Preparing a Quadrotor for Following a

Human Being Indoors While Avoiding

Obstacles
Alexander Cebulla
Bachelor Thesis
April 2014
Supervisors:
Tobias Ngeli
Prof. Dr. Otmar Hilliges
Abstract
Smartphones are widely used throughout the society. Providing them with the ability of au-
tonomous motion would lead to many interesting new uses, for example, taking images of one-
self while exercising. A possible way to achieve this is through utilizing a quadrotor. Therefore,
this thesis discusses the preparation of one, such that it is able to follow a human being indoors
while avoiding obstacles.
An existing marker detection system was extended with a lightweight rc servo-based pan/tilt
platform, which allows the quadrotor to keep track of a human carrying the marker without
an immediate need to move itself. The design of the platform and the implementation of the
necessary driver and controller to use it are presented.
Ultrasonic distance sensors were used to detect and avoid obstacles. To read them out an exist-
ing driver was extended such that it works with multiple sensors. Ultrasonic distance sensors
can inuence the measurements of each other when they are used at the same time and it was
investigated how this could be reduced through attaching funnels to them.
An extension of an existing quadrotor position controller was devised, which can navigate the
quadrotor to a predened position relative to the marker while avoiding obstacles. The func-
tionality of this extension was tested through a simulation.
i
ii
Acknowledgments
I want to thank Prof. Dr. Hilliges for providing me with the opportunity to do this thesis and my
supervisor Tobias Ngeli, who always supported me with encouragements and helpful sugges-
tions.
Furthermore, I want to thank my family and my friends for their moral support.
Finally, I extend my gratitude to the members of the Bastli
1
, the electronic laboratory of the
AMIV students union, for the many insightful discussions.
1
http://bastli.ethz.ch/
iii
iv
E11I
Eidgenossische Technische Hochschule Zurich
Swiss Federal Institute of Technology Zurich
Eigenstandigkeitserklarung
Die unterzeichnete Eigenstandigkeitserklarung ist Bestandteil jeder wahrend des Studiums verfassten
Semester-, Bachelor- und Master-Arbeit oder anderen Abschlussarbeit (auch der jeweils elektronischen
Version).
Die Dozentinnen und Dozenten konnen auch fOr andere bei ihnen verfasste schriftliche Arbeiten eine
Eigenstandigkeitserklarung verlangen.
lch bestatige, die vorliegende Arbeit selbstandig und in eigenen Worten verfasst zu haben. Davon
ausgenommen sind sprachliche und inhaltliche Korrekturvorschlage durch die Betreuer und Betreuerinnen
der Arbeit.
Titel der Arbeit (in Druckschrift):
'-'par/n!J a Guaalrof "'r {"'r Fo-{lo.yillJ or "1utnQA-
9ein.,9 Jndoor.s Wh;(e, AvoirfinJ Ob.rf qc,{e.J
Verfasst von (in Druckschrift) :
Bei Gruppenarbeiten sind die Namen al/er
Verfasserinnen und Verfasser erforderlich.
Name(n):

lch bestatige mil meiner Unterschrift:
Vorname(n):
A(exa11Jec
- lch habe keine im Merkblatt .Zit1er-Knigqe ' beschriebene Form des Plagiats begangen.
- lch habe alle Methoden, Oaten und Arbeitsablaufe wahrheitsgetreu dokumentiert.
- lch habe keine Oaten manipuliert.
- lch habe alle Personen erwahnt, welche die Arbeit wesentli ch unterstUtzt haben.
lch nehme zur Kenntnis, dass die Arbeit mil elektronischen Hilfsmitteln auf Plagiate uberpruft werden kann .
Ort, Datum
3Q. Qf;
Unterschrift(en)
Bei Gruppenarbeiten sind die Namen al/er Verfasserinnen und
Verfasser erforder/ich. Durch die Unterschriften bUrgen sie
gemeinsam far den gesamten lnhalt dieser schriftlichen Arbeit.
-
vi
Contents
List of Figures ix
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problems and Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Related Work 5
2.1 Tracking Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Collision Avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3 Mathematical Representation 7
3.1 Preliminary: Position and Attitude of a Quadrotor . . . . . . . . . . . . . . . . 7
3.2 Pan/tilt Platform Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.3 Distance Sensor Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4 Controller 15
4.1 Pan/tilt Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.2 Extension of a Quadrotors Position Controller: Following a Leader While
Avoiding Obstacles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5 Hardware and Implementation 21
5.1 Pixhawk 4 Flight Management Unit (PX4FMU) . . . . . . . . . . . . . . . . . 21
5.2 Pan/tilt platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.3 Distance sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
vii
Contents
6 Experiment Setup and Simulation 27
6.1 Investigating the Inuence Ultrasonic Distance Sensor Might Have on the Mea-
surements of Each Other . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
6.2 Servo Modication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.3 Testing the Functionality of the Pan/tilt Platforms Position Controller . . . . . 29
6.4 Simulating the Quadrotor System . . . . . . . . . . . . . . . . . . . . . . . . . 30
7 Results and Discussion 33
7.1 Inuence Ultrasonic Distance Sensor Might Have on the Measurements of Each
Other . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
7.2 Functionality of the Pan/tilt Platforms Position Controller . . . . . . . . . . . 37
7.3 Simulation of the Quadrotor System . . . . . . . . . . . . . . . . . . . . . . . 39
8 Conclusion and Outlook 47
8.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
8.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Bibliography 49
viii
List of Figures
3.1 Position and attitude of the modelled quadrotor. . . . . . . . . . . . . . . . . . 8
3.2 Outline of the pan/tilt platform model. . . . . . . . . . . . . . . . . . . . . . . 9
3.3 Step response of a servo. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4 Field of view of the camera. . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.5 Overview over the distance sensor model. . . . . . . . . . . . . . . . . . . . . 12
4.1 Controlled system of the pan/tilt platform. . . . . . . . . . . . . . . . . . . . . 15
4.2 Calculation of the angles
C
and
C
from r
CM
. . . . . . . . . . . . . . . . . . . 18
4.3 Display of the vectors involved in calculating the position set point p for the
quadrotors position controller. . . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.1 3D models of the pan/tilt platform. . . . . . . . . . . . . . . . . . . . . . . . . 22
5.2 Marker detection system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.3 MB1222 ultrasonic distance sensor. . . . . . . . . . . . . . . . . . . . . . . . 26
6.1 Experiment setup for testing the distance sensors. . . . . . . . . . . . . . . . . 28
6.2 Modication of a servo. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.3 Experiment setup for testing the functionality of the pan/tilt platforms position
controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
7.1 Experiment results: Inuence ultrasonic distance sensors have on each other at
1 m. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
7.3 Experiment results: Inuence ultrasonic distance sensors have on each other at
70 cm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
7.5 Experiment results: Comparison between three executions of the pan/tilt plat-
forms position controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
ix
List of Figures
7.6 Experiment results: Comparison between two executions of the pan/tilt plat-
forms position controller with different marker positions. . . . . . . . . . . . . 38
7.7 Validation of the servo model. . . . . . . . . . . . . . . . . . . . . . . . . . . 40
7.8 Validation of the pan/tilt platform model. . . . . . . . . . . . . . . . . . . . . . 42
7.9 Simulation of the quadrotor ight. . . . . . . . . . . . . . . . . . . . . . . . . 44
7.10 Advantages of using a pan/tilt platform. . . . . . . . . . . . . . . . . . . . . . 45
x
1
Introduction
In this chapter we rst explain our motivation for choosing this topic. After that we describe
the problems faced during this work and the goals derived from them. Finally, an outline of this
thesis is presented.
1.1 Motivation
In 2013 more than a third of the Swiss population had a smartphone
1 2
. Such a device already
offers a range of possibilities for simplifying our daily lives, for example, it can manage our
appointments, guide us through a foreign town and, with the help of specialized applications,
even track how much we exercise. Furthermore, almost all smartphones have a camera and
there are attempts to add a picoprojector to them. Nonetheless, these devices are limited by
their inability to move autonomously and it is interesting to think about the possibilities which
stem from providing them with this ability. Imagine you want to take an image of yourself
or use the projector to give a presentation. As it is today, you either need somebody to hold
your phone or nd a suitable place where you can place it. However, we believe it would be
much more convenient if the phone could assume a favourable position on its own. Another
potential use for autonomous motion might be that the phone really guides you through a town
as opposed to only displaying the correct way.
Encouraged by these possibilities, we thought about the requirements which an autonomous
moving smartphone should meet. Clearly, it must be able to keep track of and follow its owner
1
http://en.comparis.ch/telecom/mobile/news/2013/02/smartphone-verbreitung-schweiz.
aspxIn
2
http://www.bfs.admin.ch/bfs/portal/de/index/themen/01/01/key.html
1
1 Introduction
while avoiding (sometimes moving) obstacles. Furthermore, to full the tasks described above,
it would be preferable if it (or at least its camera) could assume different altitudes.
One possible way to meet these requirements would be to combine a smartphone with a quadro-
tor. Quadrotors can not only go to different locations in 3D space, but also easily avoid some of
the common obstacles ground-based robots have to deal with, like, for example, stairs or table
legs. In addition, in recent years there has been a lot of research into quadrotors which amongst
other things concerned itself with following a leader and collision avoidance.
We, therefore, decided to work on this thesis topic, because it allowed us to contribute to
one of the many projects at ETH Zurich on quadrotors: The PIXHAWK Research Project
3
.
Through preparing the hard- and software, which assists one of their PIXHAWK quadrotors
with following a human being indoors while avoiding collisions, we gained valuable insights
in the inner workings of a quadrotor and went a rst step in the direction of an autonomous
moving smartphone.
1.2 Problems and Goals
To prepare a quadrotor for following a human being indoors while avoiding obstacles, the au-
thors needed to solve the following problems:
1. Identifying a human being and determining the position relative to her
The quadrotor needs to be able to identify the correct human and then measure its relative
position with respect to this human.
This problem was solved by attaching a passive marker to the human in question. This
marker was then detected through a system already set up by another work based on the
PIXHAWK platform [Ngeli 2013]. However, this system used a camera rigidly attached
to the quadrotor.
This can lead to different problems:
First, if the human is quickly moving sideways the dynamics of the quadrotor might be to
slow to follow the marker.
Second, if the human moves forwards or backwards the quadrotor needs to lean in the
corresponding direction in order to perform a translative ight and keep its predened
position relative to the marker. However, this might move the marker out of the eld of
view of the camera.
2. Registering nearby obstacles and measuring the distance towards them
Even while automatically avoiding obstacles such as table legs or stairs, a quadrotor still
needs to deal with obstacles like human beings, walls or columns. Therefore, the quadro-
tor must be able to register nearby objects and measure the distance towards them.
3. Using the position and distance measurements for following a human being while
avoiding obstacles
Finally, the quadrotor must use the measured distances towards obstacles and the position
measurements provided by the marker detection system to follow a human being while
3
https://pixhawk.ethz.ch/
2
1.3 Thesis Outline
avoiding obstacles.
From these problems we then derived the following goals and subgoals
1. Building a pan/tilt platform and implementing a controller for it so that the platform
follows a marker detected by the marker detection system
With such a platform the quadrotor can track the marker without an immediate need to
move. In addition, the pan/tilt platform can compensate for tilting or rotational motions
of the quadrotor.
Subgoals are:
Designing a rc servo-based pan/tilt platform.
Implementing a driver to control the rc servos (for the rest of this thesis we drop the
rc) of the platform.
Implementing a position controller for the platform.
Testing the functionality of this controller.
2. Using ultrasonic distance sensors to register nearby obstacles and to measure the
distance towards them
Ultrasonic distance sensors report the distance towards the nearest obstacle inside their
cone-shaped detection zone. However, multiple of these sensors used at the same time
can interfere with each other, since one sensor might record the ultrasound emitted by
another sensor
4
.
Therefore, we set the following subgoals:
Attaching the sensors to the quadrotor.
Extending an exiting driver such that it can read out multiple distance sensors.
Exploring how to use these sensors such that they do not interfere with each other.
3. Extending an existing position controller for following a human being while avoiding
obstacles and testing this extension through a simulation
The last goal was to devise an extension for an existing position controller such that it
keeps a predened position relative to the marker and deviates from this position only if
the measured distance towards an obstacle becomes too small. Furthermore, since this
controller extension could not be tested during an actual quadrotor ight, its functionality
was tested through a simulation.
1.3 Thesis Outline
Chapter 1:
Explains the motivation behind this thesis and discusses the problems posed by it and the
goals which were derived from them.
Chapter 2:
4
http://www.maxbotix.com/articles/031.htm
3
1 Introduction
Gives a short overview over other work related to following an object and avoiding colli-
sions with a quadrotor.
Chapter 3:
Introduces the mathematical representation of the pan/tilt platform and the distance sen-
sors.
Chapter 4:
Explains the controllers used for simulating a servo and to control the pan/tilt platform.
Furthermore, the extension of the quadrotors position controller is described.
Chapter 5:
Provides an overview over the used hardware and the software implemented for it.
Chapter 6:
Explains how the conducted experiments were set up. Furthermore, the simulation of the
quadrotor system and how this simulation was evaluated is described.
Chapter 7:
Discusses the results obtained through the experiments. In addition, the outcomes of the
simulation are shown.
Chapter 8:
Gives a conclusion and discusses possible future work.
4
2
Related Work
We now present a short overview over other work related to following an object, and avoiding
collisions, with a quadrotor.
2.1 Tracking Objects
In [Ngeli 2013] it was demonstrated how quadrotors could y in a formation without requiring
support from their environment. Thereby, it was necessary that the quadrotors could measure
their position relative to each other. So markers were attached to each quadrotor and a marker
detection system, which could track these markers, was set up. This system is the same which
was used for this thesis. The problem that it might lose track of the marker, when the quadrotor
needed to tilt in order to follow it, was also described in this work.
A quadrotor which accompanies a jogger in order to make the exertion experience more engag-
ing and to maybe act as a motivational partner was presented in [Graether and Mueller 2012].
The tracking of the jogger was, thereby, made possible through a visual marker printed on a
T-shirt.
Another marker-based tracking system was presented in [Wenzel et al. 2012]. Four IR diodes,
arranged in a known pattern, were tracked by an IR camera, which reported their xy-positions in
its image frame. The authors used a method developed by them to compute the 3D marker posi-
tion from this information, which allowed a quadrotor to successfully follow another quadrotor.
Similar to this thesis, the authors attached the IR camera to a servo-based pan/tilt platform,
which allowed the follower to keep track of the leader even while the leader was not directly in
front of it.
A marker-less approach for following a human was discussed in [Naseer et al. 2013]. Thereby,
the idea was to a attach a depth camera to a quadrotor and then use an existing person tracker
5
2 Related Work
to obtain the position of a persons body parts. However, a problem was that the person tracker
required a static camera in order to segment a human from the background. The authors pro-
vided a solution to this through warping the depth images to a virtual, static camera. It was then
shown that the depth images of the virtual camera were sufciently stable to be used with the
existing person tracker. The position of the body parts computed by it allowed a quadrotor to
follow a human.
A color-based approach to object tracking was shown in [Teuliere et al. 2011], where the to-be
tracked object was specied through providing an image of it to the quadrotor. The position
of the object was then estimated with a particle lter, which improved the robustness of the
system against large displacements and occlusions of the object. However, because only the 2D
position of the object in the image plane of the camera was estimated, the quadrotor had to y
above the tracked object at a constant altitude.
2.2 Collision Avoidance
A minimal sensor setup enabling a quadrotor to y autonomously indoors was described in
[Roberts et al. 2007]. Thereby, four perpendicular infrared sensors were used to measure the
distance towards obstacles and the authors successfully let the quadrotor y along a random
path while it avoided collisions. However, this was done in a controlled environment where the
only obstacles were walls which offered a large surface for detection.
Five different ways how a quadrotor could react to obstacles in its ight path were discussed
in [Bouabdallah et al. 2007]. Furthermore, a collision avoidance for a hovering quadrotor was
implemented. Similar to this thesis, four ultrasonic distance sensors were used to measure
the distance to obstacles. The authors found that the measurements of theses sensors could
be extremely noisy, because of various reasons such as sound reecting from the ground when
ying at a lowaltitude, the propeller downwash inuencing the ultrasound waves and, as already
mentioned, one sensor recording the ultrasound emitted by another sensor. Through adding a
lter the sensor noise could be successfully reduced.
Other problems of ultrasonic sensors are their low resolution and that an object needs to actually
reect the sound emitted by the sensors back to them. Therefore, for a successful detection the
angles at which the surface of the object might face a sensor and the materials it could be made
of are limited. The authors of [Gageik et al. 2012] solved some of these problems through
fusing together the measurements of multiple ultrasonic distance sensors. Concerning the last
problem they suggested to further add distance sensor which use other methods than measuring
the time-of-ight of ultrasound.
A completely different approach to obstacle avoidance was presented in [Grzonka et al. 2008].
There the quadrotor was provided with a map of its environment in which it localized itself
through the use of a laser range nder attached to it.
6
3
Mathematical Representation
In this chapter we present the mathematical models we used to model the pan/tilt platform and
the distance sensors.
3.1 Preliminary: Position and Attitude of a Quadrotor
A detailed discussion of how a quadrotor could be modelled is, for example, given in
[Mahony et al. 2012].
For this thesis, it was important to model the position and the attitude of a quadrotor relative to
its environment.
In order to do so the quadrotor was assumed to be a point-mass. Then two right-handed coor-
dinate systems were introduced: The world coordinate system W with basis {e
W
x
, e
W
y
, e
W
z
}
and the quadrotors body coordinate system B with basis {e
B
x
, e
B
y
, e
B
z
}, dened with respect
to W. The quadrotors centre of mass coincides with the origin of B.
The position of the quadrotor is, thus, given by (see Figure 3.1) the vector p = [p
x
, p
y
, p
z
] W
pointing to the origin of B and its attitude by the rotation matrix R
B
W
: B W, where
R SO(3).
3.2 Pan/tilt Platform Model
There are different ways to construct a pan/tilt platform, however, they all have in common that,
as their name suggests, they allow the camera attached to them to be moved in a pan and tilt
motion. The setup discussed in this thesis consists of a pan and a tilt lever, named according to
7
3 Mathematical Representation
Figure 3.1: Position and attitude of the modelled quadrotor.
their type of movement. The camera is attached to the tilt lever, while the pan lever is mounted
on the quadrotor such that it can be rotated by a servo. The two levers are connected through an
other servo. For the rest of this thesis, we will refer to them as pan and tilt servo, according to
the lever they moved.
We modelled this setup with the two vectors l
pan
and l
tilt
.
The orientation of the attached camera was described through the orientation of the camera
coordinate system C with respect to B. We followed the convention
1
that C is a right-handed
coordinate system, where the z-axis points towards the scenery the camera observes and the
y-axis points downwards. Its basis is given by {e
C
x
, e
C
y
, e
C
z
} which is dened with respect to
B.
The following assumptions were made about the system setup:
The pan lever is directly mounted on top of the centre of mass of the quadrotor, therefore, l
pan
points in the same direction as the z-axis of B.
The camera is attached orthogonal to the tilt lever such that l
tilt
coincides with the y-axis of C
and points to its origin.
Let nowthe position of the shaft of the pan servo be given by , the angle between the projection
of the z-axis of C onto the xy-plane of B, and the x-axis of B (

2
< <

2
).
In addition, let the position of the shaft of the tilt servo be given by , the angle between the
z-axis of B and l
tilt
(

2
< <

2
) (see Figure 3.2).
Furthermore, let d
pan
and d
tilt
be the lengths of the pan and the tilt lever.
1
http://www.hitl.washington.edu/artoolkit/documentation/cs.htm
8
3.2 Pan/tilt Platform Model
Figure 3.2: Outline of the pan/tilt platform model showing the body and camera coordinate system.
Furthermore, the position of the tilt servos motor shaft and the pan servos motor shaft
is displayed.
The two vectors l
pan
and l
tilt
are then calculated by:
l
pan
=
_

_
0
0
d
pan
_

_
(3.2.1)
l
tilt
=
_

_
d
tilt
sin cos
d
tilt
sin sin
d
tilt
cos
_

_
. (3.2.2)
In addition, the rotation matrix R
C
B
, which transforms camera coordinates to body coordinates
can be computed through:
e
C
x
=
_

_
cos /2
sin /2
0
_

_
(3.2.3)
e
C
y
= l
tilt
/|l
tilt
| (3.2.4)
e
C
z
= e
C
x
e
C
y
(3.2.5)
R
C
B
=
_
e
C
x
, e
C
y
, e
C
z

, (3.2.6)
where equation (3.2.3) is based on the observation that by construction the x-axis of C is always
parallel to the xy-plane of B. Therefore, this axis can be calculated through rotating the x-axis
of B clock-wise according to the angle /2.
9
3 Mathematical Representation
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
80
60
40
20
0
20
40
60
80
time [s]
p
o
s
i
t
i
o
n

o
f

t
h
e

s
e
r
v
o

s

m
o
t
o
r

s
h
a
f
t

[

]


Step response of a servo
Figure 3.3: The step response of a servo, averaged over 10 samples.
3.2.1 Servo Model
To control the motion of the the pan/tilt platformwe used two servos. In general, a servo consists
of a control circuit and an electro motor, which is connected to a gear box and a feedback
potentiometer
2
. A position can be provided to it modulated as a PWM signal. The control
circuit decodes this position and compares it to the position measured by the potentiometer.
If these two position differ the control circuit starts to pulse the motor (switches it on/off) at
a xed rate per second, where the speed of the motor can be controlled through the length of
these pulses. Therefore, the motor moves to the set position and then remains there.
In order to create a model of a servo, we read out its feedback potentiometer to measure its step
response. We did so ten times and then calculated an average (for details see Chapter 6.2), the
result can be seen in Figure 3.3.
Based on this average step response, we made the following assumptions:
If the motor of the servo is turned on it quickly accelerates with a constant angular acceleration

s
until it reaches a maximum angular velocity
max
, where the sign of
s
and
max
is deter-
mined by the direction in which the motor should turn.
If the motor is turned off it is decelerated with
s
.
2
http://www.futaba-rc.com/servos/digitalservos.pdf
10
3.2 Pan/tilt Platform Model
Let now be the position of the servos motor shaft, then the motion of a servo can be modelled
by:
If the motor is turned on
=
_

s
if | | < |
max
|
0 otherwise
and if the motor is turned off
=
_

s
if = 0
0 otherwise
3.2.2 Marker Detection Model
The location of the marker in camera coordinates is measured by the marker detection system
if the marker is inside the eld of view (FOV) of the camera.
Let the position of the marker in camera coordinates be r
CM
. Furthermore, let and be the
horizontal and the vertical angle of view of the camera (see Figure 3.4).
One can now calculate the relative position of the quadrotor to the marker in its body coordi-
nates:
r
BM
=
_
l
pan
+l
tilt
+R
C
B
r
CM
if

arctan
_
r
CM
x
r
CM
z
_

2
and

arctan
_
r
CM
y
r
CM
z
_

2
0 otherwise
, (3.2.7)
where l
pan
and l
tilt
are calculated according to (3.2.1) and (3.2.2) and R
C
B
according to (3.2.6).
Figure 3.4: Field of view of the camera.
11
3 Mathematical Representation
Figure 3.5: Overview over the distance sensor model.
3.3 Distance Sensor Model
We used ultrasonic distance sensors to measure the distance towards an object. They emit
ultrasound and then measure the time it needs to be reected back to them. Since the sound
waves expand as they travel through the air, these sensors have a cone-shaped detection area,
where the opening angles of this cone is predened by the manufacturer. The distance to the
nearest object inside this cone is then measured.
Hence, we modelled a distance sensor as a cone dened through t, d, h, (see Figure 3.5),
where t W is the apex of the cone (the position of the sensor) in world coordinates, d W
is its normalized axis direction (the direction in which the sensor is oriented), h is its height and
is half its opening angle. Furthermore, let l
max
=
h
cos()
be the reach of the sensor.
The obstacles were modelled as planes, given in Hesse normal form. Therefore, a plane is
specied by its unit normal vector n W, and d its distance to the origin of W, where d =
p
T
n, with p W a point on the plane. We dened the unit normal vectors, such that they point
towards the quadrotors starting point.
Since the quadrotor should never y through such a plane, one can now check if the sensor
detected an obstacle by testing if there exists a point on the cone, which lies in the negative
halfspace of the plane. For that, we used the procedure outlined in [Ericson 2004, Section 5.2.4]:
First, one must calculate q the point furthermost in the direction of n, located on the circular
endcap of the cone:
q = t + hd + r m,
where r = h tan and m = (n d) d.
Second, it must be tested if this point is in the negative halfspace of the plane:
n
T
q < d
If this is true, the modelled distance sensor detected an obstacle.
In this case, the intersection of a line between t and q, and the plane is found, where the
12
3.3 Distance Sensor Model
intersection is given as:
0 = (t +l
O
v)
T
n d,
with v =
qt
|qt|
. After that the distance from the apex t to this intersection is calculated.
Otherwise, if no object was detected, it is assumed that the sensor reports its maximum reach
l
max
.
Therefore, the distance measurement of the sensor is given through:
l
O
=
_
dt
T
n
v
T
n
if n
T
q < d
l
max
otherwise
(3.3.1)
13
3 Mathematical Representation
14
4
Controller
In this chapter the controllers used for simulating the pan and the tilt servo and to control the
pan/tilt platform are shown. Furthermore, the extension of the quadrotor position controller is
described.
Figure 4.1: The controlled system of the pan/tilt platform, with and

the set points for, and , the
current position of the pan servos and the tilt servos motor shaft, respectively.
x
and
z
are the angular velocities around the z- and the x-axis of the quadrotors body frame. r
CM
is a vector in camera coordinates pointing from the camera to the marker.
15
4 Controller
4.1 Pan/tilt Platform
The controlled system of the pan/tilt platform can be seen in Figure 4.1. Since its behaviour
should be simulated, we devised a controller for the servo model presented in Chapter 3.2.1.
After that the controller of the pan/tilt platform is shown.
4.1.1 Bang-bang Servo Motor Controller
The task of this controller was to simulate the behaviour of a servo, hence, it was not used to
control an actual servo motor.
With this in mind, we simplied the controllers design by omitting the pulsing behaviour men-
tioned in Chapter 3.2.1. Furthermore, we made the assumption that the controller can observe
the rate at which the motor decelerates (
s
) when it is turned off.
Therefore, it can compute the time the motor stops its movement if it is turned off
t
h
=

s

s
and the position the servos motor shaft will be at this time

h
= +
s
t
h


s
2
t
2
h
,
where the current position of the servos motor shaft and
s
its current angular velocity.
Based on this we devised a simple on-off controller, which calculates the input to the motor as:
u =
_

_
1 if > 0 and =
h
1 if < 0 and =
h
0 otherwise
, (4.1.1)
where is the set point for the position of the servos motor shaft and the sign of u determines
the direction in which the motor should turn. Notice that if the set position is equal to the
position
h
, where the servos motor shaft will halt if the motor is turned off, then the motor is
turned off (u = 0).
4.1.2 Position Controller for the Pan/tilt Platform
A quadrotor, which uses a marker detection system with a rigidly attached camera, might not
always be able to follow the marker, if it is moved rapidly sideways. Furthermore, a quadrotor
needs to tilt in order to perform translative movements, which might move the marker out of the
FOV of the camera.
Both of these problems can be mitigated through attaching the camera to a pan/tilt platform,
which can follow the movements of the marker independently from the movements of the
quadrotor. Therefore, the pan/tilt platform should always be oriented in such a way that the
marker remains in the middle of the cameras FOV.
16
4.2 Extension of a Quadrotors Position Controller: Following a Leader While Avoiding Obstacles
To achieve this, we used a controller, which calculates a position set point for each of the pan
and the tilt servo, where it was assumed that the camera is directly attached to them, i.e. |l
pan
| =
|l
tilt
| = 0. Therefore, changing the position of a motor shaft does only cause a rotation of the
camera coordinate systemC, but not any translation of its origin, relative to the quadrotors body
coordinate system B.
Let now
C
be the angle between the projection of r
CM
onto the zy-plane of C and the z-axis
of C (see Figure 4.2 (a)). Similarly, let
C
be the angle between the projection of r
CM
onto the
zx-plane of C and the z-axis of C (see Figure 4.2 (b)):

C
= arctan
_
r
CM
y
r
CM
z
_

C
= arctan
_
r
CM
x
r
CM
z
_
If these two angles are zero then r
CM
coincides with the z-axis of C and the marker is in the
midpoint of the cameras FOV as required.
Hence, the error term for the controller of the tilt servo is dened as
e

C
= 0
C
and the error term for the controller of the pan servo is dened as
e

C
= 0
C
In order to compensate for changes of the quadrotors attitude, its angular velocities around the
z- and the x-axis of its body frame,
z
and
x
, are integrated and subtracted.
The set point for the tilt servo is then given as

= K

C

x
dt (4.1.2)
and, analogous, for the pan servo
= K

C

z
dt, (4.1.3)
where dt the sample time of the respective controller and , the current position of the pan
servos and the tilt servos motor shaft, respectively. The controller gains K

and K

were
found through manual tuning.
4.2 Extension of a Quadrotors Position Controller:
Following a Leader While Avoiding Obstacles
The quadrotor should keep a predened position relative to the marker and deviate from this
position if the measured distance towards an obstacle becomes too small. In order to achieve
17
4 Controller
(a) (b)
Figure 4.2: (a)
C
is the angle between the projection of r
CM
onto the zy-plane of C and the z-axis of C.
(b)
C
is the angle between the projection of r
CM
onto the zx-plane of C and the z-axis of C.
this, an extension for an existing quadrotor controller (see [Mellinger and Kumar 2011]) was
devised.
Given a set point specied through a position p W and a yaw angle

, the original controller
rst computes the errors on position and velocity
e
p
= p p (4.2.1)
e
v
= p

p
and then the desired force vector:
F
des
= K
p
e
p
K
v
e
v
+ mg e
W
z
+m

p, (4.2.2)
where K
p
and K
v
are positive denite gain matrices, m the mass of the quadrotor and e
W
z
the
unit vector for the z-axis of the world coordinate system W.
After that the desired attitude of the quadrotor R
des
is calculated:
e
B
z
,des
=
F
des
|F
des
|
x
des
=
_

_
cos(

)
sin(

)
0
_

_
e
B
y
,des
=
e
B
z
,des
x
des
|e
B
z
,des
x
des
|
e
B
x
,des
= e
B
y
,des
e
B
z
,des
R
des
=
_
e
B
x
,des
e
B
y
,des
e
B
z
,des
_
18
4.2 Extension of a Quadrotors Position Controller: Following a Leader While Avoiding Obstacles
Figure 4.3: Display of the various vectors involved in calculating the set point position p for the exten-
sion of the quadrotors position controller. Only distance sensor 1 has detected an obstacle,
therefore, all other force vectors F
dist
i
i {2, ..., N} have length zero.
F
des
and R
des
are then used to compute the desired inputs u.
Based on this we wanted to use the measured relative position of the marker to set p such that
it points from the origin of W to the predened position relative to the marker.
For that let r
p
W be a vector pointing from the marker to the predened position relative to
it. Furthermore, let p
M
W be the position of the marker in world coordinates and r
WM
=
R
W
B
r
BM
a vector pointing from the quadrotor to the marker, with r
BM
calculated according to
(3.2.7) (see Figure 4.3). It follows:
p = p
M
+ r
p
= p +r
WM
+ r
p
From this the error term (4.2.1) can now be dened as:
e
p
=
_
p p = (r
WM
+ r
p
) if r
WM
= 0
0 otherwise
(4.2.3)
Furthermore, since the quadrotor should always be "looking" at the marker, we set

= arctan
_
r
WM
y
r
WM
x
_
19
4 Controller
This concludes the rst part of the extension. However, we also wanted the quadrotor to deviate
from p if the measured distance towards an obstacle becomes too small. For this we had the
idea to let the obstacles exert a force on the quadrotor which is inversely proportional to the
distance towards it. Therefore, for each distance sensor attached to the quadrotor a force vector
F
dist
, which acts opposite to the orientation of the sensor, is calculated.
Let now d
i
be the orientation of the i
th
distance sensor attached to the quadrotor, l
O
i
its current
distance measurement and l
max
i
its maximum reach, as specied in Chapter 3.3, then its force
vector is computed as:
F
dist
i
= d
i
_
1
l
max
i

1
l
O
i
_
(4.2.4)
Notice that the force goes to innity as the distance becomes smaller.
Finally, one can combine these force vectors and the new error term (4.2.3) with the equation
of the old desired force vector (4.2.2) to get:
F
des
= K
p
e
p
K
v
e
v
+mg e
W
z
+m

p +
N

i=0
K
s
F
dist
i
,
where K
s
is a diagonal gain matrix found through manual tuning and N the number of attached
distance sensors.
20
5
Hardware and Implementation
In this chapter we give an overview over the hardware we used and the software we imple-
mented.
5.1 Pixhawk 4 Flight Management Unit (PX4FMU)
The PX4FMU was developed by the PIXHAWK Reasearch Project and "is a high-performance
autopilot-on-module suitable for xed wing, multi rotors, helicopters, cars, boats and any other
robotic platform that can move."
1
It runs the NuttX
2
real-time OS and applications can be de-
veloped for it in C or C++. The communication between the various applications is realised
through an micro object request broker (uORB), which implements the publish-subscribe pat-
tern. The PX4FMU offers, among others, four UART and two I2C ports, which we used to
interact with the ultrasonic distance sensors and the pan/tilt camera platform. Furthermore, it
has four different sensors: MPU-6000 (3D ACC / Gyro), L3GD20 (3D Gyro), HMC5883L (3D
Mag) and MS5611 (barometric pressure), the measurements of which are fused and published
via the uORB.
5.2 Pan/tilt platform
The requirements for the pan/tilt platform were:
1
http://pixhawk.org/modules/px4fmu
2
http://www.nuttx.org/
21
5 Hardware and Implementation
(a) (b)
(c)
Figure 5.1: The 3D models of the pan (a) and the tilt (b) lever. The assembled pan/tilt platform (c).
22
5.2 Pan/tilt platform
1. compact dimension
2. low weight
3. simple adaptation to different servos and cameras
Based on these we designed a platform consisting of a pan and a tilt lever in Autodesk Inventor
and then 3D printed it on a MakerBot
3
Replicator 2 (see Figure 5.1). This approach allowed
us to adjust the weight and stability of the platform through controlling the inll parameter of
the 3D printer. Furthermore, the shape of the levers could easily be changed to allow the use of
different servos and cameras.
5.2.1 Servos
For moving the pan/tilt platform two Graupner
4
DES 488 BB MG servos were used, which we
connected to a 12 channel servo controller IC (SERVO12C) from Hobbytronics
5
. This chip
is connected to the PX4FMU via I2C and allows to specify the position of a servo through a
dimensionless scalar value ranging from 0 to 255. In order to refer to this value we invented the
unit ABS.
We implemented a driver for the SERVO12C, which allows an application to change the position
of the pan/tilt camera platform. For that it sends a message via uORB to the driver, which
contains the desired position of the pan and the tilt servo, where the position can be specied
either in ABS or in radian. If the latter is the case, the position is converted to ABS by the
driver.
In both cases the driver must check if the positions lie inside the predened range of positions
the pan/tilt platform can assume and needs to adjust them if necessary.
Finally, the positions are sent to the SERVO12C via the I2C protocol. The chip then converts
them to PWM signals which are applied to the corresponding servos.
In order to enable the driver to convert from radians to ABS we wrote a calibration application.
This application uses the marker detection system to rst place the marker in the midpoint of
the cameras FOV and then gradually changes the position of the pan servos motor shaft while
measuring the corresponding angle. After that it does a linear regression on these data points
and provides the computed parameters to the driver. This procedure is then repeated for the tilt
servo.
5.2.2 Marker Detection System: PointGrey Firey Camera and
ODROID-U2
The marker detection system we utilized was already set up by another work [Ngeli 2013]
and uses a 0.3 MP PointGrey
6
black and white Firey camera running at 30 Hz to observe the
3
http://www.makerbot.com/
4
http://www.graupner.de/
5
http://www.hobbytronics.co.uk/datasheets/slave_servo_datasheet_v02.pdf
6
http://ww2.ptgrey.com/
23
5 Hardware and Implementation
Figure 5.2: The marker detection system consists of a PointGrey black and white Firey camera and an
ODROID-U2 single-board computer
marker. Captured frames from this camera are transmitted via USB to an ODROID-U2
7
single-
board computer, which runs the Ubuntu operating system. If a marker is present, its 3D position
in camera coordinates is then extracted from these frames by the ARToolKitPlus
8
library and
sent to the PX4FMUover an UART connection. On the PX4FMUa receiver application extracts
the relevant information and publishes it via uORB.
5.2.3 Implementation of the Position Controller
The implementation of position controller followed the ideas presented in Chapter 4.1.2. The
sample time dt was, thereby, set to 33 ms, since the Firey camera was running at 30 Hz and
we made the simplifying assumption that the time needed to extract the position of the marker
was constant.
A problem was that the current position of the servo motor shafts, and , could not be ob-
served. We, therefore, estimated them as the last position, which was set. However, to do so it
had to be ensured that

and lie inside the predened range of positions the pan/tilt platform
could assume. If this check would be omitted the position sent to the driver might be adjusted
by it, which led to wrong estimates.
5.3 Distance sensors
For measuring the distance towards an obstacle, we used the MB1222 I2CXL-MaxSonar-EZ2
ultrasonic distance sensor. It is suggested for collision avoidance by its manufacturer and can
detect persons up to approximately 2.4 m.
9
Furthermore, its measurements can be triggered and
read out via I2C.
7
http://www.hardkernel.com
8
http://studierstube.icg.tugraz.at/handheld_ar/artoolkitplus.php
9
http://www.maxbotix.com/documents/I2CXL-MaxSonar-EZ_Datasheet.pdf
24
5.3 Distance sensors
We extended an already existing driver for this sensor, such that it can trigger the measurement
and read out multiple sensors. However, when using multiple sensors they can inuence each
other, since one sensor might record the ultrasound emitted by another sensor. A save method to
avoid this is to let one sensor complete its measurement before starting the measurement of the
next sensor. Even so, this method rapidly reduces the measurement rate of the sensors, which,
if only one sensor is used, can vary between 10 and 40 Hz depending on the environment and
the farthest distance one wants to measure.
Thus, we decided to implement a function, which allows to group sensors together, such that
the measurements of all the sensors in one group must be completed before the measurements
of the next group are started. With this it is possible to group sensors together, which are
unlikely to inuence each other, for example, sensors which are oriented in opposite directions.
Furthermore, we designed and 3D printed funnels, which we attached to the sensors, thereby,
physically shielding them from each other. In addition, we also designed brackets, which can
be attached to the quadrotors frame, to mount the sensors on the quadrotor (see Figure 5.3).
25
5 Hardware and Implementation
(a) (b)
Figure 5.3: The MB1222 I2CXL-MaxSonar-EZ2 ultrasonic distance sensor attached to the quadrotor
with (a) and without (b) funnel
26
6
Experiment Setup and Simulation
In the next sections we describe the experiments, which were carried out to investigate how
the ultrasonic distance sensors might inuence the measurements of each other and to test the
functionality of the pan/tilt platforms position controller.
For the latter experiment the servos had to be modied such that the position of their motor
shafts could be read out directly. This modication is also described.
Furthermore, we explain the simulation of the quadrotor system and how this simulation was
evaluated.
6.1 Investigating the Inuence Ultrasonic Distance
Sensor Might Have on the Measurements of Each
Other
As mentioned previously one ultrasonic distance sensor might inuence the measurements of
another ultrasonic distance sensor if both are measuring at the same time.
We conducted an experiment, where we tried to nd out how strong this inuence was. We then
went on and attached funnels to the sensors, thereby, shielding them from each other. It was
then tested if this reduces the inuence they had on one another.
For this experiment, two distance sensors were attached to a quadrotors body frame, using
the brackets described in Chapter 5.3, which was then placed in a corner (see Figure 6.1), rst
at a distance of 70 cm and afterwards of 100 cm from each of the two corner walls. The
measurements of the sensors were then started at the same time and their reported distances
were recorded.
27
6 Experiment Setup and Simulation
Figure 6.1: Experiment setup for testing the inuence two ultrasonic distance sensors might have on
each other, when used at the same time. The body frame was rst placed at a distance of
70 cm and then of 100 cm from each of the two walls.
28
6.2 Servo Modication
Figure 6.2: Modication of a servo. The white, green and black wires are connected to the control circuit
of the servo. The blue and the second black wire were added in order to measure the position
of the servos motor shaft.
6.2 Servo Modication
A servo contains a feedback potentiometer, which informs its position controller about the cur-
rent position of its motor shaft.
We opened the pan and the tilt servo and soldered a wire to each of their potentiometers (see Fig-
ure 6.2), which we then lead out of the servo casing. The voltage measured on these wires is
directly related to the position of the servos motor shafts. To convert it to radian we measured
the corresponding voltage for different positions of the servos motor shaft and then calculated
a linear regression to obtain the conversion factor.
6.3 Testing the Functionality of the Pan/tilt Platforms
Position Controller
In order to test the functionality of the pan/tilt platforms position controller a marker was placed
at different positions relative to the platform and it was then measured how long the controller
needed to orient the platform, starting from a known initial position, such that in the end the
marker was placed in the middle of the PointGrey Firey cameras FOV.
Because the position of the pan and the tilt servo was controlled independently by the controller,
we decided to test its functionality for each of the platforms two degrees of freedom separately
(see Figure 6.3). Therefore, the rst two marker positions were chosen such that the pan servos
motor shaft needed to be rotated by 24

and by 16

degrees, respectively. The next two positions

were then selected such that the tilt servos motor shaft needed to be rotated by 16

and by 10

,
respectively.
Thereby, we measured the angle of view of the PointGrey Firey camera to be 48

horizontally
and 32

vertically.
Because of the sensitivity of the marker detection system to a changing illumination of the
29
6 Experiment Setup and Simulation
(a) (b)
Figure 6.3: Experiment setup for testing the functionality of the pan/tilt platforms position controller.
The marker was positioned such that the platform either moved in a tilting (a) or a panning
(b) motion.
marker, it was important to control the lighting conditions while these measurements were
taken.
6.4 Simulating the Quadrotor System
We gradually implemented a simulation based on the models and controllers described in Chap-
ter 3 and 4. First, the bang-bang position controller for the servo model was implemented and
the model was tted to our measurements of the servos system responds. This was then used
as a building block for the simulation of the pan/tilt platform.
We then extended a simulation of a quadrotor model provided to us according to the ideas de-
scribed in Chapter 4.2. Finally, this simulation was combined with the simulation of the pan/tilt
platform.
6.4.1 Simulating and Evaluating the Servo Model
The servo model should be used in the simulation of the pan/tilt platform. Therefore, we
recorded the system response of the two servos ten times with an oscilloscope while they were
incorporated in the pan/tilt platform with the PointGrey Firey camera attached.
These measurements were then put on top of each other and an average curve was calculated.
In order to do that the time delays between them were estimated through computing their cross-
correlation. For each of the two servos the model parameters
s
and
max
were then read out
from their corresponding averaged system responds.
For evaluating the model the movements of the servos were recorded while they headed for var-
30
6.4 Simulating the Quadrotor System
ious positions from a known starting position. For each of the provided positions the recording
was repeated ten times and an average was calculated as described above. Then the same posi-
tions were provided to the model and the differences between the simulation and the averaged
measurements were calculated.
6.4.2 Evaluating the Pan/tilt Platform Model
To evaluate the pan/tilt platform model we used a similar procedure to the one described in the
previous section. We, thereby, calculated an average of the measurements obtained through the
experiment for testing the functionality of the pan/tilt platforms position controller and then
compared it to the simulated movement of the pan/tilt platform model.
6.4.3 Simulating the Quadrotor Flight
The pan/tilt camera platform simulation was incorporated in an already existing simulation of a
quadrotor, which we extended as described in Chapter 4.2.
With this simulation we then tested the functionality of the extension for the quadrotors position
controller, therefore, if it could follow a marker while avoiding obstacles. In addition, we tested
if the problems the marker detection system might has through using a camera rigidly attached
to the quadrotor, namely, the loss of the marker, due to rapid sideways or forwards/backwards
movement of the marker (compare Chapter 1.2), could be mitigated with the pan/tilt platform.
31
6 Experiment Setup and Simulation
32
7
Results and Discussion
We now present and discuss the results of the experiments and the simulations described in the
previous chapter.
7.1 Inuence Ultrasonic Distance Sensor Might Have
on the Measurements of Each Other
As one can see from Figure 7.1 both distance sensors reported 1 m when they were used with
funnels. This corresponded to the actual distance at which the quadrotors body frame was
placed from the corner walls. However, if the sensors were used without the funnels, one sensor
repeatedly reported 38 cm (see Figure 7.2).
After the quadrotors frame was placed at a distance of 70 cm from the corner walls, the
experiment was repeated and we obtained a similar result. With funnels both sensors re-
ported 70 cm (see Figure 7.3), without funnels one sensor started to report distances of around
38 cm (see Figure 7.4).
The reason for why only the measurements of one sensor started to oscillate if no funnels were
used, while the measurements of the other one remained relatively stable, was that these sensors
received their take measurement command via I2C. Hence, they were not exactly triggered at
the same time, but with a slight delay, which we measured with a logic analyzer to be smaller
than half a millisecond. One can, therefore, see from these measurements, which sensor was
triggered rst and which one second, since the rst one inuenced the measurements of the
second one.
It is interesting that the measurement error, which one could see if no funnels were used, was of
33
7 Results and Discussion
a similar magnitude for both experiments. So far we have no explanation for this phenomena.
We tried to exclude other sources of measurement errors through ensuring that the sensors, when
used individually, reported the correct distance every time we attached a funnel or removed one
from them. Furthermore, to rule out a programming error, we also made an experiment where
we removed the funnels while not stopping the measurements. The results were similar to the
ones already discussed.
From these experiments we concluded that funnels can diminish the inuence ultrasonic dis-
tance sensor might have on the measurement of each other. However, they were conducted in
a static environment and the use of these sensors in a dynamic environment might introduce
additional noise. To deal with this a lter, as suggested by [Bouabdallah et al. 2007], might be
required.
34
7.1 Inuence Ultrasonic Distance Sensor Might Have on the Measurements of Each Other
0 2 4 6 8 10 12 14 16 18 20
0
10
20
30
40
50
60
70
80
90
100
110
Time [s]
M
e
a
s
u
r
e
d

d
i
s
t
a
n
c
e

[
c
m
]


Sensor 2
Sensor 1
Figure 7.1: 1 m with funnels.
0 2 4 6 8 10 12 14 16 18 20
0
10
20
30
40
50
60
70
80
90
100
110
Time [s]
M
e
a
s
u
r
e
d

d
i
s
t
a
n
c
e

[
c
m
]


Sensor 2
Sensor 1
Figure 7.2: 1 m without funnels.
35
7 Results and Discussion
0 2 4 6 8 10 12 14 16 18 20
0
10
20
30
40
50
60
70
80
Time [s]
M
e
a
s
u
r
e
d

d
i
s
t
a
n
c
e

[
c
m
]


Sensor 2
Sensor 1
Figure 7.3: 70 cm with funnels.
0 2 4 6 8 10 12 14 16 18 20
0
10
20
30
40
50
60
70
80
Time [s]
M
e
a
s
u
r
e
d

d
i
s
t
a
n
c
e

[
c
m
]


Sensor 2
Sensor 1
Figure 7.4: 70 cm without funnels.
36
7.2 Functionality of the Pan/tilt Platforms Position Controller
7.2 Functionality of the Pan/tilt Platforms Position
Controller
Through the measurements we obtained, it could be veried that the pan/tilt platforms position
controller we implemented was indeed able to control the position of the platform such that the
marker is placed in the FOV of the marker detection systems camera. Its average precision,
thereby, was better than 2

.
However, as one can see from the red and the green curve in Figure 7.5 the performance of
the controller could vary noticeably from test to test. Furthermore, it could happen that the
controller might overshot for one test (see the blue curve in Figure 7.5) and then not produced
any overshoots for the following tests.
We also observed that the time the controller needed to orient the pan/tilt platform, such that
the marker is placed in the FOV of the camera, might not get shorter if the distance, by which
servos motor shaft needed to be moved, was reduced (see Figure 7.6).
A possible explanation for this variance of the pan/tilt platforms position controllers perfor-
mance might be that the marker detection system used the Ubuntu OS, which is not a real-time
OS and so there are no guarantees given for the time needed to compute the position of the
marker from a captured frame. Hence, the system has not a constant rate at which it sends the
messages, containing the measurements of the markers position, to the PX4FMU, even though
the frame rate of the camera was set to 30 Hz. Therefore, it is not possible for the pan/tilt
platforms position controller to synchronize its execution with the image-taking cycle of the
camera or, for that matter, to predict when the camera might take an image of the marker.
This could lead to two different problems:
First, since the positions of the servos motor shafts could not be observed by the controller
and, thus, their last set positions were used as an estimation, it was implicitly assumed that
the marker detection system took its measurements always after the servos had nished their
movement. Violating this assumption might be the reason why the controller, from time to time,
produced overshoots even though its gain was not changed.
The second problem was that the interval between two position messages could vary and since
the controller was only able to carry out its computation when a position message arrived from
the marker detection system, the time between two outputs from the controller could also vary
to the same extent. This might explain the variance of the time needed to orient the platform.
37
7 Results and Discussion
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
1
1
3
5
7
9
11
13
15
17
19
time [s]
p
o
s
i
t
i
o
n

o
f

t
h
e

p
a
n

s
e
r
v
o

s

m
o
t
o
r

s
h
a
f
t

[

]


test run 1 (target pos.: 16)
test run 2 (target pos.: 16)
test run 3 (target pos.: 16)
Figure 7.5: Three measurements of the movement of the pan servos motor shaft, where the position of
the marker and the controllers gain were left unchanged. The measurements were ltered
in order to remove measurement noise.
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35
2
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
time [s]
p
o
s
i
t
i
o
n

o
f

t
h
e

p
a
n

s
e
r
v
o

s

m
o
t
o
r

s
h
a
f
t


[

]


test run 1 (target pos.: 16)
test run 2 (target pos.: 24)
Figure 7.6: Two measurements of the motion of the pan servos motor shaft, where the marker was rst
positioned such that the servo needed to rotate its shaft by 16

(blue curve) and then by 24

(red curve). The controllers gain remained unchanged. The measurements were ltered in
order to remove measurement noise.
38
7.3 Simulation of the Quadrotor System
7.3 Simulation of the Quadrotor System
We now present the results obtained through simulating the quadrotor system. Furthermore, in
case of the servos and the pan/tilt platforms simulation, these results were also compared with
the measured real-world behaviour.
7.3.1 Evaluation of the Servo Model
A comparison between the simulated and the averaged measured movement of the tilt servos
motor shaft by 20

is displayed in Figure 7.7. One can see a jump at the start of the movement of
the servos motor shaft and spikes at its end. These phenomena are present in all measurements
of movements of the servos motor shaft, however, we have no explanation for them. Ignoring
these, the biggest difference between the simulation and the averaged measurements in Figure
7.7 has a magnitude of around 2.7

and originates from the overshooting of the servo, which

was not modelled.
39
7 Results and Discussion
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14
0
2
4
6
8
10
12
14
16
18
20
22
24
time [s]
p
o
s
i
t
i
o
n

o
f

t
h
e

t
i
l
t

s
e
r
v
o

s

m
o
t
o
r

s
h
a
f
t


[

]


Average of 10 measurements
Simulation
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14
2
1
0
1
2
3
4
5
time [s]
D
i
f
f
e
r
e
n
c
e

b
e
t
w
e
e
n

t
h
e

r
e
a
l

a
n
d

t
h
e

s
i
m
u
l
a
t
e
d

p
o
s
i
t
i
o
n

[

]


(Average of 10 measurements simulation)
Figure 7.7: Validation of the servo model. The movement of the tilt servos motor shaft was recorded
while it should move by 20

. The blue curve is the average of ten measurements while the

green curve is the simulations output. On the bottom, the difference between these two
curves is displayed.
40
7.3 Simulation of the Quadrotor System
7.3.2 Evaluation of the Pan/tilt Platform Model
As one can see from Figure 7.8 the movement of our model was slower than the measured
movement of the real-world pan/tilt platform.
A possible explanation for this might be that in our simulation we did not consider the varying
rate at which the platforms position controller receives the position messages from the marker
detection system (see Chapter 7.2).
41
7 Results and Discussion
0 0.2 0.4 0.6 0.8 1 1.2
2
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
time [s]
p
o
s
i
t
i
o
n

o
f

t
h
e

p
a
n

s
e
r
v
o

s

m
o
t
o
r

s
h
a
f
t

[

]


Average of 10 measurements
Simulation
0 0.2 0.4 0.6 0.8 1 1.2
1
0
1
2
3
4
5
6
7
8
9
10
time [s]
D
i
f
f
e
r
e
n
c
e

b
e
t
w
e
e
n

t
h
e

r
e
a
l

a
n
d

t
h
e

s
i
m
u
l
a
t
e
d

p
o
s
i
t
i
o
n

[

]


(Average of 10 measurements simulation)
Figure 7.8: Validation of the pan/tilt platform model. The movement of the pan servos motor shaft was
recorded while it should move 24

. The blue curve is the average of ten measurements while

the red curve is the simulations output. On the bottom, the difference between these two
curves is displayed.
42
7.3 Simulation of the Quadrotor System
7.3.3 Simulation of the Quadrotor Flight
We were able to successfully simulate a quadrotor, which could follow a marker while avoiding
obstacles (see Figure 7.9).
However, the dynamics of our simulated pan/tilt platform were not fast enough to allow for the
tracking of a fast sideways moving marker or to compensate for tilting motions of the simu-
lated quadrotor. Only after we increased the platforms speed through manipulating the model
parameters, we were able to simulate how these problems could be mitigated through using a
pan/tilt platform (see Figure 7.10 and 7.11).
43
7 Results and Discussion
(a) (b)
(c) (d)
Figure 7.9: The quadrotor follows a marker ( (a) and (b)). It then stops before it collides with a wall
(symbolized by the green plane) ( (c) and (d)).
44
7.3 Simulation of the Quadrotor System
1 0.5 0 0.5 1 1.5 2 2.5 3
0
0.5
1
1.5
2
2.5
3
3.5
4
x position [m]
z

p
o
s
i
t
i
o
n

[
m
]
1 0.5 0 0.5 1 1.5 2 2.5 3
0
0.5
1
1.5
2
2.5
3
3.5
4
x position [m]
z

p
o
s
i
t
i
o
n

[
m
]
(a) (b)
Figure 7.10: Without the pan/tilt platform the marker is lost (symbolized by the red marker) if the
quadrotor tilts in order to perform a translative ight (a). The pantilt platform can compen-
sate for this movement (b) and the marker remains in the FOV of the camera (symbolized
by the blue marker).
1 0.5 0 0.5 1 1.5 2 2.5 3
3
2
1
0
1
2
3
x position [m]
y

p
o
s
i
t
i
o
n

[
m
]
1 0.5 0 0.5 1 1.5 2 2.5 3
3
2
1
0
1
2
3
x position [m]
y

p
o
s
i
t
i
o
n

[
m
]
(a) (b)
Figure 7.11: Without the pan/tilt platform the marker is lost if it is moved rapidly sideways (a). With the
platform it is possible to still keep track of it (b).
45
7 Results and Discussion
46
8
Conclusion and Outlook
8.1 Conclusion
In this thesis preparations were made to allow a quadrotor to follow a human being indoors
while avoiding collisions.
The tracking of the human was, thereby, realized through attaching a visual marker to her, which
was tracked by an existing marker detection system. However, this system used a rigidly at-
tached camera that might lose the marker if it was moved rapidly sideways or when the quadro-
tor needed to tilt for transitional ight in case the marker was moved forwards or backwards.
Furthermore, the positions, which the quadrotor could assume relative to the marker, were lim-
ited. Therefore, to mitigate these problems, a lightweight servo-based pan/tilt platform was
built and a position controller for it was implemented. Through a simulation and experiments it
was shown that this controller was successfully able to orient the platform such that the marker
stayed in the middle of the cameras FOV.
However, its performance varied noticeable during the experiments and we believe a reason for
this might be that the marker detection system used a non real-time OS to compute the posi-
tion of the marker from a captured frame and, therefore, the rate at which it sent the computed
positions to the PX4FMU might vary to a great degree.
To take measurements of the distances to nearby obstacles, which are needed to avoid colli-
sions with them, an existing driver for an ultrasonic distance sensor was extended such that it
could read out multiple sensors. Furthermore, to reduce the inuence these sensors had on the
measurements of each other, they could be grouped together such that only sensors, which were
unlikely to inuence each other, measured at the same time. In addition to that, funnels were
designed and it was shown that they can reduce the inuence these sensors have on one another.
47
8 Conclusion and Outlook
Finally, an extension of an existing quadrotor position controller was devised and, through a
simulation, its capability to follow a marker while avoiding collisions was shown.
8.2 Future Work
We believe that the following extensions would be useful for this system:
Implementation of a lter for the ultrasonic sensors such as described in
[Bouabdallah et al. 2007]:
While it was shown that the measurements of the sensors could be stabilized through the
use of funnels, this was only tested for a static environment. The use of these sensors
in a dynamic environment might introduce additional noise, which would require their
measurements to be ltered in order to be useful for collision avoidance.
Emergency landing:
If the system notices that the distance measurements of the all distance sensors become
smaller than a dened threshold the quadrotor should perform an emergency landing in
order to avoid damage to nearby objects or the system itself.
Alternative marker detection system:
We believe that one reason for the varying performance of the pan/tilt platforms position
controller was the varying rate at which the marker detection system sent the computed
marker positions to the PX4FMU. It would, therefore, be interesting to test the platform
with an alternative marker detection system. We started to implement the one described
by [Wenzel et al. 2012], however, due to time constraints we were not able to nish it.
Automatically searching for markers with the pan/tilt platform:
If the marker is lost the pan/tilt platform could begin to search for it, starting with the
direction in which the marker left the FOV of the camera.
Finally, we would like to evaluate the functionality of the system during a real quadrotor ight.
48
BIBLIOGRAPHY
Bibliography
BOUABDALLAH, S., BECKER, M., DE PERROT, V., AND SIEGWART, R. 2007. Toward Obsta-
cle Avoidance on Quadrotors. Eidgenssische Technische Hochschule Zrich, Autonomous
Systems Lab.
ERICSON, C. 2004. Real-time collision detection. CRC Press.
GAGEIK, N., MLLER, T., AND S., M. 2012. Obstacle detection and collision avoidance
using ultrasonic distance sensors for an autonomous quadrocopter. In UAVveek.
GRAETHER, E., AND MUELLER, F. 2012. Joggobot: a ying robot as jogging companion. In
CHI Extended Abstracts, ACM, J. A. Konstan, E. H. Chi, and K. Hk, Eds., 10631066.
GRZONKA, S., GRISETTI, G., AND BURGARD, W. 2008. Autonomous indoors navigation
using a small-size quadrotor. In Workshop Proc. of Intl. Conf. on Simulation, Modeling and
Programming for Autonomous Robots (SIMPAR).
MAHONY, R. E., KUMAR, V., AND CORKE, P. 2012. Multirotor aerial vehicles: Modeling,
estimation, and control of quadrotor. IEEE Robot. Automat. Mag. 19, 3, 2032.
MELLINGER, D., AND KUMAR, V. 2011. Minimum snap trajectory generation and control for
quadrotors. In ICRA, IEEE, 25202525.
NGELI, T. 2013. Environment-independent Formation Flight For Micro Aerial Vehicles.
Masters thesis, ETH Zurich.
NASEER, T., STURM, J., AND CREMERS, D. 2013. Followme: Person following and gesture
recognition with a quadrocopter. In IROS, IEEE, 624630.
ROBERTS, J. F., STIRLING, T., ZUFFEREY, J.-C., AND FLOREANO, D. 2007. Quadrotor
Using Minimal Sensing For Autonomous Indoor Flight. In European Micro Air Vehicle
Conference and Flight Competition (EMAV2007).
TEULIERE, C., ECK, L., AND MARCHAND, E. 2011. Chasing a moving target from a ying
uav. In IROS, IEEE, 49294934.
WENZEL, K. E., MASSELLI, A., AND ZELL, A. 2012. Visual tracking and following of a
quadrocopter by another quadrocopter. In IROS, IEEE, 49934998.
49