Theory - Enabling AR Drone to Track and Follow Objects


Make AR Drone Follow Objects it sees keeping a constant distance.

[singlepic id=58 w=800 h=600 float=]

Image Reference -

* No Copyright violation intended - References are provided to source of the images

Sub Goals

  1. Enable AR Drone to Track Objects it sees
  2. Enable AR Drone to Measure Depth (Distance)
  3. Control AR Drone's flight to ensure that the Object in field of view

Enable AR Drone to Track Objects

The idea is to use OpenCV and one of its many techniques to Track Object. Simplest of them being Track Objects by color.


  1. Provide the Camera feed to OpenCV
  2. Select a color to track
  3. Convert that Color to 1 and everything else to 0
  4. Now the Program can see following
  5. Track the moment of the white part on the black background

This is just one of the ways, there are many other ways to do so. To keep things simple in the first attempt, this method is preferred


Reference -

Enable AR Drone to Measure Depth (Distance)

Before we talk about using OpenCV to measure Depth, we need to understand more about Computer Vision.

Human Beings have 2 Eyes to give them a Perception of Depth. We focus on an Object and that Object is in center for both of our eyes. Rest of the objects which we do not focus on either move left or right. Our Brain uses this information to figure out the depth/distance.


Theory of Stereopsis (How Humans Perceive Depth)

Here is an example taken from

Human Eye focusing on the Whale. Note, it is not focusing on the Cube infront of the Whale

Left Eye

Note the Cube appears on the Right

Right Eye

Note the Cube appears on the Left


Human brain uses this disparity of position of Cube to compute Depth

If same has to be applied to Computer Vision (CV), then in the above picture the Light Objects are closer and Darker Objects are farther.

Problem - AR Drone only has one Eye

The biggest problem to apply the above theory to AR Drone to track object and especially its Depth is that AR Drone has only one Camera (One Eye).

Now We can bring in 2 cameras into picture as follows

and this setup works well, but for Computer Vision if you are experimenting in a Lab

What we need is a light weight solution which gives our AR Drone Camera a Stereoscopic Vision.

Here comes another theory and believe me there is no claim from myside on getting the following scientific concept of prisms right. (Probably more reading, buying prisms and working with it would help me ascertain the theory right, but for now its a concept, so don't yell if I got it wrong)

[singlepic id=57 w=800 h=600 float=center]

The idea is to use some light weight prisms and mirrors to make the AR Drone see things as follows

Then use Open CV to create a Depth Map and then AR Drone Program will see Objects as follows

The Lighter parts are closer than the darker part

References -


Control AR Drone's flight to keep Object in field of view

The Idea is to use

  1. Input from Track Object Program (see above), to control AR Drones Flight Left and Right
  2. Input from Depth Tracking Program (see above), to control AR Drones Flight to Forward and Backwards


While all this is theory at the present moment, I hope you enjoyed reading it. The next steps for me are to play with Open CV on Node Js and do the following things

  1. Write a JavaScript Program to track Objects by Color (for now) and come up with Algorithm to measure how much the camera should be moved to keep the object at center
  2. Build a Stereoscopic Vision set for Laptop Camera and use Open CV to take 2 feeds from same Image and measure depth
  3. Build a Stereoscopic Vision set light enough to be mounted on AR Drone and give the Node JS program Stereoscopic feed
  4. Wire all these together and make AR Drone to see more like we do and make it follow objects