Drone equipped with special cameras can dodge fast-moving objects

Researchers from the University of Zurich have equipped a drone with a novel type of camera to give it the ability to detect and avoid fast-moving objects.

According to the researchers, drones that are equipped with cameras typically take 20 to 40 milliseconds to process an image and react in order to detect obstacles, which is not quick enough to avoid a bird or another drone. It also isn't quick enough to avoid a static obstacle when the drone itself is flying at high speed.

To solve this problem, the researchers have equipped a quadcopter with special cameras and algorithms that reduced its reaction time down to a few milliseconds, which is quick enough for it to avoid a ball thrown at it from a short distance. This type of reaction time could make drones especially effective in situations such as the aftermath of a natural disaster.

“For search and rescue applications, such as after an earthquake, time is very critical, so we need drones that can navigate as fast as possible in order to accomplish more within their limited battery life,” explains Davide Scaramuzza, who leads the Robotics and Perception Group at the University of Zurich as well as the NCCR Robotics Search and Rescue Grand Challenge.

“However, by navigating fast drones are also more exposed to the risk of colliding with obstacles, and even more if these are moving. We realized that a novel type of camera, called Event Camera, are a perfect fit for this purpose.”

The researchers explain that traditional video cameras such as the ones found in smartphones work by regularly taking snapshots of the whole scene, which is done by exposing the pixels of the image all at the same time. With this technique, though, a moving object can only be detected after all the pixels have been analyzed by the on-board computer.

Event cameras, which are a recent innovation, have smart pixels that work independently of each other, and the pixels that detect no changes remain silent, while the ones that see a change in light intensity immediately send out the information, which means that only a tiny fraction of all the pixels of the image will need to be processed by the onboard computer. This speeds up the computation “a lot,” according to the researchers.

Researchers note that existing object-detection algorithms for drones do not work well with event cameras, so with this in mind, they developed their own algorithms that collect all the events recorded by the camera over a very short time, then subtracts the effect of the drone’s own movement, which typically account for most of the changes in what the camera sees.

Initially, Scaramuzza and his team tested the cameras and algorithms alone. They threw objects of various shapes and sizes towards the camera, and measured how efficient the algorithm was in detecting them. Depending on the size of the object and the distance of the throw, the success rate varied between 81 and 97 percent. The system took just 3.5 milliseconds to detect incoming objects.

Next, the researchers put the cameras on an actual drone, and threw objects directly at it while conducting indoor and outdoor flights. The drone was able to avoid the objects more than 90 percent of the time, including when a ball was thrown from a three-meter distance while traveling at 10 meters per second. Researchers say that when the drone “knew” the size of the object in advance, one camera was enough, but when it had to face objects of varying size, two cameras were used to give it stereoscopic vision.

Scaramuzza says that these results show that event cameras can increase the speed at which drones can navigate by up to ten times, which greatly expands their possible applications.

“One day drones will be used for a large variety of applications, such as delivery of goods, transportation of people, aerial filmography and, of course, search and rescue,” Scaramuzza says.

“But enabling robots to perceive and make decision faster can be a game changer for also for other domains where reliably detecting incoming obstacles plays a crucial role, such as automotive, good delivery, transportation, mining, and remote inspection with robots.”

The team would like to test this system on an even more agile quadrotor in the near future.

“Our ultimate goal is to make one day autonomous drones navigate as good as human drone pilots. Currently, in all search and rescue applications where drones are involved, the human is actually in control,” says Davide Falanga, the PhD student who is the primary author of the article.

“If we could have autonomous drones navigate as reliable as human pilots we would then be able to use them for missions that fall beyond line of sight or beyond the reach of the remote control.”