EPFL scientists have developed a deep-learning based motion-capture software that uses multiple camera views to model the movements of a fly in three dimensions. The ultimate aim is to use this knowledge to design fly-like robots.
“Just think about what a fly can do,” says Professor Pavan Ramdya, whose lab at EPFL’s Brain Mind Institute, with the lab of Professor Pascal Fua at EPFL’s Institute for Computer Science, led the study. “A fly can climb across terrain that a wheeled robot would not be able to.”
Flies aren’t exactly endearing to humans. We rightly associate them with less-than-appetizing experiences in our daily lives. But there is an unexpected path to redemption: Robots. It turns out that flies have some features and abilities that can inform a new design for robotic systems.
“Unlike most vertebrates, flies can climb nearly any terrain,” says Ramdya. “They can stick to walls and ceilings because they have adhesive pads and claws on the tips of their legs. This allows them to basically go anywhere. That’s interesting also because if you can rest on any surface, you can manage your energy expenditure by waiting for the right moment to act.”
It was this vision of extracting the principles that govern fly behavior to inform the design of robots that drove the development of DeepFly3D, a motion-capture system for the fly Drosophila melanogaster, a model organism that is nearly ubiquitously used across biology.
In Ramdya’s experimental setup, a fly walks on top of a tiny floating ball – like a miniature treadmill – while seven cameras record its every movement. The fly’s top side is glued onto an unmovable stage so that it always stays in place while walking on the ball. Nevertheless, the fly “believes” that it is moving freely.
Credit: “DeepFly3D: the deep-learning way to design fly-like robots”, Nick Papageorgiou, EPFL
