Optic Flow-Based Robotics
Julien Serres
Aix-Marseille University, CNRS, ISM, Inst Movement Sci Marseille, France
Search for more papers by this authorFranck Ruffier
Aix-Marseille University, CNRS, ISM, Inst Movement Sci Marseille, France
Search for more papers by this authorJulien Serres
Aix-Marseille University, CNRS, ISM, Inst Movement Sci Marseille, France
Search for more papers by this authorFranck Ruffier
Aix-Marseille University, CNRS, ISM, Inst Movement Sci Marseille, France
Search for more papers by this authorAbstract
Flying insects and birds are able to fly smartly in an unpredictable environment. Many animals have been found to rely mainly on optic flow. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Optic flow is particularly useful for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment. However, this does not require any actual measurement of either speed or distance. Optic flow is therefore suitable for various navigational tasks, such as takeoff or landing along vertical or longitudinal axes, terrain following, speed control in a cluttered environment, lateral and frontal obstacle avoidance, and visual odometry. This article focuses on feedback loops that use optic flow to control robots in the same way as the Gibsonian approach, which sometimes enhances robot perception, by a distance or speed measurement, even though the direct measurement of distance or linear speed does not exist in flying insects and birds. Optic flow is likely to be one of the most important visual cues that could be used during the next decade to enhance robot reactivity in unpredictable environments. Conversely, the biorobotic approach can therefore help to better understand how flying animals can move smartly in such an environment.
Bibliography
- 1 G. K. Taylor and H. G. Krapp. Adv. Insect Phys. 2007, 34, pp 231–316.
- 2 H. G. Krapp, R. Hengstenberg et al. Nature 1996, 384, pp 463–466.
- 3 N. Franceschini, F. Ruffier, J. Serres, and S. Viollet. Optic Flow Based Visual Guidance: From Flying Insects to Miniature Aerial Vehicles. INTECH Open Access Publisher, 2009.
- 4 M. V. Srinivasan. Physiol. Rev. 2011, 91, pp 413–460.
- 5 N. Franceschini. Proc. IEEE 2014, 102, pp 751–781.
- 6 C. T. David. J. Comp. Physiol. A 1982, 147, pp 485–493.
- 7 W. Kirchner and M. Srinivasan. Naturwissenchaften 1989, 76, pp 281–282.
- 8 M. Srinivasan, M. Lehrer, W. Kirchner, and S. Zhang. Vis. Neurosci. 1991, 6, pp 519–535.
- 9 E. Baird, M. V. Srinivasan, S. Zhang, and A. Cowling. J. Exp. Biol. 2005, 208, pp 3895–3905.
- 10 M. Ibbotson. Proc. R. Soc. Lond. B Biol. Sci. 2001, 268, pp 2195–2201.
- 11 J. Gibson. The Perception of the Visual World. Houghton Mifflin: Boston, 1950.
- 12 T. C. Whiteside and G. Samuel. Nature 1970, 225, pp 94–95.
- 13 K. Nakayama and J. Loomis. Perception 1974, 3, pp 63–80.
- 14 J. J. Koenderink. Vis. Res. 1986, 26, pp 161–179.
- 15 J. J. Koenderink and A. J. van Doorn. Biol. Cybern. 1987, 56, pp 247–254.
- 16
E. Buchner. In
Photoreception and Vision in Invertebrates, NATO ASI Series, Series A: Life Science;
Springer,
1984;
vol. 74,
pp 561–621.
10.1007/978-1-4613-2743-1_16 Google Scholar
- 17
R. Moeckel and
S.-C. Liu. In
Flying Insects and Robots.
Springer:
Berlin,
2009;
pp 101–114.
10.1007/978-3-540-89393-6_8 Google Scholar
- 18 D. Floreano, R. Pericet-Camara, S. Viollet, F. Ruffier, A. Brückner, R. Leitel, W. Buss, M. Menouni, F. Expert, R. Juston et al. Proc. Natl. Acad. Sci. 2013, 110, pp 9267–9272.
- 19 H. Chao, Y. Gu, and M. Napolitano. J. Intell. Robot. Syst. 2014, 73, pp 361–372.
- 20 D. Lucas and T. Kanade. An Iterative Image Registration Technique with an Application to Stereo Vision, in Proc. 7th International Joint Conference on Artificial Intelligence (IJCAI); Vancouver, BC, 1981; pp 674–679.
- 21 B. K. P. Horn and B. Schunck. Artif. Intell. 1981, 17, pp 185–203.
- 22 C. Blanes. Appareil visuel elementaire pour la navigation a vue d'un robot mobile autonome. Master thesis in Neurosciences (DEA in French), University of Aix-Marseille II, Marseille, 1986.
- 23 J.-M. Pichon, C. Blanes, and N. Franceschini. Visual Guidance of a Mobile Robot Equipped with a Network of Self-Motion Sensors, in Proc. SPIE Conference on Mobile Robots IV, vol. 1195; W. J. Wolfe and W. H. Chun, Eds.; SPIE: Bellingham, 1989, pp 44–53.
- 24 N. Franceschini, J. M. Pichon, and C. Blanes. Philos. Trans. Biol. Sci. 1992, 337, pp 283–294.
- 25 M. F. Land and T. Collett. J. Comp. Physiol. 1974, 89, pp 331–357.
- 26 C. Schilstra and J. Hateren. J. Exp. Biol. 1999, 202, pp 1481–1490.
- 27 L. F. Tammero and M. H. Dickinson. J. Exp. Biol. 2002, 205, pp 327–343.
- 28 R. Kern, N. Boeddeker, L. Dittmar, and M. Egelhaaf. J. Exp. Biol. 2012, 215, pp 2501–2514.
- 29 D. Coombs and K. Roberts. Bee-Bot: Using Peripheral Optical Flow to Avoid Obstacles, in Proc. SPIE on Intelligent Robots and Computer Vision XI; SPIE, 1992; vol. 1825, pp 714–721.
- 30 D. Coombs and K. Roberts. Centering Behavior Using Peripheral Vision, in 1993 Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'93); IEEE, 1993, pp 440–445.
- 31
S. Griffiths,
J. Saunders,
A. Curtis,
B. Barber,
T. McLain, and
R. Beard. In
Advances in Unmanned Aerial Vehicles.
Springer,
2007,
pp 213–244.
10.1007/978-1-4020-6114-1_7 Google Scholar
- 32 M. A. Garratt and J. S. Chahl. J. Field Robot. 2008, 25, pp 284–301.
- 33 A. Beyeler, J.-C. Zufferey, and D. Floreano. Autonom. Robots 2009, 27, pp 201–219.
- 34 F. Mura and N. Franceschini. In From Animals to Animats III; D. Cliff et al., Eds.; MIT Press: Cambridge, 1994; pp 91–99.
- 35 F. Ruffier and N. Franceschini. Robot. Autonom. Syst. 2005, 50, pp 177–194.
- 36 N. Franceschini, F. Ruffier, and J. Serres. Curr. Biol. 2007, 17, pp 329–335.
- 37 J. Serres, D. Dray, F. Ruffier, and N. Franceschini. Autonom. Robots 2008, 25, pp 103–122.
- 38 J. Santos-Victor, G. Sandini, F. Curotto, and S. Garibaldi. Divergent Stereo for Robot Navigation: Learning from Bees, in 1993 Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'93); IEEE, 1993; pp 434–439.
- 39 J. Santos-Victor, G. Sandini, F. Curotto, and S. Garibaldi. Int. J. Comput. Vis. 1995, 14, pp 159–177.
- 40 A. P. Duchon and W. H. Warren. Robot Navigation from a Gibsonian Viewpoint, in Proc. 1994 IEEE International Conference on Systems, Man, and Cybernetics: Humans, Information and Technology; IEEE, 1994; vol. 3, pp 2272–2277.
- 41 S. V. K. Weber and M. Srinivasan. In From Living Eyes to Seeing Machines. Oxford University Press: Oxford, UK, 1997, pp 226–248.
- 42 A. Dev, B. Krose, and F. Groen, Navigation of a Mobile Robot on a Temporal Development of the Optic Flow, in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); Grenoble, France, 1997; pp 558–563.
- 43 G. Baratoff, C. Toepfer, and H. Neumann. Biol. Cybern. 2000, 83, pp 199–209.
- 44 R. Carelli, C. Soria, O. Nasisi, and E. Freire. Stable AGV Corridor Navigation with Fused Vision-Based Controls Signals, in Proc. 28th IEEE Conference of Industrial Electronics Society (IECON); Sevilla, Spain, 2002; vol. 3, pp 2433–2438.
- 45 C. McCarthy and N. Barnes. Performance of Temporal Filters for Optical Flow Estimation in Mobile Robot Corridor Centring and Visual Odometry, in Proc. 2003 Australasian Conference on Robotics & Automation, 2003.
- 46 A. Argyros, D. Tsakiris, and C. Groyer. IEEE Robot. Autom. Mag. 2004, 11, pp 21–30.
- 47 S. Hrabar, G. Sukatme, P. Corke, K. Usher, and J. Roberts. Combined Optic-Flow and Stereo-Based Navigation of Urban Canyons for a UAV, in Proc. IEEE/RSJ International Conference on Intellignent Robots and Systems (IROS); Edmonton, Alberta, Canada, 2005; pp 3309–3316.
- 48 K. Souhila and A. Karim. Int. J. Adv. Robot. Syst. 2007, 4, pp 13–16.
- 49 J. S. Humbert, A. Hyslop, and M. Chinn. Experimental Validation of Wide-Field Integration Methods for Autonomous Navigation, in Proc. 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'07); IEEE, 2007; pp 2144–2149.
- 50 F. Iida. Goal-Directed Navigation of an Autonomous Flying Robot Using Biogically Inspired Cheap Vision, in Proc. 32nd International Symposium on Robotics (ISR); 2001; pp 1404–1409.
- 51 J. S. Humbert, R. M. Murray, and M. H. Dickinson. Sensorimotor Convergence in Visual Navigation and Flight Control Systems, in Proc. 16th IFAC World Congress; Praha, Czech Republic, 2005.
- 52 L. Muratet, S. Doncieux, Y. Briere, and J. Meyer. A Contribution to Vision-Based Autonomous Helicopter Flight in Urban Environments. Robot. Autonom. Syst. 2005, 50, pp 195–209.
- 53 S. Griffiths, J. Saunders, A. Curtis, B. Barber, T. McLain, and R. Beard. Robot. Autom. Mag. 2006, 13, pp 34–43.
- 54 S. Hrabar and G. Sukatme. Optimum Camera Angle for Optic Flow-Based Centring Response, in Proc. IEEE International Conference on Robotics and Automation (ICRA); Beijing, China, 2006; pp 3922–3927.
- 55 D. N. Lee. Perception 1976, 5, pp 437–459.
- 56 J. Serres, F. Ruffier, and N. Franceschini. Biomimetic Visual Navigation in a Corridor: To Centre or Not To Centre, in Proc. International Mediterranean Modeling Multiconference (I3M'05); 2005; pp 91–97.
- 57 J. Serres, F. Ruffier, S. Viollet, and N. Franceschini. Int. J. Adv. Robot. Syst. 2006, 3, pp 147–154.
- 58 J.-C. Zufferey and D. Floreano. Toward 30-Gram Autonomous Indoor Aircraft: Vision-Based Obstacle Avoidance and Altitude Control, in Proc. 2005 IEEE International Conference on Robotics and Automation (ICRA'05); IEEE, 2005; pp 2594–2599.
- 59 M. A. Lewis. Visual Navigation in a Robot Using Zig-Zag Behavior, in 1997 Conference on Advances in Neural Information Processing Systems (NIPS'97); MIT Press: Cambridge, MA, 1997; pp 822–828.
- 60 J. Serres, F. Ruffier, and N. Franceschini. Two Optic Flow Regulators for Speed Control and Obstacle Avoidance, in 2006 IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob'06); IEEE, 2006; pp 750–757.
- 61 F. L. Roubieu, J. Serres, N. Franceschini, F. Ruffier, and S. Viollet. A Fully-Autonomous Hovercraft Inspired by Bees: Wall Following and Speed Control in Straight and Tapered Corridors, in 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO); IEEE, 2012; pp 1311–1318.
- 62 F. L. Roubieu, J. R. Serres, F. Colonnier, N. Franceschini, S. Viollet, and F. Ruffier. Bioinspir. Biomim. 2014, 9, p 036003.
- 63 N. Martin and N. Franceschini. Obstacle Avoidance and Speed Control in a Mobile Vehicle Equipped with a Compound Eye, in Proc. Intelligent Vehicles' 94 Symposium; IEEE, 1994; pp 381–386.
- 64 M. V. Srinivasan, J. S. Chahl, K. Weber, S. Venkatesh, M. G. Nagle, and S.-W. Zhang. Robot. Autonom. Syst. 1999, 26, pp 203–216.
- 65 J. S. Humbert and A. M. Hyslop. IEEE Trans. Robot. 2010, 26, pp 121–130.
- 66 M. Srinivasan, S. Zhang, M. Lehrer, and T. Collett. J. Exp. Biol. 1996, 199, pp 237–244.
- 67 J. Serres, M. G. F. Ruffier, and N. Franceschini. Naturwissenschaften 2008, 95, pp 1181–1187.
- 68 J. S. Humbert. Bio-Inspired Visuomotor Convergence in Navigation and Flight Control Systems. Ph.D. dissertation, California Institute of Technology, 2005.
- 69 J. S. Humbert, R. M. Murray, and M. H. Dickinson. A Control-Oriented Analysis of Bio-Inspired Visuomotor Convergence, in Proc. 44th IEEE Conference on Decision and Control, 2005 and 2005 European Control Conference (CDC-ECC'05); IEEE, 2005; pp 245–250.
- 70 J. S. Humbert, R. M. Murray, and M. H. Dickinson. Pitch-Altitude Control and Terrain Following Based on Bio-Inspired Visuomotor Convergence, in AIAA Conference on Guidance, Navigation and Control, vol. AIAA 2005-6280; San Francisco, CA, 2005.
- 71 K. Hausen. Biol. Cybern. 1982, 45, pp 143–156.
- 72 H. G. Krapp, B. Hengstenberg, and R. Hengstenberg. J. Neurophysiol. 1998, 79, pp 1902–1917.
- 73 A. Borst and J. Haag. J. Comp. Physiol. A 2002, 188, pp 419–437.
- 74 A. Hyslop, H. G. Krapp, and J. S. Humbert. Biol. Cybern. 2010, 103, pp 353–364.
- 75 J. Conroy, G. Gremillion, B. Ranganathan, and J. S. Humbert. Autonom. Robots 2009, 27, pp 189–198.
- 76
N. Franceschini,
A. Riehle, and
A. Le Nestour. In
Facets of Vision.
Springer,
1989;
pp 360–390.
10.1007/978-3-642-74082-4_17 Google Scholar
- 77 A. Hyslop, H. G. Krapp, and J. S. Humbert. Biol. Cybern. 2010, 103, pp 353–364.
- 78 J. Keshavan, G. Gremillion, H. Escobar-Alvarez, and J. Humbert. Bioinspir. Biomim. 2014, 9, p 025011.
- 79 J. Keshavan, G. Gremillion, H. Alvarez-Escobar, and J. S. Humbert. Int. J. Micro Air Veh. 2015, 7, pp 111–124.
- 80 B. Herissé, T. Hamel, R. Mahony, and F.-X. Russotto. IEEE Trans. Robot. 2012, 28, pp 77–89.
- 81 R. Nelson and J. Aloimonos. Using Flow Field Divergence for Obstacle Avoidance in Visual Navigation, in Science Applications International Corp, Proceedings: Image Understanding Workshop, vol. 2, 1988.
- 82 N. Ancona and T. Poggio. Optical Flow from 1D Correlation: Application to a Simple Time-to-Crash Detector, in Proc. 4th International Conference on Computer Vision; Berlin, Germany, 1993; pp 209–214.
- 83 S. B. I. Badia, U. Bernardet, and P. F. Verschure. PLoS Comput. Biol. 2010, 6, p e1000701.
- 84 J.-C. Zufferey and D. Floreano. IEEE Trans. Robot. 2006, 22, pp 137–146.
- 85 M. B. Reiser and M. H. Dickinson. Philos. Trans. R. Soc.Lond. A 2003, 361, pp 2267–2285.
- 86 A. Beyeler, J.-C. Zufferey, and D. Floreano. 3D Vision-Based Navigation for Indoor Microflyers, in 2007 IEEE International Conference on Robotics and Automation; IEEE, 2007; pp 1336–1341.
- 87 G. Barrows, C. Neely, and K. Miller. In Fixed and Flapping Wing Aerodynamics for Micro Air Vehicle Applications: Progress in Astronautics and Aeronautics. AIAA: Bellingham, 2001; vol. 195, pp 557–574.
- 88 W. E. Green, P. Y. Oh, and G. Barrows. Flying Insect Inspired Vision for Autonomous Aerial Robot Maneuvers in Near-Earth Environments, in Proc. 2004 IEEE International Conference on Robotics and Automation (ICRA'04); IEEE, 2004; vol. 3, pp 2347–2352.
- 89 J. P. Lindemann, H. Weiss, R. Möller, and M. Egelhaaf. Biol. Cybern. 2008, 98, pp 213–227.
- 90 M. Rezaei and F. Saghafi. Aircr. Eng. Aerosp. Technol. 2011, 83, pp 85–93.
- 91 J. R. Serres and F. Ruffier. J. Bionic Eng. 2015, 12, pp 47–60.
- 92 M. Srinivasan, S. Zhang, J. Chahl, E. Barth, and S. Venkatesh. Biol. Cybern. 2000, 83, pp 171–183.
- 93 J. Chahl, M. Srinivasan, and S. Zhang. Int. J. Rob. Res. 2004, 23, pp 101–110.
- 94 T. Netter and N. Franceschini. In Proc. IEEE Conference on Intelligent Robots and Systems (IROS): Progress in Astronautics and Aeronautics. AIAA: Lausanne, Switzerland, 2002; vol. 195, pp 129–134.
- 95 F. Ruffier, S. Viollet, S. Amic, and N. Franceschini. Bio-Inspired Optical Flow Circuits for the Visual Guidance of Micro Air Vehicles, in Proceedings of the 2003 International Symposium on Circuits and Systems (ISCAS'03); IEEE, 2003; vol. 3, pp III–846.
- 96 F. Ruffier and N. Franceschini. J. Intell. Robot. Syst. 2015, 79, pp 275–293.
- 97 S. Zhang, W. Xiang, L. Zili, and M. Srinivasan. Vis. Neurosci. 1990, 4, pp 379–386.
- 98 B. Hérissé, T. Hamel, R. Mahony, and F.-X. Russotto. Autonom. Robots 2010, 29, pp 381–399.
- 99 G. Portelli, J. Serres, F. Ruffier, and N. Franceschini. J. Physiol. Paris 2010, 104, pp 27–39.
- 100 F. Expert and F. Ruffier. Bioinspir. Biomim. 2015, 10, p 026003.
- 101 C. Sabo, A. Cope, K. Gurny, E. Vasilaki, and J. A. Marshall. Bio-Inspired Visual Navigation for a Quadcopter Using Optic Flow. AIAA Infotech@ Aerospace, 2016; p 0404.
- 102 D. Izzo and G. D. Croon. J. Guid. Control Dyn. 2012, 35, pp 1362–1367.
- 103 F. Kendoul. Int. J. Robot. Res. 2014, 33, pp 237–267.
- 104 M. T. Alkowatly, V. M. Becerra, and W. Holderbaum. Int. J. Model. Identif. Control 2015, 24, pp 29–41.
- 105 G. C. de Croon. Bioinspir. Biomim. 2016, 11, p 016004.
- 106 E. Baird, N. Boeddeker, M. R. Ibbotson, and M. V. Srinivasan. Proc. Natl. Acad. Sci. 2013, 110, pp 18686–18691.
- 107 P. Serra, R. Cunha, T. Hamel, D. Cabecinhas, and C. Silvestre. Landing on a Moving Target Using Image-Based Visual Servo Control, in 2014 IEEE 53rd Annual Conference on Decision and Control (CDC); IEEE, 2014; pp 2179–2184.
- 108 M. V. Srinivasan. J. Comp. Physiol. A 2014, 200, pp 563–573.
- 109 N. Nourani-Vatani, J. Roberts, and M. V. Srinivasan. Practical Visual Odometry for Car-Like Vehicles, in 2009 IEEE International Conference on Robotics and Automation (ICRA'09); IEEE, 2009; pp 3551–3557.
- 110 H. Dahmen and H. A. Mallot. Sensors 2014, 14, pp 21045–21064.
- 111 R. Strydom, S. Thurrowgood, and M. Srinivasan. Visual Odometry: Autonomous UAV Navigation Using Optic Flow and Stereo, in Australasian Conference on Robotics and Automation (ACRA), 2014; pp 1–10.
- 112 M. B. Milde, O. J. Bertrand, R. Benosmanz, M. Egelhaaf, and E. Chicca. Bioinspired Event-Driven Collision Avoidance Algorithm Based on Optic Flow, in 2015 International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP); IEEE, 2015; pp 1–7.
- 113 A. Denuelle and M. V. Srinivasan. Snapshot-Based Navigation for the Guidance of UAS, in Australasian Conference on Robotics and Automation (ACRA'15), 2015.
- 114 A. Briod, J.-C. Zufferey, and D. Floreano. Autonom. Robots 2016, 40, pp 789–803.
- 115 S. Mafrica, S. Godiot, M. Menouni, M. Boyron, F. Expert, R. Juston, N. Marchand, F. Ruffier, and S. Viollet. Opt. Express 2015, 23, pp 5614–5635.
Citing Literature
Wiley Encyclopedia of Electrical and Electronics Engineering
Browse other articles of this reference work: