Biologically Inspired Computer Vision (2015)

Part I
Fundamentals

Chapter 1
Introduction

Gabriel Cristóbal, Laurent U. Perrinet and Matthias S. Keil

And first he will see the shadows best, next the reflections of men and other objects in the water, and then the objects themselves; then he will gaze upon the light of the moon and the stars and the spangled heaven; and he will see the sky and the stars by night better than the sun or the light of the sun by day?

Plato, The Republic, Book VII

As state-of-the-art imaging technologies become more and more advanced, yielding scientific data of unprecedented detail and volume, the need to process and interpret all this data has made image processing and computer vision also increasingly important. Sources of data that have to be routinely dealt with for today's applications include video transmission, wireless communication, automatic fingerprint processing, massive databanks, non-weary and accurate automatic airport screening, and robust night vision to name a few. These technological advances were closely followed by the increase in computer power of traditional, von-Neumann architectures. However, a major concern is that such architectures are efficient only for specific computations such as retrieving an item in a database or to apply a precise sequence of operations. These usually fail when the application comes closer to biological vision, such as the categorization of images independently of changes in lighting, movements, and the environment or the imaging device. Surprisingly, while computers can handle most complex tasks (such as computing hash numbers, logarithms, and linear algebra), most of the time, the “simpler” the task is for humans (such as “find the animal in the image”), the more complex it becomes to implement on a computing device. Multidisciplinary inputs from other disciplines such as computational neuroscience, cognitive science, mathematics, physics, and biology have had, and will have, a fundamental impact on the progress of imaging and vision sciences. One main advantage of the study of biological organisms is to devise very different types of computational paradigms beyond the usual von Neumann architecture, for example, by implementing a neural network with a high degree of local connectivity, which may be more easily adapted to solve the problems of modern imaging.

This book serves as a comprehensive but rigorous reference in the area of biologically inspired computer vision modeling. Biologically inspired vision, that is, the study of visual systems of living beings, can be considered as a two-way process. On the one hand, living organisms can provide a source of inspiration for new computationally efficient and robust vision models, and on the other hand, machine vision approaches can provide new insights into understanding biological visual systems. Over the different chapters, this book covers a wide range from the fundamental to the more specialized topics. This book often follows Marr's classical, three-level approach to vision (computational, algorithmic, and hardware implementation) [1], but also goes beyond Marr's approach in the design of novel and more advanced vision sensors. In particular, the last section of the book provides an overview of a few representative applications and current state of the art of the research in this area.

The scope of this book somewhat overlaps that of a few other books published in this area, most of them corresponding to conference proceedings, for example, Refs [2, 3]. More recently, several special sessions on this same topic were organized at different workshops such as the ones on “Biologically consistent vision” in conjunction with the 2011 Computer Vision and Pattern Recognition Conference and “Biological and Computer Vision Interfaces” in conjunction with the 2012 European Conference on Computer Vision. The first monograph on this topic was published more than 20 years ago [4] and therefore does not reflect the latest advances in the field. A very good reference in the field is the book by Frisby and Stone [5] that provides the foundation for computational and physiological research on vision. Another relevant reference is the book by Petrou and Bharath although it is more focused toward specialized hardware both at low power and high speed [6]. A more recent reference in the area is the book by Pomplun and Suzuki [7] which is more focused on specific aspects of visual function such as attention, binocularity, or cortical structures. At the time of writing, it is worth mentioning a special issue on bioinspired imaging which highlights recent progress in the domain of vision and biological optics [8]. As a consequence, this book is valuable for both undergraduate and graduate students and also for specialized researchers as it presents information that is usually spread out over different sources into a single and comprehensive monograph.

1.1 Why Should We Be Inspired by Biology?

A central question in this area is to understand why and how is it useful to build technical systems with a biological inspiration. Or, in other words, why should biologically-motivated studies be useful for constructing artificial systems? One answer is that living systems are engineered to perfection by evolution, and thus they provide a seemingly inexhaustible source of inspiration for engineering. In this book, to cover few examples of how such biological findings will be of direct benefit, for example, for engineering new devices and sensors. More generally, one objective of this book is to cover “two-way process” analogy previously mentioned. On the one hand, we have chapters where findings from biology, neurobiology, or psychophysics guide the development of computer vision algorithms. On the other hand, we have more technical chapters that are motivated by biology.

In general, we can argue that Nature has been continuously a source of inspiration not only for constructing artificial vision systems but also in other domains such as materials science. The term “biomimetics” has been coined for the field of science and engineering that seeks to understand and to use nature as a model for copying, adapting, and inspiring concepts. An amazingly early design of Nature can be observed in the Erbenochile erbeni, a Devonian trilobite with calcite-derived compound eyes, allowing the animal to see not only 360 on the horizontal plane but also to cover a significant vertical area. Even more intriguing is the fact that their eyes have eye shades for blocking glare, suggesting that trilobites were diurnal and not nocturnal (see Figure 1.1) [9]. Often, the visual system of non-human organisms surpasses the performance of our visual system. Examples include ultraviolet vision of honey bees, infrared vision of some snakes, or the high temporal resolution of the blow fly visual system. All species were optimized by evolution over large time scales. This has led to efficient solutions to many of the challenges they face. It is therefore not surprising that humans always make efforts to use Nature as a model for innovation and problem solving.

Figure 1.1 The Erbenochile erbeni eyes are distributed in a vertical half-cylinder surface with 18 or 19 lenses per vertical row. Note the striking similarity of this natural eye with the CurvACE sensor presented in Figure 22 of Chapter 7. Image credit: Wikipedia

Another intriguing example are the eyes of the peacock mantis shrimp (Odontodactylus japonicus), which are capable of serial or parallel vision, trinocular vision, and depth perception; the color vision is extended to the ultraviolet and infrared ranges (see left cover image of this book). They also exhibit at least 16 different photoreceptors in addition to detection of polarized light and circular polarized light [10]. Inspired by the mantis shrimp's compound eye and polarized vision, an international team of researchers has recently built a miniature sensor that can detect subtle differences in early stage cancerous cells from in vivo endoscopy of the mouse colon [11]. The eyes of other animals use other strategies to solve predatory tasks. The jumping spider uses special staircase-shaped retinas to produce sharp and out-of-focus images simultaneously (see right cover image of this book). By comparing the focused and defocused images, the spider can estimate depth through unmoving eyes. Nagata et al. have developed a mathematical model that predicts the accuracy of the jumps for differentwavelengths [12]. These recent promising results could open up new avenues both in mathematical modeling and signal processing for more challenging applications. In the context of animal vision studies, Land and Nilsson's book [13] provides a fascinating comparative account of the evolution and function of animal eyes with emphasis on the description of the optical system. Furthermore, a very recent book on the same topic is the one by Cronin et al. [14]. As a consequence, biologically inspired computer vision is not only about mimicking the structure of animals' visual systems but also to gain insights from the vast range of different solutions that have emerged through natural selection to provide efficient solutions to vision-based problems.

1.2 Organization of Chapters in the Book

This book contains 18 chapters that have been organized in four different parts:

· Fundamentals

· Sensing

· Modeling

· Applications

The cross-links between the different chapters have been sketched in Figure 1.2. This book aims at providing an overview of bioinspired computer vision, starting from fundamentals to the most recent advances and applications in the field. The three chapters selected in the section “Applications” are good representatives of how the transfer of ideas from biology to computer vision can be done in practice. Figure 1.3shows a picture of a tag cloud that has been generated from the table of contents of the book.

Figure 1.2 Mindmap of the book contents. Cross-links between chapters have been indicated as thin lines.

Figure 1.3 Tag cloud of the abstracts and table of contents of the book. Credit: www.wordle.net

The structure of the book is as follows:

· Section 1: Fundamentals.

· Chapter 2 describes basic bioinspired vision technology, which has the aim of outperforming conventional frame-based vision systems in many fields of application. It provides an overview of biosensors and neuromorphic retinas.

· Chapter 3 describes how the retina is capable of much more complicated information processing than was initially thought.

· Chapter 4 describes how natural image statistics can be exploited to effectively improve visual inference in computer vision systems.

· Chapter 5 provides the basics of visual psychophysics, that is, how to measure the performance of observers in predetermined visual tasks.

· Section 2: Sensing.

· In Chapter 6, algorithms inspired by the compound eyes of insects, whose vision is based on sensing the polarization of light are described, illustrating how this can be modeled to enhance the visual perception of standard cameras.

· Chapter 7 describes how natural concepts for miniaturization could be imitated for building computer vision systems with perfect adaptation to factors such as small size, special tasks, and specific applications.

· Chapter 8 describes the basics of plenoptic sensing and how these new devices can extend the capabilities of current standard cameras.

· Section 3: Modeling.

· Chapter 9 describes Bayesian models as a useful modeling approach for describing perception and behavior at the computational level.

· Chapter 10 explains how neurodynamical models could be used not only as biologically inspired models for processing images but also for explaining perceptual phenomena.

· Chapter 11 presents models of bottom-up visual attention and their applications.

· Chapter 12 presents a review of several recent studies focusing on the integration of retinal and extra-retinal information for visual motion processing and human tracking behavior.

· Chapter 13 describes cortical models for image recognition mainly based on the HMAX architecture of Riesenhuber and Poggio [15].

· Chapter 14 describes how bioinspired approaches may be applied to computer vision problems using predictive coding schemes focusing on sparse models as simple and efficient instances of such techniques.

· Chapter 15 describes methods for extracting and representing key points motivated from a biological standpoint.

· Section 4: Applications.

· Chapter 16 describes how by mimicking neural processes of nocturnal animals, efficient computer vision algorithms can be devised.

· Chapter 17 provides an overview of elementary motion detectors (EMDs) oriented to computer vision applications when resources available are limited (e.g., power consumption).

· Finally, Chapter 18 describes in detail a bioinspired model (ViSTARS) oriented to visually guided navigation in a cluttered world.

1.3 Conclusion

Biological vision shows excellence in terms of performance and robustness. Following one of the recommendations of the book referees, one of the aims of the book is to make it multidisciplinary, although perhaps in the future, the topic of biologically inspired computer vision could become a single discipline by itself. One of the reasons of the resurging interest in the topic of the book has been both the availability of massive computing power (e.g., cloud computing) and high-performant computing power (GPU, FPGA, etc). This has been illustrated in the Chapters 16 and 17 of this book.

For the reader's convenience, there is an accompanying website with supplementary material at bicv.weebly.com. It contains selected MATLAB and Python codes, testing images and errata.

Acknowledgments

We would like to express our appreciation for the quality of the chapters delivered by the authors and for their efforts to keep the chapter length within the limits given. This project could not have been achieved without the valuable contributions made by a significant number of experts in the field from both the academia and industry. We are grateful to their willingness to contribute to this groundbreaking resource. We would like to extend thanks to all the Wiley VCH members who have helped us manage the project; in particular, thanks are due to Val Moliere for her enthusiastic support. We want to express also our gratitude to Roy L. Caldwell and Thomas Shahan for providing us the macro pictures of the book cover (Odontodactylus japonicus male (left) and Phidippus audax male I(right)).

References

1. 1. Marr, D. (2010) Vision: A Computational Investigation into the Human Representation and Processing of Visual Information, MIT Press.

2. 2. Lee, S., Bultoff, H., and Poggio, T. (eds) (2000) Biologically Motivated Computer Vision, Lecture Notes in Computer Science, vol. 1811, Springer-Verlag.

3. 3. Bultoff, H. et al. (eds) (2002) Biologically Motivated Computer Vision, Lecture Notes in Computer Science, vol. 2525, Springer-Verlag.

4. 4. Overington, I. (1992) Computer Vision: A Unified Biologically Inspired Approach, Elsevier Science.

5. 5. Frisby, J. and Stone, J. (2010) Seeing, 2nd edn, MIT Press.

6. 6. Petrou, M. and Bharath, A. (eds) (2008) Next Generation Artificial Vision Systems: Reverse Engineering the Human Visual System, Artech House.

7. 7. Pomplun, M. and Suzuki, J. (eds) (2012) Developing and Applying Biologically-Inspired Vision Systems: Interdisciplinary Concepts, IGI Global.

8. 8. Larkin, W., Etienne-Cummings, R., and Van der Spiegel, J. (2014) Bioinspired imaging: discovery, emulation and future prospects. Proc. IEEE, 102 (10), 1404–1410.

9. 9. Fortey, R. and Chatterton, B. (2003) A Devonian trilobite with an eyeshade. Science, 301, 1689.

10.10. Horvath, G. (ed.) (2014) Polarized Light and Polarization Vision in Animal Sciences, 2nd edn, Springer.

11.11. York, T. et al. (2014) Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications. Proc. IEEE, 102, 1450–1469.

12.12. Nagata, T. et al. (2012) Depth perception from image defocus in a jumping spider. Science, 335 (6067), 469–471.

13.13. Land, M. and Nilsson, D. (2012) Animal Eyes, 2nd edn, Oxford Animal Biology, Oxford University Press.

14.14. Cronin, T. et al. (2014) Visual Ecoloy, Princeton University Press.

15.15. Riesenhuber, M. and Poggio, T. (1999) Hierarchical models of object recognition in cortex. Nat. Neurosci., 2, 1019–1025.