Associate Professor John A. Perrone
Research: 2-D Motion Sensors
Background: Visual motion is one of the most important sources of information processed by our senses. The perception of movement underlies the ability of humans and animals to locomote and navigate in complex environments, perceive the depth and shapes of objects and to coordinate eye and limb movements. It is not surprising, therefore, that a large area of neuroscience is dedicated to the understanding of visual motion processing in biological systems and that the goal of many artificial vision systems is to emulate this ability. The picture that has emerged after a number of theoretical, psychophysical, and physiological studies (e.g., Watson & Ahumada, 1983; Burr, Ross & Morrone, 1986; Emerson, et al, 1987) is that motion is processed initially by sets of spatiotemporal tuned filters located in the primary visual cortex (area V1). The functioning of these cells is reasonably well understood and a number of computational models have been developed which simulate their basic characteristics (e.g., Watson & Ahumada, 1985, Adelson & Bergen, 1985).
These early models of V1 cells fall short in some key areas related to our ability to perceive motion. The V1 cells (in primates at least) cannot account for many higher level aspects of motion perception. For instance, their broad speed tuning is not compatible with our ability to finely discriminate the speeds of two objects differing by as little as 5% (McKee, 1981). In contrast, the cells at the next level of the primate visual pathway (area MT) are tightly tuned for speed (Maunsell & Van Essen, 1983; Perrone & Thiele, 2001). The challenge is to explain how the visual system refines the V1 cell properties to generate cells with MT-like properties that are more consistent with our perceptions.
We have already successfully modelled MST cells, thought to be involved in higher levels of motion processing such as the extraction of observer self-motion and depth (Perrone, 1992; Perrone & Stone, 1994, 1998; see also: Visual navigation). This model was based on template networks of MT-like motion sensors but it could only be tested using inputs from theoretical MT cells. A working model of MT cells would greatly enhance the power of this earlier model and provide a better understanding of the V1-MT-MST motion pathway.
Goal: To develop a model of a visual motion sensor constructed from V1 cells, with properties similar to those of MT cells.
Progress: A working sensor was initially developed which simulated the MT neuron speed and direction tuning properties (Perrone, 1997). However no data were available at that time to compare the model predictions against the MT speed tuning properties. In collaboration with Alexander Thiele (at the time doing a post-doctorate at the Salk Institute) we have since demonstrated that the MT neurons are truly speed tuned and that their spectral receptive fields are inseparable (Perrone & Thiele, 2001). Alex is now in the Psychology Dept. at the University of Newcastle upon Tyne. The figure below shows examples of some of these spectral receptive fields.
We have since gone on to develop a model (Weighted Intersection Mechaminsm or WIM) which explains how the spectral receptive fields seen in MT neurons can develop from V1 neuron properties (Perrone & Thiele, 2002). This model explains how the speed tuning observed in MT neurons can develop from V1 neurons. I have also extended the model to include direction tuning (Perrone, 2004). This model has many of the properties of MT neurons and is able to be tested with 2-dimensional image sequences. The WIM model has also been used to explain the speed tuning recently found by Priebe et al., (2006) in some complex, directional V1 neurons (Perrone, 2006). Current work on the model includes testing a system which extracts velocity information from a number of the MT-model sensors. The aim is to integrate the 2-D motion sensors (based on MT neurons) into the heading templates described in the human visual navigation section: Visual navigation). Initial tests with digital image sequences have been very encouraging and further tests are underway.
Adelson, E.H. & Bergen, J.R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America, A, 2, 284-299.
Burr, D. C., Ross, J. & Morrone, M. C. (1986). Seeing objects in motion. Proceedings of the Royal Society of London B, 227, 249-265.
Emerson, R.C., Citron, M.C., Vaughn, W.J. & Klein, S.A. (1987). Nonlinear directionally selective subunits in complex cells of cat striate cortex. Journal of Neurophysiology, 58, 33-65.
Kulikowski, J.J. & Tolhurst, D.J. (1973). Psychophysical evidence for sustained and transient detectors in human vision. Journal of Physiology, 232, 149-162.
Maunsell, J.H.R. & Van Essen, D.C. (1983). Functional properties of neurons in the middle temporal visual area of the Macaque monkey. I. Selectivity for stimulus direction, speed, orientation. Journal of Neurophysiology, 49, 1148-1167.
McKee, S.P. (1981) A local mechanism for differential velocity detection. Vision Research, 21, 491-500.
Perrone, J.A. (1992). Model for the computation of self-motion in biological systems. Journal of the Optical Society of America, A, 9, 177-194.
Perrone, J.A. & Stone, L.S. (1994). A model of self-motion estimation within primate extrastriate visual cortex. Vision Research, 34, 2917-2938.
Perrone, J. A. (1997). Extracting observer heading and scene layout from image sequences. Investigative Ophthalmology and Visual Science, 35, 2158
Perrone, J. A. & Stone, L.S. (1998) Emulating the visual receptive field properties of MST neurons with a template model of heading estimation. J.Neuroscience, 18, 5958-5975.
Perrone, J. A. & Thiele, A. (2001). Speed skills: measuring the visual speed analyzing properties of primate MT neurons. Nature Neuroscience, 4(5), 526-532.
Perrone, J. A., & Thiele, A. (2002). A model of speed tuning in MT neurons. Vision Research, 42, 1035-1051.
Perrone, J. A. (2004). A visual motion sensor based on the properties of V1 and MT neurons. Vision Research, 44, 1733-55.
Perrone, J. A. (2005). Economy of scale: A motion sensor with variable speed tuning. Journal of Vision, 5(1), 53-58.
Perrone, J.A. (2006). A single mechanism can explain the speed tuning properties of MT and V1 complex neurons. The Journal of Neuroscience, 24, 11987-11991.
Watson, A.B. & Ahumada, A.J., Jr (1983) A look at motion in the frequency domain. In Tsotsos J.K. (Ed.), Motion: Perception and Representation (pp 1-10). New York: Association for Computing Machinery.
Watson, A.B. & Ahumada, A.J., Jr (1985) Model of human motion sensing. Journal of the Optical Society of America, A, 2, 322-342.