Motion Perception

Abstract

Motion perception, a form of visual perception, is the process by which the visual system recovers the speed and direction of objects, and of the observer, from the changing pattern of light on the retina. Its central difficulty is that a local motion measurement is ambiguous: a moving contour seen through a small receptive field specifies only the component of motion perpendicular to itself, so true motion must be inferred by combining many such measurements. The article traces this recovery from the motion-energy computation of low-level detectors, through the direction-selective neurons of primary visual cortex and their integration in area MT, to the linking of MT activity to perceptual decisions and the striking case of biological motion. Three interactive demonstrations model the aperture problem, the speed tuning of an MT neuron, and the reading of a noisy motion signal from a population of detectors.

Keywords: motion perception, aperture problem, motion energy, area MT, biological motion

Motion perception is the recovery of movement from a stream of images, and it is as constructive as any other part of vision. The retina registers only changing light at each location; that the world is full of objects moving at definite speeds in definite directions is a conclusion the brain reaches, not a fact it receives. The problem is hard because the local signal is ambiguous and the global percept is not, so the visual system must pool many uncertain local measurements into a single confident estimate of what is moving where. This article follows that pooling from the earliest receptive fields that first register local motion, up through the cortical area that integrates them and the experiments that tied its neurons to what an animal actually perceives, to the recovery of a walking figure from nothing but moving dots.

What Motion Perception Is

Motion perception is the set of processes that recover, from the time-varying image on the retina, the velocities of the objects in a scene and of the observer moving through it. It is a distinct achievement rather than a by-product of seeing successive positions, because the visual system contains dedicated machinery that responds to motion itself: a smoothly moving spot excites motion-sensitive neurons that a spot jumped between the same positions may not, and prolonged viewing of one direction leaves a vivid aftereffect of motion in the opposite direction, the waterfall illusion, which shows that motion is coded by channels that can be selectively fatigued (Burr & Thompson, 2011). Motion is therefore a primary visual dimension, on a footing with orientation and color, not an inference drawn after the fact from a sequence of static snapshots.

The functions that motion perception serves are correspondingly basic. It segments a scene, because a region that moves together is almost always a single object, so common motion acts as a grouping cue that binds an object's features and separates it from its background. It supports the control of action and locomotion, because the expanding flow of the image as one moves forward specifies heading and time to contact. And it recovers three-dimensional structure, because the relative motion of features as an object or observer moves reveals the object's shape, the kinetic depth effect in which a rotating tangle of lines is seen as a rigid form the instant it begins to move (Wallach & O'Connell, 1953). These uses explain why motion is extracted early, quickly, and by specialized circuitry.

The Aperture Problem

The founding difficulty of motion perception is that a local detector cannot measure true motion. A neuron early in the visual system sees the world through a small receptive field, in effect a small aperture, and a moving straight contour viewed through such an aperture provides no information about motion along its own length; only the component of motion perpendicular to the contour can be measured, because motion parallel to the contour leaves the image within the aperture unchanged (Nakayama, 1985). A single oriented detector responding to a moving edge is therefore consistent with an entire family of true motions, all sharing the same perpendicular component and differing in the unseen parallel component, and it cannot by itself say which of them is occurring.

The visual system solves this ambiguity by combining measurements from detectors tuned to different orientations. Where two differently oriented contours belong to the same object, their perpendicular components jointly constrain a single true velocity, the intersection of the two families of possible motions, and pooling across many local detectors recovers the object's real direction (Simoncelli & Heeger, 1998). This two-stage logic, local measurement of ambiguous components followed by global integration into a true velocity, is the organizing principle of cortical motion processing and the reason the system needs an area devoted to integration beyond the local detectors themselves. The demonstration below makes the ambiguity concrete: a grating drifts behind a circular aperture, and the reader sees that the same perpendicular motion is consistent with many true directions until a second constraint is added.

The Motion-Energy Model

How a neuron measures even the perpendicular component of motion was made precise by the motion-energy model. Its insight is that a feature moving through space is, in the joint domain of space and time, a structure tilted in space-time: plotting position against time, a rightward-moving edge traces a line sloping one way and a leftward-moving edge a line sloping the other, so the direction of motion becomes an orientation in space-time that an appropriately oriented filter can detect (Adelson & Bergen, 1985). A motion-energy unit is built by combining filters that are oriented in space and time and offset in phase, squaring and summing their outputs so that the unit responds to the amount of motion energy in its preferred direction regardless of the exact pattern, and subtracting the energy for the opposite direction yields an opponent signal that reports direction and speed.

An equivalent formulation, developed in parallel, casts the same computation as a pair of linear filters separated in space whose outputs are multiplied after one is delayed, the correlation detector inherited from earlier work on insect vision, which likewise responds preferentially when a feature traverses the two locations in the right order and timing (Watson & Ahumada, 1985). The motion-energy and correlation formulations turn out to be mathematically closely related, two descriptions of one detector, and together they provide the standard account of how the first stage of the visual system converts a moving image into a neural signal for direction. What they compute is a local, still somewhat ambiguous, motion measurement, exactly the input the next stage must integrate. Figure 1 shows the central intuition: a feature moving through space traces an oriented streak in the joint space-time domain, and its direction becomes a slope that an oriented filter can detect.

Cortical Motion Processing

The first cortical neurons selective for the direction of motion are found in primary visual cortex, area V1, where a subset of cells fire to an edge moving one way through their receptive field but not the other, implementing something close to a motion-energy computation on a local patch of the image. These direction-selective V1 neurons are the physiological realization of the low-level detectors, but they inherit the aperture problem in full: because each responds to an oriented contour within a small receptive field, it signals only the component of motion perpendicular to that contour, and its preferred direction for a plaid made of two gratings tracks the individual gratings rather than their combined motion (Rust et al., 2006). V1 thus supplies the ambiguous local measurements but does not resolve them.

Resolution requires a further stage, and the anatomy provides one in a set of areas beyond V1 that pool these signals. A direction-selective V1 cell projects to the middle temporal area, and the receptive fields there are large enough, and drawn from enough differently oriented inputs, to integrate the local components into a measurement of true velocity. The systematic study of this projection established that motion is processed along a distinct cortical route, an early example of the broader principle that different visual attributes are analyzed in partly separate streams (Maunsell & Van Essen, 1983). The transformation from V1 to the middle temporal area, from local and ambiguous to global and veridical, is the heart of cortical motion perception.

Area MT and the Integration of Motion

The middle temporal area, called MT in the macaque and V5 in humans, is the cortical region most specialized for motion. It was identified as a small area in the superior temporal sulcus in which almost every neuron is direction-selective, an unusually pure functional specialization that marked it as a dedicated motion area (Zeki, 1974). Its neurons are organized into columns of common preferred direction, much as V1 is organized for orientation, and they are tuned not only for direction but for speed, so that MT carries a map of image velocity. Human functional imaging localized a homologous motion-selective area, confirming that the primate arrangement is conserved and giving a target for the study of motion perception in people (Tootell et al., 1995).

What makes MT more than a relay is that many of its neurons solve the aperture problem. Presented with a plaid of two superimposed gratings drifting in different directions, a class of MT cells, the pattern cells, respond to the single coherent direction in which the plaid is seen to move, whereas V1 cells and other MT cells, the component cells, respond to the directions of the individual gratings; the pattern cells have therefore integrated the ambiguous components into true motion (Rust et al., 2006). This integration can be reproduced by a two-stage model in which direction-selective inputs are linearly combined and then divisively normalized, the same normalization that appears throughout cortex, showing that the pattern computation follows from a small set of canonical operations (Simoncelli & Heeger, 1998). MT neurons are also tuned for binocular disparity, which is why the area contributes to the recovery of depth from motion as well (Born & Bradley, 2005). The demonstration below shows the speed side of this tuning, letting the reader vary a stimulus speed and read off the response of a model MT neuron tuned to a preferred speed.

Linking Neurons to Perception

Area MT became a model system for cognitive neuroscience because its link to perception could be tested directly, and the tests are among the clearest demonstrations that a specific population of neurons underlies a specific percept. Removing or inactivating MT selectively impairs motion perception: after such lesions an animal's threshold for detecting the direction of a noisy motion display rises severalfold while its sensitivity to stationary contrast is spared, showing that MT is necessary for motion perception in particular rather than for vision in general (Newsome & Paré, 1988). MT is thus not merely correlated with motion but required for it.

Two further experiments tightened the link from necessity to sufficiency and to trial-by-trial correspondence. Recording from single MT neurons while a monkey judged the direction of a random-dot display, the sensitivity of an individual neuron was found to rival that of the whole animal, so that one well-tuned cell carried nearly as much information about the motion as the perceptual decision itself (Britten et al., 1992). And electrically microstimulating a column of MT neurons that preferred a given direction biased the monkey's choices toward that direction, as though the added neural signal were added motion, which showed that MT activity does not merely reflect the percept but helps cause it (Salzman et al., 1990). Table 1 sets these converging lines of evidence side by side.

Global and Pattern Motion

The experiments that tied MT to perception relied on a stimulus that isolates global motion, the random-dot kinematogram, and the stimulus deserves its own account because it defines what integration means. In such a display a fraction of the dots move coherently in one direction while the rest are replotted at random, and the coherence, the proportion moving together, sets the strength of the global motion signal; the observer must pool the motion of many dots over space and time to recover the net direction, because no single dot is reliable (Newsome et al., 1989). Threshold coherence, the smallest fraction that supports a reliable direction judgment, measures the efficiency of this pooling, and it is low, a few percent, which shows how effectively the system integrates weak, distributed motion evidence.

Pooling of this kind is the perceptual counterpart of the pattern computation in MT, and the two together define a hierarchy of motion. Local detectors report ambiguous components; MT pattern cells integrate components across orientation into true velocity at each location; and global-motion mechanisms pool velocity across space and time to extract a coherent direction from noise. Each stage trades spatial detail for reliability, converting many uncertain local signals into a confident global estimate, which is the general strategy by which motion perception defeats the ambiguity it starts with (Rust et al., 2006). The demonstration below realizes the read-out stage: the reader sets the coherence of a motion signal and sees how reliably a population of detectors, and so an observer, can report its direction.

Biological Motion

Nowhere is the power of motion to specify structure clearer than in biological motion. Attaching a dozen small lights to the major joints of a person and filming them moving in the dark yields a display that, held still, looks like a meaningless scatter of dots, but that the instant it moves is seen unmistakably as a walking human, whose gait, and often whose sex or mood, can be read at once (Johansson, 1973). The point-light walker shows that motion alone, stripped of form, color, and every static cue, carries enough information to recover a jointed body and its action, because the relative motions of the dots obey the constraints of a linked skeleton and the visual system is exquisitely tuned to those constraints.

Biological motion is a special case of the general principle that relative motion reveals structure, the same principle behind the kinetic depth effect, but it is a striking one because the structure recovered is a familiar, articulated object rather than an abstract shape. Its perception is fast, robust to noise, and present early in development, and it engages motion-sensitive cortex together with regions specialized for bodies and actions, which is why it has become a standard probe of how the motion system feeds into social and action perception. The phenomenon underlines the article's theme: motion is not a thin attribute added to an already-seen object but a rich source of information from which objects, depth, and even agents are constructed.

What the Framework Does and Does Not Explain

The two-stage account, local motion-energy detection followed by integration in MT, is one of the best-supported models in visual neuroscience, but it is not the whole of motion perception. It captures first-order motion, defined by moving luminance, yet observers also see second-order motion, defined by moving contrast or texture with no net luminance displacement, which a pure motion-energy detector cannot register; explaining second-order motion requires an additional nonlinear stage before the energy computation, and whether the two kinds of motion share a common pathway remains debated (Nakayama, 1985). There is also long-range apparent motion, the sense of movement between two well-separated flashes, which behaves more like a higher-level correspondence process than like local energy detection, suggesting that more than one mechanism deserves the name motion perception.

A second open question concerns how the read-out from MT is actually performed. That MT activity supports and biases motion decisions is established, but the decision itself, the accumulation of noisy motion evidence to a threshold, unfolds in downstream areas, and how the graded, distributed signal in MT is converted into a committed perceptual choice is a problem shared with the broader study of decision making (Britten et al., 1992). What is not in dispute is the core picture: motion is a constructed attribute, computed by dedicated detectors, integrated in a specialized cortical area, and tied by direct experiment to what an animal perceives and chooses. The remaining debates are about the mechanisms at the edges, not about the architecture at the center.

Worked Example

Consider the aperture demonstration with its default constants. A detector viewing a moving contour through an aperture measures only the motion component perpendicular to the contour, which is the true speed multiplied by the cosine of the angle between the true direction and the contour's normal. With a true speed of 4 degrees per second and an angle of 60 degrees between the true motion and the normal, the measured perpendicular component is 4 times the cosine of 60 degrees, and since the cosine of 60 degrees is 0.5, the measured component is 4 times 0.5, or 2 degrees per second. The detector reports 2 degrees per second, but the same 2 could arise from a true speed of 4 at 60 degrees, of 2 at 0 degrees, or of any velocity whose perpendicular component is 2, which is precisely the ambiguity a second differently oriented detector is needed to resolve.

The speed-tuning demonstration reaches its numbers from a log-Gaussian model of an MT neuron. The firing rate is the maximum rate times the exponential of minus the squared logarithm of the ratio of stimulus speed to preferred speed, divided by twice the squared tuning width, R equals R-max times the exponential of minus the squared natural log of speed over preferred-speed, over twice sigma squared. With a maximum rate of 60 spikes per second, a preferred speed of 16 degrees per second, and a tuning width of 1 in log units, a stimulus at the preferred 16 degrees per second gives the full 60 spikes per second, while a stimulus at 8 degrees per second, one octave below, gives a log ratio of the natural log of one half, or minus 0.693, whose square is 0.480, so the exponent is minus 0.240 and the rate is 60 times 0.787, or 47.2 spikes per second. Because the tuning is symmetric in log speed, a stimulus at 32 degrees per second, one octave above, gives the same 47.2 spikes per second, which is why speed tuning is naturally plotted on a logarithmic axis.

Discussion

Motion perception is best understood as the staged defeat of ambiguity. The retinal image offers only local, ambiguous motion measurements, each detector limited by the aperture problem to the component of motion perpendicular to the contour it sees; the motion-energy computation extracts even that component by treating direction as an orientation in space-time; and the integration of many such components, accomplished by the pattern neurons of area MT, recovers the true velocity that no local detector could measure (Adelson & Bergen, 1985; Rust et al., 2006). Global-motion pooling then extracts a coherent direction from noise, and the whole cascade trades spatial detail for reliability at every step.

The framework's authority rests on an unusually direct chain from neuron to percept. Area MT was localized as a nearly pure motion area, its pattern computation was reproduced by a canonical model, and its activity was shown to be necessary for motion perception, to carry enough information to support the decision, and to bias that decision when stimulated (Zeki, 1974; Newsome & Paré, 1988; Salzman et al., 1990). Biological motion then shows how far the recovered signal reaches, specifying a jointed, animate body from moving dots alone (Johansson, 1973). The open problems, second-order and long-range motion and the downstream read-out of MT, concern the boundaries of the account rather than its core, and the enduring lesson is that a percept as immediate as movement is the outcome of a deep, well-mapped computation.

Glossary

Aperture problem.

The ambiguity by which a local detector viewing a moving contour through a small receptive field can measure only the motion component perpendicular to the contour.

Apparent motion.

The perception of movement between two stimuli presented in different places at different times, as in film, without any continuous physical displacement.

Biological motion.

The perception of a living, articulated body and its action from the motion of a few points attached to its joints, as in a point-light display.

Component cell.

A motion-selective neuron that responds to the direction of an individual grating in a plaid, signalling a local, still-ambiguous motion component.

Direction selectivity.

The property of a neuron that fires to motion one way through its receptive field but not to motion the opposite way, the elementary signature of a motion detector.

Kinetic depth effect.

The perception of rigid three-dimensional shape from the relative motion of an object's features, so that a moving tangle of lines is seen as a solid form.

Middle temporal area.

Area MT or V5, a cortical region in which nearly all neurons are direction-selective, specialized for integrating local motion signals into true object velocity.

Motion aftereffect.

The illusory motion, opposite in direction, seen in a stationary scene after prolonged viewing of steady motion, revealing direction-selective channels that adapt.

Motion coherence.

In a random-dot display, the proportion of dots moving in a common direction, setting the strength of the global motion signal an observer must extract.

Motion energy.

The output of a detector built from space-time oriented filters that responds to the amount of movement in its preferred direction, the standard low-level motion computation.

Opponent motion.

A signal formed by subtracting the motion energy for one direction from that for the opposite direction, yielding a measure of net direction and speed.

Pattern cell.

An MT neuron that responds to the single coherent direction of a plaid rather than to its component gratings, having integrated local components into true motion.

Random-dot kinematogram.

A display of many dots in which a controlled fraction move coherently while the rest move at random, used to measure the pooling of global motion.

Second-order motion.

Movement defined by a drifting change in contrast or texture rather than luminance, which a simple motion-energy detector cannot register without an added nonlinear stage.

Speed tuning.

The property of a motion neuron of responding maximally to a preferred stimulus speed and less to slower or faster motion, typically symmetric on a logarithmic speed axis.

V5.

The human motion-selective visual area homologous to macaque MT, localized by functional imaging in the lateral occipitotemporal cortex.

Key Researchers

Semir Zeki. Professor at University College London; he identified area V5, also called MT, as a cortical region in which almost every neuron is selective for the direction of motion, establishing motion as the province of a distinct visual area. ORCID - Faculty Page - Wikipedia

Edward H. Adelson. Professor of vision science at the Massachusetts Institute of Technology; with Bergen he formulated the motion-energy model, showing that the direction of motion can be recovered by filters oriented in the joint domain of space and time. Faculty Page - Google Scholar - Wikipedia

William T. Newsome. Professor at Stanford University; he established the causal link between area MT and motion perception, showing that MT lesions impair motion sensitivity, that single MT neurons rival the whole animal, and that microstimulating MT biases perceptual choices. ORCID - Faculty Page - Google Scholar

J. Anthony Movshon. Professor of neural science and psychology at New York University; he showed that MT integrates the ambiguous local motion signals of a plaid into a single pattern-motion percept, and with Newsome tied single-neuron activity to perceptual decisions. ORCID - Faculty Page - Google Scholar

David J. Heeger. Professor of psychology and neural science at New York University; with Simoncelli he built a two-stage normalization model of area MT that accounts quantitatively for how its neurons compute pattern motion from direction-selective inputs. ORCID - Faculty Page - Google Scholar

Gunnar Johansson. Perception psychologist at Uppsala University until his death in 1998; he invented the point-light display, showing that a handful of moving dots attached to the joints convey a vivid impression of a walking person and founding the study of biological motion. Wikipedia) - Wikidata

Frequently Asked Questions

What is motion perception?
It is the process by which the visual system recovers the speed and direction of moving objects, and of the observer, from the changing pattern of light on the retina, treating motion as a primary visual dimension computed by dedicated detectors (Burr and Thompson, 2011).

What is the aperture problem?
It is the ambiguity that a local motion detector, viewing a moving contour through its small receptive field, can measure only the component of motion perpendicular to the contour, so its true direction cannot be known until several detectors are combined (Nakayama, 1985).

How does the brain detect the direction of motion?
Low-level detectors treat a moving feature as a structure tilted in the joint domain of space and time and use filters oriented in space-time to measure motion energy in a preferred direction, subtracting the opposite direction to report net motion (Adelson and Bergen, 1985).

What is area MT?
Area MT, or V5, is a cortical region in which nearly all neurons are direction-selective and many integrate local motion components into true object velocity, making it the brain's specialized center for motion processing (Zeki, 1974; Rust et al., 2006).

How do we know that MT causes motion perception?
Lesioning MT selectively raises motion thresholds, single MT neurons are as sensitive as the whole animal, and microstimulating a direction column biases an animal's choices toward that direction, so MT activity helps cause the percept (Newsome and Paré, 1988; Salzman et al., 1990).

What is a random-dot kinematogram?
It is a display in which a controlled fraction of dots move in a common direction while the rest move at random; the coherence sets the strength of the global motion signal, and the smallest detectable fraction measures how well the brain pools motion (Newsome et al., 1989).

What is biological motion?
It is the perception of a living, articulated body and its action from the motion of a few points of light attached to the joints, which looks like a random scatter when still but like a walking person the moment it moves (Johansson, 1973).

Can motion reveal the shape of an object?
Yes; the relative motion of an object's features as it or the observer moves specifies its three-dimensional structure, the kinetic depth effect, so that a moving tangle of lines is immediately seen as a rigid form (Wallach and O'Connell, 1953).

References