Neuronal mechanisms of visual stability Vision

What he found was hundreds of saccades that rapidly move the fine grained receptors of the ...... might at this point say that we have three legs of a four legged.
2MB taille 12 téléchargements 336 vues
Vision Research 48 (2008) 2070–2089

Contents lists available at ScienceDirect

Vision Research journal homepage: www.elsevier.com/locate/visres

Neuronal mechanisms of visual stability Robert H. Wurtz * Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bldg. 49, RM 2A50, Bethesda, MD 20892-4435, USA

a r t i c l e

i n f o

Article history: Received 4 February 2008 Received in revised form 22 March 2008

Keywords: Stable vision Saccades Corollary discharge Efference copy Visual masking Transsaccadic memory Proprioception

a b s t r a c t Human vision is stable and continuous in spite of the incessant interruptions produced by saccadic eye movements. These rapid eye movements serve vision by directing the high resolution fovea rapidly from one part of the visual scene to another. They should detract from vision because they generate two major problems: displacement of the retinal image with each saccade and blurring of the image during the saccade. This review considers the substantial advances in understanding the neuronal mechanisms underlying this visual stability derived primarily from neuronal recording and inactivation studies in the monkey, an excellent model for systems in the human brain. For the first problem, saccadic displacement, two neuronal candidates are salient. First are the neurons in frontal and parietal cortex with shifting receptive fields that provide anticipatory activity with each saccade and are driven by a corollary discharge. These could provide the mechanism for a retinotopic hypothesis of visual stability and possibly for a transsaccadic memory hypothesis, The second neuronal mechanism is provided by neurons whose visual response is modulated by eye position (gain field neurons) or are largely independent of eye position (real position neurons), and these neurons could provide the basis for a spatiotopic hypothesis. For the second problem, saccadic suppression, visual masking and corollary discharge are well established mechanisms, and possible neuronal correlates have been identified for each. Published by Elsevier Ltd.

1. Introduction We think of our visual perception as a unified continuous panorama of the world, present at all times and seamless in its continuity. Only when we consider how we acquire this visual knowledge do we appreciate that the underlying mechanisms are not nearly as seamless as our perception. The biggest challenges arise from the occurrence of rapid or saccadic eye movements that have been widely recognized following the illustrations of the frequency of these movements by Alfred Yarbus (1967). He used the then newly developed techniques for recording eye movements to record the shifts of the eye while a subject looked at a visual scene. What he found was hundreds of saccades that rapidly move the fine grained receptors of the fovea from point to point in the visual scene. Consider the consequences of these movements for vision. If we look at a series of schematic saccades (Fig. 1A) we see that this visual motor sequence has two phases: fixation at a particular location (blue dots in Fig. 1A) when virtually all vision occurs, and the rapidly moving saccades (blue lines in Fig. 1A) when virtually no vision occurs. As a consequence of this, the visual system receives a series of high resolution snapshots centered on different points in the scene (Fig. 1B). But the problem becomes even more challeng* Fax: +1 301 402 0511. E-mail addresses: [email protected], [email protected] 0042-6989/$ - see front matter Published by Elsevier Ltd. doi:10.1016/j.visres.2008.03.021

ing. The snapshots transmitted up the optic nerve with successive fixations are all centered on the fovea. There is no information in this transmission about where in the scene this snapshot is located (Fig. 1C). Finally, interspersed amongst the snapshots are the blurred images of the wide field motion produced by the rapidly moving eye during the saccade (indicated by the white patches in Fig. 1C). How our sense of visual stability survives such disruptions of the visual input has been the subject of speculation at least since the 1600s and of behavioral investigation for over a hundred years. Only since techniques became available to record neuronal activity in the brains of awake animals able to move their eyes, however, has it been possible to look at the brain mechanisms underlying this stable visual perception. This review attempts to summarize the major findings that the study of awake animals has permitted and to address the two major questions emphasized in Fig. 1: the displacement of the retinal image resulting from the saccade and the suppression of the blur and motion during the saccade. A substantial literature has accumulated on visual stability at both behavioral and neuronal levels, and this has required some selection for this review. First, I will concentrate on the problems of stability generated by saccadic eye movements which means neglecting the equally important questions of stability arising from such eye movements as smooth pursuit or those made during our movement through the environment. Second,

2071

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

WholeVisual Field Moves

(Reafference)

Proprioception (inflow)

}

Retinal

}

Extra Extraretinal

Corollary Discharge (outflow) Fig. 2. Three signals available for producing visual stability: Visual field motion (reafference), Proprioception (extraretinal inflow), Corollary Discharge (CD, extraretinal outflow).

systematic investigation of the relation between neurons and behavior. 2. Signals that contribute to visual stability In order to evaluate the neuronal mechanisms of visual stability, we need to know when changes in the image on the retina result from real world movement and when from eye movement. Only three signals and the interactions between them are used to make this distinction (Fig. 2). The first signal is visual reafference from the retina; when the eye moves it generates visual motion and displacement signals that can indicate eye movement. The second signal is an extraretinal one; when the eye muscles contract, proprioceptors are activated, and these indicate the eye has moved. This proprioception is an inflow signal from the periphery into the brain just as is visual input. The third signal is also an extraretinal one. It is a corollary or copy of the command to move the eyes. It is referred to as an outflow signal because it copies what flows out to activate the eye muscles. That’s it. All the mechanisms considered in this review are derived from these three signals and their interactions. But their contributions are not equal, and we first separate the major ones from the minor ones. 2.1. Visual reafference

Fig. 1. The two problems saccades produce for the visual system. (A) Schematic saccades (blue lines) and fixations (blue dots) used to illustrate the consequences of the saccades for vision. The painting is titled An Unexpected Visitor by Ilya Repin, and was made famous in vision research by Alfred Yarbus who used it as one of the illustrations he had his subjects look at while he recorded their eye movements. (B) The snapshots at the location of each fixation. Shading is intended to indicate reduction of acuity as the distance from the fovea increases. (C) The transmission in the optic nerve of the snapshots from three fixations without any information about location but with the interspersed blurs from the intervening saccades.

I will concentrate on the neuronal correlates of stability because the goal is to identify possible underlying mechanisms of stability to which EEG and fMRI make minimal contributions. Third, while it is essential to start each topic by specifying the behavior to be explained, I will concentrate on those behavioral attributes for which I see some evidence of at least a potential neuronal mechanism. Finally, the neuronal evidence will be from monkeys unless otherwise indicated because of the similarity of the visual and oculomotor system of old world monkeys and humans and because the monkeys can be trained to perform tasks that allow

With each eye movement, the whole visual field moves, and this optic flow can be taken as indication of self motion rather than motion in the environment. This visual input is referred to as ‘‘reafference” (von Holst & Mittelstaedt, 1950) in recognition of its selfgenerated nature in contrast to the ‘‘exafference” that results from motion in the real world in front of a stationary eye. With saccades, such full field motion is usually blurred because of their rapidity but a visual cue remains because the whole field is displaced. For slower eye movements such as with movement of the subject through the environment such optic flow can be critical. This ‘‘optic array” was regarded as the central factor for stability by Gibson (1966), but for saccades the contribution is probably minimal as indicated by the evidence presented below related to corollary discharge. 2.2. Proprioception Eye position information from eye muscle proprioceptors could flow into the brain after the eye moves and interact with the visual input to produce visual stability. The preponderance of evidence available, however, is that proprioception does not make a substantial contribution to perceptual stability. A critical test has been to artificially change the position of the eye by electrically stimulating the brain while the monkey was

2072

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

preparing a saccade to a visual target. After the stimulation, the saccade had to be made to the same visual target, but from a new position. If proprioception provides position information, every time stimulation changed the initial eye position, proprioceptive feedback should have indicated that new position, and the subsequent visually guided saccade should have compensated for the change in position. When the eyes were moved by stimulating the motor neurons, compensation did not occur, so that proprioception could not have conveyed sufficient information about the new eye position (Mays, Sparks, & Porter, 1987; Schiller & Sandell, 1983; Sparks & Mays, 1983). In contrast, when the eyes were moved by stimulating several synapses above the motor neurons, in the superior colliculus (SC), the saccade did compensate for the new initial eye position. The simplest interpretation is that stimulation at higher levels produced not only proprioception but an internal signal, and it was this internal signal that provided information on eye position, not proprioception. A second set of experiments has relied on cutting the proprioceptive afferents from the eye. When the nerves were cut, (Guthrie, Porter, & Sparks, 1983), compensation still occurred after SC evoked saccades which indicated again that proprioception was not required. In a subsequent set of experiments, Lewis, Zee, Hayman, and Tamargo (2001) found that saccadic accuracy was not significantly altered after a monkey’s proprioceptive afferents were cut. Thus, assuming all the afferents were cut, these experiments also indicate that proprioception does not provide an adequate eye position signal for saccades. Finally, the nature of the eye proprioceptive signal itself has been revealed by the recent discovery by Wang, Zhang, Cohen, and Goldberg (2007) that such an input reaches the face region of somatosensory cortex (area 3a). They found long latencies for these proprioceptive signals, frequently about 100 ms (personal communication, M.E. Goldberg and Wang et al., 2007), which probably is too late to influence the next saccade. In net, there is now no good evidence that proprioception is used to provide an extraretinal position signal that could be used to correct for the effect of eye movement after each saccade. Instead, the proprioceptive information might be providing the long term calibration of saccades that are actually made as opposed to the ones that are planned, particularly in cases where visual information is not available (Lewis et al., 2001; Wang et al., 2007). It might well be that rapid information is provided by a corollary or copy for each saccade while longer term information is provided by proprioception (Lewis et al., 2001; Steinbach, 1987) including cases in which static eye deviation leads to misperception of direction (Gauthier, Nommay, & Vercher, 1990). 2.3. Corollary discharge (CD) The other source of extraretinal information is from a corollary or copy of the command to move the eye. The basic concept is that at the same time that instructions are issued for the muscles to produce a movement, a copy or corollary of those instructions is sent to other regions of the brain to inform them of the impending movement. The term efference copy was coined by von Holst and Mittelstaedt (1950) and the term corollary discharge was coined in the same year by Sperry (1950)—1950 was a big year for the concept. In the Sperry experiments, he surgically rotated the eyes of fish by 180 degrees and found as others had that the fish swam in circles. After exploring alternatives, he reached the conclusion that ‘‘. . .any excitation pattern that normally results in a movement that will cause a displacement of the visual image on the retina may have a corollary discharge into the visual centers to compensate for the retinal displacement.” Thus as the fish normally moved forward a corollary discharge indicated forward motion and this compensated for the backward image motion on the

retina. As the fish with the rotated eye moved forward, the corollary stayed the same, but the image motion direction conveyed to the brain was now also forward and this combined with the corollary. The corollary now did not compensate for the retinal motion but accentuated it, leading to the circling. In the von Holst and Middlestadt experiments (1950), the test was on the optokinetic response of a fly to rotation of vertical stripes in front of it, with similar logic and interpretation. While the concepts of efference copy and corollary discharge are essentially identical, in this review I will use corollary discharge (CD). First, the term efference copy implies a copy of the output close to the muscles, while the neuronal mechanisms relevant to visual stability are at higher levels of the brain including the cerebral cortex. Second, the term copy implies that the signal is an exact replica of what is sent to generate the movement, but corollary need have no such exact identity. Finally the efference copy term has been closely associated with the cancellation hypothesis, in which the reafferent input and the efferent copy output match, and there is little evidence for such cancellation in the saccadic system (for discussion see Sommer & Wurtz, 2008). Whatever the name, the concept is not new, and its history is chronicled in excellent summaries (Bridgeman, 2007; Bridgeman, Van der Heijden, & Velichkovsky, 1994; Grusser, 1995). A few salient points from these reviews illustrate how central the concept has been for visual stability. Descartes in his Treatise on Man inferred what must be the mechanistic difference between the stability of the visual scene when he moved his eye as opposed to its instability when he deflected his eye ball by pushing on it. Even earlier a Dutch scientist Aquilonius in 1613 made the same point and concluded that ‘‘an internal faculty of the soul perceives the movement of the eye”. In the 19th century those who considered the issue reads like a visual science who’s who: Bell, Purkinje, Mach, and von Helmholtz. Helmholtz referred to the concept as the ‘‘effort of will”, and marshaled the evidence in favor of his view. Among the lines of evidence, three are as significant now as they were then (see Bridgeman, Hendry, & Stark, 1975; von Helmholtz, 1925). First was the previously recognized apparent motion of the visual world when the eye is moved passively by pushing on the eyeball. The perceived motion was taken to result from the movement of the eye without CD. More recent experiments that measured the passive eye movements (Bridgeman & Stark, 1991; Ilg, Bridgeman, & Hoffmann, 1989; Stark & Bridgeman, 1983) showed that the perceived motion of the visual world resulted from a subject’s attempt while fixating to counter the external push to rotate the eye rather than the rotation of the eye itself. The CD accompanying the attempted but blocked eye movement was therefore taken as the source of the perceived motion. Second was the apparent displacement of the visual world when a saccade is attempted but the eye muscles are paralyzed. Considerable controversy followed this observation, because several subsequent experimenters failed to obtain such a simple finding (see Stevens et al. 1976 for a more complete discussion). This uncertainty might well have been related to residual eye movement due to partial paralysis in some experiments. This ambiguity was resolved by heroic experiments in which the whole subject, John Stevens, was paralyzed (Stevens et al., 1976). Under such complete paralysis, with each attempted saccade the visual field was displaced in the direction of the intended saccade. There was also a greatly increased sense of effort in trying to make the saccade, and there was fading of the image over time. These experiments also carefully delineated the perception of motion, the sweeping motion of objects in the visual field, from the perception of displacement, the sudden shift of an object from one position to another position. A perception of motion was absent with complete paralysis, but was reported with partial paralysis, presum-

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

ably due to residual eye movements. Thus, with complete paralysis and the delineation of motion and displacement, the conclusions of Helmholtz were reinstated. Subsequent paralysis experiments (Matin et al., 1982) went on to show that in the presence of visual input, perception was controlled by vision rather than any extraretinal input. This ‘‘visual capture” provided a critical perspective on the relative role of CD: when both visual and CD information are available, vision prevails. The third observation was based on images that are fixed on the retina, afterimages. They remain stable when the eye is moved passively in the dark, but they are displaced with saccades (Bridgeman, 2007). The perception of displacement of an image frozen on the retina must be provided by an extraretinal signal, presumably CD. 2.4. Conclusion Of the three signals available to distinguish eye movement from world movement, CD is the critical one for saccadic eye movements. The contribution of the other extraretinal signal, proprioception, seems unlikely to be significant on a saccade by saccade basis but rather may be important for longer term calibration. The signature visual input from eye movement, full field motion, may be of less use with the rapid sweep of saccades than with slower pursuit movements or the optic flow with observer movement. CD will therefore be a central focus of this review, but only with respect to visual stability since two recent reviews have considered the neural basis of the CD more generally (Sommer & Wurtz, 2008). In addition previous reviews and commentaries provide substantial details and evaluation of the CD mechanism (Bridgeman, 1995, 2007; Bridgeman et al., 1994).

2073

ing the saccade (Bridgeman et al., 1975). Displacements up to one third the size of the saccade go undetected. Even though such a displacement is not perceived, however, the displaced target can still be accurately located when used as a saccade target (Bridgeman, Lewis, Heit, & Nagle, 1979; McLaughlin, 1967). The displacement information for the saccade is independent of that for perception. The extent to which neuronal activity is devoted to perception rather than to the control of movement is a recurring issue with respect to the neuronal correlates of visual stability. 4. Displacement: Retinotopic maps In this hypothesis, there is no higher order spatial map in the brain but instead the representation of the visual world always remains in retinotopic coordinates. At a neuronal level, we would expect to find a mechanism for updating a retinotopic map, and as complicated as it initially may seem, this updating probably has accumulated the largest body of neuronal evidence. 4.1. The concept of shifting receptive fields The landmark experiment of Duhamel, Colby, and Goldberg (1992) showed that neurons in the parietal cortex had the remarkable property that the location of their visual sensitivity shifted in anticipation of the upcoming saccade. This anticipatory change has been referred to as remapping or spatial updating (which emphasizes the conceptual significance) or as shifting receptive fields (which just describes the neuronal activity). The concept is best explained by returning to the same illustrations used to show the problems generated by saccades but now with the aim of understanding how the problem of displacement might be resolved (Fig. 3). If the subject is looking at one point in the figure (Fig. 3A, blue dot on the face on the left) and we record from a neuron in the brain, it will be sensitive to visual stimuli

3. Saccadic displacement 3.1. Multiple hypotheses Conceptually, the most challenging problem is the perception of visual stability in spite of the displacement of the image on the retina with each saccade (as illustrated in Fig. 1). How this problem is solved by the brain has been the topic of speculation involving many hypotheses with many variations. We will consider two broad categories of hypotheses and the possible neuronal correlates for each. The first hypotheses posit that we maintain only a map of what is currently on the retina, and that this retinotopic map is simply updated with each saccade. Included here are hypotheses that require updating only of regions to which attention is directed, particularly the small region of the map near the fovea. Possible neuronal mechanisms for this are neurons with a shifting receptive field (RF) in parietal and frontal cortex. The second group of hypotheses assume that there is a spatiotopic map within the brain and that each successive retinal image updates this higher order map. The neuronal evidence for this rests on the discovery of gain field neurons, whose visual responses are modulated by the position of the eye in the orbit, and real position neurons, whose responses are independent of orbital position. 3.2. Visual stability and visually guided movement Before considering these hypotheses, it is essential to recognize that maintaining perceptual stability and generating movement occur simultaneously, and neuronal activity may be related to one and not to the other. A case of separation between perception and movement that illustrates the issue is failure to see the displacement of a saccade target when the displacement occurs dur-

Fig. 3. The logic of shifting receptive fields and remapping. (A) Receptive Field and Future Field at the time of the saccade. (B) Future field has become the new Receptive Field after the saccade. See text for explanation.

2074

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

falling in one part of the visual field, the RF (Fig. 3A, darkened circle). Such localized RFs are what we expect of visual neurons from the retina onward. The cortical neurons considered here, however, have an added property: as the monkey prepares to make the saccade, the sensitivity at the site the RF will occupy after the saccade becomes more sensitive even before the saccade occurs. Thus for a time the neuron can respond to stimuli in its current RF (not surprising) and in its future field (FF in Fig. 3A)—very surprising. These neurons must have information about the amplitude and direction of the impending saccade because the FF sensitivity is located at the site where the RF will be after a saccade. The location of the FF must depend on a CD of the impending saccade since here the FF activity precedes the saccade just as the CD does. After the saccade, the FF becomes the RF of the neuron (Fig. 3B). The cycle then begins again with preparation to make the next saccade. So what does this have to do with visual stability? In its simplest form, the idea is that it is the increased sensitivity at the FF before the saccade that highlights the same object before the saccade that will fall on the RF after the saccade. When there is congruity between what is there before and after saccade, the eye must have moved from one fixation point to another. There would then be two successive increases of activity, one reflecting the increased sensitivity to the object in the FF, and one as the eye brings the RF onto the object. In contrast, if there is just a single increase in visual activity, an object must have appeared in the environment, and it was that appearance that activated the neuron rather than a saccade. While the sequence of events shown in Fig. 3 illustrates the anticipatory events that might be associated with saccades, recent psychophysical experiments by Melcher (2005,2007) provide evidence for such anticipatory changes. In one experiment, for example, subjects adapted to a tilted grating pattern overlying the fixation point, and then were tested for a tilt after effect as they continued to fixate. On subsequent adaptation trials, instead of continuing to fixate, the subjects made saccades to a target. With saccades, the adaptation increased by 60% at the new target while at the same time it decreased by 80% at the current fixation point. Both these changes occurred before the saccade began. Thus these changes in adaptation show experimentally essentially what is shown conceptually in Fig. 3. More generally, both the concept and the experiment emphasize that ‘‘postsaccadic perception does not begin anew but rather takes into account previous visual experience” (Melcher, 2007). 4.2. Neurons with shifting receptive fields Neurons that show evidence of shifting RFs have been studied in LIP where they were initially discovered (Batista, Buneo, Snyder, & Andersen, 1999; Colby, Duhamel, & Goldberg, 1996; Duhamel et al., 1992; Heiser & Colby, 2006; Kusunoki & Goldberg, 2003), and have subsequently been found in the visual-motor area of frontal cortex, FEF (Sommer & Wurtz, 2006; Umeno & Goldberg, 1997, 2001). The shifting RFs have also been seen in earlier extrastriate visual areas (Nakamura & Colby, 2002; Tolias et al., 2001), and the SC (Walker, FitzGibbon, & Goldberg, 1994). There is also fMRI evidence for them in the human brain (Medendorp, Goltz, Vilis, & Crawford, 2003; Merriam, Genovese, & Colby, 2003, 2007). Fig. 4 shows an example of a shifting RF of a frontal eye field (FEF) neuron. The basic strategy is to probe the sensitivity of the RF and FF locations at varying times before and after the saccade with brief light flashes. While the monkey fixated, the neuron responded only to probes flashed in the RF (Fig. 4 top row), but just before the saccade, it responded to probes flashed in the FF (Fig. 4 middle row). After the saccade the probe was only effective in the area of the FF which, because of the saccade, was now the RF. The increased sensitivity at the FF was not activity simply related to the

Probe onset (during fixation)

Probe onset (just before saccade)

RF probe

FF probe

No probe

Fig. 4. An example FEF neuron with a shifting RF. The visual response is aligned on the visual probe flashed during fixation (left column) and just before the saccade (right column). Probes were in the RF (top row), in the Future Field (FF, middle row), or absent as a control in the bottom row. The visual response shifts from the RF (magenta, left) to the FF (magenta, right) just before the saccade. Each trace is the mean and SEM; vertical scale is 110 sp/s; horizontal tick marks, 100 ms. Modified from Sommer and Wurtz (2006).

saccade because when there was no probe, there was no increased activity (Fig. 4 bottom row). The shifting RF is a change in sensitivity in the FF that is only revealed when probed with a stimulus. The shifting RFs neurons become understandable if they receive not only visual input, but also a CD input that provides the vector of the impending saccade that would point to the new fixation location. The RFs with a fixed retinotopic relation to the current fixation point would have the same relation to the target of the saccade (which becomes the new fixation point after the saccade). While identified visual pathways to frontal cortex are numerous (Salin & Bullier, 1995), pathways carrying a corollary discharge are not, so the next issue is identifying such a CD pathway. 4.3. A CD to cerebral cortex The fundamental problem in identifying a CD in the primate brain is two fold. First it is necessary to understand the brain circuits for pre-motor processing in order to identify the level from which the CD is drawn. Second, the CD has to be identified as separate from the signal that actually produces the movement. In the monkey both of these requirements have been met. The outline of the circuit for the generation of saccades is probably better understood than that for any other coordinated movement circuit in the primate brain in part because the saccade is a relatively simple movement (Robinson, 1964, 1968) and in part because of the substantial number of studies devoted to it since the 1960s. Fig. 5A shows an outline of this circuit extending from the retina to the eye muscles. In outline, the visual input arrives at striate or V1 cortex through a retinal-lateral geniculate pathway, and then is distributed to extrastriate areas. From there the visual information reaches two critical cortical regions: the posterior parietal cortex (particularly the lateral intraparietal region, LIP) and the dorsal frontal cortex (particularly the frontal eye field, FEF). From these two regions pathways descend to the superior colliculus (SC) on the roof of the midbrain. The SC in turn projects to the midbrain and pontine reticular formation, which project to the oculomotor nuclei that innervate each of the six eye muscles. Other descending pathways (including one from FEF through the basal ganglia to SC) and a second visual input pathway (from retina

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

Frontal Cortex

Posterior Parietal Cortex

V1

LGN Midbrain & Pons

Visual Processing

Superior Colliculus

Frontal Eye Field

MD Thalamus

have been discussed in detail in a recent review by Sommer and Wurtz (2008), and only the conclusions need be stated here. They pointed out that the neurons in the MD relay in the pathway to cortex had the characteristics expected of CD neurons in that they discharged before saccades and only before saccades of given amplitude and direction, reflecting their SC origins. The MD neurons were not in the pathway that led to saccades because inactivation did not disrupt either visually or memory guided saccades. But MD inactivation did disrupt the monkey’s ability to do a double step saccade task that requires a CD for its performance (Hallett & Lightstone, 1976). Thus the neuronal activity in the SC–MD–FEF pathway did meet the criteria for CD for saccades (Sommer & Wurtz, 2008). But as has already been discussed, a CD could be related to control of movement and not be used for perception. The next question is whether this CD contributes to shifting RFs and possibly to the stability of perception. 4.4. Does the CD to cortex drive the shifting RFs?

Corollary Discharge Sensorimotor Processing

2075

Superior Colliculus

Movement Command

Fig. 5. Brain circuits for visually guided saccades (A) and saccade based corollary discharge (CD, B). See text for description.

to SC superficial layers and then through pulvinar to visual cortex) are not shown, nor are the pathways through the cerebellum. The point, however, is that while this is just a sketch of our admittedly incomplete knowledge of these pathways, it is complete enough to understand the type of signal conveyed by neurons at multiple levels within the brain. Which brings us to the second question: locating a CD that branches off from this descending motor pathway. For a saccadic CD, we obviously look for neurons whose discharge is correlated with saccade generation and this activity is found first in cortex (FEF and to some extent LIP), then prominently in the intermediate layers of the SC, and then in the pontine and midbrain areas to which it projects. One pathway that is particularly intriguing (Fig. 5B, right) extends from the intermediate SC layers to the medial dorsal nucleus (MD) of the thalamus to the FEF (Lynch, Hoover, et al., 1994). Saccadic information in these SC layers is still organized in a retinotopic map so that the saccade related information from SC would be in the same coordinate system as the target of the pathway, the FEF. To more specifically relate this pathway to a CD function, the outline of a CD circuit is indicated on the left in Fig. 5B. The sensorimotor area in which the CD originates would be the SC, the visual processing area would be the FEF, and the connection between them would be the SC–MD–FEF pathway. The issue is whether this pathway actually conveys a CD signal. In contrast to invertebrates, where transmission of a CD can be related to a specific neuron type (Poulet & Hedwig, 2006), the CD in the primate is embedded in a complex of pathways. In addition, if we isolate a neuron in a putative CD pathway, how do we know it carries a CD signal or is just part of a loop in the movement generation circuit? To address this problem, Sommer and Wurtz (2002) relied upon previous experience in studying CD particularly in invertebrates and identified four criteria that they argued would qualify the candidate neurons as those carrying a CD. These points

We now have identified two factors that might contribute to visual stability. The first is the shifting RFs that could contribute to remapping of the visual field with each saccade and that depend upon a CD to do so. The second is a CD signal that reaches the FEF as just described. Does the identified CD fit the requirements of the shifting RFs? To answer this question it was necessary to establish that shifting RFs have the characteristics we would expect if the shifts resulted from the CD in the SC–MD–FEF pathway and then to see if the CD is necessary to drive shifts. 4.4.1. Is the CD appropriate for driving RF shifts? First, do the shifting RFs have the expected spatial configuration? Neuronal CD activity in MD (as in SC activity) indicates the vector of the upcoming saccade: the discharge increases before saccades of given amplitude and direction. Therefore we would expect the RF shifts in FEF to act as if they resulted from such a vector input: they should jump from one location to another (Fig. 6A top) rather than just expand and then contract around the FF. If there were such a slide or expansion, there would be activity between the RF and the FF whereas with the jump there would not be. There was no indication of activity between the RF and the FF (Fig. 6A bottom) and thus no indication of a slide (Sommer & Wurtz, 2006). The shift behaves spatially as if it received input from the CD vector that jumps the activity from the RF to the FF. The second expected characteristic is a temporal one: is the time of the RF shifts consistent with the timing of the CD? The CD activity in MD is synchronized with saccade onset (Sommer & Wurtz, 2004) and if this CD is driving the shift of activity in FF, this shift should also be synchronized with the saccade onset. The increase in the activity with the shift for the example neuron in Fig. 6B top shows a highly significant correlation with saccade onset (p < 0.01, Sommer & Wurtz, 2006), as expected if the CD drives the shift. For the same reason, the increased activity with the shift should not be at the latency of the neuron’s visual response. To show this, the results for the same example neuron are aligned on the visual probe rather than the saccade in Fig. 6B bottom. The visual latency to a stimulus flashed in the RF of the neuron was 80 ms (black arrow) which occurs long before the activity associated with the shift (bold orange trace of average firing rate). Therefore the shift is synchronized with the saccade, and it is more appropriate to refer to it as visual activity (it requires a visual stimulus) rather than as a visual response (it has no brisk onset or fixed latency). The range of onset times for FF activity extended from about 100 ms before the saccade to about 200 ms after it, with a mean onset at 24 ms after the saccade had started (Sommer & Wurtz,

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

Firing rate

2076

RF

FF Midpoint n.s.d.

in most cases. So the shift is anticipatory because its visual activity still occurs before the visual response generated as a result of the saccade. Of course regardless of the time at which the visual activity occurs, the flash that generates the activity is always before the saccade.

Cross section through fields Initial fixation Just before saccade

Example Visual probe 50 sp/s

n.s.d.

100ms

Saccade initiation

Range of probe onset times

4.4.2. Is the CD necessary for driving RF shifts? While the shifting RFs in FEF have spatial and temporal characteristics consistent with input from the identified CD, this only demonstrates a correlation between the shifting RFs and the CD. The critical question is whether the CD is necessary to produce the shift. Inactivation of the region of identified relay neurons in MD, while measuring the shifting RFs in an FEF neuron, makes it possible to test this necessity. Fig. 7A shows an example of one of these experiments (Sommer & Wurtz, 2006). Black traces in Fig. 7A show activity before inactivation; orange traces are activity after inactivation. The neuron had a strong visual response in the RF during fixation (left upper panel) and a strong shift to the FF just before the saccade (right lower panel). During inactivation, the FF response was greatly reduced. The response in the RF response

before

Saccade

250 sp/s

Saccade

neuron’s visual latency

100 sp/s

~70 ms

Long before

Probe onset

RF:

Probe

0

100

200

300

100 ms tics

400ms

Future

2006). The mean onset time of the CD activity in MD was 72 ms before the saccade. The mean onset time of the CD activity in MD preceded the mean for shifting activity in FEF by 96 ms so that the CD is early enough to drive the shifts. Such a long delay between the CD and the shift indicates that the shift is not the action at a single or even only a few synapses, but must instead result from activity in a multisynaptic network which remains to be explored. With the timing of the shifts now specified, it is worth clarifying the assertion that the shifting RF activity provides an ‘‘anticipatory” signal before the saccade. If the mean time of the shift is 24 ms after the saccade has started for FEF neurons, and the range of shift times extends into the period after most saccades have ended, is it appropriate to refer to the shifts as being anticipatory? No, if anticipatory refers to the onset saccade. Yes, if it refers to the time of the reafferent visual input produced by the saccade reaches the FEF neuron. After a stimulus falls on the RF as a result of the saccade, there is a visual latency for the response in the neuron. So even if the shifting RF activity occurs after the saccade it will still activate the neuron before the reafferent visual stimulation

Field: Control Inactivation

Ipsilateral

Contralateral

FF for Ipsi Saccades

FF for Contra Saccades RF

deficit (% decrease in firing rate)

Fig. 6. The spatial (A) and temporal (B) characteristics for shifting RFs are consistent with an input from the CD passing through MD thalamus. (A) The shifting RFs would be expected to jump from the RF to the FF with no significant change in activity at the midpoint if the shift resulted from the vector input of a CD. An example FEF neuron shows this lack of midpoint activity consistent with the shift being driven by a CD. From Sommer and Wurtz (2008). (B) The timing of the shifting RF should be synchronized with the saccade as is the CD. In the upper record the activity of an example FEF neuron aligned on saccade onset shows that the increased activity with the shift is synchronized with saccade initiation. In the lower record, the same neuronal responses are aligned on visual probe onset, and show that the increase in activity with the shift occurs long after the visual latency (black arrow). The green dots show the time of saccade onset in each trial. From Sommer and Wurtz (2006).

80

**

60 40 20

RF

FF for Contra Saccades

FF for Ipsi Saccades

Fig. 7. Necessity of CD input from MD thalamus for shifting RFs in FEF. (A) Example of a shifting RF impaired by MD inactivation. During inactivation (orange traces) the activity in the future field decreased by 78%. (B) The deficit in the population of neurons. Reductions in activity were seen only in the FF, not the RF, and only for contralateral, not ipsilateral, saccades. **Significant changes at p < 0.0001 level. From Sommer and Wurtz (2006).

2077

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

4.5. Temporal distribution of shifts: Saccadic compression Stimuli presented at the time of the saccade are mislocalized, and this may be related to the underlying mechanism of shifting receptive fields. Matin and his collaborators (Matin, Matin, & Pearce, 1969; Matin, Matin, & Pola, 1970; Matin & Pearce, 1965) flashed spots of light just before a saccade and found that the stimuli were mislocalized. A substantial number of subsequent experiments have verified this (Morrone, Ross, & Burr, 1997; Ross, Morrone, & Burr, 1997), and also have demonstrated mislocalization of a flash after the saccade. As illustrated in Fig. 8A, the mislocalization is in the direction of the saccade or opposite to the direction depending on flash location and timing. This has been described as a compression of the visual field at the time of the saccade. Ross, Morrone, Goldberg, and Burr (2001) suggested that this compression might be related to relative activity in the RF and the FF of the shifting neurons. For example, for the FEF neuron in Fig. 4, the activity declined in the RF field as it increased in the FF, and this was the case in 1/3 of the neurons, with the remaining 2/3 showing continuing activity in the RF as the activity developed in the FF. LIP also has a mixture of shifting RF types (Colby & Goldberg, 1999; Colby et al., 1996; Kusunoki & Goldberg, 2003). Neu-

40

Apparent position (˚)

was unchanged showing that the neuron still had its sensitivity to visual stimuli. The monkey also could still make the saccade to the target confirming that the MD inactivation did not block saccade generation. But the connection between the FEF visual processing and the CD was impaired because now the FEF neuron lost much of its advance information about the impending saccade. This in turn led to the impaired RF shift. The deficit was present in 7 of the 8 neurons in which the experiment was completed, and was highly significant in the pooled results (Fig. 7B) with the shift reduced by on average about 50%. The activity of the MD neurons was related to saccades to the contralateral visual field, and so too was the deficit in the shifting RFs; the deficit was related to contralateral saccades leaving ipsilateral saccades unimpaired. The combination of these correlation and inactivation experiments provides firm evidence that the identified CD from the SC is critical for RF shifts in FEF neurons. The working assumption is that the functions of a CD input established for FEF also apply to the shifting receptive fields found in LIP (Colby & Goldberg, 1999) and even in earlier cortical areas (Nakamura & Colby, 2002). One possibility is that there are additional pathways conveying a CD directly to parietal cortex but at this point none has been identified. Another possibility is that the CD reaches parietal cortex transcortically via the extensive functional projections between the two structures (Chafee & Goldman-Rakic, 1998; Chafee & Goldman-Rakic, 2000). One clue as to whether there is a direct CD to parietal cortex or an input through FEF might be obtained by a detailed comparison of the shifts in the two areas but that has not yet been done. Such a comparison might also provide insight into the relative roles of frontal and parietal cortex in visual stability. What must be true for both frontal and parietal neurons with shifting RFs is that they do not receive input from just one localized region of the retinotopic map as do neurons in the early visual areas. Because the FEF neurons responded to visual stimuli in both the RF and the FF, these neurons must have connections to both those retinotopic areas. Since a saccade can be made to any region of the visual field, the FF can fall in any part of the field, and so the neuron must be connected to every part of the retinotopic map. This requires a tremendous increase in the connections to each of the shifting RF neurons, and a model simulation by Quaia, Optican, and Goldberg (1998) shows how these multiple connections could be changed at the time of the saccade.

20

0

–20

–40 –200 –100

0

100

200

100

200

Time (ms)

shift onset

–200 –100

0

Time (ms) Saccade Initiation Fig. 8. Saccadic mislocalization (A) and a possible correlate in shifting RFs (B). (A) Mislocalization of the position of a bar briefly presented around the time of a horizontal saccade from 10° to +10° (black stepped line-0 refers to screen center). The apparent position judged by the subject (ordinate) is plotted against the display times relative to the onset of a saccade (abscissa). The bar was presented at one of three positions on the screen ( 20°, 0°, 20°, indicated by the arrows). The result was that subjects systematically mislocated the bar. Bars presented at 20° or 0° were mislocalized in the direction of the saccade at about the same time as the onset of a saccade. Bars presented at +20° were mislocated in the opposite direction (against the direction of the saccade). The combination produced an effective compression of visual space. Adapted, with permission, from Ross et al. (2001). (B) Spread in shift onset times given by four example FEF neurons. The orange trace is from the neuron in Fig. 6B. Shift magnitudes are normalized to each other for comparison of timing. From Sommer and Wurtz (2008).

rons with activities in both the RF and FF might effectively indicate that a flashed stimulus fell between the RF and FF locations, which would put the implied location closer to the saccade endpoint than it actually was, and this could lead to the apparent compression. A second source of the compression might be related to the timing of the shift in different neurons. Recall that in the FEF experiments a brief flash (as in Fig. 4) led to increased activity in the FF. The time at which the shift became evident, however, varied from neuron to neuron as illustrated for four FEF neurons in Fig. 8B. The distribution of the shift onset times across the sample of FEF neurons ranged from about 100 ms before to 200 ms after the saccade onset (Sommer & Wurtz, 2008). This distribution prolongs the period over which the shifting occurs, and it may be in part this prolongation of the updating that contributes to the mislocalization of brief flashes occurring while the process is still underway. More generally, the long time course over which the shifting of the RFs occurs implies that any consequent updating of a

2078

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

retinotopic map also might be spread over time. If so, this long time for shifting completion might underlie the behaviorally based conclusion that the shift in visual stability is a sluggish system as summarized in a recent critique by Bridgeman (2007). By this logic, the mislocalization at the time of the saccade is a flaw in the system rather than a feature. Such a slow updating might be tolerated by the system because there is no necessity for a temporally perfect shift; the flaws are covered up by the suppression considered later (Matin et al., 1969). 4.6. Conclusion: Shifting fields for updating a retinotopic map Shifting RFs could provide the neuronal basis for visual stability in the presence of saccades: they provide a potential link between the concept of updating a retinotopic map with each saccade and the CD that would be necessary to accomplish that updating. We might at this point say that we have three legs of a four legged stool supporting the retinotopic hypothesis of visual stability. The first leg is the demonstration that neurons in FEF, LIP and other areas show shifting receptive fields that could contribute to updating or remapping of the visual field with each saccade. The second leg is the identification of a CD from the SC–MD–FEF pathway that reaches these neurons. The third leg is the demonstration that the CD is necessary for the shift to occur in FEF. Before we try to support a hypothesis of visual stability on this stool, we need a fourth leg: we need to demonstrate that the shifting RFs actually underlie perceptual stability. Up to this point such a demonstration has been largely beyond testing even in monkeys because any inactivation at the site of the shifting RFs either in FEF or LIP would disrupt the visual processing as well as the shifting RF mechanisms. With the identification of the MD relay in the CD pathway, however, it should be possible to inactivate MD, reduce the CD to cortex, and reduce the FEF shift. With the shift reduced, the perception of stability should be reduced. The challenge then becomes one of showing that a monkey with reduced shifts has reduced ability to distinguish object displacement due its own movement from that in the environment. A successful experiment would provide the final leg of the visual stability stool by relating an internal input (CD) to the modulation of visual processing (shifting RFs) to the perception of visual stability with saccades. 5. Displacement: Transsaccadic memory and attention The function of CD was put into a different light by MacKay (1972) who argued that a CD should be regarded as a question asked as a movement is generated. The answer is given by the visual input resulting from the movement, and it is evaluated in light of the question. He hypothesized that the world is assumed to be stable unless the answer with a given movement gave evidence to the contrary. One way of looking at the neuronal shifting receptive fields illustrated in Fig. 3 is that they provide that mechanism: the CD-generated future field (FF) is the question and the visual response in the new RF after the saccade is the answer. Prophetically, MacKay also expected to find such activity at a point where visual and movement activity were both represented as is the case in frontal and parietal cortex. Mackay’s approach might also be regarded as a forerunner of the optimal inference approach based on Bayes’ theorem that has been used to explain visual perception at the time of saccades (Binda, Bruno, Burr, & Morrone, 2007; Niemeier, Crawford, & Tweed, 2003). 5.1. Transsaccadic memory A recent hypothesis uses the assumption of stability across saccades to explain the perception of visual stability, and implements the idea by using transsaccadic memory to determine whether the

assumption is correct. Deubel and his collaborators (Deubel, 2004; Deubel, Bridgeman, & Schneider, 2004; Deubel, Schneider, & Bridgeman, 2002) proposed the following steps. First, the features of the saccadic target and of objects immediately surrounding the target are stored in memory. This transsaccadic memory serves as the reference for determining whether the target object and its surround after the saccade are the same as before the saccade. After the saccade, if the objects around what is now the fixation point match those in memory, the assumption of a stable world is correct. If the match fails, the assumption of visual stability is abandoned, and it is assumed the target moved. A key observation of Deubel et al. (Deubel, Bridgeman, & Schneider, 1998; Deubel, Schneider, & Bridgeman, 1996) provided the impetus for this hypothesis. After verifying that even relatively large displacements of the saccade target made during the saccade were not detected (the suppression of displacement considered previously, Bridgeman et al., 1975), they were able to make these displacements obvious with a simple manipulation. If the displaced target was blanked out and not restored until at least 50 ms after the end of the saccade, the displacement was easily detected. When target blanking was entirely within the saccade, displacement was not detected. There is something special about the presence of the target immediately after the saccade which had been emphasized previously (Wolf, Hauske, & Lupp, 1980). This led to the transsaccadic hypothesis that makes the fundamental assumption that if the target remains in the same position before and after the saccade, it and the visual world are stable. This assumption is rejected by the total absence of the target right after the saccade as in this experiment and with large displacements brought about by real world displacements during the saccade. A specific neuronal basis for this transsaccadic memory hypothesis is lacking, but two neuronal observations are relevant. The first is derived from an area that is probably on the pathway to object vision, V4. Moore, Tolias, and Schiller (1998) showed that the visual response of V4 neurons to an optimally oriented stimulus was enhanced just before a saccade was made to the RF of the neuron. It was not enhanced when the saccade was not to the RF. They suggested that the enhanced activity just before the saccade provided reactivation of the response to the visual stimulus before the saccade. This reactivation would facilitate comparison to the post-saccadic stimulus, and this is what is needed for the transsaccadic hypothesis. A second neuronal finding relevant to the transsaccadic memory hypothesis is the activity related to the shifting RFs already considered in detail above. The activity at the future field could be regarded as representing a transsaccadic memory. As shown conceptually in Fig. 3, the activity at the FF indicates the stimulus falling there before the saccade which can then be compared to what falls on the RF after the saccade. This appears to be exactly the mechanism required for the transsaccadic memory. These shifting RF neurons, however, are not specifically coding for objects as required by the transsaccadic hypothesis, but if the object memory could be conveyed by the retinotopic based RFs of frontal or parietal neurons, these neurons could provide the required transsaccadic memory. Alternatively, the same shifting field mechanism might be present in cortical areas devoted to object vision. 5.2. Attention restricts the displacement problem A critical point of the transsaccadic memory hypothesis is that it is only the region close to the saccade target (the future fixation point) for which a transsaccadic memory is relevant; the comparison is made for objects only at or near this point. The rationale for this restriction results from a series of experiments showing increased efficiency of visual processing for visual stimuli located around the target of the upcoming saccade (Deubel & Schneider, 1996; Hoffman

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

& Subramaniam, 1995; Kowler, Anderson, Dosher, & Blaser, 1995). For example, discrimination of one letter among a cluster of letters is best when that letter is the target of the saccade (Deubel & Schneider, 1996). Furthermore in these experiments, it was not possible to direct attention to one location and make a saccade to an adjacent location, which was taken as an indication of the close coupling between this spatial attention and saccades. In the transsaccadic memory hypothesis, therefore, only the objects close to the saccade target are selected for memory across the saccade. More generally, the coupling of visual spatial attention and saccades has implications for any hypothesis that incorporates updating of a retinotopic map. Each time a saccade occurs there is an accompanying shift of visual attention as indicated by the saccade and attention experiments just recounted. The importance of attention for limiting perception to selected locations is further emphasized by the phenomenon referred to as change blindness or inattentional blindness (for reviews see Mack & Rock, 1998; Rensink, 2002; Rensink, O’Regan & Clark, 1997; Simons, 2000), and comparable effects have been demonstrated in monkeys (Cavanaugh & Wurtz, 2004). In a change blindness experiment, a substantial change in the visual scene is not reported by an observer when the sudden onset transient associated with the change is eliminated. Changes occurring in the visual scene during saccadic eye movements also go unnoticed (O’Regan and Noe, 2001). But if the observer’s attention is directed to the region of the visual field with the change, the change is seen instantly. The implication of these experiments is that we probably do not maintain a perceptual image or memory of the entire visual scene but only that part to which we pay attention. The consequence of this for a retinopically based hypothesis for visual stability is that it need not account for a remapping of the whole visual field but just that part of the field to which attention is directed at the time of the saccade. In fact in the experiments described on shifting fields, attention may have a particularly salient role because the experiment is done with flashing lights that attract attention simply by their onset. If there is no such attention, LIP neurons show no RF shift (Gottlieb, Kusunoki, & Goldberg, 1998). The shifting RFs may only be required in the region to which attention is directed, but the extent to which this is true is unknown. 5.3. Conclusion: Transsaccadic memory and updated retinotopic maps The simplifying assumption that there is stability across saccades unless there is evidence to the contrary (MacKay, 1972) has been applied to a hypothesis using object matching across saccades to verify or reject that assumption (Deubel et al., 2002). An intriguing feature of this transsaccadic memory hypothesis is that the neuronal mechanism required for the memory is conceptually similar to that considered for updating a retinotopic map. The activity in the future field (FF) before the saccade is available for comparison to the RF after the saccade. In the remapping hypothesis, the FF activity is used to update a retinotopic map; in the transsaccadic hypothesis the FF activity is used to test the match to the RF after the saccade. At the neuronal level these two hypotheses would be difficult to distinguish. Both might also apply primarily to the region around the saccade target because there is substantial evidence that only those regions attended to at the time of the saccade are relevant to perception. The shifts are still retinotopic, but the retinotopic area is not the full field but just the limited area to which attention is directed. 6. Displacement: Spatiotopic maps A logically attractive hypothesis is that by the time visual input reaches the brain regions related to visual perception, the representation of the visual world is not in the retinotopic coordinates

2079

of the eye, but rather in coordinates of visual space. That is, visual perception is based on a spatiotopic map that supersedes the retinotopic map. As the eye moves, the information with each fixation is incorporated into this higher order spatial map. This hypothesis is appealing because it fits with our personal visual experience: the spatial map is what we are aware of, not the series of snapshots depicted in Fig. 1. At the neuronal level this hypothesis predicts that a spatiotopic visual map should be found at some point in the visual system. The distinguishing feature of a neuron lying in a spatial map is that as the eye moves the RF of the neuron remains activated by a stimulus that remains at one point in the visual field. Neurons on the spatiotopic map are related to different points in space and taken together form a map of space. (For simplicity I refer to eye movements, but head and body movement have the same requirements). At this point no spatiotopic map has been identified in the monkey brain. Neurons have been identified, however, that respond to a visual stimulus but have a response modulated or gated by eye position. The hypothesis is that these neurons form the input to a map or are themselves part of a distributed map that can not be identified at the single neuron level. 6.1. Gain fields and real position neurons Neurons in area V1 fall into a clearly delineated retinotopic map and neither large saccades (Bridgeman, 1973) nor micro-saccades (Gur & Snodderly, 1987, 1997) disrupt that spatial map (for interpretive review see Bridgeman, 1999). In striking contrast, clear modulation of the visual response by eye position was first reported for neurons in posterior parietal cortex by Andersen and Mountcastle (1983). The magnitude of the visual response changed systematically with eye position (Fig. 9A). These neurons were still retinopically organized, but the gain of their visual response was modulated up or down as the eye was directed to different regions of the visual field. Subsequent experiments (Andersen, Essick, & Siegel, 1985) have provided further information on these gain fields (for a review see Andersen, 1989). The basic mapping in other extrastriate areas also appears to be largely retinotopic with a major change being the progressive increase in RF size in successive visual areas which provides a possible mechanism for increased space constancy (Kjaer, Gawne, Hertz, & Richmond, 1997; Salinas & Abbott, 1997). But extrastriate areas also show modulation of visual responses as demonstrated by Galletti and Battaglini (1989); nearly half of the neurons in V3A are modulated by eye position. Such modulation subsequently has been found in a number of areas along the dorsal visual pathway (for a summary see Galletti & Fattori, 2002) as well as in FEF (Cassanello & Ferrera, 2007), supplementary eye field (Schlag, Schlag-Rey, & Pigarev, 1992), and dorsolateral prefrontal cortex (Funahashi & Takeda, 2002). It has been suggested that these gain field neurons might themselves represent a distributed spatiotopic map that is not revealed by the activity of any individual neuron (Andersen et al., 1985; Galletti & Battaglini, 1989). Several models have shown how spatial information could be represented if the gain field neurons were envisioned as one of the steps in producing a spatiotopic map (Galletti & Fattori, 2002; Zipser & Andersen, 1988). For example, in the network model of Zipser and Andersen (1988), ‘‘neurons” in the middle layer were found to have gain fields similar to those of parietal neurons. This of course does not demonstrate the presence of a spatiotopic map, but it does show that such information in a distributed map is conceptually possible. Other neurons have been identified that do respond to stimuli in one region of visual space rather than one point on the retina. These spatiotopic ‘‘real position” neurons were first found in

2080

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

Fig. 9. Possible inputs to a spatial map. (A) An example gain field neuron from posterior parietal cortex. On the left drawing of the screen in front of the monkey, when the monkey looked at the central fixation point (o), the RF was mapped out and was found to be in the lower right quadrant (dashed circle). On the right, the response of the neuron aligned on stimulus onset is shown for fixation at the center of the screen by the record at 0, 0. Then the monkey fixated at one of the other eight positions on the screen (left drawing) each separated by 20°. At each fixation position the stimulus was moved so that it fell on the RF. Even though the stimulus fell on the same retinotopic position, the gain of the visual response changed systematically as the monkey’s eye position changed. These responses are shown on the right. The strongest response was with fixation in the upper left, the weakest with fixation in the lower right. Modified from Andersen et al. (1985). (B) An example real position neuron from area V6A. The stimulus was always in the lower right quadrant (dashed line), and +s indicate the five fixation points on an 80° wide screen. The visual response was nearly the same regardless of the fixation point. Modified from Galletti and Fattori (2002).

parietal cortex (Galletti, Battaglini, & Fattori, 1993), and Fig. 9B shows an example; the response of the neuron remains strong when the visual stimulus remains in one part of the field even when fixation is moved to different points in the visual field. Neurons that ranged from retinotopic to clearly spatiotopic have been found in V6A (Galletti et al., 1993) and in the ventral intraparietal area (Duhamel, Bremmer, BenHamed, & Graf, 1997), and in general real position neurons have been found in areas that overlap those where the gain field neurons are found (Galletti & Fattori, 2002). Thus, these real position neurons directly represent a location in visual space, and a map built up of these real position neurons could represent visual space. Furthermore, multiple gain field neurons could provide the input necessary to produce real position neurons as proposed by Galletti, Battaglini, and Fattori (1995). While there is not enough evidence to say that these real space neurons form a spatiotopic map, they have characteristics that are probably sufficient for them to represent elements of such a map which in turn could be the basis of our perception of visual stability. There is also the possibility that these neurons (as is also the

case for those with shifting RFs) are unrelated to visual stability but instead are part of the transformation leading to movement. For example some neurons in posterior parietal cortex are active before saccades while others are active before reaching (Snyder, Batista, & Andersen, 1997) suggesting a dedication to movement rather than general visual processing. Similarly the gain field neurons in several regions of parietal cortex could contribute to the coordinate conversions preceding the necessary coordination of eye and hand movements (Andersen, Snyder, Bradley, & Xing, 1997; Galletti, Kutz, Gamberini, Breveglieri, & Fattori, 2003; Snyder, Grieve, Brotchie, & Andersen, 1998) rather than producing the spatiotopic reference necessary for visual stability. 6.2. Conclusion: Updating maps Recording neurons in the monkey brain during saccades has provided neuronal evidence for the updating of either retinotopic or spatiotopic maps as the mechanism underlying stable visual perception. In the retinotopic hypothesis, the visual world remains

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

represented in a retinotopic map centered on the fovea. With each saccade, there is a brief transition period in which the map is updated by anticipatory activity in the visual field area where the future receptive field will be located. After the saccade the map is still a retinotopic one. In the spatiotopic hypothesis, there is a spatially based map of the visual world that is updated with each saccade. Or alternatively, in one version of the spatiotopic hypothesis, positions are calculated anew with each new fixation (a calibration hypothesis, Bridgeman et al., 1994). In either case both before and after the saccade the map is in spatial coordinates. In spite of these differences between the retinotopic and spatiotopic hypotheses, there are a few striking similarities in their neuronal requirements. Both require a mechanism for updating a map with each saccade. The mechanisms of the updating might be different for the two types of maps, but both require it. Furthermore, both require vastly increased connections over the simple one to one connection for a retinotopic map. Because both types of map require an updating with each saccade, and this updating can come from any part of a retinotopic map, neurons in both maps must have connections to all parts of the entire retinotopic map. How this could be accomplished has been specified by several models and hypotheses (Galletti et al., 1995; Quaia et al., 1998; Salinas & Sejnowski, 2001; Trehub, 1991). I do not think that the current evidence for the two hypotheses is equal but rather that there are two reasons to favor the retinotopic hypothesis. The first is that the neuronal evidence for the contribution of the shifting RFs to updating a retinotopic map has become substantial and coherent. The studies on shifting RFs have established a possible mechanism for shifting the visual sensitivity on the retinotopic map, and the CD required for driving such a shift has been identified. This detailed knowledge in turn allows a test of the relation of the shifts to the perception of visual stability. In contrast, a spatiotopic map, distributed or not, has yet to be identified, the signal that contributes the eye position signal is not established, and a test is not obvious. Second, at a behavioral level, there is growing evidence from experiments on change blindness that there may be no spatial map in the brain, but rather a transient retinotopic map with each new fixation. If there were a spatiotopic map independent of the current retinal image, why are such changes missed? The spatiotopic map should be representing the whole field all of the time. In contrast, if there were a retinotopic map that changes with each saccade, such omissions are readily understandable. In net, the evidence from change blindness argues against a spatial map, an argument that is independent of the issue of visual stability. Future evidence, particularly at the neuronal level, may change this conclusion on the preeminence of updated retinotopic maps for visual stability, but for now these maps are the core of a working hypothesis—with emphasis on working.

7. Saccadic suppression Saccades move the eye rapidly with peak speeds of about 500°/s for a 20° saccade in humans (Westheimer, 1954) and substantially faster (800°/s) in monkeys (Fuchs, 1967). This high speed of the saccade sweeps the image of the visual scene across the retina producing a blur. Yet we are usually unaware of this blur. This lack of vision during saccades is referred to as saccadic suppression or, to emphasize the lack of awareness of sweep, as saccadic omission (Campbell & Wurtz, 1978). There is also a more specific suppression of motion detection (Burr, Holt, Johnstone, & Ross, 1982) and displacement detection (Bridgeman et al., 1975). Anyone who has looked at an amateur home video appreciates how critical it is to eliminate these disturbing interruptions, be the recording device a camera or the eye.

2081

The first reports of suppression were from the eminently practical experiments of Erdman and Dodge in 1898 who found that during reading, letters presented during saccades were not seen and that all reading occurred during the fixations between saccades (for a concise summary and references for this early history see Volkmann, 1986). The conclusion was that we are functionally blind during saccades, but there was no consensus on the source. Holt argued that during saccades the visual input was ‘‘blanked out” producing a ‘‘central anesthesia”. In contrast, Dodge believed the lack of vision was due to both a central factor and to the interaction of the clear images outside the saccade with the blurred images during it. Interest in these turn of the century issues revived with the ability to accurately record saccades in the 1960s, and these experiments are summarized in excellent reviews (Bridgeman, 1986; Matin, 1974; Volkmann, 1986). A half century of experimentation has established that there are two sources of saccadic suppression: an extraretinal CD and a visual masking mechanism. While there is strong behavioral and neuronal evidence for both mechanisms, it seems likely that in normal high contrast environments visual masking is the dominant mechanism with the contribution of the CD being more modest. 8. Suppression by CD 8.1. Behavioral evidence When saccades are made across normal visual scenes, the speed of the eye movement usually produces a brief blurred image, and it was natural to assume that nothing was needed to account for the suppression of such minimal stimulation. Establishing a role for an extraretinal input for suppression was the first success of quantitative measurements of saccadic suppression in the 1960’s. In these experiments blur was eliminated by using flashed stimuli during saccades made in the dark or against a uniform background. Detection during saccades required flashes about three times as bright as those during fixation (Volkmann, 1962). This was a threshold elevation of about 0.5 log units which was small compared to the brightness of stimuli that might be experienced during a saccade given the many log unit sensitivity range of the visual system. The use of a brief flash also revealed that the suppression began as long as 50 ms before the saccade (Latour, 1962). These two early observations on threshold and timing showed that the suppression resulted from an extraretinal signal which had to be a CD because its effect preceded any proprioceptive input from eye muscle contraction. Detection of motion is also suppressed during saccades as indicated by the observation that a drifting grating moved suddenly required a larger movement during a saccade than during fixation to be detected (Burr et al., 1982). A larger change in motion of random dot patterns also was required for detection during a saccade than during fixation (Bridgeman et al., 1975). We have already noted that target displacement during a saccade is suppressed (Bridgeman et al., 1975), and Shioiri and Cavanagh (1989) showed that motion detection occurred without any displacement by using motion of random dot patterns that had no displacement. This reduction in motion sensitivity is specific for saccades; Burr and Ross (1982) showed that during fixation there was no lack of sensitivity to high speed motion. As motion speed increased, the direction of the motion was simply discriminated at lower and lower spatial frequencies. One of the most striking changes during saccades is the selective reduction of sensitivity to low spatial frequencies (Burr et al., 1982; Volkmann, Riggs, White, & Moore, 1978). In these experiments, horizontal saccades were made over horizontal gratings so that blur was minimal, but reduced visibility of the lower spatial frequencies persisted. This makes good functional sense because

2082

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

the selective suppression of the lower spatial frequencies during saccades would take care of what the high speed of the retinal image had not eliminated. Burr, Morrone, and Ross (1994) showed in addition that while saccades reduced the visibility of luminancemodulated gratings at low spatial frequencies, they did not reduce the perception of equiluminant color-modulated gratings at any spatial frequency. They noted that the reduced sensitivity during saccades to motion, to low spatial frequencies, and specifically to luminance and not color dependent spatial frequencies were characteristics expected from a contribution of the magnocellular layers of the lateral geniculate nucleus (LGN). They suggested that the saccadic suppression seen psychophysically could result from suppression of this segment of the LGN. The identification of other factors contributing to suppression are summarized by Volkmann (1986), but two other observations in particular lend support to the conclusion that a fraction of saccadic suppression results from a CD. First, afterimages show suppression during saccades, and afterimages have the advantage of no possible movement on the retina and no optical changes associated with a saccade. The suppression lasts longer with larger saccades (of consequently longer durations), but is evident even with the saccades as small as 0.5 deg. (Kennard, Hartmann, Kraft, & Boshes, 1970). Second, suppression occurs during blinks which can obscure vision for up to 100–150 ms (longer than the 50 ms of most saccades), and the suppression has characteristics similar to that during saccades (Volkmann, 1986). The magnitude of suppression to a full field flash is about 0.5 log unit threshold elevation (Volkmann, Riggs, & Moore, 1980). Thus, there is clearly a modest reduction in visual sensitivity (about 0.5 log unit) during saccades that results form a CD because it occurs in the absence of any possible visual interactions and begins before the saccade. There are also selective changes in sensitivity to motion, displacement, and low spatial frequency that reflect the action of a CD. The combination of these sensitivity changes are consistent with the LGN as the site of the saccadic suppression. 8.2. Neuronal evidence 8.2.1. The LGN to visual cortex pathway Extraretinal input to the LGN with eye movements has been known since the discovery of pontine-geniculate-occipital (PGO) waves of rapid eye movement sleep (see for example Bizzi, 1966). Multiple possible sources of such modulation have also been identified (Singer, 1977). Whether these modulations contribute to saccadic suppression has been studied more recently usually using full field visual stimulation and looking for changes in LGN visual responses associated with saccades. Reppas, Usrey, and Reid (2002) found changes with saccades. Magnocellular LGN neurons showed the largest changes and the largest proportion of neurons (90%) showing a change. The largest modulation, however, was not primarily a suppression, but a biphasic-small suppression and a later and larger enhancement. The suppression ends about 50 ms after saccade onset and reduces the amplitude of the initial visual response. This should reduce at least the initial burst of visual input generated as a result of the saccade. The cat LGN shows a similar suppression before and facilitation after the saccade (Lee & Malpeli, 1998). Ramcharan, Gnadt, and Sherman (2001) in the monkey found primarily increased visual responses after the saccade also primarily in magnocellular neurons. Much earlier Buttner and Fuchs (1973) found little consistent modulation of response to full field stimuli with saccades and again most of what was found was an increase of activity after the saccade. Other experiments have not measured the suppression of specific visual input, but have looked at the modulation of the background activity. Royal

et al. (Royal, Sary, Schall, & Casagrande, 2006) have summarized previous studies, and in their own studies have found reduction of background activity in all LGN layers beginning on average over 100 ms before the saccade and extending about 50 ms after the saccade. If this background reduction was accompanied by a suppression of the visual response, it would last long enough to act on at least some of the visual input generated by a saccade. They also found that the most consistent and largest modulation was an increase in background activity beginning about 100 ms after the saccade end. In net, while the more prominent modulations in several studies have been in the magnocellular layers, which is consistent with the expectations from psychophysical experiments (summarized by Ross, Burr, & Morrone, 1996), the suppression of responses to visual stimulation have been both short in duration and limited in magnitude. The neurons in primary visual cortex (V1) were the first to be checked for suppression during saccades (Wurtz, 1968, 1969a, 1969b)—indeed, it was the first question to be investigated in the awake monkey trained to fixate in order to provide controlled visual stimulation to one area of the visual field. The response of V1 to a light bar stimulus swept across the receptive field during fixation was similar to that when the saccade swept the eye across the same stationary stimulus. Similar qualitative experiments done subsequently also generally did not observe suppression in V1 (Fischer, Boch, & Bach, 1981) although a few neurons were reported to fail to respond during saccades (Battaglini, Galletti, Aicardi, Squatrito, & Maioli, 1986, see summary in Galletti & Fattori, 2003). The question of a central anesthesia was answered just as it had been answered by psychophysics: there is none. Subtle changes in neuronal responses were not measured, so that changes comparable to the slight rise in threshold seen psychophysically would probably not have been detected. Subsequent experiments on V1 quantified saccade related activity that occurred during memory guided saccades but the change was an increase not a suppression and it preceded saccades by 100–200 ms rather than followed them (Super, van der Togt, Spekreijse, Lamme, & Zd, 2004). Background V1 neuronal activity is modulated with spontaneous saccades in the dark, including some cases with suppression after the saccade (Kayama, Riso, Bartlett, & Doty, 1979), and there is ample evidence of subcortical input to V1 (Doty & Ra, 1983). If this input led to suppression of the visual response, however, we would expect it to be substantial more frequently in the V1 visual responses. While there seems to be little indication of a substantial suppressive mechanism in the LGN to V1 pathway, there is clear evidence of suppression in the extrastriate areas related to visual motion, MT and MST. Thiele, Henning, Kubischik, and Hoffmann (2002) found that the directionally selective neurons in these areas differentiate between motion during fixation and during saccades. About two thirds of the neurons responded to stimuli moving at saccadic velocities while the monkeys fixated but failed to respond when the monkey made saccades across the stationary stimulus (Fig. 10A). For some neurons, the preferred direction during saccades actually reversed and Thiele et al. suggested that this reversal might contribute to reducing any residual motion responses generated by the saccade over a visual background. Ibbotson, Price, Crowder, Ono, and Mustari (2007) subsequently found that over 80% of MT/MST neurons showed a reduced response to the motion produced by saccades compared to the same motion during fixation. While neuronal correlates of pursuit eye movements have not been considered in this review, it is worth noting that as with saccades, during pursuit relatively few neurons in V1 have been found that show evidence of a CD (Dicke, Chakraborty, & Thier, 2008; Galletti, Squatrito, Battaglini, & Grazia Maioli, 1984; Ilg & Thier, 1996). In contrast, the fraction of neurons with such a CD input increases

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

2083

Fig. 10. Saccadic suppression by CD. (A) Response of an MT neuron to a moving visual stimulus (lower record) but failure to do so during a saccadic eye movement (upper record). From Thiele et al. (2002). (B) Example SC superficial layer neuron that responds to a moving visual stimulus in front of the stationary eye but not during a saccade. The visual stimulus moved across the RF (left record) or was moved across the RF by a saccade (right record). Peak of ordinate is 250 sp/s; tick marks on the abscissa are 100 ms. From Robinson and Wurtz (1976). (C) An example SC superficial neuron demonstrated to have input from a corollary discharge. On the left and right is the suppression during normal saccades (in the light to maintain background discharge rate). In the center is the suppression that continues during retrobulbar block of both eyes. All of the records are triggered on the integrated burst of activity of the oculomotor nucleus (Oc. Nuc), that remained even when the EOG for horizontal and vertical components of the saccade were eliminated by the block (center). Peak of ordinate is 20 sp/s; tick marks on the abscissa are 100 ms. From Richmond and Wurtz (1980).

substantially at the levels of MT, MST and posterior parietal cortex (Dicke et al., 2008; Erickson & Thier, 1991; Ilg, Schumann, & Thier, 2004; Newsome, Wurtz, & Komatsu, 1988). Given the limited suppression in LGN and V1, there is the possibility that at least some of the suppression seen in MT results not from V1 input but from the now established pathway from SC to pulvinar to MT. This would be consistent with the frequent failure to find suppression in V1 and success in finding it in areas anterior to V1 (Fischer et al., 1981). A recent study determined the effect of saccades on phosphenes produced by transcranial magnetic stimu-

lation of occipital cortex and concluded that the suppression alters early cortical visual processing (Thilo, Santoro, Walsh, & Blakemore, 2004). This would be consistent with the neuronal evidence if the cortical stimulation affected primarily V2 rather than V1. Finally an fMRI study in humans also showed correlates of suppression at higher levels in the visual pathway, including the MT region, but not at the level of V1 (Kleiser, Seitz, & Krekelberg, 2004). It therefore becomes relevant to consider the modulation with saccades in the other pathway to visual cortex, that leading from SC to MT.

2084

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

8.2.2. The SC–pulvinar–cortex pathway The major source of the visual input to cortex from the SC is via the superficial layers, and in contrast to LGN and V1, neurons in these layers show striking suppression with saccades. Of the SC neurons that responded to rapid stimulus motion, Robinson and Wurtz (1976) found that roughly 2/3 failed to respond when the receptive field was swept across a slit of light by a saccade (Fig. 10B). The SC neurons responded to all four orthogonal motion directions during fixation, and saccades in all these directions produced suppression. Suppression was effective for stimuli at least one log unit above background for all neurons tested, and it lasted from saccade onset to about 125 ms after saccade end. Neurons showing a visual suppression during saccades frequently also showed a suppression of background activity (Goldberg & Wurtz, 1972; Robinson & Wurtz, 1976), and the duration of the two suppressions frequently matched. Thus the saccadic suppression of the SC neurons was robust, came at a time after the saccade when the visual stimulation resulting from the saccade reached the neurons, and correlated with a suppression of the background activity. One source for the suppression could simply be a response to the visual background during the saccade, but the suppression occurred with saccades made in total darkness. This implied that an extraretinal signal was the source of the suppression, and because the neuronal suppression was after the saccade, this signal could logically be either proprioceptive input from the periphery or a CD generated internally. While there were a number of characteristics that suggested the source was not proprioception (Robinson & Wurtz, 1976), this possibility was tested explicitly by Richmond and Wurtz (1980) who reversibly paralyzed the eye muscles in order to eliminate muscle contraction and the resulting proprioception (Fig. 10C). Eye movements were recorded to make certain the eye did not move, and the monkey’s attempts to make saccades were monitored by recording the bursts of activity of extraocular muscle motor neurons. The suppression of the background activity persisted in spite of the lack of movement (10C, center), and because the background suppression accompanied the visual stimulus suppression, it indicated the continued presence of the extraretinal signal in the absence of proprioception. The source of the neuronal suppression must have been a CD. The source of the CD input remains unknown. Robinson and Wurtz (1976) suggested that the source might be the FEF because neurons there discharge after saccades and with saccades in all directions, and it had been suggested that these neurons might represent a CD (Bizzi, 1968). Another source might be the saccade related neurons in the adjacent SC intermediate layers particularly because this connection has recently been demonstrated in the rat (Lee et al., 2007). The intermediate layer neurons connect only to superficial neurons directly above them so this connection would convey activity for just the one saccadic direction represented in the intermediate layers just below. To get the multidirectional suppression shown by the superficial layer neurons, other indirect connections would be required. Regardless of the specific source of input, the neurons in the SC superficial layers show a striking suppression of the visual response, and the most likely source of this suppression is a corollary discharge. The issue is whether this suppression of visual response in the SC during saccades contributes to visual perception. Since suppression with saccades has been relatively minimal in LGN and V1 compared to that in SC and MT, an intriguing possibility is that the perceptual suppression associated with saccades is derived from the SC. The contribution of the SC suppression would be compelling if it were known that the SC neurons were selectively sensitive to low spatial frequencies and could thus provide the selectivity that Burr et al. (1994) envisioned for the LGN. Unfortunately the spatial frequency tuning for the SC visual neurons is unknown.

If SC superficial layer neurons are a potential source of perceptual suppression during saccades, that suppression would probably reach visual cortex via the projection from SC to the pulvinar nucleus of the thalamus. Robinson and collaborators (Robinson, McClurkin, Kertzman, & Petersen, 1991; Robinson & Petersen, 1985) showed that the suppression is clearly present in the pulvinar, and that it has properties similar to those seen in the superior colliculus. Subsequently, Berman (2007) have identified relay neurons in the pulvinar that convey visual information from SC to MT. These relay neurons are centered on the medial region of the inferior pulvinar (using the nomenclature of Stepniewska & Kaas, 1997; Stepniewska, Ql, & Kaas, 2000). Some of these neurons clearly showed the suppression of background activity during saccades (Berman and Wurtz, personal communication) so that this pathway now is known to provide an input to MT that is suppressed during saccades. 8.3. Conclusion: Finding a CD for saccadic suppression Perception shows a suppression that results from a CD because it occurs in the absence of visual interactions and begins before the saccade. The evidence that suppression begins in the magnocellular layers of LGN and is conveyed to V1 cortex is limited. The neuronal suppression seems to be most prominent in visual areas beyond V1, particularly in the motion areas, MT and MST. One idea about a possible added source of that suppression, though a novel one, is that it could be provided by the SC superficial layer neurons, which show a CD based suppression, and that could project upward via the pathway from SC through pulvinar to cortex. In this scenario suppression may not be a function of the geniculocortical path but rather of a collicular cortical path. Regardless of this speculation about pathway, neuronal correlates of saccadic suppression that probably depend on CD have been identified, and this fully validates the psychophysical evidence for a CD based saccadic suppression. 9. Suppression by visual masking 9.1. Behavioral evidence A purely visual mechanism proposed to explain our lack of awareness of the visual blur during a saccade is masking, the interaction between stimuli presented one after the other. Forward masking refers to a masking stimulus blocking perception of a later test stimulus, and backward masking refers to a masking stimulus blocking the perception of a test stimulus that came before it. Both masking types could act during saccades: the blur would be a low contrast test stimulus and the mask would be the high contrast images before and after the saccade. The first experiment to show how potent such a masking effect could be was by Matin, Clymer, and Matin (1972) who showed first that a light bar limited to the duration of the saccade was seen as a longer and longer blur as the bar’s duration increased within the saccade duration. But as soon as the duration extended about 100 ms beyond the end of the saccade, only a bright bar was seen; the blur was gone. Campbell and Wurtz (1978) illustrated this masking further by showing that a contoured visual image (a laboratory room) was blurred when illuminated just during the saccade but became clear when the illumination extended 40– 100 ms after the end of the saccade. The key point in both of these experiments is that with illumination extending beyond the end of the saccade, the perception was not a blur and then a clear image, but just the clear image. Masking had obliterated the blur or ‘‘greyout” (Campbell & Wurtz, 1978). A clear image before the saccade of more than 100 ms produced a forward making effect (Campbell & Wurtz, 1978) which also eliminated perception of the blur. Subse-

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

2085

quent experiments (Corfield, Frosdick, & Campbell, 1978) showed that the elimination of the blur was a purely visual effect. The type of stimulus in the intra-saccadic interval made little difference (Chekaluk & Llewellyn, 1990). Reduced motion perception during the saccade was also shown to be a result of masking by Castet and Masson (2002), 2000). When their subjects viewed stationary vertical gratings briefly presented during the saccade, they appeared to move in the direction opposite to the saccade, that is, in the direction of the grating motion on the retina. As the grating duration extended beyond the duration of the saccade, the probability of reporting this motion dropped precipitously even though the same motion remained during the saccade. The point is that masking contributes to saccadic suppression in a normal high contrast environment whereas the CD related to the saccade itself is independent of the visual background during the saccade (Diamond, Ross, & Morrone, 2000). On the relative role of masking and corollary discharge in saccadic suppression there is little reason to alter the conclusion reached by Volkmann (1986) in her review of 20 years ago: ‘‘Masking effects, therefore, undoubtedly play a primary role in threshold elevations which accompany saccades in lighted, contoured environments. It is equally clear, however, that saccadic suppression can be shown to exist under experimental conditions in which masking effects are minimized . . ..” Visual masking has been extensively studied (for reviews see Breitmeyer, 1984; Bridgeman, 1971; Macknik, 2006; Matin, 1975), and it may well be that one of the most important contributions that masking makes to visual perception is the elimination of the blur during saccades. 9.2. Neuronal correlates The effectiveness of visual masking during saccades was first tested in V1 of awake monkeys by Judge, Wurtz, and Richmond (1980). They first tested how frequently V1 neurons responded to velocities comparable to those of a saccade by using an optimal stimulus: a slit moved across the RF along its long axis so that the integration time of light in the RF was maximal. Even under these optimal conditions only about 50% of the supragranular neurons responded, while over 90% of the infragranular neurons responded. Because the anatomical connections of the supragranular layers are primarily to subsequent cortical visual areas (Van Essen, 1983), these neurons would seem to be the most likely to contribute to perception. Because no more than half of V1 neurons responded to even optimal stimuli moving at saccade speeds, any masking effect need only act on a fraction of V1 neurons, and on weak responses at that. Masking was then studied primarily in the supragranular neurons by determining the interaction between a stimulus falling on the RF before a saccade (comparable to the mask in human psychophysical experiments) and the slit stimulus swept across the RF by the saccade. The response of the V1 neuron to the slit during the saccade was reduced by at least half in 95% of the neurons and eliminated in 35% of them. That this was independent of any CD accompanying saccades was shown by a comparable interaction between a stationary mask and a moving stimulus while the monkey remained fixating (Fig. 11A). The mask was most effective when there was no interval between it and the RF stimulus which would be the case during a saccade. When the masking stimulus fell on the RF after the saccade, there was a backward reduction of the response during the sweep in only a few neurons (Judge et al., 1980). Instead, the mask produced a possible confounding merging of the responses to the two stimuli. A similar effect had been seen in the cat LGN by (Schiller, 1968). Finally, psychophysical experiments measured detection of the stimuli with and without a masking stimulus and showed approximately the same time course for humans as did

Fig. 11. Saccadic suppression by visual masking. (A) Forward masking in a V1 neuron by a masking stimulus falling on the RF (line under each trace) on a subsequent brief RF stimulus (carrot under the trace). In the left column, the eye sweeps the RF across a stationary RF stimulus, and on the right the RF stimulus is swept across a stationary RF during fixation. The top records show the visual response to the RF stimulus only, the middle trace the reduced response to the RF stimulus when preceded by the mask, and the bottom trace to the mask only. Ordinate tick marks are 100 sp/s; abscissa marks are 100 ms. From Judge et al. (1980). (B) Forward and backward masking in a V1 neuron in an alert monkey. Black trace is the response of the neuron to the RF stimulus alone (Target Only). Pink trace shows the elimination of the on response by forward masking. Blue trace shows the elimination of the transient after discharge by backward masking. From Macknik et al. (1998).

the neuronal responses in monkeys, with forward masking being more effective than backward masking. Masking effects have subsequently been quantitatively studied in V1 neurons by Macknik and Livingstone (1998) in order to understand visual masking rather than suppression during saccades. They found a forward masking effect similar to that in the previous observations (Judge et al., 1980). Their masks were adjacent to the target in the RF instead of in the RF, which is probably why their masking reduced responses to about 70% of the unmasked response whereas the reduction during saccades approached 100% (Judge et al., 1980). A key addition in the Macknik and Livingstone experiments (1998) was that for backward

2086

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

masking the effect resulted from the elimination of an off transient occurring about 100 ms after RF stimulus offset (Fig. 11B). Further work showed the importance of spatiotemporal edges in producing such masking effects (Macknik, Martinez-Conde, & Haglund, 2000), factors that must also contribute to the effectiveness of masking during saccades. Other experiments on stimulus interactions have demonstrated that masking effects in V1 have substantial effects on visual responses including those responses generated by saccades (Bridgeman, 1980; Gawne, Woods, & Fp, 2003; Richmond, Hertz, Gawne, & Uk, 1999), and such masking interactions have recently been reviewed by Macknik (2006). Visual interactions also are clear in the responses of the SC superficial layer visual neurons (Bender & Davidson, 1986; Rizzolatti, Camarda, Grupp, & Pisa, 1973; Rizzolatti, Camarda, Grupp, & Pisa, 1974; Wurtz, Richmond, & Judge, 1980). Macknik et al. (1998) showed that an optimal combination of forward and backward masking produced an illusion of stimulus disappearance (the Standing Wave of Invisibility). Since the intervals used in this illusion are in the range of those occurring with saccadic eye movements it is an excellent illustration of the elimination of visual stimulation during saccades. It demonstrates that the stimulus is not so much suppressed as eliminated; it demonstrates the ‘‘saccadic omission” postulated earlier (Campbell & Wurtz, 1978). 9.3. Conclusion: Visual masking rules saccadic suppression Visual masking is likely to be the major factor producing saccadic suppression in normal high contrast environments. Suppression of V1 neuronal responses to the brief stimuli during saccades by stimuli before the saccades (forward masking) alone appears to be substantial enough to provide the underlying mechanism. Furthermore, demonstrations of combined forward and backward masking show that masking can produce not only suppression but the perceptual omission of briefly presented stimuli. Such saccadic omission is what is required to avoid a disruptive pause between successive fixations.

10. Summary and conclusions Understanding why our perception is stable in spite of saccadic interruptions depends on resolving two problems: displacement of the image on the retina after each saccade and suppression of the blur during each saccade. As in many biological systems, there are multiple solutions for each of these problems. For saccadic suppression, the multiple mechanisms are well established: visual masking and corollary discharge (CD). Each mechanism can be dominant depending on visual conditions, with masking being prominent in the normal high contrast environments and CD in low contrast ones. Neuronal mechanisms for suppression by CD have been identified, but rather than acting on the LGN and V1, they seem to act beyond V1. The clearest cortical suppression has been demonstrated in MT, and it might result from input from SC superficial layer neurons where the suppression depends upon a CD input. Neuronal mechanisms for visual masking that are adequate to eliminate the blur during a saccade have been identified in V1. Thus, while a number of critical details are still missing, neuronal correlates for both the CD and the masking mechanisms have been identified. Saccadic displacement is the more central, more difficult, and more interesting problem, both historically and presently. It is hardly surprising that the neuronal mechanism has not been identified, and here we still do not know whether there are multiple neuronal mechanisms. There are two salient neuronal candidates. First are the neurons in frontal and parietal cortex with shifting

receptive fields that provide anticipatory activity with each saccade. These could provide support for the retinotopic hypothesis of visual stability based on updating a retinotopic map. The anticipatory shift in activity might also provide the memory for the transsaccadic memory hypothesis of visual stability. The second neuronal mechanism is provided by neurons whose visual response is modulated by eye position (gain field neurons) or largely independent of eye position (real position neurons). These neurons could provide the basis for a higher order map for a spatiotopic hypothesis of visual stability. On the basis of the current evidence, I conclude that the shifting receptive fields provide the most attractive neuronal basis for understanding the mechanisms for visual stability. At the neuronal level the shift provides a mechanism for shifting the visual sensitivity on a retinotopic map, the CD driving such a shift has been identified, and a test of the relation of this neuronal activity to perceptual stability is now possible. Furthermore, at a behavioral level, the evidence from change blindness suggests that there is no spatial map in the brain but rather a transient retinotopic one with each saccade. Thus, both the neuronal and the behavioral evidence available now make the updating of a retinotopic map an attractive hypothesis. The shifting receptive fields also provide a possible mechanism for a transsaccadic memory hypothesis of visual stability. Happily, even with the techniques now available, these speculative conclusions can be modified, rejected, or replaced by further experimental evidence. Finally, it is important to realize how far we have come in understanding what the neuronal mechanisms underlying visual stability might be. We have moved from general conceptions of the importance of a CD over several centuries, to fortifying the idea with direct experimental evidence from flies and fish beginning in the 1950s, to the more recent neuronal correlates of CD mechanisms in the primate brain. The initial search for neuronal correlates was for a mechanism that might underlie saccadic suppression. Such a neuronal mechanism was thought to simply suppress vision, to act relatively early in the visual processing sequence, and to serve as a cudgel on any visual input following any saccade. What has introduced a more profound understanding of how precisely a CD might function, however, is the understanding of its action in producing shifting receptive fields in parietal and frontal cortex. Starting with the discovery by Duhamel et al. (1992), studies of these neurons have revealed how a motor based CD and visual processing can be seamlessly combined: the vector of the upcoming saccade is used to heighten the sensitivity of neurons on a retinotopic map. All the information both before and after the saccade remains in retinotopic coordinates, but the changes on the map incorporate the motor information. This illustrates an organization that was not envisioned on the basis of the CD concept, that expands our conception of possible ways in which a CD can act, and that emphasizes how far our understanding of possible neuronal mechanisms underlying visual stability have come—and how far they have to go. Acknowledgments I am grateful for assistance on the illustrations by M. Smith, for library assistance by A. Griswold, and for critical comments from the readers of an earlier draft: R. Berman, D. Burr, T. Crapse, H. Deubel, and C. Galletti. References Andersen, R. A. (1989). Visual and eye movement functions of the posterior parietal cortex. Annual Review of Neuroscience, 12, 377–403. Andersen, R. A., Essick, G. K., & Siegel, R. M. (1985). Encoding of spatial location by posterior parietal neurons. Science, 230(4724), 456–458.

R.H. Wurtz / Vision Research 48 (2008) 2070–2089 Andersen, R. A., & Mountcastle, V. B. (1983). The influence of the angle of gaze upon the excitability of the light-sensitive neurons of the posterior parietal cortex. Journal of Neuroscience, 3(3), 532–548. Andersen, R. A., Snyder, L. H., Bradley, D. C., & Xing, J. (1997). Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Annual Review of Neuroscience, 20, 303–330. Batista, A. P., Buneo, C. A., Snyder, L. H., & Andersen, R. A. (1999). Reach plans in eyecentered coordinates. Science, 285(5425), 257–260. Battaglini, P. P., Galletti, C., Aicardi, G., Squatrito, S., & Maioli, M. G. (1986). Effect of fast moving stimuli and saccadic eye movements on cell activity in visual areas V1 and V2 of behaving monkeys. Archives Italiennes de Biologie, 124(2), 111–119. Bender, D. B., & Davidson, R. M. (1986). Global visual processing in the monkey superior colliculus. Brain Research, 381(2), 372–375. Berman, R. A. W. R. H. (2007). The pulvinar pathway to extrastriate cortex. Society of Neuroscience. Abstract, 122.12. Binda, P., Bruno, A., Burr, D. C., & Morrone, M. C. (2007). Fusion of visual and auditory stimuli during saccades: a Bayesian explanation for perisaccadic distortions. Journal of Neuroscience, 27(32), 8525–8532. Bizzi, E. (1966). Discharge patterns of single geniculate neurons during the rapid eye movements of sleep. Journal of Neurophysiology, 29(6), 1087–1095. Bizzi, E. (1968). Discharge of frontal eye field neurons during saccadic and following eye movements in unanesthetized monkeys. Experimental Brain Research, 6, 69–80. Breitmeyer, B. (1984). Visual masking: An integrative approach. Oxford: Clarendon Press. Bridgeman, B. (1971). Metacontrast and lateral inhibition. Psychological Review, 78(6), 528–539. Bridgeman, B. (1973). Receptive fields in single cells of monkey visual cortex during visual tracking. International Journal of Neuroscience, 6(3), 141–152. Bridgeman, B. (1980). Temporal response characteristics of cells in monkey striate cortex measured with metacontrast masking and brightness discrimination. Brain Research, 196(2), 347–364. Bridgeman, B. (1986). Multiple sources of outflow in processing spatial information. Acta Psychology (Amst), 63(1–3), 35–48. Bridgeman, B. (1995). A review of the role of efference copy in sensory and oculomotor control systems. Annals of Biomedical Engineering, 23(4), 409–422. Bridgeman, B. (1999). Neither strong nor weak space constancy is coded in striate cortex. Psychological Research, 62(4), 261–265. Bridgeman, B. (2007). Efference copy and its limitations. Computers in Biology and Medicine, 37(7), 924–929. Bridgeman, B., Hendry, D., & Stark, L. (1975). Failure to detect displacement of the visual world during saccadic eye movements. Vision Research, 15(6), 719–722. Bridgeman, B., Lewis, S., Heit, G., & Nagle, M. (1979). Relation between cognitive and motor-oriented systems of visual position perception. Journal of Experimental Psychology. Human Perception and Performance, 5(4), 692–700. Bridgeman, B., & Stark, L. (1991). Ocular proprioception and efference copy in registering visual direction. Vision Research, 31(11), 1903–1913. Bridgeman, B., Van der Heijden, A. H. C., & Velichkovsky, B. M. (1994). A theory of visual stability across saccadic eye movements. Behavioral Brain Science, 17, 247–292. Burr, D. C., Holt, J., Johnstone, J. R., & Ross, J. (1982). Selective depression of motion sensitivity during saccades. Journal of Physiology, 333, 1–15. Burr, D. C., Morrone, M. C., & Ross, J. (1994). Selective suppression of the magnocellular visual pathway during saccadic eye movements. Nature, 371(6497), 511–513. Burr, D. C., & Ross, J. (1982). Contrast sensitivity at high velocities. Vision Research, 22(4), 479–484. Buttner, U., & Fuchs, A. F. (1973). Influence of saccadic eye movements on unit activity in simian lateral geniculate and pregeniculate nuclei. Journal of Neurophysiology, 36(1), 127–141. Campbell, F. W., & Wurtz, R. H. (1978). Saccadic omission: Why we do not see a grey-out during a saccadic eye movement. Vision Research, 18(10), 1297–1303. Cassanello, C. R., & Ferrera, V. P. (2007). Computing vector differences using a gain field-like mechanism in monkey frontal eye field. J. Physiol., 582(Pt 2), 647–664. Castet, E., Jeanjean, S., & Masson, G. S. (2002). Motion perception of saccade-induced retinal translation. Proc. Natl. Acad. Sci. USA, 99(23), 15159–15163. Castet, E., & Masson, G. S. (2000). Motion perception during saccadic eye movements. Nature Neuroscience, 3(2), 177–183. Cavanaugh, J., & Wurtz, R. H. (2004). Subcortical modulation of attention counters change blindness. Journal of Neuroscience, 24(50), 11236–11243. Chafee, M. V., & Goldman-Rakic, P. S. (1998). Matching patterns of activity in primate prefrontal area 8a and parietal area 7ip neurons during a spatial working memory task. Journal of Neurophysiology, 79(6), 2919–2940. Chafee, M. V., & Goldman-Rakic, P. S. (2000). Inactivation of parietal and prefrontal cortex reveals interdependence of neural activity during memory-guided saccades. Journal of Neurophysiology, 83(3), 1550–1566. Chekaluk, E., & Llewellyn, K. R. (1990). Visual stimulus input, saccadic suppression, and detection of information from the postsaccade scene. Perception & Psychophysics, 48(2), 135–142. Colby, C. L., Duhamel, J. R., & Goldberg, M. E. (1996). Visual, presaccadic, and cognitive activation of single neurons in monkey lateral intraparietal area. Journal of Neurophysiology, 76(5), 2841–2852. Colby, C. L., & Goldberg, M. E. (1999). Space and attention in parietal cortex. Annual Review of Neuroscience, 22, 319–349. Corfield, R., Frosdick, J. P., & Campbell, F. W. (1978). Grey-out elimination: The roles of spatial waveform, frequency and phase. Vision Research, 18(10), 1305–1311.

2087

Deubel, H. (2004). Localization of targets across saccades: Role of landmark objects. Visual Cognition, 11(2-3), 173–202. Deubel, H., Bridgeman, B., & Schneider, W. X. (1998). Immediate post-saccadic information mediates space constancy. Vision Research, 38(20), 3147–3159. Deubel, H., Bridgeman, B., & Schneider, W. X. (2004). Different effects of eyelid blinks and target blanking on saccadic suppression of displacement. Perception & Psychophysics, 66(5), 772–778. Deubel, H., & Schneider, W. X. (1996). Saccade target selection and object recognition: Evidence for a common attentional mechanism. Vision Research, 36(12), 1827–1837. Deubel, H., Schneider, W. X., & Bridgeman, B. (1996). Postsaccadic target blanking prevents saccadic suppression of image displacement. Vision Research, 36(7), 985–996. Deubel, H., Schneider, W. X., & Bridgeman, B. (2002). Transsaccadic memory of position and form. Progress in Brain Research, 140, 165–180. Diamond, M. R., Ross, J., & Morrone, M. C. (2000). Extraretinal control of saccadic suppression. Journal of Neuroscience, 20(9), 3449–3455. Dicke, P. W., Chakraborty, S., & Thier, P. (2008). Neuronal correlates of perceptual stability during eye movements. European Journal of Neuroscience, 27(4), 991–1002. Doty, R. W., & Ra (1983). Nongeniculate afferents to striate cortex in macaques. Journal of Comparative Neurology, 218(2), 159–173. Duhamel, J. R., Bremmer, F., BenHamed, S., & Graf, W. (1997). Spatial invariance of visual receptive fields in parietal cortex neurons. Nature, 389(6653), 845–848. Duhamel, J. R., Colby, C. L., & Goldberg, M. E. (1992). The updating of the representation of visual space in parietal cortex by intended eye movements. Science, 255(5040), 90–92. Erickson, R. G., & Thier, P. (1991). A neuronal correlate of spatial stability during periods of self-induced visual motion. Experimental Brain Research, 86(3), 608–616. Fischer, B., Boch, R., & Bach, M. (1981). Stimulus versus eye movements: comparison of neural activity in the striate and prelunate visual cortex (A17 and A19) of trained rhesus monkey. Experimental Brain Research, 43(1), 69–77. Fuchs, A. F. (1967). Saccadic and smooth pursuit eye movements in the monkey. Journal of Physiology, 191(3), 609–631. Funahashi, S., & Takeda, K. (2002). Information processes in the primate prefrontal cortex in relation to working memory processes. Review of Neuroscience, 13(4), 313–345. Galletti, C., & Battaglini, P. P. (1989). Gaze-dependent visual neurons in area V3A of monkey prestriate cortex. Journal of Neuroscience, 9(4), 1112–1125. Galletti, C., Battaglini, P. P., & Fattori, P. (1993). Parietal neurons encoding spatial locations in craniotopic coordinates. Experimental Brain Research, 96(2), 221–229. Galletti, C., Battaglini, P. P., & Fattori, P. (1995). Eye position influence on the parieto-occipital area PO (V6) of the macaque monkey. European Journal of Neuroscience, 7(12), 2486–2501. Galletti, C., & Fattori, P. (2002). Posterior parietal networks coding visual space. In H.-O. M. Karnath & A. D. G. Vallar (Eds.), The cognitive and neural bases of spatial neglect (pp. 59–69). Oxford, UK: Oxford University Press. Galletti, C., & Fattori, P. (2003). Neuronal mechanisms for detection of motion in the field of view. Neuropsychologia, 41(13), 1717–1727. Galletti, C., Kutz, D. F., Gamberini, M., Breveglieri, R., & Fattori, P. (2003). Role of the medial parieto-occipital cortex in the control of reaching and grasping movements. Experimental in Brain Research, 153(2), 158–170. Galletti, C., Squatrito, S., Battaglini, P. P., & Grazia Maioli, M. (1984). ‘Real-motion’ cells in the primary visual cortex of macaque monkeys. Brain Research, 301(1), 95–110. Gauthier, G. M., Nommay, D., & Vercher, J. L. (1990). The role of ocular muscle proprioception in visual localization of targets. Science, 249(4964), 58–61. Gawne, T. J., Woods, J. M., & Fp (2003). The responses of visual cortical neurons encode differences across saccades. Neuroreport, 14(1), 105–109. Gibson, J. J. (1966). The senses considered as perceptual systems. Boston: Houghton Mifflin. Goldberg, M. E., & Wurtz, R. H. (1972). Activity of superior colliculus in behaving monkey: I. Visual receptive fields of single neurons. Journal of Neurophysiology, 35, 542–559. Gottlieb, J., Kusunoki, M., & Goldberg, M. E. (1998). The representation of visual salience in monkey parietal cortex. Nature, 391, 481–484. Grusser, O. J. (1995). On the history of the ideas of efference copy and reafference. Clio Med., 33, 35–55. Gur, M., & Snodderly, D. M. (1987). Studying striate cortex neurons in behaving monkeys: Benefits of image stabilization. Vision Research, 27(12), 2081–2087. Gur, M., & Snodderly, D. M. (1997). Visual receptive fields of neurons in primary visual cortex (V1) move in space with the eye movements of fixation. Vision Research, 37(3), 257–265. Guthrie, B. L., Porter, J. D., & Sparks, D. L. (1983). Corollary discharge provides accurate eye position information to the oculomotor system. Science, 221, 1193–1195. Hallett, P. E., & Lightstone, A. D. (1976). Saccadic eye movements to flashed targets. Vision Research, 16(1), 107–114. Heiser, L. M., & Colby, C. L. (2006). Spatial updating in area LIP is independent of saccade direction. Journal of Neurophysiology, 95(5), 2751–2767. Hoffman, J. E., & Subramaniam, B. (1995). The role of visual attention in saccadic eye movements. Percept & Psychophysics, 57(6), 787–795.

2088

R.H. Wurtz / Vision Research 48 (2008) 2070–2089

Ibbotson, M. R., Price, N. S., Crowder, N. A., Ono, S., & Mustari, M. J. (2007). Enhanced motion sensitivity follows saccadic suppression in the superior temporal sulcus of the macaque cortex. Cereb. Cortex, 17(5), 1129–1138. Ilg, U. J., Bridgeman, B., & Hoffmann, K. P. (1989). Influence of mechanical disturbance on oculomotor behavior. Vision Res., 29(5), 545–551. Ilg, U. J., Schumann, S., & Thier, P. (2004). Posterior parietal cortex neurons encode target motion in world-centered coordinates. Neuron, 43(1), 145–151. Ilg, U. J., & Thier, P. (1996). Inability of rhesus monkey area V1 to discriminate between self-induced and externally induced retinal image slip. European Journal of Neuroscience, 8(6), 1156–1166. Judge, S. J., Wurtz, R. H., & Richmond, B. J. (1980). Vision during saccadic eye movements. I. Visual interactions in striate cortex. Journal of Neurophysiology, 43, 1133–1155. Kayama, Y., Riso, R. R., Bartlett, J. R., & Doty, R. W. (1979). Luxotonic responses of units in macaque striate cortex. Journal of Neurophysiology, 42(6), 1495–1517. Kennard, D. W., Hartmann, R. W., Kraft, D. P., & Boshes, B. (1970). Perceptual suppression of afterimages. Vision Research, 10(7), 575–585. Kjaer, T. W., Gawne, T. J., Hertz, J. A., & Richmond, B. J. (1997). Insensitivity of V1 complex cell responses to small shifts in the retinal image of complex patterns. Journal of Neurophysiology, 78(6), 3187–3197. Kleiser, R., Seitz, R. J., & Krekelberg, B. (2004). Neural correlates of saccadic suppression in humans. Current Biology, 14(5), 386–390. Kowler, E., Anderson, E., Dosher, B., & Blaser, E. (1995). The role of attention in the programming of saccades. Vision Research, 35(13), 1897–1916. Kusunoki, M., & Goldberg, M. E. (2003). The time course of perisaccadic receptive field shifts in the lateral intraparietal area of the monkey. Journal of Neurophysiology, 89(3), 1519–1527. Latour, P. L. (1962). Visual threshold during eye movements. Vision Research, 2, 261–262. Lee, D., & Malpeli, J. G. (1998). Effects of saccades on the activity of neurons in the cat lateral geniculate nucleus. Journal of Neurophysiology, 79(2), 922–936. Lee, P. H., Sooksawate, T., Yanagawa, Y., Isa, K., Isa, T., & Hall, W. C. (2007). Identity of a pathway for saccadic suppression. Proceedings of the National Academy of Sciences of the United States of America, 104(16), 6824–6827. Lewis, R. F., Zee, D. S., Hayman, M. R., & Tamargo, R. J. (2001). Oculomotor function in the rhesus monkey after deafferentation of the extraocular muscles. Experimental Brain Research, 141(3), 349–358. Lynch, J. C., Hoover, J. E., et al. (1994). Input to the primate frontal eye field from the substantia nigra, superior colliculus, and dentate nucleus demonstrated by transneuronal transport. Experimental Brain Research, 100, 181–186. Mack, A., & Rock, I. (1998). Inattentional blindness. Cambridge MA: MIT Press. MacKay, D. M. (1972). Voluntary eye movements as questions. Bibliography of Ophthalmology, 82, 369–376. Macknik, S. L. (2006). Visual masking approaches to visual awareness. Visual perception, Pt 2: Fundamentals of awareness: multi-sensory integration and highorder perception (Vol. 155, pp. 177–215). Amsterdam: Elsevier Science Bv. Macknik, S. L., & Livingstone, M. S. (1998). Neuronal correlates of visibility and invisibility in the primate visual system. Nature Neuroscience, 1(2), 144–149. Macknik, S. L., Martinez-Conde, S., & Haglund, M. M. (2000). The role of spatiotemporal edges in visibility and visual masking. Proceedings of the National Academy of Sciences of the United States of America, 97(13), 7556–7560. Matin, E. (1974). Saccadic suppression: A review and an analysis. Psychological Bulletin, 81(12), 899–917. Matin, E. (1975). The two-transient (masking) paradigm. Psychological Review, 82(6), 451–461. Matin, E., Clymer, A. B., & Matin, L. (1972). Metacontrast and saccadic suppression. Science, 178(57), 179–182. Matin, L., Matin, E., & Pearce, D. G. (1969). Visual perception of direction when voluntary saccades occur: I. Relation of visual direction of a fixation target extinguished before a saccade to a subsequent test flash present during the saccade. Perception & Psychophysics, 8, 9–14. Matin, L., Matin, E., & Pola, J. (1970). Visual perception of direction when voluntary saccades occur: II. Relation of visual direction of a fixation target extinguished before a saccade to a subsequent test flash presented before the saccade. Perception & Psychophysics, 8, 9–14. Matin, L., & Pearce, D. G. (1965). Visual perception of direction for stimuli flashed during voluntary eye movements. Science, 148, 1485–1488. Matin, L., Picoult, E., Stevens, J. K., Edwards, M. W., Jr., Young, D., & MacArthur, R. (1982). Oculoparalytic illusion: Visual-field dependent spatial mislocalizations by humans partially paralyzed with curare. Science, 216(4542), 198–201. Mays, L. E., Sparks, D. L., & Porter, J. D. (1987). Eye movements induced by pontine stimulation: Interaction with visually triggered saccades. Journal of Neurophysiology, 58, 300–317. McLaughlin, S. C. (1967). Parametric adjustment in saccadic eye movements. Perception and Psychophysics, 2, 359–362. Medendorp, W. P., Goltz, H. C., Vilis, T., & Crawford, J. D. (2003). Gaze-centered updating of visual space in human parietal cortex. Journal of Neuroscience, 23(15), 6209–6214. Melcher, D. (2005). Spatiotopic transfer of visual-form adaptation across saccadic eye movements. Current Biology, 15(19), 1745–1748. Melcher, D. (2007). Predictive remapping of visual features precedes saccadic eye movements. Nature Neuroscience, 10(7), 903–907. Merriam, E. P., Genovese, C. R., & Colby, C. L. (2003). Spatial updating in human parietal cortex. Neuron, 39(2), 361–373. Merriam, E. P., Genovese, C. R., & Colby, C. L. (2007). Remapping in human visual cortex. Journal of Neurophysiology, 97(2), 1738–1755.

Moore, T., Tolias, A. S., & Schiller, P. H. (1998). Visual representations during saccadic eye movements. Proceedings of the National Academy of Sciences of the United States of America, 95(15), 8981–8984. Morrone, M. C., Ross, J., & Burr, D. C. (1997). Apparent position of visual targets during real and simulated saccadic eye movements. Journal of Neuroscience, 17(20), 7941–7953. Nakamura, K., & Colby, C. L. (2002). Updating of the visual representation in monkey striate and extrastriate cortex during saccades. Proceedings of the National Academy of Sciences of the United States of America, 99(6), 4026–4031. Newsome, W. T., Wurtz, R. H., & Komatsu, H. (1988). Relation of cortical areas MT and MST to pursuit eye movements. II. Differentiation of retinal from extraretinal inputs. Journal of Neurophysiology, 60(2), 604–620. Niemeier, M., Crawford, J. D., & Tweed, D. B. (2003). Optimal transsaccadic integration explains distorted spatial perception. Nature, 422(6927), 76–80. O’Regan, J. K., & Noe, A. (2001). A sensorimotor account of vision and visual consciousness. The Behavioral and the Brain Sciences, 24(5), 939–973. discussion 973–1031. Poulet, J. F., & Hedwig, B. (2006). The cellular basis of a corollary discharge. Science, 311(5760), 518–522. Quaia, C., Optican, L. M., & Goldberg, M. E. (1998). The maintenance of spatial accuracy by the perisaccadic remapping of visual receptive fields. Neural Networks, 11(7–8), 1229–1240. Ramcharan, E. J., Gnadt, J. W., & Sherman, S. M. (2001). The effects of saccadic eye movements on the activity of geniculate relay neurons in the monkey. Visual Neuroscience, 18(2), 253–258. Rensink, R. A. (2002). Change detection. Annual Review of Psychology, 53, 245–277. Rensink, R. A., O’Regan, J. K., & Clark, J. J. (1997). To see or not to see: the need for attention to perceive changes in scenes. Psychological Science, 8, 368–373. Reppas, J. B., Usrey, W. M., & Reid, R. C. (2002). Saccadic eye movements modulate visual responses in the lateral geniculate nucleus. Neuron, 35(5), 961–974. Richmond, B. J., Hertz, J. A., Gawne, T. J., & Uk (1999). The relation between V1 neuronal responses and eye movement-like stimulus presentations. Neurocomputing, 247–254. Richmond, B. J., & Wurtz, R. H. (1980). Vision during saccadic eye movements. II. A corollary discharge to monkey superior colliculus. Journal of Neurophysiology, 43(4), 1156–1167. Rizzolatti, G., Camarda, R., Grupp, L. A., & Pisa, M. (1973). Inhibition of visual responses of single units in the cat superior colliculus by the introduction of a second visual stimulus. Brain Research, 61, 390–394. Rizzolatti, G., Camarda, R., Grupp, L. A., & Pisa, M. (1974). Inhibitory effect of remote visual stimuli on visual responses of cat superior colliculus: Spatial and temporal factors. Journal of Neurophysiology, 37(6), 1262–1275. Robinson, D. A. (1964). The mechanics of human saccadic eye movement. Journal of Physiology (London), 174, 245–264. Robinson, D. A. (1968). Eye movement control in primates. Science, 161, 1219–1224. Robinson, D. L., McClurkin, J. W., Kertzman, C., & Petersen, S. E. (1991). Visual responses of pulvinar and collicular neurons during eye movements of awake, trained macaques. Journal of Neurophysiology, 66(2), 485–496. Robinson, D. L., & Petersen, S. E. (1985). Response of pulvinar neurons to real and self-induced stimulus movement. Brain Research, 338, 392–394. Robinson, D. L., & Wurtz, R. H. (1976). Use of an extraretinal signal by monkey superior colliculus neurons to distinguish real from self-induced stimulus movement. Journal of Neurophysiology, 39(4), 852–870. Ross, J., Burr, D., & Morrone, C. (1996). Suppression of the magnocellular pathway during saccades. Behavioral Brain Researh, 80(1–2), 1–8. Ross, J., Morrone, M. C., & Burr, D. C. (1997). Compression of visual space before saccades. Nature, 386(6625), 598–601. Ross, J., Morrone, M. C., Goldberg, M. E., & Burr, D. C. (2001). Changes in visual perception at the time of saccades. Trends in Neuroscience, 24(2), 113–121. Royal, D. W., Sary, G., Schall, J. D., & Casagrande, V. A. (2006). Correlates of motor planning and postsaccadic fixation in the macaque monkey lateral geniculate nucleus. Experimental Brain Research, 168(1–2), 62–75. Salin, P. A., & Bullier, J. (1995). Corticocortical connections in the visual system: Structure and function. Physiological Review, 75(1), 107–154. Salinas, E., & Abbott, L. F. (1997). Invariant visual responses from attentional gain fields. Journal of Neurophysiology, 77(6), 3267–3272. Salinas, E., & Sejnowski, T. J. (2001). Gain modulation in the central nervous system: Where behavior, neurophysiology, and computation meet. Neuroscientist, 7(5), 430–440. Schiller, P. H. (1968). Single unit analysis of backward visual masking and metacontrast in the cat lateral geniculate nucleus. Vision Research, 8(7), 855–866. Schiller, P. H., & Sandell, J. H. (1983). Interactions between visually and electrically elicited saccades before and after superior colliculus and frontal eye field ablations in the rhesus monkey. Experimental Brain Research, 49, 381–392. Schlag, J., Schlag-Rey, M., & Pigarev, I. (1992). Supplementary eye field: Influence of eye position on neural signals of fixation. Experimental Brain Research, 90(2), 302–306. Shioiri, S., & Cavanagh, P. (1989). Saccadic suppression of low-level motion. Vision Research, 29(8), 915–928. Simons, D. J. (2000). Attentional capture and inattentional blindness. Trends in Cognitive Sciences, 4(4), 147–155. Singer, W. (1977). Control of thalamic transmission by corticofugal and ascending reticular pathways in the visual system. Physiological Review, 57(3), 386–420. Snyder, L. H., Batista, A. P., & Andersen, R. A. (1997). Coding of intention in the posterior parietal cortex. Nature, 386(6621), 167–170.

R.H. Wurtz / Vision Research 48 (2008) 2070–2089 Snyder, L. H., Grieve, K. L., Brotchie, P., & Andersen, R. A. (1998). Separate body- and world-referenced representations of visual space in parietal cortex. Nature, 394(6696), 887–891. Sommer, M. A., & Wurtz, R. H. (2002). A pathway in primate brain for internal monitoring of movements. Science, 296(5572), 1480–1482. Sommer, M. A., & Wurtz, R. H. (2004). What the brain stem tells the frontal cortex. I. Oculomotor signals sent from superior colliculus to frontal eye field via mediodorsal thalamus. Journal of Neurophysiology, 91(3), 1381–1402. Sommer, M. A., & Wurtz, R. H. (2006). Influence of the thalamus on spatial visual processing in frontal cortex. Nature, 444(7117), 374–377. Sommer, M. A., & Wurtz, R. H. (2008). Brain circuits for the internal monitoring of movements. Annual Review of Neuroscience, 31, 317–338. Sparks, D. L., & Mays, L. E. (1983). Spatial localization of saccade targets. I. Compensation for stimulation-induced perturbations in eye position. Journal of Neurophysiology, 49, 45–63. Sperry, R. W. (1950). Neural basis of the spontaneous optokinetic response produced by visual inversion. Journal of Comparative Physiology and Psychology, 43(6), 482–489. Stark, L., & Bridgeman, B. (1983). Role of corollary discharge in space constancy. Perception & Psychophysics, 34(4), 371–380. Steinbach, M. J. (1987). Proprioceptive knowledge of eye position. Vision Research, 27(10), 1737–1744. Stepniewska, I., & Kaas, J. H. (1997). Architectonic subdivisions of the inferior pulvinar in New World and Old World monkeys. Visual Neuroscience, 14(6), 1043–1060. Stepniewska, I., Ql, H. X., & Kaas, J. H. (2000). Projections of the superior colliculus to subdivisions of the inferior pulvinar in New World and Old World monkeys. Visual Neuroscience, 17(4), 529–549. Stevens, J. K., Emerson, R. C., Gerstein, G. L., Kallos, T., Neufeld, G. R., Nichols, C. W., et al. (1976). Paralysis of the awake human: Visual perceptions. Vision Research, 16(1), 93–98. Super, H., van der Togt, C., Spekreijse, H., Lamme, V. A. F., & Zd (2004). Correspondence of presaccadic activity in the monkey primary visual cortex with saccadic eye movements. Proceedings of the National Academy of Sciences of the United States of America, 101(9), 3230–3235. Thiele, A., Henning, P., Kubischik, M., & Hoffmann, K. P. (2002). Neural mechanisms of saccadic suppression. Science, 295(5564), 2460–2462. Thilo, K. V., Santoro, L., Walsh, V., & Blakemore, C. (2004). The site of saccadic suppression. Nature Neuroscience, 7(1), 13–14. Tolias, A. S., Moore, T., Smirnakis, S. M., Tehovnik, E. J., Siapas, A. G., & Schiller, P. H. (2001). Eye movements modulate visual receptive fields of V4 neurons. Neuron, 29(3), 757–767. Trehub, A. (1991). The cognitive brain. Cambridge, MA: MIT Press.

2089

Umeno, M. M., & Goldberg, M. E. (1997). Spatial processing in the monkey frontal eye field. I. Predictive visual responses. Journal of Neurophysiology, 78(3), 1373–1383. Umeno, M. M., & Goldberg, M. E. (2001). Spatial processing in the monkey frontal eye field. II. Memory responses. Journal of Neurophysiology, 86(5), 2344–2352. Van Essen, D. C. M., J. H. R. (1983). Hierarchical organization and functional streams in the visual cortex. Trends in Neuroscience, 6, 370–375. Volkmann, F. C. (1962). Vision during voluntary saccadic eye movements. Journal of Optical Society of America, 52, 571–578. Volkmann, F. C. (1986). Human visual suppression. Vision Research, 26(9), 1401–1416. Volkmann, F. C., Riggs, L. A., & Moore, R. K. (1980). Eyeblinks and visual suppression. Science, 207(4433), 900–902. Volkmann, F. C., Riggs, L. A., White, K. D., & Moore, R. K. (1978). Contrast sensitivity during saccadic eye movements. Vision Research, 18(9), 1193–1199. von Helmholtz, H. (1925). Helmholtz’s treatise on physiological optics (J.P.C. Southall, Trans, 1910) (3rd ed.). New York: Optical Society of America. von Holst, E., & Mittelstaedt, H. (1950). Das Reafferenzprinzip. Wechselwirkungen zwischen Zentralnervensystem und Peripherie. Naturwissenschaften, 37, 464–476. Walker, M. F., FitzGibbon, E. J., & Goldberg, M. E. (1994). Predictive visual responses in monkey superior colliculus. In U. Büttner, T. Brandt, A. Fuchs, & D. Zee (Eds.), Contemporary ocular motor and vestibular research: A tribute to David A. Robinson (pp. 512–519). Stuttgart: Thieme. Wang, X., Zhang, M., Cohen, I. S., & Goldberg, M. E. (2007). The proprioceptive representation of eye position in monkey primary somatosensory cortex. Nature Neuroscience, 10(5), 640–646. Westheimer, G. (1954). Eye movement responses to a horizontally moving visual stimulus. AMA Archives of Ophthalmology, 52(6), 932–941. Wolf, W., Hauske, G., & Lupp, U. (1980). Interaction of pre- and postsaccadic patterns having the same coordinates in space. Vision Research, 20(2), 117–125. Wurtz, R. H. (1968). Visual cortex neurons: Response to stimuli during rapid eye movements. Science, 162(858), 1148–1150. Wurtz, R. H. (1969a). Comparison of effects of eye movements and stimulus movements on striate cortex neurons of the monkey. Journal of Neurophysiology, 32, 987–994. Wurtz, R. H. (1969b). Response of striate cortex neurons during rapid eye movements in the monkey. Journal of Neurophysiology, 32, 975–986. Wurtz, R. H., Richmond, B. J., & Judge, S. J. (1980). Vision during saccadic eye movements. III. Visual interactions in monkey superior colliculus. Journal of Neurophysiology, 43(4), 1168–1181. Yarbus, A. L. (1967). Eye movements and vision. New York: Plenum. Zipser, D., & Andersen, R. A. (1988). A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature, 331(6158), 679–684.