Search

Head and eye movements incessantly modulate the luminance signals impinging onto the retina during natural intersaccadic fixation. Yet, little is known about how these fixational movements influence the statistics of retinal stimulation. Here, we provide the first detailed characterization of the visual input to the human retina during normal head-free fixation. We used high-resolution recordings of head and eye movements in a natural viewing task to examine how they jointly transform spatial information into temporal modulations. In agreement with previous studies, we report that both the head and the eyes move considerably during fixation. However, we show that fixational head and eye movements mostly compensate for each other, yielding a spatiotemporal redistribution of the input power to the retina similar to that previously observed under head immobilization. The resulting retinal image motion counterbalances the spectral distribution of natural scenes, giving temporal modulations that are equalized in power over a broad range of spatial frequencies. These findings support the proposal that "ocular drift," the smooth fixational motion of the eye, is under motor control, and indicate that the spatiotemporal reformatting caused by fixational behavior is an important computational element in the encoding of visual information., (Copyright © 2014 the authors 0270-6474/14/3412701-15$15.00/0.)

The early visual system is a model for understanding the roles of cell populations in parallel processing. Cells in this system can be classified according to their responsiveness to different stimuli; a prominent example is the division between cells that respond to stimuli of opposite contrasts (ON vs OFF cells). These two cell classes display many asymmetries in their physiological characteristics (including temporal characteristics, spatial characteristics, and nonlinear characteristics) that, individually, are known to have important roles in population coding. Here we describe a novel distinction between the information that ON and OFF ganglion cell populations carry in mouse--that OFF cells are able to signal motion information about both light and dark objects, while ON cells have a selective deficit at signaling the motion of dark objects. We found that none of the previously reported asymmetries in physiological characteristics could account for this distinction. We therefore analyzed its basis via a recently developed linear-nonlinear-Poisson model that faithfully captures input/output relationships for a broad range of stimuli (Bomash et al., 2013). While the coding differences between ON and OFF cell populations could not be ascribed to the linear or nonlinear components of the model individually, they had a simple explanation in the way that these components interact. Sensory transformations in other systems can likewise be described by these models, and thus our findings suggest that similar interactions between component properties may help account for the roles of cell classes in population coding more generally.

Several recent studies have shown that the ON and OFF channels of the visual system are not simple mirror images of each other, that their response characteristics are asymmetric (Chichilnisky and Kalmar, 2002; Sagdullaev and McCall, 2005). How the asymmetries bear on visual processing is not well understood. Here, we show that ON and OFF ganglion cells show a strong asymmetry in their temporal adaptation to photopic (day) and scotopic (night) conditions and that the asymmetry confers a functional advantage. Under photopic conditions, the ON and OFF ganglion cells show similar temporal characteristics. Under scotopic conditions, the two cell classes diverge-ON cells shift their tuning to low temporal frequencies, whereas OFF cells continue to respond to high. This difference in processing corresponds to an asymmetry in the natural world, one produced by the Poisson nature of photon capture and persists over a broad range of light levels. This work characterizes a previously unknown divergence in the ON and OFF pathways and its utility to visual processing. Furthermore, the results have implications for downstream circuitry and thus offer new constraints for models of downstream processing, since ganglion cells serve as building blocks for circuits in higher brain areas. For example, if simple cells in visual cortex rely on complementary interactions between the two pathways, such as push-pull interactions (Alonso et al., 2001; Hirsch, 2003), their receptive fields may be radically different under scotopic conditions, when the ON and OFF pathways are out of sync.

To make efficient use of their limited signaling capacity, sensory systems often use predictive coding. Predictive coding works by exploiting the statistical regularities of the environment--specifically, by filtering the sensory input to remove its predictable elements, thus enabling the neural signal to focus on what cannot be guessed. To do this, the neural filters must remove the environmental correlations. If predictive coding is to work well in multiple environments, sensory systems must adapt their filtering properties to fit each environment's statistics. Using the visual system as a model, we determine whether this happens. We compare retinal ganglion cell dynamics in two very different environments: white noise and natural. Because natural environments have more power than that of white noise at low temporal frequencies, predictive coding is expected to produce a suppression of low frequencies and an enhancement of high frequencies, compared with the behavior in a white-noise environment. We find that this holds, but only in part. First, predictive coding behavior is not uniform: most on cells manifest it, whereas off cells, on average, do not. Overlaid on this nonuniformity between cell classes is further nonuniformity within both cell classes. These findings indicate that functional considerations beyond predictive coding play an important role in shaping the dynamics of sensory adaptation. Moreover, the differences in behavior between on and off cell classes add to the growing evidence that these classes are not merely homogeneous mirror images of each other and suggest that their roles in visual processing are more complex than expected from the classic view.

The intrinsic dynamics of bipolar cells and rod photoreceptors isolated from tiger salamanders were studied by a patch-clamp technique combined with estimation of effective impulse responses across a range of mean membrane voltages. An increase in external K(+) reduces the gain and speeds the response in bipolar cells near and below resting potential. High external K(+) enhances the inward rectification of membrane potential, an effect mediated by a fast, hyperpolarization-activated, inwardly rectifying potassium current (K(IR)). External Cs(+) suppresses the inward-rectifying effect of external K(+). The reversal potential of the current, estimated by a novel method from a family of impulse responses below resting potential, indicates a channel that is permeable predominantly to K(+). Its permeability to Na(+), estimated from Goldman-Hodgkin-Katz voltage equation, was negligible. Whereas the activation of the delayed-rectifier K(+) current causes bandpass behavior (i.e., undershoots in the impulse responses) in bipolar cells, activation of the K(IR) current does not. In contrast, a slow hyperpolarization-activated current (I(h)) in rod photoreceptors leads to pronounced, slow undershoots near resting potential. Differences in the kinetics and ion selectivity of hyperpolarization-activated currents in bipolar cells (K(IR)) and in rod photoreceptors (I(h)) confer different dynamical behavior onto the two types of neurons.

Characterization of the intrinsic dynamics of isolated retinal bipolar cells by a whole-cell patch-clamp technique combined with estimation of effective impulse responses across a range of mean injected currents reveals strikingly adaptive behavior. At resting potential, bipolar cells' effective impulse response is slow, high gain, and low pass. Depolarization speeds up response, decreases gain, and, in most cells, induces bandpass behavior. This adaptive behavior involves two K(+) currents. The delayed-rectifier accounts for the observed gain reduction, speed increase, and bandpass behavior. The A-channel further shortens the impulse responses but suppresses bandpass features. Computer simulations of model neurons with a delayed-rectifier and varying A-channel conductances reveal that impulse responses largely reflect the flux of electrical charge through the two K(+) channels. The A-channel broadens the frequency response and preempts the action of the delayed-rectifier, thereby reducing the associated bandpass features. Admixtures of the two K(+) channels produce the observed variety of dynamics of retinal bipolar cells.

It has long been debated how humans resolve fine details and perceive a stable visual world despite the incessant fixational motion of their eyes. Current theories assume these processes to rely solely on the visual input to the retina, without contributions from motor and/or proprioceptive sources. Here we show that contrary to this widespread assumption, the visual system has access to high-resolution extra-retinal knowledge of fixational eye motion and uses it to deduce spatial relations. Building on recent advances in gaze-contingent display control, we created a spatial discrimination task in which the stimulus configuration was entirely determined by oculomotor activity. Our results show that humans correctly infer geometrical relations in the absence of spatial information on the retina and accurately combine high-resolution extraretinal monitoring of gaze displacement with retinal signals. These findings reveal a sensory-motor strategy for encoding space, in which fine oculomotor knowledge is used to interpret the fixational input to the retina. It is unknown how humans establish stable visual percepts despite the incessant motion of their eyes. Here the authors report that visual judgments of spatial relations incorporate fine-scale motor knowledge of eye position. [ABSTRACT FROM AUTHOR]

How is space represented in the visual system? At first glance, the answer to this fundamental question appears straightforward: spatial information is directly encoded in the locations of neurons within maps. This concept has long dominated visual neuroscience, leading to mainstream theories of how neurons encode information. However, an accumulation of evidence indicates that this purely spatial view is incomplete and that, even for static images, the representation is fundamentally spatiotemporal. The evidence for this new understanding centers on recent experimental findings concerning the functional role of fixational eye movements, the tiny movements humans and other species continually perform, even when attending to a single point. We review some of these findings and discuss their functional implications. [ABSTRACT FROM AUTHOR]

Understanding how the brain performs computations requires understanding neuronal firing patterns at successive levels of processing—a daunting and seemingly intractable task. Two recent studies have made dramatic progress on this problem by showing how its dimensionality can be reduced. Using the retina as a model system, they demonstrated that multineuronal firing patterns can be predicted by pairwise interactions. [Copyright &y& Elsevier]

Although binocular neurons in the primary visual cortex are sensitive to retinal disparity, their activity does not constitute an unambiguous disparity signal. A multi-spatial-scale neural model for disparity computation is developed to examine how population activity might be interpreted to overcome ambiguities at the single neuron level. The model incorporates a front end that encodes disparity by a family of complex cell-like energy units and a second stage that reads the population activity. Disparity is recovered by matching the population response to a set of canonical templates, derived from the mean response to white noise stimuli at a range of disparities. Model predictions are qualitatively consistent with a variety of psychophysical results in the literature, including the effects of spatial frequency on stereoacuity and bias in perceived depths, and the effect of standing disparity on increment thresholds. Model predictions are also consistent with data on qualitative appearance of complex stimuli, including depth averaging, transparency, and corrugation. The model also accounts for the non-linear interaction of disparities in compound grating stimuli. These results show that a template-match approach reduces ambiguities in individual and pooled neuronal responses, and allows for a broader range of percepts, consistent with psychophysics, than other models. Thus, the pattern of neural population activity across spatial scales is a better candidate for the neural correlate of depth perception than the activity of single neurons or the pooled activity of multiple neurons. [Copyright &y& Elsevier]

Summary Much evidence indicates that humans and other species process large-scale visual information before fine spatial detail. Neurophysiological data obtained with paralyzed eyes suggest that this coarse-to-fine sequence results from spatiotemporal filtering by neurons in the early visual pathway. However, the eyes are normally never stationary: rapid gaze shifts (saccades) incessantly alternate with slow fixational movements. To investigate the consequences of this oculomotor cycle on the dynamics of perception, we combined spectral analysis of visual input signals, neural modeling, and gaze-contingent control of retinal stimulation in humans. We show that the saccade/fixation cycle reformats the flow impinging on the retina in a way that initiates coarse-to-fine processing at each fixation. This finding reveals that the visual system uses oculomotor-induced temporal modulations to sequentially encode different spatial components and suggests that, rather than initiating coarse-to-fine processing, spatiotemporal coupling in the early visual pathway builds on the information dynamics of the oculomotor cycle. [ABSTRACT FROM AUTHOR]

What determines spatial tuning in the visual system? Standard views rely on the assumption that spatial information is directly inherited from the relative position of photoreceptors and shaped by neuronal connectivity. 1,2 However, human eyes are always in motion during fixation, 3,4,5,6 so retinal neurons receive temporal modulations that depend on the interaction of the spatial structure of the stimulus with eye movements. It has long been hypothesized that these modulations might contribute to spatial encoding, 7,8,9,10,11,12 a proposal supported by several recent observations. 13,14,15,16 A fundamental, yet untested, consequence of this encoding strategy is that spatial tuning is not hard-wired in the visual system but critically depends on how the fixational motion of the eye shapes the temporal structure of the signals impinging onto the retina. Here we used high-resolution techniques for eye-tracking 17 and gaze-contingent display control 18 to quantitatively test this distinctive prediction. We examined how contrast sensitivity, a hallmark of spatial vision, is influenced by fixational motion, both during normal active fixation and when the spatiotemporal stimulus on the retina is altered to mimic changes in fixational control. We showed that visual sensitivity closely follows the strength of the luminance modulations delivered within a narrow temporal bandwidth, so changes in fixational motion have opposite visual effects at low and high spatial frequencies. By identifying a key role for oculomotor activity in spatial selectivity, these findings have important implications for the perceptual consequences of abnormal eye movements, the sources of perceptual variability, and the function of oculomotor control. • Human eyes are always in motion, even during fixation • This fixational eye drift transforms spatial information into temporal modulations • Visual sensitivity closely follows the strength of these fixational modulations • Changes in drift speed have opposite effects on coarse and fine pattern detection Intoy et al. show that the spatial selectivity of the visual system critically depends on the strength of the temporal signals resulting from small eye movements that humans continually perform during fixation. [ABSTRACT FROM AUTHOR]

We perceive the world based on visual information acquired via oculomotor control, 1 an activity intertwined with ongoing cognitive processes. 2,3,4 Cognitive influences have been primarily studied in the context of macroscopic movements, like saccades and smooth pursuits. However, our eyes are never still, even during periods of fixation. One of the fixational eye movements, ocular drifts, shifts the stimulus over hundreds of receptors on the retina, a motion that has been argued to enhance the processing of spatial detail by translating spatial into temporal information. 5 Despite their apparent randomness, ocular drifts are under neural control. 6,7,8 However little is known about the control of drift beyond the brainstem circuitry of the vestibulo-ocular reflex. 9,10 Here, we investigated the cognitive control of ocular drifts with a letter discrimination task. The experiment was designed to reveal open-loop effects, i.e., cognitive oculomotor control driven by specific prior knowledge of the task, independent of incoming sensory information. Open-loop influences were isolated by randomly presenting pure noise fields (no letters) while subjects engaged in discriminating specific letter pairs. Our results show open-loop control of drift direction in human observers. [Display omitted] • Seemingly involuntary fixational eye movements are under cognitive influence • Specific task knowledge influences the dominant orientation of these ocular drifts • This influence is open-loop, as it can occur in the absence of visual information • Cognitive drift modulations can predict the specific task with above-chance accuracy Task demands are known to direct gaze to informative locations. Lin et al. show that task also modulates tiny, seemingly involuntary eye movements present during fixation, even when no visual information is present. Cognitive modulation of drift is of particular interest because it reveals a new aspect of cortical/brainstem functional interaction. [ABSTRACT FROM AUTHOR]

Humans use rapid gaze shifts, known as saccades, to explore visual scenes. These movements yield abrupt luminance changes on the retina, which elicit robust neural discharges at fixation onsets. Yet little is known about the spatial content of saccade transients. Here, we show that saccades redistribute spatial information within the temporal range of retinal sensitivity following two distinct regimes: saccade modulations counterbalance (whiten) the spectral density of natural scenes at low spatial frequencies and follow the external power distribution at higher frequencies. This redistribution is a consequence of saccade dynamics, particularly the speed/amplitude/duration relation known as the main sequence. It resembles the redistribution resulting from inter-saccadic eye drifts, revealing a continuum in the modulations given by different eye movements, with oculomotor transitions primarily acting by regulating the bandwidth of whitening. Our findings suggest important computational roles for saccade transients in the establishment of spatial representations and lead to testable predictions about their consequences for visual functions and encoding mechanisms. • By formatting the input flow, saccades play an important computational role in vision • At low spatial frequencies, saccades discard redundant information in natural scenes • The bandwidth of this effect (whitening) increases as saccade amplitude decreases • The whitening bandwidth oscillates during the natural saccade/fixation cycle Humans use saccades (rapid gaze shifts) to explore visual scenes. Mostofi et al. show that these eye movements flexibly reformat the luminance flow on the retina in a way that selectively discards redundant information present in natural scenes. These results reveal a matching between saccade dynamics and the characteristics of the natural world. [ABSTRACT FROM AUTHOR]