Search

Statistical models are increasingly being used to understand the complexity of stimulus selectivity in primary visual cortex (V1) in the context of complex time-varying stimuli, replacing averaging responses to simple parametric stimuli. Although such models often can more accurately reflect the computations performed by V1 neurons in more natural visual environments, they do not by themselves provide insight into V1 neural selectivity to basic stimulus features such as receptive field size, spatial frequency tuning, and phase invariance. Here, we present a battery of analyses that can be directly applied to encoding models to link complex encoding models to more interpretable aspects of stimulus selectivity. We apply this battery to nonlinear models of V1 neurons recorded in awake macaque during random bar stimuli. In linking model properties to more classical measurements, we demonstrate several novel aspects of V1 selectivity not available to simpler experimental measurements. For example, this approach reveals that individual spatiotemporal elements of the V1 models often have a smaller spatial scale than the neuron as a whole, resulting in nontrivial tuning to spatial frequencies. In addition, we propose measures of nonlinear integration that suggest that classical classifications of V1 neurons into simple versus complex cells will be spatial-frequency dependent. In total, rather than obfuscate classical characterizations of V1 neurons, model-based characterizations offer a means to more fully understand their selectivity, and link their classical tuning properties to their roles in more complex, natural, visual processing. NEW & NOTEWORTHY Visual neurons are increasingly being studied with more complex, natural visual stimuli, and increasingly complex models are necessary to characterize their response properties. Here, we describe a battery of analyses that relate these more complex models to classical characterizations. Using such model-based characterizations of V1 neurons furthermore yields several new insights into V1 processing not possible to capture in more classical means to measure their visual selectivity. [ABSTRACT FROM AUTHOR]

Neurons throughout the brain modulate their firing rate lawfully in response to sensory input. Theories of neural computation posit that these modulations reflect the outcome of a constrained optimization in which neurons aim to robustly and efficiently represent sensory information. Our understanding of how this optimization varies across different areas in the brain, however, is still in its infancy. Here, we show that neural sensory responses transform along the dorsal stream of the visual system in a manner consistent with a transition from optimizing for information preservation towards optimizing for perceptual discrimination. Focusing on the representation of binocular disparities—the slight differences in the retinal images of the two eyes—we re-analyze measurements characterizing neuronal tuning curves in brain areas V1, V2, and MT (middle temporal) in the macaque monkey. We compare these to measurements of the statistics of binocular disparity typically encountered during natural behaviors using a Fisher Information framework. The differences in tuning curve characteristics across areas are consistent with a shift in optimization goals: V1 and V2 population-level responses are more consistent with maximizing the information encoded about naturally occurring binocular disparities, while MT responses shift towards maximizing the ability to support disparity discrimination. We find that a change towards tuning curves preferring larger disparities is a key driver of this shift. These results provide new insight into previously-identified differences between disparity-selective areas of cortex and suggest these differences play an important role in supporting visually-guided behavior. Our findings emphasize the need to consider not just information preservation and neural resources, but also relevance to behavior, when assessing the optimality of neural codes. Author summary: The nervous system needs to transform information from the sensory organs into signals that can be used to guide behavior. Neural activity is noisy and can consume large amount of energy, so sensory neurons must optimize their information processing so as to limit energy consumption while maintaining key behaviorally-relevant information. In this report, we re-examine classically-defined brain areas in the visual processing hierarchy, and ask whether neurons in these areas vary lawfully in how they represent sensory information. Our results suggest that neurons in these brain areas shift from being an optimal conduit for conveying sensory information towards prioritizing information that supports key perceptual discriminations during natural tasks. [ABSTRACT FROM AUTHOR]

Fine judgments of stereoscopic depth rely mainly on relative judgments of depth (relative binocular disparity) between objects, rather than judgments of the distance to where the eyes are fixating (absolute disparity). In macaques, visual area V2 is the earliest site in the visual processing hierarchy for which neurons selective for relative disparity have been observed (Thomas et al., 2002). Here, we found that, in macaques trained to perform a fine disparity discrimination task, disparity-selective neurons in V2 were highly selective for the task, and their activity correlated with the animals' perceptual decisions (unexplained by the stimulus). Thismaypartially explain similar correlations reported in downstream areas. Although compatible with a perceptual role of these neurons for the task, the interpretation of such decision-related activity is complicated by the effects of interneuronal "noise" correlations between sensory neurons. Recent work has developed simple predictions to differentiate decoding schemes (Pitkow et al., 2015) without needing measures of noise correlations, and found that data from early sensory areas were compatible with optimal linear readout of populations with informationlimiting correlations. In contrast, our data here deviated significantly from these predictions. We additionally tested this prediction for previously reported results of decision-related activity in V2 for a related task, coarse disparity discrimination (Nienborg and Cumming, 2006), thought to rely on absolute disparity. Although these data followed the predicted pattern, they violated the prediction quantitatively. This suggests that optimal linear decoding of sensory signals is not generally a good predictor of behavior in simple perceptual tasks. [ABSTRACT FROM AUTHOR]

The ability to distinguish between elements of a sensory neuron's activity that are stimulus independent versus driven by the stimulus is critical for addressing many questions in systems neuroscience. This is typically accomplished by measuring neural responses to repeated presentations of identical stimuli and identifying the trial-variable components of the response as noise. In awake primates, however, small "fixational" eye movements (FEMs) introduce uncontrolled trial-to-trial differences in the visual stimulus itself, potentially confounding this distinction. Here, we describe novel analytical methods that directly quantify the stimulus-driven and stimulusindependent components of visual neuron responses in the presence of FEMs. We apply this approach, combined with precise model-based eye tracking, to recordings from primary visual cortex (V1), finding that standard approaches that ignore FEMs typically miss more than half of the stimulus-driven neural response variance, creating substantial biases in measures of response reliability. We show that these effects are likely not isolated to the particular experimental conditions used here, such as the choice of visual stimulus or spike measurement time window, and thus will be a more general problem for V1 recordings in awake primates. We also demonstrate that measurements of the stimulus-driven and stimulus-independent correlations among pairs of V1 neurons can be greatly biased by FEMs. These results thus illustrate the potentially dramatic impact of FEMs on measures of signal and noise in visual neuron activity and also demonstrate a novel approach for controlling for these eye-movement-induced effects. [ABSTRACT FROM AUTHOR]

In order to extract retinal disparity from a visual scene, the brain must match corresponding points in the left and right retinae. This computationally demanding task is known as the stereo correspondence problem. The initial stage of the solution to the correspondence problem is generally thought to consist of a correlation-based computation. However, recent work by Doi et al suggests that human observers can see depth in a class of stimuli where the mean binocular correlation is 0 (half-matched random dot stereograms). Half-matched random dot stereograms are made up of an equal number of correlated and anticorrelated dots, and the binocular energy model—a well-known model of V1 binocular complex cells—fails to signal disparity here. This has led to the proposition that a second, match-based computation must be extracting disparity in these stimuli. Here we show that a straightforward modification to the binocular energy model—adding a point output nonlinearity—is by itself sufficient to produce cells that are disparity-tuned to half-matched random dot stereograms. We then show that a simple decision model using this single mechanism can reproduce psychometric functions generated by human observers, including reduced performance to large disparities and rapidly updating dot patterns. The model makes predictions about how performance should change with dot size in half-matched stereograms and temporal alternation in correlation, which we test in human observers. We conclude that a single correlation-based computation, based directly on already-known properties of V1 neurons, can account for the literature on mixed correlation random dot stereograms. [ABSTRACT FROM AUTHOR]

How the processing of signals carried by sensory neurons supports perceptual decisions is a long-standing question in neuroscience. The ability to record neuronal activity in awake animals while they perform psychophysical tasks near threshold has been a key advance in studying these questions. Trial-to-trial correlations between the activity of sensory neurons and the decisions reported by animals (‘choice probabilities’), even when measured across repeated presentations of an identical stimulus provide insights into this problem. But understanding the sources of such co-variability between sensory neurons and behavior has proven more difficult than it initially appeared. Below, we discuss our current understanding of what gives rise to these correlations. [ABSTRACT FROM AUTHOR]

The stereo correspondence problem poses a challenge to visual neurons because localized receptive fields potentially cause false responses. Neurons in the primary visual cortex (V1) partially resolve this problem by combining excitatory and suppressive responses to encode binocular disparity. We explored the time course of this combination in awake, monkey V1 neurons using subspace mapping of receptive fields. The stimulus was a binocular noise pattern constructed from discrete spatial frequency components. We forward correlated the firing of the V1 neuron with the occurrence of binocular presentations of each spatial frequency component. The forward correlation yielded a complete set of response time courses to every combination of spatial frequency and interocular phase difference. Some combinations produced suppressive responses. Typically, if an interocular phase difference for a given spatial frequency produced strong excitation, we saw suppression in response to the opposite interocular phase difference at lower spatial frequencies. The suppression was delayed relative to the excitation, with a median difference in latency of 7 ms. We found that the suppressive mechanism explains a well-known mismatch of monocular and binocular signals. The suppressive components increased power at low spatial frequencies in disparity tuning, whereas they reduced the monocular response to low spatial frequencies. This long-recognized mismatch of binocular and monocular signals reflects a suppressive mechanism that helps reduce the response to false matches. [ABSTRACT FROM AUTHOR]

Many studies have reported correlations between the activity of sensory neurons and animals' judgments in discrimination tasks. Here, we suggest that such neuron-behavior correlations may require a cortical map for the task relevant features. This would explain why studies using discrimination tasks based on disparity in area V1 have not found these correlations: V1 contains no map for disparity. This scheme predicts that activity of V1 neurons correlates with decisions in an orientation-discrimination task. To test this prediction, we trained two macaque monkeys in a coarse orientation discrimination task using band-pass-filtered dynamic noise. The two orientations were always 90° apart and task difficulty was controlled by varying the orientation bandwidth of the filter. While the trained animals performed this task, we recorded from orientation-selective V1 neurons (n=82, n=31 for Monkey 1,n=51 for Monkey2). For both monkeys, we observed significant correlation (quantified as "choice probabilities") of the V1 activity with the monkeys' perceptual judgments (mean choice probability 0.54, p= 10–5). In one of these animals, we had previously measured choice probabilities in a disparity discrimination task in V1, which had been at chance (0.49, not significantly different from 0.5). The choice probabilities in this monkey for the orientation discrimination task were significantly larger than those for the disparity discrimination task (p=0.032). These results are predicted by our suggestion that choice probabilities are only observed for cortical sensory neurons that are organized in maps for the task-relevant feature. [ABSTRACT FROM AUTHOR]

Direction selective neurons in macaque primary visual cortex are narrowly tuned for orientation, and are thus afflicted by the aperture problem. At the next stage of motion processing, in the middle temporal (MT) area, some cells appear to solve this problem, responding to the pattern motion direction of plaids. Models have been proposed to account for this computation, but they do not replicate the diversity of responses observed in MT. We recorded from 386 cells in area MT of two male macaques, while presenting a wide range of random-line stimuli and their compositions into noise plaids. As we broadened the range of stimuli used to probe the cells, yielding ever more challenging conditions for extracting pattern motion, the diversity of the responses observed increased, and the fraction of cells that faithfully encoded pattern motion direction shrank. However, we show here that a pattern motion signal is present at the population level. We identified four mechanisms, one never proposed before, that together might account for the observed diversity in single-cell responses. Pattern motion is thus extracted in area MT, but it is encoded across the population, and not in a small subset of pattern neurons. [ABSTRACT FROM AUTHOR]

During perceptual decisions, the activity of sensory neurons correlates with a subject’s percept, even when the physical stimulus is identical. The origin of this correlation is unknown. Current theory proposes a causal effect of noise in sensory neurons on perceptual decisions, but the correlation could result from different brain states associated with the perceptual choice (a top-down explanation). These two schemes have very different implications for the role of sensory neurons in forming decisions. Here we use white-noise analysis to measure tuning functions of V2 neurons associated with choice and simultaneously measure how the variation in the stimulus affects the subjects’ (two macaques) perceptual decisions. In causal models, stronger effects of the stimulus upon decisions, mediated by sensory neurons, are associated with stronger choice-related activity. However, we find that over the time course of the trial these measures change in different directions—at odds with causal models. An analysis of the effect of reward size also supports this conclusion. Finally, we find that choice is associated with changes in neuronal gain that are incompatible with causal models. All three results are readily explained if choice is associated with changes in neuronal gain caused by top-down phenomena that closely resemble attention. We conclude that top-down processes contribute to choice-related activity. Thus, even forming simple sensory decisions involves complex interactions between cognitive processes and sensory neurons. [ABSTRACT FROM AUTHOR]

Stereo vision relies on cortical signals that encode binocular disparity. In V1, the disparity energy model explains many features of binocular interaction, but it overestimates the responses to anticorrelated images. Combining the outputs of two, or more, energy model-like subunits [two-subunit (2SU) model] can resolve this discrepancy and provides an alternative explanation for disparity signals previously thought to indicate phase disparity between the receptive fields (RFs) of each eye. The 2SU model naturally explains how "near/far" (odd-symmetric) tuning becomes dominant in extrastriate cortex. To compare the energy and the 2SU models, we used a broadband compound grating and applied a common interocular phase difference to all spatial frequency components (a stimulus phase disparity), combined with a common spatial displacement (a stimulus position disparity). This produces binocular images that never occur in natural viewing, for which the 2SU model and the energy model make distinctively different predictions. Responses of neurons recorded from both V1 and V2 of awake rhesus macaques systematically deviated from the predictions of the energy model, in accordance with the 2SU model. These deviations correlated with the symmetry of the tuning curve, indicating that the 2SU mechanism is exploited to produce odd symmetry. Nonetheless, individual subunits also contain RF phase disparity that contributes to odd symmetry. The results suggest that neurons in V2 probably inherit phase disparity signals from V1 neurons, but systematically combine input from V1 neurons with different position disparities, in a way that elaborates odd-symmetric tuning and extends the range of disparities encoded by single neurons. [ABSTRACT FROM AUTHOR]

Useful representations of the three-dimensional (3D) world go beyond assigning depth to individual points, building maps of surfaces and shapes. Studies in a wide range of extrastriate cortical areas have shown that single neurons show selective responses to 3D surfaces. The extent to which this advances the representation beyond that provided by the earliest binocular signals requires careful evaluation. We conclude that current data are not sufficient to identify distinctive contributions from different cortical areas to the binocular representation of 3D surfaces. [Copyright &y& Elsevier]

Summary: Sensory processing in the brain is thought to have evolved to encode naturally occurring stimuli efficiently. We report an adaptation in binocular cortical neurons that reflects the tight constraints imposed by the geometry of 3D vision. We show that the widely used binocular energy model predicts that neurons dedicate part of their dynamic range to impossible combinations of left and right images. Approximately 42% of the neurons we record from V1 of awake monkeys behave in this way (a powerful confirmation of the model), while about 58% deviate from the model in a manner that concentrates more of their dynamic range on stimuli that obey the constraints of binocular geometry. We propose a simple extension of the energy model, using multiple subunits, that explains the adaptation we observe, as well as other properties of binocular neurons that have been hard to account for, such as the response to anticorrelated stereograms. [Copyright &y& Elsevier]

In perceptual tasks, subjects attempt to rely on their most informative cues. Such strategic choices should be reflected in the types of sensory neurons that are used. We investigated this in a binocular-disparity discrimination task. Using psychophysical reverse-correlation, also known as image classification, we identified the perceptual strategies of two macaques (Macaca mulatta). Correlations between reported disparity signs and disparity noise samples for each trial yielded detection 'filters'. Filter amplitude was greater at near disparities than at far disparities, indicating that the subjects relied more on near disparities. Recordings from both macaques' disparity-selective V2 neurons showed a correlation between neuronal responses and perceptual judgment in near-preferring, but not far-preferring, units, mirroring the psychophysically measured strategy. After one monkey learned to weight near and far disparities equally, activity in its far-preferring neurons correlated with choice. Thus, the pattern of correlations between neuronal activity and perceptual reports indicates how subjects use their neuronal signals. [ABSTRACT FROM AUTHOR]

One of the fundamental challenges of binocular vision is that objects project to different positions on the two retinas (binocular disparity). Neurons in visual cortex show two distinct types of tuning to disparity, position and phase disparity, which are the results of differences in receptive field location and profile, respectively. Here, we point out that phase disparity does not occur in natural images. Why, then, should the brain encode it? We propose that phase-disparity detectors help to work out which feature in the left eye corresponds to a given feature in the right. This correspondence problem is plagued by false matches: regions of the image that look similar, but do not correspond to the same object. We show that phase-disparity neurons tend to be more strongly activated by false matches. Thus, they may act as 'lie detectors', enabling the true correspondence to be deduced by a process of elimination. [ABSTRACT FROM AUTHOR]

Because the eyes are displaced horizontally, binocular vision is inherently anisotropic. Recent experimental work has uncovered evidence of this anisotropy in primary visual cortex (V1): neurons respond over a wider range of horizontal than vertical disparity, regardless of their orientation tuning. This probably reflects the horizontally elongated distribution of two-dimensional disparity experienced by the visual system, but it conflicts with all existing models of disparity selectivity, in which the relative response range to vertical and horizontal disparities is determined by the preferred orientation. Potentially, this discrepancy could require us to abandon the widely held view that processing in V1 neurons is initially linear. Here, we show that these new experimental data can be reconciled with an initial linear stage; we present two physiologically plausible ways of extending existing models to achieve this. First, we allow neurons to receive input from multiple binocular subunits with different position disparities (previous models have assumed all subunits have identical position and phase disparity). Then we incorporate a form of divisive normalization, which has successfully explained many response properties of V1 neurons but has not previously been incorporated into a model of disparity selectivity. We show that either of these mechanisms decouples disparity tuning from orientation tuning and discuss how the models could be tested experimentally. This represents the first explanation of how the cortical specialization for horizontal disparity may be achieved. [ABSTRACT FROM AUTHOR]

We address two unresolved issues concerning the coding of binocular disparity in primary visual cortex. Experimental studies and theoretical models have suggested a relationship between a cell's ocular dominance, assessed with monocular stimuli, and its tuning to binocular disparity. First, the disparity energy model of disparity selectivity suggests that there should be a correlation between ocular dominance and the strength of disparity tuning. Second, several studies have reported a relationship between ocular dominance and the shape of the disparity tuning curve, with cells dominated by one eye more likely to have disparity tuning of the tuned-inhibitory type. We investigated both of these relationships in single neurons recorded from the primary visual cortex of awake fixating macaques, using dynamic random-dot patterns as a stimulus. To classify disparity tuning curves quantitatively, we develop a new measure of symmetry, which can be applied to any function. We find no evidence for any correlation between ocular dominance and the nature of disparity tuning. This places constraints on the circuitry underlying disparity tuning. [ABSTRACT FROM AUTHOR]

Disparity-selective neurons in striate cortex (V1) probably implement the initial processing that supports binocular vision. Recently, much progress has been made in understanding the computations that these neurons perform on retinal inputs. The binocular energy model has been highly successful in providing a simple theory of these computations. A key feature of the energy model is that it is linear until after inputs from the two eyes are combined. Recently, however, a modified version of the energy model, incorporating threshold nonlinearities before binocular combination, has been proposed to account for the weaker disparity tuning observed with anticorrelated stimuli. In this study, we present new data needed for a critical assessment of these two models. We compare two key predictions of the models with responses of disparity-selective neurons recorded from V1 of awake fixating monkeys. We find that the original energy model, and a family of generalizations retaining linear binocular combination, are quantitatively inconsistent with the response of V1 neurons. In contrast, the modified version incorporating threshold nonlinearities can explain both sets of observations. We conclude that the energy model can be reconciled with experimental observations by adding a threshold before binocular combination. This gives us the clearest picture yet of the computation being carried out by disparity-selective V1 neurons. [ABSTRACT FROM AUTHOR]

Stereoscopic depth perception relies on binocular disparities, or small geometric differences between the retinal images of each eye. The most reliable binocular depth judgments are those that are based on relative disparities between two simultaneously visible features in a scene. Many cortical areas contain neurons that are sensitive to disparity, but it is unclear whether any areas show a specific sensitivity to relative disparity. We recorded from neurons in the early cortical visual area V2 of the awake macaque during presentation of random-dot patterns. The depth of a central region ('center'), and that of an annular surrounding region ('surround'), were manipulated independently in these stimuli. Some cells were fully selective for the resulting relative disparities. Most showed partial selectivity, which nonetheless indicated a sensitivity for the depth relationship between center and surround. Both types of neural response could support psychophysical judgments of relative depth. [ABSTRACT FROM AUTHOR]

Presents research which showed that visual area MT in the brain is important in stereoscopic vision. What stereopsis is; Role of disparity-selective neurons in stereopsis; Behaviorally relevant signals concerning stereoscopic depth present in MT.

The article presents the author's views on the results of the study by H. R. Kim and colleagues on primate cortical area middle temporal (NT) neurons. The results reportedly revealed that primate cortical area MT is sensitive to the depth that are defined by dynamical perspective cues. It notes that the depth can be computed via recurrent circuits that involves signals in MT's downstream.

Summary How does our brain use differences between the images in our two eyes, binocular disparities, to generate depth perception? New work shows that a type of neural network trained on natural binocular images can learn parameters that match key properties of visual cortex. Most information is conveyed by cells which sense differences between the two eyes’ images. [ABSTRACT FROM AUTHOR]

Sensory neurons are activated by a range of stimuli to which they are said to be tuned. Usually, they are also suppressed by another set of stimuli that have little effect when presented in isolation. The interactions between preferred and suppressive stimuli are often quite complex and vary across neurons, even within a single area, making it difficult to infer their collective effect on behavioral responses mediated by activity across populations of neurons. Here, we investigated this issue by measuring, in human subjects (three males), the suppressive effect of static masks on the ocular following responses induced by moving stimuli. We found a wide range of effects, which depend in a nonlinear and nonseparable manner on the spatial frequency, contrast, and spatial location of both stimulus and mask. Under some conditions, the presence of the mask can be seen as scaling the contrast of the driving stimulus. Under other conditions, the effect is more complex, involving also a direct scaling of the behavioral response. All of this complexity at the behavioral level can be captured by a simple model in which stimulus and mask interact nonlinearly at two stages, one monocular and one binocular. The nature of the interactions is compatible with those observed at the level of single neurons in primates, usually broadly described as divisive normalization, without having to invoke any scaling mechanism. [ABSTRACT FROM AUTHOR]

Human stereopsis can operate in dense "cyclopean" images containing no monocular objects. This is believed to depend on the computation of binocular correlation by neurons in primary visual cortex (VI). The observation that humans perceive depth in half-matched random-dot stereograms, although these stimuli have no net correlation, has led to the proposition that human depth perception in these stimuli depends on a distinct "matching" computation possibly performed in extrastriate cortex. However, recording from disparityselective neurons in V1 of fixating monkeys, we found that they are in fact able to signal disparity in half-matched stimuli. We present a simple model that explains these results. This reinstates the view that disparity-selective neurons in VI provide the initial substrate for perception in dense cyclopean stimuli, and strongly suggests that separate correlation and matching computations are not necessary to explain existing data on mixed correlation stereograms. [ABSTRACT FROM AUTHOR]

Ocular following eye movements have provided insights into how the visual system of humans and monkeys processes motion. Recently, it has been shown that they also reliably reveal stereoanomalies, and, thus, might have clinical applications. Their translation from research to clinical setting has however been hindered by their small size, which makes them difficult to record, and by a lack of data about their properties in sizable populations. Notably, they have so far only been recorded in adults. We recorded ocular following responses (OFRs)–defined as the change in eye position in the 80–160 ms time window following the motion onset of a large textured stimulus–in 14 school-age children (6 to 13 years old, 9 males and 5 females), under recording conditions that closely mimic a clinical setting. The OFRs were acquired non-invasively by a custom developed high-resolution video-oculography system, described in this study. With the developed system we were able to non-invasively detect OFRs in all children in short recording sessions. Across subjects, we observed a large variability in the magnitude of the movements (by a factor of 4); OFR magnitude was however not correlated with age. A power analysis indicates that even considerably smaller movements could be detected. We conclude that the ocular following system is well developed by age six, and OFRs can be recorded non-invasively in young children in a clinical setting. [ABSTRACT FROM AUTHOR]

It has been shown recently that neurons in V2 respond selectively to the edges of figures defined only by disparity (cyclopean edges). These responses are orientation selective, often preferring similar orientations for cyclopean and luminance contours, suggesting that they may support a cue-invariant representation of contours. Here, we investigate the extent to which processing of purely local visual information (in the vicinity of the receptive field) might explain such results, using the most impoverished stimulus possible containing a cyclopean edge (a circular patch of random dots divided into two regions by a single edge). Many V2 cells responded better to the cyclopean edge than to uniform disparities, and most of these were at least broadly selective for the orientation of the cyclopean edge. Two characteristics argue against a cue-invariant contour representation: (1) the cyclopean edge response was frequently abolished by small changes to the component disparities; and (2) although V2 cells frequently responded to both signs of a cyclopean edge (defined by which side of the edge is in front), they did so at different edge locations. These characteristics are consistent with a simple feed forward scheme in which V2 neurons receive inputs from several V1 subunits with different disparity selectivity. We also found a correlation between the preferred orientations for cyclopean edges and contrast stimuli, suggesting that this feedforward wiring is not random. These characteristics suggest that V2 responses to cyclopean edges may be useful in supporting a cue-invariant contour representation higher in the visual pathway. [ABSTRACT FROM AUTHOR]

One difficulty with measuring receptive fields in the awake monkey is that even well-trained animals make small eye movements during fixation. These complicate the measurement of receptive fields by blurring out the region where a response is observed, causing underestimates of the ability of individual neurons to signal changes in stimulus position. In simple cells, this blurring may severely disrupt estimates of receptive field structure. An accurate measurement of eye movements would allow correction of this blurring. Scleral search coils have been used to provide such measurements, although little is known about their accuracy. We have devised a range of approaches to address this issue: implanting two coils into a single eye, exploiting the small size of V1 receptive fields and developing maximum-likelihood fitting techniques to extract receptive field parameters in the presence of eye movements. All our investigations lead to the same conclusion: our scleral search coils (which were not sutured to the globe) are subject to an error of approximately the same magnitude as the small eye movements which occur during fixation: SD approximately 0.1 degrees. This error is large enough to explain the SD of measured vergence in the absence of any real changes in vergence state. This, and a variety of other arguments, indicate that the real variation in vergence is much smaller than coil measurements suggest. These results suggest that monkeys, like humans, maintain very stable vergence. The error has a slower time course than fixational eye movements so that search coils report the difference in eye position between two consecutive trials more accurately than the eye position itself on either trial. Receptive field estimates are unlikely to be improved by assuming the coil record is veridical and correcting for eye position accordingly. However, receptive field parameters can reliably be determined by a fitting technique that allows for eye movements. It is possible that suturing coils to the globe reduces the artifacts, but no method has been available to demonstrate this. These receptive field measurements provide a general means by which the reliability of eye-position measurements can be assessed. [ABSTRACT FROM AUTHOR]

Since the discovery of neurons selective for pattern motion direction in primate middle temporal area MT (Albright, 1984; Movshon et al., 1985), the neural computation of this signal has been the subject of intense study. The bulk of this work has explored responses to plaids obtained by summing two drifting sinusoidal gratings. Unfortunately, with these stimuli, many different mechanisms are similarly effective at extracting pattern motion. We devised a new set of stimuli, obtained by summing two random line stimuli with different orientations. This allowed several novel manipulations, including generating plaids that do not contain rigid 2D motion. Importantly, these stimuli do not engage most of the previously proposed mechanisms. We then recorded the ocular following responses that such stimuli induce in human subjects. We found that pattern motion is computed even with stimuli that do not cohere perceptually, including those without rigid motion, and even when the two gratings are presented separately to the two eyes. Moderate temporal and/or spatial separation of the gratings impairs the computation. We show that, of the models proposed so far, only those based on the intersection-of-constraints rule, embedding a motion-from-form mechanism (in which orientation signals are used in the computation of motion direction signals), can account for our results. At least for the eye movements reported here, a motion-from-form mechanism is thus involved in one of the most basic functions of the visual motion system: extracting motion direction from complex scenes. [ABSTRACT FROM AUTHOR]

In the context of motion detection, the endings (or terminators) of 1-D features can be detected as 2-D features, affecting the perceived direction of motion of the 1 -D features (the barber-pole illusion) and the direction of tracking eye movements. In the realm of binocular disparity processing, an equivalent role for the disparity of terminators has not been established. Here we explore the stereo analogy of the barber-pole stimulus, applying disparity to a 1-D noise stimulus seen through an elongated, zero-disparity, aperture. We found that, in human subjects, these stimuli induce robust short-latency reflexive vergence eye movements, initially in the direction orthogonal to the 1-D features, but shortly thereafter in the direction predicted by the disparity of the terminators. In addition, these same stimuli induce vivid depth percepts, which can only be attributed to the disparity of line terminators. When the 1-D noise patterns are given opposite contrast in the two eyes (anticorrelation), both components of the vergence response reverse sign. Finally, terminators drive vergence even when the aperture is defined by a texture (as opposed to a contrast) boundary. These findings prove that terminators contribute to stereo matching, and constrain the type of neuronal mechanisms that might be responsible for the detection of terminator disparity. [ABSTRACT FROM AUTHOR]

Neurons in early sensory cortex show weak but systematic correlations with perceptual decisions when trained animals perform at psychophysical threshold. These correlations are observed across repeated presentations of identical stimuli and cannot be explained by variation in external factors. The relationship between the activity of individual sensory neurons and the animal's behavioral choice means that even neurons in early sensory cortex carry information about an upcoming decision. This relationship, termed choice probability, may reflect the effect of fluctuations in neuronal firing rate on the animal's decision, but it can also reflect modulation of sensory responses by cognitive factors, or network properties such as variability that is shared among populations of neurons. Here, we review recent work clarifying the relationship among fluctuations in the responses of individual neurons, correlated variability, and behavior in a variety of tasks and cortical areas. We also discuss the possibility that choice probability may in part reflect the influence of cognitive factors on sensory neurons and explore the situations in which choice probability can be used to make inferences about the role of particular sensory neurons in the decision-making process. [ABSTRACT FROM AUTHOR]

Neurons encode the depth in stereoscopic images by combining the signals from the receptive fields in the two eyes. Local variations in single images can activate neurons that do not signal the correct disparity (false matches), giving rise to the stereo correspondence problem. We used binocular white-noise stimuli to decompose the responses of monkey primary visual cortex V1 neurons into the elements of a linear–nonlinear model (via spike-triggered covariance analysis). In our population of disparity-selective neurons, we find both excitatory and suppressive elements in many of the neurons. Their binocular receptive fields were aligned in a specific push–pull manner for disparity. We demonstrate that this arrangement reduces the responses to false matches but preserves the responses to true matches. The responses of the cells to the noise stimuli were well explained by a linear summation of the elements, followed by a nonlinearity. This model also explained the shape of independently measured disparity-tuning curves, although it overestimated the response magnitude. This study constitutes the first direct physiological evidence for the contribution of suppressive mechanisms to disparity selectivity. This new mechanism contributes to solving the stereo correspondence problem. [ABSTRACT FROM AUTHOR]

When light falls within a neuronal visual receptive field (RF) the resulting activity is referred to as the visual response. Recent work suggests this activity is in response to both the visual stimulation and the abrupt appearance, or salience, of the presentation. Here we present a novel method for distinguishing the two, based on the timing of random and nonrandom presentations. We examined these contributions in frontal eye field (FEF; N = 51) and as a comparison, an early stage in the primary visual cortex (V1; N = 15) of male monkeys (Macaca mulatto). An array of identical stimuli was presented within and outside the neuronal RF while we manipulated salience by varying the time between stimulus presentations. We hypothesized that the rapid presentation would reduce salience (the sudden appearance within the visual field) of a stimulus at any one location, and thus decrease responses driven by salience in the RF. We found that when the interstimulus interval decreased from 500 to 16 ms there was an approximate 79% reduction in the FEF response compared with an estimated 17% decrease in V1. This reduction in FEF response for rapid presentation was evident even when the random sequence preceding a stimulus did not stimulate the RF for 500 ms. The time course of these response changes in FEF suggest that salience is represented much earlier (< 100 ms following stimulus onset) than previously estimated. Our results suggest that the contribution of salience dominates at higher levels of the visual system. [ABSTRACT FROM AUTHOR]

Summary: Judgments about the perceptual appearance of visual objects require the combination of multiple parameters, like location, direction, color, speed, and depth. Our understanding of perceptual judgments has been greatly informed by studies of ambiguous figures, which take on different appearances depending upon the brain state of the observer. Here we probe the neural mechanisms hypothesized as responsible for judging the apparent direction of rotation of ambiguous structure from motion (SFM) stimuli. Resolving the rotation direction of SFM cylinders requires the conjoint decoding of direction of motion and binocular depth signals [1, 2]. Within cortical visual area V5/MT of two macaque monkeys, we applied electrical stimulation at sites with consistent multiunit tuning to combinations of binocular depth and direction of motion, while the monkey made perceptual decisions about the rotation of SFM stimuli. For both ambiguous and unambiguous SFM figures, rotation judgments shifted as if we had added a specific conjunction of disparity and motion signals to the stimulus elements. This is the first causal demonstration that the activity of neurons in V5/MT contributes directly to the perception of SFM stimuli and by implication to decoding the specific conjunction of disparity and motion, the two different visual cues whose combination drives the perceptual judgment. [ABSTRACT FROM AUTHOR]

Stereo matching, i.e., the matching by the visual system of corresponding parts of the images seen by the two eyes, is inherently a 2D problem. To gain insights into how this operation is carried out by the visual system, we measured, in human subjects, the reflexive vergence eye movements elicited by the sudden presentation of stereo plaids. We found compelling evidence that the 2D pattern disparity is computed by combining disparities first extracted within orientation selective channels. This neural computation takes 10-15 ms, and is carried out even when subjects perceive not a single plaid but rather two gratings in different depth planes (transparency). However, we found that 1D disparities are not always effectively combined: When spatial frequency and contrast of the gratings are sufficiently different pattern disparity is not computed, a result that cannot be simply attributed to the transparency of such stimuli. Based on our results, we propose that a narrow-band implementation of the IOC (Intersection of Constraints) rule (Fennema and Thompson, 1979; Adelson and Movshon, 1982), preceded by cross-orientation suppression, underlies the extraction of pattern disparity. [ABSTRACT FROM AUTHOR]

The human ability to detect modulation of binocular disparity over time is poor compared with detection of luminance modulation. We examined the physiological origin of this limitation by analyzing neuronal responses to temporal modulation of binocular disparity in striate cortex of awake monkeys. When neurons were presented with random-dot stereograms in which disparity varied sinusoidally over time, their responses modulated at the stimulus temporal frequency, with little change in mean firing rate. We calculated modulation amplitude as a function of temporal frequency and compared this with the psychophysical performance of four human observers. Neuronal and psychophysical functions showed similar peak frequencies (2 Hz) and comparable high-cut frequencies (10 and 5.5 Hz, respectively). Thus, V1 (primary visual cortex) neurons appear to limit psychophysical performance. The temporal resolution of the same neurons for contrast modulation was ∼2.5 times greater, which parallels the superior psychophysical performance for contrast. There is a simple mathematical explanation for this difference: it results from calculating cross-correlation between temporally broadband monocular images that are bandpass filtered before measuring correlation. The limit on temporal resolution is a direct consequence of the binocular energy model that adds to the list of properties of human stereoscopic performance that are explained by this simple model of disparity encoding in V1: the same neurons can account for the performance of psychophysical tasks that result in either high (contrast) or low (disparity) temporal resolution. Because this principle holds whenever a broadband input is bandpass filtered before computing correlation, it may limit the resolution of other neuronal systems. [ABSTRACT FROM AUTHOR]

Disparity selectivity in the striate cortex has generally been studied with uniform disparity fields covering the receptive field (RF). In four awake behaving monkeys, we quantitatively characterized the spatial three-dimensional structure of 55 V1 RFs using random dot stereograms in which disparity varied as a sinusoidal function of vertical position ("corrugations"). At low spatial frequencies, this produced a modulation in neuronal firing at the temporal frequency of the stimulus. As the spatial frequency increased, the modulation reduced. The mean response rate changed little and was close to that produced by a uniform stimulus at the mean disparity of the corrugation. In 48 of 55 (91%) neurons, the modulation strength was a lowpass function of spatial frequency. These results are compatible with a response determined only by the weighted mean of the disparities of the dots (the weights being set by the RF envelope) and suggest that there is no disparity-based surround inhibition or selectivity for disparity gradients. This simple weighting scheme predicts a relationship between RF size and the high-frequency cutoff. Comparison with independent measurements of RF size was compatible with this. All of this behavior closely matches the binocular energy model. The mean cutoff frequency, 0.5 cycles per degree, is similar to equivalent measures of decline in human psychophysical sensitivity for such depth corrugations as a function of frequency (Tyler, 1974; Prince and Rogers, 1998; Banks et al., 2004). This suggests that human cyclopean acuity for disparity modulations is limited by the selectivity of V1 neurons. This in turn is primarily limited by the RF size, because we find no sensitivity for disparity gradients or other disparity differences within the RFs. [ABSTRACT FROM AUTHOR]