Search

Whenever a visual scene is cast onto the retina, much of it will appear degraded due to poor resolution in the periphery; moreover, optical defocus can cause blur in central vision. However, the pervasiveness of blurry or degraded input is typically overlooked in the training of convolutional neural networks (CNNs). We hypothesized that the absence of blurry training inputs may cause CNNs to rely excessively on high spatial frequency information for object recognition, thereby causing systematic deviations from biological vision. We evaluated this hypothesis by comparing standard CNNs with CNNs trained on a combination of clear and blurry images. We show that blur-trained CNNs outperform standard CNNs at predicting neural responses to objects across a variety of viewing conditions. Moreover, blur-trained CNNs acquire increased sensitivity to shape information and greater robustness to multiple forms of visual noise, leading to improved correspondence with human perception. Our results provide multi-faceted neurocomputational evidence that blurry visual experiences may be critical for conferring robustness to biological visual systems., (© 2024. The Author(s).)

It is widely believed that visual expectations can change the subjective experiences of humans. We investigated how visual expectations in a recognition task affected objective performance and subjective perception. Using a 2-alternative-forced-choice task based on digit recognition of briefly presented and visually masked digits, we found over two experiments that expectations changed the quality of the experiences without changing the performance capabilities associated with the quality of experience. Expectations were manipulated by providing a cue indicating the set of possible digits that might appear on each trial. The results also inform the debate about whether subjective experiences can be categorized in a dichotomous manner or in a graded manner. We found that subjective experiences were graded near the objective threshold and more dichotomous away from the threshold. Furthermore, distinct expectations resulted in a more dichotomous distribution of subjective experience. We also provide evidence of an interesting relationship between stimulus duration, objective performance and subjective ratings. Only experiences that were rated as evoking some degree of perception showed systematic improvements in objective performance as a function of stimulus duration. These findings suggest that subjective experience cannot be understood without considering the broader cognitive context, namely that the quality of subjective experiences is dependent on a multitude of factors such as attention, task requirements and cognitive expectations., (Copyright © 2019 Elsevier Inc. All rights reserved.)

The visual system employs a sophisticated balance of attentional mechanisms: salient stimuli are prioritized for visual processing, yet observers can also ignore such stimuli when their goals require directing attention elsewhere. A powerful determinant of visual salience is local feature contrast: if a local region differs from its immediate surround along one or more feature dimensions, it will appear more salient. We used high-resolution functional MRI (fMRI) at 7T to characterize the modulatory effects of bottom-up salience and top-down voluntary attention within multiple sites along the early visual pathway, including visual areas V1-V4 and the lateral geniculate nucleus (LGN). Observers viewed arrays of spatially distributed gratings, where one of the gratings immediately to the left or right of fixation differed from all other items in orientation or motion direction, making it salient. To investigate the effects of directed attention, observers were cued to attend to the grating to the left or right of fixation, which was either salient or nonsalient. Results revealed reliable additive effects of top-down attention and stimulus-driven salience throughout visual areas V1-hV4. In comparison, the LGN exhibited significant attentional enhancement but was not reliably modulated by orientation- or motion-defined salience. Our findings indicate that top-down effects of spatial attention can influence visual processing at the earliest possible site along the visual pathway, including the LGN, whereas the processing of orientation- and motion-driven salience primarily involves feature-selective interactions that take place in early cortical visual areas. NEW & NOTEWORTHY While spatial attention allows for specific, goal-driven enhancement of stimuli, salient items outside of the current focus of attention must also be prioritized. We used 7T fMRI to compare salience and spatial attentional enhancement along the early visual hierarchy. We report additive effects of attention and bottom-up salience in early visual areas, suggesting that salience enhancement is not contingent on the observer's attentional state., (Copyright © 2017 the American Physiological Society.)

Humans reliably recognize faces across a range of viewpoints, but the neural substrates supporting this ability remain unclear. Recent work suggests that neural selectivity to mirror-symmetric viewpoints of faces, found across a large network of visual areas, may constitute a key computational step in achieving full viewpoint invariance. In this study, we used repetitive transcranial magnetic stimulation (rTMS) to test the hypothesis that the occipital face area (OFA), putatively a key node in the face network, plays a causal role in face viewpoint symmetry perception. Each participant underwent both offline rTMS to the right OFA and sham stimulation, preceding blocks of behavioral trials. After each stimulation period, the participant performed one of two behavioral tasks involving presentation of faces in the peripheral visual field: (1) judging the viewpoint symmetry; or (2) judging the angular rotation. rTMS applied to the right OFA significantly impaired performance in both tasks when stimuli were presented in the contralateral, left visual field. Interestingly, however, rTMS had a differential effect on the two tasks performed ipsilaterally. Although viewpoint symmetry judgments were significantly disrupted, we observed no effect on the angle judgment task. This interaction, caused by ipsilateral rTMS, provides support for models emphasizing the role of interhemispheric crosstalk in the formation of viewpoint-invariant face perception., Significance Statement: Faces are among the most salient objects we encounter during our everyday activities. Moreover, we are remarkably adept at identifying people at a glance, despite the diversity of viewpoints during our social encounters. Here, we investigate the cortical mechanisms underlying this ability by focusing on effects of viewpoint symmetry, i.e., the invariance of neural responses to mirror-symmetric facial viewpoints. We did this by temporarily disrupting neural processing in the occipital face area (OFA) using transcranial magnetic stimulation. Our results demonstrate that the OFA causally contributes to judgments facial viewpoints and suggest that effects of viewpoint symmetry, previously observed using fMRI, arise from an interhemispheric integration of visual information even when only one hemisphere receives direct visual stimulation., (Copyright © 2015 the authors 0270-6474/15/3516398-06$15.00/0.)

What neural mechanisms underlie the ability to attend to a complex object in the presence of competing overlapping stimuli? We evaluated whether object-based attention might involve pattern-specific feedback to early visual areas to selectively enhance the set of low-level features corresponding to the attended object. Using fMRI and multivariate pattern analysis, we found that activity patterns in early visual areas (V1-V4) are strongly biased in favor of the attended object. Activity patterns evoked by single faces and single houses reliably predicted which of the 2 overlapping stimulus types was being attended with high accuracy (80-90% correct). Superior knowledge of upright objects led to improved attentional selection in early areas. Across individual blocks, the strength of the attentional bias signal in early visual areas was highly predictive of the modulations found in high-level object areas, implying that pattern-specific attentional filtering at early sites can determine the quality of object-specific signals that reach higher level visual areas. Through computational modeling, we show how feedback of an average template to V1-like units can improve discrimination of exemplars belonging to the attended category. Our findings provide a mechanistic account of how feedback to early visual areas can contribute to the attentional selection of complex objects., (© The Author 2013. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.)

Orientation selectivity is a cornerstone property of vision, commonly believed to emerge in the primary visual cortex. We found that reliable orientation information could be detected even earlier, in the human lateral geniculate nucleus, and that attentional feedback selectively altered these orientation responses. This attentional modulation may allow the visual system to modify incoming feature-specific signals at the earliest possible processing site.

Scenes and objects are effortlessly processed and integrated by the human visual system. Given the distinct neural and behavioral substrates of scene and object processing, it is likely that individuals sometimes preferentially rely on one process or the other when viewing canonical "scene" or "object" stimuli. This would allow the visual system to maximize the specific benefits of these 2 types of processing. It is less obvious which of these modes of perception would be invoked during naturalistic visual transition between a focused view of a single object and an expansive view of an entire scene, particularly at intermediate views that may not be assigned readily to either stimulus category. In the current study, we asked observers to report their online perception of such dynamic image sequences, which zoomed and panned between a canonical view of a single object and an entire scene. We found a large and consistent effect of prior perception, or hysteresis, on the classification of the sequence: observers classified the sequence as an object for several seconds longer if the trial started at the object view and zoomed out, whereas scenes were perceived for longer on trials beginning with a scene view. This hysteresis effect resisted several manipulations of the movie stimulus and of the task performed, but hinged on the perceptual history built by unidirectional progression through the image sequence. Multiple experiments confirmed that this hysteresis effect was not purely decisional and was more prominent for transitions between corresponding objects and scenes than between other high-level stimulus classes. This finding suggests that the competitive mechanisms underlying hysteresis may be especially prominent in the perception of objects and scenes. We propose that hysteresis aids in disambiguating perception during naturalistic visual transitions, which may facilitate a dynamic balance between scene and object processing to enhance processing efficiency.

The visual system is remarkably proficient at extracting relevant object information from noisy, cluttered environments. Although attention is known to enhance sensory processing, the mechanisms by which attention extracts relevant information from noise are not well understood. According to the perceptual template model, attention may act to amplify responses to all visual input, or it may act as a noise filter, dampening responses to irrelevant visual noise. Amplification allows for improved performance in the absence of visual noise, whereas a noise-filtering mechanism can only improve performance if the target stimulus appears in noise. Here, we used fMRI to investigate how attention modulates cortical responses to objects at multiple levels of the visual pathway. Participants viewed images of faces, houses, chairs, and shoes, presented in various levels of visual noise. We used multivoxel pattern analysis to predict the viewed object category, for attended and unattended stimuli, from cortical activity patterns in individual visual areas. Early visual areas, V1 and V2, exhibited a benefit of attention only at high levels of visual noise, suggesting that attention operates via a noise-filtering mechanism at these early sites. By contrast, attention led to enhanced processing of noise-free images (i.e., amplification) only in higher visual areas, including area V4, fusiform face area, mid-Fusiform area, and the lateral occipital cortex. Together, these results suggest that attention improves people's ability to discriminate objects by de-noising visual input in early visual areas and amplifying this noise-reduced signal at higher stages of visual processing.

Working memory serves as an essential workspace for the mind, allowing for the active maintenance of information to support short-term cognitive goals. Although people can readily report the contents of working memory, it is unknown whether they might have reliable metacognitive knowledge regarding the accuracy of their own memories. We investigated this question to better understand the core properties of the visual working memory system. Observers were briefly presented with displays of three or six oriented gratings, after which they were cued to report the orientation of a specific grating from memory as well as their subjective confidence in their memory. We used a mixed-model approach to obtain separate estimates of the probability of successful memory maintenance and the precision of memory for successfully remembered items. Confidence ratings strongly predicted the likelihood that the cued grating was successfully maintained, and furthermore revealed trial-to-trial variations in the visual precision of memory itself. Our findings provide novel evidence indicating that the precision of visual working memory is variable in nature. These results inform an ongoing debate regarding whether this working memory system relies on discrete slots with fixed visual resolution or on representations with variable precision, as might arise from variability in the amount of resources assigned to individual items on each trial.

Although practice has long been known to improve perceptual performance, the neural basis of this improvement in humans remains unclear. Using fMRI in conjunction with a novel signal detection-based analysis, we show that extensive practice selectively enhances the neural representation of trained orientations in the human visual cortex. Twelve observers practiced discriminating small changes in the orientation of a laterally presented grating over 20 or more daily 1 h training sessions. Training on average led to a twofold improvement in discrimination sensitivity, specific to the trained orientation and the trained location, with minimal improvement found for untrained orthogonal orientations or for orientations presented in the untrained hemifield. We measured the strength of orientation-selective responses in individual voxels in early visual areas (V1-V4) using signal detection measures, both before and after training. Although the overall amplitude of the BOLD response was no greater after training, practice nonetheless specifically enhanced the neural representation of the trained orientation at the trained location. This training-specific enhancement of orientation-selective responses was observed in the primary visual cortex (V1) as well as higher extrastriate visual areas V2-V4, and moreover, reliably predicted individual differences in the behavioral effects of perceptual learning. These results demonstrate that extensive training can lead to targeted functional reorganization of the human visual cortex, refining the cortical representation of behaviorally relevant information.

Can people evaluate phenomenal qualities of internally generated experiences, such as whether a mental image is vivid or detailed? This question exemplifies a problem of metacognition: How well do people know their own thoughts? In the study reported here, participants were instructed to imagine a specific visual pattern and rate its vividness, after which they were presented with an ambiguous rivalry display that consisted of the previously imagined pattern plus an orthogonal pattern. On individual trials, higher ratings of vividness predicted a greater likelihood that the imagined pattern would appear dominant when the participant was subsequently presented with the binocular rivalry display. Off-line self-report questionnaires measuring imagery vividness also predicted individual differences in the strength of imagery bias over the entire study. Perceptual bias due to mental imagery could not be attributed to demand characteristics, as no bias was observed on catch-trial presentations of mock rivalry displays. Our findings provide novel evidence that people have a good metacognitive understanding of their own mental imagery and can reliably evaluate the vividness of single episodes of imagination.

In this issue of Neuron, Haxby and colleagues describe a new method for aligning functional brain activity patterns across participants. Their study demonstrates that objects are similarly represented across different brains, allowing for reliable classification of one person's brain activity based on another's., (Copyright © 2011 Elsevier Inc. All rights reserved.)

Recent studies indicate that expertise with objects can interfere with face processing. Although competition occurs between faces and objects of expertise, it remains unclear whether this reflects an expertise-specific bottleneck or the fact that objects of expertise grab attention and thereby consume more central resources. We investigated the perceptual costs of expertise by measuring visual thresholds for identifying targets embedded within RSVP sequences presented at varying temporal rates. Car experts and novices searched for face targets among face and car distractors, or watch targets among watch and car distractors. Remarkably, car experts were slower than novices at identifying faces among task-irrelevant cars, yet faster than novices at identifying watches among cars. This suggests that car expertise leads to greater functional overlap between cars and faces while reducing the functional overlap between cars and objects, a result incompatible with the notion of an encapsulated module for exclusive processing of faces., (Copyright © 2010 Elsevier B.V. All rights reserved.)

Visual working memory provides an essential link between perception and higher cognitive functions, allowing for the active maintenance of information about stimuli no longer in view. Research suggests that sustained activity in higher-order prefrontal, parietal, inferotemporal and lateral occipital areas supports visual maintenance, and may account for the limited capacity of working memory to hold up to 3-4 items. Because higher-order areas lack the visual selectivity of early sensory areas, it has remained unclear how observers can remember specific visual features, such as the precise orientation of a grating, with minimal decay in performance over delays of many seconds. One proposal is that sensory areas serve to maintain fine-tuned feature information, but early visual areas show little to no sustained activity over prolonged delays. Here we show that orientations held in working memory can be decoded from activity patterns in the human visual cortex, even when overall levels of activity are low. Using functional magnetic resonance imaging and pattern classification methods, we found that activity patterns in visual areas V1-V4 could predict which of two oriented gratings was held in memory with mean accuracy levels upwards of 80%, even in participants whose activity fell to baseline levels after a prolonged delay. These orientation-selective activity patterns were sustained throughout the delay period, evident in individual visual areas, and similar to the responses evoked by unattended, task-irrelevant gratings. Our results demonstrate that early visual areas can retain specific information about visual features held in working memory, over periods of many seconds when no physical stimulus is present.

We employed a parametric psychophysical design in combination with functional imaging to examine the influence of metric changes in perceptual incongruence on perceptual alternation rates and cortical responses. Subjects viewed a bistable stimulus defined by incongruent depth cues; bistability resulted from incongruence between binocular disparity and monocular perspective cues that specify different slants (slant rivalry). Psychophysical results revealed that perceptual alternation rates were positively correlated with the degree of perceived incongruence. Functional imaging revealed systematic increases in activity that paralleled the psychophysical results within anterior intraparietal sulcus, prior to the onset of perceptual alternations. We suggest that this cortical activity predicts the frequency of subsequent alternations, implying a putative causal role for these areas in initiating bistable perception. In contrast, areas implicated in form and depth processing (LOC and V3A) were sensitive to the degree of slant, but failed to show increases in activity when these cues were in conflict.

Mental imagery has been proposed to contribute to a variety of high-level cognitive functions, including memory encoding and retrieval, navigation, spatial planning, and even social communication and language comprehension. However, it is debated whether mental imagery relies on the same sensory representations as perception, and if so, what functional consequences such an overlap might have on perception itself. We report novel evidence that single instances of imagery can have a pronounced facilitatory influence on subsequent conscious perception. Either seeing or imagining a specific pattern could strongly bias which of two competing stimuli reach awareness during binocular rivalry. Effects of imagery and perception were location and orientation specific, accumulated in strength over time, and survived an intervening visual task lasting several seconds prior to presentation of the rivalry display. Interestingly, effects of imagery differed from those of feature-based attention. The results demonstrate that imagery, in the absence of any incoming visual signals, leads to the formation of a short-term sensory trace that can bias future perception, suggesting a means by which high-level processes that support imagination and memory retrieval may shape low-level sensory representations.

How do selective and constructive visual mechanisms interact to determine the outcome of conscious perception? Binocular rivalry involves selective perception of one of two competing monocular images, whereas visual phantoms involve perceptual filling-in between two low-contrast collinear gratings. Recently, we showed that visual phantoms lead to neural filling-in of activity in V1 and V2, which can be dynamically gated by rivalry suppression (M. Meng, D. A. Remus, & F. Tong, 2005). Here, we used psychophysical methods to study the temporal dynamics of filling-in, by applying rivalry or flash suppression to trigger the suppression or appearance of visual phantoms. Experiments revealed that phantom filling-in involves an active, time-dependent process that depends on the phenomenal visibility of the phantom-inducing gratings. Shortly after the inducing gratings became dominant during rivalry, the likelihood of perceiving phantoms in the intervening gap increased over time, with larger gaps requiring more time for filling-in. In contrast, suppression of the inducing gratings promptly led to the disappearance of visual phantoms, with response times independent of gap size. The fact that binocular rivalry can prevent the formation of visual phantoms rules out the possibility that rivalry suppression occurs after the site of phantom filling-in. This study provides novel evidence that visual phantoms result from a slow time-dependent filling-in mechanism; possible models to account for its time course are discussed.

The neural mechanism that mediates perceptual filling-in of the blind spot is still under discussion. One hypothesis proposes that the cortical representation of the blind spot is activated only under conditions that elicit perceptual filling-in and requires congruent stimulation on both sides of the blind spot. Alternatively, the passive remapping hypothesis proposes that inputs from regions surrounding the blind spot infiltrate the representation of the blind spot in cortex. This theory predicts that independent stimuli presented to the left and right of the blind spot should lead to neighboring/overlapping activations in visual cortex when the blind-spot eye is stimulated but separated activations when the fellow eye is stimulated. Using functional MRI, we directly tested the remapping hypothesis by presenting flickering checkerboard wedges to the left or right of the spatial location of the blind spot, either to the blind-spot eye or to the fellow eye. Irrespective of which eye was stimulated, we found separate activations corresponding to the left and right wedges. We identified the centroid of the activations on a cortical flat map and measured the distance between activations. Distance measures of the cortical gap across the blind spot were accurate and reliable (mean distance: 6-8 mm across subjects, SD approximately 1 mm within subjects). Contrary to the predictions of the remapping hypothesis, cortical distances between activations to the two wedges were equally large for the blind-spot eye and fellow eye in areas V1 and V2/V3. Remapping therefore appears unlikely to account for perceptual filling-in at an early cortical level.

The constructive nature of perception can be demonstrated under viewing conditions that lead to vivid subjective impressions in the absence of direct input. When a low-contrast moving grating is divided by a large gap, observers report seeing a 'visual phantom' of the real grating extending through the blank gap region. Here, we report fMRI evidence showing that visual phantoms lead to enhanced activity in early visual areas that specifically represent the blank gap region. We found that neural filling-in effects occurred automatically in areas V1 and V2, regardless of where the subject attended. Moreover, when phantom-inducing gratings were paired with competing stimuli in a binocular rivalry display, subjects reported spontaneous fluctuations in conscious perception of the phantom accompanied by tightly coupled changes in early visual activity. Our results indicate that phantom visual experiences are closely linked to automatic filling-in of activity at the earliest stages of cortical processing.

The potential for human neuroimaging to read out the detailed contents of a person's mental state has yet to be fully explored. We investigated whether the perception of edge orientation, a fundamental visual feature, can be decoded from human brain activity measured with functional magnetic resonance imaging (fMRI). Using statistical algorithms to classify brain states, we found that ensemble fMRI signals in early visual areas could reliably predict on individual trials which of eight stimulus orientations the subject was seeing. Moreover, when subjects had to attend to one of two overlapping orthogonal gratings, feature-based attention strongly biased ensemble activity toward the attended orientation. These results demonstrate that fMRI activity patterns in early visual areas, including primary visual cortex (V1), contain detailed orientation information that can reliably predict subjective perception. Our approach provides a framework for the readout of fine-tuned representations in the human brain and their subjective contents.

It is debated whether different forms of bistable perception result from common or separate neural mechanisms. Binocular rivalry involves perceptual alternations between competing monocular images, whereas ambiguous figures such as the Necker cube lead to alternations between two possible pictorial interpretations. Previous studies have shown that observers can voluntarily control the alternation rate of both rivalry and Necker cube reversal, perhaps suggesting that bistable perception results from a common mechanism of top-down selection. However, according to the biased competition model of selective attention, attention should be able to enhance the attended percept and suppress the unattended percept. Here, we investigated selective attentional modulation of dominance durations in bistable perception. Observers consistently showed much weaker selective attentional control for rivalry than for Necker cube reversal, even for rivalry displays that maximized the opportunities for feature-, object-, or space-based attentional selection. In contrast, nonselective control of alternation rate was comparably strong for both forms of bistable perception and corresponded poorly with estimates of selective attentional control. Our results support the notion that binocular rivalry involves a more automatic, stimulus-driven form of visual competition than Necker cube reversal, and as a consequence, is less easily biased by selective attention.

The primary visual cortex (V1) is probably the best characterized area of primate cortex, but whether this region contributes directly to conscious visual experience is controversial. Early neurophysiological and neuroimaging studies found that visual awareness was best correlated with neural activity in extrastriate visual areas, but recent studies have found similarly powerful effects in V1. Lesion and inactivation studies have provided further evidence that V1 might be necessary for conscious perception. Whereas hierarchical models propose that damage to V1 simply disrupts the flow of information to extrastriate areas that are crucial for awareness, interactive models propose that recurrent connections between V1 and higher areas form functional circuits that support awareness. Further investigation into V1 and its interactions with higher areas might uncover fundamental aspects of the neural basis of visual awareness.

We report evidence from visual search that people can develop robust representations for highly overlearned faces. When observers searched for their own face versus the face of an unfamiliar observer, search slopes and intercepts revealed consistently faster processing of self than stranger. These processing advantages persisted even after hundreds of presentations of the unfamiliar face and even for atypical profile and upside-down views. Observers not only showed rapid asymptotic recognition of their own face as the target, but could reject their own face more quickly as the distractor. These findings suggest that robust representations for a highly overlearned face may (a) mediate rapid asymptotic visual processing, (b) require extensive experience to develop, (c) contain abstract or view-invariant information, (d) facilitate a variety of processes such as target recognition and distractor rejection, and (e) demand less attentional resources.

We used functional magnetic resonance imaging (fMRI) to monitor stimulus-selective responses of the human fusiform face area (FFA) and parahippocampal place area (PPA) during binocular rivalry in which a face and a house stimulus were presented to different eyes. Though retinal stimulation remained constant, subjects perceived changes from house to face that were accompanied by increasing FFA and decreasing PPA activity; perceived changes from face to house led to the opposite pattern of responses. These responses during rivalry were equal in magnitude to those evoked by nonrivalrous stimulus alternation, suggesting that activity in the FFA and PPA reflects the perceived rather than the retinal stimulus, and that neural competition during binocular rivalry has been resolved by these stages of visual processing.