Search

Binocular disparity provides critical information about three-dimensional (3D) structures to support perception and action. In the past decade significant progress has been made in uncovering human brain areas engaged in the processing of binocular disparity signals. Yet, the fine-scale brain processing underlying 3D perception remains unknown. Here, we use ultra-high-field (7T) functional imaging at submillimeter resolution to examine fine-scale BOLD fMRI signals involved in 3D perception. In particular, we sought to interrogate the local circuitry involved in disparity processing by sampling fMRI responses at different positions relative to the cortical surface (i.e., across cortical depths corresponding to layers). We tested for representations related to 3D perception by presenting participants (male and female, N = 8) with stimuli that enable stable stereoscopic perception [i.e., correlated random dot stereograms (RDS)] versus those that do not (i.e., anticorrelated RDS). Using multivoxel pattern analysis (MVPA), we demonstrate cortical depth-specific representations in areas V3A and V7 as indicated by stronger pattern responses for correlated than for anticorrelated stimuli in upper rather than deeper layers. Examining informational connectivity, we find higher feedforward layer-to-layer connectivity for correlated than anticorrelated stimuli between V3A and V7. Further, we observe disparity-specific feedback from V3A to V1 and from V7 to V3A. Our findings provide evidence for the role of V3A as a key nexus for disparity processing, which is implicated in feedforward and feedback signals related to the perceptual estimation of 3D structures. SIGNIFICANCE STATEMENT Binocular vision plays a significant role in supporting our interactions with the surrounding environment. The fine-scale neural mechanisms that underlie the brain9s skill in extracting 3D structures from binocular signals are poorly understood. Here, we capitalize on recent advances in ultra-high-field functional imaging to interrogate human brain circuits involved in 3D perception at submillimeter resolution. We provide evidence for the role of area V3A as a key nexus for disparity processing, which is implicated in feedforward and feedback signals related to the perceptual estimation of 3D structures from binocular signals. These fine-scale measurements help bridge the gap between animal neurophysiology and human fMRI studies investigating cross-scale circuits, from micro circuits to global brain networks for 3D perception.

Background. The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted both safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the fore as a promising solution for rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. Here, we examine the opportunities and challenges afforded by the societal shift toward web-based testing, highlight an urgent need to establish a standard data quality assurance framework for online studies, and develop and validate a new supervised online testing methodology, remote guided testing (RGT). Methods. A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (N=41) or online using remote guided testing (N=44), delivered using identical web-based platforms (CANTAB, Inquisit and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, response times), as well as overall task performance measures. Results. The results indicated that, across all measures of data quality and performance, RGT data was statistically-equivalent to data collected in person in the lab. Moreover, RGT participants out-performed the lab group on measured verbal intelligence, which could reflect test environment differences, including possible effects of mask-wearing on communication. Conclusions. These data suggest that the RGT methodology could help to ameliorate concerns regarding online data quality and - particularly for studies involving high-risk or rare cohorts - offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.

Although language comprehension usually requires multimodal information, no study to date has investigated how comprehenders deal with the revision of a text's interpretation when different modalities are involved. Twenty-four young adults listened to a story prompting an inference (e.g., polar bear), and then saw a picture that was either consistent (polar bear) or inconsistent but still plausible (penguin). Larger negativity (N400) in the inconsistent picture indicated successful inferential monitoring. Subsequently, a sentence carried the disambiguating word which was either expected (“bear”) or unexpected (“penguin”) in relation to the auditory-verbal information. Larger negativity in the unexpected word coming from the consistent picture suggested that comprehenders had difficulties selecting the unexpected concept when previous information was contradictory. More importantly, this effect was modulated by inhibitory control, where a higher resistance to distractor interference (flanker task) was associated with a better ability to suppress pictorial information, therefore preventing semantic competition. Similarly, accuracy measured in a final comprehension question demonstrated that higher inhibitory control was related to a more efficient ability to revise the situation model across modalities. Our findings speak to a relationship between story comprehension and mental flexibility during multimodal processing.

Learning is known to facilitate performance in a range of perceptual tasks. Behavioral improvement after training is typically shown after practice with highly similar stimuli that are difficult to discriminate (i.e., hard training), or after exposure to dissimilar stimuli that are highly discriminable (i.e., easy training). However, little is known about the processes that mediate learning after training with difficult compared to easy stimuli. Here we investigate the time course of learning when observers were asked to discriminate similar global form patterns after hard vs. easy training. Hard training required observers to discriminate highly similar global forms, while easy training to judge clearly discriminable patterns. Our results demonstrate differences in learning and transfer performance for hard compared to easy training. Hard training resulted in stronger behavioral improvement than easy training. Further, for hard training, performance improved during single sessions, while for easy training performance improved across but not within sessions. These findings suggest that training with difficult stimuli may result in online learning of specific stimulus features that are similar between the training and test stimuli, while training with easy stimuli involves transfer of learning from highly to less discriminable stimuli that may require longer periods of consolidation.

Training is known to improve performance in a variety of perceptual and cognitive skills. However, there is accumulating evidence that mere exposure (i.e. without supervised training) to regularities (i.e. patterns that co-occur in the environment) facilitates our ability to learn contingencies that allow us to interpret the current scene and make predictions about future events. Recent neuroimaging studies have implicated fronto-striatal and medial temporal lobe brain regions in the learning of spatial and temporal statistics. Here, we ask whether patients with mild cognitive impairment due to Alzheimer′s disease (MCI-AD) that are characterized by hippocampal dysfunction are able to learn temporal regularities and predict upcoming events. We tested the ability of MCI-AD patients and age-matched controls to predict the orientation of a test stimulus following exposure to sequences of leftwards or rightwards orientated gratings. Our results demonstrate that exposure to temporal sequences without feedback facilitates the ability to predict an upcoming stimulus in both MCI-AD patients and controls. However, our fMRI results demonstrate that MCI-AD patients recruit an alternate circuit to hippocampus to succeed in learning of predictive structures. In particular, we observed stronger learning-dependent activations for structured sequences in frontal, subcortical and cerebellar regions for patients compared to age-matched controls. Thus, our findings suggest a cortico-striatal–cerebellar network that may mediate the ability for predictive learning despite hippocampal dysfunction in MCI-AD.

Learning the statistics of the environment is critical for predicting upcoming events. However, little is known about how we translate previous knowledge about scene regularities to sensory predictions. Here, we ask whether patients with mild cognitive impairment due to Alzheimer's disease (MCI-AD) that are known to have spared implicit but impaired explicit recognition memory are able to learn temporal regularities and predict upcoming events. We tested the ability of MCI-AD patients and age-matched controls to predict the orientation of a test stimulus following exposure to sequences of leftwards or rightwards oriented gratings. Our results demonstrate that exposure to temporal sequences without feedback facilitates the ability to predict an upcoming stimulus in both MCI-AD patients and controls. Further, we show that executive cognitive control may account for individual variability in predictive learning. That is, we observed significant positive correlations of performance in attentional and working memory tasks with post-training performance in the prediction task. Taken together, these results suggest a mediating role of circuits involved in cognitive control (i.e. frontal circuits) that may support the ability for predictive learning in MCI-AD.

The brain integrates discrete but collinear stimuli to perceive global contours. Previous contour integration (CI) studies mainly focus on integration over space, and CI is attributed to either V1 long-range connections or contour processing in high-visual areas that top-down modulate V1 responses. Here, we show that CI also occurs over time in a design that minimizes the roles of V1 long-range interactions. We use tilted contours embedded in random orientation noise and moving horizontally behind a fixed vertical slit. Individual contour elements traveling up/down within the slit would be encoded over time by parallel, rather than aligned, V1 neurons. However, we find robust contour detection even when the slit permits only one viewable contour element. Similar to CI over space, CI over time also obeys the rule of collinearity. fMRI evidence shows that while CI over space engages visual areas as early as V1, CI over time mainly engages higher dorsal and ventral visual areas involved in shape processing, as well as posterior parietal regions involved in visual memory that can represent the orientation of temporally integrated contours. These results suggest at least partially dissociable mechanisms for implementing the Gestalt rule of continuity in CI over space and time.

Visual judgments critically depend on (1) the detection of meaningful items from cluttered backgrounds and (2) the discrimination of an item from highly similar alternatives. Learning and experience are known to facilitate these processes, but the specificity with which these processes operate is poorly understood. Here we use psychophysical measures of human participants to test learning in two types of commonly used tasks that target segmentation (signal-in-noise, or “coarse” tasks) versus the discrimination of highly similar items (feature difference, or “fine” tasks). First, we consider the processing of binocular disparity signals, examining performance on signal-in-noise and feature difference tasks after a period of training on one of these tasks. Second, we consider the generality of learning between different visual features, testing performance on both task types for displays defined by disparity, motion, or orientation. We show that training on a feature difference task also improves performance on signal-in-noise tasks, but only for the same visual feature. By contrast, training on a signal-in-noise task has limited benefits for fine judgments of the same feature but supports learning that generalizes to signal-in-noise tasks for other features. These findings indicate that commonly used signal-in-noise tasks require at least three distinct components: feature representations, signal-specific selection, and a generalized process that enhances segmentation. As such, there is clear potential to harness areas of commonality (both within and between cues) to improve impaired perceptual functions.

Despite the central role of learning in visual recognition, it is largely unknown whether visual form learning is maintained in older age. We examined whether training improved performance in both young and older adults at two key stages of visual recognition: integration of local elements and global form discrimination. We used a shape-discrimination task (concentric vs. radial patterns) in which young and older adults showed similar performance before training. Using a parametric stimulus space that allowed us to manipulate global features and background noise, we were able to distinguish integration and discrimination processes. We found that training improves global form discrimination in both young and older adults. However, learning to integrate local elements is impaired in older age, possibly because of reduced tolerance to external noise. These findings suggest that visual selection processes, rather than global feature representations, provide a fundamental limit for learning-dependent plasticity in the aging brain.

Early diagnosis of dementia is critical for assessing disease progression and potential treatment. State-or-the-art machine learning techniques have been increasingly employed to take on this diagnostic task. In this study, we employed Generalised Matrix Learning Vector Quantization (GMLVQ) classifiers to discriminate patients with Mild Cognitive Impairment (MCI) from healthy controls based on their cognitive skills. Further, we adopted a ``Learning with privileged information'' approach to combine cognitive and fMRI data for the classification task. The resulting classifier operates solely on the cognitive data while it incorporates the fMRI data as privileged information (PI) during training. This novel classifier is of practical use as the collection of brain imaging data is not always possible with patients and older participants. MCI patients and healthy age-matched controls were trained to extract structure from temporal sequences. We ask whether machine learning classifiers can be used to discriminate patients from controls based on the learning performance and whether differences between these groups relate to individual cognitive profiles. To this end, we tested participants in four cognitive tasks: working memory, cognitive inhibition, divided attention, and selective attention. We also collected fMRI data before and after training on the learning task and extracted fMRI responses and connectivity as features for machine learning classifiers. Our results show that the PI guided GMLVQ classifiers outperform the baseline classifier that only used the cognitive data. In addition, we found that for the baseline classifier, “divided attention” is the only relevant cognitive feature. When PI was incorporated, divided attention remained the most relevant feature while cognitive inhibition became also relevant for the task. Interestingly, this analysis for the fMRI GMLVQ classifier suggests that (1) when overall fMRI signal for structured stimuli is used as inputs to the classifier, the post-training session is most relevant; and (2) when the graph feature reflecting underlying spatiotemporal fMRI pattern is used, the pre-training session is most relevant. Taken together these results suggest that brain connectivity before training and overall fMRI signal after training are both diagnostic of cognitive skills in MCI.

Background Limited information processing capacity in the brain necessitates task prioritisation and subsequent adaptive behavioural strategies for the dual-task coordination of locomotion with severe concurrent cognitive loading. Commonly observed strategies include prioritisation of gait at the cost of reduced performance in the cognitive task. Alternatively alterations of gait parameters such as gait velocity have been reported presumably to free processing capacity for the benefit of performance in the cognitive task. The aim of this study was to describe the neuroanatomical correlates of adaptive behavioural strategies in cognitive-motor dual-tasking when the competition for information processing capacity is severe and may exceed individuals’ capacity limitations. Methods During an fMRI experiment, 12 young adults performed slow continuous, auditorily paced bilateral anti-phase ankle dorsi-plantarflexion movements as an element of normal gait at .5 Hz in single and dual task modes. The secondary task involved a visual, alphabetic N-back task with presentation rate jittered around .7 Hz. The N-back task, which randomly occurred in 0-back or 2-back form, was modified into a silent counting task to avoid confounding motor responses at the cost of slightly increasing the task′s general coordinative complexity. Participants’ ankle movements were recorded using an optoelectronic motion capture system to derive kinematic parameters representing the stability of the movement timing and synchronization. Participants were instructed to perform both tasks as accurately as possible. Results Increased processing complexity in the dual-task 2-back condition led to significant changes in movement parameters such as the average inter-response interval, the coefficient of variation of absolute asynchrony and the standard deviation of peak angular velocity. A regions-of-interest analysis indicated correlations between these parameters and local activations within the left inferior frontal gyrus (IFG) such that lower IFG activations coincided with performance decrements. Conclusions Dual-task interference effects show that the production of periodically timed ankle movements, taken as modelling elements of the normal gait cycle, draws on higher-level cognitive resources involved in working memory. The interference effect predominantly concerns the timing accuracy of the ankle movements. Reduced activations within regions of the left IFG, and in some respect also within the superior parietal lobule, were identified as one factor affecting the timing of periodic ankle movements resulting in involuntary ‘hastening’ during severe dual-task working memory load. This ‘hastening’ phenomenon may be an expression of re-automated locomotor control when higher-order cognitive processing capacity can no longer be allocated to the movements due to the demands of the cognitive task. The results of our study also propose the left IFG as a target region to improve performance during dual-task walking by techniques for non-invasive brain stimulation., Highlights • Neural correlates of involuntary ‘hastening’ of movements during cognitive-motor dual-tasking. • Role of left inferior frontal gyrus and left superior parietal lobe in the temporal regulation of dual-task bilateral movements. • Dissociation between left-hemisphere parietal involvement in external timing of bilateral movements and right hemisphere parietal involvement in interlimb coordination.

Learning is known to facilitate our ability to detect targets in clutter and optimize brain processes for successful visual recognition. Previous brain-imaging studies have focused on identifying spatial patterns (i.e., brain areas) that change with learning, implicating occipitotemporal and frontoparietal areas. However, little is known about the interactions within this network that mediate learning-dependent improvement in complex perceptual tasks (i.e., discrimination of visual forms in clutter). Here we take advantage of the complementary high spatial and temporal resolution of simultaneous EEG-fMRI to identify the learning-dependent changes in spatiotemporal brain patterns that mediate enhanced behavioral sensitivity in the discrimination of global forms after training. We measured the observers' choices when discriminating between concentric and radial patterns presented in noise before and after training. Similarly, we measured the choices of a pattern classifier when predicting each stimulus from EEG-fMRI signals. By comparing the performance of human observers and classifiers, we demonstrated that learning alters sensitivity to visual forms and EEG-fMRI activation patterns related to distinct visual recognition processes. In particular, behavioral improvement after training was associated with changes in (1) early processes involved in the integration of global forms in higher occipitotemporal and parietal areas, and (2) later processes related to categorical judgments in frontal circuits. Thus, our findings provide evidence that learning acts on distinct visual recognition processes and shapes feedforward interactions across brain areas to support performance in complex perceptual tasks.

Learning is thought to facilitate our ability to perform complex perceptual tasks and optimize brain circuits involved in decision making. However, little is known about the experience-dependent mechanisms in the human brain that support our ability to make fine categorical judgments. Previous work has focused on identifying spatial brain patterns (i.e., areas) that change with learning. Here, we take advantage of the complementary high spatial and temporal resolution of simultaneous electroencephalography-functional magnetic resonance imaging (EEG-fMRI) to identify the spatiotemporal dynamics between cortical networks involved in flexible category learning. Observers were trained to use different decision criteria (i.e., category boundaries) when making fine categorical judgments on morphed stimuli (i.e., radial vs. concentric patterns). Our findings demonstrate that learning acts on a feedback-based circuit that supports fine categorical judgments. Experience-dependent changes in the behavioral decision criterion were associated with changes in later perceptual processes engaging higher occipitotemporal and frontoparietal circuits. In contrast, category learning did not modulate early processes in a medial frontotemporal network that are thought to support the coarse interpretation of visual scenes. These findings provide evidence that learning flexible criteria for fine categorical judgments acts on distinct spatiotemporal brain circuits and shapes the readout of sensory signals that provide evidence for categorical decisions.

Object perception is one of the most remarkable capacities of the primate brain. Owing to the large and indeterminate dimensionality of object space, the neural basis of object perception has been difficult to study and remains controversial. Recent work has provided a more precise picture of how 2D and 3D object structure is encoded in intermediate and higher-level visual cortices. Yet, other studies suggest that higher-level visual cortex represents categorical identity rather than structure. Furthermore, object responses are surprisingly adaptive to changes in environmental statistics, implying that learning through evolution, development, and also shorter-term experience during adulthood may optimize the object code. Future progress in reconciling these findings will depend on more effective sampling of the object domain and direct comparison of these competing hypotheses.

Cross-cultural studies have shown that independence in individualistic societies is associated with analytic systems of thoughts, whereas collectivistic societies which place greater emphasis on interdependence are generally predisposed to holistic thinking (Masuda & Nisbett, 2011; Bang, 2015). Further, cultural identity has been shown to effect picture perception and cognitive processes (Nisbett & Miyamoto, 2005). For example, Asians were shown to be more sensitive to contextual rather than focal information compared to Americans in a change-blindness task in which observers had to detect either a focal or contextual change within pairs of images (Masuda & Nisbett, 2006). Here, we build on previous work on perceptual learning (Mayhew, Li, & Kourtzi, 2012) and test the role of individualistic vs. collectivistic influences on learning ability. Eighty-three participants of different cultural backgrounds – consisting of Asian (collectivistic) and European (individualistic) students – were asked to discriminate between radial and concentric Glass patterns embedded in background noise. We employed the Singelis’ (1994) self-construal scale (SCS) to examine whether differences in task performance due to training were influenced by independent or interdependent cultural values. Visual perceptual learning was evident in both groups; that is, all participants improved in accuracy and reaction times through training. Importantly, Asian participants showed higher performance before and during training than European participants, suggesting an advantage in learning to extract global shapes embedded in cluttered backgrounds. This is consistent with the previously reported tendency of collectivists for global processing under perceptual uncertainty. Our findings provide evidence for the role of cross-cultural influences on visual processes that relate to our ability to improve in making perceptual judgements through training and experience. Keywords: visual perception, learning, self-construal

We describe a compelling motion illusion elicited by a huge billboard placed along a street, depicting a building that contains strong perspective cues. When observers move fast along the opposite sidewalk, they perceive the depicted building as rotating in their direction of travel. This is a special case of the ‘following’, or ‘pointing out of the picture’, illusion that elicits a strong illusory motion percept. Here we discuss the cause of the illusory motion and suggest that the brain relies on the depicted perspective cues to infer a 3-D shape and a concomitant motion that is incompatible with the physical pictorial surface.

Recent modelling studies (Hadjipapas et al. [2009]: Neuroimage 44:1290-1303) have shown that it may be possible to distinguish between different neuronal populations on the basis of their macroscopically measured (EEG/MEG) mean field. We set out to test whether the different orientation columns contributing to a signal at a specific cortical location could be identified based on the measured MEG signal. We used 1.5deg square, static, obliquely oriented grating stimuli to generate sustained gamma oscillations in a focal region of primary visual cortex. We then used multivariate classifier methods to predict the orientation (left or right oblique) of the stimuli based purely on the time-series data from this one location. Both the single trial evoked response (0–300 ms) and induced post-transient power spectra (300–2,300 ms, 20–70 Hz band) due to the different stimuli were classifiable significantly above chance in 11/12 and 10/12 datasets respectively. Interestingly, stimulus-specific information is preserved in the sustained part of the gamma oscillation, long after perception has occurred and all neuronal transients have decayed. Importantly, the classification of this induced oscillation was still possible even when the power spectra were rank-transformed showing that the different underlying networks give rise to different characteristic temporal signatures. Hum Brain Mapp, 2010. © 2009 Wiley-Liss, Inc.

In our everyday interactions we encounter a plethora of novel experiences in different contexts that require prompt decisions for successful actions and social interactions. Despite the seeming ease with which we perform these interactions, extracting the key information from the highly complex input of the natural world and deciding how to interpret it is a computationally demanding task for the visual system. Accumulating evidence suggests that the brain solves this problem by combining sensory information and previous knowledge about the environment. Here, we review the neural mechanisms that mediate experience-based plasticity and shape perceptual decisions. We propose that learning plays an important role in the adaptive optimization of visual functions that translate sensory experiences to decisions by shaping neural representations across cortical circuits in the primate brain.

SUMMARY Making successful decisions under uncertainty due to noisy sensory signals is thought to benefit from previous experience. However, the human brain mechanisms that mediate flexible decisions through learning remain largely unknown. Comparing behavioral choices of human observers with those of a pattern classifier based on multivoxel single-trial fMRI signals, we show that category learning shapes processes related to decision variables in frontal and higher occipitotemporal regions rather than signal detection or response execution in primary visual or motor areas. In particular, fMRI signals in prefrontal regions reflect the observers’ behavioral choice according to the learned decision criterion only in the context of the categorization task. In contrast, higher occipitotemporal areas show learning-dependent changes in the representation of perceived categories that are sustained after training independent of the task. These findings demonstrate that learning shapes selective representations of sensory readout signals in accordance with the decision criterion to support flexible decisions.

Perceiving the three-dimensional (3D) properties of the environment relies on the brain bringing together ambiguous cues (e.g., binocular disparity, shading, texture) with information gained from short- and long-term experience. Perceptual aftereffects, in which the perception of an ambiguous 3D stimulus is biased away from the shape of a previously viewed stimulus, provide a sensitive means of probing this process, yet little is known about their neural basis. Here, we investigate 3D aftereffects using psychophysical and functional MRI (fMRI) adaptation paradigms to gain insight into the cortical circuits that mediate the perceptual interpretation of ambiguous depth signals. Using two classic bistable stimuli (Mach card, kinetic depth effect), we test aftereffects produced by 3D shapes defined by binocular (disparity) or monocular (texture, shading) depth cues. We show that the processing of ambiguous 3D stimuli in dorsal visual cortical areas (V3B/KO, V7) and posterior parietal regions is modulated by adaptation in line with perceptual aftereffects. Similar behavioral and fMRI adaptation effects for the two types of bistable stimuli suggest common neural substrates for depth aftereffects independent of the inducing depth cues (disparity, texture, shading). In line with current thinking about the role of adaptation in sensory optimization, our findings provide evidence that estimation of 3D shape in dorsal cortical areas takes account of the adaptive context to resolve depth ambiguity and interpret 3D structure.

Biederman and Cooper (1991a) showed that the presentation of a briefly presented image of an object at one position in the visual field facilitated its identification, as assessed by naming speed and accuracy, several minutes later. The facilitation was unaffected by a translation or a reflection of the stimulus. A component of this priming was visual rather than basic-level conceptual or lexical in that there was less facilitation for an object with the same name (and basic-level class) but a different shape. The invariance of priming to view variables has stood up well over the years and appears to be a general phenomenon--as long as the original structural description can be readily resolved--in that it has also been observed for variations in size and orientation in depth. Although priming was unaffected by a change in position, we documented that there was explicit memory for the position (and orientation and size) of the stimulus. The existence of two forms of representation from the identical stimulus presentation--one invariant and the other dependent on view variables--poses a challenge as to what can be concluded about view invariance from single-unit activity.

Processing of binocular disparity is thought to be widespread throughout cortex, highlighting its importance for perception and action. Yet the computations and functional roles underlying this activity across areas remain largely unknown. Here, we trace the neural representations mediating depth perception across human brain areas using multivariate analysis methods and high-resolution imaging. Presenting disparity-defined planes, we determine functional magnetic resonance imaging (fMRI) selectivity to near versus far depth positions. First, we test the perceptual relevance of this selectivity, comparing the pattern-based decoding of fMRI responses evoked by random dot stereograms that support depth perception (correlated RDS) with the decoding of stimuli containing disparities to which the perceptual system is blind (anticorrelated RDS). Preferential disparity selectivity for correlated stimuli in dorsal (visual and parietal) areas and higher ventral area LO (lateral occipital area) suggests encoding of perceptually relevant information, in contrast to early (V1, V2) and intermediate ventral (V3v, V4) visual cortical areas that show similar selectivity for both correlated and anticorrelated stimuli. Second, manipulating disparity parametrically, we show that dorsal areas encode the metric disparity structure of the viewed stimuli (i.e., disparity magnitude), whereas ventral area LO appears to represent depth position in a categorical manner (i.e., disparity sign). Our findings suggest that activity in both visual streams is commensurate with the use of disparity for depth perception but the neural computations may differ. Intriguingly, perceptually relevant responses in the dorsal stream are tuned to disparity content and emerge at a comparatively earlier stage than categorical representations for depth position in the ventral stream. ispartof: Journal of Neuroscience vol:28 issue:44 pages:11315-27 ispartof: location:United States status: published

Despite the importance of visual categorization for interpreting sensory experiences, little is known about the neural representations that mediate categorical decisions in the human brain. Here, we used psychophysics and pattern classification for the analysis of functional magnetic resonance imaging data to predict the features critical for categorical decisions from brain activity when observers categorized the same stimuli using different rules. Although a large network of cortical and subcortical areas contain information about visual categories, we show that only a subset of these areas shape their selectivity to reflect the behaviorally relevant features rather than simply physical similarity between stimuli. Specifically, temporal and parietal areas show selectivity for the perceived form and motion similarity, respectively. In contrast, frontal areas and the striatum represent the conjunction of spatiotemporal features critical for complex and adaptive categorization tasks and potentially modulate selectivity in temporal and parietal areas. These findings provide novel evidence for flexible neural coding in the human brain that translates sensory experiences to categorical decisions by shaping neural representations across a network of areas with dissociable functional roles in visual categorization.

Binocular disparity, the slight differences between the images registered by our two eyes, provides an important cue when estimating the three-dimensional (3D) structure of the complex environment we inhabit. Sensitivity to binocular disparity is evident at multiple levels of the visual hierarchy in the primate brain, from early visual cortex to parietal and temporal areas. However, the relationship between activity in these areas and key perceptual functions that exploit disparity information for 3D shape perception remains an important open question. Here we investigate the link between human cortical activity and the perception of disparity-defined shape, measuring fMRI responses concurrently with psychophysical shape judgments. We parametrically degraded the coherence of shapes by shuffling the spatial position of dots whose disparity defined the 3D structure and investigated the effect of this stimulus manipulation on both cortical activity and shape discrimination. We report significant relationships between shape coherence and fMRI response in both dorsal (V3, hMT+/V5) and ventral (LOC) visual areas that correspond to the observers' discrimination performance. In contrast to previous suggestions of a dichotomy of disparity-related processes in the ventral and dorsal streams, these findings are consistent with proposed interactions between these pathways that may mediate a continuum of processes important in perceiving 3D shape from coarse contour segmentation to fine curvature estimation.

Predicting future events based on previous knowledge about the environment is critical for successful everyday interactions. Here, we ask which brain regions support our ability to predict the future based on implicit knowledge about the past in young and older age. Combining behavioral and fMRI measurements, we test whether training on structured temporal sequences improves the ability to predict upcoming sensory events; we then compare brain regions involved in learning predictive structures between young and older adults. Our behavioral results demonstrate that exposure to temporal sequences without feedback facilitates the ability of young and older adults to predict the orientation of an upcoming stimulus. Our fMRI results provide evidence for the involvement of corticostriatal regions in learning predictive structures in both young and older learners. In particular, we showed learning-dependent fMRI responses for structured sequences in frontoparietal regions and the striatum (putamen) for young adults. However, for older adults, learning-dependent activations were observed mainly in subcortical (putamen, thalamus) regions but were weaker in frontoparietal regions. Significant correlations of learning-dependent behavioral and fMRI changes in these regions suggest a strong link between brain activations and behavioral improvement rather than general overactivation. Thus, our findings suggest that predicting future events based on knowledge of temporal statistics engages brain regions involved in implicit learning in both young and older adults.

Visual object perception has been suggested to follow two different routes in the human brain: a ventral, view-invariant occipital-temporal route processes object identity, whereas a dorsal, view-dependent occipital-parietal route processes spatial properties of an object. Using fMRI, we addressed the question whether these routes are exclusively involved in either object recognition or spatial representation. We presented subjects with images of natural objects and involved them either in object identification or object orientation judgment task. For both tasks, we observed activation in ventro-temporal as well as parietal areas bilaterally, with significantly stronger responses for the orientation judgment in both ventro-temporal as well as parietal areas. Our findings suggest that object identification and orientation judgment do not follow strictly separable cortical pathways, but rather involve both the dorsal and the ventral stream.

We used human functional magnetic resonance imaging (fMRI) to test whether the human lateral occipital complex (LOC), an area known to be involved in the analysis of visual shape, represents the perceived 3-D shape of objects or simply their 2-D contours. We employed an fMRI adaptation paradigm, in which repeated presentation of a stimulus results in decreased responses compared to responses to different stimuli. We found adaptation in the LOC for images of objects with the same perceived 3-D shape structure but different 2-D contours that resulted from small rotations of the objects in the frontal plane or in depth. However, no adaptation was observed in the LOC for images of objects that had the same 2-D contours but differed in their perceived 3-D shape; namely, 2-D silhouettes versus 3-D shaded images of objects, or convex versus concave objects. Differences in the fMRI adaptation responses across subregions in the LOC suggest that different neural populations in the LOC may mediate different mechanisms for the processing of object features.

The integration of local image features into global shapes was investigated in monkeys and humans using fMRI. An adaptation paradigm was used, in which stimulus selectivity was deduced by changes in the course of adaptation of a pattern of randomly oriented elements. Accordingly, we observed stronger activity when orientation changes in the adapting stimulus resulted in a collinear contour than a different random pattern. This selectivity to collinear contours was observed not only in higher visual areas that are implicated in shape processing, but also in early visual areas where selectivity depended on the receptive field size. These findings suggest that unified shape perception in both monkeys and humans involves multiple visual areas that may integrate local elements to global shapes at different spatial scales.

Here we review recent findings that reveal the functional properties of extra-striate regions in the human visual cortex that are involved in the representation and perception of objects. We characterize both the invariant and non-invariant properties of these regions and we discuss the correlation between activation of these regions and recognition. Overall, these results indicate that the lateral occipital complex plays an important role in human object recognition.

Most studies and theories of object recognition have addressed the perception of rigid objects. Yet, physical objects may also move in a nonrigid manner. A series of priming studies examined the conditions under which observers can recognize novel views of objects moving nonrigidly. Observers were primed with 2 views of a rotating object that were linked by apparent motion or presented statically. The apparent malleability of the rotating prime object varied such that the object appeared to be either malleable or rigid. Novel deformed views of malleable objects were primed when falling within the object's motion path. Priming patterns were significantly more restricted for deformed views of rigid objects. These results suggest that moving malleable objects may be represented as continuous events, whereas rigid objects may not. That is, object representations may be "dynamically remapped" during the analysis of the object's motion.

Depth rotations can reveal new object parts and result in poor recognition of “static” objects (Biederman & Gerhardstein, 1993) . Recent studies have suggested that multiple object views can be associated through temporal contiguity and similarity Edelman & Weinshall, 1991 , Lawson et al., 1994 , Wallis, 1996 . Motion may also play an important role in object recognition since observers recognize novel views of objects rotating in the picture plane more readily than novel views of statically re-oriented objects (Kourtzi & Shiffrar, 1997) . The series of experiments presented here investigated how different views of a depth-rotated object might be linked together even when these views do not share the same parts. The results suggest that depth rotated object views can be linked more readily with motion than with temporal sequence alone to yield priming of novel views of 3D objects that fall in between “known” views. Motion can also enhance path specific view linkage when visible object parts differ across views. Such results suggest that object representations depend on motion processes.

How do people recognize an object in a novel orientation? Psychophysical and neurophysiological studies have suggested that extensive practice is required before observers can recognize an object that has been rotated to a new orientation Because object orientation frequently varies with object movement, we examined whether observers might more readily recognize a moving object in a new orientation Results from a priming study indicate that motion significantly and readily enhances the recognition of new object orientations when those orientations fall within the path of the motion That is motion promotes view-invariant object recognition without practice

A striking example of our sensitivity to dynamic information is our ability to infer motion from still images depicted in paintings, photographs or cartoons. What are the neural mechanisms that mediate this implied motion perception? In a recent paper, Krekelberg et al. demonstrate that form cues that imply motion are integrated with real motion information, and influence perception in both humans and monkeys and the neural processing in prototypical motion areas of the monkey brain. Perception and successful interaction with moving objects entail that the visual system integrates information from form and motion cues into unified dynamic perceptual events. However, traditionally, shape and motion processing have been attributed to anatomically and functionally separable neural pathways in the primate brain [1]. An extrastriate visual area in the medial temporal monkey brain (MT/V5) and its human analogue in the ascending limb of the inferior temporal sulcus (hMT+/V5) have been identified as one of the main regions involved in the analysis of visual motion [2] and [3]. By contrast, regions in the occipitotemporal cortex (V4, IT) have been implicated in the analysis of shape properties and object recognition in the monkey and the human brain [4] and [5]. Although significant progress has been made in uncovering the neural mechanisms that mediate motion and form perception, surprisingly little is known about possible interactions of these mechanisms that may underlie the unified perception of moving objects in our dynamic visual environments. Krekelberg et al.[6] provide evidence for such interactions by showing similar responses in monkey MT and MST cells to real motion and static form patterns that have no coherent physical motion but imply motion. These physiological findings are consistent with the perception of their human and monkey subjects.

Recognizing objects in cluttered scenes is vital for successful interactions in our complex environments. Learning is known to play a key role in facilitating performance in a wide range of perceptual skills not only in young but also older adults. However, the neural mechanisms that support our ability to improve visual form recognition with training in older age remain largely unknown. Here, we combine behavioral and fMRI measurements to identify the brain circuits involved in the learning of global visual forms in the aging human brain. Our findings demonstrate the learning enhances perceptual sensitivity in the discrimination of visual forms similarly in both young and older adults. However, using fMRI we show that the neural circuits involved in visual form learning differ with age. Our results show that in young adults visual shape learning engages a network of occipitotemporal, parietal, and frontal regions that is known to be involved in perceptual decisions. In contrast, in older adults visual shape learning engages primarily parietal regions, suggesting a stronger role of attentionally-guided learning in older age. Interestingly, learning-dependent changes are maintained in higher occipitotemporal and posterior parietal regions, but not in frontal circuits, when observers perform a control task rather than engaging in a visual form discrimination task. Thus, learning may modulate read-out signals in posterior regions related to global form representations independent of the task, whereas task-dependent frontal activations may reflect changes in sensitivity with training in the context of perceptual decision making.

Successful interactions with the environment entail interpreting ambiguous sensory information. To address this challenge it has been suggested that the brain optimizes performance through experience. Here we used functional magnetic resonance imaging (fMRI) to investigate whether perceptual experience modulates the cortical circuits involved in visual awareness. Using ambiguous visual stimuli (binocular rivalry or ambiguous structure-from-motion) we were able to disentangle the co-occurring influences of stimulus repetition and perceptual repetition. For both types of ambiguous stimuli we observed that the mere repetition of the stimulus evoked an entirely different pattern of activity modulations than the repetition of a particular perceptual interpretation of the stimulus. Regarding stimulus repetition, decreased fMRI responses were evident during binocular rivalry but weaker during 3-D motion rivalry. Perceptual repetition, on the other hand, entailed increased activity in stimulus-specific visual brain regions – for binocular rivalry in the early visual regions and for ambiguous structure-from-motion in both early as well as higher visual regions. This indicates that the repeated activation of a visual network mediating a particular percept facilitated its later reactivation. Perceptual repetition was also associated with a response change in the parietal cortex that was similar for the two types of ambiguous stimuli, possibly relating to the temporal integration of perceptual information. We suggest that perceptual repetition is associated with a facilitation of neural activity within and between percept-specific visual networks and parietal networks involved in the temporal integration of perceptual information, thereby enhancing the stability of previously experienced percepts.

Learning is thought to facilitate the recognition of objects by optimizing the tuning of visual neurons to behaviorally relevant features. However, the learning mechanisms that shape neural selectivity for visual forms in the human brain remain essentially unknown. Here, we combine behavioral and functional magnetic resonance imaging (fMRI) measurements to test the mechanisms that mediate enhanced behavioral sensitivity in the discrimination of visual forms after training. In particular, we used high-resolution fMRI and multivoxel pattern classification methods to investigate fine learning-dependent changes in neural preference for global forms. We measured the observers' choices when discriminating between concentric and radial patterns presented in noise before and after training. Similarly, we measured the choices of a pattern classifier when predicting each stimulus from fMRI activity. Comparing the performance of human observers and classifiers demonstrated that learning alters the observers' sensitivity to visual forms and the tuning of fMRI activation patterns in visual areas selective for task-relevant features. In particular, training on low-signal stimuli enhanced the amplitude but reduced the width of pattern-based tuning functions in higher dorsal and ventral visual areas. Thus, our findings suggest that learning of visual patterns is implemented by enhancing the response to the preferred stimulus category and reducing the response to nonpreferred stimuli in higher extrastriate visual cortex.

In perceiving 3D shape from ambiguous shading patterns, humans use the prior knowledge that the light is located above their head and slightly to the left. Although this observation has fascinated scientists and artists for a long time, the neural basis of this “light from above left” preference for the interpretation of 3D shape remains largely unexplored. Combining behavioral and functional MRI measurements coupled with multivoxel pattern analysis, we show that activations in early visual areas predict best the light source direction irrespective of the perceived shape, but activations in higher occipitotemporal and parietal areas predict better the perceived 3D shape irrespective of the light direction. These findings demonstrate that illumination is processed earlier than the representation of 3D shape in the visual system. In contrast to previous suggestions, we propose that prior knowledge about illumination is processed in a bottom-up manner and influences the interpretation of 3D structure at higher stages of processing.

The ability to make categorical decisions and interpret sensory experiences is critical for survival and interactions across the lifespan. However, little is known about the human brain mechanisms that mediate the learning and representation of visual categories in aging. Here we combine behavioral measurements and fMRI measurements to investigate the neural processes that mediate flexible category learning in the aging human brain. Our findings show that training changes the decision criterion (i.e., categorical boundary) that young and older observers use for making categorical judgments. Comparing the behavioral choices of human observers with those of a pattern classifier based upon multivoxel fMRI signals, we demonstrate learning-dependent changes in similar cortical areas for young and older adults. In particular, we show that neural signals in occipito-temporal and posterior parietal regions change through learning to reflect the perceived visual categories. Information in these areas about the perceived visual categories is preserved in aging, whereas information content is compromised in more anterior parietal and frontal circuits. Thus, these findings provide novel evidence for flexible category learning in aging that shapes the neural representations of visual categories to reflect the observers' behavioral judgments.

Recognition of actions and complex movements is fundamental for social interactions and action understanding. While the relationship between motor expertise and visual recognition of body movements has received a vast amount of interest, the role of visual learning remains largely unexplored. Combining psychophysics and fMRI experiments, we investigated neural correlates of visual learning of complex movements. Subjects were trained to visually discriminate between very similar complex movement stimuli generated by motion morphing that were either compatible (Experiments 1 and 2) or incompatible (Experiment 3) with human movement execution. Employing an fMRI adaptation paradigm as index of discriminability, we scanned human subjects before and after discrimination training. The results of Experiment 1 revealed three different effects as a consequence of training; 1) Emerging fMRI-selective adaptation in general motion related areas (hMT/V5+, KO/V3b) for the differences between human-like movements. 2) Enhanced of fMRI-selective adaptation already present before training in biological motion related areas (pSTS, FBA). 3) Changes covarying with task difficulty in frontal areas. Moreover, the observed activity changes were specific to the trained movement patterns (Experiment 2). The results of Experiment 3, testing artificial movement stimuli, were strikingly similar to the results obtained for human movements. General and biological motion related areas showed movement-specific changes in fMRI-selective adaptation for the differences between the stimuli after training. These results support the existence of a powerful visual machinery for the learning of complex motion patterns that is independent of motor execution. We propose thus a key role of visual learning in action recognition. ispartof: Journal of Neuroscience vol:29 issue:44 pages:14026-14038 ispartof: location:United States status: published

The ability to detect and identify targets in cluttered scenes is a critical skill for survival and interactions. To solve this challenge the brain has optimized mechanisms for capitalizing on frequently occurring regularities in the environment. Although evolution and development have been suggested to shape the brain's architecture in a manner that resembles these natural statistics, we provide novel evidence that short-term experience in adulthood may modify the brain's functional organization to support integration of signals atypical of shape contours in natural scenes. Although collinearity is a prevalent principle for perceptual integration in natural scenes, we show that observers learn to exploit other image regularities (i.e., orthogonal alignments of segments at an angle to the contour path) that typically signify discontinuities. Combining behavioral and functional MRI measurements, we demonstrate that this flexible learning is mediated by changes in the neural representations of behaviorally relevant image regularities primarily in dorsal visual areas. These changes in neural sensitivity are in line with changes in perceptual sensitivity for the detection of orthogonal contours and are evident only in observers that show significant performance improvement. In contrast, changes in the activation extent in frontoparietal regions are evident independent of performance changes, may support the detection of salient regions, and modulate perceptual integration in occipitotemporal areas in a top-down manner. Thus experience at shorter timescales in adulthood supports the adaptive functional optimization of visual circuits for flexible interpretation of natural scenes.

Extensive psychophysical and computational work proposes that the perception of coherent and meaningful structures in natural images relies on neural processes that convert information about local edges in primary visual cortex to complex object features represented in the temporal cortex. However, the neural basis of these mid-level vision mechanisms in the human brain remains largely unknown. Here, we examine functional MRI (fMRI) selectivity for global forms in the human visual pathways using sensitive multivariate analysis methods that take advantage of information across brain activation patterns. We use Glass patterns, parametrically varying the perceived global form (concentric, radial, translational) while ensuring that the local statistics remain similar. Our findings show a continuum of integration processes that convert selectivity for local signals (orientation, position) in early visual areas to selectivity for global form structure in higher occipitotemporal areas. Interestingly, higher occipitotemporal areas discern differences in global form structure rather than low-level stimulus properties with higher accuracy than early visual areas while relying on information from smaller but more selective neural populations (smaller voxel pattern size), consistent with global pooling mechanisms of local orientation signals. These findings suggest that the human visual system uses a code of increasing efficiency across stages of analysis that is critical for the successful detection and recognition of objects in complex environments.

Understanding dynamic events entails the integration of information about form and motion that is crucial for fast and successful interactions in complex environments. A striking example of our sensitivity to dynamic information is our ability to recognize animate figures by the way they move and infer motion from still images. Accumulating evidence for form and motion interactions contrasts with the traditional dissociation between shape and motion-related processes in the ventral and dorsal visual pathways. By combining findings from physiology and brain imaging it can be demonstrated that the primate brain converts information about spatiotemporal sequences into meaningful actions through interactions between early and higher visual areas processing form and motion and frontal-parietal circuits involved in the understanding of actions.

In its search for neural codes, the field of visual neuroscience has uncovered neural representations that reflect the structure of stimuli of variable complexity from simple features to object categories. However, accumulating evidence suggests an adaptive neural code that is dynamically shaped by experience to support flexible and efficient perceptual decisions. Here, we review work showing that experience plays a critical role in molding midlevel visual representations for perceptual decisions. Combining behavioral and brain imaging measurements, we demonstrate that learning optimizes feature binding for object recognition in cluttered scenes, and tunes the neural representations of informative image parts to support efficient categorical judgements. Our findings indicate that similar learning mechanisms may mediate long-term optimization through development, tune the visual system to fundamental principles of feature binding, and optimize feature templates for perceptual decisions.

The way in which input noise perturbs the behavior of a system depends on the internal processing structure of the system. In visual psychophysics, there is a long tradition of using external noise methods (i.e., adding noise to visual stimuli) as tools for system identification. Here, we demonstrate that external noise affects processing of visual scenes at different cortical areas along the human ventral visual pathway, from retinotopic regions to higher occipitotemporal areas implicated in visual shape processing. We found that when the contrast of the stimulus was held constant, the further away from the retinal input a cortical area was the more its activity, as measured with functional magnetic resonance imaging (fMRI), depended on the signal-to-noise ratio (SNR) of the visual stimulus. A similar pattern of results was observed when trials with correct and incorrect responses were analyzed separately. We interpret these findings by extending signal detection theory to fMRI data analysis. This approach reveals the sequential ordering of decision stages in the cortex by exploiting the relation between fMRI response and stimulus SNR. In particular, our findings provide novel evidence that occipitotemporal areas in the ventral visual pathway form a cascade of decision stages with increasing degree of signal uncertainty and feature invariance.

The recognition of complex body movements and actions is a fundamental visual capacity very important for social communication. It seems possible that movement recognition is based on a general capability of the visual system to learn complex visual motion patterns. Alternatively, this visual function might exploit specialized mechanisms for the analysis of biologically relevant movements, for example, of humans or animals. To investigate this question, we trained human observers to discriminate novel motion patterns that were generated, exploiting a new technique for stimulus generation by motion morphing. We tested the learning of different classes of novel movement stimuli. One group of stimuli was fully consistent with human movements. A second class of stimuli was based on artificial skeleton models that were inconsistent with human and animal bodies. A third group of stimuli specified the same local motion information as human movements but was inconsistent with an underlying articulated shape. Participants learned both classes of articulated movements very fast in an orientation-dependent manner. Learning speed and accuracy were strikingly similar and independent of the similarity of the stimuli with biologically relevant body shapes. For the class of stimuli without underlying articulated shape, however, we did not observe significant improvements of the discrimination performance after training. Our results indicate the existence of a fast visual learning process for complex articulated movement patterns, which likely is relevant for biological motion perception. This process seems to operate independently of the consistency of the patterns with biologically relevant body shapes but seems to require the compatibility of the learned movements with a global underlying shape. ispartof: Journal of vision vol:6 issue:8 pages:791-804 ispartof: location:United States status: published

The capability of the adult primate visual system for rapid and accurate recognition of targets in cluttered, natural scenes far surpasses the abilities of state-of-the-art artificial vision systems. Understanding this capability remains a fundamental challenge in visual neuroscience. Recent experimental evidence suggests that adaptive coding strategies facilitated by underlying neural plasticity enable the adult brain to learn from visual experience and shape its ability to integrate and recognize coherent visual objects.

The integration of local elements to coherent forms is at the core of understanding visual perception. Accumulating evidence suggests that both early retinotopic and higher occipitotemporal areas contribute to the integration of local elements to global forms. However, the spatiotemporal characteristics of form analysis in the human visual cortex remain largely unknown. The aim of this study was to investigate form analysis at different spatial (global vs. local structure) and temporal (different stimulus presentation rates) scales across stages of visual analysis (from V1 to the lateral occipital complex-LOC) in the human brain. We used closed contours rendered by Gabor elements and manipulated either the global contour structure or the orientation of the local Gabor elements. Our rapid event-related fMRI adaptation studies suggest that contour integration and form processing in early visual areas is transient and limited within the local neighborhood of their cells' receptive field. In contrast, higher visual areas appear to process the perceived global form in a more sustained manner. Finally, we demonstrate that these spatiotemporal properties of form processing in the visual cortex are modulated by attention. Attention to the global form maintains sustained processing in occipitotemporal areas, whereas attention to local elements enhances their integration in early visual areas. These findings provide novel neuroimaging evidence for form analysis at different spatiotemporal scales across human visual areas and validate the use of rapid event-related fMRI adaptation for investigating processing across stages of visual analysis in the human brain.

Our perception of the world's three-dimensional (3D) structure is critical for object recognition, navigation and planning actions. To accomplish this, the brain combines different types of visual information about depth structure, but at present, the neural architecture mediating this combination remains largely unknown. Here, we report neuroimaging correlates of human 3D shape perception from the combination of two depth cues. We measured fMRI responses while observers judged the 3D structure of two sequentially presented images of slanted planes defined by binocular disparity and perspective. We compared the behavioral and fMRI responses evoked by changes in one or both of the depth cues. fMRI responses in extrastriate areas (hMT+/V5 and lateral occipital complex), rather than responses in early retinotopic areas, reflected differences in perceived 3D shape, suggesting 'combined-cue' representations in higher visual areas. These findings provide insight into the neural circuits engaged when the human brain combines different information sources for unified 3D visual perception.