Search

A major principle of organization of the visual system is between a dorsal stream that processes visuomotor information and a ventral stream that supports object recognition. Most research has focused on dissociating processing across these two streams. Here we focus on how the two streams interact. We tested neurologically-intact and impaired participants in an object categorization task over two classes of objects that depend on processing within both streams-hands and tools. We measured how unconscious processing of images from one of these categories (e.g., tools) affects the recognition of images from the other category (i.e., hands). Our findings with neurologically-intact participants demonstrated that processing an image of a hand hampers the subsequent processing of an image of a tool, and vice versa. These results were not present in apraxic patients (N = 3). These findings suggest local and global inhibitory processes working in tandem to co-register information across the two streams.

Multivoxel pattern analysis (MVPA) over functional MRI data can distinguish neural representational states that do not differ in their overall amplitude of BOLD contrast. Here we used MVPA to test whether simple intransitive actions can be distinguished in primary motor cortex. Participants rotated and flexed each of their extremities (hands and feet) during fMRI scanning. The primary motor cortex for the hand/wrist was functionally defined in each hemisphere in each subject. Within those subject-specific ROIs, we found that the average amplitude of BOLD contrast for two different movements of the contralateral hand (rotation, flexion) were higher than for the ipsilateral hand, as well as movements by both feet; however, there was no difference in amplitude between the two different types of movements for the contralateral hand. Using multivoxel pattern analysis (linear correlation), we were able to distinguish the two movements for the contralateral hand. These findings demonstrate that simple intransitive actions can be distinguished in primary motor areas using multivoxel pattern analysis.

Producing a tool use gesture is a complex process drawing upon the integration of stored knowledge of tools and their associated actions with sensory-motor mechanisms supporting the planning and control of hand and arm actions. Understanding how sensory-motor systems in parietal cortex interface with semantic representations of actions and objects in the temporal lobe remains a critical issue, and is hypothesized to be a key determinant of the severity of limb apraxia, a deficit in producing skilled action after left hemisphere stroke. We used voxel-based and connectome-based lesion symptom mapping with data from 57 left hemisphere stroke participants to assess the lesion sites and structural disconnection patterns associated with poor tool use gesturing. We found that structural disconnection between the left inferior parietal lobule, lateral temporal lobe (left middle temporal gyrus) and ventral temporal cortex (left medial fusiform gyrus) predicted the severity of tool use gesturing performance. Control analyses demonstrated that reductions in right-hand grip strength were associated with motor system disconnection, bypassing regions supporting tool use gesturing. Our findings provide causal evidence that limb apraxia may arise, in part, from disconnection of conceptual representations in the temporal lobe from mechanisms enabling skilled action production in the inferior parietal lobule.

The Translational Brain Mapping Program at the University of Rochester is an interdisciplinary effort that integrates cognitive science, neurophysiology, neuroanesthesia, and neurosurgery. Patients who have tumors or epileptogenic tissue in eloquent brain areas are studied preoperatively with functional and structural MRI, and intraoperatively with direct electrical stimulation mapping. Post-operative neural and cognitive outcome measures fuel basic science studies about the factors that mediate good versus poor outcome after surgery, and how brain mapping can be further optimized to ensure the best outcome for future patients. In this article, we describe the interdisciplinary workflow that allows our team to meet the synergistic goals of optimizing patient outcome and advancing scientific understanding of the human brain.

Interacting with manipulable objects (tools) requires the integration of diverse computations supported by anatomically remote regions. Previous functional neuroimaging research has demonstrated the left supramarginal (SMG) exhibits functional connectivity to both ventral and dorsal pathways, supporting the integration of ventrally-mediated tool properties and conceptual knowledge with dorsally-computed volumetric and structural representations of tools. This architecture affords us the opportunity to test whether interactions between the left SMG, ventral visual pathway, and dorsal visual pathway are differentially modulated when participants plan and generate tool-directed gestures emphasizing functional manipulation (tool use gesturing) or structure-based grasping (tool transport gesturing). We found that functional connectivity between the left SMG, ventral temporal cortex (bilateral fusiform gyri), and dorsal visual pathway (left superior parietal lobule/posterior intraparietal sulcus) was maximal for tool transport planning and gesturing, whereas functional connectivity between the left SMG, left ventral anterior temporal lobe, and left frontal operculum was maximal for tool use planning and gesturing. These results demonstrate that functional connectivity to the left SMG is differentially modulated by tool use and tool transport gesturing, suggesting that distinct tool features computed by the two object processing pathways are integrated in the parietal lobe in the service of tool-directed action.

When pantomiming the use of tools, patients with limb apraxia after left hemisphere stroke (LCVA) produce more spatiotemporal hand action errors with tools associated with conflicting actions for use versus grasp-to-pick-up (e.g., corkscrew) than tools having a single action for both use and grasp (e.g., hammer). There are two possible accounts for this pattern of results. Reduced performance with ‘conflict’ tools may simply reflect weakened automaticity of use action activation, which is evident only when the use and grasp actions are not redundant. Alternatively, poor use performance may reflect reduced ability of appropriate tool use actions to compete with task-inappropriate action representations. To address this issue, we developed a Stroop-like experiment in which 21 LCVA and 8 neurotypical participants performed pantomime actions in blocks containing two tools that were similar (“neighbors”) in terms of hand action or function, or unrelated on either dimension. In a congruent condition, they pantomimed the use action associated with the visually presented tool, whereas in an incongruent condition, they pantomimed the use action for the other tool in the block. Relative to controls and other task conditions, LCVA participants showed reductions in hand action errors in incongruent relative to congruent action trials; furthermore, the degree of reduction in this incongruence effect was related to the participants’ susceptibility to grasp-on-use conflict in a separate test of pantomime to the sight of tools. Support vector regression lesion-symptom mapping analyses identified the left inferior frontal gyrus, supramarginal gyrus, and superior longitudinal fasciculus as core neuroanatomical sites associated with abnormal performance on both tasks. Collectively, the results indicate that weakened activation of tool use actions in limb apraxia gives rise to reduced ability of these actions to compete for task-appropriate selection when competition arises within single tools (grasp-on-use conflict) as well as between two tools (reduced neighborhood effects).

Neural responses to small manipulable objects (“tools”) in high-level visual areas in ventral temporal cortex (VTC) provide an opportunity to test how anatomically remote regions modulate ventral stream processing in a domain-specific manner. Prior patient studies indicate that grasp-relevant information can be computed about objects by dorsal stream structures independently of processing in VTC. Prior functional neuroimaging studies indicate privileged functional connectivity between regions of VTC exhibiting tool preferences and regions of parietal cortex supporting object-directed action. Here we test whether lesions to parietal cortex modulate tool preferences within ventral and lateral temporal cortex. We found that lesions to the left anterior intraparietal sulcus, a region that supports hand-shaping during object grasping and manipulation, modulate tool preferences in left VTC and in the left posterior middle temporal gyrus. Control analyses demonstrated that neural responses to “place” stimuli in left VTC were unaffected by lesions to parietal cortex, indicating domain-specific consequences for ventral stream neural responses in the setting of parietal lesions. These findings provide causal evidence that neural specificity for “tools” in ventral and lateral temporal lobe areas may arise, in part, from online inputs to VTC from parietal areas that receive inputs via the dorsal visual pathway.

Frontal and temporal white matter pathways play key roles in language processing, but the specific computations supported by different tracts remain a matter of study. A role in speech planning has been proposed for a recently described pathway, the frontal aslant tract (FAT), which connects the posterior inferior frontal gyrus to the pre-SMA. Here, we use longitudinal functional and structural MRI and behavioral testing to evaluate the behavioral consequences of a lesion to the left FAT that was incurred during surgical resection of a frontal glioma in a 60-year-old woman, Patient AF. The pattern of performance in AF is compared, using the same measures, with that in a 37-year-old individual who underwent a left anterior temporal resection and hippocampectomy (Patient AG). AF and AG were both cognitively intact preoperatively but exhibited specific and doubly dissociable behavioral deficits postoperatively: AF had dysfluent speech but no word finding difficulty, whereas AG had word finding difficulty but otherwise fluent speech. Probabilistic tractography showed that the left FAT was lesioned postoperatively in AF (but not AG) whereas the inferior longitudinal fasciculus was lesioned in AG (but not AF). Those structural changes were supported by corresponding changes in functional connectivity to the posterior inferior frontal gyrus: decreased functional connectivity postoperatively between the posterior inferior frontal gyrus and pre-SMA in AF (but not AG) and decreased functional connectivity between the posterior inferior frontal gyrus and the middle temporal gyrus in AG (but not AF). We suggest from these findings that the left FAT serves as a key communicative link between sentence planning and lexical access processes.

The appropriate use of everyday objects requires the integration of action and function knowledge. Previous research suggests that action knowledge is represented in frontoparietal areas while function knowledge is represented in temporal lobe regions. Here we used multivoxel pattern analysis to investigate the representation of object-directed action and function knowledge while participants executed pantomimes of familiar tool actions. A novel approach for decoding object knowledge was used in which classifiers were trained on one pair of objects and then tested on a distinct pair; this permitted a measurement of classification accuracy over and above object-specific information. Region of interest (ROI) analyses showed that object-directed actions could be decoded in tool-preferring regions of both parietal and temporal cortex, while no independently defined tool-preferring ROI showed successful decoding of object function. However, a whole-brain searchlight analysis revealed that while frontoparietal motor and peri-motor regions are engaged in the representation of object-directed actions, medial temporal lobe areas in the left hemisphere are involved in the representation of function knowledge. These results indicate that both action and function knowledge are represented in a topographically coherent manner that is amenable to study with multivariate approaches, and that the left medial temporal cortex represents knowledge of object function.

A whole-brain network of regions collectively supports the ability to recognize and use objects—the Tool Processing Network. Little is known about how functional interactions within the Tool Processing Network are modulated in a task-dependent manner. We designed an fMRI experiment in which participants were required to either generate object pantomimes or to carry out a picture matching task over the same images of tools, while holding all aspects of stimulus presentation constant across the tasks. The Tool Processing Network was defined with an independent functional localizer, and functional connectivity within the network was measured during the pantomime and picture matching tasks. Relative to tool picture matching, tool pantomiming led to an increase in functional connectivity between ventral stream regions and left parietal and frontal-motor areas; in contrast, the matching task was associated with an increase in functional connectivity among regions in ventral temporo-occipital cortex, and between ventral temporal regions and the left inferior parietal lobule. Graph-theory analyses over the functional connectivity data indicated that the left premotor cortex and left lateral occipital complex were hub-like (exhibited high betweenness centrality) during tool pantomiming, while ventral stream regions (left medial fusiform gyrus and left posterior middle temporal gyrus) were hub-like during the picture matching task. These results demonstrate task-specific modulation of functional interactions among a common set of regions, and indicate dynamic coupling of anatomically remote regions in task-dependent manner.

Prior research using functional magnetic resonance imaging (fMRI) [1-4] and behavioral studies of patients with acquired or congenital amusia [5-8] suggest that the right posterior superior temporal gyrus (STG) in the human brain is specialized for aspects of music processing (for review, see [9-12]). Intracranial electrical brain stimulation in awake neurosurgery patients is a powerful means to determine the computations supported by specific brain regions and networks [13-21] because it provides reversible causal evidence with high spatial resolution (for review, see [22, 23]). Prior intracranial stimulation or cortical cooling studies have investigated musical abilities related to reading music scores [13, 14] and singing familiar songs [24, 25]. However, individuals with amusia (congenitally, or from a brain injury) have difficulty humming melodies but can be spared for singing familiar songs with familiar lyrics [26]. Here we report a detailed study of a musician with a low-grade tumor in the right temporal lobe. Functional MRI was used pre-operatively to localize music processing to the right STG, and the patient subsequently underwent awake intraoperative mapping using direct electrical stimulation during a melody repetition task. Stimulation of the right STG induced "music arrest" and errors in pitch but did not affect language processing. These findings provide causal evidence for the functional segregation of music and language processing in the human brain and confirm a specific role of the right STG in melody processing. VIDEO ABSTRACT.

Recent efforts to characterize visual object representations in the ventral object processing pathway in the human brain have led to contrasting proposals about the causes of neural specificity for different categories. Here we use multivariate techniques in a novel way to relate patterns of functional connectivity to patterns of stimulus preferences. Stimulus preferences were measured throughout the ventral stream to tools, animals, faces and places; separately, we measured the strength of functional connectivity of each voxel in the ventral stream to category-preferring regions outside the ventral stream. Multivariate analyses were then performed over ventral stream voxels, relating 'category-preferences' to 'functional connectivity preferences'. We show that the relation of those two measures doubly dissociates 'tools' and 'places', within what is ostensibly 'place' selective cortex (parahippocampal gyrus). Specifically, in the parahippocampal gyrus, functional connectivity to the left inferior parietal lobule is selectively related to stimulus preferences for tools (and not places), while functional connectivity to retrosplenial cortex is selectively related to place preferences (and not tools preferences). These findings indicate that functional connectivity can be used to index representational content rather than just provide an understanding of 'which regions are talking to which regions'. We suggest that the connectivity of the brain is what drives category-specificity in the ventral stream, and that if this is correct, then understanding the connectivity of the ventral stream will be key to understanding the causes and function of category-specific neural organization.

Prior neuroimaging and neuropsychological research indicates that the left inferior parietal lobule in the human brain is a critical substrate for representing object manipulation knowledge. In the present functional MRI study we used multivoxel pattern analyses to test whether action similarity among objects can be decoded in the inferior parietal lobule independent of the task applied to objects (identification or pantomime) and stimulus format in which stimuli are presented (pictures or printed words). Participants pantomimed the use of objects, cued by printed words, or identified pictures of objects. Classifiers were trained and tested across task (e.g., training data: pantomime; testing data: identification), stimulus format (e.g., training data: word format; testing format: picture) and specific objects (e.g., training data: scissors vs. corkscrew; testing data: pliers vs. screwdriver). The only brain region in which action relations among objects could be decoded across task, stimulus format and objects was the inferior parietal lobule. By contrast, medial aspects of the ventral surface of the left temporal lobe represented object function, albeit not at the same level of abstractness as actions in the inferior parietal lobule. These results suggest compulsory access to abstract action information in the inferior parietal lobe even when simply identifying objects.

The format of high-level object representations in temporal-occipital cortex is a fundamental and as yet unresolved issue. Here we use fMRI to show that human lateral occipital cortex (LOC) encodes novel 3-D objects in a multisensory and part-based format. We show that visual and haptic exploration of objects leads to similar patterns of neural activity in human LOC and that the shared variance between visually and haptically induced patterns of BOLD contrast in LOC reflects the part structure of the objects. We also show that linear classifiers trained on neural data from LOC on a subset of the objects successfully predict a novel object based on its component part structure. These data demonstrate a multisensory code for object representations in LOC that specifies the part structure of objects.

Visual processing of complex objects is supported by the ventral visual pathway in the service of object identification and by the dorsal visual pathway in the service of object-directed reaching and grasping. Here, we address how these two streams interact during tool processing, by exploiting the known asymmetry in projections of subcortical magnocellular and parvocellular inputs to the dorsal and ventral streams. The ventral visual pathway receives both parvocellular and magnocellular input, whereas the dorsal visual pathway receives largely magnocellular input. We used fMRI to measure tool preferences in parietal cortex when the images were presented at either high or low temporal frequencies, exploiting the fact that parvocellular channels project principally to the ventral but not dorsal visual pathway. We reason that regions of parietal cortex that exhibit tool preferences for stimuli presented at frequencies characteristic of the parvocellular pathway receive their inputs from the ventral stream. We found that the left inferior parietal lobule, in the vicinity of the supramarginal gyrus, exhibited tool preferences for images presented at low temporal frequencies, whereas superior and posterior parietal regions exhibited tool preferences for images present at high temporal frequencies. These data indicate that object identity, processed within the ventral stream, is communicated to the left inferior parietal lobule and may there combine with inputs from the dorsal visual pathway to allow for functionally appropriate object manipulation.

Viewing images of manipulable objects elicits differential blood oxygen level-dependent (BOLD) contrast across parietal and dorsal occipital areas of the human brain that support object-directed reaching, grasping, and complex object manipulation. However, it is unknown which object-selective regions of parietal cortex receive their principal inputs from the ventral object-processing pathway and which receive their inputs from the dorsal object-processing pathway. Parietal areas that receive their inputs from the ventral visual pathway, rather than from the dorsal stream, will have inputs that are already filtered through object categorization and identification processes. This predicts that parietal regions that receive inputs from the ventral visual pathway should exhibit object-selective responses that are resilient to contralateral visual field biases. To test this hypothesis, adult participants viewed images of tools and animals that were presented to the left or right visual fields during functional magnetic resonance imaging (fMRI). We found that the left inferior parietal lobule showed robust tool preferences independently of the visual field in which tool stimuli were presented. In contrast, a region in posterior parietal/dorsal occipital cortex in the right hemisphere exhibited an interaction between visual field and category: tool-preferences were strongest contralateral to the stimulus. These findings suggest that action knowledge accessed in the left inferior parietal lobule operates over inputs that are abstracted from the visual input and is contingent on analysis by the ventral visual pathway, consistent with its putative role in supporting object manipulation knowledge.

A number of studies have observed that the motor system is activated when processing the semantics of manipulable objects. Such phenomena have been taken as evidence that simulation over motor representations is a necessary and intermediary step in the process of conceptual understanding. Cognitive neuropsychological evaluations of patients with impairments for action knowledge permit a direct test of the necessity of motor simulation in conceptual processing. Here, we report the performance of a forty-seven year-old male individual (Case AA) and six age-matched control participants on a number of tests probing action and object knowledge. Case AA had a large left hemisphere frontal-parietal lesion and hemiplegia affecting his right arm and leg. Case AA presented with impairments for object-associated action production, and his conceptual knowledge of actions was severely impaired. In contrast, his knowledge of objects such as tools and other manipulable objects was largely preserved. The dissociation between action and object knowledge is difficult to reconcile with strong forms of the embodied cognition hypothesis. We suggest that these, and other similar findings, point to the need to develop tractable hypotheses about the dynamics of information exchange among sensory, motor and conceptual processes.

Information about object-associated manipulations is lateralized to left parietal regions, while information about the visual form of tools is represented bilaterally in ventral occipito-temporal cortex. It is unknown how lateralization of motor-relevant information in left-hemisphere dorsal stream regions may affect the visual processing of manipulable objects. We used a lateralized masked priming paradigm to test for a right visual field (RVF) advantage in tool processing. Target stimuli were tools and animals, and briefly presented primes were identical to or scrambled versions of the targets. In Experiment 1, primes were presented either to the left or to the right of the centrally presented target, while in Experiment 2, primes were presented in one of eight locations arranged radially around the target. In both experiments, there was a RVF advantage in priming effects for tool but not for animal targets. Control experiments showed that participants were at chance for matching the identity of the lateralized primes in a picture-word matching experiment and also ruled out a general RVF speed-of-processing advantage for tool images. These results indicate that the overrepresentation of tool knowledge in the left hemisphere affects visual object recognition and suggests that interactions between the dorsal and ventral streams occurs during object categorization.