Search

Converging results suggest that perception is controlled by rhythmic processes in the brain. In the auditory domain, neuroimaging studies show that the perception of brief sounds is shaped by rhythmic activity prior to the stimulus and electrophysiological recordings have linked delta band (1-2 Hz) activity to the functioning of individual neurons. These results have promoted theories of rhythmic modes of listening and generally suggest that the perceptually relevant encoding of acoustic information is structured by rhythmic processes along auditory pathways. A prediction from this perspective – which so far has not been tested – is that such rhythmic processes also shape how acoustic information is combined over time to judge extended soundscapes. The present study was designed to directly test this prediction. Human participants judged the overall change in perceived frequency content in temporally extended (1.2 to 1.8 s) soundscapes, while the perceptual use of the available sensory evidence was quantified using psychophysical reverse correlation. Model-based analysis of individual participant’s perceptual weights revealed a rich temporal structure, including linear trends, a U-shaped profile tied to the overall stimulus duration, and importantly, rhythmic components at the time scale of 1 to 2Hz. The collective evidence found here across four versions of the experiment supports the notion that rhythmic processes operating on the delta band time scale structure how perception samples temporally extended acoustic scenes.

Proceedings of the ICA 2019 and EAA Euroregio : 23rd International Congress on Acoustics, integrating 4th EAA Euroregio 2019 : 9-13 September 2019, Aachen, Germany / proceedings editor: Martin Ochmann, Michael Vorländer, Janina Fels 23rd International Congress on Acoustics, integrating 4th EAA Euroregio 2019, ICA 2019, Aachen, Germany, 9 Sep 2019 - 13 Sep 2019; Aachen (2019)., Published by Aachen

Visual speech is an integral part of communication. Yet it remains unclear whether semantic information carried by movements of the lips or tongue is represented in the same brain regions that mediate acoustic speech representations. Behaviourally, our ability to understand acoustic speech seems independent from that to understand visual speech, but neuroimaging studies suggest that acoustic and visual speech representations largely overlap. To resolve this discrepancy, and to understand whether acoustic and lip-reading speech comprehension are mediated by the same cerebral representations, we systematically probed where the brain represents acoustically and visually conveyed word identities in a human MEG study. We designed a single-trial classification paradigm to dissociate where cerebral representations merely reflect the sensory stimulus and where they are predictive of the participant’s percept. In general, those brain regions allowing for the highest word classification were distinct from those in which cerebral representations were predictive of participant’s percept. Across the brain, word representations were largely modality-specific and auditory and visual comprehension were mediated by distinct left-lateralised ventral and dorsal fronto-temporal regions, respectively. Only within the inferior frontal gyrus and the anterior temporal lobe did auditory and visual representations converge. These results provide a neural explanation for why acoustic speech comprehension is a poor predictor of lip-reading skills and suggests that those cerebral speech representations that encode word identity may be more modality-specific than often upheld.

Research has shown participants associate high pitch tones with small objects, and low pitch tones with large objects. Yet it remains unclear when these associations emerge in neural signals, and whether or not they are likely the result of predictive coding mechanisms being influenced by multisensory priors. Here we investigated these questions using a modified version of the implicit association task, 128-channel human EEG, and two approaches to single-trial analysis (linear discriminant and mutual information). During two interlaced discrimination tasks (auditory high/low tone and visual small/large circle), one stimulus was presented per trial and the auditory stimulus-response assignment was manipulated. On congruent trials preferred pairings (high tone, small circle) were assigned to the same response key, and on incongruent trials non-preferred pairings were (low tone, small circle). The results showed participants (male and female) responded faster during auditory congruent than incongruent trials. The EEG results showed that acoustic pitch and visual size were represented early in the trial (~100 ms and ~220 ms), over temporal and frontal regions. Neural signals were also modulated by congruency early in the trial for auditory (

Sensory discriminations, such as judgements about visual motion, often benefit from multisensory evidence. Despite many reports of enhanced brain activity during multisensory conditions, it remains unclear which dynamic processes implement the multisensory benefit for an upcoming decision in the human brain. Specifically, it remains difficult to attribute perceptual benefits to specific processes, such as early sensory encoding, the transformation of sensory representations into a motor response, or to more unspecific processes such as attention. We combined an audio-visual motion discrimination task with the single-trial mapping of dynamic sensory representations in EEG activity to localize when and where multisensory congruency facilitates perceptual accuracy. Our results show that a congruent sound facilitates the encoding of motion direction in occipital sensory - as opposed to parieto-frontal - cortices, and facilitates later - as opposed to early (i.e. below 100 ms) - sensory activations. This multisensory enhancement was visible as an earlier rise of motion-sensitive activity in middle-occipital regions about 350 ms from stimulus onset, which reflected the better discriminability of motion direction from brain activity and correlated with the perceptual benefit provided by congruent multisensory information. This supports a hierarchical model of multisensory integration in which the enhancement of relevant sensory cortical representations is transformed into a more accurate choice., Highlights • Feature specific multisensory integration occurs in sensory not amodal cortex. • Feature specific integration occurs late, i.e. around 350 ms post stimulus onset. • Acoustic and visual representations interact in occipital motion regions.

The qualities of perception depend not only on the sensory inputs but also on the brain state before stimulus presentation. Although the collective evidence from neuroimaging studies for a relation between prestimulus state and perception is strong, the interpretation in the context of sensory computations or decision processes has remained difficult. In the auditory system, for example, previous studies have reported a wide range of effects in terms of the perceptually relevant frequency bands and state parameters (phase/power). To dissociate influences of state on earlier sensory representations and higher-level decision processes, we collected behavioral and EEG data in human participants performing two auditory discrimination tasks relying on distinct acoustic features. Using single-trial decoding, we quantified the relation between prestimulus activity, relevant sensory evidence, and choice in different task-relevant EEG components. Within auditory networks, we found that phase had no direct influence on choice, whereas power in task-specific frequency bands affected the encoding of sensory evidence. Within later-activated frontoparietal regions, theta and alpha phase had a direct influence on choice, without involving sensory evidence. These results delineate two consistent mechanisms by which prestimulus activity shapes perception. However, the timescales of the relevant neural activity depend on the specific brain regions engaged by the respective task.

The phase of low‐frequency network activity in auditory cortex captures changes in neural\ud excitability, entrains to the temporal structure of natural sounds and correlates with the perceptual\ud performance in acoustic tasks. While these observations suggest a causal link between network\ud rhythms and perception, it remains unknown how precisely they affect the processes by which\ud neural populations encode sounds. We addressed this question by analysing neural responses in the\ud auditory cortex of anesthetized rats using stimulus‐response models. These models included a\ud parametric dependence on the phase of local field potential rhythms in both stimulus‐unrelated\ud background activity and the stimulus‐response transfer function. We found that phase‐dependent\ud models better reproduced the observed responses than static models, both during stimulation with a\ud series of natural sounds and during epochs of silence. This was due to two factors: phase‐dependent\ud variations in background firing (most prominent for delta; 1‐4Hz) and modulations of response gain\ud that rhythmically amplify and attenuate the responses at specific phases of the rhythm (prominent\ud for frequencies between 2Hz and 12Hz). These results provide a quantitative characterization of how\ud slow auditory cortical rhythms shape sound encoding and suggest a differential contribution of\ud network activity at different time scales. In addition, they highlight a putative mechanism that may\ud implement the selective amplification of appropriately timed sound tokens relative to the phase of\ud rhythmic auditory cortex activity.

The entrainment of slow rhythmic auditory cortical activity to the temporal regularities in speech is considered to be a central mechanism underlying auditory perception. Previous work has shown that entrainment is reduced when the quality of the acoustic input is degraded, but has also linked rhythmic activity at similar time scales to the encoding of temporal expectations. To understand these bottom-up and top-down contributions to rhythmic entrainment, we manipulated the temporal predictive structure of speech by parametrically altering the distribution of pauses between syllables or words, thereby rendering the local speech rate irregular while preserving intelligibility and the envelope fluctuations of the acoustic signal. Recording EEG activity in human participants, we found that this manipulation did not alter neural processes reflecting the encoding of individual sound transients, such as evoked potentials. However, the manipulation significantly reduced the fidelity of auditory delta (but not theta) band entrainment to the speech envelope. It also reduced left frontal alpha power and this alpha reduction was predictive of the reduced delta entrainment across participants. Our results show that rhythmic auditory entrainment in delta and theta bands reflect functionally distinct processes. Furthermore, they reveal that delta entrainment is under top-down control and likely reflects prefrontal processes that are sensitive to acoustical regularities rather than the bottom-up encoding of acoustic features.The entrainment of rhythmic auditory cortical activity to the speech envelope is considered to be critical for hearing. Previous work has proposed divergent views in which entrainment reflects either early evoked responses related to sound encoding or high-level processes related to expectation or cognitive selection. Using a manipulation of speech rate, we dissociated auditory entrainment at different time scales. Specifically, our results suggest that delta entrainment is controlled by frontal alpha mechanisms and thus support the notion that rhythmic auditory cortical entrainment is shaped by top-down mechanisms.

Effective interactions between conspecific individuals can depend upon the receiver forming a coherent multisensory representation of communication signals, such as merging voice and face content. Neuroimaging studies have identified face- or voice-sensitive areas (Belin et al., 2000; Petkov et al., 2008; Tsao et al., 2008), some of which have been proposed as candidate regions for face and voice integration (von Kriegstein et al., 2005). However, it was unclear how multisensory influences occur at the neuronal level within voice- or face-sensitive regions, especially compared with classically defined multisensory regions in temporal association cortex (Stein and Stanford, 2008). Here, we characterize auditory (voice) and visual (face) influences on neuronal responses in a right-hemisphere voice-sensitive region in the anterior supratemporal plane (STP) of Rhesus macaques. These results were compared with those in the neighboring superior temporal sulcus (STS). Within the STP, our results show auditory sensitivity to several vocal features, which was not evident in STS units. We also newly identify a functionally distinct neuronal subpopulation in the STP that appears to carry the area's sensitivity to voice identity related features. Audiovisual interactions were prominent in both the STP and STS. However, visual influences modulated the responses of STS neurons with greater specificity and were more often associated with congruent voice-face stimulus pairings than STP neurons. Together, the results reveal the neuronal processes subserving voice-sensitive fMRI activity patterns in primates, generate hypotheses for testing in the visual modality, and clarify the position of voice-sensitive areas within the unisensory and multisensory processing hierarchies.