Search

Summary Humans show a remarkable ability to understand continuous speech even under adverse listening conditions. This ability critically relies on dynamically updated predictions of incoming sensory information, but exactly how top-down predictions improve speech processing is still unclear. Brain oscillations are a likely mechanism for these top-down predictions [ 1, 2 ]. Quasi-rhythmic components in speech are known to entrain low-frequency oscillations in auditory areas [ 3, 4 ], and this entrainment increases with intelligibility [ 5 ]. We hypothesize that top-down signals from frontal brain areas causally modulate the phase of brain oscillations in auditory cortex. We use magnetoencephalography (MEG) to monitor brain oscillations in 22 participants during continuous speech perception. We characterize prominent spectral components of speech-brain coupling in auditory cortex and use causal connectivity analysis (transfer entropy) to identify the top-down signals driving this coupling more strongly during intelligible speech than during unintelligible speech. We report three main findings. First, frontal and motor cortices significantly modulate the phase of speech-coupled low-frequency oscillations in auditory cortex, and this effect depends on intelligibility of speech. Second, top-down signals are significantly stronger for left auditory cortex than for right auditory cortex. Third, speech-auditory cortex coupling is enhanced as a function of stronger top-down signals. Together, our results suggest that low-frequency brain oscillations play a role in implementing predictive top-down control during continuous speech perception and that top-down control is largely directed at left auditory cortex. This suggests a close relationship between (left-lateralized) speech production areas and the implementation of top-down control in continuous speech perception. [ABSTRACT FROM AUTHOR]

When we listen to speech, we have to make sense of a waveform of sound pressure. Hierarchical models of speech perception assume that, to extract semantic meaning, the signal is transformed into unknown, intermediate neuronal representations. Traditionally, studies of such intermediate representations are guided by linguistically defined concepts, such as phonemes. Here, we argue that in order to arrive at an unbiased understanding of the neuronal responses to speech, we should focus instead on representations obtained directly from the stimulus. We illustrate our view with a data-driven, information theoretic analysis of a dataset of 24 young, healthy humans who listened to a 1 h narrative while their magnetoencephalogram (MEG) was recorded. We find that two recent results, the improved performance of an encoding model in which annotated linguistic and acoustic features were combined and the decoding of phoneme subgroups from phoneme-locked responses, can be explained by an encoding model that is based entirely on acoustic features. These acoustic features capitalize on acoustic edges and outperform Gabor-filtered spectrograms, which can explicitly describe the spectrotemporal characteristics of individual phonemes. By replicating our results in publicly available electroencephalography (EEG) data, we conclude that models of brain responses based on linguistic features can serve as excellent benchmarks. However, we believe that in order to further our understanding of human cortical responses to speech, we should also explore low-level and parsimonious explanations for apparent high-level phenomena. • 24 participants listened to a 1 h story in MEG • Encoding and decoding analyses are evaluated and merged using information theory • Simple acoustic predictors of MEG responses can explain apparent high-level phenomena • Results are repeated in openly available EEG dataset Daube et al. use a data-driven information theoretic analysis of auditory cortex MEG responses to speech to demonstrate that complex models of such responses relying on annotated linguistic features can be explained more parsimoniously with simple models relying on the acoustics only. [ABSTRACT FROM AUTHOR]

Summary Over the past decade, extensive studies of the brain regions that support face, object, and scene recognition suggest that these regions have a hierarchically organized architecture that spans the occipital and temporal lobes [ 1–14 ], where visual categorizations unfold over the first 250 ms of processing [ 15–19 ]. This same architecture is flexibly involved in multiple tasks that require task-specific representations—e.g. categorizing the same object as "a car" or "a Porsche." While we partly understand where and when these categorizations happen in the occipito-ventral pathway, the next challenge is to unravel how these categorizations happen. That is, how does high-dimensional input collapse in the occipito-ventral pathway to become low dimensional representations that guide behavior? To address this, we investigated what information the brain processes in a visual perception task and visualized the dynamic representation of this information in brain activity. To do so, we developed stimulus information representation (SIR), an information theoretic framework, to tease apart stimulus information that supports behavior from that which does not. We then tracked the dynamic representations of both in magneto-encephalographic (MEG) activity. Using SIR, we demonstrate that a rapid (∼170 ms) reduction of behaviorally irrelevant information occurs in the occipital cortex and that representations of the information that supports distinct behaviors are constructed in the right fusiform gyrus (rFG). Our results thus highlight how SIR can be used to investigate the component processes of the brain by considering interactions between three variables (stimulus information, brain activity, behavior), rather than just two, as is the current norm. Highlights • We show how the brain reduces high-dimensional input into decision representations • Occipital cortex reduces features irrelevant for behavior ∼170 ms post stimulus • Past 170 ms, fusiform gyrus combines the features supporting behavioral decisions In a decision task, Zhan et al. visualize within a new information theoretic framework the dynamic representation of visual information in brain activity. They demonstrate rapid reduction of behaviorally irrelevant information in the occipital cortex and a combination of the features that supports distinct decisions in the right fusiform gyrus. [ABSTRACT FROM AUTHOR]

Human facial expressions are complex, multi-component signals that can communicate rich information about emotions, 1–5 including specific categories, such as "anger," and broader dimensions, such as "negative valence, high arousal." 6–8 An enduring question is how this complex signaling is achieved. Communication theory predicts that multi-component signals could transmit each type of emotion information—i.e., specific categories and broader dimensions—via the same or different facial signal components, with implications for elucidating the system and ontology of facial expression communication. 9 We addressed this question using a communication-systems-based method that agnostically generates facial expressions and uses the receiver's perceptions to model the specific facial signal components that represent emotion category and dimensional information to them. 10–12 First, we derived the facial expressions that elicit the perception of emotion categories (i.e., the six classic emotions 13 plus 19 complex emotions 3) and dimensions (i.e., valence and arousal) separately, in 60 individual participants. Comparison of these facial signals showed that they share subsets of components, suggesting that specific latent signals jointly represent—i.e., multiplex—categorical and dimensional information. Further examination revealed these specific latent signals and the joint information they represent. Our results—based on white Western participants, same-ethnicity face stimuli, and commonly used English emotion terms—show that facial expressions can jointly represent specific emotion categories and broad dimensions to perceivers via multiplexed facial signal components. Our results provide insights into the ontology and system of facial expression communication and a new information-theoretic framework that can characterize its complexities. [Display omitted] • Examined facial signals of broad-plus-specific emotion categories and dimensions • Used data-driven, perception-based modeling and information-theoretic analyses • Disentangled facial signals that multiplex broad-plus-specific emotion information • Provides insights into facial expression ontology and new methodological framework Liu et al. examine how facial expressions signal broad-plus-specific emotion category and dimensional information. Using a perception-based facial-signal-modeling technique and information-theoretic analyses, they find a latent set of facial signals that can multiplex categorical and dimensional information and a subset uniquely signaling either. [ABSTRACT FROM AUTHOR]

Communicating emotional intensity plays a vital ecological role because it provides valuable information about the nature and likelihood of the sender's behavior. 1,2,3 For example, attack often follows signals of intense aggression if receivers fail to retreat. 4,5 Humans regularly use facial expressions to communicate such information. 6,7,8,9,10,11 Yet how this complex signaling task is achieved remains unknown. We addressed this question using a perception-based, data-driven method to mathematically model the specific facial movements that receivers use to classify the six basic emotions—"happy," "surprise," "fear," "disgust," "anger," and "sad"—and judge their intensity in two distinct cultures (East Asian, Western European; total n = 120). In both cultures, receivers expected facial expressions to dynamically represent emotion category and intensity information over time, using a multi-component compositional signaling structure. Specifically, emotion intensifiers peaked earlier or later than emotion classifiers and represented intensity using amplitude variations. Emotion intensifiers are also more similar across emotions than classifiers are, suggesting a latent broad-plus-specific signaling structure. Cross-cultural analysis further revealed similarities and differences in expectations that could impact cross-cultural communication. Specifically, East Asian and Western European receivers have similar expectations about which facial movements represent high intensity for threat-related emotions, such as "anger," "disgust," and "fear," but differ on those that represent low threat emotions, such as happiness and sadness. Together, our results provide new insights into the intricate processes by which facial expressions can achieve complex dynamic signaling tasks by revealing the rich information embedded in facial expressions. [Display omitted] • Emotion categories and intensity are represented by specific facial movements • Emotion classifier facial movements are highly distinct; intensifiers are not • Emotion classifiers and intensifiers have distinct temporal signatures • Cultural variance in facial signals may impact cross-cultural communication Chen et al. examine how facial expressions dynamically represent emotion category and intensity information. Using a perception-based data-driven method and information-theoretic analyses, they reveal, in two distinct cultures, that facial expressions represent emotion and intensity information using a specific compositional dynamic structure. [ABSTRACT FROM AUTHOR]

Prediction-for-perception theories suggest that the brain predicts incoming stimuli to facilitate their categorization. 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17 However, it remains unknown what the information contents of these predictions are, which hinders mechanistic explanations. This is because typical approaches cast predictions as an underconstrained contrast between two categories 18,19,20,21,22,23,24 —e.g., faces versus cars, which could lead to predictions of features specific to faces or cars, or features from both categories. Here, to pinpoint the information contents of predictions and thus their mechanistic processing in the brain, we identified the features that enable two different categorical perceptions of the same stimuli. We then trained multivariate classifiers to discern, from dynamic MEG brain responses, the features tied to each perception. With an auditory cueing design, we reveal where, when, and how the brain reactivates visual category features (versus the typical category contrast) before the stimulus is shown. We demonstrate that the predictions of category features have a more direct influence (bias) on subsequent decision behavior in participants than the typical category contrast. Specifically, these predictions are more precisely localized in the brain (lateralized), are more specifically driven by the auditory cues, and their reactivation strength before a stimulus presentation exerts a greater bias on how the individual participant later categorizes this stimulus. By characterizing the specific information contents that the brain predicts and then processes, our findings provide new insights into the brain's mechanisms of prediction for perception. • Investigated the visual content that the brain predicts for categorization • Showed that predicted content flows down from the ventral to the occipital cortex • Showed that content lateralizes in the occipital cortex before stimulus is shown • Demonstrated that per-trial strength of predicted content biases subsequent behavior Yan et al. reveal the visual contents that the brain predicts before the stimulus is shown. They show that these contents flow down from ventral to occipital cortex, where they are lateralized. Critically, strength of predicted content in the brain biases subsequent perceptual categorization behavior when the stimulus is shown. [ABSTRACT FROM AUTHOR]

Facial attractiveness confers considerable advantages in social interactions, 1,2 with preferences likely reflecting psychobiological mechanisms shaped by natural selection. Theories of universal beauty propose that attractive faces comprise features that are closer to the population average 3 while optimizing sexual dimorphism. 4 However, emerging evidence questions this model as an accurate representation of facial attractiveness, 5–7 including representing the diversity of beauty preferences within and across cultures. 8–12 Here, we demonstrate that Western Europeans (WEs) and East Asians (EAs) evaluate facial beauty using culture-specific features, contradicting theories of universality. With a data-driven method, we modeled, at both the individual and group levels, the attractive face features of young females (25 years old) in two matched groups each of 40 young male WE and EA participants. Specifically, we generated a broad range of same- and other-ethnicity female faces with naturally varying shapes and complexions. Participants rated each on attractiveness. We then reverse correlated the face features that drive perception of attractiveness in each participant. From these individual face models, we reconstructed a facial attractiveness representation space that explains preference variations. We show that facial attractiveness is distinct both from averageness and from sexual dimorphism in both cultures. Finally, we disentangled attractive face features into those shared across cultures, culture specific, and specific to individual participants, thereby revealing their diversity. Our results have direct theoretical and methodological impact for representing diversity in social perception and for the design of culturally and ethnically sensitive socially interactive digital agents. • We modeled individual preferences for attractive faces in two cultures • Attractive face features differ from the face average and sexual dimorphism • Instead, culture and individual preferences shape attractive face features • Attractive face features from a culture are used to judge other-ethnicity faces Zhan et al. refute theories of universal beauty, showing that Western and Eastern cultures and individual preferences shape attractive face features. Individual preference models show that attractive features differ from the average and sexual dimorphism to form a space that cultural members use to perceive face attractiveness in other ethnicities. [ABSTRACT FROM AUTHOR]