Search

Many animal vocalizations contain nonlinear acoustic phenomena as a consequence of physiological arousal. In humans, nonlinear features are processed early in the auditory system, and are used to efficiently detect alarm calls and other urgent signals. Yet, high-level emotional and semantic contextual factors likely guide the perception and evaluation of roughness features in vocal sounds. Here we examined the relationship between perceived vocal arousal and auditory context. We presented listeners with nonverbal vocalizations (yells of a single vowel) at varying levels of portrayed vocal arousal, in two musical contexts (clean guitar, distorted guitar) and one non-musical context (modulated noise). As predicted, vocalizations with higher levels of portrayed vocal arousal were judged as more negative and more emotionally aroused than the same voices produced with low vocal arousal. Moreover, both the perceived valence and emotional arousal of vocalizations were significantly affected by both musical and non-musical contexts. These results show the importance of auditory context in judging emotional arousal and valence in voices and music, and suggest that nonlinear features in music are processed similarly to communicative vocal signals.

Social interaction research is lacking an experimental paradigm enabling researchers to make causal inferences in free social interactions. For instance, the expressive signals that causally modulate the emergence of romantic attraction during interactions remain unknown. To disentangle causality in the wealth of covarying factors that govern social interactions, we developed an open-source video-conference platform enabling researchers to covertly manipulate the social signals produced by participants during interactions. Using this platform, we performed a speed-dating experiment where we aligned or misaligned the facial smiles of participants in real time with face transformation algorithms. Even though participants remained totally unaware that their faces were being manipulated, aligning their smiles causally enhanced the romantic attraction they felt toward each other, compared to unaligned scenarios. Manipulations also influenced how participants synchronized and vocally reacted to each other. This paradigm causally manipulates the emergence of romantic attraction in free social interactions. Moreover, our methodology opens the possibility to perform causal inferences during free social interactions., Competing Interests: Competing interests statement:The authors declare no competing interest.

The interplay between the different components of emotional contagion (i.e. emotional state and facial motor resonance), both during implicit and explicit appraisal of emotion, remains controversial. The aims of this study were (i) to distinguish between these components thanks to vocal smile processing and (ii) to assess how they reflect implicit processes and/or an explicit appraisal loop. Emotional contagion to subtle vocal emotions was studied in 25 adults through motor resonance and Autonomic Nervous System (ANS) reactivity. Facial expressions (fEMG: facial electromyography) and pupil dilation were assessed during the processing and judgement of artificially emotionally modified sentences. fEMG revealed that Zygomaticus major was reactive to the perceived valence of sounds, whereas the activity of Corrugator supercilii reflected explicit judgement. Timing analysis of pupil dilation provided further insight into both the emotional state and the implicit and explicit processing of vocal emotion, showing earlier activity for emotional stimuli than for neutral stimuli, followed by valence-dependent variations and a late judgement-dependent increase in pupil diameter. This innovative combination of different electrophysiological measures shed new light on the debate between central and peripherical views within the framework of emotional contagion., (© 2024. The Author(s).)

The human voice is a potent social signal and a distinctive marker of individual identity. As individuals go through puberty, their voices undergo acoustic changes, setting them apart from others. In this article, we propose that hormonal fluctuations in conjunction with morphological vocal tract changes during puberty establish a sensitive developmental phase that affects the monitoring of the adolescent voice and, specifically, self-other distinction. Furthermore, the protracted maturation of brain regions responsible for voice processing, coupled with the dynamically evolving social environment of adolescents, likely disrupts a clear differentiation of the self-voice from others' voices. This socioneuroendocrine framework offers a holistic understanding of voice monitoring during adolescence., Competing Interests: Declaration of interests The authors declare no competing interests., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

A wealth of behavioral evidence indicate that sounds with increasing intensity (i.e. appear to be looming towards the listener) are processed with increased attentional and physiological resources compared to receding sounds. However, the neurophysiological mechanism responsible for such cognitive amplification remains elusive. Here, we show that the large differences seen between cortical responses to looming and receding sounds are in fact almost entirely explained away by nonlinear encoding at the level of the auditory periphery. We collected electroencephalography (EEG) data during an oddball paradigm to elicit mismatch negativity (MMN) and others Event Related Potentials (EPRs), in response to deviant stimuli with both dynamic (looming and receding) and constant level (flat) differences to the standard in the same participants. We then combined a computational model of the auditory periphery with generative EEG methods (temporal response functions, TRFs) to model the single-participant ERPs responses to flat deviants, and used them to predict the effect of the same mechanism on looming and receding stimuli. The flat model explained 45% variance of the looming response, and 33% of the receding response. This provide striking evidence that difference wave responses to looming and receding sounds result from the same cortical mechanism that generate responses to constant-level deviants: all such differences are the sole consequence of their particular physical morphology getting amplified and integrated by peripheral auditory mechanisms. Thus, not all effects seen cortically proceed from top-down modulations by high-level decision variables, but can rather be performed early and efficiently by feed-forward peripheral mechanisms that evolved precisely to sparing subsequent networks with the necessity to implement such mechanisms., (Copyright © 2024 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

After a right hemisphere stroke, more than half of the patients are impaired in their capacity to produce or comprehend speech prosody. Yet, and despite its social-cognitive consequences for patients, aprosodia following stroke has received scant attention. In this report, we introduce a novel, simple psychophysical procedure which, by combining systematic digital manipulations of speech stimuli and reverse-correlation analysis, allows estimating the internal sensory representations that subtend how individual patients perceive speech prosody, and the level of internal noise that govern behavioral variability in how patients apply these representations. Tested on a sample of N = 22 right-hemisphere stroke survivors and N = 21 age-matched controls, the representation + noise model provides a promising alternative to the clinical gold standard for evaluating aprosodia (MEC): both parameters strongly associate with receptive, and not expressive, aprosodia measured by MEC within the patient group; they have better sensitivity than MEC for separating high-functioning patients from controls; and have good specificity with respect to non-prosody-related impairments of auditory attention and processing. Taken together, individual differences in either internal representation, internal noise, or both, paint a potent portrait of the variety of sensory/cognitive mechanisms that can explain impairments of prosody processing after stroke., (© 2024. The Author(s).)

Recent deep-learning techniques have made it possible to manipulate facial expressions in digital photographs or videos, however, these techniques still lack fine and personalized ways to control their creation. Moreover, current technologies are highly dependent on large labeled databases, which limits the range and complexity of expressions that can be modeled. Thus, these technologies cannot deal with non-basic emotions. In this paper, we propose a novel interdisciplinary approach combining the Generative Adversarial Network (GAN) with a technique inspired by cognitive sciences, psychophysical reverse correlation. Reverse correlation is a data-driven method able to extract an observer's 'mental representation' of what a given facial expression should look like. Our approach can generate 1) personalized facial expression prototypes, 2) of basic emotions, and non-basic emotions that are not available in existing databases, and 3) without the need for expertise. Personalized prototypes obtained with reverse correlation can then be applied to manipulate facial expressions. In addition, our system challenges the universality of facial expression prototypes by proposing the concepts of dominant and complementary action units to describe facial expression prototypes. The evaluations we conducted on a limited number of emotions validate the effectiveness of our proposed method. The code is available at https://github.com/yansen0508/Mental-Deep-Reverse-Engineering., Competing Interests: The authors have declared that no competing interests exist., (Copyright: © 2023 Yan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

People have a well-described advantage in identifying individuals and emotions in their own culture, a phenomenon also known as the other-race and language-familiarity effect. However, it is unclear whether native-language advantages arise from genuinely enhanced capacities to extract relevant cues in familiar speech or, more simply, from cultural differences in emotional expressions. Here, to rule out production differences, we use algorithmic voice transformations to create French and Japanese stimulus pairs that differed by exactly the same acoustical characteristics. In two cross-cultural experiments, participants performed better in their native language when categorizing vocal emotional cues and detecting non-emotional pitch changes. This advantage persisted over three types of stimulus degradation (jabberwocky, shuffled and reversed sentences), which disturbed semantics, syntax, and supra-segmental patterns, respectively. These results provide evidence that production differences are not the sole drivers of the language-familiarity effect in cross-cultural emotion perception. Listeners' unfamiliarity with the phonology of another language, rather than with its syntax or semantics, impairs the detection of pitch prosodic cues and, in turn, the recognition of expressive prosody., Competing Interests: The authors have declared that no competing interests exist., (Copyright: © 2023 Nakai et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

Emotional speech perception is a multisensory process. When speaking with an individual we concurrently integrate the information from their voice and face to decode e.g., their feelings, moods, and emotions. However, the physiological reactions-such as the reflexive dilation of the pupil-associated to these processes remain mostly unknown. That is the aim of the current article, to investigate whether pupillary reactions can index the processes underlying the audiovisual integration of emotional signals. To investigate this question, we used an algorithm able to increase or decrease the smiles seen in a person's face or heard in their voice, while preserving the temporal synchrony between visual and auditory channels. Using this algorithm, we created congruent and incongruent audiovisual smiles, and investigated participants' gaze and pupillary reactions to manipulated stimuli. We found that pupil reactions can reflect emotional information mismatch in audiovisual speech. In our data, when participants were explicitly asked to extract emotional information from stimuli, the first fixation within emotionally mismatching areas (i.e., the mouth) triggered pupil dilation. These results reveal that pupil dilation can reflect the dynamic integration of audiovisual emotional speech and provide insights on how these reactions are triggered during stimulus perception., (© 2023. The Author(s).)

Objective: The acoustic characteristics of stimuli influence the characteristics of the corresponding evoked potentials in healthy subjects. Own-name stimuli are used in clinical practice to assess the level of consciousness in intensive care units. The influence of the acoustic variability of these stimuli has never been evaluated. Here, we explored the influence of this variability on the characteristics of the subject's own name (SON) P300., Methods: We retrospectively analyzed 251 disorders of consciousness patients from Lyon and Paris Hospitals who underwent an "own-name protocol". A reverse correlation analysis was performed to test for an association between acoustic properties of own-names stimuli used and the characteristics of the P300 wave observed., Results: Own-names pronounced with increasing pitch prosody showed P300 responses 66 ms earlier than own-names that had a decreasing prosody [IC 95% = 6.36; 125.9 ms]., Conclusions: Speech prosody of the stimuli in the "own name protocol" is associated with latencies differences of the P300 response among patients for whom these responses were observed. Further investigations are needed to confirm these results., Significance: Speech prosody of the stimuli in the "own name protocol" is a non-negligible parameter, associated with P300 latency differences. Speech prosody should be standardized in SON P300 studies., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 International Federation of Clinical Neurophysiology. Published by Elsevier B.V. All rights reserved.)

Rapid technological advances in artificial intelligence are creating opportunities for real-time algorithmic modulations of a person's facial and vocal expressions, or 'deep-fakes'. These developments raise unprecedented societal and ethical questions which, despite much recent public awareness, are still poorly understood from the point of view of moral psychology. We report here on an experimental ethics study conducted on a sample of N = 303 participants (predominantly young, western and educated), who evaluated the acceptability of vignettes describing potential applications of expressive voice transformation technology. We found that vocal deep-fakes were generally well accepted in the population, notably in a therapeutic context and for emotions judged otherwise difficult to control, and surprisingly, even if the user lies to their interlocutors about using them. Unlike other emerging technologies like autonomous vehicles, there was no evidence of social dilemma in which one would, for example, accept for others what they resent for themselves. The only real obstacle to the massive deployment of vocal deep-fakes appears to be situations where they are applied to a speaker without their knowing, but even the acceptability of such situations was modulated by individual differences in moral values and attitude towards science fiction. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part II)'.

A wealth of theoretical and empirical arguments have suggested that music triggers emotional responses by resembling the inflections of expressive vocalizations, but have done so using low-level acoustic parameters (pitch, loudness, speed) that, in fact, may not be processed by the listener in reference to human voice. Here, we take the opportunity of the recent availability of computational models that allow the simulation of three specifically vocal emotional behaviours: smiling, vocal tremor and vocal roughness. When applied to musical material, we find that these three acoustic manipulations trigger emotional perceptions that are remarkably similar to those observed on speech and scream sounds, and identical across musician and non-musician listeners. Strikingly, this not only applied to singing voice with and without musical background, but also to purely instrumental material. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part I)'.

Imitation is one of the core building blocks of human social cognition, supporting capacities as diverse as empathy, social learning, and knowledge acquisition 1 . Newborns' ability to match others' motor acts, while quite limited initially, drastically improves during the first months of development 2 . Of notable importance to human sociality is our tendency to rapidly mimic facial expressions of emotion. Facial mimicry develops around six months of age 3 , but because of its late emergence, the factors supporting its development are relatively unknown. One possibility is that the development of facial mimicry depends on seeing emotional imitative behavior in others 4 . Alternatively, the drive to imitate facial expressions of emotion may be independent of visual learning and be supported by modality-general processes. Here we report evidence for the latter, by showing that congenitally blind participants facially imitate smiles heard in speech, despite having never seen a facial expression., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Whether speech prosody truly and naturally reflects a speaker's subjective confidence, rather than other dimensions such as objective accuracy, is unclear. Here, using a new approach combing psychophysics with acoustic analysis and automatic classification of verbal reports, we tease apart the contributions of sensory evidence, accuracy, and subjective confidence to speech prosody. We find that subjective confidence and objective accuracy are distinctly reflected in the loudness, duration and intonation of verbal reports. Strikingly, we show that a speaker's accuracy is encoded in speech prosody beyond their own metacognitive awareness, and that it can be automatically decoded from this information alone with performances up to 60%. These findings demonstrate that confidence and accuracy have separable prosodic signatures that are manifested with different timings, and on different acoustic dimensions. Thus, both subjective mental states of confidence, and objective states related to competence, can be directly inferred from this natural behavior., (Copyright © 2021 Elsevier B.V. All rights reserved.)

The success of human cooperation crucially depends on mechanisms enabling individuals to detect unreliability in their conspecifics. Yet, how such epistemic vigilance is achieved from naturalistic sensory inputs remains unclear. Here we show that listeners' perceptions of the certainty and honesty of other speakers from their speech are based on a common prosodic signature. Using a data-driven method, we separately decode the prosodic features driving listeners' perceptions of a speaker's certainty and honesty across pitch, duration and loudness. We find that these two kinds of judgments rely on a common prosodic signature that is perceived independently from individuals' conceptual knowledge and native language. Finally, we show that listeners extract this prosodic signature automatically, and that this impacts the way they memorize spoken words. These findings shed light on a unique auditory adaptation that enables human listeners to quickly detect and react to unreliability during linguistic interactions.

Emotions are often accompanied by vocalizations whose acoustic features provide information about the physiological state of the speaker. Here, we ask if perceiving these affective signals in one's own voice has an impact on one's own emotional state, and if it is necessary to identify these signals as self-originated for the emotional effect to occur. Participants had to deliberate out loud about how they would feel in various familiar emotional scenarios, while we covertly manipulated their voices in order to make them sound happy or sad. Perceiving the artificial affective signals in their own voice altered participants' judgements about how they would feel in these situations. Crucially, this effect disappeared when participants detected the vocal manipulation, either explicitly or implicitly. The original valence of the scenarios also modulated the vocal feedback effect. These results highlight the role of the exteroception of self-attributed affective signals in the emergence of emotional feelings., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Human interactions are often improvised rather than scripted, which suggests that efficient coordination can emerge even when collective plans are largely underspecified. One possibility is that such forms of coordination primarily rely on mutual influences between interactive partners, and on perception-action couplings such as entrainment or mimicry. Yet some forms of improvised joint actions appear difficult to explain solely by appealing to these emergent mechanisms. Here, we focus on collective free improvisation, a form of highly unplanned creative practice where both agents' subjective reports and the complexity of their interactions suggest that shared intentions may sometimes emerge to support coordination during the course of the improvisation, even in the absence of verbal communication. In four experiments, we show that shared intentions spontaneously emerge during collective musical improvisations, and that they foster coordination on multiple levels, over and beyond the mere influence of shared information. We also show that musicians deploy communicative strategies to manifest and propagate their intentions within the group, and that this predicts better coordination. Overall, our results suggest that improvised and scripted joint actions are more continuous with one another than it first seems, and that they differ merely in the extent to which they rely on emergent or planned coordination mechanisms., (© 2021 Cognitive Science Society, Inc.)

Many animal vocalizations contain nonlinear acoustic phenomena as a consequence of physiological arousal. In humans, nonlinear features are processed early in the auditory system, and are used to efficiently detect alarm calls and other urgent signals. Yet, high-level emotional and semantic contextual factors likely guide the perception and evaluation of roughness features in vocal sounds. Here we examined the relationship between perceived vocal arousal and auditory context. We presented listeners with nonverbal vocalizations (yells of a single vowel) at varying levels of portrayed vocal arousal, in two musical contexts (clean guitar, distorted guitar) and one non-musical context (modulated noise). As predicted, vocalizations with higher levels of portrayed vocal arousal were judged as more negative and more emotionally aroused than the same voices produced with low vocal arousal. Moreover, both the perceived valence and emotional arousal of vocalizations were significantly affected by both musical and non-musical contexts. These results show the importance of auditory context in judging emotional arousal and valence in voices and music, and suggest that nonlinear features in music are processed similarly to communicative vocal signals., (Copyright © 2020 Elsevier B.V. All rights reserved.)

During auditory perception, neural oscillations are known to entrain to acoustic dynamics but their role in the processing of auditory information remains unclear. As a complex temporal structure that can be parameterized acoustically, music is particularly suited to address this issue. In a combined behavioral and EEG experiment in human participants, we investigated the relative contribution of temporal (acoustic dynamics) and nontemporal (melodic spectral complexity) dimensions of stimulation on neural entrainment, a stimulus-brain coupling phenomenon operationally defined here as the temporal coherence between acoustical and neural dynamics. We first highlight that low-frequency neural oscillations robustly entrain to complex acoustic temporal modulations, which underscores the fine-grained nature of this coupling mechanism. We also reveal that enhancing melodic spectral complexity, in terms of pitch, harmony, and pitch variation, increases neural entrainment. Importantly, this manipulation enhances activity in the theta (5 Hz) range, a frequency-selective effect independent of the note rate of the melodies, which may reflect internal temporal constraints of the neural processes involved. Moreover, while both emotional arousal ratings and neural entrainment were positively modulated by spectral complexity, no direct relationship between arousal and neural entrainment was observed. Overall, these results indicate that neural entrainment to music is sensitive to the spectral content of auditory information and indexes an auditory level of processing that should be distinguished from higher-order emotional processing stages. NEW & NOTEWORTHY Low-frequency (<10 Hz) cortical neural oscillations are known to entrain to acoustic dynamics, the so-called neural entrainment phenomenon, but their functional implication in the processing of auditory information remains unclear. In a behavioral and EEG experiment capitalizing on parameterized musical textures, we disentangle the contribution of stimulus dynamics, melodic spectral complexity, and emotional judgments on neural entrainment and highlight their respective spatial and spectral neural signature.

In social interactions, people have to pay attention both to the 'what' and 'who'. In particular, expressive changes heard on speech signals have to be integrated with speaker identity, differentiating e.g. self- and other-produced signals. While previous research has shown that self-related visual information processing is facilitated compared to non-self stimuli, evidence in the auditory modality remains mixed. Here, we compared electroencephalography (EEG) responses to expressive changes in sequence of self- or other-produced speech sounds using a mismatch negativity (MMN) passive oddball paradigm. Critically, to control for speaker differences, we used programmable acoustic transformations to create voice deviants that differed from standards in exactly the same manner, making EEG responses to such deviations comparable between sequences. Our results indicate that expressive changes on a stranger's voice are highly prioritized in auditory processing compared to identical changes on the self-voice. Other-voice deviants generate earlier MMN onset responses and involve stronger cortical activations in a left motor and somatosensory network suggestive of an increased recruitment of resources for less internally predictable, and therefore perhaps more socially relevant, signals., (© The Author(s) 2019. Published by Oxford University Press.)

Over the past few years, the field of visual social cognition and face processing has been dramatically impacted by a series of data-driven studies employing computer-graphics tools to synthesize arbitrary meaningful facial expressions. In the auditory modality, reverse correlation is traditionally used to characterize sensory processing at the level of spectral or spectro-temporal stimulus properties, but not higher-level cognitive processing of e.g. words, sentences or music, by lack of tools able to manipulate the stimulus dimensions that are relevant for these processes. Here, we present an open-source audio-transformation toolbox, called CLEESE, able to systematically randomize the prosody/melody of existing speech and music recordings. CLEESE works by cutting recordings in small successive time segments (e.g. every successive 100 milliseconds in a spoken utterance), and applying a random parametric transformation of each segment's pitch, duration or amplitude, using a new Python-language implementation of the phase-vocoder digital audio technique. We present here two applications of the tool to generate stimuli for studying intonation processing of interrogative vs declarative speech, and rhythm processing of sung melodies., Competing Interests: Author J.J.B., affiliated as Independent Researcher, is registered as a sole-proprietorship company in France dedicated to research and development consulting. This does not alter our adherence to PLOS ONE policies on sharing data and materials. Author J.J.B. declares that there is no competing interest, financial or otherwise, related to the publication of this study.

Competing Interests: The authors declare no conflict of interest.

Human listeners excel at forming high-level social representations about each other, even from the briefest of utterances. In particular, pitch is widely recognized as the auditory dimension that conveys most of the information about a speaker's traits, emotional states, and attitudes. While past research has primarily looked at the influence of mean pitch, almost nothing is known about how intonation patterns, i.e., finely tuned pitch trajectories around the mean, may determine social judgments in speech. Here, we introduce an experimental paradigm that combines state-of-the-art voice transformation algorithms with psychophysical reverse correlation and show that two of the most important dimensions of social judgments, a speaker's perceived dominance and trustworthiness, are driven by robust and distinguishing pitch trajectories in short utterances like the word "Hello," which remained remarkably stable whether male or female listeners judged male or female speakers. These findings reveal a unique communicative adaptation that enables listeners to infer social traits regardless of speakers' physical characteristics, such as sex and mean pitch. By characterizing how any given individual's mental representations may differ from this generic code, the method introduced here opens avenues to explore dysprosody and social-cognitive deficits in disorders like autism spectrum and schizophrenia. In addition, once derived experimentally, these prototypes can be applied to novel utterances, thus providing a principled way to modulate personality impressions in arbitrary speech signals., Competing Interests: The authors declare no conflict of interest., (Copyright © 2018 the Author(s). Published by PNAS.)

We present an open-source software platform that transforms emotional cues expressed by speech signals using audio effects like pitch shifting, inflection, vibrato, and filtering. The emotional transformations can be applied to any audio file, but can also run in real time, using live input from a microphone, with less than 20-ms latency. We anticipate that this tool will be useful for the study of emotions in psychology and neuroscience, because it enables a high level of control over the acoustical and emotional content of experimental stimuli in a variety of laboratory situations, including real-time social situations. We present here results of a series of validation experiments aiming to position the tool against several methodological requirements: that transformed emotions be recognized at above-chance levels, valid in several languages (French, English, Swedish, and Japanese) and with a naturalness comparable to natural speech.

Which spectral cues underlie the perceptual processing of smiles in speech? Here, the question was addressed using reverse-correlation in the case of the isolated vowel [a]. Listeners were presented with hundreds of pairs of utterances with randomly manipulated spectral characteristics and were asked to indicate, in each pair, which was the most smiling. The analyses revealed that they relied on robust spectral representations that specifically encoded vowel's formants. These findings demonstrate the causal role played by formants in the perception of smile. Overall, this paper suggests a general method to estimate the spectral bases of high-level (e.g., emotional/social/paralinguistic) speech representations.

A recently emerging view in music cognition holds that music is not only social and participatory in its production, but also in its perception, i.e. that music is in fact perceived as the sonic trace of social relations between a group of real or virtual agents. While this view appears compatible with a number of intriguing music cognitive phenomena, such as the links between beat entrainment and prosocial behaviour or between strong musical emotions and empathy, direct evidence is lacking that listeners are at all able to use the acoustic features of a musical interaction to infer the affiliatory or controlling nature of an underlying social intention. We created a novel experimental situation in which we asked expert music improvisers to communicate 5 types of non-musical social intentions, such as being domineering, disdainful or conciliatory, to one another solely using musical interaction. Using a combination of decoding studies, computational and psychoacoustical analyses, we show that both musically-trained and non musically-trained listeners can recognize relational intentions encoded in music, and that this social cognitive ability relies, to a sizeable extent, on the information processing of acoustic cues of temporal and harmonic coordination that are not present in any one of the musicians' channels, but emerge from the dynamics of their interaction. By manipulating these cues in two-channel audio recordings and testing their impact on the social judgements of non-musician observers, we finally establish a causal relationship between the affiliation dimension of social behaviour and musical harmonic coordination on the one hand, and between the control dimension and musical temporal coordination on the other hand. These results provide novel mechanistic insights not only into the social cognition of musical interactions, but also into that of non-verbal interactions as a whole., (Copyright © 2017. Published by Elsevier B.V.)

In humans as well as other animals, displays of body strength such as power postures or deep masculine voices are associated with prevalence in conflicts of interest and facilitated access to resources. We conduct here an ecological and highly critical test of this hypothesis in a domain that, on first thought, would appear to be shielded from such influences: access to emergency medical care. Using acoustic manipulations of vocal masculinity, we systematically varied the perceived level of physical dominance of mock patients calling a medical call center simulator. Callers whose voice were perceived as indicative of physical dominance (i.e. those with low fundamental and formant frequency voices) obtained a higher grade of response, a higher evaluation of medical emergency and longer attention from physicians than callers with strictly identical medical needs whose voice signaled lower physical dominance. Strikingly, while the effect was important for physician participants, it was virtually non-existent when calls were processed by non-medically-trained phone operators. This finding demonstrates an unprecedented degree of vulnerability of telephone-based medical decisions to extra-medical factors carried by vocal cues, and shows that it may not simply be assumed that more medical training will shield decisions from such influences.

Research has shown that people often exert control over their emotions. By modulating expressions, reappraising feelings, and redirecting attention, they can regulate their emotional experience. These findings have contributed to a blurring of the traditional boundaries between cognitive and emotional processes, and it has been suggested that emotional signals are produced in a goal-directed way and monitored for errors like other intentional actions. However, this interesting possibility has never been experimentally tested. To this end, we created a digital audio platform to covertly modify the emotional tone of participants' voices while they talked in the direction of happiness, sadness, or fear. The result showed that the audio transformations were being perceived as natural examples of the intended emotions, but the great majority of the participants, nevertheless, remained unaware that their own voices were being manipulated. This finding indicates that people are not continuously monitoring their own voice to make sure that it meets a predetermined emotional target. Instead, as a consequence of listening to their altered voices, the emotional state of the participants changed in congruence with the emotion portrayed, which was measured by both self-report and skin conductance level. This change is the first evidence, to our knowledge, of peripheral feedback effects on emotional experience in the auditory domain. As such, our result reinforces the wider framework of self-perception theory: that we often use the same inferential strategies to understand ourselves as those that we use to understand others.

The "bag-of-frames" (BOF) approach, which encodes audio signals as the long-term statistical distribution of short-term spectral features, is commonly regarded as an effective and sufficient way to represent environmental sound recordings (soundscapes). The present paper describes a conceptual replication of a use of the BOF approach in a seminal article using several other soundscape datasets, with results strongly questioning the adequacy of the BOF approach for the task. As demonstrated in this paper, the good accuracy originally reported with BOF likely resulted from a particularly permissive dataset with low within-class variability. Soundscape modeling, therefore, may not be the closed case it was once thought to be.

The mammalian auditory system extracts features from the acoustic environment based on the responses of spatially distributed sets of neurons in the subcortical and cortical auditory structures. The characteristic responses of these neurons (linearly approximated by their spectro-temporal receptive fields, or STRFs) suggest that auditory representations are formed, as early as in the inferior colliculi, on the basis of a time, frequency, rate (temporal modulations) and scale (spectral modulations) analysis of sound. However, how these four dimensions are integrated and processed in subsequent neural networks remains unclear. In this work, we present a new methodology to generate computational insights into the functional organization of such processes. We first propose a systematic framework to explore more than a hundred different computational strategies proposed in the literature to process the output of a generic STRF model. We then evaluate these strategies on their ability to compute perceptual distances between pairs of environmental sounds. Finally, we conduct a meta-analysis of the dataset of all these algorithms' accuracies to examine whether certain combinations of dimensions and certain ways to treat such dimensions are, on the whole, more computationally effective than others. We present an application of this methodology to a dataset of ten environmental sound categories, in which the analysis reveals that (1) models are most effective when they organize STRF data into frequency groupings-which is consistent with the known tonotopic organization of receptive fields in auditory structures -, and that (2) models that treat STRF data as time series are no more effective than models that rely only on summary statistics along time-which corroborates recent experimental evidence on texture discrimination by summary statistics.

While background music is often used during osteopathic treatment, it remains unclear whether it facilitates treatment, and, if it does, whether it is listening to music or jointly listening to a common stimulus that is most important. We created three experimental situations for a standard osteopathic procedure in which patients and practitioner listened either to silence, to the same music in synchrony, or (unknowingly) to different desynchronized montages of the same material. Music had no effect on heart rate and arterial pressure pre- and posttreatment compared to silence, but EEG measures revealed a clear effect of synchronized versus desynchronized listening: listening to desynchronized music was associated with larger amounts of mu-rhythm event-related desynchronization (ERD), indicating decreased sensorimotor fluency compared to what was gained in the synchronized music listening condition. This result suggests that, if any effect can be attributed to music for osteopathy, it is related to its capacity to modulate empathy between patient and therapist and, further, that music does not systematically create better conditions for empathy than silence., (Copyright © 2013 Society for Psychophysiological Research.)

Listening to music engages the whole brain, thus stimulating cognitive performance in a range of non-purely musical activities such as language and memory tasks. This article addresses an ongoing debate on the link between music and memory for words. While evidence on healthy and clinical populations suggests that music listening can improve verbal memory in a variety of situations, it is still unclear what specific memory process is affected and how. This study was designed to explore the hypothesis that music specifically benefits the encoding part of verbal memory tasks, by providing a richer context for encoding and therefore less demand on the dorsolateral prefrontal cortex (DLPFC). Twenty-two healthy young adults were subjected to functional near-infrared spectroscopy (fNIRS) imaging of their bilateral DLPFC while encoding words in the presence of either a music or a silent background. Behavioral data confirmed the facilitating effect of music background during encoding on subsequent item recognition. fNIRS results revealed significantly greater activation of the left hemisphere during encoding (in line with the HERA model of memory lateralization) and a sustained, bilateral decrease of activity in the DLPFC in the music condition compared to silence. These findings suggest that music modulates the role played by the DLPFC during verbal encoding, and open perspectives for applications to clinical populations with prefrontal impairments, such as elderly adults or Alzheimer's patients.

The paper describes an application of machine learning techniques to identify expiratory and inspiration phases from the audio recording of human baby cries. Crying episodes were recorded from 14 infants, spanning four vocalization contexts in their first 12 months of age; recordings from three individuals were annotated manually to identify expiratory and inspiratory sounds and used as training examples to segment automatically the recordings of the other 11 individuals. The proposed algorithm uses a hidden Markov model architecture, in which state likelihoods are estimated either with Gaussian mixture models or by converting the classification decisions of a support vector machine. The algorithm yields up to 95% classification precision (86% average), and its ability generalizes over different babies, different ages, and vocalization contexts. The technique offers an opportunity to quantify expiration duration, count the crying rate, and other time-related characteristics of baby crying for screening, diagnosis, and research purposes over large populations of infants.

This study uses an audio signal transformation, splicing, to create an experimental situation where human listeners judge the similarity of audio signals, which they cannot easily categorize. Splicing works by segmenting audio signals into 50-ms frames, then shuffling and concatenating these frames back in random order. Splicing a signal masks the identification of the categories that it normally elicits: For instance, human participants cannot easily identify the sound of cars in a spliced recording of a city street. This study compares human performance on both normal and spliced recordings of soundscapes and music. Splicing is found to degrade human similarity performance significantly less for soundscapes than for music: When two spliced soundscapes are judged similar to one another, the original recordings also tend to sound similar. This establishes that humans are capable of reconstructing consistent similarity relations between soundscapes without relying much on the identification of the natural categories associated with such signals, such as their constituent sound sources. This finding contradicts previous literature and points to new ways to conceptualize the different ways in which humans perceive soundscapes and music.

The "bag-of-frames" approach (BOF) to audio pattern recognition represents signals as the long-term statistical distribution of their local spectral features. This approach has proved nearly optimal for simulating the auditory perception of natural and human environments (or soundscapes), and is also the most predominent paradigm to extract high-level descriptions from music signals. However, recent studies show that, contrary to its application to soundscape signals, BOF only provides limited performance when applied to polyphonic music signals. This paper proposes to explicitly examine the difference between urban soundscapes and polyphonic music with respect to their modeling with the BOF approach. First, the application of the same measure of acoustic similarity on both soundscape and music data sets confirms that the BOF approach can model soundscapes to near-perfect precision, and exhibits none of the limitations observed in the music data set. Second, the modification of this measure by two custom homogeneity transforms reveals critical differences in the temporal and statistical structure of the typical frame distribution of each type of signal. Such differences may explain the uneven performance of BOF algorithms on soundscapes and music signals, and suggest that their human perception rely on cognitive processes of a different nature.