Search

Within certain categories of geometric shapes, prototypical exemplars that best characterize the category have been evidenced. These geometric prototypes are classically identified through the visual and haptic perception or motor production and are usually characterized by their spatial dimension. However, whether prototypes can be recalled through the auditory channel has not been formally investigated. Here we address this question by using auditory cues issued from timbre-modulated friction sounds evoking human drawing elliptic movements. Since non-spatial auditory cues were previously found useful for discriminating distinct geometric shapes such as circles or ellipses, it is hypothesized that sound dynamics alone can evoke shapes such as an exemplary ellipse. Four experiments were conducted and altogether revealed that a common elliptic prototype emerges from auditory, visual, and motor modalities. This finding supports the hypothesis of a common coding of geometric shapes according to biological rules with a prominent role of sensory-motor contingencies in the emergence of such prototypical geometry.

BackgroundMany scientific fields now use machine-learning tools to assist with complex classification tasks. In neuroscience, automatic classifiers may be useful to diagnose medical images, monitor electrophysiological signals, or decode perceptual and cognitive states from neural signals. However, such tools often remain black-boxes: they lack interpretability. A lack of interpretability has obvious ethical implications for clinical applications, but it also limits the usefulness of these tools to formulate new theoretical hypotheses.New methodWe propose a simple and versatile method to help characterize the information used by a classifier to perform its task. Specifically, noisy versions of training samples or, when the training set is unavailable, custom-generated noisy samples, are fed to the classifier. Multiplicative noise, so-called “bubbles”, or additive noise are applied to the input representation. Reverse correlation techniques are then adapted to extract either the discriminative information, defined as the parts of the input dataset that have the most weight in the classification decision, and represented information, which correspond to the input features most representative of each category.ResultsThe method is illustrated for the classification of written numbers by a convolutional deep neural network; for the classification of speech versus music by a support vector machine; and for the classification of sleep stages from neurophysiological recordings by a random forest classifier. In all cases, the features extracted are readily interpretable.Comparison with Existing MethodsQuantitative comparisons show that the present method can match state-of-the art interpretation methods for convolutional neural networks. Moreover, our method uses an intuitive and well-established framework in neuroscience, reverse correlation. It is also generic: it can be applied to any kind of classifier and any kind of input data.ConclusionsWe suggest that the method could provide an intuitive and versatile interface between neuroscientists and machine-learning tools.HighlightsThe heuristics of black-box classifiers can be probed with noisy inputsThe relevant features can be visualised in the input representation spaceThe method applies to any kind of data such as 2D images or 1D time seriesIt applies to any classifier such as deep neural networks, support vector machines, random forests

International audience; Humans excel at using sounds to make judgements about their immediate environment. In particular, timbre is an auditory attribute that conveys crucial information about the identity of a sound source, especially for music. While timbre has been primarily considered to occupy a multidimensional space, unravelling the acoustic correlates of timbre remains a challenge. Here we re-analyse 17 datasets from published studies between 1977 and 2016 and observe that original results are only partially replicable. We use a data-driven computational account to reveal the acoustic correlates of timbre. Human dissimilarity ratings are simulated with metrics learned on acoustic spectrotemporal modulation models inspired by cortical processing. We observe that timbre has both generic and experiment-specific acoustic correlates. These findings provide a broad overview of former studies on musical timbre and identify its relevant acoustic substrates according to biologically inspired models.

Introduction Lack of sleep drastically affects many aspects of human behavior. The early detection of sleepiness is thus a major challenge for health and security reasons. Here we investigated the effect of sleep deprivation on the acoustic properties of human speech. Methods Twenty-four participants were sleep deprived for two days (two successive nights with only 3 hours of sleep). They were recorded reading a short text aloud before and after sleep deprivation. An auditory model, based on spectro-temporal modulations, was used to analyse the acoustic properties of their speech and served as a front-end to machine-learning classifiers. Results Results showed that sleepiness could be accurately detected with individually-trained classifiers. However,we were not able to fit a generic classifier for all participants. As we relied on an auditory-inspired model,we could identify and interpret the acoustic features impacted by sleep deprivation. Again,no simple diagnostic feature could be easily identified in the group- level analyses of the speech signals. We therefore developed a novel probing method, combining signal detection theory and noise activation of the classifier, to understand what made the classifier successful for each participant. This led to a diagnostic map for each participant, specifying which frequency region and modulation rates were impacted by sleep deprivation for this particular individual Conclusion In addition to suggesting a practical machine learning algorithm to detect sleep deprivation, combining our probing method with considerations about voice production could help uncover the physiological impact of sleep deprivation at the level of each individual. Support

International audience; Natural soundscapes correspond to the acoustical patterns produced by biological and geophysical sound sources at different spatial and temporal scales for a given habitat. This pilot study aims to characterize the temporal-modulation information available to humans when perceiving variations in soundscapes within and across natural habitats. This is addressed by processing soundscapes from a previous study [Krause, Gage, and Joo. (2011). Landscape Ecol. 26, 1247] via models of human auditory processing extracting modulation at the output of cochlear filters. The soundscapes represent combinations of elevation, animal, and vegetation diversity in four habitats of the biosphere reserve in the Sequoia National Park (Sierra Nevada, USA). Bayesian statistical analysis and support vector machine classifiers indicate that: (i) amplitude-modulation (AM) and frequency-modulation (FM) spectra distinguish the soundscapes associated with each habitat; and (ii) for each habitat, diurnal and seasonal variations are associated with salient changes in AM and FM cues at rates between about 1 and 100 Hz in the low (1-3 kHz) audio-frequency range. Support vector machine classifications further indicate that soundscape variations can be classified accurately based on these perceptually inspired representations.

Musical instrument timbre has been intensively investigated through dissimilarity rating tasks. It is now well known that audio descriptors such as attack time and spectral centroid, among others, account well for the dimensions of the timbre spaces underlying these dissimilarity ratings. Nevertheless, it remains very difficult to reproduce these perceptual judgments from distances computed on acoustical representations such as the waveform or the spectrogram. Interestingly, biologically inspired representations based on spectrotemporal modulation spectra such as spectrotemporal receptive fields (STRF) have been shown to be well-suited to reproduce human dissimilarity ratings (Patil et al., 2012). Here, we propose a meta-analysis of seven former studies on timbre spaces in light of these recently developed representations. We implemented a computational framework that optimizes the correlation between the perceptual results and distances obtained from a set of different acoustic representations, in particular through the STRF. We observed that distances computed from spectrotemporal modulation representations provide the best correlation with the perceptual results across the seven timbre spaces. Finally, we highlighted the parts of the representations contributing the most to the correlation suggesting new insights into the underlying perceptual metrics. [Supported by Canada Research Chair, NSERC (RGPIN-2015-05208, RGPAS-478121-15), (RGPIN-262808-2012), and EU MSCf (Project MIM, H2020-MSCA-IF-2014, GA no. 659).]Musical instrument timbre has been intensively investigated through dissimilarity rating tasks. It is now well known that audio descriptors such as attack time and spectral centroid, among others, account well for the dimensions of the timbre spaces underlying these dissimilarity ratings. Nevertheless, it remains very difficult to reproduce these perceptual judgments from distances computed on acoustical representations such as the waveform or the spectrogram. Interestingly, biologically inspired representations based on spectrotemporal modulation spectra such as spectrotemporal receptive fields (STRF) have been shown to be well-suited to reproduce human dissimilarity ratings (Patil et al., 2012). Here, we propose a meta-analysis of seven former studies on timbre spaces in light of these recently developed representations. We implemented a computational framework that optimizes the correlation between the perceptual results and distances obtained from a set of different acoustic representations, in partic...

International audience; In this paper, a flexible control strategy for a synthesis model dedicated to nonlinear friction phenomena is proposed. This model enables to synthesize different types of sound sources, such as creaky doors, singing glasses, squeaking wet plates or bowed strings. Based on the perceptual stance that a sound is perceived as the result of an action on an object we propose a genuine source/filter synthesis approach that enables to elude physical constraints induced by the coupling between the interacting objects. This approach makes it possible to independently control and freely combine the action and the object. Different implementations and applications related to computer animation, gesture learning for rehabilitation and expert gestures are presented at the end of this paper.

International audience; The perception and production of biological movements is characterized by the 1/3 power law, a relation linking the curvature and the velocity of an intended action. In particular, motions are perceived and reproduced distorted when their kinematics deviate from this biological law. Whereas most studies dealing with this perceptual-motor relation focused on visual or kinaesthetic modalities in a unimodal context, in this paper we show that auditory dynamics strikingly biases visuomotor processes. Biologically consistent or inconsistent circular visual motions were used in combination with circular or elliptical auditory motions. Auditory motions were synthesized friction sounds mimicking those produced by the friction of the pen on a paper when someone is drawing. Sounds were presented diotically and the auditory motion velocity was evoked through the friction sound timbre variations without any spatial cues. Remarkably, when subjects were asked to reproduce circular visual motion while listening to sounds that evoked elliptical kinematics without seeing their hand, they drew elliptical shapes. Moreover, distortion induced by inconsistent elliptical kinematics in both visual and auditory modalities added up linearly. These results bring to light the substantial role of auditory dynamics in the visuo-motor coupling in a multisensory context.

International audience; Many studies stressed that the human movement execution but also the perception of motion are constrained by specific kinematics. For instance, it has been shown that the visuo-manual tracking of a spotlight was optimal when the spotlight motion complies with biological rules such as the so-called 1/3 power law, establishing the co-variation between the velocity and the trajectory curvature of the movement. The visual or kinesthetic perception of a geometry induced by motion has also been shown to be constrained by such biological rules. In the present study, we investigated whether the geometry induced by the visuo-motor coupling of biological movements was also constrained by the 1/3 power law under visual open loop control, i.e. without visual feedback of arm displacement. We showed that when someone was asked to synchronize a drawing movement with a visual spotlight following a circular shape, the geometry of the reproduced shape was fooled by visual kinematics that did not respect the 1/3 power law. In particular, elliptical shapes were reproduced when the circle is trailed with a kinematics corresponding to an ellipse. Moreover, the distortions observed here were larger than in the perceptual tasks stressing the role of motor attractors in such a visuo-motor coupling. Finally, by investigating the direct influence of visual kinematics on the motor reproduction, our result conciliates previous knowledge on sensorimotor coupling of biological motions with external stimuli and gives evidence to the amodal encoding of biological motion.

International audience; The present study investigated the effect of handwriting sonification on graphomotor learning. Thirty-two adults, distributed in two groups, learned four new characters with their non-dominant hand. The experimental design included a pre-test, a training session, and two post-tests, one just after the training sessions and another 24h later. Two characters were learned with and two without real-time auditory feedback (FB). The first group first learned the two non-sonified characters and then the two sonified characters whereas the reverse order was adopted for the second group. Results revealed that auditory FB improved the speed and fluency of handwriting movements but reduced, in the short-term only, the spatial accuracy of the trace. Transforming kinematic variables into sounds allows the writer to perceive his/her movement in addition to the written trace and this might facilitate handwriting learning. However, there were no differential effects of auditory FB, neither long-term nor short-term for the subjects who first learned the characters with auditory FB. We hypothesize that the positive effect on the handwriting kinematics was transferred to characters learned without FB. This transfer effect of the auditory FB is discussed in light of the Theory of Event Coding.

International audience; This study investigates the human ability to perceive biological movements through friction sounds produced by drawings and, furthermore, the ability to recover drawn shapes from the friction sounds generated. In a first experiment, friction sounds, real-time synthesized and modulated by the velocity profile of the drawing gesture, revealed that subjects associated a biological movement to those sounds whose timbre variations were generated by velocity profiles following the 1/3 power law. This finding demonstrates that sounds can adequately inform about human movements if their acoustic characteristics are in accordance with the kinematic rule governing actual movements. Further investigations of our ability to recognize drawn shapes were carried out in two association tasks in which both recorded and synthesized sounds had to be associated to both distinct and similar visual shapes. Results revealed that, for both synthesized and recorded sounds, subjects made correct associations for distinct shapes, while some confusion was observed for similar shapes. The comparisons made between recorded and synthesized sounds lead to conclude that the timbre variations induced by the velocity profile enabled the shape recognition. The results are discussed in the context of the ecological and ideomotor frameworks.

International audience; This paper presents an experiment dealing with the sensorimotor relation between auditory perception and graphical movements. Subjects were asked to synchronize their gestures with synthetic friction sounds. Some geometrical and dynamical parameters of the motor productions are analyzed according to the different mappings. This experiment provides a formal framework for a wider study which aims to evaluate the relation between audition, vision and gestures.

The dynamic features of sounds make them particularly appropriate for assessing the spatiotemporal characteristics of movements. Furthermore, sounds can inform about the correctness of an ongoing movement without directly interfering with the visual and proprioceptive feedback. Finally, because of their playful characteristics, sounds are potentially effective for motivating writers in particular need of any writing assistance. By associating relevant sounds to the specific variables of handwriting movement, the present chapter aimed at reporting how supplementary auditory information allows an examiner (teacher or therapist) to assess the movement quality from his/her hearing. Furthermore, a writer could also improve his/her movement from this real-time auditory feedback. Sonification of some movement characteristics would be a relevant tool for the diagnosis and the rehabilitation of some developmental disabilities (e.g. dysgraphia) or acquired disorders (e.g. Parkinson’s disease).

International audience; Even though we generally don't pay attention to the friction sounds produced when we are writing or drawing, these sounds are recordable, and can even evoke the underlying gesture. In this paper, auditory perception of such sounds, and the internal representations they evoke when we listen to them, is considered from the sensorimotor learning point of view. The use of synthesis processes of friction sounds makes it possible to investigate the perceptual influence of each gestures parameter separately. Here, the influence of the velocity profile on the mental representation of the gesture induced by a friction sound was investigated through 3 experiments. The results reveal the perceptual relevance of this parameter, and particularly a specific morphology corresponding to biological movements, the so-called 1/3-power law. The experiments are discussed according to the sensorimotor theory and the invariant taxonomy of the ecological approach.