Search

International audience; We propose a new type of interaction dedicated to virtual reality (VR). We have designed a futuristic scenario promoting a dialogue with a humanoid. The participant is submitted to a reverse Turing test and has to answer questions addressed by the humanoid. The participant uses his/her own voice to respond, which is transformed in real time in terms of timbre and spatialization, in correspondence with special effects applied to the 360-degree video. Behavioral assessments are proposed after each vocal answer given, to determine the quality of the vocal interaction. Overall, the potential of this new narrative vector is confirmed by the very positive feelings of the participants, while promoting the extension of interactivity in 360-degree VR.

Cette installation propose un nouveau type d'interaction, vocale, dédié à la réalité virtuelle (RV). Un scénario futuriste sert de prétexte au dialogue avec le personnage virtuel, Pieter Musk, incarnation anthropoïde d'une intelligence artificielle faisant passer au participant un test de Turing inversé. Celui-ci utilise alors sa propre voix pour répondre, laquelle est transformée en temps réel en timbre et en spatialisation, en correspondance avec la vidéo 360°. Dans la fiction, ce test quantifie le degré d'humanité du participant, tandis que des évaluations comportementales déterminent en parallèle la qualité de l'interaction. Globalement, le potentiel de ce nouveau vecteur narratif est confirmé par le ressenti très positif des participants, tout en favorisant l'extension de l'interactivité en RV à 360°.

International audience; Human listeners are able to recognize accurately an impressive range of complex sounds, such as musical instruments or voices. The underlying mechanisms are still poorly understood. Here, we aimed to characterize the processing time needed to recognize a natural sound. To do so, by analogy with the "rapid visual sequential presentation paradigm", we embedded short target sounds within rapid sequences of distractor sounds. The core hypothesis is that any correct report of the target implies that sufficient processing for recognition had been completed before the time of occurrence of the subsequent distractor sound. We conducted four behavioral experiments using short natural sounds (voices and instruments) as targets or distractors. We report the effects on performance, as measured by the fastest presentation rate for recognition, of sound duration, number of sounds in a sequence, the relative pitch between target and distractors and target position in the sequence. Results showed a very rapid auditory recognition of natural sounds in all cases. Targets could be recognized at rates up to 30 sounds per second. In addition, the best performance was observed for voices in sequences of instruments. These results give new insights about the remarkable efficiency of timbre processing in humans, using an original behavioral paradigm to provide strong constraints on future neural models of sound recognition. Anecdotally, we as human listeners seem remarkably apt at recognizing sound sources: the sound of a voice, approaching footsteps, or musical instruments in each of our cultures. There is now quantitative behavioral evidence supporting this idea (for a review, see Agus et al., in press 1). However, the underlying neural mechanisms for such an impressive feat remain unclear. One way to constrain the range of possible mechanisms is to measure the temporal characteristics of sound source recognition. Using a straightforward operational definition of recognition as a correct response to a target sound defined by its category (e.g., a voice among musical instruments), Agus et al. 2 have shown that reaction times for recognition were remarkably short, with an overhead compared to simple detection between 145 ms and 250 ms depending on target type. When natural sounds were artificially shortened by applying an amplitude "gate" of variable duration, it was observed that recognition remained above chance for durations in the milliseconds range 3-5. However, none of these results speak directly to the processing time required for sound recognition. For reaction times, the comparison of recognition and simple detection times cannot be unequivocally used to estimate processing time 6. For gating, recognizing a very short sound presented in isolation could still require a very long processing time: the short sound duration only constrain the type of acoustic features that are used 7. Similar questions about the processing time required for visual recognition of natural objects have been asked 8,9. They have typically been addressed by what is known as the now-classic "rapid sequential visual presentation task" (RSVP) 10-13 (for a review, see Keysers et al. 13). Briefly, in RSVP, images are flashed in a rapid sequence, with images from one target category presented among many distractors belonging to other categories. Participants are asked to report the sequences containing an image from the target category. The fastest presentation rate for which target recognition remains accurate is taken as a measure of processing time. The core hypothesis is that, for a target to be accurately recognized, it needs to have been sufficiently processed before the next distractor is flashed 13 .

International audience; Classical timbre studies have modeled timbre as the integration of a limited number of auditory dimensions and proposed acoustic correlates to these dimensions to explain sound identification. Here, the goal was to highlight time-frequency patterns subserving identification of musical voices and instruments, without making any assumption about these patterns. We adapted a “random search method” proposed in vision. The method consists of synthesizing sounds by randomly selecting “auditory bubbles” (small time-frequency glimpses) from the original sounds’ spectrograms, and then inverting the resulting sparsified representation. For each bubble selection, a decision procedure categorizes the resulting sound as a voice or an instrument. After hundreds of trials, the whole time-frequency space is explored, and adding together the correct answers reveals the relevant time-frequency patterns for each category. We used this method with two decision procedures: human listeners and a decision algorithm using auditory distances based on spectro-temporal excitation patterns (STEPs). The patterns were strikingly similar for the two procedures: they highlighted higher frequencies (i.e., formants) for the voices, whereas instrument identification was based on lower frequencies (particularly during the onset). Altogether these results show that timbre can be analyzed as time-frequency weighted patterns corresponding to the important cues subserving sound identification.

International audience; Sounds in our environment like voices, animal calls, or musical instruments can be recognized on the basis of timbre alone. The objective of this study was to unravel the features underlying this robust sound recognition. We investigated how severely sounds can be simplified while still being recognizable. A large number of natural sounds were simplified into “auditory or acoustic sketches:” energy peaks were selected on the auditory or acoustic spectrogram, at three levels of simplification (high, medium, and low). The remaining non-sparse information was removed. Listeners were asked to classify the simplified sounds into their four original categories: instruments, birds, vehicles, or voices. We adapted a recent signal detection model to dissociate the perceptual sensitivity (d' scores) from the bias, for each category (4-AFC task). Overall, severely simplified sounds could still be recognized above chance by the listeners. Auditory distances, based on spectro-temporal excitation patterns (STEPs), were then computed. Participants' performances on the 4-AFC task were well correlated with the auditory distances between the four categories. Altogether, our results suggest that sound recognition is a very robust perceptual process, and that basic spectral and spectro-temporal differences between sounds, captured by the auditory distances, can account for this robust recognition.

Sounds in our environment like voices, animal calls or musical instruments are easily recognized by human listeners. Understanding the key features underlying this robust sound recognition is an important question in auditory science. Here, we studied the recognition by human listeners of new classes of sounds: acoustic and auditory sketches, sounds that are severely impoverished but still recognizable. Starting from a time-frequency representation, a sketch is obtained by keeping only sparse elements of the original signal, here, by means of a simple peak-picking algorithm. Two time-frequency representations were compared: a biologically grounded one, the auditory spectrogram, which simulates peripheral auditory filtering, and a simple acoustic spectrogram, based on a Fourier transform. Three degrees of sparsity were also investigated. Listeners were asked to recognize the category to which a sketch sound belongs: singing voices, bird calls, musical instruments, and vehicle engine noises. Results showed that, with the exception of voice sounds, very sparse representations of sounds (10 features, or energy peaks, per second) could be recognized above chance. No clear differences could be observed between the acoustic and the auditory sketches. For the voice sounds, however, a completely different pattern of results emerged, with at-chance or even below-chance recognition performances, suggesting that the important features of the voice, whatever they are, were removed by the sketch process. Overall, these perceptual results were well correlated with a model of auditory distances, based on spectro-temporal excitation patterns (STEPs). This study confirms the potential of these new classes of sounds, acoustic and auditory sketches, to study sound recognition.

This paper proposes a first attempt to visualize and analyze the vibrations induced by a bone-conduction device and propagating at the surface of the skin of a human face. The method is based on a new approach in a so-called quasi-time-averaging regime, resulting in the retrieval of the vibration amplitude and phase from a sequence of digital Fresnel holograms recorded with a high image rate. The design of the algorithm depends on the ratio between the exposure time and the vibration period. The results show the propagation of vibrations at the skin surface, and quantitative analysis is achieved by the proposed approach.

This paper proposes a first attempt to visualize and analyze the vibrations propagating at the surface of a human face's skin induced by a bone conduction device. The proposed method allows the qualitative visualization and quantitative measurement of the surface movements illuminated by a coherent laser beam. To do this, we developed a new approach in a so-called "quasi-time-averaging regime" allowing the retrieval of the vibration amplitude and phase from a sequence of digital Fresnel holograms recorded with a high image rate. The experimental set-up is based on off-axis digital Fresnel holography and a high power continuous wave laser. The sensor is a high speed CMOS camera permitting recordings with a high spatial resolution (1024×1024 pixels) up to 2.4kHz. The set-up is able to provide full field measurements in the frequency bandwidth 100Hz-600Hz. Recording in the quasi-time-averaging regime leaded to the development of a dedicated algorithm able to extract the vibration using only three holograms from the sequence. The design of the algorithm depends on the ratio between exposure time and vibration period. Results exhibit propagation of vibrations at the skin surface, amplitudes being at most at 200nm, and speed velocity can be estimated at each frequency.

This paper proposes a first attempt to visualize and analyze sound propagating at the surface of a human skin induced by a bone conduction device. The proposed method is based on a so-called “quasi-time-averaging regime”.