Search

Unlike vision, the mechanisms underlying auditory motion perception are poorly understood. Here we describe an auditory motion illusion revealing a novel cue to auditory speed perception: the temporal frequency of amplitude modulation (AM-frequency), typical for rattling sounds. Naturally, corrugated objects sliding across each other generate rattling sounds whose AM-frequency tends to directly correlate with speed. We found that AM-frequency modulates auditory speed perception in a highly systematic fashion: moving sounds with higher AM-frequency are perceived as moving faster than sounds with lower AM-frequency. Even more interestingly, sounds with higher AM-frequency also induce stronger motion aftereffects. This reveals the existence of specialized neural mechanisms for auditory motion perception, which are sensitive to AM-frequency. Thus, in spatial hearing, the brain successfully capitalizes on the AM-frequency of rattling sounds to estimate the speed of moving objects. This tightly parallels previous findings in motion vision, where spatio-temporal frequency of moving displays systematically affects both speed perception and the magnitude of the motion aftereffects. Such an analogy with vision suggests that motion detection may rely on canonical computations, with similar neural mechanisms shared across the different modalities.

Recent studies have proposed that some cross-modal illusions might be expressed in what were previously thought of as sensory-specific brain areas. Therefore, one interesting question is whether auditory-driven visual illusory percepts respond to manipulations of low-level visual attributes (such as luminance or chromatic contrast) in the same way as their nonillusory analogs. Here, we addressed this question using the double flash illusion (DFI), whereby one brief flash can be perceived as two when combined with two beeps presented in rapid succession. Our results showed that the perception of two illusory flashes depended on luminance contrast, just as the temporal resolution for two real flashes did. Specifically we found that the higher the luminance contrast, the stronger the DFI. Such a pattern seems to contradict what would be predicted from a maximum likelihood estimation perspective, and can be explained by considering that low-level visual stimulus attributes similarly modulate the perception of sound-induced visual phenomena and "real" visual percepts. This finding provides psychophysical support for the involvement of sensory-specific brain areas in the expression of the DFI. On the other hand, the addition of chromatic contrast failed to produce a change in the strength of the DFI despite it improved visual sensitivity to real flashes. The null impact of chromaticity on the cross-modal illusion might suggest a weaker interaction of the parvocellular visual pathway with the auditory system for cross-modal illusions. This research was supported by the Spanish Ministry of Science and Innovation (PSI2013-41568-P, PSI2013-42626-P), Comissionat per a Universitats i Recerca del DIUEGeneralitat de Catalunya (2014SGR856, SGR2014-079), ICREA-Academy and the European Research Council (StG-2010263145).

We experience the world mostly in a multisensory fashion using a combination of all of our senses. Depending on the modality we can select different exploration strategies for extracting perceptual information. For instance, using touch we can enclose an object in our hand to explore parts of the object in parallel. Alternatively, we can trace the object with a single finger to explore its parts in a serial fashion. In this study we investigated whether the exploration mode (parallel versus serial) affects the way sensory signals are combined. To this end, participants visually and haptically explored surfaces that varied in roll angle and indicated which side of the surface was perceived as higher. In Experiment 1, the exploration mode was the same for both modalities (i.e., both parallel or both serial). In Experiment 2, we introduced a difference in exploration mode between the two modalities (visual exploration was parallel while haptic exploration was serial or vice versa). The results showed that visual and haptic signals were combined in a statistically optimal fashion only when the exploration modes were the same. In case of an asymmetry in the exploration modes across modalities, integration was suboptimal. This indicates that spatial-temporal discrepancies in the acquisition of information in the two senses (i.e., haptic and visual) can lead to the break-down of sensory integration.

We often handle an object with both of our hands. The information from the two hands has to be combined to arrive at a single percept of the object. Research on multi-sensory perception has shown that redundant information between the senses is integrated such that the combined percept is more precise than either of the two individual inputs. However, while bimanual information can be redundant, it is not necessarily the case because both hands are usually touching different parts of the same object. To investigate whether there is a difference in precision of unimanual and bimanual information, we asked participants to discriminate stiffness unimanually as well as bimanually. Our results clearly show that bimanual perception of stiffness was more precise than unimanual perception. The precision of the bimanual percept was in agreement with the precision predicted from combining the two unimanual inputs in a statistical optimal fashion.

Human perception is inherently multisensory: we perceive the world simultaneously with multiple senses. While strolling the farmers market, for example, we might become aware of the presence of a delicious fruit by its characteristic smell. We might use our senses of vision and touch to identify the fruit by its typical size and shape and touch it to select only that one with the distinctive soft texture that signals ‘ripe’. When we take a bite of the fruit, we taste its characteristic flavour and hear a slight smacking sound which confirms that the fruit we perceive with our senses of vision, touch, audition, smell and taste is a ripe, delicious peach. That is, in the natural environment the information delivered by our sense of touch is combined with information gathered by each of the other senses to create a robust percept. Combining information from multiple systems is essential because no information-processing system, neither technical nor biological, is powerful enough to provide a precise and accurate sensory estimate under all conditions.

This chapter examines the application of a psychophysical evaluation technique to quantify the fidelity of haptic rendering methods. The technique is based on multidimensional scaling analysis of similarity ratings provided by users comparing pairs of haptically-presented objects. Unbeknownst to the participants, both real and virtual deformable objects were presented. In addition, virtual objects were either rendered under high fidelity condition or under lower-fidelity condition in which filtering quality was reduced. The analysis of pairwise similarity data provides quantitative confirmation that users perceived a clear difference between real and virtual objects in the lower-fidelity, but not in the higher-fidelity condition. In the latter, a single perceptual dimension, corresponding to stiffness, sufficed to explain similarity data, while two perceptual dimensions were needed in the former condition. This demonstrates how multidimensional scaling analysis can be used in the evaluation of haptic renderin g scenarios, providing perceptual maps of real and virtual objects. It offers an opportunity to visualize and quantify the perceptual effects of changes in rendering parameters.

This chapter presents an overview of interesting scientific findings related to human haptic perception and discuss the usability of these scientific findings for the design and development of virtual environments including haptic rendering. The first section of the chapter deals with pure haptic perception whereas the second and third sections are devoted to the integration of kinesthetic information with other sensory inputs like vision and audition.

In this work, we analyze whether oscillatory motion between two extreme positions could be used to create a robotic dancing partner that provides natural haptic feedback. To this end, we compared the pattern of hand movements performed following a pacing signal while participants were instructed to either move rhythmically or to dance. Furthermore, we analyzed the influence of the frequency and type of pacing signal on the two kinds of movements. Trajectories were analyzed in terms of: frequency of movement, spatial and temporal synchronization, and jerk. Results indicate that it is easier to perform synchronized movements while dancing, even though these movements partially deviate from the pacing frequency. Dance movements are in fact more complex than the ones produced to keep the rhythm and for this reason they should be modeled accordingly in order to provide realistic haptic feedback.

This paper describes the systematic design of a modular setup for several integrated kinesthetic and cutaneous (tactile) display configurations. The proposed modular integration of a kinesthetic display and several tactile displays in serial configuration provides a versatile experimental setup to explore the integration of the kinesthetic and tactile modality of the human perception. The kinesthetic base display is a hyper-redundant device and sufficiently powerful to carry each of the compact tactile displays. In addition to a detailed description of the partly novel displays, a series of preliminary evaluation experiments is presented.

In this paper, we examine the application of a psychophysical evaluation technique to quantify the fidelity of haptic rendering methods. The technique is based on multidimensional scaling (MDS) analysis of similarity ratings provided by users comparing pairs of haptically-presented objects. Unbeknownst to the participants, both real and virtual deformable objects were presented to them. In addition, virtual objects were either presented under higher-fidelity rendering condition or under lower-fidelity condition in which force filtering and proxy-point filtering were removed. We hypothesized that reducing fidelity of virtual rendering would exaggerate the difference between real and virtual objects. MDS analysis of pairwise similarity data provided quantitative confirmation that users perceived a clear difference between real and virtual objects in the lower-fidelity, but not in the higher-fidelity condition. In the latter, a single perceptual dimension, corresponding to stiffness, sufficed to explain similarity data, while two perceptual dimensions were needed in the former condition. This study demonstrates how MDS analysis provides an opportunity to visualize and quantify the perceptual effects of changes in rendering parameters and how it can be used in the evaluation of haptic rendering scenarios.

For perceiving the environment our brain uses multiple sources of sensory information derived from several different modalities, including vision, touch and audition. Some sources of sensory information derived from different modalities provide information about the same object property or event. For example, the size of an object can both be seen with the eyes and felt with the hands. This we call redundant sources of sensory information. In this report we will show how such redundant sources of sensory information are used by the human brain in order to interact with the environment in a goal-directed fashion. Further, we show what role prior knowledge about the statistical regularities in the world plays and how it can affect the process of perception. As a model for describing such somatosensory interactions we here use Bayesian Decision Theory (BDT).

Tactile feedback is among haptics one of the more recent modalities for human-system interaction. Research in tactile feedback using pin-array type actuators has been going on during the past years or so. A survey about technological achievements, human sensing capabilities, and psychophysical evaluation in this area is presented. Then the focus is on novel approaches in actuator technology and tactile feedback systems providing shear force (tangential force to the finger-tip).