Search

The use of haptic technologies has recently become immensely essential in Human-Computer Interaction to improve user experience and performance. With the introduction of tactile feedback on a touchscreen device, commonly known as surface haptics, several applications and interaction paradigms have become a reality. However, the effects of tactile feedback on the preference of 2D images in visuo-tactile exploration task on touchscreen devices remain largely unknown. In this study, we investigated differences of preference score (the tendency of participants to like/dislike a 2D image based on its visual and tactile properties), reach time, interaction time, and response time under four conditions of feedback: no tactile feedback, high-quality of tactile information (sharp tactile texture), low-quality of tactile information (blurred tactile texture), and incorrect tactile information (mismatch tactile texture). The tactile feedback is rendered in the form of roughness that is simulated by modulating the friction between the finger and the surface and is derived from the 2D image. Thirty-six participants completed visuo-tactile exploration tasks for a total of 36 trials (3 2D images × 4 tactile textures × 3 repetitions). Results showed that the presence of tactile feedback enhanced users’ preference (tactile feedback conditions were rated significantly higher than the no tactile feedback condition for preference regardless of the quality/correctness of tactile feedback). This finding is also supported through results from self-reporting where 88.89% of participants preferred to experience the 2D image with tactile feedback. Additionally, the presence of tactile feedback resulted in significantly larger interaction time and response time compared to the no tactile feedback condition. Furthermore, the quality and correctness of tactile information significantly impacted the preference rating (sharp tactile textures were rated statistically higher than blurred tactile and mismatched tactile textures). All of these findings demonstrate that tactile feedback plays a crucial role in users’ preference and thus motivates further the development of surface haptic technologies.

Peripersonal space (PPS) is the space immediately surrounding the body, conceptualised as a sensory-motor interface between body and environment. PPS size differs between individuals and contexts, with intrapersonal traits and states, as well as social factors having a determining role on the size of PPS. Testosterone plays an important role in regulating social-motivational behaviour and is known to enhance dominance motivation in an implicit and unconscious manner. We investigated whether the dominance-enhancing effects of testosterone reflect as changes in the representation of PPS in a within-subjects testosterone administration study in women (N = 19). Participants performed a visuo-tactile integration task in a mixed-reality setup. Results indicated that the administration of testosterone caused a significant enlargement of participants’ PPS, suggesting that testosterone caused participants to implicitly appropriate a larger space as their own. These findings suggest that the dominance-enhancing effects of testosterone reflect at the level of sensory-motor processing in PPS.

Aim: Evidence has revealed a relationship between pain and the observation of limb movement, but it is unknown whether different types of movements have diverse modulating effects. In this immersive virtual reality study, we explored the effect of the vision of different virtual arm movements (arm vs wrist) on heat pain threshold of healthy participants. Patients & methods: 40 healthy participants underwent four conditions in virtual reality, while heat pain thresholds were measured. Visuo–tactile stimulation was used to attempt to modulate the feeling of virtual limb ownership while the participants kept their arms still. Results: Effects on pain threshold were present for type of stimulation but not type of movement. Conclusion: The type of observed movement does not appear to influence pain modulation, at least not during acute pain states.

People react faster to visuo-tactile stimuli presented near the body (i.e., in peripersonal space) than to tactile stimuli presented alone. This multi-sensory facilitation effect has been used as a measurement of peripersonal space. Previous research has reported that peripersonal space representations can be modulated by actively using hand-held tools or disconnected hand avatars. However, previous research has ignored the possibility that the attentional effect of active tool use could affect multi-sensory facilitation. In the present study, we delivered tactile stimuli to participants’ left or right hand concurrently with visual stimuli presented near a virtual hand avatar operated by the movements of participants’ left or right hand, which was shown far in a virtual environment and disconnected from the body. Participants reacted to tactile stimuli while ignoring the visual stimuli. The results indicated a multi-sensory facilitation effect when tactile stimuli were delivered to the hand used to operate the hand avatar. In contrast, the facilitation was not observed when the tactile stimuli were delivered to the hand that is not operating the hand avatar. These results suggest that the strength of the multi-sensory facilitation effect differed across conditions, even though the visual attention captured around the hand avatar was controlled across conditions. We concluded that the modulation of peripersonal space resulting from using tools or avatars is nearly independent of visual attention captured around tools or avatars.

Behavioural studies on Multisensory Peripersonal Space (PPS) representations are based on seminal findings in single-cell recordings.A classic experimental paradigm used to assess PPS is the Multisensory Integration Paradigm. In this paradigm Visuo-Tactile stimuli are administrated to participants. According to the paradigm, the tactile component vibrates when the visual component, approaching the participant, reaches a specific distance. Participants must detect the tactile component. Reaction times (RTs) are faster with closer visual stimuli. This effect is classically interpreted as a manifestation of PPS representation.However, the Visual and Tactile components are usually targeting at the same body part, and Tactile-Only RTs are considered constant.In this study we formally addressed two research questions: i) whether the Multisensory Visuo-Tactile RTs are independent from Tactile-Only RTs and whether the latter are influenced by time-dependency effects; and ii) whether PPS estimations derived from Multisensory Visuo-Tactile RTs depend on the location of the Visual or Tactile component of the stimuli.We studied 40 right-handed participants, in a repeated-measure design, crossing the position of the Tactile and Visual components of stimuli across the face, the hand and the foot, and 5 distances. Each trial could be Visuo-Tactile, Tactile-Only or a Catch trial.Frequentist and Bayesian analysis demonstrated that Tactile-Only RTs follow a non-monotonic trend, depending on the delay of stimulus administration. Furthermore, according to our findings, Visuo-Tactile RTs were proved to be dependent on Distance and location of the Visual component of the stimulus. Our results lead to the conclusion that Tactile-Only RTs should be removed from Visuo-Tactile RTs and that the Visual and Tactile components of Visuo-Tactile stimuli do not necessarily have to target at the same body part.These results may have an advantageous impact on the study of PPS representations, providing new and important methodological information.

Estimation of tactile properties from vision, such as slipperiness or roughness, is important to effectively interact with the environment. These tactile properties help humans, as well as robots, decide which actions they should choose and how to perform them. We, therefore, propose a model to estimate the degree of tactile properties from visual perception alone (e.g., the level of slipperiness or roughness). Our method extends an encoder-decoder network, in which the latent variables are visual and tactile features. In contrast to previous works, our method does not require manual labeling, but only RGB images and the corresponding tactile sensor data. All our data is collected with a webcam and tactile sensor mounted on the end-effector of a robot, which strokes the material surfaces. We show that our model generalizes to materials not included in the training data.

SUMMARYThis work presents the multilayered caries model with a visuo-tactile virtual reality simulator and a randomized controlled trial protocol to determine the effectiveness of the simulator in training for minimally invasive caries removal. A three-dimensional, multilayered caries model was reconstructed from 10 micro-computed tomography (CT) images of deeply carious extracted human teeth before and after caries removal. The full grey scale 0-255 yielded a median grey scale value of 0-9, 10-18, 19-25, 26-52, and 53-80 regarding dental pulp, infected carious dentin, affected carious dentin, normal dentin, and normal enamel, respectively. The simulator was connected to two haptic devices for a handpiece and mouth mirror. The visuo-tactile feedback during the operation varied depending on the grey scale. Sixth-year dental students underwent a pretraining assessment of caries removal on extracted teeth. The students were then randomly assigned to train on either the simulator (n=16) or conventional extracted teeth (n=16) for 3 days, after which the assessment was repeated. The posttraining performance of caries removal improved compared with pretraining in both groups (Wilcoxon, p

For a comprehensive understanding of the environment, the brain must constantly decide whether the incoming information originates from the same source and needs to be integrated into a coherent percept. This integration process is believed to be mediated by temporal integration windows. If presented with temporally asynchronous stimuli for a few minutes, the brain adapts to this new temporal relation by recalibrating the temporal integration windows. Such recalibration can occur even more rapidly after exposure to just a single trial of asynchronous stimulation. While rapid recalibration has been demonstrated for audio–visual stimuli, evidence for rapid recalibration of visuo–tactile stimuli is lacking. Here, we investigated rapid recalibration in the visuo–tactile domain. Subjects received visual and tactile stimuli with different stimulus onset asynchronies (SOA) and were asked to report whether the visuo–tactile stimuli were presented simultaneously. Our results demonstrate visuo–tactile rapid recalibration by revealing that subjects’ simultaneity reports were modulated by the temporal order of stimulation in the preceding trial. This rapid recalibration effect, however, was only significant if the SOA in the preceding trial was smaller than 100 ms, while rapid recalibration could not be demonstrated for SOAs larger than 100 ms. Since rapid recalibration in the audio–visual domain has been demonstrated for SOAs larger than 100 ms, we propose that visuo–tactile recalibration works at shorter SOAs, and thus faster time scales than audio–visual rapid recalibration. Electronic supplementary material The online version of this article (10.1007/s00221-017-5132-z) contains supplementary material, which is available to authorized users.

Since the discovery of neural regions in the monkey brain that respond preferentially to multisensory stimuli presented in proximal space, researchers have been studying this specialised spatial representation in humans. It has been demonstrated that approaching auditory or visual stimuli modulate tactile processing, while they are within the peripersonal space (PPS). The aim of the current study is to investigate the additional effects of tactile expectation on the PPS-related multisensory interactions. Based on the output of a computational simulation, we expected that as tactile expectation increases rapidly during the course of the motion of the visual stimulus, the outcome RT curves would mask the multisensory contribution of PPS. When the tactile expectation remains constant during the motion, the PPS-related spatially selective multisensory processes become apparent. The behavioural results on human experiments followed the pattern predicted by the simulation. That is, rapidly changing levels of tactile expectation, caused by dynamic visual stimuli, masks the outcome of the multisensory processes within peripersonal space. This indicates that both PPS-related multisensory interactions and tactile expectations play an important role in anticipating and responding to interactions with the body.

The connection between visual input and tactile sensing is critical for object manipulation tasks such as grasping and pushing. In this work, we introduce the challenging task of estimating a set of tactile physical properties from visual information. We aim to build a model that learns the complex mapping between visual information and tactile physical properties. We construct a first of its kind image-tactile dataset with over 400 multiview image sequences and the corresponding tactile properties. A total of fifteen tactile physical properties across categories including friction, compliance, adhesion, texture, and thermal conductance are measured and then estimated by our models. We develop a cross-modal framework comprised of an adversarial objective and a novel visuo-tactile joint classification loss. Additionally, we introduce a neural architecture search framework capable of selecting optimal combinations of viewing angles for estimating a given physical property.

International audience; Context: Raised grain occurs on wood surfaces after the application of a waterborne varnish and forces manufacturers to sand the surfaces between coats. Actually, little research has characterised this phenomenon and no techniques have been discovered to avoid its occurrence.Aims: This study aims to identify the topographic parameters that explain the visuo-tactile sensation of raised grain and to define a finished surface quality acceptable to consumers and industry.Methods: Oak (Quercus robur L.) and beech (Fagus sylvatica L.) wood surfaces were planed and sanded in order to have various levels of raised grain. Visuo-tactile analyses were carried out on surfaces having received one coat of varnish to characterise raised grain and having two coats to characterise the acceptable finished surface quality without sanding. Topographic parameters were measured on each type of varnished surface and correlated with the visuo-tactile scores.Results: Raised grain was characterised by the visuo-tactile sensation of protruding peaks and a certain amount of material in the core of the roughness profile for both wood species. Industrials overestimated the surface quality required by consumers. Thresholds of topographic parameters were determined to define acceptable finished surface quality.Conclusion: These findings allowed objective criteria to be defined for describing raised grain and to help industries to optimise their wood machining and finishing processes.

Enhancing motor imagery (MI) results in amplified event-related desynchronization (ERD) and is important for MI-based rehabilitation and brain–computer interface (BCI) applications. Many attempts to enhance the MI by providing a visual guidance have been reported. We believe that the rubber hand illusion (RHI), which induces body ownership over an external object, can provide better guidance to enhance MI; thus, an RHI-based paradigm with motorized moving rubber hand was proposed. To validate the proposed MI enhancing paradigm, we conducted an experimental comparison among paradigms with 20 healthy subjects. The peak amplitude and arrival times of ERD were compared at contralateral and ipsilateral electroencephalogram channels. We found significantly amplified ERD caused by the proposed paradigm, which is similar to the ERD caused by motor execution. In addition, the arrival time suggests that the proposed paradigm is applicable for BCI. In conclusion, the proposed paradigm can significantly enhance the MI with better characteristics for use with BCI.

The crossmodal congruency task is a consolidated paradigm for investigating interactions between vision and touch. In this task, participants judge the elevation of a tactile target stimulus while ignoring a visual distracter stimulus that may occur at a congruent or incongruent elevation, thus engendering a measure of visuo-tactile interference (crossmodal congruency effect, CCE). The CCE reflects perceptual, attentional, and response-related factors, but their respective roles and interactions have not been set out yet. In two experiments, we used the original version of the crossmodal congruency task as well as ad hoc manipulations of the experimental setting and of the participants’ posture for characterizing the contributions of multisensory integration, body-mediated attention, and response conflict to the CCE. Results of the two experiments consistently showed that the largest amount of variance in the CCE is explained by the reciprocal elevation of visual and tactile stimuli. This finding is compatible with a major role of response conflict for the CCE. Weaker yet distinguishable contributions come from multisensory integration, observed in the absence of response conflict, and from hand-mediated attentional binding, observed with the modified posture and in the presence of response conflict. Overall, this study informs the long-standing debate about mechanisms underlying the CCE by revealing that the visuo-tactile interference in this task is primarily due to the competition between opposite response tendencies, with an additional contribution of multisensory integration and hand-mediated attentional binding

The rubber hand illusion (RHI) and its variant the invisible hand illusion (IHI) are useful for investigating multisensory aspects of bodily self-consciousness. Here, we explored whether auditory conditioning during an RHI could enhance the trisensory visuo-tactile-proprioceptive interaction underlying the IHI. Our paradigm comprised of an IHI session that was followed by an RHI session and another IHI session. The IHI sessions had two parts presented in counterbalanced order. One part was conducted in silence, whereas the other part was conducted on the backdrop of metronome beats that occurred in synchrony with the brush movements used for the induction of the illusion. In a first experiment, the RHI session also involved metronome beats and was aimed at creating an associative memory between the brush stroking of a rubber hand and the sounds. An analysis of IHI sessions showed that the participants' perceived hand position drifted more towards the body-midline in the metronome relative to the silent condition without any sound-related session differences. Thus, the sounds, but not the auditory RHI conditioning, influenced the IHI. In a second experiment, the RHI session was conducted without metronome beats. This confirmed the conditioning-independent presence of sound-induced proprioceptive drift in the IHI. Together, these findings show that the influence of visuo-tactile integration on proprioceptive updating is modifiable by irrelevant auditory cues merely through the temporal correspondence between the visuo-tactile and auditory events.

Graphical abstract, Highlights • Visuo-tactile congruency influences the body schema, but not the body image. • Movement behavior may provide a window into the unconscious body schema. • We present novel methods to distort avatars and Euclidean space in virtual reality., Previous research showed that full body ownership illusions in virtual reality (VR) can be robustly induced by providing congruent visual stimulation, and that congruent tactile experiences provide a dispensable extension to an already established phenomenon. Here we show that visuo-tactile congruency indeed does not add to already high measures for body ownership on explicit measures, but does modulate movement behavior when walking in the laboratory. Specifically, participants who took ownership over a more corpulent virtual body with intact visuo-tactile congruency increased safety distances towards the laboratory’s walls compared to participants who experienced the same illusion with deteriorated visuo-tactile congruency. This effect is in line with the body schema more readily adapting to a more corpulent body after receiving congruent tactile information. We conclude that the action-oriented, unconscious body schema relies more heavily on tactile information compared to more explicit aspects of body ownership.

Previous studies revealed that a compelling illusion of virtual body ownership (VBO) might be achieved under a condition of recognizable anatomical and morphological similarities to human body. Though certain deviations from morphology might be acceptable (e.g. virtual tails, longer arm or larger belly), if external limbs are responsible for specific functions in conjunction with a certain virtual scenario. Thus, this study aims at showing a possibility to achieve a compelling VBO illusion over a non-human avatar in a virtual environment. The paper describes a within-subjects study exploring if immersed users could achieve a compelling VBO illusion when inhabiting a virtual body of a bat. Test subjects experienced visuo-tactile stimulation of their arms while seeing an object touching their virtual wings. The mapping between the real and virtual touch points varied across three conditions: no offset between the visual and the tactile input, 50% offset, and 70% offset from the tactile input. The results revealed variations in the degree of experienced VBO across the conditions. The illusion was broken in the absence of visuo-tactile stimulation.

The peripersonal space (PPS) is the space surrounding our body, represented in a multisensory fashion by integrating stimuli of different modalities. Recently, it has been demonstrated that PPS is emotionally connoted, being sensitive to the different affective valence of the stimuli located inside it. However, how visuo-tactile interactions can be spatially shaped by intrinsic or acquired valence of stimuli is not clear. To investigate this, we conducted three experiments in which participants performed a visuo-tactile interaction task, while the intrinsic valence (Exp. 1 and 2) or the learned valence (Exp. 3) of visual stimuli was manipulated. Participants were asked to respond as fast as possible to a tactile stimulus that was delivered while a visual stimulus was approaching (Exp.1 and 3) or receding (Exp.2) from the hand. Touch was synchronized with different distances of the visual stimulus from the hand. We found that both the expectancy of stimulus and the distance of the visual one from the hand impact RTs to tactile targets. Crucially, we found that spatial modulation was also influenced by stimulus valence, but only for the approaching and not the receding stimuli. At far distances, neutral stimuli yielded overall slower RTs than intrinsically positive or negative stimuli (Experiment 1), while no modulation was exerted by the level of conditioning (Experiment 3). At near distances, response to touches accompanied by looming neutral stimuli became as fast as that occurring with positive and negative ones. Stimulus valence did not interact with the expectancy of a tactile stimulus (Experiment 2). Overall, these findings support the vision that visuo-tactile interactions can be dynamically modulated by the valence of looming visual stimuli when these are located at longer distances from the body. When closer to it, all stimuli acquire saliency, regardless of their intrinsic or acquired valence, due to their proximity, and then relevance, to the body. Overall, a view of PPS as a gradient modulating visuo-tactile integration, also based on stimulus valence, is discussed.

RatSLAM is a rat hippocampus-inspired visual Simultaneous Localization and Mapping (SLAM) framework capable of generating semi-metric topological representations of indoor and outdoor environments. Whisker-RatSLAM is a 6D extension of the RatSLAM and primarily focuses on object recognition by generating point clouds of objects based on whisking information. This paper introduces a novel extension to both former works that is referred to as ViTa-SLAM that harnesses both vision and tactile information for performing SLAM. This not only allows the robot to perform natural interaction with the environment whilst navigating, as is normally seen in nature, but also provides a mechanism to fuse non-unique tactile and unique visual data. Compared to the former works, our approach can handle ambiguous scenes in which one sensor alone is not capable of identifying false-positive loop-closures., Comment: 7 pages, 10 figures. IEEE Cyborg and Bionic Systems (CBS) 2019

International audience; This study proposes a visuo-tactile Zero-Shot object recognition framework. The proposed framework recognizes a set of novel objects for which no tactile or visual training data are available. It uses visuo-tactile training data collected from known objects to recognize the novel ones, given their attributes. This framework extends the haptic Zero-Shot Learning framework that we proposed in [1] with vision, which enables a multimodal recognition system. In our test with the PHAC-2 dataset, the system was able to get a recognition accuracy of 72% among 6 objects that were never touched or seen during the training phase.

Perceiving, recognizing and remembering 3-dimensional (3-D) objects encountered in the environment has a very high survival value; unsurprisingly, this ability is shared among many animal species, including humans. The psychological, psychophysical and neural basis for object perception, discrimination, recognition and memory has been extensively studied in humans, monkeys, pigeons and rodents, but is still far from understood. Nearly all 3-D object recognition studies in the rodent used the "novel object recognition" paradigm, which relies on innate rather than learned behavior; however, this procedure has several important limitations. Recently, investigators have begun to recognize the power of behavioral tasks learned through reinforcement training (operant conditioning) to reveal the sensorimotor and cognitive abilities of mice and to elucidate their underlying neural mechanisms. Here, we describe a novel method for training and testing mice in visual and tactile object discrimination, recognition and memory, and use it to begin to examine the underlying sensory basis for these cognitive capacities. A custom-designed Y maze was used to train mice to associate one of two 3-D objects with a food reward. Out of nine mice trained in two cohorts, seven reached performance criterion in about 20-35 daily sessions of 20 trials each. The learned association was retained, or rapidly re-acquired, after a 6 weeks hiatus in training. When tested under low light conditions, individual animals differed in the degree to which they used tactile or visual cues to identify the objects. Switching to total darkness resulted only in a transient dip in performance, as did subsequent trimming of all large whiskers (macrovibrissae). Additional removal of the small whiskers (microvibrissae) did not degrade performance, but transiently increased the time spent inspecting the object. This novel method can be combined in future studies with the large arsenal of genetic tools available in the mouse, to elucidate the neural basis of object perception, recognition and memory.

Wearable technologies have been supporting and augmenting our body and sensory functions for a long time. Skin+ introduces a novel bidirectional on-skin interface that serve not only as haptic feedback to oneself but also as a visual display to mediate touch sensation to others as well. In this paper, we describe the design of Skin+ and its usability in a variety of applications. We use a shape-changing auxetic structure to build this programmable coherent visuo-tactile interface. The combination of shape-memory alloy with an auxetic structure enables a lightweight haptic device that can be worn seamlessly on top of our skin.

Previous studies have examined the neural correlates for crossmodal paired-associate (PA) memory and the temporal dynamics of its formation. However, the neural dynamics for feedback processing of crossmodal PA learning remain unclear. To examine this process, we recorded event-related scalp electrical potentials for PA learning of unimodal visual-visual pairs and crossmodal visual-tactile pairs when participants performed unimodal and crossmodal tasks. We examined event-related potentials (ERPs) after the onset of feedback in the tasks for three effects: feedback type (positive feedback vs. negative feedback), learning (as the learning progressed) and the task modality (crossmodal vs. unimodal). The results were as follows: (1) feedback type: the amplitude of P300 decreased with incorrect trials and the P400/N400 complex was only present in incorrect trials; (2) learning: progressive positive voltage shifts in frontal recording sites and negative voltage shifts in central and posterior recording sites were identified as learning proceeded; and (3) task modality: compared with the unimodal PA learning task, positive voltage shifts in frontal sites and negative voltage shifts in posterior sites were found in the crossmodal PA learning task. To sum up, these results shed light on cortical excitability related to feedback processing of crossmodal PA learning.

Estimation of tactile properties from vision, such as slipperiness or roughness, is important to effectively interact with the environment. These tactile properties help us decide which actions we should choose and how to perform them. E.g., we can drive slower if we see that we have bad traction or grasp tighter if an item looks slippery. We believe that this ability also helps robots to enhance their understanding of the environment, and thus enables them to tailor their actions to the situation at hand. We therefore propose a model to estimate the degree of tactile properties from visual perception alone (e.g., the level of slipperiness or roughness). Our method extends a encoder-decoder network, in which the latent variables are visual and tactile features. In contrast to previous works, our method does not require manual labeling, but only RGB images and the corresponding tactile sensor data. All our data is collected with a webcam and uSkin tactile sensor mounted on the end-effector of a Sawyer robot, which strokes the surfaces of 25 different materials. We show that our model generalizes to materials not included in the training data by evaluating the feature space, indicating that it has learned to associate important tactile properties with images., 7 pages. Accepted to International Conference on Robotics and Automation (ICRA 2019). Best Paper Award finalist at ICRA2019. An accompanied video is available at https://youtu.be/ys0QtKVVlOQ

Human infants begin very early in life to take advantage of multisensory information by extracting the invariant amodal information that is conveyed redundantly by multiple senses. Here we addressed the question as to whether infants can bind multisensory moving stimuli, and whether this occurs even if the motion produced by the stimuli is only illusory. Three- to 4-month-old infants were presented with two bimodal pairings: visuo-tactile and audio-visual. Visuo-tactile pairings consisted of apparently vertically moving bars (the Barber Pole illusion) moving in either the same or opposite direction with a concurrent tactile stimulus consisting of strokes given on the infant's back. Audio-visual pairings consisted of the Barber Pole illusion in its visual and auditory version, the latter giving the impression of a continuous rising or ascending pitch. We found that infants were able to discriminate congruently (same direction) vs. incongruently moving (opposite direction) pairs irrespective of modality (Experiment 1). Importantly, we also found that congruently moving visuo-tactile and audio-visual stimuli were preferred over incongruently moving bimodal stimuli (Experiment 2). Our findings suggest that very young infants are able to extract motion as amodal component and use it to match stimuli that only apparently move in the same direction.

In this presentation, we describe an HMD-based immersive VR environment which allow us to touch virtual objects and to apply actual kinetic effects on corresponding objects in real world. We utilize foci of airborne ultrasound for generating tactile feedback to users’ hands and applying kinetic force to the real objects. Our system offers a new tele-operation system, where bare-handed users can manipulate real objects located away from them with tactile feedback.

International audience; In wood machining operations, target surfaces are chosen to achieve technical functions (gluing, finishing), or aesthetic functions (raw wood, varnishing) in order to produce a surface which consumers will appreciate. Although the literature often refers to the optimization of cutting conditions to improve the surface quality, there is currently no specific criterion to define what good surface quality is. The purpose of this study was to investigate quantitative criteria related to consumer preferences and to find an acceptability threshold for each criterion in order to determine the best cutting conditions. To this end, 32 surfaces from Chrysophilum boivinianum (Sapotaceae) were machined in various cutting conditions which yielded surfaces ranging from very rough to smooth. The primary surface profile, roughness and waviness parameters, and machining defects (raised grain, torn grain, chip marks, cutting traces) were measured on each surface. Visual and visuo-tactile tests were then carried out with a panel of 174 consumers. The results show that touch allowed better appreciation of surface defects than a simple visual observation. Consumers like smooth surfaces without visible defects and less visible peaks of waviness. The acceptability thresholds of surface parameters correlated with consumer preferences were determined. The rotational speed and the feed speed affect the most the surface quality. To obtain good surface quality for consumers, the rotation speed should be greater than 5000 rpm, with a maximum feed rate per tooth of 0.5 mm, and a maximum average chip thickness of 0.18 mm. This knowledge will help industries to better optimize the cutting of wood.

The space immediately surrounding our body, or peripersonal space, is crucial for interaction with the environment. In primate brains, specific neural circuitry is responsible for its encoding. An important component is a safety margin around the body that draws on visuo-tactile interactions: approaching stimuli are registered by vision and processed, producing anticipation or prediction of contact in the tactile modality. The mechanisms of this representation and its development are not understood. We propose a computational model that addresses this: a neural network composed of a Restricted Boltzmann Machine and a feedforward neural network. The former learns in an unsupervised manner to represent position and velocity features of the stimulus. The latter is trained in a supervised way to predict the position of touch (contact). Unique to this model, it considers: (i) stimulus position and velocity, (ii) uncertainty of all variables, and (iii) not only multisensory integration but also prediction.

A methodological approach to perceptual learning was used to allow both early blind subjects (experimental group) and blindfolded sighted subjects (control group) to experience optical information and spatial phenomena, on the basis of visuo-tactile information transmitted by a 64-taxel pneumatic sensory substitution device. The learning process allowed the subjects to develop abilities in spatial localisation, shape recognition (with generalisation to different points of view), and monocular depth cue interpretation. During the training phase, early blind people initially experienced more difficulties than blindfolded sighted subjects (having previous perceptual experience of perspective) with interpreting and using monocular depth cues. The amelioration of the performance for all blind subjects during training sessions and the quite similar level of performance reached by two groups in the final navigation tasks suggested that early blind people were able to develop and apply cognitive understanding of depth cues. Both groups showed generalisation of the learning from the initial phases to cue identification in the maze, and subjectively experienced shapes facing them. Subjects' performance depended not only on their perceptual experience but also on their previous spatial competencies.

The human visual system is good at discriminating speed but not acceleration. However, as speed is seldom constant, it is important to be able to extract speed in conditions of acceleration and deceleration. We measured visual, tactile and bimodal speed-matching over a wide range of accelerations and decelerations in a 2IFC procedure. Both visual and tactile stimuli were generated on physical wheels etched with a sinusoidal profile. During different experimental sessions the wheels could be seen, or touched, or both. Comparisons between different unimodal and bimodal matched speeds revealed similar integration times for the two modalities, in both cases around one second, suggesting that it occurs at a relatively high level of processing. Bimodal precision of speed discrimination was better than unimodal discrimination, as predicted by the maximum likelihood model of optimal integration.

Previous studies have revealed that interruption induces disruptive influences on the performance of cognitive tasks. While much research has focused on the use of multimodal channels to reduce the cost of interruption, few studies have utilized haptic information as more than an associative cue. In the present study, we utilized a multimodal task interruption scenario involving the simultaneous presentation of visual information and haptic stimuli in order to investigate how the combined stimuli affect performance on the primary task (cost of interruption). Participants were asked to perform a two-back visuo-tactile task, in which visual and haptic stimuli were presented simultaneously, which was interrupted by a secondary task that also utilized visual and haptic stimuli. Four experimental conditions were evaluated: (1) paired information (visual stimulus + paired haptic stimulus) with interruption; (2) paired information without interruption; (3) non-paired information (visual stimulus + non-paired haptic stimulus) with interruption; and (4) non-paired information without interruption. Our findings indicate that, within a visuo-tactile task environment, redundant haptic information may not only increase accuracy on the primary task but also reduce the cost of interruption in terms of accuracy. These results suggest a new way of understanding the task recovery process within a multimodal environment.

An intriguing paper by Guterstam et al. (2016) has recently shown a new perceptual illusion, namely the “magnetic touch illusion.” It is elicited by synchronous brushstrokes applied to a participant's hidden real hand and brushstrokes delivered at some distance above the rubber hand, but never actually touching it.

Recent research suggests visuo-tactile binding is temporally extended in autism spectrum disorders (ASD), although it is not clear whether this specifically underlies altered body representation in this population. In the current study children and adolescents with ASD, and typically developing controls, placed their hand into mediated reality system (MIRAGE) and saw two identical live video images of their own right hand. One image was in the proprioceptively correct location (veridical hand) and the other was displaced to either side. While visuo-tactile feedback was applied via brushstroke to the participant’s (unseen) right finger, they viewed one hand image receiving synchronous brushstrokes and the other receiving brushstrokes with a temporal delay (60, 180 and 300 ms). After brushing, both images disappeared from view and participants pointed to a target, with direction of movement indicating which hand was embodied. ASD participants, like younger mental aged-matched controls, showed reduced embodiment of the spatially incongruent, but temporally congruent, hand compared to chronologically age-matched controls at shorter temporal delays. This suggests development of visuo-tactile integration may be delayed in ASD. Findings are discussed in relation to atypical body representation in ASD and how this may contribute to social and sensory difficulties within this population. Keywords: Autism spectrum disorder, Temporal binding window, Visuo-tactile processing, Embodied action

A growing body of literature associates musculoskeletal disorders with cortical reorganisation. One condition in which reorganisation is established and treatments that 'train the brain' are being widely used is chronic back pain. Recent evidence suggests that treatments that involve tactile training are more effective if they incorporate multisensory mechanisms, most obviously vision. With regard to back pain however, we must first determine if tactile function is enhanced by incorporating other modalities. A series of three cross-over experiments were conducted in healthy pain-free subjects to determine whether tactile acuity is enhanced when participants can see the skin of their back during testing. An initial randomised cross-over experiment suggested tactile acuity was significantly enhanced when participants could see their backs (t(25) = similar to 4.226, p < 0.001, r = 0.65). However, a second replication experiment was not corroborative. Both the second (F(3,66) = 1.00, p = 0.398) and third (t(9) = 0.969, p = 0.358) experiments suggested that seeing the back did not significantly affect tactile acuity, confirming that our initial results were likely due to chance. The principle that visual feedback improves tactile acuity at the hand does not apply to the back. These results strongly suggest that attempts to enhance tactile training by incorporating vision will not offer the benefit to treatment of back pain that has been observed for treatment of hand pain. Refereed/Peer-reviewed

Primary somatosensory cortex (S1) is known to rapidly adapt to task demands and to intermodal information (e.g. from vision). Here we show that also intramodal information (i.e. posture) can affect tactile attentional selection processes and the intermodal effects of vision on those processes at S1 stages of processing. We manipulated the spatial separation between adjacent fingers; that is, thumb and index finger where either close, far apart or touching. Participants directed their attention to either the index finger or thumb to detect infrequent tactile targets at that location while they either saw their fingers or these were covered from view. In line with previous results we found that attentional selection affected early somatosensory processing (P45, N80) when fingers were near and this attention effect was abolished when fingers were viewed. When fingers were far or touching, attentional modulations appeared reliably only from the P100, and furthermore, enhanced tactile spatial selection was found when touching fingers were viewed. Taken together, these results show for the first time a profound effect of finger posture on attentional selection between fingers and its modulations by vision at early cortical stages of processing. They suggest that the adverse effects of vision on tactile attention are not driven by a conflict between the selected information in vision (two fingers) and touch (one finger), and imply that external spatial information (i.e. finger posture) rapidly affects the organisation of primary somatosensory finger representations and that this further affects vision and tactile spatial selection effects on S1.

Twech is a mobile platform that enables users to share visuo-tactile experience and search other experiences for tactile data. User can record and share visuo-tactile experiences by using a visuo-tactile recording and displaying attachment for smartphone, allows the user to instantly such as tweet, and re-experience shared data such as visuo-motor coupling. Further, Twech's search engine finds similar other experiences, which were scratched material surfaces, communicated with animals or other experiences, for uploaded tactile data by using search engine is based on deep learning that ware expanded for recognizing tactile materials. Twech provides a sharing and finding haptic experiences and users re-experience uploaded visual-tactile data from cloud server.

Two experiments investigated 5-month-old infants’ amodal sensitivity to numerical correspondences between sets of objects presented in the tactile and visual modes. A classical cross-modal transfer task from touch to vision was adopted. Infants were first tactually familiarized with two or three different objects presented one by one in their right hand. Then, they were presented with visual displays containing two or three objects. Visual displays were presented successively (Experiment 1) or simultaneously (Experiment 2). In both experiments, results showed that infants looked longer at the visual display which contained a different number of objects from the tactile familiarization phase. Taken together, the results revealed that infants can detect numerical correspondences between a sequence of tactile and visual stimulation, and they strengthen the hypothesis of amodal and abstract representation of small numbers of objects (two or three) across sensory modalities in 5-month-old infants.

The rubber hand illusion is one reliable way to experimentally manipulate the experience of body ownership. However, debate continues about the necessary and sufficient conditions eliciting the illusion. We measured proprioceptive drift and the subjective experience (via questionnaire) while manipulating two variables that have been suggested to affect the intensity of the illusion. First, the rubber hand was positioned either in a posturally congruent position, or rotated by 180°. Second, either the anatomically same rubber hand was used, or an anatomically incongruent one. We found in two independent experiments that a rubber hand rotated by 180° leads to increased proprioceptive drift during synchronous visuo-tactile stroking, although it does not lead to feelings of ownership (as measured by questionnaire). This dissociation between drift and ownership suggests that proprioceptive drift is not necessarily a valid proxy for the illusion when using hands rotated by 180°.

Perceptual systems can distinguish among a variety of inputs in the temporal domain, including even different sensory inputs. This process has been investigated mainly by using a temporal task (temporal order judgment: TOJ). For example, studies have reported estimated critical limits (just noticeable difference: JND) of the TOJ between a visual stimulus and a tactile stimulus (visuo–tactile TOJ, e.g., flashes and vibrations) fell within a certain temporal range. Recent studies have also suggested that the visual presentation of a hand image could modulate visuo–tactile integrations in the temporal domain, but these studies did not thoroughly examine such effects by using temporal tasks. Here, we investigated the effect of visual presentation of a hand image on visuo–tactile TOJ. In our experiments, a visual stimulus was presented on the index finger of a hand image and a tactile stimulus was presented on the index finger of a participant’s hand. We found that the JND of visuo–tactile TOJ became larger when a forward hand image was presented than when inverted hand or arrow images were presented. However, this effect was not observed for the TOJ between an auditory stimulus and a visual stimulus. Thus, the visual presentation of a hand image whose angle corresponds to that of one’s own hand could selectively degrade visuo–tactile TOJ. This finding indicates that visual hand images implicitly enhance the internal proximity between the visual and tactile stimuli and make them difficult to distinguish from each other in the temporal domain.

The paper describes an application based on low cost Virtual Reality technologies whose aim is to help designers in testing some functional aspects of their product concepts without the need of building physical mockups. In particular the application allows the designers to test aspects related to the usage of products. A case study that enables designer to verify the ergonomics aspects of the interactive Virtual Prototype (iVP) of a commercial refrigerator has been implemented. The iVP is based on a three-dimensional scale representation of the product and the use of a low cost user interface as input/output device. The paper describes the implementation of the iVP and the results of some preliminary user study that has been performed in order to validate the visuo/haptic interaction approach.

Multisensory integration involves bottom-up as well as top-down processes. We investigated the influences of top-down control on the neural responses to multisensory stimulation using EEG recording and time-frequency analyses. Participants were stimulated at the index or thumb of the left hand, using tactile vibrators mounted on a foam cube. Simultaneously they received a visual distractor from a light emitting diode adjacent to the active vibrator (spatially congruent trial) or adjacent to the inactive vibrator (spatially incongruent trial). The task was to respond to the elevation of the tactile stimulus (upper or lower), while ignoring the simultaneous visual distractor. To manipulate top-down control on this multisensory stimulation, the proportion of spatially congruent (vs. incongruent) trials was changed across blocks. Our results reveal that the behavioral cost of responding to incongruent than congruent trials (i.e., the crossmodal congruency effect) was modulated by the proportion of congruent trials. Most importantly, the EEG gamma band response and the gamma-theta coupling were also affected by this modulation of top-down control, whereas the late theta band response related to the congruency effect was not. These findings suggest that gamma band response is more than a marker of multisensory binding, being also sensitive to the correspondence between expected and actual multisensory stimulation. By contrast, theta band response was affected by congruency but appears to be largely immune to stimulation expectancy.

Visuo-tactile integration occurs in a privileged way in peripersonal space, namely when visual and tactile stimuli are in spatial proximity. Here, we investigated whether crossmodal spatial effects (i.e. stronger crossmodal interactions for spatially congruent compared to incongruent visual and tactile stimuli) are also present when visual stimuli presented near the body are indirectly viewed in a mirror, thus appearing in far space. Participants had to attend to one of their hands throughout a block of stimuli in order to detect infrequent tactile target stimuli at that hand while ignoring tactile targets at the unattended hand, all tactile non-target stimuli, and any visual stimuli. Visual stimuli were presented simultaneously with tactile stimuli, in the same (congruent) or opposite (incongruent) hemispace with respect to the tactile stimuli. In one group of participants the visual stimuli were delivered near the participants’ hands and were observed as indirect mirror reflections (‘mirror’ condition), while in the other group these were presented at a distance from the hands (‘far’ condition). The main finding was that crossmodal spatial modulations of ERPs recorded over and close to somatosensory cortex were present in the ‘mirror’ condition but not the ‘far’ condition. That is, ERPs were enhanced in response to tactile stimuli coupled with spatially congruent versus incongruent visual stimuli when the latter were viewed through a mirror. These effects emerged around 190 ms after stimuli onset, and were modulated by the focus of spatial attention. These results provide evidence that visual stimuli observed in far space via a mirror are coded as near-thebody stimuli according to their known rather than to their perceived location. This suggests that crossmodal interactions between vision and touch may be modulated by previous knowledge of reflecting surfaces (i.e. top-down processing).

An important aim in cognitive neuroscience is to identify the networks connecting different brain areas and their role in executing complex tasks. In this study, visuo-tactile tasks were employed to assess the functional correlation underlying the cooperation process between visual and tactile regions. MEG data were recorded from eight healthy subjects while performing a visual, a tactile, and a visuo-tactile task. To define regions of interest (ROIs), event-related fields (ERFs) were estimated from MEG data related to visual and tactile areas. The ten channels with the highest increase in ERF variance, moving from rest to task, were selected. Cross-spectral analysis was then performed to assess potential changes in the activity of the involved regions and quantify the coupling between visual and tactile ROIs. A significant decrease (p

Gestalt rules that describe how visual stimuli are grouped also apply to sounds, but it is unknown if the Gestalt rules also apply to tactile or uniquely multimodal stimuli. To investigate these rules, we used lights, touches, and a combination of lights and touches, arranged in a classic Ternus configuration. Three stimuli (A, B, C) were arranged in a row across three fingers. A and B were presented for 50 ms and, after a delay, B and C were presented for 50 ms. Subjects were asked whether they perceived AB moving to BC (group motion) or A moving to C (element motion). For all three types of stimuli, at short delays, A to C dominated, while at longer delays AB to BC dominated. The critical delay, where perception changed from group to element motion, was significantly different for the visual Ternus (3 lights, 162 ms) and the tactile Ternus (3 touches, 195 ms). The critical delay for the multimodal Ternus (3 light – touch pairs, 161 ms) was not different from the visual or tactile Ternus effects. In a second experiment, subjects were exposed to 2.5 min of visual group motion (stimulus onset asynchrony = 300 ms). The exposure caused a shift in the critical delay of the visual Ternus, a trend in the same direction for the multimodal Ternus, but no shift in the tactile Ternus. These results suggest separate but similar grouping rules for visual, tactile, and multimodal stimuli.

In this study we aimed to explore the predictive link between visual stimuli moving towards the body and the tactile consequences that follow. More specifically, we tested if information derived from an approaching visual stimulus in the region directly surrounding the body (the peripersonal space) could be used to make judgments about the location and time of impending tactile contact. We used moving arm stimuli, displayed on a computer screen, which appeared to travel either towards the face (middle of the left/right cheek) or slightly a000354596900036way from the subject's face. This stimulus was followed by tactile stimulation of the left/right cheek. The time lag between the visual stimulus and tactile stimulation was also manipulated to simulate tactile contact at a time that was either consistent or inconsistent with the speed of the approaching hand. Reaction time information indicated that faster responses were produced when the arm moved towards the hemispace in which the tactile stimulation was delivered and was insensitive to whether the arm was moving towards the cheek or slightly away from the cheek. Furthermore, response times were fastest when the tactile stimulation arrived at the moment that was consistent with the speed of the moving arm. The effects disappeared when the arm appeared to be retracting from the subject's face. These results suggest the existence of a predictive mechanism that exploits the visual information derived from objects moving towards the body for making judgments about the time and location of impending tactile contact.

In this study, the spatial limits of referred touch to a rubber hand were investigated. Participants rated the strength of the perceived illusion when the rubber hand was placed in one of six different spatial positions (at a distance of 17.5–67.5 cm horizontal from the participant’s own hand). The results revealed a significant nonlinear relationship in the strength of the illusion, with the strongest ratings given when the two hands were closest; decaying significantly after a distance of 30 cm. The time taken to elicit the illusion followed a similar trend. These results may reflect the response properties of bimodal visuo-tactile cells encoding peripersonal space around the hand.

Spatial navigation within a real 3-D maze was investigated to study space perception on the sole basis of tactile information transmitted by means of a ‘tactile vision substitution system' (TVSS) allowing the conversion of optical images—collected by a micro camera—into ‘tactile images’ via a matrix in contact with the skin. The development of such a device is based on concepts of cerebral and functional plasticity, enabling subjective reproduction of visual images from tactile data processing. Blindfolded sighted subjects had to remotely control the movements of a robot on which the TVSS camera was mounted. Once familiarised with the cues in the maze, the subjects were given two exploration sessions. Performance was analysed according to an objective point of view (exploration time, discrimination capacity), as well as a subjective one (speech). The task was successfully carried out from the very first session. As the subjects took a different path during each navigation, a gradual improvement in performance (discrimination and exploration time) was noted, generating a phenomenon of learning. Moreover, subjective analysis revealed an evolution of the spatialisation process towards distal attribution. Finally, some emotional expressions seemed to reflect the genesis of ‘qualia’ (emotional qualities of stimulation).