Search

The Nash equilibrium concept has previously been shown to be an important tool to understand human sensorimotor interactions, where different actors vie for minimizing their respective effort while engaging in a multi-agent motor task. However, it is not clear how such equilibria are reached. Here, we compare different reinforcement learning models to human behavior engaged in sensorimotor interactions with haptic feedback based on three classic games, including the prisoner’s dilemma, and the symmetric and asymmetric matching pennies games. We find that a discrete analysis that reduces the continuous sensorimotor interaction to binary choices as in classical matrix games does not allow to distinguish between the different learning algorithms, but that a more detailed continuous analysis with continuous formulations of the learning algorithms and the game-theoretic solutions affords different predictions. In particular, we find that Q-learning with intrinsic costs that disfavor deviations from average behavior explains the observed data best, even though all learning algorithms equally converge to admissible Nash equilibrium solutions. We therefore conclude that it is important to study different learning algorithms for understanding sensorimotor interactions, as such behavior cannot be inferred from a game-theoretic analysis alone, that simply focuses on the Nash equilibrium concept, as different learning algorithms impose preferences on the set of possible equilibrium solutions due to the inherent learning dynamics.

Artificially created tactile feedback is in high demand due to fast developments in robotics, remote control in medicine, virtual reality, and smart electronics. Despite significant progress, high-quality haptic feedback devices remain challenging mainly due to the lack of stability and spatiotemporal resolution. In this work, we address these issues by the application of dynamic coatings, based on photo-responsive liquid crystal network (LCN) material. This material adapts upon an external stimulus (UV light with a power intensity of 50–90 mW/cm2) that changes its elastic properties (87% decrease of the modulus for 90 mW/cm2 power intensity of 365 nm UV light). Localized change of adaptive modulus with very high resolution (2 μm) was demonstrated.

The use of haptic technologies in modern life scenarios is becoming the new normal particularly in rehabilitation, medical training, and entertainment applications. An evident challenge in haptic telepresence systems is the delay in haptic information. How humans perceive delayed visual and audio information has been extensively studied, however, the same for haptically delayed environments remains largely unknown. Here, we develop a visuo-haptic experimental setting that simulates pick and place task and involves continuous haptic feedback stimulation with four possible haptic delay levels. The setting is built using a haptic device and a computer screen. We use electroencephalography (EEG) to study the neural correlates that could be used to identify the amount of the experienced haptic delay. EEG data were collected from 34 participants. Results revealed that midfrontal theta oscillation plays a pivotal role in quantifying the amount of haptic delay while parietal alpha showed a significant modulation that encodes the presence of haptic delay. Based on the available literature, these results suggest that the amount of haptic delay is proportional to the neural activation that is associated with conflict detection and resolution as well as for multi-sensory divided attention.

Dashboard-mounted touchscreen tablets are now common in vehicles. Screen/phone use in cars likely shifts drivers' attention away from the road and contributes to risk of accidents. Nevertheless, vision is subject to multisensory influences from other senses. Haptics may help maintain or even increase visual attention to the road, while still allowing for reliable dashboard control. Here, we provide a proof-of-concept for the effectiveness of digital haptic technologies (hereafter digital haptics), which use ultrasonic vibrations on a tablet screen to render haptic perceptions. Healthy human participants (N = 25) completed a divided-attention paradigm. The primary task was a centrally-presented visual conjunction search task, and the secondary task entailed control of laterally-presented sliders on the tablet. Sliders were presented visually, haptically, or visuo-haptically and were vertical, horizontal or circular. We reasoned that the primary task would be performed best when the secondary task was haptic-only. Reaction times (RTs) on the visual search task were fastest when the tablet task was haptic-only. This was not due to a speed-accuracy trade-off; there was no evidence for modulation of VST accuracy according to modality of the tablet task. These results provide the first quantitative support for introducing digital haptics into vehicle and similar contexts.

Haptic exploration produces mental object representations that can be memorized for subsequent object-directed behaviour. Storage of haptically-acquired object images (HOIs), engages, besides canonical somatosensory areas, the early visual cortex (EVC). Clear evidence for a causal contribution of EVC to HOI representation is still lacking. The use of visual information by the grasping system undergoes necessarily a frame of reference shift by integrating eye-position. We hypothesize that if the motor system uses HOIs stored in a retinotopic coding in the visual cortex, then its use is likely to depend at least in part on eye position. We measured the kinematics of 4 fingers in the right hand of 15 healthy participants during the task of grasping different unseen objects behind an opaque panel, that had been previously explored haptically. The participants never saw the object and operated exclusively based on haptic information. The position of the object was fixed, in front of the participant, but the subject’s gaze varied from trial to trial between 3 possible positions, towards the unseen object or away from it, on either side. Results showed that the middle and little fingers’ kinematics during reaching for the unseen object changed significantly according to gaze position. In a control experiment we showed that intransitive hand movements were not modulated by gaze direction. Manipulating eye-position produces small but significant configuration errors, (behavioural errors due to shifts in frame of reference) possibly related to an eye-centered frame of reference, despite the absence of visual information, indicating sharing of resources between the haptic and the visual/oculomotor system to delayed haptic grasping.

In this paper, a bilateral haptic virtual surgery simulation system under a hybrid controller was studied. An analogue controller realized by a field programmable analogue array (FPAA) was paralleled in the operator robot side, which reduced the impact of controller discretisation on the system. A system stability conditions under hybrid control with multiple-operators were deduced. The stability analysis indicates that the addition of analogue derivative term widens the range of haptic controls gains that satisfy the multiple-users’ stability conditions. Finally, the human’s performance of a stiffness discrimination task was studied in an independently developed minimally invasive surgical (MIS) platform. The experiment results show that, human operators under the hybrid controller achieve the highest task success rates.

Loss of tactile sensations is a major roadblock preventing upper limb-absent people from multitasking or using the full dexterity of their prosthetic hands. With current myoelectric prosthetic hands, limb-absent people can only control one grasp function at a time even though modern artificial hands are mechanically capable of individual control of all five digits. In this paper, we investigated whether people could precisely control the grip forces applied to two different objects grasped simultaneously with a dexterous artificial hand. Toward that end, we developed a novel multichannel wearable soft robotic armband to convey artificial sensations of touch to the robotic hand users. Multiple channels of haptic feedback enabled subjects to successfully grasp and transport two objects simultaneously with the dexterous artificial hand without breaking or dropping them, even when their vision of both objects was obstructed. Simultaneous transport of the objects provided a significant time savings to perform the deliveries in comparison to a one-at-a-time approach. This paper demonstrated that subjects were able to integrate multiple channels of haptic feedback into their motor control strategies to perform a complex simultaneous object grasp control task with an artificial limb, which could serve as a paradigm shift in the way prosthetic hands are operated.

While the nervous system can coordinate muscles’ activation to shape the mechanical interaction with the environment, it is unclear if and how the arm’s coactivation influences visuo-haptic perception and motion planning. Here we show that the nervous system can voluntarily coactivate muscles to improve the quality of the haptic percept. Subjects tracked a randomly moving visual target they were physically coupled to through a virtual elastic band, where the stiffness of the coupling increased with wrist coactivation. Subjects initially relied on vision alone to track the target, but with practice they learned to combine the visual and haptic percepts in a Bayesian manner to improve their tracking performance. This improvement cannot be explained by the stronger mechanical guidance from the elastic band. These results suggest that with practice the nervous system can learn to integrate a novel haptic percept with vision in an optimal fashion.

A surface texture is perceived through both the sound and vibrations produced while being explored by our fingers. Because of their common origin, both modalities have a strong influence on each other, particularly at above 60 Hz for which vibrotactile perception and pitch perception share common neural processes. However, whether the sensation of rhythm is shared between audio and haptic perception is still an open question. In this study, we show striking similarities between the audio and haptic perception of rhythmic changes, and demonstrate the interaction of both modalities below 60 Hz. Using a new surface-haptic device to synthesize arbitrary audio-haptic textures, psychophysical experiments demonstrate that the perception threshold curves of audio and haptic rhythmic gradients are the same. Moreover, multimodal integration occurs when audio and haptic rhythmic gradients are congruent. We propose a multimodal model of rhythm perception to explain these observations. These findings suggest that audio and haptic signals are likely to be processed by common neural mechanisms also for the perception of rhythm. They provide a framework for audio-haptic stimulus generation that is beneficial for nonverbal communication or modern human-machine interfaces.

As touch screen technologies advanced, a digital stylus has become one of the essential accessories for a smart device. However, most of the digital styluses so far provide limited tactile feedback to a user. Therefore we focused on the limitation and noted the potential that a digital stylus may offer the sensation of realistic interaction with virtual environments on a touch screen using a 2.5D haptic system. Thus, we developed a haptic stylus with SMA (Shape Memory Alloy) and a 2.5D haptic rendering algorithm to provide lateral skin-stretch feedback to mimic the interaction force between fingertip and a stylus probing over a bumpy surface. We conducted two psychophysical experiments to evaluate the effect of 2.5D haptic feedback on the perception of virtual object geometry. Experiment 1 investigated the human perception of virtual bump size felt via the proposed lateral skin-stretch stylus and a vibrotactile stylus as reference. Experiment 2 tested the participants’ ability to count the number of virtual bumps rendered via the two types of haptic styluses. The results of Experiment 1 indicate that the participants felt the size of virtual bumps rendered with lateral skin-stretch stylus significantly sensitively than the vibrotactile stylus. Similarly, the participants counted the number of virtual bumps rendered with the lateral skin-stretch stylus significantly better than with the vibrotactile stylus. A common result of the two experiments is a significantly longer mean trial time for the skin-stretch stylus than the vibrotactile stylus.

Haptic technologies aim to simulate tactile or kinesthetic interactions with a physical or virtual environment in order to enhance user experience and/or performance. However, due to stringent communication and computational needs, the user experience is influenced by delayed haptic feedback. While delayed feedback is well understood in the visual and auditory modalities, little research has systematically examined the neural correlates associated with delayed haptic feedback. In this paper, we used electroencephalography (EEG) to study sensory and cognitive neural correlates caused by haptic delay during passive and active tasks performed using a haptic device and a computer screen. Results revealed that theta power oscillation was significantly higher at the midfrontal cortex under the presence of haptic delay. Sensory correlates represented by beta rebound were found to be similar in the passive task and different in the active task under the delayed and synchronous conditions. Additionally, the event related potential (ERP) P200 component is modulated under the haptic delay condition during the passive task. The P200 amplitude significantly reduced in the last 20% of trials during the passive task and in the absence of haptic delay. Results suggest that haptic delay could be associated with increased cognitive control processes including multi-sensory divided attention followed by conflict detection and resolution with an earlier detection during the active task. Additionally, haptic delay tends to generate greater perceptual attention that does not significantly decay across trials during the passive task.

Limb amputation not only reduces the motor abilities of an individual, but also destroys afferent channels that convey essential sensory information to the brain. Significant efforts have been made in the area of upper limb prosthetics to restore sensory feedback, through the stimulation of residual sensory elements. Most of the past research focused on the replacement of tactile functions. On the other hand, the difficulties in eliciting proprioceptive sensations using either haptic or (neural) electrical stimulation, has limited researchers to rely on sensory substitution. Here we propose the myokinetic stimulation interface, that aims at restoring natural proprioceptive sensations by exploiting the so-called tendon illusion, elicited through the vibration of magnets implanted inside residual muscles. We present a prototype which exploits 12 electromagnetic coils to vibrate up to four magnets implanted in a forearm mockup. The results demonstrated that it is possible to generate highly directional and frequency-selective vibrations. The system proved capable of activating a single magnet, out of many. Hence, this interface constitutes a promising approach to restore naturally perceived proprioception after an amputation. Indeed, by implanting several magnets in independent muscles, it would be possible to restore proprioceptive sensations perceived as coming from single digits.

Accurate localization of touch requires the integration of two reference frames—an internal (e.g., anatomical) and an external (e.g., spatial). Using a tactile temporal order judgement task with the hands crossed over the midline, we investigated the integration of these two reference frames. We manipulated the reliability of the visual and vestibular information, both of which contribute to the external reference frame. Visual information was manipulated between experiments (Experiment 1 was done with full vision and Experiment 2 was done while wearing a blindfold). Vestibular information was manipulated in both experiments by having the two groups of participants complete the task in both an upright posture and one where they were lying down on their side. Using a Bayesian hierarchical model, we estimated the perceptual weight applied to these reference frames. Lying participants on their side reduced the weight applied to the external reference frame and produced a smaller deficit; blindfolding resulted in similar reductions. These findings reinforce the importance of the visual system when weighting tactile reference frames, and highlight the importance of the vestibular system in this integration.

Cortical body size representations are distorted in the adult, from low-level motor and sensory maps to higher levels multisensory and cognitive representations. Little is known about how such representations are built and evolve during infancy and childhood. Here we investigated how hand size is represented in typically developing children aged 6 to 10. Participants were asked to estimate their hand size using two different sensory modalities (visual or haptic). We found a distortion (underestimation) already present in the youngest children. Crucially, such distortion increases with age and regardless of the sensory modality used to access the representation. Finally, underestimation is specific for the body as no bias was found for object estimation. This study suggests that the brain does not keep up with the natural body growth. However, since motor behavior nor perception were impaired, the distortion seems functional and/or compensated for, for proper interaction with the external environment.

Responses elicited after the shortest latency spinal reflexes but prior to the onset of voluntary activity can display sophistication beyond a stereotypical reflex. Two distinct behaviors have been identified for these rapid motor responses, often called long-latency reflexes. The first is to maintain limb stability by opposing external perturbations. The second is to quickly release motor actions planned prior to the disturbance, often called a triggered reaction. This study investigated their interaction when motor tasks involve both limb stabilization and motor planning. We used a robotic manipulator to change the stability of the haptic environment during 2D arm reaching tasks, and to apply perturbations that could elicit rapid motor responses. Stabilizing reflexes were modulated by the orientation of the haptic environment (field effect) whereas triggered reactions were modulated by the target to which subjects were instructed to reach (target effect). We observed that there were no significant interactions between the target and field effects in the early (50–75 ms) portion of the long-latency reflex, indicating that these components of the rapid motor response are initially controlled independently. There were small but significant interactions for two of the six relevant muscles in the later portion (75–100 ms) of the reflex response. In addition, the target effect was influenced by the direction of the perturbation used to elicit the motor response, indicating a later feedback correction in addition to the early component of the triggered reaction. Together, these results demonstrate how distinct components of the long-latency reflex can work independently and together to generate sophisticated rapid motor responses that integrate planning with reaction to uncertain conditions.

Hearing aid and cochlear implant (CI) users often struggle to locate and segregate sounds. The dominant sound-localisation cues are time and intensity differences across the ears. A recent study showed that CI users locate sounds substantially better when these cues are provided through haptic stimulation on each wrist. However, the sensitivity of the wrists to these cues and the robustness of this sensitivity to aging is unknown. The current study showed that time difference sensitivity is much poorer across the wrists than across the ears and declines with age. In contrast, high sensitivity to across-wrist intensity differences was found that was robust to aging. This high sensitivity was observed across a range of stimulation intensities for both amplitude modulated and unmodulated sinusoids and matched across-ear intensity difference sensitivity for normal-hearing individuals. Furthermore, the usable dynamic range for haptic stimulation on the wrists was found to be around four times larger than for CIs. These findings suggest that high-precision haptic sound-localisation can be achieved, which could aid many hearing-impaired listeners. Furthermore, the finding that high-fidelity across-wrist intensity information can be transferred could be exploited in human–machine interfaces to enhance virtual reality and improve remote control of military, medical, or research robots.

Group decisions can outperform the choices of the best individual group members. Previous research suggested that optimal group decisions require individuals to communicate explicitly (e.g., verbally) their confidence levels. Our study addresses the untested hypothesis that implicit communication using a sensorimotor channel -- haptic coupling -- may afford optimal group decisions, too. We report that haptically coupled dyads solve a perceptual discrimination task more accurately than their best individual members; and five times faster than dyads using explicit communication. Furthermore, our computational analyses indicate that the haptic channel affords implicit confidence sharing. We found that dyads take leadership over the choice and communicate their confidence in it by modulating both the timing and the force of their movements. Our findings may pave the way to negotiation technologies using fast sensorimotor communication to solve problems in groups., 13 pages, 3 figures, preprint

Seventy-two older and younger adults haptically discriminated the solid shape of natural objects (bell peppers, Capsicum annuum). Plastic copies of the original-sized fruits were used as experimental stimuli, as well as copies that were reduced in size to 1/8th and 1/27th of the original object volumes. If haptic object shape is represented in a part-based manner, then haptic shape discrimination performance should be at least partly size invariant, since changes only in scale do not affect an object’s constituent parts. On any given trial, participants sequentially explored two bell pepper replicas and were required to judge whether they possessed the same shape or had different shapes. For some participants, the objects to be discriminated possessed the same size, while for others, the two objects had different sizes. It was found that variations in scale did significantly reduce the participants’ haptic sensitivities to shape. Nevertheless, the discrimination performance obtained for large variations in size was no lower than that obtained for smaller variations in size. The results also demonstrated that increases in age modestly affect haptic shape discrimination performance: the d′ values of the older participants were 15.5% lower than those of the younger participants.

Hemispheric asymmetry in hand preference for passive cutaneous perception compared to active haptic perception is not well known. A functional near-infrared spectroscopy was used to evaluate the laterality of cortical facilitation when 31 normal right-handed participants were involved in 205 Hz passive vibrotactile cutaneous stimuli on their index fingers of preferred and less-preferred hand. Passive cutaneous perception resulted that preferred (right) hand stimulation was strongly leftward lateralized, whereas less-preferred (left) hand stimulation was less lateralized. This confirms that other manual haptic exploration studies described a higher hemispheric asymmetry in right-handers. Stronger cortical facilitation was found in the right primary somatosensory cortex (S1) and right somatosensory association area (SA) during left-hand stimulation but not right-hand stimulation. This finding suggests that the asymmetric activation in the S1 and SA for less-preferred (left) hand stimulation might contribute to considerably reinforce sensorimotor network just with passive vibrotactile cutaneous stimulation.

Users of hearing-assistive devices often struggle to locate and segregate sounds, which can make listening in schools, cafes, and busy workplaces extremely challenging. A recent study in unilaterally implanted CI users showed that sound-localisation was improved when the audio received by behind-the-ear devices was converted to haptic stimulation on each wrist. We built on this work, using a new signal-processing approach to improve localisation accuracy and increase generalisability to a wide range of stimuli. We aimed to: (1) improve haptic sound-localisation accuracy using a varied stimulus set and (2) assess whether accuracy improved with prolonged training. Thirty-two adults with normal touch perception were randomly assigned to an experimental or control group. The experimental group completed a 5-h training regime and the control group were not trained. Without training, haptic sound-localisation was substantially better than in previous work on haptic sound-localisation. It was also markedly better than sound-localisation by either unilaterally or bilaterally implanted CI users. After training, accuracy improved, becoming better than for sound-localisation by bilateral hearing-aid users. These findings suggest that a wrist-worn haptic device could be effective for improving spatial hearing for a range of hearing-impaired listeners.

The cochlear implant (CI) is the most widely used neuroprosthesis, recovering hearing for more than half a million severely-to-profoundly hearing-impaired people. However, CIs still have significant limitations, with users having severely impaired pitch perception. Pitch is critical to speech understanding (particularly in noise), to separating different sounds in complex acoustic environments, and to music enjoyment. In recent decades, researchers have attempted to overcome shortcomings in CIs by improving implant technology and surgical techniques, but with limited success. In the current study, we take a new approach of providing missing pitch information through haptic stimulation on the forearm, using our new mosaicOne_B device. The mosaicOne_B extracts pitch information in real-time and presents it via 12 motors that are arranged in ascending pitch along the forearm, with each motor representing a different pitch. In normal-hearing subjects listening to CI simulated audio, we showed that participants were able to discriminate pitch differences at a similar performance level to that achieved by normal-hearing listeners. Furthermore, the device was shown to be highly robust to background noise. This enhanced pitch discrimination has the potential to significantly improve music perception, speech recognition, and speech prosody perception in CI users.

Haptic interface technologies for virtual reality applications have been developed to increase the reality and manipulability of a virtual object by creating a diverse tactile sensation. Most evaluation of the haptic technologies, however, have been limited to the haptic perception of the tactile stimuli via static virtual objects. Noting this, we investigated the effect of lateral cutaneous feedback, along with kinesthetic feedback on the perception of virtual object weight during manipulation. We modeled the physical interaction between a participant’s finger avatars and virtual objects. The haptic stimuli were rendered with custom-built haptic feedback systems that can provide kinesthetic and lateral cutaneous feedback to the participant. We conducted two virtual object manipulation experiments, 1. a virtual object manipulation with one finger, and 2. the pull-out and lift-up of a virtual object grasped with a precision grip. The results of Experiment 1 indicate that the participants felt the virtual object rendered with lateral cutaneous feedback significantly heavier than with only kinesthetic feedback (p mref = 100 and 200 g). Similarly, the participants of Experiment 2 felt the virtual objects significantly heavier when lateral cutaneous feedback was available (p mref = 100, 200, and 300 g). Therefore, the additional lateral cutaneous feedback to the force feedback led the participants to feel the virtual object heavier than without the cutaneous feedback. The results also indicate that the contact force applied to a virtual object during manipulation can be a function of the perceived object weight (p = 0.005 for Experiment 1 and p = 0.02 for Experiment 2).

Extralegal factors may influence judicial outcomes. Here we investigated the experience of incidental haptic sensations on the harshness of punishment recommendations. Based on recent theories of embodiment, which claim that cognitive representations are structured by metaphorical mappings from sensory experience, we hypothesized that tactile priming with hard objects would cause subjects to recommend harsher sentences (to be ‘hard on crime’). Furthermore, the theory of embodiment predicts that this effect should be based on sensorimotor brain activation during the judging process. In order to test this we presented participants with scenarios that described various crimes while scanning their brain activity with fMRI. Participants were then asked to rate how severely they would sentence the delinquents. Before the scenarios, the participants were primed by touching either a hard or a soft object. Results revealed tha t hard priming led participants to recommend harder punishments. These results were accompanied by activation of somatosensory brain areas during the judging phase. This outcome is in line with simulation assumptions of the embodiment theory and proposes a central role of the sensorimotor cortices for embodied metaphors. Thus, incidental tactile experiences can influence our abstract cognitions and even how hard we are on criminals.

Changes in the retinal size of stationary objects provide a cue to the observer’s motion in the environment: Increases indicate the observer’s forward motion, and decreases backward motion. In this study, a series of images each comprising a pair of pine-tree figures were translated into auditory modality using sensory substitution software. Resulting auditory stimuli were presented in an ascending sequence (i.e. increasing in intensity and bandwidth compatible with forward motion), descending sequence (i.e. decreasing in intensity and bandwidth compatible with backward motion), or in a scrambled order. During the presentation of stimuli, blindfolded participants estimated the lengths of wooden sticks by haptics. Results showed that those exposed to the stimuli compatible with forward motion underestimated the lengths of the sticks. This consistent underestimation may share some aspects with visual size-contrast effects such as the Ebbinghaus illusion. In contrast, participants in the other two conditions did not show such magnitude of error in size estimation; which is consistent with the “adaptive perceptual bias” towards acoustic increases in intensity and bandwidth. In sum, we report a novel cross-modal size-contrast illusion, which reveals that auditory motion cues compatible with listeners’ forward motion modulate haptic representations of object size.

Cochlear implants (CIs) have enabled hundreds of thousands of profoundly hearing-impaired people to perceive sounds by electrically stimulating the auditory nerve. However, CI users are often very poor at locating sounds, which leads to impaired sound segregation and threat detection. We provided missing spatial hearing cues through haptic stimulation to augment the electrical CI signal. We found that this “electro-haptic” stimulation dramatically improved sound localisation. Furthermore, participants were able to effectively integrate spatial information transmitted through these two senses, performing better with combined audio and haptic stimulation than with either alone. Our haptic signal was presented to the wrists and could readily be delivered by a low-cost wearable device. This approach could provide a non-invasive means of improving outcomes for the vast majority of CI users who have only one implant, without the expense and risk of a second implantation.

Grasping movements are typically performed toward visually sensed objects. However, planning and execution of grasping movements can be supported also by haptic information when we grasp objects held in the other hand. In the present study we investigated this sensorimotor integration process by comparing grasping movements towards objects sensed through visual, haptic or visuo-haptic signals. When movements were based on haptic information only, hand preshaping was initiated earlier, the digits closed on the object more slowly, and the final phase was more cautious compared to movements based on only visual information. Importantly, the simultaneous availability of vision and haptics led to faster movements and to an overall decrease of the grip aperture. Our findings also show that each modality contributes to a different extent in different phases of the movement, with haptics being more crucial in the initial phases and vision being more important for the final on-line control. Thus, vision and haptics can be flexibly combined to optimize the execution of grasping movement.

Minimally invasive robotic surgery allows for many advantages over traditional surgical procedures, but the loss of force feedback combined with a potential for strong grasping forces can result in excessive tissue damage. Single modality haptic feedback systems have been designed and tested in an attempt to diminish grasping forces, but the results still fall short of natural performance. A multi-modal pneumatic feedback system was designed to allow for tactile, kinesthetic, and vibrotactile feedback, with the aims of more closely imitating natural touch and further improving the effectiveness of HFS in robotic surgical applications and tasks such as tissue grasping and manipulation. Testing of the multi-modal system yielded very promising results with an average force reduction of nearly 50% between the no feedback and hybrid (tactile and kinesthetic) trials (p

Haptic interaction between two humans, for example, a physiotherapist assisting a patient regaining the ability to grasp a cup, likely facilitates motor skill acquisition. Haptic human–human interaction has been shown to enhance individual performance improvement in a tracking task with a visuomotor rotation perturbation. These results are remarkable given that haptically assisting or guiding an individual rarely benefits their individual improvement when the assistance is removed. We, therefore, replicated a study that reported that haptic interaction between humans was beneficial for individual improvement for tracking a target in a visuomotor rotation perturbation. In addition, we tested the effect of more interaction time and a stronger haptic coupling between the partners on individual improvement in the same task. We found no benefits of haptic interaction on individual improvement compared to individuals who practised the task alone, independent of interaction time or interaction strength.

Direct cortical stimulation (DCS) of primary somatosensory cortex (S1) could help restore sensation and provide task-relevant feedback in a neuroprosthesis. However, the psychophysics of S1 DCS is poorly studied, including any comparison to cutaneous haptic stimulation. We compare the response times to DCS of human hand somatosensory cortex through electrocorticographic grids with response times to haptic stimuli delivered to the hand in four subjects. We found that subjects respond significantly slower to S1 DCS than to natural, haptic stimuli for a range of DCS train durations. Median response times for haptic stimulation varied from 198 ms to 313 ms, while median responses to reliably perceived DCS ranged from 254 ms for one subject, all the way to 528 ms for another. We discern no significant impact of learning or habituation through the analysis of blocked trials, and find no significant impact of cortical stimulation train duration on response times. Our results provide a realistic set of expectations for latencies with somatosensory DCS feedback for future neuroprosthetic applications and motivate the study of neural mechanisms underlying human perception of somatosensation via DCS.

Humans constantly combine multi-sensory spatial information to successfully interact with objects in peripersonal space. Previous studies suggest that sensory inputs of different modalities are encoded in different reference frames. In cross-modal tasks where the target and response modalities are different, it is unclear which reference frame these multiple sensory signals are transformed to for comparison. The current study used a slant perception and parallelity paradigm to explore this issue. Participants perceived (either visually or haptically) the slant of a reference board and were asked to either adjust an invisible test board by hand manipulation or to adjust a visible test board through verbal instructions to be physically parallel to the reference board. We examined the patterns of constant error and variability of unimodal and cross-modal tasks with various reference slant angles at different reference/test locations. The results revealed that rather than a mixture of the patterns of unimodal conditions, the pattern in cross-modal conditions depended almost entirely on the response modality and was not substantially affected by the target modality. Deviations in haptic response conditions could be predicted by the locations of the reference and test board, whereas the reference slant angle was an important predictor in visual response conditions.

Recently, various methods using, simultaneously, two types of tactile feedback have been proposed to emulate a real object. However, the possible masking effect when providing two types of tactile feedback has been scarcely reported. In this study, we investigated the masking effect caused by mechanical vibration on the perception of electrovibration. The absolute and difference thresholds of the electrovibration were measured according to the presence/absence, frequency, and intensity of the mechanical vibration. The absolute threshold of electrovibration tended to increase in the form of a ramp function, as the intensity of the masking stimulus (mechanical vibration) increased. Particularly, the masking effect was more remarkable when the frequency of both the target and the masking stimulus was the same (up to 13 dB increase with 25 dB SL masker). Furthermore, the difference in the threshold (average of 1.21 dB) did not significantly change due to the masking stimulus, when the sensation level intensity of the target stimulus was within the section following the Weber’s law. The results further indicated that electrovibration contributes to the activation of slowly adapting afferents as well. This investigation will provide important guidelines for the design of haptic interface that employs multiple types of tactile feedback.

It is reasonable to assume that when we grasp an object we carry out the movement based only on the currently available sensory information. Unfortunately, our senses are often prone to err. Here, we show that the visuomotor system exploits the mismatch between the predicted and sensory outcomes of the immediately preceding action (sensory prediction error) to attain a degree of robustness against the fallibility of our perceptual processes. Participants performed reach-to-grasp movements toward objects presented at eye level at various distances. Grip aperture was affected by the object distance, even though both visual feedback of the hand and haptic feedback were provided. Crucially, grip aperture as well as the trajectory of the hand were systematically influenced also by the immediately preceding action. These results are well predicted by a model that modifies an internal state of the visuomotor system by adjusting the visuomotor mapping based on the sensory prediction errors. In sum, the visuomotor system appears to be in a constant fine-tuning process which makes the generation and control of grasping movements more resistant to interferences caused by our perceptual errors.

In three experiments participants haptically discriminated object shape using unimanual (single hand explored two objects) and bimanual exploration (both hands were used, but each hand, left or right, explored a separate object). Such haptic exploration (one versus two hands) requires somatosensory processing in either only one or both cerebral hemispheres; previous studies related to the perception of shape/curvature found superior performance for unimanual exploration, indicating that shape comparison is more effective when only one hemisphere is utilized. The current results, obtained for naturally shaped solid objects (bell peppers, Capsicum annuum) and simple cylindrical surfaces demonstrate otherwise: bimanual haptic exploration can be as effective as unimanual exploration, showing that there is no necessary reduction in ability when haptic shape comparison requires interhemispheric communication. We found that while successive bimanual exploration produced high shape discriminability, the participants’ bimanual performance deteriorated for simultaneous shape comparisons. This outcome suggests that either interhemispheric interference or the need to attend to multiple objects simultaneously reduces shape discrimination ability. The current results also reveal a significant effect of age: older adults’ shape discrimination abilities are moderately reduced relative to younger adults, regardless of how objects are manipulated (left hand only, right hand only, or bimanual exploration).

This work presents an evaluation study using a force feedback evaluation framework for a novel direct needle force volume rendering concept in the context of liver puncture simulation. PTC/PTCD puncture interventions targeting the bile ducts have been selected to illustrate this concept. The haptic algorithms of the simulator system are based on (1) partially segmented patient image data and (2) a non-linear spring model effective at organ borders. The primary aim is to quantitatively evaluate force errors caused by our patient modeling approach, in comparison to haptic force output obtained from using gold-standard, completely manually-segmented data. The evaluation of the force algorithms compared to a force output from fully manually segmented gold-standard patient models, yields a low mean of 0.12 N root mean squared force error and up to 1.6 N for systematic maximum absolute errors. Force errors were evaluated on 31,222 preplanned test paths from 10 patients. Only twelve percent of the emitted forces along these paths were affected by errors. This is the first study evaluating haptic algorithms with deformable virtual patients in silico. We prove haptic rendering plausibility on a very high number of test paths. Important errors are below just noticeable differences for the hand-arm system., 15 pages, 16 figures, 1 tables, 11 equations, 39 references