Search

Abstract Introduction There is currently a lack of easy-to-use and effective robotic devices for upper-limb rehabilitation after stroke. Importantly, most current systems lack the provision of somatosensory information that is congruent with the virtual training task. This paper introduces a novel haptic robotic system designed for upper-limb rehabilitation, focusing on enhancing sensorimotor rehabilitation through comprehensive haptic rendering. Methods We developed a novel haptic rehabilitation device with a unique combination of degrees of freedom that allows the virtual training of functional reach and grasp tasks, where we use a physics engine-based haptic rendering method to render whole-hand interactions between the patients’ hands and virtual tangible objects. To evaluate the feasibility of our system, we performed a clinical mixed-method usability study with seven patients and seven therapists working in neurorehabilitation. We employed standardized questionnaires to gather quantitative data and performed semi-structured interviews with all participants to gain qualitative insights into the perceived usability and usefulness of our technological solution. Results The device demonstrated ease of use and adaptability to various hand sizes without extensive setup. Therapists and patients reported high satisfaction levels, with the system facilitating engaging and meaningful rehabilitation exercises. Participants provided notably positive feedback, particularly emphasizing the system’s available degrees of freedom and its haptic rendering capabilities. Therapists expressed confidence in the transferability of sensorimotor skills learned with our system to activities of daily living, although further investigation is needed to confirm this. Conclusion The novel haptic robotic system effectively supports upper-limb rehabilitation post-stroke, offering high-fidelity haptic feedback and engaging training tasks. Its clinical usability, combined with positive feedback from both therapists and patients, underscores its potential to enhance robotic neurorehabilitation.

Introduction: There is currently a lack of easy-to-use and effective robotic devices for upper-limb rehabilitation after stroke. Importantly, most current systems lack the provision of somatosensory information that is congruent with the virtual training task. This paper introduces a novel haptic robotic system designed for upper-limb rehabilitation, focusing on enhancing sensorimotor rehabilitation through comprehensive haptic rendering. Methods: We developed a novel haptic rehabilitation device with a unique combination of degrees of freedom that allows the virtual training of functional reach and grasp tasks, where we use a physics engine-based haptic rendering method to render whole-hand interactions between the patients' hands and virtual tangible objects. To evaluate the feasibility of our system, we performed a clinical mixed-method usability study with seven patients and seven therapists working in neurorehabilitation. We employed standardized questionnaires to gather quantitative data and performed semi-structured interviews with all participants to gain qualitative insights into the perceived usability and usefulness of our technological solution. Results: The device demonstrated ease of use and adaptability to various hand sizes without extensive setup. Therapists and patients reported high satisfaction levels, with the system facilitating engaging and meaningful rehabilitation exercises. Participants provided notably positive feedback, particularly emphasizing the system's available degrees of freedom and its haptic rendering capabilities. Therapists expressed confidence in the transferability of sensorimotor skills learned with our system to activities of daily living, although further investigation is needed to confirm this. Conclusion: The novel haptic robotic system effectively supports upper-limb rehabilitation post-stroke, offering high-fidelity haptic feedback and engaging training tasks. Its clinical usability, combined with positive feedback from both therapists and patients, underscores its potential to enhance robotic neurorehabilitation. [ABSTRACT FROM AUTHOR]

In the face of challenges encountered during femur fracture surgery, such as the high rates of malalignment and X-ray exposure to operating personnel, robot-assisted surgery has emerged as an alternative to conventional state-of-the-art surgical methods. This paper introduces the development of a leader-follower robot-assisted system for femur fracture surgery, called Robossis. Robossis comprises a 7-DOF haptic controller and a 6-DOF surgical robot. A control architecture is developed to address the kinematic mismatch and the motion transfer between the haptic controller and the Robossis surgical robot. A motion control pipeline is designed to address the motion transfer and evaluated through experimental testing. The analysis illustrates that the Robossis surgical robot can adhere to the desired trajectory from the haptic controller with an average translational error of 0.32 mm and a rotational error of 0.07°. Additionally, a haptic rendering pipeline is developed to resolve the kinematic mismatch by constraining the haptic controller's (user's hand) movement within the permissible joint limits of the Robossis surgical robot. Lastly, in a cadaveric lab test, the Robossis system was tested during a mock femur fracture surgery. The result shows that the Robossis system can provide an intuitive solution for surgeons to perform femur fracture surgery. [ABSTRACT FROM AUTHOR]

In current virtual reality settings for motor skill training, only visual information is usually provided regarding the virtual objects the trainee interacts with. However, information gathered through cutaneous (tactile feedback) and muscle mechanoreceptors (kinesthetic feedback) regarding, e.g., object shape, is crucial to successfully interact with those objects. To provide this essential information, previous haptic interfaces have targeted to render either tactile or kinesthetic feedback while the effectiveness of multimodal tactile and kinesthetic feedback on the perception of the characteristics of virtual objects still remains largely unexplored. Here, we present the results from an experiment we conducted with sixteen participants to evaluate the effectiveness of multimodal tactile and kinesthetic feedback on shape perception. Using a within-subject design, participants were asked to reproduce virtual shapes after exploring them without visual feedback and with either congruent tactile and kinesthetic feedback or with only kinesthetic feedback. Tactile feedback was provided with a cable-driven platform mounted on the fingertip, while kinesthetic feedback was provided using a haptic glove. To measure the participants' ability to perceive and reproduce the rendered shapes, we measured the time participants spent exploring and reproducing the shapes and the error between the rendered and reproduced shapes after exploration. Furthermore, we assessed the participants' workload and motivation using well-established questionnaires. We found that concurrent tactile and kinesthetic feedback during shape exploration resulted in lower reproduction errors and longer reproduction times. The longer reproduction times for the combined condition may indicate that participants could learn the shapes better and, thus, were more careful when reproducing them. We did not find differences between conditions in the time spent exploring the shapes or the participants' workload and motivation. The lack of differences in workload between conditions could be attributed to the reported minimal-to-intermediate workload levels, suggesting that there was little room to further reduce the workload. Our work highlights the potential advantages of multimodal congruent tactile and kinesthetic feedback when interacting with tangible virtual objects with applications in virtual simulators for hands-on training applications. [ABSTRACT FROM AUTHOR]

In stroke rehabilitation, simple robotic devices hold the potential to increase the training dosage in group therapies and to enable continued therapy at home after hospital discharge. However, we identified a lack of portable and cost-e [ABSTRACT FROM AUTHOR]

In the present study, we investigate the effect of rendering virtual vibrotactile motion to the perception of lateral force during planar sweeping motion. The virtual vibrotactile motion was rendered by an algorithm to create the sensation of resistive lateral force utilizing illusory haptic effects. The direction of the virtual vibrotactile motion was the opposite of the hand-sweeping motion to create the sensation of resistive force. We conducted two experiments that mapped the lateral resistive haptic feedback rendered by the virtual vibrotactile motion and force feedback to the perceived force magnitude. In Experiment 1, the test was conducted for three reference stimulus force and two maximum signal intensities. The results indicate significant effect of the two experimental parameters. The perceived lateral force was significantly larger with the virtual vibrotactile motion than the force feedback only. Also, the increase in the maximum signal intensity led to a larger perceived lateral force. Experiment 2 tested the effect of vibrotactile signal envelope function on the perceived lateral force by conducting a comparative experiment for linear and logarithmic envelope functions. The experimental results indicate a significantly larger perceived lateral force for the logarithmic signal envelope function than the linear signal envelope function. Overall, this study suggests that rendering virtual vibrotactile motion at the fingertip during swiping motion can create the sensation of additive lateral force and that the perceived intensity can be controlled by modulating the vibrotactile signal intensity and the signal envelope functions.

In this paper, a method for creating texture models with physical significance using a parametric equalizer (PEQ)-based approach is presented. The model is based on the frequency, amplitude, bandwidth, and noise ratio corresponding to texture profiles. Additionally, the method simplifies the modeling process by enabling the selective omission of imperceptible roughness layers based on human tactile thresholds. The effectiveness of the simplified model was validated through subjective evaluations comprising both absolute and relative assessments. The results confirmed that layers with imperceptible roughness could be excluded without compromising perceived similarity, streamlining the texture modeling process. Regression analysis revealed that PEQ parameters reflect physical interactions in tactile sensations, such as the relationship between texture roughness and vibration amplitude. This study contributes to haptic texture modeling by offering a method that efficiently reproduces actual textures and mirrors the fundamental physical principles of touch. The findings hold promise for applications in texture authoring and material selection, indicating potential advancements in the development of more intuitive and physically transparent texture models.

In stroke rehabilitation, simple robotic devices hold the potential to increase the training dosage in group therapies and to enable continued therapy at home after hospital discharge. However, we identified a lack of portable and cost-effective devices that not only focus on improving motor functions but also address sensory deficits. Thus, we designed a minimally-actuated hand training device that incorporates active grasping movements and passive pronosupination, complemented by a rehabilitative game with meaningful haptic feedback. Following a human-centered design approach, we conducted a usability study with 13 healthy participants, including three therapists. In a simulated unsupervised environment, the naive participants had to set up and use the device based on written instructions. Our mixed-methods approach included quantitative data from performance metrics, standardized questionnaires, and eye tracking, alongside qualitative feedback from semi-structured interviews. The study results highlighted the device's overall ease of setup and use, as well as its realistic haptic feedback. The eye-tracking analysis further suggested that participants felt safe during usage. Moreover, the study provided crucial insights for future improvements such as a more intuitive and comfortable wrist fixation, more natural pronosupination movements, and easier-to-follow instructions. Our research underscores the importance of continuous testing in the development process and offers significant contributions to the design of user-friendly, unsupervised neurorehabilitation technologies to improve sensorimotor stroke rehabilitation.

In current virtual reality settings for motor skill training, only visual information is usually provided regarding the virtual objects the trainee interacts with. However, information gathered through cutaneous (tactile feedback) and muscle mechanoreceptors (kinesthetic feedback) regarding, e.g., object shape, is crucial to successfully interact with those objects. To provide this essential information, previous haptic interfaces have targeted to render either tactile or kinesthetic feedback while the effectiveness of multimodal tactile and kinesthetic feedback on the perception of the characteristics of virtual objects still remains largely unexplored. Here, we present the results from an experiment we conducted with sixteen participants to evaluate the effectiveness of multimodal tactile and kinesthetic feedback on shape perception. Using a within-subject design, participants were asked to reproduce virtual shapes after exploring them without visual feedback and with either congruent tactile and kinesthetic feedback or with only kinesthetic feedback. Tactile feedback was provided with a cable-driven platform mounted on the fingertip, while kinesthetic feedback was provided using a haptic glove. To measure the participants’ ability to perceive and reproduce the rendered shapes, we measured the time participants spent exploring and reproducing the shapes and the error between the rendered and reproduced shapes after exploration. Furthermore, we assessed the participants’ workload and motivation using well-established questionnaires. We found that concurrent tactile and kinesthetic feedback during shape exploration resulted in lower reproduction errors and longer reproduction times. The longer reproduction times for the combined condition may indicate that participants could learn the shapes better and, thus, were more careful when reproducing them. We did not find differences between conditions in the time spent exploring the shapes or the participants’ workload and motivation. The lack of differences in workload between conditions could be attributed to the reported minimal-to-intermediate workload levels, suggesting that there was little room to further reduce the workload. Our work highlights the potential advantages of multimodal congruent tactile and kinesthetic feedback when interacting with tangible virtual objects with applications in virtual simulators for hands-on training applications.

This work presents the design of the Rehab-Exos, a novel upper limb exoskeleton designed for rehabilitation purposes. It is equipped with high-reduction-ratio actuators and compact elastic joints to obtain torque sensors based on strain gauges. In this study, we address the torque sensor performances and the design aspects that could cause unwanted non-axial moment load crosstalk. Moreover, a new full-state feedback torque controller is designed by modeling the multi-DOF, non-linear system dynamics and providing compensation for non-linear effects such as friction and gravity. To assess the proposed upper limb exoskeleton in terms of both control system performances and mechanical structure validation, the full-state feedback controller was compared with two other benchmark-state feedback controllers in both a transparency test—ten subjects, two reference speeds—and a haptic rendering evaluation. Both of the experiments were representative of the intended purpose of the device, i.e., physical interaction with patients affected by limited motion skills. In all experimental conditions, our proposed joint torque controller achieved higher performances, providing transparency to the joints and asserting the feasibility of the exoskeleton for assistive applications. [ABSTRACT FROM AUTHOR]

Touch interactions are central to many human activities, but there are few technologies for computationally augmenting free-hand interactions with real environments. Here, we describe Tactile Echoes, a finger-wearable system for augmenting touch interactions with physical objects. This system captures and processes touch-elicited vibrations in real-time in order to enliven tactile experiences.  We process these signals via a parametric signal processing network in order to generate responsive tactile and auditory feedback. Just as acoustic echoes are produced through the delayed replication and modification of sounds, so are Tactile Echoes produced through transformations of vibrotactile inputs in the skin. The echoes also reflect the contact interactions and touched objects involved. A transient tap produces discrete echoes, while a continuous slide yields sustained feedback. We also demonstrate computational and spatial tracking methods that allow these effects to be selectively assigned to different objects or actions. A large variety of distinct multisensory effects can be designed via ten processing parameters. We investigated how Tactile Echoes are perceived in several perceptual experiments using multidimensional scaling methods. This allowed us to deduce low-dimensional, semantically grounded perceptual descriptions.  We present several virtual and augmented reality applications of Tactile Echoes. In a user study, we found that these effects made interactions more responsive and engaging. Our findings show how to endow a large variety of touch interactions with expressive multisensory effects.

Optimizing skill acquisition during novel motor tasks and regaining lost motor functions have been the interest of many researchers over the past few decades. One approach shown to accelerate motor learning involves haptically coupling two individuals through robotic interfaces. Studies have shown that an individual’s solo performance during upper-limb tracking tasks may improve after haptically-coupled training with a partner. In this study, our goal was to investigate whether these findings can be translated to lower-limb motor tasks, more specifically, during an ankle position tracking task. Using one-degree-of-freedom ankle movements, pairs of participants (i.e., dyads) tracked target trajectories independently. Participants alternated between tracking trials with and without haptic coupling, achieved by rendering a virtual spring between two ankle rehabilitation robots. In our analysis, we compared changes in task performance across trials while training with and without haptic coupling. The tracking performance of both individuals (i.e., dyadic task performance) improved during haptic coupling, which was likely due to averaging of random errors of the dyadic pair during tracking. However, we found that dyadic haptic coupling did not lead to faster individual learning for the tracking task. These results suggest that haptic coupling between unimpaired individuals may not be an effective method of training ankle movements during a simple, one-degree-of-freedom task.

While treating sensorimotor impairments, a therapist may provide physical assistance by guiding their patient’s limb to teach a desired movement. In this scenario, a key aspect is the compliance of the interaction, as the therapist can provide subtle cues or impose a movement as demonstration. One approach to studying these interactions involves haptically connecting two individuals through robotic interfaces. Upper-limb studies have shown that pairs of connected individuals estimate one another’s goals during tracking tasks by exchanging haptic information, resulting in improved performance dependent on the ability of one’s partner and the stiffness of the virtual connection. In this study, our goal was to investigate whether these findings generalize to the lower limb during an ankle tracking task. Pairs of healthy participants (i.e., dyads) independently tracked target trajectories with and without connections rendered between two ankle robots. We tested the effects of connection stiffness as well as visual noise to manipulate the correlation of tracking errors between partners. In our analysis, we compared changes in task performance across conditions while tracking with and without the connection. We found that tracking improvements while connected increased with connection stiffness, favoring the worse partner in the dyad during hard connections. We modeled the interaction as three springs in series, considering the stiffness of the connection and each partners’ ankle, to show that improvements were likely due to a cancellation of random tracking errors between partners. These results suggest a simplified mechanism of improvements compared to what has been reported during upper-limb dyadic tracking.

While treating sensorimotor impairments, a therapist may provide physical assistance by guiding their patient’s limb to teach a desired movement. In this scenario, a key aspect is the compliance of the interaction, as the therapist can provide subtle cues or impose a movement as demonstration. One approach to studying these interactions involves haptically connecting two individuals through robotic interfaces. Upper-limb studies have shown that pairs of connected individuals estimate one another’s goals during tracking tasks by exchanging haptic information, resulting in improved performance dependent on the ability of one’s partner and the stiffness of the virtual connection. In this study, our goal was to investigate whether these findings generalize to the lower limb during an ankle tracking task. Pairs of healthy participants (i.e., dyads) independently tracked target trajectories with and without connections rendered between two ankle robots. We tested the effects of connection stiffness as well as visual noise to manipulate the correlation of tracking errors between partners. In our analysis, we compared changes in task performance across conditions while tracking with and without the connection. We found that tracking improvements while connected increased with connection stiffness, favoring the worse partner in the dyad during hard connections. We modeled the interaction as three springs in series, considering the stiffness of the connection and each partners’ ankle, to show that improvements were likely due to a cancellation of random tracking errors between partners. These results suggest a simplified mechanism of improvements compared to what has been reported during upper-limb dyadic tracking. [ABSTRACT FROM AUTHOR]

Haptic rendering of surface textures enhances user immersion of human–computer interaction. However, strict input conditions and measurement methods limit the diversity of rendering algorithms. In this regard, we propose a neural network-based approach for vibrotactile haptic rendering of surface textures under unconstrained acquisition conditions. The method first encodes the interactions based on human perception characteristics, and then utilizes an autoregressive-based model to learn a non-linear mapping between the encoded data and haptic features. The interactions consist of normal forces and sliding velocities, while the haptic features are time–frequency amplitude spectrograms by short-time Fourier transform of the accelerations corresponding to the interactions. Finally, a generative adversarial network is employed to convert the generated time–frequency amplitude spectrograms into the accelerations. The effectiveness of the proposed approach is confirmed through numerical calculations and subjective experiences. This approach enables the rendering of a wide range of vibrotactile data for surface textures under unconstrained acquisition conditions, achieving a high level of haptic realism. • A neural network-based method for vibrotactile haptic rendering of surface textures under unconstrained acquisition conditions is proposed. • A coding method is designed based on human physiological physics characteristics. • Required no strict data segmentation methods. • Tactile perception of vibration with high realism is achieved. [ABSTRACT FROM AUTHOR]

Optimizing skill acquisition during novel motor tasks and regaining lost motor functions have been the interest of many researchers over the past few decades. One approach shown to accelerate motor learning involves haptically coupling two individuals through robotic interfaces. Studies have shown that an individual’s solo performance during upper-limb tracking tasks may improve after haptically-coupled training with a partner. In this study, our goal was to investigate whether these findings can be translated to lower-limb motor tasks, more specifically, during an ankle position tracking task. Using one-degree-of-freedom ankle movements, pairs of participants (i.e., dyads) tracked target trajectories independently. Participants alternated between tracking trials with and without haptic coupling, achieved by rendering a virtual spring between two ankle rehabilitation robots. In our analysis, we compared changes in task performance across trials while training with and without haptic coupling. The tracking performance of both individuals (i.e., dyadic task performance) improved during haptic coupling, which was likely due to averaging of random errors of the dyadic pair during tracking. However, we found that dyadic haptic coupling did not lead to faster individual learning for the tracking task. These results suggest that haptic coupling between unimpaired individuals may not be an effective method of training ankle movements during a simple, one-degree-of-freedom task. [ABSTRACT FROM AUTHOR]

Multi-modal haptic rendering is an important research direction to improve realism in haptic rendering. It can produce various mechanical stimuli that render multiple perceptions, such as hardness and roughness. This paper proposes a multi-modal haptic rendering method based on a genetic algorithm (GA), which generates force and vibration stimuli of haptic actuators according to the user's target hardness and roughness. The work utilizes a back propagation (BP) neural network to implement the perception model f that establishes the mapping (I = f (G)) from objective stimuli features G to perception intensities I. We use the perception model to design the fitness function of GA and set physically achievable constraints in fitness calculation. The perception model is transformed into the force/vibration control model by GA. Finally, we conducted realism evaluation experiments between real and virtual samples under single or multi-mode haptic rendering, where subjects scored 0-100. The average score was 70.86 for multi-modal haptic rendering compared with 57.81 for hardness rendering and 50.23 for roughness rendering, which proved that the multi-modal haptic rendering is more realistic than the single mode. Based on the work, our method can be applied to render objects in more perceptual dimensions, not only limited to hardness and roughness. It has significant implications for multi-modal haptic rendering. [ABSTRACT FROM AUTHOR]

Abstract Background Current robot-aided training allows for high-intensity training but might hamper the transfer of learned skills to real daily tasks. Many of these tasks, e.g., carrying a cup of coffee, require manipulating objects with complex dynamics. Thus, the absence of somatosensory information regarding the interaction with virtual objects during robot-aided training might be limiting the potential benefits of robotic training on motor (re)learning. We hypothesize that providing somatosensory information through the haptic rendering of virtual environments might enhance motor learning and skill transfer. Furthermore, the inclusion of haptic rendering might increase the task realism, enhancing participants’ agency and motivation. Providing arm weight support during training might also enhance learning by limiting participants’ fatigue. Methods We conducted a study with 40 healthy participants to evaluate how haptic rendering and arm weight support affect motor learning and skill transfer of a dynamic task. The task consisted of inverting a virtual pendulum whose dynamics were haptically rendered on an exoskeleton robot designed for upper limb neurorehabilitation. Participants trained with or without haptic rendering and with or without weight support. Participants’ task performance, movement strategy, effort, motivation, and agency were evaluated during baseline, short- and long-term retention. We also evaluated if the skills acquired during training transferred to a similar task with a shorter pendulum. Results We found that haptic rendering significantly increases participants’ movement variability during training and the ability to synchronize their movements with the pendulum, which is correlated with better performance. Weight support also enhances participants’ movement variability during training and reduces participants’ physical effort. Importantly, we found that training with haptic rendering enhances motor learning and skill transfer, while training with weight support hampers learning compared to training without weight support. We did not observe any significant differences between training modalities regarding agency and motivation during training and retention tests. Conclusion Haptic rendering is a promising tool to boost robot-aided motor learning and skill transfer to tasks with similar dynamics. However, further work is needed to find how to simultaneously provide robotic assistance and haptic rendering without hampering motor learning, especially in brain-injured patients. Trial registration https://clinicaltrials.gov/show/NCT04759976

Discretization of signals often generates additional energy in the haptic systems, and makes them unstable. The currently popular controls based on the passivity stabilize the systems by limiting the rendered force of the haptic systems so that the generated energy from the system stays below zero. The passivity-based methods are, however, often conservative and sacrifice the performance of haptic rendering. This paper proposes a control method to adjust the change rate of the stiffness of virtual environment connected to the haptic systems to satisfy the input-to-state stability (ISS) for better stability and transparency. The ISS conditions for the systems are derived by modeling the system as a linear time-varying system. The systems become dissipative if they satisfy the ISS conditions. The generated surplus energy of the dissipative system is less than a positive finite value while maintaining stability. Since the passive system is a special case of the dissipative system, the proposed ISS-based control is less conservative than the passivity-based approaches. Performance of the proposed control is experimentally compared with the virtual coupling and the force-bounding method that are widely used passivity-based approaches. The energy generated from the system using the proposed method is positive finite whereas the generated energy using the passivity-based approaches is less than zero. This means that more energy is transferred to the operator. Increased stiffness corresponding to this additionally transferred energy can be rendered. Root-mean-square and maximum errors of the rendered forces are, therefore, reduced by at least 86 % and 50 %, respectively.

The use of digital twins to represent a product or process digitally is trending in many engineering disciplines. This term has also been recently introduced in the medical field. In arthroscopic surgery education, the paradigm shift from apprenticeship to simulation training has driven the need for better simulators, and the current focus is on improving simulators with respect to computational efficiency and system accuracy. However, expanding surgical simulations towards digital twins has not yet been explored. This paper introduces the digital twin concept for arthroscopic surgery, and explores its potential in light of the existing scientific literature. Thus, a systematic review was conducted to summarize and analyze the literature with respect to fast and robust design of an arthroscopic digital twin using patient-specific information, and methods for interactive surgical soft tissue simulation. The review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol with three reliable scientific search engines: IEEE Xplore, ScienceDirect and PubMed. Eighty papers were included in the review, and the extracted data included modeling methods, tissue types, constitutive behavior, computational efficiency or accuracy, hardware configuration, haptic device description, software tools, and system architectures. Considering the review, a novel macro-level conceptual arthroscopic digital twin system is presented, and the applicability of the review findings for the identified subsystems are discussed. The proposed system integrates patient-specific images, diagnostic data, intraoperative sensor data, and surgical practice as inputs, and conceptually enables surgical skills training, preoperative planning, and a database of virtual surgeries.

The use of intraoperative tool-tissue interaction data for arthroscopic digital twins has the potential to enhance haptic feedback in surgical simulations. This study demonstrates the implementation of an interpolation-based method, and a Kalman filter-based method for using intraoperative interaction data to provide nonlinear haptic feedback in a partial meniscectomy punch simulation. The interpolation-based method was implemented by interpolation in tool-tissue interaction data collected from a cadaveric meniscus. The Kalman filter-based method takes a state-space formulation of the arthroscopic punch force in the one-dimensional space domain, and performs sensor fusion of the interpolation-based force signal, and a reference finite element force, to form a new haptic signal. These methods were demonstrated using a novel inexpensive haptic system for simulating an arthroscopic partial meniscectomy punch. The software, computer-aided design models, and collected data are provided open-source on GitHub. The face validity of the methods was evaluated in user studies, including six experienced orthopedic surgeons, and five non-professionals. The findings showed that experienced surgeons could distinguish between a nonlinear interpolation-based force signal, and a linear-elastic ideally-plastic finite element-based reference signal in a partial meniscectomy punch simulation. Surgeons preferred the interpolation method as a haptic rendering strategy in this study. The Kalman filter method was not as effective as interpolation in recreating nonlinear haptic feedback for the tested parameters, but demonstrated coupling of haptic force signals from intraoperative data and an idealized finite element method in a partial meniscectomy punch simulation.

Haptic feedback is critical in a broad range of human-machine/computer-interaction applications. However, the high cost and low portability/wearability of haptic devices remain unresolved issues, severely limiting the adoption of this otherwise promising technology. Electrotactile interfaces have the advantage of being more portable and wearable due to their reduced actuators’ size, as well as their lower power consumption and manufacturing cost. The applications of electrotactile feedback have been explored in human-computer interaction and human-machine-interaction for facilitating hand-based interactions in applications, such as prosthetics, virtual reality, robotic teleoperation, surface haptics, portable devices, and rehabilitation. This article presents a technological overview of electrotactile feedback, as well a systematic review and meta-analysis of its applications for hand-based interactions. We discuss the different electrotactile systems according to the type of application. We also discuss over a quantitative congregation of the findings, to offer a high-level overview into the state-of-art and suggest future directions. Electrotactile feedback systems showed increased portability/wearability, and they were successful in rendering and/or augmenting most tactile sensations, eliciting perceptual processes, and improving performance in many scenarios. However, knowledge gaps (e.g., embodiment), technical (e.g., recurrent calibration, electrodes’ durability) and methodological (e.g., sample size) drawbacks were detected, which should be addressed in future studies. [ABSTRACT FROM AUTHOR]

Unsorted three dimensional (3D) points are commonly acquired from modern tools and they become popular in many virtual reality applications. In order to produce the haptic feedback to enrich the interaction with the captured models, the point clouds are usually converted to structured meshes or implicit representations. The conversion is either time-consuming or not precise, making the haptic rendering with a low fidelity especially for small haptic proxies. We propose to locally reconstruct the points to balance the performance and quality for the haptic rendering of point clouds. We introduce visible patches on the point clouds by noticing that only the points which are visible to the haptic proxy form the candidate contact region. A computational model for the visible patches is introduced and a virtual coupling model is built to update the visible patches online for haptic rendering. The cases with noises and nonuniform samples are also discussed. We demonstrate our method on a set of synthesized and captured 3D point clouds. Various experimental results are collected and show the efficiency of our method. [ABSTRACT FROM AUTHOR]

Virtual reality (VR) is a valuable experimental tool for studying human movement, including the analysis of interactions during locomotion tasks for developing crowd simulation algorithms. However, these studies are generally limited to distant interactions in crowds, due to the difficulty of rendering realistic sensations of collisions in VR. In this article, we explore the use of wearable haptics to render contacts during virtual crowd navigation. We focus on the behavioral changes occurring with or without haptic rendering during a navigation task in a dense crowd, as well as on potential after-effects introduced by the use haptic rendering. Our objective is to provide recommendations for designing VR setup to study crowd navigation behavior. To the end, we designed an experiment (N=23) where participants navigated in a crowded virtual train station without, then with, and then again without haptic feedback of their collisions with virtual characters. Results show that providing haptic feedback improved the overall realism of the interaction, as participants more actively avoided collisions. We also noticed a significant after-effect in the users’ behavior when haptic rendering was once again disabled in the third part of the experiment. Nonetheless, haptic feedback did not have any significant impact on the users’ sense of presence and embodiment. [ABSTRACT FROM AUTHOR]

It is commonly known that a physical textured path can be followed by indirect touch through a probe also in absence of vision if sufficiently informative cues are delivered by the other sensory channels, but prior research indicates that the level of performance while following a virtual path on a touchscreen depends on the type and channel such cues belong to. The re-enactment of oriented forces, as they are induced by localized obstacles in probe-based exploration, may be important to equalize the performance between physical and virtual path following. Using a stylus attached to a force-feedback arm, an uneven path marked by virtual bars was traversed while time and positions were measured under normal sensory conditions, as well as in absence of vision or hearing. Alternatively, participants followed the same path on a wooden tablet provided with physical bars in relief (i.e., without receiving synthetic force) under the same conditions. The visual conditions were found to be significantly faster than the non-visual conditions. However, there was no significant advantage of traversing either path. In contrast to previous experiments in which the virtual bars were rendered using vibrotactile and/or auditory cues, comparable times to traverse the physical and virtual path were found also when vision was disabled. Our results hence suggest that users who are deprived of vision follow textured virtual paths as efficiently as physical paths, if unevenness is rendered using restorative force cues through a stylus. [ABSTRACT FROM AUTHOR]

Stroke survivors are commonly affected by somatosensory impairment, hampering their ability to interpret somatosensory information. Somatosensory information has been shown to critically support movement execution in healthy individuals and stroke survivors. Despite the detrimental effect of somatosensory impairments on performing activities of daily living, somatosensory training—in stark contrast to motor training—does not represent standard care in neurorehabilitation. Reasons for the neglected somatosensory treatment are the lack of high-quality research demonstrating the benefits of somatosensory interventions on stroke recovery, the unavailability of reliable quantitative assessments of sensorimotor deficits, and the labor-intensive nature of somatosensory training that relies on therapists guiding the hands of patients with motor impairments. To address this clinical need, we developed a virtual reality-based robotic texture discrimination task to assess and train touch sensibility. Our system incorporates the possibility to robotically guide the participants' hands during texture exploration (i.e., passive touch) and no-guided free texture exploration (i.e., active touch). We ran a 3-day experiment with thirty-six healthy participants who were asked to discriminate the odd texture among three visually identical textures –haptically rendered with the robotic device– following the method of constant stimuli. All participants trained with the passive and active conditions in randomized order on different days. We investigated the reliability of our system using the Intraclass Correlation Coefficient (ICC). We also evaluated the enhancement of participants' touch sensibility via somatosensory retraining and compared whether this enhancement differed between training with active vs. passive conditions. Our results showed that participants significantly improved their task performance after training. Moreover, we found that training effects were not significantly different between active and passive conditions, yet, passive exploration seemed to increase participants' perceived competence. The reliability of our system ranged from poor (in active condition) to moderate and good (in passive condition), probably due to the dependence of the ICC on the between-subject variability, which in a healthy population is usually small. Together, our virtual reality-based robotic haptic system may be a key asset for evaluating and retraining sensory loss with minimal supervision, especially for brain-injured patients who require guidance to move their hands.

Haptic rendering enables people to touch, perceive, and manipulate virtual objects in a virtual environment. Using six cascaded identical hollow disk electromagnets and a small permanent magnet attached to an operator's finger, this paper proposes and develops an untethered haptic interface through magnetic field control. The concentric hole inside the six cascaded electromagnets provides the workspace, where the 3D position of the permanent magnet is tracked with a Microsoft Kinect sensor. The driving currents of six cascaded electromagnets are calculated in real-time for generating the desired magnetic force. Offline data from an FEA (finite element analysis) based simulation, determines the relationship between the magnetic force, the driving currents, and the position of the permanent magnet. A set of experiments including the virtual object recognition experiment, the virtual surface identification experiment, and the user perception evaluation experiment were conducted to demonstrate the proposed system, where Microsoft HoloLens holographic glasses are used for visual rendering. The proposed magnetic haptic display leads to an untethered and non-contact interface for natural haptic rendering applications, which overcomes the constraints of mechanical linkages in tool-based traditional haptic devices. [ABSTRACT FROM AUTHOR]

The tactile information to be presented to a user during interaction with a virtual object is calculated by simulating the contact between the object model and user model. In the simulation, a distributed force is applied to the contact area on the skin tissue of users’ hands and results in deformation of the skin tissue. The skin deformation caused by the distributed force is the target contact state that should be presented by the device. However, most multipoint haptic displays do not have sufficient degrees of freedom (DoF) to represent the target contact state. This paper presents the concept and formulation of “deformation matching,” whereby the output force is calculated to minimize the error between the target skin deformation and skin deformation that can be realized by the limited DoF device's output force. For comparison, the conventional concept of “force matching” was also formulated. The difference in human perception between these two concepts in the expression of friction was investigated through experiments using a pin-array tactile display capable of stimulating 128 points. It was demonstrated that the perception of the friction coefficient was more sensitive and the perception of the friction direction was more accurate in deformation matching than in force matching. [ABSTRACT FROM AUTHOR]

In interactive Internet of Things (IoT) environment, a solenoid-based tiny pin-array tactile device can become a key module to haptically simulate a surface of interconnected objects. The solenoid-based pin-array tactile device creates a large enough force and stroke to stimulate human skin and generates a wide frequency range. The solenoid-based tiny pin-array tactile device, however, brings a new issue of controlling the pin's stroke of the tactile device. To overcome the limitation of the solenoid-based device, a new tactile rendering method is needed. In the proposed tactile rendering method, we control the operating frequency of pins, instead of controlling their stroke, to haptically simulate the surface of interconnected objects. Our experiments demonstrate that our proposed method with a pin-array tactile device is suitable for simulating the surface of interconnected objects. [ABSTRACT FROM AUTHOR]

Background: Current robot-aided training allows for high-intensity training but might hamper the transfer of learned skills to real daily tasks. Many of these tasks, e.g., carrying a cup of coffee, require manipulating objects with complex dynamics. Thus, the absence of somatosensory information regarding the interaction with virtual objects during robot-aided training might be limiting the potential benefits of robotic training on motor (re)learning. We hypothesize that providing somatosensory information through the haptic rendering of virtual environments might enhance motor learning and skill transfer. Furthermore, the inclusion of haptic rendering might increase the task realism, enhancing participants' agency and motivation. Providing arm weight support during training might also enhance learning by limiting participants' fatigue.Methods: We conducted a study with 40 healthy participants to evaluate how haptic rendering and arm weight support affect motor learning and skill transfer of a dynamic task. The task consisted of inverting a virtual pendulum whose dynamics were haptically rendered on an exoskeleton robot designed for upper limb neurorehabilitation. Participants trained with or without haptic rendering and with or without weight support. Participants' task performance, movement strategy, effort, motivation, and agency were evaluated during baseline, short- and long-term retention. We also evaluated if the skills acquired during training transferred to a similar task with a shorter pendulum.Results: We found that haptic rendering significantly increases participants' movement variability during training and the ability to synchronize their movements with the pendulum, which is correlated with better performance. Weight support also enhances participants' movement variability during training and reduces participants' physical effort. Importantly, we found that training with haptic rendering enhances motor learning and skill transfer, while training with weight support hampers learning compared to training without weight support. We did not observe any significant differences between training modalities regarding agency and motivation during training and retention tests.Conclusion: Haptic rendering is a promising tool to boost robot-aided motor learning and skill transfer to tasks with similar dynamics. However, further work is needed to find how to simultaneously provide robotic assistance and haptic rendering without hampering motor learning, especially in brain-injured patients. Trial registration https://clinicaltrials.gov/show/NCT04759976. [ABSTRACT FROM AUTHOR]

The electrovibration tactile display could render the tactile feeling of different textured surfaces by generating the frictional force through voltage modulation. When a user is sliding his/her finger on the display surface, he/she can feel the frictional texture. However, it is not trivial to prepare and fine-tune the appropriate frictional signals for haptic design and texture simulation. In this paper, we present a deep-learning-based framework to generate the frictional signals from the textured images of fabric materials. The generated frictional signal can be used for the tactile rendering on the electrovibration tactile display. Leveraging GANs (Generative Adversarial Networks), our system could generate the displacement-based data of frictional coefficients for the tactile display to simulate the tactile feedback of different fabric materials. Our experimental results show that the proposed generative model could generate the frictional-coefficient signals visually and statistically close to the ground-truth signals. The following user studies on fabric-texture simulation show that users could not discriminate the generated and the ground-truth frictional signals being rendered on the electrovibration tactile display, suggesting the effectiveness of our deep-frictional-signal-generation model. [Display omitted] • The deep-learning-based image-to-friction generation framework for tactile simulation of fabric material. • The augmented visual-to-frictional database based on HapTex for image-to-friction generation. • The technical experiment of frictional-coefficient signal generation evidencing the performance of the proposed generative model. • The user-perception experiment validating the effectiveness of generated signals for tactile simulation of fabrics on the electrovibration tactile display. [ABSTRACT FROM AUTHOR]

A complete set of diverse haptic feedbacks is essential for a highly realistic and immersive virtual environment. In this sense, a multi-mode haptic interface that simultaneously generates multiple kinds of haptic signals is highly desirable. In this paper, we propose a new silicone-made pneumatically controlled fingertip actuator to render pressure and vibrotactile feedback concurrently to offer a realistic and effective haptic sensation. A new silicone-based stacked dual-layer air chamber was designed. The volume of the chambers is controlled by pneumatic valves with compressed air tanks. The top/upper air chamber renders vibration feedback, whereas the bottom/lower air chamber renders pressure feedback. The proposed silicone-made fingertip actuator is designed so that it can be easily worn at the fingertips. To demonstrate the potential of the system, a virtual environment for rendering three different types of haptic textures was implemented. Extensive performance evaluation and user studies were carried out to demonstrate the proposed actuator's effectiveness compared to an actuator with single vibrotactile feedback. [ABSTRACT FROM AUTHOR]

Haptic rendering often deals with interactions between stiff objects. A traditional way of force computing models the interaction using a spring-damper system, which suffers from stability issues when the desired stiffness is high. Instead of computing a force, this paper continues to explore shifting the focus to rendering an interaction with no penetration, which can be accomplished by using a position controller in the joint space using the encoders as feedback directly. In order to make this approach easily adaptable to any device, an alternative way to model the dynamics of the device is also presented, which is to linearize a detailed simulation model. As a family of linearized models is used to approximate the full dynamic model of the system, it is important to have a smooth transition between multiple sets of controller gains generated based on these models. Gain scheduling is introduced to improve the performance in certain areas and a comparison among three controllers is conducted in a simulation setup. [ABSTRACT FROM AUTHOR]

Goal: Simulators that incorporate haptic feedback for clinical skills training are increasingly used in medical education. This study addresses the neglected aspect of rendering simulated feedback for vascular palpation skills training by systematically examining the effect of haptic factors on performance. Methods: A simulator-based approach to examine palpation skill is presented. Novice participants with and without minimal previous palpation training performed a palpation task on a simulator that rendered controlled vibratory feedback under various conditions. Results: Five objective metrics were employed to analyze participants' performance that yielded key findings in quantifying palpation performance. Participants' palpation accuracy was influenced by all three haptic factors, ranging from moderate to statistically significant. Duration, Total Path Length and Ratio of Correct Movement also demonstrated utility for quantifying performance. Conclusions: We demonstrate that our affordable simulator is capable of rendering controlled haptic feedback suitable for skills training. Further, metrics presented in this study can be used for structured palpation skills assessment and training, potentially improving healthcare delivery.

This paper describes the creation and testing of a prototype device for rendering texture, using a touch-sensitive surface consisting of a linear soft potentiometer (LSP) attached to a 3D-printed platform and mounted on the cone of a 5-inch loudspeaker. Displacement of the cone is determined by finger position along the LSP. The roughness quality of rendered textures was evaluated psychophysically: A magnitude-estimation task measured how changes in the amplitude and spatial frequency of the rendered texture translated into perceptual change. A just-noticeable difference (JND) task measured the threshold for detection of change in amplitude or frequency, proportional to a base value. Magnitude estimation demonstrated sensitivity to both variables across the physical scales presented, but with stronger effects for amplitude: A doubling of frequency led to an approximate doubling of reported magnitude, whereas a 60% increase in amplitude led to an 86% increase in magnitude. The amplitude thresholds averaged 24%, whereas the frequency thresholds were substantially higher, averaging 64% but with substantial inter-participant variability. We conclude that the device has promise for conveying a broad range of vibratory effects and hence may simulate textural variations, but additional research is necessary to further its capabilities for differentiating vibrations close in frequency. [ABSTRACT FROM AUTHOR]