Search

Blind or low-vision (BLV) individuals often have reduced independent mobility, yet new aids fails in increasing it, are not adopted enough, or both. A major cause is a severe deficiency in how mobility aids are assessed in the field: there are no established methods or measures and those used often have poor relevancy, insight affordances, and reproducibility; probing how actual BLV participants regard a proposed aid and how they compare to current aids is rare; and crucially, tests feature too few BLV participants. In this work two tools are introduced to alleviate this: a portable, large-scale-exploration, virtual reality (VR) system; and a comprehensive, aid-agnostic questionnaire focused on BLV mobility. The questionnaire has been validated once with eight orientation and mobility experts and six BLV respondents. Further, both it and the VR system have been applied in aid assessment with 19 BLV participants in a separate study. The VR system is to our knowledge the first in the field designed for portable evaluation, helping considerably in recruiting adequate numbers of BLV participants, for instance by allowing for testing in participants’ homes; while also supporting reproducible and motivated tests and analyses. The questionnaire provides a systematic method to investigate respondents’ views of numerous important facets of a proposed mobility aid, and how they relate to other aids. These tools should assist in achieving a widely adopted aid that meaningfully improves its users’ mobility.

Recent studies have revealed that visual-auditory sensory substitution devices (SSDs) can effectively convey visual information to visually impaired or blind individuals through sound. However, SSDs are still not widely available to the visually impaired and blind community. Addressing these challenges requires not only the development of efficient SSD algorithms but also the evaluation of the impact of SSDs on the devices and environments in which they are used. This study represents the first attempt to analyze the impact of the device or environment used for SSDs on users’ perceptual abilities. To achieve this goal, we developed an experimental procedure that involves both the training of the SSD algorithm and the changing environment and devices that receive the audio signal. Two user experiments were conducted and revealed that user perception is significantly affected by the device and environment used for SSDs. These findings underscore the importance of considering the effect of the device and environment in which it will be used when designing an SSD algorithm or training system.

The loss of sensitivity of the upper limb due to neurological injuries severely limits the ability to manipulate objects, hindering personal independence. Non-invasive augmented sensory feedback techniques are used to promote neural plasticity hence to restore the grasping function. This work presents a wearable device for restoring sensorimotor hand functions based on Discrete Event-driven Sensory Control policy. It consists of an instrumented glove that, relying on piezoelectric sensors, delivers short-lasting vibrotactile stimuli synchronously with the relevant mechanical events (i.e., contact and release) of the manipulation. We first performed a feasibility study on healthy participants (20) that showed overall good performances of the device, with touch-event detection accuracy of 96.2% and a response delay of 22 ms. Later, we pilot tested it on two participants with limited sensorimotor functions. When using the device, they improved their hand motor coordination while performing tests for hand motor coordination assessment (i.e., pick and place test, pick and lift test). In particular, they exhibited more coordinated temporal correlations between grip force and load force profiles and enhanced performances when transferring objects, quantitatively proving the effectiveness of the device.

Goal: Distance information is highly requested in assistive smartphone Apps by people who are blind or low vision (PBLV). However, current techniques have not been evaluated systematically for accuracy and usability. Methods: We tested five smartphone-based distance-estimation approaches in the image center and periphery at 1–3 meters, including machine learning (CoreML), infrared grid distortion (IR_self), light detection and ranging (LiDAR_back), and augmented reality room-tracking on the front (ARKit_self) and back-facing cameras (ARKit_back). Results: For accuracy in the image center, all approaches had Conclusions: We provide key information that helps design reliable smartphone-based visual assistive technologies to enhance the functionality of PBLV.

Numerous electronic travel aids (ETAs) to increase the mobility of blind or low-vision (BLV) individuals have been proposed. However, the lack of established and well-motivated methods, and of recruiting enough BLV test participants, keeps a successful aid illusory. To combat this, a new aid-agnostic questionnaire focused on mobility, the Desire of Use Questionnaire for Mobility of BLV individuals (DoUQ-MoB) and a new portable, large-scale-exploration virtual reality (VR) system, the Parrot-VR, were employed to evaluate the ETA Audomni. Through VR and Audomni, 19 heterogenous BLV participants traversed large-scale urban environments. Their experiences were probed through the DoUQ-MoB, and their movement analyzed. Numerous results are presented, a highlight being that most participants, 76 %, responded that it was very or extremely likely that they would want to use Audomni along with their current aid. Further, Parrot-VR assists in recruiting a satisfying number of diverse BLV participants; and DoUQ-MoB allows to systematically probe their opinions of an aid, and how it relates to others aids, in a considerable quantity of mobility aid aspects. This work illuminates some shortcomings of Audomni, but also shows a majority of BLV participants actually wanting to use a proposed ETA — a result rarely seen so distinctly in the field, and which encourages the continuing efforts of the project. The results are supported by a novel test procedure, which might serve as future inspiration to the field.

Visual–auditory sensory substitution systems can aid visually impaired people in traveling to various places and recognizing their own environments without help from others. Although several such systems have been developed, they are either not widely used or are limited to laboratory-scale research. Among various factors that hinder the widespread use of these systems, one of the most important issues to consider is the optimization of the algorithms for sensory substitution. This study is the first attempt at exploring the possibility of using deep learning for the objective quantification of sensory substitution. To this end, we used generative adversarial networks to investigate the possibility of optimizing the vOICe algorithm, a representative visual–auditory sensory substitution method, by controlling the parameters of the method for converting an image to sound. Furthermore, we explored the effect of the parameters on the conversion scheme for the vOICe system and performed frequency-range and frequency-mapping-function experiments. The process of sensory substitution in humans was modeled to use generative models to assess the extent of visual perception from the substituted sensory signals. We verified the human-based experimental results against the modeling results. The results suggested that deep learning could be used for evaluating the efficiency of algorithms for visual–auditory sensory substitutions without labor-intensive human behavioral experiments. The introduction of deep learning for optimizing the visual–auditory conversion method is expected to facilitate studies on various aspects of sensory substitution, such as generalization and estimation of algorithm efficiency.

With the increased demands of human-machine interaction, haptic feedback is becoming increasingly critical. However, the high cost, large size and low efficiency of current haptic systems severely hinder further development. As a portable and efficient technology, cutaneous electrotactile stimulation has shown promising potential for these issues. This paper presents a review on and insight into cutaneous electrotactile perception and its applications. Research results on perceptual properties and evaluation methods have been summarized and discussed to understand the effects of electrotactile stimulation on humans. Electrotactile applications are presented in categories to understand the methods and progress in various fields such as prostheses control, sensory substitution, sensory restoration and sensorimotor restoration. State of the art has demonstrated the superiority of electrotactile feedback, its efficiency and its flexibility. However, the complex factors and the limitations of evaluation methods made it challenging for precise electrotactile control. Groundbreaking innovation in electrotactile theory is expected to overcome challenges such as precise perception control, information capacity increasing, comprehension burden reducing and implementation costs. [ABSTRACT FROM AUTHOR]

Electrotactile displays can open a new sensory substitution channel to be utilized in a vast array of applications. But there are still challenges in different aspects of such systems. One of the most crucial concerns, is the effects of receptor fatigue. Our tests show that during prolonged exposure of receptors to the electrical signals, their sensitivity to the signal level changes gradually and adjustments in the signal's power are required to keep the receptors is the stimulated state. This was confirmed by monitoring the electrical current passing through the skin and calculating the corresponding impedance. More interestingly, the rates of the impedance changes are different for each part of the skin, indicating inconsistent rates of receptor fatigue for each region of the skin. These electrical properties of the skin were addressed in this research for the purpose of rendering consistent sensations for the users regardless of the person or skin conditions. To solve these challenges, two methods are employed: a voltage control system based on pulse-width modulation is used to adjust the signal power; and Kalman filtering is used to predict impedance changes in advance and supply the skin with the proper signal. The result is a self-contained automated system capable of managing the signal power for any user at any given time or skin condition.

The task of encoding visual information into tactile information has been studied since the 1960s. There is still an open challenge in converting the data of an image into a small set of signals that will be sent to the user as tactile input. In this study, we evaluated two methods that have never been used for encoding vision-to-touch using convolutional neural networks, a bag of convolutional features (BoF) and a vector of locally aggregated descriptors (VLAD). We also present here a very new method for evaluating the semantic property of the encoded signal by taking the idea that objects with similar features must have similar signals in the tactile interface; we created a semantic property evaluation (SPE) metric. Using this metric, we proved the advantage of using the BoF and VLAD methods, obtaining an SPE of 70.7% and 64.5%, respectively, which is a considerable improvement over the downscaling method used by many systems such as BrainPort, with 56.2%.

Pressure ulcers (PU) are known to be a high-cost disease with a risk of severe morbidity. This paper evaluates a new clinical strategy based on an innovative medical device [Tongue Display Unit (TDU)] that implements perceptive supplementation in order to reduce prolonged excessive pressure, recognized as one of the main causes of PU. A randomized, controlled, and parallel-group trial was carried out with 12 subjects with spinal cord injuries (SCI). Subjects were assigned to the control (without TDU, n = 6) or intervention (with TDU, n = 5) group. Each subject took part in two sessions, during which the subject, seated on a pressure map sensor, watched a movie for one hour. The TDU was activated during the second session of the intervention group. Intention-to-treat analysis showed that the improvement in adequate weight shifting between the two sessions was higher in the intervention group (0.84 [0.24; 0.89]) than in the control group (0.01 [-0.01; 0.09]; p = 0.004) and that the ratio of prolonged excessive pressure between the two sessions was lower in the intervention group (0.74 [0.37; 1.92]) than in the control group (1.72 [1.32; 2.56]; p = 0.06). The pressure map sensor was evaluated as being convenient for use in daily life; however, this was not the case for the TDU. This paper shows that persons with SCI could benefit from a system based on perceptive supplementation that alerts and guides the user on how to adapt their posture in order to reduce prolonged excessive pressure, one of the main causes of PU.

Recently, there has been substantial progress in the mechatronic design and myoelectric control of active prostheses. However, a significant unmet need is the lack of sensory feedback in commercial prostheses for upper-limb amputees. The lack of sensory perception impacts on the control performance and embodiment, determining relatively high rejection rates. Previous research has been conducted to evaluate various non-invasive substitutional sensory channels, mainly to regenerate haptic perception. However, providing sensory channels for stiffness perception has been much less explored. In this short paper, we propose a non-invasive wearable sensory armband, named vibrotactile frequency modulation (V-FM) system. Using the V-FM, we implement a closed-loop myocontrol system with force and stiffness perception. A user study was conducted based on a repetitive two-forced alternative choice discrimination test in six able-bodied participants. The study was designed according to the method of constant stimuli. Results showed that using the V-FM armband, the participants recovered sensation comparable (in terms of difference threshold) to the natural stiffness perception. This demonstrated the potential of the proposed V-FM armband in restoring haptic and stiffness perception non-invasively. [ABSTRACT FROM AUTHOR]

Goal: Distance information is highly requested in assistive smartphone Apps by people who are blind or low vision (PBLV). However, current techniques have not been evaluated systematically for accuracy and usability. Methods: We tested five smartphone-based distance-estimation approaches in the image center and periphery at 1-3 meters, including machine learning (CoreML), infrared grid distortion (IR&#95;self), light detection and ranging (LiDAR&#95;back), and augmented reality room-tracking on the front (ARKit&#95;self) and back-facing cameras (ARKit&#95;back). Results: For accuracy in the image center, all approaches had <±2.5 cm average error, except CoreML which had ±5.2-6.2 cm average error at 2-3 meters. In the periphery, all approaches were more inaccurate, with CoreML and IR&#95;self having the highest average errors at ±41 cm and ±32 cm respectively. For usability, CoreML fared favorably with the lowest central processing unit usage, second lowest battery usage, highest field-of-view, and no specialized sensor requirements. Conclusions: We provide key information that helps design reliable smartphone-based visual assistive technologies to enhance the functionality of PBLV., (© 2024 The Authors.)

We present a novel device mounted on the fingertip for acquiring and transmitting visual information through haptic channels. In contrast to previous systems in which the user interrogates an intermediate representation of visual information, such as a tactile display representing a camera generated image, our device uses a fingertip-mounted camera and haptic stimulator to allow the user to feel visual features directly from the environment. Visual features ranging from simple intensity or oriented edges to more complex information identified automatically about objects in the environment may be translated in this manner into haptic stimulation of the finger. Experiments using an initial prototype to trace a continuous straight edge have quantified the user's ability to discriminate the angle of the edge, a potentially useful feature for higher levels analysis of the visual scene.

Myoelectric prostheses are successfully controlled using muscle electrical activity, thereby restoring lost motor functions. However, the somatosensory feedback from the prosthesis to the user is still missing. The sensory substitution methods described in the literature comprise mostly simple position and force sensors combined with discrete stimulation units. The present study describes a novel system for sophisticated electrotactile feedback integrating advanced distributed sensing (electronic skin) and stimulation (matrix electrodes). The system was tested in eight healthy subjects who were asked to recognize the shape, trajectory, and direction of a set of dynamic movement patterns (single lines, geometrical objects, letters) presented on the electronic skin. The experiments demonstrated that the system successfully translated the mechanical interaction into the moving electrotactile profiles, which the subjects could recognize with a good performance (shape recognition: $86\pm 8\%$ lines, $73\pm 13\%$ geometries, $72\pm 12\%$ letters). In particular, the subjects could identify the movement direction with a high confidence. These results are in accordance with previous studies investigating the recognition of moving stimuli in human subjects. This is an important development towards closed-loop prostheses providing comprehensive and sophisticated tactile feedback to the user, facilitating the control and the embodiment of the artificial device into the user body scheme. [ABSTRACT FROM PUBLISHER]

The lack of haptic feedback in robotics-assisted surgery can result in tissue damage or accidental tool-tissue hits. This paper focuses on exploring the effect of haptic feedback via direct force reflection and visual presentation of force magnitudes on performance during suturing in robotics-assisted minimally invasive surgery (RAMIS). For this purpose, a haptics-enabled dual-arm master-slave teleoperation system capable of measuring tool-tissue interaction forces in all seven Degrees-of-Freedom (DOFs) was used. Two suturing tasks, tissue puncturing and knot-tightening, were chosen to assess user skills when suturing on phantom tissue. Sixteen subjects participated in the trials and their performance was evaluated from various points of view: force consistency, number of accidental hits with tissue, amount of tissue damage, quality of the suture knot, and the time required to accomplish the task. According to the results, visual force feedback was not very useful during the tissue puncturing task as different users needed different amounts of force depending on the penetration of the needle into the tissue. Direct force feedback, however, was more useful for this task to apply less force and to minimize the amount of damage to the tissue. Statistical results also reveal that both visual and direct force feedback were required for effective knot tightening: direct force feedback could reduce the number of accidental hits with the tissue and also the amount of tissue damage, while visual force feedback could help to securely tighten the suture knots and maintain force consistency among different trials/users. These results provide evidence of the importance of 7-DOF force reflection when performing complex tasks in a RAMIS setting. [ABSTRACT FROM PUBLISHER]

Human grasping and manipulation control critically depends on tactile feedback. Without this feedback, the ability for fine control of a prosthesis is limited in upper limb amputees. Although various approaches have been investigated in the past, at present there is no commercially available device able to restore tactile feedback in upper limb amputees. Based on the Discrete Event-driven Sensory feedback Control (DESC) policy we present a device able to deliver short-lasting vibrotactile feedback to transradial amputees using commercially available myoelectric hands. The device (DESC-glove) comprises sensorized thimbles to be placed on the prosthesis digits, a battery-powered electronic board, and vibrating units embedded in an arm-cuff being transiently activated when the prosthesis makes and breaks contact with objects. The consequences of using the DESC-glove were evaluated in a longitudinal study. Five transradial amputees were equipped with the device for one month at home. Through a simple test proposed here for the first time—the virtual eggs test—we demonstrate the effectiveness of the device for prosthetic control in daily life conditions. In the future the device could be easily exploited as an add-on to complement myoelectric prostheses or even embedded in prosthetic sockets to enhance their control by upper limb amputees. [ABSTRACT FROM PUBLISHER]

We propose a dynamic interactive system for conveying visual information via hearing and touch to blind and visually impaired people. The system is implemented with a touch screen that allows the user to actively explore a two-dimensional layout consisting of one or more objects with the finger while listening to auditory feedback. Sound is used as the primary source of information for object localization, identification, and shape, while touch is used for pointing and kinesthetic feedback. A static overlay of raised-dot tactile patterns can also be added. The head-related transfer function is used for rendering sound directionality, and variations of sound intensity or other features are used for rendering proximity. The main focus is on conveying the shape of an object, but the rendering of a simple scene layout, that consists of objects in a linear arrangement, each with a distinct tapping sound, is also considered and compared to a “virtual cane.” We consider a number of acoustic-tactile configurations and use empirical studies with visually blocked sighted participants to compare their effectiveness. Our findings demonstrate the advantages of spatial sound (directionality and proximity cues) for dynamic display of information (localization, identification, shape), while raised-dot patterns provide the best static shape rendition. We also show that the proposed configurations outperform existing techniques. The proposed approach is also expected to impact other applications where vision cannot be used. [ABSTRACT FROM PUBLISHER]

In this contribution, we consider the factors that influence the information processing capacity of the person using sensory substitution devices, and the influence of how the translated information, here in audio, impacts performance. First, we review aspects of vision substitution by tactile and audio devices, and then we review key theory in human information processing limitations to devise and test use of an audio–vision substitution device, The vOICe, for recognizing visual objects with audio substitution for vision. Participants heard sonifications of two-dimensional (2-D) images and had to match them to alternatives presented either in visual or tactile modalities. To assess whether capacity limits constrain performance, objects were either presented with all information simultaneously (top and bottom as whole objects), or successively (top and bottom of the object one after the other). Performance was superior in the successive trials, indicative of a capacity limit in processing the auditory information. We discuss the implications for training protocols and design to provide a useful accessibility device for blind individuals. [ABSTRACT FROM PUBLISHER]

Tactile feedback is essential to intuitive control and to promote the sense of self-attribution of a prosthetic limb. Recent findings showed that amputees can be tricked to experience this embodiment, when synchronous and modality-matched stimuli are delivered to biological afferent structures and to an alien rubber hand. Hence, it was suggested to exploit this effect by coupling touch sensors in a prosthesis to an array of haptic tactile stimulators in the prosthetic socket. However, this approach is not clinically viable due to physical limits of current haptic devices. To address this issue we have proposed modality-mismatched stimulation and demonstrated that this promotes self-attribution of an alien hand on normally limbed subjects. In this work we investigated whether similar effects could be induced in transradial amputees with referred phantom sensations in a series of experiments fashioned after the Rubber Hand Illusion using vibrotactile stimulators. Results from three independent measures of embodiment demonstrated that vibrotactile sensory substitution elicits body-ownership of a rubber hand in transradial amputees. These results open up promising possibilities in this field; indeed miniature, safe and inexpensive vibrators could be fitted into commercially available prostheses and sockets to induce the illusion every time the prosthesis manipulates an object. [ABSTRACT FROM AUTHOR]

Electro- or vibro-tactile stimulations were used in the past to provide sensory information in many different applications ranging from human manual control to prosthetics. The two modalities were used separately in the past, and we hypothesized that a hybrid vibro-electrotactile (HyVE) stimulation could provide two afferent streams that are independently perceived by a subject, although delivered in parallel and through the same skin location. We conducted psychophysical experiments where healthy subjects were asked to recognize the intensities of electro- and vibro-tactile stimuli during hybrid and single modality stimulations. The results demonstrated that the subjects were able to discriminate the features of the two modalities within the hybrid stimulus, and that the cross-modality interaction was limited enough to allow better transmission of discrete information (messages) using hybrid versus single modality coding. The percentages of successful recognitions (mean ± standard deviation) for nine messages were 56 ±11% and 72 ±8% for two hybrid coding schemes, compared to 29 ±7% for vibrotactile and 44 ±4% for electrotactile coding. The HyVE can be therefore an attractive solution in numerous application for providing sensory feedback in prostheses and rehabilitation, and it could be used to increase the resolution of a single variable or to simultaneously feedback two different variables. [ABSTRACT FROM AUTHOR]

A novel sensory substitution technique is presented. Kinesthetic and cutaneous force feedback are substituted by cutaneous feedback (CF) only, provided by two wearable devices able to apply forces to the index finger and the thumb, while holding a handle during a teleoperation task. The force pattern, fed back to the user while using the cutaneous devices, is similar, in terms of intensity and area of application, to the cutaneous force pattern applied to the finger pad while interacting with a haptic device providing both cutaneous and kinesthetic force feedback. The pattern generated using the cutaneous devices can be thought as a subtraction between the complete haptic feedback (HF) and the kinesthetic part of it. For this reason, we refer to this approach as sensory subtraction instead of sensory substitution. A needle insertion scenario is considered to validate the approach. The haptic device is connected to a virtual environment simulating a needle insertion task. Experiments show that the perception of inserting a needle using the cutaneous-only force feedback is nearly indistinguishable from the one felt by the user while using both cutaneous and kinesthetic feedback. As most of the sensory substitution approaches, the proposed sensory subtraction technique also has the advantage of not suffering from stability issues of teleoperation systems due, for instance, to communication delays. Moreover, experiments show that the sensory subtraction technique outperforms sensory substitution with more conventional visual feedback (VF). [ABSTRACT FROM PUBLISHER]

A multisite, vibrotactile sensory substitution system, that could be used in conjunction with artificial touch sensors in multifingered prostheses, to deliver sensory feedback to upper limb amputees is presented. The system is based on a low cost/power/size smart architecture of off-the-shelf miniaturized vibration motors; the main novelty is that it is able to generate stimuli where both vibration amplitude and frequency as well as beat interference can be modulated. This paper is aimed at evaluating this system by investigating the capability of healthy volunteers to perceive—on their forearms—vibrations with different amplitudes and/or frequencies. In addition, the ability of subjects in spatially discriminating stimulations on three forearm sites and recognizing six different combinations of stimulations was also addressed. Results demonstrate that subjects were able to discriminate different force amplitudes exerted by the device (accuracies greater than 75%); when both amplitude and frequency were simultaneously varied, the pure discrimination of amplitude/frequency variation was affected by the variation of the other. Subjects were also able to discriminate with an accuracy of 93% three different sites and with an accuracy of 78% six different stimulation patterns. [ABSTRACT FROM PUBLISHER]

Tactile sensory feedback is essential for dexterous object manipulation. Users of hand myoelectric prostheses without tactile feedback must depend essentially on vision to control their device. Indeed, improved tactile feedback is one of their main priorities. Previous research has provided evidence that conveying tactile feedback can improve prostheses control, although additional effort is required to solve problems related to pattern recognition learning, unpleasant sensations, sensory adaptation, and low spatiotemporal resolution. Still, these studies have mainly focused on providing stimulation to hairy skin regions close to the amputation site, i.e., usually to the upper arm. Here, we explored the possibility to provide tactile feedback to the glabrous skin of toes, which have mechanical and neurophysiological properties similar to the fingertips. We explored this paradigm in a grasp-and-lift task, in which healthy participants controlled two opposing digits of a robotic hand by changing the spacing of their index finger and thumb. The normal forces applied by the robotic fingertips to a test object were fed back to the right big and second toe. We show that within a few lifting trials, all the participants incorporated the force feedback received by the foot in their sensorimotor control of the robotic hand. [ABSTRACT FROM AUTHOR]

Electrotactile (electrocutaneous) stimulation at currents greater than sensation threshold causes sensory adaptation, which temporarily raises the sensation threshold and reduces the perceived magnitude of stimulation. After 15 min of moderately intense exposure to a conditioning stimulus (10 s on, 10 s off), the sensation threshold elevation for seven observers was 60-270%, depending on the current, frequency, and number of pulses in the burst structure of the conditioning stimulus. Increases in any of these parameters increased the sensation threshold elevation. Adaptation and recovery were each complete in approximately 15 min [ABSTRACT FROM PUBLISHER]