Search

Abstract\nIMPLICATIONS FOR REHABILITATIONPeople with visual impairments commonly rely on the use of a white cane to navigate and avoid obstacles. Although this analog tool is highly reliable and easy to use, its drawback is the impossibility to anticipate obstacles beyond reach and routes, as well as obstacles above waist level. Electronic travel aids (ETAs) and sensory substitution devices (SSDs) are new technological solutions designed to enhance the tactile and/or auditory capabilities to access the information needed to overcome those drawbacks. In the present study, 25 individuals with visual impairments used the T-Sight, a vibrotactile SSD, and/or the white cane in a navigation task involving obstacle avoidance. While the performance achieved with the device, measured by the number of collisions and walking speed, did not surpass the white cane, the SSD did have a positive impact on ambulation. Participants reduced the number of white cane touches towards environmental obstacles and performed obstacle avoidance maneuvers earlier. These results demonstrate the potential of vibrotactile devices to address the limitations of the white cane. Assessments of secondary travel aids for people with visual impairments should include performance measures beyond speed and number of collisions.These include the number of alerts, white cane touches, and trajectory analysis.Our vibrotactile secondary ETA allows improvements in navigation and obstacle avoidance shown as a reduction in the use of the white cane and an anticipation of the avoidance maneuvers.Assessments of secondary travel aids for people with visual impairments should include performance measures beyond speed and number of collisions.These include the number of alerts, white cane touches, and trajectory analysis.Our vibrotactile secondary ETA allows improvements in navigation and obstacle avoidance shown as a reduction in the use of the white cane and an anticipation of the avoidance maneuvers. [ABSTRACT FROM AUTHOR]

Playing games is an important way to promote the integration, inclusion, and socialization of participants. This is especially the case of persons with disabilities, such as visually impaired people. Unfortunately, very few games are accessible to such persons. Hopefully, in many digital games, this accessibility can be enabled in principle by assistive technologies, such as screen readers. The aim of this work consists in reviewing the recent literature on game accessibility for people with visual impairment and discussing benefits, limitations, and possible improvements of currently available accessibility solutions. After providing a definition of visual impairment and describing its relationship with gaming, the work reviews general techniques for designing more accessible games. Subsequently, it focuses on specific techniques based on replacing visual stimuli with auditory stimuli (e.g., sonification and sound-source simulation), also presenting some recently proposed sonification-mapping strategies. Then, the application of machine-learning techniques to the development of accessible interfaces for online versions of board games is illustrated by a recent case study. Finally, a discussion and some conclusions are provided, with a particular focus on policy implications of improvements in game accessibility for visually impaired people. [ABSTRACT FROM AUTHOR]

There is increasing interest in the use of virtual reality (VR) for training and simulation. While visual and auditory VR displays have improved markedly, the presentation of haptic (tactile and proprioceptive) cues remains challenging and poses limits on users' manual interaction with the virtual environment. We examined interaction with a physical touchscreen and with a virtual touchscreen in VR using a standardized Fitts' tapping task. We examined the effect on the performance of cues to touch of the screen (contact feedback) or to the accuracy of pointing to targets (error feedback) delivered via the auditory modality. For the touchscreen, error feedback reduced target misses while maintaining the efficiency of movement (throughput). In VR, error feedback reduced misses but also reduced throughput due to a disproportionate slowing of movements. Auditory cues to contact significantly improved throughput in VR but not for the touchscreen, possibly because the touchscreen provided robust tactile and proprioceptive cues to contact. The improvement in throughput with auditory contact feedback in VR was of a similar magnitude to that previously reported for haptic contact feedback, but smaller than that reported for veridical haptic cues. These results suggest that auditory stimuli are able to substitute in a limited manner in the absence of tactile or proprioceptive stimulation in VR environments. [ABSTRACT FROM AUTHOR]

Abstract Sensory substitution offers a promising approach to restore lost sensory functions. Here we show that spinal cord stimulation (SCS), typically used for chronic pain management, can potentially serve as a novel auditory sensory substitution device. We recruited 13 patients undergoing SCS implantation and translated everyday sound samples into personalized SCS patterns during their trial phase. In a sound identification task—where chance-level performance was 33.3%—participants ( $$n=8$$ n = 8 ) achieved a mean accuracy of 72.8% using only SCS input. We observed a weak positive correlation between stimulation bitrate and identification accuracy. A follow-up discrimination task ( $$n=5$$ n = 5 ) confirmed that reduced bitrates significantly impaired participants’ ability to distinguish between consecutive SCS patterns, indicating effective processing of additional information at higher bitrates. These findings demonstrate the feasibility of using existing SCS technology to create a novel neural interface for a sound prosthesis. Our results pave the way for future research to enhance stimulation fidelity, assess long-term training effects, and explore integration with other auditory aids for comprehensive hearing rehabilitation.

Abstract Exploring a novel approach to mental health technology, this study illuminates the intricate interplay between exteroception (the perception of the external world), and interoception (the perception of the internal world). Drawing on principles of sensory substitution, we investigated how interoceptive signals, particularly respiration, could be conveyed through exteroceptive modalities, namely vision and hearing. To this end, we developed a unique, immersive multisensory environment that translates respiratory signals in real-time into dynamic visual and auditory stimuli. The system was evaluated by employing a battery of various psychological assessments, with the findings indicating a significant increase in participants' interoceptive sensibility and an enhancement of the state of flow, signifying immersive and positive engagement with the experience. Furthermore, a correlation between these two variables emerged, revealing a bidirectional enhancement between the state of flow and interoceptive sensibility. Our research is the first to present a sensory substitution approach for substituting between interoceptive and exteroceptive senses, and specifically as a transformative method for mental health interventions, paving the way for future research.

Improving the quality of life for people with vision impairments has been an important goal in the research and design of assistive devices for several decades. This paper seeks to further that goal by introducing a novel assistive technology platform that leverages real-time 3D spatial audio to promote safe and efficient navigation for people who are blind or visually impaired (PVI). The presented platform, EchoSee, uses modern 3D scanning technology on a mobile device to construct a live, digital 3D map of a user's environment as they move about their surroundings. Spatialized, virtual audio sources (i.e., virtual speakers) are dynamically placed within the digital 3D scan of the world, providing the navigator with a real-time 3D stereo audio "soundscape." The digital 3D map, and its resultant soundscape, are continuously updated as the user moves about their environment. The generated soundscape is played back through headphones connected to the navigator's device. This paper details (1) the underlying technical components and how they were integrated to produce the mobile application that generates a dynamic soundscape on a consumer mobile device, (2) a methodology for analyzing navigation performance with the application, (3) the design and execution of a user study investigating the effectiveness of the presented system, and (4) a discussion of the results of that study along with a proposed future study and possible improvements. Altogether, this paper presents a novel software platform aimed at assisting individuals with vision impairments to navigate and understand spaces safely, efficiently, and independently and the results of a feasibility study analyzing the viability of the approach. [ABSTRACT FROM AUTHOR]

The vestibular system is dedicated to gaze stabilization, postural balance, and spatial orientation; this makes vestibular function crucial for our ability to interact effectively with our environment. Vestibular hypofunction (VH) progresses over time, and it presents differently in its early and advanced stages. In the initial stages of VH, the effects of VH are mitigated using vestibular rehabilitation therapy (VRT), which can be facilitated with the aid of technology. At more advanced stages of VH, novel techniques that use wearable technologies for sensory augmentation and sensory substitution have been applied to manage VH. Despite this, the potential of assistive technologies for VH management remains underexplored over the past decades. Hence, in this review article, we present the state-of-the-art technologies for facilitating early-stage VRT and for managing advanced-stage VH. Also, challenges and strategies on how these technologies can be improved to enable long-term ambulatory and home use are presented. [ABSTRACT FROM AUTHOR]

Exploring a novel approach to mental health technology, this study illuminates the intricate interplay between exteroception (the perception of the external world), and interoception (the perception of the internal world). Drawing on principles of sensory substitution, we investigated how interoceptive signals, particularly respiration, could be conveyed through exteroceptive modalities, namely vision and hearing. To this end, we developed a unique, immersive multisensory environment that translates respiratory signals in real-time into dynamic visual and auditory stimuli. The system was evaluated by employing a battery of various psychological assessments, with the findings indicating a significant increase in participants' interoceptive sensibility and an enhancement of the state of flow, signifying immersive and positive engagement with the experience. Furthermore, a correlation between these two variables emerged, revealing a bidirectional enhancement between the state of flow and interoceptive sensibility. Our research is the first to present a sensory substitution approach for substituting between interoceptive and exteroceptive senses, and specifically as a transformative method for mental health interventions, paving the way for future research. [ABSTRACT FROM AUTHOR]

The purpose was to assess the effectiveness of three sliding tactile probes placed on the forelimb skin to provide proprioceptive feedback for the detection of hand gestures and discrimination of object size. Tactile contactors representing the first three fingers were driven along the proximodistal axis by linear servo motors. Twenty healthy subjects were involved in the gesture detection test, with 10 of them also participating in the object-size discrimination task. Motors were controlled by computer in the first four sessions of the gesture detection experiment, while the fifth session utilized a sensorized glove. Both the volar and dorsal sides of the forearm were examined. In the object-size discrimination experiment, the method was exclusively assessed on the volar surface under four distinct feedback conditions, including all fingers and each finger separately. The psychophysical data were further analyzed using a structural equation model (SEM) to evaluate the specific contributions of each individual contactor. Subjects consistently outperformed the chance level in detecting gestures. Performance improved up to the third session, with better results obtained on the volar side. The performances were similar in the fourth and fifth sessions. The just noticeable difference for achieving a 75% discrimination accuracy was found to be 2.90 mm of movement on the skin. SEM analysis indicated that the contactor for the index finger had the lowest importance in gesture detection, while it played a more significant role in object-size discrimination. However, all fingers were found to be significant predictors of subjects' responses in both experiments, except for the thumb, which was deemed insignificant in object-size discrimination. The study highlights the importance of considering the partial contribution of each degree of freedom in a sensory feedback system, especially concerning the task, when designing such systems. [ABSTRACT FROM AUTHOR]

Although one can recognize the environment by soundscape substituting vision to auditory signal, whether subjects could perceive the soundscape as visual or visual-like sensation has been questioned. In this study, we investigated hierarchical process to elucidate the recruitment mechanism of visual areas by soundscape stimuli in blindfolded subjects. Twenty-two healthy subjects were repeatedly trained to recognize soundscape stimuli converted by visual shape information of letters. An effective connectivity method called dynamic causal modeling (DCM) was employed to reveal how the brain was hierarchically organized to recognize soundscape stimuli. The visual mental imagery model generated cortical source signals of five regions of interest better than auditory bottom-up, cross-modal perception, and mixed models. Spectral couplings between brain areas in the visual mental imagery model were analyzed. While within-frequency coupling is apparent in bottom-up processing where sensory information is transmitted, cross-frequency coupling is prominent in top-down processing, corresponding to the expectation and interpretation of information. Sensory substitution in the brain of blindfolded subjects derived visual mental imagery by combining bottom-up and top-down processing.

Blind or low-vision (BLV) individuals often have reduced independent mobility, yet new aids fails in increasing it, are not adopted enough, or both. A major cause is a severe deficiency in how mobility aids are assessed in the field: there are no established methods or measures and those used often have poor relevancy, insight affordances, and reproducibility; probing how actual BLV participants regard a proposed aid and how they compare to current aids is rare; and crucially, tests feature too few BLV participants. In this work two tools are introduced to alleviate this: a portable, large-scale-exploration, virtual reality (VR) system; and a comprehensive, aid-agnostic questionnaire focused on BLV mobility. The questionnaire has been validated once with eight orientation and mobility experts and six BLV respondents. Further, both it and the VR system have been applied in aid assessment with 19 BLV participants in a separate study. The VR system is to our knowledge the first in the field designed for portable evaluation, helping considerably in recruiting adequate numbers of BLV participants, for instance by allowing for testing in participants’ homes; while also supporting reproducible and motivated tests and analyses. The questionnaire provides a systematic method to investigate respondents’ views of numerous important facets of a proposed mobility aid, and how they relate to other aids. These tools should assist in achieving a widely adopted aid that meaningfully improves its users’ mobility.

Recent studies have revealed that visual-auditory sensory substitution devices (SSDs) can effectively convey visual information to visually impaired or blind individuals through sound. However, SSDs are still not widely available to the visually impaired and blind community. Addressing these challenges requires not only the development of efficient SSD algorithms but also the evaluation of the impact of SSDs on the devices and environments in which they are used. This study represents the first attempt to analyze the impact of the device or environment used for SSDs on users’ perceptual abilities. To achieve this goal, we developed an experimental procedure that involves both the training of the SSD algorithm and the changing environment and devices that receive the audio signal. Two user experiments were conducted and revealed that user perception is significantly affected by the device and environment used for SSDs. These findings underscore the importance of considering the effect of the device and environment in which it will be used when designing an SSD algorithm or training system.

The loss of sensitivity of the upper limb due to neurological injuries severely limits the ability to manipulate objects, hindering personal independence. Non-invasive augmented sensory feedback techniques are used to promote neural plasticity hence to restore the grasping function. This work presents a wearable device for restoring sensorimotor hand functions based on Discrete Event-driven Sensory Control policy. It consists of an instrumented glove that, relying on piezoelectric sensors, delivers short-lasting vibrotactile stimuli synchronously with the relevant mechanical events (i.e., contact and release) of the manipulation. We first performed a feasibility study on healthy participants (20) that showed overall good performances of the device, with touch-event detection accuracy of 96.2% and a response delay of 22 ms. Later, we pilot tested it on two participants with limited sensorimotor functions. When using the device, they improved their hand motor coordination while performing tests for hand motor coordination assessment (i.e., pick and place test, pick and lift test). In particular, they exhibited more coordinated temporal correlations between grip force and load force profiles and enhanced performances when transferring objects, quantitatively proving the effectiveness of the device.

Goal: Distance information is highly requested in assistive smartphone Apps by people who are blind or low vision (PBLV). However, current techniques have not been evaluated systematically for accuracy and usability. Methods: We tested five smartphone-based distance-estimation approaches in the image center and periphery at 1–3 meters, including machine learning (CoreML), infrared grid distortion (IR_self), light detection and ranging (LiDAR_back), and augmented reality room-tracking on the front (ARKit_self) and back-facing cameras (ARKit_back). Results: For accuracy in the image center, all approaches had Conclusions: We provide key information that helps design reliable smartphone-based visual assistive technologies to enhance the functionality of PBLV.

Numerous electronic travel aids (ETAs) to increase the mobility of blind or low-vision (BLV) individuals have been proposed. However, the lack of established and well-motivated methods, and of recruiting enough BLV test participants, keeps a successful aid illusory. To combat this, a new aid-agnostic questionnaire focused on mobility, the Desire of Use Questionnaire for Mobility of BLV individuals (DoUQ-MoB) and a new portable, large-scale-exploration virtual reality (VR) system, the Parrot-VR, were employed to evaluate the ETA Audomni. Through VR and Audomni, 19 heterogenous BLV participants traversed large-scale urban environments. Their experiences were probed through the DoUQ-MoB, and their movement analyzed. Numerous results are presented, a highlight being that most participants, 76 %, responded that it was very or extremely likely that they would want to use Audomni along with their current aid. Further, Parrot-VR assists in recruiting a satisfying number of diverse BLV participants; and DoUQ-MoB allows to systematically probe their opinions of an aid, and how it relates to others aids, in a considerable quantity of mobility aid aspects. This work illuminates some shortcomings of Audomni, but also shows a majority of BLV participants actually wanting to use a proposed ETA — a result rarely seen so distinctly in the field, and which encourages the continuing efforts of the project. The results are supported by a novel test procedure, which might serve as future inspiration to the field.

The vestibular system is dedicated to gaze stabilization, postural balance, and spatial orientation; this makes vestibular function crucial for our ability to interact effectively with our environment. Vestibular hypofunction (VH) progresses over time, and it presents differently in its early and advanced stages. In the initial stages of VH, the effects of VH are mitigated using vestibular rehabilitation therapy (VRT), which can be facilitated with the aid of technology. At more advanced stages of VH, novel techniques that use wearable technologies for sensory augmentation and sensory substitution have been applied to manage VH. Despite this, the potential of assistive technologies for VH management remains underexplored over the past decades. Hence, in this review article, we present the state-of-the-art technologies for facilitating early-stage VRT and for managing advanced-stage VH. Also, challenges and strategies on how these technologies can be improved to enable long-term ambulatory and home use are presented.

Spaceflight can make astronauts susceptible to spatial disorientation which is one of the leading causes of fatal aircraft accidents. In our experiment, blindfolded participants used a joystick to balance themselves while inside a multi-axis rotation device (MARS) in either the vertical or horizontal roll plane. On Day 1, in the vertical roll plane (Earth analog condition) participants could use gravitational cues and therefore had a good sense of their orientation. On Day 2, in the horizontal roll plane (spaceflight analog condition) participants could not use gravitational cues and rapidly became disoriented and showed minimal learning and poor performance. One potential countermeasure for spatial disorientation is vibrotactile feedback that conveys body orientation provided by small vibrating devices applied to the skin. Orientation-dependent vibrotactile feedback provided to one group enhanced performance in the spaceflight condition but the participants reported a conflict between the accurate vibrotactile cues and their erroneous perception of their orientation. Specialized vibrotactile training on Day 1 provided to another group resulted in significantly better learning and performance in the spaceflight analog task with vibrotactile cueing. In this training, participants in the Earth analog condition on Day 1 were required to disengage from the task of aligning with the gravitational vertical encoded by natural vestibular/somatosensory afference and had to align with randomized non-vertical directions of balance signaled by vibrotactile feedback. At the end of Day 2, we deactivated the vibrotactile feedback after both vibration-cued groups had practiced with it in the spaceflight analog condition. They performed as well as the group who did not have any vibrotactile feedback. We conclude that after appropriate training, vibrotactile orientation feedback augments dynamic spatial orientation and does not lead to any negative dependence. [ABSTRACT FROM AUTHOR]

Spaceflight can make astronauts susceptible to spatial disorientation which is one of the leading causes of fatal aircraft accidents. In our experiment, blindfolded participants used a joystick to balance themselves while inside a multi-axis rotation device (MARS) in either the vertical or horizontal roll plane. On Day 1, in the vertical roll plane (Earth analog condition) participants could use gravitational cues and therefore had a good sense of their orientation. On Day 2, in the horizontal roll plane (spaceflight analog condition) participants could not use gravitational cues and rapidly became disoriented and showed minimal learning and poor performance. One potential countermeasure for spatial disorientation is vibrotactile feedback that conveys body orientation provided by small vibrating devices applied to the skin. Orientation-dependent vibrotactile feedback provided to one group enhanced performance in the spaceflight condition but the participants reported a conflict between the accurate vibrotactile cues and their erroneous perception of their orientation. Specialized vibrotactile training on Day 1 provided to another group resulted in significantly better learning and performance in the spaceflight analog task with vibrotactile cueing. In this training, participants in the Earth analog condition on Day 1 were required to disengage from the task of aligning with the gravitational vertical encoded by natural vestibular/somatosensory afference and had to align with randomized non-vertical directions of balance signaled by vibrotactile feedback. At the end of Day 2, we deactivated the vibrotactile feedback after both vibration-cued groups had practiced with it in the spaceflight analog condition. They performed as well as the group who did not have any vibrotactile feedback. We conclude that after appropriate training, vibrotactile orientation feedback augments dynamic spatial orientation and does not lead to any negative dependence.

Nowadays, philosophers and scientists tend to agree that, even though human and artificial intelligence work quite differently, they can still illuminate aspects of each other, and knowledge in one domain can inspire progress in the other. For instance, the notion of "artificial" or "synthetic" phenomenology has been gaining some traction in recent AI research. In this paper, we ask the question: what (if anything) is the use of thinking about phenomenology in the context of AI, and in particular machine learning? We will isolate one sense of "phenomenology", namely the sense in which it is commonly understood within analytic philosophy of perception. Then, we will give examples of projects within sensory substitution and restoration science that rely heavily on machine learning and to which, according to us, phenomenology in the sense specified makes a relevant contribution. Finally, we will shed some light on what this contribution looks like and why it is important. [ABSTRACT FROM AUTHOR]

With the increase in the number of sensory substitution devices, the engineering community is confronted with a new challenge: ensuring user training in safe virtual environments before using these devices in real-life situations. We developed a game that uses an original sonification model, which, although not specific to a certain substitution device, can be an effective means of training for orientation in space based on audio stimuli. Thus, the game is not only a means of entertainment for visually impaired (VI) people but also one of training for the use of assistive devices. The game design and audio design are original contributions by the authors. The sonification model, which is crucial for a game dedicated to visually impaired people, is described in detail, both at the user and the implementation level. For better immersion, special sound design techniques have been used, such as ambisonic recordings and impulse response (IR) recordings. The game has been improved gradually, especially the sonification model, based on users' feedback. [ABSTRACT FROM AUTHOR]

Visual–auditory sensory substitution systems can aid visually impaired people in traveling to various places and recognizing their own environments without help from others. Although several such systems have been developed, they are either not widely used or are limited to laboratory-scale research. Among various factors that hinder the widespread use of these systems, one of the most important issues to consider is the optimization of the algorithms for sensory substitution. This study is the first attempt at exploring the possibility of using deep learning for the objective quantification of sensory substitution. To this end, we used generative adversarial networks to investigate the possibility of optimizing the vOICe algorithm, a representative visual–auditory sensory substitution method, by controlling the parameters of the method for converting an image to sound. Furthermore, we explored the effect of the parameters on the conversion scheme for the vOICe system and performed frequency-range and frequency-mapping-function experiments. The process of sensory substitution in humans was modeled to use generative models to assess the extent of visual perception from the substituted sensory signals. We verified the human-based experimental results against the modeling results. The results suggested that deep learning could be used for evaluating the efficiency of algorithms for visual–auditory sensory substitutions without labor-intensive human behavioral experiments. The introduction of deep learning for optimizing the visual–auditory conversion method is expected to facilitate studies on various aspects of sensory substitution, such as generalization and estimation of algorithm efficiency.

Balance disorders are highly prevalent worldwide, causing substantial disability with high personal and socioeconomic impact. The prognosis in many of these patients is poor, and rehabilitation programs provide little help in many cases. This medical problem can be addressed using microelectronics by combining the highly successful cochlear implant experience to produce a vestibular prosthesis, using the technical advances in micro gyroscopes and micro accelerometers, which are the electronic equivalents of the semicircular canals (SCC) and the otolithic organs. Reaching this technological milestone fostered the possibility of using these electronic devices to substitute the vestibular function, mainly for visual stability and posture, in case of damage to the vestibular endorgans. The development of implantable and non-implantable devices showed diverse outcomes when considering the integrity of the vestibular pathways, the device parameters (current intensity, impedance, and waveform), and the targeted physiological function (balance and gaze). In this review, we will examine the development and testing of various prototypes of the vestibular implant (VI). The insight raised by examining the state-of-the-art vestibular prosthesis will facilitate the development of new device-development strategies and discuss the feasibility of complex combinations of implantable devices for disorders that directly affect balance and motor performance. [ABSTRACT FROM AUTHOR]

Background Mounting evidence suggests that wearable technologies using peripheral neuromodulation can provide novel ways of improving mobility and gait function in various patient populations including older adults. The purpose of this narrative review is to provide an overview of wearable technologies/devices to improve mobility and gait function through noninvasive peripheral neuromodulation in older adults over the age of 65 and to indicate the suggested mechanism of action behind these technologies. Methods We performed searches for articles and conference abstracts written in English, using the following databases: Embase Classic+Embase from 1947 to July 15, 2021; Ovid MEDLINE; Epub Ahead of Print, In-Process, In-Data-Review & Other Non-Indexed Citations, Daily and Versions from 1946 to July 15, 2021; PubMed; and Scopus. Results Forty-one technologies met the inclusion/exclusion criteria. We found that the primary implementation of the 41 technologies can be divided into 3 main categories: sensory substitution, sensory augmentation (open loop, closed loop), and motor stimulation. Using these technologies, various aspects of mobility are treated or addressed, including, gait function, fall risk, foot drop, navigating environment, and postural control. Conclusions This narrative review summarizes wearable technologies that are currently commercially available and in stages of research and development. Overall, studies suggest that wearable peripheral neuromodulation technologies can improve aspects of mobility for older adults. Existing literature suggests that these technologies may lead to physiological changes in the brain through sensory reweighting or other neuroplastic mechanisms to enhance the performance of mobility and gait function in older adults over the age of 65. [ABSTRACT FROM AUTHOR]

Copyright of New Architecture is the property of New Architecture Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

A navigation system for individuals suffering from blindness or visual impairment provides information useful to reach a destination. Although there are different approaches, traditional designs are evolving into distributed systems with low-cost, front-end devices. These devices act as a medium between the user and the environment, encoding the information gathered on the surroundings according to theories on human perceptual and cognitive processes. Ultimately, they are rooted in sensorimotor coupling. The present work searches for temporal constraints due to such human–machine interfaces, which in turn constitute a key design factor for networked solutions. To that end, three tests were conveyed to a group of 25 participants under different delay conditions between motor actions and triggered stimuli. The results show a trade-off between spatial information acquisition and delay degradation, and a learning curve even under impaired sensorimotor coupling. [ABSTRACT FROM AUTHOR]

Current advancements in both technology and science allow us to manipulate our sensory modalities in new and unexpected ways. In the present study, we explore the potential of expanding what we perceive through our natural senses by utilizing a visual-to-auditory sensory substitution device (SSD), the EyeMusic, an algorithm that converts images to sound. The EyeMusic was initially developed to allow blind individuals to create a spatial representation of information arriving from a video feed at a slow sampling rate. In this study, we aimed to use the EyeMusic for the blind areas of sighted individuals. We use it in this initial proof-of-concept study to test the ability of sighted subjects to combine visual information with surrounding auditory sonification representing visual information. Participants in this study were tasked with recognizing and adequately placing the stimuli, using sound to represent the areas outside the standard human visual field. As such, the participants were asked to report shapes' identities as well as their spatial orientation (front/right/back/left), requiring combined visual (90° frontal) and auditory input (the remaining 270°) for the successful performance of the task (content in both vision and audition was presented in a sweeping clockwise motion around the participant). We found that participants were successful at a highly above chance level after a brief 1-h-long session of online training and one on-site training session of an average of 20 min. They could even draw a 2D representation of this image in some cases. Participants could also generalize, recognizing new shapes they were not explicitly trained on. Our findings provide an initial proof of concept indicating that sensory augmentation devices and techniques can potentially be used in combination with natural sensory information in order to expand the natural fields of sensory perception. [ABSTRACT FROM AUTHOR]

Blindness affects millions of people worldwide, leading to difficulties in daily travel and a loss of independence due to a lack of spatial information. This article proposes a new navigation aid to help people with severe blindness reach their destination. Blind people are guided by a short 3D spatialised sound that indicates the target point to follow. This sound is combined with other sonified information on potential obstacles in the vicinity. The proposed system is based on inertial sensors, GPS data, and the cartographic knowledge of pedestrian paths to define the trajectory. In addition, visual clues are used to refine the trajectory with ground floor information and obstacle information using a camera to provide 3D spatial information. The proposed method is based on a deep learning approach. The different neural networks used in this approach are evaluated on datasets that regroup navigations from pedestrians’ point-of-view. This method achieves low latency and real-time processing without relying on remote connections, instead using a low-power embedded GPU target and a multithreaded approach for video processing, sound generation, and acquisition. This system could significantly improve the quality of life and autonomy of blind people, allowing them to reliably and efficiently navigate in their environment.

Balance disorders are highly prevalent worldwide, causing substantial disability with high personal and socioeconomic impact. The prognosis in many of these patients is poor, and rehabilitation programs provide little help in many cases. This medical problem can be addressed using microelectronics by combining the highly successful cochlear implant experience to produce a vestibular prosthesis, using the technical advances in micro gyroscopes and micro accelerometers, which are the electronic equivalents of the semicircular canals (SCC) and the otolithic organs. Reaching this technological milestone fostered the possibility of using these electronic devices to substitute the vestibular function, mainly for visual stability and posture, in case of damage to the vestibular endorgans. The development of implantable and non-implantable devices showed diverse outcomes when considering the integrity of the vestibular pathways, the device parameters (current intensity, impedance, and waveform), and the targeted physiological function (balance and gaze). In this review, we will examine the development and testing of various prototypes of the vestibular implant (VI). The insight raised by examining the state-of-the-art vestibular prosthesis will facilitate the development of new device-development strategies and discuss the feasibility of complex combinations of implantable devices for disorders that directly affect balance and motor performance.

Current advancements in both technology and science allow us to manipulate our sensory modalities in new and unexpected ways. In the present study, we explore the potential of expanding what we perceive through our natural senses by utilizing a visual-to-auditory sensory substitution device (SSD), the EyeMusic, an algorithm that converts images to sound. The EyeMusic was initially developed to allow blind individuals to create a spatial representation of information arriving from a video feed at a slow sampling rate. In this study, we aimed to use the EyeMusic for the blind areas of sighted individuals. We use it in this initial proof-of-concept study to test the ability of sighted subjects to combine visual information with surrounding auditory sonification representing visual information. Participants in this study were tasked with recognizing and adequately placing the stimuli, using sound to represent the areas outside the standard human visual field. As such, the participants were asked to report shapes’ identities as well as their spatial orientation (front/right/back/left), requiring combined visual (90° frontal) and auditory input (the remaining 270°) for the successful performance of the task (content in both vision and audition was presented in a sweeping clockwise motion around the participant). We found that participants were successful at a highly above chance level after a brief 1-h-long session of online training and one on-site training session of an average of 20 min. They could even draw a 2D representation of this image in some cases. Participants could also generalize, recognizing new shapes they were not explicitly trained on. Our findings provide an initial proof of concept indicating that sensory augmentation devices and techniques can potentially be used in combination with natural sensory information in order to expand the natural fields of sensory perception.

In current philosophy of mind, there is lively debate over whether emotions, moods, and other affects can extend to comprise elements beyond one’s organismic boundaries. At the same time, there has been growing interest in the nature and significance of so-called existential feelings, which, as the term suggests, are feelings of one’s overall being in the world. In this article, I bring these two strands of investigation together to ask: can the material underpinnings of existential feelings extend beyond one’s skull and skin? To begin, I introduce and adopt a componential-systemic view of extended affectivity. In doing so, I specify the vehicle externalist criteria for extension employed in my analysis. I then define what existential feelings are and pinpoint several key issues in their potential extension. More specifically, I identify sensorimotor ‘know-how’ as a possibly extending component of existential feeling and posit it as the fulcrum of my argument. Finally, I move on to consider the extension of existential feeling via sensory substitution, especially through so-called tactile visual sensory substitution (TVSS) devices. Informed by both philosophical and empirical studies, I argue that, under certain conditions, (a) the agent’s implicit sensorimotor processing and a TVSS device can become coupled into a new systemic whole, which in turn (b) reconfigures the material underpinnings of the agent’s preintentional world-experience. This, I conclude, counts as a cogent case for extended existential feeling. [ABSTRACT FROM AUTHOR]

Humans, like most animals, integrate sensory input in the brain from different sensory modalities. Yet humans are distinct in their ability to grasp symbolic input, which is interpreted into a cognitive mental representation of the world. This representation merges with external sensory input, providing modality integration of a different sort. This study evaluates the Topo-Speech algorithm in the blind and visually impaired. The system provides spatial information about the external world by applying sensory substitution alongside symbolic representations in a manner that corresponds with the unique way our brains acquire and process information. This is done by conveying spatial information, customarily acquired through vision, through the auditory channel, in a combination of sensory (auditory) features and symbolic language (named/spoken) features. The Topo-Speech sweeps the visual scene or image and represents objects' identity by employing naming in a spoken word and simultaneously conveying the objects' location by mapping the x-axis of the visual scene or image to the time it is announced and the y-axis by mapping the location to the pitch of the voice. This proof of concept study primarily explores the practical applicability of this approach in 22 visually impaired and blind individuals. The findings showed that individuals from both populations could effectively interpret and use the algorithm after a single training session. The blind showed an accuracy of 74.45%, while the visually impaired had an average accuracy of 72.74%. These results are comparable to those of the sighted, as shown in previous research, with all participants above chance level. As such, we demonstrate practically how aspects of spatial information can be transmitted through non-visual channels. To complement the findings, we weigh in on debates concerning models of spatial knowledge (the persistent, cumulative, or convergent models) and the capacity for spatial representation in the blind. We suggest the present study's findings support the convergence model and the scenario that posits the blind are capable of some aspects of spatial representation as depicted by the algorithm comparable to those of the sighted. Finally, we present possible future developments, implementations, and use cases for the system as an aid for the blind and visually impaired. [ABSTRACT FROM AUTHOR]

In humans, the nature of sensory input influences body-related mental processing. For instance, behavioral differences (e.g., response time) can be found between mental spatial transformations (e.g., mental rotation) of viewed and touched body parts. It can thus be hypothesized that distinct brain activation patterns are associated with such sensory-dependent body-related mental processing. However, direct evidence that the neural correlates of body-related mental processing can be modulated by the nature of the sensory stimuli is still missing. We thus analyzed event-related functional magnetic resonance imaging (fMRI) data from thirty-one healthy participants performing mental rotation of visually- (images) and haptically-presented (plastic) hands. We also dissociated the neural activity related to rotation or task-related performance using models that either regressed out or included the variance associated with response time. Haptically-mediated mental rotation recruited mostly the sensorimotor brain network. Visually-mediated mental rotation led to parieto-occipital activations. In addition, faster mental rotation was associated with sensorimotor activity, while slower mental rotation was associated with parieto-occipital activations. The fMRI results indicated that changing the type of sensory inputs modulates the neural correlates of body-related mental processing. These findings suggest that distinct sensorimotor brain dynamics can be exploited to execute similar tasks depending on the available sensory input. The present study can contribute to a better evaluation of body-related mental processing in experimental and clinical settings. [ABSTRACT FROM AUTHOR]

ObjectiveSensory feedback of upper-limb prostheses is widely desired and studied. As important components of proprioception, position, and movement feedback help users to control prostheses better. Among various feedback methods, electrotactile stimulation is a potential method for coding proprioceptive information of a prosthesis. This study was motivated by the need for proprioception information for a prosthetic wrist. The flexion-extension (FE) position and movement information of the prosthetic wrist are transmitted back to the human body through multichannel electrotactile stimulation.ApproachWe developed an electrotactile scheme to encode the FE position and movement of the prosthetic wrist and designed an integrated experimental platform. A preliminary experiment on the sensory threshold and discomfort threshold was performed. Then, two proprioceptive feedback experiments were performed: a position sense experiment (Exp 1) and a movement sense experiment (Exp 2). Each experiment included a learning session and a test session. The success rate (SR) and discrimination reaction time (DRT) were analyzed to evaluate the recognition effect. The acceptance of the electrotactile scheme was evaluated by a questionnaire.Main resultsOur results showed that the average position SRs of five able-bodied subjects, amputee 1, and amputee 2 were 83.78, 97.78, and 84.44%, respectively. The average movement SR, and the direction and range SR of wrist movement in five able-bodied subjects were 76.25, 96.67%, respectively. Amputee 1 and amputee 2 had movement SRs of 87.78 and 90.00% and direction and range SRs of 64.58 and 77.08%, respectively. The average DRT of five able-bodied subjects was less than 1.5 s and that of amputees was less than 3.5 s.ConclusionThe results indicate that after a short period of learning, the subjects can sense the position and movement of wrist FE. The proposed substitutive scheme has the potential for amputees to sense a prosthetic wrist, thus enhancing the human-machine interaction.

Humans, like most animals, integrate sensory input in the brain from different sensory modalities. Yet humans are distinct in their ability to grasp symbolic input, which is interpreted into a cognitive mental representation of the world. This representation merges with external sensory input, providing modality integration of a different sort. This study evaluates the Topo-Speech algorithm in the blind and visually impaired. The system provides spatial information about the external world by applying sensory substitution alongside symbolic representations in a manner that corresponds with the unique way our brains acquire and process information. This is done by conveying spatial information, customarily acquired through vision, through the auditory channel, in a combination of sensory (auditory) features and symbolic language (named/spoken) features. The Topo-Speech sweeps the visual scene or image and represents objects’ identity by employing naming in a spoken word and simultaneously conveying the objects’ location by mapping the x-axis of the visual scene or image to the time it is announced and the y-axis by mapping the location to the pitch of the voice. This proof of concept study primarily explores the practical applicability of this approach in 22 visually impaired and blind individuals. The findings showed that individuals from both populations could effectively interpret and use the algorithm after a single training session. The blind showed an accuracy of 74.45%, while the visually impaired had an average accuracy of 72.74%. These results are comparable to those of the sighted, as shown in previous research, with all participants above chance level. As such, we demonstrate practically how aspects of spatial information can be transmitted through non-visual channels. To complement the findings, we weigh in on debates concerning models of spatial knowledge (the persistent, cumulative, or convergent models) and the capacity for spatial representation in the blind. We suggest the present study’s findings support the convergence model and the scenario that posits the blind are capable of some aspects of spatial representation as depicted by the algorithm comparable to those of the sighted. Finally, we present possible future developments, implementations, and use cases for the system as an aid for the blind and visually impaired.

Foot sole somatosensory impairment associated with peripheral neuropathy (PN) is prevalent and a strong independent risk factor for gait disturbance and falls in older adults. Walkasins, a lower-limb sensory prosthesis, has been shown to improve gait and mobility in people with PN by providing afferent input related to foot sole pressure distributions via lower-leg mechanical tactile stimulation. Given that gait and mobility are regulated by sensorimotor and cognitive brain networks, it is plausible improvements in gait and mobility from wearing the Walkasins may be associated with elicited neuroplastic changes in the brain. As such, this study aimed to examine changes in brain network connectivity after 26 weeks of daily use of the prosthesis among individuals with diagnosed PN and balance problems. In this exploratory investigation, assessments of participant characteristics, Functional Gait Assessment (FGA), and resting-state functional magnetic resonance imaging were completed at study baseline and 26 weeks follow-up. We found that among those who have completed the study (N = 8; mean age 73.7 years) we observed a fivepoint improvement in FGA performance as well as significant changes in network connectivity over the 26 weeks that were correlated with improved FGA performance. Specifically, greater improvement in FGA score over 26 weeks was associated with increased connectivity within the Default Mode Network (DMN; p < 0.01), the Somatosensory Network (SMN; p < 0.01), and the Frontoparietal Network (FPN; p < 0.01). FGA improvement was also correlated with increased connectivity between the DMN and the FPN (p < 0.01), and decreased connectivity between the SMN and both the FPN (p < 0.01) and cerebellum (p < 0.01). These findings suggest that 26 weeks of daily use of the Walkasins device may provide beneficial neural modulatory changes in brain network connectivity via the sensory replacement stimulation that are relevant to gait improvements among older adults with PN. [ABSTRACT FROM AUTHOR]

Abstract Background For people with lower-limb amputations, wearing a prosthetic limb helps restore their motor abilities for daily activities. However, the prosthesis's potential benefits are hindered by limited somatosensory feedback from the affected limb and its prosthesis. Previous studies have examined various sensory substitution systems to alleviate this problem; the prominent approach is to convert foot–ground interaction to tactile stimulations. However, positive outcomes for improving their postural stability are still rare. We hypothesized that the sensory substiution system based on surrogated tactile stimulus is capable of improving the standing stability among people with lower-limb amputations. Methods We designed a wearable device consisting of four pressure sensors and two vibrators and tested it among people with unilateral transtibial amputations (n = 7) and non-disabled participants (n = 8). The real-time measurements of foot pressure were fused into a single representation of foot–ground interaction force, which was encoded by varying vibration intensity of the two vibrators attached to the participants’ forearm. The vibration intensity followed a logarithmic function of the force representation, in keeping with principles of tactile psychophysics. The participants were tested with a classical postural stability task in which visual disturbances perturbed their quiet standing. Results With a brief familiarization of the system, the participants exhibited better postural stability against visual disturbances when switching on sensory substitution than without. The body sway was substantially reduced, as shown in head movements and excursions of the center of pressure. The improvement was present for both groups of participants and was particularly pronounced in more challenging conditions with larger visual disturbances. Conclusions Substituting otherwise missing foot pressure feedback with vibrotactile signals can improve postural stability for people with lower-limb amputations. The design of the mapping between the foot–ground interaction force and the tactile signals is essential for the user to utilize the surrogated tactile signals for postural control, especially for situations that their postural control is challenged.

Introduction: Sensory Substitution is a biofeedback intervention whereby at least sensory system is utilised to supplement environmental information which is traditionally gathered by another sense. Objective: To present an evidence-based overview of the feasibility and effectiveness of wearable Sensory Substitution devices on gait outcomes in orthopaedic patient populations. Methods: This Systematic Review and Meta-Analysis was reported according to the PRISMA 2020 statement. PubMed, the Cochrane Library, Web of science and PEDro were searched for relevant published literature. Inclusion criteria limited the search strictly to patients diagnosed with an orthopaedic condition and who were randomly grouped to a Sensory Substitution intervention or conventional therapy/training or an equivalent placebo intervention. Results: Nine Randomised Controlled Trials and three Crossover Trials investigating the effectiveness of Sensory Substitution supplemented gait training were identified and included participants with a variety of orthopaedic conditions. Meta-Analyses revealed positive findings of feasibility as well as statistical and clinical effect of the interventions in improving measures of gait speed, weight-bearing control, measures of functionality and subjective self-reporting. Meta-Analyses also revealed the interventions effects were not significant in the management of pain and retention of gait speed. Negatively reinforced Sensory Substitution biofeedback was statistically and clinically effective, whilst positively reinforced biofeedback was not. Conclusion: For orthopaedic patient populations to improve gait speed, weight-bearing control, functionality, pain and self-report measures, the authors recommend a Sensory Substitution supplemented gait training programme with negative biofeedback on performance. The intervention should be undertaken for 20 min per day, 3 days per week for 5 weeks. The intervention should coincide with structured analgesia administration to facilitate effective pain management. Limitations of the data included some low sample sizes and large age-ranges. No financial support was provided for this study.

With the increase in the number of sensory substitution devices, the engineering community is confronted with a new challenge: ensuring user training in safe virtual environments before using these devices in real-life situations. We developed a game that uses an original sonification model, which, although not specific to a certain substitution device, can be an effective means of training for orientation in space based on audio stimuli. Thus, the game is not only a means of entertainment for visually impaired (VI) people but also one of training for the use of assistive devices. The game design and audio design are original contributions by the authors. The sonification model, which is crucial for a game dedicated to visually impaired people, is described in detail, both at the user and the implementation level. For better immersion, special sound design techniques have been used, such as ambisonic recordings and impulse response (IR) recordings. The game has been improved gradually, especially the sonification model, based on users’ feedback.

We have tested the forearm skin of humans as a target organ to deliver proprioceptive feedback via a tactile sensory substitution method. In the proposed method, a contactor probe was actuated by a linear servo motor and moved on the skin in proximo-distal axis depending on the angle of a virtual joint moving on a 180° arc. Twenty healthy subjects were tested to stop the joint at a given target under no-feedback, visual feedback, and tactile (dorsal and volar) feedback conditions. The absolute difference between the target and the response angle was recorded. Tests were repeated 4 times with ~ 1-week intervals. Two joint movement speeds were tested. The subjects performed best with visual feedback, and worst if no feedback was provided. Their performances with tactile feedback were not as good as in the visual feedback condition, but better than in the no-feedback condition. Subjects equally performed with volar and dorsal tactile feedback. The movement speed had no significant effects on tactile feedback. The performance improved with training only in tactile feedback conditions. The proprioceptive information from a motorized prosthesis can be provided through probes moving on the forearm skin, while the efficacy of the feedback may improve with extensive training. [ABSTRACT FROM AUTHOR]

With the increased demands of human-machine interaction, haptic feedback is becoming increasingly critical. However, the high cost, large size and low efficiency of current haptic systems severely hinder further development. As a portable and efficient technology, cutaneous electrotactile stimulation has shown promising potential for these issues. This paper presents a review on and insight into cutaneous electrotactile perception and its applications. Research results on perceptual properties and evaluation methods have been summarized and discussed to understand the effects of electrotactile stimulation on humans. Electrotactile applications are presented in categories to understand the methods and progress in various fields such as prostheses control, sensory substitution, sensory restoration and sensorimotor restoration. State of the art has demonstrated the superiority of electrotactile feedback, its efficiency and its flexibility. However, the complex factors and the limitations of evaluation methods made it challenging for precise electrotactile control. Groundbreaking innovation in electrotactile theory is expected to overcome challenges such as precise perception control, information capacity increasing, comprehension burden reducing and implementation costs. [ABSTRACT FROM AUTHOR]

Movement sonification can improve motor control in both healthy subjects (e.g., learning or refining a sport skill) and those with sensorimotor deficits (e.g., stroke patients and deafferented individuals). It is not known whether improved motor control and learning from movement sonification are driven by feedback-based real-time ("online") trajectory adjustments, adjustments to internal models over multiple trials, or both. We searched for evidence of online trajectory adjustments (muscle twitches) in response to movement sonification feedback by comparing the kinematics and error of reaches made with online (i.e., real-time) and terminal sonification feedback. We found that reaches made with online feedback were significantly more jerky than reaches made with terminal feedback, indicating increased muscle twitching (i.e., online trajectory adjustment). Using a between-subject design, we found that online feedback was associated with improved motor learning of a reach path and target over terminal feedback; however, using a within-subjects design, we found that switching participants who had learned with online sonification feedback to terminal feedback was associated with a decrease in error. Thus, our results suggest that, with our task and sonification, movement sonification leads to online trajectory adjustments which improve internal models over multiple trials, but which themselves are not helpful online corrections. [ABSTRACT FROM AUTHOR]