Search

As part of research on human augmentation, multiple bodies are used in a virtual environment. For example, a study on multiple partial body parts has been conducted using up to 64 hands and showed that multiple hands reduced the distance traveled by one hand. However, body perception has yet to be verified. In this study, we investigated how body perception changes when nine hands, partial bodies, are moved synchronously in a virtual environment, compared to a single hand. In addition, we examined whether the sense of body ownership for all nine hands was elicited simultaneously or whether it was elicited for some of the hands while switching between them. Participants performed a reaching task using one or nine hands presented in a virtual environment. After the reaching task, a threat stimulus was given, and hand movements in response to the threat were measured. After completion of each condition, the subjective sense of body ownership and sense of agency was investigated using a Likert scale. The results indicated that users felt the sense of body ownership of several hands for the nine hands and manipulated them by switching their attention to multiple bodies.

Visuomotor synchrony in time and space induces a sense of embodiment towards virtual bodies experienced in first-person view using Virtual Reality (VR). Here, we investigated whether temporal visuomotor synchrony affects avatar embodiment even when the movements of the virtual arms are spatially altered from those of the user in a non-human-like manner. In a within-subjects design VR experiment, participants performed a reaching task controlling an avatar whose lower arms bent in inversed and biomechanically impossible directions from the elbow joints. They performed the reaching task using this “unnatural avatar” as well as a “natural avatar,” whose arm movements and positions spatially matched the user. The reaching tasks were performed with and without a one second delay between the real and virtual movements. While the senses of body ownership and agency towards the unnatural avatar were significantly lower compared to those towards the natural avatar, temporal visuomotor synchrony did significantly increase the sense of embodiment towards the unnatural avatar as well as the natural avatar. These results suggest that temporal visuomotor synchrony is crucial for inducing embodiment even when the spatial match between the real and virtual limbs is disrupted with movements outside the pre-existing cognitive representations of the human body.

Introduction: Incorporating an additional limb that synchronizes with multiple body parts enables the user to achieve high task accuracy and smooth movement. In this case, the visual appearance of the wearable robotic limb contributes to the sense of embodiment. Additionally, the user’s motor function changes as a result of this embodiment. However, it remains unclear how users perceive the attribution of the wearable robotic limb within the context of multiple body parts (perceptual attribution), and the impact of visual similarity in this context remains unknown.Methods: This study investigated the perceptual attribution of a virtual robotic limb by examining proprioceptive drift and the bias of visual similarity under the conditions of single body part (synchronizing with hand or foot motion only) and multiple body parts (synchronizing with average motion of hand and foot). Participants in the conducted experiment engaged in a point-to-point task using a virtual robotic limb that synchronizes with their hand and foot motions simultaneously. Furthermore, the visual appearance of the end-effector was altered to explore the influence of visual similarity.Results: The experiment revealed that only the participants’ proprioception of their foot aligned with the virtual robotic limb, while the frequency of error correction during the point-to-point task did not change across conditions. Conversely, subjective illusions of embodiment occurred for both the hand and foot. In this case, the visual appearance of the robotic limbs contributed to the correlations between hand and foot proprioceptive drift and subjective embodiment illusion, respectively.Discussion: These results suggest that proprioception is specifically attributed to the foot through motion synchronization, whereas subjective perceptions are attributed to both the hand and foot.

This paper examines an importance rank learning method of objects in urban scenes for assisting visually impaired people. Object detection methods have been used to assist visually impaired people in identifying obstacles in urban scenes, such as cars and trees. However, these existing methods are not dedicated to predicting which obstacle is important. Thus, we propose a method that estimates the importance of objects and warns them to users in order of importance ranking. We introduce a neural network-based ranking estimation method to predict the importance ranking of objects. In particular, our method uses optical flow from the previous frame and region data of detected objects as input. It helps to consider states of moving objects (e.g., cars, motorbikes, people) in a scene. Experimental results show that our model outperforms three other baselines qualitatively and quantitatively. Furthermore, our method was highly evaluated than the baseline methods by qualified caregivers of the visually impaired people.

Abstract In the illusory body ownership, humans feel as if a rubber hand or an avatar in a virtual environment is their own body through visual-tactile synchronization or visual-motor synchronization. Despite the onset time and duration of illusory body ownership has been investigated, it is not clear how the onset time and duration change when a part of the body is missing from the full-body. In this study, we investigated the completeness of the full-body for the illusion onset and duration by comparing the following conditions: complete avatar, avatar missing hands and feet, and avatar with hands and feet only. Our results suggest that avatar hands and feet only shorten the duration of the illusion, and missing body parts, such as only hands and feet or no hands and feet, reduce the sense of body ownership and of agency. However, the effects of avatar completeness on the onset time are unclear, and no conclusions can be made in either direction based on the current findings.

Abstract The supernumerary robotic limb system expands the motor function of human users by adding extra artificially designed limbs. It is important for us to embody the system as if it is a part of one’s own body and to maintain cognitive transparency in which the cognitive load is suppressed. Embodiment studies have been conducted with an expansion of bodily functions through a “substitution” and “extension”. However, there have been few studies on the “addition” of supernumerary body parts. In this study, we developed a supernumerary robotic limb system that operates in a virtual environment, and then evaluated whether the extra limb can be regarded as a part of one’s own body using a questionnaire and whether the perception of peripersonal space changes with a visuotactile crossmodal congruency task. We found that the participants can embody the extra-limbs after using the supernumerary robotic limb system. We also found a positive correlation between the perceptual change in the crossmodal congruency task and the subjective feeling that the number of one’s arms had increased (supernumerary limb sensation). These results suggest that the addition of an extra body part may cause the participants to feel that they had acquired a new body part that differs from their original body part through a functional expansion.

Abstract With the development of three‐dimensional (3D) simulation software, preoperative simulation technology is almost completely established. The remaining issue is how to recognize anatomy three‐dimensionally. Extended reality is a newly developed technology with several merits for surgical application: no requirement for a sterilized display monitor, better spatial awareness, and the ability to share 3D images among all surgeons. Various technology or devices for intraoperative navigation have also been developed to support the safety and certainty of liver surgery. Consensus recommendations regarding indocyanine green fluorescence were determined in 2021. Extended reality has also been applied to intraoperative navigation, and artificial intelligence (AI) is one of the topics of real‐time navigation. AI might overcome the problem of liver deformity with automatic registration. Including the issues described above, this article focuses on recent advances in simulation and navigation in liver surgery from 2020 to 2021.

Recognizing foot gestures can be useful for subtle inputs to appliances and machines in everyday life, but for a system to be useful, it must allow users to assume various postures and work in different spaces. Camera-based and pressure-based systems have limitations in these areas. In this paper, we introduce AnkleSens, a novel ankle-worn foot sensing device that estimates a variety of foot postures using photo reflective sensors. Since our device is not placed between the foot and the floor, it can predict foot posture, even if we keep the foot floating in the air. We developed a band prototype with 16 sensors that can be wrapped around the leg above the ankle. To evaluate the performance of the proposed method, we used eight foot postures and four foot states as preliminary classes. After assessing a test dataset with the preliminary classes, we integrated the eight foot postures into five. Finally, we classified the dataset with five postures in four foot states. For the resulting 20 classes, the average classification accuracy with our proposed method was 79.57% with user-dependent training. This study showed the potential of foot posture sensing as a new subtle input method in daily life.

Is it possible to feel part of one’s own body even when the body part is separated from the body? If so, we could exist in different locations by splitting the self-body and using our body in each location. In a study in which the illusion that two bodies are one’s own body was induced using virtual reality (VR), the participants felt as if they were at two locations through the two bodies. However, this illusion was weak and reported only subjectively. We hypothesized that this was because two presented bodies moved synchronously with one participant’s movement or the simultaneous stroking of one participant’s body and two presented bodies switched their attention and weakened body ownership. In this study, we investigated whether splitting one body into left and right in VR could induce body ownership and extend the self-location while maintaining a one-to-one correspondence between the participant’s body and the presented body. The results showed that weaker body ownership was induced in the split body than in the normal body and self-location was extended to the right side of the body. The participants did not report a sense of having more than one body but reported a sense of body spreading, suggesting that the split bodies were perceived as a single body extending to the right side.

Abstract Peripersonal space (PPS) is important for humans to perform body–environment interactions. However, many previous studies only focused on the specific direction of the PPS, such as the front space, despite suggesting that there were PPSs in all directions. We aimed to measure and compare the peri-trunk PPS in four directions (front, rear, left, and right). To measure the PPS, we used a tactile and an audio stimulus because auditory information is available at any time in all directions. We used the approaching and receding task-irrelevant sounds in the experiment. Observers were asked to respond as quickly as possible when a tactile stimulus was applied to a vibrator on their chest. We found that peri-trunk PPS representations exist with an approaching sound, irrespective of the direction.

Spatial cues play an important role in navigating people in both physical and virtual spaces. In spatial navigation, visual information with additional cues, such as haptic cues, enables effective guidance. Most haptic devices are applied to various body parts to make mechanical stimuli, while few devices stimulate a head despite the excellent sensitivity. This article presents Virtual Whiskers, a spatial directional guidance technique by cheek haptic stimulation using tiny robot arms attached to a Head-Mounted Display (HMD). The tip of the robotic arm has photo reflective sensors to detect the distance between the tip and the cheek surface. Using the robot arms, we stimulate a point on the cheek obtained by calculating an intersection between the cheek surface and the target direction. In the directional guidance experiment, we investigated how accurately participants identify the target direction provided by our guidance method. We evaluated an error between the actual target direction and the participant's pointed direction. The experimental result shows that our method achieves the average absolute directional error of 2.54° in the azimuthal plane and 6.54° in the elevation plane. We also conducted a spatial guidance experiment to evaluate task performance in a target search task. We compared the condition of visual information, visual and audio information, and visual information and cheek haptics for task completion time, System Usability Scale (SUS) score, NASA-TLX score. The averages of task completion time were M = 6.39 s, SD = 3.34 s, and M = 5.62 s, SD = 3.12 s, and M = 4.35 s, SD = 2.26 s, in visual-only condition, visual+audio condition, and visual+haptic condition, respectively. In terms of the SUS score, visual condition, visual+audio condition, and visual+haptic condition achieved M = 55.83, SD = 20.40, and M = 47.78, SD = 20.09, and M = 80.42, SD = 10.99, respectively. As for NASA-TLX score, visual condition, visual+audio condition, and visual+haptic condition resulted in M = 75.81, SD = 16.89, and M = 67.57, SD = 14.96, and M = 38.83, SD = 18.52, respectively. Statistical tests revealed significant differences in task completion time, SUS score, and NASA-TLX score between the visual and the visual+haptic condition and the visual+audio and the visual+haptic condition.

Human behavior and perception are optimized for a single body. Yet, the human brain has plasticity, which allows us to extend our body schema. By utilizing technology like robotics or virtual reality (VR), we can modify our body parts or even add a new body to our own while retaining control over these parts. However, the update of body cognition when controlling multiple bodies has not been well examined. In this study, we explore the task performance and body cognition of humans when they have multiple full bodies as an extended embodiment. Our experimental system allows a participant to control up to four bodies at the same time and perceive sensory information from them. The participant experiences synchronizing behavior and vision perception in a virtual environment. We set up three tasks for multiple bodies and evaluated the cognition of these bodies with their gazing information, task performances, and subjective ratings. We found that humans can have the sense of body ownership and agency for each body when controlling multiple bodies simultaneously. Furthermore, it was observed that people manipulate multiple bodies by actively switching their attention in a static environment and passively switching their attention in a dynamic environment. Distributed embodiment has the potential to extend human behavior in cooperative work, parallel work, group behavior, and so on.

We propose a system of detecting the position of a device embedded with an antenna by sensing the electrical power from a two-dimensional communication (2DC) sheet. The system obtains a characteristic power pattern at each position by sweeping the frequency of the microwave supplied to the 2DC sheet. Our system uses a machine learning technique to learn the accumulated power-pattern data to detect the position of a device. The position-detection accuracy of our system was 79.1% when the antenna was moved in 12mm intervals. In addition to detecting the position of a device, we also estimated the direction.

Visual perspective taking is inferring how the world looks to another person. To clarify this process, we investigated whether employing a humanoid avatar as the viewpoint would facilitate an imagined perspective shift in a virtual environment, and which factor of the avatar is effective for the facilitation effect. We used a task that involved reporting how an object looks by a simple direction judgment, either from the avatar's position or from the position of an empty chair. We found that the humanoid avatar's presence improved task performance. Furthermore, the avatar's facilitation effect was observed only when the avatar was facing the visual stimulus to be judged; performance was worse when it faced backwards than when there was only an empty chair facing forwards. This suggests that the avatar does not simply attract spatial attention, but the posture of the avatar is crucial for the facilitation effect. In addition, when the directions of the head and the torso were opposite (i.e., an impossible posture), the avatar's facilitation effect disappeared. Thus, visual perspective taking might not be facilitated by the avatar when its posture is biomechanically impossible because we cannot embody it. Finally, even when the avatar's head of the possible posture was covered with a bucket, the facilitation effect was found with the forward-facing avatar rather than the backward-facing avatar. That is, the head/gaze direction cue, or presumably the belief that the visual stimulus to be judged can be seen by the avatar, was not required. These results suggest that explicit perspective taking is facilitated by embodiment towards humanoid avatars.

Summary: Cyber space enables us to “share” bodies whose movements are a consequence of movements by several individuals. But whether and how our motor behavior is affected during body sharing remains unclear. Here we examined this issue in arm reaching performed by a shared avatar, whose movement was generated by averaging the movements of two participants. We observed that participants exhibited improved reaction times with a shared avatar than alone. Moreover, the reach trajectory of the shared avatar was straighter than that of either participant and correlated with their subjective embodiment of the avatar. Finally, the jerk of the avatar's hand was less than either participant's own hand, both when they reached alone and in the shared body. Movement straightness and hand jerk are well known characteristics of human reach behavior, and our results suggest that during body sharing, humans prioritize these movement characteristics of the shared body over their own.

The concept of minimally invasive spine therapy (MIST) has been proposed as a treatment strategy to reduce the need for overall patient care, including not only minimally invasive spine surgery (MISS) but also conservative treatment and rehabilitation. To maximize the effectiveness of patient care in spine surgery, the educational needs of medical students, residents, and patient rehabilitation can be enhanced by digital transformation (DX), including virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR), three-dimensional (3D) medical images and holograms; wearable sensors, high-performance video cameras, fifth-generation wireless system (5G) and wireless fidelity (Wi-Fi), artificial intelligence, and head-mounted displays (HMDs). Furthermore, to comply with the guidelines for social distancing due to the unexpected COVID-19 pandemic, the use of DX to maintain healthcare and education is becoming more innovative than ever before. In medical education, with the evolution of science and technology, it has become mandatory to provide a highly interactive educational environment and experience using DX technology for residents and medical students, known as digital natives. This study describes an approach to pre- and intraoperative medical education and postoperative rehabilitation using DX in the field of spine surgery that was implemented during the COVID-19 pandemic and will be utilized thereafter.

This review aims to identify the role of augmented, virtual or mixed reality (AR, VR or MR) technologies in setting of spinal surgery. The authors address the challenges surrounding the implementation of this technology in the operating room. A technical standpoint addresses the efficacy of these imaging modalities based on the current literature in the field. Ultimately, these technologies must be cost-effective to ensure widespread adoption. This may be achieved through reduced surgical times and decreased incidence of post-operative complications and revisions while maintaining equivalent safety profile to alternative surgical approaches. While current studies focus mainly on the successful placement of pedicle screws via AR-guided instrumentation, a wider scope of procedures may be assisted using AR, VR or MR technology once efficacy and safety have been validated. These emerging technologies offer a significant advantage in the guidance of complex procedures that require high precision and accuracy using minimally invasive interventions.

Illusory ownership can be induced in a virtual body by visuo-motor synchrony. Our aim was to test the possibility of a re-association of the right thumb with a virtual left arm and express the illusory body ownership of the re-associated arm through a synchronous or asynchronous movement of the body parts through action and vision. Participants felt that their right thumb was the virtual left arm more strongly in the synchronous condition than in the asynchronous one, and the feeling of ownership of the virtual arm was also stronger in the synchronous condition. We did not find a significant difference in the startle responses to a sudden knife appearance to the virtual arm between the two synchrony conditions, as there was no proprioceptive drift of the thumb. These results suggest that a re-association of the right thumb with the virtual left arm could be induced by visuo-motor synchronization; however, it may be weaker than the natural association.

Abstract The digitalization of modern imaging has led radiologists to become very familiar with computers and their user interfaces (UI). New options for display and command offer expanded possibilities, but the mouse and keyboard remain the most commonly utilized, for usability reasons. In this work, we review and discuss different UI and their possible application in radiology. We consider two-dimensional and three-dimensional imaging displays in the context of interventional radiology, and discuss interest in touchscreens, kinetic sensors, eye detection, and augmented or virtual reality. We show that UI design specifically for radiologists is key for future use and adoption of such new interfaces. Next-generation UI must fulfil professional needs, while considering contextual constraints. Teaching Points • The mouse and keyboard remain the most utilized user interfaces for radiologists. • Touchscreen, holographic, kinetic sensors and eye tracking offer new possibilities for interaction. • 3D and 2D imaging require specific user interfaces. • Holographic display and augmented reality provide a third dimension to volume imaging. • Good usability is essential for adoption of new user interfaces by radiologists.

Research in Virtual Reality (VR) showed that embodiment can influence participants' perceptions and behavior when embodied in a different yet plausible virtual body. In this paper, we study the changes an obese virtual body has on products perception (e.g., taste, etc.) and purchase behavior (e.g., number purchased) in an immersive virtual retail store. Participants (of a normal BMI on average) were embodied in a normal (N) or an obese (OB) virtual body and were asked to buy and evaluate food products in the immersive virtual store. Based on stereotypes that are classically associated with obese people, we expected that the group embodied in obese avatars would show a more unhealthy diet, (i.e., buy more food products and also buy more products with high energy intake, or saturated fat) and would rate unhealthy food as being tastier and healthier than participants embodied in “normal weight” avatars. Our participants also rated the perception of their virtual body: the OB group perceived their virtual body as significantly heavier and older. They also rated their sense of embodiment and presence within the immersive virtual store. These measures did not show any significant difference between groups. Finally, we asked them to rate different food products in terms of tastiness, healthiness, sustainability and price. The only difference we noticed is that participants embodied in an obese avatar (OB group) rated the coke as being significantly tastier and the apple as being significantly healthier. Nevertheless, while we hypothesized that participants embodied in a virtual body with obesity would show differences in their shopping patterns (e.g., more “unhealthy” products bought) there were no significant differences between the groups. Stereotype activation failed for our participants embodied in obese avatars, who did not exhibit a shopping behavior following the (negative) stereotypes related to obese people. conversely, while the opposite hypothesis (participants embodied in obese avatars would buy significantly more healthy products in order to “transform” their virtual bodies) could have been made, it was not the case either. We discuss these results and propose hypotheses as to why the behavior of the manipulated group differed from the one we expected. Indeed, unlike previous research, our participants were embodied in virtual avatars which differed greatly from their real bodies. Obese avatars should not only modify users' visual characteristics such as hair or skin color, etc. We hypothesize that an obese virtual body may require some other non-visual stimulus, e.g., the sensation of the extra weight or the change in body size. This main difference could then explain why we did not notice any important modification on participants' behavior and perceptions of food products. We also hypothesize that the absence of stereotype activation and thus of statistical difference between our N and OB groups might be due to higher-level cognitive processes involved while purchasing food products. Indeed our participants might have rejected their virtual bodies when performing the shopping task, while the embodiment and presence ratings did not show significant differences, and purchased products based on their real (non-obese) bodies. This could mean that stereotype activation is more complex that previously thought.