Search

Research in human–robot interaction for assistive robotics usually presents many technical challenges for experimenters, forcing researchers to split their time between solving technical problems and conducting experiments. In addition, previous work in virtual reality setups tends to focus on a single assistive robotics application. In order to alleviate these problems, we present ArmSym, a virtual reality laboratory with a fully simulated and developer-friendly robot arm. The system is intended as a testbed to run many sorts of experiments on human control of a robotic arm in a realistic environment, ranging from an upper limb prosthesis to a wheelchair-mounted robotic manipulator. To highlight the possibilities of this system, we perform a study comparing different sorts of prosthetic control types. Looking at nonimpaired subjects, we study different psychological metrics that evaluate the interaction of the user with the robot under different control conditions. Subjects report a perception of embodiment in the absence of realistic cutaneous touch, supporting previous studies in the topic. We also find interesting correlations between control and perceived ease of use. Overall our results confirm that ArmSym can be used to gather data from immersive experiences prosthetics, opening the door to closer collaboration between device engineers and experience designers in the future.

Anatomy is regarded as a cornerstone of health care education and is normally a pre-requisite for clinicians. Even though cadaver dissection and prosection is perceived as the "gold standard," in recent years, its use appeared to be replaced by a myriad of innovative teaching technologies. The present study incorporated a three-dimensional (3D) virtual human cadaver—Anatomage Table (AT)—in teaching human anatomy to first-year nursing students in a quasi-experimental design. The results show that the class average in mid-term and final examinations and the overall Grade Point Average (GPA) was significantly higher in students taught with the AT than students taught without the AT. On a satisfaction survey, 84.0% of students reported a positive experience with the AT, and 85.4% indicated they would recommend this teaching tool to other students. For nursing programs without cadaveric dissection, the AT may serve as an effective teaching tool to increase the knowledge of anatomy and may enhance student's long-term knowledge retention.

Purpose Given the growing popularity of virtual human resources development (VHRD) in organizations and among human resource development (HRD) professionals, it is highly essential to deeply examine the nature and scope of the affective dimensions of the VHRD approach. Over the past decade, VHRD has become an important part of the HRD process. Design/methodology/approach The present study used an integrative literature review to investigate the nature of VHRD in the literature, present a descriptive analysis of the literature and categorize the existing VHRD research. Findings The results indicated three major themes, namely, VHRD and socialization, VHRD and learning and VHRD and the psychological characteristics of the work environment. In addition, a new conceptual model was developed based on the findings. Research limitations/implications This study has reviewed the main concepts of VHRD. The potential actions which HRD researchers can take to address the identified challenges are discussed. Originality/value This integrative literature review could provide a roadmap for future research. Based on this model, the VHRD position is within the organizational context and different tools and processes in constant interaction are introduced. Finally, a general view of the VHRD approach was provided, which can help human resources experts deal with a wide range of technologies in the organization.

In this article, we investigate the effects of the physical influence of a virtual human (VH) in the context of face-to-face interaction in a mixed reality environment. In Experiment 1, participants played a tabletop game with a VH, in which each player takes a turn and moves their own token along the designated spots on the shared table. We compared two conditions as follows: the VH in the virtual condition moves a virtual token that can only be seen through augmented reality (AR) glasses, while the VH in the physical condition moves a physical token as the participants do; therefore the VH’s token can be seen even in the periphery of the AR glasses. For the physical condition, we designed an actuator system underneath the table. The actuator moves a magnet under the table which then moves the VH’s physical token over the surface of the table. Our results indicate that participants felt higher co-presence with the VH in the physical condition, and participants assessed the VH as a more physical entity compared to the VH in the virtual condition. We further observed transference effects when participants attributed the VH’s ability to move physical objects to other elements in the real world. Also, the VH’s physical influence improved participants’ overall experience with the VH. In Experiment 2, we further looked into the question how the physical-virtual latency in movements affected the perceived plausibility of the VH’s interaction with the real world. Our results indicate that a slight temporal difference between the physical token reacting to the virtual hand’s movement increased the perceived realism and causality of the mixed reality interaction. We discuss potential explanations for the findings and implications for future shared mixed reality tabletop setups.

Real chemical experiments may be dangerous or pollute the environment; meanwhile, the preparation of drugs and reagents is time-consuming. Due to the above-mentioned reasons, few experiments can be actually operated by students, which is not conducive to the chemistry learning and the phenomena principle understanding. Recently, due to the impact of Covid-19, many schools adopt online teaching, which is even more detrimental to students’ learning of chemistry. Fortunately, MR(mixed reality) technology provides us with the possibility of solving the safety issues and breaking the space-time constraints, while the theory of human needs (Maslow’s hierarchical needs) provides us with a way to design a comfortable and stimulant MR system with realistic visual presentation and interaction. The paper combines with the theory of human needs to propose a new needs model for virtual experiment. Based on this needs model, we design and develop a comprehensive MR system called MagicChem, which offers a robust 6-DoF interactive and illumination consistent experimental space with virtual-real occlusion, supporting realistic visual interaction, tangible interaction, gesture interaction with touching, voice interaction, temperature interaction, olfactory interaction and virtual human interaction. User study shows that MagicChem satisfies the needs model better than other MR experimental environments that partially meet the needs model. In addition, we explore the application of the needs model in VR environment. Supplementary Information The online version contains supplementary material available at 10.1007/s10055-021-00560-z.

Educators use various strategies to increase listening comprehension for nonnative English speakers in the classroom and multimedia environments. Research on audio reduction has shown mixed results, whereas a study that enhanced (doubled) virtual human gesturing found increased listening comprehension with procedural information (Davis and Vincent, British Journal of Educational Technology 50:3252–3263, 2019). This research examined the use of virtual human gesture frequency (enhanced, average, no) and video speed (normal, reduced 25%) on participant satisfaction and learning outcomes with procedural information. Analysis based on 234 multinational university students indicated that normal video speed significantly increased satisfaction compared to reduced speed; satisfaction was rated significantly higher with agents that gestured compared with the no-gesture condition; and enhancing the gesture frequency significantly increased learning outcomes compared to the average and no-gesture conditions. These findings support previous studies that indicated enhanced gestures significantly increase the learning of procedural information. Also, agent gesturing increased the satisfaction with the agent, which supports systematic review findings that gesturing is a principal moderator for agent persona (Davis et al., Journal of Research on Technology in Education 53:89–106, 2021). However, this research provides evidence that a 25% reduction in video speed may be too slow to maintain satisfaction with advanced foreign language users and that less reduced rates such as 15% or 10% should to be considered. Finally, this research puts forth a gesture design framework for designers to create gesturing virtual humans in multimedia environments.

There are many virtual human joints and degrees of freedom involved in man–machine simulation; however, the commonly used man–machine simulation software cannot provide enough constraints for the whole posture solution of virtual human. This study proposes a novel framework for whole posture calculation and generation. First, a new typical posture database is created. Based on these typical postures, virtual human approximates whole posture that is generated according to the new posture generation conditions and selection rules. Second, a novel method for accurate calculation of partial postures is presented. This method can calculate and adjust the partial posture of the virtual human. It can also achieve the whole posture generation to design the calculation and generation process. Then, this study proposes a meaningful calculation method of anthropomorphism which can evaluate the generated posture accurately. The experimental process is verified by abundant simulation examples compared with three state-of-the-art methods. It turns out that the novel framework proposed can reduce interaction effort and improve the effectiveness of posture generation. Meanwhile, the evaluation of anthropomorphism can achieve 0.91 to get Level 1 which indicates that the generated posture is beneficial to simulation requirements.

BACKGROUND: There is a great demand for the extraction of organ models from three-dimensional (3D) medical images in clinical medicine diagnosis and treatment. OBJECTIVE: We aimed to aid doctors in seeing the real shape of human organs more clearly and vividly. METHODS: The method uses the minimum eigenvectors of Laplacian matrix to automatically calculate a group of basic matting components that can properly define the volume image. These matting components can then be used to build foreground images with the help of a few user marks. RESULTS: We propose a direct 3D model segmentation method for volume images. This is a process of extracting foreground objects from volume images and estimating the opacity of the voxels covered by the objects. CONCLUSIONS: The results of segmentation experiments on different parts of human body prove the applicability of this method.

In this study, we presented movable surface models to help medical students understand the multiaxial movements of the hip joint. The secondary objective was to demonstrate a simple method to make movable surface models for other researchers. We used 166 surface models of the virtual human, and the commercial software was used for all the processes described in this study. Virtual joints were created for the hip joint of the surface models to simulate realistic movements of the joints. Bone surface models were processed to maintain the original shape of the bones during movement. Muscle surface models were processed to express deformation of the muscle shapes during movement. Next, the muscle and bone surface models were moved over six movements of the hip joint (flexion, extension, abduction, adduction, lateral rotation, and medial rotation). The surface models of these six movements were saved and packaged in a PDF file. The PDF file enabled users to see the stereoscopic shapes of the bones and muscles of the hip joint and to scrutinize the six movements on the X, Y, and Z axes of the joint. The movable surface models of the hip joint of this study will be helpful for medical students to learn the multiaxial movements of the hip joint. We expect to develop simulations of other joints that can be used in the education of medical students using the materials and methods described in this study.

Kenaikan jumlah penyandang autisme di Indonesia diikuti dengan bertambahnya kebutuhan terhadap pengajar. Permasalahan utama terkait pengajar anak dengan autisme adalah kurangnya keterampilan pengajar dalam menangani anak dengan autisme. Salah satu metode yang digunakan calon pengajar anak dengan autisme untuk meningkatkan keterampilan mengajarnya adalah dengan memainkan drama bertukar peran. Satu orang berperan sebagai guru, sedangkan yang lain berperan sebagai anak dengan autisme. Dari wawancara yang dilakukan terhadap beberapa mahasiswa pendidikan khusus autis, metode tersebut dinilai kurang efektif untuk memahami dan merespons perilaku anak dengan autisme. Dewasa ini, Virtual Reality (VR) memungkinkan calon pengajar berlatih meningkatkan kemampuan mengajarnya pada lingkungan yang lebih aman. Umumnya, proses simulasi pembelajaran menggunakan VR memanfaatkan manusia virtual yang memiliki fitur maupun kemampuan spesifik. Makalah ini menyajikan penggunaan model ADDIE untuk mengembangkan animasi manusia virtual yang dapat digunakan sebagai agen pembelajaran bagi calon pengajar anak dengan autisme. Animasi tersebut merupakan representasi gerakan yang sering dilakukan oleh anak dengan autisme yang dapat digunakan sebagai modul pada VR. Dari hasil pengujian yang dilakukan, diketahui bahwa animasi yang dikembangkan dianggap sudah dapat merepresentasikan perilaku anak dengan autisme.

Virtual humans are embodied agents with a human-like appearance. In educational contexts, virtual humans are often designed to help people learn. In this special issue, we see six representations o...

Virtual humans (VHs) in augmented reality (AR) can provide users an illusory sense of being together in the real space. However, such an illusion can easily break when the augmented VH conflicts (or is overlaid) with the real objects. Recent spatial understanding technology is starting to make physically plausible VHs in response to collisions, but there are still limitations (e.g., resolution, accuracy) and inevitable conflict situations (e.g., unexpected passer-by), especially in daily life. Moreover, depending on the situation, VH's plausible behavior to avoid collision may rather interfere with the original interaction with the users. In this paper, we investigate three such situations: (1) when VH appears in a room through a closed door, (2) when the VH's body overlaps with static real objects, and (3) when a real moving object passes through the VH. While we considered (2) as an avoidable situation where physically plausible behaviors of VHs might be required, (1) and (3) were considered as inevitable situations (e.g., VH appearing out of nowhere, or passer-by cannot be aware of a virtual being), and we may not present VH's plausible behaviors, so alternatives might be required. Thus, for each of these notable situations in AR, we tested different visual effects as presentation methods for physical conflicts between a VH and real objects. Our findings indicate that visual effects improve VH's social/co-presence and physicality depending on the situations and effect types as well as influence users' attention/social behaviors. We discuss the implications of our findings and future research directions.

The framework of emotional Biologically Inspired Cognitive Architecture (eBICA) is used to define a cognitive model, producing believable socially emotional behavior in a social interaction paradigm in a virtual environment. The paradigm selected for this study is a virtual pet interacting with a human. Empirical results indicate that the combination of somatic factors, moral schemas and rational constraints in one model has the potential to make behavior of a virtual actor more believable, humanlike and socially acceptable. Implications concern future intelligent collaborative robots and virtual assistants.

Extracting 3D structures from voxel based images can make doctors more directly observe the situation of the target in the clinic, making it easier for doctors to diagnose the condition and make the medicine teaching more directly and easier to understand. For this purpose, we propose a 3D volume image segmentation method based on the max-flow/min-cut algorithm. Our segmentation method can be applied directly to 3D volume image. After users marking small amount tags (foreground and background pixels), we put forward a method to use a directed connected graph structure to represent the volume image. In the directed connected graph, in order to speed up the efficiency of the segmentation in subsequent steps, we divide each voxel node in the graph into different color ranges, and each color range match up with an auxiliary node. In order to divide the color range more finely, we propose a method to calculate the color similarity. We then use the max-flow/min-cut algorithm to segment the directed connected graph. The result of experiments performed in multiple sets of slice images shows that our proposed method improves the efficiency, reduces human error on the 3D volume image segmentation task, and the result is complete and accurate.

We present an extension of the Virtual Human Toolkit to include a range of computing platforms, including mobile, web, Virtual Reality (VR) and Augmented Reality (AR). The Toolkit uses a mix of in-house and commodity technologies to support audio-visual sensing, speech recognition, natural language processing, nonverbal behavior generation and realization, text-to-speech generation and rendering. It has been extended to support computing platforms beyond Windows by leveraging microservices. The resulting framework maintains the modularity of the underlying architecture, allows re-use of both logic and content through cloud services, and is extensible by porting lightweight clients. We present the current state of the framework, discuss how we model and animate our characters, and offer lessons learned through several use cases, including expressive character animation in seated VR, shared space and navigation in room-scale VR, autonomous AI in mobile AR, and real-time user performance feedback leveraging mobile sensors in headset AR.

In this contribution we studied how different rendering styles of a virtual human impacted users’ visual attention in an interactive medical training simulator. In a mixed design experiment, 78 participants interacted with a virtual human representing a sample from the non-photorealistic (NPR) to the photorealistic (PR) rendering continuity. We presented five rendering style samples scenarios, namely All Pencil Shaded (APS), Pencil Shaded (PS), All Cartoon Shaded (ACT), Cartoon Shaded (CT), and Human-Like (HL), and compared how visual attention differed between groups of users. For this study, we employed an eye tracking system for collecting and analyzing users’ gaze during interaction with the virtual human in a failure to rescue medical training simulation. Results shows that users spent more total time in the APS and ACT conditions but users visually attended more to virtual humans in the PS, CT and HL appearance conditions.

The objective of this work was to study the dynamics of human emotional states in the process of social interaction in a virtual environment. The previously developed for this purpose prototypes of the virtual actor (NPC) and its virtual environment simulator “Teleport” underwent significant re-design and modification. The experimental platform was re-implemented and used in experiments with college student participants, combining electromyography, emotion recognition in facial recordings and model-based game log analysis in a social videogame paradigm. Participants interacted with two virtual actors implemented based on the eBICA cognitive architecture (Samsonovich, 2013, 2018). Positive correlations were found between eBICA model predictions and participant affects extracted from their facial expressions and facial muscle activity. Affective dynamics of social phenomena, such as the establishment of partnership or an act of betrayal, were characterized and found consistent with the model predictions. Other findings include a gradually developing emotional reaction, possibly due to the integration of appraisals of game events. Overall, obtained results confirm the eBICA model, suggesting its further extension and refinement.

Human gaze awareness is important for social and collaborative interactions. Recent technological advances in augmented reality (AR) displays and sensors provide us with the means to extend collaborative spaces with real-time dynamic AR indicators of one's gaze, for example via three-dimensional cursors or rays emanating from a partner's head. However, such gaze cues are only as useful as the quality of the underlying gaze estimation and the accuracy of the display mechanism. Depending on the type of the visualization, and the characteristics of the errors, AR gaze cues could either enhance or interfere with collaborations. In this paper, we present two human-subject studies in which we investigate the influence of angular and depth errors, target distance, and the type of gaze visualization on participants' performance and subjective evaluation during a collaborative task with a virtual human partner, where participants identified targets within a dynamically walking crowd. First, our results show that there is a significant difference in performance for the two gaze visualizations ray and cursor in conditions with simulated angular and depth errors: the ray visualization provided significantly faster response times and fewer errors compared to the cursor visualization. Second, our results show that under optimal conditions, among four different gaze visualization methods, a ray without depth information provides the worst performance and is rated lowest, while a combination of a ray and cursor with depth information is rated highest. We discuss the subjective and objective performance thresholds and provide guidelines for practitioners in this field.

In healthcare, the human body is a controlled input-output system, which generates different observations with the variations of external interventions. The intervention acts as the input, and the output is the phenotype observation that reflects the latent health state of the body system. The objective of healthcare is to determine effective intervention strategies that can nurse an unhealthy human body to a healthy state. With the advances of Internet-of-Things (IoT) and body sensor networks, it becomes convenient to observe the multimedia data of the human body anywhere and anytime. To aid healthcare decision making, we put forward to construct the human body simulators based on deep neural networks (DNNs) for healthcare research. At first, we formulate the model of the human body system based on DNNs. During our analysis, we realize that DNN-based models could simulate practical situations, e.g. some health states are unreachable. Then, we combine deep reinforcement learning (DRL) with conceptual embedding techniques to explore effective healthcare strategies for simulated human bodies. We implement a virtual human body simulator, which can take interventions and represent its hidden states by high-dimensional images, and a DRL-based treatment module, which can diagnose latent health state through the image observations and choose interventions to nurse the simulated body to a target state. By combining the body simulator and treatment module, we create a dynamic closed-loop for healthcare information processing. Experimental simulations are performed to validate the feasibility of the offered approach.

With the rapid development of computer technology, surgical training, and the digitalized teaching of human body morphology are gaining prominence in medical education. Accurate, true organ models are essential digital material for these computer-assisted systems. However, no direct three-dimensional (3D) true organ model acquisition method currently exists. Thus, the direct extraction of the interested organ models based on the existing Virtual Human Project (VHP) image set is urgently needed. In this paper, a closed-form solution-based volume matting method is proposed. Using a small quantity of graffiti in the foreground and background, target 3D regions can be extracted by closed-form solution computing. The upper triangular storage strategy and the preconditioned conjugate-gradient (PCG) method also promote robustness. Four image data sets (2 virtual human male and 2 virtual human female) from the United States National Library of Medicine (including brain slices, eye slices, lung slices, heart slices, liver slices, kidney slices, spine slices, arm slices, vastus slices, and foot slices) were selected to extract the 3D volume organ models. The experimental results show that the extracted 3D organs were acceptable and satisfactory. This method may provide technical support for medical and other scientific research fields.

In this article I explore the relationship between digital place and disability through an ethnographic study of disability experience in the virtual world Second Life. I discuss how forms of landscape and interface shape disability experience, how building relates to “being-inworld” in digital place, and how proximity and collaboration relate to disability embodiment in a virtual context. “Participant building” on a virtual island created for this research, “Ethnographia,” complements participant observation and other methods to investigate these questions of digital place. Through these lines of analysis, I develop a notion of “digital topography” to illuminate the implications of digital place for disability and human experience more generally. This allows for differentiating digital places from digital media and thus forging conceptual frameworks that reflect how the internet is not a unitary entity. It also allows for considering digital emplacement as related to, but distinct from, digital embodiment. This helps draw attention to questions of digital placemaking alongside the better-known phenomena of avatars. Avatars are important, but it is crucial to highlight the virtual geographies without which the emplacement of those avatars would be impossible. These materials speak to broad questions regarding embodiment, ability, the digital, and the real.

This paper describes my initial attempts to create a believable Virtual Actor: an autonomous computer graphic character depiction that can convincingly portray actions and emotions in-the-moment, i...