Search

The process of transforming data into sounds for auditory display provides unique user experiences and new perspectives for analyzing and interpreting data. A research study for data transformation to sounds based on musical elements, called data-to-music sonification, reveals how musical characteristics can serve analytical purposes with enhanced user engagement. An existing user engagement scale has been applied to measure engagement levels in three conditions within melodic, rhythmic, and chordal contexts. This article reports findings from a user engagement study with musical traits and states the benefits and challenges of using musical characteristics in sonifications. The results can guide the design of future sonifications of multivariable data.

Training in virtual reality (VR) is a valuable supplementing tool for advancing knowledge transfer that results in increased efficiency and accuracy of technicians in fieldwork. However, COVID-19 pandemic restrictions made it impossible for VR training centers to operate on a full scale, forcing traditional face-to-face learning sessions to become remote. In this article, we investigate the asymmetric use of a VR training solution—among devices with different levels of immersion and control—to enrich the content of remote training sessions. The VR in this case can be seen as a source of visual and other contextual information to advance the effects of situated learning and enhance knowledge transfer. To evaluate this approach, we conducted a remote user study with ten industrial maintenance and installation experts. We also introduce the “Research Panel” tool to gather reactions of learners during the remote training session. The expert user study results demonstrate the usefulness and relevance of asymmetric VR to improve remote training sessions and other application industrial scenarios, while the “Research Panel” data provided detailed insight into the session flow. Building on the qualitative findings, we present design guidelines to aid the adoption of asymmetric VR in the industrial context.

Technical documentation creation is a collaborative process involving several departments in R&D. Even though virtual reality (VR) has been demonstrated to facilitate industrial collaboration and advance the product development lifecycle in earlier studies, it has not been utilized for technical documentation review and risk assessment processes in industrial companies. This article presents a case study where the benefits of VR to maintenance documentation reviews and risk assessments were studied. The virtual reality environment was tested by nine domain experts from an industrial company in a user study that replicated their actual real-life industrial collaboration tasks. Both qualitative and quantitative data were collected during the study. Our findings show that collaborative VR has the potential to enhance the documentation review and risk assessment processes. Overall, the concept of using virtual reality for documentation review and risk assessment processes was rated positively by participants, and even though further development is needed for the review tools, VR was viewed as a concept that facilitates collaboration, enhances the current review practices, and increases spatial understanding. The benefits of VR are evident, especially for geographically scattered teams that rarely meet face-to-face or do not have access to the actual physical equipment. In cases where traditional means of communication are not enough, process improvements are needed for documentation review and risk assessment processes, and our proposed solution is VR.

Virtual Reality (VR) is a critical emerging technology in industrial contexts, as it facilitates collaboration and supports the product development lifecycle. However, its broad adoption is constrained by complex and high-cost integration. The use of VR among devices with various immersion and control levels may solve this obstacle, and increase the scalability of VR technologies. This article presents a case study on applying asymmetry between the COVE-VR platform and Microsoft Teams to enable distributed collaboration of multinational departments and enhance the maintenance method and documentation creation processes. Overall, five remote collaborative sessions were held with 20 experts from four countries. Our findings suggest that asymmetry between head-mounted display and Teams users enhances the quality of communication among geographically dispersed teams and their spatial understanding, which positively influences knowledge transfer and efficiency of industrial processes. Based on qualitative evaluation of the asymmetric VR setup, we further suggest a list of guidelines on how to enhance the collaboration efficiency for low-cost distributed asymmetric VR from three perspectives: organization, collaboration and technology.

With the fourth industrial revolution, Industry 4.0, many work tasks are becoming knowledge intensive. At the forefront of the change are workers that are already mobile, working in the field with customers. We describe the human-centred design process that resulted in the Mobile Service Technician 4.0 concept. The concept illustrates how industrial field maintenance work could benefit from knowledge-sharing solutions based on Industry 4.0. The solutions utilize industrial internet, virtual and augmented reality as well as wearable technologies to improve mobile service technicians’ daily work performance and work satisfaction. The Mobile Service Technician 4.0 concept illustrates the user experience of future maintenance work: feeling competent, feeling connected to the work community, and feeling of success and achievement by being better prepared for maintenance visits, getting situationally relevant support in maintenance operations, sharing knowledge with peers, and making maintenance reports effortless.

When designing extended reality (XR) applications, it is important to consider multimodal interaction techniques, which employ several human senses simultaneously. Multimodal interaction can transform how people communicate remotely, practice for tasks, entertain themselves, process information visualizations, and make decisions based on the provided information. This scoping review summarized recent advances in multimodal interaction technologies for head-mounted display-based (HMD) XR systems. Our purpose was to provide a succinct, yet clear, insightful, and structured overview of emerging, underused multimodal technologies beyond standard video and audio for XR interaction, and to find research gaps. The review aimed to help XR practitioners to apply multimodal interaction techniques and interaction researchers to direct future efforts towards relevant issues on multimodal XR. We conclude with our perspective on promising research avenues for multimodal interaction technologies.

Developments in sensor technology, artificial intelligence, and network technologies like 5G has made remote operation a valuable method of controlling various types of machinery. The benefits of remote operations come with an opportunity to access hazardous environments. The major limitation of remote operation is the lack of proper sensory feedback from the machine, which in turn negatively affects situational awareness and, consequently, may risk remote operations. This article explores how to improve situational awareness via multimodal feedback (visual, auditory, and haptic) and studies how it can be utilized to communicate warnings to remote operators. To reach our goals, we conducted a controlled, within-subjects experiment in eight conditions with twenty-four participants on a simulated remote driving system. Additionally, we gathered further insights with a UX questionnaire and semi-structured interviews. Gathered data showed that the use of multimodal feedback positively affected situational awareness when driving remotely. Our findings indicate that the combination of added haptic and visual feedback was considered the best feedback combination to communicate the slipperiness of the road. We also found that the feeling of presence is an important aspect of remote driving tasks, and a requested one, especially by those with more experience in operating real heavy machinery.

Speech technology has matured so that voice-based reporting utilizing speech-to-text can be applied in various domains. Speech has two major benefits: it enables efficient reporting and speech input improves the quality of the reports since reporting can be done as a part of the workflow without delays between work and reporting. However, designing reporting voice user interfaces (VUIs) for professional use is challenging, as there are numerous aspects from technology to organization and language that need to be considered. Based on our experience in developing professional reporting VUIs with different stakeholders representing both commercial and public sector, we define a design space for voice-based reporting systems. The design space consists of 28 dimensions grouped into five categories: Language Processing, Structure of Reporting, Technical Limitations in the Work Domain, Interaction Related Aspects in the Work Domain, and Organization. We illustrate the design space by discussing four voice-based reporting systems, designed and implemented by us, and describing a design process that utilizes it. The design space enables designers to identify critical aspects of professional reporting VUIs and optimize those for their target domain. The design space can be used as a practical tool especially by designers with limited experience on speech technologies.