Search

A growing number of studies have been conducted over the past few years on the positioning of daily massage robots. However, most methods used for research have low interactivity, and a systematic method should be designed for accurate and intelligent positioning, thus compromising usability and user experience. In this study, a massage positioning algorithm with online learning capabilities is presented. The algorithm has the following main innovations: (1) autonomous massage localization can be achieved by gaining insights into natural human-machine interaction behavior and (2) online learning of user massage habits can be achieved by integrating recursive Bayesian ideas. As revealed by the experimental results, combining natural human-computer interaction and online learning with massage positioning is capable of helping people get rid of positioning aids, reducing their psychological and cognitive load, and achieving a more desirable positioning effect. Furthermore, the results of the analysis of user evaluations further verify the effectiveness of the algorithm., Competing Interests: The authors declared that there are no conflicts of interest regarding the publication of this paper., (Copyright © 2022 Liran Zhou et al.)

At present, virtual-reality fusion smart experiments mostly employ visual perception devices to collect user behavior data, but this method faces the obstacles of distance, angle, occlusion, light, and a variety of other factors of indoor interactive input devices. Moreover, the essence of the traditional multimodal fusion algorithm (TMFA) is to analyze the user's experimental intent serially using single-mode information, which cannot fully utilize the intent information of each mode. Therefore, this paper designs a multimodal fusion algorithm (hereinafter referred to as MFA, Algorithm 4) which focuses on the parallel fusion of user's experimental intent. The essence of the MFA is the fusion of multimodal intent probability. At the same time, this paper designs a smart glove based on the virtual-reality fusion experiments, which can integrate multichannel data such as voice, visual, and sensor. This smart glove can not only capture user's experimental intent but also navigate, guide, or warn user's operation behaviors, and it has stronger perception capabilities compared to any other data glove or smart experimental device. The experimental results demonstrate that the smart glove presented in this paper can be widely employed in the chemical experiment teaching based on virtual-reality fusion., Competing Interests: The authors declare that they have no conflicts of interest., (Copyright © 2022 Hongyue Wang et al.)

The current virtual system for secondary school experiments poses several issues, such as limited methods of operation for students and an inability of the system to comprehend the users' operational intentions, resulting in a greater operational burden for students and hindering the goal of the experimental practice. However, many traditional multimodal fusion algorithms rely solely on individual modalities for the analysis of users' experimental intentions, failing to fully utilize the intention information for each modality. To rectify these issues, we present a new multimodal fusion algorithm, MFIRA, which intersects and blends intention probabilities between channels by executing parallel processing of multimodal information at the intention layer. Additionally, we developed an augmented reality (AR) virtual experiment platform based on the Hololens 2, which enables students to conduct experiments using speech, gestures, and vision. Employing the MFIRA algorithm, the system captures users' experimental intent and navigates or rectifies errors to guide students through their experiments. The experimental results indicate that the MFIRA algorithm boasts a 97.3% accuracy rate in terms of interpreting users' experimental intent. Compared to existing experimental platforms, this system is considerably more interactive and immersive for students and is highly applicable in secondary school experimental chemistry classrooms. [ABSTRACT FROM AUTHOR]

Featured Application: Authors are encouraged to provide a concise description of the specific application or a potential application of the work. This section is not mandatory. Chemistry experiments are an important part of chemistry learning, and the development and application of virtual experiments have greatly enriched experimental teaching. However, in the existing virtual experiments, there are problems such as low human–computer interaction efficiency, poor user sense of reality and operation, and a boring experimental process. Therefore, this paper designs a multimodal perception gameplay virtual and real fusion intelligence laboratory (GVRFL). GVRFL uses virtual and real fusion methods to interactively complete chemical experiments, which greatly improves the user's sense of reality and operation. This method proposes a multimodal intention active understanding algorithm to improve the efficiency of human–computer interaction and user experience and proposes a novel game-based virtual–real fusion intelligent experimental mode that adds gameplay to the process of virtual–real fusion experiments. The experimental results show that this method improves the efficiency of human–computer interaction and reduces the user's operating load. At the same time, the interaction between the real experimental equipment and the virtual experimental scene greatly improves the user's sense of reality and operation. The introduction of game elements into the process of virtual and real fusion experiments stimulates students' interest in and enthusiasm for learning. [ABSTRACT FROM AUTHOR]

Despite the emergence of various human-robot collaboration frameworks, most are not sufficiently flexible to adapt to users with different habits. In this article, a Multimodal Reinforcement Learning Human-Robot Collaboration (MRLC) framework is proposed. It integrates reinforcement learning into human-robot collaboration and continuously adapts to the user's habits in the process of collaboration with the user to achieve the effect of human-robot cointegration. With the user's multimodal features as states, the MRLC framework collects the user's speech through natural language processing and employs it to determine the reward of the actions made by the robot. Our experiments demonstrate that the MRLC framework can adapt to the user's habits after repeated learning and better understand the user's intention compared to traditional solutions. [ABSTRACT FROM AUTHOR]

While facing complex surroundings, robots need to identify the same intention which is expressed in different ways. In order to solve the problem of assisting robots to get a better intention understanding, a self-tuning multimodal fusion algorithm is put forward in this paper, which is not restricted by the expressions of interacting participants and environment. The multimodal fusion algorithm can be transferred to different application platforms. Robots can own the understanding competence and adapt new tasks by changing the content of the robot knowledge base. Compared with other multimodal fusion algorithms, this paper attempts to transfer the basic structure of feed-forward neural networks on discrete sets, which has strengthened the consistency and perfect the complementary relations between multiple mode, and has realized the simultaneous operation of fusion operator's self-tuning and intention search. There are three kinds of modes selected in the paper: speech, gesture, and scene objects, where the single modal classifiers are trained separately. This method conducted a human-computer interaction experiment on the bionic robot Pepper platform, which proved that the method can effectively improve the accuracy and robustness of robots in aspects of understanding human intentions, and reduce the uncertainty about intention judgment in a single modal interaction. [ABSTRACT FROM AUTHOR]

This paper proposes an intention understanding algorithm (KDI) based on an elderly service robot, which combines Neural Network with a seminaive Bayesian classifier to infer user's intention. KDI algorithm uses CNN to analyze gesture and action information, and YOLOV3 is used for object detection to provide scene information. Then, we enter them into a seminaive Bayesian classifier and set key properties as super parent to enhance its contribution to an intent, realizing intention understanding based on prior knowledge. In addition, we introduce the actual distance between the users and objects and give each object a different purpose to implement intent understanding based on object-user distance. The two methods are combined to enhance the intention understanding. The main contributions of this paper are as follows: (1) an intention reasoning model (KDI) is proposed based on prior knowledge and distance, which combines Neural Network with seminaive Bayesian classifier. (2) A set of robot accompanying systems based on the robot is formed, which is applied in the elderly service scene. [ABSTRACT FROM AUTHOR]

With the development of deep learning, gesture recognition systems based on the neural network have become quite advanced, but the application effect in the elderly is not ideal. Due to the change of the palm shape of the elderly, the gesture recognition rate of most elderly people is only about 70%. Therefore, in this paper, an intelligent gesture error correction algorithm based on game rules is proposed on the basis of the AlexNet. Firstly, this paper studies the differences between the palms of the elderly and young people. It also analyzes the misread gesture by using the probability statistics method and establishes a misread-gesture database. Then, based on the misreading-gesture library, the maximum channel number of different gestures in the fifth layer is studied by using the similar curve algorithm and the Pearson algorithm. Finally, error correction is completed under the game rule. The experimental results show that the gesture recognition rate of the elderly can be improved to more than 90% by using the proposed intelligent error correction algorithm. The elderly-accompanying robot can understand people's intentions more accurately, which is well received by users. [ABSTRACT FROM AUTHOR]