Search

Despite the emergence of various human-robot collaboration frameworks, most are not sufficiently flexible to adapt to users with different habits. In this article, a Multimodal Reinforcement Learning Human-Robot Collaboration (MRLC) framework is proposed. It integrates reinforcement learning into human-robot collaboration and continuously adapts to the user's habits in the process of collaboration with the user to achieve the effect of human-robot cointegration. With the user's multimodal features as states, the MRLC framework collects the user's speech through natural language processing and employs it to determine the reward of the actions made by the robot. Our experiments demonstrate that the MRLC framework can adapt to the user's habits after repeated learning and better understand the user's intention compared to traditional solutions. [ABSTRACT FROM AUTHOR]

While facing complex surroundings, robots need to identify the same intention which is expressed in different ways. In order to solve the problem of assisting robots to get a better intention understanding, a self-tuning multimodal fusion algorithm is put forward in this paper, which is not restricted by the expressions of interacting participants and environment. The multimodal fusion algorithm can be transferred to different application platforms. Robots can own the understanding competence and adapt new tasks by changing the content of the robot knowledge base. Compared with other multimodal fusion algorithms, this paper attempts to transfer the basic structure of feed-forward neural networks on discrete sets, which has strengthened the consistency and perfect the complementary relations between multiple mode, and has realized the simultaneous operation of fusion operator's self-tuning and intention search. There are three kinds of modes selected in the paper: speech, gesture, and scene objects, where the single modal classifiers are trained separately. This method conducted a human-computer interaction experiment on the bionic robot Pepper platform, which proved that the method can effectively improve the accuracy and robustness of robots in aspects of understanding human intentions, and reduce the uncertainty about intention judgment in a single modal interaction. [ABSTRACT FROM AUTHOR]

A growing number of studies have been conducted over the past few years on the positioning of daily massage robots. However, most methods used for research have low interactivity, and a systematic method should be designed for accurate and intelligent positioning, thus compromising usability and user experience. In this study, a massage positioning algorithm with online learning capabilities is presented. The algorithm has the following main innovations: (1) autonomous massage localization can be achieved by gaining insights into natural human-machine interaction behavior and (2) online learning of user massage habits can be achieved by integrating recursive Bayesian ideas. As revealed by the experimental results, combining natural human-computer interaction and online learning with massage positioning is capable of helping people get rid of positioning aids, reducing their psychological and cognitive load, and achieving a more desirable positioning effect. Furthermore, the results of the analysis of user evaluations further verify the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes an intention understanding algorithm (KDI) based on an elderly service robot, which combines Neural Network with a seminaive Bayesian classifier to infer user's intention. KDI algorithm uses CNN to analyze gesture and action information, and YOLOV3 is used for object detection to provide scene information. Then, we enter them into a seminaive Bayesian classifier and set key properties as super parent to enhance its contribution to an intent, realizing intention understanding based on prior knowledge. In addition, we introduce the actual distance between the users and objects and give each object a different purpose to implement intent understanding based on object-user distance. The two methods are combined to enhance the intention understanding. The main contributions of this paper are as follows: (1) an intention reasoning model (KDI) is proposed based on prior knowledge and distance, which combines Neural Network with seminaive Bayesian classifier. (2) A set of robot accompanying systems based on the robot is formed, which is applied in the elderly service scene. [ABSTRACT FROM AUTHOR]

Abstract: Human hand shape features extraction from image frame sequences is one of the key steps in human hand 2D/3D tracking system and human hand shape recognition system. In order to satisfy the need of human hand tracking in real time, a fast and accurate method for acquirement of edge features from human hand images without consideration of hand over face is put forward in this paper. The proposed approach is composed of two steps, the coarse location phase (CLP) and the refined location phase (RLP) from coarseness to refinement. In the phase of CLP, the hand contour is approximately described by a polygon with concave and convex, an approach to obtaining hand shape polygon using locating points and locating lines is meticulously discussed. Then, a coarse location (CL) algorithm for extraction of interested hand shape features, such as contour, fingertips, roots of fingers, joints and the intersection of knuckle on different fingers, is proposed. In the phase of RLP, a multi-scale approach is introduced into our study to refine the features obtained by the CL algorithm. By means of defining the response strength of different types of features, a refined location (RL) algorithm is proposed. The major contribution of this paper is that the novel detection operators for features of hand images are presented in the above two steps, which have been successfully applied to our 3D hand shape tracking system and 2D hand shape recognition system. A number of comparative studies with real images and online videos demonstrate that the proposed method can extract the three defined human hand image features with high accuracy and high speed. [ABSTRACT FROM AUTHOR]

The ubiquitous use and advancement in built-in smartphone sensors and the development in big data processing have been beneficial in several fields including healthcare. Among the basic vitals monitoring, pulse rate monitoring is the most important healthcare necessity. A multimedia video stream data acquired by built-in smartphone camera can be used to estimate it. In this paper, an algorithm that uses only smartphone camera as a sensor to estimate pulse rate using PhotoPlethysmograph (PPG) signals is proposed. The results obtained by the proposed algorithm are compared with the actual pulse rate and the maximum error found is 3 beats per minute. The standard deviation in percentage error and percentage accuracy is found to be 0.68 % whereas the average percentage error and percentage accuracy is found to be 1.98 % and 98.02 % respectively. [ABSTRACT FROM AUTHOR]

Coronal Mass Ejection (CME) is a major solar activity that affects the earth, thus CMEs detection is of great importance for space weather forecast, disaster prevention and reduction. We model the detection of Coronal Mass Ejections (CMEs) as the classification of the brightest block in the current running difference image. Because CMEs usually correspond to the areas with high gray values or complex texture features, multiple features including gray features and texture features are extracted to represent the brightest block. And classifier is designed based on these features. Our method includes four steps: first, because the CMEs spread along the radial direction of the sun, in order to facilitate the analysis, the original coordinate is transformed into the polar coordinate; Secondly, because the typical appearance of the CMEs is bright or complex texture enhancement, we use the brightest block to represent the whole image; Thirdly, we extract the gray, texture and HOG features of the brightest gray blocks. Finally, we use the extracted features to design decision trees as the base classifiers, and AdaBoost is used to obtain the final ensemble classifier. As far as we know, this is the first time that the learning based classification framework is presented in the CMEs detection. Moreover, multiple feature fusion is first used to model the various CMEs. Experimental results show that the integration of multi-feature based detection algorithm proposed can achieve better detection results. [ABSTRACT FROM AUTHOR]