Search

Recently, there has been an increase in the importance of community-centric systems as a new paradigm to enhance the quality of community (QOC). Social media plays an important role in creating, sharing, and exchanging information within a community. However, assistive technologies should be developed from human-centric and community-centric points of view to realize such information support. In this paper, we discuss the use of smart devices interlocked robot partners for interactive information support similar to concierge services in hotels. The interactive information support system is composed of two main subsystems, namely robot partners and informationally structured space servers. A robot partner performs communication and interaction with people through voice recognition and gesture recognition in addition to the use of touch interfaces. The informationally structured space server receives the measurement data of human motions and personal information containing human requests and preferences from the robot partner. Next, the informationally structured space server selects and recommends shops, restaurants, and sightseeing spots to the guests and visitors through utterance and display by the robot partner. First, we explain the concept of the informationally structured space to connect a person with sensory information and propose the overall architecture of the informationally structured space. Next, we explain how to provide information support using smart devices interlocked robot partners based on the informationally structured space. In addition, we describe several social experiments on interactive information support at hotels. Finally, we discuss the effectiveness of the proposed system and the future direction of community-centric systems.

In Japan, vermin such as deer and wild boar do significant harm to agriculture. Extermination with firearms and traps has been used, and the amount of damage has been steadily declining in recent years, but more effective extermination is still needed. We demonstrate the feasibility of an intelligent capture device based on off-the-shelf net launchers in this article. An image recognition system and a tracking system are included in the proposed system. To recognize animals, the image recognition system employs a technique known as Single Shot Multibox Detection (SSD), while the tracking system employs a pan-tilt process. We demonstrate that the proposed system can trap animals by integrating the two mechanisms.

In recent years, due to the shortage of labor caused by an aging population and low birth rate in Japan, various research on robots has been conducted. There are many industries in which applying autonomous mobile robots are important, for instance, the inspection system for waterway tunnels. Entering the waterway tunnel is a difficult task for humans, thus, the dark and damp environment of a waterway tunnel is unfriendly to humans both in the physical way and mental ways. To solve that problem, various vSLAM methods have been proposed in the past, including feature-based methods, direct methods, and RGB-D methods. However, bad illuminance condition and unclear texture of the ceiling and walls of a waterway tunnel brings difficulties to many vSLAM algorithms and frameworks. To improve the accuracy of vSLAM, various methods have been proposed, such as sensor fusion. In this paper, we proposed a system that contains an online tracking using Intel REALSENSE T265 and an offline map reconstruction based on opengl using the data gathered by Intel REALSENSE T265. For the experiments, we conducted two comparison experiments based on openvslam framework and our proposed system. The results show that our proposed system can successfully track the camera positions and construct a map.

In this paper, an unsupervised learning model of episodic memory is proposed. The proposed model, enhanced episodic memory adaptive resonance theory (EEM-ART), categorizes and encodes experiences of a robot to the environment and generates a cognitive map. EEM-ART consists of multilayer ART networks to extract novel events and encode spatio-temporal connection as episodes by incrementally generating cognitive neurons. The model connects episodes to construct a sensorimotor map for the robot to continuously perform path planning and goal navigation. Experimental results for a mobile robot indicate that EEM-ART can process multiple sensory sources for learning events and encoding episodes simultaneously. The model overcomes perceptual aliasing and robot localization by recalling the encoded episodes with a new anticipation function and generates sensorimotor map to connect episodes together to execute tasks continuously with little to no human intervention.

This paper presents a framework for classifying human hand pose, especially in grasping object intuitively. First, we propose a system based on the stereo infra-red image as a sensor that can produce hand coordinates in 3-dimensional space. We use egocentric vision because it can get uniform and natural data with only a single sensor module. Second, we transformed the position to get the angle information for each joint on the finger. Third, we designed an intelligent system based on Multi-Layer Perceptron (MLP) to process angular data to obtain classification results according to the Cutkosky grasp taxonomy. Finally, we compared the results on several similar objects and evaluated their classification accuracy. In the validation phase, the results yielded an accuracy of 16 grasp pose classification is 89,60%. In real-time testing, the results yielded an accuracy of 81.93%. This result shows feature-based learning can reduce the complexity and training time of the MLP. Furthermore, a small amount of training data is sufficient for the training and implementation.

This paper proposes an ecological approach for object relationship extraction. First, we explain the ecological approach used to develop our previous works in the perception of a partner robot. Second, we developed a framework for finding the relation of objects based on scenarios of daily human activities. We represent the relationships among objects in a graph consisting of nodes and edges. Graph Convolutional Networks (GCN) is used to train the graph and perform semi-supervised classification. Third, we generate and prepare the datasets and then train the GCN. Finally, we present training visualization in two different scenarios. The proposed methods allow the system to identify the object based on its relationship. This approach is then used as a starting point to implement the ecological factors in elderly care robot applications.

In recent years, social robots are widely research due to the advancement of electronics hardware and deep learning algorithms breakthrough. Hand gesture recognition playing an important role in human-robot interaction. However, to deploy state-of-art of deep learning methods for hand gesture recognition, researchers have to associate all hand gestures with their respective meanings (label). Re-train the deep learning model is required if a new gesture added for recognition. In addition, the meaning of hand gestures often depending on the environmental conditions. In this paper, we propose a framework that learns and recognizes hand gesture meaning based on surrounding objects. The proposed method consists of two sub frameworks: i) A deep learning model, Mask RCNN to detect and extract features of objects and hand gestures; ii) An incremental learning model, Declarative Memory Recurrent Neural Model to incrementally learn hand gestures meaning based on surrounding objects. The effectiveness of our proposed method is validated through several experiments.

Recently, various types of robot partners aiming at social implementation have been developed and researched, but they have not yet been popularized. In the case of communication robots, it has been reported that it is easier and more comfortable for humans to communicate with multiple robots. However, multirobots require large installation space; the role or character of each robot is not clear, and it is difficult for users to design utterance and action contents. Therefore, we propose a new design methodology of the body-sharing multi-robot system and develop a kangaroo-based robot that contains two physically interconnected objects as one of the models of multiple communication robots. Next, we show the evaluation of the proposed robot in terms of impression and discuss the possibility of social implementation of the developed body sharing multirobot system.

Personal mobility vehicles (PMV) with the autonomous driving system is required for supporting movement-limited persons. To realize their safe movement, researchers extract pedestrian features from images without considering pedestrian characteristics. However, before these extractions, it is also important to estimate which person should be paid attention and assigning priority order by the degree of danger for PMV’s collision avoidance. In this paper, we propose a deep neural model for pedestrian detection with danger estimation. Finally, we show some experimental results and discuss parameters that are important for pedestrian danger estimation for each experimental scenario.

Falls are a major health issue, particularly among the elderly. Increasing fall events require high service quality and dedicated medical treatment which is an economic burden. In the lack of appropriate care and support, serious injuries caused by fall will cost lives. Therefore, tracking systems with fall detection capabilities are required. Static-view sensors with machine learning techniques for human fall detection have been widely studied and achieved significant results. However, these systems unable to monitor a person if he or she is out of viewing angle which greatly impedes its performance. Mobile robots are an alternative for keeping the person in sight. However, existing mobile robots are unable to operate for a long time due to battery issues and movement constraints in complex environments. In this paper, we proposed a lightweight deep learning vision-based model for human fall detection with an assistive robot to provide assistance when a fall happens. The proposed detection system requires less computational power which can be implemented in a low-cost 2D camera and GPU board for real-time monitoring. The assistive robot equipped with various sensors that can perform SLAM, obstacle avoidance and navigation autonomously. Our proposed system integrates these two sub-systems to compensate for the weakness of each other to constitute a system that robust, adaptable, and high performance. The proposed method has been validated through a series of experiments.

Recently, as the rapid progress of information and communication technology, robot technology, and artificial intelligence, we have become to build a higher level of safe, comfortable, and smart society coexisting with advanced technology. Methamphetamine addiction has become a major human social problem in the world. Traditional approaches of detecting methamphetamine through hair, skin, urine, and blood fluid are financially inefficient, time-consuming, and in some cases too complicated. Providing reliable and trustworthy detection with the highest precision and accuracy is of a challenging task. This paper proposes ensemble learning using a soft voting approach to improve the accuracy of detection. First, we trained five individual classifiers, namely adaptive neuro-fuzzy inference system (ANFIS), random forest, multilayer perceptron (MLP), k-nearest neighbor (k-NN), and support vector machine (SVM) on the same urine dataset. We then created new ensemble learning using the soft voting approach by averaging the probability of individual ANFIS, random forest, MLP, k-NN, and SVM. Firefly algorithm for weight optimization is used to strengthen individual classifiers to form an ensemble and increase the prediction accuracy. Our proposed ensemble produces an accuracy value of 100% compared to the individual classifiers mentioned above.

Activity recognition which aims to accurately distinguish human actions in complex environments plays a key role in human-robot/computer interaction. However, long-lasting and similar actions will cause poor feature sequence extraction and thus lead to a reduction of the recognition accuracy. We propose a novel discriminative deep model (D3D-LSTM) based on 3D-CNN and LSTM for both single-target and interaction action recognition to improve the spatiotemporal processing performance. Our models have several notable properties: 1) A real-time feature fusion method is used to obtain a more representative feature sequence through composition of local mixtures for enhancing the performance of discriminating similar actions; 2) We introduce an improved attention mechanism that focuses on each frame individually by assigning different weights in real-time; 3) An alternating optimization strategy is proposed for our model to obtain parameters with the best performance. Because the proposed D3D-LSTM model is efficient enough to be used as a detector that recognizes various activities, a Real-set database is collected to evaluate action recognition in complex real-world scenarios. For long-term relations, we update the present memory state via the weight-controlled attention module that enables the memory cell to store better long-term features. The densely connected bimodal modal makes local perceptrons of 3D-Conv motion-aware and stores better short-term features. The proposed D3D-LSTM model has been evaluated through a series of experiments on the Real-set and open-source datasets, i.e. SBU-Kinect and MSR-action-3D. Experimental results show that the proposed D3D-LSTM model achieves new state-of-the-art results, including pushing the average rate of the SBU-Kinect to 92.40% and the average rate of the MSR-action-3D to 95.40%.

This paper presents a method with dynamic density topological structure generation for low-cost real-time vertical ladder detection from 3D point cloud data. Dynamic Density Growing Neural Gas (DD-GNG) is proposed to generate a dynamic density of the topological structure. The density of the structure and the number of nodes will be increased in the targeted object area. Feature extraction model is required to classify suspected objects for being processed in the next time process. After that, rungs of the vertical ladder is processed using an inlier-outlier method. Thus, the ladder detection model represents the ladder with a set of nodes and edges. Next, affordance detection is processed for detecting the feasible grasped location. To validate the effectiveness of the proposed method, a series of experiments are conducted on a 4-legged robot with a non-GPU board for real-time vertical ladder detection and climbing to validate the effectiveness of the proposed method. Results show that our proposed method able to detect and track the ladder structure in real-time with a much lower computational cost. The affordance of the ladder provides safety information for robot grasping.

Biological information inspires the advancement of a navigational mechanism for autonomous robots to help people explore and map real-world environments. However, the robot’s ability to constantly acquire environmental information in real-world, dynamic environments has remained a challenge for many years. In this paper, we propose a self-organizing adaptive recurrent incremental network that models human episodic memory to learn spatiotemporal representations from novel sensory data. The proposed method termed as SOARIN consists of two main learning process that is active learning and episodic memory playback. For active learning (robot exploration), SOARIN quickly learns and adapts incoming novel sensory data as episodic neurons via competitive Hebbian Learning. Episodic neurons are connecting with each other and gradually forms a spatial map that can be used for robot localization. Episodic memory playback is triggered whenever the robot is in an inactive mode (charging or hibernating). During playback, SOARIN gradually integrates knowledge and experience into more consolidate spatial map structures that can overcome the catastrophic forgetting. The proposed method is analyzed and evaluated in term of map learning and localization through a series of real robot experiments in real-world indoor environments.

Due to many applications such as video surveillance, human machines interaction and video recovery, human actions recognition was a significant topic in computer vision. This paper proposes a self-organizing recurrent incremental network (SORIN) for human action recognition using human skeleton information. The proposed method models human working memory and episodic memory and comprises two layers of adaptive recurrent Growing-When-Required (ar-GWR) network that connected hierarchically. The working memory layer continually learns incoming perception information and encodes the learned knowledge as neurons. Similarly, the episodic memory layer further learns the spatiotemporal relationship of neurons from working memory as episode neurons to realize human actions incrementally. The proposed method integrates with OpenPose framework for human skeleton action recognition and it is validated through a series of experiments.

Biologically inspired computational techniques play a crucial role in robotic cognition. Artificial learning agents and robots that interact in complex environments must constantly acquire and refine knowledge over long periods of time. In this paper, we propose a novel recurrent neural architecture that mimics humans’ declarative memory system for continuously generating a cognitive map during robot navigation. The proposed method termed as Declarative Memory Adaptive Recurrent Model (DM-ARM), and consists of three hierarchical memory courses: (i) Working Memory, (ii) Episodic Memory and (iii) Semantic Memory layer. Each memory layer comprises a self-organizing adaptive recurrent incremental network (SOARIN) with a different learning task respectively. The Working Memory layer quickly clusters sensory information while the Episodic Memory layer learns fine-grained spatiotemporal relationships of clusters (temporal encoding). Both the memory layer learning is in an unsupervised manner. The Semantic Memory layer utilizes task-relevant cues to adjust the level of architectural flexibility and generate a semantic map that contains more compact episodic representations. The effectiveness of the proposed recurrent neural architecture is evaluated through a series of experiments. We implemented and validated our proposed work on the tasks of robot navigation.

This paper presents a new integration model for on-line topological map building with environment slow features detection. The proposed model is formed by the integration of Bayesian Adaptive Resonance Associative Memory (BARAM) and Incremental Slow Feature Analysis (IncSFA). IncSFA incrementally extracts slowly varying features from a rapidly changing input signal. These slow features will be fed to BARAM for environment learning to build a topological map. The explored environment is represented as a set of neurons (nodes) and edges that connecting all nodes. Each neuron represents a distinct place and edges store robot traverse information that leads the robot to travel from one node to another. The proposed model is an unsupervised learning technique that does not require any prior knowledge of what an environment is supposed to be for ease of implementation. The effectiveness of our proposed method is validated by several standardized benchmark datasets.

Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and recognition.

In this paper, an episodic memory learning framework is proposed for categorizing and encoding sensory information that acquired from a robot for environment adaptation and sensorimotor map building. The proposed learning model termed as Incremental Episodic Memory Adaptive Resonance Theory (In-EMART), consists two layers of ART networks which used to detect novel event encountered by the robot and learn the spatio-temporal relationship by creating neurons incrementally. A set of connected episodes forms a sensorimotor map that can be used for path planning and goal navigation autonomously. The experimental results for a mobile robot show that: (i) In-EMART can learn sensory data in real time which is important for robot implementation; (ii) the model solves the perceptual aliasing issue by recalling the connected episode neurons; (iii) compared with previous works, the proposed method further generates a sensorimotor map for connecting episodes together to navigate from one place to another continuously.

In this paper, we present a novel biologically inspired evolving neural oscillator for quadruped robot locomotion to minimize constraints during the locomotion process. The proposed sensory-motor coordination model is formed by the interconnection between motor and sensory neurons. The model utilizes Bacterial Programming to reconstruct the number of joints and neurons in each joint based on environmental conditions. Bacterial Programming is inspired by the evolutionary process of bacteria that includes bacterial mutation and gene transfer process. In this system, either the number of joints, the number of neurons, or the interconnection structure are changing dynamically depending on the sensory information from sensors equipped on the robot. The proposed model is simulated in computer for realizing the optimization process and the optimized structure is then applied to a real quadruped robot for locomotion process. The optimizing process is based on tree structure optimization to simplify the sensory-motor interconnection structure. The proposed model was validated by series of real robot experiments in different environmental conditions.

Topological structure learning methods are expected for the field of data mining for extracting multiscale topological structures from an unknown dataset. In this paper, we introduce the unsupervised neural network method for topological structure learning method from point clouds for map building. We propose Batch Learning GNG (BL-GNG) in order to improve the learning convergence. BL-GNG uses an objective function based on Fuzzy C-means for improving the learning convergence. Finally, we conduct on several experiments for evaluating our proposed method by comparing to other hierarchical approaches, and discuss the effectiveness of our proposed method.

In this article, we propose a new approach for mobile robot localization and hybrid map building simultaneously without using any odometry hardware system. The proposed method termed as Genetic Bayesian ARAM comprises two main components: (1) steady-state genetic algorithm (SSGA) for self-localization and occupancy grid map building and (2) Bayesian Adaptive Resonance Associative Memory (ARAM) for online topological map building. The model of the explored environment is formed as a hybrid representation, both topological and grid based, and it is incrementally constructed during the exploration process. During occupancy map building, the robot-estimated self-position is updated by SSGA. At the same time, the robot-estimated self-position is transmitted to Bayesian ARAM for topological map building and localization. The effectiveness of our proposed approach is validated by a number of standardized benchmark datasets and real experimental results carried on the mobile robot. Benchmark datasets are used to verify the proposed method capable of generating topological map in different environment conditions. Real robot experiment to verify the proposed method can be implemented in real world.

This paper presents a new framework for mobile robot to perform localization and build topological-metric hybrid map simultaneously. The proposed framework termed as Genetic Bayesian ARAM consists of two main components: 1) Steady state genetic algorithm (SSGA) for self-localization and occupancy grid map building and 2) Bayesian Adaptive Resonance Associative Memory (ARAM) for topological map building. The proposed method is validated using a mobile robot. Result show that Genetic Bayesian ARAM capable of generate hybrid map online and perform localization simultaneously.

This paper proposes a new neural architecture called Topological Gaussian ARAM (TGARAM) for Simultaneous Localization and Mapping (SLAM). TGARAM is integrating the Gaussian classifier with the incremental topology-learning mechanisms of the Growing Neural Gas (GNG) model for online learning of multidimensional inputs and topological map building. By using the Gaussian classifier, the sensitivity to noise on a number of benchmarks data sets is diminished, and it learns a more efficient internal representation of a mapping. The incremental topology-learning mechanisms of GNG enable TGARAM to connect the generated nodes and build a topology-preserving map. In addition, TGARAM retains multi-channel ARAM network architecture and thus capable to learn multiple mappings simultaneously across multi-modal input patterns, in an online and incremental manner. Multiple sensory sources can be transmitted to TGARAM to build a topological map and improve the estimation of localization, in order to be as generic as possible. The proposed method enables an autonomous agent to perform SLAM in an unknown environment. Finally, we validate the proposed method, through several experiments with several benchmark datasets.