Search

Autonomously exploring and mapping is one of the open challenges of robotics and artificial intelligence. Especially when the environments are unknown, choosing the optimal navigation directive is not straightforward. In this paper, we propose a reinforcement learning framework for navigating, exploring, and mapping unknown environments. The reinforcement learning agent is in charge of selecting the commands for steering the mobile robot, while a SLAM algorithm estimates the robot pose and maps the environments. The agent, to select optimal actions, is trained to be curious about the world. This concept translates into the introduction of a curiosity-driven reward function that encourages the agent to steer the mobile robot towards unknown and unseen areas of the world and the map. We test our approach in explorations challenges in different indoor environments. The agent trained with the proposed reward function outperforms the agents trained with reward functions commonly used in the literature for solving such tasks.

We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83% compared to 56% in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot.

Reinforcement Learning has been able to solve many complicated robotics tasks without any need for feature engineering in an end-to-end fashion. However, learning the optimal policy directly from the sensory inputs, i.e the observations, often requires processing and storage of a huge amount of data. In the context of robotics, the cost of data from real robotics hardware is usually very high, thus solutions that achieve high sample-efficiency are needed. We propose a method that aims at learning a mapping from the observations into a lower-dimensional state space. This mapping is learned with unsupervised learning using loss functions shaped to incorporate prior knowledge of the environment and the task. Using the samples from the state space, the optimal policy is quickly and efficiently learned. We test the method on several mobile robot navigation tasks in a simulation environment and also on a real robot., Paper Accepted at ICPR2020

A lot of research has been done on the detection of mental workload (MWL) using various bio-signals. Recently, deep learning has allowed for novel methods and results. A plethora of measurement modalities have proven to be valuable in this task, yet studies currently often only use a single modality to classify MWL. The goal of this research was to classify perceived mental workload (PMWL) using a deep neural network (DNN) that flexibly makes use of multiple modalities, in order to allow for feature sharing between modalities. To achieve this goal, an experiment was conducted in which MWL was simulated with the help of verbal logic puzzles. The puzzles came in five levels of difficulty and were presented in a random order. Participants had 1 h to solve as many puzzles as they could. Between puzzles, they gave a difficulty rating between 1 and 7, seven being the highest difficulty. Galvanic skin response, photoplethysmograms, functional near-infrared spectrograms and eye movements were collected simultaneously using LabStreamingLayer (LSL). Marker information from the puzzles was also streamed on LSL. We designed and evaluated a novel intermediate fusion multimodal DNN for the classification of PMWL using the aforementioned four modalities. Two main criteria that guided the design and implementation of our DNN are modularity and generalisability. We were able to classify PMWL within-level accurate (0.985 levels) on a seven-level workload scale using the aforementioned modalities. The model architecture allows for easy addition and removal of modalities without major structural implications because of the modular nature of the design. Furthermore, we showed that our neural network performed better when using multiple modalities, as opposed to a single modality. The dataset and code used in this paper are openly available.

Autonomous robots require high degrees of cognitive and motoric intelligence to come into our everyday life. In non-structured environments and in the presence of uncertainties, such degrees of intelligence are not easy to obtain. Reinforcement learning algorithms have proven to be capable of solving complicated robotics tasks in an end-to-end fashion without any need for hand-crafted features or policies. Especially in the context of robotics, in which the cost of real-world data is usually extremely high, reinforcement learning solutions achieving high sample efficiency are needed. In this paper, we propose a framework combining the learning of a low-dimensional state representation, from high-dimensional observations coming from the robot's raw sensory readings, with the learning of the optimal policy, given the learned state representation. We evaluate our framework in the context of mobile robot navigation in the case of continuous state and action spaces. Moreover, we study the problem of transferring what learned in the simulated virtual environment to the real robot without further retraining using real-world data in the presence of visual and depth distractors, such as lighting changes and moving obstacles., Comment: Paper Accepted at IROS2021. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Human Activity Recognition is focused on the use of sensing technology to classify human activities and to infer human behavior. While traditional machine learning approaches use hand-crafted features to train their models, recent advancements in neural networks allow for automatic feature extraction. Auto-encoders are a type of neural network that can learn complex representations of the data and are commonly used for anomaly detection. In this work we propose a novel multi-class algorithm which consists of an ensemble of auto-encoders where each auto-encoder is associated with a unique class. We compared the proposed approach with other state-of-the-art approaches in the context of human activity recognition. Experimental results show that ensembles of auto-encoders can be efficient, robust and competitive. Moreover, this modular classifier structure allows for more flexible models. For example, the extension of the number of classes, by the inclusion of new auto-encoders, without the necessity to retrain the whole model.

In this research, we investigate the use of Reinforcement Learning (RL) for an effective and robust solution for exploring unknown and indoor environments and reconstructing their maps. We benefit from a Simultaneous Localization and Mapping (SLAM) algorithm for real-time robot localization and mapping. Three different reward functions are compared and tested in different environments with growing complexity. The performances of the three different RL-based path planners are assessed not only on the training environments, but also on an a priori unseen environment to test the generalization properties of the policies. The results indicate that RL-based planners trained to maximize the coverage of the map are able to consistently explore and construct the maps of different indoor environments.

Full-body motion capture typically requires sensors/markers to be placed on each rigid body segment, which results in long setup times and is obtrusive. The number of sensors/markers can be reduced using deep learning or offline methods. However, this requires large training datasets and/or sufficient computational resources. Therefore, we investigate the following research question: &ldquo, What is the performance of a shallow approach, compared to a deep learning one, for estimating time coherent full-body poses using only five inertial sensors?&rdquo, We propose to incorporate past/future inertial sensor information into a stacked input vector, which is fed to a shallow neural network for estimating full-body poses. Shallow and deep learning approaches are compared using the same input vector configurations. Additionally, the inclusion of acceleration input is evaluated. The results show that a shallow learning approach can estimate full-body poses with a similar accuracy (~6 cm) to that of a deep learning approach (~7 cm). However, the jerk errors are smaller using the deep learning approach, which can be the effect of explicit recurrent modelling. Furthermore, it is shown that the delay using a shallow learning approach (72 ms) is smaller than that of a deep learning approach (117 ms).

Background: Most people experience low back pain (LBP) at least once in their life and for some patients this evolves into a chronic condition. One way to prevent acute LBP from transiting into chronic LBP, is to ensure that patients receive the right interventions at the right moment. We started research in the design of a clinical decision support system (CDSS) to support patients with LBP in their self-referral to primary care. For this, we explored the possibilities of using supervised machine learning. We compared the performances of the three classification models -i.e. 1. decision tree, 2. random forest, and 3. boosted tree - to get insight in which model performs best and whether it is already acceptable to use this model in real practice.Methods: The three models were generated by means of supervised machine learning with 70% of a training dataset (1288 cases with 65% GP, 33% physio, 2% self-care cases). The cases in the training dataset were fictive cases on low back pain collected during a vignette study with primary healthcare professionals. We also wanted to know the performance of the models on real-life low back pain cases that were not used to train the models. Therefore we also collected real-life cases on low back pain as test dataset. These cases were collected with the help of patients and healthcare professionals in primary care. For each model, the performance was measured during model validation - with 30% of the training dataset -as well as during model testing -with the test dataset containing real-life cases. The total observed accuracy as well as the kappa, and the sensitivity, specificity, and precision were used as performance measures to compare the models.Results: For the training dataset, the total observed accuracies of the decision tree, the random forest and boosted tree model were 70%, 69%, and 72% respectively. For the test dataset, the total observed accuracies were 71%, 53%, and 71% respectively. The boosted tree appeared to be the best for predicting a referral advice with a fair accuracy (Kappa between 0.2 and 0.4). Next to this, the measured evaluation measures show that all models provided a referral advice better than just a random guess. This means that all models learned some implicit knowledge of the provided referral advices in the training dataset.Conclusions: The study showed promising results on the possibility of using machine learning in the design of our CDSS. The boosted tree model performed best on the classification of low back pain cases, but still has to be improved. Therefore, new cases have to be collected, especially cases that are classified as self-care cases. This to be sure that also the self-care advice can be predicted well by the model.

Social robots should be able to automatically understand and respond to human touch. The meaning of touch does not only depend on the form of touch but also on the context in which the touch takes place. To gain more insight into the factors that are relevant to interpret the meaning of touch within a social context we elicited touch behaviors by letting participants interact with a robot pet companion in the context of different affective scenarios. In a contextualized lab setting, participants (n = 31) acted as if they were coming home in different emotional states (i.e., stressed, depressed, relaxed, and excited) without being given specific instructions on the kinds of behaviors that they should display. Based on video footage of the interactions and interviews we explored the use of touch behaviors, the expressed social messages, and the expected robot pet responses. Results show that emotional state influenced the social messages that were communicated to the robot pet as well as the expected responses. Furthermore, it was found that multimodal cues were used to communicate with the robot pet, that is, participants often talked to the robot pet while touching it and making eye contact. Additionally, the findings of this study indicate that the categorization of touch behaviors into discrete touch gesture categories based on dictionary definitions is not a suitable approach to capture the complex nature of touch behaviors in less controlled settings. These findings can inform the design of a behavioral model for robot pet companions and future directions to interpret touch behaviors in less controlled settings are discussed.

Contains fulltext : 135011.pdf (Publisher’s version ) (Closed access) Visual attention is used to selectively filter relevant information depending on current task demands and goals. Visual attention is called object-based attention when it is directed to coherent forms or objects in the visual field. This study used real-time functional magnetic resonance imaging for moment-to-moment decoding of attention to spatially overlapped objects belonging to two different object categories. First, a whole-brain classifier was trained on pictures of faces and places. Subjects then saw transparently overlapped pictures of a face and a place, and attended to only one of them while ignoring the other. The category of the attended object, face or place, was decoded on a scan-by-scan basis using the previously trained decoder. The decoder performed at 77.6% accuracy indicating that despite competing bottom-up sensory input, object-based visual attention biased neural patterns towards that of the attended object. Furthermore, a comparison between different classification approaches indicated that the representation of faces and places is distributed rather than focal. This implies that real-time decoding of object-based attention requires a multivariate decoding approach that can detect these distributed patterns of cortical activity. 11 p.

Item does not contain fulltext A brain-computer interface (BCI) enables direct communication from the brain to devices, bypassing the traditional pathway of peripheral nerves and muscles. Traditional approaches to BCIs require the user to train for weeks or even months to learn to control the BCI. In contrast, BCIs based on machine learning only require a calibration session of less than an hour before the system can be used, since the machine adapts to the user's existing brain signals. However, this calibration session has to be repeated before each use of the BCI due to inter-session variability, which makes using a BCI still a time-consuming and an error-prone enterprise. In this work, we present a second-order baselining procedure that reduces these variations, and enables the creation of a BCI that can be applied to new subjects without such a calibration session. The method was validated with a motor-imagery classification task performed by 109 subjects. Results showed that our subject-independent BCI without calibration performs as well as the popular common spatial patterns (CSP)-based BCI that does use a calibration session.

We propose a novel approach to developing a tractable affective dialogue model for probabilistic frame-based dialogue systems. The affective dialogue model, based on Partially Observable Markov Decision Process (POMDP) and Dynamic Decision Network (DDN) techniques, is composed of two main parts: the slot-level dialogue manager and the global dialogue manager. It has two new features: (1) being able to deal with a large number of slots and (2) being able to take into account some aspects of the user's affective state in deriving the adaptive dialogue strategies. Our implemented prototype dialogue manager can handle hundreds of slots, where each individual slot might have hundreds of values. Our approach is illustrated through a route navigation example in the crisis management domain. We conducted various experiments to evaluate our approach and to compare it with approximate POMDP techniques and handcrafted policies. The experimental results showed that the DDN–POMDP policy outperforms three handcrafted policies when the user's action error is induced by stress as well as when the observation error increases. Further, performance of the one-step look-ahead DDN–POMDP policy after optimizing its internal reward is close to state-of-the-art approximate POMDP counterparts.

Touch is an important interaction modality in social interaction, for instance touch can communicate emotions and can intensify emotions communicated by other modalities. In this paper we explore the use of Neural Networks for the classification of touch. The exploration and assessment of Neural Networks (NNs) is based on the Corpus of Social Touch established by Jung et al. This corpus was split in a train set (65%) and test set (35%), the train set was used to find the optimal parameters for the NN and for training the final model. Also different feature sets were investigated; the basic feature set included in the corpus, energy-histogram and dynamical features. Using all features led to the best performance of 64% on the test set, using a NN consisting of one hidden layer with 46 neurones. The confusion matrix showed the expected high confusion between pat-tap and grab-squeeze. A leave-one-subject-out approach lead to a performance of 54%, which is comparable with the results of Jung et al.

Automatic sleep staging on an online basis has recently emerged as a research topic motivated by fundamental sleep research. The aim of this paper is to find optimal signal processing methods and machine learning algorithms to achieve online sleep staging on the basis of a single EEG signal. The classification performance obtained using six different EEG signals and various signal processing feature sets is compared using the kappa statistic which has very recently become popular in sleep staging research. A variable duration of the EEG segment (or epoch) to decide on the sleep stage is also analyzed. Spectral-domain, time-domain, linear, and nonlinear features are compared in terms of performance and two types of machine learning approaches (random forests and support vector machines) are assessed. We have determined that frontal EEG signals, with spectral linear features, epoch durations between 18 and 30 seconds, and a random forest classifier lead to optimal classification performance while ensuring real-time online operation.

Brain-computer interfaces offer a valuable input modality, which unfortunately comes also with a high degree of uncertainty. There are simple methods to improve detection accuracy after the incoming brain activity has already been classified, which can be divided into (1) gathering additional evidence from other sources of information, and (2) transforming the unstable classification results to be more easy to control. The methods described are easy to implement, but it is essential to apply them in the right way. This paper provides an overview of the different techniques, showing where to apply them and comparing the effects. Detection accuracy is important, but there are trade-offs to consider. Future research should investigate the effectiveness of these methods in their context of use, as well as the optimal settings to obtain the right balance between functionality and meeting the user's expectations for maximum acceptance.

Offline analysis pipelines have been developed and evaluated for the detection of covert attention from electroen-cephalography recordings, and the detection of overt attention in terms of eye movement based on electrooculographic measurements. Some additional analysis were done in order to prepare the pipelines for use in a real-time system. This real-time system and a game application in which these pipelines are to be used were implemented. The game is set in a virtual environment where player is a wildlife photographer on an uninhabited island. Overt attention is used to adjust the angle of the first person camera, when the player is tracking animals. When making a photograph, the animal will flee when it notices it is looked at directly, so covert attention is required to get a good shot. Future work will entail user tests with this system to evaluate usability, user experience, and characteristics of the signals related to overt and covert attention when used in such an immersive environment.

The multimodal, multi-paradigm brain-computer interfacing (BCI) game Bacteria Hunt was used to evaluate two aspects of BCI interaction in a gaming context. One goal was to examine the effect of feedback on the ability of the user to manipulate his mental state of relaxation. This was done by having one condition in which the subject played the game with real feedback, and another with sham feedback. The feedback did not seem to affect the game experience (such as sense of control and tension) or the objective indicators of relaxation, alpha activity and heart rate. The results are discussed with regard to clinical neurofeedback studies. The second goal was to look into possible interactions between the two BCI paradigms used in the game: steady-state visually-evoked potentials (SSVEP) as an indicator of concentration, and alpha activity as a measure of relaxation. SSVEP stimulation activates the cortex and can thus block the alpha rhythm. Despite this effect, subjects were able to keep their alpha power up, in compliance with the instructed relaxation task. In addition to the main goals, a new SSVEP detection algorithm was developed and evaluated. © 2010 The Author(s).

This paper covers part of a study which concerns the feasibility of real-time word spotting with partially recurrent neural networks (PRNN’s). PRNN’s have already proven appropriate for other examples of pure sequence recognition [1, 2]. However choices concerning architectural and learning aspects are still hard to make. One of the questions still to be answered, is how these aspects influence the term of memory of a PRNN. This paper tries to obtain some directives regarding architectures and learning algorithms.

In this paper a rule extraction method for a multilayered neural network is proposed. The method is not only able to extract rules from a neural network trained with boolean valued inputs but also from a neural network trained with continuous valued inputs.