Search

Locomotion in animals provides a model for adaptive behavior as it is able to deal with various kinds of perturbations. Work in insects suggests that this evolved flexibility results from a modular architecture, which can be characterized by a recurrent neural network allowing for various emerging attractor states. Whereas a lower control-level coordinates joint movements on a short timescale, a higher-level handles action selection on longer timescales. Implementation of such a control system on a walking hexapod robot was able to deal with various walking patterns including disturbances such as uneven terrain or loss of a leg. Here, we propose a cognitive expansion to the adaptive control system that allows dealing with novel challenging situations. This approach makes use of an internal simulation-based planner that is triggered when the model-free controller fails to recover from an unstable pose. Using a grounded internal body model, the planner then tries, in internal simulation, different solutions out of context, and thus, proposes a new plan to be executed on the real robot. We demonstrate the feasibility of this control approach for walking over terrain with uncertain footholds in three scenarios.

For incremental machine-learning applications it is often important to robustly estimate the system accuracy during training, especially if humans perform the supervised teaching. Cross-validation and interleaved test/train error are here the standard supervised approaches. We propose a novel semi-supervised accuracy estimation approach that clearly outperforms these two methods. We introduce the Configram Estimation (CGEM) approach to predict the accuracy of any classifier that delivers confidences. By calculating classification confidences for unseen samples, it is possible to train an offline regression model, capable of predicting the classifier&rsquo, s accuracy on novel data in a semi-supervised fashion. We evaluate our method with several diverse classifiers and on analytical and real-world benchmark data sets for both incremental and active learning. The results show that our novel method improves accuracy estimation over standard methods and requires less supervised training data after deployment of the model. We demonstrate the application of our approach to a challenging robot object recognition task, where the human teacher can use our method to judge sufficient training.

With this work we are explaining the "You Only Look Once" (YOLO) single-stage object detection approach as a parallel classification of 10647 fixed region proposals. We support this view by showing that each of YOLOs output pixel is attentive to a specific sub-region of previous layers, comparable to a local region proposal. This understanding reduces the conceptual gap between YOLO-like single-stage object detection models, RCNN-like two-stage region proposal based models, and ResNet-like image classification models. In addition, we created interactive exploration tools for a better visual understanding of the YOLO information processing streams: https://limchr.github.io/yolo_visualization

We examine the mechanisms required to handle everyday activities from the standpoint of cognitive robotics, distinguishing activities on the basis of complexity and transparency. Task complexity (simple or complex) reflects the intrinsic nature of a task, while task transparency (easy or difficult) reflects an agent's ability to identify a solution strategy in a given task. We show how the CRAM cognitive architecture allows a robot to carry out simple and complex activities such as laying a table for a meal and loading a dishwasher afterward. It achieves this by using generalized action plans that exploit reasoning with modular, composable knowledge chunks representing general knowledge to transform underdetermined everyday action requests into motion plans that successfully accomplish the required task. Noting that CRAM does not yet have the ability to deal with difficult activities, we leverage insights from the situation model perspective on the cognitive mechanisms underlying flexible context-sensitive behavior with a view to extending CRAM to overcome thisdeficit. © 2021 The Authors. Topics in Cognitive Science published by Wiley Periodicals LLC on behalf of Cognitive Science Society.

We present an approach based on conditional generative adversarial networks (GANs) to generate grasps directly and in a feed-forward manner from a raw depth image input. Building on the recently introduced StyleGAN architecture we extend results from an earlier proof-of-concept paper [1] and demonstrate successful sim2real transfer of grasp outputs for a robot arm with a Shadow Dexterous Hand. We find that the GAN model, which was only trained on a limited set of primitive objects, was able to generalize to a range of everyday real-world objects that differed significantly from the primitive objects used in simulation training. In contrast to discriminative models, the approach learns a latent representation in the set of feasible grasps that can be used for navigation in grasp space and thus allows smooth integration with other motion planning tools.

In daily life, humans use their limbs to perform various movements to interact with an external environment. Thanks to limb's variable and adaptive stiffness, humans can adapt their movements to unstable dynamics of the external environments. The underlying adaptive mechanism has been investigated, employing a simple planar device perturbed by external 2D force patterns. In the present work, we will employ a more advanced, compliant robot arm to extend previous work to a more realistic 3D-setting. We study the adaptive mechanism and use machine learning to capture the human adaptation behavior. In order to model human's stiffness adaptive skill, we give human subjects the task to reach for a target by moving a handle assembled on the end-effector of a compliant robotic arm. The arm is force controlled and the human is required to navigate the handle inside a non-visible, virtual maze and explore it only through robot force feedback when contacting maze virtual walls. By sampling the hand's position and force data, a computational model based on a combination of model predictive control and nonlinear regression is used to predict participants' successful trials. Our study shows that participants selectively increased the stiffness within the axis direction of uncertainty in order to compensate for instability caused by a divergent external force field. The learned controller was able to successfully mimic this behavior. When it is deployed on the robot for the navigation task, the robot arm successfully adapt to the unstable dynamics in the virtual maze, in a similar manner as observed in the participants' adaptation skill.

Decentralization is a central characteristic of biological motor control that allows for fast responses relying on local sensory information. In contrast, the current trend of Deep Reinforcement Learning (DRL) based approaches to motor control follows a centralized paradigm using a single, holistic controller that has to untangle the whole input information space. This motivates to ask whether decentralization as seen in biological control architectures might also be beneficial for embodied sensori-motor control systems when using DRL. To answer this question, we provide an analysis and comparison of eight control architectures for adaptive locomotion that were derived for a four-legged agent, but with their degree of decentralization varying systematically between the extremes of fully centralized and fully decentralized. Our comparison shows that learning speed is significantly enhanced in distributed architectures—while still reaching the same high performance level of centralized architectures—due to smaller search spaces and local costs providing more focused information for learning. Second, we find an increased robustness of the learning process in the decentralized cases—it is less demanding to hyperparameter selection and less prone to becoming trapped in poor local minima. Finally, when examining generalization to uneven terrains—not used during training—we find best performance for an intermediate architecture that is decentralized, but integrates only local information from both neighboring legs. Together, these findings demonstrate beneficial effects of distributing control into decentralized units and relying on local information. This appears as a promising approach towards more robust DRL and better generalization towards adaptive behavior.

This work discusses a learning approach to mask rewarding objects in images using sparse reward signals from an imitation learning dataset. For that we train an Hourglass network using only feedback from a critic model. The Hourglass network learns to produce a mask to decrease the critic’s score of a high score image and increase the critic’s score of a low score image by swapping the masked areas between these two images. We trained the model on an imitation learning dataset from the NeurIPS 2020 MineRL Competition Track, where our model learned to mask rewarding objects in a complex interactive 3D environment with a sparse reward signal. This approach was part of the 1st place winning solution in this competition. Video demonstration and code: https://rebrand.ly/critic-guided-segmentation.

Variational Auto Encoder (VAE) provide an efficient latent space representation of complex data distributions which is learned in an unsupervised fashion. Using such a representation as input to Reinforcement Learning (RL) approaches may reduce learning time, enable domain transfer or improve interpretability of the model. However, current state-of-the-art approaches that combine VAE with RL fail at learning good performing policies on certain RL domains. Typically, the VAE is pre-trained in isolation and may omit the embedding of task-relevant features due to insufficiencies of its loss. As a result, the RL approach can not successfully maximize the reward on these domains. Therefore, this paper investigates the issues of joint training approaches and explores incorporation of policy gradients from RL into the VAE's latent space to find a task-specific latent space representation. We show that using pre-trained representations can lead to policies being unable to learn any rewarding behaviour in these environments. Subsequently, we introduce two types of models which overcome this deficiency by using policy gradients to learn the representation. Thereby the models are able to embed features into its representation that are crucial for performance on the RL task but would not have been learned with previous methods.

In the experiments of comparative visual search reported here, each half of a display contains simple geometrical objects of three different colors and forms. The two hemifields are identical except for one mismatch either in color or form. The subject’s task is to find this difference. Eye-movement recording yields insight into the interaction of mental processes involved in the completion of this demanding task. We present a hierarchical model of comparative visual search and its implementation as a computer simulation. The evaluation of simulation data shows that this Three-Level Model is able to explain about 98% of the empirical data collected in six different experiments.

We present our second generation tactile sensor for the Shadow Dexterous Hand's palm. We were able to significantly improve the tactile sensor characteristics by utilizing our latest barometer-based tactile sensing technology with linear (R² ≥ 0.9996) sensor output and no noticeable hysteresis. The sensitivity threshold of the tactile cells and the spatial density were both dramatically increased. We demonstrate the benefits of the new sensor by re-running an experiment to estimate the stiffness of different objects that we originally used to test our first generation palm sensor. The results underline a considerable performance boost in estimation accuracy, just due to the improved tactile skin. We also propose a revised neural network architecture that even further improves the average classification accuracy to 96% in a 5-fold cross-validation.

Because of the complex anatomy of the human hand, in the absence of external constraints, a large number of postures and force combinations can be used to attain a stable grasp. Motor synergies provide a viable strategy to solve this problem of motor redundancy. In this study, we exploited the technical advantages of an innovative sensorized object to study unconstrained hand grasping within the theoretical framework of motor synergies. Participants were required to grasp, lift, and hold the sensorized object. During the holding phase, we repetitively applied external disturbance forces and torques and recorded the spatiotemporal distribution of grip forces produced by each digit. We found that the time to reach the maximum grip force during each perturbation was roughly equal across fingers, consistent with a synchronous, synergistic stiffening across digits. We further evaluated this hypothesis by comparing the force distribution of human grasping vs. robotic grasping, where the control strategy was set by the experimenter. We controlled the global hand stiffness of the robotic hand and found that this control algorithm produced a force pattern qualitatively similar to human grasping performance. Our results suggest that the nervous system uses a default whole hand synergistic control to maintain a stable grasp regardless of the number of digits involved in the task, their position on the objects, and the type and frequency of external perturbations. NEW & NOTEWORTHY We studied hand grasping using a sensorized object allowing unconstrained finger placement. During object perturbation, the time to reach the peak force was roughly equal across fingers, consistently with a synergistic stiffening across fingers. Force distribution of a robotic grasping hand, where the control algorithm is based on global hand stiffness, was qualitatively similar to human grasping. This suggests that the central nervous system uses a default whole hand synergistic control to maintain a stable grasp.

Haptic exploration is a key skill for both robots and humans to discriminate and handle unknown objects or to recognize familiar objects. Its active nature is evident in humans who from early on reliably acquire sophisticated sensory-motor capabilities for active exploratory touch and directed manual exploration that associates surfaces and object properties with their spatial locations. This is in stark contrast to robotics. In this field, the relative lack of good real-world interaction models-along with very restricted sensors and a scarcity of suitable training data to leverage machine learning methods-has so far rendered haptic exploration a largely underdeveloped skill. In robot vision however, deep learning approaches and an abundance of available training data have triggered huge advances. In the present work, we connect recent advances in recurrent models of visual attention with previous insights about the organisation of human haptic search behavior, exploratory procedures and haptic glances for a novel architecture that learns a generative model of haptic exploration in a simulated three-dimensional environment. This environment contains a set of rigid static objects representing a selection of one-dimensional local shape features embedded in a 3D space: an edge, a flat and a convex surface. The proposed algorithm simultaneously optimizes main perception-action loop components: feature extraction, integration of features over time, and the control strategy, while continuously acquiring data online. Inspired by the Recurrent Attention Model, we formalize the target task of haptic object identification in a reinforcement learning framework and reward the learner in the case of success only. We perform a multi-module neural network training, including a feature extractor and a recurrent neural network module aiding pose control for storing and combining sequential sensory data. The resulting haptic meta-controller for the rigid 16 × 16 tactile sensor array moving in a physics-driven simulation environment, called the Haptic Attention Model, performs a sequence of haptic glances, and outputs corresponding force measurements. The resulting method has been successfully tested with four different objects. It achieved results close to 100% while performing object contour exploration that has been optimized for its own sensor morphology.

We present a distributed supervised learning architecture, which can generate trajectory data conditioned by control commands and learned from demonstrations. The architecture consists of an ensemble of neural networks (NNs) which learns the dynamic model and a separate addressing NN that decides from which NN to draw a prediction. We introduce an error-based method for automatic assignment of data subsets to the ensemble NNs for training using the loss profile of the ensemble. Our code is publicly available (Code: https://github.com/NicoBach/distributed-dynamics-model).

This work presents a novel object-level control framework for the dexterous in-hand manipulation of objects with torque-controlled robotic hands. The proposed impedance-based controller realizes the compliant 6-DOF positioning of a grasped object. Enabled by the in-hand localization, the slippage of contacts is avoided by actively maintaining the desired grasp configuration. The internal forces on the object are determined by solving a quadratic optimization problem, which explicitly considers friction constraints on the contacts and allows to gradually shift the load between fingers. The proposed framework is capable of dealing with dynamic changes in the grasp configuration, which makes it applicable for the control of the object during grasp acquisition or the reconfiguration of fingers (i.e. finger gaiting). A nullspace controller avoids joint limits and singularities. Experiments with the DLR robot David demonstrate the efficiency and performance of the controller in various grasping scenarios.

Deep Reinforcement Learning has recently seen progress for continuous control tasks, driven by yearly challenges such as the NeurIPS Competition Track. This work combines complementary characteristics of two current state of the art methods, Twin-Delayed Deep Deterministic Policy Gradient and Distributed Distributional Deep Deterministic Policy Gradient, and applied this in the state-of-the-art Learn to move—Walk Around locomotion control challenge which was part of the NeurIPS 2019 Competition Track. The combined approach showed improved results and achieved the 4th place in this competition. The article presents this combination and evaluates the performance.

Recent developments in machine-learning algorithms have led to impressive performance increases in many traditional application scenarios of artificial intelligence research. In the area of deep reinforcement learning, deep learning functional architectures are combined with incremental learning schemes for sequential tasks that include interaction-based, but often delayed feedback. Despite their impressive successes, modern machine-learning approaches, including deep reinforcement learning, still perform weakly when compared to flexibly adaptive biological systems in certain naturally occurring scenarios. Such scenarios include transfers to environments different than the ones in which the training took place or environments that dynamically change, both of which are often mastered by biological systems through a capability that we here term "fluid adaptivity" to contrast it from the much slower adaptivity ("crystallized adaptivity") of the prior learning from which the behavior emerged. In this article, we derive and discuss research strategies, based on analyzes of fluid adaptivity in biological systems and its neuronal modeling, that might aid in equipping future artificially intelligent systems with capabilities of fluid adaptivity more similar to those seen in some biologically intelligent systems. A key component of this research strategy is the dynamization of the problem space itself and the implementation of this dynamization by suitably designed flexibly interacting modules.

How much information with regard to identity and further individual participantcharacteristics are revealed by relatively short spatio-temporal motion trajectories of a person?We study this question by selecting a set of individual participant characteristics and analysingmotion captured trajectories of an exemplary class of familiar movements, namely handover of anobject to another person. The experiment is performed with different participants under different,predefined conditions. A selection of participant characteristics, such as the Big Five personalitytraits, gender, weight, or sportiness, are assessed and we analyse the impact of the three factor groups&ldquo, participant identity&rdquo, &ldquo, participant characteristics&rdquo, and &ldquo, experimental conditions&rdquo, on the observedhand trajectories. The participants&rsquo, movements are recorded via optical marker-based hand motioncapture. One participant, the giver, hands over an object to the receiver. The resulting time courses ofthree-dimensional positions of markers are analysed. Multidimensional scaling is used to projecttrajectories to points in a dimension-reduced feature space. Supervised learning is also applied.We find that &ldquo, seems to have the highest correlation with the trajectories, withfactor group &ldquo, ranking second. On the other hand, it is not possible to find acorrelation between the &ldquo, and the hand trajectory features.

A lot of motion planning research in robotics focuses on efficient means to find trajectories between individual start and goal regions, but it remains challenging to specify and plan robotic manipulation actions which consist of multiple interdependent subtasks. The Task Constructor framework we present in this work provides a flexible and transparent way to define and plan such actions, enhancing the capabilities of the popular robotic manipulation framework MoveIt!.11The Task Constructor framework is publicly available at https://github.com/ros-planning/moveit_task_constructor Subproblems are solved in isolation in black-box planning stages and a common interface is used to pass solution hypotheses between stages. The framework enables the hierarchical organization of basic stages using containers, allowing for sequential as well as parallel compositions. The flexibility of the framework is illustrated in multiple scenarios performed on various robot platforms, including bimanual ones.

Brain-computer interfaces provide a means for controlling a device by brain activity alone. One major drawback of noninvasive BCIs is their low information transfer rate, obstructing a wider deployment outside the lab. BCIs based on codebook visually evoked potentials (cVEP) outperform all other state-of-the-art systems in that regard. Previous work investigated cVEPs for spelling applications. We present the first cVEP-based BCI for use in real-world settings to accomplish everyday tasks such as navigation or action selection. To this end, we developed and evaluated a cVEP-based on-line BCI that controls a virtual agent in a simulated, but realistic, 3-D kitchen scenario. We show that cVEPs can be reliably triggered with stimuli in less restricted presentation schemes, such as on dynamic, changing backgrounds. We introduce a novel, dynamic repetition algorithm that allows for optimizing the balance between accuracy and speed individually for each user. Using these novel mechanisms in a 12-command cVEP-BCI in the 3-D simulation results in ITRs of 50 bits/min on average and 68 bits/min maximum. Thus, this work supports the notion of cVEP-BCIs as a particular fast and robust approach suitable for real-world use.

This work proposes a new approach to robotic grasping exploiting conditional Wasserstein generative adversarial networks (WGANs), which output promising grasp candidates from depth image inputs. In contrast to discriminative models, the WGAN approach enables deliberative navigation in the set of feasible grasps and thus allows a smooth integration with other motion planning tools. We find that the training autonomously partitioned the space of feasible grasps into several regions corresponding to different grasp types. Each region forms a smooth grasp manifold with latent parameters corresponding to important grasp parameters like approach direction. We evaluate the model in simulation on the multi-fingered Shadow Robot hand, comparing it a) to a classical grasp planner for primitive geometric object shapes and b) to a state-of-the-art discriminative network model. The proposed generative model matches the grasp success rate of its trainer models and exhibits better generalization.

We propose a reinforcement learning approach that combines an asynchronous actor-critic model with a recurrent model of visual attention. Instead of using the full visual information of the scene, the resulting model accumulates the foveal information of controlled glimpses and is thus able to reduce the complexity of the network. Using the designed model, an artificial agent is able to solve a challenging “mediated interaction” task. In these tasks, the desired effects cannot be created through direct interaction, but instead require the learner to discover how to exert suitable effects on the target object through involving a “tool”. To learn the given mediated interaction task, the agent is “actively” searching for salient points within the environment by taking a limited number of fovea-like glimpses. It then uses the accumulated information to decide which action to take next.

This research examines two factors whether they have an influence on human performance during haptic rotation. The first one is the number of fingers used to perform the haptic rotation. The second one is the grasping orientation of the fingers on the rotary knob. Twenty blindfolded participants had to rotate a rotary knob 90 degrees counterclockwise around its own axis. In the first part of the experiment we varied the number of fingers conducting the rotation from two to five. In the second part four different grasping orientations were evaluated: vertical, horizontal, oblique and unfamiliar. To determine the performance, we calculated two measures, the accuracy and the precision based on the rotation angles. The first part of the evaluation showed, that the number of fingers involved in the rotation had a significant influence on the accuracy, while more fingers lead to a better accuracy. Precision was not dependent on the number of fingers. The second evaluation showed that the grasping orientation had no effect on the accuracy. However, there was a significant impact of the grasping position on the precision. The vertical grasping orientation led to a better precision than the horizontal or unfamiliar grasping position.

Single-trial classification of EEG data from Disorder of Consciousness patients (DoC) has proved particularly challenging. We present an approach that establishes a measure to relate the performance of single-trial classification of DoC patient EEG data with relational frequency bands and thus with their mental state. We evaluate our approach on 31 patient data sets from two studies, showing that our measure indicates for different data sets a particular likelihood for misclassifying either target or non-target class samples.

We investigate the effectiveness of transfer learn- ing for accelerating shallow and deep reinforcement learning of “mediated interaction tasks”. In these tasks, the desired effects cannot be created through direct interaction, but instead require the learner to discover how to exert suitable effects on the target object through involving a suitable “mediator object”. We focus on the case where transfer learning is applied to generalize experiences from source tasks that are solvable through direct, unmediated interaction, to target tasks that require mediated interaction for their solution. We find that transfer learning employed in this context leads to a significant acceleration of learning. A refinement of the basic transfer learning strategy that is motivated from the principle of scaffolding in psychology leads to further improvements, totalling to an overall speed-up factor of almost one order of magnitude for a reinforcement learner solving an “extension- of-reach” task in a 2D world with simulated physics.

Our fingernails help us to accomplish a variety of manual tasks, but surprisingly only a few robotic hands are equipped with nails. In this paper, we present a sensorized fingernail for mechatronic hands that can capture static and dynamic interaction forces with the nail. Over the course of several iterations, we have developed a very compact working prototype that fits together with our previously developed multi-cell tactile fingertip sensor into the cavity of the distal phalange of a human-sized robotic hand. We present the construction details, list the key performance characteristics and demonstrate an example application of finding the end of an adhesive tape roll using the signals captured by the sensors integrated in the nail. We conclude with a discussion about improvement ideas for future versions.

The ability to discriminate between target and distractors, using the information perceived by the hand over time, is essential to perform haptic search successfully, be it for a human hand or a suitably sensorized anthropomorphic robot hand. To address the latter, we train a binary classifier to perform this discrimination during unconstrained haptic search performed by sighted study participants who were blindfolded. In this work, we test different representational concepts and compare the results with the human classification performance. This approach both guides our understanding of human haptic interaction with the 3D environment and aids future modeling of artificial touch for anthropomorphic robot hands. Our contribution is three-fold. Firstly, we are able to acquire a synchronized multimodal time series of exceptionally high spatio-temporal resolution of both the 3D environment and the hand with our novel experimental setup. It includes our Modular Haptic Stimulus Board to represent a 3D environment and a novel tactile glove equipped with position tracking markers and joint angle sensors. Secondly, we introduce a machine learning approach inspired by a novel application of the feature guidance concept for vision (Wolfe et al., 2007 [1]) to modeling of haptic search in a 3D environment, focusing on the target-distractor discrimination. Finally, we compare results for two different types of artificial neural networks, a feedforward and a recurrent network. We show that using recurrent networks, and therefore integrating information over time, improves the classification results. The evaluation also shows that classification accuracy is the highest for the combination of both the tactile and the joint angle modalities.

Formulating problems rigorously in terms of optimization principles has become a dominating approach in the fields of machine learning and computer vision. However, the systems described in these fields are in some respects different to integrated, modular, and embodied systems, such as the ones we aim to build in robotics. While representing systems via optimality principles is a powerful approach, relying on it as the sole approach to robotics raises substantial challenges. In this article, we take this as a starting point to discuss which ways of representing problems should be best-suited for robotics. We argue that an adequate choice of system representation—e.g. via optimization principles—must allow us to reflect the structure of the problem domain. We discuss system design principles, such as modularity, redundancy, stability, and dynamic processes, and the degree to which they are compatible with the optimization stance or instead point to alternative paradigms in robotics research. This discussion, we hope, will bring attention to this important and often ignored system-level issue in the context of robotics research.

In the NIPS 2017 Learning to Run challenge, participants were tasked with building a controller for a musculoskeletal model to make it run as fast as possible through an obstacle course. Top participants were invited to describe their algorithms. In this work, we present eight solutions that used deep reinforcement learning approaches, based on algorithms such as Deep Deterministic Policy Gradient, Proximal Policy Optimization, and Trust Region Policy Optimization. Many solutions use similar relaxations and heuristics, such as reward shaping, frame skipping, discretization of the action space, symmetry, and policy blending. However, each of the eight teams implemented different modifications of the known algorithms., Comment: 27 pages, 17 figures

International audience; Tactile sensing technology has achieved much progress in last decades. Recently developed tactile sensing systems have been utilized for implementation of bodily-aware robots, enhancement of robotic hand's dexterous manipulation, development of wearable devices for facilitating the human robot interaction, and so on. While tactile sensing has resulted in many possibilities in robotic research, robot's manipulation is still far from that of its counterpart-human-in term of dexterity. To overcome this issue, we are considering the following three aspects that need to be tackled: - Available tactile sensing technology is struggling to meet the capabilities of humans’ sense of touch; - Tactile data processing algorithms and data-driven learning technology are still immature; - General and robust robot manipulation control frameworks are missing