Search

This paper provides a structured and practical roadmap for practitioners to integrate learning from demonstration (LfD) into manufacturing tasks, with a specific focus on industrial manipulators. Motivated by the paradigm shift from mass production to mass customization, it is crucial to have an easy-to-follow roadmap for practitioners with moderate expertise, to transform existing robotic processes to customizable LfD-based solutions. To realize this transformation, we devise the key questions of "What to Demonstrate", "How to Demonstrate", "How to Learn", and "How to Refine". To follow through these questions, our comprehensive guide offers a questionnaire-style approach, highlighting key steps from problem definition to solution refinement. This paper equips both researchers and industry professionals with actionable insights to deploy LfD-based solutions effectively. By tailoring the refinement criteria to manufacturing settings, this paper addresses related challenges and strategies for enhancing LfD performance in manufacturing contexts. [ABSTRACT FROM AUTHOR]

The requirement for a high number of training episodes has been a major limiting factor for the application of Reinforcement Learning (RL) in robotics. Learning skills directly on real robots requires time, causes wear and tear and can lead to damage to the robot and environment due to unsafe exploratory actions. The success of learning skills in simulation and transferring them to real robots has also been limited by the gap between reality and simulation. This is particularly problematic for tasks involving contact with the environment as contact dynamics are hard to model and simulate. In this paper we propose a framework which leverages a shared control framework for modeling known constraints defined by object interactions and task geometry to reduce the state and action spaces and hence the overall dimensionality of the reinforcement learning problem. The unknown task knowledge and actions are learned by a reinforcement learning agent by conducting exploration in the constrained environment. Using a pouring task and grid-clamp placement task (similar to peg-in-hole) as use cases and a 7-DoF arm, we show that our approach can be used to learn directly on the real robot. The pouring task is learned in only 65 episodes (16 min) and the grid-clamp placement task is learned in 75 episodes (17 min) with strong safety guarantees and simple reward functions, greatly alleviating the need for simulation. [ABSTRACT FROM AUTHOR]

Currently, the standard method of programming industrial robots is to perform it manually, which is cumbersome and time-consuming. Thus, it can be a burden for the flexibility of inspection systems when a new component with a different design needs to be inspected. Therefore, developing a way to automate the task of generating a robotic trajectory offers a substantial improvement in the field of automated manufacturing and quality inspection. This paper proposes and evaluates a methodology for automatizing the process of scanning a 3D surface for the purpose of quality inspection using only visual feedback. The paper is divided into three sub-tasks in the same general setting: (1) autonomously finding the optimal distance of the camera on the robot's end-effector from the surface, (2) autonomously generating a trajectory to scan an unknown surface, and (3) autonomous localization and scan of a surface with a known shape, but with an unknown position. The novelty of this work lies in the application that only uses visual feedback, through the image focus measure, for determination and optimization of the motion. This reduces the complexity and the cost of such a setup. The methods developed have been tested in simulation and in real-world experiments and it was possible to obtain a precision in the optimal pose of the robot under 1 mm in translational, and 0.1 ∘ in angular directions. It took less than 50 iterations to generate a trajectory for scanning an unknown free-form surface. Finally, with less than 30 iterations during the experiments it was possible to localize the position of the surface. Overall, the results of the proposed methodologies show that they can bring substantial improvement to the task of automatic motion generation for visual quality inspection. [ABSTRACT FROM AUTHOR]

Ground robots require the crucial capability of traversing unstructured and unprepared terrains and avoiding obstacles to complete tasks in real-world robotics applications such as disaster response. When a robot operates in off-road field environments such as forests, the robot's actual behaviors often do not match its expected or planned behaviors, due to changes in the characteristics of terrains and the robot itself. Therefore, the capability of robot adaptation for consistent behavior generation is essential for maneuverability on unstructured off-road terrains. In order to address the challenge, we propose a novel method of self-reflective terrain-aware adaptation for ground robots to generate consistent controls to navigate over unstructured off-road terrains, which enables robots to more accurately execute the expected behaviors through robot self-reflection while adapting to varying unstructured terrains. To evaluate our method's performance, we conduct extensive experiments using real ground robots with various functionality changes over diverse unstructured off-road terrains. The comprehensive experimental results have shown that our self-reflective terrain-aware adaptation method enables ground robots to generate consistent navigational behaviors and outperforms the compared previous and baseline techniques. [ABSTRACT FROM AUTHOR]

Data generation and labeling are often expensive in robot learning. Preference-based learning is a concept that enables reliable labeling by querying users with preference questions. Active querying methods are commonly employed in preference-based learning to generate more informative data at the expense of parallelization and computation time. In this article, we develop a set of novel algorithms, batch active preference-based learning methods, that enable efficient learning of reward functions using as few data samples as possible while still having short query generation times and also retaining parallelizability. We introduce a method based on determinantal point processes for active batch generation and several heuristic-based alternatives. Finally, we present our experimental results for a variety of robotics tasks in simulation. Our results suggest that our batch active learning algorithm requires only a few queries that are computed in a short amount of time. We showcase one of our algorithms in a study to learn human users' preferences. [ABSTRACT FROM AUTHOR]

Recent advancements in quadrupedal robotics have explored the motor potential of these machines beyond simple walking, enabling highly dynamic skills such as jumping, backflips, and even bipedal locomotion. While reinforcement learning has demonstrated excellent performance in this domain, it often relies on complex reward function tuning and prolonged training times, and the interpretability is not satisfactory. Riemannian motion policies, a reactive control method, excel in handling highly dynamic systems but are generally limited to fully actuated systems, making their application to underactuated quadrupedal robots challenging. To address these limitations, we propose a novel framework that treats each leg of a quadrupedal robot as an intelligent agent and employs multi-agent reinforcement learning to coordinate the motion of all four legs. This decomposition satisfies the conditions for utilizing Riemannian motion policies and eliminates the need for complex reward functions, simplifying the learning process for high-level motion modalities. Our simulation experiments demonstrate that the proposed method enables quadrupedal robots to learn stable locomotion using three, two, or even a single leg, offering advantages in training speed, success rate, and stability compared to traditional approaches, and better interpretability. This research explores the possibility of developing more efficient and adaptable control policies for quadrupedal robots. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Beijing University of Technology is the property of Journal of Beijing University of Technology, Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Agile maneuvers such as sprinting and high-speed turning in the wild are challenging for legged robots. We present an end-to-end learned controller that achieves record agility for the MIT Mini Cheetah, sustaining speeds up to 3.9 m/s. This system runs and turns fast on natural terrains like grass, ice, and gravel and responds robustly to disturbances. Our controller is a neural network trained in simulation via reinforcement learning and transferred to the real world. The two key components are (i) an adaptive curriculum on velocity commands and (ii) an online system identification strategy for sim-to-real transfer. Videos of the robot's behaviors are available at https://agility.csail.mit.edu/. [ABSTRACT FROM AUTHOR]

Modeling and manipulating elasto-plastic objects are essential capabilities for robots to perform complex industrial and household interaction tasks (e.g., stuffing dumplings, rolling sushi, and making pottery). However, due to the high degrees of freedom of elasto-plastic objects, significant challenges exist in virtually every aspect of the robotic manipulation pipeline, for example, representing the states, modeling the dynamics, and synthesizing the control signals. We propose to tackle these challenges by employing a particle-based representation for elasto-plastic objects in a model-based planning framework. Our system, RoboCraft, only assumes access to raw RGBD visual observations. It transforms the sensory data into particles and learns a particle-based dynamics model using graph neural networks (GNNs) to capture the structure of the underlying system. The learned model can then be coupled with model predictive control (MPC) algorithms to plan the robot's behavior. We show through experiments that with just 10 min of real-world robot interaction data, our robot can learn a dynamics model that can be used to synthesize control signals to deform elasto-plastic objects into various complex target shapes, including shapes that the robot has never encountered before. We perform systematic evaluations in both simulation and the real world to demonstrate the robot's manipulation capabilities. [ABSTRACT FROM AUTHOR]

Human brain and behavior provide a rich venue that can inspire novel control and learning methods for robotics. In an attempt to exemplify such a development by inspiring how humans acquire knowledge and transfer skills among tasks, we introduce a novel multi-task reinforcement learning framework named Episodic Return Progress with Bidirectional Progressive Neural Networks (ERP-BPNN). The proposed ERP-BPNN model 1) learns in a human-like interleaved manner by 2) autonomous task switching based on a novel intrinsic motivation signal and, in contrast to existing methods, 3) allows bidirectional skill transfer among tasks. ERP-BPNN is a general architecture applicable to several multi-task learning settings; in this paper, we present the details of its neural architecture and show its ability to enable effective learning and skill transfer among morphologically different robots in a reaching task. The developed Bidirectional Progressive Neural Network (BPNN) architecture enables bidirectional skill transfer without requiring incremental training and seamlessly integrates with online task arbitration. The task arbitration mechanism developed is based on soft Episodic Return progress (ERP), a novel intrinsic motivation (IM) signal. To evaluate our method, we use quantifiable robotics metrics such as ‘expected distance to goal’ and ‘path straightness’ in addition to the usual reward-based measure of episodic return common in reinforcement learning. With simulation experiments, we show that ERP-BPNN achieves faster cumulative convergence and improves performance in all metrics considered among morphologically different robots compared to the baselines. Overall, our method provides a human-inspired and efficient multi-task reinforcement learning approach with interleaved learning, making it highly suitable for lifelong learning applications.

Autonomous navigation in unstructured vegetated environments remains an open challenge. To successfully operate in these settings, autonomous ground vehicles (AGVs) must assess the environment and determine which vegetation is pliable enough to safely traverse. In this paper, we propose ForestTrav (Forest Traversability): a novel lidar-only (geometric), online traversability estimation (TE) method that can accurately generate a per-voxel traversability estimate for densely vegetated environments, demonstrated in dense subtropical forests. The method leverages a salient, probabilistic 3D voxel representation, continuously fusing incoming lidar measurements to maintain multiple, per-voxel ray statistics, in combination with the structural context and compactness of sparse convolutional neural networks (SCNNs) to perform accurate TE in densely vegetated environments. The proposed method is real-time capable and is shown to outperform state-of-the-art volumetric and 2.5D TE methods by a significant margin (0.62 vs. 0.41 Matthews correlation coefficient (MCC) score at qty 0.1 m voxel resolution) in challenging scenes and to generalize to unseen environments. ForestTrav demonstrates that lidar-only (geometric) methods can provide accurate, online TE in complex, densely-vegetated environments. This capability has not been previously demonstrated in the literature in such complex environments. Further, we analyze the response of the TE methods to the temporal and spatial evolution of the probabilistic map as a function of information accumulated over time during scene exploration. It shows that our method performs well even with limited information in the early stages of exploration, and this provides an additional tool to assess the expected performance during deployment. Finally, to train and assess TE methods in highly-vegetated environments, we collected and labeled a novel, real-world data set and provide it to the community as an open-source resource.

Dynamic Motion Primitives (DMPs) only address the generalization problem for target positions that are close to the demonstration position and in order to enhance the generalization capability, by making the learned movements executable in the entire reachable robot workspace, multiple demonstrations are needed, resulting in an increase in the time required to teach new tasks to the robot. This work aims to propose a novel approach to scale the DMP parameters through two demonstrations in order to enhance the DMP’s generalization capability in the robot reachable workspace while guaranteeing a fast and easy learning phase. The proposed method to scale the DMP parameters relies on a linear interpolation performed on the DMP parameters extracted by two demonstrations and is applied to the agricultural field, where the adoption of DMPs can provide a promising solution to meet the demand for a wide range of tasks in an ever-changing and mutable agricultural environment. Four agricultural activities, namely digging, seeding, irrigation, and harvesting, have been learned by the robot using DMPs. The experimental validation was carried out on the Tiago robot and the proposed approach, based on two demonstrations, was compared to two literature methods based on a single demonstration and multiple demonstrations in terms of accuracy of the motion reconstruction, success rate of task execution and speed of the learning process. The obtained results demonstrated that the proposed approach, based on two demonstrations, guarantees a successful execution of the four tasks without exceeding the robot reachable workspace and with acceptable accuracy (mean success rate of task execution is about 95.6%) and a fast training phase (about 70 minutes less than the database approach that is built on multiple demonstrations).

Task-oriented robot learning has shown significant potential with the development of Reinforcement Learning (RL) algorithms. However, the learning of long-horizon tasks for robots remains a formidable challenge due to the inherent complexity of tasks, typically comprising multiple diverse stages. Universal RL algorithms commonly encounter issues such as slow convergence or even failure to converge altogether when applied to such tasks. The reasons behind these challenges lie in the local optima trap or redundant exploration during the new stages or the junction of two continuous stages. To address these challenges, we propose a novel state-dependent maximum entropy (SDME) reinforcement learning algorithm. This algorithm effectively balances the trade-off between exploration and exploitation around three kinds of critical states arising from the unique nature of long-horizon tasks. We conducted experiments within an open-source simulation environment, focusing on two representative long-horizon tasks. The proposed SDME algorithm exhibits faster and more stable learning capabilities, requiring merely one-third of the number of learning samples necessary for baseline approaches. Furthermore, we assess the generalization ability of our method under randomly initialized conditions, and the results show that the success rate of the SDME algorithm is nearly twice that of the baselines. Our code will be available at . [ABSTRACT FROM AUTHOR]

Robot grasping constitutes an essential capability in fulfilling the complexities of advanced industrial operations. This field has been extensively investigated to address a range of practical applications. However, the generation of a stable grasp remains challenging, principally due to the constraints imposed by object geometries and the diverse objectives of the tasks. In this work, we propose a novel learning from demonstration-based grasp-planning framework. This framework is designed to extract crucial human grasp skills, namely the contact region and approach direction, from a single demonstration. Then, it formulates an optimization problem that integrates the extracted skills to generate a stable grasp. Distinct from conventional methods that rely on learning implicit synergies through human demonstration or on mapping the dissimilar kinematics between human hands and robot grippers, our approach focuses on learning the intuitive human intent that involves the potential contact regions and the grasping approach direction. Furthermore, our optimization formulation is capable of identifying the optimal grasp by minimizing the surface fitting error between the demonstrated contact regions on the object and the gripper finger surface and imposing a penalty for any misalignment between the demonstrated and the gripper's approach directions. A series of experiments is conducted to verify the effectiveness of the proposed algorithm through both simulations and real-world scenarios. [ABSTRACT FROM AUTHOR]

The convergence of cloud and edge computing, along with distributed AI and the latest 5G/6G communications, is revolutionizing collaboration, connectivity, and interaction. These digital advancements pave the way for a new era of AI-powered robots, enabling them to navigate unfamiliar scenarios and adapt in the long term by seamlessly engaging with the digital realm. Consequently, these innovative applications generate diverse, continuous, and rapidly evolving data transmission requirements that traditional network resource management struggles to satisfy in terms of Quality of Service (QoS). In this paper, we take a step beyond focusing solely on network-side traffic engineering efforts. Instead, we explore the potential of application-side traffic shaping within non-public networks to address these demanding transmission needs. Within the framework of optimizing Quality of Experience, we discuss on how the 5G-ERA project focuses on a multi-domain learning process for autonomous robotics using an intent-based networking approach for optimized resource management within the network validated by the use cases within the project enabling the transition from Industry 4.0 to Industry 5.0. Our central aim is to efficiently control and direct data traffic within the confines of the private network, ensuring it aligns with predefined objectives. While this methodology can be applied in various contexts, it is essential to establish precise intentions rooted in domain expertise. This innovative approach serves as a valuable complement to conventional network resource management methods typically employed at the network infrastructure level. [ABSTRACT FROM AUTHOR]

Abstract Bagging is an essential skill that humans perform in their daily activities. However, deformable objects, such as bags, are complex for robots to manipulate. A learning‐based framework that enables robots to learn bagging is presented. The novelty of this framework is its ability to learn and perform bagging without relying on simulations. The learning process is accomplished through a reinforcement learning (RL) algorithm introduced and designed to find the best grasping points of the bag based on a set of compact state representations. The framework utilises a set of primitive actions and represents the task in five states. In our experiments, the framework reached 60% and 80% success rates after around 3 h of training in the real world when starting the bagging task from folded and unfolded states, respectively. Finally, the authors test the trained RL model with eight more bags of different sizes to evaluate its generalisability.

This paper provides a structured and practical roadmap for practitioners to integrate learning from demonstration (LfD) into manufacturing tasks, with a specific focus on industrial manipulators. Motivated by the paradigm shift from mass production to mass customization, it is crucial to have an easy-to-follow roadmap for practitioners with moderate expertise, to transform existing robotic processes to customizable LfD-based solutions. To realize this transformation, we devise the key questions of “What to Demonstrate”, “How to Demonstrate”, “How to Learn”, and “How to Refine”. To follow through these questions, our comprehensive guide offers a questionnaire-style approach, highlighting key steps from problem definition to solution refinement. This paper equips both researchers and industry professionals with actionable insights to deploy LfD-based solutions effectively. By tailoring the refinement criteria to manufacturing settings, this paper addresses related challenges and strategies for enhancing LfD performance in manufacturing contexts.

This work proposes a Learning by Demonstration framework based on Dynamic Movement Primitives (DMPs) that could be effectively adopted to plan complex activities in robotics such as the ones to be performed in agricultural domains and avoid orientation discontinuity during motion learning. The approach resorts to Lie theory and integrates into the DMP equations the exponential and logarithmic map, which converts any element of the Lie group S O (3) into an element of the tangent space s o (3) and vice versa. Moreover, it includes a dynamic parameterization for the tangent space elements to manage the discontinuity of the logarithmic map. The proposed approach was tested on the Tiago robot during the fulfillment of four agricultural activities, such as digging, seeding, irrigation and harvesting. The obtained results were compared to the one achieved by using the original formulation of the DMPs and demonstrated the high capability of the proposed method to manage orientation discontinuity (the success rate was 100 % for all the tested poses). [ABSTRACT FROM AUTHOR]

Learning from visual observations is a significant yet challenging problem in Reinforcement Learning (RL). Two respective problems, representation learning and task learning, need to solve to infer an optimal policy. Some methods have been proposed to utilize data augmentation in reinforcement learning to directly learn from images. Although these methods can improve generation in RL, they are often found to make the task learning unsteady and can even lead to divergence. We investigate the causes of instability and find it is usually rooted in high-variance of Q-functions. In this paper, we propose an easy-to-implement and unified method to solve above-mentioned problems, Data-augmented Reinforcement Learning with Ensemble Exploration and Exploitation (DAR-EEE). Bootstrap ensembles are incorporated into data augmented reinforcement learning and provide uncertainty estimation of both original and augmented states, which can be utilized to stabilize and accelerate the task learning. Specially, a novel strategy called uncertainty-weighted exploitation is designed for rational utilization of transition tuples. Moreover, an efficient exploration method using the highest upper confidence is used to balance exploration and exploitation. Our experimental evaluation demonstrates the improved sample efficiency and final performance of our method on a range of difficult image-based control tasks. Especially, our method has achieved the new state-of-the-art performance on Reacher-easy and Cheetah-run tasks. [ABSTRACT FROM AUTHOR]

Copyright of Acta Scientiarum Naturalium Universitatis Pekinensis is the property of Editorial Office of Acta Scientiarum Naturalium Universitatis Pekinensis and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

For robots in human environments, learning complex and demanding interaction skills from humans and responding quickly to human motions are highly desirable. A common challenge for interaction tasks is that the robot has to satisfy both the task space and the joint space constraints on its motion trajectories in real time. Few studies have addressed the issue of hyperspace constraints in human-robot interaction, whereas researchers have investigated it in robot imitation learning. In this work, we propose a method of dual-space feature fusion to enhance the accuracy of the inferred trajectories in both task space and joint space; then, we introduce a linear mapping operator (LMO) to map the inferred task space trajectory to a joint space trajectory. Finally, we combine the dual-space fusion, LMO, and phase estimation into a unified probabilistic framework. We evaluate our dual-space feature fusion capability and real-time performance in the task of a robot following a human-handheld object and a ball-hitting experiment. Our inference accuracy in both task space and joint space is superior to standard Interaction Primitives (IP) which only use single-space inference (by more than 33%); the inference accuracy of the second order LMO is comparable to the kinematic-based mapping method, and the computation time of our unified inference framework is reduced by 54.87% relative to the comparison method. [ABSTRACT FROM AUTHOR]

Legged robots that can instantly change motor patterns at different walking speeds are useful and can accomplish various tasks efficiently. However, state-of-the-art control methods either are difficult to develop or require long training times. In this study, we present a comprehensible neural control framework to integrate probability-based black-box optimization (PI BB) and supervised learning for robot motor pattern generation at various walking speeds. The control framework structure is based on a combination of a central pattern generator (CPG), a radial basis function (RBF) -based premotor network and a hypernetwork, resulting in a so-called neural CPG-RBF-hyper control network. First, the CPG-driven RBF network, acting as a complex motor pattern generator, was trained to learn policies (multiple motor patterns) for different speeds using PI BB. We also introduce an incremental learning strategy to avoid local optima. Second, the hypernetwork, which acts as a task/behavior to control parameter mapping, was trained using supervised learning. It creates a mapping between the internal CPG frequency (reflecting the walking speed) and motor behavior. This map represents the prior knowledge of the robot, which contains the optimal motor joint patterns at various CPG frequencies. Finally, when a user-defined robot walking frequency or speed is provided, the hypernetwork generates the corresponding policy for the CPG-RBF network. The result is a versatile locomotion controller which enables a quadruped robot to perform stable and robust walking at different speeds without sensory feedback. The policy of the controller was trained in the simulation (less than 1 h) and capable of transferring to a real robot. The generalization ability of the controller was demonstrated by testing the CPG frequencies that were not encountered during training. [ABSTRACT FROM AUTHOR]

Goal-conditioned rearrangement of deformable objects (e.g. straightening a rope and folding a cloth) is one of the most common deformable manipulation tasks, where the robot needs to rearrange a deformable object into a prescribed goal configuration with only visual observations. These tasks are typically confronted with two main challenges: the high dimensionality of deformable configuration space and the underlying complexity, nonlinearity and uncertainty inherent in deformable dynamics. To address these challenges, we propose a novel representation strategy that can efficiently model the deformable object states with a set of keypoints and their interactions. We further propose local-graph neural network (GNN), a light local GNN learning to jointly model the deformable rearrangement dynamics and infer the optimal manipulation actions (e.g. pick and place) by constructing and updating two dynamic graphs. Both simulated and real experiments have been conducted to demonstrate that the proposed dynamic graph representation shows superior expressiveness in modeling deformable rearrangement dynamics. Our method reaches much higher success rates on a variety of deformable rearrangement tasks (96.3% on average) than state-of-the-art method in simulation experiments. Besides, our method is much more lighter and has a 60% shorter inference time than state-of-the-art methods. We also demonstrate that our method performs well in the multi-task learning scenario and can be transferred to real-world applications with an average success rate of 95% by solely fine tuning a keypoint detector. A supplementary video can be found at https://youtu.be/AhwTQo6fCM0. [ABSTRACT FROM AUTHOR]

Recent advancements in quadrupedal robotics have explored the motor potential of these machines beyond simple walking, enabling highly dynamic skills such as jumping, backflips, and even bipedal locomotion. While reinforcement learning has demonstrated excellent performance in this domain, it often relies on complex reward function tuning and prolonged training times, and the interpretability is not satisfactory. Riemannian motion policies, a reactive control method, excel in handling highly dynamic systems but are generally limited to fully actuated systems, making their application to underactuated quadrupedal robots challenging. To address these limitations, we propose a novel framework that treats each leg of a quadrupedal robot as an intelligent agent and employs multi-agent reinforcement learning to coordinate the motion of all four legs. This decomposition satisfies the conditions for utilizing Riemannian motion policies and eliminates the need for complex reward functions, simplifying the learning process for high-level motion modalities. Our simulation experiments demonstrate that the proposed method enables quadrupedal robots to learn stable locomotion using three, two, or even a single leg, offering advantages in training speed, success rate, and stability compared to traditional approaches, and better interpretability. This research explores the possibility of developing more efficient and adaptable control policies for quadrupedal robots.

Abstract Goal-conditioned rearrangement of deformable objects (e.g. straightening a rope and folding a cloth) is one of the most common deformable manipulation tasks, where the robot needs to rearrange a deformable object into a prescribed goal configuration with only visual observations. These tasks are typically confronted with two main challenges: the high dimensionality of deformable configuration space and the underlying complexity, nonlinearity and uncertainty inherent in deformable dynamics. To address these challenges, we propose a novel representation strategy that can efficiently model the deformable object states with a set of keypoints and their interactions. We further propose local-graph neural network (GNN), a light local GNN learning to jointly model the deformable rearrangement dynamics and infer the optimal manipulation actions (e.g. pick and place) by constructing and updating two dynamic graphs. Both simulated and real experiments have been conducted to demonstrate that the proposed dynamic graph representation shows superior expressiveness in modeling deformable rearrangement dynamics. Our method reaches much higher success rates on a variety of deformable rearrangement tasks (96.3% on average) than state-of-the-art method in simulation experiments. Besides, our method is much more lighter and has a 60% shorter inference time than state-of-the-art methods. We also demonstrate that our method performs well in the multi-task learning scenario and can be transferred to real-world applications with an average success rate of 95% by solely fine tuning a keypoint detector. A supplementary video can be found at https://youtu.be/AhwTQo6fCM0 .

In recent years, soft robots gain increasing attention as a result of their compliance when operating in unstructured environments, and their flexibility that ensures safety when interacting with humans. However, challenges lie on the difficulty to develop control algorithms due to various limitations induced by their soft structure. In this paper, we introduce a novel technique that aims to perform motion control of a modular bio-inspired soft-robotic arm, with the main focus lying on facilitating the qualitative reproduction of well-specified periodic trajectories. The introduced method combines the notion behind two previously developed methodologies both based on the Movement Primitive (MP) theory, by exploiting their capabilities while coping with their main drawbacks. Concretely, the requested actuation is initially computed using a Probabilistic MP (ProMP)-based method that considers the trajectory as a combination of simple movements previously learned and stored as a MP library. Subsequently, the key components of the resulting actuation are extracted and filtered in the frequency domain. These are eventually used as input to a Central Pattern Generator (CPG)-based model that takes over the generation of rhythmic patterns at the motor level. The proposed methodology is evaluated on a two-module soft arm. Results show that the first algorithmic component (ProMP) provides an immediate estimation of the requested actuation by avoiding time-consuming training, while the latter (CPG) further simplifies the execution by allowing its control through a low-dimensional parameterization. Altogether, these results open new avenues for the rapid acquisition of periodic movements in soft robots, and their compression into CPG parameters for long-term storage and execution.

In this research, a biped robot is programmed to learn to walk using genetic algorithms. The biped robot has the ability to rise after a fall. The system is made according to human genetics, where a child tries to learn to walk, takes a few steps, falls, and then stands up somehow after a fall. After this process, the system restarts and tries to walk again. The biped robot evolves after each generation, and after each generation it will have a more natural walking posture. The system is unpredictable, because the biped robot sometimes evolves faster and sometimes evolves slower, like in human genetics where some children are faster and other children are slower to walk. The biped robot is considered to have evolved enough after having a walking posture similar to that of a human and does not fall. [ABSTRACT FROM AUTHOR]

A novel robot skill learning method using dynamic movement primitive (DMP) and adaptive control is proposed. The existing DMP method learns actions from a single teaching trajectory, and its Gaussian basis function distribution mode is fixed, which is not suitable for multiple movement trajectories with different characteristics. Therefore, the Gaussian mixture model (GMM) and Gaussian mixture regression are introduced into DMP to enable the robot to learn skills from multi-teaching trajectory. Moreover, radial basis function neural network (RBFNN) is introduced into DMP to establish the RBF-DMP method, which is able to learn the central position and weight of Gaussian basis through gradient descent and improves the accuracy of skill modeling. Furthermore, an adaptive neural network controller is designed to control the learned actions of the manipulator in redemonstration. Finally, experiments on Franka Emika Panda manipulator prove the effectiveness of the proposed method.

Manipulators motion planning faces new challenges as robots are increasingly used in dense, cluttered and dynamic environments. The recently proposed technique called Riemannian motion policies(RMPs) provides an elegant solution with clear mathematical interpretations to such challenging scenarios. It is based on differential geometry policies that generate reactive motions in dynamic environments with real-time performance. However, designing and combining RMPs is still a difficult task involving extensive parameter tuning, and typically seven or more RMPs need to be combined by using RMPflow to realize motions of a robot manipulator with more than 6 degrees-of-freedoms, where the RMPs parameters have to be empirically set each time. In this paper, we take a policy to decompose such complex policies into multiple learning modules based on reinforcement learning. Specifically, we propose a three-layer robot learning framework that consists of the basic-level, middle-level and top-level layers. At the basic layer, only two base RMPs i.e. target and collision avoidance are used to output reactive actions. At the middle-level layer, a hierarchical reinforcement learning approach is used to train an agent that automatically selects those RMPs and their parameters based on environmental changes and will be deployed at each joint. At the top-level layer, a multi-agent reinforcement learning approach trains all the joints with high-level collaborative policies to accomplish actions such as tracking a target and avoiding obstacles. With simulation experiments, we compare the proposed method with the baseline method and find that our method effectively produces superior actions and is better at avoiding obstacles, handling self-collisions, and avoiding singularities in dynamic environments. In addition, the proposed framework possesses higher training efficiency while leveraging the generalization ability of reinforcement learning to dynamic environments and improving safety and interpretability.

This study introduces a novel, hand-drawn language designed to foster human-robot collaboration in wood stereotomy, central to carpentry and joinery professions. Based on skilled carpenters’ line and symbol etchings on timber, this language signifies the location, geometry of woodworking joints, and timber placement within a framework. A proof-of-concept prototype has been developed, integrating object detectors, keypoint regression, and traditional computer vision techniques to interpret this language and enable an extensive repertoire of actions. Empirical data attests to the language’s efficacy, with the successful identification of a specific set of symbols on various wood species’ sawn surfaces, achieving a mean average precision (mAP) exceeding 90%. Concurrently, the system can accurately pinpoint critical positions that facilitate robotic comprehension of carpenter-indicated woodworking joint geometry. The positioning error, approximately 3 pixels, meets industry standards.

Despite the great promise of service robots in everyday tasks, many roboethics issues remain to be addressed before these robots can physically work in human environments. Robot safety is one of the essential concerns for roboethics which is not just a design-time issue. It is also crucial to devise the required onboard monitoring and control strategies to enable robots to be aware of and react to anomalies (i.e., unexpected deviations from intended outcomes) that arise during their operations in the real world. The detection and identification of these anomalies is an essential first step toward fulfilling these requirements. Although several architectures have been proposed for anomaly detection; identification has not yet been thoroughly investigated. This task is challenging since indicators may appear long before anomalies are detected. In this paper, we propose a ConvoLUtional threE-stream Anomaly Identification (CLUE-AI) framework to address this problem. The framework fuses visual, auditory and proprioceptive data streams to identify everyday object manipulation anomalies. A stream of 2D images gathered through an RGB-D camera placed on the head of the robot is processed within a self-attention-enabled visual stage to capture visual anomaly indicators. The auditory modality provided by the microphone placed on the robot’s lower torso is processed within a designed convolutional neural network (CNN) in the auditory stage. Last, the force applied by the gripper and the gripper state is processed within a CNN to obtain proprioceptive features. These outputs are then combined with a late fusion scheme. Our novel three-stream framework design is analyzed on everyday object manipulation tasks with a Baxter humanoid robot in a semi-structured setting. The results indicate that CLUE-AI achieves an f-score of 94%, outperforming the other baselines in classifying anomalies.

Planning precise manipulation in robotics to perform grasp and release-related operations, while interacting with humans is a challenging problem. Reinforcement learning (RL) has the potential to make robots attain this capability. In this paper, we propose an affordance-based human-robot interaction (HRI) framework, aiming to reduce the action space size that would considerably impede the exploration efficiency of the agent. The framework is based on a new algorithm called Contextual Q-learning (CQL). We first show that the proposed algorithm trains in a reduced amount of time (2.7 seconds) and reaches an 84% of success rate. This suits the robot’s learning efficiency to observe the current scenario configuration and learn to solve it. Then, we empirically validate the framework for implementation in HRI real-world scenarios. During the HRI, the robot uses semantic information from the state and the optimal policy of the last training step to search for relevant changes in the environment that may trigger the generation of a new policy.