Search

The intelligent soft robot has received wide attention from both academia and the industry due to its remarkable adaptability. It performs intelligent behavioral learning and evolved morphologies in unpredictable environmental conditions. However, designing a soft robot with a well-adapted morphology involves searching through a large number of possible structures. Furthermore, to learn control tasks in diverse environments, the robot performs computationally intensive numerical simulations, which is time-consuming for evaluating the performance of robots. To address both issues, a multi-fidelity evolutionary algorithm is proposed, which consists of three main components. Firstly, a niching-based fidelity adjustment strategy is introduced to significantly reduce the evaluation cost by training the controller of each robot for only a small number of simulation steps. In particular, considering the estimation errors of the low-fidelity evaluation, the population is divided into multiple subpopulations with different fidelity levels for parallel optimization. Secondly, an effective morphology transfer strategy is proposed to improve the quality of offspring by transferring the local structure of robots in different subpopulations. Finally, a fast local search is developed to enhance the search efficiency of the algorithm without performing additional control simulations. The experimental results on 31 test tasks demonstrate that the proposed algorithm outperforms the SOTA design algorithms on 25 test tasks, especially when the computational budget is limited. Compared to the baseline algorithms, our algorithm reduces the computational cost by 60$\%$% while achieving similar performance. [ABSTRACT FROM AUTHOR]

The development of intelligent robot has always been an important research direction in the field of artificial intelligence, and the object detection of robot is the basis of intelligent perception and autonomous action. This study proposes an improved object detection algorithm which integrates two kinds of intelligent robot object detection techniques. In this process, the relationship between the center position of the target in the frame used for object detection and tracking is analyzed. It uses Laplace probability density as the parameter to calculate the center position relationship, and joins the correction network to correct the feature point positioning. The object re-capture function is added to solve the problem of object loss in the long-term object detection task, and the classifier is used to realize the object recognition. The results show that when the number of targets in the image reaches 20, the capture accuracy of the two data sets remains above 98.7%. In the intersection to union ratio test, when the real rectangular box contained 2M pixels, the intersection to union ratio of the method proposed in this study remains at or above 0.989. When conducting actual application memory usage tests, the proposed method maintains a memory usage of less than 2000Mb at runtime. It is shown that this method has better target detection efficiency and quality, and the requirement of hardware is lower.

To make mobile robot move considerately, an adaptive motion control method integrated potential field method is proposed. Risk map is firstly defined to show the magnitude of risks in the empty spaces around objects, then the considerate path is generated based on based on risk map. The robot is controlled by using integrated potential field method. The robot can automatically move with a considerate path to avoid disturbing others without make multiple moving rules, and areas with medium level risk are still passable, when there is no better choice. Besides, the robot will still make the users feel conformable by using effective interactions, like moving slowly to make sure safety or warning the users by sound. By comparing the experimental results with those controlled by conventional potential field method, the proposed method can control the robot move more adaptively and considerately with appropriate interactions.

Guide mobile robot have been researched and developed for decades. However, current robot cannot guide the user to the destination considerately, since the status of the user and the properties of the obstacles are not considered, especially for the guiding work of servicing visually impaired people. In this paper, a guide mobile robot with an easy-to-hold handle is designed, a generation method of spatial risk map is proposed to evaluate the influences of potential spaces of objects and a motion control method based on spatial risk map considering potential occupied spaces of objects is proposed. The users are successfully guided to the destination naturally without influencing other pedestrians by avoiding entering the potential spaces of objects automatically, and they can adjust their moving status in their own will since the considerate robot can adaptively adjust its guide status to adapt to the user. Through comparing guiding experiments of different tasks for 10 blindfolded users, the proposed guide mobile robot is proven considerate by guiding the users adaptively and make other pedestrians feel comfortable by avoiding entering the potential spaces of objects automatically.

Human-robot teaming receives an ever-increasing level of attention in research, development and industry. Novel approaches to task sharing in hybrid teams range from optimized schedules to intelligent cobot assistants with a high degree of autonomy. These approaches must prove their usefulness and benefits compared to manual work or full automation – particularly when it comes to assessing their potential for productive industrial use. This leverages demand for standardized, repeatable benchmarks to compare approaches and measure improvements in a reproducible way. Designing such benchmarks is challenging as numerous aspects, from safety considerations to human factors and team performance, must be considered. This survey seeks to contribute to the future development of benchmarks for the field of collaborative assembly, handling, and industrial cobot applications by giving a comprehensive overview of relevant metrics, evaluation strategies, and tasks for human-robot teams.

Koopman operator theory has proven to be a promising approach to nonlinear system identification and global linearization. For nearly a century, there had been no efficient means of calculating the Koopman operator for applied engineering purposes. The introduction of a recent computationally efficient method in the context of fluid dynamics, which is based on the system dynamics decomposition to a set of normal modes in descending order, has overcome this long-lasting computational obstacle. The purely data-driven nature of Koopman operators holds the promise of capturing unknown and complex dynamics for reduced-order model generation and system identification, through which the rich machinery of linear control techniques can be utilized. Given the ongoing development of this research area and the many existing open problems in the fields of smart mobility and vehicle engineering, a survey of techniques and open challenges of applying Koopman operator theory to this vibrant area is warranted. This review focuses on the various solutions of the Koopman operator which have emerged in recent years, particularly those focusing on mobility applications, ranging from characterization and component-level control operations to vehicle performance and fleet management. Moreover, this comprehensive review of over 100 research papers highlights the breadth of ways Koopman operator theory has been applied to various vehicular applications with a detailed categorization of the applied Koopman operator-based algorithm type. Furthermore, this review paper discusses theoretical aspects of Koopman operator theory that have been largely neglected by the smart mobility and vehicle engineering community and yet have large potential for contributing to solving open problems in these areas.

Robotics has made it possible to change and improve many support processes for vulnerable people in different settings. In recent years, its use has been oriented toward supporting therapeutic interventions of neurodevelopmental disorders (NDD), including attention deficit hyperactivity disorder (ADHD). This review of the literature highlights how advances in robotics have evolved in different scenarios of ADHD treatment, its collaboration with other emerging technologies, its results, its limitations, and the research challenges for the future development of robotics in the field of supporting children with ADHD. The authors conducted a literature review based on the location of keywords ‘robotics’ and several NNDs such as ‘ADHD’, ’Autism Spectrum Disorder (ASD)’, ‘cerebral palsy’, and ‘dementia’ in titles, abstracts, and introduction of scientific articles in the Scopus and Web of Science (WoS) database. The reviewed literature was classified according to the type of therapy supported by the robots, the type of robot and the associated technologies. From this analysis, we can solve the research question: Which types of robots have the potential for specific applications in ADHD treatment? Furthermore, this article shows that despite favorable technical results, robotic technologies that support ADHD therapies require significant improvements in terms of scalability, human-machine interaction, and treatment and processing of acquired information to be applied effectively in real-world therapies. The most significant research challenges are proposed to drive research efforts to develop new approaches to enable robotic assistants to participate in ADHD therapies.

Perception and interpretation of the terrain is essential for robot navigation, particularly in off-road areas, where terrain characteristics can be highly variable. When planning a path, features such as the terrain gradient and roughness should be considered, and they can jointly represent the traversability cost of the terrain. Despite this range of contributing factors, most cost maps are currently binary in nature, solely indicating traversible versus non-traversible areas. This work presents a joint local and global planning methodology for building continuous cost maps using LIDAR, based on a novel traversability representation of the environment. We investigate two approaches. The first, a statistical approach, computes terrain cost directly from the point cloud. The second, a learning-based approach, predicts an IMU response solely from geometric point cloud data using a 2D-Convolutional-LSTM neural network. This allows us to estimate the cost of a patch without directly driving over it, based on a data set that maps IMU signals to point cloud patches. Based on the terrain analysis, two continuous cost maps are generated to jointly select the optimal path considering distance and traversability cost for local navigation. We present a real-time terrain analysis strategy applicable for local planning, and furthermore demonstrate the straightforward application of the same approach in batch mode for global planning. Off-road autonomous driving experiments in a large and hybrid site illustrate the applicability of the method. We have made the code available online for users to test the method. [ABSTRACT FROM AUTHOR]

Currently, visual-tactile fusion learning for robotic object recognition has achieved appealing performance, due to the fact that visual and tactile data can offer complementary information. However: 1) the distinct gap between vision and touch makes it difficult to fully explore the complementary information, which would further lead to performance degradation and 2) most of the existing visual-tactile fused learning methods assume that visual and tactile data are complete, which is often difficult to be satisfied in many real-world applications. In this article, we propose a partial visual-tactile fused (PVTF) framework for robotic object recognition to address these challenges. Specifically, we first employ two modality-specific (MS) encoders to encode partial visual-tactile data into two incomplete subspaces (i.e., visual subspace and tactile subspace). Then, a modality gap mitigated (MGM) network is adopted to discover modality-invariant high-level label information, which is utilized to generate gap loss and further help updating the MS encoders for relatively consistent visual and tactile subspaces generation. In this way, the huge gap between vision and touch is mitigated, which would further contribute to mine the complementary visual-tactile information. Finally, to achieve data completeness and complementary visual-tactile information exploration simultaneously, a cycle subspace leaning technique is proposed to project the incomplete subspaces into a complete subspace by fully exploiting all the obtainable samples, where complete latent representations with maximum complementary information can be learned. A lot of comparative experiments conducted on three visual-tactile datasets validate the advantage of the proposed PVTF framework, by comparing with state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

Perceiving humans in the context of Intelligent Transportation Systems (ITS) often relies on multiple cameras or expensive LiDAR sensors. In this work, we present a new cost-effective vision-based method that perceives humans’ locations in 3D and their body orientation from a single image. We address the challenges related to the ill-posed monocular 3D tasks by proposing a neural network architecture that predicts confidence intervals in contrast to point estimates. Our neural network estimates human 3D body locations and their orientation with a measure of uncertainty. Our proposed solution (i) is privacy-safe, (ii) works with any fixed or moving cameras, and (iii) does not rely on ground plane estimation. We demonstrate the performance of our method with respect to three applications: locating humans in 3D, detecting social interactions, and verifying the compliance of recent safety measures due to the COVID-19 outbreak. We show that it is possible to rethink the concept of “social distancing” as a form of social interaction in contrast to a simple location-based rule. We publicly share the source code towards an open science mission. [ABSTRACT FROM AUTHOR]

With the development of smart manufacturing technology, manufacturers all over the world have begun using automated processing and unmanned factories. China already has many high-end production lines that use intelligent robots, where Artificial Intelligence (AI) and robot vision play a critical role in facilitating robot-driven production. This paper provides a review of the state-of-the-art in Deep Learning (DL) techniques and the applications of DL-based robot vision in smart manufacturing, with an emphasis on the contributions made by China. We conclude by summarizing challenges and prospects in the area. [ABSTRACT FROM AUTHOR]

In recent years, reinforcement learning (RL) has been widely used to solve multi-agent navigation tasks, and a high-fidelity level for the simulator is critical to narrow the gap between simulation and real-world tasks. However, high-fidelity simulators have high sampling costs and bottleneck the training model-free RL algorithms. Hence, we propose a Multi-Fidelity Simulator framework to train Multi-Agent Reinforcement Learning (MFS-MARL), reducing the total data cost with samples generated by a low-fidelity simulator. We apply the depth-first search to obtain local feasible policies on the low-fidelity simulator as expert policies to help the original reinforcement learning algorithm explore. We built a multi-vehicle simulator with variable fidelity levels to test the proposed method and compared it with the vanilla Soft Actor-Critic (SAC) and expert actor methods. The results show that our method can effectively obtain local feasible policies and can achieve a 23% cost reduction in multi-agent navigation tasks.

Self-aware robots rely on depth sensing to interact with the surrounding environment, e.g. to pursue object grasping. Yet, dealing with tiny items, often occurring in industrial robotics scenarios, may represent a challenge due to lack of sensors yielding sufficiently accurate depth measurements. Existing active sensors fail at measuring details of small objects (< 1cm) because of limitations in the working range, e.g. usually beyond 50 cm away, while off-the-shelf stereo cameras are not suited to close-range acquisitions due to the need for extremely short baselines. Therefore, we propose a framework designed for accurate depth sensing and particularly amenable to reconstruction of miniature objects. By leveraging on a single camera mounted in eye-on-hand configuration and the high repeatability of a robot, we acquire multiple images and process them through a stereo algorithm revised to fully exploit multiple vantage points. Using a novel dataset addressing performance evaluation in industrial applications, our Single camera Stereo Robot (SiSteR) delivers high accuracy even when dealing with miniature objects. We will provide a public dataset and an open-source implementation of our proposal to foster further development in this field.

Recent developments in the field of service robots have led to a renewed interest in human-robot coexistence environments such as at home and office. In this regard, this study focuses on one of such service robots, a human-following mobile robot. In particular, we consider predicting the future trajectory of pedestrians using a machine learning algorithm to improve the accuracy of tracking people. Massive trajectory data is required in existing methods to train the prediction model; however, collecting a sufficient amount of data in general public places before providing services is challenging. Therefore, in this study, we propose a trajectory prediction method based on extracting similar datasets from a large-size dataset and generating a pre-trained prediction model using the extracting datasets. We express the data features in the source and target environments as probability distributions and evaluate the divergence between them. Specifically, the dataset features are expressed as a multidimensional Gaussian distribution and discrete distribution of samples. Then, similarities using the Kullback-Leibler divergence are compared. To verify the effectiveness of the proposed method, we compare the prediction results of the LSTM-based algorithm with those obtained by extracting multiple source datasets from a large dataset and training prediction models using these datasets. The result shows that the proposed method makes it possible to construct an appropriate prediction model with high accuracy in trajectory prediction.

Industrial robots are increasingly used to perform tasks that require an interaction with the surrounding environment (e.g., assembly tasks). Such environments are usually (partially) unknown to the robot (in terms of dynamic characteristics), demanding the implemented controllers to suitably react to the established interaction. Standard controllers require force/torque measurements to close the loop, making it, if possible, to adapt the robot behavior to the specific environment. However, most of the industrial manipulators do not have embedded force/torque sensor(s), which entails additional effort in terms of costs and implementation for their integration in the robotic setup. To extend the use of sensorless compliant controllers to force control, a robot–environment interaction dynamics model-based methodology is presented in this article. Relying on the sensorless Cartesian impedance control, an extended Kalman filter (EKF) is proposed to estimate the stiffness of an interaction environment. Exploiting the provided estimation, the robot–environment coupled dynamic modeling is used to design an optimal LQR interaction controller to close the force loop. The control gains can be analytically computed by solving the related Riccati equation, as such gains are a function of the impedance control and environment parameters. In addition, the interaction force can be estimated to close the force loop, making the sensorless robot able to perform the target interaction task. The described approach has been validated with experiments by analyzing two scenarios: a probing task and a closing of a plastic (i.e., compliant) box with a snap-fit closure mechanism. The performance of the proposed control framework has been evaluated, highlighting the capabilities of the EKF and the optimal LQR interaction controller. Finally, the proposed control schema is enhanced by the adaptation of the EKF for the estimation of the external wrench. Two additional experiments are provided to show the improvements on the control schema (a polishing-like task and an assembly task). The Franka EMIKA panda robot has been used as the reference robotic platform for experimental validation. [ABSTRACT FROM AUTHOR]

Sampling-based planning methods often become inefficient due to narrow passages. Narrow passages induce a higher runtime, because the chance to sample them becomes vanishingly small. In recent work, we showed that narrow passages can be approached by relaxing the problem using admissible lower dimensional projections of the state space. Those relaxations often increase the volume of narrow passages under projection. Solving the relaxed problem is often efficient and produces an admissible heuristic we can exploit. However, given a base path, i.e., a solution to a relaxed problem, there are currently no tailored methods to efficiently exploit the base path. To efficiently exploit the base path and thereby its admissible heuristic, we develop section patterns, which are solution strategies to efficiently exploit base paths in particular around narrow passages. To coordinate section patterns, we develop the pattern dance algorithm, which efficiently coordinates section patterns to reactively traverse narrow passages. We combine the pattern dance algorithm with previously developed multilevel planning algorithms and benchmark them on challenging planning problems like the Bugtrap, the double L-shape, an egress problem, and on four pregrasp scenarios for a 37 degrees-of-freedom shadow hand mounted on a KUKA LWR robot. Our results confirm that section patterns are useful to efficiently solve high-dimensional narrow passage motion planning problems. [ABSTRACT FROM AUTHOR]

This paper proposed an improved incremental smoothing and mapping (iSAM) which combines flexible re-linearization threshold with error learning model to improve the efficiency and accuracy of navigation for autonomous underwater vehicle (AUV). The flexible threshold of the proposed method, which can avoid periodic re-linearization of iSAM, is controlled by adaptive threshold. Through the use of flexible re-linearization, the proposed method can reduce the running time and carry out re-linearization timely. Simultaneously, for the first time, the proposed method takes advantages of the fast training time of hidden-layer neural networks and Gaussian Process Regression which is more suitable for non-linearity to get error learning model for iSAM. The proposed method can achieve better accuracy with only a rough model and can be easily transplanted to other systems without cumbersome calculations of the precise model. Our algorithm has been demonstrated by a range of simulated and real datasets, especially in practical application it outperforms current available iSAM algorithms in efficiency and accuracy. The RMSE of proposed method increases by 21.7% and efficiency also has been improved by 56% than iSAM2 in practical application for AUV. [ABSTRACT FROM AUTHOR]

Data association is the foundation of state estimation in mobile robot simultaneous localization and mapping. Aiming at the problems of false association, high computational complexity in joint compatible branch and bound algorithm, we propose an optimized joint compatible branch and bound data association algorithm based on Gaussian mixture clustering. Firstly, the local association strategy is adopted to limit data association in local region, so as to reduce the number of features involved in data association at the current moment. Secondly, the Gaussian mixture clustering algorithm is used in local areas to group the observed values at the current moment, so as to get several groups that have little correlation with each other. Finally, joint compatible branch and bound data association algorithm is used in each group for data association, and the optimal solution is obtained according to mutual exclusion criteria and optimal criteria. The experiment results verify that the algorithm improved the accuracy of data association, reduced the computational complexity and improved the efficiency of data association.

We present a novel implementation of a Rock-Paper-Scissors (RPS) game interaction with a social robot. The framework is tailored to be computationally lightweight, as well as entertaining and visually appealing through collaboration with designers and animators. The fundamental gesture recognition pipeline employs a Leap motion device and two separate machine learning architectures to evaluate kinematic hand data on-the-fly. The first architecture is used to recognize and segment human motion activity in order to initialize the RPS play, and the second architecture is used to classify hand gestures into rock, paper or scissors. The employed tabletop robot interacts in the RPS play through unique animated gestural movements and vocalizations designed by animators which communicate the robot's choices as well as cognitive reflection on winning, losing and draw states. Performance of both learning architectures is carefully evaluated with respect to accuracy, reliability and run time performance under different feature and classifier types. Moreover, we assess our system during an interactive RPS play between robot and human. Experimental results show that the proposed system is robust to user variations and play style in real environment conditions. As such, it offers a powerful application for the subsequent exploration of social human-machine interaction.

The goal of the study was to research different communication modalities needed for intuitive Human-Robot Interaction. This study utilizes a Wizard of Oz prototyping method to enable a restriction-free, intuitive interaction with an industrial robot. The data from 36 test subjects suggests a high preference for speech input, automatic path planning and pointing gestures. The catalogue developed during this experiment contains intrinsic gestures suggesting that the two most popular gestures per action can be sufficient to cover the majority of users. The system scored an average of 74% in different user interface experience questionnaires, while containing forced flaws. These findings allow a future development of an intuitive Human-Robot interaction system with high user acceptance.

Increasing the flexibility of robots needs systems more capable in perceiving and interacting with the environment. A challenge is still to easy design the robotic system around the application, especially when the objects to be manipulated are bulky, and the relative positions between the robot and the objects to be manipulated are uncertain and high precision is required to successfully complete a task. In this paper a possible guideline to design a system capable to localize itself, identify a target, bulky, object and manipulate it, is presented. A method for tuning the impedance control parameters is shown, to keep interaction forces below dangerous values. The autonomous localization, grasping and assembly of a sidewall panel of an airplane is used as test. Experiments show that the success rate of completing a task increases, combining vision perception and force control, with respect to the single use of visual localization and position control.

Robots, in contrast to typical computational systems, affect the physical environment directly. Therefore, other assumptions must be considered for task management procedures in these system types. Robots coexist with humans in the environment and act upon potentially dangerous objects (e.g., a cooker); hence, extra safety procedures in robot task harmonisation must be ensured. Additionally, an algorithm that schedules tasks for a robot and optimises the robot's operation needs to consider the robot motion time, dynamics of the physical processes, changes in the robot user preferences and changes in the environment made by other habitants. In this article, we investigate the problem of switching between various independent tasks safely and state requirements for a control system resolving it. The tasks are uploaded to a store, launched on a robot at the user's request (similar to smartphone applications) and scheduled following a configurable algorithm. Furthermore, we design a model of systems satisfying the requirements. The systems are structured with agents of different classes. We propose a task-switching procedure and dedicated states of the finite-state machines describing the operation of the agents. Finally, we present a TaskER framework implementing the model, and we verify the model through the execution of an exemplary system in scenarios showing the benefits of the model implementation. As a result of our approach application, the robot tasks can be safely interrupted, postponed, resumed, and potential danger (e.g., leaving a cooker on for a long time) can be minimised.

This paper presents a system enabling a mobile robot to autonomously pick-up objects a human is pointing at from the floor. The system does not require object models and is designed to grasp unknown objects. The robot decides by itself if an object is suitable for grasping by considering measures of size, position and the environment suitability. The implementation is built on the second prototype of the home care robot Hobbit, thereby verifying that complex robotic manipulation tasks can be performed with economical hardware. The presented system was already tested in real apartments with elderly people. We highlight this by discussing the additional complexity for complete autonomous behavior in apartments compared with tests in labs.

In order to maintain the submarine equipment, autonomous underwater vehicle (AUV) is usually assigned to track the submarine cables or pipes. The capabilities of navigation and control are critical to track the target accurately. Ultra-short baseline (USBL) is essential equipment for AUV, which uses sound waves for positioning. Unfortunately, due to the low frequency of USBL, it inevitably limits the frequency of control and ultimately affects the tracking effect. In order to improve the aforementioned issue and achieve better tracking tasks, intelligent collaborative navigation and control (CNaC) was herein proposed in this article. First, we proposed nonlinear state reconstruction neural network navigation, which used the neural networks to reconstruct the state between two adjacent USBL valid values online. Combined with the valid USBL and reconstructed states, the online process model generated by neural networks are applied to give the estimate position for AUV. At last, intelligent CNaC use the estimated position and valid USBL as inputs to control AUV to achieve tracking tasks. This strategy makes the control frequency free from the limitation of the USBL frequency. The proposed intelligent CNaC is demonstrated by simulation and real experiments. Compared to mechanically combining the traditional navigation and control algorithm, the tracking accuracy of intelligent CNaC improves by 81.96%. [ABSTRACT FROM AUTHOR]

The introduction of service robots in the public domain has introduced a paradigm shift in how robots are interacting with people, where robots must learn to autonomously interact with the untrained public instead of being directed by trained personnel. As an example, a hospital service robot is told to deliver medicine to Patient Two in Ward Three. Without awareness of what “Patient Two” or “Ward Three” is, a service robot must systematically explore the environment to perform this task, which requires a long time. The implementation of a Semantic Map allows for robots to perceive the environment similar to people by associating semantic information with spatial information found in geometric maps. Currently, many semantic mapping works provide insufficient or incorrect semantic-metric information to allow a service robot to function dynamically in human-centric environments. This paper proposes a semantic map with a hierarchical semantic organization structure based on a hybrid metric-topological map leveraging convolutional neural networks and spatial room segmentation methods. Our results are validated using multiple simulated and real environments on our lab's custom developed mobile service robot and demonstrate an application of semantic maps by providing only vocal commands. We show that this proposed method provides better capabilities in terms of semantic map labeling and retain multiple levels of semantic information.

The purpose of this research is to analyze and upgrade the performances of the Baxter intelligent robot, through data mining methods. The case study belongs to the robotics domain, being integrated in the context of manufacturing execution systems and product lifecycle management, aiming to overcome the lack of vertical integration inside a company. The explored data comprises the parameters registered during the activities of the Baxter intelligent robot, as, for example, the movement of the left or right arm. First, the state of the art concerning the data mining methods is presented, and then the solution is detailed by describing the data mining techniques. The final purpose was that of improving the speed and robustness of the robot in the production. Specific techniques and sometimes their combinations are experimented and assessed, in order to perform root cause analysis, then powerful classifiers and metaclassifiers, as well as deep learning methods, in optimum configuration, are analyzed for prediction. The experimental results are described and discussed in details, then the conclusions and further development possibilities are formulated. Based on the experiments, important relationships among the robot parameters were discovered, the obtained accuracy for predicting the target variables being always above 96%.

Traditional offline programming or teach pendant-based methods limit the collaboration capabilities of users and robots to cope with complex and changing industrial tasks. To address efficient robot manipulation in high-mix and low-volume tasks, especially for skillful tasks involving both trajectory and force control requirements, fast robot teaching and skill transferability are critical. Compared to manually dragging a heavy robot, or programming a trajectory by complex calculations, we believe that robots can efficiently learn skills from direct teaching through telemanipulation, and improve the skills based on optimization with sensory feedback. In aerospace engine, maintenance, repair, and operations, the surfaces of aerospace components are required to be masked by tapes. We propose a fast and intuitive telemanipulation-based method to teach a robot these masking skills and compare the performance of the proposed method with teach pendant–based methods among several users. This study aims to prove the efficiency and intuitiveness of the telemanipulation-based method proposed herein for enabling a robot to learn skillful and complex manipulation tasks. [ABSTRACT FROM AUTHOR]

Snap-fit assemblies are widely used in the manufacturing of several product types, allowing part joining, while the parts remain unprocessed. The locking mechanism of a snap-fit is usually done within the object structure, not allowing visual identification of the successful process completion. Humans consider the forces developed between the two parts or the snapping sound, as an indication of success. This is difficult to realize in robotic assembly, and the process success is usually identified at a product quality control stage. The aim of this article is to migrate the human ability to identify a successful snap assembly to autonomous robotic assembly, via a machine learning framework, enabled by human–robot collaboration for rich data collection and labeling. The proposed framework allows learning while minimizing complexity, cost, and time. A generic feature set is proposed, which can produce good identification results in different snap assembly types. A feature transformation is also introduced that is fundamental for the real-time operation of the proposed framework and the identification of successful snap-assemblies. Three different objects are used to experimentally validate the approach using a KUKA LWR4+ robotic arm, resulting in high classification and real-time identification accuracy. Finally, a comparison with a model-based method is conducted. Note to Practitioners—This article is motivated by the need for flexible robotic assembly processes, and specifically the characterization of snap-fits, an assembly-type widely used in automated robotic processes. The snap-fit completion cannot be easily identified from a visual inspection, but the final result is usually realized by sensing the developed forces or identifying the snapping sound during a manual assembly. Training industrial robots to imitate the human snap-fit identification will allow accurate monitoring of such assemblies and the automation of the process with certainty for the final result. A quick and accurate perception of the process status can improve the overall process deployment speed and efficiency of such assemblies. The proposed framework allows the real-time characterization of the snap-fit process with very high accuracy and is easily implemented with collaborative robots. The framework can be further extended to other assembly processes using assembly forces as an indication signal. [ABSTRACT FROM AUTHOR]

An innovative strategy is proposed to investigate the problem of synchronization control for the unreliable network systems in this article. It is shown that the dynamics for nodes in the heterogeneous network are nonidentical, which can describe more complicated processes. The proposed distributed controller consists of the reference generator (RG) and regulator. First, any two RGs can exchange effective information by the communication channels. Time-varying delay may occur in the communication channels due to the resource constraints and the limited to network bandwidth. By designing the distributed control law, the closed-loop of the leader and all RGs can be equivalently rewritten as the feedback interconnection of a linear time-invariant system and an integral operator. In the light of the small gain theorem, the stability criteria for the feedback interconnection is obtained. Second, one leader and all RGs are considered as the exosystem of each nonidentical network node. The designed regulator can copy the dynamics of the exosystem and enforce that each nonidentical node tracks the corresponding exosystem. Finally, the control effectiveness of this novel strategy is substantiated through a numerical simulation. [ABSTRACT FROM AUTHOR]

The fusion of sensor data is an essential requirement for active perception in autonomous robots in order to accurately perceive an environment, make appropriate adjustments that improve the understanding of the environment and generate actionable insights. Despite a large body of literature in this domain, only a handful of research focuses on unsupervised machine learning, which is increasingly important in unlabeled, unknown environments. This paper proposes a scalable self-organizing neural architecture for environmental perception based on multimodal fusion using unsupervised machine learning. Inspired by the biological counterpart, the neural architecture consists of topographic maps across each modality and incorporates a novel scalable self-organization process that handles high volume, high-velocity sensor data. The co-occurrence relationships across modalities are captured in cross-modal connections and the reverse Hebbian projection is used for multimodal representation. This proposed unsupervised machine learning-based scalable fusion technique was implemented on Apache Hadoop and Apache Spark. It was validated using an extensive multimodal human activity sensor dataset to demonstrate efficient representation and multimodal fusion for active perception. Note to Practitioners—Heterogeneous sensors are increasingly used to capture different aspects of an event or an environment. This paper presents a multimodal fusion mechanism to fuse such heterogeneous sensor data to support the accurate perception of the surrounding. The proposed mechanism uses natural regularities in patterns observed across multiple modalities and enriches each modality with information from other co-occurring modalities to achieve a multimodal clustering. The cluster can be used for online prediction and offline action planning of the robot by matching the current sensory inputs. The algorithm uses unsupervised machine learning, and hence, can be used with any multimodal dataset without being restricted by the labeling of the dataset. In addition, we adapt the algorithm for multiple big data platforms so that it is effective and pragmatic for large-scale sensor data fusion tasks. [ABSTRACT FROM AUTHOR]

This paper describes a set of software modules and algorithms for obstacle avoidance. The approach described here is designed to work in conjunction with any path planning method that generates a viable sequence of waypoints, with obstacle avoidance regarded as a local deviation of the planned path of points. The helm architecture used here will temporarily spawn a new obstacle avoidance behavior for each near detected obstacle, and generate a maneuvering objective function to avoid the obstacle. Once the vehicle passes the obstacle, the helm deletes the behavior and frees its memory space. The helm, obstacle manager, and obstacle avoidance behavior are described here. This software arrangement was used by the MIT 2014 RobotX team, and will be included in the next software release of the open source MOOS-IvP marine autonomy project. [ABSTRACT FROM AUTHOR]

Previous work on ethical machine reasoning has largely been theoretical, and where such systems have been implemented, it has, in general, been only initial proofs of principle. Here, we address the question of desirable attributes for such systems to improve their real world utility, and how controllers with these attributes might be implemented. We propose that ethically critical machine reasoning should be proactive, transparent, and verifiable. We describe an architecture where the ethical reasoning is handled by a separate layer, augmenting a typical layered control architecture, ethically moderating the robot actions. It makes use of a simulation-based internal model and supports proactive, transparent, and verifiable ethical reasoning. To do so, the reasoning component of the ethical layer uses our Python-based belief–desire–intention (BDI) implementation. The declarative logic structure of BDI facilitates both transparency, through logging of the reasoning cycle, and formal verification methods. To prove the principles of our approach, we use a case study implementation to experimentally demonstrate its operation. Importantly, it is the first such robot controller where the ethical machine reasoning has been formally verified. [ABSTRACT FROM AUTHOR]

The research and development of intelligent automation solutions is a ground-breaking point for the factory of the future. A promising and challenging mission is the use of autonomous robot systems to automate tasks in the field of maintenance. For this purpose, the robot system must be able to plan autonomously the different manipulation tasks and the corresponding paths. Basic requirements are the development of algorithms with a low computational complexity and the possibility to deal with environmental uncertainties. In this paper, an approach is presented, which is especially suited to solve the problem of maintenance automation. For this purpose, offline data from CAD is combined with online data from an RGBD vision system via a probabilistic filter, to compensate uncertainties from offline data. For planning the different tasks, a method is explained, which uses a symbolic description, founded on a novel sampling-based method to compute the disassembly space. For path planning, we use global state-of-the-art algorithms with a method that allows the adaption of the exploration stepsize in order to reduce the planning time. Every method is experimentally validated and discussed. Note to Practitioners—This paper was motivated by the current deficit to automate maintenance tasks in manufacturing systems. Today there exist many automatic methods for fault detection, but there is no possibility to restore the desired state. A typical application is the replacement of a defective component or service tasks such as refilling of cooling lubricant in machine tools. The focus of this paper is to provide automatic planning concepts for robot systems which deal with this problem. For this, a task planning algorithm is proposed which uses CAD and vision data. The advantage of this method is that it works on a contact analysis, based on a discrete approximation of the disassembly space, allowing a fast determination of possible robot manipulations. Also, a new preprocessing concept for path planning is developed which enables the adaptation of the stepsize of the path planner, based on the local occupancy of the environment. Through this, it is possible to drastically reduce the required planning effort. [ABSTRACT FROM AUTHOR]

This paper presents an algorithm called \varepsilon ^\star , for online coverage path planning of unknown environment. The algorithm is built upon the concept of an Exploratory Turing Machine (ETM), which acts as a supervisor to the autonomous vehicle to guide it with adaptive navigation commands. The ETM generates a coverage path online using Multiscale Adaptive Potential Surfaces (MAPS), which are hierarchically structured and dynamically updated based on sensor information. The \varepsilon ^\star -algorithm is computationally efficient, guarantees complete coverage, and does not suffer from the local extrema problem. Its performance is validated by 1) high-fidelity simulations on Player/Stage and 2) actual experiments in a laboratory setting on autonomous vehicles. [ABSTRACT FROM AUTHOR]

This paper describes a novel method for guiding a robot in an environment containing moving objects. The robot predicts the motions of the objects and plans a collision-free path to the goal. It is a hybrid method that combines local and global approaches. This paper describes two new predicting algorithms, which are based on the AprioriAll algorithm, and one planning algorithm that is based on heuristic search. This method has been tested via a framework using a simulation environment. The results have shown that the robot succeeds in reaching the goal via collision-free paths at an average rate of about 94%. [ABSTRACT FROM PUBLISHER]

Soft matter capacitive sensors are developed and integrated into the silicone skin of finger tips and palm of a robotic hand. Distributed sensing nodes are used for obtaining preliminary pre-touch information about the object that should be grasped. This includes conductive objects taking advantage of proximity sensing from changes in electric field and non-conductive objects taking advantage of pressure sensing from the deforming capacitive electrodes. We argue that this system can eventually address one important challenge in control of multi-DOF bionic hands, i.e., autonomously choosing the right closing pattern. Sensor fabrication, characterization, and integration into the hand are described. An algorithm for detection and activation of closing patterns is suggested, and results are discussed. [ABSTRACT FROM PUBLISHER]

Mobile robots with complex morphology are essential for traversing rough terrains in Urban Search & Rescue missions. Since teleoperation of the complex morphology causes high cognitive load of the operator, the morphology is controlled autonomously. The autonomous control measures the robot state and surrounding terrain which is usually only partially observable, and thus the data are often incomplete. We marginalize the control over the missing measurements and evaluate an explicit safety condition. If the safety condition is violated, tactile terrain exploration by the body-mounted robotic arm gathers the missing data. [ABSTRACT FROM AUTHOR]

We present algorithms for environment exploration in the context of a habitat monitoring task, where the goal is to track radio-tagged invasive fish with autonomous surface or ground robots. The first task is navigation around an unknown obstacle using an input from a front-facing sonar. This capability is important for navigation on inland lakes, because plants and shallow shorelines are hard to map in advance. The second task involves energy harvesting for long-term operation. We address the problem of exploring the solar map of the environment which is used for energy-efficient navigation. For both problems, we present online algorithms and examine their performance using competitive analysis. In competitive analysis, the performance of an online algorithm is compared against the optimal offline algorithm. For obstacle avoidance, the offline algorithm knows the shape of the obstacle. For solar exploration, the offline algorithm knows the geometry of the shadow-casting objects. We obtain an $O(1)$ competitive ratio for obstacle avoidance and an $O(\log n)$ competitive ratio for solar exploration, where $n$ is the number of critical points to observe. The strategies for obstacle avoidance are validated through extensive field experiments, and the strategies for exploration are validated with simulations. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of simultaneously achieving low trajectory tracking errors and compliant control without using explicit mathematical models of task dynamics. To achieve this goal, we propose a new movement representation called compliant movement primitives (CMPs), which encodes position trajectory and associated torque profiles and can be learned from a single user demonstration. With the proposed control framework, the robot can remain compliant and consequently safe for humans sharing its workspace, even if high trajectory tracking accuracy is required. We developed a statistical learning approach that can use a database of existing CMPs and compute new ones, adapted for novel task variations. The proposed approach was evaluated on a Kuka LWR-4 robot performing 1) a discrete pick-and-place task with objects of varying weight and 2) a periodic handle turning operation. The evaluation of the discrete task showed a 15-fold decrease of the tracking error while exhibiting compliant behavior compared to the standard feedback control approach. It also indicated no significant rise in the tracking error while using generalized primitives computed by the statistical learning method. With respect to unforeseen collisions, the proposed approach resulted in a 75% drop of contact forces compared to standard feedback control. The periodic task demonstrated on-line use of the proposed approach to accomplish a task of handle turning. [ABSTRACT FROM PUBLISHER]

Ongoing technological advances in the areas of computation, sensing, and mechatronics enable robotic-based systems to interact with humans in the real world. To succeed against a human in a competitive scenario, a robot must anticipate the human behavior and include it in its own planning framework. Then it can predict the next human move and counter it accordingly, thus not only achieving overall better performance but also systematically exploiting the opponent’s weak spots. Pool is used as a representative scenario to derive a model-based planning and control framework where not only the physics of the environment but also a model of the opponent is considered. By representing the game of pool as a Markov decision process and incorporating a model of the human decision-making based on studies, an optimized policy is derived. This enables the robot to include the opponent’s typical game style into its tactical considerations when planning a stroke. The results are validated in simulations and real-life experiments with an anthropomorphic robot playing pool against a human. [ABSTRACT FROM PUBLISHER]

We investigate the modeling of inverse dynamics without prior kinematic information for holonomic rigid-body robots. Despite success in compensating robot dynamics and friction, general inverse dynamics models are nontrivial. Rigid-body models are restrictive or inefficient; learning-based models are generalizable yet require large training data. The structured kernels address the dilemma by embedding the robot dynamics in reproducing kernel Hilbert space. The proposed kernels autonomously converge to rigid-body models but require fewer samples; with a semi-parametric framework that incorporates additional parametric basis for friction, the structured kernels can efficiently model general rigid-body robots. We tested the proposed scheme in simulations and experiments; the models that consider the structure of function space are more accurate. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a collective decision-making method for swarms of robots. The method enables a robot swarm to select, from a set of possible actions, the one that has the fastest mean execution time. By means of positive feedback the method achieves consensus on the fastest action. The novelty of our method is that it allows robots to collectively find consensus on the fastest action without measuring explicitly the execution times of all available actions. We study two analytical models of the decision-making method in order to understand the dynamics of the consensus formation process. Moreover, we verify the applicability of the method in a real swarm robotics scenario. To this end, we conduct three sets of experiments that show that a robotic swarm can collectively select the shortest of two paths. Finally, we use a Monte Carlo simulation model to study and predict the influence of different parameters on the method. [ABSTRACT FROM PUBLISHER]

The last two decades have seen a number of developments in creating robots to play billiards. The designed robotic systems have successfully incorporated the kinematics required and have had appropriate machine vision elements for a decent gameplay. Independently, computer scientists have also developed the artificial intelligence programs needed for the strategy to play billiards. Despite these developments, the accurate ball manipulation aspect of the game, needed for good performance, has not been addressed enough; two important parameters are the potting accuracy and advantageous cue ball positioning for next shot. In this regard, robotic ball manipulation by predicting the ball trajectories under the action of various dynamic phenomena, such as ball spin, impacts and friction, is the key consideration of this research. By establishing a connection to the methods used in nonprehensile robotic manipulation, a forward model is developed for the rolling, sliding, and two distinct types of frictional impacts of billiards balls are developed. High-speed camera-based tracking is performed to determine the physical parameters required for the developed dynamic models. To solve the inverse manipulation problem, i.e., the decision on shot parameters, for accurate ball positioning, an optimization based solution is proposed. A simplistic ball manipulator is designed and used to test the theoretical developments. Experimental results show that a 90% potting accuracy and a 100–200 mm post-shot cue ball positioning accuracy has been achieved by the autonomous system within a table area of $6 \times 5$ ft2. [ABSTRACT FROM PUBLISHER]