Search

Tele-impedance augments classic teleoperation by enabling the human operator to actively command remote robot stiffness in real-time, which is an essential ability to successfully interact with the unstructured and unpredictable environment. However, the literature is missing a study on benefits and drawbacks of different types of stiffness command interfaces used in bilateral tele-impedance. In this article, we introduce a term called coupling effect, which pertains to the coupling between human-commanded stiffness going to the remote robot and force feedback coming from the remote robot. We hypothesize that, whenever the operator's commanded stiffness and force feedback are subject to coupling effect (e.g., muscle activity based stiffness command interfaces), force feedback can invoke involuntary changes in the commanded stiffness due to human reflexes. Although the coupling effect takes away some degree of the operator's control over the commanded stiffness, these involuntary changes can be either beneficial (e.g., during position tracking) or detrimental (e.g., during force tracking) to the task performance on the remote robot side. We examined the coupling effect in an experimental study with $\mathbf {16}$ participants, who performed position and force tracking tasks by using both coupled type (muscle activity based) and decoupled type (external device based) of interface. The results demonstrate a benefit of the coupling effect when the remote robot is operating in presence of unexpected force perturbations, where lower absolute error in position tracking task was observed. On the other hand, the decoupled type of interface is beneficial for force tracking tasks on the remote robot side, such as establishing or maintaining a stable contact with objects. However, the coupling effect negatively influences the commanding of reference stiffness to the remote robot in both position and force tracking tasks for the coupled type of interface, compared to the decoupled type of interface, which is not affected.

In this paper, we propose a method for selective monitoring and management of human muscle fatigue in human-robot co-manipulation scenarios. The proposed approach uses a machine learning technique to learn the complex relationship between individual human muscle forces, arm configuration and arm endpoint force that are provided by a sophisticated offline musculoskeletal model. The estimated muscle forces are used in the fatigue model to estimate the individual muscle fatigue levels online. Two fatigue management protocols are proposed that enable the robot to handle and reduce the human fatigue by altering the configuration of task execution. The first protocol uses optimisation technique to find the optimal position for task execution, where the fatigue-related endurance time can be maximised. The second protocol divides the arm muscles into groups and then alters the direction of endpoint force so that the fatigued muscle group can relax and the relaxed muscle group becomes active. The proposed method has a potential to enable the robot to facilitate safer and more ergonomic working conditions for the human coworker. The main advantage of this approach is that it can operate online, and that all the measurements can be performed by the robot sensory system, which can significantly increase the applicability in real world scenarios. To validate the proposed method, we performed multiple experiments with two collaborative tasks (polishing and drilling) under different conditions.

The science and technology of wearable robots are steadily advancing, and the use of such robots in our everyday life appears to be within reach. Nevertheless, widespread adoption of wearable robots should not be taken for granted, especially since many recent attempts to bring them to real-life applications resulted in mixed outcomes. The aim of this article is to address the current challenges that are limiting the application and wider adoption of wearable robots that are typically worn over the human body. We categorized the challenges into mechanical layout, actuation, sensing, body interface, control, human–robot interfacing and coadaptation, and benchmarking. For each category, we discuss specific challenges and the rationale for why solving them is important, followed by an overview of relevant recent works. We conclude with an opinion that summarizes possible solutions that could contribute to the wider adoption of wearable robots.

This letter proposes a novel human–robot collaboration (HRC) control approach to alert and reduce the static joint torque overloading of a human partner while executing shared tasks with a robot. Using a preidentified statically equivalent serial chain model, variations of the centre-of-pressure and ground reaction force are calculated, and the overloading joint torques are evaluated in real time to initially alert the human about consequent injuries. An online optimization technique is implemented to adjust the robot trajectories facilitating the human to achieve more ergonomic body poses throughout the HRC task. The optimized human configurations are calculated by taking into account the human stability, the robot, and the human workspaces, and the task constraints, and illustrated to the human in real time. The experimental evaluation of the proposed technique is achieved in a human–robot load sharing task as a representative example of coassembly or transportation scenarios in industrial settings.

In this paper, we propose a novel method for human---robot collaboration, where the robot physical behaviour is adapted online to the human motor fatigue. The robot starts as a follower and imitates the human. As the collaborative task is performed under the human lead, the robot gradually learns the parameters and trajectories related to the task execution. In the meantime, the robot monitors the human fatigue during the task production. When a predefined level of fatigue is indicated, the robot uses the learnt skill to take over physically demanding aspects of the task and lets the human recover some of the strength. The human remains present to perform aspects of collaborative task that the robot cannot fully take over and maintains the overall supervision. The robot adaptation system is based on the Dynamical Movement Primitives, Locally Weighted Regression and Adaptive Frequency Oscillators. The estimation of the human motor fatigue is carried out using a proposed online model, which is based on the human muscle activity measured by the electromyography. We demonstrate the proposed approach with experiments on real-world co-manipulation tasks: material sawing and surface polishing.

We propose a novel human-in-the-loop approach for teaching robots how to solve assembly tasks in unpredictable and unstructured environments. In the proposed method the human sensorimotor system is integrated into the robot control loop though a teleoperation setup. The approach combines a 3-DoF end-effector force feedback with an interface for modulation of the robot end-effector stiffness. When operating in unpredictable and unstructured environments, modulation of limb impedance is essential in terms of successful task execution, stability and safety. We developed a novel hand-held stiffness control interface that is controlled by the motion of the human finger. A teaching approach was then used to achieve autonomous robot operation. In the experiments, we analysed and solved two part-assembly tasks: sliding a bolt fitting inside a groove and driving a self-tapping screw into a material of unknown properties. We experimentally compared the proposed method to complementary robot learning methods and analysed the potential benefits of direct stiffness modulation in the force-feedback teleoperation.

In this letter, we propose a novel method that enables the robot to autonomously devise an appropriate control strategy from human demonstrations without a prior knowledge of the demonstrated task. The method is primarily based on observing the patterns and consistency in the observed dataset. This is obtained through a demonstration setting that uses a motion capture system, a force sensor, and muscle activity measurements. The variables (position and force) in the collected dataset are then segmented and analysed for each axis of the observed task frame separately. While checking several conditions based on the consistency, value range, and magnitude of repeated observations, the appropriate controller (i.e., position or force) is delegated to each axis of the task frame. In the final stage, the method also checks for a correlation between variables and muscle activity patterns to determine the desired stiffness behaviour. The robot then uses the derived control strategies in autonomous operation through a hybrid force/impedance controller. To validate the proposed method, we performed experiments on real-life tasks involving physical interaction with the environment, where we considered surface wiping, material sawing, and drilling.

In this paper, we introduce and explore a concept called coupling effect, which pertains to the influence of force feedback on the commanded stiffness that is voluntarily controlled by the operator through the stiffness interface during bilateral tele-impedance. The degree of coupling effect depends on the type of interface used to control the impedance of the remote robot. In case of muscle activity based stiffness command interfaces, the force feedback can invoke involuntary changes in the commanded stiffness due to human reflexes. These involuntary changes can be either beneficial (e.g., during position tracking) or detrimental (e.g., during force tracking) to the task performance on the remote robot side. To investigate the coupling effect in different types of stiffness command interfaces (i.e., coupled and decoupled), we conduct an experimental study in which participants are asked to perform position and force tracking tasks. The results show that in both position and force tracking tasks a lower tracking error of the reference stiffness is obtained with a decoupled interface (p

In this paper we propose a method for addressing the human operator's ergonomics during bilateral teleoperation. The method is based on human operator's musculoskeletal and fatigue models, whose input is the force produced by the operator through the haptic interface (master) in order to control the slave robot in the remote environment. The system then estimates the fatigue-based endurance time for producing the desired task force in different human arm configurations within the selected workspace. We perform an online optimisation process to find the optimal configuration that has the longest endurance time, which signifies that the operator can perform the task in that configuration for a longer period. Next, a trajectory is generated on the master robot in order to guide the human arm into the optimal configuration. The teleoperation is temporarily suspended by decoupling the master from the slave robot when the master robot is being reconfigured. The teleoperation will be resumed and the slave robot will be teleoperated again from where it stopped after the master robot guided the operator's arm to the optimised configuration. The main advantage of the proposed method is that the human operator can perform the task with less muscle fatigue, which increases the endurance time. To validate our approach, we performed proof-of-concept experiments on a teleoperation system composed of two Panda robots by Franka Emika, where one was serving as master and the other as slave. The main experimental task was for the human operator to produce a reference force with the slave robot on an object in a remote environment. A supplementary task involved putting an object of unknown mass on the slave robot end-effector.

In this work, we present a novel control approach to human-robot collaboration that takes into account ergonomic aspects of the human co-worker during power tool operations. The method is primarily based on estimating and reducing the overloading torques in the human joints that are induced by the manipulated external load. The human overloading joint torques are estimated and monitored using a whole-body dynamic state model. The appropriate robot motion that brings the human into the suitable ergonomic working configuration is obtained by an optimisation method that minimises the overloading joint torques. The proposed optimisation process includes several constraints, such as the human arm muscular manipulability and safety of the collaborative task, to achieve a task-relevant optimised configuration. We validated the proposed method by a user study that involved a human-robot collaboration task, where the subjects operated a polishing machine on a part that was brought to them by the collaborative robot. A statistical analysis of ten subjects as an experimental evaluation of the proposed control framework is provided to demonstrate the potential of the proposed control framework in enabling ergonomic and task-optimised human-robot collaboration.

This paper introduces a novel control framework for an arm exoskeleton that takes into account force of the human arm. In contrast to the conventional exoskeleton controllers where the assistance is provided without considering the human arm biomechanical force manipulability properties, we propose a control approach based on the arm muscular manipulability. The proposed control framework essentially reshapes the anisotropic force manipulability into the endpoint force manipulability that is invariant with respect to the direction in the entire workspace of the arm. This allows users of the exoskeleton to perform tasks effectively in the whole range of the workspace, even in areas that are normally unsuitable due to the low force manipulability of the human arm. We evaluated the proposed control framework with real robot experiments where subjects wearing an arm exoskeleton were asked to move a weight between several locations. The results show that the proposed control framework does not affect the normal movement behavior of the users while effectively reduces user effort in the area of low manipulability. Particularly, the proposed approach augments the human arm force manipulability to execute tasks equally well in the entire workspace of the arm.

Industry 4.0 envisions the integration of flexible and quickly reconfigurable robotic systems in assembly lines. This has led to the development of light-weight and adaptive collaborative robots with limited power and payload constraints. Hence, repetitive tasks or those that demand high forces may exceed such limits, resulting in robot damage and lost productivity. To address this problem, we propose a novel framework to prolong the lifetime of collaborative robots while guaranteeing the desired level of productivity. To address this, we propose a method that minimises a function of robot fatigue, an index able to model the robot motor usage at the joint level. This index features the desired external force, task duration, hardware parameters, and fatigue history. Moreover, future tasks are considered through fatigue thresholds imposed on specific joints, computed according to the robot safety requirements. We proved the effectiveness of the approach by comparing its results in terms of fatigue and torque with the well-known minimum effort approach. The results showed that our method ensures that the fatigue thresholds are not exceeded.

Collaborative robots are often designed with limited power and force capacity, with the aim to provide affordable solutions and ensure human safety in case of accidental collisions and impacts. If a task requires a power beyond this capacity, or is performed repeatedly over long periods, such limits may be exceeded, which can cause inevitable robot damage and contribute to the lost productivity. In such cases, where hardware solutions and improvements are not applicable, effective software frameworks can prolong robot productivity and lifetime. To this end, in this paper we propose a novel technique for the monitoring and management of robot fatigue in repetitive or high-effort task execution scenarios. The robot fatigue is estimated by the measured temperature of motors in the joints. The proposed fatigue management system is composed of two-stage reaction process that is triggered by different levels of the estimated fatigue. The first stage exploits the kinematic redundancy of robot structure in attempt to minimise the load in the specific joints that under fatigue by reconfiguration in the joint space through the null space of the Cartesian task production. If the first stage is not successful in reducing the fatigue, the second stage is activated that gradually reduces the forces of hybrid controller. At that point, the human co-worker can temporarily take over the task execution until the robot will be recovered from the excessive fatigue. To validate the proposed approach we conducted experiments on KUKA Lightweight Robot performing two interaction tasks: autonomous surface wiping and collaborative human-robot surface polishing.

In this paper, we propose a novel method for selective management of muscle fatigue in human-robot co-manipulation. The proposed framework enables the detection of excessive fatigue levels of an individual muscle group while executing a certain task, and provides anticipatory robotic responses to distribute the effort among less-fatigued muscles of human arm. Our approach uses a machine learning technique to enable online predictions of muscle forces in different arm configurations and endpoint interaction forces. The estimated muscle forces are then used for the model-based estimation of muscle fatigue levels. Through optimisation, the fatigue management system can alter the task execution in a way that specific fatigued muscles are offloaded, while at the same time enables the production of task force using muscles with lower levels of fatigue. The main advantage of the proposed method is that it can operate online, and that all the measurements are performed by the robot sensory system, which can significantly increase the applicability in real-world scenarios. To validate the proposed method, we performed proof-of-concept experiments where the task of the human operator was to use a tool to polish an object that was manipulated by the robot.

In this paper, we propose a novel method for the control of human-robot eo-manipulation that takes into aeeount the ergonomie requirements for the human eo-worker. The robot uses a whole-body dynamic model of the human to optimise for the position of the eo-manipulation task in the workspace. In this configuration, the overloading joint torques, i.e. the effects of an external load in human body joints, are minimised. In addition, the optimisation process ineludes several constraints, sueh as human arm manipulability properties, to ensure that the human has a good manipulation eapaeity in the given eonfiguration. The main advantage of this approaeh is that the robot ean potentially help to reduee the work-related strain and increase the productivity of the human eo-worker. In addition, the on-line estimation of the overloading joint torques does not require an external foree plate or sensor insoles. We validated the proposed method with experiments in two eo-manipulation tasks: human using a device to polish an object that is delivered by the robot and a human-robot object handover. The results show that the proposed method is successful in achieving the ergonomie conditions for the human during the eollaboration.

In this paper we study the concept of robots learning from collaboration with skilled robots. The advantage of this concept is that the human involvement is reduced, while the skill can be propagated faster among the robots performing similar collaborative tasks or the ones being executed in hostile environments. The expert robot initially obtains the skill through the observation of, and physical collaboration with the human. We present a novel approach to how a novice robot can learn the specifics of the co-manipulation task from the physical interaction with an expert robot. The method consists of a multi-stage learning process that can gradually learn the appropriate motion and impedance behaviour under given task conditions. The trajectories are encoded with Dynamical Movement Primitives and learnt by Locally Weighted Regression, while their phase is estimated by adaptive oscillators. The learnt trajectories are replicated by a hybrid force/impedance controller. To validate the proposed approach we performed experiments on two robots learning and executing a challenging co-manipulation task.

The paper presents a novel control method for the arm exoskeletons that takes into account the muscular force manipulability of the human arm. In contrast to classical controllers that provide assistance without considering the biomechanical properties of the human arm, we propose a control method that takes into account the configuration of the arm and the direction of the motion to effectively compensate the anisotropic property of the muscular manipulability of the human arm. Consequently, the proposed control method effectively maintains a spherical endpoint manipulability in the entire workspace of the arm. As a result, the proposed method allows the human using the exoskeleton to efficiently perform tasks in arm configurations that are normally unsuitable due to the low manipulability. We evaluated the proposed approach by a preliminary experimental study where a subject wearing a 2 DOF arm-exoskeleton had to move a 4 kg weight between several locations. The results of our study demonstrate that the proposed approach effectively augments the ability of human motor control to perform tasks equally well in the whole arm workspace that include configurations with low intrinsic manipulability.

We present a novel approach where a human demonstrator can intuitively teach robot full-body skills. The aim of this approach is to exploit human sensorimotor ability to learn how to operate a huma...

We propose a novel method that arbitrates the control between the human and the robot actors in a teaching-by-demonstration setting to form synergy between the two and facilitate effective skill synthesis on the robot. We employed the human-in-the-loop teaching paradigm to teleoperate and demonstrate a complex task execution to the robot in real-time. As the human guides the robot to perform the task, the robot obtains the skill online during the demonstration. To encode the robotic skill we employed Locally Weighted Regression that fits local models to specific state region of the task based on the human demonstration. If the robot is in the state region where no local models exist, the control over the robotic mechanism is given to the human to perform the teaching. When local models are gradually obtained in that region, the control is given to the robot so that the human can examine its performance already during the demonstration stage, and take actions accordingly. This enables a co-adaptation between the agents and contributes to a faster and more efficient teaching. As a proof-of-concept, we realised the proposed robot teaching system on a haptic robot with the task of generation of a desired vertical force on a horizontal plane with unknown stiffness properties. European Commission

This paper presents a novel approach for human-robot cooperation in tasks with dynamic uncertainties. The essential element of the proposed method is a multi-modal interface that provides the robot with the feedback about the human motor behaviour in real-time. The human muscle activity measurements and the arm force manipulability properties encode the information about the motion and impedance, and the intended configuration of the task frame, respectively. Through this human-in-the-loop framework, the developed hybrid controller of the robot can adapt its actions to provide the desired motion and impedance regulation in different phases of the cooperative task. We experimentally evaluate the proposed approach in a two-person sawing task that requires an appropriate complementary behaviour from the two agents.

We propose an approach that combines human demonstrated posture-control skill defined by the motion in the lower limb joints with an inverse kinematics solution of an arbitrary hand motion. The posture-control skill for humanoid robot was obtained through the human-in-the-loop teaching approach. The collected data during the teaching phase was used to approximate functional relation between the state of the robot's centre-of-pressure and the appropriate motion in the joints of the leg. The motion of the robot's hand was prescribed using the inverse kinematic solution. The posture-control skill and the motion of the hand were combined together using the task priority resolution strategy. We tested our approach with an experiment where the humanoid robot had to spin a spindle device with the hand and simultaneously maintain balance in the presence of external perturbations. In addition, we propose an upgrade that allows switching of control over the leg joints between arm task and posture-control task.

Due to copyright restrictions, the access to the full text of this article is only available via subscription. We propose an approach to efficiently teach robots how to perform dynamic anipulation tasks in cooperation with a human partner. The approach utilises human sensorimotor learning ability where the human tutor controls the robot through a multi-modal interface to make it perform the desired task. During the tutoring, the robot simultaneously learns the action policy of the tutor and through time gains full autonomy. We demonstrate our approach by an experiment where we taught a robot how to perform a wood sawing task with a human partner using a two-person crosscut saw. The challenge of this experiment is that it requires precise coordination of the robot’s motion and complianceaccording to the partner’s actions. To transfer the sawing skill from the tutor to the robot we used Locally Weighted Regression for trajectory generalisation, and adaptive oscillators for adaptation of the robot to the partner’s motion. Slovenian Research Agency ; Slovenian Ministry of Higher Education, Science and Technology.

In this paper we propose a system capable of teaching humanoid robots new skills in real-time. The system aims to simplify the robot control and to provide a natural and intuitive interaction between the human and the robot. The key element of the system is exploitation of the human sensorimotor learning ability where a human demonstrator learns how to operate a robot in the same fashion as humans adapt to various everyday tasks. Another key aspect of the proposed system is that the robot learns the task simultaneously while the human is operating the robot. This enables the control of the robot to be gradually transferred from the human to the robot during the demonstration. The control is transferred based on the accuracy of the imitated task. We demonstrated our approach using an experiment where a human demonstrator taught a humanoid robot how to maintain the postural stability in the presence of the perturbations. To provide the appropriate feedback information of the robot's postural stability to the human sensorimotor system, we utilized a custom-built haptic interface. To absorb the demonstrated skill by the robot, we used Locally Weighted Projection Regression machine learning method. A novel approach was implemented to gradually transfer the control responsibility from the human to the incrementally built autonomous robot controller.

In this paper we propose an exoskeleton control method for adaptive learning of assistive joint torque profiles in periodic tasks. We use human muscle activity as feedback to adapt the assistive joint torque behaviour in a way that the muscle activity is minimised. The user can then relax while the exoskeleton takes over the task execution. If the task is altered and the existing assistive behaviour becomes inadequate, the exoskeleton gradually adapts to the new task execution so that the increased muscle activity caused by the new desired task can be reduced. The advantage of the proposed method is that it does not require biomechanical or dynamical models. Our proposed learning system uses Dynamical Movement Primitives (DMPs) as a trajectory generator and parameters of DMPs are modulated using Locally Weighted Regression. Then, the learning system is combined with adaptive oscillators that determine the phase and frequency of motion according to measured Electromyography (EMG) signals. We tested the method with real robot experiments where subjects wearing an elbow exoskeleton had to move an object of an unknown mass according to a predefined reference motion. We further evaluated the proposed approach on a whole-arm exoskeleton to show that it is able to adaptively derive assistive torques even for multiple-joint motion.

In this paper, we propose and evaluate methods for the local navigation using only visual perception for the skiing robot. Our skiing robot, capable of skiing using the carving technique, has no direct control on the velocity of skiing as it cannot break or accelerate, therefore well known navigation methods for nonholonomic mobile robots cannot be directly applied. We consider the following methods: an intuitive method of aiming at the closest gates, a human obstacle avoidance movement model, neural networks learning from a set of human demonstrations, and a global method that uses a predefined, spline-encoded path. The navigation performance of the robot on unknown ski courses is evaluated using two criteria: successful completion of the course and the time required to complete the course. Simulation results show the applicability and drawbacks of presented methods. Finally, the method using the neural networks was applied on a real-world skiing robot and we tested navigating a slalom course on both roller blades and skies.

In this paper, we propose a method that allows the robot to adapt its physical behaviour to the human fatigue in human-robot co-manipulation tasks. The robot initially imitates the human to perform the collaborative task in a leader-follower setting, using a feedback about the human motor behaviour. Simultaneously, the robot obtains the skill in online manner. When a predetermined level of human fatigue is detected, the robot uses the learnt skill to take over the physically demanding aspect of the task and contributes to a significant reduction of the human effort. The human, on the other hand, controls and supervises the high-level interaction behaviour and performs the aspects that require the contribution of both agents in such a dynamic co-manipulation setup. The robot adaptation system is based on the Dynamical Movement Primitives, Locally Weighted Regression and Adaptive Frequency Oscillators. The estimation of the human motor fatigue is carried out using a proposed online model, which is based on the human muscle activity measured by the electromyography. We demonstrate the proposed method with experiments on a real-world co-manipulation task with environmental constraints and dynamic uncertainties.