Search

Humans have the amazing ability to learn the dynamics of the body and environment to develop motor skills. Traditional motor studies using arm reaching paradigms have viewed this ability as the process of ‘internal model adaptation’. However, the behaviors have not been fully explored in the case when reaches fail to attain the intended target. Here we examined human reaching under two force fields types; one that induces failures (i.e., target errors), and the other that does not. Our results show the presence of a distinct failure-driven adaptation process that enables quick task success after failures, and before completion of internal model adaptation, but that can result in persistent changes to the undisturbed trajectory. These behaviors can be explained by considering a hierarchical interaction between internal model adaptation and the failure-driven adaptation of reach direction. Our findings suggest that movement failure is negotiated using hierarchical motor adaptations by humans., Author summary How do we improve actions after a movement failure? Although negotiating movement failures is obviously crucial, previous motor-control studies have predominantly examined human movement adaptations in the absence of failures, and it remains unclear how failures affect subsequent movement adaptations. Here we examined this issue by developing a novel force field adaptation task where the hand movement during an arm reaching is perturbed by novel forces that induce a large target error, that is a failure. Our experimental observation and computational modeling show that, in addition to the popular ‘internal model learning’ process of motor adaptations, humans also utilize a ‘failure-negotiating’ process, that enables them to quickly improve movements in the presence of failure, even at the expense of increased arm trajectory deflections, which are subsequently reduced gradually with training after the achievement of the task success. Our results suggest that a hierarchical interaction between these two processes is a key for humans to negotiate movement failures.

When using an automated system, user trust in the automation is an important factor influencing performance. Prior studies have analyzed trust during supervisory control of automation, and how trust influences reliance: the behavioral correlate of trust. Here, we investigated how reliance on haptic assistance affects performance during shared control with an automated system. Subjects made reaches towards a hidden target using a visual cue and haptic cue (assistance from the automation). We sought to influence reliance by changing the variability of trial-by-trial random errors in the haptic assistance. Reliance was quantified in terms of the subject's position at the end of the reach relative to the two cues. Our results show that subjects aimed more towards the visual cue when the variability of the haptic cue errors increased, resembling cue weighting behavior. Similar behavior was observed both when subjects had explicit knowledge about the haptic cue error variability, as well as when they had only implicit knowledge (from experience). However, the group with explicit knowledge was able to more quickly adapt their reliance on the haptic assistance. The method we introduce here provides a quantitative way to study user reliance on the information provided by automated systems with shared control.

Shared control is an increasingly popular approach to facilitate control and communication between humans and intelligent machines. However, there is little consensus in guidelines for design and evaluation of shared control, or even in a definition of what constitutes shared control. This lack of consensus complicates cross fertilization of shared control research between different application domains. This paper provides a definition for shared control in context with previous definitions, and a set of general axioms for design and evaluation of shared control solutions. The utility of the definition and axioms are demonstrated by applying them to four application domains: automotive, robot-assisted surgery, brain–machine interfaces, and learning. Literature is discussed for each of these four domains in light of the proposed definition and axioms. Finally, to facilitate design choices for other applications, we propose a hierarchical framework for shared control that links the shared control literature with traded control, co-operative control, and other human–automation interaction methods. Future work should reveal the generalizability and utility of the proposed shared control framework in designing useful, safe, and comfortable interaction between humans and intelligent machines.

To effectively interpret and interact with the world, humans weight redundant estimates from different sensory cues to form one coherent, integrated estimate. Recent advancements in physical assistance systems, where guiding forces are computed by an intelligent agent, enable the presentation of augmented cues. It is unknown, however, if cue weighting can be extended to augmented cues. Previous research has shown that cue weighting is determined by the reliability (inversely related to uncertainty) of cues within a trial, yet augmented cues may also be affected by errors that vary over trials. In this study, we investigate whether people can learn to appropriately weight a haptic cue from an intelligent assistance system based on its error history. Subjects held a haptic device and reached to a hidden target using a visual (Gaussian distributed dots) and haptic (force channel) cue. The error of the augmented haptic cue varied from trial to trial based on a Gaussian distribution. Subjects learned to estimate the target location by weighting the visual and augmented haptic cues based on their perceptual uncertainty and experienced errors. With both cues available, subjects were able to find the target with an improved or equal performance compared to what was possible with one cue alone. Our results show that the brain can learn to reweight augmented cues from intelligent agents, akin to previous observations of the reweighting of naturally occurring cues. In addition, these results suggest that the weighting of a cue is not only affected by its within-trial reliability but also the history of errors.

When grasping and manipulating objects, people are able to efficiently modulate their grip force according to the experienced load force. Effective grip force control involves providing enough grip force to prevent the object from slipping, while avoiding excessive force to avoid damage and fatigue. During indirect object manipulation via teleoperation systems or in virtual environments, users often receive limited somatosensory feedback about objects with which they interact. This study examines the effects of force feedback, accuracy demands, and training on grip force control during object interaction in a virtual environment. The task required subjects to grasp and move a virtual object while tracking a target. When force feedback was not provided, subjects failed to couple grip and load force, a capability fundamental to direct object interaction. Subjects also exerted larger grip force without force feedback and when accuracy demands of the tracking task were high. In addition, the presence or absence of force feedback during training affected subsequent performance, even when the feedback condition was switched. Subjects' grip force control remained reminiscent of their employed grip during the initial training. These results motivate the use of force feedback during telemanipulation and highlight the effect of force feedback during training.

During robot-assisted minimally invasive surgery, teleoperation systems allow surgeons to perform operations at a distance via instruments inserted through small incisions in the body, thereby minimizing patient pain and recovery time. While the patient-side manipulator allows precise, dexterous gripping and manipulation by the surgical tools, current clinical systems provide the surgeon with limited haptic feedback about tool-environment interactions. This differs from direct grasp and manipulation of hand-held objects, during which we receive feedback that provides cues regarding object surface properties, slip, and load force. We use a custom research version of the da Vinci Surgical System to study the control of grip force during teleoperated manipulation of an elastic environment. We tested a placement task that involved stretching of a rubber band, with and without feedback of the patient-side load forces to the user. We hypothesized that there is greater coupling between the applied grip force and the patient-side load force when force feedback is provided, as is observed during direct manipulation of hand-held objects. With an experienced surgeon user, coupling between the applied grip force and the load force was greater with force feedback than without.

Cerebellar damage impairs the control of complex dynamics during reaching movements. It also impairs learning of predictable dynamic perturbations through an error-based process. Prior work suggests that there are distinct neural mechanisms involved in error-based learning that depend on the size of error experienced. This is based, in part, on the observation that people with cerebellar degeneration may have an intact ability to learn from small errors. Here we studied the relative effect of specific dynamic perturbations and error size on motor learning of a reaching movement in patients with cerebellar damage. We also studied generalization of learning within different coordinate systems (hand vs. joint space). Contrary to our expectation, we found that error size did not alter cerebellar patients' ability to learn the force field. Instead, the direction of the force field affected patients' ability to learn, regardless of whether the force perturbations were introduced gradually (small error) or abruptly (large error). Patients performed best in fields that helped them compensate for movement dynamics associated with reaching. However, they showed much more limited generalization patterns than control subjects, indicating that patients rely on a different learning mechanism. We suggest that patients typically use a compensatory strategy to counteract movement dynamics. They may learn to relax this compensatory strategy when the external perturbation is favorable to counteracting their movement dynamics, and improve reaching performance. Altogether, these findings show that dynamics affect learning in cerebellar patients more than error size.

Human-machine interfaces (HMIs) are widely used in biomedical applications, from teleoperated surgical systems to rehabilitation devices. This paper investigates a method of control that allows an HMI to transition from anisometric to isometric mode, shifting the control input from position to force as the user's movement is gradually reduced. Two different approaches for achieving this transition are discussed: one is based on the natural system dynamics, whereas the other involves selecting and controlling dynamics. The two approaches were implemented on a custom haptic device in a targeting task. Anisometric to isometric transitioning can potentially be used for training purposes, enabling transfer of what was learned in one mode to the other, as well as novel studies of the human sensorimotor system.

Haptic virtual fixtures have been shown to improve user performance and increase the safety of robot-assisted tasks, particularly for surgical applications. However, little research has studied virtual fixtures that provide moving force constraints based on motion of the environment, e.g., organ movement due to heartbeat or respiration. This work discusses design considerations of moving forbidden-region virtual fixtures and presents two methods of implementation: predicted-position and current-position virtual fixtures. Human subject experiments were performed to determine the effectiveness of moving virtual fixtures when interacting with an object in motion using a teleoperator. Results show that moving virtual fixtures can help improve user precision and decrease the amount of force applied.