Search

Integration of a tactile display onto the end effector of a robot allows free-hand exploration of an encountered-type environment that provides both kinesthetic and cutaneous feedback. A novel tactile display approach, called Haptic Jamming, uses a combination of particle jamming and pneumatics to control the stiffness and shape of a surface. The tactile display mounts to the cable-driven platform of a kinesthetic system for medical simulation, called KineSys MedSim. The parallel structure of the robot provides a high strength-to-weight ratio for kinesthetic feedback in combination with a spatially aligned visual display. Its controller uses hand tracking to move the platform to the portion of the workspace the user is reaching toward. Data from a lump localization simulation demonstrates that the integrated system successfully tracks the user’s hand and reconfigures the tactile display according to the location of the end effector.

A method for segmenting the 3D shape of curved needles in solid tissue is described. An actuator attached to the needle outside the tissue vibrates at frequencies between 600 Hz and 6500 Hz, while 3D power Doppler ultrasound imaging is applied to detect the resulting motion of the needle shaft and surrounding tissue. The cross section of the vibrating needle is detected across the series of 2D images produced by a mechanical 3D ultrasound transducer, and the needle shape is reconstructed by fitting a 3D curve to the resulting points. The sensitivity of segmentation accuracy to tissue composition, vibration frequency, and Doppler pulse repetition frequency (PRF) was examined. Comparison with manual segmentation demonstrates that this method results in an average error of 1.09 mm in ex vivo tissue. This segmentation method may be useful in the future for providing feedback on curved needle shape for control of robotic needle steering systems.

During upper-limb prosthesis use, proprioception is not available so visual cues are used to identify the location of the artificial limb. We investigate the efficacy of a skin stretch device for artificially relaying proprioception during a spring discrimination task, with the goal of enabling the task to be achieved in the absence of vision. In this study, intact users perceive the location of a virtual prosthetic limb using each of four sensory conditions: Vision, Proprioception, Skin Stretch, and Skin Stretch with Vision. For the conditions with skin stretch, a haptic device stretches the forearm skin by an amount proportional to the angular rotation of a virtual prosthetic limb. Sensory condition was not found to significantly influence task performance, exploration methods, or perceived usefulness. We conclude that, in the absence of vision, artificial skin stretch could be used by prosthesis wearers to obtain position/motion information and identify the behavior of a spring.

Robot-assisted minimally invasive surgery (RMIS) holds great promise for improving the accuracy and dexterity of a surgeon while minimizing trauma to the patient. However, widespread clinical success with RMIS has been marginal and it is hypothesized by engineers and surgeons alike that the lack of haptic feedback presented to the surgeon is a limiting factor. The objective of our research is to acquire, display, and determine the utility of haptic information during RMIS. This overview paper examines the design, analysis, practicality, and effectiveness of various force estimation and display methods. In particular, we describe our experience in adding force feedback to an experimental version of the da Vinci surgical system, a commercially available teleoperated RMIS system.

This work develops a methodology for building haptic reality-based modeling systems by exploiting the complementary goals of haptic exploration and display. While the generally unstructured nature of haptic exploration makes it difficult to develop and control autonomous robotic fingers, simultaneous consideration of exploration and display provides a way to characterize the required sensing, control, finger geometry, and haptic modeling components. Both the exploring robot and haptic display must complement an appropriate virtual model, which can accurately recreate remote or hazardous environments. We present an example of haptic exploration, modeling and display for surface features explored using a three-degree-of-freedom spherical robotic fingertip with a tactile sensor.

Haptic virtual fixtures are software-generated force and position signals applied to human operators in order to improve the safety, accuracy, and speed of robot-assisted manipulation tasks. Virtual fixtures are effective and intuitive because they capitalize on both the accuracy of robotic systems and the intelligence of human operators. In this paper, we discuss the design, analysis, and implementation of two categories of virtual fixtures: guidance virtual fixtures, which assist the user in moving the manipulator along desired paths or surfaces in the workspace, and forbidden-region virtual fixtures, which prevent the manipulator from entering into forbidden regions of the workspace. Virtual fixtures are analyzed in the context of both cooperative manipulation and telemanipulation systems, considering issues related to stability, passivity, human modeling, and applications.

“The Publisher wishes to include a reference to Dr. Rajesh Kumar’s Ph.D. thesis, entitled.” An Augmented Steady Hand System for Precise Micromanipulation,” Johns Hopkins University, Baltimore MD (April 2001), which describes a framework for task level control on the Steady Hand Robot at JHU and which reports demonstrations of several representative tasks, including retinal cannulation, on dry lab and ex-vivo phantoms. The Publisher also refers to Figure 2 of the Article and wishes to reference the depiction of retinal cannulation in Dr. Kumar’s thesis, which reflects a task sequence that was performed for Dr. Kumar’s thesis and is depicted in the thesis as Figure 5.13.”

Robotic surgical systems such as Intuitive Surgical’s da Vinci system provide a rich source of motion and video data from surgical procedures. In principle, this data can be used to evaluate surgical skill, provide surgical training feedback, or document essential aspects of a procedure. If processed online, the data can be used to provide context-specific information or motion enhancements to the surgeon. However, in every case, the key step is to relate recorded motion data to a model of the procedure being performed. This paper examines our progress at developing techniques for “parsing” raw motion data from a surgical task into a labelled sequence of surgical gestures. Our current techniques have achieved > 90% fully automated recognition rates on 15 datasets.

We describe our current progress in developing Human-Machine Collaborative Systems (HMCSs) for microsurgical applications such as vitreo-retinal eye surgery. Three specific problems considered here are (1) developing of systems tools for describing and implementing an HMCS, (2) segmentation of complex tasks into logical components given sensor traces of a human performing the task, and (3) measuring HMCS performance. Our goal is to integrate these into a full microsurgical workstation with the ability to automatically “parse” traces of user execution into a task model which is then loaded into the execution environment, providing the user with assistance using online recognition of task state. The major contributions of our work to date include an XML task graph modeling framework and execution engine, an algorithm for real-time segmentation of user actions using continuous Hidden Markov Models, and validation techniques for analyzing the performance of HMCSs.

This paper details the development and preliminary testing of a system for 6-DOF human-machine cooperative motion using vision-based virtual fixtures for applications in retinal micro-surgery. The system makes use of a calibrated stereo imaging system to track surfaces in the environment, and simultaneously tracks a tool held by the JHU Steady-Hand Robot. As the robot is guided using force inputs from the user, a relative error between the estimated surface and the tool position is established. This error is used to generate an anisotropic stiffness matrix that in turn guides the user along the surface in both position and orientation. Preliminary results show the effectiveness of the system in guiding a user along the surface and performing different sub-tasks such as tool alignment and targeting within the resolution of the visual system.The accuracy of surface reconstruction and tool tracking obtained from stereo imaging was validated through comparison with measurements made by an infrared optical position tracking system.

Minimally Invasive Surgery (MIS) has enjoyed increasing attention and development over the last two decades. As MIS systems evolve, the surgeon is increasingly insulated from patient contact, creating a trade-off between surgi- cal sensory information and patient invasiveness. Incorporation of haptic feed- back into MIS systems promises to restore sensory information surrendered in favor of minimal invasiveness. We have developed a novel, biocompatible 2- DOF force-sensing sleeve that can be used modularly with a variety of 5mm laparoscopic instruments. The functional requirements for such a device are de- fined, and design strategies are explored. Our formal device design is outlined and device calibration is presented with derived calibration functions. Illustrative experimental force data from a porcine model is presented. This device can be used for intra-abdominal force recording and feedback in laparoscopic environ- ments; the implications and future potential for this technology are explored.

Surgical robots, including those with remote center of motion (RCM) mechanisms, have demonstrated utility in image-guided percutaneous needle placement procedures. However, widespread clinical application of these robots is hindered by not only complicated mechanical design but also the need for calibration and registration of the robot to the medical imager prior to each use. In response, we propose a Virtual RCM algorithm that requires only online tracking or registering the surgical tool to the imager, and a five degree-of-freedom (DOF) robot comprised of three prismatic DOF decoupled from two rotational DOF. The robot can be unencoded, uncalibrated, and does not require pre-operative registration. An incremental adaptive motion control cycle both guides the needle to the insertion point and orients it to align with the target. The robot executes RCM motion “virtually” without having a physically constrained fulcrum point. The proof-of-concept prototype system achieved 0.78 mm translation and 1.4 degrees rotational accuracy (within the tracker accuracy), within 17 iterative steps (0.5–1s).

We present the tip and friction forces acting on a needle during penetration into a canine prostate, independently measured by a 7-axis load cell newly developed for this purpose. This experimental apparatus clarifies the mechanics of needle penetration, potentially improving the development of surgical simulations. The behavior of both tip and friction forces can be used to determine the mechanical characteristics of the prostate tissue upon penetration, and the detection of the surface puncture, which appears in the friction force, makes it possible to estimate the true insertion depth of the needle in the tissue. The friction model caused by the clamping force on the needle can also be determined from the measured friction forces.

In this paper, we present a study that evaluates the effect of visual and haptic feedbacks and their relevance to human performance in a needle insertion task. A virtual needle insertion simulator with a four-layer tissue model (skin, fat, muscle, and bone) and haptic feedback is used for perceptual experiments. The task was to detect the puncture of a particular layer using haptic and visual cues provided by the simulation. The results show that the addition of force feedback reduces error in detection of transitions between tissue layers by at least 55%. The addition of real-time visual feedback (image overlay) improved user performance by at least 87% in scenarios without force feedback. Presentation of both force and visual feedback improved performance by at least 43% in comparison to scenarios without feedback. These results suggest that real-time image overlay provides greater improvement in performance than force feedback.

Despite many successes with teleoperated robotic surgical systems, some surgeons feel that the lack of haptic (force or tactile) feedback is detrimental in applications requiring fine suture manipulation. In this paper, we study the difference between applied suture forces in three knot tying exercises: hand ties, instrument ties (using needle drivers), and robot ties (using the da Vinci? Surgical System from Intuitive Surgical, Inc.). Both instrument and robot-assisted ties differ from hand ties in accuracy of applied force. However, only the robot ties differ from hand ties in repeatability of applied force. Furthermore, comparison between attendings and residents revealed statistically significant differences in the forces used during hand ties, although attendings and residents perform similarly when comparing instrument and robot ties to hand ties. These results indicate that resolved force feedback would improve robot-assisted performance during complex surgical tasks such as knot tying with fine suture.

The potato (Solanum tuberosum L.) is one of the world’s most valuable basic food crops. It is a tetraploid (2n = 48), highly heterozygous, and vegetatively propagated crop. Polyploidy and sterility are a serious constraint in potato breeding through sexual crossing. The outcome of crosses between tetraploid parental lines is often unpredictable. Genotypes superior for a sufficient number of desired characters are extremely rare in such progenies. Spontaneous somatic mutations occur in potatoes with quite a low frequency (Ross 1986).