Search

This paper presents an online dynamic hand gesture recognition system with an RGB-D camera, which can automatically recognize hand gestures against complicated background. For background subtraction, we use a model-based method to perform human detection and segmentation in the depth map. Since a robust hand tracking approach is crucial for the performance of hand gesture recognition, our system uses both color information and depth information in the process of hand tracking. To extract spatio-temporal hand gesture sequences in the trajectory, a reliable gesture spotting scheme with detection on change of static postures is proposed. Then discrete HMMs with Left-Right Banded (LRB) topology are utilized to model and classify gestures based on multi-feature representation and quantization of the hand gesture sequences. Experimental evaluations on two self-built databases of dynamic hand gestures show the effectiveness of the proposed system. Furthermore, we develop a human-robot interactive system, and the performance of this system is demonstrated through interactive experiments in the dynamic environment.

Space robots are in huge demand due to the rapid growth of their service targets, i.e., spacecraft. There are generally large flexible components on spacecraft, such as antenna reflectors and solar paddles. Due to the vibratility of their structure, it is challenging for a space robot with flexible appendages (the base is then called flexible-base) to capture and repair the large flexible spacecraft. After capturing, the two spacecraft with flexible appendages are connected by a space manipulator, and a compounded system is formed. In this paper, we developed a dynamic model and a closed-loop simulation system, to provide a means to verify path planning and control algorithms. Initially, the dynamic characteristics of different capturing stages (preimpact and post-impact) were analyzed. The topologies of a flexible-base space robot and the compounded system were described based on incidence and channel matrices. Secondly, the recursive dynamics was formulated and resolved by an effective numerical method. The modeling was verified by Adams’ model. Thirdly, we implemented a dynamics calculation block in Matlab/Simulink environment using the S-function package for the C program, and developed a closed-loop simulation system, which was composed of the Planning and Controller, the Multibody Dynamic, and the 3D Display modules. Finally, based on the simulation system, two typical missions—target berthing and on-orbital manipulation of the target along a circle, were simulated and evaluated. Dynamics analysis results presented some useful rules for the path planning and control to suppress the vibration of the flexible structure.

Space robotic systems are expected to play an increasingly important role in the future on-orbit service. The applications include repairing, refueling or de-orbiting of a satellite, or removal of the space debris. The dynamical performances of space robotic system result from the multi-physics interactions between mechanical, electrical, electronic, control, etc. In this paper, we developed a unified multi-domain modelling and simulation system. The system is composed of the following modules: the path planner, joint controllers, motor and its driver, gearing mechanism of the space manipulators, the Guidance, Navigation, and Control (GNC) system, the actuators of the base, and the orbital environment, orbital dynamic and the multi-body dynamic of the whole system, etc. Based on this system, the operation during different stages, including far range rendezvous, close range rendezvous (is usually divided into two sub-phases: closing and final approach) and target capturing can be studied from the view of multi-physics domains. The key algorithms, such as pose (position and attitude) measurement, GNC of the base, path planning and control of the space manipulator, and so on, can be validated using the system. As examples, the capturing processes of a moving target under free-floating and attitude-controlled modes are simulated and the simulation results are given.

For the aging population, surveillance in household environments has become more and more important. In this paper, we present a household robot that can detect abnormal events by utilizing video and audio information. In our approach, moving targets can be detected by the robot using a passive acoustic location device. The robot then tracks the targets by employing a particle filter algorithm. To adapt to different lighting conditions, the target model is updated regularly based on an update mechanism. To ensure robust tracking, the robot detects abnormal human behavior by tracking the upper body of a person. For audio surveillance, Mel frequency cepstral coefficients (MFCC) is used to extract features from audio information. Those features are input to a support vector machine classifier for analysis. Experimental results show that the robot can detect abnormal behavior such as "falling down" and "running". Also, a 88.17% accuracy rate is achieved in the detection of abnormal audio information like "crying", "groan", and "gun shooting". To lower the false alarms by abnormal sound detection system, the passive acoustic location device directs the robot to the scene where abnormal events occur and the robot can employ its camera to further confirm the occurrence of the events. At last, the robot will send the image captured by the robot to the mobile phone of master.

In this paper, we discuss the problem of how human control strategy can be represented as a parametric model using a support vector machine (SVM) and how an SVM-based controller can be used in controlling dynamically stable systems. The SVM approach has been implemented in the balance control of Gyrover, which is a dynamically stable, statically unstable, single-wheel mobile robot. The experimental results that compare SVM with general artificial neural network approaches clearly demonstrate the superiority of the SVM approach with regard to human control strategy learning.

Space robotic systems are expected to play an increasingly important role in the future. The robotic on-orbital service, whose key is the capturing technology, becomes research hot in recent years. In this paper, the authors propose an autonomous path planning method for target capturing. The task is described in Cartesian space and it can drive the manipulator to approach the target along the closest path. Firstly, the target feature is extracted based on the measured information via the hand-eye camera, and the target pose (position and orientation) and velocities (linear velocity and angular velocity) are estimated using Kalman filtering technology. Then, a numerically feasible approach is presented to plan the manipulator motion and avoid the dynamic singularities, which are transformed into real-time kinematic singularities avoiding problem. Thirdly, the potential disturbance on the base due to the manipulator's motion is estimated, and the joint rates are autonomously adjusted to reduce the disturbance if it is beyond the allowed bound. At last, a ground experiment system is set up based on the concept of dynamic emulation and kinematic equivalence. With the experiment system, the autonomous target capturing experiments are conducted. The experiment results validate the proposed algorithm.

Technology for the capturing of space target is very important for on-orbit servicing. In order to assure the task is accomplished successfully, ground experimentations are required for the verification of the planning and control algorithms of space robotic system before it is launched. In this paper, an experiment concept is used, which is a hybrid approach, i.e. it combines the mathematical model with the physical model. The key issues of the concept are dynamic emulation and kinematic equivalence, in which the behaviors of the space robotic system are calculated by its dynamic equations. The motion of its end-effector and the space target is realized by two industrial robots. According to different observation spots, two modes of capturing process are emulated: one is observed from the inertial frame, the other is from the space base. Based on the concept proposed above, a ground experiment system is set up, which is composed of two industrial robots, a set of global visual system and five industrial computers. Using the system, algorithms of space robot of any geometry and mass properties can be tested. As an example, the autonomous trajectory planning algorithm is verified by the experiment of capturing a moving target. Moreover, a real-time 3D simulation system is developed to emulate the capturing process in 3D space. Numeric simulation and experiment results show that the ground system is effective in evaluating the planning and control algorithms of space robot.

Global manufacturing chains have become common in the 21st century. To perform remote monitoring and diagnosis in such chains, real-time data compression has become a core factor in the efficient and effective exchange of information via computer networks. This paper presents a new technique for compressing data using a kernel-based method. Overcoming the drawbacks of support vector techniques – that is, fast decompression but slow compression – the new method exhibits high speed in both phases. In addition, the new method can also be applied for pattern classification. Based on strain signal, example tests derived from sheet metal stamping operations, the new method is shown to be very effective. The proposed technology has enormous potential in the application of advanced manufacturing processes monitoring and control through Internet or Intranet.

Sensor signals produced in industrial manufacturing processes contain valuable information about the condition of operations. However, extracting the appropriate feature for effective fault diagnosis is difficult. Moreover, the adaptability and flexibility of current fault diagnosis systems are often found wanting in real-world applications. Unfortunately, it is essential to rebuild most fault diagnosis systems when new fault types emerge. This paper presents an intelligent fault diagnosis system based on a hidden Markov model. Introducing the concepts of time marginal energy and frequency marginal energy, the features of which can be acquired by the wavelet packet technique satisfy the requirements for fault diagnosis. By utilizing the best tree principle, this method not only extracts the feature automatically without a priori experience but also compresses the data; both of which ensure a system that is practical for real-time application. The new diagnosis system developed here is efficient and effective, as demonstrated by the model developed and applied to a real-time sheet metal stamping process. Based on tests conducted during two experiments (one based on simple blanking, the other on progressive operations) and related comparisons, the proposed method is substantially more effective than other approaches. In addition, the new method requires that only related models be created for new fault types, which results show are ideal for shop floor applications.

In this paper, we present a method for high-level control of robots whose low-level software is based on dynamically reconfigurable, reusable real-time software modules. Our approach is to use an embedded interpreter for a general-purpose programming language to direct the operation of the low-level modules toward meeting the task-level goals of the robot. To this end, we present RSK, a virtual-machine kernel implementing a Scheme interpreter capable of hard real-time operation, and employing a method of code execution we call “message-based evaluation” (MBE) . MBE is a novel combination of a traditional code execution model and a message-passing architecture, which simplifies the process of writing code for managing the robot‘s reconfigurable subsystem.

The shared control system is a modular real-time system designed to execute complex tasks through the intelligent coordination of task modules. A state machine is used to control task sequencing, and due to the automatic switching, the accuracy and reliability with which tasks are executed, is greatly improved. Tasks consist of sets of independent, modular, and reusable subtasks whose outputs are combined to create the control. This system has proved itself useful for rapid development of reliable high level multiple sensor based manipulation and control tasks. Additionally, we have integrated a neural network based visual servoing system, a semi-compliant Cartesian trajectory following heuristics, and a real-time graphical user interface in the system. The shared control system was implemented for the Self-Mobile Space Manipulator (SM 2) to handle a range of tasks associated with locomotion, manipulation, and material transportation on Space-Station Freedom.

TheSelf-Mobile Space Manipulator (SM2) is a 5-DOF, 1/3-scale, laboratory version of a robot designed to walk on the trusswork and other exterior surfaces of Space Station Freedom. It will be capable of routine tasks such as inspection, parts transportation and simple maintenance and repair. We have designed and built the robot and gravity compensation system to permit simulated zero-gravity experiments. The control ofSM2 is challenging because of significant structural flexibility, relatively high friction at the joints, positioning error amplified from joint errors due to the long reach, and high performance requirements for general 3-D locomotion. In this paper, we focus on the modeling of the robot system and the design of the control system based on the model. We address the kinematics and dynamic modeling of the 3-D motion ofSM2 and demonstrate the simulation and experimental modal analysis results. The robot dynamic characteristics vary significantly when the robot configuration changes. To consider this effect, we develop a control system that is composed of two basic parts, the model-based part and the servo part. The model-based loop can be updated based on the off-line dynamic model, and the servo control loop is updated by a gain schedule according to the off-line relationship between the closed-loop frequency and the modal frequency estimated from the off-line dynamic model. By taking dynamic variation into account in the controller, the control system is independent of the robot configuration, and the motion performance ofSM2 is greatly enhanced in implementation.

Many robotic applications require the direct contact of the end-effector with the environment. Passive compliance attached to the robot wrist, hand, or finger is desirable to produce smooth transitions between the free motion and contact, as well as to allow self-correction in order to accommodate geometric uncertainties in assembly and manufacturing. However, the use of passive compliance degenerates the positioning capability of the manipulator when the robot moves in free space. When the robot makes contact on workpiece, active adjustment of stiffness for various tasks in different directions is needed. We proposed to use passive compliance that is instrumented so that the system provides the necessary flexibility and also sensing to actively control the contact forces or to compensate the positioning error during motion and contact. In this article, the dynamic control of the manipulator with a compliant wrist is addressed. The measured deformation information of the instrumented compliant wrist is utilized in the feedback loop to increase the stiffness of the overall system in position control and to decrease the stiffness in force control. The dynamics model for both unconstrained and constrained cases is established. Applying nonlinear feedback control techniques, the dynamics of the manipulator-wrist system is linearized and decoupled, which allows the controller design to be carried out by using the linear system theory.