Search

Humans have a remarkable ability to learn continuously from th e environment and inner experience. One of the grand goals of robots is to build an artificial "lifelong learning" agent that can shape a cultivated understanding of the world from the current scene and previous knowledge via an autonomous lifelong development. It is challenging for the robot learning process to retain earlier knowledge when robots encounter new tasks or information. Recent advances in computer vision and deep -learning methods have been impressive due to large-scale data sets, such as ImageNet and COCO. However, robotic vision poses unique new challenges for applying visual algorithms developed from these computer vision data sets because they implicitly assume a fixed set of categories and time -invariant task distributions.

Hand gesture recognition plays an essential role in the human-robot interaction (HRI) field. Most previous research only studies hand gesture recognition in a short distance, which cannot be applied for interaction with mobile robots like unmanned aerial vehicles (UAVs) at a longer and safer distance. Therefore, we investigate the challenging long-range hand gesture recognition problem for the interaction between humans and UAVs. To this end, we propose a novel attention-based single shot multibox detector (SSD) model that incorporates both spatial and channel attention for hand gesture recognition. We notably extend the recognition distance from 1 meter to 7 meters through the proposed model without sacrificing speed. Besides, we present a long-range hand gesture (LRHG) dataset collected by the USB camera mounted on mobile robots. The hand gestures are collected at discrete distance levels from 1 meter to 7 meters, where most of the hand gestures are small and at low resolution. Experiments with the self-built LRHG dataset show our methods reach the surprising performance-boosting over the state-of-the-art method like the SSD network on both short-range (1 meter) and long-range (up to 7 meters) hand gesture recognition tasks.

In this paper, a novel soft inflatable arm is proposed for telepresence robots. The new proposed structure of the arm is achieved by a very common and low-cost inflatable material and it is very light, weighing only about 50 g. However, it can realize agile movement by driving six tiny cables installed in the shoulder and elbow joints. The soft inflatable arm can work by pumping air at a very low pressure ( $7.32 \pm 3.45$ kPa) and allows direct and soft human contact without any external force sensors. This paper proposes joint compressed models, joint kinematic models, and kinematic models for the whole inflatable robot arm. These models can be easily applied to multijoint arms. In addition, a new redundancy resolution method is also developed for the inflatable arm, which makes it easier to control resolution and is less complex than other traditional approaches. Numerous experiments have been conducted, including performances of accuracy, repeatability, motion trajectories, human-safe interaction, and remote interaction. Results are satisfactory and validate the expected performance of the proposed robotic arm.

The challenges in transforming paintings into music is well-known, since the relationship between two kinds of art is unclear. Different composers write different music when the same painting is presented to them. In this paper, a cross-mordality model has been proposed for transforming images into multitrack music based on the framework of deep convolutional generative adversarial networks (DCGANs). The proposed model is trained on a classical music dataset and a dataset of impressionist paintings. The model can be applied to transfer impressionist paintings into classical music with two tracks. By using music evaluation methods, the harmonicity of the generated music can be confirmed. Our model is the first attempt of our knowledge at transforming paintings into music segments.

Redundant space manipulators, including Space Station Remote Manipulator System (SSRMS), Special Purpose Dexterous Manipulator (SPDM) and European Robotic Arm (ERA), have been playing important roles in the construction and maintenance of International Space Station (ISS). They all have 7 revolute joints arranged in similar configurations, and are referred to as SSRMS-type manipulators. When a joint is locked in an arbitrary position due to some failures, a 7R manipulator degrades to a 6R manipulator. Without a spherical wrist or three consecutive parallel joints, the inverse kinematics of the 6R manipulator is very complex. In this paper, we propose effective methods to resolve the inverse kinematics for different cases of any joint locked in an arbitrary position. Firstly, configuration characteristics of the SSRMS-type redundant manipulators are analyzed. Then, an existing of closed-form inverse kinematics is discussed for locking different joints. Secondly, D–H frames and corresponding D–H parameters of the new 6-DOF manipulator formed by locking a joint in an arbitrary position are re-constructed. A unified table is then created to describe the kinematics for all possible cases of single joint locking failure. Thirdly, completely analytical and semi-analytical methods are presented to solve the inverse kinematics equations, and the former is used for locking joint 1, 2, 6 or 7 while the latter for locking joint 3, 4 or 5. Finally, typical cases for single joint locking are studied. The results verify the proposed methods.

This paper presents the gait design and comparison study of a quadruped robot. The main contributions of this paper are: (i) Explore the modularized system design of an 8 degree-of-freedom quadruped robot with a power management system. (ii) Propose a gait design method based on drive function. (iii) Propose another gait design method based on foot trajectory. (iv) The simulation of the walk and trot gaits with above-mentioned methods are validated in Webots. (v) The walk and trot gait are successfully implemented in the real robot. Experimental results show that the stable locomotion can be achieved with improved elliptical foot trajectory.

Currently, the shock absorption system for outdoor AGV is mainly built with identical-spring solution, in which all the damping springs have the identical elastic coefficient. In this paper, we introduce a method to design the shock absorption system of the AGV using dual-spring solution. Springs of two different elastic coefficients are used, and an enhanced performance in obstacle-crossing capability can be achieved. The proposed dual-spring solution design method takes several critical outdoor working requirements as design constraints, and integrates damping system analysis to determine the optimal coefficients for the dual-spring system. To validate the effectiveness of this dual-spring design method, theoretical analysis is fully derived and a AGV chassis is physically built using dual-spring solution and is subjected to a series of validation tests.

In this paper, we propose a novel design of gaits for space manipulators so as to minimize the dynamic disturbance on a space station. Without modifying their joint configuration, we first propose the use of our previous developed gripping methodology. It combines the concepts of wheels motion in parallel grippers to reduce the swing motions in conventional gaits. For the proposed gaits, three major aspects are considered including 1) linear motion; 2) turning; and 3) exterior transition. Based on the conventional modeling approach for fixed base space manipulators, we extend the method to formulate the dynamic coupling during wheels motion. It is to visualize the manipulator as a prismatic joint with its gripping base moving with wheels motion. Also, we develop an experimental platform for verification. For disturbance analysis, various simulations have been conducted. Results have validated that the proposed gaits generate the lowest disturbance on a space station.

This paper presents an online dynamic hand gesture recognition system with an RGB-D camera, which can automatically recognize hand gestures against complicated background. For background subtraction, we use a model-based method to perform human detection and segmentation in the depth map. Since a robust hand tracking approach is crucial for the performance of hand gesture recognition, our system uses both color information and depth information in the process of hand tracking. To extract spatio-temporal hand gesture sequences in the trajectory, a reliable gesture spotting scheme with detection on change of static postures is proposed. Then discrete HMMs with Left-Right Banded (LRB) topology are utilized to model and classify gestures based on multi-feature representation and quantization of the hand gesture sequences. Experimental evaluations on two self-built databases of dynamic hand gestures show the effectiveness of the proposed system. Furthermore, we develop a human-robot interactive system, and the performance of this system is demonstrated through interactive experiments in the dynamic environment.

Ocean exploration has attracted enormous interest from human-kind for thousands of years. The sailing boat, propelled by wind is a historical invention. But human-involved marine exploration has high risk and low efficiency. This paper presents a new autonomous surface vehicle (ASV) based on retrofitting a trimaran sailing boat. The previous human-based maneuvering interfaces, i.e. sail and rudder control, are motorized by three electric actuators. The ASV acquires signals of wind speed and direction, GPS position, and rolling angle, determine the heading based on various wind direction. Experiments validates the effectiveness of the ASV.

Monocular cameras are not generally considered as a 3D measurement tool in the field of intelligent transportations. This belief limits the application of monocular cameras in many applications such as guiding robots or vehicles for autonomous driving. In this paper, we demonstrate that with the reference of a license plate, a monocular camera can be used as a tool to measure 3 dimensional parameters (width, height and length) of a passenger vehicle down to 10 cm accuracy. The proposed measurement method does not reply on any training process, but is designed as a fast and direct approach utilizing a series of fundamental computer vision techniques such as ROI extraction, edge/line detection, space coordinates transformations, etc. A series of validation tests were conducted and the results demonstrated the robustness of this measurement method and implies the possibility to broaden the application scope of monocular cameras in the field of autonomous driving and intelligent transportation systems.

Localization and navigation are fundamental issues to autonomous mobile robotics. In the case of the environmental map has been built, the traditional two-dimensional (2D) lidar localization and navigation system can't be matched to the initial position of the robot in dynamic environment and will be unreliable when kidnapping occurs. Moreover, it relies on high-cost lidar for high accuracy and long range. In view of this, the paper presents a low cost navigation system based on a low cost lidar and a cheap webcam. In this approach, 2D-codes are attached to the ceiling, to provide reference points to aid the indoor robot localization. The mobile robot is equipped with webcam pointing to the ceiling to identify 2D-codes. On the other hand, a low-cost 2D laser scanner is applied to build a map in unknown environment and detect obstacles. Adaptive Monte Carlo Localization (AMCL) is implements for lidar positioning, A* and Dynamic Window Approach (DWA) are applied in path planning based on a 2D grid map. The error analysis and experiments has validated the proposed method.

Space robots are in huge demand due to the rapid growth of their service targets, i.e., spacecraft. There are generally large flexible components on spacecraft, such as antenna reflectors and solar paddles. Due to the vibratility of their structure, it is challenging for a space robot with flexible appendages (the base is then called flexible-base) to capture and repair the large flexible spacecraft. After capturing, the two spacecraft with flexible appendages are connected by a space manipulator, and a compounded system is formed. In this paper, we developed a dynamic model and a closed-loop simulation system, to provide a means to verify path planning and control algorithms. Initially, the dynamic characteristics of different capturing stages (preimpact and post-impact) were analyzed. The topologies of a flexible-base space robot and the compounded system were described based on incidence and channel matrices. Secondly, the recursive dynamics was formulated and resolved by an effective numerical method. The modeling was verified by Adams’ model. Thirdly, we implemented a dynamics calculation block in Matlab/Simulink environment using the S-function package for the C program, and developed a closed-loop simulation system, which was composed of the Planning and Controller, the Multibody Dynamic, and the 3D Display modules. Finally, based on the simulation system, two typical missions—target berthing and on-orbital manipulation of the target along a circle, were simulated and evaluated. Dynamics analysis results presented some useful rules for the path planning and control to suppress the vibration of the flexible structure.

In this paper, we study the parameter setting for a set of intelligent multi-category classifiers in wall-following robot navigation. Based on the swarm optimization theory, a particle selecting approach is proposed to search for the optimal parameters, a key property of this set of multi-category classifiers. By utilizing the particle swarm search, it is able to obtain higher classification accuracy with significant savings on the training time compared to the conventional grid search. For wall-following robot navigation, the best accuracy (98.8%) is achieved by the particle swarm search with only 1/4 of the training time by the grid search. Through communicating the social information available in particle swarms in the training process, classification-based learning can achieve higher classification accuracy without prematurity. One of such learning classifiers has been implemented in SIAT mobile robot. Experimental results validate the proposed search scheme for optimal parameter settings.

This paper proposes a global path- and motion-planning algorithm that enables inchworm-like robots to navigate their way up tree branches. The intuitive climbing space representation method proposed here greatly simplifies the path-planning problem. The dynamic programming algorithm can be used to identify the optimal path leading to the target position in the target direction according to the constraints and requirements specified. The planned path can be applied in any tree-climbing robot that utilizes the nonenclosure gripping method. An efficient motion-planning algorithm for continuum inchworm-like robots is then developed to enable them to climb along the planned path with a high degree of accuracy. In comparison with the method proposed in our previous study, the method proposed herein significantly improves consistency between the planned path and the motions of the robot, and therefore makes it more practical to implement the motion-planning algorithm in trees of different shapes. The paper also describes hardware experiments in which the proposed planning algorithm is applied to enable inchworm-like robots to climb real trees, thus validating the proposed planning algorithm in practice. © 2012 Wiley Periodicals, Inc. © 2013 Wiley Periodicals, Inc.

Abnormal crowd behavior detection plays an important role in surveillance applications. We propose a camera parameter independent and perspective distortion invariant approach to detect two types of abnormal crowd behavior. The two typical abnormal activities are people gathering and running. Since people counting is necessary for detecting the abnormal crowd behavior, we present an potential energy-based model to estimate the number of people in public scenes. Building histograms on the X- and Y-axes, respectively, we can obtain probability distribution of the foreground object and then define crowd entropy. We define the Crowd Distribution Index by combining the people counting results with crowd entropy to represent the spatial distribution of crowd. We set a threshold on Crowd Distribution Index to detect people gathering. To detect people running, the kinetic energy is determined by computation of optical flow and Crowd Distribution Index. With a threshold, kinetic energy can be used to detect people running. To test the performance of our algorithm, videos of different scenes and different crowd densities are used in the experiments. Without camera calibration and training data, our method can robustly detect abnormal behaviors with low computation load.

Treebot is the first tree-climbing robot that is capable of climbing from a tree trunk to a branch. The robot employs several design principles adapted from arboreal animals, including claw gripping and inchworm locomotion, with a certain artificial optimization to achieve high maneuverability on irregular-shaped trees. Treebot is composed of a pair of tree grippers that permits Treebot to attach to a wide variety of trees with a wide range of gripping curvature, and a novel continuum maneuvering structure that provides high maneuverability and adaptability. In the robot actuation, only five actuators are necessary. Although Treebot weighs only 600 gr, it has a payload capability of 1.75 kg, which is nearly three times its own weight. This paper describes the design process and specifically addresses the robot locomotion and optimization of gripping force. Experimental results demonstrate the robot's ability to climb trees with high maneuverability. © 2012 Wiley Periodicals, Inc. © 2012 Wiley Periodicals, Inc.

Improvements in hybrid electric vehicle (HEV) fuel economy and emissions heavily depend on an efficient energy management strategy (EMS). However, the uncertainty of future driving conditions generally cannot be easily tackled in EMS design. Most existing EMSs act upon fixed parameters and cannot adapt to varying driving conditions. Therefore, they usually fail to fully explore the potential of these advanced vehicles. In this paper, a novel EMS design procedure based on neural dynamic programming (NDP) is proposed. The NDP is a generic online learning algorithm, which combines stochastic dynamic programming (SDP) and the temporal difference (TD) method. Instead of computing the utility function and optimal control actions through Bellman equations, the NDP algorithm uses two neural networks to approximate them. The weights of these neural networks are updated online by the TD method. It avoids the high computational cost that SDP suffers from and is suitable for real-time implementation. The main advantages of NDP EMS is that it does not rely on prior information related to future driving conditions, and can self-tune with a wide variance in operating conditions. The NDP EMS has been applied to “Qianghua-I”, a prototype of a parallel HEV, using a revolving drum test bench for verification. Experiment results illustrate the potential of the proposed EMS in terms of fuel economy and in keeping state of charge (SOC) deviations at a low level. The proposed research ensures the optimality of NDP EMS, as well as real-time applicability.

In this paper, a fast charging system (FCS) is designed for service robots. In this design, batteries are used to due to its energy density, whereas supercapacitors (SCs) are used to enhance the power density. DC-DC converter controls the FCS. The main advantage of the design of fast charging system is that, the SCs are charged with a large current fast and then discharge to the batteries with small current slowly. This feature is beneficial for batteries' long-life and store more energy for service robots which have been in great demand for multiple purposes, such as elderly care, child education etc. Some experimental and testing results are presented to validate the design of FCS for service robots.

In this paper, we propose an autonomous tree climbing strategy for a novel tree climbing robot that is named Treebot. The proposed algorithm aims to guide Treebot in climbing along an optimal path by the use of minimal sensing resources. Inspired by inchworms, the algorithm reconstructs the shape of a tree simply by the use of tactile sensors. It reveals how the realization of an environment can be achieved with limited tactile information. An efficient nonholonomic motion planning strategy is also proposed to make Treebot climb on an optimal path. This is accomplished by the prediction of the future shape of the tree. The study that is presented in this paper also includes the formulation of Treebot kinematics and an analysis of the workspace of Treebot on different shapes of a tree. Numerous experiments have been conducted to evaluate the proposed autonomous climbing algorithm and to unveil the ability of Treebot.

Space robotic systems are expected to play an increasingly important role in future space activities. Nevertheless, dynamics modeling and motion planning of a space robot are much more complex than those of a fixed-base robot, due to the dynamic coupling between the manipulator and its base. On the other hand, in order to assure the success of on-orbital missions, many experiments are required to verify the key algorithms on the ground before the space robot is launched. In this paper, the main research achievements on dynamics modeling, path planning, and ground verification are reviewed, and future studies are recommended. Firstly, we summarize the essential modeling concepts, and deduce the kinematics and dynamics equations of a space robot. Secondly, the main motion planning approaches are discussed. Then, different ground verification systems, including the air-bearing table, neutral buoyancy, airplane flying, free-falling motion, suspension system, and hybrid system, are introduced. Finally, the future research trends are forecasted.

Battery fast charging is one of the most significant and difficult techniques affecting the commercialization of electric vehicles (EVs). In this paper, we propose a fast charge framework based on model predictive control, with the aim of simultaneously reducing the charge duration, which represents the out-of-service time of vehicles, and the increase in temperature, which represents safety and energy efficiency during the charge process. The RC model is employed to predict the future State of Charge (SOC). A single mode lumped-parameter thermal model and a neural network trained by real experimental data are also applied to predict the future temperature in simulations and experiments respectively. A genetic algorithm is then applied to find the best charge sequence under a specified fitness function, which consists of two objectives: minimizing the charging duration and minimizing the increase in temperature. Both simulation and experiment demonstrate that the Pareto front of the proposed method dominates that of the most popular constant current constant voltage (CCCV) charge method.

Early automatic detection of cardiovascular diseases is of great importance to provide timely treatment and reduce fatality rate. Although many efforts have been devoted to detecting various arrhythmias, classification of other common cardiovascular diseases still lacks comprehensive and intensive studies. This work aims at developing an automatic diagnosis system for myocardial infarction, valvular heart disease, cardiomyopathy, hypertrophy, and bundle branch block, based on the clinic recordings provided by PTB Database. The proposed diagnosis system consists of the components as baseline wander reduction, beat segmentation, feature extraction, feature reduction and classification. The selected features are the location, amplitude and width of each wave, exactly the parameters of ECG dynamical model. We also propose a mean shift algorithm based method to extract these features. To demonstrate the availability and efficacy of the proposed system, we use a total of 13,564 beats to conduct a large scale experiment, where only 25% beats are utilized to train the eigenvectors of generalized discriminant analysis in the feature reduction phase and 25% beats are applied to train the support vector machine in the classification phase. The average sensitivity, specificity and positive predicitivity for the test set, containing 75% beats, are respectively 96.06%, 99.32% and 97.29%.

This paper presents an omnidirectional steer-by-wire system for a four-wheel independent steering vehicle. The system consists of an extended steering interface and a behavior-based steering controller. The extended steering interface provides a novel manipulation way to vehicle driver. The driver can control the vehicle in traditional way or omnidirectionally without any mode switching operation. The reservation of the traditional driving way makes the driver adapt the novel steering interface easily. The steering interface also has a force-feedback function to synchronize the extended steering interface and the orientations of wheels so as to improve vehicle handling. The behavior-based steering controller is designed for controlling the orientations of wheels. It acts as virtual linkage among each wheel to minimize wheel slip resulted by the misalignment of wheels. As there is no trajectory planning needed in the control algorithm, the proposed controller is especially suitable for a nonautonomous vehicle, the driving path of which cannot be predetermined.

Denoising of electrocardiogram (ECG) is the fundamental technique for manual or automatic ECG diagnosis. Model-based denoising has attracted initial studies since the ECG dynamical model was established in 2003 and been demonstrated to outperform most model-less denoising methods. The focus of this paper is robust denoising of abnormal ECG signals, which do not satisfy the assumption in previous model-based studies that morphological or physiological variations are small from one beat to another. A mean shift based initializer is proposed to provide a much more robust estimation of initial model parameters for each heart beat. Together with physiological knowledge based wave sub-segmentation and enhanced strategies, the novel initializer has been demonstrated to achieve satisfactory performance for both normal and abnormal heart beats under both white and pink noises. Utilizing records from Massachusetts Institute of Technology (MIT)-Beth Israel Hospital (BIH) database, this paper also applies various filters to denoise noisy signals and the denoising performances verify the availability and efficacy of the proposed denoising method.

Space robotic systems are expected to play an increasingly important role in the future on-orbit service. The applications include repairing, refueling or de-orbiting of a satellite, or removal of the space debris. The dynamical performances of space robotic system result from the multi-physics interactions between mechanical, electrical, electronic, control, etc. In this paper, we developed a unified multi-domain modelling and simulation system. The system is composed of the following modules: the path planner, joint controllers, motor and its driver, gearing mechanism of the space manipulators, the Guidance, Navigation, and Control (GNC) system, the actuators of the base, and the orbital environment, orbital dynamic and the multi-body dynamic of the whole system, etc. Based on this system, the operation during different stages, including far range rendezvous, close range rendezvous (is usually divided into two sub-phases: closing and final approach) and target capturing can be studied from the view of multi-physics domains. The key algorithms, such as pose (position and attitude) measurement, GNC of the base, path planning and control of the space manipulator, and so on, can be validated using the system. As examples, the capturing processes of a moving target under free-floating and attitude-controlled modes are simulated and the simulation results are given.

In this paper, we present an innovative design of a solder paste inspection system that can be practically integrated into existing solder paste printing machines. Since solder paste inspection systems usually occupy a large vertical space, we designed a mirror box that can redirect the transmission of fringe patterns. In this way, a new parallel-structure solder paste inspection system with a significant reduction in the vertical constraint is developed. We also developed a hybrid weighting algorithm that applies the distance and fringe contrast to acquire the height of solder pastes. Furthermore, we developed an algorithm that generates the 2-D image from the fringe pattern images during the four-step algorithm. It reduces the time required for solder paste inspection compared to traditional approaches that use special lighting systems to create the 2-D image. Based on the results of the height acquisition algorithm, 2-D and 3-D solder paste inspections are performed. Experimental results show that our system can inspect a 20 mm times 20 mm printed circuit board area within 2 s to detect common 2-D and 3-D defects, and the maximum standard deviation for the average height is 3 mum.

SUMMARYThe technologies of autonomous rendezvous and robotic capturing of non-cooperative targets are very crucial for the future on-orbital service. In this paper, we proposed a method to achieve this aim. Three problems were addressed: the target recognition and pose (position and attitude) measurement based on the stereo vision, the guidance, navigation and control (GNC) of the chaser, and the coordinated plan and control of space robot (CP&C). The pose measurement algorithm includes image filtering, edge detection, line extraction, stereo match and pose computing, et al. Based on the measured values, a certain GNC algorithm was designed for the chaser to approach and rendezvous with the target. Then the CP&C algorithm, which is proved to be advantageous over the traditional separated method, was used to plan and track the trajectories of the base pose and the joint angle. At last, a 3D simulation system was developed to evaluate the proposed method. Simulation results verified the corresponding algorithms.

SUMMARYSpace robotic systems are expected to play an increasingly important role in the future. Unlike on the earth, space operations require the ability to work in the unstructured environment. Some autonomous behaviors are necessary to perform complex and difficult tasks in space. This level of autonomy relies not only on vision, force, torque, and tactile sensors, but also the advanced planning and decision capabilities. In this paper, the authors study the autonomous target capturing from the issues of theory and experiments. Firstly, we deduce the kinematic and dynamic equations of space robotic system. Secondly, the visual measurement model of hand–eye camera is created, and the image processing algorithms to extract the target features are introduced. Thirdly, we propose an autonomous trajectory planning method, directly using the 2D image features. The method predicts the target motion, plans the end-effector's velocities and solves the inverse kinematic equations using practical approach to avoid the dynamic singularities. At last, numeric simulation and experiment results are given. The ground experiment system is set up based on the concept of dynamic simulation and kinematic equivalence. With the system, the experiments of autonomous capturing a target by a free-floating space robot, composed of a 6-DOF manipulator and a satellite as its base, are conducted, and the results validate the proposed algorithm.

For the aging population, surveillance in household environments has become more and more important. In this paper, we present a household robot that can detect abnormal events by utilizing video and audio information. In our approach, moving targets can be detected by the robot using a passive acoustic location device. The robot then tracks the targets by employing a particle filter algorithm. To adapt to different lighting conditions, the target model is updated regularly based on an update mechanism. To ensure robust tracking, the robot detects abnormal human behavior by tracking the upper body of a person. For audio surveillance, Mel frequency cepstral coefficients (MFCC) is used to extract features from audio information. Those features are input to a support vector machine classifier for analysis. Experimental results show that the robot can detect abnormal behavior such as "falling down" and "running". Also, a 88.17% accuracy rate is achieved in the detection of abnormal audio information like "crying", "groan", and "gun shooting". To lower the false alarms by abnormal sound detection system, the passive acoustic location device directs the robot to the scene where abnormal events occur and the robot can employ its camera to further confirm the occurrence of the events. At last, the robot will send the image captured by the robot to the mobile phone of master.

Space robots are playing an increasingly important role in on-orbital servicing, including repairing, refueling, or de-orbiting the satellite. The target must be captured and berthed before the servicing task starts. However, the attitude of the base may lean much and needs re-orientating after capturing. In this paper, a method is proposed to berth the target, and re-orientate the base at the same time, using manipulator motion only. Firstly, the system state is formed of the attitude quaternion and joint variables, and the joint paths are parameterized by sinusoidal functions. Then, the trajectory planning is transformed to an optimization problem. The cost function, defined according to the accuracy requirements of system variables, is the function of the parameters to be determined. Finally, we solve the parameters using the particle swarm optimization algorithm. Two typical cases of the spacecraft with a 6-DOF manipulator are dynamically simulated, one is that the variation of base attitude is limited; the other is that both the base attitude and the joint rates are constrained. The simulation results verify the presented method.

In this paper, we discuss the problem of how human control strategy can be represented as a parametric model using a support vector machine (SVM) and how an SVM-based controller can be used in controlling dynamically stable systems. The SVM approach has been implemented in the balance control of Gyrover, which is a dynamically stable, statically unstable, single-wheel mobile robot. The experimental results that compare SVM with general artificial neural network approaches clearly demonstrate the superiority of the SVM approach with regard to human control strategy learning.

This paper introduces a face detection, classification and counting system that is robust and works in real-time. It tracks multiple people, which is useful for face counting. The classification problem is defined as differentiating and then classifying the front of a face into Asian or non-Asian categories. The first step, is principal component analysis (PCA) for feature generation and independent component analysis (ICA) for feature extraction. Then, we employ support vector machine (SVM) for the training process and combine different SVM classifiers to create new classifiers, which improves the classification rate. Based on this, we can count the number of Asians and non-Asians. Experiments show that our system achieves a classification rate of 82.5% based on a database containing 750 face images from FERET.

Space robotic systems are expected to play an increasingly important role in the future. The robotic on-orbital service, whose key is the capturing technology, becomes research hot in recent years. In this paper, the authors propose an autonomous path planning method for target capturing. The task is described in Cartesian space and it can drive the manipulator to approach the target along the closest path. Firstly, the target feature is extracted based on the measured information via the hand-eye camera, and the target pose (position and orientation) and velocities (linear velocity and angular velocity) are estimated using Kalman filtering technology. Then, a numerically feasible approach is presented to plan the manipulator motion and avoid the dynamic singularities, which are transformed into real-time kinematic singularities avoiding problem. Thirdly, the potential disturbance on the base due to the manipulator's motion is estimated, and the joint rates are autonomously adjusted to reduce the disturbance if it is beyond the allowed bound. At last, a ground experiment system is set up based on the concept of dynamic emulation and kinematic equivalence. With the experiment system, the autonomous target capturing experiments are conducted. The experiment results validate the proposed algorithm.

Detecting human abnormal behaviors in a crowded situation is a challenging problem for public security departments. Human abnormal behaviors are seen either in an individual or in a crowd. This paper is focused on the latter, which is more crucial at some important spots and has been less studied. To achieve this goal, we define two categories of video energy based on intensity variation and motion features and adopt two surveillance methods for the two energy accordingly. Using wavelet analysis of the energy curves, we have obtained a result which shows that both methods can be used to deal with crowd modeling and real-time surveillance satisfactorily. A comparison of the two methods is then made in the actual environment in a metro surveillance system.

This paper presents a method for modeling abnormal human gait using hidden Markov model under the framework of a shoe-integrated system. The intelligent system focuses on modeling the following patterns: normal gait, toe in abnormality, and toe out abnormality. In the developed prototype, an inertial measurement unit (IMU) consisting of three-dimensional gyroscopes and accelerometers is employed to measure the angular velocities and accelerations of the foot. Four force sensing resistors (FSRs) and one bend sensor are arranged on the insole of each foot for force and flexion information acquisition. The proposed method is mainly based on supervised Principal Component Analysis (SPCA) for feature generation and hidden Markov model (HMM) for multi-pattern modeling. The "similarity distance measure" criterion is introduced to do model-to-model evaluation. Experimental results demonstrate the proposed approach is robust and efficient in detecting abnormal gait patterns. Our goal is to provide a cost-effective system for detecting gait abnormalities in order to assist persons with abnormal gaits in developing a normal walking pattern in their daily life.

Coded structured light is an active approach that obtains the shape of object. In our previous work, we proposed a new binary projection mechanism for inspecting semiconductor products. The mechanism consists of only a single light source in combination with a binary grating for projecting binary pattern. By shifting the binary grating in space and in every shifting taking a separate image of the illuminated surface, each position on the illuminated surface is attached with a unique binary string for correspondence. In the mechanism the inspection speed is governed by the number of needed images which also equals to the number of shiftings of the grating. To reduce image number, we propose that two neighbor bits are combined together to produce unique codeword, which is referred to as bit-pairing mechanism. This paper addresses how the spatial shifting of bit-pairing mechanism can be designed optimally for minimizing this number for faster inspection speed. An optimal solution to shifting strategy optimization is proposed that is applicable to any given binary patterns. Theoretical analysis and real image experiments are presented to illustrate the workability of the solutions.

Human gait is a dynamic biometrical feature which is complex and difficult to imitate. It is unique and more secure than static features such as passwords, fingerprints and facial features. In this paper, we present intelligent shoes for human identification based on human gait modeling and similarity evaluation with hidden Markov models (HMMs). Firstly we describe the intelligent shoe system for collecting human dynamic gait performance. Using the proposed machine learning method hidden Markov models, an individual wearer's gait model is derived and we then demonstrate the procedure for recognizing different wearers by analyzing the corresponding models. Next, we define a hidden-Markov-model-based similarity measure which allows us to evaluate resultant learning models. With the most likely performance criterion, it will help us to derive the similarity of individual behavior and its corresponding model. By utilizing human gait modeling and similarity evaluation based on hidden Markov models, the proposed method has produced satisfactory results for human identification during testing.

With the emerging need for household applications, service robotics has become a hot spot attracting researchers globally. Universities, institutes, and industrial companies, either independently or collaboratively, have been focusing on research and development for household service robots, resulting in interesting and well-performing household service robotic systems. In this chapter, we first give a categorization of the various applications, followed by an elaboration of some highlighted cases with their overall system designs.

Mapping, localization, and navigation are functions necessitated by a diverse range of household service robot solutions. This chapter presents a wide variety of mapping, localization, and navigation techniques that suit a wide range of robot platforms and applications. The problem of mapping and localization has been approached in many ways and we review successful results attained by traditional radio tracking, ultra wideband sensing, RFID communication, North-Star infrared sensing, two-dimensional special pattern detection, laser-range surveying, various camera system sensing, time-of-flight camera ranging, and finally low-cost contact and near-field sensing. The problem of navigation, path planning, and obstacle avoidance requires not only sensing described above but adequate robot responses in a dynamic situation. There are many approaches to successful obstacle avoidance and our review of the literature covers the use of structured light as well as linear and planar sensors, along with visual processing techniques such as optic flow, ground plane segmentation, and feature-based obstacle avoidance, in addition to the optimization of path planning, robot interaction, and behavior.

Household service is one of the most important fields in robot applications. Compared with applications in industry, the household environment is more challenging. First, the household environment is usually unstructured, which makes it more difficult to handle simple tasks such as what the assembly line robot can easily achieve. Second, the object for household services is human, sometimes a small child. For safety reasons, the robot should be very careful when taking actions. Third, the household robot is expected to be cleverer in dealing with complicated situations, which requires flexibility in responding. All these challenges are, to some extent, related to the capability of perceiving various objects, from an old table to a sleeping baby. In this chapter, we discuss the state of the art in object recognition for household service.

Human–robot interaction (HRI) studies on how humans collaborate and interact with robots in an effective and natural way have received considerable attention in the robot research community, especially with regard to the design and development of household service robots, owing to their highly interactive characteristics in domestic service activities. Many researchers have proposed a variety of principles, algorithms, and systems to achieve an effective and natural HRI in the domestic environment. In this chapter, we first give a brief illustration of the state of the art for HRI, divided into several categories, and then elaborate on some selected cases with regard to the design and implementation of HRI systems.

Robotic grasping and manipulation comprise three subtasks, i.e., target detection, planning, and control. The robot first needs to detect the target. Here “detect” means to find (or recognize) the target and locate it. The target could be the object to be manipulated (e.g., electrical plugs) or the destination of the object (electrical sockets). Then the robot approaches the target in an optimized trajectory. In the process of approaching and grasping, the robot should be controlled to perform the task correctly and smoothly. In this chapter, we review the approaches to accomplish these three subtasks, i.e., target detection, planning, and control.

In this chapter, an efficient and low-cost cellphone-commandable mobile manipulation system is described. Aiming at home use and elderly caring, this system can be easily commanded through a common cellphone network to efficiently grasp objects in a household environment, utilizing several low-cost off-the-shelf devices. Unlike the visual servo technology using a high quality vision system with the associated high cost, the household-service robot would not be able to afford such a high quality vision servo system, and thus it is essential to use some low-cost devices. However, it is extremely challenging to create such a vision system with precise localization, as well as motion control. To tackle this challenge, we developed a real-time vision system with which a reliable grasping algorithm combining machine vision, robotic kinematics and motor control technology is presented. After the target is captured by the arm camera, the arm camera keeps tracking the target while the arm keeps stretching until the end effector reaches the target. However, if the target is not captured by the arm camera, the arm will make a move to help the arm camera capture the target under the guidance of the head camera. This algorithm is implemented on two robot systems: one with a fixed base and another with a mobile base. The results demonstrate the feasibility and efficiency of the algorithm and system we developed, and our study shows the significance of developing a service robot in a modern household environment.

For an aging population, surveillance in household environments has become more and more important. In this chapter, we present a household robot that can detect abnormal events by utilizing video and audio information. In our approach, moving targets can be detected by the robot using a passive acoustic location device. The robot then tracks the targets by employing a particle filter algorithm. To adapt to different lighting conditions, the target model is updated regularly based on an update mechanism. To ensure robust tracking, the robot detects abnormal human behavior by tracking the upper body of a person. For audio surveillance, Mel frequency cepstral coefficients is used to extract features from audio information. Those features are input to a support vector machine classifier for analysis. Experimental results show that the robot can detect abnormal behavior such as “falling down” and “running.” Also, an 88.17% accuracy rate is achieved in the detection of abnormal audio information like “crying,” “groaning,” and “gun shooting.” To lower the false alarms by the abnormal sound detection system, the passive acoustic location device directs the robot to the scene where the abnormal events are occuring and the robot can employ its camera to further confirm the occurrence of the events. At last, the robot will send the image captured by the robot to the mobile phone of its master.

With the significant development of robotic technologies, the applications of robots have been extended broadly from their traditional industry role to military, medical field, and even daily services. Robotics has gradually become a very large industry with tremendous influence on the sustainable development of the world.

Technology for the capturing of space target is very important for on-orbit servicing. In order to assure the task is accomplished successfully, ground experimentations are required for the verification of the planning and control algorithms of space robotic system before it is launched. In this paper, an experiment concept is used, which is a hybrid approach, i.e. it combines the mathematical model with the physical model. The key issues of the concept are dynamic emulation and kinematic equivalence, in which the behaviors of the space robotic system are calculated by its dynamic equations. The motion of its end-effector and the space target is realized by two industrial robots. According to different observation spots, two modes of capturing process are emulated: one is observed from the inertial frame, the other is from the space base. Based on the concept proposed above, a ground experiment system is set up, which is composed of two industrial robots, a set of global visual system and five industrial computers. Using the system, algorithms of space robot of any geometry and mass properties can be tested. As an example, the autonomous trajectory planning algorithm is verified by the experiment of capturing a moving target. Moreover, a real-time 3D simulation system is developed to emulate the capturing process in 3D space. Numeric simulation and experiment results show that the ground system is effective in evaluating the planning and control algorithms of space robot.

In practice, the environments in which mobile robots operate are usually modeled in highly complex forms and, as a result, autonomous navigation and localization can be difficult. The difficulties are exacerbated for practical robots with limited on-board computational resources and complex planning algorithms, since this paradigm of environmental modeling requires enormous computational power. A novel navigation/localization learning methodology is presented to abstract and transfer the human sequential navigational skill to a robotic wheelchair by showing the platform how to respond in different local environments along a demonstrated, static cluttered route using a lookup table representation. This method utilizes limited on-board range sensing information to concisely model local unstructured environments, with respect to the robot, for navigation or localization along the learned route in order to achieve good performance with low on-line computational demand and low-cost hardware requirements. Experimental study demonstrates the feasibility of this method and some interesting characteristics of navigation, localization, and environmental modeling problems. Analysis is also conducted to investigate performance evaluation, advantages of the approach, choices of lookup table inputs and outputs, and potential generalization of this paper

Surgical training systems using virtual reality simulation techniques offer a cost-effective alternative to traditional training methods. In this sense, techniques for interactive visualization and virtual reality surgery have been one of the very important research areas. We describe various techniques we have used in developing a virtual reality system for anatomic visualization and training arthroscopic knee surgeons. Virtual models used in our systems are constructed from the Visible Human Project and Chinese Visible Human data sets. We present our various developments in segmentation, personal-computer-based real-time volume visualization, soft tissue deformation with topological change in real-time using finite element analysis, and soft tissue cutting with tactile feedback.