Search

Humans have a remarkable ability to learn continuously from th e environment and inner experience. One of the grand goals of robots is to build an artificial "lifelong learning" agent that can shape a cultivated understanding of the world from the current scene and previous knowledge via an autonomous lifelong development. It is challenging for the robot learning process to retain earlier knowledge when robots encounter new tasks or information. Recent advances in computer vision and deep -learning methods have been impressive due to large-scale data sets, such as ImageNet and COCO. However, robotic vision poses unique new challenges for applying visual algorithms developed from these computer vision data sets because they implicitly assume a fixed set of categories and time -invariant task distributions.

Hand gesture recognition plays an essential role in the human-robot interaction (HRI) field. Most previous research only studies hand gesture recognition in a short distance, which cannot be applied for interaction with mobile robots like unmanned aerial vehicles (UAVs) at a longer and safer distance. Therefore, we investigate the challenging long-range hand gesture recognition problem for the interaction between humans and UAVs. To this end, we propose a novel attention-based single shot multibox detector (SSD) model that incorporates both spatial and channel attention for hand gesture recognition. We notably extend the recognition distance from 1 meter to 7 meters through the proposed model without sacrificing speed. Besides, we present a long-range hand gesture (LRHG) dataset collected by the USB camera mounted on mobile robots. The hand gestures are collected at discrete distance levels from 1 meter to 7 meters, where most of the hand gestures are small and at low resolution. Experiments with the self-built LRHG dataset show our methods reach the surprising performance-boosting over the state-of-the-art method like the SSD network on both short-range (1 meter) and long-range (up to 7 meters) hand gesture recognition tasks.

The ability to maintain balance and stability is a major challenge for quadruped robots in locomotion. For example, when walking in the unstructured environment, the robot not only needs to adjust its gait according to the terrain information, but also needs to tune the body posture based on the real-time body posture acquired from inertial measurement unit (IMU) to maintain stability. In this paper, we present a method that focuses on the design of the posture correction controller based on the IMU feedback for adaptive slope walking for a quadruped robot. To this end, the posture correction controller is integrated into the foot trajectory generator to maintain balance. We experimentally validated our approach both on simulation and real quadruped robot. Compared with the one only with the foot trajectory generator, the quadruped robot can walk more stable with the posture correction controller integrated into the foot trajectory generator and has a better locomotion ability. Our method increases locomotion stability when walking on the slopes.

The challenges in transforming paintings into music is well-known, since the relationship between two kinds of art is unclear. Different composers write different music when the same painting is presented to them. In this paper, a cross-mordality model has been proposed for transforming images into multitrack music based on the framework of deep convolutional generative adversarial networks (DCGANs). The proposed model is trained on a classical music dataset and a dataset of impressionist paintings. The model can be applied to transfer impressionist paintings into classical music with two tracks. By using music evaluation methods, the harmonicity of the generated music can be confirmed. Our model is the first attempt of our knowledge at transforming paintings into music segments.

Currently, AGVs (Auto Guided Vehicles) are the most widely used type of mobile robot in real world for logistics and intelligent manufacturing. While most of indoor AGVs do not have a special requirement on shock absorption systems, it remains a major concern for the outdoor AGVs' application. Compared with most of the current mono-spring absorption system designs, which means that all the spring dampers have identical elastic coefficient, dual-spring designs have an advantage in obstacle-crossing capability and shock absorption performances. This paper studies and validates the enhanced outdoor working performances of AGVs with dual-spring shock absorption systems. Theoretical dynamic analysis has been conduct to reveal the advantages in shock absorption characteristics. A series of simulation experiments have been performed to verify its advantages in stability and obstacle-crossing capabilities compared to conventional mono-spring absorption designs.

Recent studies have witnessed a significant progress in developing legged robots with rapid and stable locomotion speeds and adaptability to complex roads, but their high energy consumption limited their development and applications. This paper analyzes the energy efficiency of a bionic electric-drive panda robot serving as an emotional companion for robot-enhanced therapy. Unlike other quadruped robots, our bionic robot focuses on emotional interaction rather than capability of dexterous locomotion and operations. This results in the need to change working period. Then we proposed four strategies to reduce energy consumption. The energy efficiency optimization strategy described in this paper is suitable for quadruped robots focusing on none-highly dynamic motions, and can be treated as locomotion optimization strategy based on energy efficiency optimization for quadruped robots to some extent.

Accurate turning is not an easy task for the quadruped robot and previous work mainly addresses the problem with complex control strategy or systems. This paper proposes a design method for the accurate turning motion of a quadruped robot with 12 degrees of freedom (DOF). Unlike others, we have achieved precise turning motion with a relatively simple geometric approach. Once the parameters of desired movement are set, such as the turning radius and angles, the robot can realize corresponding circle locomotion. A design architecture including geometric analysis, 3D foot trajectory design and inverse kinematics is presented to solve the problem systematically. In addition, according to the different conditions, the corresponding compensation coefficient is necessarily considered in order to acquire precise values of leg joints. At last, comparison results by several experiments have identified the feasibility and accuracy of turning along a circle trajectory based on our proposed method.

Trajectory tracking control is one of the core techniques that impacts the auto-driving performance of a mobile robot. Whereas, there lacks enough work on reference trajectory generation and controller design for practical usage. This paper considers mobile robots with unicycle vehicle model on which most of automatic guided vehicles (AGVs) in real world are built. A new trajectory planning algorithm is developed, and is applied along with a control law considering constraints of the unicycle model and limited motor capabilities. The proposed algorithm is easy to be implemented on real world AGVs, and it yields a fast, accurate and robust trajectory tracking performance. The effectiveness of the algorithm is validated by simulation tests.

Localization and navigation are fundamental issues to autonomous mobile robotics. In the case of the environmental map has been built, the traditional two-dimensional (2D) lidar localization and navigation system can't be matched to the initial position of the robot in dynamic environment and will be unreliable when kidnapping occurs. Moreover, it relies on high-cost lidar for high accuracy and long range. In view of this, the paper presents a low cost navigation system based on a low cost lidar and a cheap webcam. In this approach, 2D-codes are attached to the ceiling, to provide reference points to aid the indoor robot localization. The mobile robot is equipped with webcam pointing to the ceiling to identify 2D-codes. On the other hand, a low-cost 2D laser scanner is applied to build a map in unknown environment and detect obstacles. Adaptive Monte Carlo Localization (AMCL) is implements for lidar positioning, A* and Dynamic Window Approach (DWA) are applied in path planning based on a 2D grid map. The error analysis and experiments has validated the proposed method.

In this paper, we study the parameter setting for a set of intelligent multi-category classifiers in wall-following robot navigation. Based on the swarm optimization theory, a particle selecting approach is proposed to search for the optimal parameters, a key property of this set of multi-category classifiers. By utilizing the particle swarm search, it is able to obtain higher classification accuracy with significant savings on the training time compared to the conventional grid search. For wall-following robot navigation, the best accuracy (98.8%) is achieved by the particle swarm search with only 1/4 of the training time by the grid search. Through communicating the social information available in particle swarms in the training process, classification-based learning can achieve higher classification accuracy without prematurity. One of such learning classifiers has been implemented in SIAT mobile robot. Experimental results validate the proposed search scheme for optimal parameter settings.

This paper proposes a global path- and motion-planning algorithm that enables inchworm-like robots to navigate their way up tree branches. The intuitive climbing space representation method proposed here greatly simplifies the path-planning problem. The dynamic programming algorithm can be used to identify the optimal path leading to the target position in the target direction according to the constraints and requirements specified. The planned path can be applied in any tree-climbing robot that utilizes the nonenclosure gripping method. An efficient motion-planning algorithm for continuum inchworm-like robots is then developed to enable them to climb along the planned path with a high degree of accuracy. In comparison with the method proposed in our previous study, the method proposed herein significantly improves consistency between the planned path and the motions of the robot, and therefore makes it more practical to implement the motion-planning algorithm in trees of different shapes. The paper also describes hardware experiments in which the proposed planning algorithm is applied to enable inchworm-like robots to climb real trees, thus validating the proposed planning algorithm in practice. © 2012 Wiley Periodicals, Inc. © 2013 Wiley Periodicals, Inc.

Abnormal crowd behavior detection plays an important role in surveillance applications. We propose a camera parameter independent and perspective distortion invariant approach to detect two types of abnormal crowd behavior. The two typical abnormal activities are people gathering and running. Since people counting is necessary for detecting the abnormal crowd behavior, we present an potential energy-based model to estimate the number of people in public scenes. Building histograms on the X- and Y-axes, respectively, we can obtain probability distribution of the foreground object and then define crowd entropy. We define the Crowd Distribution Index by combining the people counting results with crowd entropy to represent the spatial distribution of crowd. We set a threshold on Crowd Distribution Index to detect people gathering. To detect people running, the kinetic energy is determined by computation of optical flow and Crowd Distribution Index. With a threshold, kinetic energy can be used to detect people running. To test the performance of our algorithm, videos of different scenes and different crowd densities are used in the experiments. Without camera calibration and training data, our method can robustly detect abnormal behaviors with low computation load.

Early automatic detection of cardiovascular diseases is of great importance to provide timely treatment and reduce fatality rate. Although many efforts have been devoted to detecting various arrhythmias, classification of other common cardiovascular diseases still lacks comprehensive and intensive studies. This work aims at developing an automatic diagnosis system for myocardial infarction, valvular heart disease, cardiomyopathy, hypertrophy, and bundle branch block, based on the clinic recordings provided by PTB Database. The proposed diagnosis system consists of the components as baseline wander reduction, beat segmentation, feature extraction, feature reduction and classification. The selected features are the location, amplitude and width of each wave, exactly the parameters of ECG dynamical model. We also propose a mean shift algorithm based method to extract these features. To demonstrate the availability and efficacy of the proposed system, we use a total of 13,564 beats to conduct a large scale experiment, where only 25% beats are utilized to train the eigenvectors of generalized discriminant analysis in the feature reduction phase and 25% beats are applied to train the support vector machine in the classification phase. The average sensitivity, specificity and positive predicitivity for the test set, containing 75% beats, are respectively 96.06%, 99.32% and 97.29%.

Denoising of electrocardiogram (ECG) is the fundamental technique for manual or automatic ECG diagnosis. Model-based denoising has attracted initial studies since the ECG dynamical model was established in 2003 and been demonstrated to outperform most model-less denoising methods. The focus of this paper is robust denoising of abnormal ECG signals, which do not satisfy the assumption in previous model-based studies that morphological or physiological variations are small from one beat to another. A mean shift based initializer is proposed to provide a much more robust estimation of initial model parameters for each heart beat. Together with physiological knowledge based wave sub-segmentation and enhanced strategies, the novel initializer has been demonstrated to achieve satisfactory performance for both normal and abnormal heart beats under both white and pink noises. Utilizing records from Massachusetts Institute of Technology (MIT)-Beth Israel Hospital (BIH) database, this paper also applies various filters to denoise noisy signals and the denoising performances verify the availability and efficacy of the proposed denoising method.

SUMMARYSpace robotic systems are expected to play an increasingly important role in the future. Unlike on the earth, space operations require the ability to work in the unstructured environment. Some autonomous behaviors are necessary to perform complex and difficult tasks in space. This level of autonomy relies not only on vision, force, torque, and tactile sensors, but also the advanced planning and decision capabilities. In this paper, the authors study the autonomous target capturing from the issues of theory and experiments. Firstly, we deduce the kinematic and dynamic equations of space robotic system. Secondly, the visual measurement model of hand–eye camera is created, and the image processing algorithms to extract the target features are introduced. Thirdly, we propose an autonomous trajectory planning method, directly using the 2D image features. The method predicts the target motion, plans the end-effector's velocities and solves the inverse kinematic equations using practical approach to avoid the dynamic singularities. At last, numeric simulation and experiment results are given. The ground experiment system is set up based on the concept of dynamic simulation and kinematic equivalence. With the system, the experiments of autonomous capturing a target by a free-floating space robot, composed of a 6-DOF manipulator and a satellite as its base, are conducted, and the results validate the proposed algorithm.

This paper introduces a face detection, classification and counting system that is robust and works in real-time. It tracks multiple people, which is useful for face counting. The classification problem is defined as differentiating and then classifying the front of a face into Asian or non-Asian categories. The first step, is principal component analysis (PCA) for feature generation and independent component analysis (ICA) for feature extraction. Then, we employ support vector machine (SVM) for the training process and combine different SVM classifiers to create new classifiers, which improves the classification rate. Based on this, we can count the number of Asians and non-Asians. Experiments show that our system achieves a classification rate of 82.5% based on a database containing 750 face images from FERET.

Detecting human abnormal behaviors in a crowded situation is a challenging problem for public security departments. Human abnormal behaviors are seen either in an individual or in a crowd. This paper is focused on the latter, which is more crucial at some important spots and has been less studied. To achieve this goal, we define two categories of video energy based on intensity variation and motion features and adopt two surveillance methods for the two energy accordingly. Using wavelet analysis of the energy curves, we have obtained a result which shows that both methods can be used to deal with crowd modeling and real-time surveillance satisfactorily. A comparison of the two methods is then made in the actual environment in a metro surveillance system.

This paper presents a method for modeling abnormal human gait using hidden Markov model under the framework of a shoe-integrated system. The intelligent system focuses on modeling the following patterns: normal gait, toe in abnormality, and toe out abnormality. In the developed prototype, an inertial measurement unit (IMU) consisting of three-dimensional gyroscopes and accelerometers is employed to measure the angular velocities and accelerations of the foot. Four force sensing resistors (FSRs) and one bend sensor are arranged on the insole of each foot for force and flexion information acquisition. The proposed method is mainly based on supervised Principal Component Analysis (SPCA) for feature generation and hidden Markov model (HMM) for multi-pattern modeling. The "similarity distance measure" criterion is introduced to do model-to-model evaluation. Experimental results demonstrate the proposed approach is robust and efficient in detecting abnormal gait patterns. Our goal is to provide a cost-effective system for detecting gait abnormalities in order to assist persons with abnormal gaits in developing a normal walking pattern in their daily life.

Human gait is a dynamic biometrical feature which is complex and difficult to imitate. It is unique and more secure than static features such as passwords, fingerprints and facial features. In this paper, we present intelligent shoes for human identification based on human gait modeling and similarity evaluation with hidden Markov models (HMMs). Firstly we describe the intelligent shoe system for collecting human dynamic gait performance. Using the proposed machine learning method hidden Markov models, an individual wearer's gait model is derived and we then demonstrate the procedure for recognizing different wearers by analyzing the corresponding models. Next, we define a hidden-Markov-model-based similarity measure which allows us to evaluate resultant learning models. With the most likely performance criterion, it will help us to derive the similarity of individual behavior and its corresponding model. By utilizing human gait modeling and similarity evaluation based on hidden Markov models, the proposed method has produced satisfactory results for human identification during testing.

For an aging population, surveillance in household environments has become more and more important. In this chapter, we present a household robot that can detect abnormal events by utilizing video and audio information. In our approach, moving targets can be detected by the robot using a passive acoustic location device. The robot then tracks the targets by employing a particle filter algorithm. To adapt to different lighting conditions, the target model is updated regularly based on an update mechanism. To ensure robust tracking, the robot detects abnormal human behavior by tracking the upper body of a person. For audio surveillance, Mel frequency cepstral coefficients is used to extract features from audio information. Those features are input to a support vector machine classifier for analysis. Experimental results show that the robot can detect abnormal behavior such as “falling down” and “running.” Also, an 88.17% accuracy rate is achieved in the detection of abnormal audio information like “crying,” “groaning,” and “gun shooting.” To lower the false alarms by the abnormal sound detection system, the passive acoustic location device directs the robot to the scene where the abnormal events are occuring and the robot can employ its camera to further confirm the occurrence of the events. At last, the robot will send the image captured by the robot to the mobile phone of its master.

Recently, audio analysis has been proposed by some researchers for surveillance application. Compared with video surveillance, the effectiveness of audio surveillance is not influenced by the occlusions. In this paper, we propose a learning based approach to intelligent audio surveillance in household environment. This approach can be applied to static or mobile audio surveillance systems which work solely or work as the supplements of video surveillance systems. After extracting Mel frequency cepstral coefficients (MFCC) from a 1.0 s waveform, a classifier trained from a labeled dataset using modest AdaBoost is employed to determine whether this waveform is normal or abnormal. The result and simulation show the effectiveness of this approach.

In practice, the environments in which mobile robots operate are usually modeled in highly complex forms and, as a result, autonomous navigation and localization can be difficult. The difficulties are exacerbated for practical robots with limited on-board computational resources and complex planning algorithms, since this paradigm of environmental modeling requires enormous computational power. A novel navigation/localization learning methodology is presented to abstract and transfer the human sequential navigational skill to a robotic wheelchair by showing the platform how to respond in different local environments along a demonstrated, static cluttered route using a lookup table representation. This method utilizes limited on-board range sensing information to concisely model local unstructured environments, with respect to the robot, for navigation or localization along the learned route in order to achieve good performance with low on-line computational demand and low-cost hardware requirements. Experimental study demonstrates the feasibility of this method and some interesting characteristics of navigation, localization, and environmental modeling problems. Analysis is also conducted to investigate performance evaluation, advantages of the approach, choices of lookup table inputs and outputs, and potential generalization of this paper

Localization is one of the key research topics in robotics. Although computer vision based methodologies such as Augmented Markers provides a realtime localization approach, the sensing range and localization accuracy are the main limitations for applications. In this paper, we proposed the use of a problem solving tool, TRIZ to overcome these limitations. Among the tools provided by TRIZ, we illustrated the selection of principles for problem solving. To enhance the sensing range, a novel AR marker is presented. To improve the localization accuracy, we presented the use of an AR polyhedron for system localization. With the use of an AR cube for illustration, we further developed a new optimization strategy. It is a weighted sum method which aims to optimize the position of the AR cube by considering the orientation of different AR markers captured by a camera. To evaluate the proposed methods, different experiments have been conducted including static and dynamic localizations. Results show that the proposed solutions and strategy can significantly improve the sensing range and accuracy of the conventional AR localization system.

This paper introduces a real-time video surveillance system which can track people and detect human abnormal behaviors. In the blob detection part, an optical flow algorithm for crowd environment is studied experimentally and a comparison study with respect to traditional subtraction approach is carried out. The different approaches in segmentation and tracking enable the system to track persons when they change movement unpredictably in occlusion. We developed two methods for the human abnormal behavior analysis. The first one employs Principal Component Analysis for feature selection and Support Vector Machine for classification of human behaviors. The proposed feature selection method is based on the border information of four consecutive blobs. The second approach computes optical flow to obtain the velocity of each pixel for determining whether a human behavior is normal or not. Both algorithms are successfully developed in crowded environments to detect the following human abnormal behaviors: (1) Running people in a crowded environment; (2) falling down movement while most are walking or standing; (3) a person carrying an abnormal bar in a square; (4) a person waving hand in the crowd. Experimental results demonstrate these two methods are robust in detecting human abnormal behaviors.

Surveillance of public places has become a worldwide concern in recent years. The ability to identify abnormal human behaviors in real-time is fundamental to the success of intelligent surveillance systems. The recognition of abnormal and suspicious human walking patterns is an important step towards the achievement of this goal. In this research, we develop an intelligent visual surveillance system that can classify normal and abnormal human walking trajectories in outdoor environments by learning from demonstration. The system takes into account both the local and global characteristics of the observed trajectories and is able to identify their normality in real-time. By utilizing support vector learning and a similarity measure based on hidden Markov models, the developed system has produced satisfactory results on real-life data during testing. Moreover, we utilize the approach of longest common subsequence (LCSS) in determining the similarity between different types of walking trajectories. In order to establish the position and speed boundaries required for the similarity measure, we compare the performance of a number of approaches, including fixed boundary values, variable boundary values, learning boundary by support vector regression, and learning boundary by cascade neural networks.

In this research, computational intelligence techniques are applied towards the modeling of human sensations in virtual environments. We specifically focus on the following important questions: (1) how to efficiently model the relationship between human sensations and the physical stimuli presented to humans, (2) how to validate the human sensation models, and (3) how to reduce the size of the input data when it gets large and how to select the information which is most important to human sensation modeling. In order to provide an experimental testbed for the implementation of the proposed learning and analysis techniques, a full-body motion virtual reality interface capable of recording human sensations is developed. We propose using cascade neural networks with node-decoupled extended Kalman filter training for modeling human sensation in virtual environments. For the purpose of sensation model validation, we propose using a stochastic similarity measure based on hidden Markov models to calculate the relative similarity between model-generated sensation and actual human sensation. Next, we investigate a number of feature extraction and input selection techniques for reducing the input data size in human sensation modeling. We propose and develop a new input selection method based on independent component analysis, which is capable of reducing the data size and selecting the stimuli information that is most important to the human sensation.

Modeling human control strategy (HCS) is becoming an increasingly popular paradigm in a number of different research areas, ranging from robotics and intelligent vehicle highway systems to expert training and virtual reality computer games. Usually, HCS models are derived empirically, rather than analytically, from real-time human input–output data. While these empirical models offer an effective means of transferring intelligent behaviors from humans to robots and other machines, there is a great need to develop adequate performance criteria for these models. It is our goal in this paper to develop several such criteria for the task of human driving. We first collect driving data from different individuals through a real-time graphic driving simulator that we have developed, and identify each individual’s control strategy model through the flexible cascade neural network learning architecture. We then define performance measures for evaluating two aspects of the resultant HCS models. The first is based on event analysis, while the second is based on inherent analysis. Using the proposed performance criteria, we demonstrate the procedure for evaluating the relative skill of different HCS models. Finally, we propose an iterative algorithm for optimizing an initially stable HCS model with respect to independent, user-specified performance criteria, by applying the simultaneously perturbed stochastic approximation (SPSA) algorithm. The methods proposed herein offer a means for modeling and transferring HCS in response to real-time inputs, and improving the intelligent behaviors of artificial machines.

Owing to the ever-increased demand for product quality improvement and production cost reduction, on-line monitoring of stamping operations has become a common practice in shop floors around the world today. Based on a market survey, for most stamping processes, monitoring systems use tonnage signals and/or strain signals. Recently, a few attempts have been reported using acceleration signals as they contain rich information and are relatively inexpensive. However, it is known that acceleration signals are vulnerable to the noise disturbances, and hence are less robust. This paper presents a study that uses bispectrum to analyze the acceleration signals. It is shown that the bispectrum can suppress Gaussian color noise to boost the signal-to-noise ratio. It also extracts the features of the signal that are related to the defective parts (such as material too thick or slug). The experimental results demonstrate that the method presented is effective and has a good potential for applications in shop floor. We also present new method for reducing the computation load in the process.

The human behavior, such as gait, is controlled by the brain and other peripheral neural system, and thus is significantly affected by the internal emotion. It is important to carry out the affective sensation of abnormal internal emotion in a non-invasive and non-privacy-infringed way. This paper presents an experiment method and platform to sense the abnormal affective status through gaits. We used a wearable intelligent shoe system for affective sensing, designed an experiment for the human subject to have the normal gait and the gait with emotional disorder, and finally classified the gaits through machine learning.

A transmission from the engine crank to the main transmission with variable reduction ratio is desirable for the hybrid electric vehicles, especially for micro/mild hybrid vehicles with auto start-stop function, for the sake of reducing the design requirements for integrated starter-generator and thus reducing the cost. This paper presents a solution to the issue by introducing a Controller-Free Ratio-Variable gearbox. Unlike other solutions, the proposed gearbox avoids from using actuators to change gear ratios but realize the function in a purely mechanical and fully automatic way. This feature reduces system complexity and improves system stability, as electronic-based info sensing and actuating are not as robust in the engine compartment. Also, the feature allows to change gear ratio faster and thus better protecting integrated starter-generator from over speeding. In this paper, the principle of the gearbox is explained and its performance under different conditions is studied.

As a key device in nuclear station, the steam generator is vulnerable to certain kinds of damages, which make it necessary to check the generator annually. However, it is impossible for human to finish the examine because of that the tremendous tubes inside the generator makes its inner space too crowded. In this paper, we introduced a novel designed climbing robot system which is competent to the work. The main part of the robot system is a vehicle which can moves inside the steam generator carrying a PT Arm. Four cameras are installed in the vehicle and PT Arm to send the real time video back for analysis. We built a control panel as the interface for users to control the vehicle and a cabinet as the container of all the processors int the control system. Experiments show that our system works stably inside the steam generator.

In this paper, we present a method for modeling human strategies in controlling a light source in a dynamic environment. In order to illustrate the procedure and method, we take a simple example of how to control a light source so as to avoid a shadow and to maintain appropriate illumination conditions on the target area of attention. This work is likely to be valuable in various applications of automatic light control-from surgical rooms and space applications to inspections.

In this paper, the authors present a stochastic,discontinuous modeling framework that is designed for abstracting human control strategies and is based on hidden Markov models (HMM). A real-time driving simulator is developed for investigating human control strategies. The shortcomings of a typical continuous modeling approach for modeling discontinuous human control strategies are discussed. The HMM-based method is shown to demonstrate greater fidelity to the human training data.

In this paper ceiling affixed ARToolKitPlus 2D code artificial landmarks are evaluated for purposes of robot localization and navigation. Ceiling affixed codes rarely come in contact with people, equipment or robots, and for this reason they are more likely to stay detectable over a longer period of time. Multi threshold averaging, light gradient compensation and neighbourhood search techniques further enhanced AR-ToolKitPlus performance. Multi threshold averaging collects positioning results at each gray scale threshold level. After all of the threshold levels are analyzed, the related results are averaged into a final result. Light gradient compensation eliminates effects of uneven lighting in the neighbourhood of a 2D code. Neighbourhood search for a 2D code requires fewer computational resources than a global search. Further repeatability improvements are achieved by means of averaging localizations. Localization performance is evaluated at varying distances of the 2D code from the camera. Experimental results show substantial improvements in repeatability and reliability over the baseline ARToolKitPlus performance. Improved performance will allow for realtime 2D code based localization for indoor robot navigation.

In this paper, we present a method for high-level control of robots whose low-level software is based on dynamically reconfigurable, reusable real-time software modules. Our approach is to use an embedded interpreter for a general-purpose programming language to direct the operation of the low-level modules toward meeting the task-level goals of the robot. To this end, we present RSK, a virtual-machine kernel implementing a Scheme interpreter capable of hard real-time operation, and employing a method of code execution we call “message-based evaluation” (MBE) . MBE is a novel combination of a traditional code execution model and a message-passing architecture, which simplifies the process of writing code for managing the robot‘s reconfigurable subsystem.

In this paper, we discuss the problem of how a free-floating space manipulator can be mapped to a conventional, fixed-base manipulator which preserves both its dynamic and kinematic properties. This manipulator is called Dynamically Equivalent Manipulator (DEM). The DEM concept not only allows us to model a free-floating space manipulator system with simple, well-understood methods, but also to build a conventional manipulator system to experimentally study the dynamic performance and task execution of a space manipulator system, without having to resort to complicated experimental set-ups to simulate the space environment. This paper presents the theoretical development of the DEM concept, demonstrates the dynamic and kinematic equivalence, and presents simulation results to illustrate the equivalence under open-loop and closed-loop control strategies.

To successfully interact with and learn from humans in cooperative modes, robots need a mechanism for recognizing, characterizing, and emulating human skills. In particular, it is our interest to develop the mechanism for recognizing and emulating simple human actions, i.e., a simple activity in a manual operation where no sensory feedback is available. To this end, we have developed a method to model such actions using a hidden Markov model (HMM) representation. We proposed an approach to address two critical problems in action modeling: classifying human action-intent, and learning human skill, for which we elaborated on the method, procedure, and implementation issues in this paper. This work provides a framework for modeling and learning human actions from observations. The approach can be applied to intelligent recognition of manual actions and high-level programming of control input within a supervisory control paradigm, as well as automatic transfer of human skills to robotic systems.

In decades, many researchers have came up with many path planning algorithms. Robots rely on sensor data to deduce inner and outer state as its position. Many localization algorithm suggests robot's knowledge of its position should be a probability distribution. However, previews path planning algorithms fell to utilize this information and just simply assume the robots to be in a single certain point. In this paper, we described the implementation of a new algorithm M* that takes the uncertainty of the robots location into detailed consideration. M* uses Monto Carlo method to represent probability distribution and by considering problem in high dimension space it transfer the problem into a neatly form. Some important concepts are introduced and pseudo code is used to show how them work.

For surveillance robots, road detection is of high importance for other functionalities such as pedestrian detection, obstacle avoidance, autonomous running, etc. The vision-based road detection is to classify image pixels belonging to road surface or not. Up to now, most algorithms are designed for working during daytime. In this paper, we focus on road detection at night. Firstly a planar reflection model is used to fit the intensity distribution of the images pixels got from a near-infrared camera. After that, we use a pixel-based classification to determine whether the pixel belongs to the road surface or not. In the experiments, we compare our algorithm with the region growing method. The results show that our approach works better in several aspects.

The energy of a space station is a precious resource, and the minimization of energy consumption of a space manipulator is crucial to maintain its normal functionalities. This paper first presents novel gaits for space manipulators by equipping a new gripping mechanism. With the use of wheels locomotion, lower energy demand gaits can be achieved. With the use of the proposed gaits, we further develop a global path planning algorithm for space manipulators which can plan a moving path on a space station with a minimum total energy demand. Different from existing approaches, we emphasize both the use of the proposed low energy demand gaits and the gaits composition during the path planning process. To evaluate the performance of the proposed gaits and path planning algorithm, numerous simulations are performed. Results show that the energy demand of both the proposed gaits and the resultant moving path is also minimum.

As the most critical function of mobile robots, navigation substantially relies on self-localization. Among all the approaches, vision based localization is widely adopted. Unlike the complex algorithms developed before, we present a simple real-time visual localization approach which regards the circular lamps on the ceiling as the landmarks. Two sets of point correspondences are applied to solving Perspective-n-Point (PnP) problem. One set contains the centers of the ellipses in image and the centers of the circles in 3D. The other set contains the tangent points on the circles and ellipses. At least two lamps are required in the field of view to avoid ambiguities. The experimental results show that no blind zone exists since the required number of lamps is small. The position and orientation errors are small enough for indoor navigation. This approach is validated by experiments for real-time application.

As one of the most natural and intuitive way of communication between people and machines, hand gesture is widely used in HCI (Human-Computer-interaction). In this paper, we proposed a novel method for hand tracking and pose recognition based on Kinect. For hand tracking, skin information is used for initialization of hand segmentation, and then a region growing algorithm is applied in the depth image to separate hand from other skin colored objects. Finally, a Kalman filter is used for tracking hand in 3D space. For hand recognition, we decompose the problem of recognizing hand pose into recognizing different finger states. Both contour information of the whole hand and depth information inside the contour are considered for finger states recognition. It is shown in the experiments that our system can track the hand robustly and recognize more than 90% of the hand poses we define for our depth image database.

The shared control system is a modular real-time system designed to execute complex tasks through the intelligent coordination of task modules. A state machine is used to control task sequencing, and due to the automatic switching, the accuracy and reliability with which tasks are executed, is greatly improved. Tasks consist of sets of independent, modular, and reusable subtasks whose outputs are combined to create the control. This system has proved itself useful for rapid development of reliable high level multiple sensor based manipulation and control tasks. Additionally, we have integrated a neural network based visual servoing system, a semi-compliant Cartesian trajectory following heuristics, and a real-time graphical user interface in the system. The shared control system was implemented for the Self-Mobile Space Manipulator (SM 2) to handle a range of tasks associated with locomotion, manipulation, and material transportation on Space-Station Freedom.

The global motion planning problem for tree climbing is challenging, as trees have an irregular and complex shape. To the best of the author’s knowledge, there is no related study that focuses on the global motion planning problem for tree-climbing robot. Aracil [17] proposed a motion planning method to allow the Parallel Climbing Robot to climb a trunk. However, this work merely discussed the local motion planning problem according to local information. There are many motion planning approaches for climbing in artificial structures, such as walls and glass windows [10, 11, 21]. However, these structures are different from trees, and the approaches are thus not suitable for tree-climbing problems.

The proposed tree gripper introduced in Chapter 4 is simple in control. It also implies that it has less controllability. In order to tackle the variant environment in limited controllability, the setting of the gripper should be optimized so as to maximize the fastening force in a wide range of surface curvature. The optimized items include the spine installation angle and the spring force distribution so as to provide optimal spine insert angle and direction of action force over a wide range of surface curvatures. On-tree experiments of the proposed mechanism will be presented at the end of this chapter to evaluate the optimization results.

There are several robots developed in the literature that are capable of climbing trees. Some of the robots are specifically designed for tree climbing while some of them are designed for moving in multiple terrains including trees. In this chapter, the details of these robots are introduced to provide a whole picture of the field of the tree-climbing technology.

Based on the limitation of the existing tree-climbing robots, one of the motivations of the work presented in this book is to develop a novel type of tree-climbing robot that can assist or replace humans in performing tasks on trees. The robot should be applicable to various pursuits such as harvesting, tree maintenance and observation of arboreal animals.