Search

Humans have a remarkable ability to learn continuously from th e environment and inner experience. One of the grand goals of robots is to build an artificial "lifelong learning" agent that can shape a cultivated understanding of the world from the current scene and previous knowledge via an autonomous lifelong development. It is challenging for the robot learning process to retain earlier knowledge when robots encounter new tasks or information. Recent advances in computer vision and deep -learning methods have been impressive due to large-scale data sets, such as ImageNet and COCO. However, robotic vision poses unique new challenges for applying visual algorithms developed from these computer vision data sets because they implicitly assume a fixed set of categories and time -invariant task distributions.

Hand gesture recognition plays an essential role in the human-robot interaction (HRI) field. Most previous research only studies hand gesture recognition in a short distance, which cannot be applied for interaction with mobile robots like unmanned aerial vehicles (UAVs) at a longer and safer distance. Therefore, we investigate the challenging long-range hand gesture recognition problem for the interaction between humans and UAVs. To this end, we propose a novel attention-based single shot multibox detector (SSD) model that incorporates both spatial and channel attention for hand gesture recognition. We notably extend the recognition distance from 1 meter to 7 meters through the proposed model without sacrificing speed. Besides, we present a long-range hand gesture (LRHG) dataset collected by the USB camera mounted on mobile robots. The hand gestures are collected at discrete distance levels from 1 meter to 7 meters, where most of the hand gestures are small and at low resolution. Experiments with the self-built LRHG dataset show our methods reach the surprising performance-boosting over the state-of-the-art method like the SSD network on both short-range (1 meter) and long-range (up to 7 meters) hand gesture recognition tasks.

The challenges in transforming paintings into music is well-known, since the relationship between two kinds of art is unclear. Different composers write different music when the same painting is presented to them. In this paper, a cross-mordality model has been proposed for transforming images into multitrack music based on the framework of deep convolutional generative adversarial networks (DCGANs). The proposed model is trained on a classical music dataset and a dataset of impressionist paintings. The model can be applied to transfer impressionist paintings into classical music with two tracks. By using music evaluation methods, the harmonicity of the generated music can be confirmed. Our model is the first attempt of our knowledge at transforming paintings into music segments.

This paper presents an online dynamic hand gesture recognition system with an RGB-D camera, which can automatically recognize hand gestures against complicated background. For background subtraction, we use a model-based method to perform human detection and segmentation in the depth map. Since a robust hand tracking approach is crucial for the performance of hand gesture recognition, our system uses both color information and depth information in the process of hand tracking. To extract spatio-temporal hand gesture sequences in the trajectory, a reliable gesture spotting scheme with detection on change of static postures is proposed. Then discrete HMMs with Left-Right Banded (LRB) topology are utilized to model and classify gestures based on multi-feature representation and quantization of the hand gesture sequences. Experimental evaluations on two self-built databases of dynamic hand gestures show the effectiveness of the proposed system. Furthermore, we develop a human-robot interactive system, and the performance of this system is demonstrated through interactive experiments in the dynamic environment.

Monocular cameras are not generally considered as a 3D measurement tool in the field of intelligent transportations. This belief limits the application of monocular cameras in many applications such as guiding robots or vehicles for autonomous driving. In this paper, we demonstrate that with the reference of a license plate, a monocular camera can be used as a tool to measure 3 dimensional parameters (width, height and length) of a passenger vehicle down to 10 cm accuracy. The proposed measurement method does not reply on any training process, but is designed as a fast and direct approach utilizing a series of fundamental computer vision techniques such as ROI extraction, edge/line detection, space coordinates transformations, etc. A series of validation tests were conducted and the results demonstrated the robustness of this measurement method and implies the possibility to broaden the application scope of monocular cameras in the field of autonomous driving and intelligent transportation systems.

Localization and navigation are fundamental issues to autonomous mobile robotics. In the case of the environmental map has been built, the traditional two-dimensional (2D) lidar localization and navigation system can't be matched to the initial position of the robot in dynamic environment and will be unreliable when kidnapping occurs. Moreover, it relies on high-cost lidar for high accuracy and long range. In view of this, the paper presents a low cost navigation system based on a low cost lidar and a cheap webcam. In this approach, 2D-codes are attached to the ceiling, to provide reference points to aid the indoor robot localization. The mobile robot is equipped with webcam pointing to the ceiling to identify 2D-codes. On the other hand, a low-cost 2D laser scanner is applied to build a map in unknown environment and detect obstacles. Adaptive Monte Carlo Localization (AMCL) is implements for lidar positioning, A* and Dynamic Window Approach (DWA) are applied in path planning based on a 2D grid map. The error analysis and experiments has validated the proposed method.

In this paper, we study the parameter setting for a set of intelligent multi-category classifiers in wall-following robot navigation. Based on the swarm optimization theory, a particle selecting approach is proposed to search for the optimal parameters, a key property of this set of multi-category classifiers. By utilizing the particle swarm search, it is able to obtain higher classification accuracy with significant savings on the training time compared to the conventional grid search. For wall-following robot navigation, the best accuracy (98.8%) is achieved by the particle swarm search with only 1/4 of the training time by the grid search. Through communicating the social information available in particle swarms in the training process, classification-based learning can achieve higher classification accuracy without prematurity. One of such learning classifiers has been implemented in SIAT mobile robot. Experimental results validate the proposed search scheme for optimal parameter settings.

This paper proposes a global path- and motion-planning algorithm that enables inchworm-like robots to navigate their way up tree branches. The intuitive climbing space representation method proposed here greatly simplifies the path-planning problem. The dynamic programming algorithm can be used to identify the optimal path leading to the target position in the target direction according to the constraints and requirements specified. The planned path can be applied in any tree-climbing robot that utilizes the nonenclosure gripping method. An efficient motion-planning algorithm for continuum inchworm-like robots is then developed to enable them to climb along the planned path with a high degree of accuracy. In comparison with the method proposed in our previous study, the method proposed herein significantly improves consistency between the planned path and the motions of the robot, and therefore makes it more practical to implement the motion-planning algorithm in trees of different shapes. The paper also describes hardware experiments in which the proposed planning algorithm is applied to enable inchworm-like robots to climb real trees, thus validating the proposed planning algorithm in practice. © 2012 Wiley Periodicals, Inc. © 2013 Wiley Periodicals, Inc.

Abnormal crowd behavior detection plays an important role in surveillance applications. We propose a camera parameter independent and perspective distortion invariant approach to detect two types of abnormal crowd behavior. The two typical abnormal activities are people gathering and running. Since people counting is necessary for detecting the abnormal crowd behavior, we present an potential energy-based model to estimate the number of people in public scenes. Building histograms on the X- and Y-axes, respectively, we can obtain probability distribution of the foreground object and then define crowd entropy. We define the Crowd Distribution Index by combining the people counting results with crowd entropy to represent the spatial distribution of crowd. We set a threshold on Crowd Distribution Index to detect people gathering. To detect people running, the kinetic energy is determined by computation of optical flow and Crowd Distribution Index. With a threshold, kinetic energy can be used to detect people running. To test the performance of our algorithm, videos of different scenes and different crowd densities are used in the experiments. Without camera calibration and training data, our method can robustly detect abnormal behaviors with low computation load.

In this paper, we propose an autonomous tree climbing strategy for a novel tree climbing robot that is named Treebot. The proposed algorithm aims to guide Treebot in climbing along an optimal path by the use of minimal sensing resources. Inspired by inchworms, the algorithm reconstructs the shape of a tree simply by the use of tactile sensors. It reveals how the realization of an environment can be achieved with limited tactile information. An efficient nonholonomic motion planning strategy is also proposed to make Treebot climb on an optimal path. This is accomplished by the prediction of the future shape of the tree. The study that is presented in this paper also includes the formulation of Treebot kinematics and an analysis of the workspace of Treebot on different shapes of a tree. Numerous experiments have been conducted to evaluate the proposed autonomous climbing algorithm and to unveil the ability of Treebot.

Early automatic detection of cardiovascular diseases is of great importance to provide timely treatment and reduce fatality rate. Although many efforts have been devoted to detecting various arrhythmias, classification of other common cardiovascular diseases still lacks comprehensive and intensive studies. This work aims at developing an automatic diagnosis system for myocardial infarction, valvular heart disease, cardiomyopathy, hypertrophy, and bundle branch block, based on the clinic recordings provided by PTB Database. The proposed diagnosis system consists of the components as baseline wander reduction, beat segmentation, feature extraction, feature reduction and classification. The selected features are the location, amplitude and width of each wave, exactly the parameters of ECG dynamical model. We also propose a mean shift algorithm based method to extract these features. To demonstrate the availability and efficacy of the proposed system, we use a total of 13,564 beats to conduct a large scale experiment, where only 25% beats are utilized to train the eigenvectors of generalized discriminant analysis in the feature reduction phase and 25% beats are applied to train the support vector machine in the classification phase. The average sensitivity, specificity and positive predicitivity for the test set, containing 75% beats, are respectively 96.06%, 99.32% and 97.29%.

Denoising of electrocardiogram (ECG) is the fundamental technique for manual or automatic ECG diagnosis. Model-based denoising has attracted initial studies since the ECG dynamical model was established in 2003 and been demonstrated to outperform most model-less denoising methods. The focus of this paper is robust denoising of abnormal ECG signals, which do not satisfy the assumption in previous model-based studies that morphological or physiological variations are small from one beat to another. A mean shift based initializer is proposed to provide a much more robust estimation of initial model parameters for each heart beat. Together with physiological knowledge based wave sub-segmentation and enhanced strategies, the novel initializer has been demonstrated to achieve satisfactory performance for both normal and abnormal heart beats under both white and pink noises. Utilizing records from Massachusetts Institute of Technology (MIT)-Beth Israel Hospital (BIH) database, this paper also applies various filters to denoise noisy signals and the denoising performances verify the availability and efficacy of the proposed denoising method.

SUMMARYThe technologies of autonomous rendezvous and robotic capturing of non-cooperative targets are very crucial for the future on-orbital service. In this paper, we proposed a method to achieve this aim. Three problems were addressed: the target recognition and pose (position and attitude) measurement based on the stereo vision, the guidance, navigation and control (GNC) of the chaser, and the coordinated plan and control of space robot (CP&C). The pose measurement algorithm includes image filtering, edge detection, line extraction, stereo match and pose computing, et al. Based on the measured values, a certain GNC algorithm was designed for the chaser to approach and rendezvous with the target. Then the CP&C algorithm, which is proved to be advantageous over the traditional separated method, was used to plan and track the trajectories of the base pose and the joint angle. At last, a 3D simulation system was developed to evaluate the proposed method. Simulation results verified the corresponding algorithms.

SUMMARYSpace robotic systems are expected to play an increasingly important role in the future. Unlike on the earth, space operations require the ability to work in the unstructured environment. Some autonomous behaviors are necessary to perform complex and difficult tasks in space. This level of autonomy relies not only on vision, force, torque, and tactile sensors, but also the advanced planning and decision capabilities. In this paper, the authors study the autonomous target capturing from the issues of theory and experiments. Firstly, we deduce the kinematic and dynamic equations of space robotic system. Secondly, the visual measurement model of hand–eye camera is created, and the image processing algorithms to extract the target features are introduced. Thirdly, we propose an autonomous trajectory planning method, directly using the 2D image features. The method predicts the target motion, plans the end-effector's velocities and solves the inverse kinematic equations using practical approach to avoid the dynamic singularities. At last, numeric simulation and experiment results are given. The ground experiment system is set up based on the concept of dynamic simulation and kinematic equivalence. With the system, the experiments of autonomous capturing a target by a free-floating space robot, composed of a 6-DOF manipulator and a satellite as its base, are conducted, and the results validate the proposed algorithm.

For the aging population, surveillance in household environments has become more and more important. In this paper, we present a household robot that can detect abnormal events by utilizing video and audio information. In our approach, moving targets can be detected by the robot using a passive acoustic location device. The robot then tracks the targets by employing a particle filter algorithm. To adapt to different lighting conditions, the target model is updated regularly based on an update mechanism. To ensure robust tracking, the robot detects abnormal human behavior by tracking the upper body of a person. For audio surveillance, Mel frequency cepstral coefficients (MFCC) is used to extract features from audio information. Those features are input to a support vector machine classifier for analysis. Experimental results show that the robot can detect abnormal behavior such as "falling down" and "running". Also, a 88.17% accuracy rate is achieved in the detection of abnormal audio information like "crying", "groan", and "gun shooting". To lower the false alarms by abnormal sound detection system, the passive acoustic location device directs the robot to the scene where abnormal events occur and the robot can employ its camera to further confirm the occurrence of the events. At last, the robot will send the image captured by the robot to the mobile phone of master.

In this paper, we discuss the problem of how human control strategy can be represented as a parametric model using a support vector machine (SVM) and how an SVM-based controller can be used in controlling dynamically stable systems. The SVM approach has been implemented in the balance control of Gyrover, which is a dynamically stable, statically unstable, single-wheel mobile robot. The experimental results that compare SVM with general artificial neural network approaches clearly demonstrate the superiority of the SVM approach with regard to human control strategy learning.

This paper introduces a face detection, classification and counting system that is robust and works in real-time. It tracks multiple people, which is useful for face counting. The classification problem is defined as differentiating and then classifying the front of a face into Asian or non-Asian categories. The first step, is principal component analysis (PCA) for feature generation and independent component analysis (ICA) for feature extraction. Then, we employ support vector machine (SVM) for the training process and combine different SVM classifiers to create new classifiers, which improves the classification rate. Based on this, we can count the number of Asians and non-Asians. Experiments show that our system achieves a classification rate of 82.5% based on a database containing 750 face images from FERET.

Space robotic systems are expected to play an increasingly important role in the future. The robotic on-orbital service, whose key is the capturing technology, becomes research hot in recent years. In this paper, the authors propose an autonomous path planning method for target capturing. The task is described in Cartesian space and it can drive the manipulator to approach the target along the closest path. Firstly, the target feature is extracted based on the measured information via the hand-eye camera, and the target pose (position and orientation) and velocities (linear velocity and angular velocity) are estimated using Kalman filtering technology. Then, a numerically feasible approach is presented to plan the manipulator motion and avoid the dynamic singularities, which are transformed into real-time kinematic singularities avoiding problem. Thirdly, the potential disturbance on the base due to the manipulator's motion is estimated, and the joint rates are autonomously adjusted to reduce the disturbance if it is beyond the allowed bound. At last, a ground experiment system is set up based on the concept of dynamic emulation and kinematic equivalence. With the experiment system, the autonomous target capturing experiments are conducted. The experiment results validate the proposed algorithm.

Detecting human abnormal behaviors in a crowded situation is a challenging problem for public security departments. Human abnormal behaviors are seen either in an individual or in a crowd. This paper is focused on the latter, which is more crucial at some important spots and has been less studied. To achieve this goal, we define two categories of video energy based on intensity variation and motion features and adopt two surveillance methods for the two energy accordingly. Using wavelet analysis of the energy curves, we have obtained a result which shows that both methods can be used to deal with crowd modeling and real-time surveillance satisfactorily. A comparison of the two methods is then made in the actual environment in a metro surveillance system.

This paper presents a method for modeling abnormal human gait using hidden Markov model under the framework of a shoe-integrated system. The intelligent system focuses on modeling the following patterns: normal gait, toe in abnormality, and toe out abnormality. In the developed prototype, an inertial measurement unit (IMU) consisting of three-dimensional gyroscopes and accelerometers is employed to measure the angular velocities and accelerations of the foot. Four force sensing resistors (FSRs) and one bend sensor are arranged on the insole of each foot for force and flexion information acquisition. The proposed method is mainly based on supervised Principal Component Analysis (SPCA) for feature generation and hidden Markov model (HMM) for multi-pattern modeling. The "similarity distance measure" criterion is introduced to do model-to-model evaluation. Experimental results demonstrate the proposed approach is robust and efficient in detecting abnormal gait patterns. Our goal is to provide a cost-effective system for detecting gait abnormalities in order to assist persons with abnormal gaits in developing a normal walking pattern in their daily life.

Coded structured light is an active approach that obtains the shape of object. In our previous work, we proposed a new binary projection mechanism for inspecting semiconductor products. The mechanism consists of only a single light source in combination with a binary grating for projecting binary pattern. By shifting the binary grating in space and in every shifting taking a separate image of the illuminated surface, each position on the illuminated surface is attached with a unique binary string for correspondence. In the mechanism the inspection speed is governed by the number of needed images which also equals to the number of shiftings of the grating. To reduce image number, we propose that two neighbor bits are combined together to produce unique codeword, which is referred to as bit-pairing mechanism. This paper addresses how the spatial shifting of bit-pairing mechanism can be designed optimally for minimizing this number for faster inspection speed. An optimal solution to shifting strategy optimization is proposed that is applicable to any given binary patterns. Theoretical analysis and real image experiments are presented to illustrate the workability of the solutions.

Human gait is a dynamic biometrical feature which is complex and difficult to imitate. It is unique and more secure than static features such as passwords, fingerprints and facial features. In this paper, we present intelligent shoes for human identification based on human gait modeling and similarity evaluation with hidden Markov models (HMMs). Firstly we describe the intelligent shoe system for collecting human dynamic gait performance. Using the proposed machine learning method hidden Markov models, an individual wearer's gait model is derived and we then demonstrate the procedure for recognizing different wearers by analyzing the corresponding models. Next, we define a hidden-Markov-model-based similarity measure which allows us to evaluate resultant learning models. With the most likely performance criterion, it will help us to derive the similarity of individual behavior and its corresponding model. By utilizing human gait modeling and similarity evaluation based on hidden Markov models, the proposed method has produced satisfactory results for human identification during testing.

With the emerging need for household applications, service robotics has become a hot spot attracting researchers globally. Universities, institutes, and industrial companies, either independently or collaboratively, have been focusing on research and development for household service robots, resulting in interesting and well-performing household service robotic systems. In this chapter, we first give a categorization of the various applications, followed by an elaboration of some highlighted cases with their overall system designs.

Mapping, localization, and navigation are functions necessitated by a diverse range of household service robot solutions. This chapter presents a wide variety of mapping, localization, and navigation techniques that suit a wide range of robot platforms and applications. The problem of mapping and localization has been approached in many ways and we review successful results attained by traditional radio tracking, ultra wideband sensing, RFID communication, North-Star infrared sensing, two-dimensional special pattern detection, laser-range surveying, various camera system sensing, time-of-flight camera ranging, and finally low-cost contact and near-field sensing. The problem of navigation, path planning, and obstacle avoidance requires not only sensing described above but adequate robot responses in a dynamic situation. There are many approaches to successful obstacle avoidance and our review of the literature covers the use of structured light as well as linear and planar sensors, along with visual processing techniques such as optic flow, ground plane segmentation, and feature-based obstacle avoidance, in addition to the optimization of path planning, robot interaction, and behavior.

Household service is one of the most important fields in robot applications. Compared with applications in industry, the household environment is more challenging. First, the household environment is usually unstructured, which makes it more difficult to handle simple tasks such as what the assembly line robot can easily achieve. Second, the object for household services is human, sometimes a small child. For safety reasons, the robot should be very careful when taking actions. Third, the household robot is expected to be cleverer in dealing with complicated situations, which requires flexibility in responding. All these challenges are, to some extent, related to the capability of perceiving various objects, from an old table to a sleeping baby. In this chapter, we discuss the state of the art in object recognition for household service.

Robotic grasping and manipulation comprise three subtasks, i.e., target detection, planning, and control. The robot first needs to detect the target. Here “detect” means to find (or recognize) the target and locate it. The target could be the object to be manipulated (e.g., electrical plugs) or the destination of the object (electrical sockets). Then the robot approaches the target in an optimized trajectory. In the process of approaching and grasping, the robot should be controlled to perform the task correctly and smoothly. In this chapter, we review the approaches to accomplish these three subtasks, i.e., target detection, planning, and control.

In this chapter, an efficient and low-cost cellphone-commandable mobile manipulation system is described. Aiming at home use and elderly caring, this system can be easily commanded through a common cellphone network to efficiently grasp objects in a household environment, utilizing several low-cost off-the-shelf devices. Unlike the visual servo technology using a high quality vision system with the associated high cost, the household-service robot would not be able to afford such a high quality vision servo system, and thus it is essential to use some low-cost devices. However, it is extremely challenging to create such a vision system with precise localization, as well as motion control. To tackle this challenge, we developed a real-time vision system with which a reliable grasping algorithm combining machine vision, robotic kinematics and motor control technology is presented. After the target is captured by the arm camera, the arm camera keeps tracking the target while the arm keeps stretching until the end effector reaches the target. However, if the target is not captured by the arm camera, the arm will make a move to help the arm camera capture the target under the guidance of the head camera. This algorithm is implemented on two robot systems: one with a fixed base and another with a mobile base. The results demonstrate the feasibility and efficiency of the algorithm and system we developed, and our study shows the significance of developing a service robot in a modern household environment.

For an aging population, surveillance in household environments has become more and more important. In this chapter, we present a household robot that can detect abnormal events by utilizing video and audio information. In our approach, moving targets can be detected by the robot using a passive acoustic location device. The robot then tracks the targets by employing a particle filter algorithm. To adapt to different lighting conditions, the target model is updated regularly based on an update mechanism. To ensure robust tracking, the robot detects abnormal human behavior by tracking the upper body of a person. For audio surveillance, Mel frequency cepstral coefficients is used to extract features from audio information. Those features are input to a support vector machine classifier for analysis. Experimental results show that the robot can detect abnormal behavior such as “falling down” and “running.” Also, an 88.17% accuracy rate is achieved in the detection of abnormal audio information like “crying,” “groaning,” and “gun shooting.” To lower the false alarms by the abnormal sound detection system, the passive acoustic location device directs the robot to the scene where the abnormal events are occuring and the robot can employ its camera to further confirm the occurrence of the events. At last, the robot will send the image captured by the robot to the mobile phone of its master.

With the significant development of robotic technologies, the applications of robots have been extended broadly from their traditional industry role to military, medical field, and even daily services. Robotics has gradually become a very large industry with tremendous influence on the sustainable development of the world.

Technology for the capturing of space target is very important for on-orbit servicing. In order to assure the task is accomplished successfully, ground experimentations are required for the verification of the planning and control algorithms of space robotic system before it is launched. In this paper, an experiment concept is used, which is a hybrid approach, i.e. it combines the mathematical model with the physical model. The key issues of the concept are dynamic emulation and kinematic equivalence, in which the behaviors of the space robotic system are calculated by its dynamic equations. The motion of its end-effector and the space target is realized by two industrial robots. According to different observation spots, two modes of capturing process are emulated: one is observed from the inertial frame, the other is from the space base. Based on the concept proposed above, a ground experiment system is set up, which is composed of two industrial robots, a set of global visual system and five industrial computers. Using the system, algorithms of space robot of any geometry and mass properties can be tested. As an example, the autonomous trajectory planning algorithm is verified by the experiment of capturing a moving target. Moreover, a real-time 3D simulation system is developed to emulate the capturing process in 3D space. Numeric simulation and experiment results show that the ground system is effective in evaluating the planning and control algorithms of space robot.

In practice, the environments in which mobile robots operate are usually modeled in highly complex forms and, as a result, autonomous navigation and localization can be difficult. The difficulties are exacerbated for practical robots with limited on-board computational resources and complex planning algorithms, since this paradigm of environmental modeling requires enormous computational power. A novel navigation/localization learning methodology is presented to abstract and transfer the human sequential navigational skill to a robotic wheelchair by showing the platform how to respond in different local environments along a demonstrated, static cluttered route using a lookup table representation. This method utilizes limited on-board range sensing information to concisely model local unstructured environments, with respect to the robot, for navigation or localization along the learned route in order to achieve good performance with low on-line computational demand and low-cost hardware requirements. Experimental study demonstrates the feasibility of this method and some interesting characteristics of navigation, localization, and environmental modeling problems. Analysis is also conducted to investigate performance evaluation, advantages of the approach, choices of lookup table inputs and outputs, and potential generalization of this paper

Localization is one of the key research topics in robotics. Although computer vision based methodologies such as Augmented Markers provides a realtime localization approach, the sensing range and localization accuracy are the main limitations for applications. In this paper, we proposed the use of a problem solving tool, TRIZ to overcome these limitations. Among the tools provided by TRIZ, we illustrated the selection of principles for problem solving. To enhance the sensing range, a novel AR marker is presented. To improve the localization accuracy, we presented the use of an AR polyhedron for system localization. With the use of an AR cube for illustration, we further developed a new optimization strategy. It is a weighted sum method which aims to optimize the position of the AR cube by considering the orientation of different AR markers captured by a camera. To evaluate the proposed methods, different experiments have been conducted including static and dynamic localizations. Results show that the proposed solutions and strategy can significantly improve the sensing range and accuracy of the conventional AR localization system.

This paper introduces a real-time video surveillance system which can track people and detect human abnormal behaviors. In the blob detection part, an optical flow algorithm for crowd environment is studied experimentally and a comparison study with respect to traditional subtraction approach is carried out. The different approaches in segmentation and tracking enable the system to track persons when they change movement unpredictably in occlusion. We developed two methods for the human abnormal behavior analysis. The first one employs Principal Component Analysis for feature selection and Support Vector Machine for classification of human behaviors. The proposed feature selection method is based on the border information of four consecutive blobs. The second approach computes optical flow to obtain the velocity of each pixel for determining whether a human behavior is normal or not. Both algorithms are successfully developed in crowded environments to detect the following human abnormal behaviors: (1) Running people in a crowded environment; (2) falling down movement while most are walking or standing; (3) a person carrying an abnormal bar in a square; (4) a person waving hand in the crowd. Experimental results demonstrate these two methods are robust in detecting human abnormal behaviors.

Surveillance of public places has become a worldwide concern in recent years. The ability to identify abnormal human behaviors in real-time is fundamental to the success of intelligent surveillance systems. The recognition of abnormal and suspicious human walking patterns is an important step towards the achievement of this goal. In this research, we develop an intelligent visual surveillance system that can classify normal and abnormal human walking trajectories in outdoor environments by learning from demonstration. The system takes into account both the local and global characteristics of the observed trajectories and is able to identify their normality in real-time. By utilizing support vector learning and a similarity measure based on hidden Markov models, the developed system has produced satisfactory results on real-life data during testing. Moreover, we utilize the approach of longest common subsequence (LCSS) in determining the similarity between different types of walking trajectories. In order to establish the position and speed boundaries required for the similarity measure, we compare the performance of a number of approaches, including fixed boundary values, variable boundary values, learning boundary by support vector regression, and learning boundary by cascade neural networks.

Sensor signals produced in industrial manufacturing processes contain valuable information about the condition of operations. However, extracting the appropriate feature for effective fault diagnosis is difficult. Moreover, the adaptability and flexibility of current fault diagnosis systems are often found wanting in real-world applications. Unfortunately, it is essential to rebuild most fault diagnosis systems when new fault types emerge. This paper presents an intelligent fault diagnosis system based on a hidden Markov model. Introducing the concepts of time marginal energy and frequency marginal energy, the features of which can be acquired by the wavelet packet technique satisfy the requirements for fault diagnosis. By utilizing the best tree principle, this method not only extracts the feature automatically without a priori experience but also compresses the data; both of which ensure a system that is practical for real-time application. The new diagnosis system developed here is efficient and effective, as demonstrated by the model developed and applied to a real-time sheet metal stamping process. Based on tests conducted during two experiments (one based on simple blanking, the other on progressive operations) and related comparisons, the proposed method is substantially more effective than other approaches. In addition, the new method requires that only related models be created for new fault types, which results show are ideal for shop floor applications.

In this research, computational intelligence techniques are applied towards the modeling of human sensations in virtual environments. We specifically focus on the following important questions: (1) how to efficiently model the relationship between human sensations and the physical stimuli presented to humans, (2) how to validate the human sensation models, and (3) how to reduce the size of the input data when it gets large and how to select the information which is most important to human sensation modeling. In order to provide an experimental testbed for the implementation of the proposed learning and analysis techniques, a full-body motion virtual reality interface capable of recording human sensations is developed. We propose using cascade neural networks with node-decoupled extended Kalman filter training for modeling human sensation in virtual environments. For the purpose of sensation model validation, we propose using a stochastic similarity measure based on hidden Markov models to calculate the relative similarity between model-generated sensation and actual human sensation. Next, we investigate a number of feature extraction and input selection techniques for reducing the input data size in human sensation modeling. We propose and develop a new input selection method based on independent component analysis, which is capable of reducing the data size and selecting the stimuli information that is most important to the human sensation.

This paper presents a new method for fault diagnosis using a newly developed method, support vector machine (SVM). First, the basic theory of the SVM is briefly reviewed. Next, a fast implementation algorithm is given. Then the method is applied for the fault diagnosis in sheet metal stamping processes. According to the tests on two different examples, one is a simple blanking and the other is a progressive operation, the new method is very effective. In both cases, its success rate is over 96.5%. In comparison, the success rate of the popular artificial neural network (ANN) is just 93.3%. In addition, the new method requires only few training samples, which is an attractive feature for shop floor applications.

Modeling human control strategy (HCS) is becoming an increasingly popular paradigm in a number of different research areas, ranging from robotics and intelligent vehicle highway systems to expert training and virtual reality computer games. Usually, HCS models are derived empirically, rather than analytically, from real-time human input–output data. While these empirical models offer an effective means of transferring intelligent behaviors from humans to robots and other machines, there is a great need to develop adequate performance criteria for these models. It is our goal in this paper to develop several such criteria for the task of human driving. We first collect driving data from different individuals through a real-time graphic driving simulator that we have developed, and identify each individual’s control strategy model through the flexible cascade neural network learning architecture. We then define performance measures for evaluating two aspects of the resultant HCS models. The first is based on event analysis, while the second is based on inherent analysis. Using the proposed performance criteria, we demonstrate the procedure for evaluating the relative skill of different HCS models. Finally, we propose an iterative algorithm for optimizing an initially stable HCS model with respect to independent, user-specified performance criteria, by applying the simultaneously perturbed stochastic approximation (SPSA) algorithm. The methods proposed herein offer a means for modeling and transferring HCS in response to real-time inputs, and improving the intelligent behaviors of artificial machines.

Owing to the ever-increased demand for product quality improvement and production cost reduction, on-line monitoring of stamping operations has become a common practice in shop floors around the world today. Based on a market survey, for most stamping processes, monitoring systems use tonnage signals and/or strain signals. Recently, a few attempts have been reported using acceleration signals as they contain rich information and are relatively inexpensive. However, it is known that acceleration signals are vulnerable to the noise disturbances, and hence are less robust. This paper presents a study that uses bispectrum to analyze the acceleration signals. It is shown that the bispectrum can suppress Gaussian color noise to boost the signal-to-noise ratio. It also extracts the features of the signal that are related to the defective parts (such as material too thick or slug). The experimental results demonstrate that the method presented is effective and has a good potential for applications in shop floor. We also present new method for reducing the computation load in the process.

The fault diagnosis is important in continuously monitoring the performance and quality of manufacturing processes. Overcoming the drawbacks of threshold approach, artificial neural network may extract the symptom of the faults through learning from the samples, but it is difficult to design its structure. Moreover, it needs a large numbers of samples in practice. In this paper, support vector machine approach was proposed to overcome these limitations based on statistics learning theory, and a new fault diagnosis system is developed. The experimental results showed that it is an efficient and practical on-line intelligent monitoring system for the stamping processes.

Localization of an autonomous mobile robot has always been a topic of interest in Robotics community. The constraints of working under different environments are constantly a challenge in designing an efficient localization system. We propose a localization algorithm for a mobile robot, which will be used for home security. The objective is to perform indoor localization efficiently and keeping the cost as low as possible so as to make it commercializable. Here, we introduce an approach based on recognizing a pattern generated by laser pointers on the wall, which is captured by an infrared camera. The image of this pattern is used to locate the position of a robot in the room. The problem of working under black ceiling has been overcomed because the projection is on the wall and is moving which also increases the range of operation. As the pattern is unique, the interference of strong sunlight has also been fixed to some extent. Experimental results show that the proposed approach is effective in real environment and opens new doors for future research.

This paper presents a novel shape representation approach, Finite Element Skeleton (FES), based on studies of planar shape analysis from the perspective of Finite Element Method (FEM). First, we derive the shape contour of an object in image with edge detection approach. Then, the approach builds connections of all pixels on the shape contour with the inner center point of the shape, and models these virtual connections as bendable beam elements. Finally, we obtain a multi-degree-of-freedom (MDOF) model of the original shape. By this model, we can calculate its natural frequency, which is one of mechanical properties that directly related to the geometric shape, and employed it as the shape representation. FES shape descriptor possesses the natural properties on translation and rotation invariant. Especially, FES descriptor has the feature of articulation insensitivity which is the most important property of a shape descriptor to ensure a good performance of the intra-class similarity measurement. Experimental results validated that the proposed FES descriptor is capable of identifying shape for object recognition.

The human behavior, such as gait, is controlled by the brain and other peripheral neural system, and thus is significantly affected by the internal emotion. It is important to carry out the affective sensation of abnormal internal emotion in a non-invasive and non-privacy-infringed way. This paper presents an experiment method and platform to sense the abnormal affective status through gaits. We used a wearable intelligent shoe system for affective sensing, designed an experiment for the human subject to have the normal gait and the gait with emotional disorder, and finally classified the gaits through machine learning.

In this paper, we present a method for modeling human strategies in controlling a light source in a dynamic environment. In order to illustrate the procedure and method, we take a simple example of how to control a light source so as to avoid a shadow and to maintain appropriate illumination conditions on the target area of attention. This work is likely to be valuable in various applications of automatic light control-from surgical rooms and space applications to inspections.

In this paper, the authors present a stochastic,discontinuous modeling framework that is designed for abstracting human control strategies and is based on hidden Markov models (HMM). A real-time driving simulator is developed for investigating human control strategies. The shortcomings of a typical continuous modeling approach for modeling discontinuous human control strategies are discussed. The HMM-based method is shown to demonstrate greater fidelity to the human training data.

Combining functions with aesthetics, characters with drawings, Chinese calligraphy is a unique type of traditional art in China. To mimic the special writing skills, we develop a calligraphy robot which consists of one 6-DOF robot arm, one linear rail and one paper conveyor. This calligraphy robot, called Callibot, has several features. Firstly, it has sufficient degrees of freedom and very few singularities which improve the manipulability of the robot. Secondly, Callibot could mimic real motion of human writing because playback from demonstration approach is adopted. The encoders will record the joint positions while demonstrating. Then Callibot filters out the noises (error readings in serial port communication) and repeats the motion. Thirdly, the workspace of the robot arm is largely expanded due to the linear rail and the paper conveyor. The experimental results show that Callibot is capable of writing a large piece of aesthetic calligraphic work.