Search

Sailing robots can contribute significantly to maritime surface exploration, due to its potential for long-range and long-duration motions in the environment with abundant wind. However, energy, the critical factor for their long-term missions, shall be carefully investigated, so as to achieve sustainability in distance and time. In this survey, we have conducted a comprehensive investigation on numerous sailing robots, developed in academia and industry. Some of them have achieved long-term operation, and some are motivated by, but still on the way to this ambitious goal. Prototypes are grouped in each team, so as to view the development path. We further investigate the existing design and control strategies for energy sufficiency from three perspectives: actuation, harvesting, and energy management. In propulsion and steering, i.e., two major actuations, researchers have accumulated effective sail and rudder designs. The motorized propeller and wave-glider–inspired mechanism also contribute as compliments for propulsion. Electricity harvesting based on solar or wind energies is also discussed to gather more power from nature. Pros and cons in strategies of energy management, which are valuable tools to enhance power utilization efficiency, are elaborated. This article is hoped to provide researchers in long-term robotic sailing with a comprehensive reference from the perspectives of energy.

This paper proposes an interactive system for the non-musician learners to get inspired from a song. Differing from complex models of deep learning or simple Markov models sparse of music inter-features, in this research, we unify the composing of a song in a general architecture with music theory, and thus provide a much more understandable view of the music generation for non-musician learners. The proposed model focuses on extracting the extant feature from a target song and recreating different phrases with the representing probabilistic graph underlying the target song based on the relationship among notes in a phrase. Furthermore, an interactive interface between the users and the proposed system is built with a tunable parameter for them to be involved in the music generation and creating procedure. This procedure provides practical experience in aiding the non-musicians to understand and learn from composing a song. Approximately 700 samples of preferences questionnaire survey about the generated music and original music and more than 3000 samples for interactive preferences voting for the tunable parameter have been collected. Quantities of experiments have proved the validation of the proposed system.

The new era with prosperous artificial intelligence (AI) and robotics technology is reshaping the materials discovery process in a more radical fashion. Here we present authentic intelligent robotics for chemistry (AIR-Chem), integrated with technological innovations in the AI and robotics fields, functionalized with modules including gradient descent-based optimization frameworks, multiple external field modulations, a real-time computer vision (CV) system, and automated guided vehicle (AGV) parts. AIR-Chem is portable and remotely controllable by cloud computing. AIR-Chem can learn the parametric procedures for given targets and carry on laboratory operations in standalone mode, with high reproducibility, precision, and availability for knowledge regeneration. Moreover, an improved nucleation theory of size focusing on inorganic perovskite quantum dots (IPQDs) is theoretically proposed and experimentally testified to by AIR-Chem. This work aims to boost the process of an unmanned chemistry laboratory from the synthesis of chemical materials to the analysis of physical chemical properties, and it provides a vivid demonstration for future chemistry reshaped by AI and robotics technology.

In this paper, we propose a navigation algorithm oriented to multi-agent environment. This algorithm is expressed as a hierarchical framework that contains a Hidden Markov Model (HMM) and a Deep Reinforcement Learning (DRL) structure. For simplification, we term our method Hierarchical Navigation Reinforcement Network (HNRN). In high-level architecture, we train an HMM to evaluate the agents perception to obtain a score. According to this score, adaptive control action will be chosen. While in low-level architecture, two sub-systems are introduced, one is a differential target-driven system, which aims at heading to the target; the other is a collision avoidance DRL system, which is used for avoiding dynamic obstacles. The advantage of this hierarchical structure is decoupling the target-driven and collision avoidance tasks, leading to a faster and more stable model to be trained. The experiments indicate that our algorithm has higher learning efficiency and rate of success than traditional Velocity Obstacle (VO) algorithms or hybrid DRL method.

Personal handwriting style fonts generation is a diverting but time-consuming task due to the large size of Chinese character set. In addition, unlike standard printed style fonts, hand-writing style fonts are of more complicated stroke and glyph feature. In this paper, an improved network architecture is proposed for learning and generation of personal hand-writing style fonts based on small character set. The network is composed of three sub-networks: 1) a classification network for identifying the general style of the target fonts; 2) a generating network for transferring the identified fonts to the target fonts; 3) a discriminating network for differentiating the generated image from real ones. The experiments revealed the effectiveness of the model for generating personal hand-writing style font with relatively small data size, reduction by a scale of 10 comparing to previous reported works.

This paper presents a new model of automatic music generation which is based on simulated annealing and genetic algorithm. Our model uses simulated annealing algorithm to create melody units on different chords, and regards them as the genes in genetic algorithm, which can pick out a fine combination of these genes. This model is efficient for producing massive fair-sounding pieces of double-track music, when provided with a customized metric. Hence, it can be especially useful for massively producing background musics for video games, or providing inspirations for music composers. In this paper, we detailedly state how to transform the prior knowledge about music theory into a reasonable metric, and propose a sample metric for reference. Using this sample metric, we generate a set of music pieces as test cases. The evaluation results show that these pieces fully satisfy the need of quality and creativity.

In this paper, we present an accurate approach to estimate vehicles' pose and shape from off-board multiview images. The images are taken by monocular cameras and have small overlaps. We utilize state-of-the-art convolutional neural networks (CNNs) to extract vehicles' semantic keypoints and introduce a Cross Projection Optimization (CPO) method to estimate the 3D pose. During the iterative CPO process, an adaptive shape adjustment method named Hierarchical Wireframe Constraint (HWC) is implemented to estimate the shape. Our approach is evaluated under both simulated and real-world scenes for performance verification. It's shown that our algorithm outperforms other existing monocular and stereo methods for vehicles' pose and shape estimation. This approach provides a new and robust solution for off-board visual vehicle localization and tracking, which can be applied to massive surveillance camera networks for intelligent transportation., Comment: 8 pages, 8 figures, published in ROBIO 2018

In the field of autonomous driving, abundant researches have been carried out in development of advanced Lidar or vision algorithms for mapping and localization, and some efforts in the field of sensor fusion have been made to combine the advantages of multiple sensor types. However, most works focused on the 2D map construction, which had its own limitation with complex environments. In this paper, a new algorithm based on the fusion of Lidar and Inertial Measurement Unit (IMU) data is developed to construct a 2.5D map. We integrate the height information transformed from IMU sensor with laser scan results to construct a 2.5D occupied grid map. The proposed algorithm is convenient to implement on real-world mobile vehicles, and it is fast and robust. The effectiveness of our algorithm is validated by both simulation and practical experiment.

The challenges in transforming paintings into music is well-known, since the relationship between two kinds of art is unclear. Different composers write different music when the same painting is presented to them. In this paper, a cross-mordality model has been proposed for transforming images into multitrack music based on the framework of deep convolutional generative adversarial networks (DCGANs). The proposed model is trained on a classical music dataset and a dataset of impressionist paintings. The model can be applied to transfer impressionist paintings into classical music with two tracks. By using music evaluation methods, the harmonicity of the generated music can be confirmed. Our model is the first attempt of our knowledge at transforming paintings into music segments.

A vision-based perception system for understanding the outdoor environment has been proposed. This system utilizes and applies the useful information from the binocular cameras of a legged robot for landform — terrain — ground environmental perception. The ground texture recognition can provide information about the material of the ground then a legged robot can choose an appropriate walking pose accordingly. With the terrain perception and obstacle detection, combined with semantic segmentation of the environment, a legged robot can move towards the target with intelligent obstacle avoidance strategy. Based on this vision-based perception system, we propose a vision-based perception framework for outdoor navigation tasks applicable to legged robots. In this framework, environmental modeling and situation assessment will be firstly carried out combined with multi-mode sensor fusion, and then footsteps and local path planning can be generated.

In this paper, we present a novel approach for video-anomaly detection in crowded and complicated scenes. The proposed approach detects anomalies based on a hierarchical activity-pattern discovery framework, comprehensively considering both global and local spatio-temporal contexts. The discovery is a coarse-to-fine learning process with unsupervised methods for automatically constructing normal activity patterns at different levels. A unified anomaly energy function is designed based on these discovered activity patterns to identify the abnormal level of an input motion pattern. We demonstrate the effectiveness of the proposed method on the UCSD anomaly-detection datasets and compare the performance with existing work.

Monocular cameras are not generally considered as a 3D measurement tool in the field of intelligent transportations. This belief limits the application of monocular cameras in many applications such as guiding robots or vehicles for autonomous driving. In this paper, we demonstrate that with the reference of a license plate, a monocular camera can be used as a tool to measure 3 dimensional parameters (width, height and length) of a passenger vehicle down to 10 cm accuracy. The proposed measurement method does not reply on any training process, but is designed as a fast and direct approach utilizing a series of fundamental computer vision techniques such as ROI extraction, edge/line detection, space coordinates transformations, etc. A series of validation tests were conducted and the results demonstrated the robustness of this measurement method and implies the possibility to broaden the application scope of monocular cameras in the field of autonomous driving and intelligent transportation systems.

Localization and navigation are fundamental issues to autonomous mobile robotics. In the case of the environmental map has been built, the traditional two-dimensional (2D) lidar localization and navigation system can't be matched to the initial position of the robot in dynamic environment and will be unreliable when kidnapping occurs. Moreover, it relies on high-cost lidar for high accuracy and long range. In view of this, the paper presents a low cost navigation system based on a low cost lidar and a cheap webcam. In this approach, 2D-codes are attached to the ceiling, to provide reference points to aid the indoor robot localization. The mobile robot is equipped with webcam pointing to the ceiling to identify 2D-codes. On the other hand, a low-cost 2D laser scanner is applied to build a map in unknown environment and detect obstacles. Adaptive Monte Carlo Localization (AMCL) is implements for lidar positioning, A* and Dynamic Window Approach (DWA) are applied in path planning based on a 2D grid map. The error analysis and experiments has validated the proposed method.

In this paper, we discuss the problem of how human control strategy can be represented as a parametric model using a support vector machine (SVM) and how an SVM-based controller can be used in controlling dynamically stable systems. The SVM approach has been implemented in the balance control of Gyrover, which is a dynamically stable, statically unstable, single-wheel mobile robot. The experimental results that compare SVM with general artificial neural network approaches clearly demonstrate the superiority of the SVM approach with regard to human control strategy learning.

This paper introduces a face detection, classification and counting system that is robust and works in real-time. It tracks multiple people, which is useful for face counting. The classification problem is defined as differentiating and then classifying the front of a face into Asian or non-Asian categories. The first step, is principal component analysis (PCA) for feature generation and independent component analysis (ICA) for feature extraction. Then, we employ support vector machine (SVM) for the training process and combine different SVM classifiers to create new classifiers, which improves the classification rate. Based on this, we can count the number of Asians and non-Asians. Experiments show that our system achieves a classification rate of 82.5% based on a database containing 750 face images from FERET.

With the emerging need for household applications, service robotics has become a hot spot attracting researchers globally. Universities, institutes, and industrial companies, either independently or collaboratively, have been focusing on research and development for household service robots, resulting in interesting and well-performing household service robotic systems. In this chapter, we first give a categorization of the various applications, followed by an elaboration of some highlighted cases with their overall system designs.

Mapping, localization, and navigation are functions necessitated by a diverse range of household service robot solutions. This chapter presents a wide variety of mapping, localization, and navigation techniques that suit a wide range of robot platforms and applications. The problem of mapping and localization has been approached in many ways and we review successful results attained by traditional radio tracking, ultra wideband sensing, RFID communication, North-Star infrared sensing, two-dimensional special pattern detection, laser-range surveying, various camera system sensing, time-of-flight camera ranging, and finally low-cost contact and near-field sensing. The problem of navigation, path planning, and obstacle avoidance requires not only sensing described above but adequate robot responses in a dynamic situation. There are many approaches to successful obstacle avoidance and our review of the literature covers the use of structured light as well as linear and planar sensors, along with visual processing techniques such as optic flow, ground plane segmentation, and feature-based obstacle avoidance, in addition to the optimization of path planning, robot interaction, and behavior.

Robotic grasping and manipulation comprise three subtasks, i.e., target detection, planning, and control. The robot first needs to detect the target. Here “detect” means to find (or recognize) the target and locate it. The target could be the object to be manipulated (e.g., electrical plugs) or the destination of the object (electrical sockets). Then the robot approaches the target in an optimized trajectory. In the process of approaching and grasping, the robot should be controlled to perform the task correctly and smoothly. In this chapter, we review the approaches to accomplish these three subtasks, i.e., target detection, planning, and control.

With the significant development of robotic technologies, the applications of robots have been extended broadly from their traditional industry role to military, medical field, and even daily services. Robotics has gradually become a very large industry with tremendous influence on the sustainable development of the world.

This paper introduces a real-time video surveillance system which can track people and detect human abnormal behaviors. In the blob detection part, an optical flow algorithm for crowd environment is studied experimentally and a comparison study with respect to traditional subtraction approach is carried out. The different approaches in segmentation and tracking enable the system to track persons when they change movement unpredictably in occlusion. We developed two methods for the human abnormal behavior analysis. The first one employs Principal Component Analysis for feature selection and Support Vector Machine for classification of human behaviors. The proposed feature selection method is based on the border information of four consecutive blobs. The second approach computes optical flow to obtain the velocity of each pixel for determining whether a human behavior is normal or not. Both algorithms are successfully developed in crowded environments to detect the following human abnormal behaviors: (1) Running people in a crowded environment; (2) falling down movement while most are walking or standing; (3) a person carrying an abnormal bar in a square; (4) a person waving hand in the crowd. Experimental results demonstrate these two methods are robust in detecting human abnormal behaviors.

This paper presents a novel shape representation approach, Finite Element Skeleton (FES), based on studies of planar shape analysis from the perspective of Finite Element Method (FEM). First, we derive the shape contour of an object in image with edge detection approach. Then, the approach builds connections of all pixels on the shape contour with the inner center point of the shape, and models these virtual connections as bendable beam elements. Finally, we obtain a multi-degree-of-freedom (MDOF) model of the original shape. By this model, we can calculate its natural frequency, which is one of mechanical properties that directly related to the geometric shape, and employed it as the shape representation. FES shape descriptor possesses the natural properties on translation and rotation invariant. Especially, FES descriptor has the feature of articulation insensitivity which is the most important property of a shape descriptor to ensure a good performance of the intra-class similarity measurement. Experimental results validated that the proposed FES descriptor is capable of identifying shape for object recognition.

The human behavior, such as gait, is controlled by the brain and other peripheral neural system, and thus is significantly affected by the internal emotion. It is important to carry out the affective sensation of abnormal internal emotion in a non-invasive and non-privacy-infringed way. This paper presents an experiment method and platform to sense the abnormal affective status through gaits. We used a wearable intelligent shoe system for affective sensing, designed an experiment for the human subject to have the normal gait and the gait with emotional disorder, and finally classified the gaits through machine learning.

In this paper ceiling affixed ARToolKitPlus 2D code artificial landmarks are evaluated for purposes of robot localization and navigation. Ceiling affixed codes rarely come in contact with people, equipment or robots, and for this reason they are more likely to stay detectable over a longer period of time. Multi threshold averaging, light gradient compensation and neighbourhood search techniques further enhanced AR-ToolKitPlus performance. Multi threshold averaging collects positioning results at each gray scale threshold level. After all of the threshold levels are analyzed, the related results are averaged into a final result. Light gradient compensation eliminates effects of uneven lighting in the neighbourhood of a 2D code. Neighbourhood search for a 2D code requires fewer computational resources than a global search. Further repeatability improvements are achieved by means of averaging localizations. Localization performance is evaluated at varying distances of the 2D code from the camera. Experimental results show substantial improvements in repeatability and reliability over the baseline ARToolKitPlus performance. Improved performance will allow for realtime 2D code based localization for indoor robot navigation.

Hand posture is a natural and effective human robot interaction way. In this paper, an user-independent hand posture recognition system using depth and color images captured from an RGB-D camera is presented. To recognize hand posture against complicated background conditions, we propose a novel method for automatic and accurate hand posture segmentation which detects the hand with Chamfer matching, tracks the hand with Kalman filter and segments the hand with region growing algorithm only in the depth space. A new hand posture descriptor invariant to scale, shift and in-plane rotation is constructed with the combination of local contour Fourier descriptor and global Bag-of-Features (BoF) descriptor based on Scale Invariance Feature Transform (SIFT). The sparse representation-based classification (SRC) is applied to perform the hand posture recognition task in the system. Experiments with a self-built large scale hand posture database collected online show the robustness and effectiveness of the proposed system.

This paper1 presents a novel shape representation approach, Finite Element Contour (FEC), based on studies of shape analysis from the perspective of Finite Element Method (FEM). We assume that an edge of a contour can be modeled as a bendable beam element. Linking finite number of beam elements end to end along the contour, we obtain a closed-loop Finite Element Contour model as an approximate physical model of the original shape. By this model, we can calculate its natural frequency, which is one of mechanical properties that directly related to the geometric shape, and employed it as the shape representation. FEC shape feature possesses translation and rotation invariant properties naturally. We also realized scale and unique affine normalization in few simple steps based on intrinsic physical properties of shape from FEM viewpoint. Experimental results validated that the proposed FEC feature is capable of identifying shape in object recognition task. It also can describe shape deformation. In the well-known MPEG 7 shape retrieval task, the enhanced FEC approach obtains Bullseye score 87.11%.

This paper presents a novel shape representation approach based on Finite Element Region (FER). We assume an image blob can be treated as a mesh model that consists of a finite number of deformable quadrilateral pixel elements. These elements connecting side by side form an approximate physical model of the surface of real object. By employing Finite Element Method (FEM), the model's natural frequency can be derived, which is one of mechanical properties that only related to geometric shape, as the shape representation. Experimental results validated the proposed FER descriptor possessing good performance on shape representation. It also has translation, rotation invariant properties naturally, and scale and affine normalization can be realized in few steps. Experiments also demonstrated that the FER descriptor is sensitive to shape deformation. It can precisely describe subtle changing processes between different gestures. This property allows us to establish a conceptual graph model of hand gestures changing to improve the performance of ambiguous gesture recognition.

This paper mainly presents the survey of mobility approaches for the Extra Vehicular Robotics (EVR) application in space station. Seven different kinds of mobility approaches are identified, including straight rail mobile approach, inchworm-like movement approach, free flying vehicle/robot with docking approach, handrail climbing approach, mobile base with circular climbing approach and long manipulator transferring approach. The performance and difference of each approach are elaborated carefully, which offers reference for further research.

As one of the most unique types of art in Chinese culture, nowadays Chinese calligraphy is attracting increasing interests from researchers. It will be a big step if we have a robot to write Chinese calligraphy, especially in various styles, and it will form a bridge to combine science with art directly. However, there are so many different types and styles in Chinese calligraphy, and to distinguish them is the most basic quality to a green hand, but it is a big challenge for a robot to do so. For the lack of exploration about this, we conduct a lot of experiments to help the robot to accomplish it automatically. We first propose a parametric representation of calligraphic characters, and then adopt the Mahalanobis distance for similarity measurement and classification. The average accuracies of classifying types and styles of the Chinese calligraphy are 96.36% and 95.61% respectively. During the experiments, some interesting phenomena are discovered through similarity measure. Meanwhile, the parametric representation also has some potential applications, such as defining aesthetic grading standards of calligraphy and synthesizing calligraphy. Based on our research, the calligraphy robot can tell which style of calligraphy it sees for mimicking.

As the most critical function of mobile robots, navigation substantially relies on self-localization. Among all the approaches, vision based localization is widely adopted. Unlike the complex algorithms developed before, we present a simple real-time visual localization approach which regards the circular lamps on the ceiling as the landmarks. Two sets of point correspondences are applied to solving Perspective-n-Point (PnP) problem. One set contains the centers of the ellipses in image and the centers of the circles in 3D. The other set contains the tangent points on the circles and ellipses. At least two lamps are required in the field of view to avoid ambiguities. The experimental results show that no blind zone exists since the required number of lamps is small. The position and orientation errors are small enough for indoor navigation. This approach is validated by experiments for real-time application.

The internet and computer are more popular in the world, and the keyboard is the important input for the computer. However, in our society, many disable people cannot type by using keyboard. In order to assist them, we developed a shoe platform which senses the foot motion and use it as the input for the computer. In this system, we used force sensing resistors (FSRs) and a 3-axis accelerometer to detect foot movements. Based on these sensors, we analyzed human foot motions in different positions, and selected those with more comfortable motions when people sitting in front of the computer with a healthy attitude. To reduce the computational cost and power consumption, and enhance the real-time performance, we use the principle component analysis (PCA) for sensor reduction. Through PCA, we selected some important FSRs in this system. Also we compared two classification methods, chose two layer back propagation(BP) networks, to detect the foot movements for typing.

With the acquisition of an image, the first step is to distinguish objects of interest from the background. In surveillance applications, those objects of interest are usually humans. Their various shapes and different motions, including walking, jumping, bending down, and so forth, represent significant challenges in the extraction of foreground pixels from the image.

An important feature of crowd surveillance is the estimation of the number of persons in the monitored scene. People counting is especially important for crowd modeling as it not only gives a direct estimation of the current situation in the field of view but also contributes to other tasks including behavior analysis and crowd simulation.

In many cases, we cannot obtain the background image easily, and if the occlusion is serious, then it is difficult to segment masses of blobs into single blobs.We therefore use optical flow to detect abnormal behavior in crowded environments [57], [58].

After successful segmentation and tracking, the temporal-spatial information of each blob sequence can be indexed for further behavior analysis. In this chapter, we elaborate two kinds of approaches for the behavior analysis of individuals: learningbased and rule-based.

After foreground extraction, if nonsignificant merging occurs among people, then segmentation is necessary, and the merged blobs can be split into separate ones equal in number to the number of people in the merged blob. Thereafter, all of the blobs can be indexed and tracked for motion and behavior analysis.

In the previous chapter, we presented behavior analysis, which is based on dynamic information, including background extraction based on a series of frames, blob detection based on two consecutive frames, and so forth. However, for intelligent surveillance, we can also explore static information, i.e., facial images. Facial detection is straightforward as it does not depend on whether or not background information is extractable. Hence, it is useful in highly cluttered environments, with the only requirement being sufficient image resolution of the image.

We present an intelligent driver identification system to handle vehicle theft based on modeling dynamic human behaviors. We propose to recognize illegitimate drivers through their driving behaviors. Since human driving behaviors belong to a dynamic biometrical feature which is complex and difficult to imitate compared with static features such as passwords and fingerprints, we find that this novel idea of utilizing human dynamic features for enhanced security application is more effective. In this paper, we first describe our experimental platform for collecting and modeling human driving behaviors. Then we compare fast Fourier transform (FFT), principal component analysis (PCA), and independent component analysis (ICA) for data preprocessing. Using machine learning method of support vector machine (SVM), we derive the individual driving behavior model and we then demonstrate the procedure for recognizing different drivers by analyzing the corresponding models. The experimental results of learning algorithms and evaluation are described.

In this paper, an integrated video surveillance system for robust tracking is introduced. In the blob detection part, an optical flow algorithm for crowded environment is studied experimentally and a comparison study with respect to traditional subtraction approach is carried out. In the segmentation part, different algorithms are fused to develop a hybrid algorithm for stable segmentation, and validation rules for successful segmentation are also presented preventing from false segmentation. In the tracking part, a blob's parameter, which we call color spectrum, is developed to identify different persons and track them robustly. A hybrid algorithm for tracking is also developed to combine color tracking with traditional distance tracking. The hybrid algorithms in segmentation and tracking enable the system to track persons when they change movement unpredictably in occlusion. Experimental results validate the proposed algorithm.

This paper presents the progress toward a face detection and race classification system that is robust and works in real-time. We address the race classification problem as classifying a frontal face into Asian or non-Asian. Firstly, we propose principal component analysis (PCA) for feature generation and independent component analysis (ICA) for feature extraction. Then, we use SVM for training process and combine different SVM classifiers to some new classifiers, which improve the classification rate to a new level. Experiments show that our system achieves a classification rate of 82.5 % based on a database containing 750 face images from FERET.

This paper introduces a real-time video surveillance system which detects human abnormal behaviors. We present two approaches to such a problem. The first one employs principal component analysis for feature selection and support vector machine for classification of human behaviors. The proposed feature selection method is based on the border information of four consecutive blobs. The second approach computes optical flow to obtain the velocity of each pixel for determining whether a human behavior is normal or not. Both algorithms are successfully implemented in crowded environments for detecting the human abnormal behaviors, such as (1) running people in a crowded environment, (2) bending down movement while most are walking or standing, (3) a person carrying a long bar and (4) a person waving hand in the crowd. Experimental results demonstrate the two methods proposed are robust and efficient in detecting human abnormal behaviors.