Search

Recurrent neural network (RNN) has become popular for human motion prediction thanks to its ability to capture temporal dependencies. However, it has limited capacity in modeling the complex spatial relationship in the human skeletal structure. In this work, we present a novel diffusion convolutional recurrent predictor for spatial and temporal movement forecasting, with multi-step random walks traversing bidirectionally along an adaptive graph to model interdependency among body joints. In the temporal domain, existing methods rely on a single forward predictor with the produced motion deflecting to the drift route, which leads to error accumulations over time. We propose to supplement the forward predictor with a forward discriminator to alleviate such motion drift in the long term under adversarial training. The solution is further enhanced by a backward predictor and a backward discriminator to effectively reduce the error, such that the system can also look into the past to improve the prediction at early frames. The two-way spatial diffusion convolutions and two-way temporal predictors together form a quadruple network.\ud \ud Furthermore, we train our framework by modeling the velocity from observed motion dynamics instead of static poses to predict future movements that effectively reduces the discontinuity problem at early prediction. Our method outperforms the state of the arts on both 3D and 2D datasets, including the Human3.6M, CMU Motion Capture and Penn Action datasets. The results also show that our method correctly predicts both high-dynamic and low-dynamic moving trends with less motion drift.

The target classification from synthetic aperture radar (SAR) imagery is entering a bottleneck stage for the extensive use of a deep learning technology. Researchers have deployed various deep neural networks to extract the target features from the original SAR image in Euclidean space, which requires a large number of training data and cost lots of time to train the deep neural networks well generalized. Aiming at this problem, this letter introduces a novel method of target classification from SAR imagery based on the target pixel grayscale decline by a graph representation, which is different from the conventional deep learning methods so far. We separate the whole grayscale interval of one SAR image into several subintervals and assign a node to represent each pixel with the declined order of pixel grayscale in the subinterval. Then, a graph structure could be constructed to transform the raw SAR image from Euclidean data to graph-structured data. Finally, we construct a graph convolutional neural network to extract the features of graph-structured data we constructed previously and output the target classification result. The experiment result on the MSTAR dataset shows that our method achieved the average classification accuracy with 100%, which surpasses all the state-of-the-art methods for the first time in SAR automatic target recognition field.

In this paper, a Chinese character educational system based on Kinect is proposed for guiding beginners to learn the basics of Chinese characters in a more intuitive way. It extracts 19 common components, denoted as alphabets, from Chinese characters. Nineteen postures were designed according to the shapes of these alphabets. Instead of memorizing Chinese characters through repetitive copying, students can first associate an alphabet with the corresponding designed posture. Then, they break down a Chinese character into a set of alphabets in order to perform the sequence of corresponding postures so that they can easily remember the whole character. Our proposed system contains two major functions: 1) the learning function that is responsible for delivering the courseware, and 2) the testing function that is used to let students acquire their learning progress through some tests. A rule-based algorithm is designed to recognize the students’ input postures captured from the Kinect motion sensor so as to determine whether the students have performed the correct postures. We conducted a survey which involved 90 students to try our proposed system as well as two other learning modes for comparison. Moreover, we have interviewed those 30 students who had tried our proposed system with some open-ended questions. The positive results show that the proposed system can promote students’ experiences in learning Chinese characters.

This paper addresses the problem of recognizing human skeletal interactions using multiview data captured from depth sensors. The interactions among people are important cues for group and crowd human behavior analysis. In this paper, we focus on modeling the person–person skeletal interactions for human activity recognition. First, we propose a novel graph model in each single-view case to encode class-specific person–person interaction patterns. Particularly, we model each person–person interaction by an attributed graph, which is designed to preserve the complex spatial structure among skeletal joints according to their activity levels as well as the spatio-temporal joint features. Then, combining the graph models for each single-view case, we propose the multigraph model to characterize each multiview interaction. Finally, we apply a general multiple kernel learning method to determine the optimal kernel weights for the proposed multigraph model while the optimal classifier is jointly learned. We evaluate the proposed approach on the M $^2$ I dataset, the SBU Kinect interaction dataset, and our interaction dataset. The experimental results show that our proposed approach outperforms several existing interaction recognition methods.

The popularity and influence of social media have been continuously expanding worldwide, and a similar trend is visible in educational settings. Social media tools have been reportedly used for a variety of educational purposes and in wide-ranging contexts, bridging formal and informal, as well as individual and collaborative learning. Currently, the trend is toward the integration of social media services with mobile and ubiquitous learning and adaptive educational technologies. To provide an overview of this interplay by reviewing the applicable techniques and relevant works in the area, we supplement this editorial with a tutorial survey. In particular, the following three main topics are examined and studied here: ubiquitous learning via social media services, intelligent tutoring in adaptive e-learning, and multimedia enabled social learning. The authors also introduced the articles featured in this special issue on social media for learning.

In this paper, an automatic lesson generation system is presented which is suitable in a learning-by-mimicking scenario where the learning objects can be represented as multiattribute time series data. The dance is used as an example in this paper to illustrate the idea. Given a dance motion sequence as the input, the proposed lesson generation system automatically generates the lesson plan for students. It first extracts patterns from the input dance sequence to form the learning objects. The prerequisite structure is then built by considering the relations between the learning objects. Afterward the knowledge structure is constructed from the prerequisite structure based on the knowledge space theory. Finally, the learning path is derived according to an easy-to-complex manner while respecting the prerequisite relations. A user study that involved 40 students was conducted to evaluate the proposed work. The average learning time required for the treatment group (learning with the proposed system) was found to be lower than that of the control group (learning by free browsing) thus demonstrating the learning efficiency of the proposed system. The feedback from the questionnaires indicated that a majority of the subjects showed positive response toward the usefulness and rationality of our proposed system.

In this paper, a new dance training system based on the motion capture and virtual reality (VR) technologies is proposed. Our system is inspired by the traditional way to learn new movements-imitating the teacher's movements and listening to the teacher's feedback. A prototype of our proposed system is implemented, in which a student can imitate the motion demonstrated by a virtual teacher projected on the wall screen. Meanwhile, the student's motions will be captured and analyzed by the system based on which feedback is given back to them. The result of user studies showed that our system can successfully guide students to improve their skills. The subjects agreed that the system is interesting and can motivate them to learn.

In this paper, we propose a new method to simulate human gait motion when muscles are weakened. The method is based on the enhanced version of three-dimensional linear inverted pendulum model that is used for generation of gait in robotics. After the normal gait motion is generated by setting the initial posture and the parameters that decide the trajectories of the center of mass and angular momentum, the muscle to be weakened is specified. By minimizing an objective function based on the force exerted by the specified muscle during the motion, the set of parameters that represent the pathological gait was calculated. Since the number of parameters to describe the motion is small in our method, the optimization process converges much more quickly than in previous methods. The effects of weakening the gluteus medialis, the gluteus maximus, and vastus were analyzed. Important similarities were noted when comparing the predicted pendulum motion with data obtained from an actual patient.