Search

Creating realistic characters that can react to the users’ or another character’s movement can benefit computer graphics, games and virtual reality hugely. However, synthesizing such reactive motions in human–human interactions is a challenging task due to the many different ways two humans can interact. While there are a number of successful researches in adapting the generative adversarial network (GAN) in synthesizing single human actions, there are very few on modeling human–human interactions. In this paper, we propose a semi-supervised GAN system that synthesizes the reactive motion of a character given the active motion from another character. Our key insights are two-fold. First, to effectively encode the complicated spatial–temporal information of a human motion, we empower the generator with a part-based long short-term memory (LSTM) module, such that the temporal movement of different limbs can be effectively modeled. We further include an attention module such that the temporal significance of the interaction can be learned, which enhances the temporal alignment of the active–reactive motion pair. Second, as the reactive motion of different types of interactions can be significantly different, we introduce a discriminator that not only tells if the generated movement is realistic or not, but also tells the class label of the interaction. This allows the use of such labels in supervising the training of the generator. We experiment with the SBU, the HHOI and the 2C datasets. The high quality of the synthetic motion demonstrates the effective design of our generator, and the discriminability of the synthesis also demonstrates the strength of our discriminator.

We present a novel graph-based model for 3D human skeletal action representation.We present a graph kernel to measure the similarity between two graphs.The temporal pyramid covariance descriptor is proposed to preserve the individual joint features.Trained SVM classifier with proposed graph kernel is used for action recognition. Human action recognition is a challenging task due to the articulated and complex nature of actions. Recently developed commodity depth sensors coupled with the skeleton estimation algorithm have generated a renewed interest in human skeletal action recognition. In this paper, we characterize the human actions with a novel graph-based model which preserves complex spatial structure among skeletal joints according to their activity levels as well as the spatio-temporal joint features. In particular, the proposed top-K Relative Variance of Joint Relative Distance (RVJRD)s determine which joint pairs should be selected in the resulting graph according to normalized activity levels. In addition, the temporal pyramid covariance descriptors are adopted to represent joint locations. The graph kernel is used for measuring the similarity between two graphs by matching the walks from each of the two graphs to be matched. We evaluate the proposed approach on three challenging action recognition datasets captured by depth sensors. The experimental results show our proposed approach outperforms several state-of-the-art human skeletal action recognition approaches.

Graphs are frequently used to provide a powerful representation for structured data. However, it is still a challenging task to model 3D human motions due to its large spatio-temporal variations. This paper proposes a novel graph-based method for real time 3D human motion retrieval. Firstly, we propose a novel graph construction method which connects the joints that are deemed important for a given motion. In particular, the top-N Relative Ranges of Joint Relative Distances (RRJRD) were proposed to determine which joints should be connected in the resulting graph because these measures indicate the normalized activity levels among the joint pairs. Different motions may thus result in different graph structures so the construction of the graphs is made adaptive to the characteristics of a given motion and is able to represent a meaningful spatial structure. In addition to the spatial structure, the temporal pyramid of covariance descriptors was adopted to preserve certain level of spatio-temporal local features. The graph kernel is computed by matching the walks from each of the two graphs to be matched. Furthermore, multiple kernel learning was applied to determine the optimal weights for combining the graph kernels to measure the overall similarity between two motions. The experimental results show that our method is robust under several variations, and demonstrates superior performance in comparison to three state-of-the-art methods. Graphical abstractDisplay Omitted HighlightsWe propose a novel adaptive graph model for representation of 3D human motions.We propose an AGK to measure the similarity of two adaptive graphs.We propose a method which allows AGK set weights to the different matched walk groups.

In this paper, an automated lesson generation system for guiding beginners to learn basic dance movements is proposed. It analyzes the dance to generate a two-phase lesson which can provide a suitable cognitive load thus offering an efficient learning experience. In the first phase, the dance is divided into small pieces which are patterns, and then those patterns are arranged to form an easy-to-complex learning path according to the prerequisite relations. The courseware of the second phase is responsible for guiding students to assemble all the patterns into the full dance. The learning process of the second phase adapts to the student's background knowledge and preference. It keeps monitoring the students' performances during the learning process, and adaptively adding new patterns for the students to learn. A user study which involved 52 college students who had never been trained to dance before was conducted and the results are optimistic implying that the two-phase lesson is a practical way for people to learn dance.

A system that can retrieve logically relevant 3D captured motions is useful in game and animation production. We presented a robust logical relevance metric based on the relative distances among the joints. Existing methods select a universal subset of features for all kinds of queries which may not well characterize the variations in different queries. To break through this limitation we proposed an Adaptive Feature Selection (AFS) method that abstracts the characteristics of the query by a Linear Regression Model, and different feature subsets can be selected according to the properties of the specific query. With a Graded Relevance Feedback (GRF) algorithm, we refined the feature subset that enhances the retrieval performance according to the graded relevance of the feedback samples. With an ontology that predefines the logical relevance between motion classes in terms of graded relevance, the performance of the proposed AFS-GRF algorithm is evaluated and shown to outperform other class-specific feature selection and motion retrieval methods.

The purpose of the present study was to investigate the handwriting characteristics of secondary school students with and without physical disabilities (PD). With the use of a computerized Chinese Handwriting Assessment Tool (CHAT), it was made possible to objectively assess and analyze in detail the handwriting characteristics of individual students. Fifty participants (age range: 15-19-years-old) were recruited from one mainstream secondary school and 20 participants (age range: 17-24-years-old) were recruited from two secondary schools for students with PD. They were asked to perform three consecutive handwriting tasks: copying 90 characters using the computerized CHAT, an English passage copying task, and a Chinese passage copying task. The data indicated that students with PD were significantly slower in copying both Chinese and English characters in passages when compared to the typical students. Significant differences in the measures of writing speed, air/ground time ratio, standard deviation of speed, standard deviation of size per character, and number of stroke errors measured by the CHAT were found between the two groups of students. Further analysis on the data of typical students indicated no significant difference in handwriting speed among students of different classes (i.e. arts or science) on copying Chinese and English passages, and on individual Chinese words (from CHAT). The academic results of students also showed no significant correlation with their handwriting speed measured by the three writing tasks. To conclude, the CHAT system was able to identify a number of characteristics of handwriting on students with and without PD. It was suggested that the CHAT should further be developed into an objective evaluation tool to explore the handwriting characteristics of the students with a wider range of disabilities in the future, and to make recommendations to arrange special examination arrangements (SEA) for students with physical disabilities or other special needs.

Membrane protein plays an important role in some biochemical process such as signal transduction, transmembrane transport, etc. Membrane proteins are usually classified into five types [Chou, K.C., Elrod, D.W., 1999. Prediction of membrane protein types and subcellular locations. Proteins: Struct. Funct. Genet. 34, 137-153] or six types [Chou, K.C., Cai, Y.D., 2005. J. Chem. Inf. Modelling 45, 407-413]. Designing in silico methods to identify and classify membrane protein can help us understand the structure and function of unknown proteins. This paper introduces an integrative approach, IAMPC, to classify membrane proteins based on protein sequences and protein profiles. These modules extract the amino acid composition of the whole profiles, the amino acid composition of N-terminal and C-terminal profiles, the amino acid composition of profile segments and the dipeptide composition of the whole profiles. In the computational experiment, the overall accuracy of the proposed approach is comparable with the functional-domain-based method. In addition, the performance of the proposed approach is complementary to the functional-domain-based method for different membrane protein types.