Search

Affective computing is a field of great interest in many computer vision applications, including video surveillance, behaviour analysis, and human-robot interaction. Most of the existing literature has addressed this field by analysing different sets of face features. However, in the last decade, several studies have shown how body movements can play a key role even in emotion recognition. The majority of these experiments on the body are performed by trained actors whose aim is to simulate emotional reactions. These unnatural expressions differ from the more challenging genuine emotions, thus invalidating the obtained results. In this paper, a solution for basic non-acted emotion recognition based on 3D skeleton and Deep Neural Networks (DNNs) is provided. The proposed work introduces three majors contributions. First, unlike the current state-of-the-art in non-acted body affect recognition, where only static or global body features are considered, in this work also temporal local movements performed by subjects in each frame are examined. Second, an original set of global and time-dependent features for body movement description is provided. Third, to the best of out knowledge, this is the first attempt to use deep learning methods for non-acted body affect recognition. Due to the novelty of the topic, only the UCLIC dataset is currently considered the benchmark for comparative tests. On the latter, the proposed method outperforms all the competitors.

Most automatic expression analysis systems attempt to recognize a conventional set of expressions such as happiness, sadness, anger, surprise and fear, etc. Although this set of expressions is the most typical of the face, it is not the most representative/relevant for what the body expressions tell us. This paper presents a novel and generic approach for the recognition of body expressions using human postures. Our method is based on the notion of neutral motion generated from a given expressive one. In a second time, we estimate a residue function, as the difference between the two associated motions, namely the expressive and the neutral motion. More precisely, this function that is inspired by studies from psychology domain, gives a “neutrality” score of a motion. Using this “neutrality score”, we propose a cost function which enables to synthesis the neutral motion from any input expressive motion. The synthesis of neutral motion process is based on two nested Principal Component Analysis providing a space where moving and selecting realistic human animations become possible. Proposed approach is evaluated on four databases with heterogeneous movements and body expressions and it achieved recognition results for body expression recognition that exceed state of the art.

Skeleton-based human action recognition has made great progress, especially with the development of a graph convolution network (GCN). The most important work is ST-GCN, which automatically learns both spatial and temporal patterns from skeleton sequences. However, this method still has some imperfections: only short-range correlations are appreciated, due to the limited receptive field of graph convolution. However, long-range dependence is essential for recognizing human action. In this work, we propose the use of a spatial-temporal relative transformer (ST-RT) to overcome these defects. Through introducing relay nodes, ST-RT avoids the transformer architecture, breaking the inherent skeleton topology in spatial and the order of skeleton sequence in temporal dimensions. Furthermore, we mine the dynamic information contained in motion at different scales. Finally, four ST-RTs, which extract spatial-temporal features from four kinds of skeleton sequence, are fused to form the final model, multi-stream spatial-temporal relative transformer (MSST-RT), to enhance performance. Extensive experiments evaluate the proposed methods on three benchmarks for skeleton-based action recognition: NTU RGB+D, NTU RGB+D 120 and UAV-Human. The results demonstrate that MSST-RT is on par with SOTA in terms of performance.

Skeleton-based Human Activity Recognition has achieved great interest in recent years as skeleton data has demonstrated being robust to illumination changes, body scales, dynamic camera views, and complex background. In particular, Spatial-Temporal Graph Convolutional Networks (ST-GCN) demonstrated to be effective in learning both spatial and temporal dependencies on non-Euclidean data such as skeleton graphs. Nevertheless, an effective encoding of the latent information underlying the 3D skeleton is still an open problem, especially when it comes to extracting effective information from joint motion patterns and their correlations. In this work, we propose a novel Spatial-Temporal Transformer network (ST-TR) which models dependencies between joints using the Transformer self-attention operator. In our ST-TR model, a Spatial Self-Attention module (SSA) is used to understand intra-frame interactions between different body parts, and a Temporal Self-Attention module (TSA) to model inter-frame correlations. The two are combined in a two-stream network, whose performance is evaluated on three large-scale datasets, NTU-RGB+D 60, NTU-RGB+D 120, and Kinetics Skeleton 400, consistently improving backbone results. Compared with methods that use the same input data, the proposed ST-TR achieves state-of-the-art performance on all datasets when using joints' coordinates as input, and results on-par with state-of-the-art when adding bones information., Accepted at Computer Vision and Image Understanding (CVIU) 12 pages, 8 figures

International audience; We present a novel and generic framework for the recognition of body expressions using human postures. Motivated by the state of the art from the domain of psychology, our approach recognizes expression by analyzing sequence of pose. Features proposed in this article are computationally simple and intuitive to understand. They are based on visual cues and provide in-depth understanding of body postures required to recognize body expressions. We have evaluated our approach on different databases with heterogeneous movements and body expressions. Our recognition results exceeds state of the art for some database and for others we obtain results at par with state of the art.

Adding heat conducting particles to a polymer matrix to prepare thermally conductive and electrical insulation materials is an effective approach to address the safety issues arising from the accumulation of heat in the working process of electronic devices. In this work, thermally conductive and electrical insulation nano-paper, consisting of Boron Nitride nano-sheet (BNNS) and cellulose nanofiber (CNF), was prepared using an aerogel 3D skeleton template method. For comparison, BNNS/CNF nano-paper was also produced using a simple blending method. At a BNNS loading of 50 wt%, the thermal conductivity of BNNS/CNF aerogel nano-paper and blended nano-paper at 70 °C are 2.4 W/mK and 1.2 W/mK respectively, revealing an increase of 94.4%. Under similar conditions, the volume resistivity of BNNS/CNF aerogel nano-paper and blended nano-paper are 4.0 × 1014 and 4.2 × 1014 Ω·cm respectively. In view of its excellent thermal conductivity and electrical insulation performance, therefore, BNNS/CNF aerogel nano-paper holds great potential for electronic-related applications.