Back to Search
Start Over
Spatial Temporal Transformer Network for Skeleton-based Action Recognition
- Source :
- Pattern Recognition. ICPR International Workshops and Challenges ISBN: 9783030687953, ICPR Workshops (3)
- Publication Year :
- 2020
-
Abstract
- Skeleton-based human action recognition has achieved a great interest in recent years, as skeleton data has been demonstrated to be robust to illumination changes, body scales, dynamic camera views, and complex background. Nevertheless, an effective encoding of the latent information underlying the 3D skeleton is still an open problem. In this work, we propose a novel Spatial-Temporal Transformer network (ST-TR) which models dependencies between joints using the Transformer self-attention operator. In our ST-TR model, a Spatial Self-Attention module (SSA) is used to understand intra-frame interactions between different body parts, and a Temporal Self-Attention module (TSA) to model inter-frame correlations. The two are combined in a two-stream network which outperforms state-of-the-art models using the same input data on both NTU-RGB+D 60 and NTU-RGB+D 120.<br />Accepted as ICPRW2020 (FBE2020, Workshop on Facial and Body Expressions, micro-expressions and behavior recognition) 8 pages, 2 figures. arXiv admin note: substantial text overlap with arXiv:2008.07404
- Subjects :
- FOS: Computer and information sciences
3D skeleton
Computer science
Computer Vision and Pattern Recognition (cs.CV)
Open problem
Computer Science - Computer Vision and Pattern Recognition
02 engineering and technology
Skeleton (category theory)
Self-attention
Action recognition
Representation learning
Operator (computer programming)
Graph CNN
Encoding (memory)
0202 electrical engineering, electronic engineering, information engineering
Transformer (machine learning model)
business.industry
Self attention
020207 software engineering
Pattern recognition
020201 artificial intelligence & image processing
Artificial intelligence
business
Feature learning
Subjects
Details
- Language :
- English
- ISBN :
- 978-3-030-68795-3
- ISBNs :
- 9783030687953
- Database :
- OpenAIRE
- Journal :
- Pattern Recognition. ICPR International Workshops and Challenges ISBN: 9783030687953, ICPR Workshops (3)
- Accession number :
- edsair.doi.dedup.....de456541ebf81809b9ff09bc5cfd636d