Back to Search
Start Over
Watch-n-Patch: Unsupervised Learning of Actions and Relations.
- Source :
-
IEEE Transactions on Pattern Analysis & Machine Intelligence . Feb2018, Vol. 40 Issue 2, p467-481. 15p. - Publication Year :
- 2018
-
Abstract
- There is a large variation in the activities that humans perform in their everyday lives. We consider modeling these composite human activities which comprises multiple basic level actions in a completely unsupervised setting. Our model learns high-level co-occurrence and temporal relations between the actions. We consider the video as a sequence of short-term action clips, which contains human-words and object-words. An activity is about a set of action-topics and object-topics indicating which actions are present and which objects are interacting with. We then propose a new probabilistic model relating the words and the topics. It allows us to model long-range action relations that commonly exist in the composite activities, which is challenging in previous works. We apply our model to the unsupervised action segmentation and clustering, and to a novel application that detects forgotten actions, which we call action patching. For evaluation, we contribute a new challenging RGB-D activity video dataset recorded by the new Kinect v2, which contains several human daily activities as compositions of multiple actions interacting with different objects. Moreover, we develop a robotic system that watches and reminds people using our action patching algorithm. Our robotic setup can be easily deployed on any assistive robots. [ABSTRACT FROM PUBLISHER]
Details
- Language :
- English
- ISSN :
- 01628828
- Volume :
- 40
- Issue :
- 2
- Database :
- Academic Search Index
- Journal :
- IEEE Transactions on Pattern Analysis & Machine Intelligence
- Publication Type :
- Academic Journal
- Accession number :
- 127253142
- Full Text :
- https://doi.org/10.1109/TPAMI.2017.2679054