Search

The presented work is aimed at designing a system that will model and recognize actions and its interaction with objects. Such a system is aimed at facilitating robot task learning. Activity modeling and recognition is very important for its potential applications in surveillance, human-machine interface, entertainment, biomechanics etc. Recent developments in neuroscience suggest that all actions are a compositions of smaller units called primitives. Current works based on primitives for action recognition uses a supervised framework for specifying the primitives. We propose a method to extract primitives automatically. These primitives are to be used to generate actions based on certain rules for combining. These rules are expressed as a stochastic context free grammar. A model merging approach is adopted to learn a Hidden Markov Model to t the observed data sequences. The states of the HMM approximates local properties of a long sequence. Observation sequences are used to learn a model expressing the data in a structured way. Based on the learned model, recurring parts in the sequences are identied. Primitives that make up the observation sequences are identied as the recurring and unique parts appearing in the sequences. Extracted primitives are used to make a primitive graph from which a grammar for the observed primitives are derived. This method is further extended to include object context. We observe that human actions and objects can be seen as being intertwined: we can interpret actions from the way the body parts are moving, but as well from how their eect on the involved object. While human movements can look vastly dierent even under minor changes in location, orientation and scale, the use of the object can provide a strong invariant for the detection of motion primitives. Movements that produce the same state change in the object state space are classied to be instances of the same action primitive. This allows us to dene action primitives as sets of movements where the movements of each primitive are connected through the object state change they induce.

There has been a recent interest in segmenting action sequences into meaningful parts (action primitives) and to model actions on a higher level based on these action primitives. Unlike previous works where action primitives are defined a-priori and search is made for them later, we present a sequential and statistical learning algorithm for automatic detection of the action primitives and the action grammar based on these primitives. We model a set of actions using a single HMM whose structure is learned incrementally as we observe new types. Actions are modeled with sufficient number of Gaussians which would become the states of an HMM for an action. For different actions we find the states that are common in the actions which are then treated as an action primitive.