1. Speech to Head Gesture Mapping in Multimodal Human-Robot Interaction
- Author
-
Adriana Tapus, Amir Aly, Robotique et Vision (RV), Unité d'Informatique et d'Ingénierie des Systèmes (U2IS), École Nationale Supérieure de Techniques Avancées (ENSTA Paris)-École Nationale Supérieure de Techniques Avancées (ENSTA Paris), and École Nationale Supérieure de Techniques Avancées (ENSTA Paris)
- Subjects
0209 industrial biotechnology ,Focus (computing) ,Index Terms— Coupled HMM ,Computer science ,Head (linguistics) ,Speech recognition ,02 engineering and technology ,signal mapping ,Human–robot interaction ,020901 industrial engineering & automation ,audio-video signal synchroniza-tion ,0202 electrical engineering, electronic engineering, information engineering ,[INFO.INFO-RB]Computer Science [cs]/Robotics [cs.RO] ,Robot ,020201 artificial intelligence & image processing ,Hidden Markov model ,Prosody ,Utterance ,Gesture - Abstract
International audience; In human-human interaction, para-verbal and non-verbal communication are naturally aligned and synchronized. The difficulty encountered during the coordination between speech and head gestures concerns the conveyed meaning, the way of performing the gesture with respect to speech characteristics , their relative temporal arrangement, and their coordinated organization in a phrasal structure of utterance. In this research, we focus on the mechanism of mapping head gestures and speech prosodic characteristics in a natural human-robot interaction. Prosody patterns and head gestures are aligned separately as a parallel multi-stream HMM model. The mapping between speech and head gestures is based on Coupled Hidden Markov Models (CHMMs), which could be seen as a collection of HMMs, one for the video stream and one for the audio stream. Experimental results with Nao robot are reported.
- Published
- 2012
- Full Text
- View/download PDF