Forming Shared Attention between Robot and Users in Conversational Situations

The ability to interact with multiple users as well as to recognize the ways in which people are interacting is essential for a robot participating in society. Forming shared attention is regarded as fundamental in improving the flexibility of human-robot interaction. This paper proposes an algorithm that enables shared attention between a robot and speakers in alternative conversational situations. We firstly present a microphone array technique and a method to combine auditory and visual information for estimating the physical location of the sound source. Secondly, the speaker is detected according to the linear combination of the result of sound source localization and the likelihood map representing the distribution of pixels having skin color. Finally, in the conversational situations, the speaker's focus of attention is alternatively detected and shared by the robot for upcoming communication based on the proposed algorithm. Several experimental results are presented that demonstrate the effectiveness of the proposed method.