Back to Search Start Over

Alexa, play with robot: Introducing the First Alexa Prize SimBot Challenge on Embodied AI

Authors :
Shi, Hangjie
Ball, Leslie
Thattai, Govind
Zhang, Desheng
Hu, Lucy
Gao, Qiaozi
Shakiah, Suhaila
Gao, Xiaofeng
Padmakumar, Aishwarya
Yang, Bofei
Chung, Cadence
Guthy, Dinakar
Sukhatme, Gaurav
Arumugam, Karthika
Wen, Matthew
Ipek, Osman
Lange, Patrick
Khanna, Rohan
Pansare, Shreyas
Sharma, Vasu
Zhang, Chao
Flagg, Cris
Pressel, Daniel
Vaz, Lavina
Dai, Luke
Goyal, Prasoon
Sahai, Sattvik
Liu, Shaohua
Lu, Yao
Gottardi, Anna
Hu, Shui
Liu, Yang
Hakkani-Tur, Dilek
Bland, Kate
Rocker, Heather
Jeun, James
Rao, Yadunandana
Johnston, Michael
Iyengar, Akshaya
Mandal, Arindam
Natarajan, Prem
Ghanadan, Reza
Publication Year :
2023

Abstract

The Alexa Prize program has empowered numerous university students to explore, experiment, and showcase their talents in building conversational agents through challenges like the SocialBot Grand Challenge and the TaskBot Challenge. As conversational agents increasingly appear in multimodal and embodied contexts, it is important to explore the affordances of conversational interaction augmented with computer vision and physical embodiment. This paper describes the SimBot Challenge, a new challenge in which university teams compete to build robot assistants that complete tasks in a simulated physical environment. This paper provides an overview of the SimBot Challenge, which included both online and offline challenge phases. We describe the infrastructure and support provided to the teams including Alexa Arena, the simulated environment, and the ML toolkit provided to teams to accelerate their building of vision and language models. We summarize the approaches the participating teams took to overcome research challenges and extract key lessons learned. Finally, we provide analysis of the performance of the competing SimBots during the competition.

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2308.05221
Document Type :
Working Paper