Task-driven Embodied Agents that Chat

Aishwarya Padmakumar Jesse Thomason Ayush Shrivastava
Patrick Lange Anjali Narayan-Chen Spandana Gella
Robinson Piramithu Gokhan Tur Dilek Hakkani-Tur

TEACH (Task-drive Embodied Agents that Chat) is a set of episodes in which a human Commander who knows what task needs to be accomplished and where objects in a scene are located works with a human Follower who controls a virtual agent in the scene to accomplish household chores. The Commander and Follower communicate via natural language text. TEACh enables researchers to study strategies for natural language cooperation, instruction following from egocentric vision, question and answer generation, and partner modeling. We believe these skills will be necessary for real-world language-driven robotics applications.

Paper on ArXiv» GitHub» Simulator» Leaderboard (Coming Soon)

  title ={{TEACh:
          Task-driven Embodied Agents that Chat}},
  author={Aishwarya Padmakumar and Jesse Thomason and
          Ayush Shrivastava and Patrick Lange and
          Anjali Narayan-Chen and Spandana Gella and
          Robinson Piramithu and Gokhan Tur and,
          Dilek Hakkani-Tur},
  booktitle = {Conference on Artificial
               Intelligence (AAAI)},
  year = {2022},
  url  = {https://arxiv.org/abs/2110.00534}