Addressing Ambiguity in Human Robot Interaction using Compositional Reinforcement Learning and User Preference

Abstract

The ability for social robots to integrate naturally with the lives of humans has many advantages in industry and assisted services. For effective Human Robot Interaction (HRI), social robots require communication abilities to understand an instruction from the user and perform tasks accordingly. Verbal communication is an intuitive natural interaction for non-expert users but it can also be a source of ambiguity, especially when there is also ambiguity in the environment (i.e. similar objects to be retrieved). Addressing ambiguity for task inference in HRI is an unsolved problem. Current approaches, that have been implemented in collaborative robots, include asking for clarifications from the user. Related research shows the promising results of using user preference in HRI, but no work has been found where user preference is employed specifically to address ambiguity in conjunction with clarifying questions. Additionally, these methods do not leverage knowledge learned from previous interactions with the environment and the life-long learning capabilities of Reinforcement Learning (RL) agents. Based on the related work and shortfalls, we propose a framework to address ambiguity in HRI (resulting from natural language instructions), that leverages the compositionality of learned object-attribute base-tasks in conjunction with user preference and clarifying questions for adaptive task inference. Evaluating our method in the BabyAI domain, we extensively test all components of our system and determine that our framework provides a viable solution for addressing the problem of ambiguity in HRI. We experimentally prove that our method improves user experience by decreasing the number of clarifying questions asked, while maintaining a high level of accuracy.