Bridging Learning and Planning for Construction in a 3D environment

Abstract

Building structures in a partially observable state space can be seen as a tedious and complex task for reinforcement learning. This is due to a partial observable state space only containing a fraction of the required information a reinforcement learning agent requires as well as the fact that the agent would be required to explore and perform thousands of actions to achieve anything notable. One way of reducing the complexity is using pre-learnt skills to aid the agent by decreasing the learning needed to accomplish small tasks within this domain. This, however, is still largely complex as an agent would need to select an action or skill within the pixel domain. A domain that is less complex for representing the environment and skills is that of the symbolic domain. Often used for planning, the symbolic domain contains abstracted information about the environment which decreases the complexity of the problem. Thus, we propose a framework that leverages the convenient outcome of planning methods to plan complex tasks that can be completed in any complex environment with a partially observable state space. We conduct experiments in various construction tasks in Minecraft and find that our framework outperforms a basic DQN as well as a DQN with options.