Teaching a Robot to Build Duplo Structures using Reinforcement Learning

Abstract

Consider the task of configuring a robot arm to build Duplo structures to recreate a desired goal structure. This is a challenging task because it requires long-term planning. Briefly, we solve the task in the following manner. First, we provide images of the desired goal structure we wish to recreate. Using these images, we generate a list of sequential block placing actions using a Simulation Environment and a learned model. Finally, we execute this list of actions using a YuMi robot and Duplo blocks to recreate the structure. We experiment with different design decisions such as biasing our training data as well as providing the goal structures in the form of an instruction manual to see how they affect the overall system. The instruction manual outlines some high-level steps required to recreate a goal structure by providing ordered images of intermediate structures that can be constructed to recreate the final goal structure. Our final model manages to recreate unseen goal structures to a high degree of accuracy. We also show that the use of an instruction manual is beneficial to the performance of the model. We contribute a novel solution to the task of configuring a robot arm to build Duplo structures.