Learning Operators with NEAT for Boolean Composition in Reinforcement Learning

Abstract

The idea of skill composition has been gaining traction within reinforcement learning research. This compositional approach promotes efficient use of knowledge and represents a realistic, human-like style of learning. Existing work has demonstrated how simple skills can be composed using Boolean operators to solve new, unseen tasks without the need for further learning. However, this approach assumes that the learned value functions for each atomic skill are optimal—an assumption that is violated in most practical cases. We thus propose a method that instead learns operators for composition using evolutionary strategies. Our approach is empirically verified first in a tabular setting and then in a high dimensional function approximation environment. Results demonstrate outperformance of existing composition methods when faced with learned, suboptimal behaviours, while also promoting the development of robust agents and allowing for fluid transfer between domains.