Investigating the integration of interoception in a reinforcement learning pipeline

Abstract

Reinforcement learning (RL), inspired by neuroscience, trains artificial agents to maximise external rewards. However, decision-making in humans involves complex evaluation and planning processes that initiate actions beyond external reward including the incentive to maintain internal homeostasis. Interoception is the brain’s processing of signals from the body which convey hunger, thirst and body temperature. The initiation of actions beyond external reward is demonstrated in behaviours which address interoceptive needs like hunger. The prefrontal cortex (PFC), in coordination with other brain regions, balances responding to these intrinsically rewarding internal needs while engaging in externally rewarding activities. This study models this holistic evaluation process conducted by the PFC to manage an organism’s hunger levels alongside extrinsically rewarding activities. The study draws on an analogy of a mouse in a flat room with access to a running wheel (extrinsic reward) and a block of cheese (intrinsic reward), to an RL agent in a MiniGrid environment with a green goal square and a food location. It does this by equipping an RL agent with ghrelin levels and the capacity to manage its subsequent hunger thresholds by visiting the predefined food location. The behaviour of the RL agent is consistent with the naturally observed behaviour of a hungry mouse, in that a “hungry” RL agent (i.e. when its ghrelin levels equal the hunger threshold) prioritises visiting the food location before the green goal square, emulating survival-based behaviours. By varying key parameters, an RL agent which incorporates interoception can reproduce behaviours consistent with organisms suffering from obesity and anhedonia. Hence, this study demonstrates the potential of integrating the holistic role of the PFC in human decision making onto the RL pipeline.