Optimisation of Kick Latency for Enhanced Performance of Robots in the RoboCup Three-Dimensional League through Proximal Policy Optimisation (PPO)

Abstract

This study aimed to enhance the kicking ability of Nao robots in the three-dimensional RoboCup simulation by addressing a crucial challenge observed in the University of Witwatersrand RoboCup team. The focal challenge revolved around a noticeable delay and slow movement manifested by the robot during ball kicks, leading to vulnerabilities in ball possession against opposing teams. To surmount this challenge, the implementation of Proximal Policy Optimisation (PPO), a methodology pioneered by OpenAI, was advocated. The precise objective was to optimise kick parameters, with a primary emphasis on curtailing kick latency. This optimisation aimed to ensure swift and accurate execution across various kicking scenarios, encompassing actions like propelling the ball into the opponent’s territory to bolster ball possession and thwart adversary manoeuvres. Harnessing the iterative advancements embedded in PPO, the successor to Trust Region Policy Optimisation (TRPO), the endeavour was to refine the kicking behaviour of Nao robots. This optimisation process significantly reduced the observed kick delay, and this made the robot more agile and effective at competing in the complex three-dimensional RoboCup simulation environment. The study’s outcomes highlighted substantial progress in reducing kick latency and improving the adaptability of robotic soccer players, opening up possibilities for further exploration in reinforcement learning for autonomous agents.