Estimating skills in discrete pursuit-evasion games
Date
2023
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of the Witwatersrand, Johannesburg
Abstract
Game Theory is a well-established field in mathematics, economics, and computer science, with a rich history of studying n-person, zero-sum games. Researchers have utilized the best computational power of their time to create computational players that are able to beat the best human players at complex two-player, zero-sum games such as Chess and Go. In the field of Reinforcement Learning and Robotics, these types of games are considered useful environments to conduct experiments about agent behavior and learning. In this research report we explore a subset of discrete skill-dependent pursuit-evasion games upon which we build a framework to estimate player skills. In this game environment a player’s skill determines the actions available to them in each state and the transition dynamics resulting from the chosen action. The game offers a simplified depresentation of more complex games which often have vast state and action spaces, making it difficult to model and analyze player behavior.
In this game environment we find that players with incorrect assumptions about an opponent’s skill perform sub-optimally at winning games. Given that knowledge of an opponent’s skill impacts on player performance, we demonstrate that players can use Bayesian inference to estimate their opponent’s skill, based on the action outcomes of an opponent. We also demonstrate that skill estimation is a valuable exercise for players to undertake and show that the performance of players that estimate their opponent’s skill converges to the performance of players given perfect knowledge of their opponent’s skill. This research contributes to our understanding of Bayesian skill estimation in skill-dependent pursuit-evasion games which may be useful in the fields of Multi-agent Reinforcement Learning and Robotics.
Description
A research report submitted to the Faculty of Science in partial fulfilment of the requirements for the degree of Master of Science in the School of Computer Science and Applied Mathematics at University of the Witwatersrand, Johannesburg, 2023
Keywords
Skills estimation, Extended form game, Game Theory, Robotics