A fully-decentralised general-sum approach for multi-agent reinforcement learning using minimal modelling
| dc.contributor.author | Kruger, Marcel Matthew Anthony | |
| dc.contributor.supervisor | Rosman, Benjamin | |
| dc.contributor.supervisor | James, Steven | |
| dc.contributor.supervisor | Shipton, Jarrod | |
| dc.date.accessioned | 2024-10-26T19:31:22Z | |
| dc.date.available | 2024-10-26T19:31:22Z | |
| dc.date.issued | 2023-08 | |
| dc.description | A dissertation submitted in fulfilment of the requirements for the degree of Master of Science, to the Faculty of Science, School of Computer Science & Applied Mathematics, University of the Witwatersrand, Johannesburg, 2023. | |
| dc.description.abstract | Multi-agent reinforcement learning is a prominent area of research in machine learning, extending reinforcement learning to scenarios where multiple agents concurrently learn and interact within the same environment. Most existing methods rely on centralisation during training, while others employ agent modelling. In contrast, we propose a novel method that adapts the role of entropy to assist in fully-decentralised training without explicitly modelling other agents using additional information to which most centralised methods assume access. We augment entropy to encourage more deterministic agents, and instead, we let the non-stationarity inherent in MARL serve as a mode for exploration. We empirically evaluate the performance of our method across five distinct environments, each representing unique challenges. Our assessment encompasses both cooperative and competitive cases. Our findings indicate that the approach of penalising entropy, rather than rewarding it, enables agents to perform at least as well as the prevailing standard of entropy maximisation. Moreover, our alternative approach achieves several of the original objectives of entropy regularisation in reinforcement learning, such as increased sample efficiency and potentially better final rewards. Whilst entropy has a significant role, our results in the competitive case indicate that position bias is still a considerable challenge. | |
| dc.description.submitter | MM2024 | |
| dc.faculty | Faculty of Science | |
| dc.identifier | 0000-0003-4803-5037 | |
| dc.identifier.citation | Kruger, Marcel Matthew Anthony. (2023). A fully-decentralised general-sum approach for multi-agent reinforcement learning using minimal modelling. [Master's dissertation, University of the Witwatersrand, Johannesburg]. https://hdl.handle.net/10539/41978 | |
| dc.identifier.uri | https://hdl.handle.net/10539/41978 | |
| dc.language.iso | en | |
| dc.publisher | University of the Witwatersrand, Johannesburg | |
| dc.rights | ©2023 University of the Witwatersrand, Johannesburg. All rights reserved. The copyright in this work vests in the University of the Witwatersrand, Johannesburg. No part of this work may be reproduced or transmitted in any form or by any means, without the prior written permission of University of the Witwatersrand, Johannesburg. | |
| dc.rights.holder | University of the Witwatersrand, Johannesburg | |
| dc.school | School of Computer Science and Applied Mathematics | |
| dc.subject | Reinforcement Learning | |
| dc.subject | Multi-Agent Reinforcement Learning | |
| dc.subject | Deep Learning | |
| dc.subject | Multi-Agent Systems | |
| dc.subject | Entropy | |
| dc.subject | UCTD | |
| dc.subject.other | SDG-9: Industry, innovation and infrastructure | |
| dc.title | A fully-decentralised general-sum approach for multi-agent reinforcement learning using minimal modelling | |
| dc.type | Dissertation |