MultiI-View Ranking: Tasking Transformers to Generate and Validate Solutions to Math Word Problems
| dc.contributor.author | Mzimba, Rifumo | |
| dc.contributor.supervisor | Klein, Richard | |
| dc.contributor.supervisor | Rosman, Benjamin | |
| dc.date.accessioned | 2024-10-26T07:26:36Z | |
| dc.date.available | 2024-10-26T07:26:36Z | |
| dc.date.issued | 2023-11 | |
| dc.description | A dissertation submitted in fulfillment of the requirements for the degree of Master of Science,to the Faculty of Science, School of Computer Science & Applied Mathematics, University of the Witwatersrand, Johannesburg, 2023. | |
| dc.description.abstract | The recent developments and success of the Transformer model have resulted in the creation of massive language models that have led to significant improvements in the comprehension of natural language. When fine-tuned for downstream natural language processing tasks with limited data, they achieve state-of-the-art performance. However, these robust models lack the ability to reason mathematically. It has been demonstrated that, when fine-tuned on the small-scale Math Word Problems (MWPs) benchmark datasets, these models are not able to generalize. Therefore, to overcome this limitation, this study proposes to augment the generative objective used in the MWP task with complementary objectives that can assist the model in reasoning more deeply about the MWP task. Specifically, we propose a multi-view generation objective that allows the model to understand the generative task as an abstract syntax tree traversal beyond the sequential generation task. In addition, we propose a complementary verification objective to enable the model to develop heuristics that can distinguish between correct and incorrect solutions. These two goals comprise our multi-view ranking (MVR) framework, in which the model is tasked to generate the prefix, infix, and postfix traversals for a given MWP, and then use the verification task to rank the generated expressions. Our experiments show that the verification objective is more effective at choosing the best expression than the widely used beam search. We further show that when our two objectives are used in conjunction, they can effectively guide our model to learn robust heuristics for the MWP task. In particular, we achieve an absolute percentage improvement of 9.7% and 5.3% over our baseline and the state-of-the-art models on the SVAMP datasets. Our source code can be found on https://github.com/ProxJ/msc-final. | |
| dc.description.submitter | MM2024 | |
| dc.faculty | Faculty of Science | |
| dc.identifier | 0000-0002-1713-4094 | |
| dc.identifier.citation | Mzimba, Rifumo. (2023). MultiI-View Ranking: Tasking Transformers to Generate and Validate Solutions to Math Word Problems. [Master's dissertation, University of the Witwatersrand, Johannesburg]. https://hdl.handle.net/10539/41971 | |
| dc.identifier.uri | https://hdl.handle.net/10539/41971 | |
| dc.language.iso | en | |
| dc.publisher | University of the Witwatersrand, Johannesburg | |
| dc.rights | ©2023 University of the Witwatersrand, Johannesburg. All rights reserved. The copyright in this work vests in the University of the Witwatersrand, Johannesburg. No part of this work may be reproduced or transmitted in any form or by any means, without the prior written permission of University of the Witwatersrand, Johannesburg. | |
| dc.rights.holder | University of the Witwatersrand, Johannesburg | |
| dc.school | School of Computer Science and Applied Mathematics | |
| dc.subject | Multiview | |
| dc.subject | Math word problems | |
| dc.subject | Transformers | |
| dc.subject | Multi-tasking | |
| dc.subject | Verifiers | |
| dc.subject | UCTD | |
| dc.subject.other | SDG-9: Industry, innovation and infrastructure | |
| dc.title | MultiI-View Ranking: Tasking Transformers to Generate and Validate Solutions to Math Word Problems | |
| dc.type | Dissertation |