Evaluating elastic weight consolidation across a taxonomy of continual learning scenarios

dc.contributor.authorLaxton, Gregory Stephen
dc.contributor.supervisorJarvis, Devon
dc.date.accessioned2026-11-12T15:43:37Z
dc.date.issued2025-11
dc.descriptionA research report submitted in fulfilment of the requirements for the degree of Master of Science in Data Science, to the Faculty of Science, School of Computer Science and Applied Mathematics, University of the Witwatersrand, Johannesburg, 2025
dc.description.abstractThe dynamic nature of real-world datasets poses significant challenges for machine learning, especially in continual learning, where catastrophic forgetting occurs as neural networks forget prior knowledge when acquiring new information, known as catastrophic forgetting. Various methods have been investigated in order to overcome catastrophic forgetting, such as Elastic Weight Consolidation (EWC). This study aims to determine the effectiveness of EWC in diverse real-world datasets characterised by different continual learning scenarios. We compare EWC against a base model with no regularisation, models with different continual learning approaches, and a method that incorporates data rehearsal from the previous task. Through experiments and performance comparisons using datasets specifically designed to represent three key iterated learning scenarios—class-iterated learning, domain-iterated learning, and task-iterated learning—this research aims to provide a comprehensive assessment of the capacity of EWC to facilitate continual learning. To ensure a thorough evaluation, these scenarios were carefully selected to include a toy domain, an MNIST-based domain, and a practical real-world domain, reflecting a progression from abstract conceptual tasks to more complex and realistic challenges. This meticulous approach highlights the core contribution of the research, emphasising its relevance and applicability across varying levels of complexity in continual learning. Our findings show that while EWC consistently delivers stable performance across diverse continual learning scenarios, it struggles with adaptability in tasks involving high feature overlap or dynamic shifts. In contrast, Synaptic Intelligence offers greater flexibility but exhibits higher variability, especially when task boundaries are ambiguous or overlap occurs at deeper representation levels.
dc.description.sponsorshipRonald Laxton
dc.description.submitterMMM2026
dc.facultyFaculty of Science
dc.identifier0009-0003-3799-2461
dc.identifier.citationLaxton, Gregory Stephen. (2025). Evaluating elastic weight consolidation across a taxonomy of continual learning scenarios. [Master's dissertation, University of the Witwatersrand, Johannesburg]. WIReDSpace. https://hdl.handle.net/10539/50305
dc.identifier.urihttps://hdl.handle.net/10539/50305
dc.language.isoen
dc.publisherUniversity of the Witwatersrand, Johannesburg
dc.rights©2025 University of the Witwatersrand, Johannesburg. All rights reserved. The copyright in this work vests in the University of the Witwatersrand, Johannesburg. No part of this work may be reproduced or transmitted in any form or by any means, without the prior written permission of University of the Witwatersrand, Johannesburg.
dc.rights.holderUniversity of the Witwatersrand, Johannesburg
dc.schoolSchool of Computer Science and Applied Mathematics
dc.subjectContinual learning
dc.subjectElastic Weight Consolidation (EWC)
dc.subjectMNIST-based domain
dc.subjectSynaptic Intelligence (SI)
dc.subjectTask-iterated learning
dc.subjectClass iterated learning
dc.subjectUCTD
dc.subject.primarysdgSDG-4: Quality education
dc.subject.secondarysdgSDG-9: Industry, innovation and infrastructure
dc.titleEvaluating elastic weight consolidation across a taxonomy of continual learning scenarios
dc.typeDissertation

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Laxton_Evaluating_2025.pdf
Size:
4.43 MB
Format:
Adobe Portable Document Format

License bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
license.txt
Size:
2.43 KB
Format:
Item-specific license agreed upon to submission
Description: