Empirical analysis of neural networks training optimisation
Date
2016
Authors
Kayembe, Mutamba Tonton
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Neural networks (NNs) may be characterised by complex error functions with attributes such as saddle-points, local minima, even-spots and plateaus. This complicates the associated training process in terms of efficiency, convergence and accuracy given that it is done by minimising such complex error functions. This study empirically investigates the performance of two NNs training algorithms which are based on unconstrained and global optimisation theories, i.e. the Resilient propagation (Rprop) and the Conjugate Gradient with Polak-Ribière updates (CGP). It also shows how the network structure plays a role in the training optimisation of NNs. In this regard, various training scenarios are used to classify two protein data, i.e. the Escherichia coli and Yeast data. These training scenarios use varying numbers of hidden nodes and training iterations. The results show that Rprop outperforms CGP. Moreover, it appears that the performance of classifiers varies under various training scenarios.
Description
A Dissertation submitted to the Faculty of Science, University of the Witwatersrand, Johannesburg, in fulfilment of the requirements for the degree of Master of Science in
Mathematical Statistics,School of Statistics and Actuarial Science.
October 2016.
Keywords
Citation
Kayembe, Mutamba Tonton (2016) Empirical analysis of neural networks training optimisation, University of Witwatersrand, Johannesburg, <http://wiredspace.wits.ac.za/handle/10539/21679>