Learning to backpropagate
| dc.contributor.author | Eyono, Roy Henha | |
| dc.date.accessioned | 2021-07-20T11:58:09Z | |
| dc.date.available | 2021-07-20T11:58:09Z | |
| dc.date.issued | 2020 | |
| dc.description | A dissertation submitted in fulfillment of the requirements for the degree of Master of Science to the Faculty of Science, University of the Witwatersrand, Johannesburg, 2020 | en_ZA |
| dc.description.abstract | The backpropagation algorithm is regarded as the de-facto standard for gradient optimization in artificial neural networks. Since the conception of the method, modifications of the algorithm have been proposed. Adaptive learning rate methods are examples of custom modification to the back propagation equation as they scale the gradient equation during training. Existing gradient modifications are largely based on theoretical fundamen tals of gradient optimization, but few methods optimize for modifications that are based on data. We are motivated by the idea of discovering better custom gradient update equations for gradient optimization. In this paper, we present our parametrized backpropagation learning frame work (PBLF) which learns modifications of the backpropagation gradient update equation for stochastic gradient optimization. We achieve this by optimizing parts of the backpropagation equation to produce custom gra dient update equations for gradient optimization. We evaluate our custom equations by training our target network on a validation dataset. In our dissertation, we provide empirical analysis and evidence to support PBLF as a competitive alternative to standard backpropagation. In our experiments, we report competitive empirical performances on CI FAR10 with our custom gradient update equations sampled from PBLF. Our data-driven method offers promising custom update equations for gradient optimization | en_ZA |
| dc.description.librarian | TL (2021) | en_ZA |
| dc.faculty | Faculty of Science | en_ZA |
| dc.format.extent | Online resource (52 leaves) | |
| dc.identifier.citation | Eyono Henha, Roy Pavel Samuel (2020) Learning to backpropagate, University of the Witwatersrand, Johannesburg, <http://hdl.handle.net/10539/31427> | |
| dc.identifier.uri | https://hdl.handle.net/10539/31427 | |
| dc.language.iso | en | en_ZA |
| dc.subject.lcsh | Neural networks (Computer science) | |
| dc.subject.lcsh | Back propagation | |
| dc.title | Learning to backpropagate | en_ZA |
| dc.type | Thesis | en_ZA |
Files
Original bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- Eyono Roy Final_MSc_Dissertation_1723980.pdf
- Size:
- 1.43 MB
- Format:
- Adobe Portable Document Format
- Description:
- Main Work
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- license.txt
- Size:
- 1.71 KB
- Format:
- Item-specific license agreed upon to submission
- Description: