A reinforcement learning design for HIV clinical trials
Files
Date
2014-07-30
Authors
Parbhoo, Sonali
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Determining e ective treatment strategies for life-threatening illnesses such as HIV is
a signi cant problem in clinical research. Currently, HIV treatment involves using
combinations of anti-HIV drugs to inhibit the formation of drug-resistant strains. From
a clinician's perspective, this usually requires careful selection of drugs on the basis of an
individual's immune responses at a particular time. As the number of drugs available for
treatment increases, this task becomes di cult. In a clinical trial setting, the task is even
more challenging since experience using new drugs is limited. For these reasons, this
research examines whether machine learning techniques, and more speci cally batch
reinforcement learning, can be used for the purposes of determining the appropriate
treatment for an HIV-infected patient at a particular time. To do so, we consider using
tted Q-iteration with extremely randomized trees, neural tted Q-iteration and least
squares policy iteration. The use of batch reinforcement learning means that samples
of patient data are captured prior to learning to avoid imposing risks on a patient.
Because samples are re-used, these methods are data-e cient and particularly suited to
situations where large amounts of data are unavailable. We apply each of these learning
methods to both numerically generated and real data sets. Results from this research
highlight the advantages and disadvantages associated with each learning technique.
Real data testing has revealed that these batch reinforcement learning techniques have
the ability to suggest treatments that are reasonably consistent with those prescribed
by clinicians. The inclusion of additional state variables describing more about an
individual's health could further improve this learning process. Ultimately, the use of
such reinforcement learning methods could be coupled with a clinician's knowledge for
enhanced treatment design.
Description
A dissertation submitted to the Faculty of Science, University of the Witwatersrand, Johannesburg, in fulfilment of the requirements for the degree of Master of Science. Johannesburg, 2014.