Comparison of accuracy of HIV diagnosis between rapid HIV test kits conducted in non-laboratory settings and laboratory-based methods in South Africa
Date
2016
Authors
Chidarikire, Thato Nelly
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Introduction
South Africa has the largest absolute number of individuals living with human
immuno-deficiency virus (HIV) in the world. The quality assurance (QA) of HIV
rapid diagnostic tests (RDT) has not kept pace with the rate of expanded testing
and utilisation of RDT. This has made it difficult to assess the accuracy of testing.
In South Africa HIV counselling and testing (HCT) and the use of HIV RDT is the
point of entry to HIV prevention, management, care, treatment and support.
HCT in public health facilities is delivered mainly through rapid testing by nonprofessional
staff. Implementation of QA processes is crucial for accurate
diagnosis of HIV. However, accuracy of HCT using rapid test kits in non-laboratory
settings in South Africa will remain a challenge unless there is evidence that nonlaboratory
rapid HIV testing results are as reliable as the laboratory-based enzyme
immunoassays.
This study aimed to determine the accuracy of HIV RDT in the context of an
intervention. The objectives of the study were: i. To assess the sensitivity and
specificity of rapid test kits in two provinces; ii. To assess the sensitivity and
specificity of rapid test kits between the two provinces and New Start nongovernmental
organisation (NGO) which implemented a more comprehensive
quality management system (QMS); iii. To assess the accuracy of HIV RDT in the
two provinces; iv. To assess the accuracy of HIV RDT between the two provinces
and New Start sites.
The hypothesis was ‘the accuracy of HIV diagnosis using HIV RDT kits in nonlaboratory
settings in which an intervention has been introduced (internal quality
control), also known as IQC, will not be different compared to settings that do not
utilize IQC’.
Methods
In South Africa, the current laboratory-based gold standard for diagnosis of HIV
infection in adults in the public sector as recommended by the National Health
Laboratory Services (NHLS) Virology expert committee is a serial 2-test algorithm.
Thus, a reactive enzyme immunoassay (EIA) test result must be confirmed by a
second confirmatory EIA that must be different in terms of antigens and
technology. The Expert Committee recommendation is that positive results should
be confirmed by a separate sample 14 days later. In the case of HIV rapid testing
the national HIV counselling and testing (HCT) policy, 2010, similarly recommends
a serial 2-test algorithm for diagnosis where a reactive screening test is confirmed
by a different confirmatory test. If the confirmatory test is reactive the diagnosis is
positive. If test 1 is non-reactive then the diagnosis is negative. In case of
discrepant results an enzyme-linked immunosorbent assay (ELISA) test was
recommended as a tiebreaker. A new HIV testing services (HTS) policy was
approved in South Africa in 2016 and it further recommended that the first time
discrepant results are found, the counsellor must repeat the algorithm and if on
repeat, the results are still discrepant, then reflex testing is recommended where
the blood (whole blood) of a client is taken to the laboratory for ELISA (NDOH,
2016).This algorithm has replaced the use of Western Blot is South Africa. The
rationale for the change was based on the sensitivity and specificity of 3rd and 4th
generation ELISAs, workload, costs and expertise.
With the introduction of the 3rd and latterly 4th generation EIA tests the above
algorithm is in use in South Africa and has replaced the use of the Western blot as
a confirmatory test. The rationale for the change is based on earlier detection of
HIV infection, workload, costs and expertise. Further developments for a
diagnostic algorithm include the use of a fourth generation test and if reactive to
use a HIV-1 and HIV-2 discriminatory test and HIV viral load.
This study was cross-sectional and compared the performance of HIV RDT in
selected sites in Limpopo province that had introduced an intervention viz., an
internal quality control (IQC) as part of quality management system (QMS)
implementation, and compared to Mpumalanga province that had not introduced
the IQC and performed limited QMS activities. The sample size calculated for the
study was N = 717. IQC is an independent internal quality control that is used to
check that an analytical phase or test precision is optimal. The introduction of
routine QMS in Limpopo was through implementation of IQC supported by
appropriate training and certification of implementers. IQC was implemented
routinely as part of the provincial QA initiatives with the aim of supporting the
implementation of HIV RDT in non- laboratory settings. There are other QA
measures that may be implemented to support HIV RDT programmes including
external quality assessment (EQA) such as proficiency testing (PT) which is a tool
used to assess the testing process independently. EQA implementation was
however not part of the Limpopo (LP) QMS implementation. Six high volume
testing sites comprising of 3 hospitals and 3 clinics were selected per province.
This was to avoid the risk of not meeting the required number of participants due
to refusals, lack of results and challenges with reporting.
In order to mitigate risk, the study was oversampled, where a total of 457
participants from the LP sites were enrolled in the study and results were analysed
and compared to those of 361 participants from the Mpumalanga (MP) sites
resulting in a total sample size of 818. The analyses included demographics,
performance of RT as measured by the number of discordant results, reliability
and validity of rapid tests RT as measured by the sensitivity, specificity, positive
predictive value (PPV) and negative predictive value (NPV) results. The data
between Limpopo and Mpumalanga were further analysed together with the data
from selected sites from a non-governmental organisation (NGO) called New Start
and the performance, reliability and validity of the HIV test results were compared.
The main role of New Start was to offer HCT in support of the government
priorities and it implemented several different QMS measures for HIV rapid testing,
namely, IQC, EQA, PT and re-testing, training for implementers, development and
implementation of standard operating procedures (SOPs), and ensuring that all
commodities were stored under appropriate conditions including temperature
monitoring.
In order to determine the validity and reliability of HIV RDT against the gold
standard ELISA in Limpopo, Mpumalanga and New Start sites, the rate of
discordance, the sensitivity, specificity, PPV and NPV were determined. Logistic
regression models were constructed to assess the association between the
interventions in the provinces. Crude and adjusted odds ratios were used as a
measure of association between exposure and outcome and a 95% precision of
estimate was used to ascertain statistical significance. Exposure factors with
p<0.05 were considered statistically significant.
Results
A total of 947 attendees for HCT services in selected sites in Mpumalanga and
Limpopo provinces between August and April 2012, were screened and of these,
818 were enrolled into the study according to the study inclusion criteria. There
was no significant difference (p=0.05) between the number of participants enrolled
in Limpopo (457) as compared to Mpumalanga (361) though Limpopo enrolled
more participants than Mpumalanga. All available data from New Start sites for the
period 2008 was analysed. The gender, rate of discordance and HIV positivity rate
were significantly different between the two provinces (p<0.05). The study showed
that the laboratory-based HIV prevalence rate in each setting was 22.9% in
Limpopo, 26% in Mpumalanga and 11% in New Start sites. The prevalence rates
reported by Shisana, 2014, were 21.8% for Mpumalanga and 13.9% for Limpopo.
The rate of discordant HIV test results between the 2 provinces and New Start
sites was also measured where discordant results were defined as those that were
different between HIV rapid test and the ELISA test. The rate of discordant HIV
test results was 5.9% (27) in Limpopo, 11.0% (40) in Mpumalanga p= 0.010 and
1.4% (68) in New Start sites. False negative results accounted for all the
discordant results.
Logistic regression models were used to estimate the Odds Ratio (OR) and the
95% confidence interval of the association between implementation of QA
programme and the HIV test accuracy or the HIV discordance rate. Facilities
without a QA intervention programme had an approximately 2-fold increased odds
of HIV test discordance compared to facilities with a QA programme in place
(crude OR 1.86, 95% CI: 1.10 – 3.12 and adjusted OR 1.90, 95% CI:1.08 - 3.30).
This association was statistically significant. The sex and age of the participants
was not associated with discordance rate.
The sensitivities of the HIV RDT in Limpopo, Mpumalanga and New Start sites
were 86% (CI: 83.9-89.4), 72% (CI: 64.2-79.0) and 98% (CI: 97.6-98.4)
respectively. In this study, specificity ranged within 99% (CI: 98.9-99.9) in all sites
(Provinces and New Start sites). The PPV in Limpopo, Mpumalanga and New
Start sites were 98% (CI: 93.2-99.6), 97% (CI: 91.0-99.2) and 93% (CI: 92.3-93.7)
respectively, The NPV results in Limpopo were 93% (90.5-95.2), Mpumalanga at
86% (CI:81.3-90.7). For New Start sites, the NPV was 99.6% (CI: 99.4-99.8). The
sensitivities and specificities of the sites were used at a national prevalence rate of
18.8% to determine the national PPV and NPV and these were found to be 100%
(CI: 100-100) and 91.3% (CI: 89.04-92.96) respectively.
Discussion
In all three settings the World health Organisation (WHO) recommended sensitivity
(>99%) and specificity (>98%) were not met. There was a gradient of sensitivities
and specificities that was associated with the extent of QA implementation. Thus,
New Start sites with a more extensive set of QA activities had the highest
sensitivity; LP with introduction of IQC, had an intermediate sensitivity and MP the
lowest. Despite the introduction of an intervention LP was not able to meet the
required level of QA implementation compared to New Start. Increased
discordance was associated with the extent of implementation of QA as shown by
the results of the logistic regression model (crude and adjusted). In this study there
was a decline in sensitivity that resulted in some false negative results. To a lesser
extent, some false positive results were also identified in New Start sites. In the
case of LP and MP the potential contributory factors to false negative results
xi
would include the extent of QA implementation and training. Further evidence of
the relative poor implementation would include the M&E assessments and in the
course of the study there lost results, poorly taken and missing specimens that led
to data being excluded.
Conclusion
On the basis of these results, it is concluded that implementation of quality
assurance measures is critical to ensure correct diagnosis of rapid HIV testing.
Furthermore, implementation of a combination of aspects of QA is urgently
required including training of all implementing staff on quality assurance of rapid
HIV testing, monitoring and evaluation to assess kit performance through IQC and
PT, as well as implementation of the current South African HIV testing Services
(HTS) Policy. All PT methods should be explored for implementation and training
and certification of implementers must be ensured.
Description
A thesis submitted to the Faculty of Health Sciences, University of the
Witwatersrand, in fulfilment of the requirements for the degree of Doctor of
Philosophy
Johannesburg, 2016