International Journal of Psychology International Journal of Psychology, 2021 Vol. 56, No. 6, 895–907, DOI: 10.1002/ijop.12764 Validation of Oxford Cognitive Screen: Executive Function (OCS-EF), a tablet-based executive function assessment tool amongst adolescent females in rural South Africa Kirsten Rowe1,3 , Mihaela Duta2 , Nele Demeyere2 , Ryan G. Wagner3,4 , Audrey Pettifor3,5,6 , Kathleen Kahn3,4,7 , Stephen Tollman3,4,7 , Gaia Scerif2 , and Alan Stein1,3 1Department of Psychiatry, University of Oxford, Oxford, UK 2Department of Experimental Psychology, University of Oxford, Oxford, UK 3MRC/Wits Rural Public Health and Health Transitions Research Unit (Agincourt), School of Public Health, Faculty of Health Sciences, University of Witwatersrand, Johannesburg, South Africa 4Centre for Global Health Research, Umeå, University, Umeå, Sweden 5Department of Epidemiology, Carolina Population Centre, University of North Carolina, Chapel Hill, NC, USA 6Wits Reproductive Health and HIV Institute, University of the Witwatersrand, Johannesburg, South Africa 7INDEPTH Network, Accra, Ghana S hort, reliable, easily administered executive function (EF) assessment tools are needed to measure EF in low- and middle-income countries, particularly in sub-Saharan Africa given the prevalence of human immunodeficiency virus (HIV)-associated neurocognitive disorder. We administered Oxford Cognitive Screen—Executive Function (OCS-EF) to 932 rural South African females (mean age 19.7 years). OCS-EF includes seven tasks: two hot inhibition tasks (a modified Iowa Gambling Task, emotional go/no-go) and five cool EF tasks, two switching tasks (visuospatial rule-finding, geometric trails) and three working memory tasks (digit recall, selection and figure drawing). We performed confirmatory factor analysis testing whether a three-factor, two-factor hot-cool, two-factor working memory and inhibition/switching, or one-factor EF model fitted the data better. The three-factor (switching, inhibition and working memory) model had the best local and global fit (χ2 (11) 24.21, p = 0.012; RMSEA 0.036; CFI 0.920; CD 0.617). We demonstrated the feasibility of OCS-EF administration by trained laypeople, the tripartite structure of EF amongst adolescent females and the factorial validity of OCS-EF in this population and context. OCS-EF tablet-based cognitive assessment tool can be administered by trained laypeople and is a valid tool for assessing cognition at scale amongst adolescents in rural South Africa and similar environments. Keywords: Cognition; Executive function; Low- and middle-income countries; Africa; Adolescents. Correspondence should be addressed to Kirsten Rowe, Department of Psychiatry, University of Oxford, Warneford Hospital, Warneford Lane, Oxford, OX3 7JX, UK. ((E-mail: kirsten.rowe@psych.ox.ac.uk) / (E-mail: kirstenrowe@gmail.com)) The first author is funded by the Rhodes Trust, the Africa-Oxford initiative (AfiOx-32), the Society for Education and Music Psychology Research (SEMPRE) and the Gerry Farrell scholarship. Overall support for the HIV Prevention Trials Network (HPTN) was provided by the National Institute of Allergy and Infectious Diseases (NIAID), the National Institute of Mental Health (NIMH) and the National Institute on Drug Abuse (NIDA) of the National Institutes of Health (NIH) under Award Numbers UM1AI068619 (HPTN Leadership and Operations Centre), UM1AI068617 (HPTN Statistical and Data Management Centre) and UM1AI068613 (HPTN Laboratory Centre). The study was also funded under Award Number 5R01MH087118–02 and R24 HD050924 to the Carolina Population Centre. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Allergy and Infectious Diseases or the National Institutes of Health. AP and KK conceptualised and designed the parent HPTN 068 study. KK, ST, GS and MD conceptualised this substudy. KR, AS, GS, RGW, MD and ND designed the substudy. KR analysed the data and drafted the manuscript. All authors assisted with the interpretation of the data, revising the manuscript critically for important intellectual content and approved the final version of the manuscript for submission. © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. https://orcid.org/0000-0002-2005-3666 https://orcid.org/0000-0002-0435-571X https://orcid.org/0000-0003-0416-5147 https://orcid.org/0000-0003-2741-3676 https://orcid.org/0000-0002-3387-0817 https://orcid.org/0000-0003-3339-3931 https://orcid.org/0000-0003-0744-7588 https://orcid.org/0000-0002-6371-8875 https://orcid.org/0000-0001-8207-2822 896 ROWE ET AL. Research on executive function (EF) factor structure in adolescents from low- and middle-income countries (LMICs) in Africa is limited. EF is important for concentration and self-control. EF is associated with various important outcomes, for example, occupational performance, physical health, mental health and relational health (Diamond, 2013). EF is still developing during adolescence. Research within high-income countries suggests that EF differentiates with age, starting as a unidimensional ability in pre-schoolers then differen- tiating into two/three distinct abilities by adolescence or adulthood; however, models across the lifespan are inconsistent (Hughes et al., 2010; Karr et al., 2018; Shing et al., 2010). Re-analysis of factor-analytic studies found greater unidimensionality amongst child/adolescent samples, with unity and diversity in adults; three- and nested-factors (bifactor without inhibition) were most common amongst adolescent/adult samples (Karr et al., 2018). Three core EFs described in the three-factor model are working memory (maintaining and manipulat- ing retained information), switching (shifting from one learned rule/pattern to another) and inhibition (inhibiting automatic responses or ignoring distrac- tors); more complex EFs include planning (Friedman et al., 2008). Much of existing EF research focuses on cool EF (no affective/emotional component). An alternative model posits a broader EF conceptualisation including hot/affective EF. Hot EF involves cognitive control in affective contexts with choices between instant gratification and longer-term rewards (e.g., gam- bling tasks). Factor-analytic work in younger children found a two-factor hot-cool EF model superior to a one-factor model (Willoughby et al., 2011), with a single-factor model being superior at the transition to adolescence (Prencipe et al., 2011). During adolescence, developmental trajectories differ between dorsolateral prefrontal cortex-mediated cool EF (ascends linearly), and orbitofrontal cortex-mediated hot EF (U-shaped curve) (Poon, 2017). There is limited work examining a two-factor hot-cool model in adolescence. This study only included females; however, the literature shows that there are no sex-specific differences in overall EF (Grissom & Reyes, 2019). Research on EF in adolescents in Africa is lim- ited. Studies have examined intelligence or cognition broadly, rather than EF, with validation in small samples (Shuttleworth-Edwards et al., 2013). Minimal research on the validation of cognitive assessment tools has occurred in large, local samples. The Siyakhula cohort found an acceptable fit of a four-factor cognitive model amongst 7–11-year-olds including two non-EF factors (learning, simultaneous processing) and two EF-related factors (planning/inhibition/switching, working memory) (Rochat et al., 2017). Measuring EF is particularly rele- vant in human immunodeficiency virus (HIV)-prevalent settings, such as southern Africa, where HIV-associated neurocognitive disorder (HAND) is common. Despite antiretroviral therapy availability, milder forms of HAND affecting EF (including working memory, inhibition and switching) remain common in HIV-infected adolescents and adults (Hoare et al., 2016; Walker & Brown, 2018). Women with HIV tend to have greater neurocognitive impairment than men (Rubin et al., 2019). Young women in southern Africa also have a heightened risk of acquiring HIV compared to young men (Harrison et al., 2015). To determine the cognitive profile associated with HAND, it is important to establish the validity and local population norms of sensitive neuropsychological tasks to measure EF accurately. It is also vital to develop platforms for simple standardised administration by non-specialists in LMICs where there is limited access to formal neuropsy- chological testing. A task-shifting approach, involving the delegation of cognitive assessment from highly quali- fied medical doctors or psychologists to trained lay health workers with fewer qualifications, is necessary to enable widespread testing and diagnosis. Past paper-based screening by trained community health workers in South Africa (SA) highlighted the pitfalls of this approach; some of which were mitigated using highly automated tablet-based cognitive screening (Robbins et al., 2018). The original intention with the Oxford Cognitive Screen (OCS) was to develop an intuitive tablet-based cognitive screening platform for use globally across the lifespan. OCS was initially piloted in stroke pop- ulations (Demeyere et al., 2015). Oxford Cognitive Screen Plus (OCS-Plus) was then developed and tested more widely, including in ageing rural South African populations within the Medical Research Council (MRC)/Witwatersrand (Wits) Agincourt Health and Demographic Surveillance System (HDSS) (Demeyere et al., 2021; Humphreys et al., 2017). This demonstrated that large-scale fieldworker-administered tablet-based cognitive testing was feasible in rural SA. Three paper-based OCS-Plus EF tasks (trails, rule-finding, figure drawing) were then piloted amongst adolescents in this setting (Rosenberg et al., 2018). Challenges included standardising implementation; and increased time, cost and risk of data-capturing errors. A tablet-based version was then developed to help standardise administration with on-screen instructions, pre-programmed dura- tion cut-offs, and minimising examiner interpretation by automating most scoring. The current paper intro- duces this tablet-based version, the Oxford Cognitive Screen—Executive Function (OCS-EF), OCS-Plus adapted for adolescents. A verbal working memory task (digit recall) and two hot EF tasks (Iowa Gambling Task; emotional go/no-go) were added to the OCS-Plus EF tasks (trails, figure drawing, rule-finding, selection). The OCS-EF was then included in the Year 4 wave of assessments within the HIV Prevention Trials Net- work (HPTN) 068 study, focused on HIV prevention © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. VALIDATION OF OCS-EF 897 in young women in rural SA. This enabled OCS-EF validation amongst adolescent girls and young women (reported here), and examination of associations between EF, risk-taking and HIV (Rowe et al., 2020). Objectives of this paper were to determine the: 1. feasibility of large-scale fieldworker-administered adolescent cognitive assessment using OCS-EF in rural SA. 2. OCS-EF task-by-task performance by a community-based adolescent sample in rural SA. 3. OCS-EF factorial validity in rural SA by confirming the factor structure. METHODS Study design This validation study utilised cross-sectional data collected during HPTN 068, a longitudinal cohort post-randomised controlled trial in the MRC/Wits Rural (Agincourt) Research Unit in SA. This region has high youth unemployment (∼ 75%), poverty and lacks quality education and work opportunities. Most households rely on government social grants and have limited access to water and basic sanitation. The population is predomi- nantly black Tsonga-speaking Africans. Adolescent HIV prevalence is high affecting females disproportionately (> 1/4 20–24-year-olds) (Gomez-Olive et al., 2013). HPTN 068 trial results are reported elsewhere (Pettifor et al., 2016). The aim of the trial was to test whether providing cash transfers to young women and their households, conditional on school attendance, reduced young women’s risk of acquiring HIV, compared to a control group. Eligibility criteria for trial participa- tion included: 13–20-year-old females in grades 8–11 at government schools; not married or pregnant; able to complete tablet-based questionnaires alone; having documentation to open a bank account (to receive cash transfers safely); having a parent/guardian at home with similar documentation; currently living and intending to reside in the region until study completion. Eligibility criteria for this study (which occurred 5-year post-trial enrolment) included: 17–25-year-old females; confirmed HIV-negative status; a complete, single cognitive dataset. Compliance with Ethical Standards: Ethical approval was obtained for the study from the University of Witwa- tersrand Human Research Ethics Committee, the Univer- sity of North Carolina Institutional Review Board and the Oxford Tropical Research Ethics Committee. All study procedures were performed in accordance with the eth- ical standards of the 1964 Helsinki Declaration and its amendments. Informed consent was obtained from all individual adult participants included in the study; caregiver consent Table 1 Sociodemographic characteristics of study participants Characteristic (n = 932) Female sex (%) 100 Black African ethnicity (%) 100 Home language Tsonga (%) 100 Age Mean (SD) 19.71 (1.34) Range 17–25 Highest level of education (%) Some high school 27.71 Completed high school 33.30 More than high school 38.99 Socio-economic status (%) Low (lowest quartile) 27.07 Middle (middle two quartiles) 49.51 High (highest quartile) 23.42 with participant assent was obtained for minors under 18-years-old. Sample characteristics See Table 1 for sample sociodemographic characteristics. Materials and administration Socio-demographic data were obtained with Audio Computer-Assisted Self-Interview (ACASI) (self-administration being useful for sensitive data) and fieldworker-administered Computer-Assisted Personal Interview (CAPI), both allowing immediate tablet-based data-capturing. Questions and task instructions were translated into Tsonga by research staff. HIV screening, including pre- and post-test counselling, was performed at each visit. OCS-EF, developed using MATLAB and Psy- chophysics Toolbox, runs as a stand-alone Win- dows/Android application with a fixed task order and full written on-screen task instructions to standardise administration. OCS-EF aims to be culturally unbiased, brief (∼ 30 minutes) and easy for trained laypeople to administer. We administered OCS-EF on Windows Sur- face Pro tablets using dedicated styluses (touch input disabled). An overview of the seven tasks follows. Four tasks have easier baseline conditions before test conditions: digit recall; trails; selection; figure drawing. Trails is a geometric trail-making task assessing planning and switching. A practice round precedes two baseline condi- tions (connecting circles in ascending order of size then connecting squares in descending order) and test condi- tion (switching between circles and squares, squares in descending order of size and circles in ascending order). Figure drawing assesses visuospatial constructional © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. 898 ROWE ET AL. ability, visuospatial working memory and planning. The baseline condition involves copying a 20-component composite drawing and the test condition (immediate recall) involves drawing the same figure from memory after seeing it again briefly. Rule-finding assesses visu- ospatial problem-solving and switching. A red dot moves from one shape to another in a matrix of 24 shapes (three columns: squares-triangles-squares) following one of five predefined rules. Participants predict the dot’s next move; the preceding position is highlighted. Switches to a new rule are unsignalled. Selection assesses selective attention (baseline visible), and visuospatial working memory and planning (test invisible). Participants are presented with 60 items: 30 vegetables (10 of each type); 30 fruit (10 of each type). They have to select fruit (targets) only, ignoring vegetables (distractors). In the visible condition, selected items remain highlighted throughout; whereas in the invisible condition, they only remain highlighted for a few seconds. Digit recall is a classic, widely used task. Digit span is the longest sequence length recalled correctly in each condition. Forwards (baseline) span assesses verbal short-term memory (storage); backwards (test) span assesses verbal working memory. Examiners read progressively longer digit sequences (two to nine digits’ long) out loud. Participants recite the sequences forwards or backwards. Correct recital results in progres- sion to the next level. Three failed attempts (different sequences) at a level terminates the task condition. Emo- tional go/no-go is based on Hare’s task implementation (Hare et al., 2008). Participants are presented sequen- tially with facial visual stimuli (three women with happy, neutral and fearful facial expressions). Participants have to tap neutral faces (targets), ignoring happy/fearful faces (distractors). There is a practice block (six trials) and four test blocks (36 trials each). The Iowa Gambling Task is based on the Children’s Gambling Task implementa- tion (Kerr & Zelazo, 2004) assessing risk-taking in the context of uncertain reward. Participants are presented with two decks of cards: the advantageous, low-risk deck (low rewards, low penalties with net gain); and the disadvantageous, high-risk deck (high reward, high loss with net loss). Participants need to maximise their total score by learning the nature of the decks. They select a card from each deck for 50 trials and see their updated total score on-screen after each selection. See Supplementary Material S1 for full task and outcome measure descriptions. Stimuli were adapted for the cultural context (selec- tion using common local fruit and vegetables; go/no-go using grayscale photographs of three NimStim African American women’s faces) (Tottenham et al., 2009). Task instructions were read in Tsonga. Testing was performed in testing rooms or at participants’ homes (< 10%) by locally employed, trained female fieldwork- ers, fluent in English and Tsonga. Fieldworkers required a 12th grade formal education and preferably previous research fieldwork experience. None had tertiary quali- fications. Quality control measures included fieldworker self-checks, cross-checks and random supervisor checks. After each task, examiners were prompted to select a single test condition: • no issue • participant issues: no speech, visual problems, motor problems, auditory problems, fatigue, refused, ran out of time • environmental/technical: technical problems, interrup- tions, other • examiner error. Selection of anything other than “no issue” resulted in participant task invalidation. Task timing was recorded automatically. Accuracy was scored automatically except for tasks requiring interpretation of drawings (figure) or speech (digits). The figures were marked blindly by a researcher using a standardised figure-scoring application with cross-checks. Each drawing component was scored for presence, accuracy and position. In 2018, a post-study fieldworker feedback session with anonymous questionnaires containing both general and task-specific open-ended and Likert-style questions obtained qualitative data about OCS-EF administration. Data analysis Data analysis was performed in Stata 14. Participants with invalidated data due to test condition errors had data points estimated using maximum likelihood estima- tion. Analyses of variance (ANOVAs) by age, education level and socio-economic status (asset index) were per- formed to assess relationships between all the EF task outcome measures and these sociodemographic variables. Comparison between baseline and test condition accu- racy was performed graphically for trails, figure, selection and digit to see if the expected shifts-to-the-left occurred. Confirmatory factor analyses compared four different EF models: 1. three-factor (Miyake et al., 2000): switching (indi- cators: trails, rule-finding), inhibition (indicators: go/no-go, Iowa), working memory (indicators: digit, selection, figure). 2. hot-cool (Zelazo & Müller, 2002): hot (indicators: go/no-go, Iowa), cool (indicators: trails, rule-finding, digit, selection, figure). 3. two-factor (switching/inhibition, working mem- ory): working memory (indicators: digit, selection, figure) and inhibition/switching (indicators: trails, rule-finding, go/no-go, Iowa). 4. one-factor: same indicators all loading onto a single factor. © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. VALIDATION OF OCS-EF 899 A single reflective indicator per task was included. For trails, other indicators were tested with similar results (not reported). We selected at least two indicators per latent variable (factor) so the models could be identi- fied. We assumed latent variables have a standardised measurement unit fixing variances to 1 (Schumacker & Lomax, 2004). These indicators were selected: • Trails: switch accuracy cost (alternative scores tested: trails switch accuracy; trails switch duration cost). • Iowa: net score. • Emotional go/no-go: D-prime. • Rule-finding: accuracy. • Selection: invisible accuracy. • Figure: figure recall accuracy. • Digit: backwards span. Other task scores derived to assess estimates of local cognitive performance: • Trails: baseline combined accuracy. • Iowa: learning slope regression coef ficient (beta). • Emotional go/no-go: – Response bias criterion C. – Mean go reaction time. Selection: visible accuracy. • Figure: copy accuracy; copy duration; recall duration. • Digit: forwards span. Extreme collinearity (R2 smc > 0.90) was not present. Maximum likelihood with missing values with observed information matrix estimation was used. The model goodness-of-fit was assessed locally by examining parameter estimate sizes, standardised resid- ual variances and squared multiple correlation coef- ficients, a measure of indicator reliability. Standard- ised regression coefficients were interpreted as Pear- son’s correlations using Cohen’s rule: large r > |0.5|; medium r > |0.3|; small r > |0.1| (Cohen, 1988). Standard- ised residual variances were interpreted like standard- ised z scores (>1.96 suggesting model misspecification) (Schumacker & Lomax, 2004). Global model fit was assessed using these fit indices: likelihood-ratio χ2 test, Steiger-Lind root mean squared error of approximation (RMSEA), comparative fit index (CFI) and coefficient of determination (CD) (close to 1 indicating good fit). Comparative model fit was assessed using a 𝜒2 difference test. We used these cut-offs for good model fit: RMSEA < 0.06; CFI> 0.95 (Hu & Bentler, 1999). The saturated model is delineated in Supplementary Material S2. Qualitative data from fieldworker questionnaires were summarised by the cognitive researcher and short responses analysed using a semantic, inductive approach to thematic analysis. RESULTS Sample characteristics and feasibility A total of 954 eligible 17–25-year-old females completed the OCS-EF; 22 were excluded due to missing/duplicate files; 932 (98%) participants were included (see Figure 1). Participants with invalid task data (see Figure 2) (n = 144; 15%) were included using maximum likelihood esti- mates. All tasks had > 90% valid data. The most common technical problem was tablet freezing during hot weather. This sometimes resulted in examiners double-clicking (due to tablets being unresponsive) and accidentally skip- ping trails/selection task sections (most examiner errors). Tasks with baseline and test conditions (i.e., multiple task sections) tended to have more issues and resul- tant invalidation. Trails had the most issues (7.8%: 5% task-skipping examiner error; 2% tablet freezing; 0.8% other with two interruptions). Selection also had a num- ber of issues (3.6%: 2% task-skipping examiner error; 1% tablet freezing; 0.6% other including one partici- pant refusal and one interruption). Digit recall had a few participant issues (2.6%: 1.2% participant fatigue; 0.8% technical issue; 0.6% other including three interruptions). Figure drawing also had > 1% invalidated data (1.1%: 0.8% technical issue; 0.3% other with one participant refusal). The remaining tasks had no or minimal inval- idation: Iowa 0.2% technical issues; Rule-finding 0.1% examiner error; Go/no-go no issues. See Table 2 for task durations (all positively skewed). Fieldworker feedback Eight of the 19 fieldworkers (42%) completed post-study feedback questionnaires. Questionnaire data analysis indicated that most fieldworkers found OCS-EF easy to administer citing the app’s similarity to a mobile phone menu and the clear task instructions in the local language as reasons. They felt it was more meaningful to admin- ister the tasks when they understood the rationale and the abilities being assessed by each task. Fieldworkers suggested improving tablets to withstand the hot climate and adapting software to minimise/allow correction of resulting examiner error, particularly during the trails task. Fieldworkers felt participants found the trails base- line condition easy and the switching test condition more challenging. They described varying perceptions of rule-finding and Iowa Gambling task difficulty; some participants learned the rules/patterns while others did not. They reported that most participants found selection and figure easy. Emotional go/no-go was challenging to © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. 900 ROWE ET AL. Figure 1. OCS-EF validation substudy flow diagram. 0 1 2 3 4 5 6 7 8 9 10 Trails Iowa Go/no-go Rules Figure Selection Digit % in va li d Participant Examiner Environment/technical Figure 2. Reasons for invalid task test conditions. administer; its long duration sometimes resulted in par- ticipant fatigue. Digit recall was the most intensive task to administer. The task duration was long if participants reached the lengthier sequences, sometimes resulting in participant fatigue. It also required fieldworkers to concentrate intensely while listening to and capturing participant responses on screen. The main OCS-EF administration challenge was interruptions, particularly at participant homes. Interruptions made it harder for participants to concentrate and perform well, particularly on the more challenging tasks. Fieldworkers reported that most participants found it resulted in them struggling to complete the task. Easier tasks, specifically trails baseline and selection, were sometimes associated with participant boredom, according to the fieldworkers. Task performance Summary statistics are presented in Table 3. A pairwise correlation matrix is presented in Supplementary Mate- rial S3. There were ceiling effects (negative skewness) for Table 2 Table of summary statistics of recorded task durations Task Condition Mean (SD) duration (seconds) Median (IQR) duration (seconds) Range (seconds) Trails Baseline 1 15.85 (9.91) 14.07 (11.09, 17.76) 2.89, 184.98 Baseline 2 15.78 (8.50) 14.097 (11.13, 18.23) 2.73, 129.69 Switch 35.18 (23.00) 32.15 (22.89, 42.94) 6.76, 423.10 Selection Visible 42.96 (18.03) 38.605 (32.76, 48.51) 12.90, 218.69 Invisible 34.16 (11.52) 32.460 (26.73, 39.77) 5.62, 115.65 Figure drawing Copy 75.32 (27.80) 70.12 (57.81, 87.16) 13.78, 294.54 Recall 56.59 (24.57) 52.16 (40.74, 68.01) 4.59, 183.09 Rule-finding — 121.38 (48.46) 114.42 (92.60, 144.10) 22.17, 442.30 © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. VALIDATION OF OCS-EF 901 Table 3 Table of task performance summary variables Task Measure N Mean (SD) Median (IQR) Range (min, max) Shape of distribution Trails Baseline accuracy 851 0.79 (0.23) 0.86 (0.64, 1.00) 0.00, 1.00 Negative-skew Switch accuracy 851 0.52 (0.31) 0.50 (0.29, 0.79) 0.00, 1.00 Negative-skew Switch accuracy costa 827 0.62 (0.32) 0.64 (0.36, 1.00) 0.00, 1.00 Negative-skew Switch duration cost 851 1.19 (0.63) 1.09 (0.77, 1.47) 0.11, 4.74 Positive-skew Iowa Coefficient beta of slope gradient 931 11.78 (30.66) 12.16 (−11.33, 41.50) −65.26, 67.67 Bimodal: central and positive peaks Net scorea 931 0.31 (0.57) 0.13 (−0.07, 1.00) −1.00, 1.00 Bimodal: central and positive peaks Go/no-go D primea 930 2.12 (0.96) 2.18 (1.54, 2.78) −1.11, 4.87 Normal Bias criterion C 930 0.10 (0.55) 0 (−0.26, 0.42) −1.95, 1.62 Normal Mean go trial reaction time (milliseconds) 932 511.33 (66.33) 501.29 (467.44, 543.33) 335.99, 847.53 Slight positive-skew Rule-finding Accuracya 931 0.48 (0.23) 0.46 (0.30, 0.65) 0.00, 1.00 Slight positive-skew Figure Copy accuracy 921 0.89 (0.11) 0.92 (0.83, 0.97) 0.00, 1.00 Negative-skew Recall accuracy* 921 0.80 (0.15) 0.82 (0.70, 0.90) 0.03, 1.00 Negative-skew Selection Visible accuracy 898 0.94 (0.11) 0.97 (0.93, 1.00) −0.27, 1.00 Negative-skew Invisible accuracya 898 0.72 (0.22) 0.77 (0.60, 0.87) −0.87, 1.00 Negative-skew Digit Forwards 905 5.40 (1.49) 5 (5, 6) 0, 9 Normal Backwardsa 905 3.67 (1.55) 4 (3, 4) 0, 9 Normal a Indicator in confirmatory factor analysis. trails, selection and figure accuracy, especially the base- line conditions. Iowa net scores had a bimodal distribution (central and high peaks). Plots of the four tasks with base- line and test conditions are presented in Supplementary Material S4. Task performance by sociodemographic variables Late adolescence (≥20 years) was associated with poorer performance on digits forwards [F (1, 866) = 8.47, p = 0.0037] and digits backwards [F (1, 866) = 18.09, p< 0.0001], trails baseline accuracy [F (1, 818) = 12.88, p = 0.0004] and switch duration cost [F (1, 818) = 6.75, p = 0.0095], figure recall accuracy [F (1, 883) = 4.27, p = 0.0391] and rule-finding accuracy [F (1, 892) = 8.20, p = 0.0043]. Education level was associated with bet- ter performance on the trails baseline accuracy [F (2, 818) = 4.84, p = 0.0081] and switch duration cost [F (2, 818) = 3.23, p = 0.0399], figure recall accuracy [F (2, 883) = 4.32, p = 0.0136] and rule-finding accuracy [F (2, 892) = 9.21, p = 0.0001]; however, it was nega- tively associated with visible selection [F (2, 859) = 3.37, p = 0.0349] and invisible selection [F (2, 859) = 4.09, p = 0.0171]. Socio-economic status was not associated with performance on any EF task. Confirmatory factor analysis Estimates of standardised pattern coefficients between factors and indicators (measured variables), latent fac- tors (and their covariances), intercepts and residuals for trails 1.9 ε1 .93 rules 2.1 ε2 .89 Iowa .54 ε3 .87 gonogo 2.2 ε4 .82 digits 2.4 ε5 .97 selection 5.2 ε6 .95 TS 1 IC 1 WM 1 figure 5.4 ε7 .59 .26 .33 .36 .43 .18 .23 1.3 .7 -.0023 .64 Figure 3. Model 1, three-factor, confirmatory factor analysis estimates. IC, inhibition; TS, switching; WM, working memory. the four EF models are shown in Figure 3 (Model 1: three-factor), Figure 4 (Model 2: two-factor hot-cool), Figure 5 (Model 3: two-factor working memory inhibi- tion/switching) and Figure 6 (Model 4: one-factor). For Figures 3-6, values on curved arrows are standardised co-variances between latent factors. Values inside ovals are latent variable variances (fixed to 1). Values on straight arrows (paths) are standardised beta coefficients. Values © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. 902 ROWE ET AL. trails 1.9 ε1 .91 rules 2.1 ε2 .77 Iowa .54 ε3 .89 gonogo 2.2 ε4 .78 digits 2.4 ε5 .98 selection 5.2 ε6 .97 Cool 1 Hot 1 figure 5.4 ε7 .93 .3 .48 .33 .47 .13 .17 .73 .27 Figure 4. Model 2, hot-cool, confirmatory factor analysis estimates. trails 1.9 ε1 .9 rules 2.1 ε2 .81 Iowa .54 ε3 .89 gonogo 2.2 ε4 .82 digits 2.4 ε5 .96 selection 5.2 ε6 .92 WM 1 IC/TS 1 figure 5.4 ε7 .74 .31 .44 .34 .42 .21 .29 .32 .51 Figure 5. Model 3, two-factor working memory inhibition/switching, confirmatory factor analysis estimates. IC, inhibition; TS, switching; WM, working memory. in each indicator block are standardised intercepts. Values to the right of error terms (ε) are residual variances. trails 1.9 ε1 .9 rules 2.1 ε2 .8 Iowa .54 ε3 .91 gonogo 2.2 ε4 .84 digits 2.4 ε5 .99 selection 5.2 ε6 .98 EF 1 figure 5.4 ε7 .95 .31 .45 .31 .4 .1 .15 .22 Figure 6. Model 4, one-factor, confirmatory factor analysis estimates. EF, executive function. Coefficient and covariance estimates are also presented with standard errors, 95% confidence intervals, z statis- tics, probability values and squared multiple correla- tion coefficients in Tables 4-7 for Models 1–4, respec- tively. With regards to local fit, no standardised residuals were high enough to suggest model misspecification. Correlations between indicators and latent variables var- ied in size. They were medium in all four models for the rule-finding, go/no-go and Iowa Gambling Task indi- cators with models explaining 10–25% of each indi- cator’s variance. The correlations were small for the digit and selection indicators with models explaining <10% of each indicator’s variance. Sometimes they var- ied between models. They were large for figure in the three-factor and two-factor models and small in the one-factor and hot-cool models. They were medium for trails in the two- and one-factor models, and small in the three-factor and hot-cool models. In the three-factor model, switching was significantly and positively corre- lated with both inhibition and working memory; how- ever, inhibition and working memory were uncorrelated. Global goodness-of-fit indices are presented in Table 8. RMSEA, CFI and CD were all most favourable in model 1 (three-factor) with model 3 (working memory inhibi- tion/switching) having the next best fit (RMSEA < 0.06 for both). Models 2 (hot-cool) and 4 (one-factor) had a higher RMSEA (> 0.06) suggesting worse fit. No model had CFI> 0.95 but model 1 (three-factor) came clos- est (CFI 0.92). Likelihood-ratio χ2 tests revealed that © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. VALIDATION OF OCS-EF 903 Table 4 Model 1 (three-factor) parameter estimates Estimate Coefficient SE z p> |z| 95% CI SMC TS ➔ Trails .27 .06 4.68 < 0.001 .16, .38 .07 TS ➔ Rules .35 .07 5.18 < 0.001 .22, .49 .12 IC ➔ Iowa .37 .05 6.95 < 0.001 .27, .47 .14 IC➔ Go/no-go .43 .06 7.29 < 0.001 .31, .54 .18 WM➔ Digit .18 .06 3.29 0.001 .07, .29 .03 WM➔ Selection .23 .06 4.03 < 0.001 .12, .34 .05 WM➔ Figure .62 .12 5.23 < 0.001 .39, .85 .38 Cov (TS, IC) 1.26 .26 4.91 < 0.001 .76, 1.77 — Cov (TS, WM) .71 .19 3.76 < 0.001 .34, 1.08 — Cov (IC, WM) .02 .02 0.19 0.850 −.18, .21 — Note: Cov = covariance; Coefficient = standardised coefficient; IC = inhibition; SE = standard error; SMC = squared multiple correlation coefficient; TS = switching; z = z statistic; WM = working memory. Table 5 Model 2 (hot-cool) parameter estimates Estimate Coefficient SE z p> |z| 95% CI SMC Cool ➔ Trails .30 .06 4.76 < 0.001 .18, .42 .09 Cool ➔ Rules .50 .07 7.29 < 0.001 .37, .64 .25 Cool ➔ Digit .13 .05 2.36 0.018 .02, .23 .02 Cool ➔ Selection .17 .06 2.93 0.003 .06, .28 .03 Cool ➔ Figure .27 .06 4.60 < 0.001 .15, .38 .07 Hot ➔ Iowa .34 .06 5.95 < 0.001 .23, .46 .12 Hot ➔ Go/no-go .46 .07 6.39 < 0.001 .32, .60 .21 Cov (cool, hot) .74 .13 5.83 < 0.001 .49, .99 — Note: Cov = covariance; Coefficient = standardised coefficient; SE = standard error; SMC = squared multiple correlation coefficient; z = z statistic. Table 6 Model 3 (two-factor working memory, inhibition/switching) parameter estimates Estimate Coefficient SE z p> |z| 95% CI SMC WM ➔ Digit .21 .06 3.40 0.001 .09, .33 .04 WM ➔ Selection .28 .07 3.92 < 0.001 .14, .42 .08 WM ➔ Figure .51 .11 4.55 < 0.001 .29, .74 .26 IC/TS ➔ Trails .31 .06 5.51 < 0.001 .20, .42 .10 IC/TS ➔ Rules .46 .06 7.49 < 0.001 .34, .58 .21 IC/TS ➔ Iowa .34 .05 6.47 < 0.001 .24, .45 .12 IC/TS ➔ Go/no-go .41 .06 7.26 < 0.001 .30, .53 .17 Cov (WM, IC/TS) .34 .10 3.41 0.001 .15, .54 — Note: Cov = covariance; Coefficient = standardised coefficient; IC = inhibition; SE = standard error; SMC = squared multiple correlation coefficient; TS = switching; WM = working memory; z = z statistic. Table 7 Model 4 (one-factor) parameter estimates Estimate Coefficient SE z p> |z| 95% CI SMC EF ➔ Trails .31 .06 5.44 < 0.001 .20, .42 .10 EF ➔ Rules .47 .06 7.78 < 0.001 .35, .59 .22 EF ➔ Digit .11 .05 1.97 0.048 .00, .21 .01 EF ➔ Selection .15 .05 2.84 0.004 .05, .26 .02 EF ➔ Figure .23 .06 4.02 < 0.001 .12, .34 .05 EF ➔ Iowa .31 .05 5.91 < 0.001 .21, .42 .10 EF ➔ Go/no-go .39 .06 6.91 < 0.001 .28, .51 .15 Note: Coefficient = standardised coefficient; EF = executive function; SE = standard error; z = z statistic; SMC = squared multiple correlation coefficient. © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. 904 ROWE ET AL. models 2–4 were more significantly different to satu- rated models (p’s< 0.001) than the three-factor model (p = 0.012). The χ2 difference test revealed a significant difference between the two best models, three-factor and two-factor working memory inhibition/switching model (χ2 differ- ence = 25.28; df = 2; p< 0.0001). We retained the three-factor model as it had the best fit on multiple indices. DISCUSSION Main findings We demonstrated the feasibility of large-scale fieldworker-administered adolescent cognitive test- ing with the OCS-EF in rural SA. Fieldworker feedback re-administering the OCS-EF and data integrity were gen- erally good. All tasks had > 90% valid data (most tasks > 98% valid data); however, there were two tasks, trails and selection, where invalidation was more common, due to tablets freezing during these tasks in hot weather, resulting in fieldworkers sometimes double-clicking and accidentally skipping task sections. This problem was likely due to a combination of technical (both hardware and software), climate and fieldworker factors; however, the solution would primarily be technical as described in the recommendations section below. Optimising the testing environment to minimise interruptions and further automating the digit recall task could reduce data loss and the potential for human error, and thus improve data quality further. The feasibility of large-scale cogni- tive testing and research in these rural, underdeveloped villages was likely facilitated by the existing HDSS platform of the MRC/Wits Agincourt Research Unit with its skills development ethos, experience with training fieldworkers, experience with electronic tablet-based data collection with the support of a skilled data team, exist- ing quality control standard operating procedures, and well-established relationships with community leaders and members. This study, to our knowledge, represents the first attempt to confirm EF factor structure in a large community-based sample of late adolescents in Africa. The large sample size means parameter estimates are likely stable and that testing across a wide age range was possible. Similar to studies performed elsewhere amongst adolescent/adult samples, we found a differen- tiated EF model fit the data better than a single-factor or undifferentiated EF model (Karr et al., 2018); however, our models were unique because they included both hot and cool EF. We found a differentiated three-factor model including hot (inhibition) and cool (working memory, switching) EF factors fitting best, followed by the two-factor (working memory, combined inhibi- tion/switching) model. This differentiated three-factor model incorporating hot and cool EF found switching and inhibition were highly correlated; however, collapsing them into a single switching/inhibition factor (Model 3) decreased global model fit. Switching and working memory were also correlated; however, collapsing them into a single cool factor in the two-factor hot-cool model also decreased global model fit. Inhibition and working memory factors were uncorrelated, possibly because the inhibition factor only included hot EF tasks. All the differentiated models fit better than an undifferentiated model. The three-factor model fitting better than the hot-cool model suggests that the differentiation of global EF into switching, inhibition and working memory may be more important than the hot-cool EF differentiation; however, the final three-factor model incorporates the hot-cool EF differentiation, with inhibition being the hot EF factor, so this hot-cool differentiation is likely also relevant. These findings suggest that, like in high-income coun- tries, EF differentiates into distinct, related factors by late adolescence (Miyake & Friedman, 2012). In contrast to Miyake’s updated EF model, we did not assess cool inhibition as a common, underlying factor because we only used hot inhibition tasks. Although the emotional go/no-go task measures inhibition comparably to the non-emotional analogue (Schulz et al., 2007), Iowa Gambling Task performance has generally not been sig- nificantly associated with cool EF (Toplak et al., 2010). Yet, in this study, the inhibition factor onto which the Iowa Gambling Task loads was correlated with the cool EF switching factor. This commonality may represent the pattern-recognition/learning component that is also present in both the Iowa Gambling (inhibition) and rule-finding (switching) tasks. The lack of correlation between inhibition and working memory factors proba- bly explains why a one-factor model fitted poorly. Low digit factor loading was surprising given its frequent use globally. Measurement error may be responsible. It required fieldworker input to capture responses correctly while participants verbalised responses. Rule-finding, Iowa and go/no-go tasks explained greater proportions of the shared variance than selection, digit and trails tasks, with mixed findings for figure draw- ing. Some tasks may have explained lower proportions because they also measure other aspects of cognition. Digit’s low loading on the working memory factor may be because the visuospatial working memory tasks (invis- ible selection; figure recall) reflected planning more than visuospatial working memory. This might explain why factor loadings were higher for selection and figure than for digits backwards, a traditional working memory task. The three-factor model fit was likely good because the figure factor loading was high in this model; it decreased substantially when the working memory and switching factors were collapsed into a single factor (hot-cool © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. VALIDATION OF OCS-EF 905 Table 8 Goodness of fit indices: models 1–4 Goodness-of-fit index Model 1 (3-factor) Model 2 (hot-cool) Model 3 (Working memory, inhibition/ switching) Model 4 (1-factor) Likelihood ratio 24.21 69.93 49.48 72.89 χ2 test (model vs. saturated) 11 13 13 14 Degrees of freedom (df ) p value 0.012 < 0.001 < 0.001 < 0.001 Population error Root mean squared error of approximation (RMSEA) 0.036 0.070 0.056 0.068 Baseline comparison: Comparative fit index (CFI) 0.920 0.655 0.779 0.643 Size of residuals Coefficient of determination (CD) 0.617 0.506 0.600 0.427 and one-factor models). The three-factor model thus explained a higher proportion of the overall variance. There were varying findings when examining the asso- ciations between EF and sociodemographic factors. First, in terms of age associations, late adolescence being associated with poorer EF in this study does not fit with the broader literature but may be due to uninten- tional selection bias. Older participants were by definition educationally delayed at recruitment (18–20-year-olds still in grades 8–11 despite the expected age range for these grades being 13–17-years-old). Educational delay is more common in rural South Africa; exclusion of these older participants would not have been represen- tative of a typical rural secondary school population (Romero et al., 2018). Second, education level was sig- nificantly associated with better performance on trails baseline accuracy and switch duration cost, figure recall accuracy and rule-finding accuracy. It was not associ- ated with Iowa Gambling, go/no-go or digit tasks. These findings generally fit the pattern of cool EF, rather than hot EF, being associated with academic performance (Poon, 2017). Education level was negatively associated with selection, perhaps due to perceived task ease result- ing in boredom and carelessness. Finally, socio-economic status was likely not associated with cognitive perfor- mance as these stratifications were fine gradations within a relatively homogenous low-income population. Limitations There are several study limitations. It was difficult to ensure standardised laboratory-like test conditions in this context at scale. A further limitation is sample composi- tion: it is likely not fully representative of the local popu- lation as it excluded some of the most vulnerable women given the trial’s eligibility criteria. We only tested cogni- tion at one time-point so we could not assess test–retest reliability or predictive validity. Finally, task order meant the two inhibition tasks (sim- ilarly the two working memory tasks) were adminis- tered sequentially. This may have introduced error due to participant-by-order interaction and caused poorer perfor- mance on the second task each time (emotional go/no-go; digit recall) (Miyake et al., 2000). Fatigue was common in the last task, digit recall. Recommendations There are improvements that could be made at various lev- els. At implementation level, ongoing fieldworker train- ing with open communication is vital for quality assur- ance. Test conditions should be optimised within the con- textual constraints (e.g., mobile testing van). In terms of hardware, “rugged” tablets would be suitable for this environment. Software can be improved by: program- ming lock-in periods to prevent accidental section skip- ping or inserting a “back” button; amending task order to avoid sequential administration of two tasks assessing the same EF. Examiner digit recital can be pre-recorded and played off-screen to standardise administration further. Participant input of digit sequences could also minimise potential examiner input error. Full scoring automation with immediate on-screen results could be developed to increase clinical utility. Research gaps Although this study presents proof-of-concept for the fea- sibility and factorial validity of OCS-EF, there are many gaps to be explored in future research, including fur- ther work on implementation and quality control in rural African contexts, and assessing different types of valid- ity. There is scope for more in-depth qualitative research with cognitive fieldworkers and participants to assess the tasks’ face validity, and the impact of environmental fac- tors (e.g., testing environment) on data quality. In terms © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. 906 ROWE ET AL. of factorial validity, replication of the three-factor struc- ture in other African adolescent samples (especially urban and/or males) will increase confidence in result stabil- ity. Given that HAND is a common cause of cognitive deficits in young people in Africa, the utility and diag- nostic validity of OCS-EF in identifying and describing cognitive profiles in people living with HIV needs to be explored. Predictive validity should also be assessed by examining associations with relevant occupational (e.g., employment) and behavioural outcomes (e.g., risky sex- ual behaviour) (Rosenberg et al., 2018). CONCLUSION This study confirmed the factorial validity of the OCS-EF tablet-based EF assessment tool in a large community-based sample of adolescent females in rural South Africa. It is likely the largest study to examine EF factor structure in adolescents in Africa. Vitally, it has established informed estimates of cognitive sta- tus of rural adolescent females for future comparative work. OCS-EF will provide clinicians and researchers with a platform for measuring EF quickly and easily in LMICs with limited access to formal neuropsychological assessment. Manuscript received May 2020 Revised manuscript accepted March 2021 First published online May 2021 SUPPORTING INFORMATION Additional supporting information may be found online in the Supporting Information section at the end of the article. Supplemental material 1 Supplemental material 2 Supplemental material 3 Supplemental material 4 REFERENCES Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates. Demeyere, N., Haupt, M., Webb, S., Strobel, L., Milosevich, E., Moore, M., Wright, H., Finke, K., & Duta, M. (2021). Introducing the tablet-based Oxford Cognitive Screen-Plus (OCS-Plus) as an assessment tool for subtle cognitive impair- ments. Scientific Reports. https://doi.org/10.1038/s41598- 021-87287-8 Demeyere, N., Riddoch, M. J., Slavkova, E. D., Bickerton, W. L., & Humphreys, G. W. (2015). The Oxford cognitive screen (OCS): Validation of a stroke-specific short cognitive screening tool. Psychological Assessment, 27(3), 883–894. https://doi.org/10.1037/pas0000082 Diamond, A. (2013). Executive functions. Annual Review of Psychology, 64, 135–168. Friedman, N. P., Miyake, A., Young, S. E., DeFries, J. C., Cor- ley, R. P., & Hewitt, J. K. (2008). Individual differences in executive functions are almost entirely genetic in ori- gin. Journal of Experimental Psychology. General, 137(2), 201–225. https://doi.org/10.1037/0096-3445.137.2.201 Gomez-Olive, F. X., Angotti, N., Houle, B., Klipstein-Grobusch, K., Kabudula, C., Menken, J., Williams, J., Tollman, S., & Clark, S. J. (2013). Prevalence of HIV among those 15 and older in rural South Africa. AIDS Care, 25(9), 1122–1128. https://doi.org/10.1080/09540121.2012.750710 Grissom, N. M., & Reyes, T. M. (2019). Let’s call the whole thing off: Evaluating gender and sex differences in executive function. Neuropsychopharmacology, 44(1), 86–96. https:// doi.org/10.1038/s41386-018-0179-5 Hare, T. A., Tottenham, N., Galvan, A., Voss, H. U., Glover, G. H., & Casey, B. J. (2008). Biological substrates of emo- tional reactivity and regulation in adolescence during an emo- tional go-nogo task. Biological Psychiatry, 63(10), 927–934. https://doi.org/10.1016/j.biopsych.2008.03.015 Harrison, A., Colvin, C. J., Kuo, C., Swartz, A., & Lurie, M. (2015). Sustained high HIV incidence in Young women in southern Africa: Social, behavioral, and structural factors and emerging intervention approaches. Current HIV/AIDS Reports, 12(2), 207–215. https://doi.org/10.1007/s11904- 015-0261-0 Hoare, J., Phillips, N., Joska, J. A., Paul, R., Donald, K. A., Stein, D. J., & Thomas, K. G. (2016). Applying the HIV-associated neurocognitive disorder diagnostic criteria to HIV-infected youth. Neurology, 87(1), 86–93. https://doi .org/10.1212/wnl.0000000000002669 Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria ver- sus new alternatives. Structural Equation Modeling: A Mul- tidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/ 10705519909540118 Hughes, C., Ensor, R., Wilson, A., & Graham, A. (2010). Tracking executive function across the transition to school: A latent variable approach. Developmental Neuropsychology, 35(1), 20–36. https://doi.org/10.1080/87565640903325691 Humphreys, G. W., Duta, M. D., Montana, L., Demeyere, N., McCrory, C., Rohr, J., Kahn, K., Tollman, S., & Berk- man, L. (2017, Jan). Cognitive function in low-income and low-literacy settings: Validation of the tablet-based Oxford cognitive screen in the health and aging in Africa: A longi- tudinal study of an INDEPTH Community in South Africa (HAALSI). The Journals of Gerontology. Series B, Psycho- logical Sciences and Social Sciences, 72(1), 38–50. https:// doi.org/10.1093/geronb/gbw139 Karr, J. E., Areshenkoff, C. N., Rast, P., Hofer, S. M., Iverson, G. L., & Garcia-Barrera, M. A. (2018). The unity and diversity of executive functions: A systematic review and re-analysis of latent variable studies. Psycholog- ical Bulletin, 144(11), 1147–1185. https://doi.org/10.1037/ bul0000160 Kerr, A., & Zelazo, P. D. (2004, Jun). Development of "hot" executive function: The children’s gambling task. Brain and Cognition, 55(1), 148–157. https://doi.org/10.1016/s0278- 2626(03)00275-6 Miyake, A., & Friedman, N. P. (2012). The nature and Organization of Individual Differences in executive func- tions: Four general conclusions. Current Directions in © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. https://doi.org/10.1038/s41598-021-87287-8 https://doi.org/10.1038/s41598-021-87287-8 https://doi.org/10.1037/pas0000082 https://doi.org/10.1037/0096-3445.137.2.201 https://doi.org/10.1080/09540121.2012.750710 https://doi.org/10.1038/s41386-018-0179-5 https://doi.org/10.1038/s41386-018-0179-5 https://doi.org/10.1016/j.biopsych.2008.03.015 https://doi.org/10.1007/s11904-015-0261-0 https://doi.org/10.1007/s11904-015-0261-0 https://doi.org/10.1212/wnl.0000000000002669 https://doi.org/10.1212/wnl.0000000000002669 https://doi.org/10.1080/10705519909540118 https://doi.org/10.1080/10705519909540118 https://doi.org/10.1080/87565640903325691 https://doi.org/10.1093/geronb/gbw139 https://doi.org/10.1093/geronb/gbw139 https://doi.org/10.1037/bul0000160 https://doi.org/10.1037/bul0000160 https://doi.org/10.1016/s0278-2626(03)00275-6 https://doi.org/10.1016/s0278-2626(03)00275-6 VALIDATION OF OCS-EF 907 Psychological Science, 21(1), 8–14. https://doi.org/10 .1177/0963721411429458 Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The Unity and Diver- sity of executive functions and their contributions to com- plex “frontal lobe” tasks: A latent variable analysis. Cog- nitive Psychology, 41(1), 49–100. https://doi.org/10.1006/ cogp.1999.0734 Pettifor, A., MacPhail, C., Hughes, J. P., Selin, A., Wang, J., Gómez-Olivé, F. X., Eshleman, S. H., Wagner, R. G., Mabuza, W., Khoza, N., Suchindran, C., Mokoena, I., Twine, R., Andrew, P., Townley, E., Laeyendecker, O., Agyei, Y., Tollman, S., & Kahn, K. (2016). The effect of a conditional cash transfer on HIV incidence in young women in rural South Africa (HPTN 068): A phase 3, randomised controlled trial. The Lancet Global Health, 4(12), e978–e988. https:// doi.org/10.1016/S2214-109X(16)30253-4 Poon, K. (2017). Hot and cool executive functions in adoles- cence: Development and contributions to important devel- opmental outcomes. Frontiers in Psychology, 8, 2311–2311. https://doi.org/10.3389/fpsyg.2017.02311 Prencipe, A., Kesek, A., Cohen, J., Lamm, C., Lewis, M. D., & Zelazo, P. D. (2011). Development of hot and cool executive function during the transition to adolescence. Journal of Experimental Child Psychology, 108(3), 621–637. https:// doi.org/10.1016/j.jecp.2010.09.008 Robbins, R. N., Gouse, H., Brown, H. G., Ehlers, A., Scott, T. M., Leu, C.-S., Remien, R. H., Mellins, C. A., & Joska, J. A. (2018). A Mobile app to screen for neurocogni- tive impairment: Preliminary validation of NeuroScreen among HIV-infected south African adults. JMIR mHealth and uHealth, 6(1), e5. https://doi.org/10.2196/ mhealth.9148 Rochat, T. J., Houle, B., Stein, A., Pearson, R. M., Newell, M. L., & Bland, R. M. (2017). Cohort profile: The Siyakhula cohort, rural South Africa. International Journal of Epi- demiology, 46(6), 1755–1756. https://doi.org/10.1093/ije/ dyx148 Romero, R. H., Hall, J., Cluver, L., & Steinert, J. (2018). Socioeconomically disadvantaged adolescents and educa- tional delay in two provinces in South Africa: Impacts of personal, family and school characteristics. Education as Change, 22, 1–33. Rosenberg, M., Pettifor, A., Duta, M., Demeyere, N., Wagner, R. G., Selin, A., MacPhail, C., Laeyendecker, O., Hughes, J. P., Stein, A., Tollman, S., & Kahn, K. (2018, Apr). Exec- utive function associated with sexual risk in young south African women: Findings from the HPTN 068 cohort. PLoS One, 13(4), 14, e0195217. https://doi.org/10.1371/journal .pone.0195217 Rowe, K., Duta, M., Demeyere, N., Wagner, R. G., Pettifor, A., Kahn, K., Tollman, S., Scerif, G., & Stein, A. (2020). The relationship between executive function, risky behaviour and HIV in young women from the HPTN 068 study in rural South Africa. AIDS Care, 1–11. https://doi.org/10.1080/ 09540121.2020.1851016 Rubin, L. H., Neigh, G. N., Sundermann, E. E., Xu, Y., Scully, E. P., & Maki, P. M. (2019). Sex differences in neurocognitive function in adults with HIV: Patterns, predictors, and mech- anisms. Current Psychiatry Reports, 21(10), 94–94. https:// doi.org/10.1007/s11920-019-1089-x Schulz, K. P., Fan, J., Magidina, O., Marks, D. J., Hahn, B., & Halperin, J. M. (2007). Does the emotional go/no-go task really measure behavioral inhibition? Convergence with measures on a non-emotional analog. Archives of Clinical Neuropsychology, 22(2), 151–160. https://doi.org/10.1016/ j.acn.2006.12.001 Schumacker, R. E., & Lomax, R. G. (2004). A beginner’s guide to structural equation modeling (2nd ed.). Lawrence Erlbaum Associates. Shing, Y. L., Lindenberger, U., Diamond, A., Li, S.-C., & David- son, M. C. (2010). Memory maintenance and inhibitory con- trol differentiate from early childhood to adolescence. Devel- opmental Neuropsychology, 35(6), 679–697. https://doi.org/ 10.1080/87565641.2010.508546 Shuttleworth-Edwards, A. B., van der Merwe, A. S., van Tonder, P., & Radloff, S. E. (2013). WISC-IV test performance in the south African context a collation of cross-cultural norms. In S. Laher & K. Cockcroft (Eds.), Psychological assessment in South Africa (pp. 33–47). Wits University Press. https://doi .org/10.18772/22013015782.8 Toplak, M. E., Sorge, G. B., Benoit, A., West, R. F., & Stanovich, K. E. (2010). Decision-making and cognitive abilities: A review of associations between Iowa gambling task perfor- mance, executive functions, and intelligence. Clinical Psy- chology Review, 30(5), 562–581. https://doi.org/10.1016/j .cpr.2010.04.002 Tottenham, N., Tanaka, J. W., Leon, A. C., McCarry, T., Nurse, M., Hare, T. A., Marcus, D. J., Westerlund, A., Casey, B. J., & Nelson, C. (2009). The NimStim set of facial expres- sions: Judgments from untrained research participants. Psy- chiatry Research, 168(3), 242–249. https://doi.org/10.1016/ j.psychres.2008.05.006 Walker, K. A., & Brown, G. G. (2018). HIV-associated execu- tive dysfunction in the era of modern antiretroviral therapy: A systematic review and meta-analysis. Journal of Clinical and Experimental Neuropsychology, 40(4), 357–376. https://doi .org/10.1080/13803395.2017.1349879 Willoughby, M., Kupersmidt, J., Voegler-Lee, M., & Bryant, D. (2011). Contributions of hot and cool self-regulation to preschool disruptive behavior and academic achievement. Developmental Neuropsychology, 36(2), 162–180. https:// doi.org/10.1080/87565641.2010.549980 Zelazo, P., & Müller, U. (2002). Executive function in typical and atypical development. In U. Goswami (Ed.), Black- well handbook of childhood cognitive development (pp. 445–469). Blackwell Publishers Ltd.. https://doi.org/10 .1002/9780470996652.ch20 © 2021 The Authors. International Journal of Psychology published by John Wiley & Sons Ltd on behalf of International Union of Psychological Science. https://doi.org/10.1177/0963721411429458 https://doi.org/10.1177/0963721411429458 https://doi.org/10.1006/cogp.1999.0734 https://doi.org/10.1006/cogp.1999.0734 https://doi.org/10.1016/S2214-109X(16)30253-4 https://doi.org/10.1016/S2214-109X(16)30253-4 https://doi.org/10.3389/fpsyg.2017.02311 https://doi.org/10.1016/j.jecp.2010.09.008 https://doi.org/10.1016/j.jecp.2010.09.008 https://doi.org/10.2196/mhealth.9148 https://doi.org/10.2196/mhealth.9148 https://doi.org/10.1093/ije/dyx148 https://doi.org/10.1093/ije/dyx148 https://doi.org/10.1371/journal.pone.0195217 https://doi.org/10.1371/journal.pone.0195217 https://doi.org/10.1080/09540121.2020.1851016 https://doi.org/10.1080/09540121.2020.1851016 https://doi.org/10.1007/s11920-019-1089-x https://doi.org/10.1007/s11920-019-1089-x https://doi.org/10.1016/j.acn.2006.12.001 https://doi.org/10.1016/j.acn.2006.12.001 https://doi.org/10.1080/87565641.2010.508546 https://doi.org/10.1080/87565641.2010.508546 https://doi.org/10.18772/22013015782.8 https://doi.org/10.18772/22013015782.8 https://doi.org/10.1016/j.cpr.2010.04.002 https://doi.org/10.1016/j.cpr.2010.04.002 https://doi.org/10.1016/j.psychres.2008.05.006 https://doi.org/10.1016/j.psychres.2008.05.006 https://doi.org/10.1080/13803395.2017.1349879 https://doi.org/10.1080/13803395.2017.1349879 https://doi.org/10.1080/87565641.2010.549980 https://doi.org/10.1080/87565641.2010.549980 https://doi.org/10.1002/9780470996652.ch20 https://doi.org/10.1002/9780470996652.ch20