RAVEN’S ADVANCED PROGRESSIVE MATRICES WITHIN A SOUTH AFRICAN CONTEXT

Abstract

The issue of bias, whether a psychological test measures what it claims to measure similarly across different groups, remains a fundamental concern within the field of psychometrics, particularly within South Africa, where a history riddled with oppression, discrimination and malpractice in the area has led to suspicion, mistrust and legislation banning the use of many psychological tests as invalid and unfair (Foxcroft & Roodt, 2001; Murphy & Davidshofer, 2001; Nell, 1999). There is thus clearly a need for additional and more detailed investigations of the way specific individual tests function in the South African context. This study attempted to creatively examine systematic differences in performance on one specific test, the Raven’s Advanced Progressive Matrices (RAPM), on the basis of home language and gender, factors seldom investigated in the literature. A sample of one hundred Psychology first-year students completed a demographic questionnaire, the RAPM, the Similarities sub-test of the South African Wechsler Adult Intelligence Scales (SAWAIS) and an adapted version of the Reading Comprehension subtest of the Stanford Diagnostic Reading Test (SDRT). The data gathered was then utilized to explore four main research questions. Firstly, in order to establish construct comparability, the relationship between the RAPM and a more verbally-oriented measure of g, the Similarities sub-test of the SAWAIS, was assessed. Results indicated a relatively strong positive relationship between the two measures (r = 0.66), and no significant differences between the correlations on the basis of either gender or home language. Secondly, in order to explore the relationship between the RAPM and English comprehension, the study assessed the relationship between overall, literal and inferential scores on an adapted version of the Reading Comprehension sub-test of the SDRT and the RAPM. Results indicated only a moderate relationship between the two tests (r = 0.65), no difference in the relationship between RAPM performance and literal comprehension or inferential comprehension, and no difference in the relationship between the two tests on the basis of either gender or home language. Thirdly, in order to establish whether items were found to be difficult in a similar way across the different gender and home language groups, p-values and regression lines were calculated. These indicated that significant differences in level of item difficulty were experienced between English and African language speakers, although no differences were apparent in item difficulty on the basis of gender. 5 Lastly, in order to establish whether qualitative differences in performance on the basis of ability (as estimated by performance on the RAPM), gender or language existed, a discrimination analysis examining the types of errors made by each group was performed. Repeated measures ANOVAs and multiple comparison post-hoc analyses revealed significant differences in the types of errors made on the basis of ability and home language, but not gender. The post-hoc analyses suggested that those of higher ability or first language English speakers were more likely to make incomplete correlate errors, while those of lower ability or speaking African first languages were more likely to make confluence of ideas errors. In general, the findings of the study seemed to suggest that the RAPM, while not biased on the basis of gender, might contain a deep-seated language bias despite their non-verbal presentation.