The Applicability of the Neale Analysis of Reading Ability-Second Revised British Edition (NARA II) in the South African Context Student: Hansini Dhana-Dullabh Supervisors: Professor Kate Cockcroft & Professor Sumaya Laher A dissertation submitted in fulfilment of the requirements for the degree of Master of Arts by Research in the Faculty of Humanities, University of the Witwatersrand i I dedicate this research report to the vast number of individuals who are making a difference to literacy in South Africa. ii DECLARATION I, __________________________________________________, know and accept that plagiarism is wrong. Consequently, I declare that:  The dissertation is my own work.  It is being submitted for the degree of Master of Arts by Research at the University of the Witwatersrand.  It has not been submitted before for any degree or examination at any other university.  I have correctly acknowledged all direct quotations and paraphrased ideas/content. In addition I have provided a complete, alphabetized reference list, as required by the APA method of referencing.  I have not allowed, and will not allow, anyone to copy my work with the intention of passing it off as his or her own work.  I understand that the University of the Witwatersrand may take disciplinary action against me if there is a belief that this is not my own unaided work, or that I failed to acknowledge the source of the ideas or words in my writing. Signed: _____________________________ Hansini Dhana-Dullabh Date: ________________ iii ACKNOWLEDGEMENTS  Thank you to my supervisors Prof. Kate Cockcroft and Prof. Sumaya Laher for all their support, patience and input in this research project.  A special thank you to my husband Manish, for giving me this opportunity. You know how close education and literacy are to my heart and I could not have fulfilled this without you by my side. We have sacrificed a lot this year so that I could complete this dissertation. Your support and patience during this research project means a lot to me. Thank You.  Thank you to my parents Mr & Mrs Dhana for the endless opportunities that you both have provided me with throughout my life. You have encouraged me, believed in me and motivated me. I look up to you both and have learnt so much from you.  Mohini: Little do you know but you have been my pillar of strength during this journey and have always supported me through rough times. Thank you.  To all the subjects who participated in this study and the staff at the schools, thank you for your time and patience.  Thank you to Melanie Smith for your input and encouragement throughout this research project.  Thank you to Viktoria Gunkel for your assistance in the scoring of the assessments.  The financial assistance of the National Research Foundation (NRF) towards this research is hereby acknowledged. Opinions expressed and conclusions arrived at, are those of the author and are not necessarily to be attributed to the NRF. iv ABSTRACT Psychological tests are closely tied to the context in which they were designed. Within the South African context, the inequalities of apartheid played a major role in test development and use, and it relies on various international tests that may not be suitable to the context and on local tests that may not include all linguistic groups. There is a need for psychological tests that are in use to be reliable, valid and free from bias. Thus, this research examined the applicability of the Neale Analysis of Reading Ability-Second Revised British Edition (NARA II) in the South African context. Aspects of reliability, validity and bias were examined. A non-probability sample of 144 Grade Four first and second language learners attending one of four government schools in Gauteng were assessed on the NARA II. A non-probability sample of eight professionals (two teachers, one psychometrist, two educational psychologists, two speech therapists and one reading therapist and trainer) were interviewed for their views on the applicability of the NARA II. The research followed a mixed methods approach, namely a sequential explanatory strategy. The results demonstrated adequate internal consistency reliability, although this was lower than in the normative sample and other studies. Face validity and concurrent validity were adequate. The content validity of the NARA II was questionable within the South African context, suggesting it should be adapted for use in this country. Some evidence of bias based on gender, home language, population group, educational level of parents and the school learners attended was found. The qualitative data supported findings from the quantitative data. A thematic content analysis identified nine themes namely Suitability of the NARA II to the South African context, Similarities between the NARA II and reading tasks in school, Differences between the NARA II and reading tasks in school, Comprehension skills, Assessment tools utilised for reading in South Africa, Learner errors on reading, Foundations of reading, Benefits and Affordability. The overall conclusion is that adaptation of the NARA II and norming on the South African population be undertaken. v TABLE OF CONTENTS Declaration ........................................................................................................................ii Acknowledgements ........................................................................................................... iii Abstract ............................................................................................................................ iv LIST OF FIGURES ........................................................................................................... ix LIST OF TABLES ............................................................................................................. ix Introduction ....................................................................................................................... 1 CHAPTER 1: THE CONCEPT OF READING AND READING DEVELOPMENT .............. 4 1.1 Introduction ......................................................................................................... 4 1.2 Understanding Reading ...................................................................................... 4 1.3 Reading Development ........................................................................................ 7 1.3.1 The sight-vocabulary/logographic phase ...................................................... 9 1.3.2 The discrimination-net phase ....................................................................... 9 1.3.3 The phonological-recoding phase .............................................................. 10 1.3.4 The orthographic phase ............................................................................. 10 1.4 Reading Assessments ...................................................................................... 12 1.5 Conclusion ........................................................................................................ 13 CHAPTER 2: THE SOUTH AFRICAN CONTEXT ........................................................... 15 2.1 Introduction ....................................................................................................... 15 2.2 Historical Context of Education and Reading .................................................... 15 2.3 Current School Policy linked to Reading and Strategies to Improve Reading.... 16 2.4 Current Reading Statistics ................................................................................ 17 2.5 The Language of Learning and Teaching and Home Language ....................... 19 2.6 Assessing Reading in Education ....................................................................... 21 2.7 Conclusion ........................................................................................................ 23 CHAPTER 3: THE NEALE ANALYSIS OF READING ABILITY ....................................... 25 3.1 Introduction ....................................................................................................... 25 3.2 Development of the Neale Analysis of Reading Ability ...................................... 25 3.3 The Reliability of the NARA II ........................................................................... 26 3.3.1 Test-retest, parallel/alternate forms and inter-rater reliability ...................... 26 3.3.2 Internal consistency reliability .................................................................... 27 3.4 The Validity of the NARA II ............................................................................... 27 3.4.1 Face Validity .............................................................................................. 27 3.4.2 Content Validity ......................................................................................... 28 3.4.3 Criterion-related Validity ............................................................................ 28 3.4.4 Construct Validity ....................................................................................... 28 vi 3.5 Examining Bias in the NARA II.......................................................................... 29 3.5.1 Gender ...................................................................................................... 29 3.5.2 Language .................................................................................................. 30 3.5.3 Population group ....................................................................................... 30 3.5.4 Socio-economic status ............................................................................... 31 3.5.5 Quality of schooling/education ................................................................... 31 3.6 Conclusion ........................................................................................................ 33 CHAPTER 4: METHODS ................................................................................................ 34 4.1 Introduction ....................................................................................................... 34 4.2 Aims of the Study.............................................................................................. 34 4.3 Research Rationale .......................................................................................... 34 4.4 Research Questions ......................................................................................... 35 4.4.1 Quantitative Phase .................................................................................... 35 4.4.2 Qualitative Phase ...................................................................................... 36 4.5 Sample ............................................................................................................. 37 4.5.1 Quantitative Phase .................................................................................... 37 4.5.2 Qualitative Phase ...................................................................................... 39 4.6 Instruments ....................................................................................................... 39 4.6.1 Quantitative Phase .................................................................................... 39 4.6.2 Qualitative Phase ...................................................................................... 42 4.7 Research Design .............................................................................................. 43 4.7.1 Quantitative Phase .................................................................................... 43 4.7.2 Qualitative Phase ...................................................................................... 44 4.8 Procedure ......................................................................................................... 44 4.8.1 Quantitative Phase .................................................................................... 44 4.8.2 Qualitative Phase ...................................................................................... 44 4.9 Ethical Considerations ...................................................................................... 45 4.9.1 Quantitative Phase .................................................................................... 45 4.9.2 Qualitative Phase ...................................................................................... 45 4.10 Data Analysis ................................................................................................ 46 4.10.1 Quantitative Phase .................................................................................... 46 4.10.2 Qualitative Phase ...................................................................................... 48 4.11 Self-Reflexivity .............................................................................................. 48 4.12 Conclusion .................................................................................................... 49 CHAPTER 5: RESULTS ................................................................................................. 50 5.1 Introduction ....................................................................................................... 50 vii 5.2 Descriptive Statistics ......................................................................................... 50 5.3 Internal consistency reliability ........................................................................... 53 5.4 Validity .............................................................................................................. 53 5.4.1 Face Validity .............................................................................................. 53 5.4.2 Content Validity ......................................................................................... 54 5.4.3 Concurrent Validity .................................................................................... 57 5.5 Bias .................................................................................................................. 58 5.5.1 Gender and errors on the NARA II ............................................................. 59 5.5.2 Home language and errors on the NARA II ................................................ 59 5.5.3 Population group and errors on the NARA II .............................................. 60 5.5.4 Educational level of parents and errors on the NARA II ............................. 61 5.5.5 Learner school and errors on the NARA II ................................................. 62 5.5.6 Difference in chronological age and reading age by gender ....................... 62 5.5.7 Difference in chronological age and reading age by home language ......... 63 5.5.8 Difference in chronological age and reading age by population group ....... 64 5.5.9 Difference in chronological age and reading age by educational level of parents ...................................................................................................... 66 5.5.10 Difference in chronological age and reading age by learner school ........... 67 5.6 Thematic Content Analysis ............................................................................... 69 5.6.1 Suitability of the NARA II to the South African context ............................... 69 5.6.2 Similarities between the NARA II and reading tasks in school ................... 72 5.6.3 Differences between the NARA II and reading tasks in school ................... 72 5.6.4 Comprehension skills ................................................................................. 73 5.6.5 Assessment tools utilised for reading in South Africa ................................. 73 5.6.6 Learner errors on reading .......................................................................... 74 5.6.7 Foundations of reading .............................................................................. 75 5.6.8 Benefits ..................................................................................................... 76 5.6.9 Affordability ................................................................................................ 76 5.7 Conclusion ........................................................................................................ 77 CHAPTER 6: DISCUSSION ............................................................................................ 79 6.1 Introduction ....................................................................................................... 79 6.2 Descriptive Statistics: Demographic Information ............................................... 79 6.3 Descriptive Statistics: NARA II .......................................................................... 80 6.4 Internal Consistency Reliability ......................................................................... 80 6.5 Face Validity ..................................................................................................... 81 6.6 Content Validity ................................................................................................ 81 viii 6.7 Concurrent Validity ........................................................................................... 83 6.8 Bias .................................................................................................................. 83 6.8.1 Gender bias and the NARA II .................................................................... 83 6.8.2 Language bias and the NARA II................................................................. 84 6.8.3 Population group and the NARA II ............................................................. 85 6.8.4 Educational level of parents and the NARA II ............................................ 85 6.8.5 Learner school and the NARA II ................................................................ 86 6.9 Qualitative Findings .......................................................................................... 87 6.9.1 The suitability of the NARA II in the South African context in relation to norms, words and stories and reporting results ................................................................... 87 6.9.2 NARA II in relation to reading at school ..................................................... 89 6.9.3 The importance of comprehension skills in reading.................................... 90 6.9.4 Errors made in reading .............................................................................. 90 6.9.5 The skills required to be able to read ......................................................... 91 6.9.6 The benefits and affordability of reading assessments ............................... 91 6.10 Overall Findings ............................................................................................ 92 6.11 Limitations with regards to the sample size ................................................... 93 6.11.1 Learners .................................................................................................... 93 6.11.2 Professionals ............................................................................................. 93 6.12 Limitations with regards to the method .......................................................... 93 6.13 Limitations with regard to the instrument ....................................................... 94 6.13 Recommendations for Future Research ........................................................ 94 6.14 Conclusion .................................................................................................... 95 Reference List................................................................................................................. 96 APPENDIX A: INTERVIEW QUESTIONS FOR PROFESSIONALS .............................. 106 APPENDIX B: LETTER TO PARENTS/GUARDIANS ................................................... 109 APPENDIX C: ASSENT FORM FOR CHILD TO SIGN ................................................. 113 APPENDIX D: LETTER TO PROFESSIONALS AND CONSENT FORM FOR INTERVIEW AND AUDIO RECORDING ........................................................................................... 114 APPENDIX E: HUMAN RESEARCH ETHICS COMMITTEE CLEARANCE CERTFICATE ..................................................................................................................................... 117 APPENDIX F: PERMISSION FROM THE DEPARTMENT OF EDUCATION ................ 118 APPENDIX G: LETTER TO SCHOOL PRINCIPALS ..................................................... 121 ix LIST OF FIGURES Figure 2.1 Illiteracy by language in South Africa. ............................................................... 20 LIST OF TABLES Table 4.1 Frequencies for gender, population group, home language, educational level of parents and learner school................................................................................................. 38 Table 4.2 Descriptive statistics for demographic information of learners ............................ 39 Table 5.1 Descriptive statistics for the NARA II .................................................................. 50 Table 5.2 Descriptive statistics for the NARA II levels ........................................................ 51 Table 5.3 Descriptive statistics for the NARA II errors ........................................................ 52 Table 5.4 Internal consistency reliability coefficients for the comprehension questions in the NARA II.............................................................................................................................. 53 Table 5.5 Content Validity frequencies for the NARA II word list ........................................ 55 Table 5.6 Content Validity frequencies for the NARA II stories ........................................... 57 Table 5.7 Correlations between reading age and June results ........................................... 58 Table 5.8 Independent t-test comparing NARA II errors by gender .................................... 59 Table 5.9 Independent t-test comparing NARA II errors by home language ....................... 60 Table 5.10 ANOVA results for population group on the NARA II errors .............................. 61 Table 5.11 ANOVA results for educational level of parents on the NARA II errors ............. 61 Table 5.12 ANOVA results for learner school on the NARA II errors .................................. 62 Table 5.13 Independent t-tests results for the difference in chronological age and reading age by gender .................................................................................................................... 63 Table 5.14 Independent t-tests results for the difference in chronological age and reading age by home language ...................................................................................................... 64 Table 5.15 ANOVA results for the difference in chronological age and reading age by population group ................................................................................................................ 64 Table 5.16 Post-hoc results for the difference in chronological age and reading age by population group ................................................................................................................ 65 Table 5.17 ANOVA results for the difference in chronological age and reading age by educational level of parents ............................................................................................... 66 Table 5.18 Post-hoc results for the difference in chronological age and reading age by educational level of parents ............................................................................................... 67 Table 5.19 ANOVA results for the difference in chronological age and reading age by learner school .................................................................................................................... 68 Table 5.20 Post-hoc results for the difference in chronological age and reading age by learner school .................................................................................................................... 68 Table 5.21 Suggestions for possible themes suitable to the South African context ............ 70 1 INTRODUCTION The inability to read hinders any individual’s effectiveness of performing even the most basic day-to-day activity. Reading is a necessary skill that enables people to communicate, learn and integrate themselves into the world in which they live. Most information is accessible via the mode of written words in books, magazines, newspapers and computers, thus highlighting the importance of addressing illiteracy. Educational success depends on reading success since reading is the foundation of further learning. Reading performance of the South African population is among the world’s worst, with the Southern and Eastern African Consortium for Monitoring Educational Quality (SACMEQ II) project finding that learners had not reached the SACMEQ mean of 500 in reading. Listed as one of the countries at risk of not achieving the Education for All (EFA) goals (De Jager, Nassimbeni & Underwood, 2006; Fleisch, 2008), South Africa faces the challenge of being a society where one-in-five adults has had no formal education. Most studies and surveys conducted are based on adult education and literacy levels, as well as census records, therefore, it is difficult to pinpoint illiteracy in terms of age. The Progress in International Reading Literacy Study (PIRLS) was a test in which South Africa participated together with 45 education systems in 40 countries in 2006. The PIRLS is a home language test often taken by children in an additional language. South African children used for this study were in Grades Four and Five. For both grades South Africa scored the lowest out of all 40 countries (Howie et al., 2008). Difficulties in reading can lead to difficulties in learning (Oberholzer, 2005), therefore it is important to assess reading in education to determine if learners are having any difficulties and address them. Reading assessments establish if there is a difficulty, the causes, and what steps need to be taken thereafter, with intervention then based on the outcome. Early identification and intervention are important in identifying and remediating learning disorders, therefore the use of assessment can measure the level of difficulties experienced (Menzies, Mahdavi & Lewis, 2008). Strengths and weaknesses in reading ability can be carried over to later grades (Smith, 1941). Reading assessment consists of two forms, namely to find out how well a child is reading, which includes identifying any difficulties so as to assist in improvement through feedback and assistance (Pang, Muaka, Bernhardt & Kamil, 2003); and to measure the learner’s progress. For reading instruction to be effective both forms of reading assessment are important. The approach used in South Africa in terms of the development and use of psychological testing has followed a similar route to that of the United States of America (USA) and Europe. However, the political, economic and social circumstances of a country should be 2 taken into consideration, since these are closely tied to the context in which they are designed (Foxcroft, 2002). The current approach which follows trends used in USA and Europe may not be applicable to people from a South African context, given the diversity of the South African population and inequalities resulting from apartheid, without investigating reliability, validity and bias (Foxcroft, 2002; Foxcroft, Roodt & Abrahams, 2005). In addition, changes in the South African education system from apartheid to a democratic society impact on how reading assessments are used, what type are used such as word lists, comprehension, what the norm group comprises, what the results are used for, and their impact on the learner being assessed. The use of assessment tools should be reliable, valid and free from bias, as stated in the Employment Equity Act, No. 55 of 1998: Psychological testing and other similar assessments of an employee are prohibited unless the test or assessment being used (a) has been scientifically shown to be valid and reliable; (b) can be applied fairly to employees; and (c) is not biased against any employee or group (Office of the President, 1998, p.7). This has a major impact on assessors who are using various reading assessment tools. With the various language groups in South Africa, investigations into bias should be carried out for all subgroups so that the assessment tool can be deemed appropriate. If not, the investigation should state on which subgroups it can be used. The selection of assessment tools should be based on “quality, cross-cultural applicability, the appropriateness of their norms, and the availability of different language versions” (Foxcroft, Roodt & Abrahams, 2005, p.20). In order for many assessment tools to be suitable to the multilingual South African context, these tools can be adapted in one of two ways. The first would be to have appropriate South African norms, the second to make the test available in various local language versions with corresponding norms (Foxcroft, Roodt & Abrahams, 2005). In examining these alternatives, this study investigates the applicability of the Neale Analysis of Reading Ability (NARA II) in the South African context, establishing whether it is reliable, valid and free from bias for use with primary school learners. This dissertation consists of seven chapters. Chapter 1 is a literature review in which definitions of reading are explained with regards to two related processes, namely word recognition and comprehension. Reading ability in this study refers to those processes measured by the NARA II, namely, Accuracy, Comprehension and Rate. This is followed by a discussion on typical reading development and the importance of reading assessments and early intervention. The literature review concludes by describing how the applicability of the NARA II for use in the South African context will be established in this study. 3 In Chapter 2 there is a discussion on the historical context of education and reading in South Africa, followed by the current school policy on reading and strategies undertaken to improve reading, together with the current reading statistics. Given the multilingual context of the country, issues around the language of learning and teaching (LOLT) and home language are addressed. This chapter concludes with the importance of assessing reading in education. In Chapter 3, the Neale Analysis of Reading Ability (NARA II) is introduced in terms of how psychological tests in South Africa should be reliable, valid and free from bias in terms of the Employment Equity Act, No. 55 of 1998. This is followed by the types of reliability and validity that will be addressed in this study. Issues of bias in South Africa are more complex and therefore gender, home language, population group, socio-economic status and schooling are discussed in terms of how they impact on assessment. Research conducted internationally and in South Africa on the NARA II is presented in terms of its significance to the current study. The methods used for this study are presented in Chapter 4, with the aims of the study and the rationale for it being explicitly stated. The discussion in each of the sections is presented as a quantitative phase and a qualitative phase. The research questions are then posed, followed by the sample, instruments, procedure, research design, ethical considerations and analytical techniques used in this study. This chapter concludes with a discussion on self- reflexivity. Chapter 5 provides details on the results of the study. Firstly, descriptive statistics in terms of demographic information are presented. Those of the NARA II follow in terms of Accuracy, Comprehension and Rate, the difference in chronological age and reading scores, and the types of errors found in the NARA II at each level. Results on reliability, validity and bias follow. The chapter concludes with a discussion on thematic content analysis by using the qualitative data. Chapter 6 is a discussion of the results in terms of the literature and information from the first three chapters. Reliability, validity and bias are discussed in relation to the NARA II’s applicability to the South African context. This is followed by a discussion on the qualitative findings. This chapter concludes with a discussion on the limitations of the current study and makes recommendations for future research. 4 CHAPTER 1: THE CONCEPT OF READING AND READING DEVELOPMENT 1.1 Introduction The basis on which formal learning occurs, reading, is a dynamic process that involves various components which are often difficult to dissect as they overlap. Bouwer (2004, p.90) has listed reading as having nine distinct aspects, namely: “sensory, perceptual, sequential, experiential, thinking, learning, associational, affective and constructive”, which when combined produce a “reading product”. These can be combined into decoding and comprehension, both of which form part of the NARA II instrument and are examined in this chapter. Since reading involves interrelated cognitive skills, various ones related to reading are then discussed, followed by various stages of reading development and the assessment of reading ability. 1.2 Understanding Reading Reading can be defined as the understanding of speech that is written down (Day & Bamford, 1998; Ziegler & Goswami, 2005). There are prerequisite skills which need to be taught for a child to read effectively, and in terms of reading development these are word recognition/decoding and comprehension (Bouwer, 2004). ‘Reading ability’, for the purpose of this study, refers to Accuracy, Comprehension and Rate, as assessed by the NARA II (Neale, 1997). Research on reading is an active topic of investigation in both cognitive and educational psychology. Both extrinsic and intrinsic factors that influence reading development have been investigated, but the main extrinsic factors of concern here are the practical difficulties children are facing in acquiring literacy in the South African context (Eloff & Ebersӧhn, 2004). Word recognition involves the retrieval of the auditory code of written words, with comprehension being the ability to construct a mental representation of the information described and attaching meaning to it (Oakhill, Cain & Bryant 2003; Spooner, Baddeley & Gathercole, 2004). Comprehension cannot occur without the identification of most printed words in a text, therefore decoding skills, which are viewed as the building blocks of reading, need first to be mastered (Grove & Hauptfleisch, 1982; Just & Carpenter, 1987; Vauras, Kinnunen & Kuusela, 1994). Understanding written words involves making use of background knowledge, vocabulary, grammar, experience with written words and other strategies. As Beckett (2005, p.2) argues, “if a child is deemed as unable to acquire functional comprehension of oral language and its associated pragmatics, and of written language and its different communicative conventions, that child will be effectively locked out of the benefits of education.” 5 The development of speech and language abilities occurs naturally, with little effort, unlike learning to read, which is a different process that needs to be specifically taught (Harris & Coltheart, 1986). It involves knowledge about the symbolic system of writing used to represent speech, and children must learn the code used by their language for representing speech as a series of visual symbols (Ziegler & Goswami, 2005). Children learn the vocabulary, grammar and sound system of their language before learning to associate the written form with speech. Early spoken language and early reading ability show a close connection, while phonological and phonemic awareness are also closely connected. According to Adams (1996), interacting cues are used by the reader and four processors are at work: orthographic, meaning, context and phonological. Letters are processed by the orthographic cue, and then translated by the phonological processor into speech sounds to which meaning is assigned by a meaning processor that is fed into the text processor, thus constructing a continuous understanding of the text. There is a two-way communication among the processors, which send and receive information. Information from one processor can speed up the other processors, thus, if one is weak the other three would need to work harder. Parts that form the reading system are not discrete and grow together, but need to be linked at acquisition. Higher-order and lower-order processes work together, therefore those teaching children how to read need to understand the parts of the system and how they work. This would assist the educator in developing methods required for the progress and needs of each individual learner, and the facilitation to get through it. As each part is developed and improved in relation to others it guides and reinforces their growth (Adams, 1996). The results from the current study may indicate the effectiveness of the educator’s understanding of the reading system and how well it is applied when teaching reading. Research shows that the components of reading appear to be inter-related and may draw on different cognitive abilities. Linguistic ability and long-term memory are related to comprehension, while visual and auditory analytical skills are related to decoding (Oakhill et al., 2003). If each of these components is linked independently to reading ability it is said that each has different underlying skills, abilities, and independent developmental paths (Oakhill et al., 2003). The ability to read involves recognition of letters and words, association of letters and sounds, blending and understanding (Janks, 2011). The analytical process needs to achieve automaticity so that the brain can process information at the speed necessary for comprehension. Thus, fluent readers are able to read with comparatively little effort. The amount of visual information needed may diminish in proportion to one’s ability to predict what the text is going to say (Janks, 2011). The understanding of written text involves both a bottom-up word recognition process and a top-down comprehension process. The former involves blending, segmenting and phoneme 6 manipulation, with blending used to push sounds into meaningful words, segmenting to separate sounds into words, and phoneme manipulation to push sounds in and out of words (Taylor, 2002). Learning occurs at its best when information and procedures are linked, and information that comprises the written language is known as the ‘code’. The code does not exist in isolation and is therefore taught concurrently with the three skills listed above, it comprising the sound pictures found in a particular word. Top-down processing takes place when readers use prior knowledge to construct new knowledge relevant to their experiences and situations (Verhoeven & Perfetti, 2008). To comprehend text there needs to be flexibility in the use of different sources of information. Individual letters are processed through multiple channels, which work together and coincide, so a letter that is perceived incorrectly or is sometimes illegible does not stop the reading process (Adams, 1996). Visual input is changed into a linguistic representation (Verhoeven & Perfetti, 2008), and word recognition occurs through the interaction between phonological information, visual information and meaning. Word decoding develops through automatic word decoding skills and the ability of fluency, enabling readers to use reading as a vehicle to acquire new information and knowledge. According to Adams (1996, p.4), difficulties occur “when the reader cannot quickly, effortlessly and automatically recognise individual letters and spelling patterns and transform them to words and meanings”. If a reader’s word recognition skills are not functioning properly they will often choose not to read. Text that is unfamiliar in specific occurrences, or difficult in wording and structure, requires active attention for comprehension. Recognising and capturing meanings of words on a page should be rapid, effortless and automatic for cognitive energy and resources to be available and on which skilful comprehension is dependent (Adams, 1996). For text comprehension, the reader must combine the meaning of each sentence with information from previous texts. Anything taught or learnt requires sub-skills and codes, the former being required before performing a task and the latter symbols used to communicate when performing the task. Briefly, the phonological and visual sub-skills required for reading, as stated in McGuinness and McGuinness (1998), are ability to scan text from left to right, matching visual symbols to auditory sounds; competence to blend sound units into words and to segment sounds found in words; understanding that sometimes two or more letters represent a sound (for example, ‘sh’ is made up of two letters); understanding that sounds can be represented in more than one way, such as the sound ‘ee’ in green, team, happy; and realisation that some aspects of the code represent more than one sound, for example, the symbol represents the sound ‘o’ as in hot or ‘oe’ as in most. Therefore, it is important to ensure that learners are taught 7 these skills and sub-skills, to monitor their progress and to ensure that they reach their full potential of reading ability. Various components of reading are important during the process, with decoding and comprehension being the two most important. In this study, reading accuracy (decoding), comprehension and reading rate were assessed in the learner sample in order to establish the relationship between them. In addition to the reading process, it is important to discuss how reading develops. 1.3 Reading Development There is a wide variety of research on children’s reading development and reading difficulties. Current theories provide information on the processes readers use and how they differ, depending on the level of acquisition from the beginning to the advanced stage. Reading develops continuously from the emergence of one’s experience with spoken language and print, through growth and expansion as the child moves from one stage to another (Gunning, 2010). South African children begin to learn to read in the year they turn seven, which is usually Grade One (Cockcroft, 2009). The fundamentals of an alphabetic writing system are the associations between the letters and the sounds, and as alphabetic writing systems, reading English, Afrikaans, isiZulu, isiXhosa and Setho require an understanding of letter-sound rules (Chall, Jacobs & Baldwin, 1990; Harris & Coltheart, 1986). Letter-sound rules in English are not straightforward because while English has 26 letters it has more than 40 sounds (Taylor, 2002). Thus, it is described as an ‘opaque’ or ‘deep orthography’ (writing system). Research on the acquisition of reading has been concerned with how different types of reading instruction affect the rate of progress and achievement in learning (Thompson & Johnston, 1993). Children who are at the same level of reading may vary in the range of different processes they use to learn (Thompson & Johnston, 1993), thus the progress through stages of learning to read varies, depending on the language of instruction and the reading method used to teach it, for example using phonics or visual strategies (known as whole-language), or a combination. Phonics is a set of guidelines used to teach children letter-sound correspondences, “their sequences (including spelling patterns), and the pronunciation of corresponding sounds” (Thompson & Johnston, 1993, p.77). Learners go through the letters in the alphabet in a sequence, with keywords (mnemonics) used to assist them in remembering individual sounds, usually the initial sounds in words (Taylor, 2002). Phonics teaches adjacent 8 consonants as units, so when they are taught the sounds and they also learn , and now have to remember , and (McGuinness & McGuinness, 1998; Taylor, 2002). Words are taught using patterns such as ‘fr’ for frog, friend, frame, or they are taught in families, as ’at’ as in cat, fat, sat, mat, hat. The phonics approach teaches vocabulary, comprehension and decoding (Nicholson, 1993). The task of decoding is taught in chunks that are manageable by the child, making it difficult to remember mixed patterns such as two or more letters that represent a single sound (the sound in rain), where the child cannot use phonics. Children are required to memorise two or more letters that are paired together, but this may be difficult for young ones to remember. The reading process is hierarchical, starting strictly with the basics of individual letter-sound correspondences, and then working up to higher level skills. Phonics teaches 330 sounds and 1500 characters to represent the sounds; however, humans can only remember 1500 to 2000 unique signs. Phonics relies on 166 rules and 45 exceptions, because of the irregularities found in English (McGuinness & McGuinness, 1998; Taylor, 2002). On the other hand, “whole language learning builds around whole learners learning whole language in whole situations” (Goodman, 1986, p.40), thus, learning whole words in meaningful situations (McGuinness & McGuinness, 1998; Prinsloo & Bloch, 1999). According to Taylor (2002, p.34), “Language learning takes place through functional reading and writing experiences and is viewed as a whole-speaking, listening, reading and writing”. The belief is that reading instruction should focus on the construction of meaning and not on the structural units through which meaning is established, in contrast to the phonics approach to reading (Fromkin & Rodman, 1998; Treiman, 2001; Tunmer & Chapman, 1998). Whole-language does not include the importance of learning to decode or link graphemes to phonemes, as in phonics (Nicholson, 1993). The basic assumptions about the whole-language approach are that reading is a “psycholinguistic guessing game” (Janks, 2011, p.28; Liberman, 1998, p.9; Treiman, 2001), that is, readers sample the print, grasp some words and skip others. If this fails they can look at the initial letter and guess the rest (Snow & Juel, 2010; Tunmer & Chapman, 1998). The whole-language approach does not provide information on what else is needed to teach children to read, thus, if appropriate methods such as a combination of whole-language and phonics are used to teach reading it can make learners aware of phonological structure so that they can apply the alphabet principle. A combination of visual strategies and letter- sound rules are therefore required for children to be able to read. 9 Various research studies have been conducted on the stages of reading development, with Harris and Coltheart’s (1986) four phases of reading being one of the most representative approaches, based upon ideas proposed by Marsh, Friedman, Welch and Desberg (1981), Seymour and McGregor (1984), Frith (1985) and Seymour and Elder (1985). The common features in each of these theories are picture recognition, phonic decoding and orthographic word recognition. Harris and Coltheart’s (1986) four phases are detailed below. 1.3.1 The sight-vocabulary/logographic phase This phase emerges around the age of 4 to 6 years, when the child has a set of words that he or she is able to read aloud. This ability may have been acquired through using the direct procedure or spontaneously, and children as young as two years of age seem to acquire a few hundred sight words. Informal studies conducted by Harris and Coltheart (1986) showed that sight words could be read but unfamiliar words and non-words could not be read aloud. For instance, words of familiar brands, such as MacDonald’s, Steers, KFC, Uno, Opel, can be read aloud without being taught. The young child sees these names on advertising boards, buses and cars, by observing them and their surroundings, and therefore they are able to ‘read’ words that they see frequently in their environment (Beech, 2005; Ehri, 2005; Snow, Burns & Griffin, 1998). Children in this phase recognise words in terms of overall shape (i.e., logographs) and not through the use of letter-sound rules (Beech, 2005; Ehri, 2005). Words are seen as indivisible visual wholes, and even when the word-shape or visual pattern is changed a child is still able to read the word. This shows that the direct procedure does not depend on the recognition of words as wholes but a sequence of letters in that word (Harris & Coltheart, 1986), and the logographic phase is a pre-reading stage that occurs before formal learning has occurred. 1.3.2 The discrimination-net phase In the discrimination-net phase, a single word is selected from a set of words that the child can already read, and the responses are choices from words they are currently using in their reading lessons. Therefore, reading aloud is through selection from a set of words the child has learnt and is aware of, the one that matches the string of letters presented to read aloud. The word presented may be a non-word or a word that is not part of the child’s vocabulary (Harris & Coltheart, 1986). Thus, fragmentary cues are used, such as the initial letter, to increase the number of words they can read. For example, the child may know the word dog and that this word is a short word and begins with /d/, and may read other short words that begin with /d/, such as dig, dad, dim, as dog. 10 A child who has just started in this phase and who has a small reading vocabulary uses a small quantity of information from printed stimulus to select an item. He or she may use word length or single letters, for example, reading television as ‘children’ and saying that they knew that was the word children because it is a long word. Any string of letters that has k (such as likes, bkacl or pjoek) may be read as ‘black’, or smaller may be read as ‘yellow’ because yellow has two sticks. These examples indicate that children know that words that are familiar have been taught and those that are unfamiliar have not been taught (Harris & Coltheart, 1986). It becomes more difficult to read using selection techniques as the child’s vocabulary expands, and it is difficult to distinguish between words from the child’s vocabulary based on fragmentary cues. This leads to the next phase of reading development. 1.3.3 The phonological-recoding phase This phonological-recoding phase emerges from formal teaching of letter-sound rules, which are learnt once children realise that spoken words can be broken down into speech sounds. Children also begin to show the ability to read non-words aloud, and responses are no longer selections from reading vocabulary but include words that have not been used before in instruction as well as non-words. A child’s success in reading will be greater if before learning to read he or she understands that spoken words are broken down into individual sounds, each of which is represented by a letter. Having this knowledge assists in learning to read as it is the basis for the phonological-recoding phase (Harris & Coltheart, 1986; Snow, Burns & Griffin, 1998). Children apply the phonics procedure and skills learnt in the discrimination phase, such as the direct procedure, with phonics being dominant at this stage. 1.3.4 The orthographic phase The use of phonological recoding allows the reader to read unfamiliar new words, however, there are disadvantages to phonological recoding which should be discussed before presenting information on the orthographic phase. Children who rely on phonological recoding would not be able to distinguish between homophonic words (those with different spelling but the same sound) found in English, such as fare and fair, because they have the same phonological representation (Harris & Coltheart, 1986). Exception or irregular words in English, such as chair, cannot be read aloud correctly using phonological recoding, therefore, as these examples indicate, phonological recoding cannot be used to become a skilled reader. To do so, one needs to rely on orthographic recoding (Harris & Coltheart, 11 1986), which uses the way words are spelled rather than the way they sound, and is predominantly the recognition of words as orthographic units. Words are recognised through the access of a stored internal (visual) representation of letter-to-letter strings (Frith, 1985), and skilled readers use visual wholes to read words, with phonological recoding being used when they come across a word that is unfamiliar. From the above discussion on phases of reading and reading instruction arise arguments as to whether the phases of reading as proposed by Harris and Coltheart (1986) occur in a fixed sequence or whether this is variable and each phase is a prerequisite for the next. Ehri (2005) argues that there is no fixed sequence in reading development, and that a child can be in two phases at once. This concurs with the phases of reading proposed by Harris and Coltheart (1986), since a skilled reader uses both phonological recoding and orthographic units, depending on the word presented. The phases discussed do not propose a strict criterion for entry into subsequent phases, and although Harris and Coltheart (1986) mention an approximate age range for the first phase, there are no strict age norms for the others. In summary, there is therefore room for flexibility in the phases. The sample for the current study comprised children who were in transition from the phonological-recoding phase to the orthographic phase, and assessed their reading ability using the NARA II. They had some level of proficiency in reading but were neither at the beginning phase of reading nor skilled readers. In terms of this study, learners were compared with regards to their grade and age level against their reading level to determine similarities or differences. During this phase of their learning process decoding skills are strengthened, listening comprehension develops, and reading simple words becomes more automatic. Thus, the focus of reading for these learners at school would be to develop and improve decoding, comprehension and reading rate (Oberholzer, 2005). This will be further discussed in this dissertation. As mentioned in the introduction to this section, English has a non-transparent writing system compared to languages such as Sesotho, Afrikaans and isiZulu (Janks, 2011; Poulos & Msimang, 1998). The relationship between spelling-sound correspondences in languages with transparent orthographies is highly consistent, since words are spelled the way they sound. However, in non-transparent languages such as English they are inconsistent (Snow, Burns & Griffin, 1998), with words in which the grapheme-phoneme correspondences are irregular, such as yacht. Thus, first and second language learners bring different skills and understandings of spoken English to the way they learn to read (Cardoso-Martins, 1995; Greenop, 2004; Spencer & Hanley, 2003), and English may be a challenge to those whose first language is a transparent language (Snow, Burns & Griffin, 1998). The phases proposed by Harris and Coltheart (1986) do acknowledge the phases of reading that second 12 language learners undergo, but research suggests that while first and second language learners undergo the same phases of reading development they do so at different rates (Cockcroft, 2009). The differences are possibly linked to the differences in home language, therefore theories should be flexible and take into account the different developmental approaches linked to various reading methods taught. Various backgrounds and the relationship between the learner’s orthographies, which could be two or more, should also be taken into account. Everyone who learns to read should have knowledge of the letter sounds and names, which is a strong predictor of word and non-word reading development in both first and second language learners. In addition, the reading instructions should be appropriate to minimise reading difficulties (Snow, Burns & Griffin, 1998). This leads to the next section, which discusses the importance of reading assessments as a guide and intervention strategy to provide learning support to those with reading difficulties and to track progress. 1.4 Reading Assessments Assessments are usually conducted if a child is experiencing difficulties with reading but may also be undertaken in order to track progress. Reading assessments are done by teachers, psychometrists, psychologists and speech therapists, who need to establish if there is a difficulty, the causes, and what steps need to be taken thereafter. Consequently, establishing the causes of the difficulty forms an important part of the recommendations made (Beech & Singleton, 1997). Early identification and intervention are important to detect and remediate any learning disorders (Menzies, Mahdavi & Lewis, 2008). Since strengths and weaknesses in reading ability are carried over to later grades (Smith, 1941), early identification and remediation are essential. Education policies in South Africa adopted this belief after 1994 (Vandeyar, 2005), encouraging teachers to measure the level of difficulties experienced through the use of appropriate and standardised reading assessments (Menzies, Mahdavi & Lewis, 2008). Reading assessments typically consist of two forms, the first of which aims to find out how well a child is reading so as to assist in improvement which involves feedback and assisting learners (Pang et al., 2003). The second aims to measure the learner’s progress, but for reading instruction to be effective both forms are important. In the early stages of reading an assessment is made by listening to the child read aloud and measuring word recognition and fluency. For older children and those beyond the early stages, reading assessment involves reading aloud and text comprehension, which is “assessed through questions” (Pang et al., 2003, p.18). Students can respond to these questions in a written or spoken form. This study 13 used the NARA II to assess reading ability and did not address learner progress since the learners were only tested once. In the NARA II, text comprehension was assessed through spoken responses to questions. The results from this study can be used to make recommendations for improvements in reading in South African children. When testing reading ability, the aim is to find out if the individual being tested has the potential to perform academically at his or her age level, referred to as the child’s reading age. In the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev; DSM-IV- TR; American Psychiatric Association [APA], 2000), a reading disorder is defined as reading achievement that is below one’s age level, education and intelligence, and which interferes with one’s academic success and daily activities (Sadock & Sadock, 2003). Characteristics of a reading disorder include inability to recognise words, slowness and inaccuracy in reading, and poor comprehension skills (Sadock & Sadock, 2003). Thus, assessments are crucial to establish whether a child’s reading is progressing age-appropriately. In South African government schools, the Department of Basic Education (DBE) utilises the Annual National Assessments (ANA) in order to improve the quality of education and learning outcomes by assessing numeracy and literacy. The purpose is also to track learner performance and assist in improving it. The results from the ANA should be able to identify which areas require urgent attention to help improve learning, assist provincial departments to make informed decisions about which schools require urgent attention, provide the government and public with information on how well the schools are doing in terms of serving the country’s children, and provide teachers with a baseline measure of the learners’ capabilities at the beginning of the year (DBE, 2011a). This will in turn inform teachers as to where their learners are in terms of target goals, provide parents with information on learner performance, provide teachers with a gauge in developing assessment tasks, and help school management to improve learner performance. 1.5 Conclusion For learning to occur the ability to read is vital, with decoding and comprehension being the basis. The fundamental phonological skills in reading are blending, segmenting and phoneme manipulation. Visual skills include the ability to scan text from left to right and to match visual symbols to auditory sounds. How reading develops depends on the type of instruction used, with phonics, whole word or a combination of the two being most typical. There exist various phases of reading development with no fixed sequence or prerequisite to move to the next phase. Research on reading development proposes that second language learners progress through the same phases as English first language learners, but at 14 different rates. Assessments are therefore vital in establishing how well a learner is reading, assisting with improvement and tracking progress. Reading assessments provide all stakeholders (parents, teachers, schools, and government) involved in the learner’s education with information required to make decisions and changes, provide results on learners and schools, and give teachers information on the level of the learners and how it can improve. The next chapter presents a discussion on the South African context in terms of education and reading. 15 CHAPTER 2: THE SOUTH AFRICAN CONTEXT 2.1 Introduction Since 1994, when South Africa adopted universal suffrage, it has undergone transformations in various spheres, the educational sphere being the most significant for the purpose of this study. Past inequalities were to be addressed so that the country could assume a more democratic identity. This chapter begins with a discussion of the historical context of education and reading as a background to the reasons behind changes in educational policy and reading strategies to improve reading. The inequalities of the past resulted in low reading rates, which will be further discussed in terms of literacy and basic education statistics. Although South Africa is a multilingual country, English is the language of learning and teaching (LOLT) in the majority of schools (De Wet, 2002). The concepts of LOLT and home language will be discussed to background the impact of these on education and reading. The last section discusses the importance of assessing reading in education, since the focus of this research is on a particular reading assessment. The discussion examines how reading assessments can identify reading errors and provide information about learners’ progress. 2.2 Historical Context of Education and Reading Education in South Africa is linked to the dynamics of colonial conquest and missionary work which took place from the 17th to 20th centuries (Prinsloo, 1999). During the apartheid era, education was segregated with the introduction of Bantu education in 1953, by which a curriculum for Black children focussed on vocational skills and how to carry out instructions, so that they would remain dependent on the White-dominated society. English and Afrikaans were the dominant languages during apartheid (Crystal, 2002). From 1990 to 1994 there were intense policy debates facing a transition to a new government, with ‘reconstruction’ and ‘development’ being the key terms. Changes in the political circumstances led to a shift from minority rule to non-racial democracy, and illiteracy was identified as a key social problem. The perception is that, within South Africa, illiteracy is an issue that effects the Black population (Prinsloo, 1999). Failure in reading and poor literacy results are due to “the failure of the education system, a lack of exposure to instruction or formal education, especially due to oppression associated with apartheid in South Africa” (Taylor, 2002, p.35) and learning disability. Historically, literacy was not understood or valued by both the government and public because many resisted access to 16 education or were not allowed to be educated due to the policies associated with Bantu Education. Although, historically, politics have played a role in the high illiteracy rate, problems with reading persist in current society. The abovementioned historical context explains the low literacy rates, which further impacted on how education systems came about, what was done and who could be educated. The educational system in South Africa has undergone various changes since 1994, with new principles based on equity, quality and access (Howie et al., 2008). In terms of reading, the expected outcome is that “the learner is able to read and view for information and enjoyment, and respond critically to the aesthetic, cultural and emotional value of texts’” (DoE, 2002a, p.20). In relation to this study into Grade Four learners’ participation in reading, the DoE states that learners at this level need to be able to: read a variety of texts for different purposes using a variety of reading and comprehension strategies; view and comment on various visual texts; describe their feelings about texts giving reasons; discuss how the choice of language and graphical features influence the reader; identify and discuss aspects such as central idea, character, setting and plot in fiction texts; infer reasons for the actions in a story; recognise the different structures, language use, purposes and audiences of different types of texts; identify and discuss values in texts to cultural, moral, social, and environmental issues; understand and respond appropriately to information texts; interpret simple visual texts, and select information texts for own information needs (DoE, 2002b, pp.72-77). 2.3 Current School Policy linked to Reading and Strategies to Improve Reading With regards to literacy, as the single largest provider of education in South Africa the DoE wanted to increase the literacy rate by 50% by 2015. Reading is encouraged in primary school and those schools that are committed ensure that children read a number of books on a weekly basis. However, there are many challenges faced in terms of promoting literacy, such as poor access by teachers and learners to libraries, media centres and other resources. Shortage of available books is also a major issue, with many schools not having well-used libraries and many homes not having books. In addition, books in African languages are scarce, which results in the majority of the children not having an opportunity to read in their home language. Some classrooms do not have books and some of those that do have ones that are at an inappropriate level (Buthelezi, 2003; DoE, 2008). Assisting and working on changing the reading levels of South Africans would mean learners could move 17 up various grades, matriculation results and communication would improve, and the country would reap the economic benefits. Improvements in reading would lead to confidence, empowering citizens to become creative and think critically, and have access to information and knowledge in life-long learning. If reading is poor then writing is poor and in turn one’s understanding is poor (DoE, 2008). There are various policies and pieces of legislation that are related to reading (Baatjes, 2003) which shape reading in the country and have “direct or indirect implications for reading and literacy education in South Africa” (Baatjes, 2003, p.4). Policy development in terms of reading has been significant, with various educational initiatives supporting the acquisition of reading skills. In 2003, the minister of education Masibudi Mangena, stated that there was still a lack of focus on reading (Patel, 2009), therefore the development of a reading policy should incorporate: “(a) the development of tolls to measure and monitor reading; (b) the implementation of educator training programmes; (c) the mass provision of books and reading support material; and (d) monitoring and evaluation systems” (Baatjes, 2003, p.5). In addition, “government needs to recognise its constitutional obligation to literacy education and consider alternative interventions that would adequately respond to the literacy dilemma in the country” (Baatjes, 2003, p.8). The current school policy known as Curriculum and Assessment Policy Statements (CAPS) emphasizes that reading is critical to successful learning. Classroom and independent reading is encouraged so that learners can become critical and creative thinkers. Emphasis is made on reading since learners will be using English as the LOLT in Grade Four. Learners would need to read in their other subjects and use English textbooks. At the beginning of Grade Four, shared reading is encouraged. The methods of Reading with and Reading to the entire class as well as guided group and independent/paired reading are encouraged so that learners can move towards becoming independent readers. Independent reading should follow the time allocated in the teaching plans. In addition, comprehension activities are set by teachers to ensure that learners have understood what they read. The reading process consists of three stages namely pre-reading, reading and post reading (DBE, 2011c). 2.4 Current Reading Statistics From various research studies conducted on the reading achievement of first and second language learners and the gap between the two, evidence shows that many of those learners from poorer socio-economic backgrounds “cannot read for meaning in any language” (Fleisch, 2008, p.3). This is due to the learners’ parents not being taught how to read or these parents not having the resources to assist their children with reading. The 18 Southern and Eastern African Consortium for Monitoring Educational Quality (SACMEQ II) project found that learners did not reach the SACMEQ mean of 500 in reading with South Africa listed as one of the countries at risk of not achieving the Education for All (EFA) goals (De Jager, Nassimbeni & Underwood, 2006; Fleisch, 2008). Most studies and surveys conducted are based on adult education and literacy levels as well as general census, therefore it is difficult to pinpoint illiteracy in terms of age. The highest level of education of South Africans aged 20 years and older as per the 2011 General Population Census indicated that 8.6% had no schooling, 12.3% had completed some primary schooling, 4.6% had completed primary schooling, 33.9% had completed some secondary schooling, 28.9% had completed Grade 12 and 11.8% had completed higher education (Statistics South Africa, 2012). Grade Three results on literacy achievement in 2011 based on the Annual National Assessments (ANA), indicate that 53% were not achieving in literacy, 16% were partially achieving, 20% were achieving and only 11% were showing an outstanding result. This indicates that 69% of learners in Grade Three were not achieving the necessary literacy skills required at Foundation Phase learning (Bot, 2011), which would have a major impact on their future school performance and achievements. In 2001, 51 000 Grade Three learners were randomly selected and completed a government initiative assessment with the reading task comprising an oral and a reading and writing domain. When the literacy task was divided into an oral activity and a reading and writing activity, the average reading and writing score was much lower, at 39% (Fleisch, 2008). The DoE results of the Grade Six Systematic Evaluation of 34 015 learners tested in 2003 showed that these learners obtained a mean score of 35% for language, repeating the trends evident in the Grade Three test (Fleisch, 2008). Only 28% of these Grade Six learners achieved an average to above average standard as per the requirements of the curriculum for language, showing that more than two-thirds were performing below the expected level. Monitoring Learning Achievement (MLA), one of the first cross-national studies in which South Africa took part, with 10 400 Grade Four learners from nine provinces in 400 schools, revealed that South African Grade Four learners scored 48.1% for literacy, with almost 44% scoring below 25%. Scores of 75% and higher were achieved by only 12% of the learners (Fleisch, 2008). In 2006, together with 45 education systems in 40 countries, South Africa participated in the Progress in International Reading Literacy Study (PIRLS), a home language test taken by children in an additional language. South African children who participated were in Grades Four and Five, for both of which they scored the lowest out of all the countries (Howie et al., 2008). It was argued that the low literacy rates were due to lack of funding in education, 19 discrepancies in fees at schools, insufficient teacher training, lack of materials in African languages, low or no access to books, poor teaching and poor home literacy practices. In addition, in order to better understand the low literacy rates, consideration needs to be given not only to what is happening in schools but also to what takes place in the home environment. As mentioned in the previous chapter, many children from disadvantaged backgrounds do not start formal school with pre-literacy skills such as letter recognition and phonological awareness having been taught at home. 49% of the Grade Four learners’ parents in the PIRLS study expressed high involvement of early home literacy activities with their children before the start of formal schooling, whilst only 34% of the parents read on a weekly basis. Less than 50% of South African children have more than 10 books in their home compared to 78% internationally (Howie et al., 2008), which is also related to many learners not having previously enrolled in Grade R. This does not mean that attending a Grade R classroom provided learners with the literacy skills in preparation for reading and writing, as a study in four provinces found, even though the children in the study attended a Grade R class, they were still not equipped with basic early literacy skills (O’Carroll, 2011). In terms of the parents’ education and socio-economic status, 26% of the parents of Grade Four learners assessed in the PIRLS study did not have a basic school exit qualification, with another 13% having no matriculation certificate (Howie et al., 2008). The above statistics gives an indication of the South African context regarding literacy. These results are alarming and indicate a need for changes in the way children are educated at school, particularly how reading is taught and monitored. An appropriate reading policy as well as funding to further enhance the reading stimulation that occurs in schools is also required. Since various languages are spoken in South Africa, the next discussion is on home language and the LOLT. 2.5 The Language of Learning and Teaching and Home Language Languages vary based on their oral and written aspects The mismatches that occur between these aspects add to the difficulties of learning to read (Geva, 2006), in addition to which the language of the school and home may be different. In 1994, South Africa established 11 languages as official so that all would have the same status as English and Afrikaans, legally at least. Also, it was hoped that this would address issues around English not being the language used by the majority. However, many continued to regard proficiency in English as vital for educational success (Buthelezi, 2003), and illiteracy continued to be more common in first languages other than English (see Figure 2.1). 20 Figure 2.1 Illiteracy by language in South Africa. Adapted from Kha Ri Gude, DBE, 2012. Retrieved from http://www.kharigude.co.za/index.php/literacy-in-south-africa. 2008-2012 by Internet Support Services. Adapted with permission. Thus, it is important to distinguish between the LOLT and the learner’s home language, because for 45.8% of learners in this study English was a second language. The LOLT is defined as “a language medium through which learning and teaching including assessments occur” (DBE, 2011b, p.45), whilst home language is defined as “the language that is spoken most frequently at home by a learner” (DBE, 2011b, p.41). According to the Constitution (1996a) and the South African Schools Act (1996b), where practical all learners have the right to be educated in the official language or one they choose within the public educational constitutions. According to De Wet (2002), being educated in one’s home language is most appropriate for reading and writing skills, however most South Africans prefer being educated in English. African languages have orthographic structures different from those of English and Afrikaans. For some learners, the LOLT is an associated language or the same one as their home language, whilst for others the LOLT and the child’s home language are not associated. A child whose home language is isiZulu for example, may have isiXhosa as the associated language. However, English and Afrikaans are not associated languages (Janks, 2011). In most cases South African children learn to read in a language they do not know and which is vastly different from their home language. By talking to children in the early years, their knowledge of words and meanings are developed in preparation for literacy. However, many come to school with no knowledge of words in the language in which they will be taught to read (Janks, 2011) and therefore they cannot be expected to perform as well as those reading in their home language, due to the language/literacy switch. The LOLT for Grade Four learners is mostly in English (80%), which is a language spoken by less than English 1% isiZulu 29% isiXhosa 20% Sepedi 12% Setswana 9% Xitsonga 7% Sesotho 7% Afrikaans 5% Siswati 4% Tshivenda 3% isiNdebele 3% 21 10% of the population (Howie et al., 2008). However, research shows that many in Grade Four are unprepared for English as the LOLT as they do not have the vocabulary, language or literacy skills to learn in it (Janks, 2011). Learners reading in a language different from their home language find reading difficult because they use more visual information to compensate for lack of familiarity with the vocabulary, unlike those who know the words, have knowledge of the context and so are able to read faster. When some second language English learners read they are often so caught up in trying to understand the meaning of the word that they lose track of the sentence as a whole (Janks, 2011). Therefore, reading is much more difficult and slower if a learner does not have the vocabulary and understanding of meaning of the words. Research has found that second language learners take longer to develop proficiency in the LOLT than those whose home language and LOLT are the same. Thus, the current study compared the reading skills of first and second language learners in order to determine how reading in a language different from one’s home language impacts on reading skills and academic performance. In addition, knowledge of words in the language in which reading is taught impacts on reading performance. Comparing the two groups may show that first language readers in the study have a better knowledge of vocabulary in English than the second language readers, who may have no or little knowledge of the vocabulary. Further consideration will be given to potential strategies to improve this. Developing language and reading skills in the LOLT is a gradual process whereby the learner is able to comprehend and express him/herself in the LOLT, orally, in writing, and in everyday and academic contexts. This includes being familiar with the phonology of the LOLT, vocabulary, morphology and grammar (Geva, 2006). This leads to the next discussion on how assessing reading in education can assist learners in improving their reading skills. 2.6 Assessing Reading in Education Reading forms the basis for formal learning (Kirby, 2009), and high quality education depends greatly on literacy. If children become good readers in the early grades they are likely to become better learners in their school years and beyond (Prinsloo & Bloch, 1999). Within South Africa, with its generally large classrooms, teachers do not have time for one- on-one teaching, and self-study through reading is common. Many children are not keeping up with the demands of school due to their low literacy skills (Taylor, Pearson, Clark & Walpole, 1999), thus producing adult citizens who cannot readily access important information on “health, social, cultural and political issues” (Baajtes, 2003, p.1). Reading is 22 also important for personal pleasure and enrichment, but to develop the ability to read people need access to textbooks and supplementary reading material. From the above statistics it is evident that many children move through the education system without being able to read properly and then they leave school without having the necessary reading skills required thereafter (Baatjes, 2003). Assessing reading in education plays a vital role as learner progress should be tracked, thus assisting in improving performance through guided planning and resources. Learning, including reading, should improve on a continuous basis since learners come to school to be educated, search for new challenges and gain new understandings (DBE, 2011a). There is a belief that second language learners lack accuracy and fluency because they lack oral proficiency, and they may not be assessed for reading because they have not developed the adequate LOLT proficiency. This issue is twofold: firstly, avoiding interpreting poor language and literacy skills development as a reading disability in second language learners would be an over-diagnosis; and secondly, not assessing these learners’ reading ability may lead to an under-diagnosis of reading disability (Geva, 2006). The current research further contributes to how first and second languages learners are assessed, what tools are used for the assessments and how the results contribute to the over- or under- diagnosis of these learners’ reading potential. Most reading assessment tools provide reading scores which limit the interpretation of how the learner is reading and how to remediate potential problems (Bouwer, 2004). Therefore, a qualitative analysis of reading errors is necessary to establish how much of the language of assessment the learner knows, the skills the learner has acquired and the reading strategies the child utilises (Snow, Burns & Griffin, 1998). Some tests only report a score based on the number of errors made, however, this is problematic as certain errors indicate a serious reading problem. General reading skills can improve without much improvement being shown on reading scores, therefore, comprehensive reading assessments should provide a qualitative analysis of the reading errors that are made (Read America, 2005). According to the NARA II there are six types of errors. Firstly, Mispronunciations are errors made by a learner when a word is only partially decoded, which results in the word being pronounced incorrectly or distorted, resulting in no meaning. Secondly, Substitutions occur when real words are used instead of the one in the sentence the child is reading (Neale, 1997). For example, ‘home’ for house, ‘early’ for earlier or ‘airplane’ for airport. This shows that the learner is not decoding but guessing, based on some of the letters in the word. These errors occur with learners who have better general knowledge and vocabulary skills than actual reading skills. If the substitutions do not make sense there is a problem with 23 comprehension. Thirdly, Refusals are errors in which the child pauses for four to six seconds and cannot attempt to say the word. Fourthly, Additions are errors that occur when a learner inserts a word or part of it in the text. Fifthly, Omissions are when the learner omits a word from the text. Reversals are seen as substitutions but are recorded separately on the NARA II (Neale, 1997). Sixthly, Reversals can indicate neurological problems assisted through occupational therapy. These last three, additions, omissions and reversals, indicate that the learner lacks the skills of reading, and occur if the learner is not paying attention to detail, if there is a concentration problem, if the learner does not have the skills to segment or blend, or if the learner has a problem with comprehension (Read America, 2005). If reading assessments do not pick up on the reading errors that learners are making the correct intervention cannot occur. Early intervention is vital in order to remediate reading difficulties so that the learner can progress in school (Menzies, Mahdavi & Lewis, 2008). On the other hand, if reading assessments do not take place it is not known whether a learner is making the necessary progress in reading in terms of age appropriateness. Without the correct reading skills required for education, the learner will not be able to progress or keep up with the reading demands in school and beyond the school years. Thus, the results of this study established what can be achieved from assessing reading in education, what the results are and what can be done with them to improve reading through intervention strategies. 2.7 Conclusion The above discussion has indicated that the historical context of South Africa has impacted on education, which further impacts on reading ability. Political changes in 1994 included a drive for ‘Education for All’ and the recognition of 11 official languages so that African ones would have the same legal status as English and Afrikaans. Poor literacy rates and failure in reading are in part due to parents not having been exposed to education in the past. Even though the aim was to improve the literacy rate in South Africa, nearly two decades later there remain many challenges. A move towards quality education for all led to the introduction of Curriculum 2005, which did not change the academic performance but resulted in the issue of reading not being addressed. There are many challenges to improving reading in South Africa, including lack of funding, teacher training, materials and use of school libraries. Many homes have few or no books, and those found in classrooms are at an inappropriate level. Books are not readily available in African languages and even though there are policies and legislation related to reading there is no specific reading policy. South Africa has the worse reading scores out of 24 40 countries (De Jager, Nassimbeni & Underwood, 2006; Fleisch, 2008). History and statistics in South Africa indicate the importance of assessing reading in education to track learner progress, indicating reading in comparison to his or her age and grade, and also to provide qualitative information to plan and intervene accordingly. 25 CHAPTER 3: THE NEALE ANALYSIS OF READING ABILITY 3.1 Introduction As addressed in previous chapters, the ability to read is vital for learning to occur. In terms of the development of reading, first and second language learners undergo the same phases of reading development but at different rates, therefore assessments are vital in establishing how well a learner is reading, assisting with improvement and tracking progress. The historical context of South Africa has impacted on education, which includes reading ability; however, psychological tests used in this country have mostly been developed in other countries or during the apartheid era. This was not appropriate to the South African context due to the diversity within the population and the inequalities from apartheid. Against this background, this chapter addresses the development of the NARA II, together with issues of reliability, validity and bias. 3.2 Development of the Neale Analysis of Reading Ability During the 1950s, the original edition of the NARA was constructed in Britain, with a pilot study conducted on 192 children. Revisions were made based on the children’s reading performance and preferences for themes, then tested on 439 children. These results were analysed and minor amendments made. A group of 200 children aged 9, and 200 aged 11 were extensively trialled with 17 other published tests (Neale, 1997), and the final edition was published in 1958. A review of the NARA was undertaken in 1979 to check the suitability of the language and themes for children of that time, with samples tested on the existing material together with new narratives. Certain passages were retained from the original forms, some themes being as popular as they had been in the 1950s. Informal surveys were also conducted with professional workers such as teachers using the NARA to establish their views on the proposed changes. Guiding principles that emerged from the surveys indicated that the story booklet should be retained, rather than making drastic changes. However, up-to-date narratives, information on diagnosis, planning programmes and new norms were required. Thus, a new Australian Edition in 1988 was later revised to incorporate aspects of the British educational and cultural environment. Minor modifications were made to the text, with a balance made between male and female characters in the narratives and the passages updated (Neale, 1997). In 1988, the British standardisation of the NARA took place with a sample of 1760 children, under one of the age groups from 6-12 used in the norms. The sample represented children from schools in England and Wales (Neale, Christophers & Whetton, 1989), whilst other aspects of the test were included in smaller studies, such as a comparison of the scores 26 between the NARA-Revised and the original NARA (Neale, 1997). In 1996, further standardisation took place, resulting in the Neale Analysis of Reading Ability-Second Revised British Edition (NARA II), which is the version used in this study and will be further discussed in Chapter 4 of this research report. Since the focus of this research was the applicability of the NARA II for use with South African learners, the various psychometric properties examined in the study are elaborated on below. 3.3 The Reliability of the NARA II Reliability refers to how consistent a measure is in terms of what it is measuring (Gunning, 2010; Wolfaardt & Roodt, 2005). If a measure is reliable it should provide the same results each time the same thing is measured, thus the information provided by the instrument does not change as a result of its characteristics. The measure would be unreliable if every time the instrument was used it provided varied results (Neuman, 1997). There are four types of reliability, namely test-retest, parallel/alternative forms, inter-rater and internal consistency reliability, each of which will be discussed below in relation to the NARA II. 3.3.1 Test-retest, parallel/alternate forms and inter-rater reliability Test-retest reliability involves the administration of a test twice to the same group, thus correlating the scores from both. This should result in the same responses on both occasions, if what is being measured is stable (Neuman, 1997; Wolfaardt & Roodt, 2005). For the purpose of this study, test-retest reliability was not utilised since the sample was only tested once on the NARA II. Parallel/Alternative forms reliability refers to the administration of two equivalent forms of a test at two different times (Schweigert, 1994; Wolfaardt & Roodt, 2005). In the NARA II, this was calculated for accuracy, comprehension and rate through the examination of the two forms used at different times of administration, and coefficients were established in terms of teacher’s Form One and assessor’s Form Two, and teacher’s Form Two and assessor’s Form One. By using the parallel forms reliability coefficients for all pupils, Standard error of Measurement was calculated in the NARA II (Neale, 1997). For the purpose of this study, parallel/alternative forms reliability was not considered, given the difficulties in getting learners to participate and because it was beyond its scope and timeframe. Inter-rater reliability refers to the scoring procedure being subjective and different raters coming up with a different score (Wolfaardt & Roodt, 2005). It was not feasible to test this since the NARA II is scored using standardised scoring procedures that result in the same score even if the test is scored by different individuals (Wolfaardt & Roodt, 2005). 27 3.3.2 Internal consistency reliability Internal consistency reliability refers to “the consistency of responses to all items in the measure” (Wolfaardt & Roodt, 2005, p.30), which is the inter-item consistency that looks at how consistent a measure is in terms of what it is measuring. In the NARA II, internal consistency reliability was established for Accuracy and Comprehension using Cronbach’s alpha (Neale, 1997). Furthermore, a study by De Sousa and Broom (2011) found internal consistency coefficients of .88 for first language learners and .80 for second language learners on Comprehension using the NARA-Revised. Research by Spooner, Baddeley and Gathercole (2004), found an internal reliability coefficient of .67 for Comprehension using the NARA-Revised, which is lower than both the internal consistency coefficients from the standardised sample, which yielded internal consistency coefficients of .87 for Accuracy and .91 for Comprehension and lower than the internal consistency coefficients of .81 for Accuracy and .90 for Comprehension as found by Fung, Wilkinson and Moore (2003). Internal consistency reliability for the purpose of this study was measured using Cronbach’s Alpha (CA), by considering the internal consistency coefficients for each level of the comprehension questions of the NARA II. Thus, the NARA-Revised shows good internal consistency when used with local samples. 3.4 The Validity of the NARA II According to Wolfaardt and Roodt (2005), validity refers to what the test measures and how well it does so. A measure is valid for a particular purpose; therefore what may be valid for one could be invalid for another (Neuman, 1997). There are four types of validity, namely face, content, criterion-related and construct validity. 3.4.1 Face Validity Face validity is the judgement made by participants as to whether the test ‘looks valid’ (Neuman, 1997; Wolfaardt & Roodt, 2005). For this study it was assessed through a question in the semi-structured interview with professionals. The interviewees were given the NARA II and thereafter were asked: “Please have a look at some stories from the Neale Analysis of Reading Ability. Do you feel that this assessment tool is appropriate and suitable for the South African context and indicate what you think of it as a measure?” 28 3.4.2 Content Validity Content validity is achieved if the content of the measure takes into consideration the representative sample of the domain that is being measured (Neuman, 1997; Wolfaardt & Roodt, 2005). In the NARA II, it is stated that content validity was established since it requires reading aloud and the ability to answer comprehension questions (Neale, 1997). However, the manual makes no mention of the theory used in the development of the NARA. Content validity for this study was conducted by calculating frequencies for the word list and story titles that professionals felt were inappropriate for the South African context. The lists were selected from the NARA II passages. 3.4.3 Criterion-related Validity Criterion-related validity can be divided into concurrent validity and predictive validity, the former referring to the precision with which a test can diagnose the current behaviour regarding specific skills of an individual (Wolfaardt & Roodt, 2005; Neuman, 1997). With criterion-related validity, the NARA II showed adequate criterion-related validity when correlated with other reading scales. Concurrent validity was explored quantitatively, whereby a correlation coefficient was calculated between a predictor (the NARA II measure) and a criterion (learners’ June literacy marks) so as to determine any significant relationships (Wolfaardt & Roodt, 2005). 3.4.4 Construct Validity Construct validity refers to whether a test measures the theoretical trait it is supposed to measure (Wolfaardt & Roodt, 2005). Moore, Morton and Price (2007) expanded on this to include how closely the test reflects the model of reading underlying the test. In the NARA II, construct-related validity was established, showing an increase in mean scores with age and that age differentiation has been achieved in terms of Accuracy, Comprehension and Rate. Limits of Accuracy and Rate of reading are reached through the indication of an increase in scores from year to year, which tails off between the ages of 11 and 12 (Neale, 1997; Neale et al., 1989). For the purpose of this study, construct-related validity was not considered as it was beyond the scope and since one grade and age level was considered. 29 3.5 Examining Bias in the NARA II Bias is the “unfair disadvantage/advantage to one or more groups” (Kanjee, 2005, p.62), therefore test items need to be common to all the cultures in which they are to be used (Foxcroft, 2002). An item may present a characteristic that result in varied performance for individuals who have same ability but who are from different groups (Ronald & Rogers, 1995). This is relevant to the South African context, in which there are diverse cross-cultural settings, test scores that provide “one piece of information” (Grieve, 2005, p.225), and only a minimal understanding of an individual (Hurry & Doctor, 2007). Test scores need to be viewed together with the context in which the testee lives (Foxcroft & Davies, 2008). The NARA II is developed in a Westernised country, for use with first language English speakers. Its use in a multilingual country requires investigations for bias and cross-cultural validation to ensure that the test does not favour certain groups. Therefore there are several predominant issues that were considered when investigating potential bias in the NARA II. 3.5.1 Gender Gender differences can play a role in reading ability, with research showing that girls tend to have better reading ability than boys. In addition, there is a prevalence of boys with reading disabilities and reading problems (Cockcroft & Blackburn, 2008; Huestegge, Heim, Zettelmeyer & Lange-Küttner, in press; Wheldall & Limbrick, 2010). In order for the NARA II to be valid it should show equal performance between girls and boys, but although revisions were made there was a slight male bias in the content. Despite this, there is evidence that girls perform slightly better than boys for Rate and Accuracy, until the age of 8 years 11 months. This difference lessens with age for all three aspects assessed (Neale, 1997). A local study by Cockcroft and Blackburn (2008) identified the relationships between various sub-tests of the Senior South African Individual Scale-Revised (SSAIS-R) and the NARA- Revised. The results show that boys scored significantly higher than girls on reading comprehension, supporting the notion of potential gender differences in reading for Grade Four learners. A study by Patel (2009) looked at the relationship between the Wechsler Intelligence Scale for Children (WISC-IV) sub-tests and reading ability, using the NARA- Revised. Results revealed that girls scored significantly higher on Accuracy and Comprehension than boys, but that boys scored higher than girls for Rate. For these reasons this research study examined gender differences on the NARA II. 30 3.5.2 Language Language proficiency influences learning potential since it is a prerequisite for reading (Oberholzer, 2005). Within a multilingual country such as South Africa this can have major consequences, and learning to read in a language that is not one’s home language is common (De Sousa & Broom, 2011). Despite many speaking two or more languages, the level of proficiency in each is often not the same, and the language of reading is not the first language (Baker, 2006). Most learners are being educated in a second language and may not be able to use the richness and depth of their first language to enhance their reading experience (Pretorius, 2000). Second language learners may have language difficulties for a number of reasons. They are able to decode a text but are not able to comprehend it, which has a negative impact on learning. Poor performance in a test could be the result of language difficulties if the measure is administered in a language that is not the testee’s home language (Grieve, 2005). Language is one of the most important factors in terms of performance on a measure (Grieve, 2005), and this may be better if the learner is assessed in his/her home language. Language can therefore be seen as a source of bias (Grieve, 2005). A study by De Sousa and Broom (2011) showed that reading performance in second language learners is poorer than that in first language learners, based on a sample of 100 monolingual English and 100 bilingual Zulu-English learners in Grades Three from four public schools. Another study, by Hutchinson, Whiteley, Smith and Connors (2003), showed that second language learners were on average one year behind first language learners on the NARA, based on a three- year longitudinal study which tracked developmental progress from school years two to four on 34 children learning English as an additional language and 43 monolingual children. Another study revealed no difference between first and second language learners on the Stanford diagnostic reading test (SDRT) (Lesaux, Lipka & Siegel, 2006; Lesaux, Rupp & Siegel, 2007; Lipka & Siegel, 2007). Language as a factor was thus important for this study since it took into consideration a comparison of first and second language English learners to investigate any differences between the two groups. 3.5.3 Population group As mentioned above, South Africa’s diversity and history have influenced education. The learners’ varied backgrounds have influenced access to and opportunities in education, and in terms of assessment, sensitivity to values and beliefs should be kept in mind. Apartheid led to social and economic discrimination against Black South Africans, which resulted in income inequality. This impacted on reading since families did not have the income to be 31 educated, to educate their children or to provide educational resources. The variable population group is used in the South African context to represent the diverse groups in South Africa, with categories in line with international research. Definitions of culture often include learners’ religion, nationality or ethnicity (Laher, 2012). Although there has been a move towards a non-discriminatory school environment since 1994, and although access to schools is not based on race and religion, inequalities persist between schools (Jansen & Taylor, 2003). These inequalities included school facilities, quality of teachers and school management. For these reasons, the current study explored the influence of population group on reading ability. 3.5.4 Socio-economic status Socio-economic status is often used as an indicator of the learning experiences and available opportunities for individuals, and can assist in determining one’s educational opportunities since it marks one’s education, occupation and income. In this study, the socio- economic backgrounds of learners, as extrapolated through the educational level of parents, may explain the type of school they are attending and the quality of education they are receiving. In addition, parents who have a higher level of education generally provide more stimulation to their children than those who have a lower level of education or no education at all (Snow, Burns & Griffin, 1998). Educated parents may expose their children to literacy and reading materials which assist in promoting development (Grieve, 2005). Vocabulary knowledge is an important factor in reading comprehension and literacy. Socio- economic status has been found to affect vocabulary learning and consequently affects reading. Studies conducted by Hoff and Naigles (2002) indicate that children from low income backgrounds generally have lower language abilities when they begin school than those from higher income backgrounds. Jean and Geva (2009) report that this gives second language learners, who live in low socio-economic areas, an added disadvantage since they have lower vocabulary skills. Differences in socio-economic status were explored in the current study. 3.5.5 Quality of schooling/education Generally, cognitive test scores indirectly measure what an individual has learnt, which is largely influenced by one’s schooling (Grieve, 2005). By attending school one has the opportunity to have the problem-solving skills, cognitive skills and knowledge to be able to acquire information. However, in South Africa, accomplishing in a certain grade does not 32 necessarily indicate an individual’s achievements. Quality of schooling also plays a role, and a learner attending a school of poor quality may have a great deal of potential but not be equipped with the necessary skills and information to perform well, leaving the score as an underestimation of his or her potential (Grieve, 2005). On the other hand, and specifically relevant to reading ability, schools are where most children learn to read, and if the quality of the school is poor it can negatively influence the child’s reading development (Snow, Burns & Griffin, 1998). The role and attitude of the teachers are influential, and a child who may be struggling with reading and is criticised for reading at a slow pace may become demotivated and disinterested. It is easier for the child to give up and believe that he or she is a ‘failure’ in reading. Sometimes the child can be given reading material that is too difficult or too easy, resulting in frustration or boredom (Oberholzer, 2005). Therefore, interest and motivation are important factors in reading (French, 1986). Since this study was conducted at four DoE schools, a comparison of the results from each could be made to determine their differences and similarities in terms of quality of schooling. The DoE categorises schools into Quintiles. School 1 falls under Quintile Four and Schools 2-4 fall under Quintile Five. Quintiles are based on where the school is situated, the surrounding area, who lives in the area and how affluent the area is in terms of businesses. School 1 has mostly second language English learners from inner cities with the school located in a historically Indian area. Learners form part of a feeding scheme at the school. The school has fewer resources with funding being a major issue. The number of learners per class is also much larger. School 2 is in a well-established area but has mostly second language learners from inner cities. Learners from this school also form part of a feeding scheme, has fewer resources and the number of learners per class is much larger. School 3 is located in an Indian area with mostly first language learners attending the school. Learners come from mostly middle to upper class families. The number of learners per class is much smaller. School 4 is located in a well-established area. There are mostly first language learners who attend the school with smaller class sizes. Learners come from middle to upper class families and the school as various facilities and resources. Schools who fall under Quintiles Four and Five receive the least amount of resources from the DoE in comparison to schools under Quintiles One to Three. In terms of school fees, Quintile Four and Five schools’ fees is decided upon by the school governing body and parents who make a decision on the next year’s fees based on the budget from the previous year and the information from the school’s annual report presented at the Annual General Meeting. The above factors generally contribute to a child’s test-taking ability. Individuals who are illiterate or who receive minimal schooling will not be ‘test wise’, as they will not have the 33 skills or understanding needed to succeed in them. Assessments, like the NARA II, are developed from a monolingual perspective and may not be applicable to South Africa, which is multilingual (Foxcroft & Davies, 2008). 3.6 Conclusion This chapter began with a discussion on the development of the NARA. Definitions and discussion of the various types of reliability and validity were presented, with measure taken to ensure internal consistency reliability, and various forms of validity of the NARA II. Its appropriateness to the South African context was argued, by considering issues of bias through a discussion on gender, language, population group, socio-economic status and quality of schooling, and how these are explored. Having established that they contribute generally to reading ability, the researcher wished to examine more closely their impact within the South African context. 34 CHAPTER 4: METHODS 4.1 Introduction This chapter begins with an outline of the aims and the rationale for the current research study, followed by the research questions categorised into reliability, validity and bias. The sample, instruments, procedure, research design and ethical considerations are presented. Each section is subdivided into a quantitative and qualitative phase. This is followed by the section on data analysis. The chapter concludes with a discussion on self-reflexivity. 4.2 Aims of the Study This study took a United Kingdom-normed reading test, the Neale Analysis of Reading Ability (NARA II), and looked at its applicability in the South African context from a quantitative and qualitative perspective using various stakeholders. The quantitative component aimed to assess a group of Grade Four learners from four public schools in Johannesburg in order to consider issues of reliability, validity, and bias in the NARA II. For the qualitative component, professionals (teachers, psychometrists, psychologists and speech therapists) working with learners from the selected schools and in areas close to them were interviewed to determine their views on the applicability of the NARA II. This included an additional form with a list of words and story titles from the NARA II, as well as asking them to suggest themes they felt were appropriate in stories for South African children. These professionals contributed to what is useful and areas that need improvement for the NARA II to be used effectively to assess reading. 4.3 Research Rationale Numerous aspects of language acquisition take place when the child starts school, as this also means the development of reading, a crucial part of subsequent achievement. Children who experience reading difficulties tend also to experience learning difficulties, which hinders achievement and scholastic progress (Esterhuyse & Beukes, 1997). They need to be assessed to make sure that the skills which form the building blocks of success in school are developing appropriately. Early identification of children with scholastic difficulties allows for intervention and educational instruction to be implemented, and the sooner the child can be assisted to develop full potential at school the greater the likelihood of success (Luiz, Stroud & Jansen, 2005). Assessments are usually conducted because the child is experiencing difficulties with reading, so those assessing reading, usually teachers, psychometrists, psychologists or 35 speech therapists, need to establish if there is a difficulty, its extent, the causes, and what intervention should take place (Bouwer, 2004; Gunning, 2010). Those who have difficulty with reading are suspected of having a reading disorder while others may be assessed because of poor teaching or a lack of practice. Assessing children’s reading skills allows monitoring of their level of achievement, with assessments usually conducted in relation to others of the same age (Bouwer, 2004, Gunning, 2010). Parents are becoming more aware of the issues related to reading and require assessments to find out how well their children are progressing (Pang et al., 2003). Reading assessments are important and therefore the assessment tools need to be reliable and valid. Only a few norm-based reading tests are found in South Africa (Elkonin, Foxcroft, Roodt & Astbury, 2005), assessing individual words and spelling, such as the ESSI Reading and Spelling test (Esterhuyse & Beukes, 1997). However, they do not cover reading comprehension and some are old, making an investigation of the extent to which their content and norms apply to today’s population important (Foxcroft & Davies, 2008). For example, the NARA was developed in the 1950s, with a review in 1979, and an Australian edition developed in 1988. This strengthens the rationale for the current study, namely to find appropriate assessment tools for a particular population. Assessing children’s reading levels requires tests based on a sample of children from the same country, within similar educational settings, and set in the appropriate language and with updated norms and content (Foxcroft & Davies, 2008). If international tests continue to be used, there is a necessity for research into their suitability to local contexts. 4.4 Research Questions This study aimed to quantitatively and qualitatively explore how applicable the NARA II is to the South African context. It progressed in two phases, as follows. 4.4.1 Quantitative Phase 4.4.1.1 Reliability a) Is the NARA II scale reliable in terms of internal consistency reliability in the South African context? 4.4.1.2 Validity a) Does the Grade Four learner’s reading age on the NARA II correlate with their June marks? 36 4.4.1.3 Bias a) Are there any significant differences between genders in terms of errors made on the NARA II? b) Are there any significant differences between home languages in terms of errors made on the NARA II? c) Are there any significant differences between population groups in terms of errors made on the NARA II? d) Are there any significant differences between educational levels of parents in terms of errors made on the NARA II? e) Are there any significant differences between the schools the learners attended in terms of errors made on the NARA II? f) Are there any significant differences between the difference in chronological age and reading age and gender? g) Are there any significant differences between the difference in chronological age and reading age and home language? h) Are there any significant differences between the difference in chronological age and reading age and population group? i) Are there any significant differences between the difference in chronological age and reading age and educational level of parents? j) Are there any significant differences between the difference in chronological age and reading age and the schools the learners attended? 4.4.2 Qualitative Phase 4.4.2.1 Face Validity a) Is the NARA II appropriate and suitable for the South African context? 4.4.2.2 Content Validity a) Are the words and story titles in the NARA II appropriate for the South African context? 4.4.2.3 Applicability of the NARA II a) Is the NARA II appropriate and suitable for the South African context? 37 4.5 Sample Samples for each phase were selected as follows. 4.5.1 Quantitative Phase Data was collected using non-probability convenience sampling from children attending public schools in and around Johannesburg, to include those from different socio-economic backgrounds. Permission from four public schools, which represented a good spread of learners from various backgrounds, was sought. The sample consisted of 144 learners. Grade Four learners were chosen for the sample because the study required a level of proficiency in reading. The LOLT in the Foundation Phase for these learners was English. Grade Four learners are neither at the beginning stage of reading nor skilled readers since skilled readers are at the orthographic phase of reading development while Grade Four learners are in transition from the phonological-recoding phase to the orthographic phase. Table 4.1 represents the frequencies obtained for the demographic variables relevant to the analysis, namely gender, population group, home language, educational level of parents and the school learners attended. It is evident that the learner sample was equivalently distributed in terms of gender (50.7% males and 49.3% females). The sample comprised 54.2% of first language English speaking learners and 45.8% of second language English speaking learners. In addition, 52.1% were African, 40.3% Indian and 7.6% classified as “Other” (for example, White, Coloured, Cape Malay and Arab). In terms of the highest educational level of parents, 13.2% had less than matriculation (matric), 34% had a matric and 52.8% had a post-matric qualification. With regards to the schools, learners were sampled from four government-funded schools with the majority from School 3 (30.6%) and the fewest from School 4 (19.4%). The schools were described in terms of Quintiles and fee structure.1 1 Refer to Chapter 3 page 32 38 Table 4.1 Frequencies for gender, population group, home language, educational level of parents and learner school Variable Frequency % Gender Population group Home language Educational level of parents Male Female African Indian Other English first language English second language Less than matric matric Post-matric 73 71 75 58 11 78 66 19 49 76 50.7 49.3 52.1 40.3 7.6 54.2 45.8 13.2 34.0 52.8 School School 1 School 2 School 3 School 4 41 31 44 28 28.5 21.5 30.6 19.4 Table 4.2 presents the means, standard deviations, minimum and maximum values, and skewness coefficients for the demographic variables age of learner, number of years exposed to English, Overall June aggregate, Aggregate for English and Aggregate for Mathematics. The average age of learners was 119.04 months (9.92 years), with a standard deviation of 6.44 months (0.54 years), with ages ranging between 103 months (8.58 years) and 143 months (11.9 years). The number of years exposed to English was on average 91.83 months (7.65 years), with a standard deviation of 31.91 months (2.66 years), with a range of between 24 months (2 years) and 140 months (11.67 years). The overall June aggregate, which comprises an average of the total number subject marks on learners’ June school report, has a mean of 64.44% (SD=13.60), with scores ranging between 27% and 88%. The aggregate mark for English has a mean of 65.28% (SD=16.53), with scores ranging between 15% and 91%. The aggregate mark for Mathematics has a mean of 68.17% (SD=14.40), with scores ranging between 16% and 96%. 39 Table 4.2 Descriptive statistics for demographic information of learners Variable Mean SD Minimum Maximum Skewness Age of learner (months) Exposure to English (months) Overall June Aggregate (%) Aggregate for English (%) Aggregate for Mathematics (%) 119.04 91.83 64.44 65.28 68.17 6.44 31.91 13.60 16.53 14.40 103 24 27 15 16 143 140 88 91 96 .60 -.37 -.45 -.62 -.58 4.5.2 Qualitative Phase These participants were selected using purposive sampling, a non-probability sampling technique by which subjects are selected on the basis of available information or the researcher’s knowledge about the population (Collins et al., 2000). The sample included eight professionals working with reading problems. Respondent 1 and 2 are both educational psychologists from the White race group and in private practice, Respondent 3 is a psychometrist in private practice from the White race group, Respondent 4 and 5 are speech therapists from the Indian race group and working at a school, Respondent 6 is a reading therapist and trainer in private practice and from the White race group, Respondent 7 and 8 are both teachers with Respondent 7 coming from the Indian race group and from School 1 while Respondent 8 being from the African race group and from School 2. All eight professionals were females. One teacher per school and one professional (either psychometrist or psychologist or speech therapist) working in close proximity to the school were interviewed. 4.6 Instruments 4.6.1 Quantitative Phase Instruments used to collect data in the quantitative phase were as follows. 4.6.1.1 Demographic questionnaire Demographic information was obtained from parents/guardians after they provided consent for their child to participate in the study. This included the child’s name, date of birth, age, gender, population group, level of schooling, home language, a copy of the learner’s school report which included their June results, how many years the child had been exposed to English, any information on the child having learning disabilities and each 40 parent’s/guardian’s highest level of education. The level of education was required as this research was exploring if socio-economic status plays a role in determining reading ability extrapolated through the educational levels of parents. It should be kept in mind that the child’s name did not appear at any time in the research report and the only reason for asking for the name was for the data records, which remain confidential. Each participant was allocated a number to preserve anonymity as stipulated in the ethical guidelines and agreed upon with all the participants. A copy of the demographic questionnaire is presented in Appendix A. 4.6.1.2 The Neale Analysis of Reading Ability (NARA II) The Neale Analysis of Reading Ability (NARA II) is a test of English reading ability that assesses reading accuracy, comprehension and rate in learners aged 6 years to 12 years and 11 months. Consisting of a set of graded stories which increase in difficulty (Neale, 1997; Neale et al., 1989), it takes approximately 20 minutes to administer and involves the interaction of a child reading and an administrator listening. There are four comprehension questions in the first passage and eight for the subsequent passages, which are asked verbally and require a verbal response. When a child makes more than two errors an easier passage is presented, but if he or she makes fewer than two errors a more difficult passage is presented until a ceiling has been reached. Errors are recorded and form part of one of six types of error discussed in Chapter 2 (Neale, 1997). The NARA II is available in two forms, which are parallel standardised forms allowing the child to be assessed a second time in order to measure progress, but for the purpose of this study, Form One was utilised (Neale, 1997). The test material is in the form of a book which consists of graded narratives accompanied by pictures. The pictures serve to set the scene and have three practical purposes: to change the test material to a more appealing form, to help the reader change from one train of thought to another; and with some children to assist in spontaneous conversation that in turn help the administrator make recommendations (Neale et al., 1989; Neale, 1997). 4.6.1.3 Development of the NARA II Having discussed the development of the NARA2, the standardisation of the NARA II will be discussed below. Since the introduction and establishment of a National Curriculum in England and Wales, an associated testing administration came about after the first edition of 2 Chapter 3 pages 25-26 41 the NARA-Revised (Neale, 1997). This led to debates about reading and reading methods, giving rise in the 1990s to a new standardisation of the test. The process included the involvement of a larger number of schools and learners, with revision simplification of the administration instructions. An audio tape was created that illustrated the techniques to be used in the test, and administrators had to test four children as practice before the target children were tested. According to the NARA II Manual, repeated testing by trained assessors took place to check the scores, whilst other tests related to reading, such as the Phonological Assessment Battery (PhAB) and the British Ability Scales Second Edition (BAS II) were also administered to establish a relationship. However, the exact results are not provided. Provision was made for standard scores and weighting the sample by measures related to school attainment and gender for the norms to be nationally represented (Neale, 1997). The standardisation took place in 1996 between May and July, which was the final term of the school. There was a selection of 1558 schools with 1928 children tested on Form One and 1546 on Form Two, resulting in the NARA II (Neale, 1997). 4.6.1.4 Domains that the NARA II measures The NARA II measures reading through accuracy, comprehension and rate. Reading accuracy is word recognition in context of the stories, with a record of errors made providing an insight into it. Reading comprehension is assessed by a number of questions about each story to help measure the child’s use of contextual cues such as pictures, prompts and language. Reading rate is calculated across all stories as the average number of words read per minute (Neale, 1997). The raw scores can be converted to standardised scores, percentile ranks and reading ages for accuracy, comprehension and rate. It is a standardised test, therefore an individual can be compared to a UK population in the absence of norms for South Africa. 4.6.1.5 Psychometric properties of the NARA II In terms of reliability, two types are available for the NARA II. Internal consistency reliability coefficients for Accuracy were .82, .88, .86, .86 for Form One, and .81, .87, .86, .87 for Form Two. Reliability coefficients for Comprehension were .93, .95, .94, .93 for Form One, and .94, .95, .94, .93 for Form Two (Neale, 1997). These were based on four aged categories namely 6 years to 7 years 11 months, 8 years to 9 years 11 months, 10 years to 11 years 11 months and 12 years to 12 years 11 months. The sample for this study came from the age categories 8 years to 9 years 11 months and 10 years to 11 years 11 months. The 42 information above indicates that the reliability for both Accuracy and Comprehension were good. Parallel forms reliability coefficients were .89 for Accuracy, .82 for Comprehension and .66 for Rate (Miller-Whitehead & Rhoades, 2005; Neale, 1997). Reliability for Accuracy and Comprehension are higher with levels of reliability for Rate being much lower. This indicates that the NARA II should be used for diagnostic purposes and not for important decisions regarding children’s futures (Neale, 1997). Parallel form reliability was further checked through the examination of two different orders of administration and reliability coefficients yielded .92 and .87 for Accuracy, .84 and .78 for Comprehension and .65 and .67 for Rate. These indicate the same level of correlations as for standardised scores mentioned above (Neale, 1997). Standard Error of Measurement was calculated by using the parallel form reliability coefficients for all pupils, and yielded results of 4.9 for Accuracy, 6.1 for Comprehension and 8.4 for Rate (Neale, 1997). 4.6.2 Qualitative Phase 4.6.2.1 Semi-structured interviews Semi-structured interviews were used to investigate professionals’ views of the applicability of the NARA II in the South African context. The research schedule was devised by the researcher. A range of research questions were then formulated. Questions were developed in terms of their relevance to the aims of this research study and elicited information on their views on the appropriateness and suitability of the NARA II in the South African context; the types of errors their learners make in reading; suitable assessment tools for reading; the reading levels of the learners; and the appropriateness of international tools in the South African context. These professionals would contribute to what is useful and areas that need improvement within the area of reading assessments, and provide information on the NARA II in terms of its use with South African children, how well it was working and what changes or adaptations needed to be made. During the interview the interviewees were provided with an additional form to fill out which had a list of words used in the NARA II (Form One), story titles from the NARA II (Form One), and to fill out themes they felt were appropriate in stories for South African children. The list of words and story titles were used to establish whether they felt they were suitable for the South African context, and to establish what changes or adaptations needed to be made for the test to be suitable for the country. Professionals who consented to participate were interviewed and informed that all personal information would be kept confidential. The researcher and the research supervisors were involved in the review and revision of 43 questions for suitability through a pilot guide to identify any novel issues. The questions were revised again before a list of 14 was finalised. A specific list of questions included open-ended, close-ended and follow-up questions. The open-ended questions encouraged the interviewee to provide their own dispositions, ideas, thoughts and feelings (Collins et al., 2000), thus providing their “own answers to the questions’ (Babbie & Mouton, 1998, p.233), while the close-ended questions were easily processed and gave greater uniformity (Babbie & Mouton, 1998). For the purpose of this study, closed-ended questions were used for participants to answer yes or no questions and open-ended questions for participants to elaborate on particular questions, thus allowing them to provide their opinions, views and thoughts. Follow-up questions were used to probe the responses (Collins et al., 2000). (Refer to Appendix A for interview questions). 4.7 Research Design This research study used a mixed methods approach, thus producing both statistical evidence and rich descriptive accounts regarding the applicability of the NARA II in the South African context. The type of mixed method approach is known as the Sequential Explanatory Strategy, and is straightforward in comparison to other mixed method approaches. Quantitative data is collected and analysed, followed by qualitative data, with priority given to the former. In the current study the first phase was a quantitative approach that used the NARA II to assess the learners in order to establish their reading age, followed by a qualitative approach in which professionals were interviewed. The sets of quantitative and qualitative data were used to complement each other, and two sets of findings were integrated at the interpretation phase. The advantage of this ‘triangulation’, that is use of several methods that can be used to cross check, is that it allows for more comprehensive and in-depth exploration and results in more conclusive evidence. It is also useful when unexpected findings arise in the quantitative phase and it is relatively easy to report. The disadvantage is that it is time-consuming (Creswell & Plano Clark, 2007). 4.7.1 Quantitative Phase The quantitative phase used a non-experimental cross-sectional design, as learners were assessed at one time in a natural setting, their school environment. There was no manipulation of the independent variable or control measure, and the learners were not randomly assigned to groups. Therefore, it did not meet the requirements of an experimental design (Babbie & Mouton, 1998). 44 4.7.2 Qualitative Phase The qualitative phase employed semi-structured interviews for the professionals, aiming to describe their experiences and obtain in-depth and comprehensive understandings of the research topic, thus focusing on process rather than outcomes. It aimed to report on participants’ views in a particular context (Babbie & Mouton, 1998), therefore the interviews allowed the researcher to obtain a better understanding of the professionals’ perceptions of the applicability of the NARA II in South Africa. The semi-structured interviews were conducted face-to-face and audio-recorded, allowing the researcher to pay full attention to what was being said. 4.8 Procedure 4.8.1 Quantitative Phase Parents/Guardians were handed a letter (Refer to Appendix B) which included an envelope addressed to the researcher. Parents/guardians who consented to their child participating in the study were given the demographic questionnaire to complete and asked to hand it and the completed consent form in a sealed envelope to the class teacher. The researcher collected these sealed envelopes from the schools, and those who did not give consent for their child to participate were not included in the study. Using the NARA II, each child was assessed individually in one session of 20-30 minutes in a quiet area of the school. On the day of the assessment, learner assent was first established by the researcher through an introduction, informing the learner what the session would be about and the process. Learners were then asked to give their assent by writing their name on the assent form (Refer to Appendix C). They were told that they could stop at any point if they did not wish to continue, without reprisal. Once this was established the learner read aloud stories while the researcher counted and corrected reading errors. Comprehension questions were asked once no more than 16 errors had been made on a story. Learners moved through a number of stories that increased in difficulty until they made 16 or more errors. 4.8.2 Qualitative Phase Professionals were provided with a letter (Refer to Appendix D) which had information on the research study, including how long the interview would take and a consent form that they were required to fill out if they chose to participate in the interview. Once consent was provided a time was arranged between the researcher and the professional for the interview. 45 They were reminded that confidentiality of personal information would be maintained at all times. All interviews were audio-recorded and transcribed. 4.9 Ethical Considerations Ethical clearance was sought from the Human Research Ethics Committee of the University (Refer to Appendix E). Permission was granted by the Gauteng Department of Education to collect data at the schools (Refer to Appendix F). The school principals were then given a letter to inform them of the nature of the study and request their permission for data to be collected at the school (Appendix G). Once principals granted permission, letters were sent out to the learners’ parents/guardians for consent to continue, which included a self- addressed envelope which had to be handed to their child’s class teacher. 4.9.1 Quantitative Phase For the quantitative phase, parents/guardians were provided with a letter to explain the purpose of the study and those who gave consent for their child to participate needed to do so in writing. Each child was also asked if they wanted to participate in the study and were then asked to write their name on an assent form. This allowed the learners to be informed about the nature of the research, so that they and the parents/guardians could be informed that participation was voluntary, that they could withdraw from the research at any time without prejudice and that all research information would be kept confidential. In addition, they were informed that their participation or non-participation would have no impact on their academic input and evaluation. Parents/guardians were informed that all information would be kept confidential at all times. Once assent was given, each participant was allocated a number so the results would be anonymous. A separate database was kept by an administrator not connected to the research, with the names of the participants and their corresponding numbers, in case a longitudinal follow-up of the research be carried out at a later stage. Record forms, consent forms and biographical questionnaires were placed in a locked cupboard and captured data stored in a password-protected file. 4.9.2 Qualitative Phase All professionals received a letter informing them of the nature of the study and the length of the interview, and before giving their informed consent to participate were told that the interview would be audio-recorded. In the letter they were informed that participation was strictly voluntary and that they did not have to answer all the questions should they choose 46 not to. Participants were informed that they could withdraw from the study at any time and would not be discriminated against in any way. Written informed consent was obtained from the professionals for audio-recording and their voluntary participation. They were informed that their identity and any information they provided would be kept confidential at all times. In all results, participants would be referred to by a pseudonym and they were informed that the research results might be published in a journal article or presented at a conference. There were no benefits or risks associated with the study and they were informed that all personal information and recordings would be kept confidential at all times, and that only the researcher and supervisors would have access to the information. After the study had been completed the tapes would be kept in a locked cupboard at the University for three years and destroyed thereafter. Tapes are kept primarily to facilitate the research being presented or published. A separate database is being kept by an administrator not connected to the research, with the names of the participants and their corresponding numbers. This was done should a longitudinal follow-up of the research be carried out at a later stage. The audio-recordings, consent forms and transcriptions are being kept in a locked cupboard and all captured data stored in a password-protected file. 4.10 Data Analysis 4.10.1 Quantitative Phase The following methods of data analysis were employed in the quantitative phase. 4.10.1.1 Descriptive statistics Data was examined through descriptive statistics for both the demographic information and the NARA II domains. This involved computing the data to provide frequencies for variables such as gender, home language, population group, educational level of parents and the school that learners attended. Means, standard deviations, minimum and maximum scores and skewness coefficients were established for age, number of years exposed to English and school results (overall June aggregate, English aggregate and Mathematics aggregate). For the NARA II domains means, standard deviations, minimum and maximum scores and skewness coefficients were established for Accuracy, Comprehension and Rate overall, at each level, and for the difference in chronological age and reading scores on Accuracy, Comprehension and Rate. To determine if the data was suitable for parametric statistical analysis, skewness coefficients were calculated for the NARA II and Levene’s test for homogeneity of variance was used. For those variables that did not meet the criteria for parametric statistical analysis, non-parametric statistical analysis was used. 47 4.10.1.2 Examining the reliability of the NARA II Reliability was explored specifically for internal consistency reliability using Cronbach’s Alpha (CA) coefficients. Internal consistency coefficients were calculated for each level of the comprehension questions of the NARA II. 4.10.1.3 Examining the validity of the NARA II For face validity, frequencies were calculated for question two of the semi-structured interview, in which professionals who participated were asked: “Please have a look at some stories from the Neale Analysis of Reading Ability. Do you feel that this assessment tool is appropriate and suitable for the South African context and indicate what you think of it as a measure?” For content validity, frequencies were calculated for the word list and story titles from the NARA II, in terms of the words and stories that were inappropriate for the South African context. For concurrent validity, Pearson’s Product Moment correlation coefficients were calculated to determine if there were any significant relationships between the learners’ Accuracy, Comprehension and Rate scores and their school results (overall June aggregate, English aggregate and Mathematics aggregate). 4.10.1.4 Examining bias in the NARA II Firstly, the NARA II errors were compared with each demographic variable (gender, home language, population group, educational level of parents and the school the learners attended) to determine if there were any significant differences. Parametric independent t- tests and non-parametric Mann-Whitney U tests were calculated for gender and home language, while parametric Analysis of Variance (ANOVA) and non-parametric Kruskal Wallis tests were calculated for population group, educational level of parents and the school the learners attended. Secondly, the difference between the learners’ chronological age and reading age (Accuracy, Comprehension and Rate) was compared to the demographic variables (gender, home language, population group, home language, educational level of parents and the school the learners attended) to determine if there were any significant differences. This was also calculated through parametric and non-parametric independent t-tests and ANOVAs. 48 4.10.2 Qualitative Phase 4.10.2.1 Thematic content analysis Questions from the semi-structured interviews that were open-ended, in which each participant could make comments, were analysed qualitatively using thematic content analysis. According to Braun and Clarke (2006), this is a method that is used in order to identify, analyse and report on themes within a data set that can also interpret features of the research topic. The six steps that are recommended by Braun and Clarke (2006) guided the analysis. The data was transcribed, read through and checked, and given initial codes with the researcher’s pre-existing theoretical position being kept in mind. Possible themes and sub-themes were then created based on grouping the codes, which were modified until they captured the quality of the responses (Braun & Clarke, 2006). These were named together with frequencies and percentages. The data was supported with direct quotes from the interviewees’ comments and common themes were identified in order to express the views of the participants. Attempts were made to name themes and group themes so that they reflected the text as a whole, while interpretation was kept to a minimal. 4.11 Self-Reflexivity Reflection in relation to a particular research area assists the researcher to explore, learn and understand what is being brought forward by the researcher and how influential the researcher was. An interest in a particular field is shaped by the researcher’s identity and experiences. Thus, the researcher experiences emotions and subjectivity which affects the researcher’s understanding of the research process and what occurred during this time. The researcher’s qualification as a Psychometrist and a reading therapist has evoked a particular research interest. The researcher’s professional experiences have led to work conducted with particular groups. Since the topic is not out of the field of the researcher, it was important to reflect what was easy and difficult during the research process. The Neale Analysis of Reading Ability assessment tool is being used extensively and may have led to personal interest in the current research study, resulting in some bias both for the quantitative and qualitative phases. For these reasons, self-reflexivity was important in the current study. Self-reflexivity is defined as the ability to express oneself in terms of the past and the future, and includes looking back at what has been done and how it was done. Contributions are made to produce knowledge that assists in understanding and gaining knowledge of what the research has produced and how. Thus, the researcher is critically conscious of, for example, his or her own gender, race, ethnicity, nationality, and interests that impact on all 49 steps in the research process. The product of research is affected by the personnel and research process (Pillow, 2003). The researcher works in the field of reading and assessments and therefore understands the NARA II assessment tool since it is being used extensively in the researcher’s practice. This affects how the assessments and interviews were conducted for the research study. The researcher’s knowledge of the assessment tool made it easier to collect the quantitative data in terms of administration, scoring and interpretation. The researcher assessed each learner personally, which may have caused frustrations for the researcher in terms of the number of assessments conducted, and the process being repeated. It was time-consuming and tiring and having to work with four schools which worked slightly differently forced the researcher to be calm throughout and handle the frustrations when things went wrong or not as planned, when changes had to be made, for example, in the school’s schedule and timetable, or when learners were absent. In addition, the researcher may have had biases towards the groups used in the study. For example, bias towards assuming that learners from certain schools would perform better or assuming that certain groups would do better than others. The assumption that females read better and that males have poorer reading since they are lazy to read. Perhaps certain population groups read better than other population groups. With regards to the qualitative data, the researcher was curious in knowing how others might assess reading, especially those using the NARA and interacting with people in a similar field who may be experiencing similar difficulties to the researcher. The researcher may have probed some answers to hear what was not emerging from the interviewees’ responses. The interview process was not familiar to the researcher since the researcher has not conducted interviews before. The interviews were face-to-face, which was time- consuming in preparation, to travel, conducting and transcribing. 4.12 Conclusion This chapter has presented the methods utilised in this study. First, the aims and rationale were presented, followed by the research questions. Details of the participants and sampling were then presented followed by a description on the NARA II instrument. The procedure, research design, ethical considerations and a brief discussion of the data analysis in terms of the quantitative and qualitative techniques used in this study followed. The chapter concluded with a discussion on self-reflexivity. The results of the analyses are presented in Chapter 5. 50 CHAPTER 5: RESULTS 5.1 Introduction This chapter presents the results obtained in this study. Descriptive statistics for the NARA II are presented first. Following this are the results pertaining to internal consistency reliability, face validity, content validity, concurrent validity, and bias. The chapter concludes with the results found for content analysis, which were used to report on the qualitative data so as to complement the quantitative findings. 5.2 Descriptive Statistics Table 5.1 presents the means, standard deviations, minimum and maximum values, and skewness coefficients for the NARA II. Maximum possible scores for Accuracy, Comprehension and Rate are 99, 44 and 145. Accuracy is assessed by recording the number of errors the learners made. Comprehension is the number of correct responses to questions related to the reading passage. Rate is the speed at which learners read each passage. Means, standard deviations, minimum and maximum values and skewness coefficients are also shown for the mean difference between chronological age (CA) and reading age (RA) for Accuracy, Comprehension and Rate. Table 5.1 Descriptive statistics for the NARA II Variable Mean SD Minimum Maximum Skewness Accuracy Comprehension Rate Difference CA and RA Accuracy Difference CA and RA Comprehension Difference CA and RA Rate 48.41 17.74 64.27 13.72 16.27 6.24 18.61 7.75 17.84 22.08 20.80 22.42 12 5 21 -38 -39 -44 92 39 127 61 59 61 .53 .49 .46 -.56 -.47 -.27 Table 5.2 represents the means, standard deviations, minimum and maximum values, and skewness coefficients for each level of the NARA II. It is evident that across the levels the mean for Accuracy lies between 6 and 16, between 2 and 6 for Comprehension and between 25 and 135 for Rate. It is further evident that the means for Accuracy and Comprehension decrease as the level increases and Rate increases. This is because as the 51 level of passages increases the sample pool decreases, as fewer children were able to reach the higher levels of the NARA II. Table 5.2 Descriptive statistics for the NARA II levels Variable Mean SD Minimum Maximum Skewness n Level one Accuracy Comprehension Rate Level two Accuracy Comprehension Rate Level three Accuracy Comprehension Rate Level four Accuracy Comprehension Rate Level five Accuracy Comprehension Rate Level six Accuracy Comprehension Rate 15.69 3.92 25.33 14.23 5.75 35.95 10.50 4.02 66.92 8.01 4.06 104.31 6.53 3.83 116.09 9.00 2.65 135 .88 .39 13.34 2.65 1.60 22.12 3.97 2.03 29.45 4.48 1.69 35.76 4.07 1.66 32.95 4.67 1.55 30.38 11 0 11 1 1 17 0 0 27 0 0 43 0 0 59 0 0 79 16 4 58 16 8 164 16 8 179 16 8 183 14 7 219 16 6 204 -3.63 -7.65 1.19 -2.97 -.51 3.28 -.90 .00 1.19 -.11 -.34 .48 -.03 -.14 .85 -.38 .01 .33 144 144 144 144 144 144 125 125 125 83 83 83 58 58 58 34 34 34 Table 5.3 presents the means, standard deviation, minimum and maximum values and skewness coefficients for the types of errors found in the NARA II as well as the types of errors at each level. Mispronunciation and Substitution errors seemed to be the most common and were particularly evident at levels three to six. Table 5.3 shows a breakdown of this. In addition, Reversal errors seemed to be the least common and were particularly evident at level five. Addition errors were also found to be less common. This is likely since these errors are relatively rare and generally occur among learners with reading difficulties. 52 Table 5.3 Descriptive statistics for the NARA II errors Variable Mean SD Minimum Maximum Skewness n Overall errors in test Mispronunciations in test Substitutions in test Refusals in test Additions in test Omissions in test Reversals in test Overall errors Level one Level two Level three Level four Level five Level six Mispronunciations Level one Level two Level three Level four Level five Level six Substitutions Level one Level two Level three Level four Level five Level six Refusals Level one Level two Level three Level four Level five Level six Additions Level one Level two Level three Level four Level five Level six Omissions Level one Level two Level three Level four Level five Level six Reversals Level one Level two Level three Level four Level five Level six 17.84 8.90 5.73 2.15 .51 .55 .01 .23 1.77 6.43 7.99 9.19 11.00 .02 .51 2.04 4.43 6.10 6.47 .19 .83 2.47 2.25 1.78 2.32 .02 .23 .69 1.16 .81 1.21 0 .10 .14 .10 .21 .35 0 .09 .17 .05 .26 .65 0 0 0 0 .03 0 8.76 6.24 2.91 2.33 1.06 1.05 .08 .62 2.66 3.97 4.48 3.89 4.68 .14 .93 2.05 3.27 2.92 3.97 .49 1.33 1.79 1.77 1.62 1.87 .17 .91 1.39 1.19 1.19 1.37 0 .33 .47 .36 .49 .69 0 .31 .44 .23 .48 1.04 0 0 0 0 .18 0 3 0 0 0 0 0 0 0 0 0 0 3 4 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 44 30 14 15 7 5 1 5 15 16 16 16 20 1 5 9 12 15 18 3 8 8 6 7 8 2 6 7 5 4 6 0 2 3 2 2 3 0 2 2 1 2 4 0 0 0 0 1 0 .60 .80 .51 1.89 3.54 2.31 12.00 4.21 2.98 .58 .11 .21 .39 6.78 2.28 1.06 .65 .70 1.26 3.00 2.68 .82 .63 1.19 1.18 9.61 4.85 2.58 1.27 1.41 1.50 0 3.20 3.77 3.73 2.38 2.32 0 3.57 2.65 4.30 1.62 2.15 0 0 0 0 5.24 0 144 144 144 144 144 144 144 144 144 125 83 58 34 144 144 125 83 58 34 144 144 125 83 58 34 144 144 125 83 58 34 144 144 125 83 58 34 144 144 125 83 58 34 144 144 125 83 58 34 53 5.3 Internal consistency reliability Internal consistency reliability was examined for the NARA II in the form of Cronbach’s Alpha (CA). The results for each question on the Comprehension domain of the NARA II are presented in Table 5.4. It is evident that internal consistency was generally good to adequate, with the exception of level six, which is the hardest level, completed by a small pool of learners. Table 5.4 Internal consistency reliability coefficients for the comprehension questions in the NARA II NARA II α Level one comprehension questions Level two comprehension questions Level three comprehension questions Level four comprehension questions Level five comprehension questions Level six comprehension questions .69 .59 .72 .50 .53 .32 5.4 Validity Results for face validity, content validity and concurrent validity are presented in this section. Frequencies for face validity and content validity are presented followed by correlations between reading age for Accuracy, Comprehension and Rate and overall June aggregate, English aggregate and Mathematics aggregate. 5.4.1 Face Validity Frequencies were obtained from the second question of the semi-structured interview where professionals who participated were asked, “Please have a look at some stories from the Neale Analysis of Reading Ability. Do you feel that this assessment tool is appropriate and suitable for the South African context and indicate what you think of it as a measure?” Of the respondents, seven agreed that the NARA II appeared to be assessing reading ability. Respondent 3 stated: “I think the fundamental design, the fact that it test accuracy and comprehension is lovely and that it tells it in stories and not single words”. One respondent felt that it did not appear to assess reading ability since she felt that the assessment was too 54 difficult for children this age. Respondent 7 stated: “…I am just surprised that the entry level is so difficult… it doesn’t cater for the more basic. I just think it is not fair on the kids that might have the skills at basic three or four sound words. So I think that this would show that they are poorer readers than they actually are.” 5.4.2 Content Validity In an attempt to comment on the content validity of the NARA II, this study conducted frequencies for the word list and story titles that professionals felt were inappropriate for the South African context. These are presented in Tables 5.5 and 5.6 respectively. The list of words was selected from the NARA II passages. From the list of 82 words that were selected, a total of 70 in the NARA II were identified as problematic. These are found in Table 5.5. Of these words, 42 were identified by half or more of the participants. Twenty eight of the words were identified by less than half of the participants as problematic with a total of 12 words being identified as appropriate for the South African context. It is evident in Table 5.5 that all interviewees (n=8) indicated that the words diving belt, metal weights, dinghy and fledglings were inappropriate; seven participants indicated that the words underground room, skipper, clutching, feats, cub-rearing, surmount, hounds, haunts and abodes were inappropriate; six participants indicated that the words tree-house, tea- time, sheltered, buckled, air-hose, specimens, cunning, expeditions, uncanny, pursuers and inborn behaviour were inappropriate; five participants indicated that the words space-ships, temple, jewels, palace, launch, crayfish, retreated, welfare and alighting were inappropriate; four participants indicated that the words road safety, desert travellers, amazement, tangling, underwater world, rival, territory, swallows (type of bird) and storks were inappropriate; three participants indicated that the words bed-time, diver, mysterious, suspicious, extraordinary, bird-watchers, migrate and offspring were inappropriate; two participants indicated that the words nest, imagined, escape, assistance, creature, fox and mine-shaft were inappropriate; and one participant indicated that the words toys, play, no-one, hopped, crashed, television, camera, secret, buried, rocky, enemy, escaped and ancestors were inappropriate. Most of the problematic words came from the level four (a story about a girl who went diving) and level five (the life and survival skills of a fox) passages. 55 Table 5.5 Content Validity frequencies for the NARA II word list Word List Frequency % Word List Frequency % diving belt metal weights dinghy fledglings underground room skipper clutching feats cub-rearing surmount hounds haunts abodes tree-house tea-time sheltered buckled air-hose specimens cunning expeditions uncanny pursuers inborn behaviour space-ships temple jewels palace launch crayfish retreated welfare alighting road safety desert travellers 8 8 8 8 7 7 7 7 7 7 7 7 7 6 6 6 6 6 6 6 6 6 6 6 5 5 5 5 5 5 5 5 5 4 4 100 100 100 100 87.5 87.5 87.5 87.5 87.5 87.5 87.5 87.5 87.5 75 75 75 75 75 75 75 75 75 75 75 62.5 62.5 62.5 62.5 62.5 62.5 62.5 62.5 62.5 50 50 amazement tangling underwater world rival territory swallows (type of bird) storks bed-time diver mysterious suspicious extraordinary bird-watchers migrate offspring nest imagined escape assistance creature fox mine-shaft toys play no-one hopped crashed television camera secret buried rocky enemy escaped ancestors 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 50 50 50 50 50 50 50 37.5 37.5 37.5 37.5 37.5 37.5 37.5 37.5 25 25 25 25 25 25 25 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 56 In terms of the stories from the NARA II, a total of seven out of eight of the passages were identified as problematic for the South African context with four of the passages identified as problematic by half or more of the participants. Three of the stories were identified by less than half of the participants as problematic and one story being identified as being appropriate for the South African context. From Table 5.6 it is evident that, in terms of inappropriate stories from the NARA II for the South African context, six participants indicated that Practice passage two (a story about a child and his/her friend who made a tree-house and played in it) was inappropriate; five participants indicated that the level three (a story about a boy who got trapped in an old temple which had an underground room where he found jewels); and level six (a story about how birds migrate) passages were inappropriate. Four participants indicated that the level four (a story about a girl who went diving) passage was inappropriate; two participants indicated that the level two (a story about a road safety lesson that was filmed) passage was inappropriate; one participant indicated that Practice passage one (a story about a child who has toys in a box) and the level five (the life and survival skills of a fox) passage were inappropriate; with all participants indicating that the level one passage was appropriate for the South African context. Practice passage two (a story about a child and his/her friend who made a tree-house and played in it) was problematic in that South African children do not play with space-ships, children do not make tree-houses and the term ‘tea-time’ is a UK concept. The level four (a story about a girl who went diving) passage was inappropriate in that the vocabulary used in this story such as skipper, tangling, and dinghy are not used commonly in South Africa. The level two (a story about a road safety lesson that was filmed) passage was inappropriate since the concept of road safety may not be relevant. With Practice passage one (a story about a child who has toys in a box) there are South African children who do not have toys at home. The level five (the life and survival skills of a fox) passage was inappropriate in that perhaps the animal discussed in the story should be changed to another animal such as a warthog. 57 Table 5.6 Content Validity frequencies for the NARA II stories Levels Story Titles Frequency % Practice passage 2 Level three Level six Level four Level two Practice Passage 1 Level five Level one Tree-house Ali Migration Jan Road Safety Toys The Fox Bird 6 5 5 4 2 1 1 0 75 62.5 62.5 50 25 12.5 12.5 0 5.4.3 Concurrent Validity Concurrent validity was explored through correlations to determine if there were any significant relationships between the Accuracy, Comprehension and Rate scores and the learners’ school results (overall June aggregate, English aggregate and Mathematics aggregate). It is evident from Tables 5.7 that strong positive correlations were found between the variables Accuracy and Overall June Aggregate, Accuracy and English Aggregate and Accuracy and Mathematics Aggregate. In terms of effect size, 44% of variation in the Overall June Aggregate can be explained by Accuracy while 34% of variation in the English Aggregate can be explained by Accuracy. 30% of variation in the Mathematics Aggregate can be explained by Accuracy. Strong positive correlations were also found for Comprehension and Overall June Aggregate, Comprehension and English Aggregate and Comprehension and Mathematics Aggregate. In terms of effect sizes, 44% of variation in the Overall June Aggregate can be explained by Comprehension, 28% of variation in the English Aggregate can be explained by Comprehension with 35% of variation in the Mathematics Aggregate can be explained by Comprehension. In addition, strong positive correlations were found between Rate and Overall June Aggregate, Rate and English Aggregate and Rate and Mathematics Aggregate. In terms of effect sizes, 34% of variation in the Overall June Aggregate can be explained by 58 Rate, 22% of variation in the English Aggregate can be explained by Rate and 12% of variation in the Mathematics Aggregate can be explained by Rate. Table 5.7 Correlations between reading age and June results Variable Overall June Aggregate r r2 p English Aggregate r r2 p Mathematics Aggregate r r2 p Accuracy Comprehension Rate .66 .66 .58 .44 .44 .34 .000* .000* .000* .58 .53 .47 .34 .28 .22 .000* .000* .000* .55 .59 .34 .30 .35 .12 .000* .000* .000* *p < 0.05 5.5 Bias Results for bias are presented in this section. This was carried out by comparing the errors in the NARA II against each demographic (gender, home language, population group, educational level of parents and the school that the learners attended) to determine if there were any significant differences. These are presented in Tables 5.8 to 5.12. T-tests were conducted for the NARA II errors and gender and the NARA II errors and home language for those variables that were normally distributed. For those variables that were not normally distributed and where homogeneity of variance was not present, Mann-Whitney U tests were conducted. Analysis of variance (ANOVA) was conducted for the NARA II errors and population group, educational level of parents and schools learners attended for those variables that were normally distributed. For those variables that were not normally distributed Kruskal Wallis tests were conducted. Those variables that were not normally distributed are presented in the same table but are italicised. In addition, the difference in chronological age and reading age for Accuracy, Comprehension and Rate were compared to each demographic (gender, home language, population group, educational level of parents and learner school) to determine if there were any significant differences. These are presented in Tables 5.13 to 5.17. 59 5.5.1 Gender and errors on the NARA II A comparison of the errors between males and females can be found in Table 5.8. From this it is evident that there is a significant difference between Additions (U = -2.05; p = .041) and Omissions (U = -3.24; p = .001) errors with males making more of each type of error than females. Table 5.8 Independent t-test comparing NARA II errors by gender Male Female Variable df t p<[t] Mean/Mean rank SD Mean/Mean rank SD Overall errors in test Mispronunciations Substitutions Refusals Additions Omissions Reversals 142 142 142 142 142 142 142 1.14 -.064 .56 -1.14 -2.05 -3.24 -.99 .258 .949 .578 .255 .041* .001** .324 18.66 8.86 5.86 76.33 78.22 81.47 73.00 9.20 6.28 3.01 2.61 1.21 1.28 .12 17.00 8.93 5.86 68.56 66.62 63.28 72.00 8.27 6.26 2.83 1.99 .85 .63 .00 *p < 0.05, **p < 0.01 5.5.2 Home language and errors on the NARA II A comparison of the errors between first and second language learners are found in Table 5.9, from which it is evident that there is a significant difference between home language and the Overall errors in test, Mispronunciations, Substitutions, Additions and Omissions errors. Overall, second language learners made significantly more errors than first language learners. Effect sizes were calculated for Overall errors in test, Mispronunciations and Substitution errors all of which suggest moderate effect sizes. For Additions and Omissions errors, second language learners had higher mean ranks. 60 Table 5.9 Independent t-test comparing NARA II errors by home language English first language English second language Variable df t p<[t] Mean/ Mean rank SD Mean/ Mean rank SD Cohen’s d Overall errors in test Mis-pronunciations Substitutions Refusals Additions Omissions Reversals 142 142 142 142 142 142 142 3.69 2.15 3.53 -.80 -2.45 -3.68 -1.09 .000** .029* .001** .426 .014* .000** .277 15.03 7.70 4.83 69.60 65.00 61.30 73.10 7.08 5.09 2.59 2.06 .81 .57 .12 20.22 9.91 6.49 75.00 78.90 82.00 72.00 9.36 6.95 2.97 2.54 1.21 1.26 .00 .6 .4 .6 *p < 0.05, **p < 0.01 5.5.3 Population group and errors on the NARA II A comparison of the errors across population groups is found in Table 5.10. From this it is evident that there are significant differences between population group and Overall errors in test, Substitutions, Additions and Omissions. Furthermore, effect sizes were calculated for Overall errors in test and Substitutions errors which revealed a medium effect size for the Overall errors in test and a small effect size for Substitutions errors. The Indian group appeared to make the most Overall errors in the test, including in Substitutions. However, this may be because most of the Indian population group were first language learners who had undergone more levels on the NARA II than the African group, who completed fewer levels on the NARA II. 61 Table 5.10 ANOVA results for population group on the NARA II errors Variable df F/2 p Mean scores/ranks African Indian Other eta² Overall errors in test Mispronunciations Substitutions Refusals Additions Omissions Reversals 2,141 2,141 2,141 2 2 2 2 6.98 2.99 3.55 .52 11.12 10.54 .92 .001** .054 .031* .772 .004** .005** .631 15.33 20.67 20.00 7.71 10.31 9.55 5.12 6.41 6.27 1.96 2.36 2.36 .27 .72 1.00 .27 .86 .82 .01 0 0 .09 .05 *p < 0.05, **p < 0.01 5.5.4 Educational level of parents and errors on the NARA II A comparison of the errors and educational level of parents is shown in Table 5.11. From this it is evident that there are no significant differences between the errors and Socio- economic status as extrapolated by the educational level of parents. Table 5.11 ANOVA results for educational level of parents on the NARA II errors Variable df F/2 p Mean scores/ranks Less than matric matric Post matric Overall errors in test Mispronunciations Substitutions Refusals Additions Omissions Reversals 2,141 2,141 2,141 2 2 2 2 2.29 1.28 1.61 2.10 .03 .01 .39 .105 .288 .203 .147 .862 .925 .533 14.74 7.84 4.63 1.53 .37 .37 0 17.02 8.10 5.80 2.37 .37 .37 .02 19.14 9.67 5.96 2.17 .63 .71 14.74 62 5.5.5 Learner school and errors on the NARA II A comparison of the errors between the learners’ schools is found in Table 5.12. From this it is evident that there are significant differences between learner school and Overall errors in test and Mispronunciations errors. Furthermore, effect sizes were calculated for Overall and Mispronunciations errors which revealed a medium effect size for both. For the Overall errors, School 3 had a higher score compared to the other schools and for Mispronunciations errors School 4 had the higher score. Both School 3 and 4 were mostly first language learners who underwent more levels on the NARA II compared to learners from School 1 and School 2 who did not tend to progress that far on the NARA II. Therefore it shows that these learners made more errors but this is because they completed more of the test. Table 5.12 ANOVA results for learner school on the NARA II errors Variable df F/2 p Mean scores/ranks 1 2 3 4 eta² Overall errors in test Mispronunciations Substitutions Refusals Additions Omissions Reversals 3,140 3,140 3,140 3 3 3 3 5.63 4.09 2.32 5.34 3.49 6.98 6.65 .001** .008** .078 .148 .322 .073 .302 13.95 16.81 20.09 21.14 6.68 7.90 10.25 11.11 4.78 5.74 6.27 6.25 1.59 2.68 2.14 2.43 .46 .26 .70 .54 .44 .19 .73 .82 0 .03 0 0 .11 .08 *p < 0.05, **p < 0.01 5.5.6 Difference in chronological age and reading age by gender A comparison between gender and the difference in chronological age and reading age on Accuracy, Comprehension and Rate is presented in Table 5.13. This was done so as to determine if there were any significant differences between males and females. It is evident that there were no significant differences between gender and the difference in chronological age and reading age for Accuracy, Comprehension and Rate scores. 63 Table 5.13 Independent t-tests results for the difference in chronological age and reading age by gender Male Female Variable df t p<[t] Mean SD Mean SD Accuracy Comprehension Rate 142 142 142 1.10 .08 1.43 .273 .935 .156 15.71 16.41 8.86 21.63 22.14 21.51 11.66 16.13 3.55 22.49 19.45 23.16 5.5.7 Difference in chronological age and reading age by home language A comparison between home language and the difference in chronological age and reading age for Accuracy, Comprehension and Rate is presented in Table 5.14. It is evident that there were significant differences between home language and the difference in chronological age and reading age for Accuracy, Comprehension and Rate scores respectively. More specifically, second language learners showed a significantly higher mean score than first language learners for Accuracy, Comprehension and Rate. This shows that there is a vast discrepancy in second language learners. The gap between the second language learners’ chronological age and reading age on Accuracy, Comprehension and Rate is much larger, indicating that they are reading poorly and that their reading age is much lower than their chronological age. With first language learners, the gap between their chronological age and reading age on Accuracy, Comprehension and Rate is much smaller or their reading age is above their chronological age indicating better readers. Therefore a negative means as in Rate indicates that they are reading better than their chronological age. Furthermore, effect sizes were calculated for Accuracy, Comprehension and Rate, with strong effect sizes. 64 Table 5.14 Independent t-tests results for the difference in chronological age and reading age by home language English 1st language English 2nd language Variable df t p<[t] Mean SD Mean SD Cohen’s d Accuracy Comprehension Rate 141.07 138 142 -5.08 -5.75 -6.41 .000** .000** .000** 5.92 8.18 -3.49 22.59 21.49 21.45 22.92 25.83 17.74 17.58 15.24 17.69 -.85 -.96 -1.07 *p < 0.05, **p < 0.01 5.5.8 Difference in chronological age and reading age by population group A comparison between population group and the difference in chronological age and reading age on Accuracy, Comprehension and Rate is presented in Table 5.15. It is evident that there were significant differences between population group and the difference in chronological age and reading age on Accuracy, Comprehension and Rate. Table 5.15 ANOVA results for the difference in chronological age and reading age by population group Variable df F/2 p Mean scores African Indian Other Accuracy Comprehension Rate 2, 141 2, 141 2, 141 22.10 17.22 33.80 .000** .000** .000** 22.73 24.68 17.44 .64 5.43 -9.10 21.18 16.09 10.82 *p < 0.05, **p < 0.01 Post-hoc analyses using Tukey’s HSD as presented in Table 5.16 indicated that the Indian population group was significantly different from both the African and the Other grouping at the 0.01 level of significance on Accuracy. In both cases the Indian group obtained lower scores. This indicates that the Indian group are reading well above their chronological age and making fewer errors when reading. Furthermore, effect sizes were calculated which 65 revealed large effect sizes. There was no significant difference between the African and Other grouping on Accuracy. With regards to Comprehension, the African population was significantly different from the Indian population at the 0.01 level of significance, with the African population obtaining a higher score. This indicates that the African group provided more incorrect responses to comprehension questions. Furthermore, effect sizes were calculated which revealed large effect sizes. There was no significant difference between the African and Other grouping and the Indian and Other grouping on Comprehension. The Indian population group was significantly different from both the African and the Other grouping at the 0.01 level of significance on Rate. In both cases the Indian group obtained a lower score indicating that their rate for reading is faster and their reading age for Rate is larger than their chronological age. Furthermore, effect sizes were calculated which revealed large effect sizes. There was no significant difference between the African and Other grouping on Rate. Table 5.16 Post-hoc results for the difference in chronological age and reading age by population group Variable Population Group Population Group Mean Difference SE p Cohen’s d Accuracy African Other 1.55 6.26 .967 African Indian 22.10 3.39 .000** 1.13 Other Indian 20.54 6.38 .005** 1.05 Comprehension African Other 8.59 6.06 .335 African Indian 19.25 3.28 .000** 1.02 Other Indian 10.66 6.17 .199 Rate African Other 6.62 5.99 .513 African Indian 26.54 3.25 .000** 1.42 Other Indian 19.92 6.11 .004** 1.07 *p < 0.05, **p < 0.01 66 5.5.9 Difference in chronological age and reading age by educational level of parents The ANOVA results obtained for educational level of parents and the difference in chronological age and reading age on Accuracy, Comprehension and Rate are presented in Table 5.17. This was done to determine whether the NARA II is biased towards certain socio-economic groups i.e. whether certain groups perform significantly better than others. Significant differences were found between educational level of parents and the difference in chronological age and reading age on Accuracy, Comprehension and Rate scores. Post-hoc analyses using Tukey’s HSD as presented in Table 5.18 indicated that the Post- matric group was significantly different from the Less than matric at the 0.05 level of significance on Accuracy, Comprehension and Rate and significantly different from the matric group at the 0.01 level of significance. In all cases the Post-matric group obtained lower scores. This indicates that learners who have parents with a Post-matric, read better than those with parents who have a Less than matric or matric. Furthermore, effect sizes were calculated which revealed large effect sizes. There was no significant difference between the matric and Less than matric grouping on Accuracy, Comprehension and Rate. Table 5.17 ANOVA results for the difference in chronological age and reading age by educational level of parents Variable Df F/2 p Mean scores Less than matric matric Post matric Accuracy Comprehension Rate 2,141 2, 141 2, 141 8.46 8.48 7.83 .000** .000** .000** 20.63 24.00 13.21 21.59 23.22 13.90 6.91 9.86 -.43 * p < 0.05, ** p < 0.01 67 Table 5.18 Post-hoc results for the difference in chronological age and reading age by educational level of parents Variable Socio- Economic Status Socio- Economic Status Mean Difference SE p Cohen’s d Accuracy Less than matric matric .96 5.68 .984 Less than matric Post-matric 13.72 5.39 .032* .65 Matric Post-matric 14.68 3.85 .001** .69 Comprehension Less than matric matric .78 5.35 .988 Less than matric Post-matric 14.15 5.07 .017* .71 Matric Post-matric 13.37 3.62 .001** .67 Rate Less than matric matric .69 5.79 .992 Less than matric Post-matric 13.65 5.49 .038* .63 Matric Post-matric 14.33 3.92 .001** .66 *p < 0.05, **p < 0.01 5.5.10 Difference in chronological age and reading age by learner school The ANOVA results obtained for learner school and the difference in chronological age and reading age on Accuracy, Comprehension and Rate are presented in Table 5.19. Significant differences were found between learner school and the difference in chronological age and reading age on Accuracy, Comprehension and Rate. Post-hoc analyses using Tukey’s HSD as presented in Table 5.20 indicated that School 1 was significantly different from Schools 3 and 4 at the 0.01 level of significance on Accuracy, Comprehension and Rate. In all cases School 1 obtained a lower score. Furthermore, effect sizes were calculated which revealed large effect sizes. There was no significant difference between Schools 3 and 4 on Accuracy, Comprehension and Rate. School 2 was significantly different from Schools 3 and 4 at the 0.01 level of significance on Accuracy, Comprehension and Rate. In both cases School 2 obtained a lower score on Accuracy, Comprehension and Rate. Furthermore, effect sizes were calculated which revealed large effect sizes. 68 Table 5.19 ANOVA results for the difference in chronological age and reading age by learner school Variable df F/2 p Mean scores 1 2 3 4 Accuracy Comprehension Rate 3, 140 3, 140 3, 140 12.90 11.92 24.50 .000** .000** .000** 24.46 26.10 16.12 23.26 24.26 22.23 3.48 9.82 -.9.98 3.50 3.18 -.43 * p < 0.05, ** p < 0.01 Table 5.20 Post-hoc results for the difference in chronological age and reading age by learner school Variable Learner School Learner School Mean Difference SE p Cohen’s d Accuracy School 1 School 2 1.21 4.70 .994 School 1 School 3 20.99 4.29 .000** 1.06 School 1 School 4 20.96 4.84 .000** 1.06 School 2 School 3 19.78 4.63 .000** 1.00 School 2 School 4 19.76 5.15 .001** 1.00 School 3 School 4 .02 4.77 1.00 Comprehension School 1 School 2 1.84 4.46 .976 School 1 School 3 16.28 4.07 .001** .87 School 1 School 4 22.92 4.60 .000** 1.22 School 2 School 3 14.44 4.40 .007** .77 School 2 School 4 21.08 4.89 .000** 1.12 School 3 School 4 6.64 4.53 .461 Rate School 1 School 2 6.10 4.37 .503 School 1 School 3 26.10 3.98 .000** 1.42 School 1 School 4 16.55 4.50 .002** .90 School 2 School 3 32.20 4.30 .000** 1.76 School 2 School 4 22.65 4.78 .000** 1.23 School 3 School 4 9.55 4.44 .142 *p<0.05, **p<0.01 69 5.6 Thematic Content Analysis The next section covers the qualitative data which is based on the semi-structured interviews that were conducted with professionals so as to complement the quantitative findings. The open-ended questions from the semi-structured interviews were analysed using an inductive approach at the semantic level (Braun & Clarke, 2006). The six steps recommended by Braun and Clarke (2006) as discussed3 guided the analysis for the interviews for this study. The following themes and sub-themes were identified. 5.6.1 Suitability of the NARA II to the South African context The first theme that emerged from the interviews concerned the suitability of the NARA II in the South African context. Of the eight professionals, Respondents 1, 2 and 3 use the NARA in their practice to assess reading in South African children. Respondents felt that the NARA II was suitable in terms of the design where it assesses Accuracy, Comprehension and Rate. They also liked that reading was assessed through stories and not single word lists. Respondent 2 stated: “What I like about the NARA is the fact that you can get the reading comprehension, the reading accuracy and the reading rate of it.” Respondents 7 and 8, who are teachers, indicated that the NARA II seemed to be similar to what was done in schools during a reading activity, with Respondent 7 stating: “I actually prefer that, because we do it in class as well.” However, it was felt by Respondent 4 that learners from less affluent backgrounds might struggle with the NARA II. Respondent 6 felt that perhaps the entry level story was too difficult. This could indicate a learner being a poorer reader if he or she had knowledge of three- and four-sound words, such as the three sounds found in the word cat and the four sounds in the word milk, for example. Within Suitability of the NARA II to the South African context the sub-themes Appropriateness of words and stories from the NARA II, norms and results emerged. 5.6.1.1 Appropriateness of words and stories from the NARA II It was felt that various words used in the NARA II stories did not suit the South African population and context. For example, Respondent 3 stated: “One of the stories I end up using quite often has the word parcel in it and I find very few children know what a parcel is”. Respondent 4 mentioned that the names of people used in the stories such as Ali and Jan might not be familiar to some of the South African population. Their context might be too difficult for learners who speak English as an additional language such as stories about the 3 Refer to Chapter 4 page 48 70 sea as learners may not have experienced being at the sea. From the eight professionals, Respondents 6 and 7 felt that the stories seemed to reflect an appropriate context that their learners could relate to, with Respondent 6 saying: “I don’t think things have to be completely Africanized.” The respondent felt that many story books refer to concepts such as a white Christmas and people going down sleds which South Africans are aware of even though they don’t have a white Christmas or sleds in South Africa. Respondents made suggestions about possible themes/story titles that would be suitable to the South African context, as presented in Table 5.21. Table 5.21 Suggestions for possible themes suitable to the South African context Suggested themes Number of suggestions for the theme South Africa Sport Animals Transport Play School Food/Cooking Crime 5 4 3 3 3 3 3 2 More specifically, with regard to the possible theme of South Africa, suggestions were made in terms of a story related to going to the sea, Cape Town and Nelson Mandela. For Sport there were suggestions of a story on soccer or cricket. National parks and wild animals were specifically mentioned for stories about Animals. Related to Transport there was a preference for the story to be related to travelling to school by taxi or car and around safety. For Play, suggestions were made about play that occurs outdoors and playing with South African based toys. For a story about School the areas of learning, bullying, after-care, or friendship were suggested. Other themes/story titles included Living circumstances/Responsibility/Chores, Superheroes (fantasy, fable), Farming, Computer Games, Seasons/Weather, Body and growing up and Life lessons. These themes would assist in developing a tool or adapting the NARA II to suit the South African context. 71 5.6.1.2 Norms Within the broad theme of Suitability of the NARA II to the South African context, another sub-theme that emerged was norms. The respondents felt that the norms provided with the NARA II were not useful since they were too high for the South African population. Respondent 3 said: “… the norms I don’t find useful. I just worry whether we actually coming out with any sort of useful information. I don’t often get children, even the bright children, who score above the norm, so almost everybody is below the chronological age on those norms, so they most probably are too high.” This is supported by Respondent 1’s comments: “They [United Kingdom children] start reading at five.” Related to the discussion on norms, Respondent 1 indicated that those of the original NARA, which she was still using, seemed to be accurate for the South African population but she found the NARA II norms to be too high. Respondent 2 explicitly said: “I think if we could norm it here I think that would be a much better option.” It was evident that there is a need for a comprehensive reading assessment that has norms for South Africa. With regards to the norms, it was suggested that there could be separate norms for first and second language learners, taking into account that learners come from both urban and rural areas. Respondent 1 commented: “…my initial response would be to have two different types… and there are no reading tools for second language learners, I have to use the first language and they not, they just not helping, they not working for me and there are no tools in their home language.” Respondent 3 suggesting: “Should be normed appropriately across, especially across urban and rural… Maybe you would need different norms for children who are taking English as a first language and children who are taking English as a second language.” 5.6.1.3 Reporting results A further sub-theme that emerged from Suitability of the NARA II to the South African context is related to how the results of the NARA are reported. Respondents 2 and 3 indicated that when they used the NARA in their practice they reported on the results qualitatively and evaluated which errors were made in order to plan intervention, i.e., they do not only provide test scores or do not provide test scores at all. Respondent 2 mentioned: “…I will use it qualitatively, and just look at where the errors were and get an idea of what their reading rate was as well”. 72 5.6.2 Similarities between the NARA II and reading tasks in school Respondents were asked to compare the NARA II to reading tasks in school so as to have an idea of what the similarities were between the two. Respondents 1 and 5 indicated that they were not sure what the similarities were. Respondent 1 said: “I don’t know because I don’t work with children in schools.” The other respondents indicated that both forms evaluated reading through stories and a list of words. In Grade Four, learners are expected to know how to read and would need to use the skills learnt in foundation stages of learning, and therefore at Grade Four comprehension is taught, rather than how to read. Both tasks evaluated Comprehension, and in both tasks learners could refer to the text for answers to comprehension questions. This is supported by Respondent 2 who said: “Kids do a lot of comprehension in school…they can refer back to it as they can on the NARA as well.” 5.6.3 Differences between the NARA II and reading tasks in school In terms of the differences, the respondents mentioned that in school there was no one-on- one reading as learners were expected to read on their own and therefore learners who were struggling were often not identified. One teacher stated that there was no time for each learner to read to the teacher and therefore learners read silently and the teacher asks questions in terms of what was understood. However, in some cases those learners for whom English was an additional language struggled so stories were read to them. Respondent 6 indicated that the NARA II seemed to be a summative assessment in schools, after a range of formative exercises on reading had taken place. She added that books in schools relied on rhyme, with the last word of each sentence having the same sound, whilst rote learning and reading activities were done in isolation (list of words out of context). Respondent 4 commented on young learners (Grades One and Two), stating that books at school for younger learners had colourful pictures, would be based on target words and have much repetition. The books at school for younger children would have one picture and a single sentence on one page of the story. Respondent 4 said: “But what we do is colourful picture based like type of thing, so obviously the child must rely on the picture…the reading books that they have given at different levels target the words that they have targeted in the classroom…so I find that a lot of the teaching children to read today, a lot of repetition in the classroom of words that they have been exposed to.” Words in sentences that were not understood became words that need to be learnt for a spelling test. Since the NARA II is assessed individually, another difference as indicated by Respondent 7 is that group activities were not feasible in school while Respondent 8 indicated that learners were placed in groups for a reading activity according to their reading 73 ability. Vocabulary was introduced and explained and sentences analysed so that learners could read for meaning in context. Another difference was that learners were provided with written comprehension questions that required written comprehension answers, while in the NARA II these were read aloud by the tester and required verbal responses. 5.6.4 Comprehension skills Both teachers (Respondents 7 and 8) indicated that learners might struggle with Comprehension since although many could read they often could not do so for meaning, and therefore did not understand what they were reading. Respondent 7 said: “Comprehension here is a problem, same with the reading, it is a major problem. They don’t know how to answer a question…usually the sentence they don’t understand is taken down as spelling and vocabulary for a test” This is further supported by Respondent 8, who stated: “They can actually read it and if you give them they will actually read it but they will not understand.” 5.6.5 Assessment tools utilised for reading in South Africa The most commonly used assessment tools by the respondents were the NARA and the Burt Word reading test, which were used by three of the respondents. Two respondents make use of the Phono-graphix reading assessment, whilst other assessment tools that were mentioned but not commonly used were the Wechsler Individual Achievement Test (WIAT), the One minute reading test, Wechsler Objective Reading Dimensions (WORD), Grey Silent Reading Test (GSRT), the Edinburgh Reading Test Series, the ESSI reading and spelling test and the Schonell Reading Test. Of the assessment tools listed above, the One minute reading test and the ESSI reading and spelling tests were normed for South Africa. Using international assessment tools may provide information but the information may not be relevant to the South African context. At the same time, local practitioners have to use what is available even if these are internationally developed. Respondent 2 mentioned: ‘I kind of want to say what choice do we have if we don’t use what we have got, what is available to us, because we wouldn’t get any information at all” and Respondent 1 who said: “South Africa has nothing…there is a huge need for a South African tool.” However, international tools cannot provide an accurate score or age because children from different countries begin formally reading at different ages. However, the results can be used qualitatively. The reading standards in every country are different and international tools may not be adequate as Respondent 4 said: “[they are] normed on kids from other places and I mean obviously children are different, so are the norms.” 74 Some suggestions were made about how the assessments should be conducted, with Respondent 6 suggesting: “I think that there has to be a more of a holistic approach.” Most respondents felt that reading assessment should include a word reading list and assessment of comprehension, with Respondent 2 suggesting: “Perhaps even a combination of oral comprehension silent comprehension-written answers and verbal answers.” Accuracy and rate, and a spelling test, and it should have various forms and levels. Respondent 3 suggested that it should cover learners from Grades R to 12. Many wanted the assessment to have appropriate stories and information suitable to the South African context, with the option of the results having a qualitative component. Respondent 6 further suggested: “… making sure that you assess the skills of their knowledge out of context and in context or in isolation and in context.” In general, their responses indicated that reading assessments was rare and there was insufficient awareness on both the importance of reading and reading assessments. Respondent 4 commented: “I think before we start even with our assessments we require the awareness, a lot of the times we are assessing these kids and then parents and the teachers, they don’t even know why you doing this, so I think we must create awareness.” Another suggestion was that schools should have remedial teachers who could work with those learners identified as having a reading difficulty, since teachers did not have the time during or after class to assist them. Remedial teachers should work with learners from all grades and not just the Foundation Phase, since reading difficulties can occur with learners of any age. 5.6.6 Learner errors on reading Since the NARA II records the errors made when reading, it was important to support the results from the NARA II with the errors professionals had noticed with the learners they worked with. Of the six types of errors found on the NARA II, seven respondents mentioned Mispronunciation errors as being the most common. Four of these seven respondents (3, 4, 6 and 8) mentioned that this was due to learners not knowing the advanced phonological/orthographic code in which two or more letters make up one sound. Respondent 6 said: “They would read the word boat as /b/ /o/ /at/. They don’t realize that the oa is one picture or one unit for the o sound.” Respondent 8 stated: “They [learners] have got problems with some words maybe consonants you know joined together.” This was further explained by Respondent 6 as: “…learners not knowing enough of the sound pictures, the letters that represent the sounds.” Respondent 3 commented that this skill was 75 not explicitly taught and therefore some learners picked it up, but that second language learners struggled. Substitution and Reversal errors were mentioned by four of the respondents, with two of these (1 and 2) mentioning that Reversal errors were mainly found with learners who had a learning/reading disability. Respondent 1 said: ‘…the ones with learning difficulties are Substitutions, Additions and Reversals.” Respondents 5, 6 and 7 mentioned Omission errors. Respondent 5 said: “They do [make Omission errors], because they are not very strong with their phonics as well so it’s a lot.” Refusal and Addition errors were mentioned by one respondent. Respondent 4 added that second language learners made errors when they need to pronounce vowels. These learners also found discrimination between m/b, b/d difficult, as was pronunciation of f/v. 5.6.7 Foundations of reading Since various instructional methods may be used to teach reading it was important to gather which ones the respondents felt were important. Six of the eight (Respondents 1-6) commented on the method of teaching reading. Respondents 1, 4 and 5 believed that basic phonics should be taught first, with Respondent 6 saying that traditional phonics did not work because: “Traditional phonics programmes would teach the sounds and letters in isolation… so reading can take a whole year.” Respondent 1 said: “I think you need a combination of phonics and sight words.” There were disagreements by Respondent 2 on the letter name system (e.g., Sammy snake) and by Respondent 3, that sound families did not work (e.g., hat, cat, bat, and mat). Although they did not mention why they felt these systems did not work it was probably because phonological awareness is a skill that should be taught before, not instead of, phonics and visual skills. Respondents 1, 3 and 6 said that it was important to teach blending and segmenting with Respondent 1 saying: “…teach the child analysis and synthesis how to break up the word and how to put them back together.”, whilst for Respondents 1, 2 and 4 it was sight words with Respondent 2 saying: “I feel there needs to be a whole other method for your sight words. I feel these words need to be learned, the kids need to automatically be able to read those words.” There was no one single approach to teaching reading across the four schools sampled, but all respondents were of the opinion that a variety of skills should be taught. They also noted eye movement training, flash cards, single sound pictures, teaching the advanced code (two or more letters make up one sound) and comprehension as being important aspects of reading. 76 5.6.8 Benefits Respondents believed that reading is important for learning and schooling, children should be reading accurately at an appropriate pace, and comprehending, or there will be difficulties throughout their schooling. The respondents thus suggested that if there were no intervention to assist those struggling with reading they would be greatly disadvantaged. Therefore, reading assessments are beneficial as they can identify the skills the learner has, identify strengths and weaknesses, and gauge what progress the child is making. Respondent 1 commented: “They [reading assessments] identify all their learning problems…they give you clues when you design intervention programmes.” This is further supported by Respondent 3’s comment: “I suppose the point behind it [reading assessment] is to identify what reading strategies they [learners] using in order to try and either guide them in the right direction or provide them with the assistance that they need, so exactly for that reason of being able to manage with all the different school subjects.” Once identification occurs, intervention can be planned accordingly, and parents and teachers can be informed and work together to assist the child. Therefore, the foundation needs to be set. Respondent 5 said: “… every child has the ability to learn”, whilst Respondent 3 said: “I think with the correct technique and with the right mind frame, anybody can learn and improve their reading.” Respondent 1 noticed a difference after intervention by stating: “What I did find was that the comprehension was generally low. For all age groups and with my remedial therapy children their comprehension on second test had gone up, so I realised something that I was doing in my therapy was improving their comprehension as opposed to the children in the class, because their comprehension was also low but didn’t come up.” 5.6.9 Affordability Most respondents felt that middle income households should be able to afford a reading assessment or may have a medical aid account that would pay for it. However, those from a lower socio-economic group or attending a government school might not be able to afford it, as Respondent 1 suggested: “Yes but they would have to find somebody who is prepared to do it in a community based service.” Overall, it seems that the cost of a reading assessment is generally perceived as too high for most South Africans as Respondent 6 commented: “I think generally in South Africa for a parent to send their kids for one private session with a professional is too expensive.” And Respondent 2 saying: “I would say the prices are probably too high for the majority of South Africans.” It was therefore suggested that if teachers are trained in conducting reading assessments they should be able to assess and 77 teach at the same time, which is also-cost effective as Respondent 3 said: “And there should be people at least a couple of people at every school trained on it.” 5.7 Conclusion This chapter has reported on those results relevant to this study, which combined quantitative and qualitative data. The quantitative data produced statistical evidence regarding the applicability of the NARA II in the South African context, whilst the qualitative data obtained through semi-structured interviews with professionals produced themes and sub-themes obtained through thematic content analysis. The internal consistency reliability of the NARA II was found to be good to adequate. Face validity was adequate for the NARA II. Content validity was investigated through a selection of words from the NARA II and the story titles in order to determine which were inappropriate for the South African context. From a list of 82 words selected, 70 were identified as problematic with 42 of these suggested as being problematic by half or more of the respondents. With regards to the story titles, seven out of eight stories were identified as problematic, with four of these suggested as being problematic by half or more of the respondents. Concurrent validity was explored in order to determine if there were any significant relationships between the Accuracy, Comprehension and Rate scores on the NARA II and the learners’ school results. Strong positive correlations were found between the variables Accuracy, Comprehension and Rate and Overall June Aggregate, English Aggregate and Mathematics Aggregate. Bias was explored by comparing the errors in the NARA II against each demographic (gender, home language, population group, educational level of parents and the school that the learners attended) to determine if there were any significant differences. Some evidence for bias was found in terms of gender, home language, population group, educational level of parents and the school learners attended. However, effect sizes for these differences were small to moderate. In addition, the difference in the learners’ chronological age and reading age on Accuracy, Comprehension and Rate were compared for each demographic to determine if there were any significant differences. Some evidence of bias was found with large effect sizes for the differences on home language, population group, educational level of parents and the schools learners attended. This chapter concluded with results of the qualitative data obtained through content analysis. Nine themes were found namely Suitability of the NARA II to the South African context, Similarities between the NARA II and reading tasks in school, Differences between the 78 NARA II and reading tasks in school, Comprehension skills, Assessment tools utilised for reading in South Africa, Learner errors on reading, Foundations of reading, Benefits and Affordability. These themes and sub-themes provide details on the views of a small group of professionals regarding the current status of reading assessments in South Africa in order to conduct assessments suitable for the population and government school context and how reading assessments can be utilised. Chapter 6 discusses these results by linking them to the relevant literature. 79 CHAPTER 6: DISCUSSION 6.1 Introduction This chapter presents a discussion of the results in this research study as outlined in Chapter 5, in order to fulfil the aim of the study, namely to explore the applicability of the NARA II in the South African context from a quantitative and qualitative perspective. Firstly, descriptive statistics of the sample are discussed, followed by the descriptive statistics of the NARA II and a discussion of the internal consistency reliability of the instrument. Results associated with face validity, content validity, concurrent validity and bias are then discussed. This is followed by a discussion on the qualitative findings and an attempt to integrate a discussion of all the research findings. This chapter concludes with a discussion on the limitations of the study and recommendations for future research. 6.2 Descriptive Statistics: Demographic Information The total sample size for this study was 144 learners, adequate and appropriate for the statistical analyses conducted. There was equivalent distribution of gender while for home language there were a few more first than second language learners, but this difference was not statistically significant. There was a significant uneven distribution with regards to population group, educational level of parents and the schools learners attended. Most learners were from the African population while the majority of parents possessed a matriculation qualification. The educational level of parents in this study is similar to that found in the Progress in International Reading Literacy Study (PIRLS), where only 13% of parents had no matriculation qualification (Howie et al., 2008). In terms of the schools learners attended, most learners came from School 3 (n=44) and the fewest came from School 4 (n=28). Furthermore, the age range of learners was 8.58 years to 11.91 years, with a mean age of 9.92 years. Since this study took into consideration first and second language learners, it was important to note that the number of years for which second language learners were exposed to English was 5, which further reveals a 4.9 year gap between the second language learners’ average chronological age and average number of years exposed to English. This gap may account for some of the second language learners who begin to learn English when they start school and use different skills when learning to read than first language learners. The rate at which learning to read in English develops also differs for first and second language learners (De Sousa & Broom, 2011; Janks, 2011). In addition, learners’ June results were important for the concurrent validity, discussed later in this chapter. In terms of the demographic information it is important to note that the average June aggregate was 64.44%, 65.28% for English and 67.17% for Mathematics. 80 6.3 Descriptive Statistics: NARA II The descriptive statistics of the NARA II in the form of means, standard deviations, minimum and maximum values and skewness coefficients were examined overall for Accuracy, Comprehension and Rate and at each level presented in Tables 5.1 and 5.2. As explained in Chapter 5, the maximum possible scores for Accuracy, Comprehension and Rate are 99, 44 and 145 respectively. Furthermore, there was a larger gap between the learner’s chronological age and reading age (Comprehension) and a smaller gap between the learner’s chronological age and reading age (Rate). This indicates poorer performance in comprehension and a better reading rate. The NARA II outlines six types of errors that learners could make when reading aloud. The most common type of errors that were made by the sample were Mispronunciations and Substitutions. Mispronunciation errors indicate that the learner does not know the phonetic code or has insufficient phonemic awareness, which may be linked to a difficulty with general vocabulary knowledge (Cain, Oakhill & Bryant, 2000; Phillips, Clancy-Menchetti & Lonigan, 2008; Read America, 2005). Substitution errors indicate that learners are not decoding and therefore guess words based on the first few letters. This occurs with learners who have adequate general knowledge and vocabulary skills rather than actual reading skills. Substitution errors may indicate problems with comprehension (Read America, 2005). Learners, who struggle to decode words, may spend more time putting in an effort sounding of words with the result that they may forget what they have read already, thus decreasing their comprehension (Kirby, 2009; Oakhill et al., 2003; Verhoeven & Perfetti, 2008). The findings on Mispronunciation and Substitution errors are supported by the qualitative data in which the theme of Learner errors on reading emerged, with seven respondents mentioning Mispronunciations as a common error, due to not knowing the advanced code, and four respondents mentioning Substitutions as a common error. 6.4 Internal Consistency Reliability The internal consistency reliability coefficients for the comprehension questions at each level presented in Table 5.4 were generally good to adequate. The internal consistency coefficients were as follows: level one (α = .69), level two (α = .59), level three (α = .72), level four (α = .50), level five (α = .53) and level six (α = .32). They were determined by using the answers obtained for the comprehension questions, however, the coefficients were slightly lower than those obtained in the standardised sample (Neale, 1997), or in the study by De Sousa and Broom (2011), in which the NARA-Revised was used4, and lower than 4 Refer to Chapter 3 page 27 81 those found in the NARA-third edition (Australian Edition), as reviewed by McInerney and Rogers (2003). The internal consistency coefficients in this study were more in accordance with those found by Spooner, Baddeley and Gathercole (2004) using the NARA-Revised with UK children. Furthermore, reviews of the NARA II as indicated by Miller-Whitehead and Rhoades (2005), state that the comprehension questions are factual, with many of the questions requiring the learner to make inferences or predictions. This indicates that the NARA II might not be as reliable in the South African context as in other countries, previous research conducted on the comparison of first and second language learners, or in comparison to previous editions of the NARA. 6.5 Face Validity Frequencies pertaining to whether the NARA II appeared to be appropriate and suitable to the South African context were obtained5. It is evident that the majority of the professionals who participated in the semi-structured interviews were of the opinion that the NARA II did appear to be assessing reading ability. Those respondents who indicated that the instrument was appropriate and suitable did so in a sensible manner. One respondent who reported that the NARA II was not appropriate or suitable indicated reasoning that was more consistent with the difficulty at the entry level and that perhaps it needed to be more basic. It is therefore possible to conclude that the NARA II has sufficient face validity in the South African context. 6.6 Content Validity Tables 5.5 and 5.6 presented the frequencies pertaining to whether the words and stories in the NARA II were appropriate for the South African context. A list of 82 words was selected from the NARA II, with 42 of these identified by half or more of the participants as problematic. In terms of the stories, Table 5.6 showed the frequencies of inappropriate stories, including six passages and two practice passages. Four passages were identified by half or more of the participants, indicating that at least half of the stories from the NARA II were perceived as inappropriate for the South African context. In the NARA II Manual for Schools (Neale, 1997), there is only mention made of extra care being taken to enhance the content validity by selecting subject matter that is suited to the various age groups. Further, it states that: 5 Refer to Chapter 5 page 53 and 54 82 … content validity relates to the extent to which the task content of a test can be said to be a representative sample of the skill or ability the test is designed to measure, and how closely the behaviour demanded by the test resembles ‘real-life’ activity in that particular area of skill” (Neale, 1997, p.55). Thus, since stories are read aloud and learners are required to answer comprehension questions about them, it is apparent that on the face of it the test possesses content validity. There is mention in reviews of the NARA Australian Standardisation by McInerney and Rogers (2003) that even in the most recent forms there are still many weaknesses in the validity, which are not mentioned and have not yet been resolved. Furthermore, the review states that the evidence provided for content validity seems to be only the opinion of the authors and could have been improved if empirical evidence were provided in the manual. This may also be the case with the NARA II. Investigations of content validity in this study are further supported by the qualitative data in which the sub-theme Appropriateness of words and stories from the NARA II emerged6. Respondents commented on the words and stories used in the NARA II and suggested possible themes that would be suitable to the South African context, as presented in Table 5.19. Therefore, the stories in the NARA II may be well suited to a UK population, but not necessarily to the South African context (Miller-Whitehead & Rhoades 2005). In addition, the stories from the NARA II were the same as those used in the first publication in 1958, and perhaps these should be updated to suit the South African population and include aspects that interest learners in today’s time and age. Thus, it can be concluded that in terms of content validity, the NARA II appears to have insufficient content validity for the South African context and may need to be adapted for use in the country. This could be done by a panel of experts similar to the professionals who took part in the semi-structured interviews but on a much larger scale, and who are in the field of teaching reading, education and adaptation of tests (Woolfaardt & Roodt, 2005), or actual correlations with validated tests of reading comprehension. Within the educational setting in which the NARA II is being used, learning outcomes in the area of reading form the basis in which the constructs to be tapped can be defined (Foxcroft, 2005). Furthermore, in South Africa specifically, with its multilingual context, the constructs to be tapped should be explored in terms of language together with their value for each group (Foxcroft, 2004). 6 Refer to Chapter 5 page 69 and 70 83 6.7 Concurrent Validity Correlations were conducted to determine if there were any significant relationships between the NARA II and learners school results, as shown in Table 5.7. Significant correlations were found between the NARA II and school performance. The normative sample’s NARA II results were also correlated with other tests, such as the Phonological Assessment Battery (PhAB) and the British Ability Scales Second Edition (BAS II). However, even though the NARA II Manual for Schools indicates that this evidence can be found in the NARA II Manual for Psychological Services, this manual is no longer available and therefore information on the correlations for the normative sample cannot be commented on. The results from the current study show that reading ability as assessed by the NARA II is strongly related to academic performance in school. Good readers in the early grades become better learners in the school years to follow, and beyond. Therefore, the NARA II could be used to predict school success and for intervention by assessing individuals or groups. Results from the assessment can be used in different ways, for instance to assist with teaching and provided to other professionals who can intervene and assist learners, as well as to monitor teaching and learning (Neale, 1997). 6.8 Bias Bias was examined by comparing the errors on the NARA II by demographic variables (gender, population group, home language, educational level of parents and the school learners attended). Independent t-tests and ANOVAs were conducted as presented in Tables 5.8 to 5.12. Bias was further investigated by comparing the difference between chronological age and reading age against the demographic variables (gender, population group, home language, educational level of parents and the school learners attended), as presented in Tables 5.13 to 5.19. 6.8.1 Gender bias and the NARA II In terms of gender, significant differences were found between males and females for Addition and Omission errors, with males making more of both types. The literature indicates that these occur when there is a lack of skill in reading (Read America, 2005), which may be linked to socialisation, as boys are often not encouraged to read as much as girls (Millard, 1997). However, a comparison of the difference in chronological age and reading age and gender, revealed no significant differences between males and females. Some studies have shown that males read better than females (Cockcroft & Blackburn, 2008; Huestegge et al., in press) while other studies have shown the opposite (Patel, 2009). The current study 84 concludes that while boys made significantly more Addition and Omission errors than girls, there was no overall difference in reading ability between the genders. 6.8.2 Language bias and the NARA II With regards to home language, second language learners made significantly more errors than first language learners when reading the passages on the NARA II with moderate effect sizes. In addition, a comparison of the difference in chronological age and reading age revealed significant differences with strong effect sizes. This is not surprising given research indicating that second language learners tend to perform poorly in reading relative to their first language peers (De Sousa & Broom, 2011; Hutchinson et al., 2003). The most critical moderator variable of test performance is language (Nell, 1994). Many tests that are developed from a monolingual perspective rather than a multilingual perspective such as South Africa may disadvantage learners, for whom English is a second language (Foxcroft & Davies, 2008). Even though reviews of the NARA II indicate that it can be used with readers whose first language is not English, no further research or information is provided (McInerney & Rogers, 2005). Further research is warranted. Since many learners in South Africa are educated in a second language they therefore cannot be expected to perform at the same level as first language learners who are reading in their home language (Janks, 2011). Second language learners cannot always use their first language skills to enhance their second language reading (Pretorius, 2000). An adaptation of the content used in the NARA II, as mentioned in the discussion of content validity, may make a difference to the performance of second language learners, and South African learners in general. This was further supported by the qualitative findings in which the sub-theme Appropriateness of words and stories from the NARA II had emerged. Respondents made mention of words and names of people being used in the reading passages that were not suitable. Therefore, expert reviews would need to judge the content based on language (Foxcroft, 2005). First language learners may have performed significantly better than second language learners, but may not have met the standards set out in the NARA II norms. This is supported by evidence from the sub-theme norms, in which respondents commented that those from the NARA II were too high for the South African context. This may be because children in the UK start reading at school earlier than those in South Africa, leaving a two- year gap between the two. Related to this was the suggestion that the NARA II be normed on the South African population. 85 6.8.3 Population group and the NARA II In terms of population group, significant differences were found between the population groups with small effect sizes for Substitution errors and medium effect sizes for the Overall errors. Furthermore, large effect sizes were found for Accuracy, Comprehension and Rate when the population group was compared to the difference in chronological age and reading age. Differences were found between the Indian and African group and between the Indian and Other group for Accuracy. With regards to Comprehension, differences were found between the Indian and African group. Differences were found between the Indian and African group and the Indian and Other group for Rate. This is linked to first and second language proficiency and socio-economic status, with the African population group being mostly second language English learners in this study. The African population group were also mainly learners from a lower socio-economic status group, which indicates that population group, specifically in a multilingual South African society, is linked to language and socio-economic status, as would be expected. A test administered to an individual who has values different from those of the people for whom the test was developed may result in the test score not being a good reflection of that individual’s true level of functioning (Grieve, 2005). Within the South African context, there is a diversity of languages and values and therefore variations towards a Western norm should be considered (Foxcroft, 2004). Many tests that are available today are mainly applicable to a monolingual population (Foxcroft & Davies, 2008), therefore, scores and results cannot have equivalent meaning for different language groups and the NARA II should be linguistically appropriate. This is linked to the inappropriate words and stories selected by respondents from the semi-structured interviews. As an example, an African child may know the word ‘ancestors’ but may not know what a ‘tree-house’ is. If the NARA II were developed for one language group, research should be conducted to find evidence of appropriateness and may need to be adapted before being used on another language group (Foxcroft & Davies, 2008). 6.8.4 Educational level of parents and the NARA II With regards to educational level of parents, no significant differences were found between educational level of parents and the errors made on the NARA II. However, large effect sizes were found when educational level of parents was compared to the difference in chronological age and reading age. More specifically, differences were found between the Less than matric and Post matric group and the matric and Post matric group for Accuracy, Comprehension and Rate. This is because most of the learners who came from a lower 86 socio-economic status background had a larger gap between their chronological age and reading. Furthermore, those from a lower socio-economic status were mainly second language learners, indicating that socio-economic status as extrapolated to educational level of parents in this study, is influenced by the home environment and input from parents and in turn influences the stimulation received from the home environment. Children living in deprived environments do not often receive the resources required to enhance reading, such as books or a parent reading to their child (Patel, 2009). According to Molfese, Molfese and Modgline (2001), the educational stimulation received in the home largely influences reading skills, which is why there is a wide gap that hinders their reading ability when children attend school. Therefore, parents’ educational level does play a factor and differentiates the developmental opportunities available (Foxcroft & Davies, 2008), other factors involved in developing reading in the home setting would need to be investigated. 6.8.5 Learner school and the NARA II In terms of the schools attended by learners, significant differences were found with medium effect sizes when the school the learners attended was compared to the errors made on the NARA II. Furthermore, large effect sizes were found when the school the learners attended and the difference in chronological age and reading age were compared. These differences were between School 1 and School 3, School 1 and School 4, School 2 and School 3 and School 2 and School 4. These indicate that the type of school the learners attend influenced their reading ability and performance on the NARA II. Those who had experienced a poorer quality of education might not have had the same opportunities as those with a more affluent quality of education, in terms of developing academic proficiency and skills which impacts on test performance (Foxcroft, 2004; Foxcroft & Davies, 2008). A school that is less well equipped with resources and that provides inferior educational opportunities for their learners results in lower achievement compared to those who attend an affluent suburban school (Snow, Burns & Griffin, 1998), and results in low achievement learners not being able to catch up to their peers. A study across four provinces in South Africa found great concern about the extent to which teachers are equipped to promote language and literacy development (O’Carroll, 2011). The assumption is that reading should be facilitated and not taught. Teachers utilised rote teaching as that was the only option and developed their own materials and reading programmes to teach reading. In order for learners to improve their reading competencies, teachers should be able to assess the learners’ reading levels. Teachers are provided with reading assessment tools in the Foundation Phase by the DoE (2008); however, there are 87 no available tools beyond Foundation Phase. Therefore learners who attend an affluent school may become stronger readers and will do so throughout their lives, while learners from a disadvantage school may have weaker reading ability and therefore tend to read increasingly less, resulting in delays and impairment of reading development skills (Patel, 2009). 6.9 Qualitative Findings Various themes and sub-themes had emerged from the semi-structured interview which is complemented by the quantitative findings. This section discusses each theme and sub- theme in relation to the literature found. 6.9.1 The suitability of the NARA II in the South African context in relation to norms, words and stories and reporting results The NARA II appeared to be suitable to the South African context in terms of the design and its comprehensive ability to assess reading when the test does not assess individual words out of context but rather assesses the three main areas, namely, Accuracy, Comprehension and Rate. From the theme Suitability of the NARA II to the South African context, three sub- themes emerged, namely, Appropriateness of words and stories from the NARA II, norms and results. The content of the NARA II does not appear to be suitable to the South African population, as discussed under the Content Validity section of the Results and Discussion chapters, thus suggesting that perhaps the content of the NARA II should be reviewed for its suitability to the South African context. The use of tests in a multilingual context such as South Africa requires items or constructs suitable to that language. Tests from other linguistic backgrounds cannot be applied to other contexts without investigating issues of bias, adaptation and re-norming. Furthermore, there is a need for appropriate psychometric properties in the context for which it is being used without only using the original psychometric properties (Foxcroft, 2002). Therefore, linguistically relevant tests need to be developed or existing ones adapted to suit the diverse South African context (Foxcroft, 2004). The norms of the NARA II appeared to be too high for the South African population, as supported by professionals commenting that even bright learners did not score above the norm. Therefore, when using the NARA II on South African learners it is important that they match or are similar to the normative sample. Suggestions towards norms for first and second language learners were made linked to the discussion on LOLT and home 88 language7. Given that most South Africans prefer being educated in English (De Wet, 2002), consideration towards what norms would be appropriate would need to be made. If South African children are being educated in English, should the test be in English with norms for South African children educated in English or should separate norms be created for those who speak English as a first language and those who speak it as a second language? When there is not a good enough match caution needs to be taken in interpreting results. Scores on a norm-referenced test such as the NARA II are based on the performance of the normative group (Grieve, 2005), therefore professionals who are using the NARA II qualitatively are probably following a more ethical way of reporting results. Results of a test mean different things depending on what the results are used for and who they are being reported to. They should be provided so as to convey their practical value (Foxcroft, 2002) and a test score means nothing if it cannot be used to show how to remediate the problems. A qualitative analysis of errors can assist in improving reading, even though the actual scores may not improve (Bouwer, 2004; Snow, Burns & Griffin, 1998). Even though there is a large debate with regards to using imported instruments, adapted versions of imported instruments and instruments developed locally (Laher & Cockcroft, in press), this theme suggests that perhaps an adaption of the NARA II is required in terms of the content and re-norming, based on a South African normative sample. Adaptation and standardisation of the NARA II as an international instrument would be more viable then redeveloping a new instrument. In addition to the lengthy process of test development and complications that come with it, there are challenges with the lack of skilled personnel and funding (Laher & Cockcroft, in press). Using the NARA II in the South African context would also mean that individuals would need to have the skills and knowledge to use the instrument and there would need to be test developers who can review the test in order to analyse its suitability to the multicultural South African context. Adaptations or development of tests could lead to the test being in one language, for use in a multilingual society, with specifications being made for the level of proficiency required in the language being used in the test, and the level of proficiency by the test-taker. Furthermore, various versions of the same test in different languages could be considered (Foxcroft, 2002). Furthermore, once the content, instructions and tasks are adapted they need to be carefully piloted and refined before the adapted version is used. Psychometric properties of the adapted version are needed before the adapted test is used. Consideration towards the existence of various versions of the test should be made to suit the various linguistic backgrounds (Foxcroft, 2002) and consideration made to assessing in one’s home language 7 Refer to Chapter 2 pages 19-21 89 or LOLT. Within a multilingual assessment setting, consideration should be made towards creating an equivalent form of an existing test, defining new norms, creating new tests and researching validity threatening factors (Dhladhla & de Kock, 2008). A general wish list for an assessment tool in South Africa is for one that should be able to assess reading comprehensively through Accuracy, Comprehension and Rate; furthermore, a word reading list should be included. Comprehension should be assessed orally and silently through written questions and answers. The reading test should include a spelling test, which would be available in various forms and used to assess learners at various educational levels with suitable norms. The content material should be appropriate to the South African context. Results should have both a quantitative and qualitative component since quantitative information may be meaningless to parents who prefer a practical understanding of their child’s reading ability. Skills should be assessed both in and out of context, and therefore both word list and reading of stories are important. Furthermore, awareness of the importance of reading and reading assessments should be made within the South African context. 6.9.2 NARA II in relation to reading at school The NARA II is similar to what is being done in many schools in which comprehension skills are evaluated. Furthermore, reading is evaluated in schools through reading stories, even though many schools do not make use of one-on-one reading at Grade Four level, where the foundations of reading ability are assumed. Reading is not taught in Grade Four and learners are expected to read and evaluate various texts in various school subjects. The reading tasks in school are based on the DoE’s criteria on the outcome for Grade Four8, when reading is done in order to understand information and critically evaluate texts (DoE, 2002b). Therefore, it cannot be assumed that Grade Four learners are not struggling with reading if it is not comprehensively assessed in schools and those who are struggling are not identified. This is a further indication of the importance of assessing reading. With regards to differences between the NARA II and reading tasks in schools, it is evident that various differences exist, such as when rhyming words are used in stories, when the books being read are based on target words taught in class, and when the pictures are colourful and appealing. With regards to comprehension questions and answers, schools make use of written ones whereas those in the NARA II are administered verbally. Therefore, tests that exist in South Africa or those that should be adapted for its context should be aligned with the outcomes of the school curriculum in terms of achievement. 8 Refer to Chapter Two page 16 90 Specific measures and procedures should be created for teachers to assist learners. Those developing or adapting tests need to see how the school curriculum fits into the development or adaptation of a test (Foxcroft & Roodt, 2005). 6.9.3 The importance of comprehension skills in reading Comprehension skills emerged as a theme when teachers mentioned that learners can read but do not do so technically for meaning, and therefore do not understand what they are reading. Poor comprehension skills could be due to the poor identification of most words in a text, thus poor decoding skills may result in poorer comprehension and learners who cannot identify words in a text may not understand what they are reading. Learners who cannot read at least 95% of words may struggle with comprehension, which is also related to reading rate in that if a learner reads too slowly it interferes with what he or she comprehends (Gunning, 2010). 6.9.4 Errors made in reading The professionals mentioned the type of errors their learners made with Mispronunciation and Substitution errors being the most common, supported by the quantitative findings that these were the errors learners mostly made. Pronunciation differences in South Africa are due to accents found in various cultural groups and therefore it is important to take into account cultural competence. Cultural competence is defined as the understanding and appreciation for differences in beliefs and behaviours. This includes the recognition and respect for differences in cultural groups and the ability to adjust in order to provide effective intervention (Balcazar, Suarez-Balcazar & Taylor-Ritzler, 2009). Cultural competence is being critically aware of personal biases towards others, cultural knowledge of other cultures, developing skills to communicate with others and applying these components in practice. For example, an African child may pronounce ‘ship’ as sheep or an Indian child may say host pipe for a ‘hose pipe’. For these reasons, Mispronunciation errors may need to be assessed qualitatively and to be sensitive to pronunciation in other cultures. Reversal errors were also mentioned by professionals, but since these are characteristic of learners with learning and/or reading disability, of whom there were none in this study, such errors were found to be minimal. 91 6.9.5 The skills required to be able to read To be able to read, certain skills are required, and from this the theme Foundations of reading emerged. Basic phonics, the ability to blend and segment and being taught sight words appeared to be the basis for teaching reading, which is in line with the literature on the development of reading in which sight words are taught as visual wholes and phonics are dominant in the phonological-recoding phase (Harris & Coltheart, 1986). 6.9.6 The benefits and affordability of reading assessments Some of the above themes support the need for reading to be assessed at school level and reasons for this have been provided. By assessing reading, learners’ strengths and weaknesses can be identified, which assists in planning intervention. Assessing reading is important so as to track progress that can lead to an improvement in learner performance (Baatjies, 2003). This forms part of the theme Benefits. Related to the importance of reading assessment is the theme Affordability. Given the South African context, in which 29.8% of the South African working aged population are unemployed (Statistics South Africa, 2012), reading assessments may not be a priority or affordable to many citizens. Perhaps middle-income households may be able to afford a reading assessment but learners from a lower socio-economic background who attend a government school would not be able to afford a reading assessment. Suggestions towards community-based services and teachers being trained in the reading assessment seem to be more cost-effective. Linked to the theme Affordability, suggestions towards various individuals in each school being trained were made, which is further supported by the suggestion that every school requires remedial teachers who can work with learners identified as having a reading difficulty, since teachers do not have the time to assess and intervene in additional reading for learners who are struggling. Thus far, the quantitative and qualitative findings have been discussed. Since the former complements the latter it is important to discuss what is unique from each type and their possible implications. The quantitative findings provided information on the learner sample, including the demographic information of learners and the descriptive statistics of the NARA II. This could not possibly be established through the interviews with professionals. In order to address issues of reliability, validity and bias, learners needed to be assessed on the NARA II. Concurrent validity looked at how the NARA II is related to academic performance, an area that could not be established without the relevant statistical analysis. In order to establish whether a test is biased towards certain groups, it was important to statistically 92 analyse significant differences between the NARA II and gender, home language, population group, educational level of parents and the type of school learners attended. Any significant differences may raise the possibility of bias towards that group. In terms of face validity and content validity, learners from the sample could not possibly comment on these since professionals who are working with children in terms of reading and those using reading assessments would be able to have a better understanding based on their experience. Experts in the field of reading and reading assessments, such as the professionals used in the interviews, could further provide qualitative information on the applicability of the NARA II in the South African context through which the themes and sub-themes emerged. These themes provide further information on what adaptations should be made for the NARA II’s suitability in South Africa which the quantitative findings could not provide. Thus, the mixed methods approach in this study allowed both approaches to complement each other and provided unique information in some cases and support each other in others. 6.10 Overall Findings This study aimed to explore the applicability of the NARA II in the South African context by exploring the reliability, validity and issues of bias in the instrument. It is evident from the results presented in Chapter 5 and the discussions presented in this chapter, that internal consistency coefficients were generally good but lower than those found in the normative sample and other studies (De Sousa & Broom, 2001; Neale, 1997). Internal consistency reliability is more consistent with the reliability coefficients found by Spooner, Baddeley and Gathercole (2004), indicating better consistency with the NARA-Revised. The NARA II was found to have good face validity but content validity was insufficient for the South African context, suggesting adaptation of the NARA II for use in the country. Concurrent validity was sufficient since strong correlations were found between the NARA II results and the learners’ school performance. No bias was found for gender since males and females performed similarly on the NARA II. The educational level of parents in terms of explaining the influence of socio-economic status plays a role in influencing reading ability. Home language, population group and the schools learners attended seem to indicate some bias towards certain groups. Second language learners performed significantly poorer than first language learners with moderate to large effect sizes, indicating that the NARA II may be biased against them. Furthermore, significant effect sizes were found for population group, indicating that the NARA II developed for one cultural group may not be generalisable for other groups. In terms of the schools learners attended, those from poorer schools had poorer reading ability than those from affluent schools, indicating that the type of school attended also influences reading ability, thus suggesting that perhaps the NARA II should be 93 adapted to be appropriate for the language populations together with consideration of the quality of education and socio-economic status of learners. Previous chapters have made mention to some of the limitations to this study. However, these are discussed below in terms of the sample size, method and instrument. This is followed by recommendations for future research. 6.11 Limitations with regards to the sample size 6.11.1 Learners The sample size was reduced since many parents/guardians returned the consent form and biographical questionnaires with incomplete information and therefore had to be excluded. The learner sample came from four schools in one area in Gauteng, which may not be representative of the entire province. Furthermore, the research study only covered one of nine provinces and therefore not every Grade Four learner had an equal chance of being selected and it is not a representation of all in the country. However, the sample was adequate and yielded useful information for further research. In addition, research could be conducted to include various grades and not limited to one to increase generalisability of the results. 6.11.2 Professionals The sample size for the qualitative phase aimed at including at least one teacher and one other professional in close proximity to the schools in which the quantitative data was collected. However, the small sample size gained for the semi-structured interviews was only eight and therefore limited the input and value that professionals could have provided for this research, and its generalisability. 6.12 Limitations with regards to the method It was not possible to control for all extraneous variables that may have influenced reading ability, such as IQ, motivation, attitude, home environment, and assessment room. This was not a longitudinal study and therefore cannot report on the learners’ reading ability over time, or developmentally with age. 94 6.13 Limitations with regard to the instrument Testing individuals leaves some room for error. However, the use of standardised tests ensures reliability and validity, provided that they are based on relevant norm groups. The NARA II makes use of norms based on a UK sample, thus the lack of South African norms used in this research is a limitation in itself. In terms of reliability, test-retest reliability could not be ascertained due to practicality and time constraints. Evaluation of parallel/alternate forms reliability was beyond the scope of this study and should be considered for further research. In terms of validity, face validity is debated as it does not add value to psychometric research (Murphy & Davidshofer, 2005). 6.13 Recommendations for Future Research A replication of this study with a larger sample size is warranted in order to be representative of the South African population and for the results to be more broadly generalisable. Therefore, similar studies could be conducted across provinces. A longitudinal study is suggested so as to report on the learners’ reading ability over time or developmentally with age. This would provide details on the trends found and allow for information on test-retest reliability with larger samples over different periods of time. It would further provide information on reading ability across the developmental span. In terms of the qualitative data, perhaps a larger sample of respondents to the interview questions from various areas should participate in another interview in order to gather broader knowledge of how the NARA II would work in the South African context. Consideration towards test-retest reliability should be made in terms of a longitudinal study as mentioned above. Furthermore, this study utilised Form One of the NARA II and perhaps a replica using Form Two is required, so as to comment on the parallel/alternate forms reliability. In terms of validity, convergent validity could be explored by using the NARA II and other reading tests, especially ones with South African norms. Overall, the main suggestion is that an adaptation of the NARA II and norming on the South African population is undertaken. Given the multilingual South African society, considerations would need to be made to translate the test into various languages. This study draws some tentative conclusions in terms of the applicability of the NARA II to the South African context. . Overall, the face validity and concurrent validity proved to be good. The internal consistency reliability was adequate. However, issues content validity and bias would need to be addressed in order to make it a more appropriate instrument. The current study has served as a preliminary exploratory study which has introduced various 95 research possibilities using the NARA II. Further research studies should explore the issues that were identified in this study to determine the applicability of the NARA II in the South African context and the feasibility of adaptation for the country. 6.14 Conclusion In conclusion, this chapter has provided a discussion of the reliability, validity and bias of results found for the NARA II in this study. Furthermore, a discussion on the qualitative findings was made, linked to arguments presented in the literature review. It appears that the NARA II should be adapted for the South African context since the current content and norms are not relevant for use in the country. This chapter concluded with a discussion of the limitations in the study with regards to the sample, method and instrument used together with recommendations for future research. 96 REFERENCE LIST Adams, M.J. (1996). Beginning to Read. Massachusetts: MIT Press. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Arlington, VA: Author. Baatjes, I. (2003). Reading in South Africa: An overview of policy, programmes and campaigns since 1994. Innovation, 26, 1-10. Babbie, E & Mouton, J. (1998). The Practice of Social Research (South African ed.). Cape Town: Oxford University Press. Baker, C. (2006). Foundations of bilingual education and bilingualism. (4th ed.). Clevedon: Multilingual Matters. Balcazar, F. E., Suarez-Balcazar, Y., & Taylor-Ritzler, T. (2009). Cultural competence: Development of a conceptual framework. Disability and Rehabilitation, 2009, 31(14) 1153-1160. Beckett, G. (2005). Perspectives on reading and listening comprehension. SAALED News, 25(2), 2. Beech, J. R. (2005). Ehri’s model of phases of learning to read: a brief critique. Journal of Research in Reading, 28(1), 50-58. Beech, J. R. & Singleton, C. (1997). The psychological assessment of reading: Theoretical solutions and professional issues. In J. R. Beech & C. Singleton (Eds.), The Psychological Assessment of Reading (pp. 1-26). London: Routledge. Bot, M. (2011). An Overview of Selected Schooling Indicators, 2001-2010. Retrieved on March, 13, 2012 from www.jet.org.za Bouwer, C. (2004). Reading and Writing. In I. Eloff & L. Ebersӧhn (Eds.), Keys to Educational Psychology (pp. 83-118). Cape Town: UCT University Press. Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. Buthelezi, Z. (2003). Researchers, beware of your assumptions! The unique case of South African education. Retrieved on September 12, 2012 from www.readingonline.org Cain, K., Oakhill, J.V. & Bryant, P.E. (2000). Phonological skills and comprehension failure: A test of the phonological processing deficit hypothesis. Reading and Writing, 13(1- 2), 31-56. 97 Cardoso-Martins, C. (1995). The reading abilities of beginning readers’ of Brazilian Portuguese: implications for a theory of reading acquisition. Scientific Studies of Reading: 5, 289-317. Chall, J. S., Jacobs, V. A. & Baldwin, L. E. (1990). The reading crisis: Why poor children fall behind. Massachusetts: Harvard University Press. Chisholm, L. (2003). The state of curriculum reform in South Africa: The issue of Curriculum 2005. In J. D. Adam Habib & R. Southall (Eds.), State of the Nation: South Africa 2003-2004 (pp. 268-289). Cape Town: HSRC Press. Cockcroft, K. (2009). Language Development. In J. Watts, K. Cockcroft & D. Hook (Eds.), Developmental Psychology (pp.378-395). Landsdowne: UCT Press. Cockcroft, K. & Blackburn. M. (2008). The relationship between Senior South African Individual Scale-Revised (SSAIS-R) sub-tests and reading ability. South African Journal of Psychology, 38(2), 377-389. Collins, K. J., Du Plooy, G. M., Grobbelaar, M. M., Puttergill, C. H., Terre Blanche, M. J., van Eeden, R., van Rensburg. G. H., & Wigston, D. J. (2000). Research in the Social Sciences. University of South Africa: Pretoria. Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage. Crystal, D. (2002). The English language: A guided tour of the language. (2nd ed.). London: Penguin Books. Day, R. R. & Bamford, J. (1998). Extensive Reading in the Second Language Classroom. New York: Cambridge University Press. De Jager, K., Nassimbeni, M. & Underwood, P. (2006). Libraries, Literacy and Learning: Retrospects and Prospects. Retrieved March, 13, 2012 from www.dissanet.com De Sousa, D. S. & Broom, Y. (2011). Learning to read in English: Comparing monolingual English and bilingual Zulu-English Grade 3 learners. South African Journal of Childhood Education, 1(1), 1-18. De Wet, C. (2002). Factors influencing the choice of English as language of learning and teaching (LoLT)-a South African perspective. South African Journal of Education, 22(2), 119-124. Dehn, M. J. (2006). Essentials of Processing Assessment. New Jersey: John Wiley & Sons . Department Of Basic Education. (2011a) Action Plan to 2014: Towards the realisation of schooling 2025. Retrieved July, 14, 2012 from www.info.gov.za 98 Department of Basic Education. (2011b). Dictionary of education concepts and terms. Retrieved July, 14, 2012 from www.info.gov.za Department of Basic Education. (2011c). Curriculum and Assessment Policy Statement Grades 4-6. Retrieved June, 26, 2013 from www.education.gov.za Department of Education. (2002a). Revised National Curriculum Statement Grades R-9 (Schools). Retrieved July, 5, 2012 from www.info.gov.za Department of Education. (2002b). Revised National Curriculum Statement Grades R-9 (Schools) Languages-English Home Language. Retrieved July, 14, 2012 from www.info.gov.za Department of Education. (2008). National Reading Strategy. Retrieved July, 14, 2012 from www.info.gov.za Dhladhla, T. J. & De Kock, F. (2008). Psychological testing in South Africa: Current Issues and Challenges. Testing International, 19, 8-11. Ehri, L. C. (2005). Learning to read Words: Theory, Findings and Issues. Scientific Studies of Reading, 9(2), 167-188. Elkonin, D., Foxcroft, C., Roodt, G. & Astbury, G. (2005). The use of assessment measures in various applied contexts. In C. Foxcroft & G. Roodt (Eds.), An introduction to Psychological Assessment in the South African context (pp. 8-23). Cape Town: Oxford University Press. Eloff, I & Ebersӧhn, L. (2004). Introduction to Educational Psychology. In I. Eloff & L. Ebersӧhn (Eds.), Keys to Educational Psychology (pp. 4-9). Cape Town: UCT University Press. Esterhuyse, K. G. F. & Beukes, R. B. I. (1997). Manual for the ESSI Reading and Spelling Test. Department of Psychology. Bloemfontein. Fleisch, B. (2008). Primary Education in Crisis. Cape Town: Juta. Foxcroft, C. (2002). Ethical issues related to psychological testing in Africa: What I have learned (so far). In W. J. Lonner, D. L. Dinnel, S. A. Hayes & D. N. Sattler (Eds.), Online readings in psychology and culture (Unit 5, Chapter 4). Retrieved September 19, 2012, from: http://www.wwu.edu/culture/foxcroft.htm Foxcroft, C. D. (2004). Planning a Psychological test in the Multicultural South African Context. South African Journal of Industrial Psychology, 30(4), 8-15. 99 Foxcroft, C. (2005). Developing a psychological measure. In C. Foxcroft & G. Roodt (Eds.), An introduction to Psychological Assessment in the South African context (pp. 46- 56). Cape Town: Oxford University Press. Foxcroft, C. D. & Davies, C. (2008). Historical perspectives on psychometric testing in South Africa. In C. van Ommen & D. Painter (Eds.), Interiors: A history of psychology in South Africa (pp. 152-181). Pretoria: UNISA Press. Foxcroft, C., Roodt, G. (2005). What the future holds for psychological assessment. In C. Foxcroft & G. Roodt (Eds.), An introduction to Psychological Assessment in the South African context (pp. 242-257). Cape Town: Oxford University Press. Foxcroft, C., Roodt, G. & Abrahams, F. (2005). Psychological Assessment: a brief retrospective overview. In C. Foxcroft & G. Roodt (Eds.), An introduction to Psychological Assessment in the South African context (pp. 8-23). Cape Town: Oxford University Press. French, J. N. (1986). Reading and study skills in the secondary school: a sourcebook. New York: Garland. Frith, U. (1985). Beneath the surface of developmental dyslexia. In K. E. Patterson, J. C. Marshall & M. Coltheart. (Eds.), Surface Dyslexia: Cognitive and Neuropsychological Studies of Phonological Reading (pp. 301-330). Lawrence Erlbaum Associates: London. Fromkin, V. & Rodman, R. (1998). An Introduction to Language. (6th ed.). United States of America: Harcourt Brace College Publishers. Fung, I. Y. Y., Wilkinson, I.A. G & Moore, D.W. (2003). L1-assisted reciprocal teaching to improve ESL students’ comprehension of English expository text. Learning and Instruction, 13(1), 1-31. Geva, E. (2006). Learning to Read in a Second Language: Research, Implications, and Recommendations for Services. Retrieved June, 26, 2012 from www.child- encyclopedia.com Goodman, K. S. (1986). What’s whole in whole language? Portsmouth: Heinemann. Greenop, K. T. (2004). The development of literacy skills in Zulu and Northern Sotho speaking children: a cognitive-neuropsychological investigation. Unpublished Doctoral thesis, University of the Witwatersrand, Johannesburg. 100 Grieve, K. W. (2005). Factors affecting assessment results. In C. Foxcroft & G. Roodt (Eds.), An introduction to Psychological Assessment in the South African context (pp. 224- 241). Cape Town: Oxford University Press. Grove, M. C. & Hauptfleisch, H. M. A. M (1982). Remedial Education in the Primary School. Pretoria: HAUM Educational. Gunning, T. G. (2010). Assessing and Correcting Reading and Writing Difficulties. (4th ed.). Boston: Pearson Education. Harris, M. & Coltheart, M. (1986). Language Processing in Children and Adults. New York: Routledge & Kegan Paul. Hoff. E. & Naigles, L. (2002). How children use input in acquiring a lexicon. Child Development, 73(2), 418-433. Howie, S., Venter, E., Van Staden, S., Zimmerman, L., Long, C. Du Toit, C., Scherman, V. & Archer. E. (2008). Progress in International Reading Literacy Study (PIRLS) 2006 Summary Report: South African Children’s Reading Achievement. Pretoria: Centre for Evaluation and Assessment Huck, S. W. (2009). Reading statistics and research (5th ed.). Boston: Pearson. Huestegge, L., Heim, S., Zettelmeyer, E. & Lange-Küttner, C. (in press). Gender-specific Contribution of a Visual Cognition Network to Reading Abilities. British Journal of Psychology. Hurry, J., & Doctor, E. (2007). Assessing literacy in children and adolescents. Child and Adolescent Mental Health, 12(1), 38–45. Hutchinson, J.M., Whiteley, H.E., Smith, C.D., & Connors, L. (2003). The developmental progression of comprehension-related skills in children learning EAL. Journal of Research in Reading, 26(1), 19-32. Janks, H. (2011). Making sense of the PIRLS 2006 results for South Africa. Journal of the Reading Association of South Africa, 2(1), 27-39. Jansen, J. & Taylor, N. (2003). Educational Change in South Africa 1994-2003: Case Studies in Large-Scale Education Reform. Education Reform and Management Series. Retrieved on December, 19, 2012 from www.jet.org.za Jean, M. & Geva, E. (2009). The development of vocabulary in English as a second language children and its role in predicting word recognition ability. Applied Psycholinguistics, 30(1), 153-185. 101 Just, M. A. & Carpenter, p.A. (1987). The psychology of reading and language comprehension. Boston: Allyn & Bacon. Kanjee, A. (2005). Cross-cultural test adaptation and translation. In C. Foxcroft & G. Roodt (Eds.), An introduction to Psychological Assessment in the South African context (pp. 57-66). Cape Town: Oxford University Press. Kirby, J. R. (2009). What have we learned about reading comprehension? Retrieved March, 13, 2012 from www.edu.gov.on.ca/eng/research/kirby/pdf. Laher, S. (2012). Bias in psychometric studies in South Africa. Retrieved December, 19, 2012 from www.psychologicalassessmentinsouthafrica.com Laher, S. & Cockcroft, K. (in press). Current and Future trends in assessment in South Africa: Challenges and Opportunities. In S. Laher & K. Cockcroft (Eds.). Psychological Assessment in South Africa: Research and Applications. Wits University Press: Johannesburg. Lesaux, N.K., Lipka, O. & Siegel, L.S. (2006). The development of reading in children who speak English as a second language. Developmental Psychology, 39(6), 1005-1019. Lesaux, N.K., Rupp, A.A. & Siegel, L.S. (2007). Growth in reading skills of children from diverse linguistic backgrounds: Findings from a 5-year longitudinal study. Journal of Educational Psychology, 99(4), 821-834. Liberman, A, L. (1998). Why is Speech so much easier than Reading and Writing? In C. Hulme & R.M. Joshi (Eds.), Reading and Spelling: Development and Disorders (pp 5- 17). New Jersey: Lawrence Erlbaum. Lipka, O. & Siegel, L.S. (2007). The development of reading skills in children with English as a second language. Scientific Studies of Reading, 11(2), 105-131. Luiz, D., Stroud, L. & Jansen. J. (2005). Assessment of Young children, physically disabled individuals, mentally challenged learners, and individuals with chronic conditions. In C. Foxcroft & G. Roodt (Eds.), An introduction to Psychological Assessment in the South African context (pp. 101-118). Cape Town: Oxford University Press. Marsh, G., Friedman, M., Welch, V. & Desberg, p.(1981). A cognitive-development theory of reading acquisition. In G. E. McKinnon & T.G. Waller. (Eds.), Reading Research: Advances in Theory and Practice (pp. 199-221). New York: Academic Press. McGuinness, C. and McGuinness, G. (1998). Reading Reflex. Harmondsworth: Penguin. McInerney, V. & Rogers, B. G. (2005). [Review of the Neale Analysis of Reading Ability: 3rd Edition- Australian Standardisation]. In B. S. Plake, J. C. Impara & R. A. Spies (Eds.), 102 The fifteenth mental measurements yearbook (pp. 616-622). Lincoln, NE: Buros Institute of Mental Measurements. Menzies, H. M.; Mahdavi, J. N & Lewis, J. L. (2008). Early Intervention in Reading. Remedial and Special Education, 29(2), 67-77. Millard, E. (1997). Differently literate: Boys, girls and the schooling of literacy. London: Falmer Press. Miller-Whitehead, M. & Rhoades, E. K. (2005). [Review of the Neale Analysis of Reading Ability: Second Revised British Edition]. In R. A. Spies & B. S. Plake (Eds.), The sixteenth mental measurements yearbook (pp. 674-677). Lincoln, NE: Buros Institute of Mental Measurements. Molfese, V., Molfese, D. & Modgline, A. (2001). Newborn and Preschool Predictors of Second-Grade Reading Scores: An Evaluation of Categorical and Continuous Scores. Journal of Learning Difficulties, 34(6), 545-554. Moore,T., Morton, J., & Price, S. (2007). Construct validity in the IELTS Academic reading test: A comparison of reading requirements in IELTS test items and in university study. Retrieved May 18, 2012 from www.ielts.org Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing: Principles and applications. (6th ed.). New Jersey: Prentice Hall. Neale, M. D. (1997). Neale Analysis of Reading Ability-Second Revised British Edition. London: GL Assessment. Neale, M. D., Christophers, U., & Whetton, C. (1989). Neale Analysis of Reading Ability- Revised British Edition. Windsor: NFER-Nelson. Nell, V. (1994). Interpretation and misinterpretation of the South African Wechsler-Bellevue Adult Intelligence Scale: a history and prospectus. South African Journal of Psychology, 24(2), 100-108. Neuman, W. L. (1997). Social Research Methods: Qualitative and Quantitative Approaches. (3rd ed.). Boston: Allyn and Bacon. Nicholson, T. (1993). Reading without Context. In G.B. Thompson, W.E. Tunmer & T. Nicholson. (Eds.), Reading Acquisition Processes (pp. 105-122). Great Britain: WBC Print. Oakhill, J.V., Cain, K., & Bryant, P.E. (2003). The dissociation of word reading and text comprehension: Evidence from component skills. Language and Cognitive Processes, 18(4), 443-468. 103 Oberholzer, B. (2005). The relationship between reading difficulties and academic performance among a group of foundation phase learners. Unpublished Master’s thesis, University of Zululand, Empangeni. O’Carroll, S. (2011). An exploratory study of early letter-sound knowledge in a low socio- economic context in South Africa. Journal of the Reading Association of South Africa, 2(1), 7-25. Office of the President. (1998). Employment Equity Act (Act No. 55, 1998). Government Gazette Vol. 400, No. 19370, 19 October 1998. Pang, E. S., Muaka, A., Bernhardt, E.B. & Kamil, M.L. (2003). Teaching reading. Retrieved January, 27, 2012, from www.ibe.unesco.org Patel, S (2009). Relationship between the Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV) sub-tests and Reading Ability. Unpublished Master’s thesis, University of the Witwatersrand, Johannesburg. Phillips, B. M.; Clancy-Menchetti, J. & Lonigan, C. J. (2008). Successful Phonological Awareness Instruction with Preschool Children: Lessons From the Classroom. Topics in Early Childhood Special Education, 28(1), 3-17. Pillow, W. S. (2003). Confession, catharsis, or cure? Rethinking the uses of reflexivity as methodological power in qualitative research. Qualitative Studies in Education, 16(2), 175-196. Poulos, G. & Msimang, C. T. (1998). A linguistic analysis of Zulu. Johannesburg, Horizon. Pretorius, E. J. (2000). Reading and the Unisa student: is academic performance related to reading ability, Progressio, 22(2), 35-48. Prinsloo, M. (1999). Literacy in South Africa. In D.A. Wagner, B.V. Street and R.L. Venezky. (Eds.), Literacy: An International Handbook (pp. 418-423). Boulder: Westview Press. Prinsloo, M. & Bloch, C. (1999). In L. Chisholm. (Ed.), Critical perspectives in South African Education: Reconstituting the Educational Realm (pp. 15-27). Cape Town: Juta. Read America. (2005). Phono-graphix: Classroom Word Work Lesson Plan Manual. USA: Read America. Republic of South Africa. (1996a). Constitution of the Republic of South Africa Bill, 1996. As amended by the Constitutional Committee. Cape Town: Government Printer. Republic of South Africa. (1996b). The South African Schools Bill. Act 84 of 1996. Cape Town: Government Printer. 104 Richek, M. A., List, L.K. and Lerner, J. (1992). Reading Problems: Assessment and Teaching Strategies. (2nd ed.). United Kingdom: Prentice Hall. Ronald, H. & Rodgers, J. (1995). Item bias review. Practical Assessment, Research & Evaluation, 4(6), 1-3. Sadock, B.J. & Sadock, V.A. (2003). Synopsis of Psychiatry: Behavioural Sciences/Clinical Psychiatry. (9th ed.). Lippincott Williams & Wilkins: Philadelphia. Schweigert, W. A. (1994). Research Methods & Statistics for Psychology. California: Brookes/Cole. Seymour, P. H. K. & Elder, L. (1985). Beginning reading without phonology. Cognitive Neuropsychology (in press). Seymour, P. H. K. & McGregor, C. J. (1984). Developmental dyslexia: a cognitive experimental analysis of phonological, morphemic and visual impairments. Cognitive Neuropsychology, 1(1), 43-82. Smith, M. K. (1941). Measurement of the size of general English vocabulary through the elementary grades and high school. Genetic Psychological Monograph, 24, 311- 345. Snow, C. E., Burns. M.S. & Griffin, P. (1998). Preventing Reading Difficulties in Young Children. Washington, DC: National Academy Press. Snow, C. E. & Juel, C. (2010). Teaching Children to Read: What Do We Know about How to Do it? In M. J. Snowling & C. Hulme (Eds.), The Science of Reading: A Handbook (pp. 501-520). Oxford: Blackwell. Spencer, L. H. & Henley, J. R. (2003). Effects of orthography transparency on reading and phoneme awareness in children learning to read in Wales. British Journal of Psychology, 94(1), 1-28. Spooner, A.L.R., Baddeley, A.D., & Gathercole, S.E. (2004). Can reading accuracy and comprehension be separated in the Neale Analysis of Reading Ability? British Journal of Educational Psychology, 74, 187-204. Statistics South Africa. (2012). Census 2011 Statistical release. Retrieved on December 18, 2012 from www.statssa.gov.za/Publications/P03014/P030142011.pdf Taylor, J. M. (2002). Exploring the Phono-Graphix Reading Method with adult second language learners. Unpublished Master’s thesis, Wits University, Johannesburg. 105 Taylor, B. M., Pearson, p.D., Clark, K. F., & Walpole, S. (1999). Beating the Odds in teaching all Children to Read. Centre for the Improvement of Early Reading Achievement. Retrieved on January 25, 2012 from www.ciera.org The Kha ri Gude Mass literacy campaign, Retrieved November, 16, 2011 from www.kharigude.co.za Thompson, G. B. & Johnston, R.S. (1993). The Effects of type of Instruction on Processes of Reading Acquisition. In G.B. Thompson, W.E. Tunmer & T. Nicholson. (Eds.), Reading Acquisition Processes (pp.74-90). Great Britain: WBC Print. Treiman, R. (2001). Reading. In M. Aronoff & J. Rees-Miller (Eds.), Blackwell Handbook of Linguistics (pp. 664-672). Oxford: Blackwell. Tunmer, W. E. & Chapman, J. W. (1998). Language Prediction skill, Phonological Recoding ability, and Beginning Reading. In C. Hulme & R.M. Joshi (Eds.). Reading and Spelling: Development and Disorders. (pp. 33-67). New Jersey: Lawrence Erlbaum. Vandeyar, S. (2005). Conflicting Demands: Assessment Practices in 3 S.A. Primary Schools Undergoing Desegregation. Wiley Interscience, 35(4), 461-481. Vaurus, M., Kinnunen, R. & Kuusela, L. (1994). Development of text processing skills in high-, average-, and low-achieving primary school children. Journal of Reading Behaviour, 26(4), 361-389. Verhoeven, L. & Perfetti, C. (2008). Advances in Text Comprehension: Model, Process and Development. Applied Cognitive. Psychology, 22(3), 293–301. Wheldall, K. & Limbrick, L. (2010). Do more boys than girls have reading problems? Journal of Learning Disability, 43(5), 418-429. Wolfaardt, J. B. & Roodt, G. (2005). Basic Concepts. In C. Foxcroft & G. Roodt (Eds.), An introduction to Psychological Assessment in the South African context (pp. 24-45). Cape Town: Oxford University Press. Ziegler, J. C. & Goswami, U. (2005). Reading Acquisition, Developmental Dyslexia and Skilled Reading Across Languages: A Psycholinguistic Grain Size Theory. Psychological Bulletin, 131(1), 3-29. Ziegler, J. C. & Goswami, U. (2006). Becoming literate in different languages: similar problems, different solutions. Developmental Science, 9(5), 429-453. 106 APPENDIX A: INTERVIEW QUESTIONS FOR PROFESSIONALS 1. Please indicate how long have you been practicing and tell me a little more about your practice? 2. Please have a look at some stories from the Neale Analysis of Reading Ability. Do you feel that this assessment tool is appropriate and suitable for the South African context and indicate what you think of it as a measure? 3. Could you compare this assessment tool to the reading tasks that are given to learners in school? What are the similarities and differences between the two? 4. Please could you provide details of the type of errors your learners make in reading? 5. Do you assess reading skills? If no, what are the reasons? If yes, what assessment tools do you make use of to assess reading skills? Please list. 6. Within the assessment tools that you use, which of these are South African tools? Please list. 7. Do you feel reading assessments are useful for learners? Please comment further. 8. How would a learner benefit from a reading assessment? 9. Are reading assessments affordable to parents? Please comment further. 10. What in your opinion is the best method to teach reading? 11. Do we have adequate tools for the South African context? Please comment further. 12. What do we require within the South African context as professionals assessing reading skills? 13. Can reading skills in learners be improved? Please comment further. 14. Do international assessment tools provide adequate information when assessing reading skills in South African children? Please comment further. 15. Please have a look at the list of words and story titles from the NARA II being handed to you and tick which of these are appropriate to be used with children in South Africa. 107 List of words: Toys buckled expeditions Box diving belt uncanny Play metal weights Pursuers bed-time launch mine-shaft tree-house skipper Surmount Rope air-hose Hounds Climb tangling Territory no-one bubbles Escaped tea-time dinghy Haunts space-ships diver swallows (type of bird) Bird mysterious bird-watchers Hopped underwater world Abodes Bread clutching Migrate Nest crayfish Fledglings Traffic specimens Offspring Bicycles shark inborn behaviour Crashed retreated Storks Quickly assistance Ancestors Television creature Camera rocky road safety welfare Sheltered fox Temple rival Secret cunning underground room suspicious Jewels enemy desert travellers feats Imagined extraordinary Escape alighting Amazement sheep Palace cub-rearing Buried hunting 108 Story titles: Story title Please tick if appropriate in the South African context Toys (A story about a child who has toys in a box) Tree-house (A story about a child and his/her friend who made a tree-house and played in it) Bird (A story about a bird who hopped on the window and made a nest in the garden) Road Safety (A story about a road safety lesson that was filmed) Ali (A story about a boy who got trapped in an old temple which had an underground room where he found jewels) Jan (About a girl who went diving) The Fox (the life and survival skills of a fox) Migration (A story about how birds migrate) What themes/story titles should be developed that is appropriate for learners in South Africa? Please provide six appropriate themes/stories: a) ___________________________________________________________ b) ___________________________________________________________ c) ___________________________________________________________ d) ___________________________________________________________ e) ___________________________________________________________ f) ___________________________________________________________ 109 APPENDIX B: LETTER TO PARENTS/GUARDIANS Psychology School of Human & Community Development Private Bag 3, Wits 2050, South Africa. Telephone: +27 11-717-4500/2/3/4. Fax: +27-11-717-4559 Dear Parent(s)/ Guardian (s), Hello! My name is Mrs. Hansini Dhana-Dullabh and I am a Psychology Masters student at the University of the Witwatersrand. As part of my studies, I am required to undertake a research project. The reason for this letter is to ask your permission for your child to participate in my research project. My research involves determining the applicability of a UK reading test for South African children. My research involves Grade 4 learners in public schools. The knowledge from the research will hopefully benefit how we teach reading to students in South Africa and contribute to what is required in South Africa in terms of reading assessment tools. Each learner will be assessed using this assessment tool in one session of approximately 20-30 minutes on an individual basis at the school, during a time that does not interfere with your child’s academic activities. I am inviting your child to participate in this study. Participation is completely voluntary. Whether you give permission for your child to take part in the study or not will not affect their academic teaching or evaluation in any way. However, the school is aware of the project and has given their permission for the study to be conducted. If you give permission for your child to be assessed I will still ask for their agreement before I begin the procedure. If you/your child wish to withdraw from the study at any point you are free to do so. If you allow your child to participate, please complete and sign the form below, place it in the self- addressed envelope provided, seal the envelope and return this to your child’s class teacher by Friday 27th July 2012. Please be assured that all data collected will be kept strictly confidential. No child will be identified in any written or spoken report. A 1-2 page summary on group trends will be given to each parent whose child participated in the research and the school. No individual feedback will be given to parents but should any vulnerability exist, the child will be referred to the relevant professionals. Should you have any questions or if you wish to request 110 feedback, my contact details together with those of my supervisors appear in the signatures below. Thank you for taking the time to read this and for considering letting your child/ward participate in my study. Yours Sincerely, 111 Consent Form I,____________________________do/do not consent for my child ______________________________(name of child) in Grade____at __________________(school’s name) to participate in the research study to be conducted by Hansini Dhana-Dullabh. I am aware that all details will be kept confidential at all times and that my child’s participation or non-participation in the study will have no impact on his/her academic input and evaluation. - Participation in this study is completely voluntary. - I will be requested to provide my child’s June Examination marks - No information that may identify my child or me will be included in the research report - My child will not be harmed in any way during the assessment Signed: ____________________ (Signature) Name of Parent/Guardian: _______________________ Contact number of parent/guardian: _______________________ Email address of parent/guardian: _____________________________________________ Date: ________________ If you do consent for your child to participate please fill in the following information. Please be assured that the information that you provide below will be kept confidential at all times. Biographical Questionnaire 1. Name of child: ____________________________________ 2. Date of birth of child: _________________________ 3. Age of child:________________________________ 4. Gender of child: 5. Population Group: (This specific response is required for purposes of this research and is not meant to offend any research participant) If other, please specify: ______________________________________ MALE FEMALE OTHER WHITE INDIAN COLOURED BLACK 112 6. Is English your child’s first language? 7. If not, what is your child’s first/home language(s)? If other, please specify: _______________________________________ 8. How many years has your child been exposed to English? ________________ 9. Does your child have any learning disabilities? If yes, please specify and provide details about how it was diagnosed and when: ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ _____________________________________________________________________ 10. Father’s highest level of education: If post-matric, please specify: _____________________________________________ 11. Mother’s highest level of education: If post-matric, please specify: _____________________________________________ Once you have filled out this form, place this form and a copy of your child’s June report in the self-addressed envelope provided, seal the envelope and return this to your child’s class teacher by Friday 27th July 2012. Thank you kindly. Mrs. H. Dhana-Dullabh NO YES OTHER tshiVENDA seTWANA xiTSONGA siSWATI sePEDI isiNDEBELE AFRIKAANS seSOTHO isiZULU isiXHOSA YES NO Less than Grade 7 Grade 8-11 matric Post-matric Less than Grade 7 Grade 8-11 matric Post-matric 113 APPENDIX C: ASSENT FORM FOR CHILD TO SIGN Participants will be told: “I am interested in how well you can read. We are going to sit together for a little while and you are going to read stories to me. Is that ok with you?” If the participant agrees he/she will need to write his/her name below: Participant’s name: _______________________________ 114 APPENDIX D: LETTER TO PROFESSIONALS AND CONSENT FORM FOR INTERVIEW AND AUDIO RECORDING Psychology School of Human & Community Development Private Bag 3, Wits 2050, South Africa. Telephone: +27 11-717-4500/2/3/4. Fax: +27-11-717-4559 To whom it may concern Re: Permission to interview for my Master’s research My name is Mrs. Hansini Dhana-Dullabh. I am currently completing my Masters in Research Psychology by Dissertation at the University of the Witwatersrand. My research is in the field of education, specifically on reading. I wish to look at the utility of the Neale Analysis of Reading Ability (NARA II) in the South African context. Many professionals (teachers, psychometrists, psychologists and speech therapists) like you make use of this assessment tool and we rely on international tests as we do not have reading tests in South Africa. As part of my study, approximately 100 learners in Grade 4 attending a public school will be assessed on the NARA II. In addition, I would like to interview professionals like yourself in order to have a better understanding on your perceptions of reading, reading ability and reading assessments in the South African context. In addition, there is a list of words and list of stories from the NARA II where you would need to tick which of the words in the list and which of the story titles are appropriate for the South African context. There is space provided for you to suggest six possible themes for stories used on South African children. I would like to invite you to participate in my study. Participation is voluntary and will involve an interview of approximately 30-45 minutes. All information will be treated confidentially. Even though the interviews will be tape-recorded, only my supervisors and I will have access to the tapes and this will be kept in a locked cupboard at the university for three years and 115 destroyed thereafter. You are free to answer only the questions you feel comfortable with and to withdraw any information you provide at any time with no consequence. Should you wish to withdraw from the study at any point you may do so and you will not be discriminated against in any way. There are no benefits or risks associated with participation in this study. Your anonymity will be ensured in that no identifying information about you will be revealed in my report or subsequent publications. You will be referred to by a pseudonym, e.g. participant A or participant B. Should you volunteer to be interviewed please fill out the consent form for audio recording purposes and email it back to me. Once I receive the consent form I will contact you to arrange a time and place for the interview. Please note that a 1-2 page summary on group trends will be emailed to each interviewee. I look forward to hearing from you. Yours Sincerely, 116 I, ____________________________________ hereby consent to being interviewed by Mrs. Hansini Dhana-Dullabh for her research study on the applicability of the Neale Analysis of reading ability in the South African context. In addition, I give/do not give my consent for the interview to be audio recorded. I understand that:  Participation is strictly voluntary.  I do not have to answer all the questions should I choose not to.  I am free to withdraw from the study at any time.  My identity as well as any information I reveal will be confidential.  I will be referred to by a pseudonym (Participant A or Participant B, etc) in the research report and any subsequent presentations or publications.  There are no benefits or risks associated with the study.  The results of the study will be reported in the form of a research report for the partial completion of the degree, Masters in Research Psychology by Dissertation.  The research may also be presented at a local/international conference and published in a journal and/or book chapter.  The recordings will be confidential and only Hansini and her supervisors will have access to them.  After the study has been completed the tapes will be kept in a locked cupboard at the university for three years and destroyed thereafter.  Tapes are kept primarily to facilitate the research being presented or published. Signed: ____________________ (Signature) Name: _________________________________ Please indicate your profession: Contact number: _______________________ Email address: _____________________________________________ Date: ________________ Please email this consent form back to me by ___________________, should you be participating in the interview and research. Thank you kindly. Teacher Psychometrist Psychologist Speech and Hearing Therapist 117 APPENDIX E: HUMAN RESEARCH ETHICS COMMITTEE CLEARANCE CERTFICATE 118 APPENDIX F: PERMISSION FROM THE DEPARTMENT OF EDUCATION 119 120 121 APPENDIX G: LETTER TO SCHOOL PRINCIPALS Psychology School of Human & Community Development Private Bag 3, Wits 2050, South Africa. Telephone: +27 11-717-4500/2/3/4. Fax: +27-11-717-4559 Dear Principal Hello! My name is Mrs. Hansini Dhana-Dullabh. I am currently completing my Masters in Research Psychology by Dissertation at the University of the Witwatersrand. My research is in the field of education, specifically on reading. I wish to look at the utility of a UK-normed reading test in the South African context. Many professionals use this assessment tool and we rely on international tests as we do not have reading tests in South Africa. In order to do this, I will need to test approximately 100 learners in Grade 4 from various public schools. Participation in the study will involve learners being assessed using the Neale Analysis of Reading Ability. I will request consent from the learner’s parent/guardian and I have attached the information letter and consent form to this letter for your information. Parents/guardians who consent for their learner to participate will need to fill out a demographic questionnaire including the learner’s age, home language, number of years the learner is exposed to English, etc. If possible, I would greatly appreciate it if your school assisted in the distribution and the collection of the parental consent forms if you grant me permission to conduct my research at your school. I will provide the copies of the information letters and consent forms for the parents. I will also require approximately 20-30 minutes per learner at a time convenient to you during which to test each learner. In addition to this, I will also be interviewing teachers from Grade 4 and other professionals working with learners such as psychometrists, psychologists or speech and hearing therapists. The interview is based on their perceptions on various aspects related to reading 122 and the UK-normed reading test. Together with this a list of words from the Neale Analysis of Reading Ability and the story titles from the test will be included where they will need to tick which words and story titles they feel are appropriate for the South African context and to provide ten themes that are appropriate for stories in the South African context. I would kindly like to ask your permission to collect data from your school. I have obtained ethical clearance from the University of the Witwatersrand and am in the process of obtaining permission from the GDE. I hope to receive permission by the end of June and will fax you confirmation of this. I would like to collect my data between July and September this year. I would also appreciate it if you could provide the contact details of the psychologist, psychometrist and/or speech therapist that your school works with/refers to. A 1-2 page summary on group trends will be given to each parent whose child participates in the research, the professionals and the schools. No individual feedback will be given to parents but should any vulnerability exist, the child will be referred to the relevant professionals. I would also be very grateful if you made your Grade 4 learners and teachers aware of my research. I would also gladly address the learners and parents at your convenience if required. Should you grant me permission to conduct research; it would be appreciated if you could fax a letter to 0865534913 for the attention of my supervisors, Prof. Kate Cockcroft and Prof. Sumaya Laher confirming this. You may also email the letter or I can come collect it. My contact details together with those of my supervisors appear in the signatures below. I look forward to hearing from you. Yours Sincerely,