The social life of questionnaires: exploring respondents' understanding and interpretation of disability measures

Disability statistics are an important component of an informational base to monitor the needs and rights of people with disabilities within all spheres of life. The effective use of disability statistics is based on measures that are transparent and valid and where potential sources of sampling, interviewer or respondent error are clearly understood. This study investigates respondent sources of survey data error generated by three sets of existing questions on disability, as applied to adults in South Africa as a case study contributing to the growing work in this field at an international level. The questions are the Short and Extended sets of the Washington Group on Disability Statistics, and the Global Study on Ageing’s (SAGE) health state description questions used by the World Health Organization. Disability is defined as the outcome of the interaction between a person’s health condition or impairment, and the context in which he or she lives. The question evaluation considers: a) the wording, comparing asking about difficulties and disabilities; b) respondents’ understanding and interpretations of whole questions; and c) the association between various life factors and the type and severity of functional profiles. The study uses a mixed methods design comprising focus groups, semi-structured interviews, in-depth interviews that tapped into the thinking behind respondents’ answers to questions, and standard questionnaires. There were 21 focus groups located nationally, and 69 semi-structured interviews and ten detailed cognitive testing interviews at the Agincourt Health and Population Unit, located in a rural area in north eastern South Africa. The results provide insight into the performance of the questions, and, particularly, the need for revisions of the questions, potential respondent sources of error, and factors in people’s lives that are associated with different response profiles. The analysis of wording shows that asking about ‘difficulty’ people have is a more inclusive and transparent measure than asking about disability. Analysis of respondents’ understanding and interpretation of questions shows that questions on vision, hearing, walking and climbing, self-care and communication are well understood as intended, while the questions on remembering are misinterpreted in similar ways by a high number of respondents. Responses to non-traditional disability domains of pain, affect (anxiety and depression) and fatigue (or problems with sleep) reflect predominantly people’s reactions to living in adverse conditions. The measures provide a way to broaden the notion of who is counted as disabled because of functional limitations for statistical purposes, but the identity of being disabled remains a separate concern. The differences between measuring identity and functional status and the implications of these separate concepts is one area identified as an important focus of future research arising from this thesis. This thesis builds on existing knowledge by: providing strong evidence on the effect of asking about disability versus difficulty; illustrating the importance of question evaluation as part of validity testing and provide a further example to add to the growing literature on this approach; providing evidence on how responses to basic activity domains differ to those given to questions on feeling domains and the implications of this for how disability is understood in a low income and resources context of rural South Africa.