Bias in data used to train salesbased decision-making algorithms in a South African retail bank

Abstract

Banks are increasingly using algorithms to drive informed and automated decision-making. Due to algorithms being reliant on training data for the model to learn the correct outcome, banks must ensure that the customer data is securely and fairly used when creating product offerings as there is a risk of perpetuating intentional and unintentional bias. This bias can result from unrepresentative and incomplete training data or inherently biased data due to past social inequalities. This study aimed to understand the potential bias found in the training data used to train sales-based decision-making algorithms used by South African retail banks to create customer product offerings. The research adopted a qualitative approach and was conducted through ten virtual one-on-one interviews with semi-structured questions. Purposive sampling was used to select banking professionals from data science teams in a particular South African retail bank across demographics and levels of seniority. The data collected from the participants in the interviews were then thematically analysed to draw a conclusion based on the findings. Key findings included: An inconsistent understanding across data science teams in a South African retail bank around the prohibition of using the gender variable. This could result in certain developers using proxy variables for gender to inform certain product offerings. A potential gap in terms of the potential usage of proxy variables for disability (due to non-collection of this demographic attribute) to inform certain product offerings. Although disability was not identified as a known biased variable, it did, however, raise the question of whether banks should be collecting the customer’s disability data and doing more in terms of social responsibility to address social inequalities and enable disabled individuals to contribute as effectively as abled individuals. As algorithms tend to generalise based on the majority’s requirements, this would result in a higher error rate of underrepresented groups of individuals or minority groups. This could result in financial exclusion or incorrect products being offered to certain groups of customers iii which, if not corrected, would lead to the continued subordination of certain groups of customers based on demographic attributes.