THE IMPACT OF AUDIT AUTOMATION ON THE INFORMATION TECHNOLOGY (IT) AUDIT PROFESSION (A CASE OF A SOUTH AFRICAN AUDIT FIRM) by LYDIA DLAMINI (929481) SUPERVISOR: DR THEMBEKILE MAYAYISE Research Report Submitted in partial fulfilment of the requirements for the degree of Master of Commerce (Information Systems) in the School of Business Sciences, Faculty of Commerce, Law, and Management, University of the Witwatersrand Date of submission 27 FEBRUARY 2025 i ii TABLE OF CONTENTS 1 CHAPTER ONE – INTRODUCTION ........................................................................................ 1 1.1 BRIEF BACKGROUND TO THE TOPIC ........................................................................... 1 1.2 PROBLEM STATEMENT ................................................................................................... 2 1.3 RESEARCH QUESTIONS AND OBJECTIVES ................................................................. 3 1.4 SIGNIFICANCE OF THE STUDY ...................................................................................... 3 2 CHAPTER TWO – LITERATURE REVIEW ............................................................................ 4 2.1 INTRODUCTION ................................................................................................................. 4 2.2 IT AUDITING ....................................................................................................................... 4 2.3 AUDIT AUTOMATION ...................................................................................................... 5 2.4 IMPACT OF AUDIT AUTOMATION IN IT AUDITING .................................................. 5 2.5 IMPACT OF DATA ON THE USE OF AUTOMATED IT AUDITING PROCESSES ..... 6 2.6 GAPS IN THE LITERATURE REVIEWED ....................................................................... 8 2.7 THEORETICAL AND CONCEPTUAL FRAMEWORK ................................................. 17 2.7.1 RESEARCH MODEL/FRAMEWORK ...................................................................... 19 3 CHAPTER THREE – RESEARCH METHODOLOGY .......................................................... 24 3.1 INTRODUCTION ............................................................................................................... 24 3.2 RESEARCH PHILOSOPHY (PARADIGM) AND APPROACH ..................................... 24 3.3 RESEARCH DESIGN AND STRATEGY ......................................................................... 25 3.4 POPULATION AND SAMPLE ......................................................................................... 25 3.4.1 POPULATION............................................................................................................. 25 3.4.2 SAMPLING METHOD ............................................................................................... 25 3.5 RESEARCH INSTRUMENT ............................................................................................. 26 3.6 PROCEDURE FOR DATA COLLECTION ...................................................................... 26 3.7 DATA ANALYSIS AND INTERPRETATION ................................................................ 27 3.8 RIGOUR .............................................................................................................................. 29 3.9 ETHICAL CONSIDERATIONS ........................................................................................ 30 4 CHAPTER FOUR – PRESENTATION OF THE FINDINGS ................................................. 31 4.1 INTRODUCTION ............................................................................................................... 31 4.2 CONTEXT OF THE STUDY ............................................................................................. 31 4.3 THEMES PRESENT AND FINDINGS OF THE STUDY ................................................ 33 5 CHAPTER FIVE – DISCUSSION OF THE FINDINGS ......................................................... 57 5.1 INTRODUCTION ............................................................................................................... 57 5.2 CHAPTER CONCLUSION ................................................................................................ 62 iii 6 CHAPTER SIX – CONCLUSION AND RECOMMENDATIONS ......................................... 64 6.1 CONCLUSION ................................................................................................................... 64 6.2 RESEARCH LIMITATIONS ............................................................................................. 64 6.3 RECOMMENDATIONS FOR FUTURE RESEARCH ..................................................... 65 7 REFERENCES........................................................................................................................... 66 8 APPENDIX A: RESEARCH INSTRUMENT ............................................................................... 68 9 APPENDIX B: CONSISTENCY TABLE ................................................................................. 69 10 APPENDIX C: CONSENT FORM ........................................................................................... 71 11 APPENDIX D: ETHICS CLEARANCE ................................................................................... 72 12 APPENDIX E: TURNITIN REPORT SUMMARY ................................................................. 73 iv THE IMPACT OF AUDIT AUTOMATION ON THE INFORMATION TECHNOLOGY (IT) AUDIT PROFESSION (A CASE OF A SOUTH AFRICAN AUDIT FIRM) ABSTRACT Organisations are moving towards the automation of business processes to meet the demands of stakeholders. To meet the frequency and expectations of businesses, IT audits should be automated to ensure that organisation’s auditing/accounting practices, risk controls, compliance and information systems, and business controls are reviewed on an ongoing basis. Additionally, the IT profession is evolving as a result of the audit automation. This qualitative case study explores the impact of audit automation on the IT audit profession, focusing on how automation influences their day-today responsibilities and competencies. The study follows an interpretivism paradigm where semi-structured interviews were conducted with 21 IT auditors from a South African audit firm. The interview questions were derived from a model built from the systematic literature review conducted for the study and the affordance theory adopted for the use of Information Systems research. The data was analysed using thematic coding, which identified key themes such as data as an enabler, continuous auditing and continuous monitoring, audit automation and the role of the IT auditor. Findings indicate that although the IT audit role is evolving, it will not be fully replaced by automation anytime soon, IT auditors will work together with audit automation tools to act as information certifiers of the results from the audit automation tools. Their key role will be to apply their professional judgement and professional scepticism which audit automations tools currently lack. The study concludes that IT auditors and audit firms should invest in training and upskilling to ensure that they have the relevant skills to work together with audit automation tools. This study contributes to the understanding of how IT auditors perceive and adapt to automation and the evolving responsibilities, offering insights to practitioners and audit firms on how they can upskill and provide the relevant training to IT auditors. Future research should focus on the role of artificial intelligence on the automation of the audit profession and the capabilities and impacts that AI has on the audit profession. Keywords: IT Audit, IT audit profession, automation, affordance theory, technology adoption theories, audit automation, auditing, process automation, artificial intelligence 1 1 CHAPTER ONE – INTRODUCTION 1.1 BRIEF BACKGROUND TO THE TOPIC Organisations are moving towards the automation of business processes to meet the demands of stakeholders. To meet the frequency and expectations of businesses, IT audits should be automated to ensure that organisation’s auditing/accounting practices, risk controls, compliance and information systems, and business controls are reviewed on an ongoing basis (Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). To ensure the advancement of audit in organisations, their audit procedures must be automated (Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Traditional audits are audits where there is intensive human labor without the assistance of audit automation tools or heavy reliance on any tools (Eulerich, Masli, Pickered, & Wood, 2023; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Automated audits are an improvement from traditional audits as 1) well-defined and repetitive tasks are completed in less time (Frey & Osborne, 2017; Huang & Vasarhelyi, 2019; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021) 2) Continuous auditing and continuous monitoring are driven by audit process automation (Huang & Vasarhelyi, 2019) and enabled by data (Frey & Osborne, 2017; Huang & Vasarhelyi, 2019; Pizzi, Venturelli, Variale, & Macario, 2021), 3). Most risks that impact the organisation such as fraud risk and security risks are mitigated accordingly with early detection from the automation driven by the use of data (Setty & Rohit, 2013; Titera, 2013), 4) Fraud risk in organisations is mitigated by the use of automation and data analytics to proactively detect or identify fraud before it happens or as it occurs (Appelbaum, Kogan, & Vasarhelyi, 2017; Eulerich, Masli, Pickered, & Wood, 2023; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021) 5) Furthermore, there are fewer human errors as there is limited professional judgment required due to the reliance of automation and resulting reports (Huang & Vasarhelyi, 2019; Zhang, Thomas, & Vasarhelyi, 2022). Therefore, the study's purpose is to explore and understand the perceptions and experiences of IT auditors on IT audit automation, examine concerns, challenges, and benefits. This helps understand what the impact of automation is on the IT audit profession, what the role of the IT auditors is, how this benefits audit firms in South Africa and explore emerging innovations and trends because of IT audit automation. 2 1.2 PROBLEM STATEMENT Business process automation is the computerisation or automation of complex business processes or procedures using advanced technologies (Gartner, 2022). An IT Audit is the review of information systems to provide assurance or advice on the design and adequacy of the information systems reviewed and their operating effectiveness over a specified period (Deloitte, 2022). The purpose of IT audits is to ensure that there are controls in place to mitigate risks in the IT environment and that those controls mitigate the inherent risk such that the residual risk (remainder of risk) is at its minimum (Deloitte, 2022). Process automation in IT audit entails the automation of audit procedures such that well-defined and repetitive tasks are automated (Huang & Vasarhelyi, 2019). IT audit mainly uses robotic process automation (RPA) to automate these well- defined and repetitive audit procedures (Huang & Vasarhelyi, 2019; Zhang, Thomas, & Vasarhelyi, 2022). The earliest form of the automation of audit procedures was identified in the beginning of the 1990’s and was coined as electronic data processing (EDP) (Kogan, Sudit, & Vasarhelyi, 1999; Vasarhelyi, 1993). As a result, process automation in audits has been applied in all types of auditing including IT audits in both an external and internal audit point of view where external audits focus on the finances of the organisation per statutory requirements while internal audits focus on the operations of the organisation (Deloitte, 2022). Based on the reviewed literature covering a three decades of research (Rowe, 2014), no recent studies have been conducted on IT audits in the African region. While some studies were empirical, (Chan & Vasarhelyi, 2011; Kogan, Sudit, & Vasarhelyi, 1999; Pizzi, Venturelli, Variale, & Macario, 2021; Rikhardsson & Dull, 2016; Vasarhelyi, 1993; Zhang, Thomas, & Vasarhelyi, 2022) a few were conceptual, and this study aims to narrow the gap and make a methodological and contextual contribution through the empirical case study undertaken in the African context. Furthermore, the studies have highlighted the advantages of audit process automation such as frequent reviews with a proactive approach where volumes of information can be audited continuously (Chan & Vasarhelyi, 2011; Kogan, Sudit, & Vasarhelyi, 1999; Pizzi, Venturelli, Variale, & Macario, 2021; Rikhardsson & Dull, 2016; Vasarhelyi, 1993; Zhang, Thomas, & Vasarhelyi, 2022)and have also highlighted all the requirements that should be met to enable this automation (Alles & Gray, 2020; Chan & Vasarhelyi, 2011; Huang & Vasarhelyi, 2019; Kogan, Sudit, & Vasarhelyi, 1999; Pizzi, Venturelli, Variale, & Macario, 2021; Vasarhelyi, 1993; Zhang, Thomas, & Vasarhelyi, 2022). However, it was noted there are limited studies on the impact of audit automation on IT auditors or the IT audit profession. Problem Statement Despite the extensive research on business process automation and its application in IT audits, there is a notable gap in recent empirical studies focusing on the African region, particularly concerning the impact of audit automation on IT auditors and the IT audit profession. This study aims to address this gap by exploring the methodological and contextual implications of audit process automation in the African context, thereby contributing to the understanding of how IT audit process automation can enhance audit procedures and the IT audit profession's evolution. 3 1.3 RESEARCH QUESTIONS AND OBJECTIVES Purpose Statement The purpose of this research is to explore the impact of the automation of audit procedures in IT audit or the IT audit profession by focusing on the following objective: To determine the impacts of IT audit automation on IT audit/IT audit profession. Research Questions Based on the research objectives this research aims to address the following research questions: What impact does the automation of IT audit processes have on IT auditors/IT audit profession? The sub-research questions that aim to answer the main research question are as follows: Q1: How does data (data access, data analytics/data modelling and big data) affect the application of automation in IT auditing? Q2: How does the IT audit process automation affect the IT audit profession in IT auditing? Q3: What is the effect of affordances as a result of audit automation on the IT audit process in IT auditing? Q4: What is the effect of affordances as a result of audit automation on the IT audit profession? 1.4 SIGNIFICANCE OF THE STUDY The study helps to explore the impact of audit automation on IT auditors as most studies have highlighted the advantages of audit process automation such as frequent reviews with a proactive approach where volumes of information can be audited continuously (Chan & Vasarhelyi, 2011; Kogan, Sudit, & Vasarhelyi, 1999; Pizzi, Venturelli, Variale, & Macario, 2021; Rikhardsson & Dull, 2016; Vasarhelyi, 1993; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012) and have also highlighted all the requirements that should be met to enable this automation (Alles & Gray, 2020; Chan & Vasarhelyi, 2011; Huang & Vasarhelyi, 2019; Huerta & Jensen, 2017; Rikhardsson & Dull, 2016; Pizzi, Venturelli, Variale, & Macario, 2021). However, it was noted there are limited studies on the impact of audit automation on IT auditors or the IT audit profession in the South African context. Furthermore, there are limited studies on the value of IT audit, the effort required to perform full-scope audits with limited resources. Therefore, this study explores the relevant theories and/or models to understand the impact of the automation of IT audit on the IT audit profession. This also helps understand the impact of using audit automation and how it can empower audit firms to best leverage the capabilities of IT audit automation. 4 2 CHAPTER TWO – LITERATURE REVIEW 2.1 INTRODUCTION This chapter outlines the review of the related literature for the study and the theoretical models and frameworks. The chapter outlines what IT audit entails, what audit automation is and what the role of the IT auditor looks like. Furthermore, continuous auditing and continuous monitoring are covered as part of impacts of audit automation on IT audit. Additionally, data as an enabler of audit automation is discussed. Details of the systematic literature review and gaps are discussed. Additionally, a conceptual framework developed as part of the systematic literature review, information systems theories and the affordance theory are discussed. The review included peer reviewed journal articles that had a full text available. The studies also had to be in the English language. The studies also had to be thirty years old as, (Rowe, 2014) states that going back to 30 years is beneficial when conducting a systematic literature review where first pieces of the research are included. The study should contain automation and auditing to be included. The review excluded any research that was conducted before 1993. Additionally, it excluded work that was not peer reviewed and without full text available for download/extraction. Furthermore, work without any references was also excluded from the literature review. 2.2 IT AUDITING IT Audit is the review of information systems to provide assurance or advice on the design and adequacy of the information systems reviewed and their operating effectiveness over a specified period (Deloitte, 2022). In traditional audits, IT auditors perform the testing and reporting, where audit procedures are labour intensive and therefore, samples are utilised for the testing (Chan & Vasarhelyi, 2011; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012; Zhang, Thomas, & Vasarhelyi, 2022). Audit process automation may result in the automation of most audit processes, however, it should be noted that the following tasks cannot yet be automated 1) perception and manipulation tasks, where the tasks consists of an unstructured work environment; 2) Creative intelligence tasks where creativity is required to perform tasks and; 3) social intelligence tasks, where negotiation, persuasion and care are required to carry out the tasks (Frey & Osborne, 2017; Huang & Vasarhelyi, 2019; Zhang, Thomas, & Vasarhelyi, 2022). This means that audit tasks such as being innovative or creative to make daily audit tasks better cannot be automated. Additionally, the auditing of unstructured data cannot be automated. Furthermore, social intelligent tasks such as negotiation and persuasion (used during negotiations when findings of audits are presented to process owners) cannot be automated. Therefore, auditors can work together with audit automation tools to ensure efficiency in the audit process (Kogan, Sudit, & Vasarhelyi, 1999; Huerta & Jensen, 2017). Automation can be used for repetitive tasks that do not require human judgement and for pattern recognition in proactive auditing (Huang & Vasarhelyi, 2019). Furthermore, auditors can focus on tasks that require professional judgement (Huang & Vasarhelyi, 2019; Zhang, Thomas, & Vasarhelyi, 2022), additionally, they can upskill themselves to obtain data analytical skills (Huerta & Jensen, 2017) where they play the role of an 5 information worker/data certifier who analyses results from audit automation tools and give audit assurance to the organisation. 2.3 AUDIT AUTOMATION The earliest form of automation in auditing was identified in the 90’s and was referred to as the electronic data processing (EDP) (Kogan, Sudit, & Vasarhelyi, 1999; Vasarhelyi, 1993). Since then, audit automation has matured and has been applied in the auditing field for the automation of audit procedures. The automation of audit processes has resulted in the emergence of audit concepts such as continuous auditing (CA) and continuous monitoring (CM), where the auditing/accounting practices, risks and controls, compliance monitoring, information systems auditing, and business controls evaluations are reviewed on an ongoing basis (Appelbaum, Kogan, & Vasarhelyi, 2017; Eulerich, Masli, Pickered, & Wood, 2023; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021). The use of robotic process automation (RPA) has also been adopted in audit process automation, where well-defined, repetitive tasks are automated (Huang & Vasarhelyi, 2019). Furthermore, data analysis, data modelling, big data, data availability for robotic process automation and business intelligence were identified as enablers of the automation of audit processes, these were unpacked further in section 2.5. 2.4 IMPACT OF AUDIT AUTOMATION IN IT AUDITING Automation of tasks requires that they should be well-defined and repetitive (Frey & Osborne, 2017; Huang & Vasarhelyi, 2019), they should also be easily programmable (Frey & Osborne, 2017). However, it has also been noted that non-routine cognitive tasks can be replaced by automation (Alles & Gray, 2020; Frey & Osborne, 2017; Zhang, Thomas, & Vasarhelyi, 2022). Big data is used, where data algorithms are employed to replace tasks that require the storage and access of information and pattern recognition (Frey & Osborne, 2017). Audit has a combination of procedures that are well-defined, repetitive, and non-routine cognitive tasks, and it also relies on data storage and data access of information where pattern recognition is used in the proactive audit approach. This therefore means that automation of audit processes could pose a risk of replacing the audit profession. Continuous auditing and continuous monitoring have evolved the way in which traditional audits are performed as they are a result of audit automation (Chan & Vasarhelyi, 2011; Kogan, Sudit, & Vasarhelyi, 1999; Pizzi, Venturelli, Variale, & Macario, 2021; Rikhardsson & Dull, 2016; Vasarhelyi, 1993). Traditional audits are performed on a periodic basis, e.g., yearly; the audit approach is reactive where issues are picked up after they have materialised; the audit procedures used are manual in nature; the role of auditors entails manual labour where samples are selected from the full population to conduct testing; the results of the tests done by auditors are also reported periodically when the audit happens (Chan & Vasarhelyi, 2011; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). In contrast to traditional audits, continuous audits happen on a frequent basis, e.g., monthly, weekly, daily, or real time depending on the needs of the organisation; the audit approach is proactive where potential issues are picked up before they materialise; audit procedures are automated; data analytics and data modelling are used for testing where full populations are mostly 6 used for testing; the results of the testing are reported on a frequent basis as per the organisation’s needs (Chan & Vasarhelyi, 2011; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Based on the above, it can be deduced that automation is a driver for continuous auditing and that the automation of audit processes brings rise to benefits in the audit process. The benefits include cost reduction on audit engagements as less auditors are needed on audits to carry out testing (Rikhardsson & Dull, 2016), additionally there is high productivity as data modelling and data analytics are used to carry out audits instead of intensive manual labour (Huang & Vasarhelyi, 2019; Kogan, Sudit, & Vasarhelyi, 1999; Rikhardsson & Dull, 2016). Furthermore, continuous audits are proactive, meaning picking up issues before they materialise is cost saving to the organisation and operational and reputational risks are addressed before they have a significant impact on the organisation (Alles & Gray, 2020; Chan & Vasarhelyi, 2011; Pizzi, Venturelli, Variale, & Macario, 2021; Rikhardsson & Dull, 2016). Automated audits also mitigate the risk of human error (Alles & Gray, 2020; Caster, Elder, & Janvrin, 2021; Chan & Vasarhelyi, 2011; Huang & Vasarhelyi, 2019; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012), the automated nature of audit processes means that populations or samples that are tested, are tested the same way all the time, unlike human based tests where there may be inconsistencies and errors on some of the tests performed. Although there are benefits realised from automating audit processes, it was noted that there are also some disadvantages. Auditors must be familiar with the use of audit automating tools (Chan & Vasarhelyi, 2011; Huerta & Jensen, 2017) which means that organisations must invest in the training and development of their audit staff which might increase costs. Additionally, the automation of audit processes may lead to limited interaction with the auditee, where audit results are automatically sent to the auditee without human interaction which may lead to a disconnect between the auditee and audit department (Alles & Gray, 2020; Kogan, Sudit, & Vasarhelyi, 1999). Although automation has a lot of advantages within the audit field, it is believed that human interaction would still be required (Alles & Gray, 2020; Kogan, Sudit, & Vasarhelyi, 1999), which means that auditors would have to use data as an enabler of automation to better their jobs or audit tasks. The possibility of automating audit procedures is also driven by the availability of data for automation (Alles & Gray, 2020; Huang & Vasarhelyi, 2019; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021). 2.5 IMPACT OF DATA ON THE USE OF AUTOMATED IT AUDITING PROCESSES Access to appropriate data sets that are stored electronically are required for audit process automation (Huang & Vasarhelyi, 2019; Kogan, Sudit, & Vasarhelyi, 1999; Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Audit automation is considered matured when there are advances in the data quality and there is complete data access in an audit data warehouse (Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Furthermore, data collection must be standardised to avoid having to manually clean data before it is usable for auditing, the manual labour for cleaning data will decrease the efficiencies and benefits realised from automated audit processes (Chan & Vasarhelyi, 2011). Additionally, data 7 must be compatible with software that will be used for automation, and it must be structured to avoid high error rates and processing costs (Huang & Vasarhelyi, 2019). Data modelling and data analytics have long been utilised on traditional audits where statistical techniques were used for analytical procedures (Chan & Vasarhelyi, 2011; Dzuranin & Malaescu, 2016; Titera, 2013). However, in automated audits, data modelling and data analytics techniques are utilised in the form of data mining and machine learning techniques for testing predictive controls in the proactive audit approach (Chan & Vasarhelyi, 2011; Pizzi, Venturelli, Variale, & Macario, 2021). It was noted that an increase in the data quality and integrity used for audit automation, resulted in more the confidence placed on the data for decision making and reporting in audits which added to the quality and efficiency of reporting (Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Big data has been identified as one of the drivers of audit process automation (Alles & Gray, 2020; Caster, Elder, & Janvrin, 2021; Frey & Osborne, 2017; Huang & Vasarhelyi, 2019; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021), however, the use of big data also brings rise to other concerns (Huerta & Jensen, 2017). One of the concerns is that auditors may assume that results deduced from the use of big data are free of bias which may be incorrect as cognitive biases of the decision maker cannot be eliminated, additionally, system biases may be built into the system used for big data (Huerta & Jensen, 2017). It should be noted that these biases are not limited to big data, they may also be applicable in data analysis and data modelling. Furthermore, the use of data for automation has been deemed to pose the risk of replacing the audit profession (Frey & Osborne, 2017; Huerta & Jensen, 2017), this was discussed on section 2.3 of this research report. 8 2.6 GAPS IN THE LITERATURE REVIEWED This section discusses the literature reviewed for this study and the gaps identified in the literature review. A systematic literature review was performed to review prior studies that have been conducted on IT audit automation and its impact on the IT audit profession. Table 1 provides a summary of the selected studies based on the attributes; reference, context of the study, purpose of the study, methods used, demographics/level of study, concepts examined, key findings, and limitations. It should be noted that some of the studies identified are not limited to IT audit, however, their concepts can also be applied to IT auditing. Table 1: Summary of studies identified from the systematic literature review performed for this research. Reference Context Purpose of study Methods used Demographics/ Level of study Variables/ concepts examined Key Findings Limitations (Vasarhelyi, 1993) The implementation of the (ADAPT) audit automation tool. Describing the implementation of the ADAPT audit automation tool. • Interpretivist • Inductive • Qualitative • Descriptive Audit functions (both external and internal) • Continuous Auditing / Monitoring • Audit Process Automation Continuous auditing may be a result of the use of audit automation tools. Limited to the ADAPT tool for audit automation. (Kogan, Sudit, & Vasarhelyi, 1999) Research on online continuous auditing. Exploring the effects of continuous online auditing. • Interpretivist • Inductive • Qualitative • Descriptive Continuous online auditing (as an IT artefact) • Continuous Auditing/ Monitoring • Audit Process Automation • Data access • Data analytics/ modelling • IT audit profession automation • Role of auditor As technology advances, the frequency of audits may also increase at lower costs. Limited to the automation of continuous auditing online. Automation is also not limited IT audit it covers the broad audit. (Chan & Vasarhelyi, 2011) Practice of continuous auditing. Exploring continuous online auditing • Interpretivist • Inductive • Qualitative Continuous auditing (as an IT artefact) • Continuous Auditing/ Monitoring Automation of audits has led to the improvement of Limited to audit automation as a dimension of 9 Reference Context Purpose of study Methods used Demographics/ Level of study Variables/ concepts examined Key Findings Limitations as an audit innovation. • Descriptive • Audit Process Automation • Data analytics/ modelling • Role of auditor traditional audits and has enabled continuous auditing. continuous auditing. (Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012) The acceptance and adoption of continuous auditing by internal auditors. Explore effects of continuous auditing in small businesses. • Interpretivist • Inductive • Qualitative • Interviews Internal audit managers and internal audit staff in 9 internal audit organisations • Continuous Auditing/ Monitoring • Audit Process Automation • Data access • Data analytics/ modelling Some organisations have already adopted continuous auditing while others are looking for more advanced technologies for their audits. Limited to the adoption and acceptance of continuous auditing by internal auditors. (Titera, 2013) Application of data analytics on audit standards Explains the role and value of data analysis in the audit process. • Interpretivist • Qualitative • Explanatory American Institute of Certified Public Accountants (AICPA) board • Continuous Auditing/ Monitoring • Audit Process Automation • Data analytics/ modelling The study recommends a detailed analysis and provides guidance to changing audit standards for data analysis. Limited to the American Institute of Certified Public Accountants (AICPA) board. (Dzuranin & Malaescu, 2016) Challenges and opportunities in IT audit. Exploring emerging topics in IT audit. • Interpretivist • Qualitative • Exploratory Academic researchers and practitioners • Continuous Auditing/ Monitoring • Audit Process Automation • Data analytics/ modelling Outlines concerns facing auditors regarding the use of big data. Limited to the big data and audit analytics in the external audit context. 10 Reference Context Purpose of study Methods used Demographics/ Level of study Variables/ concepts examined Key Findings Limitations (Rikhardsson & Dull, 2016) Impact of continuous auditing in small businesses. Exploring the factors that influence the adoption of continuous auditing in small businesses. • Interpretivist • Inductive • Qualitative • Observation Auditing functions within small organisations • Continuous Auditing/ Monitoring • Audit Process Automation • Data access • Data analytics/ modelling Continuous auditing is usually seen as solution for data quality issues and inefficiencies as opposed to being a technology strategy aligned solution. Limited to the adoption and use of continuous auditing in small businesses. (Huerta & Jensen, 2017) Big data and data analytics as an audit automation enabler. Exploring the effects of applying data analytics for audit automation. • Interpretivist • Inductive • Qualitative • Exploratory Academics and practitioners • Audit Process Automation • Data analytics/ modelling • Big data • Role of auditor Automation enabled by big data may bring rise to new career opportunities in the audit space. Limited to big data as an enabler of automation in financial audits. (Frey & Osborne, 2017) The future of professions and their susceptibility due to computerization. Examines the susceptibility of professions because of automation. • Positivist • Deductive • Quantitative • Relational 702 Occupations/ professions • Data analytics/ modelling • Big data • IT audit profession automation • Role of auditor Automation is no longer limited to well-defined repetitive tasks, with big data as an enabler, it can now be applied on non-cognitive tasks where pattern recognition can be used to replace occupations that consist of non-routine cognitive tasks. Limited to predictions regarding tasks/ equipment that can be operated using automation and their expectation to perform. 11 Reference Context Purpose of study Methods used Demographics/ Level of study Variables/ concepts examined Key Findings Limitations (Huang & Vasarhelyi, 2019) Framework for using robotic process automation in audit. Examines the application of robotic process automation in auditing. • Interpretivist • Inductive • Qualitative • Exploratory Robotic process automation (RPA) (as an IT artefact) • Audit Process Automation • Data access • Data analytics/ modelling • Big data • IT audit profession automation • Role of auditor Insights on human and technology related issues were discovered where cost saving was also identified as a benefit of audit automation. Limited to the use of robotic process automation in the confirmation process for financial audits. (Alles & Gray, 2020) Framework for understanding the automation of audit processes. Examines the automations of the audit practice. • Interpretivist • Inductive • Qualitative • Exploratory Audit automation (as an IT artefact) • Audit Process Automation • Data analytics/ modelling • Big data A framework for technology as an input in audit processes was developed. Limited to the automation of financial audit processes. (Pizzi, Venturelli, Variale, & Macario, 2021) An analysis of impacts on internal audit due to digital transformation. Analyses the impacts of the automation of audit. • Interpretivist • Inductive • Qualitative • Exploratory Digital transformation (as an IT artefact) • Continuous Auditing/ Monitoring • Audit Process Automation • Data analytics/ modelling • Big data The following research areas were highlighted as part of the study; continuous auditing, data analytics, fraud detections and technological innovation. Limited to audit automation literature that was reviewed at the time and excludes conference proceedings as part of the audit automation literature review. (Caster, Elder, & Janvrin, 2021) Investigates the effects of automation on • Interpretivist • Inductive • Qualitative • Longitudinal dataset from the largest • Audit Process Automation • Big data Automation of the audit confirmation Limited to the automation of the bank 12 Reference Context Purpose of study Methods used Demographics/ Level of study Variables/ concepts examined Key Findings Limitations the confirmation process. • Exploratory • Longitudinal study supplemented by interviews • Secondary data third-party U.S. confirmation service provider • Practitioners process results in less errors by auditors. confirmation process. (Zhang, Thomas, & Vasarhelyi, 2022) Attended Process Automation in Audit: A Framework and A Demonstration Exploring auditor’s roles in RPA enabled audits. • Exploratory • Qualitative • APA Framework demonstration • Role of auditor • Audit Process Automation Professional judgement is not replaceable by automation. Limited to external audit functions. (Eulerich, Masli, Pickered, & Wood, 2023) The Impact of Audit Technology on Audit Task Outcomes: Evidence for Technology- Based Audit Techniques Explore impacts of audit technology on audit task outcomes. • Interpretivist • Qualitative • Interview • Interviews with Chief Audit Executives (CAEs) • Audit Process Automation • Data Analytics Use of technology audits results in efficiencies and higher costs. Limited to internal audit functions. 13 The systematic literature review performed revealed that several studies have identified continuous auditing (CA) and continuous monitoring (CM) as a benefit of audit automation (Chan & Vasarhelyi, 2011; Kogan, Sudit, & Vasarhelyi, 1999; Pizzi, Venturelli, Variale, & Macario, 2021; Rikhardsson & Dull, 2016; Vasarhelyi, 1993; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). CA and CM are results of audit automation where the frequency of auditing is increased (Chan & Vasarhelyi, 2011; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). CA and CM as a result of audit automation have changed the way in which traditional audits are performed, where audits are performed frequently instead of periodically, audit procedures are automated instead of manual, full population testing is performed instead of sample selection and reporting also happens frequently (Chan & Vasarhelyi, 2011; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Based on the reviewed literature it can be noted that audit process automation is the main driver of CA and CM and brings rise to many benefits in the IT audit process such as cost reduction in running IT audit engagements (Rikhardsson & Dull, 2016), reduced human error as IT auditors are not performing a lot of manual tasks (Alles & Gray, 2020; Caster, Elder, & Janvrin, 2021; Chan & Vasarhelyi, 2011; Huang & Vasarhelyi, 2019) and audits are proactive where risks are identified before they materialise (Chan & Vasarhelyi, 2011; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). The literature also revealed that the automation of IT audit processes is enabled by data and this data has to be accessible and stored electronically (Huang & Vasarhelyi, 2019; Kogan, Sudit, & Vasarhelyi, 1999; Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). The literature also revealed that for the audit automation process to be mature, there should be advances in quality of data used for the automation process (Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). The literature further revealed that data analytics and data modelling were used before the concept of audit automation was introduced where traditional statistical procedures were used for audit processes requiring analysis (Chan & Vasarhelyi, 2011). However, it has been noted that in advanced data analytical audit methods, techniques such as data mining and machine learning are used to aid the process of IT audit automation (Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Furthermore, it has been identified that big data has also significantly contributed to the audit automation process where different types of data (structured and unstructured) can be accessed to extract value adding business and IT audit insights (Alles & Gray, 2020; Caster, Elder, & Janvrin, 2021; Frey & Osborne, 2017; Pizzi, Venturelli, Variale, & Macario, 2021). Frey and Osborne (2017) predicted that there is a 94% probability that the audit profession will be computerised/automated and subsequent to the study other researchers have implied to agree with this prediction (Huang & Vasarhelyi, 2019). This prediction was made on a basis that automation is no longer limited to repetitive and well-defined task, as big data can now be used to automate non-routine cognitive tasks where algorithms are applied to replace tasks that encompass storage and access of information/knowledge and pattern recognition (Alles & Gray, 2020; Frey & Osborne, 2017). Such automation would then change the role of the IT auditor who would be more of an information worker responsible for interpreting the output for the IT audit automation tools (Frey & Osborne, 2017). 14 Although the studies focused on the benefits of audit automation and data as an enabler, there was little or no discussion of the impacts of audit automation on IT auditors or the IT audit profession. Table 2 is a summary of the concepts identified on the reviewed literature. It supports the identified pattern on the lack of discussion or contribution regarding the IT audit profession or the role of the IT auditor as a result of IT audit automation. This study therefore focuses on the impact of IT audit automation on IT auditors or the IT audit profession. 15 Table 2: Concept matrix summarising concepts examined on the reviewed literature. Articles Concepts The audit process Data The IT audit profession Continuous Auditing (CA)/ Continuous Monitoring (CM) IT Audit process automation Data Access Data Analytics/ Data Modelling Big Data IT Audit profession Automation Role of IT Auditor (Vasarhelyi, 1993) * * (Kogan, Sudit, & Vasarhelyi, 1999) * * * * * * (Chan & Vasarhelyi, 2011) * * * * (Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012) * * * * (Titera, 2013) * * * * * (Dzuranin & Malaescu, 2016) * * * * * (Rikhardsson & Dull, 2016) * * * * (Huerta & Jensen, 2017) * * * * (Frey & Osborne, 2017) * * * * 16 Articles Concepts The audit process Data The IT audit profession Continuous Auditing (CA)/ Continuous Monitoring (CM) IT Audit process automation Data Access Data Analytics/ Data Modelling Big Data IT Audit profession Automation Role of IT Auditor (Huang & Vasarhelyi, 2019) * * * * * * (Alles & Gray, 2020) * * * (Pizzi, Venturelli, Variale, & Macario, 2021) * * * * (Caster, Elder, & Janvrin, 2021) * * (Zhang, Thomas, & Vasarhelyi, 2022) * * * (Eulerich, Masli, Pickered, & Wood, 2023) * * * 17 2.7 THEORETICAL AND CONCEPTUAL FRAMEWORK Based on the literature reviewed for this research, there were no explicit theories discussed however, there were concepts identified. These concepts can be divided into three main concepts with sub-concepts namely, 1) The audit process consisting of 1a) Continuous Auditing (CA)/ Continuous Monitoring (CM) and 1b) audit process automation; 2) Data consisting of 2a) data access and 2b) Data Analytics/ Data Modelling and 2c) Big data and 3) the IT audit profession consisting of 3a) IT audit profession automation and 3b) Role of IT auditor. These are discussed further in section 2.7.1 of this research report where a model was developed to address the research question and research objective. Refer to table 2 for a detailed view of the concepts examined and the respective studies. A high-level review of socio-technical Information Systems (IS) theories was conducted, as Mkhomazi & Iyamu, 2013 suggest that socio-technical IS theories are key to underpinning IS research. A theory that could potentially address the research question and research objective is the Diffusion of Innovation Theory (DOI) which aims to explain how innovations are accepted by a population within a specified context (Mkhomazi & Iyamu, 2013). However, this theory does not fully address the research question and research objective as it focuses on explaining the acceptance or rejection of the innovation in a specified context (Mkhomazi & Iyamu, 2013) it does not explain the impact of the use of the innovation after the acceptance therefore it was not used for this study. Another theory that was reviewed is the Technology Acceptance Model (TAM) which aims to explain the relationship between perceived usefulness (PU) and perceived ease of use (PEOU) where users are more likely to adopt a system if they perceive it to improve their work (Momami, Jamous, & Hilles, 2018). Another theory that was reviewed is the Unified Theory of Acceptance and Use of Technology (UTAUT) which is the expansion of TAM to include performance expectancy, effort expectancy, social influence and facilitating conditions as key adoption factors (Venkatesh, Morris, Davis, & Davis, 2003). However, the theories do not fully address the research question and objective as they do not explore the impact of the use of the technology after the acceptance has occurred and were not selected for this study. Furthermore, the Socio-Technical Systems (STS) theory was reviewed. This theory argues that successful technology adoption reqiuires balancing technical and social factors and highlights that users have to be involved in the system design and organisational alignment during the system implementation phase (Tatnall, 2023). However, this theory does not fully address the research question and objective as it focuses on pre-implentation factors of teachnology use and acceptance and was not selected for this study. Other theories that were reviewed include the Actor-Network Theory (ANT) which views technology adoption as a result of interactions between humans and software/hardware and explains how it is more likely for a system to be adopted if it aligns user needs and workflows (Doolin & Lowe, 2002). Additionally, the Task-Technology Fit (TTF) model was reviewed. IT 18 aims to explain that technology adoption depends on how well it fits the tasks it is meant to support, similar to ANT it suggests that it is more likely for a system to be adopted if it aligns to the user needs and workflows (Goodhue & Thompson, 1995). However, these theories do not fully address the research question and objective as they focus on adoption factors that are task and workflow based instead of the impacts of the adoption of the technology. Additionally, the Institutional Theory was reviewed, which argues that technology adoption is determined by external pressures such as governements, industry norms, competitors and cultural expectations where organisations may adopt systems to maintain legitimacy instead of gaining efficiencies (DiMaggio & Powell, 1983). This theory does not fully address the research question and objective as focuses on orgnisational external factors for technology adoption instead of impacts as a results of the adoption. Another theory reviewed was the Resistance to Change theory, which explains that technology adoption can be slowed by psychological resistance, organisational inertia and lack of user training (del Val & Fuentes, 2018). However, this theory does not fully address the research question and objective as it focuses on change management strategies during technology adoption. An additional theory that was reviewed which would best address the research question and research objective is the Affordance Theory. It aims to explain the relationship between the technology and the user/human actor (Wang, Wang, & Tang, 2018). This theory is best fit to address the research question and research objective as it can help understand how auditors are impacted by the affordance provided by audit automation. Furthermore, the theory can be used to explore both the organisational and individual aspects of the effects of the affordances/ lack of (Wang, Wang, & Tang, 2018). It can be noted that several theories could potentially address the objective and questions posed on this research report, however, only two were selected and discussed to comprise the theoretical framework of this research report, these include 1) A model that was developed as a result of the concepts identified on the systematic literature review performed for this research report which addresses the research question and research objective and 2) The Affordance Theory. These theories/frameworks and their respective constructs are discussed further in section 2.7.1. 19 2.7.1 RESEARCH MODEL/FRAMEWORK Model developed from systematic literature review of this research report Table 2 of this research report summarises concepts/ key themes that were identified as a result of the systematic literature review conducted for this research report. It was noted that there are three concepts containing sub-concepts that were identified namely, 1) The audit process consisting of 1a) Continuous Auditing (CA)/ Continuous Monitoring (CM) and 1b) audit process automation; 2) Data consisting of 2a) data access and 2b) Data Analytics/ Data Modelling and 2c) Big data and 3) the IT audit profession consisting of 3a) IT audit profession automation and 3b) Role of IT auditor. As discussed on the background literature of this research report, it was noted that data is the main enabler of IT audit automation and as a result affects the audit process or how IT audits are conducted. This in turn has an impact in the IT audit profession. The relationships identified from the literature review can be explained by figure 1. Refer to the literature review of this research report for the detailed background of the identified concepts examined from literature. The linkage of this model and the affordance theory is discussed further and represented by figure 3. Figure 1: Model developed from key themes/concepts of the systematic literature review conducted for this research Data • Data access • Data Analysis/ Data Modelling • Big Data Audit process • Continuous Auditing (CA) / Continuous Monitoring (CM) • IT audit process automation IT audit profession • IT Audit profession automation • Role of auditor 20 Affordance Theory The affordance theory was first developed in 1977 by Gibson to understand how an object in an environment can afford an actor possibilities of meeting a goal based on the context of the perceived goal (Volkoff & Strong, 2017). In the Information Systems research context, the affordance theory has been contextualised to understand the relationship between the technology and the human actor (Mkhomazi & Iyamu, 2013) in this research report the relationship between the IT auditor (organisational perspective) and the IT audit automation (IT artefact). The affordance theory can also be used to identify the effects of affordances/ lack of as a result of the interaction between the human actor (IT auditor) and the technology (IT audit automation) (Mkhomazi & Iyamu, 2013). Figure 2 below explains the theoretical framework that can be used to understand the effects of these affordances. The framework below also considers the organisational lens of effects of the affordances (recognition process and behaviour of the IT auditors). Figure 2: Theoretical framework of affordances in Information Systems as depicted by Wang, et al., (2018, p. 62) 21 The framework consists of four concepts that include the 1) cognition process which is made up of the IT artefact, organisation and affordances existance; 2) the recognition process which includes the affordances perception; the behavior which consists of affordances actualisation and 4) the affordances effects. Refer to table 3 for the concepts identified. Table 3: Themes used to code the transcribed interviews adapted from the model developed in the literature review of this study and the theoretical framework of affordances in Information Systems as depicted by Wang, et al., (2018, p. 62) Concept Sub-concept Definition Data Data Access Automation of IT audit processes is enabled by data and this data must be accessible and stored electronically (Huang & Vasarhelyi, 2019; Kogan, Sudit, & Vasarhelyi, 1999; Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Data Analysis/Data Modelling Advanced data analytical audit methods, techniques such as data mining and machine learning are used to aid the process of IT audit automation (Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Big Data Big data is a significant contributor to the audit automation process where different types of data (structured and unstructured) can be accessed to extract value adding business and IT audit insights (Alles & Gray, 2020; Huang & Vasarhelyi, 2019; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021). Audit Process Continuous Auditing (CA) / Continuous Monitoring (CM) Continuous Auditing (CA) and Continuous Monitoring (CM) as a benefit of audit automation (Chan & Vasarhelyi, 2011; Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Audit Process Automation Automated audits are an improvement from traditional audits as 1) well defined and repetitive tasks are completed in less time (Frey & Osborne, 2017; Huang & Vasarhelyi, 2019; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021). IT Audit Profession IT Audit Profession Automation The automation of the IT audit profession (Eulerich, Masli, Pickered, & Wood, 2023; Pizzi, Venturelli, Variale, & Macario, 2021; Rikhardsson & Dull, 2016; Zhang, Thomas, & Vasarhelyi, 2022). 22 Concept Sub-concept Definition Role of Auditor The role of the IT auditor given the automation of the IT audit profession (Huerta & Jensen, 2017; Kogan, Sudit, & Vasarhelyi, 1999). Cognition Process IT Artefact The IT artefact based on the context of the study (Wang, Wang, & Tang, 2018). Organisation The organisation based on the context of the study (Wang, Wang, & Tang, 2018). Affordances Existence The cognitive process of where the human actor perceives potential affordances when interacting with IT artefacts (objects) (Wang, Wang, & Tang, 2018). Recognition Process Affordances Perception The recognition of the existence of affordances per the perception of the potential affordances (Wang, Wang, & Tang, 2018). Behaviour Affordances Actualisation The interaction with IT artefact by human actors to realise affordances (Wang, Wang, & Tang, 2018). N/A Affordances Effects Effects achieved as a result of the actualisation of affordances by human actors (Wang, Wang, & Tang, 2018). 23 Linking of frameworks and model Concepts from figure 2 were identified from the systemetic literature review conducted for this research report. These concepts were further grouped to develop a model per figure 3 of this research report. A framework for the automation of the audit process as identified in figure 3 was identified as per Huang & Vasarhelyi, (2019) and can guide how the automation of the IT process is conducted and the extent of the automation. The affordances that are a result of this automation can also be explained using figure 3 as guideline which in turn explains the effects of the affordances. Figure 3 depicts these relationships. Data IT Audit process IT Audit profession Cognition Process Figure 3: Conceptual framework of the study Q1 Q2 Q3 Q4 Recognition Process Behaviour Affordance Theory 24 3 CHAPTER THREE – RESEARCH METHODOLOGY 3.1 INTRODUCTION This chapter outlines the methodology used in the research, it defines all methodological terms used and justifies all methods selected for the study. The chapter will cover the research philosophy and approach, the design and strategy, population and sampling, the research instrument used to conduct the interview questions, data analysis procedures, the research rigour, and ethical considerations. 3.2 RESEARCH PHILOSOPHY (PARADIGM) AND APPROACH The research followed the interpretivism paradigm. Interpretivism is a philosophical paradigm where the world is viewed with subjectivity where knowledge creation/research is conducted based on people’s subjective perceptions of the world (Oates, Griffiths, & McLean, 2022). Ontology is how the nature of the world/reality is viewed (Oates, Griffiths, & McLean, 2022). In the interpretivism paradigm, the world is viewed as subjective reality where the reality is dependent on people’s perceptions. This means that based on the interpretivism paradigm there are multiple realities. This paradigm is therefore the best approach to addressing the objectives and questions of this research as it captures the different perspectives of the different IT auditors who formed part of the study. Epistemology is how knowledge creation/research is performed on the way reality is viewed (Oates, Griffiths, & McLean, 2022). In the interpretivist paradigm the subjective reality cannot be understood/measured objectively (Oates, Griffiths, & McLean, 2022). Therefore, interpretivist research aims to understand why certain people act the way they do and why they perceive some concepts the way they do (Oates, Griffiths, & McLean, 2022) which can yield multiple subjective results based on the individuals/groups assessed and how they perceive certain concepts. This therefore assisted with addressing the objectives and research questions of the study by understanding the perceptions of IT auditors regarding the automation of audit procedures. Interpretivist research focuses mostly on qualitative research which critically analyses people’s subjective understanding on phenomena (Oates, Griffiths, & McLean, 2022). The results on the interpretivism paradigm are not generalisable on the greater population. Smaller samples of a selected population are usually selected where the data collected is not structured as it entails subjective responses of the different subjects. Interviews are a data collecting method that can be used in social science research following the interpretivism paradigm. The interpretivism paradigm relies on inductive research where theories are developed using new empirical data (Bhattacherjee, 2012). This research aims on identifying further concepts that may impact the IT audit profession/ IT auditors due to audit automation which may bring rise to new theories/models, or which may add to the underpinning research framework of this research. Therefore, based on the nature of this research, the sample size of the IT auditors is 21 which avoids data saturation while also ensuring variability and a comprehensive understanding of the phenomenon being studied (Guest, Bunce , & Johnson, 2006; Malterud, Siersma, & Guassora, 2015). 25 3.3 RESEARCH DESIGN AND STRATEGY This research utilised a qualitative case study. Qualitative data is data that includes non-numeric data such as words, pictures, and sounds (Oates, Griffiths, & McLean, 2022). Case studies are studies that focus on a single phenomenon that is under assessment such as a department or organisation (Oates, Griffiths, & McLean, 2022). Having defined qualitative data and case study separately, a qualitative case study can be defined as a study that focuses on one instance of a single phenomenon by using non-numeric data techniques such as words, pictures, sounds, etc. This research focused on one department within an audit firm in South Africa. The case involves an IT audit department which uses audit automation tools for their IT audits. The impact of using these tools has not been explored, this study helps explore impacts that the audit automation has on the IT auditors within this department in a South African audit firm. There are three basic types of case studies namely, exploratory, descriptive, and explanatory (Oates, Griffiths, & McLean, 2022). An exploratory study is used to define question or hypotheses that are to be used in a future study and is usually selected when there is limited literature on the selected phenomenon of study (Oates, Griffiths, & McLean, 2022). Therefore, this type of case study can be considered if there is not enough literature regarding the phenomenon under review. A descriptive study is used to derive a detailed analysis of the selected phenomenon and details observed occurrences and subject’s perspectives on how they view the observed occurrences (Oates, Griffiths, & McLean, 2022). Therefore, this type of case study can be considered when analysing and detailing the perceptions on the use of a certain IT artefact. An explanatory study is like a descriptive study; however, it goes further by identifying multiple factors that affected the phenomenon being assessed and comparing them with theories from the literature and identifying the best theory for the phenomenon being assessed (Oates, Griffiths, & McLean, 2022). Therefore, this type of case study would be considered when identifying multiple factors that affect the phenomenon under review. Based on the definitions of the qualitative case studies, the type of case study that best address the research objectives and questions is the exploratory study as there is limited literature that focuses on the effects of audit automation of IT auditors/IT audit profession. 3.4 POPULATION AND SAMPLE 3.4.1 POPULATION A population is a complete group of individuals with similar characteristics that a sample can be selected from to draw conclusions (Bhattacherjee, 2012). The population for this study entails all the IT auditors within an IT audit department in a South African audit firm. 3.4.2 SAMPLING METHOD Sampling is a statistical process where a subset is selected from a population of interest for the purpose of collecting data or making observations and statistical conclusions on the population (Bhattacherjee, 2012). The sampling process incudes identifying the population, which is the group 26 of interest that the statistical conclusions can be generalisable to (Bhattacherjee, 2012). It includes the sampling frame, which is the list that the sample can be selected from and the sample, which is the which consist of the selected unit of analysis (Bhattacherjee, 2012). In this research, the population is IT auditors within a department at a South African audit firm. The research used purposive sampling, which is a non-probability sampling technique where respondents are selected based on their expertise (Bhattacherjee, 2012). Opinions from experts are more credible than those provided by non-experts (Bhattacherjee, 2012). IT auditors were selected for this study as they are more likely to provide expert/credible responses on their perception of IT audit automation. A sample size of 21 IT auditors was selected to avoid data saturation while also ensuring variability and a comprehensive understanding of the phenomenon being studied (Guest, Bunce , & Johnson, 2006; Malterud, Siersma, & Guassora, 2015). 3.5 RESEARCH INSTRUMENT The instrument/survey was assessed for validation and refined before data was collected. A panel of experts (The university’s proposal defence panel) was used to confirm content and face validity of the questions used. Face validity refers to whether an indication is a justifiable measure of a concept at face value and content validity refers to how well scale concepts match with content that the concept is intending to measure (Bhattacherjee, 2012). After the instrument was developed and tested, pilot testing was conducted to further refine the research instrument. Pilot testing is important in the research process as it detects potential issues in the research design or instrumentation which ensures the validity and reliability of the concepts (Bhattacherjee, 2012). For this study, two (2) IT auditors were interviewed before the pilot-testing to ensure that the questions asked were clear and concise, no significant changes were made to the research instrument. Pilot-testing was conducted with one (1) IT auditor, question 1a) was adjusted to add that data is an enabler for automation. Refer to Appendix A for the instrument used for this study. 3.6 PROCEDURE FOR DATA COLLECTION Interviews are a data collecting method that can be used in social science research following the interpretivism paradigm (Bhattacherjee, 2012). The interviews contained both open-ended (unstructured) and closed-ended (structured) questions and were semi-structured where a section of the interview contained questions that required pre-defined answers (Bhattacherjee, 2012). The data was collected using semi-structured interviews with 21 IT auditors at an IT audit department a South African audit firm. Permission to interview IT auditors in a department at a South African audit firm was obtained from the department’s business unit leader. The interviewees were contacted via email and those who are interested were selected to participate in the study. The interviews were recorded using Microsoft Teams. Participants’ consent was obtained before the interviews. The data was collected following an inductive strategy where questions were derived from the underpinning framework of this research. The interview questions were based on Q1-Q4 proposed on the literature review section of this research report: 27 Q1: How does data (data access, data analytics/data modelling and big data) affect the application of automation in IT auditing? Q2: How does the IT audit process automation affect the IT audit profession in IT auditing? Q3: What is the effect of affordances as a result of audit automation on the IT audit process in IT auditing? Q4: What is the effect of affordances as a result of audit automation on the IT audit profession in IT auditing? 3.7 DATA ANALYSIS AND INTERPRETATION Qualitative data is composed of non-numerical data such as words, pictures, or sounds and can be generated from interview recordings, websites, research diaries, etc. (Oates, Griffiths, & McLean, 2022). Before analysing the qualitative data, it must be prepared (Oates, Griffiths, & McLean, 2022). Data preparation is the process of getting data ready for analysis, where data is grouped/presented in a similar format (Oates, Griffiths, & McLean, 2022). For this research report, interview recordings were transcribed to ensure that collected data is in a similar format that can be easily analysed. After the data was transcribed, data analysis took place. Post reading all the transcribed data, key themes can be identified (Bhattacherjee, 2012; Oates, Griffiths, & McLean, 2022). Oates, et al. 2022, suggest using three themes during initial stages of analysis which include, 1) segments that are not relevant to the study, 2) segments that provide descriptive information for the context of the research and, 3) segments that may be relevant to address the research questions of the research (Oates, Griffiths, & McLean, 2022). After the initial analysis, segments from theme 3 can be further categorised into different themes (Oates, Griffiths, & McLean, 2022). Some of the categories identified can be broken down into sub-categories (Oates, Griffiths, & McLean, 2022). Post categorisation, an analysis of interconnections and themes between segments and categories was made (Oates, Griffiths, & McLean, 2022). The use of visual aids such as tables helped with the categorisations (Oates, Griffiths, & McLean, 2022). The data analysis process was documented step-by-step to ensure that the conclusions reached are justifiable and the analysis can be followed/understood. The concepts were derived from the systematic literature review conducted for this research report where a model with three main concepts was derived. Furthermore, the affordance theory adapted for information systems studies was used which also has three (3) main concepts. Table 4 outlines the key themes used to code the data collected from the interviews with the 21 IT auditors. 28 Table 4: Themes used to code the transcribed interviews adapted from the model developed in the literature review of this study and the theoretical framework of affordances in Information Systems as depicted by Wang, et al., (2018, p. 62) Concept Sub-concept Definition Data Data Access Automation of IT audit processes is enabled by data and this data must be accessible and stored electronically (Huang & Vasarhelyi, 2019; Kogan, Sudit, & Vasarhelyi, 1999; Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Data Analysis/Data Modelling Advanced data analytical audit methods, techniques such as data mining and machine learning are used to aid the process of IT audit automation (Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Big Data Big data is a significant contributor to the audit automation process where different types of data (structured and unstructured) can be accessed to extract value adding business and IT audit insights (Alles & Gray, 2020; Huang & Vasarhelyi, 2019; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021). Audit Process Continuous Auditing (CA) / Continuous Monitoring (CM) Continuous Auditing (CA) and Continuous Monitoring (CM) as a benefit of audit automation (Chan & Vasarhelyi, 2011; Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Audit Process Automation Automated audits are an improvement from traditional audits as 1) well defined and repetitive tasks are completed in less time (Frey & Osborne, 2017; Huang & Vasarhelyi, 2019; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021). IT Audit Profession IT Audit Profession Automation The automation of the IT audit profession (Eulerich, Masli, Pickered, & Wood, 2023; Pizzi, Venturelli, Variale, & Macario, 2021; Rikhardsson & Dull, 2016; Zhang, Thomas, & Vasarhelyi, 2022). Role of Auditor The role of the IT auditor given the automation of the IT audit profession (Huerta & Jensen, 2017; Kogan, Sudit, & Vasarhelyi, 1999). Cognition Process IT Artefact The IT artefact based on the context of the study (Wang, Wang, & Tang, 2018). Organisation The organisation based on the context of the study (Wang, Wang, & Tang, 2018). Affordances Existence The cognitive process of where the human actor perceives potential affordances when interacting with IT artefacts (objects) (Wang, Wang, & Tang, 2018). Recognition Process Affordances Perception The recognition of the existence of affordances per the perception of the potential affordances (Wang, Wang, & Tang, 2018). 29 Behaviour Affordances Actualisation The interaction with IT artefact by human actors to realise affordances (Wang, Wang, & Tang, 2018). N/A Affordances Effects Effects achieved as a result of the actualisation of affordances by human actors (Wang, Wang, & Tang, 2018). 3.8 RIGOUR Rigour refers to the methods applied to ensure that the findings are credible, trustworthy, and accurately represents the participants’ experiences and perceptions (Bhattacherjee, 2012; Oates, Griffiths, & McLean, 2022). The quality or rigour of an interpretivist research is measured using, trustworthiness, confirmability, dependability, credibility, and transferability (Oates, Griffiths, & McLean, 2022). Credibility Credibility refers to how believable the study is and if it reflects the authenticity of the participants (Bhattacherjee, 2012). “The credibility of interpretive research can be improved by providing evidence of the researcher’s extended engagement in the field, by demonstrating data triangulation across subjects or data collection techniques, and by maintaining meticulous data management and analytic procedures, such as verbatim transcription of interviews, accurate records of contacts and interviews, and clear notes on theoretical and methodological decisions, that can allow an independent audit of data collection and analysis if needed.” (Bhattacherjee, 2012, p. 110). In this study credibility was ensured by recording and transcribing all interviews conducted. Accurate records of contacts were also stored. Transferability Transferability refers to the extent to which the findings of the study can be generalisable to other cases (Bhattacherjee, 2012). This can be demonstrated by providing detailed descriptions of the research context and detailing any assumptions made from the data so that other researchers can evaluate if the reported findings are generalisable to other cases (Bhattacherjee, 2012). In this study, transferability was ensured by documenting all the findings and the corresponding quotes that the participants made to get to certain conclusions. These were broken down into themes which were also documented in detail. Dependability Dependability refers to the authenticity of the study where it is examined if another researcher assessing the same research phenomenon can reach the same or a similar conclusion (Bhattacherjee, 2012). This can be demonstrated by having a logical, traceable, and clearly documented study (Bhattacherjee, 2012). In this study this was ensured by documenting all the steps of the data analysis and a consistency table was used when coding the transcribed interviews. Refer to appendix B for the consistency table used. Confirmability Confirmability refers to the extent to which the results are based on the participants’ perceptions and not the researcher’s biases (Bhattacherjee, 2012). This can be demonstrated by using triangulation across data collection techniques and methods (Bhattacherjee, 2012). For this study, the data collection methods and techniques were described, and the research instrument used was derived/adopted from the literature reviewed. 30 Trustworthiness Trustworthiness refers the amount of trust that can be placed on the study (Oates, Griffiths, & McLean, 2022). This can be demonstrated by ensuring that the research is credible, transferable, dependable, and confirmable (Oates, Griffiths, & McLean, 2022). For this study, this was ensured by detailing every step and process undertaken to ensure the for characteristics of trustworthiness. 3.9 ETHICAL CONSIDERATIONS Ethical considerations refer to the protection of the participant’s rights, where informed consent should be obtained via a review process by an institution and obtaining ethical approval (Klopper, 2008). Research is considered ethical when it does not bring harm to human subjects (Rosenthal, 1994). This study underwent the review process by the university to ensure that it does not violate any participant’s (IT auditors) rights, and it does not harm them. This means that the participants (IT auditors) participated voluntarily in the study, meaning that they could withdraw at any point of the study without any consequences, and they remained unharmed regardless of their participation or lack of (Bhattacherjee, 2012). The participants (IT auditors) signed a consent form before participating in the study, the consent form clearly stated their rights (including the participation being voluntary and that they could withdraw at any point of the study/interview) (Bhattacherjee, 2012). Furthermore, the participants interests were protected by ensuring that they remained anonymous, or their responses were kept confidential (Bhattacherjee, 2012). Anonymity implies that a response cannot be linked back to a participant (Bhattacherjee, 2012). Although anonymity was not fully possible in this study as the interviews included face-to-face interactions, participants were guaranteed confidentiality by making sure that none of the participant’s real names or any identifying information was included in the final research report. Participants were also given full details of the research before they could participate, as this helped them decide if they wanted to participate or not (Bhattacherjee, 2012). For this research, this information was included as part of the consent form. The purpose, outcomes and benefits of the results were outlined (Bhattacherjee, 2012). Additionally, the details of the analysis of data and reporting of results were included in the final report that participants can have access to upon request (Bhattacherjee, 2012). All results were disclosed regardless of whether they were unexpected or negative (Bhattacherjee, 2012). In addition to the above considerations, the research was performed at the highest possible quality as Rosenthal 1994, states that low quality research is a waste of participants’ time and a waste of resources which is also an ethical concern. 31 4 CHAPTER FOUR – PRESENTATION OF THE FINDINGS 4.1 INTRODUCTION This section contains the context of the study based on the 21 interviews conducted and presents the findings, major themes identified and any emergent themes. 4.2 CONTEXT OF THE STUDY The interview questions aimed to answer the four (4) sub-research questions that were derived from the literature review of the study: Q1: How does data (data access, data analytics/data modelling and big data) affect the application of automation in IT auditing? Q2: How does the IT audit process automation affect the IT audit profession in IT auditing? Q3: What is the effect of affordances as a result of audit automation on the IT audit process in IT auditing? Q4: What is the effect of affordances as a result of audit automation on the IT audit profession in IT auditing? It can be noted that Q1-Q4 address the main objective and research questions posed on this research report. Objective: To determine the impacts of IT audit automation on IT audit. Main research question: What impact does the automation of IT audit processes have on IT auditors/IT audit profession? These questions were answered by themes that were derived from the model developed from the literature review of the research report and the theoretical framework of affordances in Information Systems as depicted by Wang, et al., (2018, p. 62) The study focused on IT Auditors within an IT audit department in a South African audit firm. Twenty-one IT auditors were interviewed. Purposive sampling was used, where IT Auditors who had availability and were willing to participate were selected. Additionally, Snowball sampling was used were some of the participants recommended other collegues. The context of individuals interviewed is presented on Table 5. Identifiers/names of the participants have been removed to adhere to the university’s confidentiality and ethics policies. 32 Table 5: Biographical details of participants interviewed. Name of Participant Job Title Gender Age IT Audit Experienc e Experience with IT Audit Automation Tools Participant 1 Manager Female 22-34 7+ 7+ Participant 2 Manager Male 35-47 7+ 7+ Participant 3 Senior Consultant Female 22-34 4+ 3+ Participant 4 Senior Manager Female 22-34 10+ 10+ Participant 5 Director Female 35-47 20+ 20+ Participant 6 Senior Manager Male 35-47 10+ 10+ Participant 7 Manager Male 35-47 4+ 4+ Participant 8 Manager Female 22-34 7+ 7+ Participant 9 Senior Manager Male 35-47 9+ 9+ Participant 10 Senior Consultant Male 22-34 5+ 5+ Participant 11 Manager Male 22-34 4+ 4+ Participant 12 Manager Female 35-47 7+ 7+ Participant 13 Manager Male 22-34 5+ 5+ Participant 14 Senior Associate Director Male 35-47 20+ 20+ Participant 15 Manager Male 22-34 8+ 8+ Participant 16 Associate Director Male 48-60 20+ 20+ Participant 17 Senior Consultant Female 22-34 4+ 4+ Participant 18 Director Male 35-47 21+ 21+ Participant 19 Senior Manager Male 35-47 14+ 14+ Participant 20 Senior Manager Male 22-34 5+ 5+ Participant 21 Senior Associate Director Male 35-47 22+ 22+ 33 4.3 THEMES PRESENT AND FINDINGS OF THE STUDY Figure 3 outlines key themes that were identified as a result of the literature review conducted for this study. It was noted that there were three concepts containing sub-concepts that were identified namely, 1) The audit process consisting of 1a) Continuous Auditing (CA)/ Continuous Monitoring (CM) and 1b) audit process; 2) Data consisting of 2a) data access and 2b) Data Analytics/ Data Modelling and 2c) Big data and 3) the IT audit profession consisting of 3a) IT audit profession automation and 3b) Role of IT auditor. Additionally, themes from the Theoretical framework of affordances in Information Systems as depicted by Wang, et al., (2018, p. 62) were identified namely 4) Cognition Prcess consisting of 4a) IT Artefact, 4b) Organisation and 4c) Affordances Existance, 5) the Recognition Process consisting of 5a) Affordances Perception, 6) Behavior consisting of 6a) Affordances Actualisation and Affordances 7a) Affordances Effects 34 Q1: How does data (data access, data analytics/data modelling and big data) affect the application of automation in IT auditing? DATA Data Access In the context of the study, data access refers to Automation of IT audit processes that is enabled by data and this data must be accessible and stored electronically (Huang & Vasarhelyi, 2019; Kogan, Sudit, & Vasarhelyi, 1999; Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012) Twelve (12) out of the 21 participants highlighted the significance of the availability of quality data, correct data, structured data, accessibility of data, data with high integrity, complete and accurate data, and consistent data. The participants noted that not only should the data be available, but it should also be of good quality and have integrity: Participant 1: “The availability of high-quality, structured, and accessible data significantly enhances the automation of IT audit procedures. It enables auditors to leverage tools like machine learning and data analytics to identify patterns, anomalies, and risks faster and more accurately, reducing manual intervention.” Participant 2: “Without data, there is no processing because automation is supposed to process the data and provide us information that we can use, right? So, if the integrity of the data is compromised then so will the results. So, the automation will be pretty much pointless because the automation is also dependent on the input for it to give us the design output.” Participant 3: “If you don't have the proper data for you to perform any automations in an IT audit process, you won't have the output that you need.” Participant 5: “If you don't feed the tools the right data, you will not get the desired result or the right results for the audit. So that's the biggest thing. The integrity of the data that is fed into the tools and then. In addition to having accessible good quality data, participants noted that the data should be complete and accurate to ensure that the automated outputs yield credible results: Participant 4: “If It’s not complete or accurate… I think it will then skew your audit results in terms of overall conclusions, whether it's effective or ineffective, that might give you, I suppose, like false positives as well as, false negatives… so the reliability and availability of data is important, especially from the timing perspective.” 35 Participant 7: “The more complete and accurate of the data, the more you can rely on the automation of the audit process. Participant 9: “An important part of what we do and the completeness and accuracy of that data. There's always going to be. A major focus when you when you are using. Even if you're doing it manual or you're using an automation… make sure that you confirm the accuracy on a completeness of the data before you put it through your automated process.” Furthermore, the participants noted that the data should be fit for purpose to yield any insights. They also noted that the data should be in a structured or usable format: Participant 6: “It's not just the availability of data, it's the efficient use of data, the insights you can pull from data, the structure that you can pull from there as well”. Participant 16: “It's often the format that you are able to get the data in. So, if you are using a tool like ACL, normally you would either get a download of the data and you know the format that you get in it in is very important… Is in the right format, is right and then also you know then tying having various control titles and checks to ensure the completeness and accuracy of that data is very important.” Participant 18: “In order for an automated technology or tool to work effectively right, it needs data in order to work effectively. So in in a sense, without the data, it's very difficult to automate anything, because what would you be automating? You basically have the functionality, but you wouldn't have the underlying substance that these automated.” Participant 20: “If the data that you're using is incorrect, then by default the automation won't want to yield any benefits.” Additionally, a participant also noted that consistency of data is required to ensure that the automation works consistently: Participant 21: “What we're doing is repeatable so that you can automate something that you're going to do over and over again. So, that sort of have that consistency in the in the data to be able to do that. So, it's not just availability of data itself, but actually the consistency of the data as well.” 36 Data Analysis/Data Modelling Data Analysis and Data Modelling refer to the Advanced data analytical audit methods, techniques such as data mining and machine learning are used to aid the process of IT audit automation (Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). Four (4) out of the 21 participants noted that data analytics and data modelling aid in the automation of audit processes which provides more insights into the data, results in efficiencies, full population testing and trend analysis. Participants noted that data analysis and data modelling help with predictive insights and ease of automation of repetitive tasks: Participant 1: “Yes, other enablers include, Advanced analytics tools, Cloud computing platforms. Blockchain for immutable transaction records, Artificial intelligence (AI) for predictive insights, Robotic Process Automation (RPA) for repetitive tasks.” Participant 6: “It's the availability of data. And then it's also the insight into the data… So, it's not just the availability, but it's also the fact the intelligence on extracting the right sets of data with the right integrity to make sure you know that you've got… You've got integrity over the data that's coming out to support the automation of the audit procedures that are built on that data. So, I think it's more just the availability. It's also the insight.” Additionally, participants noted that the use of data analysis and data modelling help with efficiencies as time spent delivering IT audit is shortened. Furthermore, they noted that data analysis and data modelling allow IT auditors to test full populations for their audits instead of sample-based testing: Participant 3: “Efficiency…You focus on Data Analytics… You get an output of the full population in in a few seconds or minutes.” Participant 5: “…And shortens the period that is required to audit for example. We are able to use data that is provided by clients in previous years and in the current years. To develop a trend analysis and dashboard for those for those clients. 37 Big Data Big data is a significant contributor to the audit automation process where different types of data (structured and unstructured) can be accessed to extract value adding business and IT audit insights (Alles & Gray, 2020; Huang & Vasarhelyi, 2019; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021). Three (3) out of the 21 participants indicated having one platform (preferably cloud) enables audit automation in a big scale as most of the data will be stored in one place which makes collaboration amongst team members easy. Furthermore, an integration of data sources was noted which allows the access and interaction of multiple data sources: Participant 5: “The cloud. Definitely a big enabler for automation because it helps, it allows for collaboration by people in different parts of in different geographies. People don't have to be in the same office anymore to be able to collaborate on audits because of the cloud.” Participant 6: “I'm aware of other automation audit automation enabling. I'm trying to think I know there's tools like GRC tools that are used in some audits where it like for example, you're doing different types of audits where it's ISO audits, third party assurance, SOC audits, financial audits. I know I am aware of tools that are used to Kind of store the data centrally but map it to different types of audits.” Participant 17: “I think integration right 'cause now with automation like most processes are integrated, right. And there's an interaction of data from multiple sources.” Participants whose responses were not captured only emphasised the importance of data access and availability, they did not add any additional inputs or insights. 38 Q2: How does the IT audit process automation affect the IT audit profession in IT auditing? AUDIT PROCESS Continuous Auditing (CA) and Continuous Monitoring (CM) Continuous Auditing (CA) and Continuous Monitoring (CM) as a benefit of audit automation (Chan & Vasarhelyi, 2011; Rikhardsson & Dull, 2016; Vasarhelyi, Alles, Kuenkaikaew, & Littley, 2012). A participant expressed that tools can be developed to enable real-time auditing which aligns to Continuous Auditing and Continuous Monitoring: Participant 16: “So, you can develop scripts and tools in data analytics tools such as ACL. So, that will definitely help you. And I mean, you know, if you're looking at things like continuous auditing, you know is to have direct access to databases, you know preferably obviously read only access. So that you can monitor in real time.” IT Audit Process Automation Automated audits are an improvement from traditional audits as 1) well defined and repetitive tasks are completed in less time (Frey & Osborne, 2017; Huang & Vasarhelyi, 2019; Huerta & Jensen, 2017; Pizzi, Venturelli, Variale, & Macario, 2021). Nine (9) out of the 21 participants indicated that any process automation that happens should be governed by the methodology of the organisation. They further stated that the same principles or methods that are applied when auditing manually should be applied to the audit automation for it to be a success. Participants noted that the IT audit methodology should be embedded on the automation of the IT audits. They noted that similarly to how traditional audits follow the organisation’s IT audit methodology, the IT audit process automation should also be guided by methodology: Participant 1: “The chosen methodology determines how automation is applied. Traditional audit methodologies may need adaptation to leverage automation fully. For instance, risk-based approaches can integrate automated risk analysis, while compliance audits can utilise automated rule-checking systems.” Participant 2: “The automation of the audit procedures has to be a product of the methodology, so the methodology has to be the primary source… It must be in line with the methodology of the entity that you are in, in order for you to actually build this automation for this automation process or bot (audit robot).” 39 Participant 4: “I think that that methodology, essentially it drives error risk-based approach, right? So, whether it's automated automation of ordered procedures… If it is risk based or if it is control based, you would then it determines your procedures that should apply in order to achieve your objectives.” Participant 8: “I don't know if methodology is a data aspect, but essentially you know guidance on how to do the audit. I think that's important for the automation process because it would guide it would guide the extent of automation needed.” Participant 9: “Everything that you're building. Yeah, all the any automation that you have in your audit procedures, all of it still needs to make sure that it follows our methodology.” Participant 12: “They obtain evidence, and so the way the automation tools are developed, they are developed to specifically target and collect information that is relevant for the audit per the audit methodology. So definitely the audit methodology plays an impact on what data we are collecting from the client.” Participant 19: “If you're automating all the procedures, then it has to be aligned with the relevant methodology. Otherwise, you could end up with an automated solution that actually doesn't answer the question.” Participant 13: “You know the methodology will guide the audit procedures. So, in a point which we are now automating those audit procedures, we're automating them in line with the methodology. So, in automating the audit procedures, it's important that we make sure that we don't lose sight of the actual methodology.” Additionally, a participant noted that the automation should work the same way an IT auditor would, meaning that it should follow the same principles and guidelines that an expert IT auditor would: Participant 18: “Those same principles that we used to test it manually need to then guide the automation so that automation needs to then use the methodology in order to build those methodology rules. In order to facilitate that methodology, process the same way you would do it as if you were if a human was doing it right. So in in essence the IT process and methodology are there to help guide the building and the development of the rules that the automation would use.” 40 Sixteen (16) of the 21 participants indicated the disadvantages of traditional audits (auditing without automation or having minimal automation). They mentioned that traditional audits are labour- intensive, time consuming, prone to human error, limited scope of analysis, are sample based and require specialised skills which some individuals may not have. Participants noted that traditional IT audits are labour-intensive and time consuming: Participant 1: “Traditional methods can be labour-intensive, time-consuming, and prone to human error. They may struggle to handle the large data volumes typical in modern IT systems or detect sophisticated risks without advanced tools… Other difficulties include inefficiency in manual processes, Limited scope for analysing big data. Difficulty maintaining accuracy under time constraints, Lack of real-time insights.” Participant 5: “I wouldn't say I've had difficulties, but it's obviously now in 2024 is not a preferred method of auditing… traditional audit meth