A proposed internal audit tool to improve internal audit effectiveness in South African national departments Research Report Zondre Launchya Seitei WITS Business School Thesis presented in partial fulfilment for the degree of Master of Business Administration to the Faculty of Commerce, Law, and Management, University of the Witwatersrand March 2021 ii DECLARATION I, Zondre Launchya Seitei declare that this research report entitled ‘A proposed internal audit tool to improve internal audit effectiveness in South African national departments’ is my own unaided work. I have acknowledged, attributed, and referenced all ideas sourced elsewhere. I am hereby submitting it in partial fulfilment of the requirements of the degree of Master of Business Administration at the University of the Witwatersrand, Johannesburg. I have not submitted this report before for any other degree or examination to any other institution. Zondre Launchya Seitei Signed at Johannesburg on 31st March 2021 Name of candidate Zondre Launchya Seitei Student number 1841171 Telephone numbers 072 431 5245 Email address Zondre.seitei@gmail.com First year of registration 2018 Date of proposal submission 31st October 2020 Date of report submission 31st March 2021 Name of supervisor Kato Plant and Kambidima Wotela iii ABSTRACT Author: Zondre Launchya Seitei Supervisor: Kato Plant Thesis title: A proposed internal audit tool to improve internal audit effectiveness in South African national departments Johannesburg, March 2021 Increasing governance related issues are plaguing the South African government, which have in recent times seen many state institutions facing serious financial and operational related challenges to the point where the National Treasury had to bail out public institutions. Internal audit is a vital component of governance structures within organisations, as internal audit functions (IAF) provides support to the its governing bodies through the provision of assurance and consulting services on the effectiveness of risk management, controls and governance processes. The Institute of Internal Auditors (IIA) has made the development and implementation of a quality assurance and improvement programme (QAIP) mandatory in term of standard 1300 of the International Standards for the Professional Practice of Internal Auditing (The Standards). This augments the importance of measuring the quality delivered by internal auditors to their key stakeholders. Literature revealed that the quality or effectiveness of IAFs’ management support, independence and its competence are factors that can be positively correlated to internal audit effectiveness (IAE). However, the existence of a quality assurance self-assessment preparation (QASP) tool appears to be limited. Therefore, the aim of his study was to propose such a tool to aid IAFs in South African government national departments in conducting their own self-assessments. This could capacitate IAFs in the public sector to assess their effectiveness. This study found that there is a need for quality assurance self-assessment preparation (QASP) tool to help IAFs assess their effectiveness. Four key themes emerged from the findings, namely, IAE, internal assessments, a measurement tool and external audit reliance culminating into a proposed QASP tool. It is therefore recommended that IAFs of national departments in the South African public sector take cognisance of the proposed tool to improve internal audit quality and ultimately governance and service delivery. iv TABLE OF CONTENTS DECLARATION ........................................................................................................................................ ii Abstract iii Table of contents ........................................................................................................................................ iv List of tables vii List of figures viii List of tables and figures in the appendices ............................................................................................ ix ACKNOWLEDGEMENTS ..................................................................................................................... x Definition of key terms and concepts ..................................................................................................... xi 1 Introduction to the research ...........................................................................................................12 1.1 Background and context ......................................................................................................12 1.1.1 What factors or indicators contribute to internal audit effectiveness (IAE)? ...................................................................................................................................15 1.1.2 What quality assurance tool will aid internal audit in adequately assessing its effectiveness and conformance to the IIA Standards? .....................................15 1.2 Research conceptualisation ..................................................................................................16 1.2.1 The research problem statement..........................................................................16 1.2.2 The research purpose (aim and objectives) statement .....................................17 1.2.3 The research questions as well as where applicable accompanying research hypotheses or research propositions ...................................................................18 1.3 Delimitations, assumptions and limitations of the research study ................................18 1.4 Significance of the research study ......................................................................................19 1.5 Preface to the research report .............................................................................................20 2 Literature review ...............................................................................................................................22 2.1 Research problem analysis ...................................................................................................29 2.2 Research knowledge gap analysis .......................................................................................33 2.3 Qualitative attributes or quantitative variables key to the research ...............................36 2.4 Framework(s) for interpreting research findings .............................................................39 2.5 Summary and conclusion .....................................................................................................43 2.5.1 Summary of literature reviewed ...........................................................................43 2.5.2 Proposed research strategy, design, procedure and methods arising from the literature reviewed ..................................................................................................44 3 Research strategy, design, procedure and methods .....................................................................45 3.1 Research strategy ...................................................................................................................45 3.2 Research design .....................................................................................................................46 3.3 Research procedure and methods ......................................................................................47 3.3.1 Research data and information collection instrument(s) .................................47 3.3.2 Research target population and selection of respondents ...............................49 3.3.3 Ethical considerations when collecting research data ......................................50 3.3.4 Research data and information collection process ............................................51 3.3.5 Research data and information processing and analysis ..................................52 3.3.6 Description of the research respondents ............................................................53 3.4 Research strengths—reliability and validity measures applied .......................................54 3.5 Research weaknesses—technical and administrative limitations ...................................55 4 Presentation of research results ......................................................................................................57 v 4.1 Profiling ..................................................................................................................................58 4.2 What factors or indicators contribute to internal audit effectiveness (IAE)? .............59 4.2.1 Theme 1: Internal audit effectiveness .................................................................59 4.3 What quality assurance tool will aid internal audit in adequately assessing its effectiveness and conformance to the IIA Standards? ...................................................62 4.3.1 Theme 2: Internal assessments .............................................................................62 4.3.2 Theme 3: Quality assurance self-assessment preparation (QASP) Tool .......63 4.3.3 Theme 4: External audit’s reliance on internal audit work ..............................67 5 Discussion of research findings ......................................................................................................70 5.1 Introduction ...........................................................................................................................70 5.2 Key findings: Theme 1 – Internal audit effectiveness .....................................................71 5.3 Key findings: Theme 2 – Internal Assessments ...............................................................71 5.4 Key findings: Theme 3 – Quality Assurance Self-Assessment Preparation (QASP) tool...........................................................................................................................................72 5.5 Key findings: Theme 4 – External audit's reliance on internal audit work ..................72 5.6 Proposed Quality Assurance Self-assessment Preparation (QASP) tool .....................73 6 Summary, conclusions, limitations, and recommendations .......................................................75 6.1 Summary .................................................................................................................................75 6.2 Conclusions ............................................................................................................................77 6.3 Limitations .............................................................................................................................78 6.4 Recommendations ................................................................................................................80 6.4.1 IIA Professional Body ...........................................................................................80 6.4.2 National Treasury and IAFs .................................................................................80 6.4.3 National Treasury ...................................................................................................81 6.4.4 Contribution to management knowledge ...........................................................82 6.5 Final conclusion ....................................................................................................................82 References 83 Appendices 88 Appendices 89 Appendix 1.1: Data collection instrument(s) ......................................................................................90 Appendix 1.1: Data collection instrument(s) ......................................................................................91 Appendix 2.1: One-page bio of the researcher including declaration of interest in the research and funders, if any ........................................................................................................92 Appendix 2.2: Ethics documentation ....................................................................................................93 Appendix 3.1: Dully filled in data collection instrument(s) ...............................................................94 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (1) .......95 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (2) .......96 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (3) .......97 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (4) .......98 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (5) .......99 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (6) .... 100 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (7) .... 101 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (8) .... 102 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (9) .... 103 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (10) .. 104 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (11) .. 105 vi Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (12) .. 106 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (13) .. 107 Appendix 4.1: Proposed Quality Assurance Self-assessment Preparation (QASP) Tool (14) .. 108 vii LIST OF TABLES Table 1: Participant profile analysis ......................................................................................... 59 viii LIST OF FIGURES Figure 1: Interpretive Framework (IIA, 2016e) ..................................................................... 43 Figure 2: Visual representation of the quality assurance self-assessment preparation (QASP) tool (Zondre Seitei, 2020) ................................................................................. 73 ix LIST OF TABLES AND FIGURES IN THE APPENDICES Participant Information Sheet Consent Form x ACKNOWLEDGEMENTS I owe it all to GOD who gives me strength in all I do. Thank you for bringing me through the most trying period of my adult life, which resulted in me having to delay the submission of my research, and without whom I would not have made it. Sincere thanks to my dearest and loving husband Harry Seitei for his understanding and unwavering support throughout my research. His contribution extended beyond just being my biggest cheerleader, but also assisting in pre-review of my work. Thank you for your many selfless sacrifices known and unknown, including sharing our time with my MBA journey. To my supervisor, Dr Kato Plant. A big thank you for your guidance and support. You have been amazing and I have learnt so much from you. You persuaded me to pursue the topic of which I now have no regrets. I am confident that this study as a start will contribute majorly to improving the quality of internal audit functions within the public sector. To my mother, Denise Oosthuizen, a single mother of five. Thank you for all your support and love in raising us and for the many altruistic struggles you have gone through that I know nothing of. A heartfelt thank you to National Treasury for funding my MBA studies and the enormous support received. Finally, thank you to the friends I made on my MBA journey. I have learnt so much from you, looking forward to our life-long friendships in further contributing to each other and South Africa as a whole. xi DEFINITION OF KEY TERMS AND CONCEPTS AC – Audit Committee AGSA – Auditor-General South Africa Auditee – Management responsible for designing, implementing and monitoring risks and controls of the organisation CAE – Chief Audit Executive EQAR – External Quality Assurance Review IAE – Internal audit effectiveness IAF – Internal Audit Function, also known as Internal Audit Activity IIA – Institute of Internal Auditors Global IIA SA – Institute of Internal Auditors South Africa (South African Chapter) IIA Standards – International Standards for the Professional Practice of Internal Auditing “IIA Standards”, 2016 IPPF – International Professional Practices Framework, 2016 P – Participant (are internal auditors and interviewees from which data a collected) PFMA – Public Finance Management Act, No. 1 of 1999 QAIP – Quality Assurance and Improvement Programme QASP – QASP Tool (Proposed Tool) SA – South Africa SAA – South African Airways TR – Treasury Regulations, 2005 USA – United States of America 12 1 INTRODUCTION TO THE RESEARCH 1.1 Background and context More generally, this research is proposing a practical internal audit quality assurance self- assessment tool that can be used by internal audit functions (IAF) in South African national departments to assess internal audit effectiveness. Before getting to the research conceptualisation (Section 1.2), the terms and concepts used in conceptualising this research are briefly introduced in Section 1.1 generally and broadly—while Chapter 2 has a more specific and detailed discussion on the research context. The research conceptualisation section provides for the research problem statement (Section 1.2.1) and consequently the purpose of this research (Section 1.2.2) as well as the research questions (Section 1.2.3). The delimitations and assumptions of the research study are in Section 1.3 while the significance of the research study is discussed in Section 1.4 and provides a preface to the research report in Section 1.5. A number of recent governance failures (Thabane & Snyman-Van Deventer, 2018) and serious irregularities (Staff Reporter, 2018) has seen organisations such as amongst others, Steinhoff, Nkonki Incorporated, Klynveld Peat Marwick Goerdeler (KPMG), Eskom, South African Broadcasting Corporation (SABC) and South African Airways (SAA) crumble to a point of irrefutable damage to their reputation, credibility and bottom line (Burger-Smidt, 2018; Thabane & Snyman-Van Deventer, 2018). This has casted a shadow on the once previously world renowned South African auditing fraternity in relation to being ranked as the best in auditing and reporting standards by the World Economic Forum’s Global Competitiveness Survey. These recent scandals consequently resulted in South Africa (SA) falling in rank from number one (1) out of 138 countries in 2016 (Schwab, 2016) to number 49 out of 141 countries in 2019 (Schwab, 2019). This is cause for serious concern, since governance is the life-blood of any organisation if the definition by The Chartered Governance Institute (TCGI) is conformed to (TCGI, 2020). The TCGI (2020) defines corporate governance as the process of how an organisation is governed, controlled, directed and held to account (IIA, 2012i) in line with its mandate and objectives, and identifies the decision-making structures. 13 The King IV Report (2016:p11) defines corporate governance “as the exercise of ethical and effective leadership by the governing body towards the achievement of ... ethical culture, good performance, effective control and legitimacy.” Governance plays an extremely vital role in how organisations are controlled and directed (TCGI, 2020) and could mean the difference between the sustainability of an organisation and its demise as in the case of Nkonki Incorporated and SAA (Burger-Smidt, 2018) & (Thabane & Snyman-Van Deventer, 2018). Although the South African Johannesburg Stock Exchange (JSE) listing requirements mandates companies that are listed with them to comply with the King Code, it is however not a requirement for government organisations, although these organisations are strongly encouraged to apply the recommended governance practices (Makgabo, 2007). Internal audit is a vital component of governance structures within organisations (IIA, 2016a). Key aspects of the definition of governance relates to and overlaps with the definition of internal auditing as it pertains to the improvement of governance, control and risk management processes and effectiveness (IIA, 2016b). Specifically within the public sector, the IAF provides support to the organisation’s governing bodies through the provision of assurance and consulting services on the effectiveness of risk management, controls and governance processes (IIA, 2012i). The Auditor-General South Africa (AGSA) in its 2018/19 Consolidated General Report on national and provincial audit outcomes report, attributed the improvement of departments’ governance to concerted efforts made by audit committees (AC) and the assurance provided by internal audit (AGSA, 2019a). In essence, internal audit could be regarded as what one could call the guardians of governance. The internal audit profession in SA (IIA, 2016e) is governed by amongst others the internal audit professional body, which is the Institute of Internal Auditors South Africa (IIA SA) and forms the local chapter of the Global Institute of Internal Auditors (IIA) based in the United States of America (USA) (IIA, 2020a). The profession of internal audit in the South African public sector is further directed by local statutes, being the Public Finance Management Act (1999) (PFMA) and the Treasury Regulations (2005) (TR) inter alia. 14 The IIA goes as far back as 1941 (IIA, 2020a) and has a mandatory Framework called the International Professional Practices Framework (IPPF) that requires internal auditors who are members of the IIA to conform to it (IIA, 2016d). The IPPF consists of both mandatory and recommended guidance (IIA, 2016d). The International Standards for the Professional Practice of Internal Auditing (ISPPIA – hereinafter referred to as the Standards) of the IIA which forms part of the IPPF (2016d) mandatory guidance has been adopted by the South African government and incorporated into section 3.2.6 of the TR (Treasury, 2005). The IIA has made the development and implementation of a quality assurance and improvement programme (QAIP) mandatory in term of standard 1300 of the ISPPIA (IIA, 2016a). This augments the importance of measuring the quality delivered by internal auditors to their key stakeholders. The IIA’s quality assurance assessment criteria consists mainly of the Standards against which internal auditors and the IAFs’ effectiveness are measured (IIA, 2016e). The question however remains whether the Standards alone adequately and comprehensively measure internal audit effectiveness and the value it adds (Fahlén & Langell, 2014) or does it require an accompanying tool to further enhance its implementation? From the study by Kasim and Hanafi (2012) there appears to be inadequate or insufficient practical tools and techniques available to support South African government departments IAF’s in conducting quality assurance assessments. This is further evidenced by the results emanating from Mbewu and Barac (2017) research where no form of quality assurance assessments was performed by the South African municipalities’ IAFs that partook in their study. Quality assurance mechanisms efforts are directed at overall improvement of processes and ultimately performance (Karapetrovic & Walter, 2000). It is not enough to have adequate measurement criteria, but what’s needed is a tool that aids in practically implementing (Hudson, Hunter, & Peckham, 2019) the said quality criteria (Vidhya, 2017), as this is what can inform effectiveness measurement (Marais, 2004). Continuous measurement and scrutiny of performance criteria according to Marais (2004) forms part of monitoring quality systems. The question should be, what constitutes value add, as this is perspective dependent and should be focused from the perceptions of the IAF’s key stakeholders. 15 1.1.1 What factors or indicators contribute to internal audit effectiveness (IAE)? A number of studies on factors influencing internal audit effectiveness (IAE) around the world has been conducted. Some of the pertinent findings emanating from those studies conducted in Greece, Vietnam and Ethiopia highlight the independence of internal audit (Huong, 2018), support by management (Mihret & Yismaw, 2007), internal audit competence (Mustika, 2015), internal audit quality (George, Theofanis, & Konstantinos, 2015), inter alia as indicators contributing to and influencing IAE. Another research study by Alzeban and Gwilliam (2014) revealed that management support was the key driver of IAE, whereas Mustika’s (2015) results found no correlation between management support and IAE. A number of authors have emphasised the importance of having quality IAF’s (Huong, 2018; Mihret & Yismaw, 2007); however adequate quality assurance tools do not exist that include criteria to assess indicators of effectiveness (Kasim & Hanafi, 2012). 1.1.2 What quality assurance tool will aid internal audit in adequately assessing its effectiveness and conformance to the IIA Standards? The IIA Standards (2016a), Standard 1300 requires that internal audit develops and maintains a QAIP. In order for internal audit to be a trusted advisor (IIA, 2016c) and provide quality services to its audit clients, be it consulting or assurance (IIA, 2016b), they need to be adequately equipped. This is a huge capacity challenge for many IAFs within the public sector, and their sizes and complexities differ (Mbewu & Barac, 2017) with many not adequately resourced from both a financial and human resources perspective (IKUTU & Treasury, 2014). Quality assurance processes are designed to evaluate IAE and quality (IIA, 2016a), and if not done, opens it up to inferior delivery of services. There are many public sector IAFs that are ineffective (Erasmus & Fourie, 2014) and are not implementing QAIPs and conducting quality self-assessments of their work (Mbewu & Barac, 2017), and the reasons why this is not being done is being questioned and remains an ongoing debate, hence the purpose for this study. 16 1.2 Research conceptualisation 1.2.1 The research problem statement The South African Government are experiencing increasing governance related issues, which have in recent times seen many state institutions facing serious financial and operational related challenges to the point of the National Treasury bailing them out (Devenish, 2018) (Mutize, 2017). This has had a huge strain on the fiscal budget of the country according to Mutize (2017). A recent report by the AGSA released alarming audit results for the 2018/19 financial year, one of which points out that government departments incurred increased irregular expenditure of R11 billion more than the prior year and is a serious cause for concern (AGSA, 2019b). A number of factors contribute to the poor quality and as a result, effectiveness of internal audit, particularly a lack of resources (IKUTU & Treasury, 2014) (Mbewu & Barac, 2017), independence of and competence of internal audit (Mustika, 2015). The latter is largely based on the capability of internal audit to identify risks and control deficiencies in order to further advise management and help them improve an organisation’s operations (IIA, 2016b). Mihret and Yismaw (2007) further attributed inadequate management support and the quality of internal audit as factors that negatively affect IAE. The IIA Standards (2016d) require of an IAF to put in place a QAIP in terms of Standard 1300 that measures internal audit as a whole against the IPPF (2016d). This means that if internal audit quality is measured, this will enable internal audit to not only be cognisant of its areas of weaknesses, but also put in place adequate measures to address these shortcomings and allow the AC to monitor internal audit’s progress thereon (IKUTU & Treasury, 2014). The indicators of IAE include competence (Mustika, 2015), internal audit quality, its independence (George et al., 2015), as well as management support (Huong, 2018). However, internal audit quality, if assessed using the IIA Quality Assurance Manual (2016), includes the evaluation of all these indicators, but criteria pertaining to management support needs enhancing. According to Mihret and Yismaw (2007), an effective IAF contributes immensely to the overall improvement of an organisation’s operations and control processes (Mihret & Yismaw, 2007). 17 The IIA Standards require internal audit to add value and help an organisation achieve its objectives by evaluating and improving its risk management, governance and control processes (IIA, 2016b). However, many government departments' IAFs are not performing optimally (Erasmus & Fourie, 2014) and are unable to assess the value added and their effectiveness for further improvement (Mbewu & Barac, 2017). Furthermore, the implementation of professional guidance and prescripts is a huge concern in the public sector and the same also for IAFs. With specific reference to quality assurance self-assessments, the Institute of Internal Auditors Quality Assessment Manual was revised in 2016 to assist IAFs to conduct self-assessments; however, it appears that the implementation part of the manual requirements remains a challenge (Mbewu & Barac, 2017). Hence, the need for a practical self-assessment quality tool that will enable internal audit to assess their contribution to the achievement of an organisation's objectives. 1.2.2 The research purpose (aim and objectives) statement The purpose of the research is to propose a tool through the development of a quality assurance self-assessment preparation tool (QASP) for IAFs in South African government national departments. This will capacitate IAFs in the public sector to be in a position to conduct their own self-assessments on the quality of the services they provide and its effectiveness (IIA, 2016a). There are presently inadequate or no practical tools for public sector IAFs to utilise in efficiently conducting a self-assessment of their work (Mbewu & Barac, 2017). This is presently a challenge and if addressed will aid the AC to have a more directed approach by using the result produced by this tool as the basis for a monitoring mechanism in identifying, assessing and tracking the gaps hindering the effectiveness of IAFs within government departments for further improvement, rather than having their attention scattered. The intent is to explore the design of a QASP that will enable internal audit to evaluate their effectiveness and adherence to the IIA Standards, using the IIA Quality Assessment Manual (2016) as a baseline that will ultimately result in internal audit adding value to organisations by assisting them in achieving their objectives (IIA, 2016b). In addition, to ascertain using qualitative strategies the need for enabling tools to measure the quality of IAE. 18 The research is to be performed by means of qualitative methods, thus using semi- structured interviews with a purposive sample of participants that will aid in ascertaining the adequacy of the QAIP in relation to indicators of IAE. It will help realise if any gaps exist in the existing Quality Assurance Assessment Manual (2016) that is crucial and fundamental to promote the effectiveness and quality (IIA, 2016e) of IAFs within government departments. To derive the conceptual frame for undertaking this study it is imperative to establish the extent to which the IIA’s Quality Assessment Manual (2016) includes indicators of IAE and ease of conducting a self-assessment that generates a result. This process will further help inform a practical quality assurance tool that public sector internal auditors can use to perform their own self-assessment in preparation for their external quality assurance reviews (EQAR) as required by the professional body, the IIA at least once every five years. Furthermore, such a self-assessment can be independently validated (IIA, 2016h) rather than have a full external review done (IIA, 2016g), thus saving costs to departments (IIA, 2016h). 1.2.3 The research questions as well as where applicable accompanying research hypotheses or research propositions 1.2.3.1 Question 1: What factors or indicators contribute to internal audit effectiveness (IAE)? Proposition: Internal audit effectiveness indicators are critical to the well-functioning of IAFs to be able to provide adequate assurance and consulting activities. 1.2.3.2 Question 2: What quality assurance tool will aid internal audit in adequately assessing its effectiveness and conformance to the IIA Standards? Proposition: The development of a QASP tool will aid internal audit in adequately assessing its effectiveness and conformance to the IIA Standards. 1.3 Delimitations, assumptions and limitations of the research study 19 The delimitations, assumptions and limitations of this research study are as follows: Assumptions Many public sector IAFs within the South African context lack the relevant competencies to conduct quality assurance assessments (Mbewu & Barac, 2017) and (IKUTU & Treasury, 2014) where they measure their conformance to the IIA Standards and it effectiveness, incapacitating them. Thus, not having adequate automated quality assurance self-assessment tools to administer and prepare them for this process exacerbate this issue (Kasim & Hanafi, 2012). Hence, the purpose of this study is to arm internal audit with necessary and adequate tools and techniques that will empower them to facilitate and implement adequate quality assurance self-assessments. Delimitations This study focuses only on capacitating IAFs in national government departments in the South African public sector and not any other spheres of government with the aim to propose a practical, automated QASP tool. Limitations This tool will not endeavour to test the effectiveness of the proposed tool; but rather to obtain the views and perceptions of IAFs on the feasibility of the proposed tool. Testing the effectiveness of the tool therefore falls out of the scope of this study, as this process will entail a longitudinal study, but could be explored for future research purposes. This study is further limited to only the assessment criteria informed by the IIA Standards to ensure a succinct assessment. A further limitation is the limited number of national departments that participated in this study, although the results cannot be generalised it is arguably relevant to other national departments. 1.4 Significance of the research study Internal audit is seen as an imperative governance pillar in the governance structures of government within SA (IIA, 2012i). Hence, the focus on well-run IAFs should be augmented through the execution of quality assurance reviews, as internal audit supports the highest governance structures within government. It is for this reason that developing a practical and automated QASP tool for the South African public sector 20 IAFs will not only capacitate internal audit, but aid them to ascertain their areas for development. This will put them in a position to introspect, be aware of and implement adequate control mechanisms to address any known gaps, and as a result enhance the value they deliver to their auditees, as highlighted in Standard 2000 (IIA, 2016a). It is in essence the need to ensure that the organisation’s performance is enhanced and objectives achieved continually that activate a demand and necessity for adequate quality systems. Hence, assessing internal audit’s quality and ultimately its contribution to the organisation, aids in ascertaining whether internal audit is operating efficiently and effectively and will allow them to set in motion control activities to address any gaps identified to better equip them to add value (IIA, 2016a). Developing the proposed tool will provide a systematic process for internal audit to report their shortcomings and needs from an informed standpoint to management and the AC whom are the decision- makers and hold the power to provide the necessary resources internal audit may require in order to improve their status quo. 1.5 Preface to the research report To this end, the report has six chapters. Following this introductory chapter, Chapter 2 provides a literature review covering the problem, the past studies, the explanatory framework and the conceptual framework. Chapter 3 discusses the research strategy, design, procedures, reliability and validity measures as well as limitations. Chapters 4 and 5 present and discuss the findings, respectively, to interrogating the research questions while Chapter 6 summarises and concludes the research. A detailed review and discussion of literature were performed to ascertain the extent of research conducted in the area of internal audit related quality assurance tools, of which there were very few that related to the development of practical and automated assurance tools. Using the IIA Quality Assessment Manual (2016) as the underlying framework, the idea supporting this research is to capacitate IAFs enabling them to practically conduct quality assurance self-assessments in preparation for their EQAR, which is identified as an important area for improvement in the IKUTU and Treasury report (2014). This tool should largely provide the AC with a mechanism to assess 21 internal audit quality and effectiveness and to hold them to account through monitoring their progress in addressing the findings emanating from the proposal tool (Marais, 2004) (Vidhya, 2017). This study aims to obtain an in-depth understanding of an extensive account of current senior internal auditors’ experiences and views on quality assurance processes and its implementation, as well as the importance of assessing internal audit conformance to the IIA Standards (2016a) and its effectiveness. The latter is what provides an indicator of the impact internal audit’s efforts have on the organisation achieving its objectives (IIA, 2016b) For purposes of this study and to achieve the aforementioned rich data, the semi-structured interview method will be applied in the collation of information. Since quality assurance processes require persons with in-depth knowledge and experience in the space of internal audit and quality assurance, only senior internal auditors currently in practice in national government departments within SA will be interviewed. 22 2 LITERATURE REVIEW This chapter has three broad objectives; namely to understand the research problem, to identify the knowledge gap, and to develop a framework for interpreting the research findings. Specifically, in Section 2.1, the research problem is explained. In Section 2.2, literature on studies that have attempted a similar study or research are reviewed. With information arising from Section 2.2, qualitative attributes that are key to this research are identified in Section 2.3 as well as a framework that we will used to interpret the research findings in Section 2.4. Internal audit role in governance in the public sector and its challenges Public sector governance takes into account how organisational goals are established and attained and how effectively it’s managed (Asare, 2009). Internal audit has been identified as a key governance pillar of sound governance based on its assessment of business operations of which the results are communicated to the AC and senior management (Coetzee & Erasmus, 2017). Internal audit’s contributions to governance are binary in that it provides independent and objective appraisals of the organisation’s governance system and are catalysts for improvement of risk management, governance and controls (Asare, 2009). Maintaining effective governance within public institutions has become more and more challenging over the years with the increase in irregular activities taking place in public institutions (Zulu, 2019) owing to inadequate leadership and unethical decisions (Njobeni, 2016). This has become more apparent through reports produced by organisations such as the Public Protector on allegations of state capture (Public Protector, 2016), which triggered the Zondo Commission of Enquiry (President of the RSA, 2018) with a number of other Commissions set up over recent years to probe public resources irregularities. This is cause for concern, as a number of public institutions have become technically insolvent due to financial mismanagement, such as the SAA (Zulu, 2019) and Eskom to name a few (Public Protector, 2016). The relevance and importance of internal audit in organisations are increasingly being questioned putting added pressure on them to perform (Boţa-Avram & Palfi, 2009). With all that has happened in the government domain, a number of questions have been raised as to where the auditors were to prevent these perverse governance mishaps from occurring and to the extent it has, which is a valid ask (von Eck, 2019). Internal audit’s 23 role is to provide assurance and consulting activities on the effectiveness of governance, risk management and controls of the organisation to oversight bodies charged with the responsibility for governance (IIA, 2016a). The AGSA (2019a) consolidated general report on audit outcomes highlighted with concern management’s failure to adequately address findings previously raised, which has indeed hindered internal audit and the ACs’ efforts to effectively discharge their monitoring activities. Increasing demands are placed on internal audit to provide much needed unbiased and value adding insights and foresights (IIA, 2016a). This requires an enabling environment that will assist internal audit to effectively conduct their work in meeting the mission of internal audit, which is to protect and enhance the value of organisations (IIA, 2020b). This however will depend on the resources and tools available to internal audit that will empower them to improve their service offerings by conducting quality assurance self- assessments on their IAFs in order for them to direct their efforts toward intentional and needed continuous improvements (National Treasury, 2018). South African legislation supports the IIA Standards (2016a), as it has incorporated section 3.2.6 in the Treasury Regulations (2005) enforcing internal audit engagements to be conducted in accordance with these Standards. Despite no specific mention made in the TRs (2005) of quality assurance processes, it is highlighted as a process to be followed by the National Treasury (2009) Internal Audit Framework, section 3.7. With all the professional guidance and prescripts designed to improve the effectiveness of IAFs within the South African public sector, this study aims to augment this by developing a practical QASP tool that can be implemented with ease by these functions. Internal audit effectiveness (IAE) indicators An important part of measuring the quality of internal audit’s deliverables is identifying and understanding the indicators and value chain of what makes an IAF effective. Lenz, Sarens and D’Silva (2014), and Roussy and Brivot (2016) associated IAE indicators to input, such as having people with the right skills set and knowledge, processes that entail having the right tools, technology and adequate procedures; and output deliverables which refer to the recommendations, reports, advice and presentations made to key stakeholders. The value added is all about how one’s stakeholders receive it and how useful it is to them and their work. This is why managing expectations with their key 24 stakeholders is of paramount importance (Lenz & Hahn, 2015) as it contributes to being relevant and effective as an IAF. Bardhan, Vargus, Pizzini and Lin (2011) explain quality assurance as techniques that enable the effectiveness of the IAF’s operations and include activities that consist of client feedback, peer reviews, independent review of working papers and direct supervision amongst others. The research results of George, Theofanis and Konstantinos (2015) based on the Greece context, highlight that quality of internal audit, management support, internal audit independence and its competence are all indicators that positively relate to the effectiveness of internal audit (Huong, 2018). All these indicators are aligned to the IPPF (2016d); however, when reviewing the IIA Quality Assessment Manual (2016), the criteria to assess management support is not adequately articulated and covered therein to invoke a glaring gap, should this be an area internal audit is struggling with. They further found that the indicator on internal audit quality is associated highly with the other three mentioned indicators (George et al., 2015). Although Mustika (2015) found management support to have no effect on IAE, their study found independence, competence of internal audit and strong relationships between external and internal audit to be highly correlated to IAE within the Indonesian context, with the latter indicator contributing to internal audit quality. Earlier research by Mihret and Yismaw (2007) in the Ethiopian context reveals that quality of internal audit and management support is positively associated with the effectiveness of internal audit. Hence, from the research materials presented, the IAE indicators for consideration and for the purpose of this research include quality of internal audit, independence, competence and management support. When comparing these indicators to the current IIA Quality Assessment Manual (2016) criteria, the indicator on management support could not easily be traced into the said assessment, notwithstanding that this as criteria is not easily measured, and hence this indicator being expanded on under internal audit governance should be considered for expansion. Management support of the work of auditors is inadequate as reported by the AGSA (2019a) consolidated general report, due to their failure to address audit findings raised through the implementation of adequate controls, which is one of the critical aspects toward overall organisational improvement. Another aspect that helps leverage the work done by internal audit is the reliance placed on internal audit’s work by the external auditors in terms of the International Standard 25 on Auditing (ISA) 610 and the IIA’s Standard 2050, particularly in the public sector. Their reliance is mainly premised on the measurement of quality and effectiveness outcomes of the work produced and the behavioural actions of IAF and internal auditors (Lenz & Hahn, 2015). This will further encourage the combined assurance principles to be applied (Huibers, 2015) (IoDSA, 2016) and will help save organisations costs, whilst ensuring increased assurance and or consulting coverage, thereby creating efficiencies within the assurance processes. Internal audit quality assurance and improvement programme (QAIP) A quality assurance process in respect of internal audit refers to the assessment of the IAF’s conformance to amongst others the IIA Standards (2016a), Code of Ethics and its effectiveness (IIA, 2016a). Standard 1300 of the IIA Standards (2016) requires chief audit executives (CAEs), the heads of IAFs (also referred to as internal audit directors) to develop and maintain a QAIP. Internal audit should always strive to improve their service offerings through continuous measurement against set criteria to ensure its quality is constantly enhanced (Pitt, 2014). Quality can be defined in various contexts, but it ultimately aims to measure performance (Al-Amri, Jani, & Zubairi, 2015). A study by Elassy (2015) states that there is no simple and straightforward definition for quality, as the difficulty with it is that it is a relative concept and can mean different things in diverse contexts. He further goes on to explain five various definitions of quality, with one particular definition that could relate to internal audit quality assurance, being quality in the context of conformance to standards. Despite this, Lenz and Hahn (2015:p7) gives meaning to quality as follows: “quality in a product or service is not what the supplier puts in. It is what the customer gets out and is willing to pay for”. A distinction is made by Elassy (2015) between quality and standards, with the former being the process, whereas outcomes relate to the standards, this elevating the role of the quality assurer to ensure that the process meets a high standard. Karapetrovic and Willborn (2000:p679) state, “quality audits are prominent and proven management tools for assessing compliance and effectiveness of quality systems” and further explain that its objective is ultimately to help improve an organisation’s overall performance. The IIA Netherlands Chapter has directly associated quality with the 26 effectiveness of the IAF (Netherlands, 2016). The IIA (2017:webpage) has stated that “for an IAF to be considered effective, all Principles1 should be present and operating effectively”. One of these principles that this study focuses on in particular, relates to internal audit demonstrating continuous improvement and quality (IIA, 2016d). Measurement of performance is imperative for further improvement of any process, thing or person (Al-Amri et al., 2015). The meaning of quality varies dependent on the context it’s used in, such as a product being defect-free does not mean that the product is competitive if it does not have the most contemporary features (Hoyle, 2008). Since the context in which quality is used is important, the definition by Hoyle (2008) on conformance to requirements pertains to a service of which internal audit offers their audit clients. Effectiveness on the other hand according to Bartuševičienė and Šakalytė (2013) entails measuring the degree to which goals are attained by organisations. This means that for internal audit to be effective it needs to help the organisation achieve its objectives (IIA, 2016a) through the services that it offers the organisation. This is in line with the IIA definition of internal audit (IIA, 2016b). Kasim and Hanafi (2012) once wrote that conformance to the internal audit International Professional Practices Framework (2016d) (IPPF) is the best measure of the effectiveness of internal audit. Alzeban and Gwilliam (2014) mention that internal audit quality hugely affects IAE. Hence, for purposes of this study internal audit quality and IAE will be regarded as the same, as the focus is on the criteria or indicators of IAE. The outcomes of the study by George et al. (2015) are in line with the instrument developed by Kasim and Hanafi (2012), as their model is premised on the IPPF. However, this instrument consists of just a set of criteria similar to that of the IIA’s Quality Assessment Manual (2016) and lacks a quality assurance self-assessment tool. Since the IIA’s Quality Assessment Manual’s (2016) assessment criteria will underpin the proposed practical QASP tool, it is notable that all the aforementioned IAE indicators (George et al., 2015) are reasonably covered, except for the IAE indicator on management support that needs to be expanded on to make it explicit. This tool will enable internal audit to conduct self-assessments that will prepare the IAF by simulating 1 Demonstrates integrity | Demonstrates competence and due professional care | Is objective and free from undue influence (independent) | Aligns with the strategies, objectives, and risks of the organization | Is appropriately positioned and adequ ately resourced | Demonstrates quality and continuous improvement | Communicates effectively | Provides risk-based assurance | Is insightful, proactive, and future-focused | Promotes organizational improvement. 27 the quality assurance review for the actual EQAR, and will ensure that the IAF conforms to Standard 1300 on QAIP (IIA, 2016a). The IIA welcomes the customisation of the application of the IPPF (2016d), provided that conformance to the Standards are not in any way jeopardised (IIA, 2016e). Marais (2004:p104) quoted a fundamental statement that “a profession can have great standards, but without credible mechanisms to enforce compliance, they lose their effectiveness and their status as a mark of professional quality”. This means that adequate systems that enable enforcement and prudent implementation of the standards and performance criteria that measure effectiveness and value added are underpinned by credible systems. These factors are interdependent for its successful implementation. To attain optimal levels of performance that are beneficial for an external assurance review, Sarens and Abdolmohammadi (2011) put forth that the identification and reporting of leading practices and benchmarking should be included in their area of work to ascertain and adopt efficiencies and improve effectiveness of the IAF. The current IIA Quality Assessment Manual (2016) allows IAFs to identify and explain any potential opportunities for improvement and leading practices that they may have adopted. A quality system according to Ramly, Ramly and Yusof (2008) is a tool that assesses activities against quality criteria and identifies key areas for improvement. Karapetrovic and Willborn (2000) further support this through their explanation that independent and objective verification, collation and evaluation of audit evidence against set audit criteria is a form of quality audit. Similarly, Fahlén and Langell (2014) emphasised a need for further research to be conducted in the area of complementary quality tools with a focus on improvement suggestions opposed to just compliance checks. Karapetrovic and Willborn (2000) emphasised that a perfect system does not exist, as areas for improvement are always necessary. Assessing internal audit’s quality and contribution to the organisation aids in identifying areas for further improvement and ultimately value added (IIA, 2016a, 2017). There is a need to ensure that the organisation’s performance is enhanced and objectives achieved continually that sets in motion a demand and need for quality systems (Ramly et. al, 2008). Fahlén and Langell (2014) argues that internal audit could potentially be used as a form of tool for improvement and value adding, but further highlights that no evidence exists that internal audit continually adds organisational value. 28 Karapetrovic and Willborn (2000) emphasised that applying the applicable quality assurance measures will aid in ensuring that audit processes and outcomes are of quality and build customer confidence in the services offered. The Organisation for International Standards (ISO), an international body that aims to issue global standards on standardisation, issued ISO 9001 which is a set of principles-based criteria for managing quality (ISO, 2015). However, an article by Jack Dearing (2007) questions the cost-benefit aspect of implementing this ISO stating that there is indeed room for improvement. Although Bota-Avram and Palfi’s (2009) research highlighted that instituting quality assurance sections within the IAF will enhance internal audit performance, this is a luxury that the SA public sector does not have. In the absence of this ideal, this study seeks to fill this gap through the development of an automated excel-driven QASP tool that could push for independent validation, which is less costly (IIA, 2016h). This will enable public sector internal auditors to conduct their own preliminary quality assurance self-assessments that will aid in the identification of focused areas for improvement and that can be regularly monitored by the AC (IKUTU & Treasury, 2014) toward preparation for the actual EQAR (IIA, 2016g) or Self- assessment with Independent Validation (IIA, 2016h). Adopting and adapting this tool based on IAF’s needs will help internal audit move towards becoming a creator of value opposed to a resources consumer (Boţa-Avram & Palfi, 2009). From an internal audit perspective, the quality assurance self-assessments are expected to provide much needed insights and foresights (IIA, 2016a) that will not only help internal audit identify its areas of weaknesses, but also provide potential solutions for future improvement and augmentation of its effectiveness. The QAIP has a two- pronged approach which ultimately evaluates whether or not internal audit is attaining its own objectives including those of the organisation it audits (IIA, 2012m). This study aims to augment literature and find a credible and practical QASP tool that will advance and enable IAFs to effectively, efficiently and economically, conduct quality assurance self-assessments of their IAF and auditors. It further aims to help them systematically evaluate their effectiveness, monitor and report on gaps to their key stakeholders for continuous improvement. 29 2.1 Research problem analysis Internal auditors in the South African public sector are increasingly facing challenges that inhibit inter alia their effectiveness. A study on IAE by Mbewu and Barac (2017) found that organisational governance is amplified when IAFs are effective. Gramling and Van der Velde (2006) further support this, where they highlight that having an IAF that is weak and of poor quality significantly weakens the internal control systems of that organisation. Mbewu and Barac (2017) research found that there is a lack of or no quality assurance (internal or external) assessments conducted on the IAFs in SA, severe internal audit resource constraints and developmental programmes, which all significantly hinders internal audit effectiveness. The quality of internal audit is a key factor that affects the function’s effectiveness (Alzeban & Gwilliam, 2014) and requires adequate practical tools that are able to evaluate the IAE, of which is in short supply, as major weaknesses are prevalent in the existing tools (Kasim & Hanafi, 2012). There is no perfect system and room for continuous improvement is critical (Karapetrovic & Willborn, 2000), as some mechanisms designed to augment internal audit’s role and responsibilities, are the very ones that attrite their effectiveness. Factors such as professional requirements necessitate internal audit to adhere to certain standards, such as the IIA Standard 1300, stating that internal audit should put in place a QAIP (Sarens, Jallow, Abdolmohammadi, & Lenz, 2012) (Sarens et al., 2012). However, due to a lack of functional support, systems and resources (financial and human), it does not enable internal audit efficaciously to fulfil their responsibilities (Mbewu & Barac, 2017). Notwithstanding the competing priorities that departments have to endure in allocating extremely limited resources to activities that will best ensure the effective delivery of services to the public. This being, distribution of funding to departmental core activities over the allocation of these same resources to support functions, of which internal audit is one of them, and which is similar to the scenario painted by Merino (2018). Internal audit resources are a challenge according to Mbewu and Barac (2017), and one of these key resource constraints relates to lack of quality assurance tools that will help enable internal audit to diagnose their areas of weaknesses pertaining to their effectiveness and the value they add. If they do not know what they are doing or where 30 they are going wrong, how will they know what areas within their function to improve in order to direct their already stretched resources to attain organisational objectives. Apart from resources being a challenge (IKUTU & Treasury, 2014), research revealed that many IAFs fail to conduct quality self-assessments to ascertain their level of effectiveness (Pitt, 2014). This is due to some IAFs being small in nature, assessments requiring increased time for compliance at the expense of core audit work and lack of management support (Baharuddin, 2014). Having an adequate quality assurance tool in place is imperative, and equally important is having all the required criteria that will not only measure conformance to the IIA Standards (2016a), but also criteria that assesses effectiveness and the value internal audit adds to its auditees (Kasim & Hanafi, 2012). Kasim and Hanafi, (2012) further stated that no consistency exists explaining what audit quality characteristics are. This could be due to set audit criteria according to Ramly et al., 2008 being biased (Karapetrovic & Willborn, 2000), and which do not enable enhancement of performance (Fahlén & Langell, 2014). Marais (2004) interpretation of the IIA Standard 1300 on the QAIP alludes to its intention to improve quality. The IIA’s IPPF (2016d) is a mere baseline for quality measurements to inform quality systems; however, a number of quality assurance systems do not include all of the IIA’s IPPF requirements or criteria, and these systems further do not quantify or allocate a rating to their measures (Kasim & Hanafi, 2012). Pitt (2014) also supports this notion that conformity to the IIA Standards alone are inadequate and should include efficiencies created and effectiveness of the function. This could be due to quality not having a standard definition (Elassy, 2015). Sarens and Mohammad (2011) is of a slightly different view in that adopting the full IIA Standards set will give increased coverage of quality aspects. The IIA Standards (2016a), Standard 1300 is very clear in that the QAIP does not only focus on conformance to the Standards and evaluates application of the Code of Ethics, but should also include the evaluation of how efficiently and effectively the function operates with due consideration to identifying opportunities for process improvements. This means that the QAIP's intention was always to focus on quality beyond mere conformance to the IIA Standards (2016a). There are other spin-offs from enhancing internal audit quality, in that other assurance providers, such as external audit will be able to comfortably use the work of internal 31 audit if their function is effective and produces quality results (IAASB, 2014). Endaya and Hanefah (2016) writes that only when internal audit is fully effective, will external audit rely on and use the work of internal audit. This entails conforming to the IIA Standards (2016a) if one is to consider the criteria outlined in the International Standards on Auditing (ISA) 610 applicable to external auditing (IAASB, 2014). Pitt (2014) however argues differently in that conformance to the Standards should not be the performance objective, but rather a premise for how their operations are run. This means that performance is more than just meeting the Standards, which is perhaps why most existing quality systems are not reaping the intended objectives. It is known that professional governing documents may be well written with the greatest of intention, but is futile if not institutionalised and supported with adequate tools or systems that will enable users thereof to effectively implement it in practice. The manner in which existing quality tools are designed may have unintended consequences resulting in the tools’ failure to promote effectiveness through the attainment of the intended set objectives, as well as not being able to identify areas for further improvement, although the final results of any system should be to ensure organisational sustainability (Ramly, et al., 2008:Abstract). The IIA Standards (2016a) are there to guide internal audit from amongst others a technical and behavioural perspective, however putting it into practice is a separate process. Internal audit has become so good at assessing the organisational business processes that they have neglected to take stock and assess their own effectiveness and the quality they deliver to their auditees (Peña, 2019). This may be due to their lack in skill to apply IIA Standard 1300 in practice or lack of resources; but whatever their reason is, it must be done if they are to remain relevant (IIA, 2016a). A study revealed that failure to enforce quality standards could lead to the inefficient and ineffective use of resources by internal audit (Peña, 2019). It is due to the aforesaid that it is important to not only focus on performance and quality criteria, but also the practical tools that will assist in efficiently assessing the set criteria. According to Peña (2019), a number of significant positives could be derived from having conducted a quality assurance process, such as increased training and resources that will contribute to building internal audit. It is for the above reasons that there is a significant need for an adequate, credible and practical 32 quality assurance tool in the public sector that will enable IAFs to conduct quality self- assessments of their effectiveness and conformance to the IIA Standards (2016a). What is even more important is the actual process of developing the intervention that will contribute to the impact the tool will have on improving the overall effectiveness of IAFs within the public sector and its conformance to the IIA Standards (2016a). The outcome explained by Belcher and Palenberg (2018), as the causes of impact, are premised on fully conforming and effective public sector IAFs. This aforesaid will be as a consequence of the review on the adequacy of the existing quality assessment criteria of the IIA (IIA, 2016e) to ascertain whether it amply includes criteria that assess IAE indicators. To attain the outcomes, it will be imperative to review the IIA's Quality Assessment Manual (2016) in order to ascertain if it incorporates assessment criteria pertaining to IAE. The first point of call is to identify the factors or indicators of what constitutes IAE, and then to determine if the IIA’s quality assessment criteria include these factors or indicators. For internal audit to add value and provide advice to its clients, it needs to be operating optimally. Hence, having adequate criteria alone is not sufficient, but actually conducting the self-assessment against the enhanced quality assessment criteria is imperative. However, existing tools are inadequate (Kasim & Hanafi, 2012) to enable internal audit to conduct quality self-assessments, and by converting the IIA’s Quality Assessment Manual (2016) into a practical self-assessment preparation tool could be the solution to moving public sector IAFs toward becoming effective and operating optimally. Although Mbewu and Barac (2017) suggest that training on quality assurance reviews could be undertaken by IAFs within the South African local government to address the gap of them performing no to limited assurance assessments, this may not be the most economical solution since funding in general is a challenge, let alone funding for training. Hence, the purpose of this study is to develop such a practical self-assessment preparation tool. 33 2.2 Research knowledge gap analysis This section will aid in identifying the research knowledge gap through the analysis of existing literature. A qualitative approach will be adopted using semi-structured interview methods, transcribing and analysing the interview information into themes (Mbewu & Barac, 2017) that will produce findings, recommendations, and conclusions and further evaluations of the extent of existing quality assurance mechanisms. A number of studies have explored factors affecting IAE (George et al., 2015), however limited studies have considered developing instruments that could drive effectiveness (Mbewu & Barac, 2017). Although Kasim and Hanafi's (2012) research considers a quality assurance instrument, it does not allow for an automated self-assessment that prepares internal audit for the EQAR that should take place at least once every five years (IIA, 2016a). Although questionnaires are popular mechanisms used to collect and analyse data, as used by George, et al. (2015); a number of research studies have used semi-structured interviews, as a mechanism, to extract information and has quantitatively analysed results (Kasim & Hanafi, 2012), with others analysing the results using qualitative methods (Mbewu & Barac, 2017). Focus group interviews were used to conduct research (Kasim & Hanafi, 2012), whilst Mbewu and Barac (2017) used questionnaires in their interviews to collect information. The most popular mechanism used to collect research data consists of questionnaires or surveys of which mostly quantitative methods are employed to analyse such data (George et al., 2015). Kasim and Hanafi’s (2012) research is based on the Malaysian context with a minimum total sample size of 185 (60 for attribute standards and 125 for performance standards) and included a set of two (2) interviews that targeted nine (9) CAEs’ and 22 internal auditors. With Mbewu and Barac’s (2017) study, they targeted mostly senior representatives such as accounting officers, chairpersons of ACs and CAEs, as well as internal auditors from four municipalities in Limpopo in SA. Another study by George et al. (2015) used a sample size of 140 Athens Stock Exchange listed companies out of a population of 240. A mix of focus group interviews (Kasim & Hanafi, 2012) and 34 questionnaires (Mbewu & Barac, 2017) were used in the information collection process, whilst George et al. (2015) only utilised a questionnaire to collect information. Kasim and Hanafi, (2012) firstly confirmed the existing IIA Quality Assessment Manual (2016) with the originators prior to using it in the development of the instrument and then transcribed the information obtained. However, George et al. (2015) made use of descriptive statistics and regression analysis to process data. Two (2) sets of reliability tests were used for analysing the questionnaire (Kasim & Hanafi, 2012), which included Cronbach’s coefficient alpha. George et al. (2015), however used a number of statistical analyses, which included amongst others, factor analysis, varimax orthogonal to interpret results, Barlett test to ascertain correlations and Kaiser-Meyer-Olkin measure of sampling. Mbewu and Barac (2017), to unpack and analyse the data obtained in their research used thematic analysis. George, et al. (2015) found that internal audit quality (IIA Standard 1300) was not only positively correlated with IAE, but also with other IAE indicators such as the competence (IIA Standard 1200) of the internal audit team, its independence (IIA Standard 1100) and management support. All of these IAE indicators are reasonably and adequately included in the IIA Quality Assessment Manual (2016) criteria with the exception of management support, as the criteria hereon if expanded on could enhance this indicator. This means that if the quality aspect of the IAFs are addressed, so will many other IAE indicators. Kasim and Hanafi (2012) explained that a number of methods were introduced to evaluate internal audit’s quality with little success to quantify the measures. This research attempts to bridge this gap particularly as it relates to IAFs within the South African public sector. Kasim and Hanafi (2012) further confirmed the validity of the IPPF (2016d) and the IIA Quality Assessment Manual (2016) in developing their quality assurance instrument and since the latter is aligned to the recent changes in the IIA Standards (2016a), the updated IPPF (2016d) will be used in this study. It will serve as a baseline for inclusion in the development of the automated practical QASP tool and since management support is confirmed as an IAE indicator, this will be expanded on and incorporated into the said tool. Although extensive research has been conducted on IAE indicators and some research on the existing IIA Quality Assessment Manual (2016), limited research is done on the 35 development of a practical and implementable automated quality assurance self- assessment tools that will aid internal audit in conducting quality assurance self- assessments in preparation for the EQAR. Kasim & Hanafi's (2012) research highlighted that a modified measurement scale instrument could enable quantifiable measurements of conformance to the IPPF (2016d). Despite them replacing the present Likert scales due to it not being quantitative, their instrument still does not enable internal audit to conduct a self-examination of their conformance to the IIA Standards (2016a) that concludes in the automatic calculation of a result (i.e. General Conforms, Partially Conforms or Do Not Conform) (IIA, 2016f). However, this study focuses on allocating a rating to the current Likert scales with the tool automatically generating a final assessment result. South African public sector internal auditors are always assessing organisations’ compliance to prescripts, their risk management processes, governance and control systems (IIA, 2016a) that give rise to the delivery of services. However, in order to be the referee of others, they need to conduct an introspection of their services and conduct before they can evaluate others (IIA, 2016a). Hence, the need to assess the quality of an IAF and its effectiveness, as it is an imperative in order to lead by example as the guardians’ and evaluators of an organisation’s control, risk and governance systems (IIA, 2016b). With the development of an automated practical quality assurance self- assessment preparation tool, it will contribute to enhancing and strengthening public sector IAFs’ effectiveness and enable them to adequately identify and address their shortcomings that can be monitored quarterly by the AC (IKUTU & Treasury, 2014). By IAFs being externally quality reviewed, it will further increase the functions’ chances whereby, the AGSA who is the supreme audit institution of SA and the legislated external auditor of government, could use the work of internal audit in terms of their ISA 610 Standard. This is the process whereby external audit places reliance on the work of internal audit or uses direct assistance to assist in their audit coverage (IAASB, 2017). According to Gramling and Van der Velde (2006), a weak IAF can constitute a significant weakness in the control environment of the organisation. 36 Kasim and Hanafi’s (2012) study expressed a need for further research to refine their instrument. With the limited research conducted on the development of practical quality assurance assessment tools for internal audit, it necessitates a study on the development of such a tool that will help enhance internal audit effectiveness (George et al., 2015) and their conformance to the IIA Standards (IIA, 2016a). A proposed tool of this nature aims to enable and encourage internal audit to implement and conform to professional guidance. Such a tool would be able to expose any non-conformance or effectiveness gaps that can be monitored by the AC (IKUTU & Treasury, 2014) to ensure improvement of public sector IAFs. This tool will further prepare IAFs for EQARs that must be conducted at least once every five years as required by IIA Standard 1312 (IIA, 2016a). 2.3 Qualitative attributes or quantitative variables key to the research Since a qualitative approach will be used, and hence qualitative attributes apply here, the Lexico powered by the Oxford Dictionary describes an attribute as “a quality or feature regarded as a characteristic or inherent part of someone or something” or “regard something as being caused by”(Lexico powered by Oxford, 2020). The qualitative attributes of this study will include assessment criteria for IAE that will measure the quality and effectiveness of IAFs in national government departments in SA. Research by Erasmus and Fourie (2014) reveals that IAFs within national government departments are not meeting the expectations of their key stakeholders, as they are not effective in delivering their goals, which is to help government organisations achieve its objectives. The question arises whether internal audit knows and understands why they are not delivering on their objectives and how they are measuring their effectiveness. Thus, this study aims to ascertain if developing a practical QASP tool with rating scales that automatically generate a result, will aid internal audit in adequately assessing its effectiveness and conformance to the IIA Standards (2016a). Development of the said tool could be done by using as a basis, the criteria outlined in the IIA Quality Assessment Manual (2016), which is based on the IPPF (2016d) and should further include criteria to measure IAE (Lee & Quazi, 2001). Lee and Quazi (2001) state that the purpose of self-assessment tools is to measure the quality of performance, which is 37 further supported by Jørgensen, Boer and Gertsen (2003) explaining that it is a development improvement process. Government institutions’ IAFs do not have adequate resources (IKUTU & Treasury, 2014) in particular to measure the quality of services they offer (Mbewu & Barac, 2017), for example the funds to buy licences for the use of audit software to conduct certain audit processes and or quality reviews (Janse van Rensburg & Coetzee, 2016). This research however is suggesting the development of a self-assessment tool within a programme that all government departments has access to, which is the Microsoft Excel package. There is a general consensus amongst scholars that self-assessment tools are designed to identify shortcomings (Lee & Quazi, 2001) for further improvement purposes (Jørgensen, Boer, & Gertsen, 2003). Jørgensen et al.'s (2003) study reveals and advocates the importance of having the self-assessment process facilitated, however earlier research by Lee and Quazi (2001) states that the quality criteria used in the assessment should be underpinned by the actual self-assessment tool. This means that the self- assessment tool is the underlying mechanism and vehicle to deliver the frame of the tool being the quality assessment criteria. Further research by Lee and Quazi (2001) reveals that self-assessment tools developed over the years to measure quality do provide some form of benefit to organisations in the form of a dipstick high-level analysis of the issues. Organisations’ success according to Lee and Quazi (2001) is augmented when quality processes inform management practices. On the other hand, Jørgensen et al. (2003) argues that self-assessment results may reveal symptoms of challenges, but alone it is not enough to drive change or improvement, unless fully supported by management, hence the need to expand on management support criteria in the IIA’s Quality Assessment Manual (2016). Lee and Quazi (2001) advocated for the assessment criteria to be delivered through a questionnaire driven self-assessment, which is a similar approach that this study will be taking in the development of the practical QASP tool. This study takes the form of semi-structured interview questions with two (2) sets of target audience. The first set of interviewees targeted at national government departments within SA include senior internal auditors (and those that has worked in the internal audit space within the last two years) responsible for or those implementing 38 quality assurance programmes within IAFs. This includes those individuals who were previously in quality assurance and have left the internal audit space within the last two (2) years. The second set consisting of those that do not have a quality assurance programme in place. The interview questions can be found under Annexure 1.1. The interview questions will profile the interviewees by asking about the position they hold and the department they are employed by, including the number of years’ experience in internal audit. The generic questions will follow and filter the target audience into two (2) groups, those that have a QAIP in place and those that do not, as the set of questions will differ for each group, with some questions being similar. This will be followed by ascertaining if the IAF has actually implemented the QAIP processes in place. The first group are those persons working in IAFs that have a QAIP in place. This is followed by a question to determine if the AGSA places reliance on their work. Questions are posed to determine interviewees’ views of what they consider indicators of IAE are and to ascertain if these indicators highlighted by literature is included in their internal assessment and EQAR. Further questions are posed on when they last conducted an internal assessment and an EQAR, the outcome thereof, what they thought attributed to the outcome and whether the results have been communicated to the AC. Regarding a tool, questions are directed to determining if they have quality assurance tools and techniques in place to assess the IAF against the IIA Standards and IAE indicators, and whether these tools automatically generate a simulated result. The next question is to ask if their existing quality assurance tools are effective and prepare them adequately for the actual EQAR, and if using an automated quality assurance self- assessment tool will prepare them better for the EQAR. Last, but certainly a very important question focuses on their suggestions on what functionalities would enhance a QASP tool that would simplify and facilitate the implementation of quality assurance activities. The second group consists of those persons working in IAFs that do not have a QAIP in place. The questions posed to the purposively selected audience focuses on the reasons for not having a QAIP in place and would having one in place improve internal 39 audit quality. Other questions zoom into whether their IAF is effective and their view on what constitutes IAE indicators. Then importantly a question on whether or not the development of a Microsoft Excel automated QASP tool will help internal audit improve its effectiveness and quality and what functionalities in their view would simplify the implementation of quality assurance activities. The quality of internal audit is imperative to help an organisation improve its overall risk management, controls and governance processes (IIA, 2016b). For this to happen internal audit needs to be effective in the discharge of its responsibilities. The only way the IAF can ascertain this, is if it conducts some form of a self-assessment on the performance of the IAF as a whole, which is proposed by Standard 1300, requiring the head of internal audit to develop and implement a quality assurance improvement programme (IIA, 2016a). Thus making use of qualitative attributes to gain an in-depth understanding on the viability to develop a practical QASP tool aimed at helping internal audit identify its shortcomings and improving its quality and effectiveness within national government departments in SA. A quality assurance self-assessment tool that is formula-driven and scoring enabled allows for a rating and quantification of a final result, thus providing an analysis of current situation with improvement suggestions to enhance the status quo (Jørgensen et al., 2003). 2.4 Framework(s) for interpreting research findings Applying a qualitative research method using semi-structured interviews to ascertain the target audience’s view and or experiences of the need to conduct quality assurance self- assessments on their services offered and how embarking on this process will contribute to the effectiveness of IAFs within the public sector. Further, ascertaining through this mechanism what participants perceive as indicators contributing to IAE. The results will be analysed to derive findings that either support or disprove my research question(s). The IIA Standard 1300 (IIA, 2016a) requires the head of internal audit to develop and maintain a QAIP that assesses the function’s activities against the IPPF (IIA, 2016d). In earlier years an external assessment was required to be conducted at a minimum once every three years, however was later revised to once in every five years (IIA, 2016e). The purpose of the QAIP is to measure internal audit's level of conformance to the IPPF 40 (2016d) (IIA, 2016f) and add value to an organisation’s operations (Marais, 2004). However, constructing an instrument that will enable the measurement of internal audit's conformance to the IPPF (Kasim & Hanafi, 2012) will prove more effective and useful for IAFs. The aforesaid has been cemented in the IIA Quality Assessment Manual (2016) that not only encourages heads of internal audit to develop their own QAIP, but provides a framework to perform such an assessment. However this framework lacks a structured formula driven rating scale that enables internal audit to perform a self-assessment (IIA, 2016f) on its effectiveness and compliance with the IIA Standards (2016a). Improving the quality of internal audit's level of services to the organisation (Marais, 2004) by measuring its level of conformance to the IPPF (Kasim & Hanafi, 2012) is the primary objective. However, the existing framework suggested by the IIA's Quality Assessment Manual (2016) does not provide for a scientific process to arrive at an assessment scale result of Generally Conforms, Partially Conforms and Does Not Conform, which is why there is a need to develop an automated self-assessment tool. Like most professional guidance, it provides a framework within which to work; however the “how” to implement the framework in practice is not clearly documented, which is usually what public sector practitioners struggle with (Vidhya, 2017). It is for this reason that the development of an automated Microsoft Excel tool with rating scales that are premised on the IIA’s Quality Assessment Manual (2016) and that will enable public sector internal auditors to efficiently and effectively conduct a quality assurance self-assessment is paramount. This will put them in a position to not only identify their weaknesses for further and continuous improvement in preparation for the EQAR, but also quantify it (Kasim & Hanafi, 2012). Further advantages of the practical QASP tool is that it focuses ultimately on adding value to the organisation and provides opportunities for further improvement (IIA, 2016g) through the recommendations emanating from the tool. This tool will not only assesses internal audit’s achievement of their own objectives, but that of the organisation’s as well (IIA, 2012m). Although many IAFs will opt to conduct a self- assessment with independent validation (IIA, 2016h), these mostly focus on conformance to the IIA Standards as the final goal of performance, rather than increased focus placed on achieving efficiencies and effectiveness (Pitt, 2014). Pitt 41 (2014) further emphasises the importance of basing quality (i.e. outputs and outcomes of planned goals) on stakeholder expectations and value adding services, which she labelled in her book as demand-based drivers of quality as opposed to conformance drivers (i.e. conformance to the IIA Standards). This is a very important statement, as this goes beyond mere conformance, which in itself is fundamental to ascertaining internal audit weaknesses. Internal audit cannot be perfunctory in their duties, as this will render them irrelevant and ineffective in propelling the organisation toward achievement of their goals, and as a result their mandate. A number of challenges pertaining to present quality assurance mechanisms exist, including that most of the instruments on quality assurance limits quantification of quality assessments and does not include the entire IPPF in the assessment criteria (Kasim & Hanafi, 2014), as well as criteria on efficiencies and effectiveness (Pitt, 2012). Furthermore, many public sector IAFs are small in size and have extremely limited resources, resulting in them not being able to afford the services of an external quality assurance assessor, increasing the likelihood of them not being externally reviewed (Marais, 2004). The AGSA applies the ISA 610 when assessing whether or not to use the work of internal audit (IAASB, 2014). All the criteria in the ISA 610 Standard are also assessed in the IIA’s Quality Assessment Manual (2016) that is suggested as a tool for internal auditors to use in assessing their conformance to the IPPF (2016d). However, recent research by Mbewu and Barac (2017) revealed that most IAFs at a local government level do not conduct quality assurance assessments. This is a missed opportunity, as results that emanate from such assessments could be used to explain shortcomings and substantiate the need for management support, whether in the form of resources, developmental initiatives or just their cooperation that would empower internal audit to provide augmented and value added services. The IKUTU (2014) report alludes to the need for IAFs to develop and implement a quality improvement and innovation plan that ought to be closely monitored by the AC. AGSA audit outcome reports are another report that can be used to inform the status of South African government departments’ IAFs. Using a recent audit report drafted by the AGSA (2019a), a couple of examples pertaining to national departments’ audit outcomes can be used as indicators to ascertain IAE and quality from the perspective of the organisation’s level of goal attainment. The present state of government services are 42 poor and appears to be regressing, as can be seen by the results presented by the AGSA on departments’ performance information where only 33% of departments produced quality performance reports in the 2018/19 financial year compared to 47% in 2014/15 (AGSA, 2019a). To add salt to the wound, one of the biggest government departments in SA attained 67% of their planned targets for education infrastructure development; however 94% of the allocated budget was spent (AGSA, 2019a). With such results, can one say that the department is achieving its objectives as set out if only two-thirds of its deliverables are met, with allocated funding almost depleted. If internal audit is risk-focused and are there to assist management attain its objectives (IIA, 2016b), the question arises as to why the department attained this poor result, although it has an IAF. This means that internal audit has not been as effective as they could be in terms of their definition (IIA, 2016b). If quality aims to measure performance (Al-Amri et al., 2015), it means that quality is increased with higher levels of goal attainment and that if a department is not attaining 100% or close enough to their planned deliverables, that internal audit’s contributions are not sufficiently effective. This could be due to a number of reasons such as insufficient resources (IKUTU & Treasury, 2014) and lack of competencies (National Treasury, 2018); however in order to ascertain this, internal audit needs to conduct a systematic evaluation of its own operations as required by Standard 1300 of the IIA Standards (IIA, 2016a). Developing a practical QASP tool has many benefits to the practice of internal audit, as it could entrench the concept of continuous improvement into internal audit's operating culture and quality processes that extends beyond conformance with the IPPF (2016d) (IIA, 2016f). By building in a measurement scale into present literature allows for a quantifiable self-assessment, of which the results can be monitored and used as the AC's assessment of the quality delivered by internal audit (Kasim and Hanafi (2012). Although Marais’ (2004) research is dated, it still holds true that another advantage is that a self-assessment particularly for small audit shops is cost beneficial, as less time will be spent by external validators to conduct their independent validations. The interpretative framework for this study is the IIA Quality Assurance Manual (2016) that underpins the proposed practical QASP tool. The following assessment criteria or 43 statements are eminent from this manual and form the basis for the development of the said tool. Figure 1 hereunder outlines the said interpretive framework: Figure 1: Interpretive Framework (IIA, 2016e) 2.5 Summary and conclusion 2.5.1 Summary of literature reviewed To sum up this literature review, it is evident that not much research has been conducted on adequate practical quality self-assessment preparation tools that will aid and capacitate IAFs to conduct a quality assurance self-assessment in preparation for the once in every five years external review (IIA, 2016a). Despite a number of research studies conducted on IAE indicators, not much emphasis was placed on unpacking the practical implementation of the quality aspect of internal audit, which inherently covers and assesses most of the IAE factors (George et al., 2015). The IIA has created a Quality Assessment Manual (2016) that includes a set of criteria based on the IPPF (2016d) against which internal audit operations and auditors are assessed. This provides 44 an adequate basis for the proposed practical self-assessment tool for quality assurance. The IAE indicator on management support however requires more emphasis through the design of explicit assessment criteria, as assessing this will add value to the quality of internal audit. The literature review has further revealed that South African public sector organisations, in particular national departments lack in the area of implementing quality improvement programmes and require assistance herewith (IKUTU & Treasury, 2014). Hence, this need highlighted by the IKUTU and Treasury (2014) report will likely be addressed through the development of the proposed practical QASP tool. In order to develop the said proposed tool, a qualitative approach will be followed to be able to obtain the in- depth professional experiences of those in practice and the present challenges they are facing, which will provide rich data to adequately inform this study. 2.5.2 Proposed research strategy, design, procedure and methods arising from the literature reviewed Based on prior research, it is therefore evident that both qualitative and quantitative research strategies have been used previously to explore the notion of quality assurance of IAFs. The aims of prior quantitative studies were to examine theories that are objective by studying the association between variables, whereas the qualitative studies aimed to answer the “why” questions. Due to the fact that this study attempts to obtain more rich data from a purposive sample of research participants or interviewees, a qualitative strategy and design will be followed using semi-structured interviews. Data collected will be analysed qualitatively using thematic analysis (Creswell & Creswell, 2018). This will in turn further inform and enhance the proposed practical quality assurance self-assessment preparation tool. 45 3 RESEARCH STRATEGY, DESIGN, PROCEDURE AND METHODS In Section 1.2.3, two questions that this research report intend to answer were posed— that is, ‘What factors or indicators contribute to internal audit effectiveness (IAE)?’ and ‘What quality assurance tool will aid internal audit in adequately assessing its effectiveness and conformance to the IIA Standards?’ A literature review has since been conducted and an interpretative as well as conceptual framework developed that will guide the choices of techniques to be used. This chapter identifies and describes the research approach, design as well as procedure and methods that were employed in this research to collect, process, and analyse empirical evidence. Broadly, it has three objectives; namely, to identify and describe the research strategy (Section 3.1), the research design (Section 3.2), as well as the procedure and methods (Section 3.3). The chapter also describes the reliability and validity measures (Section 3.4) that this research applies to make it credible as well as the technical and administrative limitations of the choices we make (Section 3.5). 3.1 Research strategy Research strategy refers to the manner in which a research study is carried out (Leavy, 2017) and involves the process of developing a research structure and procedures (Creswell & Creswell, 2018). The most common research strategies include quantitative, qualitative and mixed methods (Creswell & Creswell, 2018), which is a combination of the first two. A qualitative strategy was adopted, as this study entails ascertaining and understanding the challenges (Creswell & Creswell, 2018) pertaining to implementing the IIA’s Quality Assessment Manual (2016) in order to propose an easily accessible and practical automated-driven quality as