A PRACTICAL REVIEW OF THE EFFECTIVENESS OF ARTIFICIAL INTELLIGENCE IN THE AUTOMATED REVIEW OF LEGAL CONTRACTS. by Jacques Gerrit Fouché 0516171M Submitted in partial fulfilment of the requirements for the degree of Master of Laws by Coursework and Research Report at the University of the Witwatersrand, Johannesburg Date: 17 October 2023 DECLARATION I, Jacques Gerrit Fouché (0516171M), declare that this Research Report is my own unaided work. It is submitted in partial fulfillment of the requirements for the degree of Master of Laws (by Coursework and Research Report) at the University of the Witwatersrand, Johannesburg. It has not been submitted before for any degree or examination in this or any other university. I have submitted my final Research Report through TurnItIn and have attached the report to my submission. Word Count: 10,528 [Incl. the body of your report, footnotes BUT excl. title page, declaration, abstract, table of contents and bibliography] ABSTRACT This research report compares the performance of AI-driven automated contract review platforms with that of human contract reviewers, aiming to see improved effectiveness for the elements of time, cost and quality. An empirical study is done by comparing the specific performance of an automated contract review platform provider, Lawgeex, to the human contract reviewers of a business entity, Endress+Hauser. The results of the effectiveness assessment are reported on through dashboard data and questionnaires to the users of the platform. Recommendations are made both in general and specific to the two entities of the empirical study. Table of Contents I. INTRODUCTION ......................................................................................................... 2 II. LITERATURE STUDY................................................................................................. 3 (a) Fourth Industrial Revolution ......................................................................................... 3 (b) Artificial Intelligence .................................................................................................... 4 i. General Discussion ..................................................................................................... 4 ii. Current Use in Legal Profession ................................................................................ 5 iii. Potential Benefits of AI for Legal Profession .......................................................... 7 iv. Potential Risks and Hinderances of AI for the Legal Profession ........................... 10 III. RESEARCH PROBLEM AND LIMITATIONS ........................................................ 11 (a) Artificial Intelligence in Automated Contract Review ............................................... 11 (b) Hypothesis .................................................................................................................. 12 (c) Limitations .................................................................................................................. 12 (d) Artificial Intelligence Response to the Research Question ........................................ 13 IV. EMPIRICIAL STUDY ................................................................................................ 16 (a) General Description .................................................................................................... 16 (b) Data Assessment Parameters ...................................................................................... 16 V. DATA ANALYSIS REPORT ..................................................................................... 20 (a) Lawgeex Dashboard Data Analysis Report ................................................................ 20 (b) Target Group Data Analysis Report – Legal Counsel or Contract Reviewers ........... 27 (c) Target Group Data Analysis Report – Contract Uploaders ........................................ 33 (d) Data Analysis Discussion and Deductions ................................................................. 34 (e) Data Analysis Conclusion ........................................................................................... 35 VI. RECOMMENDATIONS ............................................................................................. 35 (a) General ........................................................................................................................ 35 (b) Recommendations Specific to the Target Group ........................................................ 36 VII. CONCLUSION ............................................................................................................ 37 VIII. BIBLIOGRAPHY ........................................................................................................ 38 IX. ANNEXURES ............................................................................................................. 41 (a) Lawgeex Document Examples (b) Questionnaire Formats and Analysis (c) Lawgeex Dashboard Reports (d) Respondents' Reports and Graphs 1 I INTRODUCTION In the latter part of 2022, the public was introduced to the mainstream concept of artificial intelligence. The launch of the conversational application (app) called ChatGPT, put artificial intelligence at the forefront of everyday discussions. However, artificial intelligence has been in use for decades by data scientists and data engineers to process vast amounts of data. The first use of the concept of computer intelligence was by Alan Turing when he published his work “Computer Machinery and Intelligence” in 1950.1 Computers can just process datasets faster than humans can, with a lower margin of error. Tools such as ChatGPT and Google Bard became mainstream since they could act conversationally on any topic, scourging the internet for answers. The user can very quickly gain insight into a topic that may previously have required significant time to research. In the business world, the use of artificial intelligence to process data is already long established. Consumer trends, financial market forecasting and similar large data sources have been analysed using computer processing power. Human intelligence simply does not have the capacity or time to process the data manually, with an acceptable low error percentage. Most often, the tasks are simply too mundane to waste human resources on such an activity, when a computer can do it better and faster. In the legal field, contract assessment often falls in the mundane category. Detailed contract analysis in mergers and acquisitions is necessary to ensure that both parties are protected. Due diligence tasks are necessary to make accurate decisions. Many law firms today devote numerous hours of associates and paralegals, to review contracts and write assessments on which clauses should be revised or deleted, to provide a legal service to their clients. Artificial intelligence has entered this arena as well to provide faster service to legal practitioners. This research report will analyse the practical effectiveness (in time, cost and quality) of an automated review of legal contracts by an artificial intelligence algorithm, through the data and information of an artificial intelligence algorithm contract review service provider to a multinational business entity. In the business world, terms are often used interchangeably such as “agreement” and “contract”. An agreement is considered a negotiated and typically legally binding arrangement between parties as to a course of 1 Anon What is the history of artificial intelligence (AI)? 2 action.2 A contract is a written or spoken agreement, especially one concerning employment, sales, or tenancy, that is intended to be enforceable by law.3 Subsections of these would be a nondisclosure agreement and a supply agreement. A nondisclosure agreement (NDA) is a contract by which one or more parties agree not to disclose confidential information that they have shared as a necessary part of doing business together.4 A supply agreement is a contract between two parties that details the terms of a purchase of goods from a supplier.5 These subsections of agreements and contracts are what will be analysed in the empirical study. This research report will juxtapose the performance of the automated contract review with that of the human counterparts in terms of time spent, costs allocated and quality of the output. II LITERATURE STUDY (a) Fourth Industrial Revolution The first industrial revolution was marked by the introduction of steam as a power source. The advent of electricity, ushered in the second industrial revolution and the third was earmarked by automation and machinery. The Fourth Industrial Revolution was shaped when computers breached the physical layer or “cyber-physical systems” made their appearance.6 The Fourth Industrial Revolution is characterised by many names or abbreviations – Industry 4.0; Internet of Things; 4IR; IoT – but essential refers to the same thing.7 Smart digital technologies are transforming the way companies do everything: design, innovate, improve, manufacture and distribute their products to users and consumers, all over the world. These technologies are disrupting the legal services industry as much as any other industry.8 Industry 4.0 refers to a new phase in the Industrial Revolution where smart digital technologies such as automation, artificial intelligence, 3D printing, and the Internet of Things are transforming the way companies manufacture, improve, and distribute their products. The term was coined by Klaus Schwab, Founder and Executive Chairman of the World Economic Forum, in 2017 in his book The Fourth Industrial Revolution that a 2 https://languages.oup.com/google-dictionary-en/ accessed on 12.01.2023. 3 Ibid. 4 Ibid. 5 https://www.contractscounsel.com/t/us/supply-agreement accessed on 12.01.2023. 6 Anon What is Industry 4.0, the Fourth Industrial Revolution, and 4IR? 7 Ibid. 8 Anon Introduction to AI contract review Lawgeex. 3 revolution of a technical nature is underway "that is blurring the lines between the physical, digital and biological spheres.”9 The Fourth Industrial Revolution builds on the digital revolution (The Third Industrial Revolution). Computers, electronics and the Internet from the 1950s to the 2000s are exponentially expanded by four basic types of disruptive technologies: 1) Connectivity, data and processing power; 2) computational analytics and intelligent computer systems (Artificial intelligence fits here); 3) advanced human-machine interfaces and 4) advanced engineering.10 These technological advancements are only part of The Fourth Industrial Revolution. Harnessing the power that these four disruptive technologies bring is forcing businesses to rethink their operations and how to optimally use their greatest asset: their staff. Employees must learn new skills to use new tools that the Fourth Industrial Revolution brings to stay effective as a minimum or competitive and innovative as a norm. The legal profession is not exempt from this Fourth Industrial Revolution.11 Legal practices today deal with many contracts in various forms. Mergers and acquisition departments, due diligence assessments and the like are all grounded in contracts. These contracts of clients need to be scrutinised by the legal practice to provide professional advice to clients. The contracts should be analysed by experienced legal professionals such as associates and paralegals to compile comprehensive feedback reports that can be used by the partners in law firms to brief clients. This contract review is a time-consuming task but needs to be done with accuracy, despite the nuances of contract language. Mistakes will inevitably happen due to the nature of the task – being mundane and repetitive – but there are tools available that can assist the legal practice. The Fourth Industrial Revolution bring computational analytics and advanced human-machine interface that are aimed at alleviating the mundane task and providing accurate results. (a) Artificial Intelligence i. General Discussion Artificial Intelligence (AI) explained in broad terms would be when an algorithm, based on a computer system, completes a set of tasks normally done by humans. This includes analysis, strategies and conclusions on a set of facts and compared to a defined set of rules. AI is new in the greater scheme of things but such ability has been theorised about for many years.12 A 9 Schwab, Klaus The Fourth Industrial Revolution. 10 Op cit note 7. 11 Ibid. 12 Vanni, Rachel How Artificial Intelligence Is Transforming the Legal Profession. 4 German lawyer, G.W. Leibniz, put a theory forward in the late 1600s that a futuristic machine would be able to calculate an output, based on a set of rules and a defined input. He never saw any “machine” that remotely resembled a computer, but he was accurate in the benefits that such a process would provide to humans and indeed lawyers. He prophesied that “excellent men would relegate calculation tasks to machines” to focus on more important tasks.13 Futurists and experts alike see the impact of AI as important as the innovation of steam.14 The impact will reverberate throughout all industries and professions, yet there is a fair amount of caution about the abilities of AI. Renowned physicist, Stephen Hawking, had candid responses to an online question-and-answer session called “AMA” or “Ask Me Anything”. He cautioned that super-intelligent AI would be beneficial provided that its goals were aligned with ours.15 Further, that human-level AI is a matter of time and we need to ensure that the AI is creating beneficial intelligence for us.16 Lastly, when AI will have higher intelligence than its creators, we will face an intelligence explosion bigger than what we can currently comprehend. Machine’s intelligence will surpass ours more than our intelligence surpasses that of snails.17 This is called “the singularity” where the advances in technology will cause machines to be smarter than human beings.18 But, let me leave the super-futuristic concepts for another time. ii. Current Use in Legal Profession In the legal profession, AI and humans can and have collaborated to improve the effectiveness – time, cost and quality – of legal functions. AI can analyse and summarise pages upon pages of legal precedents, can review and mark up contracts and can search through mountains of legal data to find answers. All being done in a matter of hours.19 For reference, LexisNexis alone processes 13 million new documents daily from over 50,000 data sources in varying formats. Its legal big data platform holds 60 billion documents and 2.5 petabytes of legal data.20 The ‘big data’ referred to is extremely large data sets that may be analysed computationally to reveal patterns, trends, and associations, especially relating to human behaviour and interactions.21 Although humans can work through such data volume, 13 Wolfram, Stephen Dropping In on Gottfried Leibnitz. 14 Anon Lawyer vs AI: A legal revolution. 15 Tegmark, Max Hawking Reddit AMA on AI. 16 Ibid. 17 Ibid. 18 Reedy, Christianna Kurzweil Claims That the Singularity Will Happen by 2045. 19 Op cit note 15. 20 Pfeifer, Jeff LexisNexis: Legal AI and Our Data-Driven Future. 21 Op cit note 3. 5 the time taken to do so may render the answer obsolete by the time it is found. The capabilities of AI must wow even the uninformed and casual observer but a topic that can never be overlooked is ethics. Once a system is acting intelligently and displays distinct autonomy, crucial legal and ethical questions are raised where the answers impact both the creator and consumer of AI technology. Such questions include public policy, professional ethics and philosophical ethics and the answers need to come from a combination of computer scientists and legal professionals.22 For example, if automated vehicles reduce traffic fatalities by half, would the car makers receive thank-you notes or lawsuits? Would it be ethically acceptable that a machine decides a lesser probability of human injury versus a greater probability of material damage?23 A real threat is that AI can harm by perpetuating human biases.24 Think of ethical profiling through facial recognition technology. Think invasion of privacy through digital video technology under the auspices of targeted advertising. AI, the law and ethics are a crucial short-term research priority defined by Stuart Russell, Daniel Dewey and Max Tegmark and will uncover more areas as AI develops.25 An important subsection of AI and its ability to act intelligently is machine learning. Due to the advances in computer hardware and software, machine learning has expanded rapidly. Machine learning is where computer algorithms learn independently from the data and information available. Computers autonomously change and improve their algorithms themselves. It is already used today for self-driving cars, creating sports reports or locating persons of interest. The advancement in computational power allows machine learning to improve faster and provide results with greater accuracy. These are natural progressions in any industry based on a vast body of knowledge. For data to become information, it must be standardised, classified, summarised and stored in an accessible format. This is ideally suited for AI and machine learning. These features are already used in the legal industry to remove the mundane tasks from expertly trained lawyers, allowing them to practice law.26 The First Industrial Revolution automated manual labour. In the same way, the Fourth Industrial Revolution is automating mental tasks.27 This has a definitive effect on the practice of law. AI will aid legal practitioners rather than replace them in the foreseeable future. Contract review, document discovery and legal research are all part of the current AI capabilities. However, AI has already started with contract drafting, legal outcome predictions and 22 Stuart, Dewey, and Tegmark Research Priorities for Robust and Beneficial Artificial Intelligence. 23 Ibid. 24 Anon Artificial Intelligence Law. 25 Op cit note 23. 26 Anon Big-Firm Intelligence for Small Law Firms page 4. 27 Stepka, Matthew Law Bots: How AI Is Reshaping the Legal Profession. 6 recommendations on judicial decisions.28 AI offers benefits to the legal practitioner – both current benefits and potential benefits. Increases in legal practitioner productivity and improved accuracy on repeated legal clauses are already a reality. Yet AI is not yet sufficiently developed to replace human judgement in the legal sphere. The high risk of bias used in setting the base algorithm remains real in the development of AI as well as the “explainability” of AI decisions to humans. These must be overcome first before a deeper entrenchment in advanced legal contexts can happen.29 Some legal practitioners and business entities are already using AI and machine learning in contract review for a quicker, more accurate outcome. The ability of AI to spot incompatible clauses and errors, more consistently than its human counterparts, is already providing entities with improved efficiency. Start-up businesses like Lawgeex are already providing such services in a sustainable business model.30 Traditional lawyers are rightfully cautious about the encroachment on a traditionally human area of expertise. But the objective of AI is not to replace legal practitioners. It aims to automate repetitive tasks that require time and high accuracy from their human counterparts but is not necessarily very exciting. This allows the same legal practitioners to focus on detailed assessments and personal contact with clients.31 AI and machine learning will without a doubt create new career paths and direction in the legal sphere.32 AI’s strength lies in the superior ability to process data but human traits of emotional intelligence and human judgement is (currently) beyond the reach of AI. The high complexity found in tax law will gain from AI’s processing ability, but humans are superior in people interaction areas – negotiation, mediation and ethical behaviour.33 Just as the plough didn’t make the farmer redundant; it improved their effectiveness. AI is an instrument that makes the lives of users easier by removing mundane tasks and allowing them to focus on their target market – people.34 iii. Potential Benefits of AI for Legal Profession The effectiveness of AI is evaluated for time, cost and quality. A legal practitioner’s ability to earn is directly linked to billable hours. AI drives the tools that assist legal practitioners to make better legal decisions, faster. Such decisions can be encoded as an algorithm for future 28 Op cit note 28. 29 Ibid. 30 Ibid. 31 Op cit note 13. 32 Ibid. 33 Anon Lawyer vs Artificial Intelligence: A Legal Revolution. 34 Op cit note 9. 7 machine learning, meaning that the corporate legal practitioner does not need to be everywhere at the same time. His knowledge is duplicated for use within his organisation, supported by AI.35 The mobile world required people to be available whenever their clients need them. Videoconferencing from any location; taking calls while on the daily commute; etc. are all ways in which our digital world has improved our accessibility.36 It is estimated that 23% of an average legal practitioner’s day can be freed up with AI and automation of tasks. The “paperwork-based” duties are taking away quality time from more important aspects of a legal practitioner.37 The review of documents in due diligence or litigation, contract clause assessment and qualifications are already done consistently with AI.38 As the legal industry grows and develops with the aid of AI, the know-how of practitioners is more important than standard legal practice. There are already trends that corporate entities are no longer willing to pay for the time spent by associates on contract review, because of the abilities of AI in the same field. Such shifts are opening the door for law firms to increase fees for experience engagements with clients such as consultations and assessments.39 Corporate entities dealing with high-value or high-risk opportunities need to have legal answers quickly before opportunities disappear or risks materialise. The AI-assisted legal practitioner thus acts with sensitivity towards the business.40 Increased data processing speed leads to a change in cost. Previously, such hardware was located at a client’s site, and this had to be maintained by IT specialists. Improvements in technology now leave the hardware at the service provider’s site and data is uploaded to the cloud via the internet. Legal practitioners are seeing cost reductions in client representation and practice management. Nearly all documentary processes are becoming digitalised and automated at a fraction of the cost of the same activity by people.41 Start-up enterprises such as Lawgeex have a yearly license fee of about $75,000, including the roll-out of the assessment criteria (“platform playbook”) within the service and the training of the users on the system.42 It was already stated that AI can analyse and summarise huge amounts of legal data without the help of a legion of associates.43 Legal practitioners can spend time on the complex side of their business – argument formulation, negotiation, client interaction and strategic planning – 35 Op cit note 9. 36 Op cit note 7 page 7. 37 Anon How Will Artificial Intelligence Impact the Legal Profession? 38 Ibid. 39 Ibid. 40 Op cit note 9. 41 Op cit note 34. 42 Fischer & Cohen The Total Economic Impact™ of Lawgeex page 2. 43 Op cit note 34. 8 knowing that AI will deliver.44 The baseline work requirement of all law firms is to assess agreements, whether they are non-disclosure agreements (NDA), supply agreements (SA) or service level agreements (SLA). These contracts are engaged multiple times per year with multiple clients, yet the contract clauses are not always unique or agreement specific. The standard clauses have come past in one iteration or another. Rather than reviewing these same clauses of the same agreements – day in and day out – by a human legal practitioner, AI can review these agreements without tiring and without errors. Should the same mistake be repeated, it will identify the mistake every time without wavering accuracy. Should contracting parties employ ambiguous wording, AI will highlight and identify these for changes to clearer wording.45 The accuracy of the work delivered by an AI contract review platform creates unprecedented contract quality. In corporate entities, the business units relying on the legal department for agreement review can be sure that all issues have been identified, addressed and amended to company standard wording. This gives confidence to such business units to continue with their business activities. Coupled with the turnaround speed of typically less than 48 hours, both time and quality goals have been achieved.46 Ultimately, the goal of AI development is to reach the ability of systems to learn like humans and surpass their ability in most analytical tasks, thus positively influencing society. The quality delivered by AI will consistently improve with the ability of AI to learn.47 Collectively, the effectiveness of AI, measured in time, cost and quality, leads to gains in the efficiency of the legal practice, corporate entity, or business unit. This application of AI helps small law firms to grow, increasing their efficiency to focus on clients, sales and operations.48 The efficiency of AI lies in its ability to be programmed to copy human analysis and decision criteria. It can recognise patterns from as little as 20 legal documents or 20 million documents. The efficiency of the AI system is its speed to identify patterns. Whether the source documents are judicial reports or contract documents, the AI system can analyse, summarise, draw conclusions and provide a human-understandable output format.49 Machine learning further enhances its ability to analyse and data mine through repetitive use, further aiding its human counterparts.50 This technology is available to any legal practice, whether a 44 Op cit note 13. 45 Op cit note 9. 46 Ibid. 47 Op cit note 28 page 109. 48 Op cit note 27 page 3. 49 Ibid page 11. 50 Op cit note 27 page 11. 9 single attorney for a large corporate.51 AI reduces the historical advantage of large law firms with legions of hired associates since AI can be used by anyone.52 A recent LexisNexis report on the evolution of legal research identified six ways that AI is transforming legal practice. Those six categories are 1) Due Diligence; 2) legal analytics or prediction technology; 3) document automation; 4) intellectual property, 5) electronic billing and 6) visualisation tools.53 Note that document automation – the use of AI to create or complete documents – is not automated contract review. This is creating or completing a document by filling in predefined fields that fall outside the scope of this report. Automated contract review forms part of the due diligence category and this report will only focus on automated contract review with AI. Agreements and contracts received by firms remain an obstacle to business and legal entities. Notwithstanding the volume of these, the myriad of versions, use of language and challenges to control these contract documents. AI together with machine learning can be programmed with parameters to search, identify, check and update similar concept clauses automatically.54 Firms can now manage contracts through the algorithm that identifies key components and supplies basic predictive alternatives. The AI can now, based on the company-specific contract clause criteria, assess any contract and provide alternative clauses in the case of conflicts, flag unacceptable clauses and make recommendations in case of ambiguous language.55 The negotiating legal practitioner is immediately armed with clauses to accept, amend or reject – far sooner than with a conventional contract review. As the AI body of knowledge grows, the AI software can draw conclusions, develop contract strategies and discover novel interpretations.56 iv. Potential Risks and Hindrances of AI for the Legal Profession At an almost basic level is the answer to the question “What is AI?”. The process or software or algorithm of 10 years ago is not the same as today. It can best be described as a constantly moving target. The question of “What constitutes intelligence?” is equally hard to define.57 The law functions on a set of rules that demarcate an action or a concept. There is a legal definition for terms such as “contract” or “breach”, yet the definition of AI is opaque at best. There are several risks to the use of AI in the legal profession. As it is a software program, 51 Ibid page 7. 52 Ibid. 53 Anon Six Ways Artificial Intelligence (AI) Is Transforming Law Practice. 54 Op cit note 34. 55 Op cit note 54. 56 Op cit note 13. 57 Greenstein, Stanley Preserving the Rule of Law in the Era of Artificial Intelligence (AI). 10 arguably not in the traditional sense, its functioning can be impeded by coding failures. Further, as the AI training is based on basic data or training data, the algorithms could be overfit or underfit, thus skewing results. The results can even be irrational. The AI algorithm may potentially be unusable on certain data sets where the training data were inadequate. Lastly, AI systems could actively discriminate against people, ideologies or orientations whilst carefully duplicating and advancing the current bias of society.58 Practical examples would be a persistent rejection of any African country’s law as the governing law of a contract or the continuous rejection of established seats of arbitration in place of foreign seats of arbitration. There are several examples of where AI systems made “wrong” decisions. A self-driving vehicle crash in 2016 claimed the life of a pedestrian and the Microsoft chatbot “Tay” responded with racist and sexist responses and even ended with denying that the Holocaust happened. The chatbot was shut down after a mere 16 hours of operation.59 Who would be held to account? There is no natural person or juristic person that can be sued since the response originated from an entity separate from its creators. Hence the responses of such systems should be carefully scrutinised to ensure correctness. III RESEARCH PROBLEM AND LIMITATIONS (a) Artificial Intelligence in Automated Contract Review The literature study identified the impact of AI on the legal profession. Its ability to rapidly analyse data and trends is established. Artificial intelligence tools and software are used as a blanket term for the function of automated contract review. However, the aim of this report is not to define AI or deep dive into the inner workings of the AI service provider’s intellectual property (IP). Suffice it to say that for this research report, “a computer programme” or “software” compares an uploaded contract against a set of predefined, acceptable contract clauses and identifies those clauses which are not compliant, too ambiguous, or unclear. The set of predefined, acceptable contract clauses is called a “playbook” by the AI platform service provider. The AI will render an output document that either “strikethrough” the incompatible clauses without a reworded clause or will add additional descriptive clauses, to define the limits of risk and liability. In the case of general clauses, the AI will add comments to the document to aid the receiver in further contract negotiation. This research report will 58 Kaminski, Margot E Regulating the Risks of AI. 59 Buiten, Miriam C Towards Intelligent Regulation of Artificial Intelligence. 11 analyse if the expected benefits, measured as time cost and quality, are realised in practice. Data sets from an AI service provider and their customer will be used to determine if the benefits are realised. First, the evidence of AI technology applied to contract review will determine if any efficiency gains are realised. The efficiency will be analysed as a function of time. Secondly, the costs associated with the AI application used for contract review will be countered by human contract reviewers. A licence cost versus an hourly cost for the same functions will be used to determine a cost-benefit. Thirdly, the actual revised clauses of the contract updates as done by the AI application, will be measured against the human reviewers’ output and a quality review will thus be done. This measures the accuracy of the AI application review when compared to human reviewers. It is essential to keep in mind that the empirical study is limited to the automated contract review of an artificial intelligence application. (b) Hypothesis The overarching hypothesis is that AI contract review is more effective in terms of time, cost and quality than traditional manual contract review. The three segments will be analysed individually to arrive at the answer to the overarching hypothesis. Sub-hypothesis 1: AI contract review is more efficient, i.e., faster than human reviewers. Sub-hypothesis 2: AI contract review is more cost-effective, i.e., cheaper than human reviewers. Sub-hypothesis 3: AI contract review yields higher quality, i.e., fewer mistakes and more consistent responses, than human reviewers. This research report will compare the two methods (AI contract review and human contract review) based on real-life data and through actual reviewers’ feedback to either prove or disprove the hypothesis. (c) Limitations The real limitation is that the performance of the AI platform is limited to the information entered by the employees of the Endress+Hauser Group. No other data from within the Lawgeex platform was brought into the mix. Thus, machine learning (if any) is limited to the data of the Endress+Hauser Group. The data within the AI platform has a maximum span of a rolling 12-month period. Therefore, the contracts within the system are only those uploaded in the data frame. Specific details about this will be given in the data analysis report. 12 (d) Artificial Intelligence Response to the Research Question This report would be lacking input if modern conversational apps such as ChatGPT are not polled for a response. I opted to pose the research question to three different conversational AI apps: ChatGPT, Google Bard and Shako. The responses are unedited. Question: Is legal contract review by artificial intelligence more effective in terms of time, cost and quality than traditional manual contract review? i. Chat.openai.com (ChatGPT)60 “Legal contract review by artificial intelligence (AI) can offer several advantages over traditional manual contract review in terms of time, cost, and quality. Here are some ways in which AI can be more effective: Time Efficiency: AI can process and analyze contracts much faster than humans. It can review large volumes of contracts in a fraction of the time it would take a human to do the same task. This speed allows for quicker turnaround times and enables legal professionals to focus on more complex or strategic aspects of their work. Cost Savings: AI-powered contract review can significantly reduce costs compared to manual review. By automating the review process, organizations can save on labor expenses and allocate resources more efficiently. Additionally, AI can help identify potential risks, inconsistencies, or missing clauses, reducing the likelihood of costly errors or legal disputes down the line. Quality and Accuracy: AI can enhance the quality and accuracy of contract review. Machine learning algorithms can be trained on vast amounts of data, including legal precedents and best practices, enabling them to identify important clauses, potential issues, and legal risks. AI can also help ensure compliance with regulatory requirements by flagging any deviations or non-compliant clauses. Consistency and Standardization: AI-based contract review ensures consistent application of legal rules and standards. It eliminates the variability that may arise from human interpretation and subjectivity, providing a standardized approach to contract analysis and reducing the chances of inconsistencies across different reviewers or departments. 60 ChatGPT, available at https://chat.openai.com/, accessed on 18 July 2023. 13 Scalability: AI can easily scale to handle a large number of contracts simultaneously. Whether you have dozens or thousands of contracts, AI can process them all efficiently and consistently, without compromising quality or incurring additional costs. Continuous Learning: AI can continuously learn and improve over time. As more contracts are reviewed and new legal precedents emerge, AI algorithms can adapt and incorporate this new knowledge into their analysis. This adaptive learning enables AI to enhance its performance and accuracy, making it increasingly effective over time. It is important to note that AI should be seen as a valuable tool to assist legal professionals, rather than a complete replacement for human expertise. While AI can automate routine tasks and streamline the contract review process, human oversight and judgment are still essential, particularly for complex legal issues or unique contractual arrangements.” ii. Bard.google.com61 “Yes, AI-powered legal contract review is more effective in terms of time, cost, and quality than traditional manual contract review. • Time: AI can review contracts much faster than humans. This is because AI can process large amounts of data and identify key terms and clauses much more quickly than a human reviewer. For example, a study by Gartner found that AI-powered contract review can reduce the time required to review a contract by up to 50%. • Cost: AI can also help to reduce the cost of contract review. This is because AI can automate many of the tasks that are currently performed manually, such as data extraction and keyword searching. For example, a study by CobbleStone Software found that AI-powered contract review can save businesses up to 30% on the cost of contract review. • Quality: AI can also help to improve the quality of contract review. This is because AI can be trained to identify potential risks and errors in contracts. For example, a study by Lexology found that AI-powered contract review can help to identify potential risks in contracts up to 90% of the time. In addition to the benefits of time, cost, and quality, AI-powered legal contract review can also help to improve: 61 Google Bard, available at https://bard.google.com/, accessed on 18 July 2023. 14 • Compliance: AI can be used to help businesses ensure that their contracts are compliant with all applicable laws and regulations. • Negotiation: AI can be used to help businesses negotiate better terms in their contracts. • Risk management: AI can be used to help businesses identify and mitigate potential risks in their contracts. Overall, AI-powered legal contract review is a more effective way to review contracts than traditional manual contract review. It is faster, cheaper, and more accurate, and it can also help businesses to improve compliance, negotiation, and risk management. However, it is important to note that AI-powered legal contract review is not a perfect solution. AI systems can still make mistakes, and they can only be as good as the data they are trained on. As a result, it is important to use AI-powered legal contract review in conjunction with human review.” iii. Shako.ai62 “Hello! As Shako, I can tell you that there are benefits to using artificial intelligence (AI) for contract review. One of the main advantages is that AI can process large amounts of data much faster than humans, which can save time and reduce costs. Additionally, AI can identify patterns and anomalies in data that may be difficult for humans to detect, which can improve the accuracy of contract review. However, it's important to note that AI is not a replacement for human expertise in contract review. While AI can be helpful in identifying certain types of issues, it may not be able to catch every mistake or fully understand the nuances of a contract. Additionally, there may be cases where a human review is necessary for legal or ethical reasons. Overall, I would say that AI can be a useful tool in contract review, but it should be used in conjunction with human expertise and not as a replacement for it.” 62 Shako, available at https://shako.ai/, accessed on 18 July 2023. 15 IV EMPIRICAL STUDY (a) General Description I am a long-term employee of a multinational company and have worked in a variety of roles within the organisation, both locally and abroad. A career that started as a chemical engineer, developed into a sales and marketing function, and later specialising in project management. This brought me into active contact with contracts and organisational development initiated the use of automated contract review. During this time, I was prompted to pursue a law degree to become a more skilful contract negotiator. The love of studies and self- improvement led me to a Master of Laws degree with a specialisation in commercial and business law. My current role includes the function of corporate legal counsel and gives me insight to the requirements of contract review in a multinational organisation. Endress+Hauser is a multinational business entity that receives a variety of contracts and agreements during its business dealings. Among these are non-disclosure agreements and supply agreements (contracts). I will expand on their business functions when describing the platform users. Lawgeex is a service provider of automated contract reviews by using artificial intelligence software, based on a defined set of rules provided by a business entity. Endress+Hauser is such a business entity and Lawgeex is such a service provider to them. Lawgeex self-identify what they do as: “We use Artificial Intelligence to relieve corporate attorneys from the daily tedium of contracting; with more time and creative energy, they can use their expertise to add more value and make a bigger strategic impact on their business. We’ve reviewed millions of contracts, expanding our AI’s capabilities to match the output of a human lawyer. Lawgeex harnesses contextual intelligence to understand legalese, accurately reviewing contracts, improving negotiation and accelerating approval at scale. The document is uploaded to the platform and the review parameters are set for location and document type.”63 The focus of this research report is to determine the effectiveness of AI in automated contract review, by analysing the aspects of time, cost and quality and to juxtapose that with the performance of the same aspects of human contract reviewers. The AI platform has provided 63 Anon “What we do” available at https://www.lawgeex.com/company/, accessed on 11.01.2023. 16 me access to its dashboard for all aspects. I have created a questionnaire for the human participants, analysing their responses according to selectable pre-defined options. The participants come from the entire Endress+Hauser Group and I assumed that contract review is done in the same manner across the Group. The various entities have different legal systems that may impact their natural decisions but the product offered and the customer type worldwide is similar. I have also assumed that the aspects are known and visible to the participants in that they can accurately respond to the time and cost questions. The quality aspect will be a comparative study between the AI platform dashboard and that of the human respondents and I expect to see a close similarity between the AI performance and the human reviewers. A baseline was determined by asking the contract reviewers within the target group how contract review was done before the introduction of the AI service provider’s platform. A questionnaire64 with selectable pre-defined answers, relating to effectiveness, was provided and the responses were analysed. (b) Data Assessment Parameters i. Target Group The target group for the empirical study centred on two subgroups, namely those who uploaded agreements (they were called “Uploaders” in the questionnaires.) and those who conducted reviews of the replies received (These were called “General Counsel, Legal 64 Annexure: Endress+Hauser Legal Counsel Questionnaire. 17 Counsel, Legal Support and Paralegals” or “Legal Counsels” in the questionnaires.). The “Uploaders” questionnaires were sent to everyone who uploaded an agreement to Lawgeex in 2022. This was a total of 74 people across 23 countries. The “Legal Counsels” questionnaire was sent to all staff with such designation in their job title. This was a total of 32 people across 9 countries. ii. Explanation of AI “platform” Lawgeex possesses a patent for “System and Method In Support of Digital Document Analysis” and its proprietary algorithm that analyses contracts in a contextual format. It is done by a learned understanding of contract language as well as the policies and objectives of the parties in a contract. Lawgeex can uniquely locate and respond to modulations, negating a compromise on quality assessment speed and costs of AI contract review.65 Lawgeex achieves this with its “Contextual AI Engine” that understands contract language as well as the intent and objectives of the parties in a contract. It aims to mimic the approach used by human lawyers to always consider the client’s guidelines and goals. As the contract body of knowledge grows, so new patterns are identified and machine learning is employed to grow the contextual know-how.66 Lawgeex is considered a “pioneer” in this field claiming a 10x faster speed than a human lawyer, with a better cost structure and higher quality.67 Lawgeex works on a legal “playbook” that holds the company’s legal policies and preferred clause language. It also contains the specific risk assessment and risk appetite of the company. Those legal policies must be applied consistently for them to be truly protected. When the AI engine is programmed with the company’s legal playbook, it can analyse any 3rd party contract, without prior qualification.68 When a user uploads an agreement to the Lawgeex portal, both the location country and the playbook need to be selected by the user. This determines both the applicable location law as well as the set of rules to be applied to the uploaded agreement. The user-friendliness, any pdf, doc or docx format can be uploaded to the portal. The AI-assessed result is returned to the user in MS Word format with all deletions and additions marked red and, in some instances, additional side comments are added. See ANNEXURES for examples of a base contract and a marked-up contract. 65 Anon Lawgeex granted pioneering patent. 66 Ibid. 67 Ibid. 68 Op cit note 9. 18 A dashboard is made available to the senior personnel of the corporate entity to monitor and track specific analytics regarding the performance of the Lawgeex portal. I received special permission to access this information for this research report. iii. Explanation of the “platform users” Endress+Hauser self-identify as: “…a global leader in measurement instrumentation, services and solutions for industrial process engineering. They provide process solutions for flow, level, pressure, analytics, temperature, recording and digital communications, optimizing processes in terms of economic efficiency, safety & environmental impact. Their customers come from various industries, including chemical, food & beverage, life sciences, power & energy, mining, minerals & metals, oil & gas and water & wastewater.”69 The Group has a variety of offices around the world, functioning as either a sales organisation or a production organisation.” Endress+Hauser is a corporate client, utilising the Lawgeex platform for the automated contract review of non-disclosure agreements and supply agreements. The Endress+Hauser Group General Counsel developed a set of minimum rules that is acceptable to the Group to harmonise the contractual clauses throughout the Group. Cognisance is given to common law and civil law principles as well as to international contract law. Within the Endress+Hauser sales organisation, there are several departments – General Management, Sales, Projects, Finance or Controlling – fulfilling specialised roles in the greater functioning of the company. The Sales department deals with customers on a transactional basis for daily maintenance and repair orders (MRO). The Projects department deals with customer projects where engineers, project managers and digital system engineers collaborate to support a customer project to build a new facility – mine, brewery, refinery – etc. The latter requires a larger number of items to be delivered at a specific time, hence the monetary spend is higher. Because of this, legal agreements such as non-disclosure agreements and supply contract agreements become necessary. Contract review within the company starts when an employee receives an agreement from a customer. The document is uploaded to the platform and once a response is received, it is 69 Anon About us: The Endress+Hauser Group. 19 reviewed by either legally trained staff or senior management. The final reviewed document is then returned to the customer as a baseline document for negotiation. iv. Empirical Research Methodology The research methodology employed in the empirical study was to engage with two groups of platform users – the legally trained staff and the general staff tasked with uploading agreements – to complete questionnaires regarding their work. The questions overlapped extensively to determine the understanding and accuracy of the replies received from the platform. The platform is also capable to provide reports regarding several parameters. I received special access to the back-end reports to tailor my questions to the platform users and to gain operational insight into the AI working. The data made available to me was limited to the data entered by Endress+Hauser into the platform but from any user, anywhere in the world where the company operates. The data of the questionnaires were compared to the analytics on the Lawgeex dashboard to find similarities and discrepancies. Deductions were made from these to find the reasons and develop recommendations. Additionally, specific company internal questions were asked of all respondents to provide specific data regarding costs. This is not captured by the Lawgeex platform but was deemed relevant for the research. V DATA ANALYSIS REPORT (a) Lawgeex Dashboard Data Analysis Report The Endress+Hauser Group provides access for any employee to the Lawgeex platform for the upload of non-disclosure agreements and supply agreements. In this report, I will refer to them as “Uploaders”. The data period under assessment (9 January 2022 to 9 January 2023) saw 89 different people uploading agreements from 24 different countries. A slight complexing factor was that several countries with Endress+Hauser offices have a business responsibility for a different area, region or country. For example, Endress+Hauser South Africa has business responsibility for sub-equatorial Africa with a northern territory border of Kenya, Uganda and the Democratic Republic of the Congo. All the countries south of this boundary line have existing and potential customers of Endress+Hauser South Africa. Should contracts be uploaded by a South African employee, the uploader can select the country of 20 origin of the contract. In the dashboard data, however, this location selection is not transparent. The Lawgeex dashboard provides four reports in both graphical pdf format and base data MS Excel format. Report 1 covers the Legal Operations, with a high-level view of the performance of the Endress+Hauser, uploads to the Lawgeex platform. Report 2 covers the Legal Policy: Mutual non-disclosure agreements, Report 3 covers the Legal Policy: Receiving non-disclosure agreements and Report 4 covers the Legal Policy: Supply Agreements. I have excerpted some of the graphs for illumination in the text but the complete reports are added to the ANNEXURES. For clarity, the difference between the two non- disclosure agreements is that one (mutual) relates to both parties disclosing information and the second (receiving) relates to only Endress+Hauser receiving information from an entity. There are also different playbooks for Supply Agreements such as “General” or “Common Law”, but the evaluation does not need to delve into such differentiation. In the sections below, the key graphs are duplicated to provide clarity within the text. i. Lawgeex Report 1: Legal Operations70 This covers the period from 9 January 2022 to 9 January 2023 (note 1) with a total of 443 contracts uploaded (note 2). The average turnaround time was 1 day, 1 hour and 49 minutes per contract (note 3). There seems to be a seasonal trend where fewer contracts are uploaded during the Northern Hemisphere summer (June/July) but that is a glancing observation. The bulk (146 or 33.96%) relate to Supply Agreements (General) and the rest to NDAs. 70 Annexure: Lawgeex Legal Operations Report. 21 When evaluating size, 44.47 or 197 contracts are less than 5 pages and considered “short”, while 20.09% or 89 contracts are more than 20 pages and considered “long”. ii. Lawgeex Report 2: Legal Policy – Mutual NDA71 This covers the period from 9 January 2022 to 9 January 2023 (note 1). The additional note about displaying data for 6 June 2021 to 9 January 2023 is to remind the user that no legal policy data is available before 6 June 2021. This has no bearing on the data under review. The Top Deleted Concepts out of 96 NDAs are listed as 1) Marking of Confidential Information; 2) Indemnification and 3) IP Assignment. 71 Annexure: Lawgeex Legal Policy - Mutual NDA. 22 The Top Missing Concepts out of 96 NDAs are listed as: 1) Confidentiality of Relationship; 2) No Future Obligations and 3) Standard of Care. The Top Revised Concepts out of 96 NDAs are listed as 1) Return or Destruction of Confidential Information; 2) Governing Law and Venue; and 3) Term of Confidentiality. 23 iii. Lawgeex Report 3: Legal Policy – Receiving NDA72 This covers the period from 9 January 2022 to 9 January 2023 (note 1). The additional note about displaying data for 6 June 2021 to 9 January 2023 is to remind the user that no legal policy data is available before 6 June 2021. This has no bearing on the data under review. The Top Deleted Concepts out of 93 NDAs are listed as 1) Indemnification; 2) IP Assignment and 3) Liquidated Damages. The Top Missing Concepts out of 93 NDAs are listed as: 1) Affiliate Definition; 2) No Future Obligations and 3) Compelled Disclosure. 72 Annexure: Lawgeex Legal Policy - Receiving NDA Report. 24 The Top Revised Concepts out of 93 NDAs are listed as 1) Governing Law and Venue; 2) Return or Destruction of Confidential Information; and 3) Permitted Disclosure. iv. Lawgeex Report 4: Legal Policy – Supply Agreement (General)73 This covers the period from 9 January 2022 to 9 January 2023 (note 1). The additional note about displaying data for 6 June 2021 to 9 January 2023 is to remind the user that no legal policy data is available before 6 June 2021. This has no bearing on the data under review. 73 Annexure: Lawgeex Legal Policy - Supply Agreement Report note 1. 25 The Top Deleted Concepts out of 146 Supply Agreement (General) are listed as: 1) Financial Guarantees; 2) Governing Law and Venue and 3) Most Favoured Nation. The Top Missing Concepts out of 146 Supply Agreement (General) are listed as: 1) Use Restrictions; 2) Affiliate Definition and 3) Export Limitations. 26 The Top Revised Concepts out of 146 Supply Agreement (General) are listed as: 1) Warranty; 2) Force Majeure; and 3) Limitation of Liability. The dashboards as supplied by the Lawgeex platform provide a comprehensive overview of workload and details to deleted concepts, missing concepts and revised concepts. The example contract in the ANNEXURES gives a practical example of the contract before the upload and the contract as received from the platform. Deleted concepts are struck through, missing concepts are added, and revised concepts have text added to the original version. The user has a clear view of all the amendments made by the AI platform, with explanatory notes where necessary. (b) Target Group Data Analysis Report – Legal Counsel or Contract Reviewers A baseline was determined by polling all employees worldwide that have legal training and would have responsibility for legal contract review. The questionnaire74 went to 32 employees across 13 locations and had a response rate of 41%. Although Lawgeex has been implemented for more than 3 years within the Endress+Hauser Group, the respondents were asked about their experiences before the introduction of Lawgeex. The questionnaire had 74 Annexure: Endress+Hauser Data Questionnaire: Legal Counsel or Contract Reviewer. 27 target questions and predefined responses like the available data from the Lawgeex dashboard, to determine the baseline on contract clauses but also contained effectiveness questions to determine the time and cost of the legal counsels and contract reviewers. The effectiveness results were as follows: Question 1) Number of contracts per week: 31% of respondents receive 2-5 contracts per week; 23% receive 5-8 contracts per week and 23% receive 8-10 contracts per week. Only 15% of respondents receive more than 10 contracts per week. Question 2) The average time spent per 10 contract pages: 46% of respondents spent 60-120 minutes per 10 contract pages; 38% spent 30-60 minutes and 15% spent 15-30 minutes to review 10 contract pages. Question 3) Number of NDAs per week: 15% of respondents receive 8-10 NDAs per week to review; 54% of respondents receive 2-5 NDAs to review; 31% receive less than 2 NDAs per week. Question 4) The average time spent per 3 NDA pages: 23% spent 45-30 minutes; 31% spent 15-30 minutes; 15% spent 10-15 minutes and 31% spent less than 10 minutes to review a 3-page NDA. Almost 70% of the respondents answered negatively about their knowledge of their standard production cost rate. Each employee in Endress+Hauser has a known “standard production cost” rate that reflects the accounting cost of everyone. It is more than the traditional cost-to- company as it accounts for building overheads as well as several other costs. To compare the responses of the respondents, specific questions were posed with the Lawgeex dashboard data and had the following responses: 28 Top Negotiated Terms: 1) Liability clauses; 2) Delivery penalties; 3) Warranty / Guarantee duration. Top Deleted Concepts: 1) Uncapped product liability; 2) Consequential damages; 3) Warranty / Guarantee period exceeding 36 months. 29 Respondents - Top Deleted Concepts 0 2 4 6 8 10 12 1. Uncapped product liability 2. Consequential damages due to product failure 3. Warranty guarantee period exceeding 36 months 4. Credit terms for non / new customers 5. Liability for Confidentiality breaches 6. Other (please expand) Top Missing or Added Terms: 1) Limitation of liability; 2) Payment Milestones; 3) Force majeure clauses. Top Revised Terms: 1) Uncapped liability to capped liability; 2) [Tie between three options] Penalty application against total value versus outstanding items; Contract termination clauses and Transfer of ownership and risk. 30 Respondents - Top Missing or Added Terms 0 2 4 6 8 10 12 4. Limitation of liability 2. Payment milestones 1. Force majeure / Pandemic clauses 5. Compliance with International Law 6. Other (please expand) 3. Forward Exchange Contracts Respondents - Top Revised Terms 0 2 4 6 8 10 12 1. Uncapped liability to capped liability. 4. Penalty application against total order versus outstanding items. 7. Contract termination clauses. 8. Transfer of ownership and risk. 6. Incoterms2010 – FCA; DAP; CIF; EWX; etc. 3. Offset against payment versus claims process. 2. Fixed delivery duration versus fixed delivery date. 5. Settlement discount for early payments. 9. Other (please expand) To gauge the respondents’ natural progression or regression from known legal arenas, three additional questions not covered by the Lawgeex dashboard were asked 1) Which governing law would you accept, other than your own country’s law? 2) Which governing law would you not accept, irrespective of the business potential? And 3) Which governing law would you propose as an alternative to your own country’s law? The results were as follows: Alternative Governing Law: 1) Swiss; 2) England and Wales; 3) United States 31 Respondents - Alternative Acceptable Governing Law 0 1 2 3 4 5 6 7 8 4. Switzerland 1. England and Wales 2. United States 3. Singapore 5. Other (please expand) Unacceptable Governing Law: 1) Russian Federation; 2) Any African country; 3) Any non- English language country. Alternative Language to English: 1) German; 2) [Tie between three options] French; Spanish and “local language”. 32 Respondents - Unacceptable Governing Law 0 2 4 6 8 10 12 2. Russian Federation 1. Any African country 4. Any non-English language country 3. Any Latin American Country 5. No response 6. Other (please expand) Respondents - Alternative Language to English 0 1 2 3 4 5 6 7 8 1. German 2. French 4. Spanish 5. No response 6. Other (please expand) 3. Chinese (c) Target Group Data Analysis Report – Contract Uploaders Following the baseline determination, a questionnaire75 was sent to all employees who had uploaded a contract in the year under review. These are not necessarily staff with formal legal qualifications but are perhaps dealing with a customer contract. A total of 74 employees received the questionnaire across 39 countries. A survey response of 53% was recorded. Like the legal counsel questionnaire, effectiveness questions were also asked of this larger group. The effectiveness questions had the following responses: Question 1) Who reviewed contracts before Lawgeex? 60% of respondents reported they reviewed contracts while 13% reported that the entity Managing Director reviewed contracts. 13% reported external lawyers were tasked to review contracts. Question 2) Time spent on 10 pages contract review: 48% of respondents spent 30-60 minutes reviewing a 10-page contract while 24% of respondents spent 60-120 minutes and 19% of respondents spent 15-30 minutes reviewing a 10-page contract. [The remaining 10% was split between other options.] Question 3) Who reviewed NDAs before Lawgeex? 63% of respondents reported that they reviewed NDA while 11% reported that the Managing Director of the entity reviewed NDAs and 11% reported that Heads of Department reviewed NDAs. 9% of respondents reported that external lawyers did the review of NDAs. [The remaining 7% were split between other options.] Question 4) Time spent on 10-page NDA review. 24% of respondents spent 30-45 minutes on a 10-page NDA while 22% of respondents spent 15-30 minutes on the same. 19% of respondents spent less than 10 minutes but 16% spent more than 45 minutes. 14% of respondents spent 10-15 minutes reviewing a 10-page NDA and the remaining 5% had external reviewers of NDAs. Question 4) Turnaround time if the review was done by others: 36% of respondents reported that the review was returned the same week and 33% reported that the review was received in 2-3 days. 19% reported that the review took more than 1 week. Only 8% reported that the review was received in 24 hours and only 3% (a single response) was that the review was received the same day. Also here, the positive response to the cost question was equally low under this group of respondents. 59% did not know their respective standard production cost. Also, 54% of respondents stated that they had in-house colleagues dedicated to contract or NDA review. 75 Annexure: Endress+Hauser Data Questionnaire: Contract Uploader. 33 (d) Data Analysis Discussion and Deductions The Lawgeex platform effectiveness measures in terms of time, cost and quality are reflected in the dashboard data. The average turnaround time is 1 day 1 hour and 49 minutes of human working time. Even the best reviewers would work 1 hour per 10 pages and then receive 5 contracts with 20 pages which would generate a workload of 10 hours. However, with the time that AI spent reviewing the contract, the human uploader or legal person can spend time on other activities. That is where the benefit lies. Some respondents stated that after implementing Lawgeex, the organization’s turnaround times for specific contract reviews decreased by up to two days.76 As to cost, the annual licence fee of $85,000 for the entire Endress+Hauser Group of companies, would yield a total of 460 manhours, considering that the average rate of respondents was given as €150/hour or about $163/hour. This is slightly less than 3 months’ worth of manhours. There can be no doubt that from a cost perspective, the AI platform outperforms the human counterpart by a large margin. The accuracy part of the platform yielded results that required insight. In the supply agreement review, the top deleted clauses by the platform were Financial Guarantees and Governing Law, whereas the human counterparts stated uncapped product liability and consequential damages. It is my opinion that these clauses would be amended by their human counterparts instead of outrightly deleted. When the top revised concepts are evaluated, the AI platform gave warranty and force majeure, whereas the human counterparts gave uncapped product liability again and penalties on part order instead of the entire order. In the NDA review, the AI platform gave indemnification as a top deleted concept and governing law and return or destruction of confidential information as the top revised concepts. None of these clauses featured in the human counterpart responses. It was clear that the human contract reviewers placed a fair amount of emphasis on operational contract and agreement conditions and less emphasis on purely legal aspects. The legally trained contract reviewers did identify some of the legal aspects but the results do not show a consistent identification of such clauses. As a Euro-centric organisation, the default fallback of many of the human reviewers was to known legal positions. This included the German language in case of a non- English language contract and Swiss governing law in the case of governing law alternatives. The response regarding governing law originating in Africa was the option selection to 76 Op cit note 43. 34 disregard any African country governing law. The same for any Latin American country too. This demonstrated a lack of understanding of other legal systems and would need better guidance from legal specialists if the business potential of these regions were to be tapped. (The exclusion of the governing law of the Russian Federation was not surprising, given the Russian-Ukraine conflict situation of 2022 and 2023.) The specialisation of the human reviewers demonstrated an understanding of the business environment, and the contract review was done in a similar light. Project management expertise and transactional selling skills set the tone for the contractual conditions but the NDA reviewers missed some key points from a legal perspective. The understanding of the functioning of the NDA clauses was present, but the limitation of risk and consequences of actions was not always clear. The Lawgeex solution greatly improved the accuracy of the reviewed contracts and agreements by highlighting clauses that were either lower in priority or even missed by human reviewers. The revised phrasing of the Lawgeex platform mark-ups helped to add clarity and the playbook structure also increased the accuracy and consistency of outputs.77 (e) Data Analysis Conclusion The Lawgeex platform proved to be an effective tool in the parameters of time, cost and quality, for the automated contract review when compared to human reviewers. The time and cost aspects were to a certain extent anticipated, but the quality level surpassed expectations. How important legal clauses and concepts were brought to the fore and marked up for clarity, was far better than expected. The playbook model used by the platform proves that it is a sustainable model that consistently provides good results. VI RECOMMENDATIONS (a) General The research that was done for this report would require additional development to determine its impact on the wider legal fraternity. The narrow focus may have diluted some findings. A future research project could consider different AI service providers as well as different types of firms, to support the findings. The results from the empirical study would need to be replicated in other areas with similar parameters to validate the findings. 77 Op cit note 43. 35 The introduction of an AI contract review platform into an organisation would require a clear mandate from the business partner as well as buy-in from all the users. A clear “why?” statement needs to be formulated to ensure that the tool is used as intended to deliver the results as expected. The formulation of a playbook or similar guide for the AI to follow is a critical success factor. In this step, the legal counsel of the business entity should provide clear direction on the requirements of the AI platform. The question of whether AI contract review is beneficial, I believe has been answered: the time improvement, the cost reduction and the accuracy of incompatible clause identification, are consistently better than a human contract reviewer. (b) Recommendations Specific to the Target Group Several reviewers stated that the Lawgeex platform is insufficient in their business environment purely due to language. The inability of the platform to work in languages other than English is a major drawback. Considering the global presence and regional focus of the Endress+Hauser Group, multi-language capabilities are a necessity rather than a simple feature. Even in some English-language countries, a requirement exists to be able to review contracts in other languages. In analysing the output of the AI review platform, the ability of machine learning seems to be lacking. Nowhere can the AI platform “be taught” that rejected or amended clauses are incorrect. A case in point is that all contracts sent from South Africa, have the governing law and arbitration clauses replaced with Swiss law and Swiss arbitration. Yet nowhere can the review give feedback to the platform about removed mark-ups. Internally towards Endress+Hauser, it was concerning that such a low number of associates were informed of their cost (standard production cost) towards the company. A few reviewers also commented that the response time of Lawgeex was too slow for their business requirements. I do not agree with such comments since it would be more beneficial to manage customer expectations than to rush contract review. Several respondents lamented Lawgeex mark-ups as “too much” and that such mark-ups require additional, specialised time to resolve. Whilst the latter part is true and detail mark-ups take additional time from legally trained specialists, the AI platform demonstrated the clauses are missed by the human counterparts that are not in alignment with the legal playbook. The consistency demonstrated by the AI to find and mark up the inconsistent clauses outweighs the additional specialist reviewer time requirements. If the clause's markup is incorrect, however, the feedback needs to go the machine learning route to optimise the AI platform. There is a gap in my opinion. 36 Lastly, there appears to be a gap in linking the contract size to the risk appetite of the business entity. The AI platform reviews the contract or agreement without consideration of the contract value. Much of the risks and liabilities are intricately linked to the contract value. Yet the AI platform deals with all the contracts the same way. In the Endress+Hauser business environment, an R200,000 or €10,000 contract and an R10 million or €500,000 will attract a decidedly different risk appetite. The AI platform review of the contract does not capture this in my opinion. A business decision can decide to accept all the penalties and liability clauses on a low-value contract, in the view to be expedient and secure the business. Whereas a large contract will require far more due diligence and risk assessment before the contract is signed and concluded. VII CONCLUSION The advent of high technology for big data issues will have an impact on the next generation of lawyers or legal practitioners. Gone are the days of spending hours reviewing legal contracts in a commercial environment. The new lawyer needs to be adept in the output of technology for him or her to spend quality (billable) hours in front of their client. Associates and paralegals alike must become proficient in the adept in the use of technology for him or her to become a high-performance individual. There is an inherent risk that the stagnant, traditional legal practitioner work will become obsolete in the Fourth Industrial Revolution era. A significant case was made that data processing and automated contract review can enhance the efficiency of mundane legal practice work, while at the same time reducing costs and increasing the quality of the work. There is also an opportunity for specialist legal practitioners to step forward to assist companies in defining their legal playbook for use in automated contract review platforms. An identified risk of AI is the programme background, including bias or preference, will impact the quality of the output, thus rendering the advice potentially skewed. Thus, the lawyer of the future must be able to translate legal principles and requirements into a functioning set of rules for the AI platform to compare. The platform is ultimately only as good as the background programming. The high-performance law firm of tomorrow will be the one that uses technology tools to enhance its client offering. Word Count: 10,528 37 VIII BIBLIOGRAPHY Adams, Nathan-Ross “South African Company Law in the Fourth Industrial Revolution: Does Artificial Intelligence Create a Need for Legal Reform?” (unpublished LLM thesis, University of the Western Cape, 2021), available at https://doi.org/10.2139/ssrn.4052285, accessed on 23 January 2023. Anon Introduction to AI Contract Review (2021) Lawgeex. Anon “Big-Firm Intelligence for Small Law Firms.” LexisNexis 2018, available at https://www.lexisnexis.com.au/en/insights-and-analysis/research-and- whitepapers/2018/the-rise-of-legal-ai-big-firm-intelligence-for-small-law-firms, accessed on 13 October 2022. Anon “Lawyer vs Artificial Intelligence: A Legal Revolution.” LexisNexis 2018, available at https://www.lexisnexis.com.au/en/insights-and-analysis/practice- intelligence/2018/ Lawyer-vs-AI-A-legal-revolution, accessed on 13 October 2022. Anon “Six Ways Artificial Intelligence (AI) Is Transforming Law Practice.” LexisNexis 18 September 2020, available at https://www.lexisnexis.co.za/evolution-of-legal- research/six-ways-artificial-intelligence-ai-is-transforming-law-practice, accessed on 13 October 2022. Anon “How Will Artificial Intelligence Impact the Legal Profession?” Exigent (blog) 17 March 2020, available at https://www.exigent-group.com/blog/how-will-artificial- intelligence-impact-the-legal-profession/, accessed on 13 October 2022. Anon “LAWGEEX GRANTED PIONEERING PATENT ON CONTEXTUAL AI ENGINE FOR CONTRACT ANALYSIS.” Lawgeex 24 February 2021, available at https://www.lawgeex.com/pr/lawgeex-granted-pioneering-patent-on-contextual-ai- engine-for-contract-analysis/, accessed on 8 November 2022. Anon “About us: The Endress+Hauser Group” available at https://www.endress.com/en/endress-hauser-group, accessed on 11 January 2023. Anon “Artificial Intelligence Law | AI Law.” Michalsons (undated), available at https://www.michalsons.com/focus-areas/artificial-intelligence-law, accessed on 23 January 2023. Anon “What Is Industry 4.0 and the Fourth Industrial Revolution?” McKinsey 17 August 2022, available at https://www.mckinsey.com/featured-insights/mckinsey- explainers/what-are-industry-4-0-the-fourth-industrial-revolution-and-4ir, accessed on 24 January 2023. Anon “What Is the History of Artificial Intelligence (AI)?” Tableau, (undated), available at https://www.tableau.com/data-insights/ai/history, accessed on 6 August 2023. Buiten, Miriam C “Towards Intelligent Regulation of Artificial Intelligence.” European Journal of Risk Regulation 10 no. 1 (March 2019): 41–59, available at https://doi.org/10.1017/err.2019.8, accessed 8 June 2023. Deamer, Kacey “What the First Driverless Car Fatality Means for Self-Driving Tech.” Scientific American 1 July 2016, available on 38 https://www.scientificamerican.com/article/what-the-first-driverless-car-fatality-means- for-self-driving-tech/, accessed on 6 August 2023. Dormehl, Luke “Why AI Will Never Rule the World,” Digital Trends, 25 September 2022, available at https://www.digitaltrends.com/computing/why-ai-will-never-rule-the-world/, accessed on 24 January 2023. Fischer, Christian & Cohen, Ashleigh The Total Economic Impact™ of Lawgeex Forrester May 2021, available at https://www.lawgeex.com/wp- content/uploads/2021/10/May2021-Forrester_TEI_Report-LawGeex.pdf, accessed on 13 October 2022. Galeon, Dom “Bill Gates Says We Shouldn’t Panic about Artificial Intelligence.” Futurism 26 September 2017, available at https://futurism.com/bill-gates-says-we-shouldnt-panic- about-artificial-intelligence, accessed on 23 January 2023. Greenstein, Stanley “Preserving the Rule of Law in the Era of Artificial Intelligence (AI).” Artificial Intelligence and Law 30, no. 3 (1 September 2022): 291–323, available at https://doi.org/10.1007/s10506-021-09294-4, accessed on 6 August 2023. Haggerty, Jarrod “Disruptive Technology in the Legal Profession.” Deloitte United Kingdom, 2017, available at https://www2.deloitte.com/uk/en/pages/financial-advisory/articles/the- case-for-disruptive-technology-in-the-legal-profession.html, accessed 23 January 2023. Heinlein Robert A “Quotable Quote” goodreads (undated), available at https://www.goodreads.com/quotes/12051-a-human-being-should-be-able-to-change-a- diaper, accessed 23 January 2023. Kaminski, Margot E “Regulating the Risks of AI.” University of Colorado 19 August 2022, available at https://doi.org/10.2139/ssrn.4195066, accessed on 6 August 2023. Lant, Karla “Elon Musk: Competition for AI Superiority at National Level Will Be the ‘Most Likely Cause of WW3.’” Futurism 4 September 2017, available at https://futurism.com/elon-musk-competition-for-ai-superiority-at-national-level-will-be- the-most-likely-cause-of-ww3, accessed on 23 January 2023. Lant, Karla “Experts Weigh in on AI and the Singularity.” Futurism 7 June 2017, available at https://futurism.com/experts-weigh-in-on-ai-and-the-singularity, accessed on 23 January 2023. Marwala, Tshilidzi Closing the Gap (2020) Pan Macmillan South Africa. Mik, Eliza “AI in Negotiating and Entering into Contracts.” AI in Private Law 1 June 2021. Available at https://papers.ssrn.com/abstract=3873071, accessed on 6 August 2023. Mollick, Ethan “All My Classes Suddenly Became AI Classes.” One Useful Thing (And Also Some Other Things), 17 January 2023, available at https://oneusefulthing.substack.com/p/all-my-classes-suddenly-became-ai, accessed on 23 January 2023. Parapontis, Ioannis “AI in Law: Robot Lawyer in First Ever Court Case. Lawyers to Become Obsolete in the Future? | LinkedIn.” Artificial Owl 8 January 2023 available at https://www.linkedin.com/pulse/ai-law-robot-lawyer-first-ever-court-case-lawyers- 39 ioannis/?trackingId=PKSJgCxNSHWkgYZwJh98Ow%3D%3D, accessed on 9 January 2023. Pfeifer, Jeff “LexisNexis: Legal AI and Our Data-Driven Future.” Artificial Lawyer (blog) 11 August 2017, available at https://www.artificiallawyer.com/2017/08/11/lexisnexis-legal- ai-and-our-data-driven-future/, accessed on 23 January 2023. Reedy, Christianna “Kurzweil Claims That the Singularity Will Happen by 2045.” Futurism 5 October 2017, available at https://futurism.com/kurzweil-claims-that-the-singularity- will-happen-by-2045, accessed on 23 January 2023. Russell, Stuart, Daniel Dewey, and Max Tegmark “Research Priorities for Robust and Beneficial Artificial Intelligence.” AI Magazine 36 no. 4 (31 December 2015): 105–14, available at https://doi.org/10.1609/aimag.v36i4.2577, accessed on 13 October 2022. Schwab, Klaus The Fourth Industrial Revolution (2017) Penguin UK. Stepka, Matthew “Law Bots: How AI Is Reshaping the Legal Profession.” Business Law Today 21 February 2022, available at https://businesslawtoday.org/2022/02/how-ai-is- reshaping-legal-profession/, accessed 13 October 2022. Surden, Harry “Artificial Intelligence and Law: An Overview.” Georgia State University Law Review Volume 35 Issue 4 Summer 2019 (1 June 2019), available at https:// papers.ssrn.com/abstract=3411869, accessed on 23 January 2023. Tegmark, Max “Hawking Reddit AMA on AI.” Future of Life Institute (blog), 12 October 2015, available at https://futureoflife.org/ai/hawking-reddit-ama-on-ai/, accessed on 13 October 2022. Vanni, Rachel “How Artificial Intelligence Is Transforming the Legal Profession,” Kira 8 May 2020, available at https://kirasystems.com/learn/how-artificial-intelligence-is- transforming-the-legal-profession/, accessed on 13 October 2022. Wolfram, Stephen “Dropping In on Gottfried Leibniz.” Stephen Wolfram Writings 14 May 2013, available at https://writings.stephenwolfram.com/2013/05/dropping-in-on-gottfried- leibniz/, accessed 23 January 2023. 40 IX ANNEXURES (a) Lawgeex Document Examples i Contract Draft – Original ii Contract Draft – Lawgeex Reviewed (b) Questionnaire Formats and Analysis i Endress+Hauser Legal Counsel Questionnaire and Results ii Endress+Hauser Contract Uploaders Questionnaire and Results (c) Lawgeex Dashboard Reports i Report 1: Legal Operations ii Report 2: Legal Policy – Mutual NDA iii Report 3: Legal Policy – Receiving NDA iv Report 4: Legal Policy – Supply Agreement (General) (d) Respondents' Reports and Graphs i Graph Respondents - Top Negotiated Terms ii Graph Respondents - Top Deleted Concepts iii Graph Respondents - Top Missing Terms iv Graph Respondents - Top Revised Terms v Graph Respondents – Alternative Governing Law vi Graph Respondents - Unaccepted Governing Law vii Graph Respondents - Alternative Language 41 MINOR SERVICES AGREEMENT This Minor Services Agreement (the "Agreement") is dated 18/07/2022 (the "Effective Date") and made between: XXXX a company incorporated under the law of Mozambique, with registration number XXXX, on page 57, from book C-45, on March 10, 2006, with tax registration number XXXX, with principal place of business at XXXX, Maputo- Mozambique, hereinafter referred to as the "Company" and [… …], with its registered office at [… …], (hereinafter the "Contractor"), Each the Company and the Contractor referred to as a "Party" or, collectively, as the "Parties", who have agreed to enter into the Agreement on the following terms and conditions: Minor Services Agreement DM 1043 Commissioning of Vibratory Fork Density Meters conditions of the 1/21 signed by and between XXXX and Endress Hauser. i04100186 Text Box ANNEXURE (a) i: CONTRACT DRAFT - ORIGINAL 1. Subject to instructions received by the Contractor from the Company from time to time, the Contractor shall perform the Services for the Company, at the times and in the manner, described in Schedule A (the "Services"). 2. The following Schedules attached to this Agreement shall constitute an integral part hereof: a) Schedule A (Scope of Services); b) Schedule AI - (Comercial and Technical Proposal); c) Schedule AII - (RG 0006); d) Schedule AIII - (RG 0018); e) Schedule B - (Table of Quantities and Prices); f) Schedule C – (Measurement Criteria) g) Schedule D – (insurance) 2.1. If there is any conflict or inconsistency between the provisions of this Agreement and its Schedules, the provisions of this Agreement shall prevail to the extent necessary to eliminate any conflict or inconsistency. In the event of any conflict between the Schedules, the order of such schedules listed in Clause 2 above shall be observed. 3. It is hereby agreed that the Company shall pay the Contractor the total amount of ZAR 359.304 (three hundred and fifty-nine thousand, three hundred and four rands) for the Services during the term of this Agreement, in accordance with the payment terms specified in Schedule B (Bill of quantities). 3.1 All payments shall be made via wire transfer to the Contractor's bank account as follows: [… …]. Bank: Bank Address: International Bank Code (IBAN): Account Number: Minor Services Agreement DM 1043 Commissioning of Vibratory Fork Density Meters conditions of the 2/21 signed by and between XXXX and Endress Hauser. Account Holder 3.2 Payment shall be made to the Contractor within 30 days of receipt by the Company of an invoice in form and substance previously approved by the Company. If the Company disputes any item in an invoice, the Company shall not be obliged to pay the disputed item until the Parties have resolved such dispute in an amicable way or in accordance with Clause 20, as the case may be. 3.3 All or part of any payment under an invoice may be withheld by the Company, or set-off against the payments owing to the Contractor, on account of: (i) defective Services rendered by the Contractor; (ii) all debts, damages and moneys due and unpaid from the Contractor to the Company; and (iii) any amount that the Company is required to withhold or deduct by law or by order of any relevant authority. Unless Contractor provides the Company with evidence of a waiver from the appropriate Governmental Authorities, Company may, where required by applicable law and subject to any applicable tax treaties, withhold from the aggregate amount charged all applicable withholding taxes, which Company shall remit to the appropriate Governmental Authorities. Upon Contractor’s request, Company will provide evidence of such remittance. Company shall provide Contractor with all tax information and documentation with respect to the withholdings as required by law and in the time required by law. Contractor shall be solely responsible for obtaining a refund with respect to any amounts withheld pursuant to this Clause. 4 The term of this Agreement shall be valid for 03 (three) months counted from the issuance of the first Service Order, ending at the end of the referred period, without prejudice of the provisions of this Agreement regarding the deadline for payment of the invoices. 5 The Contractor shall: 5.1.1 exercise that degree of skill, diligence, prudence and foresight that would reasonably and ordinarily be expected from time to time from a skilled and experienced professional seeking in good faith to comply with its contractual obligations, complying with all applicable legislation and laws and engaged in the same type of undertaking, and under the same or similar circumstances and conditions as those envisaged by this Agreement, and in accordance with the Company's environmental, health and safety standards and procedures contained in Schedule E (Company Procedure), and instructions, as communicated by the Company to the Contractor from time to time; Minor Services Agreement DM 1043 Commissioning of Vibratory Fork Density Meters conditions of the 3/21 signed by and between XXXX and Endress Hauser. 5.1.2 allocate sufficient resources and personnel for the performance of the Services; 5.1.3 progress the Services promptly and diligently; 5.1.4 prior to the date on which performance of the Services commences, obtain and maintain all necessary licenses and consents, and comply with all relevant legislation, in relation to the Services; and 6 The Company shall: 6.1.1 make payments to Contractor for the performance of the Services, in accordance with the terms of this Agreement and Schedule C (Commercial Conditions); 6.1.2 without warranty or liability, provide the Contractor all information in its possession which, in the opinion of the Company, is reasonably required for the performance of the Services and pertains to the Services; and 6.1.3 on all matters properly referred to it by the Contractor in connection with this Agreement, give its decision within the time periods specified in this Agreement, or within a reasonable period if no such period is specified herein. 7 Any notice, consent or approval required or permitted to be given in connection with this Agreement shall be in writing, in the English language and be addressed to a Party at the address chosen by the Party concerned below: 7.1.1 in the case of the Company to: XXXXXXXX XXXXXXXXX Tete – Moatize - Moçambique Att: XXXXX Tel: XXXXX E-mail: XXXXX 7.1.2 in the case of the Contractor to: Minor Services Agreement DM 1043 Commissioning of Vibratory Fork Density Meters conditions of the 4/21 signed by and between XXXX and Endress Hauser. [___________] Name: Fax: Telephone number: E-mail: Attn: 8 Either Party may terminate this Agreement for convenience by delivering a written notice to the other Party, and such termination will be effective from the time stated in the termination notice, or if no such time is stated in that notice, at the time such notice is delivered to the other Party. 8.1.1 In addition, if any of the Parties commits a material breach of this Agreement (which shall include, with respect to the Contractor, any Contractor Event of Default and, with respect to the Company, be limited to any Company Material Breach) and, upon receiving a Contractor Event of Default Notice or, as applicable, Company Material Breach Notice from the non-breaching Party, such material breach is capable of remedy, and, the breaching Party fails to remedy such material breach within [30 days] of receipt thereof, the non-breaching Party shall be entitled to terminate this Agreement by further written notice to the breaching Party. 8.1.2 Termination for Contractor Default 8.1.2.1 The Contractor will be in default under this Agreement if any of the following events occurs (each, a "Contractor Event of Default"): 8.1.2.1.1 dissolution, which includes the bankruptcy, insolvency, liquidation, amalgamation, reconstruction, reorganisation, administration, administrative or other receivership, winding up or dissolution of the Contractor, and any equivalent or analogous proceedings howsoever described and in whatever jurisdiction, and any step taken (including the presentation of a petition or the passing of a resolution) for or with a view to any of the foregoing (each an "Event of Insolvency") occurs and is continuing, with respect to the Contractor; 8.1.2.1.2 the Contractor commits a breach of the environmental health and safety plan or applicable laws relating to the environment, health and safety; Minor Services Agreement DM 1043 Commissioning of Vibratory Fork Density Meters conditions of the 5/21 signed by and between XXXX and Endress Hauser. 8.1.2.1.3 the Contractor makes a false or misleading representation, declaration or warranty, and, in the case of a false or misleading representation, declaration or warranty, that is capable of being remedied, such misrepresentation is not remedied within the period set forth in Clause 8.1 above; 8.1.2.1.4 the Contractor abandons performance of the Services; 8.1.2.1.5 the Contractor assigns and/or novates its rights and/or obligations under this Agreement; 8.1.2.1.6 the Contractor enters into a subcontract for any part of the Services without the prior written consent of the Company; 8.1.2.1.7 the Contractor fails to effect, have in place or otherwise maintain, or provide evidence of, insurance as required under Clause 17 and Schedule F (Insurance and Worker´s Compensation); 8.1.2.1.8 the Contractor does not progress the Services in a timely and diligent manner; or 8.1.2.1.9 the Contractor breaches Clauses 17.2.1 to Error! Reference source not found. inclusive. 8.1.2.2 If any Contractor Event of Default occurs, the Company shall be entitled to terminate this Agreement by written notice to the Contractor (each, a "Contractor Event of Default Notice") in accordance with Clause 8.1 above, without prejudice to any of its other rights and/or remedies. 8.1.3 Company Material Breach 8.1.3.1 The Company shall be in Material Breach of this Agreement if any of the following occurs (each a "Company Material Breach"): 8.1.3.1.1 an Event of Insolvency occurs and is continuing with respect to the Company; or 8.1.3.1.2 the Company fails to make an undisputed payment that is due and payable within the payment period in accordance with this Agreement and upon receiving notice of such breach from the Contractor, the Company fails to remedy such breach within 30 business days of its receipt of such notice of such breach. Minor Services Agreement DM 1043 Commissioning of Vibratory Fork Density Meters conditions of the 6/21 signed by and between XXXX and Endress Hauser. 8.1.3.2 The Contractor must give written notice to the Company of the occurrence of a Company Material Breach (each, a "Company Material Breach Notice"). 8.1.4 Automatic Termination for certain Contractor Events It is hereby agreed that if the Contractor and/or its Affiliates and Subsidiaries ever engages forced labour, child labour or any other labour conditions that oppress human rights and dignity, this Agreement and/or any other contract entered into by the Contractor and/or its Subsidiaries and/or its Affiliates, whether directly or indirectly, with the Company, will be automatically terminated. In such case, termination of this Agreement will occur immediately and automatically, the Contractor will not be entitled to any prior notice or cure or remedy rights or periods and such termination will be without prejudice to proper notice to the competent authorities and any further applicable rights and/or remedies of the Company in relation thereto. 8.1.5 In the event of termination of this Agreement by either Party under Clause 8, the Company shall pay the Contractor the aggregate amount of all undisputed invoices which remain due/not unpaid as of the relevant termination date in relation to Services performed by the Contractor hereunder. The Company shall only be obliged to make payment of any amounts under this Clause 8.1.5 30 days after the date on which Company has approved the relevant invoices submitted by the Contractor for payment on termination. 8.1.6 In the event of termination of this Agreement pursuant to Clause 8.1.1 with respect to a Contractor Event of Default or Company Material Breach, the Party in default or, as applicable, in material breach, shall pay the non-defaulting or, as applicable, non-breaching, Party liquidated damages in the aggregate amount of 10% of the total price payable under Clause 3 of this Agreement. The Parties hereby agree and declare that liquidated damages payable under this Agreement are a genuine pre-estimate of the non-defaulting or, as applicable, non-breaching, Party's loss under the relevant circumstances, and will be a debt due and payable by the defaulting or, as applicable, breaching, Party to the non-defaulting or, as applicable, non-breaching, Party. For the avoidance of doubt, the Contractor shall not be entitled to any compensation under this Sub-Clause 8.1.6, if termination of this Agreement occurs under Sub-Clause 8.1.4. 8.1.7 Termination for Force Majeure Minor Services Agreement DM 1043 Commissioning of Vibratory Fork Density Meters conditions of the 7/21 signed by and between XXXX and Endress Hauser. If a Party is relieved from the performance of any obligation under Clause 18 for a continuous period of [180] days or more; then the other Party may, at any time thereafter, by written notice to the other Party, terminate this Agreement with immediate effect. 8.1.7.1 Payment on Termination for Force Majeure If this Agreement is terminated by any Party under Sub-Clause 8.1.7, the Contractor's sole compensation will be paid for direct costs reasonably and properly incurred as a result of any such termination. 9 Each Party must keep confidential and shall not, without the prior written consent of the other Party, disclose to any Person: 9.1 the details of this Agreement, nor the details of any of the transactions or agreements contemplated by this Agreement; 9.2 any information relating to the business, operations, affairs, assets and liabilities of the other Party or any of the other Party's subsidiaries, holding companies or Affiliates; 9.3 any non-public, confidential, personal or proprietary information disclosed by one Party to the other, whether in writing or communicated orally or electronically (and including all information obtained by a Party through visual inspection of the assets of the other Party and any document prepared by a Party that contains, or otherwise reflects or is generated, by using any information specified in this Paragraph 9(9.3, irrespective of how such information came into such Party's possession, and irrespective of how such information is recorded or stored), and, in each case, whether or not the relevant information is marked as "confidential", collectively, the "Confidential Information". 9.3.1 The Parties may disclose Confidential Information to: 9.3.1.1 their respective officers, directors, employees, agents and professional advisors who: 9.3.1.1.1.1 have a need to know (and then only to the extent that each such person has a need to know); 9.3.1.1.1.2 are aware that the Confidential Information must be kept confidential; 9.3.1.1.1.3 are aware of the disclosing Party's confidentiality undertaking hereunder; and Minor Services Agreement DM 1043 Commissioning of Vibratory Fork Density Meters conditions of th