A Longitudinal Study on the Effect of Patches on Software System Maintainability and Code Coverage

Abstract

In the rapidly evolving landscape of software development, ensuring the quality of code patches could potentially improve the overall health and longevity of a software project. The significance of assessing patch quality arises from its pivotal role in the ongoing evolution of software projects. Patches represent the incremental changes made to the code-base, shaping the trajectory of a project’s development. The identification and understanding of factors influencing patch quality could possibly contribute to enhanced software maintainability, reduced technical debt, and ultimately, a more resilient and adaptive code-base. While previous research predominantly concentrates on analysing releases as static entities, this study extends an existing study of patch testing while incorporating an examination of quality from a maintainability point of view, thereby filling a void in patch-to-patch investigations. Over 90, 000 builds spanning 201 software projects written in 17 programming languages are mined from two popular coverage services, Coveralls and Codecov. To quantify maintainability, a variant of the SIG Maintainability Model, a recognised metric designed to assess the maintainability of incremental code changes is employed. Additionally, the Change Risk Anti-Patterns (CRAP) metric is utilised to identify and measure potential risks associated with code modifications. A moderate correlation of 0.4 was observed between maintainability and patch coverage, indicating that patches with higher coverage tend to exhibit improved maintainability. Similarly, a moderate correlation was identified between the CRAP metric and patch coverage, suggesting that higher patch coverage is associated with reduced change risk anti- patterns. In contrast, patch coverage demonstrates no correlation with overall coverage, underscoring the distinctive nature of patches. However, it is noted that relying solely on patch coverage lacks comprehensive overview of coverage patterns. Thus, it is recommended to supplement it with overall system coverage for a more comprehensive understanding. Moreover, patch maintainability also exhibits no correlation with overall coverage, again, highlighting the unique nature of patches. In conclusion, the study offers valuable insights into the nuanced relationships between patch coverage, maintainability, and change risk anti-patterns, contributing to a more refined understanding of software quality in the context of software evolution.