An Investigation of Integration Debt in Continuous Integration - A design science study

Abstract: Continuous integration is a commonly used method in software development. It is used to improve the development process by prescribing frequent code integration and testing and thereby creating a mechanism for early feedback. To optimize the use of continuous integration in large scale development in regards to code integration, this study introduced a new term called integration debt. The concept is borrowed from the notion of technical debt, applied to continuous integration and the factors that are involved in this integration debt (which happens due to unintegrated code) are investigated. In this study, a set of integration debt factors is defined and the relationships between the factors are mapped and analyzed. This is done through design science research methodology where the artifact is the mapping of integration debt factors and the set of factors itself. Using various data gathering methods, a set of factors that are involved in integration debt are identified and validated. These factors were lines of code in commits, the number of modified files in commits, code churn, frequency of commits, code dependency, development overlap and lastly the factors related to the organization’s CI system. Additionally, this study investigates, and evaluates, a potential threshold value for one of the factors where the threshold is for when that factor reaches a significant risk to integration. Defining a general and accurate threshold proved to be difficult which suggests that integration debt factor thresholds require significant effort to define. This could perhaps be investigated further in future research. Consequently, the findings of this research study build a foundation for integration debt and how it could be mapped and measured. This can aid companies that work with large scale development, to better manage the integration debt level across their project by taking appropriate actions to avoid large accumulated debt levels. This potentially lowers the costs for companies and reduces the extra effort required to resolve integration problems during development.