The effects of continuous integration on software development: a causal investigation

Abstract

Continuous Integration (CI) is a software engineering technique usually mentioned as one of the foundations of agile methodologies. The main objective of CI is to reduce the cost and risk of code integration among development teams. For such, it preconizes frequent commits to integrate the work from developers into a source code repository and the frequent quality verification via automated builds and tests. Through CI usage, it is expected that development teams can quickly detect and correct issues, improving team productivity and software quality, among other benefits pointed out by researchers and practitioners. Previous studies regarding CI usage highlight several benefits in software development aspects. However, such associations are not mapped as a whole and are not sufficient to conclude that CI is indeed the cause of such results. Therefore, the main goal of this work is to investigate the effects of CI adoption on software development from a causal perspective. First, we conducted a systematic literature review to catalog the findings from studies that empirically evaluated the effects of adopting CI. After exploring the existing state-of-the-art, we conducted two studies to deepen the comprehension regarding two aspects supposedly impacted by CI: software quality and teams’ productivity. We investigate if there is a causal relationship between CI adoption and such literature-reported effects. For this purpose, we employ causal Direct Acyclic Graphs (causal DAGs) combined with two other strategies: a literature review and a mining software repository (MSR) study. Our results show a panoramic view of CI literaturereported effects and point out that, indeed, there is a causal relationship between CI and software quality.