Confidence in Release Candidates : Maintaining confidence levels when moving from traditional release management to continuous delivery

Abstract: When shortening release cycles and moving towards continuous delivery, a different approach for quality assurance may be needed than in traditional release management. To allow the transition, all stakeholders must retain a sense of confidence in the quality of release candidates. This thesis proposes a definition for confidence consisting of 30 confidence factors to take into account to ensure confidence from all stakeholders. Confidence factors have been found through interviews with 11 stakeholders, analyzed and categorized using grounded theory analysis. The found factors are grouped into two main categories: Process and Verification Results. The thesis additionally contains a literature review of quality measurements and explores how confidence can be expressed in a continuous delivery pipeline. It is found that it is not possible to comprehensively express confidence only with metrics displayable in a pipeline when including only currently well-researched metrics, but with the combination of processes known to be followed in the organization some metrics provide coverage for many of the confidence factors.