Determining confidence in test quality assessment in large-scale software development

Abstract: Software testing can be cumbersome and complicated. The complexity of the tests increases when the software itself becomes larger and more complex. When continuous integration is applied to software development feedback to tests can be obtained regularly.  These tests are performed in stages and each loop provides test results.A theoretical model for assigning confidence to different testing stages is presented in this thesis to aid in the understanding of the test quality. The input to the model was based on information given in interviews in a case study performed at Ericsson AB. The model is based on the ISO/IEC 25010 standard for software product quality. The theoretical model presented is focused on the early stages of integration and evaluated qualitatively. Its input is delivery test results, trouble reports both from customers and internal testers, and continuous integration flow trends. It was concluded that the theoretical model can be easily automated as each input source can be automatically collected. For developers working in the early stages of integration the model could be of help to give insight into what confidence they can assign their to test's quality. For testers working later in the flow the suitability of the model requires alterations that cannot be deduced from this thesis alone. For other stakeholders the usefulness of the model depends on how involved their work is in the development chain.