Evaluation of a qualitative model for a company's technical maturity within Continuous Integration, Continuous Delivery and DevOps

Abstract: The purpose of this study is to continue development of a benchmarking model to help companies assess their technical maturity when it comes to adopting Continuous Integration, Continuous Delivery and DevOps in their organization. The goal of the research is to assess how to improve the quality of qualitative models. Which conclusions can be drawn from comparing companies using benchmark and to assess which actions are the most effective to take to reach higher Continuous Integration, Continuous Delivery and DevOps maturity. The benchmark consisted of a questioner of two hundred statements that were answered for level of agreement with a current situation analysis and an ought-to-be analysis to be able to draw conclusions from the possible discrepancy between these categories. The questioner was answered during an interview study with chosen clients. Conclusions drawn from this study were that a lot can be done to improve the quality of qualitative models for examining Continuous Integration, Continuous Delivery and DevOps maturity. Different actions are necessary but the most important seems to be to ask open ended questions as well ass questions about different aspects of the same problem to promote discussion. It was also showed to be important to peer review the questions in the interview material beforehand to increase quality. The study also showed that it is possible to see trends in Continuous Integration, Continuous Delivery and DevOps maturity when comparing qualitative results for research subjects. The study showed that the most effective method for increasing Continuous Integration, Continuous Delivery and DevOps maturity is to use extensive automated testing suites that covers all testing disciplines.