Software Test Automation : A qualitative study on optimizing maintenance in test automation

Abstract: In IT the waterfall model is being replaced by agile development processes. When transitioning into agile processes software products are delivered in iterations, or cycles, of the same software. That software will be tested repeatedly, by the same tests. When the same test is executed several times test automation comes into consideration. With test automation organizations aim to cut costs but also have predictable and efficient test execution. Over time though, research suggests that maintenance of test automation could become a burden rather than a success.  This thesis aims to investigate and research problem areas that affect test automation maintenance and what measurements can be taken to manage maintenance. In total seven semi-structured interviews are conducted with software testing professionals. The gathered empirical data have been analyzed using a thematic approach, which is the result of this study. The outcome of the thematic analysis resulted in four themes Maintenance Issues, Minimize Maintenance, Coding Guidelines, and Collaboration Guidelines. Findings from those themes and underlying codes revealed several sources and problem areas in test automation maintenance. Respondents mentioned several challenges with Flaky tests, unreliable environments, and usage of incorrect tools. When it comes to reducing maintenance, respondents evaluated a list of guidelines but also mentioned a set of tools or approaches to use, to keep maintenance to a minimum.