Systematic Review on Automated Testing (Types, Effort and ROI)

Abstract: Software organizations always want to build software by minimizing their resources to reduce overall cost and by maintaining high quality to produce reliable software. Software testing helps us to achieve these goals in this regard. Software testing can be manual or automated. Manual testing is a very expensive activity. It takes much time to write test cases and run them one by one. It can be error-prone due to much involvement of human throughout the process. Automated testing reduces the testing time which results in reduction of overall software cost as well as it provides other benefits i.e. early time to market, improved quality. Organizations are willing to invest in test automation. Before investment, they want to know the expected cost and benefits for AST. Effort is the main factor, which increase the cost of testing.     In this thesis, a systematic review have been conducted which identifies and summarizes  all the retrieved research concerning the automated testing types, effort estimation and return on investment (ROI) / cost-benefit analysis for automated testing. To conduct the systematic review, the author has developed a comprehensive plan which follows the procedure presented in [15]. This plan provides guidance to identify relevant research articles of a defined period. After the identification of research articles, it collects, evaluates and then interprets all the retrieved data about automated testing types, effort estimation and ROI. The results have been presented in statistical and descriptive form which provides different aspects of the data.     The statistical results have been presented with the help of tables and graphs which show different aspects of data i.e. any gaps in research work of automated testing, number of articles for each testing type. The answers of the questions have been presented in descriptive form. The descriptive results show 22 automated testing types, 17 Industrial case studies out of 60 studies, benefits of automated testing and effort estimation models. The discussion part highlighted some important facts about the retrieved data and provides practical implications for conducting systematic reviews. Finally it is concluded that systematic reviews are good means of finding and analyzing research data about a topic, phenomena and area of interest. It also provides support to researchers for conducting and investigating more research.