Efficiency determination of automated techniques for GUI testing

Abstract: Efficiency as a term in software testing is, in the research community, a term that is not so well defined. In the industry, and specifically the test tool industry, it has become a sales pitch without meaning. GUI testing in its manual form is a time consuming task, which can be thought of as repetitive and tedious by testers. Using human testers to perform a task, where focus is hard to keep, often ends in defects going unnoticed. The purpose of this thesis is to collect knowledge on the area efficiency in software testing, but focusing more on efficiency in GUI testing in order to keep the scope focused. Part of the purpose is also to test the hypothesis that automated GUI testing is more efficient than traditional, manual GUI testing. In order to reach the purpose, the choice fell to use case study research as the main research method. Through the case study, a theoretical study was performed to gain knowledge on the subject. To gain data used for an analysis in the case study, the choice fell on using a semi-experimental research approach where one automated GUI testing technique called Capture & Replay was tested against a more traditional approach towards GUI testing. The results obtained throughout the case study gives a definition on efficiency in software testing, as well as three measurements on efficiency, those being defect detection, repeatability of test cases, and time spent with human interaction. The result also includes the findings from the semi-experimental research approach where the testing tools Squish, and TestComplete, where used beside a manual testing approach. The main conclusion deducted in this work is that an automated approach towards GUI testing can become more efficient than a manual approach, in the long run. This is when efficiency is determined on the points of defect detection, repeatability, and time.