Investigating Exploratory Testing in Industrial Practice - A Case Study

Abstract: Exploratory testing (ET) is an agile approach towards software testing. It is simultaneous learning, testing, reporting of problems and then generating new tests based on the learning. The aim of this thesis is to investigate exploratory testing in the industry. This thesis was proposed by Sogeti AB in Lund. Therefore the research questions were formulated and finalized according to their requirements and consent. Initially, a literature survey was conducted to study the different concepts of ET. After that a descriptive case study was conducted to investigate ET practices in an industrial environment. The research methodology used is qualitative. It comprises of ten semistructured interviews with industrial practitioners including both ET testers and customers having different perspectives. The study explores the misconceptions about ET and also identifies its claimed pros and cons. The investigation also studies its suitability with different types of testing and its effective combination with other techniques. In addition to the interviews, a survey was conducted to further investigate and analyze our findings with a larger sample of 25 practitioners. A framework for a session-based exploratory testing as practiced by the industrial partner is presented as part of our results. ET has different opinions and it is hard to say whether they are misconceptions or not. The most prominent advantage of ET from the testers view point is utilization of tester’s creativity and experience however, customers think the biggest benefit of ET approach is its ability to provide focused testing. The major disadvantage according to the testers resulted to be, difficulty in finding testers with appropriate experience, skill set and knowledge. On the other hand Inability to produce decision material was the biggest concern of the customers. The results also showed that ET approach was preferred to be used in combination with other testing techniques. Combination of Risk Based Testing and ET was reported to be used the most. ET was considered very suitable for situations where the testing requirements included learning of the system, time constraints and complementary testing. It was considered unsuitable for testing critical systems.