Mutation Testing to Identify and Predict TestBehaviour

Abstract: Testing is one of the crucial steps in software system development, which takes a lot of resources and time. Using Behavioural-based diversity to prioritise test suites has been shown to have the potential to reduce the cost and increase the quality of testing-related software development activities. Hence, in this paper, we propose to reproduce the results of the Gomes de Oliveira Neto et al. study by following the same methodology but with different projects and a new tool. The proposed method by Gomes de Oliveira Neto, which uses mutation testing to prioritise test suites based on behavioural diversity (b-div), starts by first generating the failure data by mutation testing and using this data to calculate the diversity between pairs of tests. We used three distance measures in this study: Accuracy (ACC), Matthew’s correlation coefficient (MCC), and the Fowlkes–Mallows index (FM). The study was conducted on six different open-source projects and the results were compared to the original study. The results showed that the proposed method (b-div) was more effective in killing more mutants compared to randomized priority. Additionally, we analysed the methods we used to calculate the distance measure and found that, while the overall difference between MCC and ACC was small, the FM method, which we chose to evaluate as a new distance measure, did not perform well. Nevertheless, our analysis was still in alignment with the original conclusions regarding the effectiveness of MCC and ACC in identifying mutants.