Implementing and comparing challengers to popular multi-objective algorithms for unit test cases generation

Abstract: The topic of multi-objective algorithms has been researched for many years, where hundreds of multi-objective algorithms have been developed. With the field of search-based software engineering attracting use-cases, more research on which algorithms are fitting the area is still lacking. Comparing algorithms is key to fully understand what properties of multi-objective algorithms can bring benefit to an application area. In this case, search-based software testing. Three multi-objective algorithms—AGE-II, SIBEA and MOEA/D—are selected for implementation in the Evosuite tool to compare against the two popular algorithms NSGA-II and SPEA2, together with a random-search baseline. The algorithms create test cases for 100 randomly selected Java classes in the SF100 benchmark suite over a maximum of 1000 generations. The objectives of the algorithms are to maximize four coverage criteria—banch, line, method, and statement. The benchmarking shows that SPEA2 completes themost goals on average at 67.04%, also having the most stable results at ±20.28 goals for the ten executions. Tests generated by the random-search baseline has the highest branch, line and statement coverage, with NSGA-II generating tests with the best method coverage. Neither AGE-II or SIBEA could compete with the algorithms already implemented in Evosuite. The MOEA/D implementation could not be completed. More implementations and benchmarks of multi-objective algorithms are needed to find concrete links between technique to goals and coverage amount, especially surrounding the random-search baseline being a top performer.