Software Testing Resource Scheduling based on Artificial Intelligence

Abstract: Generally, for executing a test activity, several testing resources such as a testing environment, integrator, and test cases need to be planned and prepared in advance. In addition, each test activity might require a different system setup, installation, and preparation in the testing environment. Today, a manual mapping between the test activities and testing resources requires a team where they need to master the testing domain. However, manual scheduling of the test activities in the test environment is a resource-consuming manual process suffering from human judgments, errors, ambiguity, and uncertainty. In this thesis, we introduce, implement, and evaluate an AI-based decision support system for scheduling test activities in the testing environment. The proposed approach in this thesis includes two main phases using different AI techniques such as natural language processing, machine learning, and reinforcement learning. The phase 1 predicts compatible test resources for each test activity based on their test requests written in natural text. The phase 2 schedules test activities to test resources with respect to minimizing an overall cost considering test activities' priority, competition time, waiting time, and cost of resource utilization. The feasibility of the proposed solution in this thesis is studied by an evaluation that has been performed on a Telecom use case at Ericsson AB, Sweden. For the phase 1, the evaluation shows that our proposed machine learning-based solution can reach a best F1 score of 0.88, and the proposed rulebased solution can reach a best F1 score of 0.75 on an internal dataset of Ericsson AB. For the phase 2, the evaluation indicates that our reinforcement learning-based approach can outperform a random scheduling strategy and also reach a similar optimization level as a genetic algorithm-based solution on several artificial datasets.