Requirements Conflicts Detection Using Conversational AIs

Abstract: The success of software development projects heavily depends on effectively capturing and meeting stakeholders' requirements. However, involving multiple stakeholders with diverse backgrounds and objectives often leads to conflicts among these requirements. These conflicts represent inconsistencies in the system design, resulting in various challenges, including project delays, increased costs, and potential system failures. Previous research has primarily focused on identifying conflicts with algorithms or negotiation, while conversational AI's potential to detect conflicts in real-time has been neglected. This thesis study addresses the challenge of requirement conflicts by proposing a novel approach that leverages conversational AI in the form of a chatbot. The chatbot, developed using the Rasa platform, enables real-time detection of conflicts, focusing on three general types: duplicated (similar), incompatible, and contradictory requirements. During the study, the design science research method is employed to guide the chatbot's development. Further, an experiment is applied to evaluate the chatbot's performance compared with domain experts using four different datasets. The experiment results are presented using F1 scores, which calculate precision and recall for both the chatbot and the experts on each dataset. Overall, the chatbot scored 0.8, while the experts achieved a slightly higher score of 0.86. To determine if there was a statistically significant difference between the two performances, a Wilcoxon signed-rank test was conducted on the results. The analysis showed no significant difference in the F1 score between the chatbot and the experts, indicating the chatbot's feasibility and effectiveness in detecting conflicts. The contribution of this thesis study can advance requirements engineering by providing a user-friendly and efficient method for real-time conflict detection, enhancing the quality and overall success of software development projects.