Defect Detection in SRS using Requirement Defect Taxonomy

Abstract: Context: Defects occurred in the SRS may cause problems in project due to implementation of poor requirements which require extra time, effort, resources and budget to complete it. Reading techniques i.e., checklist based reading (CBR) helps to guide reviewers in identifying defects in software requirement specification (SRS) during individual requirement inspections. Checklists contain potential defects/problems to look for, but often lack clear definitions with examples of the problem, and also their abstractions are different. Therefore, there is a need for identifying existing defects and classifiers and to create a consolidated version of taxonomy. Objectives: We developed taxonomy for requirement defects that are in requirement specifications and compared it with the checklist based approach. The main objective was to investigate and compare the effectiveness and efficiency of inspection techniques (checklist and taxonomy) with M.Sc. software engineering students and industry practitioners by performing a both controlled student and industry experiment. Methods: Literature review, controlled student experiment and controlled industry experiment were the research methods utilized to fulfill the objectives of this study. INSPEC and Google scholar database was used to find the articles from the literature. Controlled student experiment was conducted with the M.Sc. software engineering students and controlled industry experiment was performed with the industry practitioners to evaluate the effectiveness and efficiency of the two treatments that are checklist and taxonomy. Results: An extensive literature review helped us to identify several types of defects with their definitions and examples. In this study, we studied various defect classifiers, checklists, requirement defects and inspection techniques and then built taxonomy for requirement defects. We evaluated whether the taxonomy performed better with respect to checklist using controlled experiments with students and practitioners. Moreover, the results of student experiment (p= 0.90 for effectiveness and p=0.10 for efficiency) and practitioner experiment (p=1.0 for effectiveness and p=0.70 for efficiency) did not show significant values with respect to effectiveness and efficiency. But because of less number of practitioners it is not possible to apply a statistical test since we also have used standard formulas to calculate effectiveness and efficiency. 2 out of the 3 reviewers using taxonomy found more defect types compared to 3 reviewers using checklist. 10-15% more defects have been found by reviewers using taxonomy. 2 out of the 3 reviewers using taxonomy are more productive (measuring in hours) compared to reviewers of checklist. Although the results are quite better than the student experiment but it is hard to claim that reviewers using taxonomy are more effective and efficient than the reviewers using checklist because of less subjects in number. The results of the post experiment questionnaire revealed that the taxonomy is easy to use and easy to understand but hard to remember while inspecting SRS than the checklist technique. Conclusions: Previously researchers created taxonomies for their own purpose or on industry demand. These taxonomies lack clear and understandable definitions. To overcome this problem, we built taxonomy with requirement defects which consists of definitions and examples. No claims are made based on student experiment because of insignificant values with respect to effectiveness and efficiency. Although the controlled industry experiment results showed that taxonomy performed slightly better than the checklist in efficiency i.e., in defect detection rate and effectiveness i.e., number of defect found. From this we can conclude that taxonomy helps guiding the reviewers to indentify defects from SRS but not quite much so it is recommended to perform a further study with practitioners in a large scale for effective results.