A Systematic Literature Review on the Scalability Issues in Software Requirements Prioritization

Abstract: Context: Prioritizing requirements is an important process that plays a vital role in producing a successful quality system by selecting high-risk requirements for implementation. Several studies have been conducted to analyze the performance of existing RP techniques. The inability to address a large number of requirements (Scalability issue) has been revealed as one of the major problems facing most of the existing requirement prioritization (RP) techniques. Objective: The scalability is a great challenge that has led to the RP process becoming inapplicable and inefficient in the industry, as most industrial projects nowadays have hundreds of requirements to be implemented. This paper aims to critically analyze the scalability issue in RP in terms of a) revealing and analyzing the factors that induce to the scalability issue while prioritizing a large number of requirements along with revealing the impact of each identified factor in raising the scalability issue, b) identifying and analyzing the existing techniques proposed to handle the scalability issue with respect to their usage contexts, the features used and their consequence in handling the scalability issue in their RP processes, c) investigating the evaluation extent of the identified techniques in terms of being evaluated and implemented in the industry settings, d) assessing the capabilities of the the identified RP techniques in considering the listed causative factors of the scalability issue in their performance evaluation, and e) revealing the limitations and future recommendations for further research. Method: A review protocol was designed based on the standard review guidelines by Kitchenham. Initially, 258 potential related studies were compiled through a step wise searching process. Thereafter, by thorough selection procedure, 22 relevant studies were finally selected to address the listed research questions. Results: The findings revealed four causative factors of scalability issue in RP with their descriptions: number of comparisons, time, human efforts required, and scarcity of the automation, along with revealing the impact of each identified factor in provoking the scalability issue. We found that the number of comparisons is the most significant factor that can lead to the inability to deal with large-scale requirements in prioritization with an impact degree of 48% percentage in leading to the scalability issue. Twenty three techniques that focused on addressing the scalability issue have been identified and analyzed critically in terms of their software development usage contexts, features used and the usage consequences of the utilized features in addressing the scalability issue, evaluation extents, and capability in addressing the identified causative factors of the scalability issue during their evaluation performance, and limitations. It has been found that most of the identified techniques were evaluated within the industrial settings with a percentage of 82%. However, the results reveal that these techniques still face serious limitations with respect to rank update, requirement interdependencies, substantial dependence on expert engagement, and error-prone and sensitivity to judgemental errors of the existing techniques. Conclusion: The scalability issue in the RP process has been elaborated on precisely in this study. Based on the findings, most of the identified techniques have not sufficiently covered the causative factors in their performance evaluations as at most up to 2 factors have been considered (mostly attributed to the fact that they are primarily considered as proofs-of-concept). Thus, a comprehensive performance evaluation of these techniques is needed to ensure their capabilities in addressing the scalability issue sufficiently in terms of considering the four identified causative factors. Additionally, adoption of the present strengths that have been found and attention to their underlying limitations are highly necessary. Thus, the current work represents several suggestions that can act as guides for the advancement of RP techniques in addressing the identified limitations.