Measuring Agility- A Validity Study on Tools Measuring The Agility Level of Software Development Teams

Abstract: Context: In the past two decades, an increasing number of software development teams have been transitioning to agile. As a result, a need has emerged for measuring how agile these teams are. To satisfy this need, many researchers have created their own agile measurement tools. However, none of the tools managed to provide a substantial solution. Objective: Many tools have been created for measuring the agility of software development teams, thus creating a saturation in the eld. Three tools were selected in order to validate whether they will yield similar results. These tools were the Perceptive Agile Measurement (PAM), the Team Agility Assessment (TAA) and the Objectives Principles Strategies (OPS). Method: The surveys for the three tools were given to the four software development teams of Company A. The survey questions were grouped into agile practices which were checked for correlation in order to establish convergent validity. In addition, we checked whether the questions identi ed to be the same among the tools would would be given the same replies by the respondents. Moreover, the coverage of agile practices was analysed by checking which tool covers more agile practices. The results were used to see whether the three tools yield similar results. Results: The correlations of the data gathered were very few and very low. As a result, convergent validity could not be established. In addition, the questions which were identi ed as the same among the tools did not have the same answers from the respondents. Moreover, Objectives Principles Strategies (OPS) was the tool covering the most agile practices. All the above provide evidence that the three tools do not yield similar results. Conclusion: We conclude that the area of measuring agility is still fertile and more work needs to be done. Based on the various agile practices covered by each tool, we believe that not all tools are applicable to every team but they should be selected on the basis of how a team has transitioned to agile. This study has set a milestone in the area and pinpoints the need for a better way to measure agility.