Measuring Performance in Large Scale Agile Software Development Projects

Abstract: The increased usage and need for software as part of products has challenged traditional project management, nevertheless for hardware heavy organisations that are used to rely on the linear prediction and tracking of project outcomes. The developments in projects with embedded systems have countless dependencies and almost impossible to predict. Literature shows that software development projects have problems meeting the initial goals of budget, time, and scope. This is discovered too late due to insufficient methods of tracking progress. The purpose of this thesis was to investigate how large agile software development projects can continuously be followed to evaluate their performance and meet initial customer agreements fixed in time, budget, and scope. The thesis was conducted at Saab, active in the defense and security industry. This qualitative exploratory study was conducted with semistructured interviews and focus group discussions at the case company Saab, benchmark interviews with two additional companies, and an extensive literature study. The issues with the existing tracking approach were explored to determine how progress tracking may be created to continuously measure progress and indicate if project goals will be accomplished or not. The more general challenges in software development were also investigated to provide knowledge about areas in need of additional metrics which could indicate the problem and mitigate it. One industry-specific challenge is the security aspect that is unavoidable and requires a lot of documentation that holds up the development activities. Other detected challenges were difficulties in understanding requirements that lead to faulty estimations and work in the wrong direction, undiscovered dependencies that lead to a lot of rework and waiting for additional parts, insufficient testing environments that lead to late feedback, and holds up the development. It was also visible that the projects were conducted with different management approaches and no best-proven practice existed for tracking performance. From an analysis of the empirical data and existing literature, a suggestion of method tracking design was developed for large agile software projects with fixed contracts. The models were proposed to allow flexibility, enable control, and provide a holistic view. As Saab intends to introduce Earned Value Management in their software projects, this method was complemented with COMOD, TRL, IRL, and SRL to provide these three characteristics. Transparency and visibility of both products and processes are also found to be key to project success, thus additional metrics to increase visibility in projects are suggested to enable efficient project leading.