Usability Evaluation of a Production System Development Framework : A Meta-Study Performed on the Use of a Production System Development Framework in the Development of a New Production System at Xylem

Abstract: Today’s competitive global market has placed companies under great pressure and the focus on production systems has been more prominent. Although there are several claimed benefits with using frameworks in the development of production systems, companies are reluctant to use these. Consequently, a relevant question formulation is: Are frameworks in the development of production systems usable? The purpose with this thesis work was therefore to evaluate the usability of production system development frameworks (PSDFs) in practice. In order to achieve this purpose, two research questions were established: RQ1.  How can usability of frameworks be evaluated? RQ2.  How does the use of a framework contribute to the development of a new production system? In order to answer the posed research questions, Bellgran and Säfsten’s PSDF was used in the production system development (PSD) process of a new production system at Xylem. Based on the PSD process, a meta-study was performed to evaluate the practical usability of the PSDF. Usability was defined and evaluated based on the five usability terms learnability, memorability, efficiency, effectiveness, and satisfaction. The result showed that all the five usability terms contribute to the usability evaluation of PSDFs. However, memorability was considered difficult to use on only one study since the user has to think a step further and make a qualified guess to answer if it is possible to memorize a framework. Therefore, it was considered memorability is only appropriate to use in a multiple study. The results also showed that Bellgran and Säfsten’s PSDF contributed most in the beginning of the PSD process by putting emphasis on the planning phase and providing a structure to follow. Due to the nature of a framework (i.e., to serve as a guide for structures to follow), this was not unexpected. However, the contributions from a structure or plan are hard to exactly distinguish. Since companies most often want tangible and accurate evidences, frameworks’ vague contributions are considered to be a major reason to why companies do not use frameworks more frequently.