Evaluation of Structural Testing Effectiveness in Industrial Model-driven Software Development

Abstract: Software testing is a crucial but cost consuming practice in the development of safety critical software-intensive systems. In industry software testing techniques are chosen heuristically rather than scientifically. Researchers have studied for many years which test method is more efficient and effective in an industrial setting. Nevertheless, test design decisions still remain a challenge when applied in industry. Structural testing effectiveness is influenced by program code structure and the software development method used for designing and implementing industrial software-intensive systems. In the field of structural testing, the industrial approach toward different development methods and programming standards is studied insufficiently in the literature. In this thesis, we have found that applying structural testing on the actual generated source code from a Function Block Diagram (FBD) conforming to the IEC 61131-3 industrial standard is not effective. The FBD model is transformed to an FBD program code composed of almost exclusively assignments with Boolean expressions and function calls, which are not detected in code coverage analysis. In this thesis, we use the FBD model as a good candidate for measuring structural coverage, since it expresses the system behavior and it is the model from which the actual target code is generated. We propose a new automated method with potential application to our case study that evaluates the test suite based on structural coverage criteria. This leads to a shorter lifecycle with more effective structural testing for model-driven software development.