Automated regression test of a communication stack

Abstract: This thesis investigates possible solutions for testing a communication stack. The communication stack is a part of a communication module at Scania CV AB which is called SCOMM. This module handles diagnostic communication over CAN and shall conform to CAN-related ISO standards. We evaluate these solutions based on three aspects and how the stack is implemented. These three aspects are test coverage, ease of use and integration. Based on these evaluations, a test component has been implemented. We present models, resulting from the evaluation as well as test scripts examples. This test component performs black box testing which raised the question of how to define coverage for something that is hidden. It is hard to devise a decent mathematical model from something that is unknown. Therefore we make further investigations in existing tools that may increase test quality and help in finding reliable coverage metrics. E.g. we investigate learning based testing which allows automated test case generation. One outcome of LBT research is a tool called LBTest. LBTest is the first testing tool combining automata learning methods with model checker based test case generation. So far, research concerning the performance of LBTest, are positive. Therefore, this tool was worth looking into and we tried to answer these two questions. Could this tool improve SCOMMs testing and are there any improvements that would give the tool the right prerequisites, to be more useful? When evaluating the results from LBTest case studies, its ease of use and the system of SCOMM, we made the following conclusion: Yes it may be useful to test the communication stack. With these conclusions, LBTest has a bright future with a lot of areas of use and may be might be a solution for the famous test coverage problem that concerns black box testers.