Evaluation of Model Based Testing and Conformiq Qtronic

Abstract: The Model Based Testing is one of the modern automated testing methodologies used to generate test suits automatically from the abstract behavioral or environmental models of the System Under Test (SUT). The Generated test cases are abstract like models, but these test cases can also be transformed to the different test scripts or language specific test cases to execute them. The Model based testing can be applied in different ways and it has several dimensions during implementation that can be changes with nature of the SUT. Since model based testing is directly related with models, the model based testing can be applied at early stages of development that helps in validation of both models and requirements that could save time of test development at later stages. With the automatic generation of test cases, requirements change is very easy to handle with the model based testing as it requires fewer changes in the models and reduces rework. It is also easy to generate a large number of test cases with full coverage criteria using the model based testing that was hard to produce with traditional testing methodologies. Testing non-functional requirements is one field in which the model based testing is lacking; quality related aspects of the SUT difficult to be tested with the model based testing. The effectiveness and performance of model based testing is directly related to the efficiency of CASE tool that implementing it. A variety of CASE tools based on models are currently in use in different industries. The Qtronic tool is one generating test cases from abstract model of SUT automatically. In this master thesis detailed evaluation of the Qtronic test case generation technique, generation time, coverage criterion and quality of test cases are analyzed by modeling the Session Initiating Protocol (SIP) & File Transfer Protocol. (FTP), Also generation of test cases from models manually and by using the Qtronic Tool. In order to evaluate the Qtronic tool, detailed experiments and comparisons of manually generated test cases and test case generated by the Qtronic are conducted. The results of the case studies show the efficiency of the Qtronic over traditional manual test case generation in many aspects. We also show that the model based testing is not effective applied on every system under test, for some simple systems manual test case generation might be a good choice.