Quality control of a diagnostic tool through qualitative and quantitative measurement assessment of field testing

Abstract: The purpose of this study is to develop a method to qualitatively and quantitatively measure and assess the field testing of a diagnostic tool by identifying the parameters that are relevant to assess a field test. The study is conducted at Scania CV AB, Södertälje, Sweden, a world leading manufacturer of trucks, buses and industrial and marine engines, where a method to assess the field test of their diagnostic currently does not exist. The study follows a deductive approach while taking a positivistic and hermeneutic perspective. The relevant theories and literature such as quality development and software testing are described to give a better understanding of the study. The study is conducted in four main steps- description of present situation, situation analysis, development of the assessment approach or framework and evaluation of the framework. The empirical information gathered from numerous interviews and meetings is presented in the description of present situation along with the various data sources available. The collected data from different databases is analysed where hypotheses are formulated based on the different influencing parameters for field testing. The correlations between the parameters are then calculated and analysed to verify the hypothesis as True or False. The ECU updates are also analysed to show that the ECU updates performed during field testing is a good representation of the actual usage after release. The framework to assess the field test is then developed using the available data and analysis made. A holistic view is taken to include the processes before and after the field test in the framework. The framework is in the form of an Excel workbook where data is either copied from databases or manually entered and relevant graphs describing the field test are generated automatically. The time period to be displayed on the graphs can be selected manually. This gives a good base to take decisions about how a field test has gone and whether or not the software is ready for release. Based on the correlation of the different parameters, a table with different key values of how much field test usage that should be conducted based on the number of implemented change requests are presented. Thus the result is that the most important attributes to consider for a field test are the amount of implemented changes where each field test usage occasion increases the chance of finding potential faults in the software of the diagnostic tool. An unrestricted framework is also described using data that may be available, but currently difficult to utilise effectively. Thus the recommended future work is represented by this framework which describes what information that can be obtained from different data sources and how they can be used to get a detailed understanding of what exactly has been used during field testing as well as after the software has been released. The framework is assessed in the last step and its uses along with limitations are described.  The difficulty in describing the success of software testing is also discussed to give a good context to the framework and understand its utility.