Analysis and Visualization of Validation Results

Abstract: Usage of simulation models is an essential part in many modern engineering disci- plines. Computer models of complex physical systems can be used to expedite the design of control systems and reduce the number of physical tests. Model valida- tion tries to answer the question if the model is a good enough representation of the physical system. This thesis describes techniques to visualize multi-dimensional validation results and the search for an automated validation process. The work is focused on a simulation model of the Primary Environmental Control System of Gripen E, but can be applied on validation results from other simulation models. The result from the thesis can be divided into three major components, static validation, dynamic validation and model coverage. To present the results from the static validation different multi-dimensional visualization techniques are in- vestigated and evaluated. The visualizations are compared to each other and to properly depict the static validation status of the model, a combination of visual- izations are required. Two methods for validation of the dynamic performance of the model are examined. The first method uses the singular values of an error model estimated from the residual. We show that the singular values of the error model relay important information about the model’s quality but interpreting the result is a considerable challenge. The second method aims to automate a visual inspection procedure where interesting quantities are automatically computed. Coverage is a descriptor of how much of the applicable operating conditions that has been validated. Two coverage metrics, volumetric coverage and nearest neigh- bour coverage, are examined and the strengths and weaknesses of these metrics are presented. The nearest neighbour coverage metric is further developed to account for validation performance, resulting in a total static validation quantity.