Evaluating Process Mining Techniques on PACS Command Usage Data : Exploring common process mining techniques and evaluating their applicability on PACS event log data for domain-specific workflow analysis

Abstract: Many software companies today collect command usage data by monitoring and logging user interactions within their applications. This is not always utilised to its full potential, but with the use of state-of-the-art process mining techniques, this command usage log data can be used to gain insights about the users' workflows. These insights can then be used to improve the software application and boost user productivity and efficiency. One area where this might be especially relevant is within the radiology domain, where the radiology labour shortage renders every efficiency-improvement valuable. Connected to this, this thesis aimed to evaluate a number of process mining techniques on real radiology PACS command usage data from Sectra to identify which techniques might be useful for analysing user workflows. Three process discovery algorithms (Alpha, Heuristics, and Inductive Miner - infrequent) were used on two datasets and evaluated based on a number of quantitative metrics (fitness and simplicity) and qualitative aspects (interpretability and usefulness). The qualitative aspects of the resulting process models were assessed through interviews with domain experts at Sectra, and the Heuristics Miner was found to discover the most useful models that could be interpreted and analysed by domain experts, mainly due to its simpler process model notation. To reduce model complexity, three different filtering methods based on sequential pattern mining were evaluated as a pre-processing step before applying the discovery algorithms. This resulted in improvements for the Alpha and Inductive Miner - infrequent, although none of the methods improved the Heuristic Miner models against the baseline. Trace clustering was also explored to address model complexity with the aim of identifying trace execution variants. Several configurations of trace representation techniques and clustering algorithms were used, and the neural-network-based approaches, Word2Vec and Autoencoder, emerged as the alternatives that achieved the best scores in the clustering evaluation. A few clusters with well-separated trace execution variants were identified - although most clusters were still complex and dominated by similar events. Finally, a prototype application with integrated process mining concepts was created based on the findings from the previous interviews. This was then evaluated with domain experts at Sectra, with the aim of investigating what concepts are practically useful for assisting with domain analysis. The findings indicate a clear use case for such an application to analyse sequential relations and command usage patterns in PACS user workflows, providing a data-driven and on-demand approach for hypotheses testing. Simpler concepts like manual filtering and aggregation were found to be practically useful and prioritised by the domain experts, while the opinion was more divided on the automatic pre-processing methods.