Interpretable Outlier Detection in Financial Data : Implementation of Isolation Forest and Model-Specific Feature Importance

Abstract: Market manipulation has increased in line with the number of active players in the financialmarkets. The most common methods for monitoring financial markets are rule-based systems,which are limited to previous knowledge of market manipulation. This work was carried out incollaboration with the company Scila, which provides surveillance solutions for the financialmarkets.In this thesis, we will try to implement a complementary method to Scila's pre-existing rule-based systems to objectively detect outliers in all available data and present the result onsuspect transactions and customer behavior to an operator. Thus, the method needs to detectoutliers and show the operator why a particular market participant is considered an outlier. Theoutlier detection method needs to implement interpretability. This led us to the formulation of ourresearch question as: How can an outlier detection method be implemented as a tool for amarket surveillance operator to identify potential market manipulation outside Scila's rule-basedsystems?Two models, an outlier detection model Isolation Forest, and a feature importance model (MI-Local-DIFFI and its subset Path Length Indicator) were chosen to fulfill the purpose of the study.The study used three datasets, two synthetic datasets, one scattered and one clustered, andone dataset from Scila.The results show that Isolation Forest has an excellent ability to find outliers in the various datadistributions we investigated. We used a feature importance model to make Isolation Forest’sscoring of outliers interpretable. Our intention was that the feature importance model wouldspecify how important different features were in the process of an observation being defined asan outlier. Our results have a relatively high degree of interpretability for the scattered datasetbut worse for the clustered dataset. The Path Length Indicator achieved better performancethan MI-Local-DIFFI for both datasets. We noticed that the chosen feature importance model islimited by the process of how Isolation Forest isolates an outlier.