Intrusion Detection in IT Infrastructures using Hidden Markov Models

Abstract: In the past decades, cloud based services have developed rapidly. And as a result, cybercrimehas increased in sophistication as well as frequency. It therefore becomes vital to have solidprotection against such attacks, especially for infrastructures containing sensitive information. Inthis project we aim to study Hidden Markov Models (HMM) in an intrusion detection use case,both to detect malicious activity among the regular client traffic as well as differentiatingbetween specific attack stages. In the first part of the project we compared the accuracy of asupervised and unsupervised HMM when predicting the start of an intrusion attempt. LongShort-Term Memory (LSTM) Neural Networks as well as Random Forest Classifier (RFC) wereused as baseline comparisons. The second part of the project instead focused on decoding andpredicting different attack stages. Both supervised and unsupervised HMMs were trained. Thebaseline used in the second part was LSTM. The observed results indicated that supervisedHMM managed to both score higher on average as well as show greater consistency. This wasconcluded to be due to the supervised method utilising more information. Additionally, the lackof consistency in the unsupervised HMM is believed to be a result of the Baum-Welch algorithmsometimes converging to a local maximum instead of a global maximum when training themodel. From the results we conclude that HMMs display promising results in the intrusiondetection use case.