Root Cause Analysis and Classification for Firewall Log Events Using NLP Methods

Abstract: Network log records are robust evidence for enterprises to make error diagnoses. The current method of Ericsson’s Networks team for troubleshooting is mainly by manual observation. However, as the system is getting vast and complex, the log messages show a growth trend. At this point, it is vital to accurately and quickly discern the root cause of error logs. This thesis proposes models that can address two main problems applying Natural Language Processing methods: manual log root cause classification is progressed to automated classification and Question Answering (QA) system to give root cause directly. Models are validated on Ericsson’s firewall traffic data. Different feature extraction methods and classification models are chosen, with the more effective Term Frequency-Inverse Document Frequency (TF-IDF) method combined with a Random Forest classifier obtaining the F1 score of 0.87 and Bidirectional Encoder Representations from Transformers (BERT) fine-tuned classification obtaining the F1 score of 0.90. The validated QA model also gets good performance in quality assessment. The final results demonstrate that the proposed models can optimize manual analysis. While choosing algorithms, deep learning models such as BERT can produce similar or even better results than Random Forest and Naive Bayes classifiers. However, it is complex to implement the BERT since it requires more resources compared to more straightforward solutions and more caution.