Deep Learning-­Based Traffic Classification for Network Penetration Testing

Abstract: With the increasing amount of computers in society, the need to test and evaluate security of these are crucial to prevent various cyberattacks.  During a penetration test, a common methodology to follow is OSSTMM. Sometimes this method can not be followed in whole. In these scenarios, problems needs to be identified and methods developed to be able to perform a penetration test as efficient and thorough as possible. In this thesis, interviews with IT-security consultants at Combitech were used to identify areas that could be improved with use of Deep Learning. Interviews with the same consultants are used to evaluate the benefits or drawbacks of implementations done. Such scenario were identified in analysing large capture files containing network traffic from various systems received from customers. Today, a penetration tester at Combitech perform this task manually, a method that is prone to errors due to fatigue and human errors. The network traffic is classified based on either packet or TCP payload-level. Where the classification classes aim to create subsets of the network data which would be more convenient to analyse. The most promising model performs classification in a non-protocol based manner and classify the traffic in two classes; Encrypted and Not encrypted payload. Where the second and third model performs protocol classification based on packet or TCP payload. The first model shows best results and has the most positive impact in the work of analysing a capture file. Both protocol classifiers do however lack performance and by that, it does not make it useful for an analyst since there exist other methods that performs this task with 100 % accuracy.