Dynamic Heuristic Analysis Tool for Detection of Unknown Malware

Abstract: Context: In today's society virus makers have a large set of obfuscation tools to avoid classic signature detection used by antivirus software. Therefore there is a need to identify new and obfuscated viruses in a better way. One option is to look at the behaviour of a program by executing the program in a virtual environment to determine if it is malicious or benign. This approach is called dynamic heuristic analysis. Objectives: In this study a new heuristic dynamic analysis tool for detecting unknown malware is proposed. The proposed implementation is evaluated against state-of-the-art in terms of accuracy. Methods: The proposed implementation uses Cuckoo sandbox to collect the behavior of a software and a decision tree to classify the software as either malicious or benign. In addition, the implementation contains several custom programs to handle the interaction between the components. Results: The experiment evaluating the implementation shows that an accuracy of 90% has been reached which is higher than 2 out of 3 state-of-the-art software. Conclusions: We conclude that an implementation using Cuckoo and decision tree works well for classifying malware and that the proposed implementation has a high accuracy that could be increased in the future by including more samples in the training set.