ASSESSMENT OF ROSLYN ANALYZERS FOR VISUAL STUDIO

Abstract: So‰ware security is an ever growing subject that is gett‹ing more important as we implement more soft‰ware into our daily lives. We want to protect our personal information and keep our privacy intact. Since our so‰ftware systems are gett‹ing more complex as well, soft‰ware developers need support in the form of tools that can help them to keep the so‰ftware free from vulnerabilities. ‘There are many such tools available but the focus of this study is investigating the performance of the fairly new Roslyn analyzers for security that can be embedded into Visual Studio.Since Roslyn analyzers for security are, in the time of writing (June 2019), not subject in any released studies the goal is to lay a foundation for future work regarding these types of tools. ‘Therefore three Roslyn analyzers for security are being compared, on source code in the C# programming language provided by the SAMATE project, both with each other but also against classic static analysis tools.Four vulnerability categories from the SAMATE test suite for C# are used to investigate the analyzers, namely OS command injection (CWE-078), SQL Injection (CWE-089), XML Injection (CWE-091) and Cryptography algorithms (CWE-327).Th‘e performance of the analyzers is measured with the metrics recall, precision and F-measure which are commonly used in other similar studies and makes it possible to compare the results obtained within this study with the results ofother studies within the €field.Th‘e results of this study are inconclusive as the scope chosen turns out to be to narrow. Two of the analyzers are not generating warnings for two or more of the vulnerability categories which makes it impossible to compare them with each other in a reasonable fashion. Even comparing the analyzers with classic static analysis tools is obsolete since there is only one representative from the Roslyn analyzers which does not say much about the general performance of these analyzers.‘The study reveals the necessity of a more complete and controlled test suite to evaluate security tools on source code wri‹en in C#.i