Automated Performance Tests : Challenges and Opportunities: an Industrial Case Study

Abstract: Automated software testing is often very helpful when performing functionality testing. It makes it possible to have a lot of user actions performed within the application without the need for a human interaction. But would it be possible to extend the behavior of regular functionality testing scripts and use them for performance testing? This way we could have regular application usage testing performed automatically during longer runs as well as investigate how well the application performs over time.This report presents the process of making test automation scripts run in a manner that makes it possible to analyze the tested application’s performance and limitations over time – not just its functionality. Additionally, a research is performed on how to choose proper test automation suites, appropriate tools and in which way to make it possible to efficiently gather performance related data during the test automation runs.This work has been done at the company Tobii Technology and it is used to test a desktop application they develop, called Tobii Studio. As a result of our work, we have implemented a test suite that can run automated tests over a long period of time while monitoring runtime performance for both the application and the computer’s hardware. The resulting tests can be used repeatedly by Tobii to help them identify performance issues for common test cases, and newer versions of Tobii Studio can be tested in the future to verify that a certain level of performance is maintained. The designs of our tests are so general that it will be possible for Tobii to continue extending our suite with more functionality.