Continuous Validation of Multi-Cloud Systems by Automated Test Scripting

Abstract: Scalability, pay-as-you-go, and flexibility are some factors why more and more organizations are utilizing a cloud environment for their IT systems. Specifically, the trend is towards multi-cloud, where different cloud providers manage different applications and services. This environment is however a lot less deterministic than an on premise solution. For example, if an application from one cloud provider is updated, this might result in that another application from another cloud provider stops working. This Master’s thesis investigates how a system owner can validate his/her cloud-based business system continuously, to be able to see that it functions as expected, as well as to spot anomalies. A measurement method is developed using Apache JMeter and crontab, with a setup using an Amazon EC2 instance and Raspberry Pis. To test this method, a multiple case study is performed on two different companies with two different cloud environments. Three tests are performed. One measures the login process, among other things, another measures the time required to generate a report of inventory values, and the third measures the response times of an ordering system. The method is successful for both environments, and all tests, as is shown by this thesis. For the first and third test, the response times are visualized in graphs. The result of the second test is shown in a table. The two companies are different in the sense that one is a large international company, while the other is a medium-sized company just located in Sweden. Also, the third test is performed because this company experienced its ordering system as slow, while the other company had no reported anomalies. Hence, this measurement method can be used to test different functions in a business system for both large and medium-sized companies. Furthermore, if a company suspects an error in their system, this method can be used to verify its existence, as well as verify if the error has been solved or not.