Predicting Cross-Platform Performance : A Case Study on Evaluating Predictive Models and Exploring the Economic Consequences in Software Testing

Abstract: Background: In today's digital world, there is increasing importance on cross-platform performance testing and the challenges faced by businesses in achieving efficient performance for applications across multiple platforms. Predictive models, such as machine learning and regression, have emerged as potential solutions to predict performance to be quickly analyzed, thus eliminating the need to execute an entire environment. Predicting performance can help firms save time and resources to keep pace with market demand, but potential risks and limitations need to be considered. With the increasing availability of data, predictive models have become effective problem-solving methods in various industries, including the testing industry. Objectives: This research aims to investigate the economic consequences and opportunities of implementing predictive models to predict cross-platform performance for firms operating in the software market and evaluate the performance of three models when predicting cross-platform performance. The study aims to add arguments to help businesses make informed decisions on the adoption of predictive models. Methods: The methodology employed in this research involved evaluating Multiple Linear Regression, Multiple Neural Network, and Random Forest, to gain insight into how such models perform when predicting performance. In addition to this analysis, interviews were conducted with industry experts to get an understanding of current processes and the potential benefits of adopting predictive models to identify the economic consequences of implementing such models. Results: The result shows that Multiple Linear Regression was the most promising one, with an R2 value of 0.79. Additionally, the research revealed that the current testing process faces difficulties when testing on multiple platforms. While predicting performance can provide cost and time savings, challenges and risks, such as data privacy and model trust, must also be considered. Conclusions: Multiple Linear Regression exhibited the most favorable performance among the evaluated models, with consistent results across all test runs and indicating a linear relationship. The economic consequences identified were the continuously required maintenance and updates of predictive models to remain accurate throughout the lifecycle. This implies ongoing costs, such as the complexity and cost of generating and storing the necessary data to train the models. Thus, the adoption of predictive models is still in its early stages, and while there are significant benefits, there are also challenges to address.