An Analysis to Identify the Factors thatImpact the Performance of Real-TimeSoftware Systems : A Systematic mapping study and Case Study

Abstract: Background: Many organizations lack the time, resources, or experience to derive a myriad of input factors impacting performance. Instead, developers use the trial and error approach to analyze the performance. The trial and error approach is difficult and time taking process when working with complex systems. Many factors impact the performance of real-time software systems. But the most important factors which impact the performance of the real-time software systems are identified in this research paper. Black box (performance) testing focuses solely on the outputs generated in response to the factors supplied while neglecting the internal components of the software. Objectives: The objective of this research is to identify the most important factors which impact the performance of real-time software systems. Identifying these factors helps developers in improving the performance of real-time software systems. The context in which the objective is achieved is an Online charging system, which is one of the software in Business support systems. In context, real-time systems, the traffic changes in a fraction of seconds, so it is important measuring the performance of these systems. Latency is also one of the major factors which impact the performance of any real-time system. Additionally, another motivation for this research is to explore a few other major factors which impact the performance. Methods: Systematic Mapping Study (SMS) and case study were conducted to identify the important factors which impact the performance of real-time software systems. Both the data collection methods, a survey and interviews were designed and executed to collect the qualitative data. Survey and interviews were conducted among 12 experienced experts who have prior knowledge regarding the performance of the system to know the most important factors that impact the performance of the online charging system. The qualitative data collected from the case study are categorized by using thematic analysis. From the logs, i.e., quantitative data collected from industry was analyzed by using random forest feature importance algorithm to identify the factors which have the highest impact on the performance of the online charging system. Results: Systematic mapping study was conducted in the literature to review the existing literature; 22 factors are identified from 21 articles. 12 new factors are identified from the survey, which was previously not identified in the literature study. From the available quantitative data based on the performance impact of the factors on the system, the factors are identified. Knowing these factors helps the developers to resolve the performance issues by allocating more number of virtual machines, thereby the performance of the system can be improved and also the behaviour of the system can be known. All these results are purely based on the expert's opinions. Conclusions: This study identifies the most important factors that impact the performance of real-time software systems. The identified factors are mostly technical factors such as CPU utilization, Memory, Latency, etc.. The objectives are addressed by selecting suitable research methods.