Approaches and Techniques to Assess the Accuracy of Network Emulators

Abstract: As the mobile communication world expands in size and complexity, the need for testing the solutions in a controlled environment before deployment is as pronounced as ever. Simultaneously, new tools and products are developed to optimize the process and reduce costs. One tool that allows for testing the system under varying network conditions during development is a network emulator, which lies in between a classic network simulator and real-world network testing, providing high flexibility and repeatability in experiments and is expected to behave closer to a real-world network than a network simulator would. One established network emulator is the Itrinegy Network Emulator (INE). This paid product comes with a physical device and a software GUI and has been on the market for almost two decades. On the other end of the network emulation spectrum, the Znail Network Emulator is found: an open-source software-based network emulator designed to run on a single-board computer (e.g., a Raspberry Pi), which has been developed in the last five years in a nonenterprise-manner. In this thesis, these two network emulators are evaluated and compared against each other, aiming to investigate what functionality they offer and evaluate their emulation accuracy level. To do this, a testbed is set up to perform experiments by sending network traffic from a client to a server. The network emulator is placed between the hosts, acting as a bridge and applying different network conditions to the passing network traffic. The functionality comparison demonstrates that Znail offers minimal configuration options compared to the INE: only the most basic network conditions can be achieved with Znail. The results from the testbed experiments show that both emulators perform well when only one network parameter is applied at a time. However, when using more complex configurations with combinations of multiple network parameters, INE is overall superior in accuracy and stability, and Znail is deficient in several sectors. Although, the results also show that INE has some frailties in achieving correct emulation accuracy when combined with low bandwidth levels. Through the emulator functionality comparison and accuracy evaluation, this thesis argues that Znail can be a good choice of emulator in smaller establishments and for the at-home developer, while the INE is a better choice for larger organizations where stability and reliability are crucial.