Tracing Control with Linux Tracing Toolkit, next generation in a Containerized Environment

Abstract: 5G is becoming reality with companies rolling out the technology around the world. In 5G,the Radio Access Network (RAN) is moving from a monolithic-based architecture into a cloud-based microservice architecture for the purpose of simplifying deployment and manageability,and explore scalability and flexibility. Thus, the transition of functionalities from a proprietaryhardware-based system into a more distributed and flexible virtualized system is ongoing. Insuch systems, legacy methods performance monitoring is relevant, wheresystem tracingplaysan important role. System tracing is important for the purpose of performance analysis of anygiven system. However, current tools were designed thinking about monolith architectureswhere, therefore, in new distributed architectures, new tracing tools need to be developed. System tracing often requires special permissions to be executed in applications running ina virtualized third-party environment. Unfortunately, not all applications running in a dis-tributed virtualized environment can be given such special access, at the risk of compromis-ing security and stability of the system. However, tracing data needs to be also collected fromapplications running in such environments. This thesis addresses the challenge of remotely configuring and controlling the system tracingtool with the example of LTTng in applications that run as part of a distributed virtualizedenvironment with Kubernetes. We explore the problem of remotely controlling and configuringsystem tracing as well as to optimize data collection. The main outcome is a tool able to re-motely control and configure system tracing tools. In addition, a proof-of-concept is presentedwith working demos for basic system tracing commands. It was discovered that a relay-based solution can be exposed outside the cluster via node-portwhich can relay incoming requests on-wards to any number of microservices. However, dis-covery of these microservices that are running system tracing tools is critial. Service discoverymechanism’s were brought forth and introduced to the system for the purpose of disoveringmicroservices with system tracing tools. Tracing data that is saved locally can be extracted bythe user through the relay-based solution or sent directly to any remote system using LTTngrelay daemon functionality. Comparison between directly executing commands in a bash shelland the remote CLI was measured. It has been concluded that the overall the response timeof both Linux and LTTng commands that are sent through the remote CLI is 1.96 times longerthan directly executing these commands in a bash shell. This was accounted to the fact thatcommands sent over the network traffic within the kubernetes cluster which is the cost ofremotely being able to control and configure system tracing tools. This being said, there arestill many steps that can be taken to improve the solution and to develop a more productionready solution.i