The Design and Evaluation of a Seamless Approach to Migrate the State of QUIC Connections for Load Balancing Purposes

Abstract: QUIC is an emerging connection-oriented transport layer protocol that aims to support low-latency and highly secure communication between users and cloud services. Cloud load balancers distribute incoming QUIC connections towards a pool of backend servers where applications run. The sheer size of today’s clouds requires to deploy large pools of load balancing instances across multi-core servers. Today, once a connection is handled by a load balancing instance, the handling of the connection cannot be delegated to another instance which is possibly running on an underutilized server or core. Recent work has shown that such imbalances lead to poor utilization of the server resources when implementing the load balancer component itself. Moving the handling of a connection from one load balancing instance to another one is key to achieve uniform distribution of the workload among the load balancing instances. This thesis discusses the system design, algorithmic principles, and evaluation of an approach to move the state of a QUIC connection from one QUIC instance to another one, a task that we call “connection migration”. Our preliminary investigation focuses on inter-CPU-core thread-based migration of QUIC connections within the same server machine with the goal of supporting different types of load balancers that dynamically spread the workload across the load balancing instances. We evaluated the performance of the proposed implementation in a real physical testbed in NSLab at KTH across different dimensions: the number of active connections, the size of the file transfers within each connection, and the number of load balancing instances. The results show that under ideal circumstances, the throughput of a server system built by an 8-core computer reaches 100MB per second and handles up to 500 client requests per second. Finally, this thesis analyzes the bottleneck of the system and the critical parts of the implementation that should be optimized for better performance.