Latency Aware SmartNIC based Load Balancer (LASLB)

Abstract: In the 21th century, we see a trend in which CPU processing power is not evolving at the same pace as it did in the century before. Also, in the current generation, the data requirements and the need for higher speed are increasing every day. This increasing demand requires multiple middlebox instances in order to scale. With recent progress in virtualization, middleboxes are getting virtualized and deployed as software (Network Function (NF)s) behind commodity CPUs. Various systems perform Load Balancing (LB) functionality in software, which consumes extra CPU at the NF side. There are research work in the past which tried to move the LB functionality from software to hardware. Majority of hardware­based load balancer only provides basic LB functionality and depends on NF to provide the current performance statistics. Providing statistics feedback to LB consumes processing power at the NF and creates an inter­dependency.   In this thesis work, we explore the possibility of moving the load balancing functionality to a Smart Network Interface Card (smartNIC). Our load balancer will distribute traffic among the set of CPUs where NF instances run. We will use P4 and C programming language in our design, which gives us the combination of high­speed parallel packet processing and the ability to implement relatively complex load balancing features. Our LB approach uses latency experienced by the packet as an estimate for the current CPU loading. In our design, higher latency is a sign of a more busy CPU. The Latency Aware smartNIC based Load Balancer (LASLB) also aims to reduce the tail latency by moving traffic from CPUs where traffic experiences high latency to CPU that processes traffic under low latency. The approach followed in the design does not require any statistics feedback support from the NF, which avoids the tight binding of LB with NF.   Our experiment on different traffic profiles has shown that LASLB can save ~30% CPU for NF. In terms of fairness of CPU loading, our evaluation indicates that in imbalanced traffic, the LASLB can load more evenly than other evaluated methods in smartNIC­ based LB category. Our evaluation also shows that LASLB can reduce 95th percentile tail latency by ~22% compared to software load balancing.