Comparing various methods for improving resource allocation on a single node cluster in Kubernetes

Abstract: When dealing with latency-critical applications in Kubernetes, a common strategy is to over-allocate resources to ensure the application can meet its latency guarantees during traffic surges. However, this practice often leads to resource underutilizationas the application will not fully utilize its reserved resources. The Kubernetes scheduler cannot initiate new workloads on the node because of perceived full resource utilization. This study explored the utility of two existing methods, Container Runtime Interface (CRI-RM), which we configured to use the 'balloon policy' and the vertical Pod Autoscaler (VPA) in addressing resource underutilization problems on single node Kubernetes clusters while maintaining latency-grantees of certain pods. Utilizing tc-sim, a network traffic simulator, we deployed four latency-critical and two non-latency-critical pods, all subjected to overallocation. Our finding reveals that VPA was ineffectivein detecting and addressing the underutilization of resources because of its slow response in adjusting requests inside the pods. Moreover, it worsened the underutilization issues of the node. Our configuration of the 'balloon policy' failed to detect theover-allocation issues and further led to performance degradation in the simulator, potentially due to the overhead introduced by CRI-RM. These results underscore the intricacy of over-allocation challenges in latency-critical applications, emphasizing the need for proposed-designed solutions that enable quick and dynamic exchange of resources between pods.