Investigating the performance of resizable arrays in the HPC job scheduler Slurm

Abstract: Slurm from SchedMD is an open-source job scheduler used at UPPMAX,which is Uppsala University's organization for high performancecomputing and large-scale data storage. Many of the data structures for providing scheduling in Slurm is implemented in C as linkedlists, which can lead to limitations when it comes to scalability.This is especially noticeable when many short jobs are sent to the system, which can cause effectivity problems and prolonged response times. The goal with this research paper was to find an alternative solution to the current implementation in the scheduler which provides better results. We have examined the possibility of using a mutable array asan underlying data structure and evaluated the performance differences in a local environment. Our conclusion was that it showed similar linear growth during batch scheduling operations, but hadimproved performance during independent list comparisons. It is likely that further differences can be highlighted with more tests and that the optimization may give a performance increase in more industrial contexts where Slurm is regularly used.