Creating memory bandwidth contention with best intentions

Abstract: Heterogeneous System Architecture (HSA) is a computing system architecture that integrates central processing unit (CPU) and graphics processing unit (GPU) with a shared off-chip main memory. On one hand, sharing the memory reduces the communication latency between CPU and GPU but on the other hand, sharing can lead to contention for shared resources. The programs which execute concurrently on the GPU and CPU cores, share the off-chip memory bandwidth. This sharing can result in contention for bandwidth between CPU programs and GPU kernels. The CPU programs can steal bandwidth from GPU kernels which can lead to performance degradation. Since, memory bandwidth is important for the performance of GPU workloads, it is essential to measure the sensitivity of GPU kernels to bandwidth contention from CPU programs. This thesis describes the design and implementation of a program called Bandwidth Bandit which can steal memory bandwidth from co-running programs. The Bandit which was designed to execute on a CPU, can steal the bandwidth from programs co-running on a CPU or GPU core. The Bandit was used to measure the sensitivity of three GPU applications with different values of bandwidth demand. The results showed that all the three GPU kernels experienced substantial slowdown when subjected to off-chip memory contention due to the co-running CPU application.