Optimizing on-chip Machine Learning for Data Prefetching

Abstract: The idea behind data prefetching is to speed up program execution by predicting what data is needed by the processor, before it is actually needed. Data prefetching is commonly performed by prefetching the next memory address in line, but there are other, more sophisticated approaches such as machine learning. The accuracy performance of a Machine learning prefetcher can be highly accurate and the model can be of great size, but applying it to hardware will enforce a limit regarding the size of the model. Therefore a balance between machine learning model size and performance has to be considered. This paper describes the optimization of a machine learning prefetcher’s size, whilst retaining performance, and how it was achieved by considering Recurrent Neural Networks’ in hardware. Finally this paper suggests machine learning prefetcher attributes promoting feasibility in hardware, as well as presenting a machine learning model optimized for prefetching in a hardware setting.