Effects of an LSTM Composite Prefetcher

Abstract: Recent work in computer architecture and machine learning has seen various groups begin exploring the viability of using neural networks to augment conventional processor designs. Of particular interest is using the predictive capabilities of techniques in natural language processing to assist traditional CPU memory prefetching methods. This work demonstrates one of these proposed techniques, and examines some of the challenges associated with producing satisfactory and consistently reproducible results. Special attention is given to data acquisition and preprocessing as different methods. This is important since the handling training data can enormously influence on the final prediction accuracy of the network. Finally, a number of changes to improve these methods are suggested. These include ways to raise accuracy, reduce network overhead, and to improve the consistency of results. This work shows that augmenting an LSTM prefetcher with a simple stream prefetcher leads to moderate improvements in prediction accuracy. This could be a way to start reducing the size of neural networks so they are usable in real hardware.