Transformer learning for traffic prediction in mobile networks

Abstract: The resources of mobile networks are expensive and limited, and as demand for mobile data continues to grow, improved resource utilisation is a prioritised issue. Traffic demand at base stations (BSs) vary throughout the day and week, but the capacity remains constant and utilisation could be significantly improved based on precise, robust, and efficient forecasting. This degree project proposes a fully attention- based Transformer model for traffic prediction at mobile network BSs. Similar approaches have shown to be extremely successful in other domains but there seems to be no previous work where a model fully based on the Transformer is applied to predict mobile traffic. The proposed model is evaluated in terms of prediction performance and required time for training by comparison to a recurrent long short- term memory (LSTM) network. The implemented attention- based approach consists of stacked layers of multi- head attention combined with simple feedforward neural network layers. It thus lacks recurrence and was expected to train faster than the LSTM network. Results show that the Transformer model is outperformed by the LSTM in terms of prediction error in all performed experiments when compared after training for an equal number of epochs. The results also show that the Transformer trains roughly twice as fast as the LSTM, and when compared on equal premises in terms of training time, the Transformer predicts with a lower error rate than the LSTM in three out of four evaluated cases.