Channel Estimation Optimization in 5G New Radio using Convolutional Neural Networks

Abstract: Channel estimation is the process of understanding and analyzing the wireless communication channel's properties. It helps optimize data transmission by providing essential information for adjusting encoding and decoding parameters. This thesis explores using a Convolutional Neural Network~(CNN) for channel estimation in the 5G Link Level Simulator, 5G-LLS, developed by Tietoevry. The objectives were to create a Python framework for channel estimation experimentation and to evaluate CNN's performance compared to the conventional algorithms Least Squares~(LS), Minimum Mean Square Error~(MMSE) and Linear Minimum Mean Square Error~(LMMSE). Two distinct channel model scenarios were investigated in this study. The results from the study suggest that CNN outperforms LMMSE, LS, and MMSE regarding Mean Squared Error~(MSE) for both channel models, with LMMSE at second place. It managed to lower to the MSE by 85\% compared to the LMMSE for the correlated channel and 78\% for the flat fading channel. In terms of the overall system-level performance, as measured by Bit-Error Rate (BER), the CNN only managed to outperform LS and MMSE. The CNN and the LMMSE yielded similar results. This was due to that the LMMSE's MSE was still good enough to demodulate the symbols for the QPSK modulation scheme correctly.  The insights in this thesis work enables Tietoevry to implement more machine learning algorithms and further develop channel estimation in 5G telecommunications and wireless communication networks through experiments in 5G-LLS. Given that the CNN did not increase the performance of the communication system, future studies should test a broader range of channel models and consider more complex modulation schemes. Also, studying other and more advanced machine learning techniques than CNN is an avenue for future research.