Over-the-Air Federated Learning with Compressed Sensing

Abstract: The rapid progress with machine learning (ML) technology has solved previously unsolved problems, but training these ML models requires ever larger datasets and increasing amounts of computational resources. One potential solution is to enable parallelization of the computations and allow local processing of training data in distributed nodes, such as Federated Learning (FL).  For the implementation of FL over wireless networks, the limitations of communication resources pose constraints on how many information bits can be reliably transmitted, which further affects the accuracy of the aggregated model. Over-the-Air (OtA) computation has recently appeared as an alternative solution for aggregating data over distributed nodes, by exploiting the superposition property of wireless channels. This thesis explores how the linearity of OtA computation and compressed sensing (CS) can be used in combination for update aggregation in an FL system. Particularly, the thesis uses CS with iterative hard thresholding (IHT) as the reconstruction algorithm, and the transmission of a compressed update vector from each node is subject to some total power constraints.  Simulation results show that for an OtA FL system with total power constraints, when all nodes have the same (and known) sparsity pattern, applying CS does not bring any obvious benefits as compared to the case of transmitting directly uncompressed sparsified update vectors. However, this conclusion does not rule out the possibility that CS can be an efficient method for OtA FL under different scenarios, e.g., when the sparsity patterns are unknown.