Machine Learning-based Computation Offloading in Energy-Harvesting 5G Networks

Abstract

This dissertation explores the application of machine learning for enhancing computation offloading in 5G networks, driven by three key areas in Mobile Edge Computing (MEC) research. Firstly, the work in this dissertation addresses computation offloading for enhanced Mobile Broadband (eMBB) applications requiring high transmission data rates, in a multi-user edge network with Energy-Harvesting (EH) devices. Integrating the Simultaneous Wireless Information and Power Transfer (SWIPT) technology provides a sustainable energy source for resource-constrained User Equipment (UE). An objective function that improves energy efficiency by minimizing UE energy consumption and task queueing delays while maximizing offloading transmission data rates is proposed. Deep Reinforcement Learning (DRL) is utilized in this work to overcome the limitations of traditional optimization methods, which struggle with scalability. The proposed approach uses a state-of-the-art DRL model, Twin Delayed Deep Deterministic Policy Gradient (TD3), and is shown to achieve superior optimization compared to prior works that rely on earlier DRL variants. Next, this work focuses on the coexistence of eMBB and Ultra-Reliable Low-Latency Communication (URLLC) users, addressing the same objectives. A coexistence mechanism that enables both user types to share channel resources effectively is designed. For URLLC users, who demand minimal latency and high reliability, the 5G New Radio (NR) is used for short packet transmissions in small Transmission Time Intervals (TTIs), and a strict transmission reliability constraint is introduced. Optimally slicing wireless channel resources using DRL and the 5G flexible frame structure for uplink transmissions ensures that both user types meet their Quality of Service (QoS) requirements. Lastly, as future work, this dissertation shows how to tackle the user association problem in a system with multiple Next Generation Node Bs (gNBs). A Federated Learning (FL) approach is adopted, where several gNBs share knowledge of UE states through Deep Neural Network (DNN) model parameters, which are first trained at each gNB using information about respective associated users and are then transmitted from the gNBs to a centralized Mobility Management Entity (MME). The MME aggregates these models and sends updated parameters back to the gNBs. This method saves communication bandwidth as model parameters are significantly lighter than training data. A global objective function is introduced to guide model training, optimizing data rates between users and their associated gNBs. Simulation results show that all hyperparameter tuning conducted only affects the convergence rate of the chosen DRL model. The TD3 model performs better in terms of convergence rate, and variance, than its earlier variant, Deep Deterministic Policy Gradient (DDPG), which is widely used in prior DRL-based computation offloading works. Several TD3 policies evaluated against each other have shown that reward shaping is a crucial element in DRL and, especially in this work, that it results in trade-offs among eMBB energy consumption, throughput, delay, and URLLC reliability.