Federated learning is an emerging machine-learning technique that trains an algorithm across multiple decentralized edge devices or clients holding local data samples. It involves training local models on local data and uploading model parameters to a server node at regular intervals to generate a global model which is transmitted to all clients. However, edge nodes often have limited energy resources, and hence performing energy-efficient communication of model parameters is a bottleneck problem. We propose an energy-adaptive model sparsification for Federated Learning. The central idea is to adapt the sparsification level in run-time by optimizing the ratio between information content and energy cost. We illustrate the efficiency of the proposed algorithm by comparing its performance with three baseline schemes. We validate the performance of the proposed algorithm for two cost models. Simulation results show that the proposed algorithm needs exponentially less amount of communication and energy as compared to the three baseline schemes while achieving the best accuracy and fastest convergence.

Federated learning has emerged as a popular distributed machine-learning paradigm. It involves many rounds of iterative communication between nodes to exchange model parameters. With the increasing complexity of ML tasks, the models can be large, having millions of parameters. Moreover, edge and IoT nodes often have limited energy resources and channel bandwidths. Thus, reducing the communication cost in Federated Learning is a bottleneck problem. This cost could be in terms of energy consumed, delay involved, or amount of data communicated. We propose a communication cost-adaptive model sparsification for Federated Learning with Error Compensation. The central idea is to adapt the sparsification level in run-time by optimizing the ratio between the impact of the communicated model parameters and communication cost. We carry out a detailed convergence analysis to establish the theoretical foundations of the proposed algorithm. We conduct extensive experiments to train both convex and non-convex machine learning models on a standard dataset. We illustrate the efficiency of the proposed algorithm by comparing its performance with three baseline schemes. The performance of the proposed algorithm is validated for two communication models and three cost functions. Simulation results show that the proposed algorithm needs a substantially less amount of communication than the three baseline schemes while achieving the best accuracy and fastest convergence. The results are consistent for all the considered cost models, cost functions, and ML models. Thus, the proposed FL-CATE algorithm can substantially improve the communication efficiency of federated learning, irrespective of the ML tasks, costs, and communication models.

Internet of Things (IoT) systems increasingly operate in environments where devices must respond in real time while managing fluctuating resource constraints, including energy and bandwidth. Yet, current approaches often fall short in addressing scenarios where operational constraints evolve over time. To address these limitations, we propose a novel Budgeted Multi-Armed Bandit framework tailored for IoT applications with dynamic operational limits. Our model introduces a decaying violation budget, which permits limited constraint violations early in the learning process and gradually enforces stricter compliance over time. We present the Budgeted Upper Confidence Bound (UCB) algorithm, which adaptively balances performance optimization and compliance with time-varying constraints. We provide theoretical guarantees showing that Budgeted UCB achieves sublinear regret and logarithmic constraint violations over the learning horizon. Extensive simulations in a wireless communication setting show that our approach achieves faster adaptation and better constraint satisfaction than standard online learning methods. These results highlight the framework’s potential for building adaptive, resource-aware IoT systems.

IoT networks of the future will be characterized by autonomous decision-making by individual devices. Decision-making is done with the purpose of optimizing certain objectives. A multitude of mathematically oriented algorithms exist for solving optimization problems. However, optimization in IoT networks is challenging due to a number of uncertainties, complex network topologies, and rapid changes in the environment. This makes the data-driven and machine learning (ML) approaches more suitable for effectively handling IoT environments’ dynamic and intricate nature. However, supervised and unsupervised ML approaches depend on training data, which is not always available before training. In recent years, reinforcement learning (RL) has attracted considerable attention for solving optimization problems in IoT. This is because RL has the distinguishing feature of learning with experience while interacting with the environment without training data. A central challenge in decision-making in IoT networks is that most optimization problems consist of co-optimizing multiple conflicting objectives. With the development of multi-objective RL (MORL) approaches over the last two decades, there is great potential for utilizing them for future IoT networks. Most recently developed MORL approaches have not been applied in the IoT domain. In this chapter, we will discuss the need for efficient multi-objective optimization in IoT, the fundamentals of using RL for decision-making in IoT, an overview of existing MORL approaches, and, finally, the future scope and challenges associated with utilizing MORL for IoT.

A key challenge in many IoT applications is to en-sure energy efficiency while processing large amounts of streaming data at the edge. Nodes often need to process time-sensitive data using limited computing and communication resources. To that end, we design a novel R - Learning based Offloading framework, RLO, that allows edge nodes to learn energy optimal decisions from experience regarding processing incoming data streams. In particular, when should the node process data locally? When should it transmit data to be processed by a fog node? And when should it store data for later processing? We validate our results on both real and simulated data streams. Simulation results show that RLO learns with time to achieve better overall-rewards with respect to three existing baseline schemes. Moreover, the proposed algorithm excels the existing baseline schemes when different priorities were set on the two objectives. We also illustrate how to adjust the priorities of the two objectives based on the application requirements and network constraints.

IoT networks often face conflicting routing goals such as maximizing packet delivery, minimizing delay, and conserving limited battery energy. These priorities can also change dynamically: for example, an emergency alert requires high reliability, while routine monitoring prioritizes energy efficiency to prolong network lifetime. Existing works, including many deep reinforcement learning approaches, are typically centralized and assume static objectives, making them slow to adapt when preferences shift. We propose a dynamic and fully distributed multi-objective Q-learning routing algorithm that learns multiple per-preference Q-tables in parallel and introduces a novel greedy interpolation policy to act near-optimally for unseen preferences. The algorithm learns to optimize for energy efficiency, packet delivery ratio, and the composite reward, adapting to changing trade-offs between these metrics without retraining or centralized control. A theoretical analysis further shows that the optimal value function is Lipschitz-continuous in the preference parameter, ensuring that proposed greedy interpolation policy yields provably near-optimal behavior. Simulation results show that our approach adapts in real time to shifting priorities and achieves up to 80-90% lower energy consumption and up to 5 × higher cumulative rewards and packet delivery compared to six baseline protocols, under dynamic and distributed settings. Sensitivity analysis across varying preference window lengths confirms that the proposed DPQ framework consistently achieves higher composite reward than all baseline methods, demonstrating robustness to changes in operating conditions.