Search

With the proliferation of the Internet of Things and smart devices, there exists an urge to address the critical computation demands of end users for several real-time applications. Fog computing (FC) targets to address the computation requirements nearer to the end user in almost real-time. Mainly, the fog environments (FE) exhibit characteristics of resource constraint and wide heterogeneity. In addition, the complex fog environment enforces the fog resources (FR) deployment either within fixed or distributed over random geographical locations. In the literature, there exist several solutions for the optimization of FR allocation. However, the location-dependent FR allocation has not been addressed thoroughly. Hence, this paper proposes a novel hybrid clustering-based deep Q-network for location-based optimal resource allocation (HCDQN-ORA) model. Here, the first objective is to minimize the search operation for identifying suitable FR and to meet the quality of service (QoS) demand using an enhanced fog resource clustering (E-FRC) algorithm. The second objective aims to achieve optimal FR allocation among the set of location-dependent FRs using reinforcement learning techniques. Problem formulation has been modelled using Markov decision process. A deep Q-network algorithm with two variants, namely enhanced optimal resource allocation using deep Q-network (EORA-DQN) and enhanced optimal resource allocation using deep Q-network with experience replay (EORA-DQN-ER), has been incorporated. To analyse the statically located and randomly distributed FR environment, the experimental results exhibit that, in the case of clustering of FRs, E-FRC is more efficient than conventional clustering algorithms. Next, the location-dependent optimal FR allocation problem incorporates the performance analysis based on the convergence of loss function and the parametric analysis based on batch size, respectively. Results exhibit that EORA-DQN performs better only for statically allotted resources. However, the proposed EORA-DQN-ER outperforms both types of FR with a success rate of 89.6% of optimal allocation with the batch size of 200. [ABSTRACT FROM AUTHOR]

The integration of new Internet of Things (IoT) applications and services heavily relies on task offloading to external devices due to the constrained computing and battery resources of IoT devices. Up to now, Cloud Computing (CC) paradigm has been a good approach for tasks where latency is not critical, but it is not useful when latency matters, so Multi-access Edge Computing (MEC) can be of use. In this work, we propose a distributed Deep Reinforcement Learning (DRL) tool to optimize the binary task offloading decision, this is, the independent decision of where to execute each computing task, depending on many factors. The optimization goal in this work is to maximize the Quality-of-Experience (QoE) when performing tasks, which is defined as a metric related to the battery level of the UE, but subject to satisfying tasks' latency requirements. This distributed DRL approach, specifically an Actor-Critic (AC) algorithm running on each User Equipment (UE), is evaluated through the simulation of two distinct scenarios and outperforms other analyzed baselines in terms of QoE values and/or energy consumption in dynamic environments, also demonstrating that decisions need to be adapted to the environment's evolution. [ABSTRACT FROM AUTHOR]