Search

A large number of distributed applications necessitate accurate network distance, for example, in the form of delay or latency, to ensure the Quality of Service (QoS). Due to high network measurement overhead and severe traffic congestion, network distance prediction has been introduced, instead of direct network measurements, to infer the unknown network distance with the partial measurements. However, most existing efforts neglect to fully capitalize on the potential latent factors, such as spatial correlations, long-existing temporal results and multi-rule exploration fusion, to achieve better accuracies with quicker convergence. To fill this gap, in this paper, we propose an Accurate Prediction of Network Distance (APND) solution via Federated Deep Reinforcement Learning (FDRL), which has four novel features distinguishing from the previous work. Firstly, a local feature-based matrix with low rank is established in each network cluster, referring to a set of neighbor nodes, to represent the potential spatial correlations among reachable node-pairs. Secondly, the parallel FDRL-based matrix factorization with multi-rule exploration fusion is introduced into APND and executed in all local clusters to minimize prediction errors and accelerate learning convergence. Thirdly, the long-existing learning experience is designed for local model training via Deep Reinforcement Learning (DRL) with rapid convergence. Fourthly, following the real-world routing paths, the cross-domain network nodes are simultaneously classified into adjacent clusters, built on the spatial correlations among them, and their coordinates will be further refined with error-based and average-based policies. Extensive experiments built on available real-world datasets illustrate that APND can accurately predict network distance compared with state-of-the-art approaches at the moderate computing cost.

Topology robustness is critical to the connectivity and lifetime of large-scale Internet-of-Things (IoT) applications. To improve robustness while reducing the execution cost, the existing robustness optimization methods utilize neural learning schemes, including neural networks, deep learning, and reinforcement learning. However, insufficient exploration of reinforcement learning agents for topological environments is likely to yield local optima. Moreover, convergence speed is influenced by the sparse reward problem generated while exploring topological environments. To address these problems, this study proposes a self-adaptive robustness optimization method with an evolutionary multi-agent for IoT topology (ROMEM). ROMEM introduces a new multi-agent co-evolution scheme that leverages a non-deterministic strategy to extend the exploration in multi-directions, enabling the reinforcement learning agent to transcend local optima. Furthermore, ROMEM presents a novel distributed training mechanism for multiple agents to accelerate convergence. Experimental results demonstrate that ROMEM can achieve multi-directional collaborative training and outperform other state-of-the-art learning-based robustness optimization methods in terms of convergence efficiency and robustness.