Search

We consider a public blockchain realized in the mobile edge computing (MEC) network, where the blockchain miners compete against each other to solve the proof-of-work puzzle and win a mining reward. Due to limited computing capabilities of their mobile terminals, miners offload computations to the MEC servers. The MEC servers are maintained by the service provider (SP) that sells its computing resources to the miners. The SP aims at maximizing its long-term profit subject to miners’ budget constraints. The miners decide on their hash rates, i.e., computing powers, simultaneously and independently, to maximize their payoffs without revealing their decisions to other miners. As such, the interactions between the SP and miners are modeled as a stochastic Stackelberg game under private information, where the SP assigns the price per unit hash rate, and miners select their actions, i.e., hash rate decisions, without observing actions of other miners. We develop a hierarchical learning framework for this game based on fully- and partially-observable Markov decision models of the decision processes of the SP and miners. We show that the proposed learning algorithms converge to stable states in which miners’ actions are the best responses to the optimal price assigned by the SP. [ABSTRACT FROM AUTHOR]

The paper considers a cloud-based Content Delivery Network (CDN) with Device-to-Device (D2D) and Unmanned Aerial Vehicle (UAV) enabled caching. The network is managed by the operator that offers content transfer services to the set of Content Providers (CPs). A subscriber of any CP can receive its content either by first, cellular content transfer via a terrestrial Base Station (BS) or a UAV, or by second, D2D content transfer from another subscriber of this CP. Each CP aims to maximize its expected payoff defined as a reciprocal of the weighted sum of the expected content transfer cost and delay for its subscribers. The operator's objective is to maximize its long-term average revenue and stabilize the queuing system which represents the content transfer services. To model the interactions between the operator and the CPs, a novel framework is proposed that combines: first, cooperative game that enables the CPs to form coalitions in which all subscribers can exchange content via D2D links, thereby reducing content transfer costs and delays; second, Lyapunov optimization based dynamic channel and UAVs’ activity allocation policy of the operator. Through analytical and numerical evaluations, it is proven that if the operator and each CP are rational, the network reaches a state where all the CPs are in the stable coalitional structure, whereas, the dynamic policy of the operator is optimal with the trade-off in the total queue backlog. [ABSTRACT FROM AUTHOR]

This paper investigates the problem of distributed resource sharing in a device-to-device enabled heterogeneous network, where the various device pairs choose their transmission channels, modes, base stations (BSs), and power levels without any control by the BSs based only on the locally-observable information. This problem is represented as a Bayesian coalition formation game, where the players (device pairs) create coalitions to maximize their long-term rewards with no prior knowledge of the values of potential coalitions and the types of their members. To minimize these uncertainties, a novel Bayesian reinforcement learning (RL) model is derived. In this model, the players update (through repeated coalition formation) their beliefs about the types and coalitional values to reach a stable coalitional agreement. The proposed Bayesian RL-based coalition formation algorithms are implemented in a long-term evolution advanced network and evaluated using simulations. The algorithms show a superior performance when compared with other relevant resource allocation schemes and achieve near-optimal results after a relatively small number of RL iterations. [ABSTRACT FROM PUBLISHER]

We study the problem of autonomous operation of the device-to-device (D2D) pairs in a heterogeneous cellular network with multiple base stations (BSs). The spectrum bands of the BSs (that may overlap with each other) comprise the sets of orthogonal wireless channels. We consider the following spectrum usage scenarios: 1) the D2D pairs transmit over the dedicated frequency bands and 2) the D2D pairs operate on the shared cellular/D2D channels. The goal of each device pair is to jointly select the wireless channel and power level to maximize its reward, defined as the difference between the achieved throughput and the cost of power consumption, constrained by its minimum tolerable signal-to-interference-plus-noise ratio requirements. We formulate this problem as a stochastic non-cooperative game with multiple players (D2D pairs) where each player becomes a learning agent whose task is to learn its best strategy (based on the locally observed information) and develop a fully autonomous multi-agent Q-learning algorithm converging to a mixed-strategy Nash equilibrium. The proposed learning method is implemented in a long term evolution-advanced network and evaluated via the OPNET-based simulations. The algorithm shows relatively fast convergence and near-optimal performance after a small number of iterations. [ABSTRACT FROM PUBLISHER]