Search

Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, such as deep reinforcement learning (DRL), can achieve higher convergence speed efficiency and learning accuracy based on the parametric approximation for the large-scale state-action space. This paper provides a comprehensive research review on RL-enabled MEC and offers insight for development in this area. More importantly, associated with free mobility, dynamic channels, and distributed services, the MEC challenges that can be solved by different kinds of RL algorithms are identified, followed by how they can be solved by RL solutions in diverse mobile applications. Finally, the open challenges are discussed to provide helpful guidance for future research in RL training and learning MEC.

Urban transportation network design and traffic control problems fall within the scope of infrastructural engineering sciences which become increasingly more important in ever-growing societies of today. In highly populated old cities where establishing new links are facing many human-related, social, economic, and political problems, a workaround for addressing traffic problems is to expand the capacity of existing links, so as to not only control the traffic, but also reduce the urban environmental pollutions caused by vehicles stuck in traffic and decrease the time wasted in traffic to accelerate routines of the society. In the present research, an urban transportation network design model is presented with the aim of enhancing travel time reliability by expanding the capacity of existing network links at minimum possible cost. A significant assumption taken in the present study is that demands in normal condition and peak traffic hours are treated separately, so as to prevent possible problems by congestion management. In the present study, the uncertainty associated with demand for travel, travel time, and the flow passing through different links are taken into consideration. Travel time reliability calculations are carried out assuming that the demand for travel and travel time follow lognormal distributions. In order to solve this bi-level model, particle swarm optimization algorithm was used. Incorporation of the inertial coefficients dynamics, personal learning, and communal learning into the algorithm contributes to the convergence of this algorithm for solving the bi-level model. [ABSTRACT FROM AUTHOR]

Spatial econometric models become increasingly popular in various subfields of political science. However, the necessity to specify the underlying network of dependencies, denoted by  $\boldsymbol{W}$ , prior to estimation is a prevalent source of criticism since the true dependence structure is rarely known and theories mostly provide insufficient guidance. The present study investigates the effects of this network uncertainty which is a special case of model uncertainty that arises from uncertainty about the correct specification of  $\boldsymbol{W}$. It advocates Bayesian model averaging (BMA) as a superior approach to this problem, located at the intersection of theory and empirics. Conducting Monte Carlo experiments, I demonstrate that, while the effect estimates are robust toward a misspecification in the functional form of  $\boldsymbol{W}$ , uncertainty in the neighborhood definition can bias the effect estimates derived from spatial autoregressive models. In contrast to alternative techniques, BMA directly addresses network uncertainty, correctly identifies the true network structure in the set of feasible alternatives, and provides unbiased effect estimates. Two replication studies from different subfields of the discipline illustrate the benefits of this approach for applied research. [ABSTRACT FROM AUTHOR]

The objective of this study is to develop a non UE (User Equilibrium)-based traffic assignment model which is able to be applied to a real size network. Over the last few decades, the UE principle has been used as a standard route choice principle. However, it also has received considerable criticism since it is difficult to fully describe the route choice behavior of real drivers. The real drivers select a route based on their cumulated experiences, and their perceived travel times are also not deterministic values, but probabilistic values which reflect the various uncertainties existing in the network. If a driver frequently uses a specific route, then the driver has better information on that route than other routes. As a result, a marginal superiority of other paths might not make the driver quit using the frequently selected path. Accordingly, a network traffic pattern is not so sensitive to the change of travel environment and drivers tend to select routes stably and habitually. In addition, the cumulated information in drivers' minds and the travel experiences are also important to develop traffic operation and management schemes. However, a UE-based traffic assignment model cannot take into account the previous travel experiences and network uncertainties. Therefore, the route choice in the UE-based model might be over-sensitive to the change of travel environment in the network or occasionally forecasted incorrectly because of the absence of consideration for previous travel experiences and perception limit. In order to overcome these problems, the authors developed an agent-based traffic assignment model while making use of 'reference point' theory in the field of psychology and 'bounded rationality' used in decision making theory. In the developed model, travel time is modeled not as a deterministic value, but as the concept of probability density function in order to reflect the network uncertainties with which the real drivers face in the real world. In addition, the density function is linearly simplified in order to improve applicability to the real-size transportation network problems. [ABSTRACT FROM AUTHOR]

Abstract: This paper is concerned with finding first-best tolls in static transportation networks with day-to-day variation in network capacity, as accounted for by changes in the volume-delay function. The key question in addressing this problem is that of information, namely, which agents have access to what information when making decisions. In this work, travelers are assumed to be either fully informed about network conditions before embarking on travel, or having no information except the probability distributions; likewise, the network manager (toll-setter) is either able to vary tolls in response to realized network conditions, or must apply the same tolls every day. Further, travelers’ preference for reliable travel is accounted for, representing risk aversion in the face of uncertainty. For each of the scenarios implied by combinations of these assumptions, we present methods to determine system-optimal link prices. A demonstration is provided, using the Sioux Falls test network, suggesting that attempts to incorporate uncertainty into nonresponsive tolls involve significantly higher prices. [Copyright &y& Elsevier]

Although local protection methods can perform well at speed, in a real network where network uncertainties are unavoidable, these methods may result in inaccurate performance. Therefore, the use of other methods, such as wide area protection, can lead to more accurate results because of using comprehensive information via telecommunication links. In this paper, a wide area protection algorithm is presented to cover the mal-operation of relays due to the network uncertainties. In this method, an objective function is defined based on the performance status of various zones of distance relays, the values of which will be different for each line. Depending on the position of each relay on the transmission line, weighting factors for objective function are determined based on the optimization algorithm and taking into account the possible uncertainties of the power grid. The proposed method has been implemented on IEEE 9-bus system and has proved high detection precision and performance accuracy. [ABSTRACT FROM AUTHOR]

Cancer prognosis prediction is typically carried out without integrating scientific knowledge available on genomic pathways, the effect of drugs on cell dynamics, or modeling mutations in the population. Recent work addresses some of these problems by formulating an uncertainty class of Boolean regulatory models for abnormal gene regulation, assigning prognosis scores to each network based on intervention outcomes, and partitioning networks in the uncertainty class into prognosis classes based on these scores. For a new patient, the probability distribution of the prognosis class was evaluated using optimal Bayesian classification, given patient data. It was assumed that (1) disease is the result of several mutations of a known healthy network and that these mutations and their probability distribution in the population are known and (2) only a single snapshot of the patient's gene activity profile is observed. It was shown that, even in ideal settings where cancer in the population and the effect of a drug are fully modeled, a single static measurement is typically not sufficient. Here, we study what measurements are sufficient to predict prognosis. In particular, we relax assumption (1) by addressing how population data may be used to estimate network probabilities, and extend assumption (2) to include static and time-series measurements of both population and patient data. Furthermore, we extend the prediction of prognosis classes to optimal Bayesian regression of prognosis metrics. Even when time-series data is preferable to infer a stochastic dynamical network, we show that static data can be superior for prognosis prediction when constrained to small samples. Furthermore, although population data is helpful, performance is not sensitive to inaccuracies in the estimated network probabilities.