Search

With the development of new radio (NR) technologies, sidelink (SL) transmission plays a more vital role in supporting low latency, higher reliability vehicle-to-everything (V2X) communications. Compared with the Long-Term Evolution (LTE), NR V2X aims to support enhanced V2X use cases with more stringent latency and reliability requirements. 3GPP introduces NR V2X distributed resource allocation to fulfill the low latency requirement to reduce the delay caused by requesting transmission resources from gNB. In distributed mode, UEs autonomously select resources without the scheduling from gNB. NR V2X introduces the physical sidelink feedback channel (PSFCH) to enhance transmission reliability. This paper presents the resource allocation mechanism for NR V2X distributed mode. Since the SL feedback channel is a new feature different from LTE V2X, we discuss the potential enhancements on resource allocation utilizing SL feedback channel to improve the reliability. This paper proposes a Receiver Grant-based resource allocation, which allows the receiver to send the grant with the feedback channel. By allowing the transmitter to indicate the pre-selected resources before the end of the previous reservation, the receiver can inform the transmitter about the collided resource selection through the PSFCH. The analytical models for NR V2X distributed mode with the feedback channel are also provided for UE to estimate the collision ratio. The simulation results and the analytical model show that the proposed Receiver (Rx) Grant-based resource allocation improves transmission reliability in terms of collision ratio. For future advanced V2X use cases, the analytical models provide an evaluation method and help UE make decisions to meet the reliability requirements. [ABSTRACT FROM AUTHOR]

Swift decision-making based on visual environment perception is crucial for autonomous control of visual underwater vehicles (VUVs) during underwater missions. However, learning perception and decision models individually might result in weak robustness of overall control system as the mismatched state extraction and control decision making are asynchronous. As a remedy, we will introduce in this paper an end-to-end monocular autonomous reinforcement control (MARC) framework for autonomous control of VUVs, which is performed in two cascaded procedures, i.e., 1) perception, where a geometric network (GeoNet) is designed based on a convolutional encoder-decoder network to generate depth maps from input environmental videos; 2) decision, where with depth maps as input, a reinforcement control network (CtrlNet) integrates a convolutional neural network into a deep deterministic policy gradient network and outputs action decisions, which are refined by reinforcement learning algorithm for obstacle-avoiding based autonomous control. Numerical and experimental results demonstrate that the proposed MARC exhibits high-quality depth prediction and is capable of conducting obstacle-avoiding navigation and autonomous control of VUVs with high accuracy and strong robustness. [ABSTRACT FROM AUTHOR]

For vehicle-to-network communications, handover (HO) management enables vehicles to maintain the connection with the network while transiting through coverage areas of different base stations (BSs). However, the high mobility of vehicles means shorter connection periods with each BS that leads to frequent HOs, hence raises the necessity for optimal HO decision making for high quality infotainment services. Machine learning is capable of capturing underlying pattern via data driven methods to find optimal solutions to complex problems, and much learning-based HO optimization research has been conducted focusing on specific network setups. However, attention still needs to be paid to the actual deployment aspect and standardized datasets or simulation environments for evaluation. This paper proposes a deep reinforcement learning-based HO algorithm using the input parameters that are configurable in the existing measurement report of cellular networks. The performance of the proposed algorithm is evaluated using the well-known network simulator ns-3 with its official LTE module. A realistic network setup in the city center of Glasgow (U.K.) is configured with vehicle trajectories generated by the routes mobility model using Google Maps Directions API. Evaluation results reveal that the proposed algorithm significantly outperforms the A3 RSRP baseline with an average of 25.72% packet loss reduction per HO, suggesting significant improvement in quality of service of phone call and video streaming, etc. The proposed algorithm also has a small implementation cost compared to some state-of-the-art and should be deployed by a software update to a local BS controller. [ABSTRACT FROM AUTHOR]

An emerging computational paradigm, named federated edge learning (FEL), enables intelligent computing at the network edge with the feature of preserving data privacy for edge devices. Given their constrained resources, it becomes a great challenge to achieve high execution performance for FEL. Most of the state-of-the-arts concentrate on enhancing FEL from the perspective of system operation procedures, taking few precautions during the composition step of the FEL system. Though a few recent studies recognize the importance of FEL formation and propose server-centric device selection schemes, the impact of data sizes is largely overlooked. In this paper, we take advantage of game theory to depict the decision dilemma among edge devices regarding whether to participate in FEL or not given their heterogeneous sizes of local datasets. For realizing both the individual and global optimization, the server is employed to solve the participation dilemma, which requires accurate information collection for devices’ local datasets. Hence, we utilize mechanism design to enable truthful information solicitation. With the help of correlated equilibrium, we derive a decision making strategy for devices from the global perspective, which can achieve the long-term stability and efficacy of FEL. For scalability consideration, we optimize the computational complexity of the basic solution to the polynomial level. Lastly, extensive experiments based on both real and synthetic data are conducted to evaluate our proposed mechanisms, with experimental results demonstrating the performance advantages. [ABSTRACT FROM AUTHOR]

The three main modules of autonomous vehicles, i.e., sensing, decision making, and motion controlling, have been studied separately in most existing works on autonomous driving, which overlook the correlations among these modules, leading to a result of unsatisfactory performance. In this paper, we propose a novel scheme that first tactfully processes the sensing data, then jointly learns and optimizes the decision-making and motion-controlling using reinforcement learning (RL). Specifically, the proposed scheme designs a novel state representation mechanism, where the sensing data goes through the attention layer and the convolutional neural network (CNN) layer sequentially. The attention layer focuses on extracting the most important local information and then CNN layer takes a broad view to comprehensively consider the global information for a better representation. Furthermore, the proposed scheme jointly learns decision-making and motion-controlling, therefore, the relevance of these two modules is implicitly considered, which helps to achieve a better autonomous driving policy. Extensive simulation results show that the proposed scheme is better than classic control methods and some RL methods in terms of safety, velocity, etc. We also demonstrate the respective functions of the attention layer and the CNN layer through ablation studies. Finally, we construct a traffic scene with a real autonomous vehicle, and verified the feasibility of the proposed scheme. [ABSTRACT FROM AUTHOR]

In order to solve the manual labelling, long-tail effect and driving conservatism of the existing decision-making algorithm. This paper proposed an integrated decision-making framework (IDF) for highway autonomous vehicles. Firstly, states of the highway traffic are extracted by the velocity, time headway (TH) and the probabilistic lane distribution of the surrounding vehicles. With the extracted traffic state, the reinforcement learning (RL) is adopted to learn the optimal state-action pair for specific scenario. Analogously, by mapping millions of traffic scenarios, huge amounts of state-action pairs can be stored in the experience pool. Then the imitation learning (IL) is further employed to memorize the experience pool by deep neural networks. The learning result shows that the accuracy of the decision network can reach 94.17%. Besides, for some imperfect decisions of the network, the rule-based method is taken to rectify by judging the long-term reward. Finally, the IDF is simulated in G25 highway and has promising results, which can always drive the vehicle to the state with high efficiency while ensuring safety. [ABSTRACT FROM AUTHOR]

In this paper, the problem of decision making for autonomous vehicles changing lanes is addressed by formulating multiple games in normal form for pairs of agents. This formulation generates the optimal action for the Ego vehicle at a given state and does not consider global optimality for all agents. The payoff matrices of the games are designed based on a user-defined set of rules. The constant parameters of these payoffs are then adjusted using neural learning to generate optimal behavior among the vehicles. An algorithm integrating deep reinforcement learning and game theory, regarded as Nash Q-learning, is included in the decision-making scheme. The applicability of the proposed method in a lane-changing scenario is tested via simulation. [ABSTRACT FROM AUTHOR]

Nowadays, the autonomous driving transportation systems are at the heart of both academic and industry research for the distinguished advantages including increased network capacity, enhanced punctuality, greater flexibility and improved overall safety level. With the responsibility of transporting passengers in a safe, comfortable and efficient way, the decision making method plays a critical position in the autonomous driving of high-speed trains. Focusing on solving the autonomous decision making problem, this paper proposes a novel learning based framework by combining the deep learning technology with the distributed tracking control approach. To cope with the data insufficiency problem in training the deep learning network, a generative adversarial network (GAN) based data argumentation scheme is proposed to generate data samples that have the same distribution with actual data samples, and a hybrid learning network is constructed to predict the speed trajectory from the multi-attribute data with both temporal sequences and static features. Then, based on the model predictive control (MPC) scheme, a distributed tracking control model is formulated to minimize the tracking deviations and balance the performance of punctuality, energy-efficiency and riding comfort. Further, the dual decomposition technique is adopted to deal with the coupling constraints for the safe distance headway such that the separation for the autonomous driving of high-speed trains is achieved. Finally, simulation experiments based on actual scenarios of the Beijing-Shanghai high-speed railway are conducted to illustrate the effectiveness of our methods. [ABSTRACT FROM AUTHOR]

Autonomous pilot is crucial in integrally promoting the autonomy of an unmanned surface vehicle (USV). However, the integration mechanism of decision and control is still unclear within the entire autonomy. In this paper, by organically bridging path planning and tracking, an autonomous pilot framework with waypoints generation, path smoothing and policy guidance of a USV in congested waters is established, for the first time. Incorporating elite and diversity operations into the genetic algorithm (GA), an elite-duplication GA (EGA) strategy is devised to optimally generate sparse waypoints in a constrained space. The B-spline technique is further deployed to make flexibly smooth interpolation facilitating path smoothing supported by optimal sparse-waypoints. Seamlessly bridged by the parametric smooth path, deep reinforcement learning (DRL) technique is resorted to continuously extract in-depth pilotage policies, i.e., mappings from path tracking errors, collision risks and control constraints to continuous control forces/torques. Eventually, the entire spline-bridged EGA-DRL (SED) framework merits autonomous global-pilotage and local-reaction in an organically modular manner. Comprehensive validations and comparisons in various real-world geographies demonstrate the effectiveness and superiority of the proposed SED autonomous pilot framework. [ABSTRACT FROM AUTHOR]

Channel allocation has a direct and profound impact on the performance of vehicle-to-everything (V2X) networks. Considering the dynamic nature of vehicular environments, it is appealing to devise a blended strategy to perform effective resource sharing. In this paper, we exploit deep learning techniques predict vehicles’ mobility patterns. Then we propose an architecture consisting of centralized decision making and distributed channel allocation to maximize the spectrum efficiency of all vehicles involved. To achieve this, we leverage two deep reinforcement learning techniques, namely deep Q-network (DQN) and advantage actor-critic (A2C) techniques. In addition, given the time varying nature of the user mobility, we further incorporate the long short-term memory (LSTM) into DQN and A2C techniques. The combined system tracks user mobility, varying demands and channel conditions and adapt resource allocation dynamically. We verify the performance of the proposed methods through extensive simulations and prove the effectiveness of the proposed LSTM-DQN and LSTM-A2C algorithms using real data obtained from California state transportation department. [ABSTRACT FROM AUTHOR]

Network slicing in future 5G systems enables the provision of multitenant networks in which a network infrastructure owned by an operator is shared among different tenants, such as mobile virtual operators, over-the-top providers or vertical market players. The support of network slicing within the radio access network requires the introduction of appropriate radio resource management functions to ensure that each tenant gets the required radio resources in accordance with the expected service level agreement (SLA). This paper addresses radio admission control (RAC) functionality in multiservice and multitenant scenarios as a mechanism for regulating the acceptance of new guaranteed bit rate service requests of different tenants. This paper proposes an optimization framework that models the RAC as a semi-Markov decision process and, as a result, derives an optimal decision-making policy that maximizes an average long-term function representing the desired optimization target. A reward function is proposed to capture the degree of tenant satisfaction with the received service in relation to the expected SLA, accounting for both the provision of excess capacity beyond the SLA and the cost associated with sporadic SLA breaches. The proposed approach is evaluated by means of simulations, and its superiority to other reference schemes in terms of reward and other key performance indicators is analyzed. [ABSTRACT FROM AUTHOR]

One of the pillars of the 5G architecture is network slicing, in which hardware, radio, and power resources are virtualized as a logical network taking into account the requirements of diverse applications. While ensuring performance isolation among different slices, resource allocation in 5G Radio Access Networks (RANs) is associated with different challenges due to network dynamics and the different applications’ requirements. In this paper, we have considered the allocation of power and radio resources to rate-based as well as resource-based users. We propose an energy-efficient deep reinforcement learning-assisted resource allocation (EE-DRL-RA) method for RAN slicing in 5G networks. The main idea of the proposed method is to exploit a collaborative learning framework that includes deep reinforcement learning (DRL) and deep learning (DL) to decide on resource allocation in the RAN. Specifically, we use DL for decision-making on resource allocation on a large time-scale and DRL for decision-making on resource allocation on a small time-scale. The asynchronous advantage actor-critic (A3C) and the stacked and bidirectional long-short-term-memory (SBiLSTM) network are used as DRL and supervised DL methods, respectively. Furthermore, we determine the optimal power and resource blocks (RBs) for rate-based users by formulating the energy-efficient power allocation (EE-PA) problem as a non-convex optimization problem and solve it by an efficient iterative algorithm. Our proposed approach is unique in that it simultaneously allocates power and RBs while ensuring slice isolation with low computational and time complexity. Simulation results show that EE-DRL-RA yields better performance compared to a state-of-the-art published method in terms of convergence speed, computational complexity, energy efficiency, and the number of accepted users as well as the degree of inter-slice isolation. [ABSTRACT FROM AUTHOR]

Knowledge transfer is a promising concept to achieve real-time decision-making for autonomous vehicles. This paper constructs a transfer deep reinforcement learning (RL) framework to transform the driving tasks in the intersection environments. The driving missions at the unsignalized intersection are cast into a left turn, right turn, and running straight for automated vehicles. The goal of the autonomous ego vehicle (AEV) is to drive through the intersection situation efficiently and safely. This objective promotes the studied vehicle to increase its speed and avoid crashing other vehicles. The decision-making policy learned from one driving task is transferred through three transfer rules in another driving mission and evaluated. Simulation results reveal that the decision-making strategies related to similar tasks are transferable and have a high success rate. It indicates that the presented control framework could reduce time consumption and realize online implementation. Therefore, the transfer RL concept is helpful for establishing the real-time decision-making policy for autonomous vehicles. [ABSTRACT FROM AUTHOR]

This paper presents a latent space reinforcement learning method for interpretable decision-making of autonomous vehicles at highway on-ramps. This method is based on the latent model and the combination model of the hidden Markov model and Gaussian mixture regression (HMM-GMR). It is difficult for the traditional decision-making method to understand the environment because its input is high-dimensional and lacks an understanding of the task. By utilizing the HMM-GMR model, we can obtain the interpretable state providing semantic information and environment understanding. A framework is proposed to unify representation learning with the deep reinforcement learning (DRL) approach, in which the latent model is used to reduce the dimension of interpretable state by extracting underlying task-relevant information. Experimental results are presented and the results show the right balance between driving safety and efficiency in the challenging scenarios of highway on-ramps merging. [ABSTRACT FROM AUTHOR]

In this paper, we propose a mechanism – DEFT– to facilitate mobile Internet of Vehicles (IoV) user entities (UEs) with unit tasks to cooperate among one another and offload their tasks to nearby fog nodes (FNs)/roadside units (RSUs) in a decentralized and fair manner. While existing literature depends on offloading schemes based on centralized systems, DEFT provides near real-time computation offloading in a fog-enabled IoV using a two-level game-theoretic approach. Further, this approach allows the UEs to make their own decisions in a dynamic environment. In the first level, FNs play an Oligopoly game for determining the price of their services based on their computation capability and then broadcast the price to the UEs. Further, based on the price transmitted by the FNs, UEs play a non-cooperative Stackelberg game to map the offloading among the user and fog layer. We also ensure that no UE consumes all of the resources in an FN. Additionally, we prove our objective function's convexity and show that the proposed game always attains the Nash Equilibrium. Through extensive real-world emulations, we observe that DEFT minimizes the latency and power consumption, and improves throughput compared to the existing schemes. In the presence of 40, 80, and 120 UEs, the overhead for computation in the fog layer is minimized by $70-75$ % compared to the local computation overhead. [ABSTRACT FROM AUTHOR]

In a typical traffic scenario, autonomous vehicles are required to share the road with other road participants, e.g., human driven vehicles, pedestrians, etc. To successfully navigate the traffic, an adaptive robust level-k framework can be used by the autonomous agents to categorize the agents based on their depth of strategic thought and act accordingly. However, mismatch between the vehicle dynamics and its predictions, and improper classification of the agents can lead to undesirable behavior or collision. Robust approaches can handle the uncertainties, however, might result in a conservative behavior of the autonomous vehicle. This paper proposes an adaptive robust decision making strategy for autonomous vehicles to handle model mismatches in the prediction model while utilizing the confidence of the driver behavior to obtain less conservative actions. The effectiveness of the proposed approach is validated for a lane change maneuver in a highway driving scenario. [ABSTRACT FROM AUTHOR]

This paper presents a novel integrated approach to deal with the decision making and motion planning for lane-change maneuvers of autonomous vehicle (AV) considering social behaviors of surrounding traffic occupants. Reflected by driving styles and intentions of surrounding vehicles, the social behaviors are taken into consideration during the modelling process. Then, the Stackelberg Game theory is applied to solve the decision-making, which is formulated as a non-cooperative game problem. Besides, potential field is adopted in the motion planning model, which uses different potential functions to describe surrounding vehicles with different behaviors and road constraints. Then, Model Predictive Control (MPC) is utilized to predict the state and trajectory of the autonomous vehicle. Finally, the decision-making and motion planning is then integrated into a constrained multi-objective optimization problem. Three testing scenarios considering different social behaviors of surrounding vehicles are carried out to validate the performance of the proposed approach. Testing results show that the integrated approach is able to address different social interactions with other traffic participants, and make proper and safe decisions and planning for autonomous vehicles, demonstrating its feasibility and effectiveness. [ABSTRACT FROM AUTHOR]

As electric vehicles (EVs) are connected to the distribution power system, EV owners desire to charge their EVs economically, reliably, confidentially and without requiring major modification to the charging systems in their EVs. At the same time, the increasing EV connections to the distribution power system can increase the electric load demand and decrease the power quality (PQ) of the power system due to the charging current of the EV chargers. It is, therefore, necessary to devise novel strategies for the coordination of the connection of EVs at the distribution level. Although different control methods have been proposed previously, most of these solutions require costly advanced infrastructures and computational resources. In this paper, a charging control system (CCS) using a hierarchical decision-making (HDM) approach is devised that can be easily applied for the economical and reliable charging of commercial EVs with minor modifications while improving the PQ in the power grids. The HDM approach includes a primitive energy management strategy and an adaptive droop control for the real-time operation. A thorough investigation of the effectiveness of the proposed CSS to meet the major needs of EV owners while improving the PQ of the distribution network has been carried out. The results from the case studies show that the CCS can satisfy the preferences of the EV owners, while improving the PQ of distribution grids. Specifically, the CCS can meet the economical and reliable EV charging for the EV owners, and at the same time maintain the voltage deviation within the acceptable PQ limits. [ABSTRACT FROM AUTHOR]

In this paper, we consider the source localization problem with an underwater acoustic sensor network (UASN) consisting of a number of platforms each equipped with an array, such as an autonomous underwater vehicle (AUV). Direct position determination (DPD) has been a promising topic in recent years. Existing DPD methods can be divided into two categories: global coherent processing and non-coherent processing. The former is easy to implement in practical tasks but the coherence between the received signals at different nodes are not exploited, while the requirements for coherent processing may not be fulfilled in real environments. A multi-cluster array model is constructed considering the deployment of the nodes with the intention to exploit the available coherence in the UASN and avoiding the side effects of imperfect spatial coherence. A weighted MUSIC direct source localization approach is proposed based on this model. The arrays within a cluster work coherently as a large array and contribute a MUSIC spectrum. The final source position estimate is obtained from a weighted combination of these spectra, and the weights are dependent on the eigenstucture and noise variance of each cluster. We first investigate the theoretical gain of inter-array coherence exploitation on localization accuracy by means of Cramér-Rao bound analysis. Then the model for imperfect spatial coherence is generalized to the multi-cluster system to examine the performance of the proposed approach when the inherent constraints for intra-cluster coherent processing can not be guaranteed. Numerical results demonstrate the superior resolution capability of the proposed approach and its robustness to imperfect spatial coherence. [ABSTRACT FROM AUTHOR]

In this paper, we propose a trustworthy service provisioning scheme for Safety-as-a-Service (Safe-aaS) infrastructure in IoT-based intelligent transport systems. Typically, a Safe-aaS infrastructure provides customized safety-related decisions dynamically to multiple end-users using the concept of decision virtualization. We consider road transportation as the application environment of Safe-aaS to generate trustworthy decisions. On the other hand, the efficiency and accuracy of the decisions generated depend on the security, privacy, and trustworthiness of the participating sensor nodes and the route through which data transit. We propose a trust evaluation model to compute the trustworthiness of the data generated from these nodes. Further, we consider direct and indirect trust mechanisms for each of the sensor nodes and update their trust measures at regular intervals of time. Based on these measures, we evaluate the trust of each data item sourced from the network. We formulate an integer linear programming (ILP) model to select the optimal data for decision-making, while alleviating the effects of illegitimate sensor nodes. Further, we show that the formulated ILP problem is NP-hard and use a dynamic programming approach to solve the problem. Experimental results show that our proposed trust evaluation model exhibits more than 8% attack detection rate and 13% reduction in false attack detection rate in a network with 50% malicious nodes, compared to the benchmark schemes. The proposed trustworthy data selection algorithm outperforms different existing greedy approaches. [ABSTRACT FROM AUTHOR]

Ethical decision-making during inevitable crashes, especially when humans involved, has become a big and sensitive roadblock for future mass adoption of autonomous vehicles. Towards addressing this challenge, this paper proposes a predictive control framework for ethical decision-making in autonomous driving using rational ethics. For flexibly implementing of ethical rules, the Lexicographic Optimization-based model predictive controller (LO-MPC) has been designed, in which obstacles and constraints are prioritized. Simulation environment is set up in PreScan, with different edge cases. The results show that the proposed LO-MPC approach has the capability to deal with the ethical decision-making during inevitable crashes by avoiding the obstacles with the assumed priority orders compared with traditional decision-making algorithm. [ABSTRACT FROM AUTHOR]

Recently, the network slicing technology has emerged enabling network operators to provide dedicated virtual networks to customers, over a common network infrastructure. This paper addresses the service provider maximization revenue problem in network slicing, aiming at offering customized services for different users’ classes and heterogeneous requirements. In particular, the slicing of the network is performed here on the basis of the end users’ perspective, considered at different levels. The customers demand forecasting is pursuit for service classes through the application of the federated learning framework according to which the end users act as clients. Then, considering the forecast service demands, the virtual network functions placement strategy is performed by taking into account that different network zones are characterized by diverse provision costs and prices, and it is realized by resorting to the application of matching theory principles. Furthermore, with the aim at taking into consideration the customers perspectives to obtain a realistic users’ decisions process, the prospect theory principles are applied. Finally, the goodness of the proposed framework is investigated and validated by providing numerical results derived by extensive computer simulations and proposing performance comparisons with the Kolkata game and the potential game. [ABSTRACT FROM AUTHOR]

Different studies have recently emphasized the importance of deploying clustering schemes in Vehicular ad hoc Network (VANET) to overcome challenging problems related to scalability, frequent topology changes, scarcity of spectrum resources, maintaining clusters stability, and rational spectrum management. However, most of these studies addressed the clustering problem using conventional performance metrics while spectrum shortage, and the combination of spectrum trading and VANET architecture have not been tackled so far. Thus, this paper presents a new fuzzy logic based clustering control scheme to support scalability, enhance the stability of the network topology, motivate spectrum owners to share spectrum and provide efficient and cost-effective use of spectrum. Unlike existing studies, our context-aware scheme is based on multi-criteria decision making where fuzzy logic is adopted to rank the multi-attribute candidate nodes for optimizing the selection of cluster heads (CH)s. Criteria related to each candidate node include: received signal strength, speed of vehicle, vehicle location, spectrum price, reachability, and stability of node. Our model performs efficiently, exhibits faster recovery in response to topology changes and enhances the network efficiency life time. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel adaptive cloud resource allocation model based on Semi-Markov Decision Process (SMDP) and Reinforcement Learning (RL) algorithm in vehicular cloud system. The issue of adaptive resource allocation for vehicular request is formed as an SMDP in order to gain the dynamics of vehicular requests’ arrival and departure. An optimized decision is made to guarantee the Quality of Service (QoS) of the vehicular cloud system and the Quality of Experience (QoE) of the vehicular users as well as to maximize the total system reward of the vehicular cloud system in consideration of the balance between the vehicular cloud resource expense and the system income. Furthermore, to capture the mobility feature of the vehicular cloud system, we also apply a neural-network-based RL algorithm to resolve our proposed SMDP-based adaptive cloud resource allocation model. Firstly, we use a Planning algorithm to get the action values under certain state-action pairs, which are the initial samples to train the neural network. Then the RL is used to update the parameters of the neural network, train the neural network and adaptively improve the decision strategy. Subsequently, an adaptive vehicular cloud resource allocation scheme which can approach the optimal strategy is obtained without the knowledge of the distribution function of vehicular requests’ arrival and departure during the RL process. The simulation results show that our proposed adaptive cloud resource allocation model for vehicular cloud system can reduce the probability of delay in processing requests and achieve high system rewards in comparison with the regularly used greedy resource allocation method. The performance of the RL solution approaches that of traditional value iteration solution for our proposed adaptive cloud resource allocation model. [ABSTRACT FROM AUTHOR]

We study the problem of base station (BS) dynamic switching for energy efficient design of fifth-generation (5G) cellular networks and beyond. We formulate this problem as a Markov decision process (MDP) and use an approximation method known as policy rollout to solve it. This method employs Monte Carlo sampling to approximate the Q-value. In this paper, we introduce a novel approach to design an energy-efficient BS control algorithm. We design an MDP-based algorithm to control the on/off switching of BSs in real time; we exploit user mobility and location information in the selection of the optimal control actions. We start our formulation with the simple case of one-user one-ON. We then gradually and systematically extend this formulation to the multi-user multi-ON scenario. Simulation results show the potential of our novel approach of exploiting user mobility information within the MDP framework to achieve significant energy savings while providing quality-of-service guarantees. [ABSTRACT FROM AUTHOR]

This paper focuses on a heterogeneous scenario in which cellular and wireless local area technologies coexist and mobile devices are enabled with device-to-device (D2D) communication capabilities. In this context, this paper assumes a network architecture in which a given user equipment (UE) can receive mobile service either by connecting directly to a cellular base station (BS) or by connecting through another UE that acts as an access point (AP) and relays the traffic from a cellular BS. This paper investigates the optimization of the connectivity of different UEs with the target to minimize the total transmission power. An optimization framework is presented, and a distributed strategy based on Q-learning and softmax decision-making is proposed as a means to solve the considered problem with reduced complexity. The proposed strategy is evaluated under different conditions, and it is shown that the strategy achieves a performance very close to the optimum. Moreover, significant transmission power reductions of approximately 40% are obtained with respect to the classical approach, in which all UEs are connected to the cellular infrastructure. For multicell scenarios, in which the optimum solution cannot be easily known a priori, the proposed approach is compared against a centralized genetic algorithm. The proposed approach achieves similar performance in terms of total transmitted power while exhibiting much lower computational requirements. [ABSTRACT FROM PUBLISHER]

This paper presents a comprehensive cognitive management framework for spectrum selection in cognitive radio (CR) networks. The framework uses a belief vector concept as a means to predict the interference affecting the different spectrum blocks (SBs) and relies on a smart analysis of the scenario dynamicity to properly determine an adequate observation strategy to balance the tradeoff between achievable performance and measurement requirements. In this respect, the paper shows that the interference dynamics in a given SB can be properly characterized through the second highest eigenvalue of the interference state transition matrix. Therefore, this indicator is retained in the proposed framework as a relevant parameter to drive the selection of both the observation strategy and spectrum selection decision-making criterion. This paper evaluates the proposed framework to illustrate the capability to properly choose among a set of possible observation strategies under different scenario conditions. Furthermore, a comparison against other state-of-the-art solutions is presented. [ABSTRACT FROM PUBLISHER]

To meet the real-time requirements of device-to-device (D2D) communications, we study the scheduling problems of real-time D2D communications to handle the challenges introduced by the stochastic packet arrivals, packet losses, and heterogeneous deadlines. In this paper, we first model the scheduling problem by the Markov Decision Process (MDP), and then propose an $\epsilon$ -optimal scheduling algorithm based on this model. Considering the curse of dimensionality of the MDP-based model, we further model the scheduling problem by the restless multi-armed bandit model. Based on the model, we prove the indexability of the scheduling problem and derive the closed-form Whittle's index. According to the closed-form index, we thus propose an index-based scheduling algorithm that has very low time complexity. Finally, numerous simulations demonstrate the effectiveness of the proposed algorithms. The $\epsilon$ -optimal scheduling algorithm converges very fast to optimal solutions with small-sized networks and small $\epsilon$. The index-based scheduling algorithm outperforms the compared algorithms in terms of the total discounted reward. [ABSTRACT FROM AUTHOR]

Cache management and resource scheduling schemes enhance the mobility and flow control in SDN. Dynamic and opportunistic spectrum accessing modes enhance the resource allocation and reduce the cache delays in CRN. In this paper, the proposed cognitive radio enabled cache map-and-route (CR-CMR) scheme considers the multi-layered framework using context mapping and decision making approach in SDN. User requests are classified based on mapped and unmapped contexts. CR-CMR control plane integrates the cognitive radio control unit, centralized SDN controller, context mapping unit (CMU), and decision making unit (DMU). The control plane interacts with the virtual plane and performs the cache map-and-route process. Base rule analyzes the context variance and cache variance. The decision-making rules apply step-wise decision approach to map the service requests of unmapped contexts and optimize the cache map-and-route process. The splay tree analysis applied at virtual cache centers improves the resource availability and minimizes the mapping cost. CR-CMR supports scalability and minimizes the cache variance. The virtual cache and data centers verify the route tables of one-hop, two-hop neighbors and establish the forward decision paths with optimal route cost and minimal latency in SDN. [ABSTRACT FROM AUTHOR]

This paper studies energy management in a smart grid-powered cellular network consisting of an independent system operator (ISO) and multiple geographically distributed aggregators. The aggregators have energy storage devices and can purchase energy from the electric grid via the ISO to serve their users. To account for the uncertainty of the renewable energy supply as well as the impacts of multiple aggregators on the electric grid and energy prices, a foresighted strategy combined with the adaptive $\epsilon$ -greedy method is developed for the aggregators to distributively and adaptively minimize the long-term overall cost of the system, based on the ahead-of-time decision making of the storage pre-charging amount. Simulation results validate that the proposed strategy surpasses a recent learning-based storage management design and a myopic design. [ABSTRACT FROM AUTHOR]

Network slicing in fog radio access networks is recognized as a cost-efficient solution to support future diverse use cases. However, with the number of user equipments (UE) fast increasing, the centralized resource allocation architecture for network slicing can put heavy burdens on the global radio resource manager (GRRM), and meanwhile slice customization is not easy to achieve. To overcome the two issues, a hierarchical radio resource allocation architecture is proposed in this paper, where the GRRM is responsible for allocating subchannels to local radio resource managers (LRRMs) in slices, which then allocate the assigned resources to their UEs. Under this architecture, a hierarchical resource allocation problem is formulated, and the problem is further modeled as a Stackelberg game with the GRRM as the leader and LRRMs as followers, considering the hierarchy between the GRRM and LRRMs. Due to the NP-hardness of the followers’ problems, a process based on exhaustive search is first proposed to achieve the Stackelberg equilibrium (SE). Nevertheless, when the network scale is large, achieving SE within limited decision making time is impractical for game players. Facing this challenge, the GRRM and LRRMs are seen as bounded rational players, and low complexity algorithms are developed to help them achieve local optimal solutions that lead to a weak version of SE. Simulation results show that there exists a tradeoff between the performance of slices, and the low complexity algorithms achieve close performance to that of exhaustive search and outperform other baselines significantly. [ABSTRACT FROM AUTHOR]

The employment of social similarity (SS) aware device-to-device (D2D) user equipment (UE)-to-network (UE-NW) relay helps to improve system capacity and UEs’ accessability to the evolved Node B (eNB), where a crucial aspect is the design of the relay helper UE (HUE) selection process. However, few existing studies jointly consider the diverse impacts from various design criteria on the performance of HUE selection. In this paper, we formulate the SS-aware D2D UE-NW relay selection (RS) issue as a multiobjective binary integer linear programming (MOBILP) problem under the constraints of several criteria. Then, we propose a new two-stage D2D UE-NW selection scheme. Specifically, a joint algorithm exploiting the intuitionistic fuzzy analytic hierarchy process and the entropy weight generation method is designed for stage one, which provides both subjective and objective preferences for assisting decision making. In stage two, with the constructed decision weights, the MOBILP can be transformed to a simpler binary integer linear programming problem and thus be solved distributively through an efficient message passing method. Based on realistic social data traces, our simulation results validate the benefits of the proposed two-stage scheme, which ensures high system performances in terms of throughput, fairness and energy efficiency. [ABSTRACT FROM AUTHOR]

The drastic growth of mobile traffic greatly challenges the capacity of mobile infrastructures. Dense deployment of low-power small cells helps alleviate the congestion in the radio access network, yet it also introduces large complexity for network management. Software-defined radio access network has been proposed to tackle the added complexity. However, existing software-defined solutions rely on a fully centralized control plane to make decisions for the whole network, which greatly limits the scalability and responsiveness of the control plane. In this paper, we propose a hierarchical software-defined radio access network architecture. The proposed architecture leverages the hierarchical structure of radio access networks, deploying additional local controllers near the network edge. Utilizing the intrinsic locality in radio access networks, it offloads control tasks from the central controller to local controllers with limited overhead introduced. Under the architecture, a distributed optimization framework is proposed, and a typical optimization problem is studied to illustrate the effectiveness of the proposed architecture and framework. Both analysis and experiments validate that the proposed architecture and framework can improve the network objective during the optimization, meanwhile balancing load and improving scalability and responsiveness. [ABSTRACT FROM AUTHOR]

This paper presents a novel cooperative-driving prediction and planning framework for dynamic environments based on the methods of game theory. The proposed algorithm can be used for highly automated driving on highways or as a sophisticated prediction module for advanced driver-assistance systems with no need for intervehicle communication. The main contribution of this paper is a model-based interaction-aware motion prediction of all vehicles in a scene. In contrast to other state-of-the-art approaches, the system also models the replanning capabilities of all drivers. With that, the driving strategy is able to capture complex interactions between vehicles, thus planning maneuver sequences over longer time horizons. It also enables an accurate prediction of traffic for the next immediate time step. The prediction model is supported by an interpretation of what other drivers intend to do, how they interact with traffic, and the ongoing observation. As part of the prediction loop, the proposed planning strategy incorporates the expected reactions of all traffic participants, offering cooperative and robust driving decisions. By means of experimental results under simulated highway scenarios, the validity of the proposed concept and its real-time capability is demonstrated. [ABSTRACT FROM PUBLISHER]

In this paper, we study an uplink network configuration in a two-tier macro–femto heterogeneous system with hybrid access control. We consider a system where one macro base station (MBS) and a cluster of adjacent femto base stations (FBSs) together serve a number of mobile users. In this system, base stations and users make decisions in various network configuration processes with different optimization objectives. Such decision-making processesz are usually correlated, and an efficient mechanism is needed to coordinate the decision makers. In this paper, we propose a five-stage network configuration mechanism where access control, resource allocation, and power management are sequentially performed at the base stations and users, respectively. We show that this mechanism provides incentive for the FBSs to operate at the hybrid access mode. We model the configuration mechanism as a multistage decision-making process and formulate a multilevel optimization problem. We analyze the problem in a bottom-up manner and propose efficient algorithms to solve the optimization problem in each level sequentially. Simulation results show that the proposed network configuration mechanism achieves a higher system utility than configuration mechanisms with topology-based hybrid access or closed access. [ABSTRACT FROM AUTHOR]

The American automobile industry is facing substantial environmental and energy challenges driving the pursuit of alternative powertrain technologies—“The intervening mechanism by which power is transmitted from an engine to a propeller or axle that it drives,” which nominally includes engine, transmission, drive shafts, differentials, and the final drive. Emerging alternative fuel vehicles are showing their potential to address these challenges. However, diffusion of new technologies has many complications. This paper uses system dynamics modeling to investigate the impacts of individual and organizational parameters. This model embeds two traditional modeling methods for technology adoption, the Bass diffusion model and multinomial logit regression method. Five types of powertrain systems and three major stakeholders are considered. Mathematical relationships among different variables are elaborated. The qualitative impacts of government feebate, manufacturer willingness, and consumer purchasing preferences on economic and environmental issues are addressed using scenario analysis. [ABSTRACT FROM AUTHOR]

For a diverse many-to-many-to-one (MMO) Internet of things (IoT) network, it is essential and significant to ensure the access fairness of things with limited cost of performance, i.e., each thing (denoted by sensor) has an identical probability to upload its information to support the intelligent decision making. Therefore, this paper investigates, how to balance the diversity gain and fairness issue of MMO IoT networks by designing joint thing and relay selection criteria (TRSCs). Based on cumulative distribution functions, a fairness optimal TRSC (FO-TRSC) and a fairness suboptimal TRSC (FSO-TRSC) are proposed, efficiently separating the joint selection process into two independent steps. A low-complexity distributed implementation method is presented, which has reduced the implementation complexity to $O(1)$. A rigorous mathematical analysis for the outage probability is given and closed-form expressions are obtained. It has been shown that FO-TRSC can achieve the strict sensor fairness at the cost of relatively big performance loss. On the contrary, FO-TRSC can obtain the same diversity order as the conventional outage optimal TRSC while guaranteeing the approximate sensor fairness. As a result, the two birds are catch: diversity gain and fairness. Simulation results highlight the effectiveness of our criteria and verify our analytical results. [ABSTRACT FROM AUTHOR]

Accurately predicting and inferring a driver's decision to brake is critical for designing warning systems and avoiding collisions. In this paper, we focus on predicting a driver's intent to brake in car-following scenarios from a perception–decision–action perspective according to his/her driving history. A learning-based inference method, using onboard data from CAN-Bus, radar, and cameras as explanatory variables, is introduced to infer drivers’ braking decisions by combining a Gaussian mixture model (GMM) with a hidden Markov model (HMM). The GMM is used to model stochastic relationships among variables, while the HMM is applied to infer drivers’ braking actions based on the GMM. Real-case driving data from 49 drivers (more than three years’ driving data per driver on average) have been collected from the University of Michigan Safety Pilot Model Deployment database. We compare the GMM-HMM method to a support vector machine (SVM) method and a SVM-Bayesian filtering method. The experimental results are evaluated by employing three performance metrics: accuracy, sensitivity, and specificity. The comparison results show that the GMM–HMM obtains the best performance, with an accuracy of 90%, sensitivity of 84%, and specificity of 97%. Thus, we believe that this method has great potential for real-world active safety systems. [ABSTRACT FROM PUBLISHER]

This paper considers the problem of vehicular roadside unit (RSU) placement so that the sum of capital expenditure (CAPEX) and operating expenditure (OPEX) costs is minimized. The minimum cost solution considers these two cost contributions jointly when making RSU placement decisions. The input to the placement process consists of historical vehicular traffic traces and a set of candidate site locations from which RSU placements are chosen. An integer linear program (ILP) formulation is first given that computes the minimum cost placement based on the input traffic traces and candidate locations. A practical algorithm is then introduced that solves a relaxed version of the ILP and uses a novel rounding procedure to obtain real RSU placements. The algorithm takes into account the energy costs incurred by vehicular requests when the latter are scheduled using a minimum energy online scheduler. Performance results are presented that show that the proposed algorithm performs well compared to the case where placements are done without jointly considering both CAPEX and OPEX cost components. [ABSTRACT FROM AUTHOR]

This paper proposes a methodology for overcoming Mamdani-type inference failures on a fuzzy-logic-based decision-making process applied to collision avoidance in ship navigation. The fuzzy inference failures are observed in three distinct situations: 1) intersected contradictory decision boundaries; 2) an improper transition region between the inference boundaries of nonintersected contradictory decisions; and 3) contradictory decision accumulation under multiple obstacle scenarios. The solutions for overcoming these fuzzy inference failures and their limitations are also discussed in this paper. The proposed solutions consist of insertion of smooth transition regions, determination of the proper size of the smooth transition regions, and use of multilevel decision/action formulations. Furthermore, this paper analyzes a decision-making process for ship navigation, derives input and output fuzzy membership functions (FMFs), formulates an if–then-rule-based fuzzy inference system (FIS), and presents simulation results that support recovery from rule inference failures in several contradictory decision boundary conditions. [ABSTRACT FROM PUBLISHER]

Driving in busy highways and roads is becoming complex and challenging, as more cars are hitting the roads. Safe driving requires attentive drivers, quality perception of the environment, awareness of the situation, and critical decision making to react properly in emergency situations. This paper provides an overview on driver safety monitoring systems. We study various driver sources of inattention while providing a comprehensive taxonomy. Then, different safety systems that tackle driver inattention are reported. Furthermore, we present the new generation of driver monitoring systems within the context of Internet of Cars. Thus, we introduce the concept of integrated safety, where smart cars collect information from the driver, the car, the road, and, most importantly, the surrounding cars to build an efficient environment for the driver. We conclude by highlighting issues and emerging trends envisioned by the research community. [ABSTRACT FROM AUTHOR]

The number of electric vehicles (EVs) is expected to increase significantly in the future to combat air pollution and reduce reliance on fossil fuels. This will impact the power system. However, appropriate charging and discharging of EVs through vehicle-to-grid (V2G) operations could also provide support for the power system and benefits for EV owners. This raises the questions of when and how EV battery storage should be dispatched, taking into account both vehicle users' and power system's requirements and priorities, as well as the constraints of the battery system. This paper proposes a novel decentralized dispatch strategy based on the analytic hierarchy process (AHP), taking into account the relative importance of the different criteria, including cost, battery state of charge (SOC), power system contingency, and load leveling. The proposed AHP-based dispatch strategy was tested on an IEEE Reliability Test System with different EV numbers and capacities to investigate the efficacy of such an approach. The simulation results demonstrate the feasibility and benefits of this dispatch strategy. [ABSTRACT FROM PUBLISHER]

A Bayesian decision rule for erasure insertion is derived in this paper, assuming frequency-hop multiple-access communications in Rician selective fading. The proposed method is based on a channel-identification approach together with decision feedback. Given the propagation environment under study, which can be deemed typical for broad-band mobile systems, the Rice factor, in addition to the channel power delay profile, need to be estimated by the receiver. Significant performance gain, compared to the simple ratio-threshold test (RTT), can be achieved with the proposed method at the expense of greater complexity, mainly due to the channel-identification requirement. Nevertheless, in this paper it is shown that this Bayesian decision rule exhibits high robustness on respect to the joint estimation of the Rice factor and powerdelay profile, thus proving its actual applicability in real receivers employing erasure insertion. [ABSTRACT FROM AUTHOR]

Taking the complex but regular characteristics of bus routines into account, the stochastic dynamic programming (SDP) might be a method with more potential to optimize the energy management of a plug-in hybrid electric bus (PHEB). However, the discrete transmission system and the continuous power system make it a complicated multidimensional optimal problem, particularly for PHEB with automated mechanical transmission (AMT), and the optimal decisions, which are obtained based on historical data, might not always well satisfy the driver's expectation to vehicle maneuverability under various driving conditions. To solve these problems, an adaptive approach based on the SDP is proposed in this paper. Exhaustively, the SDP is propelled into the input of the transmission to only optimize the torque split under the special gearshift logic, which reduces the dimensions of optimization and obtains more applicable optimal sequences. Then, an adaptive factor, which trades off the vehicle fuel economy and drivability in real time by dynamically adjusting the gearshift points and the torque split, is developed for the variation of the complicated bus driving cycles. The simulation results demonstrate that the proposed method could well respond to the variations of the driving conditions (e.g., road grade and vehicle load). Furthermore, the performance of the proposed method is discussed in detail by comparisons with different control strategies. More importantly, the proposed approach has great potential to be applied in practice. [ABSTRACT FROM PUBLISHER]

In this paper, observing the information asymmetry phenomenon among multiple secondary users (SUs) spectrum sharing, we analyze the hierarchical decision-making and the strategic interaction of information-poor and information-rich SUs. A Stackelberg capacity-maximization game is formulated with leaders and followers, and closed-form solutions are mathematically derived for the optimal Stackelberg equilibrium solution. Moreover, the existence and uniqueness of equilibrium solutions are investigated via the quasi-variational inequality method. Finally, the distributed algorithm with partial asymmetric information awareness is designed to reach the solution. Numerical results demonstrate that the proposed algorithm achieves improved individual and system performance with mild condition on the ratio of information-poor leaders and information-rich followers. [ABSTRACT FROM PUBLISHER]

Many future large-scale unattended sensor networks (USNs) are expected to follow a two-tier architecture with resource-poor sensor nodes at the lower tier and fewer resource-rich master nodes at the upper tier. Master nodes collect data from sensor nodes and then answer the queries from the network owner on their behalf. In hostile environments, master and sensor nodes may be compromised by the adversary and return incorrect data in response to data queries. Such application-level attacks are more harmful and difficult to detect than blind denial-of-service attacks on network communications, particularly when the query results are the basis for critical decision making. This paper presents a suite of novel schemes to enable verifiable top- $k$ query processing in USNs, which is the first work of its kind. The proposed schemes are built upon symmetric cryptographic primitives and enable the network owner to detect any incorrect top- $k$ query results. Detailed theoretical and simulation results confirm the high efficacy and efficiency of the proposed schemes. [ABSTRACT FROM AUTHOR]

Multicopy utility-based forwarding algorithms are popular in opportunistic mobile networks. They aim to gain high system throughput while keeping the cost low. However, most of them ignore the fairness issue on the successful delivery rate among users. In this paper, we analyze the fairness evaluation of the success rate distribution, and we propose a new fair packet-forwarding strategy based on packet priority. We formulate the opportunistic packet-forwarding process as a discrete-time Markov chain and deduce a stationary probability distribution vector. Instead of taking the hill-climbing heuristic on utility comparison, we introduce a lower utility tolerance mechanism for the decision-making process of each node, and we theoretically demonstrate that the proposed mechanism may be used to control the success rate of packet delivery by changing a simple parameter. In addition, we adopt a message-duplication restricting mechanism to adjust the number of replications based on packet priority. Our proposed protocol can work as a plug-in for traditional utility-based forwarding algorithms. The performance is compared with several well-known opportunistic routing protocols via both a synthetic model and real human mobility traces. The results show that our protocol improves the fairness in successful delivery rates among users while maintaining almost the same system throughput. Moreover, it reduces the cost of traditional utility-based forwarding algorithms, rendering our proposed scheme not only fair but efficient as well. [ABSTRACT FROM PUBLISHER]

In this paper, we study how multiple users can balance their traffic loads to share common resources in an efficient and distributed manner, without message exchanges. Specifically, we study a deployment scenario where users deploy delay-sensitive applications over a wireless multipath network and aim to minimize their own expected delays. Since the performance of a user's load balancing strategy depends on the strategies that are deployed by other users, it becomes important that a user considers the multiuser coupling when making its own load balancing decisions. We model this multiuser interaction as a load balancing game (LBG) and show that users can converge to a \varepsilon-consistent conjectural equilibrium by building near-accurate beliefs about the remaining capacities on each path. Based on these beliefs, users can make load balancing decisions without explicitly knowing the actions of the other users. In such a conjecture-based LBG, we analytically show that, if a leader is elected to build beliefs about how the users' aggregate transmission strategies affect the remaining resources, then this leader can use this knowledge to shape its traffic such that the multiuser interaction can achieve an efficient allocation across paths. Even if no leader is present in the game, as long as the users follow a set of prescribed rules for building beliefs, they can reach efficient outcomes in a distributed manner. Importantly, the proposed distributed load balancing solution can be also applied to other multiuser communication and networking problems where message exchanges are prohibited (or prohibitively expensive in terms of delay or bandwidth), ranging from multichannel selection in wireless networks to relay assignment in multivehicle networks. [ABSTRACT FROM PUBLISHER]

Wireless technology is a revolutionary advancement providing users with ubiquitous data and telephony access anywhere and anytime with no physical (PHY) connection. The currently deployed wireless networks like Wi-Fi, Worldwide Interoperability for Microwave Access (WiMAX), Universal Mobile Telecommunications Systems (UMTS), and Long Term Evolution (LTE) systems have different characteristics that make them complementary in terms of performance, coverage, and cost. This network variety presents an opportunity to provide better services to end users, given the advances in mobile terminals (MTs). To reach this goal, an appropriate automatic network selection (ANS) mechanism, which is able to always select the best access network, is needed. This consists of constantly monitoring any type of available access network, i.e., automatically selecting and switching to the best one, as the network that maximizes the users quality of experience, taking into account their preferences and the terminal and network conditions. ANS is a multidimensional decision-making problem that can be solved by finding an appropriate complex tradeoff between possibly conflicting criteria. In this paper, we propose an analytical model to capture the preferences of end users. Based on this model, we design an ANS mechanism that takes into account all aspects of the tradeoff between the quality of the connections, the preferences of the end users, and the cost. To highlight the benefits of our approach from the perspectives of both end users and network operators, we have implemented and tested the solution in a multitechnology simulator. Results show that the proposed solution outperforms mainstream approaches. [ABSTRACT FROM PUBLISHER]

As an initial step toward solving the spectrum-shortage problem, the Federal Communications Commission (FCC) has started the so-called opportunistic spectrum access (OSA), which allows unlicensed users to exploit the unused licensed spectrum, but in a manner that limits interference to licensed users. Fortunately, technological advances have enabled cognitive radios, which have recently been recognized as the key enabling technology for realizing OSA. In this paper, we propose a machine-learning-based scheme that will exploit the cognitive radios' capabilities to enable effective OSA, thus improving the efficiency of spectrum utilization. Our proposed learning technique requires no prior knowledge of the environment's characteristics and dynamics, yet it can still achieve high performance by learning from interaction with the environment. [ABSTRACT FROM AUTHOR]