Search

Cross-layer optimization solutions have been proposed in recent years to improve the performance of wireless users that operate in a time-varying, error-prone network environment. However, these solutions often rely on centralized cross-layer optimization solutions that violate the layered network architecture of the protocol stack by requiring layers to provide access to their internal protocol parameters to other layers. This paper presents a new systematic framework for cross-layer optimization, which allows each layer to make autonomous decisions to maximize the wireless user's utility by optimally determining what information should be exchanged among layers. Hence, this cross-layer framework preserves the current layered network architecture. Since the user interacts with the wireless environment at various layers of the protocol stack, the cross-layer optimization problem is solved in a layered fashion such that each layer adapts its own protocol parameters and exchanges information (messages) with other layers that cooperatively maximize the performance of the wireless user. Based on the proposed layered framework, we also design a message-exchange mechanism that determines the optimal cross-layer transmission strategies, given the user's experienced environment dynamics. Index Terms--Autonomous decision making, cross-layer optimization, environmental dynamics, information exchange, layered dynamic programming (DP) operator.

In this paper, we model the various users in a wireless network (e.g., cognitive radio network) as a collection of selfish autonomous agents that strategically interact to acquire dynamically available spectrum opportunities. Our main focus is on developing solutions for wireless users to successfully compete with each other for the limited and time-varying spectrum opportunities, given experienced dynamics in the wireless network. To analyze the interactions among users given the environment disturbance, we propose a stochastic game framework for modeling how the competition among users for spectrum opportunities evolves over time. At each stage of the stochastic game, a central spectrum moderator (CSM) auctions the available resources, and the users strategically bid for the required resources. The joint bid actions affect the resource allocation and, hence, the rewards and future strategies of all users. Based on the observed resource allocations and corresponding rewards, we propose a best-response learning algorithm that can be deployed by wireless users to improve their bidding policy at each stage. The simulation results show that by deploying the proposed best-response learning algorithm, the wireless users can significantly improve their own bidding strategies and, hence, their performance in terms of both the application quality and the incurred cost for the used resources. Index Terms--Delay-sensitive transmission, interactive learning, multiuser resource management, reinforcement learning, stochastic games, wireless networks.

In this paper, we consider the problem of real-time transmission scheduling over time-varying channels. We first formulate this transmission scheduling problem as a Markov decision process and systematically unravel the structural properties (e.g., concavity in the state-value function and monotonicity in the optimal scheduling policy) exhibited by the optimal solutions. We then propose an online learning algorithm that preserves these structural properties and achieves \varepsilon-optimal solutions for an arbitrarily small \varepsilon. The advantages of the proposed online method are given as follows: 1) It does not require a priori knowledge of the traffic arrival and channel statistics, and 2) it adaptively approximates the state-value functions using piecewise linear functions and has low storage and computation complexity. We also extend the proposed low-complexity online learning solution to enable prioritized data transmission. The simulation results demonstrate that the proposed method achieves significantly better utility (or delay)–energy tradeoffs compared to existing state-of-the-art online optimization methods. [ABSTRACT FROM PUBLISHER]