A POMDP Approach to Channel Sensing and Data Transmission for Opportunistic Spectrum Access

This paper considers opportunistic spectrum access (OSA) at a cognitive network. Recognizing hardware and energy constraints, we assume that a user may not be able to perform full spectrum sensing. The OSA process is usually formulated as a partially observable Markov decision process (POMDP) model with hybrid actions. In the past work, we improved this model by dividing one state transition procedure into two neighboring state transition procedures with the single action. In this study, we propose a value iteration algorithm based on the state estimation to solve the exponential complexity problem when the channels of the spectrum are relevant with each other. The algorithm uses the Markovian state estimate to replace the partially observable state, and iterates the heuristic value function of the state estimation to optimize the actions. The experiments show that the existing algorithms have better performance with the improved model, and the proposed algorithm can be used to solve the problem with relevant channels.