Joint energy allocation and multiuser scheduling in SWIPT systems with energy harvesting

The utilisation and transfer of renewable energy and grid energy in the downlink of multiuser communication systems is studied. In the considered multiuser system, the base station (BS) is powered by both harvested energy and grid. When the BS transmits data to one user terminal, other terminals can replenish energy opportunistically from received radio-frequency signals, which is called simultaneous wireless information and power transfer (SWIPT). The objective is to maximise the average throughput by multiuser scheduling and energy allocation utilising imperfect causal channel state information while satisfying the requirement for harvested energy and the average power constraint of the grid. With channel dynamics and energy arrival modelled as Markov processes, the authors characterise the problem as a Markov decision process (MDP). The standard reinforcement learning framework is considered as an effective solution to MDP. If the transition probability of MDP is known, the policy iteration (PI) algorithm is used to solve the problem; otherwise, the R-learning algorithm is adopted. Simulation results show that the proposed algorithm can improve the average throughput of the system and increase the energy harvested by idle user terminals compared with existing works. Also, R-learning can achieve performance close to the PI algorithm under the condition that the channel transition probability is unknown.