Search

We study the problem of off-policy evaluation (OPE) for episodic Partially Observable Markov Decision Processes (POMDPs) with continuous states. Motivated by the recently proposed proximal causal inference framework, we develop a non-parametric identification result for estimating the policy value via a sequence of so-called V-bridge functions with the help of time-dependent proxy variables. We then develop a fitted-Q-evaluation-type algorithm to estimate V-bridge functions recursively, where a non-parametric instrumental variable (NPIV) problem is solved at each step. By analyzing this challenging sequential NPIV problem, we establish the finite-sample error bounds for estimating the V-bridge functions and accordingly that for evaluating the policy value, in terms of the sample size, length of horizon and so-called (local) measure of ill-posedness at each step. To the best of our knowledge, this is the first finite-sample error bound for OPE in POMDPs under non-parametric models.

To design an efficient interference management and multiple access scheme for visible light communication (VLC) network, this letter leverages the non-orthogonal multiple access (NOMA), which has received significant attention in the $5^{th}$ generation wireless communication. With the residual interference from the successive interference cancellation in NOMA taken into account, we optimize the power allocation for NOMA VLC network to improve the achievable user rate under user quality of service (QoS) constraint. The performance of the proposed approaches is evaluated by the numerical results., Comment: 5 pages, 4 figures. This article has been submitted to IEEE Communication Letters for publication on July 27, 2016