Approximate solution of stochastic infinite horizon optimal control problems for constrained linear uncertain systems

We propose a Model Predictive Control (MPC) with a single-step prediction horizon to solve infinite horizon optimal control problems with the expected sum of convex stage costs for constrained linear uncertain systems. The proposed method relies on two techniques. First, we estimate the expected values of the convex costs using a computationally tractable approximation, achieved by sampling across the space of disturbances. Second, we implement a data-driven approach to approximate the optimal value function and its corresponding domain, through systematic exploration of the system's state space. These estimates are subsequently used as the terminal cost and terminal set within the proposed MPC. We prove recursive feasibility, robust constraint satisfaction, and convergence in probability to the target set. Furthermore, we prove that the estimated value function converges to the optimal value function in a local region. The effectiveness of the proposed MPC is illustrated with detailed numerical simulations and comparisons with a value iteration method and a Learning MPC that minimizes a certainty equivalent cost.
Comment: Submitted to the IEEE Transactions on Automatic Control