Drift counteraction optimal control for deterministic systems and enhancing convergence of value iteration

The paper treats a class of optimal control problems for deterministic nonlinear discrete-time systems with the objective of maximizing the time or total yield until prescribed constraints are violated. Such problems are referred to as drift counteraction optimal control (DCOC) problems as the corresponding control policy may be viewed as optimally counteracting drift imposed by disturbances or system dynamics. We derive conditions for the existence of an optimal solution. The optimal control policy is characterized by the value function and a new algorithm based on proportional feedback is presented that obtains the value function faster than conventional dynamic programming algorithms. In addition, an approximate dynamic programming (ADP) approach using Gaussian process regression is formulated based on the new algorithm. Two numerical examples are reported, a time maximization problem for a van der Pol oscillator and a satellite life extension problem.