Computational Method and a Numerical Algorithm for Finding the Optimal Control Policy for a Partially Observable System.

In this paper a computational method and numerical algorithm is developed for finding the optimal control policy for a partially observable system subject to random failure. The condition of the system is modeled by an unobservable continuous-time homogeneous Markov chain. Multivariate data stochastically related to the system state is collected at equidistant sampling times. The system is controlled by a multivariate Bayesian control chart, i.e. full inspection followed by maintenance is performed when the posterior probability that the system is in the warning state exceeds a control limit. A statistical constraint is considered which bounds the probability of a true alarm given by the control chart. The objective is to minimize the long-run expected average cost per unit time by determining the optimal values of the control limit and sampling interval subject to the statistical constraint. The stochastic evolution of the posterior probability process is analyzed and computational algorithm is developed in the semi-Markov decision process framework. Numerical examples are provided which illustrate the effect of the statistical constraint and the performance of the Bayesian control chart is compared with the traditional multivariate χ2 control chart. [ABSTRACT FROM AUTHOR]