Search

The present article enables reinforcement learning (RL)‐based controllers for process control applications. Existing instances of RL‐based solutions have significant challenges for online implementation since the training process of an RL agent (controller) presently requires practically impossible number of online interactions between the agent and the environment (process). To address this challenge, we propose an implementable model‐free RL method developed by leveraging industrially implemented model predictive control (MPC) calculations (often designed using a simple linear model identified via step tests). In the first step, MPC calculations are used to pretrain an RL agent that can mimic the MPC performance. Specifically, the MPC calculations are used to pretrain the actor, and the objective function is used to pretrain the critic(s). The pretrained RL agent is then employed within a model‐free RL framework to control the process in a way that initially imitates MPC behavior (thus not compromising process performance and safety), but also continuously learns and improve its performance over the nominal linear MPC. The effectiveness of the proposed approach is illustrated through simulations on a chemical reactor example. [ABSTRACT FROM AUTHOR]

This work addresses the problem of designing an offset-free implementable reinforcement learning (RL) controller for nonlinear processes. RL-based controllers can update the control policy using observed data obtained online based on the controller-process interactions. This allows to alleviate the regular model maintenance step that is essential in advanced control techniques such as model predictive control (MPC). However, random explorations are required for an RL agent to find optimal state–action regions so as to finally reach the optimal policy. This is not implementable in practical situations due to safety concerns and economic objectives. To address this issue, a pre-training strategy is proposed to provide a secure platform for online implementations of the RL controllers. To this end, an offset-free MPC (representative industrial MPC) optimization problem is leveraged to train the RL agent offline. Having obtained similar performance to the offset-free MPC, the RL agent is utilized for online control to interact with the actual process. The efficacy of the proposed approach to handle nonlinearity and changes in plant operating conditions (due to unmeasured disturbances) are demonstrated through simulations on a chemical reactor example for a pH neutralization process. The results show that the proposed RL controller can significantly improve the oscillatory closed-loop responses, obtained by running the offset-free MPC due to the plant-model mismatch and unmeasured disturbances. • This work proposes a practical reinforcement (RL) learning control strategy. • An RL agent is pre-trained using offset-free model predictive control (MPC) information. • The RL controller can initially replicate the offset-free MPC performance. • The pre-trained RL controller outperforms the offset-free MPC after online training. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of determining achievable set‐points for artificial neural network (ANN)‐based model predictive control (MPC) designs. In particular, this work considers a case where a first‐principles model may not be readily available for a nonlinear process, while sufficient closed‐loop data containing possibly correlated outputs is available, such that an ANN‐based model that captures the nonlinear dynamics reasonably well can be identified. The paper addresses implementation aspects with such an ANN‐based MPC design—specifically that of ensuring that achievable set‐points are prescribed to the MPC. The key idea is to perform principal component analysis (PCA) on the training data in order to recognize existing collinearity and determine the upper confidence limit of squared prediction error (SPE) statistic. An optimization problem subject to the SPE constraint is then defined to calculate the achievable set‐points, that can in turn be provided to an MPC design. The efficacy of the proposed approach is illustrated via implementations on a chemical reactor example. The results reveal the superior tracking performance of MPC using the achievable set‐points over the case where arbitrarily prescribed set‐points are used in the MPC implementations. [ABSTRACT FROM AUTHOR]

In this work, we address the problem of simultaneous fault diagnosis in nonlinear uncertain networked systems utilizing a distributed fault detection and isolation (FDI) strategy. The key idea is to design a bank of local FDI (LFDI) schemes that communicate with each other for improved FDI. The proposed distributed FDI scheme is shown to be able to handle local faults as well as those that affect more than one subsystem. This is achieved via appropriate adaptation of the LFDI filters based on information exchange with other subsystems and using the proposed notion of detectability index. The detectability index and isolability conditions are rigorously derived for the distributed FDI scheme. Effectiveness of the proposed methodology is shown via application to a reactor-separator process subject to uncertainty and measurement noise. [ABSTRACT FROM AUTHOR]

This work addresses the problem of simultaneous actuator and sensor fault detection and isolation (FDI) for control affine nonlinear uncertain systems in the absence of measurement noise. The FDI is achieved by using a bank of filters, which utilize a subset of the measurements along with prescribed values of the control actuators to estimate states and compute expected process behavior. Residuals are next defined as the difference between the observed and expected behavior. Detectability conditions are developed, which, upon satisfaction, ensure that each residual remains sensitive to a subset of fault scenarios in the presence of uncertainty. To this end, first the ability of observers in providing bounded estimation error for a generalized class of nonlinear uncertain systems is rigorously established. These bounds allow determining thresholds that account for the impact of uncertainty on each residual. Finally, the ability of the proposed framework to achieve FDI by ensuring a unique residual breaching pattern for each fault scenario is established. The efficacy of the FDI framework subject to uncertainty and measurement noise is illustrated using a chemical reactor example. [ABSTRACT FROM AUTHOR]

In this paper, we present an algorithm to address the problem of guaranteed stabilization of nonlinear systems from the largest set of initial conditions from where stabilization is possible, the so-called null controllable region (NCR), while also handling state constraints and partial unboundedness of the NCR. The controller uses the geometry of the NCR to inform a constrained control Lyapunov function which in turn provides feasibility and stability guarantees. The NCR is produced from a novel iterative growth algorithm that exploits a certain invariance property. The new controller implementation is illustrated using a chemical reactor system and compared to a conventional nonlinear model predictive control (MPC) implementation. [ABSTRACT FROM AUTHOR]

Thiswork considers the problem of handling the failure of purificationequipment in a hydrogen production process. In this process, naturalgas and superheated steam are fed to a heated chemical reactor termeda reformer to produce hydrogen. The effluent gas is further processedand finally purified in a pressure swing adsorber (PSA). The off-gasout of the PSA is used to provide heat to the reformer. The failureof the PSA results in the loss of the off-gas, precluding the possibilityof the continuation of nominal operation. If not properly handled,this fault can lead to a shutdown of the entire plant. To achievestable operation while meeting operating requirements, a model predictivecontrol (MPC) based safe-parking framework is designed for the handlingof the fault. The key idea is to drive the process to a feasible operatingpoint that enables stable operation in the faulty mode. MPC is usedto handle the multivariable nature of the process and operating constraints.It also guides the process to a different operating region while meetingoperating requirements. The effectiveness of the safe-parking designis demonstrated through simulations using a first-principles modelof the hydrogen production unit. [ABSTRACT FROM AUTHOR]

Abstract: This work considers the problem of sensor fault isolation and fault-tolerant control for nonlinear systems subject to input constraints. The key idea is to design fault detection residuals and fault isolation logic by exploiting model-based sensor redundancy through a state observer. To this end, a high-gain observer is first presented, for which the convergence property is rigorously established, forming the basis of the residual design. A bank of residuals are then designed using a bank of observers, with each driven by a subset of measured outputs. A fault is isolated by checking which residuals breach their thresholds according to a logic rule. After the fault is isolated, the state estimate generated using measurements from the healthy sensors is used in closed-loop to maintain nominal operation. The implementation of the fault isolation and handling framework subject to uncertainty and measurement noise is illustrated using a chemical reactor example. [Copyright &y& Elsevier]

Abstract: This work considers the problem of control of nonlinear process systems subject to input constraints and faults in the control actuators. Faults are considered that preclude the possibility of continued operating at the nominal equilibrium point and a framework (which we call the safe-parking framework) is developed to enable efficient resumption of nominal operation upon fault-recovery. To this end, first Lyapunov-based model predictive controllers, that allow for an explicit characterization of the stability region subject to constraints on the manipulated input, are designed. The stability region characterization is utilized in selecting ‘safe-park’ points from the safe-park candidates (equilibrium points subject to failed actuators). Specifically, a candidate parking point is termed a safe-park point if (1) the process state at the time of failure resides in the stability region of the safe-park candidate (subject to depleted control action), and (2) the safe-park candidate resides within the stability region of the nominal control configuration. Performance considerations, such as ease of transition from and to the safe-park point and cost of running the process at the safe-park point, are then quantified and utilized in choosing the optimal safe-park point. The proposed framework is illustrated using a chemical reactor example and robustness with respect to parametric uncertainty and disturbances is demonstrated on a styrene polymerization process. [Copyright &y& Elsevier]

The article presents a paper which proposes optimization-based method for the derivation of tuning guidelines for proportional-integral-derivative (PID) controllers that take nonlinear process behavior explicitly into account. A two-level PID tuning method is described. Application to a chemical reactor example is illustrated.

• A framework is presented to handle the problem of input data co-linearity in developing recurrent neural network based dynamic models for model predictive control (MPC). • Two specific solutions are developed- one that utilizes a principle component analysis as a pre-processor and another one that imposes specific constraints in the MPC implementation to handle model validity. • A re-identification based technique is developed that allows new training data to be generated that enables a larger range of model validity and improved closed-loop performance. This work addresses one aspect of the overparameterization problem in using artificial/recurrent neural networks (ANN/RNN) based dynamic models for model predictive control (MPC) implementations. The manuscript presents an approach to handle situations where the training data may not be sufficiently rich, and in particular, for handling historical data with correlated inputs. Two approaches are proposed. The key idea in the first method is to perform principal component analysis (PCA) on input space and then utilize the scores to build a PCA-RNN model. Next, a PCA-RNN-based MPC is designed to compute the optimal values of scores and subsequently determined the manipulated inputs. An alternative solution is proposed in the second approach by proposing a new constraint on squared prediction error (SPE) statistic in the RNN-based MPC to make prescribed inputs follow the PCA model constructed for training input data. Finally, an approach is presented that allows to break the correlation in the MPC implementation while maintaining model validity. This is done by first generating richer closed-loop data by implementing the SPE based MPC with slightly relaxed constraints (thus compromising only slightly on the closed loop performance). Then the new data is utilized to re-identify the model, and for use in the MPC. The efficacy of the proposed approaches to handle the problem of set-point tracking is evaluated using a chemical reactor example. The results are compared with a nominal MPC design, and the superior performance under the proposed formulations demonstrated. [ABSTRACT FROM AUTHOR]

Abstract: This work considers the problem of isolating actuator and sensor faults in nonlinear process systems. The key idea of the proposed method is to exploit the analytical redundancy in the system through state observer design. To this end, we consider subsets of faults, and design state observers that use information of inputs and outputs only subject to faults in each subset. We then design residuals using the process model and state estimates such that each residual is only sensitive to the corresponding subset of faults. The occurrence of faults in a subset is detected if the corresponding residual breaches its threshold. With the ability of detecting the occurrence of faults in a subset, faults can be isolated using a bank of residuals and a logic rule. The proposed method enables differentiation between and isolation of actuator and sensor faults while explicitly accounting for system nonlinearity. The effectiveness of the fault isolation design subject to plant-model mismatch and measurement noise is illustrated using a chemical reactor example. [Copyright &y& Elsevier]

This work considers the problem of control and fault-handling in living nitroxide-mediated radical polymerization (NMRP) of styrene subject to input constraints. First, a Lyapunov based model predictive controller is designed that allows for an explicit computation of the stability region. Next, a fault in one of the monomer feed streams is considered due to which operation of the reactor at the nominal operating point is no longer possible. It is shown that trying to continue nominal operation using the remaining functioning actuator drives the process to a point resulting in significantly off-spec product. A safe-parking framework is utilized to choose an operating point during fault rectification so that the styrene produced during fault rectification complies with the desired product specifications, and smooth resumption of nominal operation upon fault rectification is achieved. [ABSTRACT FROM AUTHOR]