Search

Model predictive control can optimally deal with nonlinear systems under consideration of constraints. The control performance depends on the model accuracy and the prediction horizon. Recent advances propose to use reinforcement learning applied to a parameterized model predictive controller to recover the optimal control performance even if an imperfect model or short prediction horizons are used. However, common reinforcement learning algorithms rely on first order updates, which only have a linear convergence rate and hence need an excessive amount of dynamic data. Higher order updates are typically intractable if the policy is approximated with neural networks due to the large number of parameters. In this work, we use a parameterized model predictive controller as policy, and leverage the small amount of necessary parameters to propose a trust-region constrained Quasi-Newton training algorithm for policy optimization with a superlinear convergence rate. We show that the required second order derivative information can be calculated by the solution of a linear system of equations. A simulation study illustrates that the proposed training algorithm outperforms other algorithms in terms of data efficiency and accuracy., Comment: European Control Conference (ECC) 2024, Code: https://github.com/DeanBrandner/ECC24_TR_improved_QN_PO_for_MPC_in_RL

We investigate the suboptimality resulting from the application of nominal model predictive control (MPC) to a nonlinear discrete time stochastic system. The suboptimality is defined with respect to the corresponding stochastic optimal control problem (OCP) that minimizes the expected cost of the closed loop system. In this context, nominal MPC corresponds to a form of certainty-equivalent control (CEC). We prove that, in a smooth and unconstrained setting, the suboptimality growth is of fourth order with respect to the level of uncertainty, a parameter which we can think of as a standard deviation. This implies that the suboptimality does not grow very quickly as the level of uncertainty is increased, providing further insight into the practical success of nominal MPC. Similarly, the difference between the optimal and suboptimal control inputs is of second order. We illustrate the result on a simple numerical example, which we also use to show how the proven relationship may cease to hold in the presence of state constraints.

Uncertainty quantification is an important task in machine learning - a task in which standardneural networks (NNs) have traditionally not excelled. This can be a limitation for safety-critical applications, where uncertainty-aware methods like Gaussian processes or Bayesian linear regression are often preferred. Bayesian neural networks are an approach to address this limitation. They assume probability distributions for all parameters and yield distributed predictions. However, training and inference are typically intractable and approximations must be employed. A promising approximation is NNs with Bayesian last layer (BLL). They assume distributed weights only in the linear output layer and yield a normally distributed prediction. To approximate the intractable Bayesian neural network, point estimates of the distributed weights in all but the last layer should be obtained by maximizing the marginal likelihood. This has previously been challenging, as the marginal likelihood is expensive to evaluate in this setting. We present a reformulation of the log-marginal likelihood of a NN with BLL which allows for efficient training using backpropagation. Furthermore, we address the challenge of uncertainty quantification for extrapolation points. We provide a metric to quantify the degree of extrapolation and derive a method to improve the uncertainty quantification for these points. Our methods are derived for the multivariate case and demonstrated in a simulation study. In comparison to Bayesian linear regression with fixed features, and a Bayesian neural network trained with variational inference, our proposed method achieves the highest log-predictive density on test data., Comment: This work has been published at IEEE Access with Digital Object Identifier 10.1109/ACCESS.2023.3329685 under a Creative Commons Attribution 4.0 License

With the growing share of renewable energy sources, the uncertainty in power supply is increasing. In addition to the inherent fluctuations in the renewables, this is due to the threat of deliberate malicious attacks, which may become more revalent with a growing number of distributed generation units. Also in other safety-critical technology sectors, control systems are becoming more and more decentralized, causing the targets for attackers and thus the risk of attacks to increase. It is thus essential that distributed controllers are robust toward these uncertainties and able to react quickly to disturbances of any kind. To this end, we present novel methods for model-based identification of attacks and combine them with distributed model predictive control to obtain a resilient framework for adaptively robust control. The methodology is specially designed for distributed setups with limited local information due to privacy and security reasons. To demonstrate the efficiency of the method, we introduce a mathematical model for physically coupled microgrids under the uncertain influence of renewable generation and adversarial attacks, and perform numerical experiments, applying the proposed method for microgrid control., Comment: Submitted for review to Springer Natural Computer Science on November 18th 2022

Based on the principle of onion routing, the Tor network achieves anonymity for its users by relaying user data over a series of intermediate relays. This approach makes congestion control in the network a challenging task. As of today, this results in higher latencies due to considerable backlog as well as unfair data rate allocation. In this paper, we present a concept study of PredicTor, a novel approach to congestion control that tackles clogged overlay networks. Unlike traditional approaches, it is built upon the idea of distributed model predictive control, a recent advancement from the area of control theory. PredicTor is tailored to minimizing latency in the network and achieving max-min fairness. We contribute a thorough evaluation of its behavior in both toy scenarios to assess the optimizer and complex networks to assess its potential. For this, we conduct large-scale simulation studies and compare PredicTor to existing congestion control mechanisms in Tor. We show that PredicTor is highly effective in reducing latency and realizing fair rate allocations. In addition, we strive to bring the ideas of modern control theory to the networking community, enabling the development of improved, future congestion control. We therefore demonstrate benefits and issues alike with this novel research direction.

Tube-enhanced multi-stage nonlinear model predictive control is a robust control scheme that can handle a wide range of uncertainties with reduced conservatism and manageable computational complexity. In this paper, we elaborate on the flexibility of the approach from an application point of view. We discuss the path to making design decisions to implement the novel scheme systematically. We illustrate the critical steps in the design and implementation of the scheme for an industrial example.

We discuss the application of a nonlinear model predictive control (MPC) and a moving horizon estimation (MHE) to achieve an optimal operation of \textit{E. coli} fed-batch cultivations with intermittent bolus feeding. 24 parallel experiments were considered in a high-throughput microbioreactor platform at a 10 mL scale. The robotic island in question can run up to 48 fed-batch processes in parallel with automated liquid handling and online and at-line analytics. The implementation of the model-based monitoring and control framework reveals that there are mainly three challenges that need to be addressed; First, the inputs are given in an instantaneous pulsed form by bolus injections, second, online and at-line measurement frequencies are severely imbalanced, and third, optimization for the distinctive multiple reactors can be either parallelized or integrated. We address these challenges by incorporating the concept of impulsive control systems, formulating multi-rate MHE with identifiability analysis, and suggesting criteria for deciding the reactor configuration. In this study, we present the key elements and background theory of the implementation with \textit{in silico} simulations for bacterial fed-batch cultivation.

Optimal experimental design for parameter precision attempts to maximize the information content in experimental data for a most effective identification of parametric model. With the recent developments in miniaturization and parallelization of cultivation platforms for high-throughput screening of optimal growth conditions massive amounts of informative data can be generated with few experiments. Increasing the quantity of the data means to increase the number of parameters and experimental design variables which might deteriorate the identifiability and hamper the online computation of optimal inputs. To reduce the problem complexity, in this work, we introduce an auxiliary controller at a lower level that tracks the optimal feeding strategy computed by a high-level optimizer in an online fashion. The hierarchical framework is especially interesting for the operation under constraints. The key aspect of this method are discussed together with an in silico study considering parallel glucose limited bacterial fed batch cultivations., Comment: 6 pages, 4 figures, submitted to IFAC Conference

Fog Computing allows applications to address their latency and privacy requirements while coping with bandwidth limitations of Internet service providers (ISPs). Existing research on fog systems has so far mostly taken a very high-level view on the actual fog infrastructure. In this position paper, we identify and discuss the problem of having multiple ISPs in edge-to-edge communication. As a possible solution we propose that edge operators create direct edge-to-edge links in a grassroots fashion and discuss different implementation options. Based on this, we highlight some important open research challenges that result from this., Comment: accepted for publication in ACM M4IoT 2021

Since the earliest conceptualizations by Lee and Markus, and Propoi in the 1960s, Model Predictive Control (MPC) has become a major success story of systems and control with respect to industrial impact and with respect to continued and wide-spread research interest. The field has evolved from conceptually simple linear-quadratic (convex) settings in discrete and continuous time to nonlinear and distributed settings including hybrid, stochastic, and infinite-dimensional systems. Put differently, essentially the entire spectrum of dynamic systems can be considered in the MPC framework with respect to both -- system theoretic analysis and tailored numerics. Moreover, recent developments in machine learning also leverage MPC concepts and learning-based and data-driven MPC have become highly active research areas. However, this evident and continued success renders it increasingly complex to live up to industrial expectations while enabling graduate students for state-of-the-art research in teaching MPC. Hence, this position paper attempts to trigger a discussion on teaching MPC. To lay the basis for a fruitful debate, we subsequently investigate the prospect of covering MPC in undergraduate courses; we comment on teaching textbooks; and we discuss the increasing complexity of research-oriented graduate teaching of~MPC.

This article considers the $\mathcal{H}_\infty$ static output-feedback control for linear time-invariant uncertain systems with polynomial dependence on probabilistic time-invariant parametric uncertainties. By applying polynomial chaos theory, the control synthesis problem is solved using a high-dimensional expanded system which characterizes stochastic state uncertainty propagation. A closed-loop polynomial chaos transformation is proposed to derive the closed-loop expanded system. The approach explicitly accounts for the closed-loop dynamics and preserves the $\mathcal{L}_2$-induced gain, which results in smaller transformation errors compared to existing polynomial chaos transformations. The effect of using finite-degree polynomial chaos expansions is first captured by a norm-bounded linear differential inclusion, and then addressed by formulating a robust polynomial chaos based control synthesis problem. This proposed approach avoids the use of high-degree polynomial chaos expansions to alleviate the destabilizing effect of truncation errors, which significantly reduces computational complexity. In addition, some analysis is given for the condition under which the robustly stabilized expanded system implies the robust stability of the original system. A numerical example illustrates the effectiveness of the proposed approach., Comment: 11 pages, 3 figures, 1 table; submitted to IEEE Transactions on Automatic Control

The trade-off between optimality and complexity has been one of the most important challenges in the field of robust Model Predictive Control (MPC). To address the challenge, we propose a flexible robust MPC scheme by synergizing the multi-stage and tube-based MPC approaches. The key idea is to exploit the non-conservatism of the multi-stage MPC and the simplicity of the tube-based MPC. The proposed scheme provides two options for the user to determine the trade-off depending on the application: the choice of the robust horizon and the classification of the uncertainties. Beyond the robust horizon, the branching of the scenario-tree employed in multi-stage MPC is avoided with the help of tubes. The growth of the problem size with respect to the number of uncertainties is reduced by handling \emph{small} uncertainties via an invariant tube that can be computed offline. This results in linear growth of the problem size beyond the robust horizon and no growth of the problem size concerning small magnitude uncertainties. The proposed approach helps to achieve a desired trade-off between optimality and complexity compared to existing robust MPC approaches. We show that the proposed approach is robustly asymptotically stable. Its advantages are demonstrated for a CSTR example., Comment: 37 pages, 10 figures, 2 tables. Journal under review

Data-enabled predictive control (DeePC) is a recently proposed approach that combines system identification, estimation and control in a single optimization problem, for which only recorded input/output data of the examined system is required. The same premise holds for the subspace predictive control (SPC) method in which a multi-step prediction model is identified from the same data as required for DeePC. This model is then used to formulate a similar optimal control problem. In this work we investigate the relationship between DeePC and SPC. Our primary contribution is to show that SPC is equivalent to DeePC in the deterministic case. We also show the equivalence of both methods in a special case for the non-deterministic formulation. We investigate the advantages and shortcomings of DeePC as opposed to SPC with and without measurement noise and illustrate them with a simulation example., Comment: Accepted at European Control Conference (ECC) 2021. 8 pages, 3 figures, 1 table

Many autonomous control systems are frequently exposed to attacks, so methods for attack identification are crucial for a safe operation. To preserve the privacy of the subsystems and achieve scalability in large-scale systems, identification algorithms should not require global model knowledge. We analyze a previously presented method for hierarchical attack identification, that is embedded in a distributed control setup for systems of systems with coupled nonlinear dynamics. It is based on the exchange of local sensitivity information and ideas from sparse signal recovery. In this paper, we prove sufficient conditions under which the method is guaranteed to identify all components affected by some unknown attack. Even though a general class of nonlinear dynamic systems is considered, our rigorous theoretical guarantees are applicable to practically relevant examples, which is underlined by numerical experiments with the IEEE~30 bus power system., Comment: in Proc. of the 2020 Conference on Decision and Control

Information and communication technologies have accompanied our everyday life for years. A steadily increasing number of computers, cameras, mobile devices, etc. generate more and more data, but at the same time we realize that the data can only partially be analyzed with classical approaches. The research and development of methods based on artificial intelligence (AI) made enormous progress in the area of interpretability of data in recent years. With growing experience, both, the potential and limitations of these new technologies are increasingly better understood. Typically, AI approaches start with the data from which information and directions for action are derived. However, the circumstances under which such data are collected and how they change over time are rarely considered. A closer look at the sensors and their physical properties within AI approaches will lead to more robust and widely applicable algorithms. This holistic approach which considers entire signal chains from the origin to a data product, "Sensor AI", is a highly relevant topic with great potential. It will play a decisive role in autonomous driving as well as in areas of automated production, predictive maintenance or space research. The goal of this white paper is to establish "Sensor AI" as a dedicated research topic. We want to exchange knowledge on the current state-of-the-art on Sensor AI, to identify synergies among research groups and thus boost the collaboration in this key technology for science and industry., Comment: 4 pages. 1st Workshop on Sensor Artificial Intelligence, Apr. 2020, Berlin, Germany

In the Tor network, anonymity is achieved through a multi-layered architecture, which comes at the cost of a complex network. Scheduling data in this network is a challenging task and the current approach shows to be incapable of avoiding network congestion and allocating fair data rates. We propose PredicTor, a distributed model predictive control approach, to tackle these challenges. PredicTor is designed to schedule incoming and outgoing data rates on individual nodes of the Tor architecture, leading to a scalable approach. We successfully avoid congestion through exchanging information of predicted behavior with adjacent nodes. Furthermore, we formulate PredicTor with a focus on fair allocation of resources, for which we present and proof a novel optimization-based fairness approach. Our proposed controller is evaluated with the popular network simulator ns-3, where we compare it with the current Tor scheduler as well as with another recently proposed enhancement. PredicTor shows significant improvements over the previous approaches, especially with respect to latency., Comment: 8 pages, 7 figures, submitted to the 4th IEEE Conference on Control Technology and Applications (CCTA 2020)

Neural networks can be used as approximations of several complex control schemes such as model predictive control. We show in this paper which properties deep neural networks with rectifier linear units as activation functions need to satisfy to guarantee constraint satisfaction and asymptotic stability of the closed-loop system. To do so, we introduce a parametric description of the neural network controller and use a mixed-integer linear programming formulation to perform output range analysis of neural networks. We also propose a novel method to modify a neural network controller such that it performs optimally in the LQR sense in a region surrounding the equilibrium. The proposed method enables the analysis and design of neural network controllers with formal safety guarantees as we illustrate with simulation results., Comment: 8 pages, 4 figures

In this paper, we consider a stochastic Model Predictive Control able to account for effects of additive stochastic disturbance with unbounded support, and requiring no restrictive assumption on either independence nor Gaussianity. We revisit the rather classical approach based on penalty functions, with the aim of designing a control scheme that meets some given probabilistic specifications. The main difference with previous approaches is that we do not recur to the notion of probabilistic recursive feasibility, and hence we do not consider separately the unfeasible case. In particular, two probabilistic design problems are envisioned. The first randomization problem aims to design \textit{offline} the constraint set tightening, following an approach inherited from tube-based MPC. For the second probabilistic scheme, a specific probabilistic validation approach is exploited for tuning the penalty parameter, to be selected \textit{offline} among a finite-family of possible values. The simple algorithm here proposed allows designing a \textit{single} controller, always guaranteeing feasibility of the online optimization problem. The proposed method is shown to be more computationally tractable than previous schemes. This is due to the fact that the sample complexity for both probabilistic design problems depends on the prediction horizon in a logarithmic way, unlike scenario-based approaches which exhibit linear dependence. The efficacy of the proposed approach is demonstrated with a numerical example., Comment: Submitted as Contributed Paper to IFAC2020 Congress

Inertial measurement units are commonly used in a growing number of application fields to track or capture motions of kinematic chains, such as human limbs, exoskeletons or robotic actuators. A major challenge is the presence of magnetic disturbances that result in unreliable magnetometer readings. Recent research revealed that this problem can be overcome by exploitation of kinematic constraints. While typically each segment of the kinematic chain is equipped with an IMU, a novel approach called sparse inertial motion tracking aims at infering the complete motion states from measurements of a reduced set of sensors. In the present contribution, we combine the magnetometer-free and the sparse approach for real-time motion tracking of double-hinge joint systems with non-parallel joint axes. Analyzing the observability of the system, we find a condition which assures that the relative orientations between all segments are uniquely determined by a kinematic constraint, which contains only the gyroscope readings. Furthermore, we propose a moving-horizon estimator and validate it in a simulation study of three movements with different degrees of excitation. The results of this study confirm all theoretical conjectures and demonstrate that magnetometer-free sparse inertial real-time motion tracking is feasible under precise and simple excitation conditions.

Solving nonlinear model predictive control problems in real time is still an important challenge despite of recent advances in computing hardware, optimization algorithms and tailored implementations. This challenge is even greater when uncertainty is present due to disturbances, unknown parameters or measurement and estimation errors. To enable the application of advanced control schemes to fast systems and on low-cost embedded hardware, we propose to approximate a robust nonlinear model controller using deep learning and to verify its quality using probabilistic validation techniques. We propose a probabilistic validation technique based on finite families, combined with the idea of generalized maximum and constraint backoff to enable statistically valid conclusions related to general performance indicators. The potential of the proposed approach is demonstrated with simulation results of an uncertain nonlinear system., Comment: 23 pages, 4 figures

Resonant power converters offer improved levels of efficiency and power density. In order to implement such systems, advanced control techniques are required to take the most of the power converter. In this context, model predictive control arises as a powerful tool that is able to consider nonlinearities and constraints, but it requires the solution of complex optimization problems or strong simplifying assumptions that hinder its application in real situations. Motivated by recent theoretical advances in the field of deep learning, this paper proposes to learn, offline, the optimal control policy defined by a complex model predictive formulation using deep neural networks so that the online use of the learned controller requires only the evaluation of a neural network. The obtained learned controller can be executed very rapidly on embedded hardware. We show the potential of the presented approach on a Hardware-in-the-Loop setup of an FPGA-controlled resonant power converter., Comment: 12 pages, 13 figures

We show that artificial neural networks with rectifier units as activation functions can exactly represent the piecewise affine function that results from the formulation of model predictive control of linear time-invariant systems. The choice of deep neural networks is particularly interesting as they can represent exponentially many more affine regions compared to networks with only one hidden layer. We provide theoretical bounds on the minimum number of hidden layers and neurons per layer that a neural network should have to exactly represent a given model predictive control law. The proposed approach has a strong potential as an approximation method of predictive control laws, leading to better approximation quality and significantly smaller memory requirements than previous approaches, as we illustrate via simulation examples. We also suggest different alternatives to correct or quantify the approximation error. Since the online evaluation of neural networks is extremely simple, the approximated controllers can be deployed on low-power embedded devices with small storage capacity, enabling the implementation of advanced decision-making strategies for complex cyber-physical systems with limited computing capabilities., Comment: 13 pages, 7 figures

Constraint based methods, such as the Flux Balance Analysis, are widely used to model cellular growth processes without relying on extensive information on the regulatory features. The regulation is instead substituted by an optimization problem usually aiming at maximal biomass accumulation. A recent extension to these methods called the dynamic enzyme-cost Flux Balance Analysis (deFBA) is a fully dynamic modeling method allowing for the prediction of necessary enzyme levels under changing environmental conditions. However, this method was designed for deterministic settings in which all dynamics, parameters, etc. are exactly known. In this work, we present a theoretical framework extending the deFBA to handle uncertainties and provide a robust solution. We use the ideas from multi-stage nonlinear Model Predictive Control (MPC) and its feature to represent the evolution of uncertainties by an exponentially growing scenario tree. While this representation is able to construct a deterministic optimization problem in the presence of uncertainties, the computational cost also increases exponentially. We counter this by using a receding prediction horizon and reshape the standard deFBA to the short-time deFBA (sdeFBA). This leads us, along with further simplification of the scenario tree, to the robust deFBA (rdeFBA). This framework is capable of handling the uncertainties in the model itself as well as uncertainties experienced by the modeled system. We applied these algorithms to two case-studies: a minimal enzymatic nutrient uptake network, and the abstraction of the core metabolic process in bacteria., Comment: 18 pages, 5 figures, 3 tables