Search

Informed learning is an emerging field in machine learning that aims to compensate for insufficient data with prior knowledge. Shape knowledge covers many types of prior knowledge concerning the relationship of a function's output with respect to input variables, for example, monotonicity, convexity, etc. This shape knowledge -- when formalized into algebraic inequalities (shape constraints) -- can then be incorporated into the training of regression models via a constraint problem formulation. The defined shape-constrained regression problem is, mathematically speaking, a semi-infinite program (SIP). Although off-the-shelf algorithms can be used at this point to solve the SIP, we recommend an adaptive feasible-point algorithm that guarantees optimality up to arbitrary precision and strict fulfillment of the shape constraints. We apply this semi-infinite approach for shape-constrained regression (SIASCOR) to three application examples from manufacturing and one artificial example. One application example has not been considered in a shape-constrained regression setting before, so we used a methodology (ISI) to capture the shape knowledge and define corresponding shape constraints. Finally, we compare the SIASCOR method with a purely data-driven automated machine learning method (AutoML) and another approach for shape-constrained regression (SIAMOR) that uses a different solution algorithm.

We develop adaptive discretization algorithms for locally optimal experimental design of nonlinear prediction models. With these algorithms, we refine and improve a pertinent state-of-the-art algorithm in various respects. We establish novel termination, convergence, and convergence rate results for the proposed algorithms. In particular, we prove a sublinear convergence rate result under very general assumptions on the design criterion and, most notably, a linear convergence result under the additional assumption that the design criterion is strongly convex and the design space is finite. Additionally, we prove the finite termination at approximately optimal designs, including upper bounds on the number of iterations until termination. And finally, we illustrate the practical use of the proposed algorithms by means of two application examples from chemical engineering: one with a stationary model and one with a dynamic model., Comment: 42 pages

We propose a general methodology of sequential locally optimal design of experiments for explicit or implicit nonlinear models, as they abound in chemical engineering and, in particular, in vapor-liquid equilibrium modeling. As a sequential design method, our method iteratively alternates between performing experiments, updating parameter estimates, and computing new experiments. Specifically, our sequential design method computes a whole batch of new experiments in each iteration and this batch of new experiments is designed in a two-stage locally optimal manner. In essence, this means that in every iteration the combined information content of the newly proposed experiments and of the already performed experiments is maximized. In order to solve these two-stage locally optimal design problems, a recent and efficient adaptive discretization algorithm is used. We demonstrate the benefits of the proposed methodology on the example of of the parameter estimation for the non-random two-liquid model for narrow azeotropic vapor-liquid equilibria. As it turns out, our sequential optimal design method requires substantially fewer experiments than traditional factorial design to achieve the same model precision and prediction quality. Consequently, our method can contribute to a substantially reduced experimental effort in vapor-liquid equilibrium modeling and beyond., Comment: 30 pages, 8 figures

We develop two adaptive discretization algorithms for convex semi-infinite optimization, which terminate after finitely many iterations at approximate solutions of arbitrary precision. In particular, they terminate at a feasible point of the considered optimization problem. Compared to the existing finitely feasible algorithms for general semi-infinite optimization problems, our algorithms work with considerably smaller discretizations and are thus computationally favorable. Also, our algorithms terminate at approximate solutions of arbitrary precision, while for general semi-infinite optimization problems the best possible approximate-solution precision can be arbitrarily bad. All occurring finite optimization subproblems in our algorithms have to be solved only approximately, and continuity is the only regularity assumption on our objective and constraint functions. Applications to parametric and non-parametric regression problems under shape constraints are discussed., Comment: 24 pages

Optimization in engineering requires appropriate models. In this article, a regression method for enhancing the predictive power of a model by exploiting expert knowledge in the form of shape constraints, or more specifically, monotonicity constraints, is presented. Incorporating such information is particularly useful when the available data sets are small or do not cover the entire input space, as is often the case in manufacturing applications. The regression subject to the considered monotonicity constraints is set up as a semi-infinite optimization problem, and an adaptive solution algorithm is proposed. The method is applicable in multiple dimensions and can be extended to more general shape constraints. It is tested and validated on two real-world manufacturing processes, namely laser glass bending and press hardening of sheet metal. It is found that the resulting models both comply well with the expert's monotonicity knowledge and predict the training data accurately. The suggested approach leads to lower root-mean-squared errors than comparative methods from the literature for the sparse data sets considered in this work., Comment: 22 pages, 6 figures

We deal with monotonic regression of multivariate functions $f: Q \to \mathbb{R}$ on a compact rectangular domain $Q$ in $\mathbb{R}^d$, where monotonicity is understood in a generalized sense: as isotonicity in some coordinate directions and antitonicity in some other coordinate directions. As usual, the monotonic regression of a given function $f$ is the monotonic function $f^*$ that has the smallest (weighted) mean-squared distance from $f$. We establish a simple general approach to compute monotonic regression functions: namely, we show that the monotonic regression $f^*$ of a given function $f$ can be approximated arbitrarily well -- with simple bounds on the approximation error in both the $2$-norm and the $\infty$-norm -- by the monotonic regression $f_n^*$ of grid-constant functions $f_n$. We also establish the continuity of the monotonic regression $f^*$ of a continuous function $f$ along with an explicit averaging formula for $f^*$. And finally, we deal with generalized monotonic regression where the mean-squared distance from standard monotonic regression is replaced by more complex distance measures which arise, for instance, in maximum smoothed likelihood estimation. We will see that the solution of such generalized monotonic regression problems is simply given by the standard monotonic regression $f^*$., Comment: 37 pages

We illustrate the advantages the high-level open-source software package Pyomo has in rapidly setting up and solving dynamic simulation and optimization problems. In order to do so, we use the example of the Williams-Otto process. We show how to simulate the process dynamics using the collocation method and the IPOPT solver provided by Pyomo. We also discuss waste minimization and yield maximization as two examplary process optimization problems. And finally, we present and compare two approaches to setpoint tracking: one based on proportional-integral feedback control and one based on optimal open-loop control., Comment: 19 pages, 10 figures

We establish the local input-to-state stability of a large class of disturbed nonlinear reaction-diffusion equations w.r.t. the global attractor of the respective undisturbed system., Comment: 16 pages

We establish asymptotic gain along with input-to-state practical stability results for disturbed semilinear systems w.r.t. the global attractor of the respective undisturbed system. We apply our results to a large class of nonlinear reaction-diffusion equations comprising disturbed Chaffee--Infante equations, for example., Comment: 29 pages. Added detailed explanations why we have to work with set-valued semiprocesses (end of Section 2.1 and beginning of Section 3)

We investigate the behavior of infinite-time admissibility under compact perturbations. We show, by means of two completely different examples, that infinite-time admissibility is not preserved under compact perturbations $Q$ of the underlying semigroup generator $A$, even if $A$ and $A+Q$ both generate strongly stable semigroups., Comment: 9 pages, workshop on control theory of infinite-dimensional systems in Hagen (January 2018)

We establish well-posedness results for non-autonomous semilinear input-output systems, the central assumption being the scattering-passivity of the considered semilinear system. We consider both systems with distributed control and observation and systems with boundary control and observation. Applications are given to nonlinearly controlled collocated systems and to nonlinearly controlled port-Hamiltonian systems., Comment: 31 pages, 1 figure. Added an application example (Section 5.1). Also, added proofs (Section 3.1 and 4.1) and stability statements to the well-posedness theorems (Section 3.2 and 4.2). And finally, removed some errors, namely a slight imprecision in the definition of generalized solutions and outputs (see discussion at the very end of Section 2.1) and an error in the assumptions of Corollary 5.1

We establish an exponential stabilization result for linear port-Hamiltonian systems of first order with quite general, not necessarily continuous, energy densities. In fact, we have only to require the energy density of the system to be of bounded variation. In particular, and in contrast to the previously known stabilization results, our result applies to vibrating strings or beams with jumps in their mass density and modulus of elasticity., Comment: 24 pages

We establish characterizations of weak input-to-state stability for abstract dynamical systems with inputs, which are similar to characterizations of uniform and of strong input-to-state stability established in a recent paper by A. Mironchenko and F. Wirth. We also answer, by means of suitable counterexamples, two open questions concerning weak input-to-state stability (and its relation to other common stability concepts) raised in the aforementioned paper., Comment: 18 pages

In this paper, we are concerned with the stabilization of linear port-Hamiltonian systems of arbitrary order $N \in \mathbb{N}$ on a bounded $1$-dimensional spatial domain $(a,b)$. In order to achieve stabilization, we couple the system to a dynamic boundary controller, that is, a controller that acts on the system only via the boundary points $a,b$ of the spatial domain. We use a nonlinear controller in order to capture the nonlinear behavior that realistic actuators often exhibit and, moreover, we allow the output of the controller to be corrupted by actuator disturbances before it is fed back into the system. What we show here is that the resulting nonlinear closed-loop system is input-to-state stable w.r.t.~square-integrable disturbance inputs. In particular, we obtain uniform input-to-state stability for systems of order $N=1$ and a special class of nonlinear controllers, and weak input-to-state stability for systems of arbitrary order $N \in \mathbb{N}$ and a more general class of nonlinear controllers. Also, in both cases, we obtain convergence to $0$ of all solutions as $t \to \infty$. Applications are given to vibrating strings and beams., Comment: 41 pages, 1 figure. Slight generalization of the regularity assumption of the input-to-state stability result (now: absolute continuity instead of Lipschitz continuity), update of the bibliography, correction of typos