Search

We propose excess Gibbs free energy graph neural networks (GE-GNNs) for predicting composition-dependent activity coefficients of binary mixtures. The GE-GNN architecture ensures thermodynamic consistency by predicting the molar excess Gibbs free energy and using thermodynamic relations to obtain activity coefficients. As these are differential, automatic differentiation is applied to learn the activity coefficients in an end-to-end manner. Since the architecture is based on fundamental thermodynamics, we do not require additional loss terms to learn thermodynamic consistency. As the output is a fundamental property, we neither impose thermodynamic modeling limitations and assumptions. We demonstrate high accuracy and thermodynamic consistency of the activity coefficient predictions.

In chemical engineering, process data are expensive to acquire, and complex phenomena are difficult to fully model. We explore the use of physics-informed neural networks (PINNs) for dynamic processes with incomplete mechanistic semi-explicit differential-algebraic equation systems and scarce process data. In particular, we focus on estimating states for which neither direct observational data nor constitutive equations are available. We propose an easy-to-apply heuristic to assess whether estimation of such states may be possible. As numerical examples, we consider a continuously stirred tank reactor and a liquid-liquid separator. We find that PINNs can infer unmeasured states with reasonable accuracy, and they generalize better in low-data scenarios than purely data-driven models. We thus show that PINNs are capable of modeling processes when relatively few experimental data and only partially known mechanistic descriptions are available, and conclude that they constitute a promising avenue that warrants further investigation., Comment: manuscript (32 pages, 9 figures, 11 tables), supporting materials (14 pages, 4 figures, 5 tables)

We propose a method to construct representative price profiles of the day-ahead (DA) and the intraday (ID) electricity spot markets and use this method to provide examples of ready-to-use price data sets. In contrast to common scenario generation approaches, the method is deterministic and relies on a small number of degrees of freedom, with the aim to be well defined and easy to use. We thereby target an enhanced comparability of future research studies on demand-side management and energy cost optimization. We construct the price profiles based on historical time series from the spot markets of interest, e.g., European Power Exchange (EPEX) spot. To this end, we extract key price components from the data while also accounting for known dominant mechanisms in the price variation. Further, the method is able to preserve key statistical features of the historical data (e.g., mean and standard deviation) when constructing the benchmark profile. Finally, our approach ensures comparability of ID and DA price profiles by design, as their cumulative (integral) price can be made identical if needed., Comment: Supplementary information (SI) included; Manuscript: 27 pages, 9 figures, 4 tables; SI: 7 pages, 5 figures, 2 tables

We present a method for end-to-end learning of Koopman surrogate models for optimal performance in control. In contrast to previous contributions that employ standard reinforcement learning (RL) algorithms, we use a training algorithm that exploits the potential differentiability of environments based on mechanistic simulation models. We evaluate the performance of our method by comparing it to that of other controller type and training algorithm combinations on a literature known eNMPC case study. Our method exhibits superior performance on this problem, thereby constituting a promising avenue towards more capable controllers that employ dynamic surrogate models., Comment: 6 pages, 4 figures, 1 table

Polymer particle size constitutes a crucial characteristic of product quality in polymerization. Raman spectroscopy is an established and reliable process analytical technology for in-line concentration monitoring. Recent approaches and some theoretical considerations show a correlation between Raman signals and particle sizes but do not determine polymer size from Raman spectroscopic measurements accurately and reliably. With this in mind, we propose three alternative machine learning workflows to perform this task, all involving diffusion maps, a nonlinear manifold learning technique for dimensionality reduction: (i) directly from diffusion maps, (ii) alternating diffusion maps, and (iii) conformal autoencoder neural networks. We apply the workflows to a data set of Raman spectra with associated size measured via dynamic light scattering of 47 microgel (cross-linked polymer) samples in a diameter range of 208nm to 483 nm. The conformal autoencoders substantially outperform state-of-the-art methods and results for the first time in a promising prediction of polymer size from Raman spectra., Comment: 51 pages, 12 figures, 4 tables

The critical micelle concentration (CMC) of surfactant molecules is an essential property for surfactant applications in industry. Recently, classical QSPR and Graph Neural Networks (GNNs), a deep learning technique, have been successfully applied to predict the CMC of surfactants at room temperature. However, these models have not yet considered the temperature dependency of the CMC, which is highly relevant for practical applications. We herein develop a GNN model for temperature-dependent CMC prediction of surfactants. We collect about 1400 data points from public sources for all surfactant classes, i.e., ionic, nonionic, and zwitterionic, at multiple temperatures. We test the predictive quality of the model for following scenarios: i) when CMC data for surfactants are present in the training of the model in at least one different temperature, and ii) CMC data for surfactants are not present in the training, i.e., generalizing to unseen surfactants. In both test scenarios, our model exhibits a high predictive performance of R$^2 \geq $ 0.94 on test data. We also find that the model performance varies by surfactant class. Finally, we evaluate the model for sugar-based surfactants with complex molecular structures, as these represent a more sustainable alternative to synthetic surfactants and are therefore of great interest for future applications in the personal and home care industries.

Constraint-based optimization of microbial strains and model-based bioprocess design have been used extensively to enhance yields in biotechnological processes. However, strain and process optimization are usually carried out in sequential steps, causing underperformance of the biotechnological process when scaling up to industrial fermentation conditions. Herein, we propose the optimization formulation SimulKnock that combines the optimization of a fermentation process with metabolic network design in a bilevel optimization program. The upper level maximizes space-time yield and includes mass balances of a continuous fermentation, while the lower level is based on flux balance analysis. SimulKnock predicts optimal gene deletions and finds the optimal trade-off between growth rate and product yield. Results of a case study with a genome-scale metabolic model of E. coli indicate higher space-time yields than a sequential approach using OptKnock for almost all target products considered. By leveraging SimulKnock, we reduce the gap between strain and process optimization.

This work studies synergies arising from combining industrial demand response and local renewable electricity supply. To this end, we optimize the design of a local electricity generation and storage system with an integrated demand response scheduling of a continuous power-intensive production process in a multi-stage problem. We optimize both total annualized cost and global warming impact and consider local photovoltaic and wind electricity generation, an electric battery, and electricity trading on day-ahead and intraday market. We find that installing a battery can reduce emissions and enable large trading volumes on the electricity markets, but significantly increases cost. Economically and ecologically-optimal operation of the process and battery are driven primarily by the electricity price and grid emission factor, respectively, rather than locally generated electricity. A parameter study reveals that cost savings from the local system and flexibilizing the process behave almost additively., Comment: manuscript (35 pages, 9 figures, 8 tables), supporting materials (14 pages, 9 figures, 7 tables)

We use Koopman theory for data-driven model reduction of nonlinear dynamical systems with controls. We propose generic model structures combining delay-coordinate encoding of measurements and full-state decoding to integrate reduced Koopman modeling and state estimation. We present a deep-learning approach to train the proposed models. A case study demonstrates that our approach provides accurate control models and enables real-time capable nonlinear model predictive control of a high-purity cryogenic distillation column.

Surfactants are of high importance in different industrial sectors such as cosmetics, detergents, oil recovery and drug delivery systems. Therefore, many quantitative structure-property relationship (QSPR) models have been developed for surfactants. Each predictive model typically focuses on one surfactant class, mostly nonionics. Graph Neural Networks (GNNs) have exhibited a great predictive performance for property prediction of ionic liquids, polymers and drugs in general. Specifically for surfactants, GNNs can successfully predict critical micelle concentration (CMC), a key surfactant property associated with micellization. A key factor in the predictive ability of QSPR and GNN models is the data available for training. Based on extensive literature search, we create the largest available CMC database with 429 molecules and the first large data collection for surface excess concentration ($\Gamma$$_{m}$), another surfactant property associated with foaming, with 164 molecules. Then, we develop GNN models to predict the CMC and $\Gamma$$_{m}$ and we explore different learning approaches, i.e., single- and multi-task learning, as well as different training strategies, namely ensemble and transfer learning. We find that a multi-task GNN with ensemble learning trained on all $\Gamma$$_{m}$ and CMC data performs best. Finally, we test the ability of our CMC model to generalize on industrial grade pure component surfactants. The GNN yields highly accurate predictions for CMC, showing great potential for future industrial applications.

Achieving real-time capability is an essential prerequisite for the industrial implementation of nonlinear model predictive control (NMPC). Data-driven model reduction offers a way to obtain low-order control models from complex digital twins. In particular, data-driven approaches require little expert knowledge of the particular process and its model, and provide reduced models of a well-defined generic structure. Herein, we apply our recently proposed data-driven reduction strategy based on Koopman theory [Schulze et al. (2022), Comput. Chem. Eng.] to generate a low-order control model of an air separation unit (ASU). The reduced Koopman model combines autoencoders and linear latent dynamics and is constructed using machine learning. Further, we present an NMPC implementation that uses derivative computation tailored to the fixed block structure of reduced Koopman models. Our reduction approach with tailored NMPC implementation enables real-time NMPC of an ASU at an average CPU time decrease by 98 %.

Microgels are cross-linked, colloidal polymer networks with great potential for stimuli-response release in drug-delivery applications, as their size in the nanometer range allows them to pass human cell boundaries. For applications with specified requirements regarding size, producing tailored microgels in a continuous flow reactor is advantageous because the microgel properties can be controlled tightly. However, no fully-specified mechanistic models are available for continuous microgel synthesis, as the physical properties of the included components are only studied partly. To address this gap and accelerate tailor-made microgel development, we propose a data-driven optimization in a hardware-in-the-loop approach to efficiently synthesize microgels with defined sizes. We optimize the synthesis regarding conflicting objectives (maximum production efficiency, minimum energy consumption, and the desired microgel radius) by applying Bayesian optimization via the solver ``Thompson sampling efficient multi-objective optimization'' (TS-EMO). We validate the optimization using the deterministic global solver ``McCormick-based Algorithm for mixed-integer Nonlinear Global Optimization'' (MAiNGO) and verify three computed Pareto optimal solutions via experiments. The proposed framework can be applied to other desired microgel properties and reactor setups and has the potential of efficient development by minimizing number of experiments and modelling effort needed., Comment: Manuscript: 24 pages, 8 figures; SI: 9 pages, 3 figures

The flexible operation of electrified chemical processes, powered by renewable electricity, offers economic and ecological incentives, paving the way for a more sustainable chemical industry. However, this requires a departure from the traditional steady-state operation, posing a challenge to process design and operation. We propose an approach for flexible biodiesel production by utilizing intermediate and final buffer tanks. These tanks serve to decouple the dynamics between different process unit operations. We investigate the flexibility of three different process configurations and compare the outcomes of their dynamic optimization strategies with those of a steady-state operation, considering a typical demand-side response scenario. For all strategies, we employ local gradient-based optimization and solve them using our open-source optimization framework DyOS. Our findings indicate that by incorporating intermediate buffer tanks, we fully exploit the process flexibility potential, leading to total energy savings of up to 29%. We also propose an implementation based on distributed optimization problems, yielding similar savings while significantly reducing computational costs by over 10-fold and improving convergence, thus paving the way toward online application. Overall, this study highlights the potential benefits of incorporating buffer tanks and optimization strategies in the flexible operation of electrified chemical processes, even those initially designed solely for steady-state operation.

(Economic) nonlinear model predictive control ((e)NMPC) requires dynamic models that are sufficiently accurate and computationally tractable. Data-driven surrogate models for mechanistic models can reduce the computational burden of (e)NMPC; however, such models are typically trained by system identification for maximum prediction accuracy on simulation samples and perform suboptimally in (e)NMPC. We present a method for end-to-end reinforcement learning of Koopman surrogate models for optimal performance as part of (e)NMPC. We apply our method to two applications derived from an established nonlinear continuous stirred-tank reactor model. The controller performance is compared to that of (e)NMPCs utilizing models trained using system identification, and model-free neural network controllers trained using reinforcement learning. We show that the end-to-end trained models outperform those trained using system identification in (e)NMPC, and that, in contrast to the neural network controllers, the (e)NMPC controllers can react to changes in the control setting without retraining., Comment: manuscript (20 pages, 7 figures, 6 tables), supplementary materials (3 pages, 2 tables)

We propose Gibbs-Duhem-informed neural networks for the prediction of binary activity coefficients at varying compositions. That is, we include the Gibbs-Duhem equation explicitly in the loss function for training neural networks, which is straightforward in standard machine learning (ML) frameworks enabling automatic differentiation. In contrast to recent hybrid ML approaches, our approach does not rely on embedding a specific thermodynamic model inside the neural network and corresponding prediction limitations. Rather, Gibbs-Duhem consistency serves as regularization, with the flexibility of ML models being preserved. Our results show increased thermodynamic consistency and generalization capabilities for activity coefficient predictions by Gibbs-Duhem-informed graph neural networks and matrix completion methods. We also find that the model architecture, particularly the activation function, can have a strong influence on the prediction quality. The approach can be easily extended to account for other thermodynamic consistency conditions.

The synthesis of methanol from captured carbon dioxide and green hydrogen could be a promising replacement for the current fossil-based production. The major energy input and cost driver for such a process is the electricity for hydrogen production. Time-variable electricity cost or availability thus motivates flexible operation. However, it is unclear if each unit of the process should be operated flexibly, and if storage of electricity or hydrogen reduces the methanol production cost. To answer these questions, we modeled a Power-to-Methanol plant with batteries and hydrogen storage. Using this model, we solved a combined design and scheduling optimization problem, which provides the optimal size of the units of the plant and their optimal (quasi-stationary) operation. The annualized cost of methanol was minimized for a grid-connected and a stand-alone case study. The optimization results confirm that storage, especially hydrogen storage, is particularly beneficial when the electricity price is high and highly fluctuating. Irrespective of the presence of storage, the whole Power-to-Methanol plant should be operated flexibly: even moderate flexibility of the methanol synthesis unit significantly reduces the production cost.

Meta-learning of numerical algorithms for a given task consists of the data-driven identification and adaptation of an algorithmic structure and the associated hyperparameters. To limit the complexity of the meta-learning problem, neural architectures with a certain inductive bias towards favorable algorithmic structures can, and should, be used. We generalize our previously introduced Runge-Kutta neural network to a recursively recurrent neural network (R2N2) superstructure for the design of customized iterative algorithms. In contrast to off-the-shelf deep learning approaches, it features a distinct division into modules for generation of information and for the subsequent assembly of this information towards a solution. Local information in the form of a subspace is generated by subordinate, inner, iterations of recurrent function evaluations starting at the current outer iterate. The update to the next outer iterate is computed as a linear combination of these evaluations, reducing the residual in this space, and constitutes the output of the network. We demonstrate that regular training of the weight parameters inside the proposed superstructure on input/output data of various computational problem classes yields iterations similar to Krylov solvers for linear equation systems, Newton-Krylov solvers for nonlinear equation systems, and Runge-Kutta integrators for ordinary differential equations. Due to its modularity, the superstructure can be readily extended with functionalities needed to represent more general classes of iterative algorithms traditionally based on Taylor series expansions., Comment: manuscript (22 pages, 9 figures), supporting information (11 pages, 9 figures)

Graph neural networks (GNNs) are emerging in chemical engineering for the end-to-end learning of physicochemical properties based on molecular graphs. A key element of GNNs is the pooling function which combines atom feature vectors into molecular fingerprints. Most previous works use a standard pooling function to predict a variety of properties. However, unsuitable pooling functions can lead to unphysical GNNs that poorly generalize. We compare and select meaningful GNN pooling methods based on physical knowledge about the learned properties. The impact of physical pooling functions is demonstrated with molecular properties calculated from quantum mechanical computations. We also compare our results to the recent set2set pooling approach. We recommend using sum pooling for the prediction of properties that depend on molecular size and compare pooling functions for properties that are molecular size-independent. Overall, we show that the use of physical pooling functions significantly enhances generalization.

Trained ML models are commonly embedded in optimization problems. In many cases, this leads to large-scale NLPs that are difficult to solve to global optimality. While ML models frequently lead to large problems, they also exhibit homogeneous structures and repeating patterns (e.g., layers in ANNs). Thus, specialized solution strategies can be used for large problem classes. Recently, there have been some promising works proposing specialized reformulations using mixed-integer programming or reduced space formulations. However, further work is needed to develop more efficient solution approaches and keep up with the rapid development of new ML model architectures.

Molecular property prediction is of crucial importance in many disciplines such as drug discovery, molecular biology, or material and process design. The frequently employed quantitative structure-property/activity relationships (QSPRs/QSARs) characterize molecules by descriptors which are then mapped to the properties of interest via a linear or nonlinear model. In contrast, graph neural networks, a novel machine learning method, directly work on the molecular graph, i.e., a graph representation where atoms correspond to nodes and bonds correspond to edges. GNNs allow to learn properties in an end-to-end fashion, thereby avoiding the need for informative descriptors as in QSPRs/QSARs. GNNs have been shown to achieve state-of-the-art prediction performance on various property predictions tasks and represent an active field of research. We describe the fundamentals of GNNs and demonstrate the application of GNNs via two examples for molecular property prediction.

Ionic liquids (ILs) are important solvents for sustainable processes and predicting activity coefficients (ACs) of solutes in ILs is needed. Recently, matrix completion methods (MCMs), transformers, and graph neural networks (GNNs) have shown high accuracy in predicting ACs of binary mixtures, superior to well-established models, e.g., COSMO-RS and UNIFAC. GNNs are particularly promising here as they learn a molecular graph-to-property relationship without pretraining, typically required for transformers, and are, unlike MCMs, applicable to molecules not included in training. For ILs, however, GNN applications are currently missing. Herein, we present a GNN to predict temperature-dependent infinite dilution ACs of solutes in ILs. We train the GNN on a database including more than 40,000 AC values and compare it to a state-of-the-art MCM. The GNN and MCM achieve similar high prediction performance, with the GNN additionally enabling high-quality predictions for ACs of solutions that contain ILs and solutes not considered during training., Comment: 16 pages, 4 figures, 5 tables

Fuels with high-knock resistance enable modern spark-ignition engines to achieve high efficiency and thus low CO2 emissions. Identification of molecules with desired autoignition properties indicated by a high research octane number and a high octane sensitivity is therefore of great practical relevance and can be supported by computer-aided molecular design (CAMD). Recent developments in the field of graph machine learning (graph-ML) provide novel, promising tools for CAMD. We propose a modular graph-ML CAMD framework that integrates generative graph-ML models with graph neural networks and optimization, enabling the design of molecules with desired ignition properties in a continuous molecular space. In particular, we explore the potential of Bayesian optimization and genetic algorithms in combination with generative graph-ML models. The graph-ML CAMD framework successfully identifies well-established high-octane components. It also suggests new candidates, one of which we experimentally investigate and use to illustrate the need for further auto-ignition training data., Comment: manuscript (26 pages, 9 figures, 2 tables), supporting information (12 pages, 8 figures, 1 table)

Electricity is traded on various markets with different time horizons and regulations. Short-term intraday trading becomes increasingly important due to the higher penetration of renewables. In Germany, the intraday electricity price typically fluctuates around the day-ahead price of the European Power EXchange (EPEX) spot markets in a distinct hourly pattern. This work proposes a probabilistic modeling approach that models the intraday price difference to the day-ahead contracts. The model captures the emerging hourly pattern by considering the four 15 min intervals in each day-ahead price interval as a four-dimensional joint probability distribution. The resulting nontrivial, multivariate price difference distribution is learned using a normalizing flow, i.e., a deep generative model that combines conditional multivariate density estimation and probabilistic regression. Furthermore, this work discusses the influence of different external impact factors based on literature insights and impact analysis using explainable artificial intelligence (XAI). The normalizing flow is compared to an informed selection of historical data and probabilistic forecasts using a Gaussian copula and a Gaussian regression model. Among the different models, the normalizing flow identifies the trends with the highest accuracy and has the narrowest prediction intervals. Both the XAI analysis and the empirical experiments highlight that the immediate history of the price difference realization and the increments of the day-ahead price have the most substantial impact on the price difference., Comment: manuscript (20 pages, 11 figures, 5 tables), supporting information (8 pages, 5 figures, 4 tables)

Innovative membrane technologies optimally integrated into large separation process plants are essential for economical water treatment and disposal. However, the mass transport through membranes is commonly described by nonlinear differential-algebraic mechanistic models at the nano-scale, while the process and its economics range up to large-scale. Thus, the optimal design of membranes in process plants requires decision making across multiple scales, which is not tractable using standard tools. In this work, we embed artificial neural networks~(ANNs) as surrogate models in the deterministic global optimization to bridge the gap of scales. This methodology allows for deterministic global optimization of membrane processes with accurate transport models -- avoiding the utilization of inaccurate approximations through heuristics or short-cut models. The ANNs are trained based on data generated by a one-dimensional extended Nernst-Planck ion transport model and extended to a more accurate two-dimensional distribution of the membrane module, that captures the filtration-related decreasing retention of salt. We simultaneously design the membrane and plant layout yielding optimal membrane module synthesis properties along with the optimal plant design for multiple objectives, feed concentrations, filtration stages, and salt mixtures. The developed process models and the optimization solver are available open-source, enabling computational resource-efficient multi-scale optimization in membrane science.

We present a specialized scenario generation method that utilizes forecast information to generate scenarios for day-ahead scheduling problems. In particular, we use normalizing flows to generate wind power scenarios by sampling from a conditional distribution that uses wind speed forecasts to tailor the scenarios to a specific day. We apply the generated scenarios in a stochastic day-ahead bidding problem of a wind electricity producer and analyze whether the scenarios yield profitable decisions. Compared to Gaussian copulas and Wasserstein-generative adversarial networks, the normalizing flow successfully narrows the range of scenarios around the daily trends while maintaining a diverse variety of possible realizations. In the stochastic day-ahead bidding problem, the conditional scenarios from all methods lead to significantly more stable profitable results compared to an unconditional selection of historical scenarios. The normalizing flow consistently obtains the highest profits, even for small sets scenarios., Comment: manuscript (18 pages, 7 figures, 6 tables), supporting information (2 pages, 1 figure, 1 table)

To model manifold data using normalizing flows, we employ isometric autoencoders to design embeddings with explicit inverses that do not distort the probability distribution. Using isometries separates manifold learning and density estimation and enables training of both parts to high accuracy. Thus, model selection and tuning are simplified compared to existing injective normalizing flows. Applied to data sets on (approximately) flat manifolds, the combined approach generates high-quality data., Comment: 11 pages, 7 figures, 4 tables

We use Koopman theory to develop a data-driven nonlinear model reduction and identification strategy for multiple-input multiple-output (MIMO) input-affine dynamical systems. While the present literature has focused on linear and bilinear Koopman models, we derive and use a Wiener-type Koopman formulation. We discuss that the Wiener structure is particularly suitable for model reduction, and can be naturally derived from Koopman theory. Moreover, the Wiener block-structure unifies the mathematical simplicity of linear dynamical blocks and the accuracy of bilinear dynamics. We present a Koopman deep-learning strategy combining autoencoders and linear dynamics that generates low-order surrogate models of MIMO Wiener type. In three case studies, we apply our framework for identification and reduction of a system with input multiplicity, a chemical reactor and a high-purity distillation column. We compare the prediction performance of the identified Wiener models to linear and bilinear Koopman models. We observe the highest accuracy and strongest model reduction capabilities of low-order Wiener-type Koopman models, making them promising for control.

The design and operation of modern energy systems are heavily influenced by time-dependent and uncertain parameters, e.g., renewable electricity generation, load-demand, and electricity prices. These are typically represented by a set of discrete realizations known as scenarios. A popular scenario generation approach uses deep generative models (DGM) that allow scenario generation without prior assumptions about the data distribution. However, the validation of generated scenarios is difficult, and a comprehensive discussion about appropriate validation methods is currently lacking. To start this discussion, we provide a critical assessment of the currently used validation methods in the energy scenario generation literature. In particular, we assess validation methods based on probability density, auto-correlation, and power spectral density. Furthermore, we propose using the multifractal detrended fluctuation analysis (MFDFA) as an additional validation method for non-trivial features like peaks, bursts, and plateaus. As representative examples, we train generative adversarial networks (GANs), Wasserstein GANs (WGANs), and variational autoencoders (VAEs) on two renewable power generation time series (photovoltaic and wind from Germany in 2013 to 2015) and an intra-day electricity price time series form the European Energy Exchange in 2017 to 2019. We apply the four validation methods to both the historical and the generated data and discuss the interpretation of validation results as well as common mistakes, pitfalls, and limitations of the validation methods. Our assessment shows that no single method sufficiently characterizes a scenario but ideally validation should include multiple methods and be interpreted carefully in the context of scenarios over short time periods., Comment: 20 pages, 8 figures, 2 tables

Neural networks-based learning of the distribution of non-dispatchable renewable electricity generation from sources such as photovoltaics (PV) and wind as well as load demands has recently gained attention. Normalizing flow density models are particularly well suited for this task due to the training through direct log-likelihood maximization. However, research from the field of image generation has shown that standard normalizing flows can only learn smeared-out versions of manifold distributions. Previous works on normalizing flow-based scenario generation do not address this issue, and the smeared-out distributions result in the sampling of noisy time series. In this paper, we exploit the isometry of the principal component analysis (PCA), which sets up the normalizing flow in a lower-dimensional space while maintaining the direct and computationally efficient likelihood maximization. We train the resulting principal component flow (PCF) on data of PV and wind power generation as well as load demand in Germany in the years 2013 to 2015. The results of this investigation show that the PCF preserves critical features of the original distributions, such as the probability density and frequency behavior of the time series. The application of the PCF is, however, not limited to renewable power generation but rather extends to any data set, time series, or otherwise, which can be efficiently reduced using PCA., Comment: 18 pages, 7 figures

Diverse domains of science and engineering use parameterised mechanistic models. Engineers and scientists can often hypothesise several rival models to explain a specific process or phenomenon. Consider a model discrimination setting where we wish to find the best mechanistic, dynamic model candidate and the best model parameter estimates. Typically, several rival mechanistic models can explain the available data, so design of dynamic experiments for model discrimination helps optimally collect additional data by finding experimental settings that maximise model prediction divergence. We argue there are two main approaches in the literature for solving the optimal design problem: (i) the analytical approach, using linear and Gaussian approximations to find closed-form expressions for the design objective, and (ii) the data-driven approach, which often relies on computationally intensive Monte Carlo techniques. Olofsson et al. (ICML 35, 2018) introduced Gaussian process (GP) surrogate models to hybridise the analytical and data-driven approaches, which allowed for computationally efficient design of experiments for discriminating between black-box models. In this study, we demonstrate that we can extend existing methods for optimal design of dynamic experiments to incorporate a wider range of problem uncertainty. We also extend the Olofsson et al. (2018) method of using GP surrogate models for discriminating between dynamic black-box models. We evaluate our approach on a well-known case study from literature, and explore the consequences of using GP surrogates to approximate gradient-based methods.

Existing open-source modeling frameworks dedicated to energy systems optimization typically utilize (mixed-integer) linear programming ((MI)LP) formulations, which lack modeling freedom for technical system design and operation. We present COMANDO, an open-source Python package for component-oriented modeling and optimization for nonlinear design and operation of integrated energy systems. COMANDO allows to assemble system models from component models including nonlinear, dynamic and discrete characteristics. Based on a single system model, different deterministic and stochastic problem formulations can be obtained by varying objective function and underlying data, and by applying automatic or manual reformulations. The flexible open-source implementation allows for the integration of customized routines required to solve challenging problems, e.g., initialization, problem decomposition, or sequential solution strategies. We demonstrate features of COMANDO via case studies, including automated linearization, dynamic optimization, stochastic programming, and the use of nonlinear artificial neural networks as surrogate models in a reduced-space formulation for deterministic global optimization., Comment: 24 pages, 1 graphical abstract, 13 figures, 4 tables

Waste heat recovery for trucks via organic Rankine cycle is a promising technology to reduce fuel consumption and emissions. As the vehicles are operated in street traffic, the heat source is subject to strong fluctuations. Consequently, such disturbances have to be considered to enable safe and efficient operation. Herein, we find optimal operating policies for several representative scenarios by means of dynamic optimization and discuss the implications on control strategy design. First, we optimize operation of a typical driving cycle with data from a test rig. Results indicate that operating the cycle at minimal superheat is an appropriate operating policy. Second, we consider a scenario where the permissible expander power is temporarily limited, which is realistic in street traffic. In this case, an operating policy with flexible superheat can reduce the losses associated with operation at minimal superheat by up to 53 % in the considered scenario. As the duration of power limitation increases, other constraints might become active which results in part of the exhaust gas being bypassed, hence reduced savings.

Nonlinear model predictive control (NMPC) requires accurate and computationally efficient plant models. Our previous work has shown that the classical compartmentalization model reduction approach for distillation columns can be enhanced by replacing parts of the system of equations by artificial neural networks (ANNs) trained on offline solved solutions to improve computational performance. In real-life applications, the absence of a high-fidelity model for data generation can, however, prevent the deployment of this approach. Therefore, we propose a method that utilizes solely plant measurement data, starting from a small initial data set and then continuously adapting to newly measured data. The efficacy of the proposed approach is examined in silico for a distillation column from literature. To this end, we first adjust our reduced hybrid mechanistic/data-driven modeling approach that originally builds on compartmentalization to a stage-aggregation procedure, tailoring it for the application within the adaptive learning framework. Second, we apply an adaptive learning algorithm that trains the ANNs replacing the stationary stage-to-stage calculations on newly available data. We apply the adaptive learning of the hybrid model within a regulatory NMPC framework and conduct closed-loop simulations. We demonstrate that by using the proposed method, the control performance can be steadily improved over time compared to a non-adaptive approach while being real-time applicable. Moreover, we show that the performance when using either a model trained on excessive amounts of offline generated data or the original high-fidelity model can be approached in the limit., Comment: 9 pages, 4 figures

Data-driven models are becoming increasingly popular in engineering, on their own or in combination with mechanistic models. Commonly, the trained models are subsequently used in model-based optimization of design and/or operation of processes. Thus, it is critical to ensure that data-driven models are not evaluated outside their validity domain during process optimization. We propose a method to learn this validity domain and encode it as constraints in process optimization. We first perform a topological data analysis using persistent homology identifying potential holes or separated clusters in the training data. In case clusters or holes are identified, we train a one-class classifier, i.e., a one-class support vector machine, on the training data domain and encode it as constraints in the subsequent process optimization. Otherwise, we construct the convex hull of the data and encode it as constraints. We finally perform deterministic global process optimization with the data-driven models subject to their respective validity constraints. To ensure computational tractability, we develop a reduced-space formulation for trained one-class support vector machines and show that our formulation outperforms common full-space formulations by a factor of over 3,000, making it a viable tool for engineering applications. The method is ready-to-use and available open-source as part of our MeLOn toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

Gaussian processes~(Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the degrees of freedom and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the "MeLOn - Machine Learning Models for Optimization" toolbox~(https://git.rwth-aachen.de/avt.svt/public/MeLOn).