Search

The design and operation of modern energy systems are heavily influenced by time-dependent and uncertain parameters, e.g., renewable electricity generation, load-demand, and electricity prices. These are typically represented by a set of discrete realizations known as scenarios. A popular scenario generation approach uses deep generative models (DGM) that allow scenario generation without prior assumptions about the data distribution. However, the validation of generated scenarios is difficult, and a comprehensive discussion about appropriate validation methods is currently lacking. To start this discussion, we provide a critical assessment of the currently used validation methods in the energy scenario generation literature. In particular, we assess validation methods based on probability density, auto-correlation, and power spectral density. Furthermore, we propose using the multifractal detrended fluctuation analysis (MFDFA) as an additional validation method for non-trivial features like peaks, bursts, and plateaus. As representative examples, we train generative adversarial networks (GANs), Wasserstein GANs (WGANs), and variational autoencoders (VAEs) on two renewable power generation time series (photovoltaic and wind from Germany in 2013 to 2015) and an intra-day electricity price time series form the European Energy Exchange in 2017 to 2019. We apply the four validation methods to both the historical and the generated data and discuss the interpretation of validation results as well as common mistakes, pitfalls, and limitations of the validation methods. Our assessment shows that no single method sufficiently characterizes a scenario but ideally validation should include multiple methods and be interpreted carefully in the context of scenarios over short time periods.

Background: Classification of binary data arises naturally in many clinical applications, such as patient risk stratification through ICD codes. One of the key practical challenges in data classification using machine learning is to avoid overfitting. Overfitting in supervised learning primarily occurs when a model learns random variations from noisy labels in training data rather than the underlying patterns. While traditional methods such as regularization and early stopping have demonstrated effectiveness in interpolation tasks, addressing overfitting in the classification of binary data, in which predictions always amount to extrapolation, demands extrapolation-enhanced strategies. One such approach is hybrid mechanistic/data-driven modeling, which integrates prior knowledge on input features into the learning process, enhancing the model's ability to extrapolate. Results: We present NoiseCut, a Python package for noise-tolerant classification of binary data by employing a hybrid modeling approach that leverages solutions of defined max-cut problems. In a comparative analysis conducted on synthetically generated binary datasets, NoiseCut exhibits better overfitting prevention compared to the early stopping technique employed by different supervised machine learning algorithms. The noise tolerance of NoiseCut stems from a dropout strategy that leverages prior knowledge of input features and is further enhanced by the integration of max-cut problems into the learning process. Conclusions: NoiseCut is a Python package for the implementation of hybrid modeling for the classification of binary data. It facilitates the integration of mechanistic knowledge on the input features into learning from data in a structured manner and proves to be a valuable classification tool when the available training data is noisy and/or limited in size. This advantage is especially prominent in medical and biomedical applications where data scarcity and noise are common challenges. The codebase, illustrations, and documentation for NoiseCut are accessible for download at https://pypi.org/project/noisecut/. The implementation detailed in this paper corresponds to the version 0.2.1 release of the software. [ABSTRACT FROM AUTHOR]

Metalearning of numerical algorithms for a given task consists of the data-driven identification and adaptation of an algorithmic structure and the associated hyperparameters. To limit the complexity of the metalearning problem, neural architectures with a certain inductive bias towards favorable algorithmic structures can, and should, be used. We generalize our previously introduced Runge-Kutta neural network to a recursively recurrent neural network superstructure for the design of customized iterative algorithms. In contrast to off-the-shelf deep learning approaches, it features a distinct division into modules for generation of information and for the subsequent assembly of this information towards a solution. Local information in the form of a subspace is generated by subordinate, inner, iterations of recurrent function evaluations starting at the current outer iterate. The update to the next outer iterate is computed as a linear combination of these evaluations, reducing the residual in this space, and constitutes the output of the network. We demonstrate that regular training of the weight parameters inside the proposed superstructure on input/output data of various computational problem classes yields iterations similar to Krylov solvers for linear equation systems, Newton--Krylov solvers for nonlinear equation systems, and Runge-Kutta integrators for ordinary differential equations. Due to its modularity, the superstructure can be readily extended with functionalities needed to represent more general classes of iterative algorithms traditionally based on Taylor series expansions. [ABSTRACT FROM AUTHOR]

Metabolic engineering relies on modifying gene expression to regulate protein concentrations and reaction activities. The gene expression is controlled by the promoter sequence, and sequence libraries are used to scan expression activities and to identify correlations between sequence and activity. We introduce a computational workflow called Exp2Ipynb to analyze promoter libraries maximizing information retrieval and promoter design with desired activity. We applied Exp2Ipynb to seven prokaryotic expression libraries to identify optimal experimental design principles. The workflow is open source, available as Jupyter Notebooks and covers the steps to 1) generate a statistical overview to sequence and activity, 2) train machine-learning algorithms, such as random forest, gradient boosting trees and support vector machines, for prediction and extraction of feature importance, 3) evaluate the performance of the estimator, and 4) to design new sequences with a desired activity using numerical optimization. The workflow can perform regression or classification on multiple promoter libraries, across species or reporter proteins. The most accurate predictions in the sample libraries were achieved when the promoters in the library were recognized by a single sigma factor and a unique reporter system. The prediction confidence mostly depends on sample size and sequence diversity, and we present a relationship to estimate their respective effects. The workflow can be adapted to process sequence libraries from other expression-related problems and increase insight to the growing application of high-throughput experiments, providing support for efficient strain engineering.

Fuels with high‐knock resistance enable modern spark‐ignition engines to achieve high efficiency and thus low CO2 emissions. Identification of molecules with desired autoignition properties indicated by a high research octane number and a high octane sensitivity is therefore of great practical relevance and can be supported by computer‐aided molecular design (CAMD). Recent developments in the field of graph machine learning (graph‐ML) provide novel, promising tools for CAMD. We propose a modular graph‐ML CAMD framework that integrates generative graph‐ML models with graph neural networks and optimization, enabling the design of molecules with desired ignition properties in a continuous molecular space. In particular, we explore the potential of Bayesian optimization and genetic algorithms in combination with generative graph‐ML models. The graph‐ML CAMD framework successfully identifies well‐established high‐octane components. It also suggests new candidates, one of which we experimentally investigate and use to illustrate the need for further autoignition training data. [ABSTRACT FROM AUTHOR]

Gaussian processes (Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the free variables and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the "MeLOn—MachineLearning Models for Optimization" toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn). [ABSTRACT FROM AUTHOR]

Extensive literature has considered reduced, but still highly accurate, nonlinear dynamic process models, particularly for distillation columns. Nevertheless, there is a need for continuing research in this field. Herein, opportunities from the integration of machine learning into existing reduction approaches are discussed. First, key concepts for dynamic model reduction and their limitations are briefly reviewed. Afterwards, promising model structures for reduced hybrid mechanistic/data‐driven models are outlined. Finally, crucial future challenges as well as promising research perspectives are presented. [ABSTRACT FROM AUTHOR]

The high computational requirements of nonlinear model predictive control (NMPC) are a long-standing issue and, among other methods, learning the control policy with machine learning (ML) methods has been proposed in order to improve computational tractability. However, these methods typically do not explicitly consider constraint satisfaction. We propose two methods based on learning the optimal control policy by an artificial neural network (ANN) and using this for initialization to accelerate computations while meeting constraints and achieving good objective function value. In the first, the ANN prediction serves as the initial guess for the solution of the optimal control problem (OCP) solved in NMPC. In the second, the ANN prediction is improved by solving a single quadratic program (QP). We compare the performance of the two proposed strategies against two benchmarks representing the extreme cases of (i) solving the NMPC problem to convergence using the shift-initialization strategy and (ii) implementing the controls predicted by the ANN prediction without further correction to reduce the computational delay. We find that the proposed ANN initialization strategy mostly results in the same control policy as the shift-initialization strategy. The computational times are on average ∼ 45% longer but the maximum time is ∼ 42% smaller and the distribution is tighter, thus more predictable. The proposed QP-based method yields a good compromise between finding the optimal control policy and solution time. Closed-loop infeasibilities are negligible and the objective function is typically greatly improved as compared to benchmark (ii). The computational time required for the necessary second-order sensitivity integration is typically an order of magnitude smaller than for solving the NMPC problem to convergence. • We present two methods to reduce computational demand of NMPC with machine learning. • Both methods obtain an initial guess for a subsequent algorithm from ML model. • First method achieves tight distribution of solution times and predictable delay. • Second method achieves good objective values and feasibility with little effort. [ABSTRACT FROM AUTHOR]

Ionic liquids (ILs) are important solvents for sustainable processes and predicting activity coefficients (ACs) of solutes in ILs is needed. Recently, matrix completion methods (MCMs), transformers, and graph neural networks (GNNs) have shown high accuracy in predicting ACs of binary mixtures, superior to well-established models, e.g., COSMO-RS and UNIFAC. GNNs are particularly promising here as they learn a molecular graph-to-property relationship without pretraining, typically required for transformers, and are, unlike MCMs, applicable to molecules not included in training. For ILs, however, GNN applications are currently missing. Herein, we present a GNN to predict temperature-dependent infinite dilution ACs of solutes in ILs. We train the GNN on a database including more than 40,000 AC values and compare it to a state-of-the-art MCM. The GNN and MCM achieve similar high prediction performance, with the GNN additionally enabling high-quality predictions for ACs of solutions that contain ILs and solutes not considered during training. • Graph neural networks for activity coefficient prediction of solutes in ionic liquids. • Fast prediction of temperature-dependent infinite dilution activity coefficients. • Comparative prediction quality to state-of-the-art matrix completion methods. • Successful generalization to previously unseen ionic liquid and solute molecules. [ABSTRACT FROM AUTHOR]

• Introduce HybridML, an open source hybrid modeling platform. • Train hybrid models including neural networks and differential equations in Tensorflow. • Build a hybrid model to predict the drug concentration in patients' blood over time. Hybrid modelling, i.e., the combination of data-driven modelling with mechanistic model components, reduces the data demand and enables extrapolation of data-driven models. However, building, training and evaluation of hybrid models is cumbersome with current frameworks. We developed HybridML, an open-source modeling platform, in which hybrid models can be trained, i.e., combinations of artificial neural networks, arithmetic expressions, and differential equations. We employ TensorFlow for artificial neural network training and Casadi to integrate ordinary differential equations and provide gradients of differential model equations enabling continuous time representations. HybridML provides also a JSON interface for the model development. We apply HybridML to an industrial case study, in which the trained model is used to predict drug concentrations over time, based on physiological information about the patients. To demonstrate its versatility, we also present a nonlinear application, where HybridML is used to model the spread of the COVID-19 pandemic in German federal states based on the state's socio-economic attributes. [ABSTRACT FROM AUTHOR]

Innovative membrane technologies optimally integrated into large separation process plants are essential for economical water treatment and disposal. However, the mass transport through membranes is commonly described by nonlinear differential-algebraic mechanistic models at the nano-scale, while the process and its economics range up to large-scale. Thus, the optimal design of membranes in process plants requires decision making across multiple scales, which is not possible using standard tools. In this work, we embed artificial neural networks (ANNs) as surrogate models in the deterministic global optimization to bridge the gap of scales. This methodology allows for deterministic global optimization of membrane processes with accurate transport models – avoiding the utilization of inaccurate approximations through heuristics or short-cut models. The ANNs are trained based on data generated by a one-dimensional extended Nernst-Planck ion transport model and extended to a more accurate two-dimensional distribution of the membrane module, that captures the filtration-related decreasing retention of salt. We simultaneously design the membrane and plant layout yielding optimal membrane module synthesis properties along with the optimal plant design for multiple objectives, feed concentrations, filtration stages, and salt mixtures. The developed process models and the optimization solver are available open-source, enabling computational resource-efficient multi-scale optimization in membrane science. Image 1 • Resolving the multi-scale physics for innovative design of membrane processes. • Simultaneous optimization of membrane synthesis and process plant. • Replacement of complex high-fidelity membrane transport model by surrogate model. • Data-driven surrogate models are created using artificial neural networks. • Rigorous optimization of plant superstructure with accurate models. [ABSTRACT FROM AUTHOR]