Search

Soft gels, formed via the self-assembly of particulate organic materials, exhibit intricate multi-scale structures that provides them with flexibility and resilience when subjected to external stresses. This work combines molecular simulations and topological data analysis (TDA) to characterize the complex multi-scale structure of soft gels. Our TDA analysis focuses on the use of the Euler characteristic, which is an interpretable and computationally-scalable topological descriptor that is combined with filtration operations to obtain information on the geometric (local) and topological (global) structure of soft gels. We reduce the topological information obtained with TDA using principal component analysis (PCA) and show that this provides an informative low-dimensional representation of gel structure. We use the proposed computational framework to investigate the influence of gel preparation (e.g., quench rate, volume fraction) on soft gel structure and to explore dynamic deformations that emerge under oscillatory shear in various response regimes (linear, nonlinear, and flow). Our analysis identifies specific scales and extents at which hierarchical structures in soft gels are affected; moreover, correlations between structural deformations and mechanical phenomena (such as shear stiffening) are explored. In summary, we show that TDA facilitates the mathematical representation, quantification, and analysis of soft gel structures, extending traditional network analysis methods to capture both local and global organization.

Datasets encountered in scientific and engineering applications appear in complex formats (e.g., images, multivariate time series, molecules, video, text strings, networks). Graph theory provides a unifying framework to model such datasets and enables the use of powerful tools that can help analyze, visualize, and extract value from data. In this work, we present PlasmoData.jl, an open-source, Julia framework that uses concepts of graph theory to facilitate the modeling and analysis of complex datasets. The core of our framework is a general data modeling abstraction, which we call a DataGraph. We show how the abstraction and software implementation can be used to represent diverse data objects as graphs and to enable the use of tools from topology, graph theory, and machine learning (e.g., graph neural networks) to conduct a variety of tasks. We illustrate the versatility of the framework by using real datasets: i) an image classification problem using topological data analysis to extract features from the graph model to train machine learning models; ii) a disease outbreak problem where we model multivariate time series as graphs to detect abnormal events; and iii) a technology pathway analysis problem where we highlight how we can use graphs to navigate connectivity. Our discussion also highlights how PlasmoData.jl leverages native Julia capabilities to enable compact syntax, scalable computations, and interfaces with diverse packages., Comment: 62 pages, 18 figures, 8 tables

The Euler characteristic (EC) is a powerful topological descriptor that can be used to quantify the shape of data objects that are represented as fields/manifolds. Fast methods for computing the EC are required to enable processing of high-throughput data and real-time implementations. This represents a challenge when processing high-resolution 2D field data (e.g., images) and 3D field data (e.g., video, hyperspectral images, and space-time data obtained from fluid dynamics and molecular simulations). In this work, we present parallel algorithms (and software implementations) to enable fast computations of the EC for 2D and 3D fields using vertex contributions. We test the proposed algorithms using synthetic data objects and data objects arising in real applications such as microscopy, 3D molecular dynamics simulations, and hyperspectral images. Results show that the proposed implementation can compute the EC a couple of orders of magnitude faster than ${\tt GUDHI}$ (an off-the-shelf and state-of-the art tool) and at speeds comparable to ${\tt CHUNKYEuler}$ (a tool tailored to scalable computation of the EC). The vertex contributions approach is flexible in that it compute the EC as well as other topological descriptors such as perimeter, area, and volume (${\tt CHUNKYEuler}$ can only compute the EC). Scalability with respect to memory use is also addressed by providing low-memory versions of the algorithms; this enables processing of data objects beyond the size of dynamic memory. All data and software needed for reproducing the results are shared as open-source code.

Bayesian optimization (BO) has proven to be an effective paradigm for the global optimization of expensive-to-sample systems. One of the main advantages of BO is its use of Gaussian processes (GPs) to characterize model uncertainty which can be leveraged to guide the learning and search process. However, BO typically treats systems as black-boxes and this limits the ability to exploit structural knowledge (e.g., physics and sparse interconnections). Composite functions of the form $f(x, y(x))$, wherein GP modeling is shifted from the performance function $f$ to an intermediate function $y$, offer an avenue for exploiting structural knowledge. However, the use of composite functions in a BO framework is complicated by the need to generate a probability density for $f$ from the Gaussian density of $y$ calculated by the GP (e.g., when $f$ is nonlinear it is not possible to obtain a closed-form expression). Previous work has handled this issue using sampling techniques; these are easy to implement and flexible but are computationally intensive. In this work, we introduce a new paradigm which allows for the efficient use of composite functions in BO; this uses adaptive linearizations of $f$ to obtain closed-form expressions for the statistical moments of the composite function. We show that this simple approach (which we call BOIS) enables the exploitation of structural knowledge, such as that arising in interconnected systems as well as systems that embed multiple GP models and combinations of physics and GP models. Using a chemical process optimization case study, we benchmark the effectiveness of BOIS against standard BO and sampling approaches. Our results indicate that BOIS achieves performance gains and accurately captures the statistics of composite functions., Comment: 6 pages, 5 figures

Industrial sectors such as urban centers, chemical companies, manufacturing facilities, and microgrids are actively exploring strategies to help reduce their carbon footprint. For instance, university campuses are complex urban districts (involving collections of buildings and utility systems) that are seeking to reduce carbon footprints that originate from diverse activities (e.g., transportation operations and production of heating, cooling, and power utilities). This work presents an optimization framework to identify technology pathways that enable decarbonization of complex industrial sectors. The framework uses a graph abstraction that compactly captures interdependencies between diverse products and technologies as well as diverse externalities (e.g., market, policy, and carbon prices). Duality analysis reveals that the formulation can be interpreted as an economy, market, or value chain that uses technologies to generate economic value (wealth) by transforming basic products into higher value products. This interpretation also reveals that the formulation identifies pathways that maximize the profit of stakeholders, helps reveal the inherent value (prices) of intermediate products, and helps analyze the impact of externalities and technology specifications on product values. Our developments are illustrated via a case study involving a prototypical university campus that seeks to identify pathways that reduce its carbon footprint (e.g., via electrification and deployment of hydrogen technologies). We use the framework to determine carbon tax values, technology specifications, and investment budgets that activate different technology pathways and that achieve different levels of decarbonization.

Hierarchical optimization architectures are used in power systems to manage disturbances and phenomena that arise at multiple spatial and temporal scales. We present a graph modeling abstraction for representing such architectures and an implementation in the ${\tt Julia}$ package ${\tt Plasmo.jl}$. We apply this framework to a tri-level hierarchical framework arising in wholesale market operations that involves day-ahead unit commitment, short-term unit commitment, and economic dispatch. We show that graph abstractions facilitate the construction, visualization, and solution of these complex problems., Comment: 7 pages, 9 figures

Graph Neural Networks (GNNs) have emerged as a prominent class of data-driven methods for molecular property prediction. However, a key limitation of typical GNN models is their inability to quantify uncertainties in the predictions. This capability is crucial for ensuring the trustworthy use and deployment of models in downstream tasks. To that end, we introduce AutoGNNUQ, an automated uncertainty quantification (UQ) approach for molecular property prediction. AutoGNNUQ leverages architecture search to generate an ensemble of high-performing GNNs, enabling the estimation of predictive uncertainties. Our approach employs variance decomposition to separate data (aleatoric) and model (epistemic) uncertainties, providing valuable insights for reducing them. In our computational experiments, we demonstrate that AutoGNNUQ outperforms existing UQ methods in terms of both prediction accuracy and UQ performance on multiple benchmark datasets. Additionally, we utilize t-SNE visualization to explore correlations between molecular features and uncertainty, offering insight for dataset improvement. AutoGNNUQ has broad applicability in domains such as drug discovery and materials science, where accurate uncertainty quantification is crucial for decision-making.

The power grid is undergoing significant restructuring driven by the adoption of wind/solar power and the incorporation of new flexible technologies that can shift load in space and time (e.g., data centers, battery storage, and modular manufacturing). Load shifting is needed to mitigate space-time fluctuations associated with wind/solar power and other disruptions (e.g., extreme weather). The impact of load shifting on electricity markets is typically quantified via sensitivity analysis, which aims to assess impacts in terms of price volatility and total welfare. This sensitivity approach does not explicitly quantify operational flexibility (e.g., range or probability of feasible operation). In this work, we present a computational framework to enable this; specifically, we quantify operational flexibility by assessing how much uncertainty in net loads (which capture uncertain power injections/withdrawals) can be tolerated by the system under varying levels of load shifting capacity. The proposed framework combines optimization formulations that quantify operational flexibility with power grid models that capture load shifting in the form of virtual links (pathways that transfer load across space-time). Our case studies reveal that adding a single virtual link that shifts load in either space or time can lead to dramatic improvements in system-wide flexibility; this is because shifting relieves space-time congestion that results from transmission constraints and generator ramping constraints. Our results provide insights into how the incorporation of flexible technologies can lead to non-intuitive, system-wide gains in flexibility., Comment: 20 pages, 11 figures

Anthropogenic pollution of hydrological systems affects diverse communities and ecosystems around the world. Data analytics and modeling tools play a key role in fighting this challenge, as they can help identify key sources as well as trace transport and quantify impact within complex hydrological systems. Several tools exist for simulating and tracing pollutant transport throughout surface waters using detailed physical models; these tools are powerful, but can be computationally intensive, require significant amounts of data to be developed, and require expert knowledge for their use (ultimately limiting application scope). In this work, we present a graph modeling framework -- which we call ${\tt HydroGraphs}$ -- for understanding pollutant transport and fate across waterbodies, rivers, and watersheds. This framework uses a simplified representation of hydrological systems that can be constructed based purely on open-source data (National Hydrography Dataset and Watershed Boundary Dataset). The graph representation provides an flexible intuitive approach for capturing connectivity and for identifying upstream pollutant sources and for tracing downstream impacts within small and large hydrological systems. Moreover, the graph representation can facilitate the use of advanced algorithms and tools of graph theory, topology, optimization, and machine learning to aid data analytics and decision-making. We demonstrate the capabilities of our framework by using case studies in the State of Wisconsin; here, we aim to identify upstream nutrient pollutant sources that arise from agricultural practices and trace downstream impacts to waterbodies, rivers, and streams. Our tool ultimately seeks to help stakeholders design effective pollution prevention/mitigation practices and evaluate how surface waters respond to such practices., Comment: 41 pages, 9 figures; minor update to analysis (e.g., urban land cover to case studies; overall results remain unchanged)

The real-time operation of large-scale infrastructure networks requires scalable optimization capabilities. Decomposition schemes can help achieve scalability; classical decomposition approaches such as the alternating direction method of multipliers (ADMM) and distributed Newtons schemes, however, often either suffer from slow convergence or might require high degrees of communication. In this work, we present new primal decomposition schemes for solving large-scale, strongly convex QPs. These approaches have global convergence guarantees and require limited communication. We benchmark their performance against the off-the-shelf interior-point method Ipopt and against ADMM on infrastructure networks that contain up to 300,000 decision variables and constraints. Overall, we find that the proposed approaches solve problems as fast as Ipopt but with reduced communication. Moreover, we find that the proposed schemes achieve higher accuracy than ADMM approaches.

We discuss basic concepts of convolutional neural networks (CNNs) and outline uses in manufacturing. We begin by discussing how different types of data objects commonly encountered in manufacturing (e.g., time series, images, micrographs, videos, spectra, molecular structures) can be represented in a flexible manner using tensors and graphs. We then discuss how CNNs use convolution operations to extract informative features (e.g., geometric patterns and textures) from the such representations to predict emergent properties and phenomena and/or to identify anomalies. We also discuss how CNNs can exploit color as a key source of information, which enables the use of modern computer vision hardware (e.g., infrared, thermal, and hyperspectral cameras). We illustrate the concepts using diverse case studies arising in spectral analysis, molecule design, sensor design, image-based control, and multivariate process monitoring.

Bayesian optimization (BO) is one of the most effective methods for closed-loop experimental design and black-box optimization. However, a key limitation of BO is that it is an inherently sequential algorithm (one experiment is proposed per round) and thus cannot directly exploit high-throughput (parallel) experiments. Diverse modifications to the BO framework have been proposed in the literature to enable exploitation of parallel experiments but such approaches are limited in the degree of parallelization that they can achieve and can lead to redundant experiments (thus wasting resources and potentially compromising performance). In this work, we present new parallel BO paradigms that exploit the structure of the system to partition the design space. Specifically, we propose an approach that partitions the design space by following the level sets of the performance function and an approach that exploits partially-separable structures of the performance function found. We conduct extensive numerical experiments using a reactor case study to benchmark the effectiveness of these approaches against a variety of state-of-the-art parallel algorithms reported in the literature. Our computational results show that our approaches significantly reduce the required search time and increase the probability of finding a global (rather than local) solution., Comment: 36 pages, 16 figures, 8 algorithms

We report numerical results on solving constrained linear-quadratic model predictive control (MPC) problems by exploiting graphics processing units (GPUs). The presented method reduces the MPC problem by eliminating the state variables and applies a condensed-space interior-point method to remove the inequality constraints in the KKT system. The final condensed matrix is positive definite and can be efficiently factorized in parallel on GPU/SIMD architectures. In addition, the size of the condensed matrix depends only on the number of controls in the problem, rendering the method particularly effective when the problem has many states but few inputs and moderate horizon length. Our numerical results for PDE-constrained problems show that the approach is an order of magnitude faster than a standard CPU implementation. We also provide an open-source Julia framework that facilitates modeling (DynamicNLPModels.jl) and solution (MadNLP.jl) of MPC problems on GPUs.

Ambitious renewable portfolio standards motivate the incorporation of energy storage resources (ESR) as sources of flexibility. While the United States government aims to promote ESR participation in electricity markets, work on market designs for properly remunerating ESRs is still lacking. In this paper, we propose a new energy market clearing framework that incorporates ESR systems. The new market design is computationally attractive in that it avoids mixed-integer formulations and formulations with complementarity constraints. Moreover, compared to previous market designs, our market decomposes the operations of ESRs using the concept of virtual links, which capture the transfer of energy across time. The virtual link representation reveals economic incentives available for ESR operations and sheds light into how electricity markets should remunerate ESRs. We also explore the role of ESR physical parameters on market behavior; we show that, while energy and power capacity defines the amount of flexibility each ESR can provide, storage charge/discharge efficiencies play a fundamental role in ESR remuneration and in mitigating market price volatility. We use our proposed framework to analyze the interplay between ESRs and independent system operators (ISOs) and to provide insights into optimal deployment strategies of ESRs in power grids., Comment: 28 pages, 8 figures

Data centers (DCs) can help decarbonize the power grid by helping absorb renewable power (e.g., wind and solar) due to their ability to shift power loads across space and time. However, to harness such load-shifting flexibility, it is necessary to understand how grid signals (carbon signals and market price/load allocations) affect DC operations. An obstacle that arises here is the lack of computationally-tractable DC operation models that can capture objectives, constraints, and information flows that arise at the interface of DCs and the power grid. To address this gap, we present a receding-horizon resource management model (a mixed-integer programming model) that captures the resource management layer between the DC scheduler and the grid while accounting for logical constraints, different types of objectives, and forecasts of incoming job profiles and of available computing capacity. We use our model to conduct extensive case studies based on public data from Microsoft Azure and MISO. Our studies show that DCs can provide significant temporal load-shifting flexibility that results in reduced carbon emissions and peak demand charges. Models and case studies are shared as easy-to-use Julia code., Comment: 12 pages, 11 figures

We explore the use of tools from Riemannian geometry for the analysis of symmetric positive definite matrices (SPD). An SPD matrix is a versatile data representation that is commonly used in chemical engineering (e.g., covariance/correlation/Hessian matrices and images) and powerful techniques are available for its analysis (e.g., principal component analysis). A key observation that motivates this work is that SPD matrices live on a Riemannian manifold and that implementing techniques that exploit this basic property can yield significant benefits in data-centric tasks such classification and dimensionality reduction. We demonstrate this via a couple of case studies that conduct anomaly detection in the context of process monitoring and image analysis., Comment: 18 pages, 10 figures

We present a novelty detection framework for Convolutional Neural Network (CNN) sensors that we call Sensor-Activated Feature Extraction One-Class Classification (SAFE-OCC). We show that this framework enables the safe use of computer vision sensors in process control architectures. Emergent control applications use CNN models to map visual data to a state signal that can be interpreted by the controller. Incorporating such sensors introduces a significant system operation vulnerability because CNN sensors can exhibit high prediction errors when exposed to novel (abnormal) visual data. Unfortunately, identifying such novelties in real-time is nontrivial. To address this issue, the SAFE-OCC framework leverages the convolutional blocks of the CNN to create an effective feature space to conduct novelty detection using a desired one-class classification technique. This approach engenders a feature space that directly corresponds to that used by the CNN sensor and avoids the need to derive an independent latent space. We demonstrate the effectiveness of SAFE-OCC via simulated control environments.

We present a new modeling paradigm for optimization that we call random field optimization. Random fields are a powerful modeling abstraction that aims to capture the behavior of random variables that live on infinite-dimensional spaces (e.g., space and time) such as stochastic processes (e.g., time series, Gaussian processes, and Markov processes), random matrices, and random spatial fields. This paradigm involves sophisticated mathematical objects (e.g., stochastic differential equations and space-time kernel functions) and has been widely used in neuroscience, geoscience, physics, civil engineering, and computer graphics. Despite of this, however, random fields have seen limited use in optimization; specifically, existing optimization paradigms that involve uncertainty (e.g., stochastic programming and robust optimization) mostly focus on the use of finite random variables. This trend is rapidly changing with the advent of statistical optimization (e.g., Bayesian optimization) and multi-scale optimization (e.g., integration of molecular sciences and process engineering). Our work extends a recently-proposed abstraction for infinite-dimensional optimization problems by capturing more general uncertainty representations. Moreover, we discuss solution paradigms for this new class of problems based on finite transformations and sampling, and identify open questions and challenges.

We present a graph sampling and coarsening scheme (gSC) for computing lower and upper bounds for large-scale supply chain models. An edge sampling scheme is used to build a low-complexity problem that is used to finding an approximate (but feasible) solution for the original model and to compute a lower bound (for a maximization problem). This scheme is similar in spirit to the so-called sample average approximation scheme, which is widely used for the solution of stochastic programs. A graph coarsening (aggregation) scheme is used to compute an upper bound and to estimate the optimality gap of the approximate solution. The coarsening scheme uses node sampling to select a small set of support nodes that are used to guide node/edge aggregation and we show that the coarsened model provides a relaxation of the original model and a valid upper bound. We provide numerical evidence that gSC can yield significant improvements in solution time and memory usage over state-of-the-art solvers. Specifically, we study a supply chain design model (a mixed-integer linear program) that contains over 38 million variables and show that gSC finds a solution with an optimality gap of <0.5% in less than 22 minutes., Comment: 23 pages, 12 figures

We propose a new class of measures for shaping time-dependent trajectories in dynamic optimization (DO). The proposed measures are analogous to risk measures used in stochastic optimization (SO) and are inspired by a recently-proposed unifying abstraction for infinite-dimensional optimization. Risk measures are summarizing statistics (e.g., average, variance, quantiles, worst-case values) that are used to shape the probability density of random objectives and constraints. We show that this extensive collection of measures can be applied in DO for computing and manipulating interesting features of time-dependent trajectories (e.g., excursion costs and quantiles). We also discuss how to implement these measures in the Julia modeling package InfiniteOpt.jl.

In this work, we analyze the spatio-temporal economic properties of multi-product supply chains. Specifically, we interpret the supply chain as a coordinated market in which stakeholders (suppliers, consumers, and providers of transportation, storage, and transformation services) bid into a market that is cleared by an independent entity to obtain allocations and prices. The proposed model provides a general graph representation of spatio-temporal product transport that helps capture geographical transport, time delays, and storage (temporal transport) in a unified and compact manner. This representation allows us to establish fundamental economic properties for the supply chain (revenue adequacy, cost recovery, and competitiveness) and to establish bounds for space-time prices. To illustrate the concepts, we consider a case study in which organic waste is used for producing biogas and electricity. Our market model shows that incentives for waste storage emerge from electricity demand dynamics and illustrates how space-time price dynamics for waste and derived products emerge from geographical transport and storage., Comment: 32 pages, 15 figures

We propose a computational framework to quantify (measure) and to optimize the reliability of complex systems. The approach uses a graph representation of the system that is subject to random failures of its components (nodes and edges). Under this setting, reliability is defined as the probability of finding a path between sources and sink nodes under random component failures and we show that this measure can be computed by solving a stochastic mixed-integer program. The stochastic programming setting allows us to account for system constraints and general probability distributions to characterize failures and allows us to derive optimization formulations that identify designs of maximum reliability. We also propose a strategy to approximately solve these problems in a scalable manner by using purely continuous formulations.

We present a computational framework for analyzing and quantifying system flexibility. Our framework incorporates new features that include: general uncertainty characterizations that are constructed using composition of sets, procedures for computing well-centered nominal points, and a procedure for identifying and ranking flexibility-limiting constraints and critical parameter values. These capabilities allow us to analyze the flexibility of complex systems such as distribution networks.

We study the problem of designing systems in order to minimize cost while meeting a given flexibility target. Flexibility is attained by enforcing a joint chance constraint, which ensures that the system will exhibit feasible operation with a given target probability level. Unfortunately, joint chance constraints are complicated mathematical objects that often need to be reformulated using mixed-integer programming (MIP) techniques. In this work, we cast the design problem as a conflict resolution problem that seeks to minimize cost while maximizing flexibility. We propose a purely continuous relaxation of this problem that provides a significantly more scalable approach relative to MIP methods and show that the formulation delivers solutions that closely approximate the Pareto set of the original joint chance-constrained problem.

We present a methodology for computing the flexibility index when uncertainty is characterized using multivariate Gaussian random variables. Our approach computes the flexibility index by solving a mixed-integer conic program (MICP). This methodology directly characterizes ellipsoidal sets to capture correlations in contrast to previous methodologies that employ approximations. We also show that, under a Gaussian representation, the flexibility index can be used to obtain a lower bound for the so-called stochastic flexibility index (i.e., the probability of having feasible operation). Our results also show that the methodology can be generalized to capture different types of uncertainty sets.

Infinite-dimensional optimization (InfiniteOpt) problems involve modeling components (variables, objectives, and constraints) that are functions defined over infinite-dimensional domains. Examples include continuous-time dynamic optimization (time is an infinite domain and components are a function of time), PDE optimization problems (space and time are infinite domains and components are a function of space-time), as well as stochastic and semi-infinite optimization (random space is an infinite domain and components are a function of such random space). InfiniteOpt problems also arise from combinations of these problem classes (e.g., stochastic PDE optimization). Given the infinite-dimensional nature of objectives and constraints, one often needs to define appropriate quantities (measures) to properly pose the problem. Moreover, InfiniteOpt problems often need to be transformed into a finite dimensional representation so that they can be solved numerically. In this work, we present a unifying abstraction that facilitates the modeling, analysis, and solution of InfiniteOpt problems. The proposed abstraction enables a general treatment of infinite-dimensional domains and provides a measure-centric paradigm to handle associated variables, objectives, and constraints. This abstraction allows us to transfer techniques across disciplines and with this identify new, interesting, and useful modeling paradigms (e.g., event constraints and risk measures defined over time domains). Our abstraction serves as the backbone of an intuitive Julia-based modeling package that we call InfiniteOpt.jl. We demonstrate the developments using diverse case studies arising in engineering.

We study an electricity market clearing formulation that seeks to remunerate spatio-temporal, load-shifting flexibility provided by data centers (DaCes). Load-shifting flexibility is a key asset for power grid operators as they aim to integrate larger amounts of intermittent renewable power and to decarbonize the grid. Central to our study is the concept of virtual links, which provide non-physical pathways that can be used by DaCes to shift power loads (by shifting computing loads) across space and time. We use virtual links to show that the clearing formulation treats DaCes as prosumers that simultaneously request load and provide a load-shifting flexibility service. Our analysis also reveals that DaCes are remunerated for the provision of load-shifting flexibility based on nodal price differences (across space and time). We also show that DaCe flexibility helps relieve space-time price volatility and show that the clearing formulation satisfies fundamental economic properties that are expected from coordinated markets (e.g., provides a competitive equilibrium and achieves revenue adequacy and cost recovery). The concepts presented are applicable to other key market players that can offer space-time shifting flexibility such as distributed manufacturing facilities and storage systems. Case studies are presented to demonstrate these properties.

We study tightness properties of a Lagrangian dual (LD) bound for the nonconvex alternating current optimal power flow (ACOPF) problem. We show an LD bound that can be computed in a parallel, decentralized manner. Specifically, the proposed approach partitions the network into a set of subnetworks, dualizes the coupling constraints (giving the LD function), and maximizes the LD function with respect to the dual variables of the coupling constraints (giving the desired LD bound). The dual variables that maximize the LD are obtained by using a proximal bundle method. We show that the bound is less tight than the popular semidefinite programming relaxation but as tight as the second-order cone programming relaxation. We demonstrate our developments using PGLib-OPF test instances.

We study a property of dynamic optimization (DO) problems (as those encountered in model predictive control and moving horizon estimation) that is known as exponential decay of sensitivity (EDS). This property indicates that the sensitivity of the solution at stage $i$ against a data perturbation at stage $j$ decays exponentially with $|i-j|$. {Building upon our previous results, we show that EDS holds under uniform boundedness of the Lagrangian Hessian, a uniform second order sufficiency condition (uSOSC), and a uniform linear independence constraint qualification (uLICQ). Furthermore, we prove that uSOSC and uLICQ can be obtained under uniform controllability and observability. Hence, we have that uniform controllability and observability imply EDS.} These results provide insights into how perturbations propagate along the horizon and enable the development of approximation and solution schemes. We illustrate the developments with numerical examples.

In this paper we review the mathematical foundations of convolutional neural nets (CNNs) with the goals of: i) highlighting connections with techniques from statistics, signal processing, linear algebra, differential equations, and optimization, ii) demystifying underlying computations, and iii) identifying new types of applications. CNNs are powerful machine learning models that highlight features from grid data to make predictions (regression and classification). The grid data object can be represented as vectors (in 1D), matrices (in 2D), or tensors (in 3D or higher dimensions) and can incorporate multiple channels (thus providing high flexibility in the input data representation). CNNs highlight features from the grid data by performing convolution operations with different types of operators. The operators highlight different types of features (e.g., patterns, gradients, geometrical features) and are learned by using optimization techniques. In other words, CNNs seek to identify optimal operators that best map the input data to the output data. A common misconception is that CNNs are only capable of processing image or video data but their application scope is much wider; specifically, datasets encountered in diverse applications can be expressed as grid data. Here, we show how to apply CNNs to new types of applications such as optimal control, flow cytometry, multivariate process monitoring, and molecular simulations.

We study solution sensitivity for nonlinear programs (NLPs) whose structures are induced by graphs. These NLPs arise in many applications such as dynamic optimization, stochastic optimization, optimization with partial differential equations, and network optimization. We show that for a given pair of nodes, the sensitivity of the primal-dual solution at one node against a data perturbation at the other node decays exponentially with respect to the distance between these two nodes on the graph. In other words, the solution sensitivity decays as one moves away from the perturbation point. This result, which we call exponential decay of sensitivity, holds under the strong second-order sufficiency condition and the linear independence constraint qualification. We also present conditions under which the decay rate remains uniformly bounded; this allows us to characterize the sensitivity behavior of NLPs defined over subgraphs of infinite graphs. The theoretical developments are illustrated with numerical examples.

Reinforcement learning (RL) is attracting attention as an effective way to solve sequential optimization problems that involve high dimensional state/action space and stochastic uncertainties. Many such problems involve constraints expressed by inequality constraints. This study focuses on using RL to solve constrained optimal control problems. Most RL application studies have dealt with inequality constraints by adding soft penalty terms for violating the constraints to the reward function. However, while training neural networks to learn the value (or Q) function, one can run into computational issues caused by the sharp change in the function value at the constraint boundary due to the large penalty imposed. This difficulty during training can lead to convergence problems and ultimately lead to poor closed-loop performance. To address this issue, this study proposes a dynamic penalty (DP) approach where the penalty factor is gradually and systematically increased during training as the iteration episodes proceed. We first examine the ability of a neural network to represent a value function when uniform, linear, or DP functions are added to prevent constraint violation. The agent trained by a Deep Q Network (DQN) algorithm with the DP function approach was compared with agents with other constant penalty functions in a simple vehicle control problem. Results show that the proposed approach can improve the neural network approximation accuracy and provide faster convergence when close to a solution., Comment: Submitted to ADCHEM 2021

We study logistical investment flexibility provided by modular processing technologies for mitigating risk. Specifically, we propose a multi-stage stochastic programming formulation that determines optimal capacity expansion plans that mitigate demand uncertainty. The formulation accounts for multi-product dependencies between small/large units and for trade-offs between expected profit and risk. The formulation uses a cumulative risk measure to avoid timeconsistency issues of traditional, per-stage risk-minimization formulations and we argue that this approach is more compatible with typical investment metrics such as the net present value. Case studies of different complexity are presented to illustrate the developments. Our studies reveal that the Pareto frontier of a flexible setting (allowing for deployment of small units) dominates the Pareto frontier of an inflexible setting (allowing only for deployment of large units). Notably, this dominance is prevalent despite benefits arising from economies of scale of large processing units.

Motivation: Estimating model parameters from experimental observations is one of the key challenges in systems biology and can be computationally very expensive. While the Julia programming language was recently developed as a high-level and high-performance language for scientific computing, systems biologists have only started to realise its potential. For instance, we have recently used Julia to cut down the optimization time of a microbial community model by a factor of 140. To facilitate access of the systems biology community to the efficient nonlinear solvers used for this optimisation, we developed SBML2Julia. SBML2Julia translates optimisation problems specified in SBML and TSV files (PEtab format) into Julia for Mathematical Programming (JuMP), executes the optimization and returns the results in tabular format. Availability and implementation: SBML2Julia is freely available under the MIT license. It comes with a command line interface and Python API. Internally, SBML2Julia calls the Julia LTS release v1.0.5 for optimisation. All necessary dependencies can be pulled from Docker Hub (https://hub.docker.com/repository/docker/paulflang/sbml2julia). Source code and documentation are available at https://github.com/paulflang/SBML2Julia., Comment: 4 pages, 1 figure

Nonlinear optimization problems are found at the heart of real-time operations of critical infrastructures. These problems are computationally challenging because they embed complex physical models that exhibit space-time dynamics. We propose modeling these problems as graph-structured optimization problems, and illustrate how their structure can be exploited at the modeling level (for parallelizing function/derivative computations) and at the solver level (for parallelizing linear algebra operations). Specifically, we present a restricted additive Schwarz scheme that enables flexible decomposition of complex graph structures within an interior-point algorithm. The proposed approach is implemented as a general-purpose nonlinear programming solver that we call MadNLP.jl; this Julia-based solver is interfaced to the graph-based modeling package Plasmo.jl. The efficiency of this framework is demonstrated via problems arising in transient gas network optimization and multi-period AC optimal power flow. We show that our framework accelerates the solution (compared to off-the-shelf tools) by over 300%; specifically, solution times are reduced from 72.36 sec to 23.84 sec for the gas problem and from 515.81 sec to 149.45 sec for the power flow problem.

We present a Bayesian optimization (BO) framework for tuning model predictive controllers (MPC) of central heating, ventilation, and air conditioning (HVAC) plants. This approach treats the functional relationship between the closed-loop performance of MPC and its tuning parameters as a black-box. The approach is motivated by the observation that evaluating the closed-loop performance of MPC by trial-and-error is time-consuming (e.g., every closed-loop simulation can involve solving thousands of optimization problems). The proposed BO framework seeks to quickly identify the optimal tuning parameters by strategically exploring and exploiting the space of the tuning parameters. The effectiveness of the BO framework is demonstrated by using an MPC controller for a central HVAC plant using realistic data. Here, the BO framework tunes back-off terms for thermal storage tanks to minimize year-long closed-loop costs. Simulation results show that BO can find the optimal back-off terms by conducting 13 year-long simulations, which significantly reduces the computational burden of a naive grid search. We also find that the back-off terms obtained with BO reduce the closed-loop costs., Comment: 7 pages, 7 figures, conference

We propose a dual dynamic integer programming (DDIP) framework for solving multi-scale mixed-integer model predictive control (MPC) problems. Such problems arise in applications that involve long horizons and/or fine temporal discretizations as well as mixed-integer states and controls (e.g., scheduling logic and discrete actuators). The approach uses a nested cutting-plane scheme that performs forward and backward sweeps along the time horizon to adaptively approximate cost-to-go functions. The DDIP scheme proposed can handle general MPC formulations with mixed-integer controls and states and can perform forward-backward sweeps over block time partitions. We demonstrate the performance of the proposed scheme by solving mixed-integer MPC problems that arise in the scheduling of central heating, ventilation, and air-conditioning (HVAC) plants. We show that the proposed scheme is scalable and dramatically outperforms state-of-the-art mixed-integer solvers., Comment: 32 pages, 13 figures. arXiv admin note: text overlap with arXiv:2002.10065

We present a data-driven model predictive control (MPC) framework for systems with high state-space dimensionalities. This work is motivated by the need to exploit sensor data that appears in the form of images (e.g., 2D or 3D spatial fields reported by thermal cameras). We propose to use dynamic mode decomposition (DMD) to directly build a low-dimensional model from image data and we use such model to obtain a tractable MPC controller. We demonstrate the scalability of this approach (which we call DMD-MPC) by using a 2D thermal diffusion system. Here, we assume that the evolution of the thermal field is captured by 50x50 pixel images, which results in a 2500-dimensional state-space. We show that the dynamics of this high-dimensional space can be accurately predicted by using a 40-dimensional DMD model and we show that the field can be manipulated satisfactorily by using an MPC controller that embeds the low-dimensional DMD model. We also show that the DMD-MPC controller significantly outperforms a standard MPC controller that uses data from a finite set of spatial locations (proxy locations) to manipulate the high-dimensional thermal field. This comparison illustrates the value of information embedded in image data., Comment: 22 pages, 14 figures

We present a general graph-based modeling abstraction for optimization that we call an OptiGraph. Under this abstraction, any optimization problem is treated as a hierarchical hypergraph in which nodes represent optimization subproblems and edges represent connectivity between such subproblems. The abstraction enables the modular construction of highly complex models in an intuitive manner, facilitates the use of graph analysis tools (to perform partitioning, aggregation, and visualization tasks), and facilitates communication of structures to decomposition algorithms. We provide an open-source implementation of the abstraction in the Julia-based package Plasmo.jl. We provide tutorial examples and large application case studies to illustrate the capabilities.

We interpret multi-product supply chains (SCs) as coordinated markets; under this interpretation, a SC optimization problem is a market clearing problem that allocates resources and associated economic values (prices) to different stakeholders that bid into the market (suppliers, consumers, transportation, and processing technologies). The market interpretation allows us to establish fundamental properties that explain how physical resources (primal variables) and associated economic values (dual variables) flow in the SC. We use duality theory to explain why incentivizing markets by forcing stakeholder participation (e.g., by imposing demand satisfaction or service provision constraints) yields artificial price behavior, inefficient allocations, and economic losses. To overcome these issues, we explore market incentive mechanisms that use bids; here, we introduce the concept of a stakeholder graph (a product-based representation of a supply chain) and show that this representation allows us to naturally determine minimum bids that activate the market. These results provide guidelines to design SC formulations that properly remunerate stakeholders and to design policy that foster market transactions. The results are illustrated using an urban waste management problem for a city of 100,000 residents., Comment: 43 pages, 9 figures

A primary hypothesis that drives scientific and engineering studies is that data has structure. The dominant paradigms for describing such structure are statistics (e.g., moments, correlation functions) and signal processing (e.g., convolutional neural nets, Fourier series). Topological Data Analysis (TDA) is a field of mathematics that analyzes data from a fundamentally different perspective. TDA represents datasets as geometric objects and provides dimensionality reduction techniques that project such objects onto low-dimensional spaces that are composed of elementary geometric objects. Key property of these elementary objects (also known as topological features) are that they persist at different scales and that they are stable under perturbations (e.g., noise, stretching, twisting, and bending). In this work, we review key mathematical concepts and methods of TDA and present different applications in chemical engineering., Comment: 42 pages, 44 figures

We study the convergence properties of an overlapping Schwarz decomposition algorithm for solving nonlinear optimal control problems (OCPs). The algorithm decomposes the time domain into a set of overlapping subdomains, and solves all subproblems defined over subdomains in parallel. The convergence is attained by updating primal-dual information at the boundaries of overlapping subdomains. We show that the algorithm exhibits local linear convergence, and that the convergence rate improves exponentially with the overlap size. We also establish global convergence results for a general quadratic programming, which enables the application of the Schwarz scheme inside second-order optimization algorithms (e.g., sequential quadratic programming). The theoretical foundation of our convergence analysis is a sensitivity result of nonlinear OCPs, which we call "exponential decay of sensitivity" (EDS). Intuitively, EDS states that the impact of perturbations at domain boundaries (i.e. initial and terminal time) on the solution decays exponentially as one moves into the domain. Here, we expand a previous analysis available in the literature by showing that EDS holds for both primal and dual solutions of nonlinear OCPs, under uniform second-order sufficient condition, controllability condition, and boundedness condition. We conduct experiments with a quadrotor motion planning problem and a PDE control problem to validate our theory; and show that the approach is significantly more efficient than ADMM and as efficient as the centralized solver Ipopt., Comment: 16 pages

We propose a multiscale model predictive control (MPC) framework for stationary battery systems that exploits high-fidelity models to trade-off short-term economic incentives provided by energy and frequency regulation (FR) markets and long-term degradation effects. We find that the MPC framework can drastically reduce long-term degradation while properly responding to FR and energy market signals (compared to MPC formulations that use low-fidelity models). Our results also provide evidence that sophisticated battery models can be embedded within closedloop MPC simulations by using modern nonlinear programming solvers (we provide an efficient and easy-to-use implementation in Julia). We use insights obtained with our simulations to design a low-complexity MPC formulation that matches the behavior obtained with high-fidelity models. This is done by designing a suitable terminal penalty term that implicitly captures longterm degradation. The results suggest that complex degradation behavior can be accounted for in low-complexity MPC formulations by properly designing the cost function. We believe that our proof-of-concept results can be of industrial relevance, as battery vendors are seeking to participate in fast-changing electricity markets while maintaining asset integrity.