Search

Maximum Likelihood Estimation of continuous variable models can be very challenging in high dimensions, due to potentially complex probability distributions. The existence of multiple interdependencies among variables can make it very difficult to establish convergence guarantees. This leads to a wide use of brute-force methods, such as grid searching and Monte-Carlo sampling and, when applicable, complex and problem-specific algorithms. In this paper, we consider inference problems where the variables are related by multiaffine expressions. We propose a novel Alternating and Iteratively-Reweighted Least Squares (AIRLS) algorithm, and prove its convergence for problems with Generalized Normal Distributions. We also provide an efficient method to compute the variance of the estimates obtained using AIRLS. Finally, we show how the method can be applied to graphical statistical models. We perform numerical experiments on several inference problems, showing significantly better performance than state-of-the-art approaches in terms of scalability, robustness to noise, and convergence speed due to an empirically observed super-linear convergence rate.

We consider a recommender system that takes into account the interaction between recommendations and the evolution of user interests. Users opinions are influenced by both social interactions and recommended content. We leverage online feedback optimization to design a recommender system that trades-off between maximizing engagement and minimizing polarization. The recommender system is agnostic about users' opinion, clicking behavior, and social interactions, and solely relies on clicks. We establish optimality and closed-loop stability of the resulting feedback interconnection between the social platform and the recommender system. We numerically validate our algorithm when the user population follows an extended Friedkin--Johnsen model. We observe that network-aware recommendations significantly reduce polarization without compromising user engagement.

Diffusion regulates a phenomenal number of natural processes and the dynamics of many successful generative models. Existing models to learn the diffusion terms from observational data rely on complex bilevel optimization problems and properly model only the drift of the system. We propose a new simple model, JKOnet*, which bypasses altogether the complexity of existing architectures while presenting significantly enhanced representational capacity: JKOnet* recovers the potential, interaction, and internal energy components of the underlying diffusion process. JKOnet* minimizes a simple quadratic loss, runs at lightspeed, and drastically outperforms other baselines in practice. Additionally, JKOnet* provides a closed-form optimal solution for linearly parametrized functionals. Our methodology is based on the interpretation of diffusion processes as energy-minimizing trajectories in the probability space via the so-called JKO scheme, which we study via its first-order optimality conditions, in light of few-weeks-old advancements in optimization in the probability space.

We study optimization problems whereby the optimization variable is a probability measure. Since the probability space is not a vector space, many classical and powerful methods for optimization (e.g., gradients) are of little help. Thus, one typically resorts to the abstract machinery of infinite-dimensional analysis or other ad-hoc methodologies, not tailored to the probability space, which however involve projections or rely on convexity-type assumptions. We believe instead that these problems call for a comprehensive methodological framework for calculus in probability spaces. In this work, we combine ideas from optimal transport, variational analysis, and Wasserstein gradient flows to equip the Wasserstein space (i.e., the space of probability measures endowed with the Wasserstein distance) with a variational structure, both by combining and extending existing results and introducing novel tools. Our theoretical analysis culminates in very general necessary optimality conditions for optimality. Notably, our conditions (i) resemble the rationales of Euclidean spaces, such as the Karush-Kuhn-Tucker and Lagrange conditions, (ii) are intuitive, informative, and easy to study, and (iii) yield closed-form solutions or can be used to design computationally attractive algorithms. We believe this framework lays the foundation for new algorithmic and theoretical advancements in the study of optimization problems in probability spaces, which we exemplify with numerous case studies and applications to machine learning, drug discovery, and distributionally robust optimization.

We study the problem of nonepisodic reinforcement learning (RL) for nonlinear dynamical systems, where the system dynamics are unknown and the RL agent has to learn from a single trajectory, i.e., without resets. We propose Nonepisodic Optimistic RL (NeoRL), an approach based on the principle of optimism in the face of uncertainty. NeoRL uses well-calibrated probabilistic models and plans optimistically w.r.t. the epistemic uncertainty about the unknown dynamics. Under continuity and bounded energy assumptions on the system, we provide a first-of-its-kind regret bound of $\setO(\beta_T \sqrt{T \Gamma_T})$ for general nonlinear systems with Gaussian process dynamics. We compare NeoRL to other baselines on several deep RL environments and empirically demonstrate that NeoRL achieves the optimal average cost while incurring the least regret.

Reinforcement learning (RL) excels in optimizing policies for discrete-time Markov decision processes (MDP). However, various systems are inherently continuous in time, making discrete-time MDPs an inexact modeling choice. In many applications, such as greenhouse control or medical treatments, each interaction (measurement or switching of action) involves manual intervention and thus is inherently costly. Therefore, we generally prefer a time-adaptive approach with fewer interactions with the system. In this work, we formalize an RL framework, Time-adaptive Control & Sensing (TaCoS), that tackles this challenge by optimizing over policies that besides control predict the duration of its application. Our formulation results in an extended MDP that any standard RL algorithm can solve. We demonstrate that state-of-the-art RL algorithms trained on TaCoS drastically reduce the interaction amount over their discrete-time counterpart while retaining the same or improved performance, and exhibiting robustness over discretization frequency. Finally, we propose OTaCoS, an efficient model-based algorithm for our setting. We show that OTaCoS enjoys sublinear regret for systems with sufficiently smooth dynamics and empirically results in further sample-efficiency gains.

Over the past decade, the continuous surge in cloud computing demand has intensified data center workloads, leading to significant carbon emissions and driving the need for improving their efficiency and sustainability. This paper focuses on the optimal allocation problem of batch compute loads with temporal and spatial flexibility across a global network of data centers. We propose a bilevel game-theoretic solution approach that captures the inherent hierarchical relationship between supervisory control objectives, such as carbon reduction and peak shaving, and operational objectives, such as priority-aware scheduling. Numerical simulations with real carbon intensity data demonstrate that the proposed approach successfully reduces carbon emissions while simultaneously ensuring operational reliability and priority-aware scheduling.

In this paper, we implement an advanced safety filter to smoothly limit the current of an inverter-based Battery Energy Storage System. The task involves finding suitable Control Barrier Function and Control Lyapunov Function via Sum-of-Squares optimization to certify the system's safety during grid transients. In contrast to the conventional safety filter, the advanced safety filter not only provides a safety certificate but also achieves finite-time convergence to a nominal region. Within this region, the action of the nominal control, i.e. the Enhanced Direct Power Control, remains unaltered by the safety filter. The advanced safety filter is implemented using a Quadratically Constrained Quadratic Program, providing the capability to also encode quadratic input constraints. Finally, we showcase the effectiveness of the implementation through simulations involving a load step at the Point of Common Coupling, and we compare the outcomes with those obtained using a standard vector current controller.

This paper develops an adaptive traffic control policy inspired by Maximum Pressure (MP) while imposing coordination across intersections. The proposed Coordinated Maximum Pressure-plus-Penalty (CMPP) control policy features a local objective for each intersection that consists of the total pressure within the neighborhood and a penalty accounting for the queue capacities and continuous green time for certain movements. The corresponding control task is reformulated as a distributed optimization problem and solved via two customized algorithms: one based on the alternating direction method of multipliers (ADMM) and the other follows a greedy heuristic augmented with a majority vote. CMPP not only provides a theoretical guarantee of queuing network stability but also outperforms several benchmark controllers in simulations on a large-scale real traffic network with lower average travel and waiting time per vehicle, as well as less network congestion. Furthermore, CPMM with the greedy algorithm enjoys comparable computational efficiency as fully decentralized controllers without significantly compromising the control performance, which highlights its great potential for real-world deployment.

Opinion dynamics models are important in understanding and predicting opinion formation processes within social groups. Although the weighted-averaging opinion-update mechanism is widely adopted as the micro-foundation of opinion dynamics, it bears a non-negligibly unrealistic implication: opinion attractiveness increases with opinion distance. Recently, the weighted-median mechanism has been proposed as a new microscopic mechanism of opinion exchange. Numerous advancements have been achieved regarding this new micro-foundation, from theoretical analysis to empirical validation, in a discrete-time asynchronous setup. However, the original discrete-time weighted-median model does not allow for "compromise behavior" in opinion exchanges, i.e., no intermediate opinions are created between disagreeing agents. To resolve this problem, this paper propose a novel continuous-time weighted-median opinion dynamics model, in which agents' opinions move towards the weighted-medians of their out-neighbors' opinions. It turns out that the proof methods for the original discrete-time asynchronous model are no longer applicable to the analysis of the continuous-time model. In this paper, we first establish the existence and uniqueness of the solution to the continuous-time weighted-median opinion dynamics by showing that the weighted-median mapping is contractive on any graph. We also characterize the set of all the equilibria. Then, by leveraging a new LaSalle invariance principle argument, we prove the convergence of the continuous-time weighted-median model for any initial condition and derive a necessary and sufficient condition for the convergence to consensus., Comment: 13 pages, 1 figure

Motivated by the complex dynamics of cooperative and competitive interactions within networked agent systems, multi-cluster games provide a framework for modeling the interconnected goals of self-interested clusters of agents. For this setup, the existing literature lacks comprehensive gradient-based solutions that simultaneously consider constraint sets and directed communication networks, both of which are crucial for many practical applications. To address this gap, this paper proposes a distributed Nash equilibrium seeking algorithm that integrates consensus-based methods and gradient-tracking techniques, where inter-cluster and intra-cluster communications only use row- and column-stochastic weight matrices, respectively. To handle constraints, we introduce an averaging procedure, which can effectively address the complications associated with projections. In turn, we can show linear convergence of our algorithm, focusing on the contraction property of the optimality gap. We demonstrate the efficacy of the proposed algorithm through a microgrid energy management application.

This article proposes a "cross-forming" control concept for grid-forming inverters operating against grid faults. Cross-forming refers to voltage angle forming and current magnitude forming. It differs from classical grid-forming and grid-following paradigms that feature voltage magnitude-and-angle forming and voltage magnitude-and-angle following (or current magnitude-and-angle forming), respectively. The cross-forming concept addresses the need for inverters to remain grid-forming (particularly voltage angle forming, as required by grid codes) while managing fault current limitation. Simple and feasible cross-forming control implementations are proposed, enabling inverters to quickly limit fault currents to a prescribed level while preserving voltage angle forming for grid-forming synchronization and providing dynamic ancillary services, during symmetrical or asymmetrical fault ride-through. Moreover, the cross-forming control yields an equivalent system featuring a constant virtual impedance and a "normal form" representation, allowing for the extension of previously established transient stability results to include scenarios involving current saturation. Simulations and experiments validate the efficacy of the proposed cross-forming control implementations.

Game-theoretic MPC (or Receding Horizon Games) is an emerging control methodology for multi-agent systems that generates control actions by solving a dynamic game with coupling constraints in a receding-horizon fashion. This control paradigm has recently received an increasing attention in various application fields, including robotics, autonomous driving, traffic networks, and energy grids, due to its ability to model the competitive nature of self-interested agents with shared resources while incorporating future predictions, dynamic models, and constraints into the decision-making process. In this work, we present the first formal stability analysis based on dissipativity and monotone operator theory that is valid also for non-potential games. Specifically, we derive LMI-based certificates that ensure asymptotic stability and are numerically verifiable. Moreover, we show that, if the agents have decoupled dynamics, the numerical verification can be performed in a scalable manner. Finally, we present tuning guidelines for the agents' cost function weights to fulfill the certificates and, thus, ensure stability.

Complex droop control, alternatively known as dispatchable virtual oscillator control (dVOC), stands out for its unique capabilities in synchronization and voltage stabilization among existing control strategies for grid-forming converters. Complex droop control leverages the novel concept of ``complex frequency'', thereby establishing a coupled connection between active and reactive power inputs and frequency and rate-of-change-of voltage outputs. However, its reliance on static droop gains limits its ability to exhibit crucial dynamic response behaviors required in future power systems. To address this limitation, this paper introduces dynamic complex-frequency control, upgrading static droop gains with dynamic transfer functions to enhance the richness and flexibility in dynamic responses for frequency and voltage control. Unlike existing approaches, the complex-frequency control framework treats frequency and voltage dynamics collectively, ensuring small-signal stability for frequency synchronization and voltage stabilization simultaneously. The control framework is validated through detailed numerical case studies on the IEEE nine-bus system, also showcasing its applicability in multi-converter setups., Comment: 6 Pages, 7 Figures

In this paper, we investigate the transient stability of a state-of-the-art grid-forming complex-droop control (i.e., dispatchable virtual oscillator control, dVOC) under current saturation. We quantify the saturation level of a converter by introducing the concept of degree of saturation (DoS), and we propose a provably stable current-limiting control with saturation-informed feedback, which feeds the degree of saturation back to the inner voltage-control loop and the outer grid-forming loop. As a result, although the output current is saturated, the voltage phase angle can still be generated from an internal virtual voltage-source node that is governed by an equivalent complex-droop control. We prove that the proposed control achieves transient stability during current saturation under grid faults. We also provide parametric stability conditions for multi-converter systems under grid-connected and islanded scenarios. The stability performance of the current-limiting control is validated with various case studies.

Feedback optimization enables autonomous optimality seeking of a dynamical system through its closed-loop interconnection with iterative optimization algorithms. Among various iteration structures, model-based approaches require the input-output sensitivity of the system to construct gradients, whereas model-free approaches bypass this need by estimating gradients from real-time evaluations of the objective. These approaches own complementary benefits in sample efficiency and accuracy against model mismatch, i.e., errors of sensitivities. To achieve the best of both worlds, we propose gray-box feedback optimization controllers, featuring systematic incorporation of approximate sensitivities into model-free updates via adaptive convex combination. We quantify conditions on the accuracy of the sensitivities that render the gray-box approach preferable. We elucidate how the closed-loop performance is determined by the number of iterations, the problem dimension, and the cumulative effect of inaccurate sensitivities. The proposed controller contributes to a balanced closed-loop behavior, which retains provable sample efficiency and optimality guarantees for nonconvex problems. We further develop a running gray-box controller to handle constrained time-varying problems with changing objectives and steady-state maps.

A system of non-tradable credits that flow between individuals like karma, hence proposed under that name, is a mechanism for repeated resource allocation that comes with attractive efficiency and fairness properties, in theory. In this study, we test karma in an online experiment in which human subjects repeatedly compete for a resource with time-varying and stochastic individual preferences or urgency to acquire the resource. We confirm that karma has significant and sustained welfare benefits even in a population with no prior training. We identify mechanism usage in contexts with sporadic high urgency, more so than with frequent moderate urgency, and implemented as an easy (binary) karma bidding scheme as particularly effective for welfare improvements: relatively larger aggregate efficiency gains are realized that are (almost) Pareto superior. These findings provide guidance for further testing and for future implementation plans of such mechanisms in the real world.

Motivated by how forecast errors exacerbate order fluctuations in supply chains, we use tools from robust control theory to characterize and compute the worst-case order fluctuation experienced by an individual supply chain vendor under bounded forecast errors and demand fluctuations. Building on existing discrete time, linear time-invariant (LTI) models of supply chains, we separately model forecast error and demand fluctuations as inputs to the inventory dynamics. We then define a transient Bullwhip measure to evaluate the vendor's worst-case order fluctuation and show that for bounded forecast errors and demand fluctuations, this measure is equivalent to the disturbance to control peak gain. To compute the controller that minimizes the worst-case peak gain, we formulate an optimization problem with bilinear matrix inequalities and show that solving this problem is equivalent to minimizing a quasi-convex function on a bounded domain. In contrast to the existing Bullwhip measure in literature, the transient Bullwhip measure has an explicit dependency on the forecast error and does not need the forecast to be a deterministic function of the demand history. This explicit dependency enables us to separately quantify the transient Bullwhip measure's sensitivity to forecast error and demand fluctuations. We empirically verify our model for vendors with non-zero perishable rates and order backlogging rates., Comment: 8 pages, 5 figures

Online feedback optimization (OFO) enables optimal steady-state operations of a physical system by employing an iterative optimization algorithm as a dynamic feedback controller. When the plant consists of several interconnected sub-systems, centralized implementations become impractical due to the heavy computational burden and the need to pre-compute system-wide sensitivities, which may not be easily accessible in practice. Motivated by these challenges, we develop a fully distributed model-free OFO controller, featuring consensus-based tracking of the global objective value and local iterative (projected) updates that use stochastic gradient estimates. We characterize how the closed-loop performance depends on the size of the network, the number of iterations, and the level of accuracy of consensus. Numerical simulations on a voltage control problem in a direct current power grid corroborate the theoretical findings.

We present SIM-FSVGD for learning robot dynamics from data. As opposed to traditional methods, SIM-FSVGD leverages low-fidelity physical priors, e.g., in the form of simulators, to regularize the training of neural network models. While learning accurate dynamics already in the low data regime, SIM-FSVGD scales and excels also when more data is available. We empirically show that learning with implicit physical priors results in accurate mean model estimation as well as precise uncertainty quantification. We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system. Using model-based RL, we demonstrate a highly dynamic parking maneuver with drifting, using less than half the data compared to the state of the art.

A promising step from linear towards nonlinear data-driven control is via the design of controllers for linear parameter-varying (LPV) systems, which are linear systems whose parameters are varying along a measurable scheduling signal. However, the interplay between uncertainty arising from corrupted data and the parameter-varying nature of these systems impacts the stability analysis, and limits the generalization of well-understood data-driven methods for linear time-invariant systems. In this work, we decouple this interplay using a recently developed variant of the Fundamental Lemma for LPV systems and the viewpoint of data-informativity, in combination with biquadratic Lyapunov forms. Together, these allow us to develop novel linear matrix inequality conditions for the existence of scheduling-dependent Lyapunov functions, incorporating the intrinsic nonlinearity. Appealingly, these results are stated purely in terms of the collected data and bounds on the noise, and they are computationally favorable to check., Comment: Submitted for CDC 2024

In competitive multi-player interactions, simultaneous optimality is a key requirement for establishing strategic equilibria. This property is explicit when the game-theoretic equilibrium is the simultaneously optimal solution of coupled optimization problems. However, no such optimization problems exist for the correlated equilibrium, a strategic equilibrium where the players can correlate their actions. We address the lack of a coupled optimization framework for the correlated equilibrium by introducing an {unnormalized game} -- an extension of normal-form games in which the player strategies are lifted to unnormalized measures over the joint actions. We show that the set of fully mixed generalized Nash equilibria of this unnormalized game is a subset of the correlated equilibrium of the normal-form game. Furthermore, we introduce an entropy regularization to the unnormalized game and prove that the entropy-regularized generalized Nash equilibrium is a sub-optimal correlated equilibrium of the normal form game where the degree of sub-optimality depends on the magnitude of regularization. We prove that the entropy-regularized unnormalized game has a closed-form solution, and empirically verify its computational efficacy at approximating the correlated equilibrium of normal-form games., Comment: 8 pages, 2 figures

A closed-loop control model to analyze the impact of recommendation systems on opinion dynamics within social networks is introduced. The core contribution is the development and formalization of model-free and model-based approaches to recommendation system design, integrating the dynamics of social interactions within networks via an extension of the Friedkin-Johnsen (FJ) model. Comparative analysis and numerical simulations demonstrate the effectiveness of the proposed control strategies in maximizing user engagement and their potential for influencing opinion formation processes.

Recent years have seen a surge of artificial currency-based mechanisms in contexts where monetary instruments are deemed unfair or inappropriate, e.g., in allocating food donations to food banks, course seats to students, and, more recently, even for traffic congestion management. Yet the applicability of these mechanisms remains limited in repeated auction settings, as it is challenging for users to learn how to bid an artificial currency that has no value outside the auctions. Indeed, users must jointly learn the value of the currency in addition to how to spend it optimally. In this work, we study the problem of learning to bid in two prominent classes of artificial currency auctions: those in which currency, which users spend to obtain public resources, is only issued at the beginning of a finite period; and those where, in addition to the initial currency endowment, currency payments are redistributed to users at each time step. In the latter class, the currency has been referred to as karma, since users do not only spend karma to obtain public resources but also gain karma for yielding them. In both classes, we propose a simple learning strategy, called adaptive karma pacing, and show that this strategy a) is asymptotically optimal for a single user bidding against competing bids drawn from a stationary distribution; b) leads to convergent learning dynamics when all users adopt it; and c) constitutes an approximate Nash equilibrium as the number of users grows. Our results require a novel analysis in comparison to adaptive pacing strategies in monetary auctions, since we depart from the classical assumption that the currency has known value outside the auctions, and moreover consider that the currency is both spent and gained in the class of auctions with redistribution., Comment: Manuscript submitted for review to the 25th ACM Conference on Economics & Computation (EC'24)

Due to more volatile generation, flexibility will become more important in transmission grids. One potential source of this flexibility can be distribution grids. A flexibility request from the transmission grid to a distribution grid then needs to be split up onto the different flexibility providing units (FPU) in the distribution grid. One potential way to do this is Online Feedback Optimization (OFO). OFO is a new control method that steers power systems to the optimal solution of an optimization problem using minimal model information and computation power. This paper will show how to choose the optimization problem and how to tune the OFO controller. Afterward, we test the resulting controller on a real distribution grid laboratory and show its performance, its interaction with other controllers in the grid, and how it copes with disturbances. Overall, the paper makes a clear recommendation on how to phrase the optimization problem and tune the OFO controller. Furthermore, it experimentally verifies that an OFO controller is a powerful tool to disaggregate flexibility requests onto FPUs while satisfying operational constraints inside the flexibility providing distribution grid.

In this paper, we propose a systematic closed-loop approach to provide optimal dynamic ancillary services with converter-interfaced generation systems based on local power grid perception. In particular, we structurally encode dynamic ancillary services such as fast frequency and voltage regulation in the form of a parametric transfer function matrix, which includes several parameters to define a set of different feasible response behaviors, among which we aim to find the optimal one to be realized by the converter system. Our approach is based on a so-called "perceive-and-optimize" (P&O) strategy: First, we identify a grid dynamic equivalent at the interconnection terminals of the converter system. Second, we consider the closed-loop interconnection of the identified grid equivalent and the parametric transfer function matrix, which we optimize for the set of transfer function parameters, resulting in a stable and optimal closed-loop performance for ancillary services provision. In the process, we ensure that grid-code and device-level requirements are satisfied. Finally, we demonstrate the effectiveness of our approach in different numerical case studies based on a modified Kundur two-area test system., Comment: 15 pages, 20 Figures

A plethora of methods have been developed in the past two decades to infer the underlying network structure of an interconnected system from its collective dynamics. However, methods capable of inferring nonpairwise interactions are only starting to appear. Here, we develop an inference algorithm based on sparse identification of nonlinear dynamics (SINDy) to reconstruct hypergraphs and simplicial complexes from time-series data. Our model-free method does not require information about node dynamics or coupling functions, making it applicable to complex systems that do not have reliable mathematical descriptions. We first benchmark the new method on synthetic data generated from Kuramoto and Lorenz dynamics. We then use it to infer the effective connectivity among seven brain regions from resting-state EEG data, which reveals significant contributions from non-pairwise interactions in shaping the macroscopic brain dynamics., Comment: Main text: 9 pages, 4 figures. Supp. Mat.: 2 pages, 3 figures

Direct data-driven design methods for the linear quadratic regulator (LQR) mainly use offline or episodic data batches, and their online adaptation has been acknowledged as an open problem. In this paper, we propose a direct adaptive method to learn the LQR from online closed-loop data. First, we propose a new policy parameterization based on the sample covariance to formulate a direct data-driven LQR problem, which is shown to be equivalent to the certainty-equivalence LQR with optimal non-asymptotic guarantees. Second, we design a novel data-enabled policy optimization (DeePO) method to directly update the policy, where the gradient is explicitly computed using only a batch of persistently exciting (PE) data. Third, we establish its global convergence via a projected gradient dominance property. Importantly, we efficiently use DeePO to adaptively learn the LQR by performing only one-step projected gradient descent per sample of the closed-loop system, which also leads to an explicit recursive update of the policy. Under PE inputs and for bounded noise, we show that the average regret of the LQR cost is upper-bounded by two terms signifying a sublinear decrease in time $\mathcal{O}(1/\sqrt{T})$ plus a bias scaling inversely with signal-to-noise ratio (SNR), which are independent of the noise statistics. Finally, we perform simulations to validate the theoretical results and demonstrate the computational and sample efficiency of our method.

Traditionally, numerical algorithms are seen as isolated pieces of code confined to an {\em in silico} existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein {\em in vivo} algorithms interact with their environment. Examples of such {\em open algorithms} include various real-time optimization-based control strategies, reinforcement learning, decision-making architectures, online optimization, and many more. Further, even {\em closed} algorithms in learning or optimization are increasingly abstracted in block diagrams with interacting dynamic modules and pipelines. In this opinion paper, we state our vision on a to-be-cultivated {\em systems theory of algorithms} and argue in favor of viewing algorithms as open dynamical systems interacting with other algorithms, physical systems, humans, or databases. Remarkably, the manifold tools developed under the umbrella of systems theory are well suited for addressing a range of challenges in the algorithmic domain. We survey various instances where the principles of algorithmic systems theory are being developed and outline pertinent modeling, analysis, and design challenges.

The vast majority of products we use daily are supplied to us through complex global supply chains that transform raw materials into finished goods and distribute them to end consumers. This paper proposes a modeling methodology for dynamic competitive supply chains based on game theory and model predictive control. We model each manufacturer in the supply chain as a rational utility maximizing agent that selects their actions by finding an open-loop generalized Nash equilibrium of a multi-stage game. To react to competitors and the state of the market, every agent re-plans their actions in a receding horizon manner based on estimates of market and supplier parameters thereby creating an approximate closed-loop equilibrium policy. We demonstrate through numerical simulations that this modeling approach is computationally tractable and generates economically interpretable behaviors in a variety of settings such as demand spikes, supply shocks, and information asymmetry.

We study control of constrained linear systems with only partial statistical information about the uncertainty affecting the system dynamics and the sensor measurements. Specifically, given a finite collection of disturbance realizations drawn from a generic distribution, we consider the problem of designing a stabilizing control policy with provable safety and performance guarantees despite the mismatch between the empirical and true distributions. We capture this discrepancy using Wasserstein ambiguity sets, and we formulate a distributionally robust (DR) optimal control problem, which provides guarantees on the expected cost, safety, and stability of the system. To solve this problem, we first present new results for DR optimization of quadratic objectives using convex programming, showing that strong duality holds under mild conditions. Then, by combining our results with the system-level parametrization of linear feedback policies, we show that the design problem can be reduced to a semidefinite program. We present numerical simulations to validate the effectiveness of our approach and to highlight the value of empirical distributions for control design.

Urban traffic congestion remains a pressing challenge in our rapidly expanding cities, despite the abundance of available data and the efforts of policymakers. By leveraging behavioral system theory and data-driven control, this paper exploits the DeePC algorithm in the context of urban traffic control performed via dynamic traffic lights. To validate our approach, we consider a high-fidelity case study using the state-of-the-art simulation software package Simulation of Urban MObility (SUMO). Preliminary results indicate that DeePC outperforms existing approaches across various key metrics, including travel time and CO$_2$ emissions, demonstrating its potential for effective traffic management, Comment: This paper has been submitted to IEEE ECC24

Online feedback optimization is a controller design paradigm for optimizing the steady-state behavior of a dynamical system. It employs an optimization algorithm as a dynamic feedback controller and utilizes real-time measurements to bypass knowing exact plant dynamics and disturbances. Different from existing centralized settings, we present a fully decentralized feedback optimization controller for networked systems to lift the communication burden and improve scalability. We approximate the overall input-output sensitivity matrix through its diagonal elements, which capture local model information. For the closed-loop behavior, we characterize the stability and bound the sub-optimality due to decentralization. We prove that the proposed decentralized controller yields solutions that correspond to the Nash equilibria of a non-cooperative game.

Reinforcement learning algorithms typically consider discrete-time dynamics, even though the underlying systems are often continuous in time. In this paper, we introduce a model-based reinforcement learning algorithm that represents continuous-time dynamics using nonlinear ordinary differential equations (ODEs). We capture epistemic uncertainty using well-calibrated probabilistic models, and use the optimistic principle for exploration. Our regret bounds surface the importance of the measurement selection strategy(MSS), since in continuous time we not only must decide how to explore, but also when to observe the underlying system. Our analysis demonstrates that the regret is sublinear when modeling ODEs with Gaussian Processes (GP) for common choices of MSS, such as equidistant sampling. Additionally, we propose an adaptive, data-dependent, practical MSS that, when combined with GP dynamics, also achieves sublinear regret with significantly fewer samples. We showcase the benefits of continuous-time modeling over its discrete-time counterpart, as well as our proposed adaptive MSS over standard baselines, on several applications.

Online feedback optimization (OFO) is an emerging control methodology for real-time optimal steady-state control of complex dynamical systems. This tutorial focuses on the application of OFO for the autonomous operation of large-scale transmission grids, with a specific goal of minimizing renewable generation curtailment and losses while satisfying voltage and current limits. When this control methodology is applied to multi-area transmission grids, where each area independently manages its congestion while being dynamically interconnected with the rest of the grid, a non-cooperative game arises. In this context, OFO must be interpreted as an online feedback equilibrium seeking (FES) scheme. Our analysis incorporates technical tools from game theory and monotone operator theory to evaluate the stability and performance of multi-area grid operation. Through numerical simulations, we illustrate the key challenge of this non-cooperative setting: on the one hand, independent multi-area decisions are suboptimal compared to a centralized control scheme; on the other hand, some areas are heavily penalized by the centralized decision, which may discourage participation in the coordination mechanism.

The paper studies conical, convex, and affine models in the framework of behavioral systems theory. We investigate basic properties of such behaviors and address the problem of constructing models from measured data. We prove that closed, shift-invariant, conical, convex, and affine models have the intersection property, thereby enabling the definition of most powerful unfalsified models based on infinite-horizon measurements. We then provide necessary and sufficient conditions for representing conical, convex, and affine finite-horizon behaviors using raw data matrices, expressing persistence of excitation requirements in terms of non-negative rank conditions. The applicability of our results is demonstrated by a numerical example arising in population ecology.

We study the controller implementability problem, which seeks to determine if a controller can make the closed-loop behavior of a given plant match that of a desired reference behavior. We establish necessary and sufficient conditions for controller implementability which only rely on raw data. Subsequently, we consider the problem of constructing controllers directly from data. By leveraging the concept of canonical controller, we provide a formula to directly construct controllers that implement plant-compatible reference behaviors using measurements of both reference and plant behaviors.

In this paper, we analytically study the transient stability of grid-connected converters with grid-forming complex droop control, also known as dispatchable virtual oscillator control. We prove theoretically that complex droop control, as a state-of-the-art grid-forming control, always possesses steady-state equilibria whereas classical droop control does not. We provide quantitative conditions for complex droop control maintaining transient stability (global asymptotic stability) under grid disturbances, which is beyond the well-established local (non-global) stability for classical droop control. For the transient instability of complex droop control, we reveal that the unstable trajectories are bounded, manifesting as limit cycle oscillations. Moreover, we extend our stability results from second-order grid-forming control dynamics to full-order system dynamics that additionally encompass both circuit electromagnetic transients and inner-loop dynamics. Our theoretical results contribute an insightful understanding of the transient stability and instability of complex droop control and offer practical guidelines for parameter tuning and stability guarantees.

We prove that the popular grid-forming control, i.e., dispatchable virtual oscillator control (dVOC), also termed complex droop control, exhibits output-feedback passivity in its large-signal model, featuring an explicit and physically meaningful passivity index. Using this passivity property, we derive decentralized stability conditions for the transient stability of dVOC in multi-converter grid-connected systems, beyond prior small-signal stability results. The decentralized conditions are of practical significance, particularly for ensuring the transient stability of renewable power plants under grid disturbances.

Conventional grid-code specifications for dynamic ancillary services provision such as fast frequency and voltage regulation are typically defined by means of piece-wise linear step-response capability curves in the time domain. However, although the specification of such time-domain curves is straightforward, their practical implementation in a converter-based generation system is not immediate, and no customary methods have been developed yet. In this paper, we thus propose a systematic approach for the practical implementation of piece-wise linear time-domain curves to provide dynamic ancillary services by converter-based generation systems, while ensuring grid-code and device-level requirements to be reliably satisfied. Namely, we translate the piece-wise linear time-domain curves for active and reactive power provision in response to a frequency and voltage step change into a desired rational parametric transfer function in the frequency domain, which defines a dynamic response behavior to be realized by the converter. The obtained transfer function can be easily implemented e.g. via a PI-based matching control in the power loop of standard converter control architectures. We demonstrate the performance of our method in numerical grid-code compliance tests, and reveal its superiority over classical droop and virtual inertia schemes which may not satisfy the grid codes due to their structural limitations., Comment: 8 pages, 11 figures

Recommendation systems are widely used in web services, such as social networks and e-commerce platforms, to serve personalized content to the users and, thus, enhance their experience. While personalization assists users in navigating through the available options, there have been growing concerns regarding its repercussions on the users and their opinions. Examples of negative impacts include the emergence of filter bubbles and the amplification of users' confirmation bias, which can cause opinion polarization and radicalization. In this paper, we study the impact of recommendation systems on users, both from a microscopic (i.e., at the level of individual users) and a macroscopic (i.e., at the level of a homogenous population) perspective. Specifically, we build on recent work on the interactions between opinion dynamics and recommendation systems to propose a model for this closed loop, which we then study both analytically and numerically. Among others, our analysis reveals that shifts in the opinions of individual users do not always align with shifts in the opinion distribution of the population. In particular, even in settings where the opinion distribution appears unaltered (e.g., measured via surveys across the population), the opinion of individual users might be significantly distorted by the recommendation system., Comment: Accepted for presentation at, and publication in the proceedings of, the 62nd IEEE Conference on Decision and Control

This paper proposes decentralized stability conditions for multi-converter systems based on the combination of the small gain theorem and the small phase theorem. Instead of directly computing the closed-loop dynamics, e.g., eigenvalues of the state-space matrix, or using the generalized Nyquist stability criterion, the proposed stability conditions are more scalable and computationally lighter, which aim at evaluating the closed-loop system stability by comparing the individual converter dynamics with the network dynamics in a decentralized and open-loop manner. Moreover, our approach can handle heterogeneous converters' dynamics and is suitable to analyze large-scale multi-converter power systems that contain grid-following (GFL), grid-forming (GFM) converters, and synchronous generators. Compared with other decentralized stability conditions, e.g., passivity-based stability conditions, the proposed conditions are significantly less conservative and can be generally satisfied in practice across the whole frequency range.

In this paper, we address the challenge of Nash equilibrium (NE) seeking in non-cooperative convex games with partial-decision information. We propose a distributed algorithm, where each agent refines its strategy through projected-gradient steps and an averaging procedure. Each agent uses estimates of competitors' actions obtained solely from local neighbor interactions, in a directed communication network. Unlike previous approaches that rely on (strong) monotonicity assumptions, this work establishes the convergence towards a NE under a diagonal dominance property of the pseudo-gradient mapping, that can be checked locally by the agents. Further, this condition is physically interpretable and of relevance for many applications, as it suggests that an agent's objective function is primarily influenced by its individual strategic decisions, rather than by the actions of its competitors. In virtue of a novel block-infinity norm convergence argument, we provide explicit bounds for constant step-size that are independent of the communication structure, and can be computed in a totally decentralized way. Numerical simulations on an optical network's power control problem validate the algorithm's effectiveness.

As renewable generation becomes more prevalent, traditional power systems dominated by synchronous generators are transitioning to systems dominated by converter-interfaced generation. These devices, with their weaker damping capabilities and lower inertia, compromise the system's ability to withstand disturbances, pose a threat to system stability, and lead to oscillations and poor frequency response performance. While some new converter-interfaced generations are capable of providing superior damping and fast frequency control, there is a lack of effective measures to incentivize manufacturers to adopt them. To address this gap, this paper defines the joint oscillation damping and inertia provision services at the system level, seeking to encourage converter-interfaced generation to provide enhanced damping and fast frequency response capabilities. Our approach is anchored in a novel convex parametric formulation that combines oscillation mode and frequency stability constraints. These constraints ensure a sufficient damping ratio for all oscillation modes and maintain transient frequency trajectories within acceptable limits. They are designed to integrate smoothly into various operational and planning optimization frameworks. Using this formulation, we introduce a joint service for oscillation damping and inertia provision based on a cost-minimization problem. This facilitates the optimal allocation of damping and virtual inertia to converters, achieving both small-signal stability and frequency stability. Furthermore, we investigate the economic effects of introducing this service into a new ancillary service market, assessing its impact on system operations and cost-efficiency. Numerical tests highlight the service's efficacy in ensuring both small-signal stability and frequency stability, and offer insights into potential economic benefits., Comment: Submitted for IEEE PES journal for possible publications

Recently, the concept of dynamic virtual power plants (DVPP) has been proposed to collectively provide desired dynamic ancillary services such as fast frequency and voltage control by a heterogeneous ensemble of distributed energy resources (DER). This paper presents an experimental validation of a recent DVPP control design approach on a multi-converter power hardware-in-the-loop (PHIL) test bed system. More specifically, we consider a DVPP composed of a wind generation system, a photovoltaic (PV) system, and a STATCOM with small storage capacity to collectively provide grid-following fast frequency regulation in the presence of grid-frequency and load variations. The performance of the aggregated DVPP response is evaluated with respect to its ability to match a desired dynamic behavior while taking practical limitations of the individual DVPP units into account., Comment: 8 Pages, 11 Figures, 22nd Wind & Solar Integration Workshop 2023

The evolution of existing transportation systems,mainly driven by urbanization and increased availability of mobility options, such as private, profit-maximizing ride-hailing companies, calls for tools to reason about their design and regulation. To study this complex socio-technical problem, one needs to account for the strategic interactions of the heterogeneous stakeholders involved in the mobility ecosystem and analyze how they influence the system. In this paper, we focus on the interactions between citizens who compete for the limited resources of a mobility system to complete their desired trip. Specifically, we present a game-theoretic framework for multi-modal mobility systems, where citizens, characterized by heterogeneous preferences, have access to various mobility options and seek individually-optimal decisions. We study the arising game and prove the existence of an equilibrium, which can be efficiently computed via a convex optimization problem. Through both an analytical and a numerical case study for the classic scenario of Sioux Falls, USA, we illustrate the capabilities of our model and perform sensitivity analyses. Importantly, we show how to embed our framework into a "larger" game among stakeholders of the mobility ecosystem (e.g., municipality, Mobility Service Providers, and citizens), effectively giving rise to tools to inform strategic interventions and policy-making in the mobility ecosystem., Comment: 8 pages, 5 figures, to appear in the proceedings of the 2023 IEEE 26th International Conference on Intelligent Transportation Systems

This work explores the stability, behavior, variants, and a controller-hardware-in-the-loop (C-HiL) verification of the recently proposed grid-forming (GFM) hybrid angle control (HAC). We revisit the foundation of GFM HAC, and highlight its behavioral properties in relation to the conventional synchronous machine (SM). Next, we introduce the required complementary controls to be combined with the HAC to realize a GFM behavior. The characterization of the analytical operating point and nonlinear energy-based stability analysis of a grid-connected converter under the HAC is presented. Further, we consider various output filter configurations and derive an approximation for the original control proposal. Moreover, we provide details on the integration of GFM HAC into a complex converter control architecture and introduce several variants of the standard HAC. Finally, the performance of GFM HAC is verified by several test scenarios in a C-HiL setup to test its behavior against real-world effect such as noise and delays.

Distribution grid flexibility is discussed as a possible measure in curative system operation, yielding a need for an efficient and robust coordination mechanism for the joint flexibility provision by individual units to the transmission grid. This paper introduces a method to coordinate distribution grid level flexibility as a fast-responding curative measure based on Online Feedback Optimization. We utilize an optimization algorithm in a closed loop with the distribution grid to dispatch set points for active and reactive power to flexibility providing units. The approach is evaluated in an experimental setup, utilizing assets connected to an exemplary low voltage grid. Online Feedback Optimization is found to be both a viable as well as a highly effective approach to coordinate distributed energy resources in real-time curative system operation.

Towards bridging classical optimal control and online learning, regret minimization has recently been proposed as a control design criterion. This competitive paradigm penalizes the loss relative to the optimal control actions chosen by a clairvoyant policy, and allows tracking the optimal performance in hindsight no matter how disturbances are generated. In this paper, we propose the first receding horizon scheme based on the repeated computation of finite horizon regret-optimal policies, and we establish stability and safety guarantees for the resulting closed-loop system. Our derivations combine novel monotonicity properties of clairvoyant policies with suitable terminal ingredients. We prove that our scheme is recursively feasible, stabilizing, and that it achieves bounded regret relative to the infinite horizon clairvoyant policy. Last, we show that the policy optimization problem can be solved efficiently through convex-concave programming. Our numerical experiments show that minimizing regret can outperform standard receding horizon approaches when the disturbances poorly fit classical design assumptions - even when the finite horizon planning is recomputed less frequently.