Search

We propose a generative adversarial network (GAN) based deep learning method that serves the dual role of both identification and mitigation of cyber-attacks in wide-area damping control loops of power systems. Two specific types of attacks considered are false data injection and denial-of-service (DoS). Unlike existing methods, which are either model-based or model-free and yet require two separate learning modules for detection and mitigation leading to longer response times before clearing an attack, our deep learner incorporate both goals within the same integrated framework. A Long Short-Term Memory (LSTM) encoder-decoder based GAN is proposed that captures the temporal dynamics of the power system significantly more accurately than fully-connected GANs, thereby providing better accuracy and faster response for both goals. The method is validated using the IEEE 68-bus power system model., Comment: AAAI 2023 workshop

We study how high charging rate demands from electric vehicles (EVs) in a power distribution grid may collectively cause its dynamic instability, and, accordingly, how a price incentivization strategy can be used to steer customers to settle for lesser charging rate demands so that these instabilities can be avoided. We pose the problem as a joint optimization and optimal control formulation. The optimization determines the optimal charging setpoints for EVs to minimize the $\mathcal{H}_2$-norm of the transfer function of the grid model, while the optimal control simultaneously develops a linear quadratic regulator (LQR) based state-feedback control signal for the battery-currents of those EVs to jointly minimize the risk of grid instability. A subsequent algorithm is developed to determine how much customers may be willing to sacrifice their intended charging rate demands in return for financial incentives. Results are derived for both unidirectional and bidirectional charging, and validated using numerical simulations of multiple EV charging stations in the IEEE 33-bus power distribution model.

We study the equilibrium-independent (EI) passivity of a nonlinear power system composed of two-axis generator models. The model of our interest consists of a feedback inter-connection of linear and nonlinear subsystems, called mechanical and electromagnetic subsystems. We mathematically prove the following three facts by analyzing the nonlinear electromagnetic subsystem. First, a lossless transmission network is necessary for the EI passivity of the electromagnetic subsystem. Second, the convexity of a strain energy function characterizes the largest set of equilibria over which the electromagnetic subsystem is EI passive. Finally, we prove that the strain energy function for the network of the two-axis generator models is convex if and only if its flux linkage dynamics is stable, and the strain energy function for the network of the classical generator models derived by singular perturbation approximation of the flux linkage dynamics is convex. Numerical simulation of the IEEE 9-bus power system model demonstrates the practical implications of the various mathematical results. In particular, we validate that the convex domain of the strain energy function over which the electromagnetic subsystem is EI passive is almost identical to the set of all stable equilibria. This result is also generalized to lossy power systems based on our finding that the convexity of the strain energy function is equivalent to the positive semidefiniteness of a synchronizing torque coefficient matrix.

We develop data-driven reinforcement learning (RL) control designs for input-affine nonlinear systems. We use Carleman linearization to express the state-space representation of the nonlinear dynamical model in the Carleman space, and develop a real-time algorithm that can learn nonlinear state-feedback controllers using state and input measurements in the infinite-dimensional Carleman space. Thereafter, we study the practicality of having a finite-order truncation of the control signal, followed by its closed-loop stability analysis. Finally, we develop two additional designs that can learn structured as well as sparse representations of the RL-based nonlinear controller, and provide theoretical conditions for ensuring their closed-loop stability. We present numerical examples to show how our proposed method generates closed-loop responses that are close to the optimal performance of the nonlinear plant. We also compare our designs to other data-driven nonlinear RL control methods such as those based on neural networks, and illustrate their relative advantages and drawbacks., Comment: 18 pages, extended version, preprint version

We develop investment approaches to secure electric power systems against load attacks where a malicious intruder (the attacker) covertly changes reactive power setpoints of loads to push the grid towards voltage instability while the system operator (the defender) employs reactive power compensation (RPC) to prevent instability. Extending our previously reported Stackelberg game formulation for this problem, we develop a robust-defense sequential algorithm and a novel genetic algorithm that provides scalability to large-scale power system models. The proposed methods are validated using IEEE prototype power system models with time-varying load uncertainties, demonstrating that reliable and robust defense is feasible unless the operator's RPC investment resources are severely limited relative to the attacker's resources., Comment: 6 pages, 6 figures, accepted by IEEE CDC 2023

Achieving distributed reinforcement learning (RL) for large-scale cooperative multi-agent systems (MASs) is challenging because: (i) each agent has access to only limited information; (ii) issues on convergence or computational complexity emerge due to the curse of dimensionality. In this paper, we propose a general computationally efficient distributed framework for cooperative multi-agent reinforcement learning (MARL) by utilizing the structures of graphs involved in this problem. We introduce three coupling graphs describing three types of inter-agent couplings in MARL, namely, the state graph, the observation graph and the reward graph. By further considering a communication graph, we propose two distributed RL approaches based on local value-functions derived from the coupling graphs. The first approach is able to reduce sample complexity significantly under specific conditions on the aforementioned four graphs. The second approach provides an approximate solution and can be efficient even for problems with dense coupling graphs. Here there is a trade-off between minimizing the approximation error and reducing the computational complexity. Simulations show that our RL algorithms have a significantly improved scalability to large-scale MASs compared with centralized and consensus-based distributed RL algorithms., Comment: This paper has been accepted by IEEE Transactions on Automatic Control as a full paper and published online. Different from the published paper, the arxiv version contains more results. Moreover, we will continuously update the arxiv version if we find any typos in the published paper. So we suggest you to read this arxiv paper instead of the published one. Thank you for your interest in our work

Existing distributed cooperative multi-agent reinforcement learning (MARL) frameworks usually assume undirected coordination graphs and communication graphs while estimating a global reward via consensus algorithms for policy evaluation. Such a framework may induce expensive communication costs and exhibit poor scalability due to requirement of global consensus. In this work, we study MARLs with directed coordination graphs, and propose a distributed RL algorithm where the local policy evaluations are based on local value functions. The local value function of each agent is obtained by local communication with its neighbors through a directed learning-induced communication graph, without using any consensus algorithm. A zeroth-order optimization (ZOO) approach based on parameter perturbation is employed to achieve gradient estimation. By comparing with existing ZOO-based RL algorithms, we show that our proposed distributed RL algorithm guarantees high scalability. A distributed resource allocation example is shown to illustrate the effectiveness of our algorithm.

We propose a novel hierarchical frequency and voltage control design for multi-area power system integrated with inverter-based resources (IBRs). The design is based on the idea of prioritizing the use of IBRs over conventional generator-based control in compensating for sudden and unpredicted changes in loads and generations, and thereby mitigate any undesired dynamics in the frequency or the voltage by exploiting their fast actuation time constants. A new sequential optimization problem, referred to as Area Prioritized Power Flow (APPF), is formulated to model this prioritization. It is shown that compared to conventional power flow APPF not only leads to a fairer balance between the dispatch of active and reactive power from the IBRs and the synchronous generators, but also limits the impact of any contingency from spreading out beyond its respective control area, thereby guaranteeing a better collective dynamic performance of the grid. This improvement, however, comes at the cost of adding an extra layer of communication needed for executing APPF in a hierarchical way. Results are validated using simulations of a 9-machine, 6-IBR, 33-bus, 3-area power system model, illustrating how APPF can mitigate a disturbance faster and more efficiently by prioritizing the use of local area-resources.

Recently introduced distributed zeroth-order optimization (ZOO) algorithms have shown their utility in distributed reinforcement learning (RL). Unfortunately, in the gradient estimation process, almost all of them require random samples with the same dimension as the global variable and/or require evaluation of the global cost function, which may induce high estimation variance for large-scale networks. In this paper, we propose a novel distributed zeroth-order algorithm by leveraging the network structure inherent in the optimization objective, which allows each agent to estimate its local gradient by local cost evaluation independently, without use of any consensus protocol. The proposed algorithm exhibits an asynchronous update scheme, and is designed for stochastic non-convex optimization with a possibly non-convex feasible domain based on the block coordinate descent method. The algorithm is later employed as a distributed model-free RL algorithm for distributed linear quadratic regulator design, where a learning graph is designed to describe the required interaction relationship among agents in distributed learning. We provide an empirical validation of the proposed algorithm to benchmark its performance on convergence rate and variance against a centralized ZOO algorithm., Comment: The arxiv version contains proofs of Lemma 3 and Lemma 5, which are missing in the published version

In the state-of-the-art literature on cryptography and control theory, there has been no systematic methodology of constructing cyber-physical systems that can achieve desired control performance while being protected against eavesdropping attacks. In this paper, we tackle this challenging problem. We first propose two novel notions referred to as sample identifying complexity and sample deciphering time in an encrypted-control framework. The former explicitly captures the relation between the dynamical characteristics of control systems and the level of identifiability of the systems while the latter shows the relation between the computation time for the identification and the key length of a cryptosystem. Based on these two tractable new notions, we propose a systematic method for designing the both of an optimal key length to prevent system identification with a given precision within a given life span of systems, and of an optimal controller to maximize both of the control performance and the difficulty of the identification. The efficiency of the proposed method in terms of security level and realtime-ness is investigated through numerical simulations. To the best of our knowledge, this paper first connect the relationship between the security of cryptography and dynamical systems from a control-theoretic perspective., Comment: 15 pages, 6 figures

We present a resource-planning game for cyber-security of networked control systems (NCS). The NCS is assumed to be operating in closed-loop using a linear state-feedback $\mathcal{H}_2$ controller. A zero-sum, two-player Stackelberg game (SG) is developed between an attacker and a defender for this NCS. The attacker aims to disable communication of selected nodes and thereby render the feedback gain matrix to be sparse, leading to degradation of closed-loop performance, while the defender aims to prevent this loss by investing in the protection of targeted nodes. Both players trade their $\mathcal{H}_2$-performance objectives for the costs of their actions. The standard backward induction method is modified to determine a cost-based Stackelberg equilibrium (CBSE) that saves the players' costs without degrading the control performance. We analyze the dependency of a CBSE on the relative budgets of the players as well as on the node "importance" order. Moreover, a robust-defense method is developed for the realistic case when the defender is not informed about the attacker's resources. The proposed algorithms are validated using examples from wide-area control of electric power systems. It is demonstrated that reliable and robust defense is feasible unless the defender's resources are severely limited relative to the attacker's resources. We also show that the proposed methods are robust to time-varying model uncertainties and thus are suitable for long-term security investment in realistic NCSs. Finally, we employ computationally efficient genetic algorithms (GA) to compute the optimal strategies of the attacker and the defender in realistic large power systems., Comment: 16 pages, 14 figures, 2 tables

We address the problem of model-free distributed stabilization of heterogeneous multi-agent systems using reinforcement learning (RL). Two algorithms are developed. The first algorithm solves a centralized linear quadratic regulator (LQR) problem without knowing any initial stabilizing gain in advance. The second algorithm builds upon the results of the first algorithm, and extends it to distributed stabilization of multi-agent systems with predefined interaction graphs. Rigorous proofs are provided to show that the proposed algorithms achieve guaranteed convergence if specific conditions hold. A simulation example is presented to demonstrate the theoretical results., Comment: This paper propose model-free RL algorithms for deriving stabilizing gains of continuous-time multi-agent systems

We address the problem of sparsity-promoting optimal control of cyber-physical systems (CPSs) in the presence of communication delays. The delays are categorized into two types - namely, an inter-layer delay for passing state and control information between the physical layer and the cyber layer, and an intra-layer delay that operates between the computing agents, referred to here as control nodes (CNs), within the cyber-layer. Our objective is to minimize the closed-loop H2-norm of the physical system by co-designing an optimal combination of these two delays and a sparse state-feedback controller while respecting a given bandwidth cost constraint. We propose a two-loop optimization algorithm for this. Based on the alternating directions method of multipliers (ADMM), the inner loop handles the conflicting directions between the decreasing H2-norm and the increasing sparsity level of the controller. The outer loop comprises a semidefinite program (SDP)-based relaxation of non-convex inequalities necessary for closed-loop stability. Moreover, for CPSs where the state and control information assigned to the CNs are not private, we derive an additional algorithm that further sparsifies the communication topology by modifying the row and column structures of the obtained controller, resulting in reassigning the communication map between the cyber and physical layers, and determining which physical agent should send its state information to which CN. Proofs for closed-loop stability and optimality are provided for both algorithms, followed by numerical simulations., Comment: arXiv admin note: text overlap with arXiv:2003.05888

Individual agents in a multi-agent system (MAS) may have decoupled open-loop dynamics, but a cooperative control objective usually results in coupled closed-loop dynamics thereby making the control design computationally expensive. The computation time becomes even higher when a learning strategy such as reinforcement learning (RL) needs to be applied to deal with the situation when the agents dynamics are not known. To resolve this problem, we propose a parallel RL scheme for a linear quadratic regulator (LQR) design in a continuous-time linear MAS. The idea is to exploit the structural properties of two graphs embedded in the $Q$ and $R$ weighting matrices in the LQR objective to define an orthogonal transformation that can convert the original LQR design to multiple decoupled smaller-sized LQR designs. We show that if the MAS is homogeneous then this decomposition retains closed-loop optimality. Conditions for decomposability, an algorithm for constructing the transformation matrix, a parallel RL algorithm, and robustness analysis when the design is applied to non-homogeneous MAS are presented. Simulations show that the proposed approach can guarantee significant speed-up in learning without any loss in the cumulative value of the LQR cost., Comment: This paper contains proofs of all the theorems in the conference paper "Decomposability and Parallel Computation of Multi-Agent LQR"

Designing the optimal linear quadratic regulator (LQR) for a large-scale multi-agent system (MAS) is time-consuming since it involves solving a large-size matrix Riccati equation. The situation is further exasperated when the design needs to be done in a model-free way using schemes such as reinforcement learning (RL). To reduce this computational complexity, we decompose the large-scale LQR design problem into multiple smaller-size LQR design problems. We consider the objective function to be specified over an undirected graph, and cast the decomposition as a graph clustering problem. The graph is decomposed into two parts, one consisting of independent clusters of connected components, and the other containing edges that connect different clusters. Accordingly, the resulting controller has a hierarchical structure, consisting of two components. The first component optimizes the performance of each independent cluster by solving the smaller-size LQR design problem in a model-free way using an RL algorithm. The second component accounts for the objective coupling different clusters, which is achieved by solving a least squares problem in one shot. Although suboptimal, the hierarchical controller adheres to a particular structure as specified by inter-agent couplings in the objective function and by the decomposition strategy. Mathematical formulations are established to find a decomposition that minimizes the number of required communication links or reduces the optimality gap. Numerical simulations are provided to highlight the pros and cons of the proposed designs., Comment: This paper proposes a hierarchical learning and control framework for model-free LQR of heterogeneous linear multi-agent systems

We propose a new reinforcement learning based approach to designing hierarchical linear quadratic regulator (LQR) controllers for heterogeneous linear multi-agent systems with unknown state-space models and separated control objectives. The separation arises from grouping the agents into multiple non-overlapping groups, and defining the control goal as two distinct objectives. The first objective aims to minimize a group-wise block-decentralized LQR function that models group-level mission. The second objective, on the other hand, tries to minimize an LQR function between the average states (centroids) of the groups. Exploiting this separation, we redefine the weighting matrices of the LQR functions in a way that they allow us to decouple their respective algebraic Riccati equations. Thereafter, we develop a reinforcement learning strategy that uses online measurements of the agent states and the average states to learn the respective controllers based on the approximate Riccati equations. Since the first controller is block-decentralized and, therefore, can be learned in parallel, while the second controller is reduced-dimensional due to averaging, the overall design enjoys a significantly reduced learning time compared to centralized reinforcement learning.

We formulate a Stackelberg game between an attacker and a defender of a power system. The attacker attempts to alter the load setpoints of the power system covertly and intelligently, so that the voltage stability margin of the grid is reduced, driving the entire system towards a voltage collapse. The defender, or the system operator, aims to compensate for this reduction by retuning the reactive power injection to the grid by switching on control devices, such as a bank of shunt capacitors. A modified Backward Induction method is proposed to find a cost-based Stackelberg equilibrium (CBSE) of the game, which saves the players' costs while providing the optimal allocation of both players' investment resources under budget and covertness constraints. We analyze the proposed game extensively for the IEEE 9-bus power system model and present an example of its performance for the IEEE 39-bus power system model. It is demonstrated that the defender is able to maintain system stability unless its security budget is much lower than the attacker's budget., Comment: 6 pages in main paper, 6 pages in supplementary material, 5 figs, Main paper has been accepted by CDC 2020

We present a set of model-free, reduced-dimensional reinforcement learning (RL) based optimal control designs for linear time-invariant singularly perturbed (SP) systems. We first present a state-feedback and output-feedback based RL control design for a generic SP system with unknown state and input matrices. We take advantage of the underlying time-scale separation property of the plant to learn a linear quadratic regulator (LQR) for only its slow dynamics, thereby saving a significant amount of learning time compared to the conventional full-dimensional RL controller. We analyze the sub-optimality of the design using SP approximation theorems and provide sufficient conditions for closed-loop stability. Thereafter, we extend both designs to clustered multi-agent consensus networks, where the SP property reflects through clustering. We develop both centralized and cluster-wise block-decentralized RL controllers for such networks, in reduced dimensions. We demonstrate the details of the implementation of these controllers using simulations of relevant numerical examples and compare them with conventional RL designs to show the computational benefits of our approach.

Eigensystem Realization Algorithm (ERA) is a data-driven approach for subspace system identification and is widely used in many areas of engineering. However, the computational cost of the ERA is dominated by a step that involves the singular value decomposition (SVD) of a large, dense matrix with block Hankel structure. This paper develops computationally efficient algorithms for reducing the computational cost of the SVD step by using randomized subspace iteration and exploiting the block Hankel structure of the matrix. We provide a detailed analysis of the error in the identified system matrices and the computational cost of the proposed algorithms. We demonstrate the accuracy and computational benefits of our algorithms on two test problems: the first involves a partial differential equation that models the cooling of steel rails, and the second is an application from power systems engineering.

We address the problem of sparsity-promoting optimal control of cyber-physical systems with feedback delays. The delays are categorized into two classes - namely, intra-layer delay, and inter-layer delay between the cyber and the physical layers. Our objective is to minimize the H2-norm of the closed-loop system by designing an optimal combination of these two delays along with a sparse state-feedback controller, while respecting a given bandwidth constraint. We propose a two-loop optimization algorithm for this. The inner loop, based on alternating directions method of multipliers (ADMM), handles the conflicting directions of decreasing H2-norm and increasing sparsity of the controller. The outer loop comprises of semidefinite program (SDP)-based relaxations of non-convex inequalities necessary for stable co-design of the delays with the controller. We illustrate this algorithm using simulations that highlight various aspects of how delays and sparsity impact the stability and $\mc{H}_2$ performance of a LTI system.

In this paper, we propose a fast reinforcement learning (RL) control algorithm that enables online control of large-scale networked dynamic systems. RL is an effective way of designing model-free linear quadratic regulator (LQR) controllers for linear time-invariant (LTI) networks with unknown state-space models. However, when the network size is large, conventional RL can result in unacceptably long learning times. The proposed approach is to construct a compressed state vector by projecting the measured state through a projective matrix. This matrix is constructed from online measurements of the states in a way that it captures the dominant controllable subspace of the open-loop network model. Next, a RL-controller is learned using the reduced-dimensional state instead of the original state such that the resultant cost is close to the optimal LQR cost. Numerical benefits as well as the cyber-physical implementation benefits of the approach are verified using illustrative examples including an example of wide-area control of the IEEE 68-bus benchmark power system., Comment: 12 pages

Multi-agent networked control systems (NCSs) are often subject to model uncertainty and are limited by large communication cost, associated with feedback of data between the system nodes. To provide robustness against model uncertainty and to reduce the communication cost, this paper investigates the mixed $H_2/H_{\infty}$ control problem for NCS under the sparsity constraint. First, proximal alternating linearized minimization (PALM) is employed to solve the centralized social optimization where the agents have the same optimization objective. Next, we investigate a sparsity-constrained noncooperative game, which accommodates different control-performance criteria of different agents, and propose a best-response dynamics algorithm based on PALM that converges to an approximate Generalized Nash Equilibrium (GNE) of this game. A special case of this game, where the agents have the same $H_2$ objective, produces a partially-distributed social optimization solution. We validate the proposed algorithms using a network with unstable node dynamics and demonstrate the superiority of the proposed PALM-based method to a previously investigated sparsity-constrained mixed $H_2/H_{\infty}$ controller.

Recent years have seen several new directions in the design of sparse control of cyber-physical systems (CPSs) driven by the objective of reducing communication cost. One common assumption made in these designs is that the communication happens over a dedicated network. For many practical applications, however, communication must occur over shared networks, leading to two critical design challenges, namely - time-delays in the feedback and fair sharing of bandwidth among users. In this paper, we present a set of sparse H2 control designs under these two design constraints. An important aspect of our design is that the delay itself can be a function of sparsity, which leads to an interesting pattern of trade-offs in the H2 performance. We present three distinct algorithms. The first algorithm preconditions the assignable bandwidth to the network and produces an initial guess for a stabilizing controller. This is followed by our second algorithm, which sparsifies this controller while simultaneously adapting the feedback delay and optimizing the H2 performance using alternating directions method of multipliers (ADMM). The third algorithm extends this approach to a multiple user scenario where optimal number of communication links, whose total sum is fixed, is distributed fairly among users by minimizing the variance of their H2 performances. The problem is cast as a difference-of-convex (DC) program with mixed-integer linear program (MILP) constraints. We provide theorems to prove convergence of these algorithms, followed by validation through numerical simulations., Comment: Preliminary version appeared at American Control Conference (ACC) 2018

This paper presents a new counter-measure to mitigate denial-of-service cyber-attacks in linear time-invariant (LTI) systems. We first design a sparse linear quadratic regulator (LQR) optimal controller for a given LTI plant and evaluate the priority of the feedback communication links in terms of the loss of closed-loop performance when the corresponding block of the feedback gain matrix is removed. An attacker may know about this priority ordering, and thereby attack the links with the highest priority. To prevent this, we present a message rerouting strategy by which the states that are scheduled to be transmitted through the high priority links can be rerouted through lower priority ones in case the former get attacked. Since the attacked link is not available for service, and the states of the low priority links can no longer be accommodated either, we run a structured $\mathcal{H}_2$ control algorithm to determine the post-attack optimal feedback gains. We illustrate various aspects of the proposed algorithms by simulations.

This paper presents a mathematical analysis of how wind generation impacts the coherency property of power systems. Coherency arises from time-scale separation in the dynamics of synchronous generators, where generator states inside a coherent area synchronize over a fast time-scale due to stronger coupling, while the areas themselves synchronize over a slower time-scale due to weaker coupling. This time-scale separation is reflected in the form of a spectral separation in the weighted Laplacian matrix describing the swing dynamics of the generators. However, when wind farms with doubly-fed induction generators (DFIG) are integrated in the system then this Laplacian matrix changes based on both the level of wind penetration and the location of the wind farms. The modified Laplacian changes the effective slow eigenspace of the generators. Depending on penetration level, this change may result in changing the identities of the coherent areas. We develop a theoretical framework to quantify this modification, and validate our results with numerical simulations of the IEEE 68-bus system with one and multiple wind farms. We compare our model based results on clustering with results using measurement-based principal component analysis to substantiate our derivations., Comment: 8 pages, 11 figures

We formulate a resource-planning game between an attacker and a defender of a network control system. We consider the network to be operating in closed-loop with a linear quadratic regulator (LQR). We construct a general-sum, two-player, mixed strategy game, where the attacker attempts to destroy communication equipment of some nodes, and thereby render the LQR feedback gain matrix to be sparse, leading to degradation of closed-loop performance. The defender, on the other hand, aims to prevent this loss. Both players trade their control performance objectives for the cost of their actions. A Mixed Strategy Nash Equilibrium (MSNE) of the game represents the allocation of the players' respective resources for attacking or protecting the network nodes. We analyze the dependence of a MSNE on the relative budgets of the players as well as on the important network nodes that must be preserved to achieve a desirable LQR performance. MSNE is computed using nonlinear programming. Results are validated using the New England power system model, and it is shown that reliable defense is feasible unless the cost of attack is very low or much smaller than the cost of protection per generator.

In this paper we present an online wide-area oscillation damping control (WAC) design for uncertain models of power systems using ideas from reinforcement learning. We assume that the exact small-signal model of the power system at the onset of a contingency is not known to the operator and use the nominal model and online measurements of the generator states and control inputs to rapidly converge to a state-feedback controller that minimizes a given quadratic energy cost. However, unlike conventional linear quadratic regulators (LQR), we intend our controller to be sparse, so its implementation reduces the communication costs. We, therefore, employ the gradient support pursuit (GraSP) optimization algorithm to impose sparsity constraints on the control gain matrix during learning. The sparse controller is thereafter implemented using distributed communication. Using the IEEE 39-bus power system model with 1149 unknown parameters, it is demonstrated that the proposed learning method provides reliable LQR performance while the controller matched to the nominal model becomes unstable for severely uncertain systems., Comment: Submitted to IEEE ACC 2019. 8 pages, 4 figures

In this paper a nonlinear control design for power balancing in networked microgrids using energy storage devices is presented. Each microgrid is considered to be interfaced to the distribution feeder though a solid-state transformer (SST). The internal duty cycle based controllers of each SST ensures stable regulation of power commands during normal operation. But problem arises when a sudden change in load or generation occurs in any microgrid in a completely unpredicted way in between the time instants at which the SSTs receive their power setpoints. In such a case, the energy storage unit in that microgrid must produce or absorb the deficit power. The challenge lies in designing a suitable regulator for this purpose owing to the nonlinearity of the battery model and its coupling with the nonlinear SST dynamics. We design an input-output linearization based controller, and show that it guarantees closed-loop stability via a cascade connection with the SST model. The design is also extended to the case when multiple SSTs must coordinate their individual storage controllers to assist a given SST whose storage capacity is insufficient to serve the unpredicted load. The design is verified using the IEEE 34-bus distribution system with nine SST-driven microgrids., Comment: 8 pages, 10 figures

This article presents a suite of new control designs for next-generation electric smart grids. The future grid will consist of thousands of non-conventional renewable generation sources such as wind, solar, and energy storage. These new components are collectively referred to as distributed energy resources (DER). The article presents a comprehensive list of dynamic models for DERs, and shows their coupling with the conventional generators and loads. It then presents several innovative control designs that can be used for facilitating large-scale DER integration. Ideas from decentralized retrofit control and distributed sparsity-promoting optimal control are used for developing these designs, followed by illustrations on an IEEE power system test model., Comment: 36 pages, 24 figures

Wide-area control (WAC) has been shown to be an effective tool for damping low-frequency oscillations in power systems. In the current state of art, WAC is challenged by two main factors - namely, scalability of design and complexity of implementation. In this paper we present a control design called control inversion that bypasses both of these challenges using the idea of clustering. The basic philosophy behind this method is to project the original power system model into a lower-dimensional state-space through clustering and aggregation of generator states, and then designing an LQR controller for the lower-dimensional model. This controller is finally projected back to the original coordinates for wide-area implementation. The main problem is, therefore, posed as finding the projection which best matches the closed-loop performance of the WAC controller with that of a reference LQR controller for damping low-frequency oscillations. We verify the effectiveness of the proposed design using the NPCC 48-machine power system model., Comment: Submitted to IEEE Transactions on Control of Network Systems

We study nonlinear power systems consisting of generators, generator buses, and non-generator buses. First, looking at a generator and its bus' variables jointly, we introduce a synchronization concept for a pair of such joint generators and buses. We show that this concept is related to graph symmetry. Next, we extend, in two ways, the synchronization from a pair to a partition of all generators in the networks and show that they are related to either graph symmetry or equitable partitions. Finally, we show how an exact reduced model can be obtained by aggregating the generators and associated buses in the network when the original system is synchronized with respect to a partition, provided that the initial condition respects the partition. Additionally, the aggregation-based reduced model is again a power system., Comment: 7 pages, 4 figures; accepted for the 17th European Control Conference, Limassol, Cyprus

The electric power grid is one of the largest and most complex infrastructures ever built by mankind. Modern civilization depends on it for industry production, human mobility, and comfortable living. However, many critical technologies such as the 60 Hz transformers were developed at the beginning of the 20th century and have changed very little since then.1 The traditional unidirectional power from the generation to the customer through the transmission-distribution grid has also changed nominally, but it no longer meets the need of the 21st century market energy customers. On one hand, 128m US residential customers pay $15B/per month for their utility bill, yet they have no option to select their energy supplier. In a world of where many traditional industries are transformed by digital Internet technology (Amazon, Ebay, Uber, Airbnb), the traditional electric energy market is lagging significantly behind. A move towards a true digital grid is needed. Such a digital grid requires a tight integration of the physical layer (energy and power) with digital and cyber information to allow an open and real time market akin to the world of e-commerce. Another major factor that is pushing for this radical transformation are the rapidly changing patterns in energy resources ownership and load flow. Driven by the decreasing cost in distributed solar, energy storage, electric vehicle, on site generation and microgrids, the high penetration of Distributed Energy Resource (DER) is shifting challenges substantially towards the edge of grid from the control point of view. The envisioned Digital Grid must facilitate the open competition and open innovation needed to accelerate of the adoption of new DER technologies while satisfying challenges in grid stability, data explosion and cyber security., Comment: A Computing Community Consortium (CCC) white paper, 3 pages

The solution space of any set of power flow equations may contain different number of real-valued solutions. The boundaries that separate these regions are referred to as power flow solution space boundaries. Knowledge of these boundaries is important as they provide a measure for voltage stability. Traditionally, continuation based methods have been employed to compute these boundaries on the basis of initial guesses for the solution. However, with rapid growth of renewable energy sources these boundaries will be increasingly affected by variable parameters such as penetration levels, locations of the renewable sources, and voltage set-points, making it difficult to generate an initial guess that can guarantee all feasible solutions for the power flow problem. In this paper we solve this problem by applying a numerical polynomial homotopy based continuation method. The proposed method guarantees to find all solution boundaries within a given parameter space up to a chosen level of discretization, independent of any initial guess. Power system operators can use this computational tool conveniently to plan the penetration levels of renewable sources at different buses. We illustrate the proposed method through simulations on 3-bus and 10-bus power system examples with renewable generation., Comment: 9 pages, 5 figures

Standard H2 optimal control of networked dynamic systems tend to become unscalable with network size. Structural constraints can be imposed on the design to counteract this problem albeit at the risk of making the solution non-convex. In this paper, we present a special class of structural constraints such that the H2 design satisfies a quadratic invariance condition, and therefore can be reformulated as a convex problem. This special class consists of structured and weighted projections of the input and output spaces. The choice of these projections can be optimized to match the closed-loop performance of the reformulated controller with that of the standard H2 controller. The advantage is that unlike the latter, the reformulated controller results in a hierarchical implementation which requires significantly lesser number of communication links, while also admitting model and controller reduction that helps the design to scale computationally. We illustrate our design with simulations of a 500-node network., Comment: Submitted to 2018 American Control Conference

In this paper we address the growing concerns of wind power integration from the perspective of power system dynamics and stability. We propose a new retrofit control technique where an additional controller is designed at the doubly-fed induction generator site inside the wind power plant. This controller cancels the adverse impacts of the power flow from the wind side to the grid side on the dynamics of the overall power system. The main advantage of this controller is that it can be implemented by feeding back only the wind states and wind bus voltage without depending on any of the other synchronous machines in the rest of the system. Through simulations of a 4-machine Kundur power system model we show that the retrofit can efficiently enhance the damping performance of the system variable despite very high values of wind penetration., Comment: in Proc. American Control Conference 2017

In this paper we present a set of projection-based designs for constructing simplified linear quadratic regulator (LQR) controllers for large-scale network systems. When such systems have tens of thousands of states, the design of conventional LQR controllers becomes numerically challenging, and their implementation requires a large number of communication links. Our proposed algorithms bypass these difficulties by clustering the system states using structural properties of its closed-loop transfer matrix. The assignment of clusters is defined through a structured projection matrix P, which leads to a significantly lower-dimensional LQR design. The reduced-order controller is finally projected back to the original coordinates via an inverse projection. The problem is, therefore, posed as a model matching problem of finding the optimal set of clusters or P that minimizes the H2-norm of the error between the transfer matrix of the full-order network with the full-order LQR and that with the projected LQR. We derive a tractable relaxation for this model matching problem, and design a P that solves the relaxation. The design is shown to be implementable by a convenient, hierarchical two-layer control architecture, requiring far less number of communication links than full-order LQR., Comment: Submitted to Transactions on Automatic Control, Oct 4, 2016. (under review)

In this paper we develop a set of algorithms that can detect the identities of malicious data-manipulators in distributed optimization loops for estimating oscillation modes in large power system models. The estimation is posed in terms of a consensus problem among multiple local estimators that jointly solve for the characteristic polynomial of the network model. If any of these local estimates are compromised by a malicious attacker, resulting in an incorrect value of the consensus variable, then the entire estimation loop can be destabilized. We present four iterative algorithms by which this instability can be quickly detected, and the identities of the compromised estimators can be revealed. The algorithms are solely based on the computed values of the estimates, and do not need any information about the model of the power system. Both large and covert attacks are considered. Results are illustrated using simulations of a IEEE 68-bus power system model., Comment: 15 pages, 23 figures, and 2 tables

Multi-agent networked linear dynamic systems have attracted attention of researchers in power systems, intelligent transportation, and industrial automation. The agents might cooperatively optimize a global performance objective, resulting in social optimization, or try to satisfy their own selfish objectives using a noncooperative differential game. However, in these solutions, large volumes of data must be sent from system states to possibly distant control inputs, thus resulting in high cost of the underlying communication network. To enable economically-viable communication, a game-theoretic framework is proposed under the \textit{communication cost}, or \textit{sparsity}, constraint, given by the number of communicating state/control input pairs. As this constraint tightens, the system transitions from dense to sparse communication, providing the trade-off between dynamic system performance and information exchange. Moreover, using the proposed sparsity-constrained distributed social optimization and noncooperative game algorithms, we develop a method to allocate the costs of the communication infrastructure fairly and according to the agents' diverse needs for feedback and cooperation. Numerical results illustrate utilization of the proposed algorithms to enable and ensure economic fairness of wide-area control among power companies.

In this paper we present a sufficient condition that guarantees identifiability of linear network dynamic systems exhibiting continuous-time weighted consensus protocols with acyclic structure. Each edge of the underlying network graph $\mathcal G$ of the system is defined by a constant parameter, referred to as the weight of the edge, while each node is defined by a scalar state whose dynamics evolve as the weighted linear combination of its difference with the states of its neighboring nodes. Following the classical definitions of identifiability and indistinguishability, we first derive a condition that ensure the identifiability of the edge weights of $\mathcal G$ in terms of the associated transfer function. Using this characterization, we propose a sensor placement algorithm that guarantees identifiability of the edge weights. We describe our results using several illustrative examples., Comment: 7 pages, 6 figures, 1 table

In this paper we investigate how the equilibrium characteristics of conventional power systems may change with an increase in wind penetration. We first derive a differential-algebraic model of a power system network consisting of synchronous generators, loads and a wind power plant modeled by a wind turbine and a doubly-fed induction generator (DFIG). The models of these three components are coupled via nonlinear power flow equations. In contrast to the traditional approach for solving the power flows via iterative methods that often lead to only local solutions, we apply a recently developed parameter-homotopy based numerical continuation algorithm to compute all possible solutions. The method solves the power flow equations over multiple values of the wind penetration level with far less computational effort instead of solving them at each value individually. We observe that depending on the penetration limit and the setpoint value for the magnitude of the wind bus voltage, the system may exhibit several undesired or even unstable equilibria. We illustrate these results through a detailed simulation of a 5-machine power system model with wind injection, and highlight how the solutions may be helpful for small-signal stability assessment., Comment: 7 pages, 14 figures. Submitted to a Special Session of American Control Conference to be held in Palmer House Hilton, Chicago, IL, USA, in July 2015