Search

The proliferation of behind-the-meter (BTM) distributed energy resources (DER) within the electrical distribution network presents significant supply and demand flexibilities, but also introduces operational challenges such as voltage spikes and reverse power flows. In response, this paper proposes a network-aware dynamic pricing framework tailored for energy-sharing coalitions that aggregate small, but ubiquitous, BTM DER downstream of a distribution system operator's (DSO) revenue meter that adopts a generic net energy metering (NEM) tariff. By formulating a Stackelberg game between the energy-sharing market leader and its prosumers, we show that the dynamic pricing policy induces the prosumers toward a network-safe operation and decentrally maximizes the energy-sharing social welfare. The dynamic pricing mechanism involves a combination of a locational {\em ex-ante} dynamic price and an {\em ex-post} allocation, both of which are functions of the energy sharing's BTM DER. The {\em ex-post} allocation is proportionate to the price differential between the DSO NEM price and the energy sharing locational price. Simulation results using real DER data and the IEEE 13-bus test systems illustrate the dynamic nature of network-aware pricing at each bus, and its impact on voltage., Comment: 10 pages, 3 figures

Purpose This article presents a case for a next-generation grid monitoring and control system, leveraging recent advances in generative artificial intelligence (AI), machine learning, and statistical inference. Advancing beyond earlier generations of wide-area monitoring systems built upon supervisory control and data acquisition (SCADA) and synchrophasor technologies, we argue for a monitoring and control framework based on the streaming of continuous point-on-wave (CPOW) measurements with AI-powered data compression and fault detection. Methods and Results: The architecture of the proposed design originates from the Wiener-Kallianpur innovation representation of a random process that transforms causally a stationary random process into an innovation sequence with independent and identically distributed random variables. This work presents a generative AI approach that (i) learns an innovation autoencoder that extracts innovation sequence from CPOW time series, (ii) compresses the CPOW streaming data with innovation autoencoder and subband coding, and (iii) detects unknown faults and novel trends via nonparametric sequential hypothesis testing. Conclusion: This work argues that conventional monitoring using SCADA and phasor measurement unit (PMU) technologies is ill-suited for a future grid with deep penetration of inverter-based renewable generations and distributed energy resources. A monitoring system based on CPOW data streaming and AI data analytics should be the basic building blocks for situational awareness of a highly dynamic future grid.

This paper presents a generative artificial intelligence approach to probabilistic forecasting of electricity market signals, such as real-time locational marginal prices and area control error signals. Inspired by the Wiener-Kallianpur innovation representation of nonparametric time series, we propose a weak innovation autoencoder architecture and a novel deep learning algorithm that extracts the canonical independent and identically distributed innovation sequence of the time series, from which samples of future time series are generated. The validity of the proposed approach is established by proving that, under ideal training conditions, the generated samples have the same conditional probability distribution as that of the ground truth. Three applications involving highly dynamic and volatile time series in real-time market operations are considered: (i) locational marginal price forecasting for self-scheduled resources such as battery storage participants, (ii) interregional price spread forecasting for virtual bidders in interchange markets, and (iii) area control error forecasting for frequency regulations. Numerical studies based on market data from multiple independent system operators demonstrate the superior performance of the proposed generative forecaster over leading classical and modern machine learning techniques under both probabilistic and point forecasting metrics.

We propose an operating envelopes (OEs) aware energy community market mechanism that dynamically charges/rewards its members based on two-part pricing. The OEs are imposed exogenously by a regulated distribution system operator (DSO) on the energy community's revenue meter and is subject to a generalized net energy metering (NEM) tariff design. By formulating the interaction of the community operator and its members as a Stackelberg game, we show that the proposed two-part pricing achieves a Nash equilibrium and maximizes the community's social welfare in a decentralized fashion while ensuring that the community's operation abides by the OEs. The market mechanism conforms with the cost-causation principle and guarantees community members a surplus level no less than their maximum surplus when they autonomously face the DSO. The dynamic and uniform community price is a monotonically decreasing function of the community's aggregate renewable generation. We also analyze the impact of exogenous parameters such as NEM rates and OEs on the value of joining the community. Lastly, through numerical studies, we showcase the community's welfare, and pricing, and compare its members' surplus to customers under the DSO's regime., Comment: 19 pages, 7 figures, 3 tables

Generative probabilistic forecasting produces future time series samples according to the conditional probability distribution given past time series observations. Such techniques are essential in risk-based decision-making and planning under uncertainty with broad applications in grid operations, including electricity price forecasting, risk-based economic dispatch, and stochastic optimizations. Inspired by Wiener and Kallianpur's innovation representation, we propose a weak innovation autoencoder architecture and a learning algorithm to extract independent and identically distributed innovation sequences from nonparametric stationary time series. We show that the weak innovation sequence is Bayesian sufficient, which makes the proposed weak innovation autoencoder a canonical architecture for generative probabilistic forecasting. The proposed technique is applied to forecasting highly volatile real-time electricity prices, demonstrating superior performance across multiple forecasting measures over leading probabilistic and point forecasting techniques., Comment: Accepted at CISS 2024. arXiv admin note: text overlap with arXiv:2306.03782

We consider the problem of co-optimized energy-reserve market clearing with state-of-charge (SoC) dependent bids from battery storage participants. While SoC-dependent bidding accurately captures storage's degradation and opportunity costs, such bids result in a non-convex optimization in the market clearing process. More challenging is the regulation reserve capacity clearing, where the SoC-dependent cost is uncertain as it depends on the unknown regulation trajectories ex-post of the market clearing. Addressing the nonconvexity and uncertainty in a multi-interval co-optimized real-time energy-reserve market, we introduce a simple restriction on the SoC-dependent bids along with a robust optimization formulation, transforming the non-convex market clearing under uncertainty into a standard convex piece-wise linear program and making it possible for large-scale storage integration. Under reasonable assumptions, we show that SoC-dependent bids yield higher profit for storage participants than that from SoC-independent bids. Numerical simulations demonstrate a 28%-150% profit increase of the proposed SoC-dependent bids compared with the SoC-independent counterpart.

We analyze the overall benefits of an energy community cooperative game under which distributed energy resources (DER) are shared behind a regulated distribution utility meter under a general net energy metering (NEM) tariff. Two community DER scheduling algorithms are examined. The first is a community with centrally controlled DER, whereas the second is decentralized letting its members schedule their own DER locally. For both communities, we prove that the cooperative game's value function is superadditive, hence the grand coalition achieves the highest welfare. We also prove the balancedness of the cooperative game under the two DER scheduling algorithms, which means that there is a welfare re-distribution scheme that de-incentivizes players from leaving the grand coalition to form smaller ones. Lastly, we present five ex-post and an ex-ante welfare re-distribution mechanisms and evaluate them in simulation, in addition to investigating the performance of various community sizes under the two DER scheduling algorithms., Comment: 8 pages

We consider the problem of merchant storage participating in the regulation market with state-of-charge (SoC) dependent bids. Because storage can simultaneously provide regulation up and regulation down capacities, the market-clearing engine faces the computation challenge of evaluating storage costs under different regulation scenarios. One approach is to employ a bilevel optimization that minimizes the worst-case storage cost among all potential regulation events. However, subproblems of such a bilevel optimization are nonconvex, resulting in prohibitive computation challenges for the real-time clearing of the regulation market. We show that the complex nonconvex market clearing problem can be convexified by a simple restriction on the SoC-dependent bid, rendering the intractable market clearing computation to standard linear programs. Numerical simulations demonstrate that SoC-dependent bids satisfying the convexification conditions increase the profits of merchant storage owners by 12.32-77.38% compared with SoC-independent bids.

We consider the problem of hydrogen storage integration in microgrids to improve the electricity supply resilience. Nonlinear effects from electrochemical models of electrolyzers and fuel cells for hydrogen storage are considered, making scheduling under the nonlinear model intractable and the conventional linear approximation infeasible. A piecewise linear model approximation with feasibility projection is proposed, resulting in a computationally efficient model predictive control for hydrogen storage operation. Several resilience performance measures, such as loss-of-load, duration-of-outage, and system cost, are used in performance evaluation. Simulations for the proposed optimization demonstrated a 13%-48% reduction in duration-of-outage, a 6.4%-21.7% reduction in system cost, and a 95% reduction in loss-of-load for critical loads compared to the scheduling algorithm involving linear model approximation. The performance gap of the proposed optimization to the benchmark involving the accurate nonlinear electrochemical model is less than 1% in most metrics., Comment: 9 pages, 7 figure, PESGM2024

We propose an operating-envelope-aware, prosumer-centric, and efficient energy community that aggregates individual and shared community distributed energy resources and transacts with a regulated distribution system operator (DSO) under a generalized net energy metering tariff design. To ensure safe network operation, the DSO imposes dynamic export and import limits, known as dynamic operating envelopes, on end-users' revenue meters. Given the operating envelopes, we propose an incentive-aligned community pricing mechanism under which the decentralized optimization of community members' benefit implies the optimization of overall community welfare. The proposed pricing mechanism satisfies the cost-causation principle and ensures the stability of the energy community in a coalition game setting. Numerical examples provide insights into the characteristics of the proposed pricing mechanism and quantitative measures of its performance., Comment: 22 pages, 5 figures, 1 table. Presented at the 59th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, Sep., 2023

We consider the aggregation of distributed energy resources (DERs) by a profit-seeking DER aggregator (DERA), participating directly in the wholesale electricity market and constrained by the distribution network access. We propose a competitive DER aggregation that maximizes the DERA's profit subject to that each customer of the DERA gains no less surplus and pays no higher energy cost than that under the regulated retail tariff. The DERA participates in the wholesale electricity market as virtual storage with optimized generation offers and consumption bids derived from the DERA's competitive aggregation. Also derived are DERA's bid curves for the distribution network access and DERA's profitability when competing with the regulated retail tariff. We show that, with the same distribution network access, the proposed DERA's wholesale market participation achieves the same welfare-maximizing outcome as when its customers participate directly in the wholesale market. We empirically evaluate how many DERAs can survive in the long-run equilibrium. Numerical studies also compare the proposed DERA with existing benchmarks on surpluses of DERA's customers and DERA profits as functions of the DER adoption level and the distribution network access., Comment: 15 pages, 8 figures

Distributed energy resource aggregators (DERAs) must share the distribution network together with the distribution utility in order to participate in the wholesale electricity markets that are operated by independent system operators (ISOs). We propose a forward auction that a distribution system operator (DSO) can utilize to allocate distribution network access limits to DERAs. As long as the DERAs operate within their acquired limits, these limits define operating envelopes that guarantee distribution network security, thus defining a mechanism that requires no real-time intervention from the DSOs for DERAs to participate in the wholesale markets. Our auctions take the form of robust and risk-sensitive markets with bids/offers from DERAs and utility's operational costs. Properties of the proposed auction, e.g., resulting surpluses of DSO and the DERAs, and the auction prices, along with empirical performance studies, are presented., Comment: 12 page, 8 figure

We propose a social welfare maximizing market mechanism for an energy community that aggregates individual and community-shared energy resources under a general net energy metering (NEM) policy. Referred to as Dynamic NEM (D-NEM), the proposed mechanism dynamically sets the community NEM prices based on aggregated community resources, including flexible consumption, storage, and renewable generation. D-NEM guarantees a higher benefit to each community member than possible outside the community, and no sub-communities would be better off departing from its parent community. D-NEM aligns each member's incentive with that of the community such that each member maximizing individual surplus under D-NEM results in maximum community social welfare. Empirical studies compare the proposed mechanism with existing benchmarks, demonstrating its welfare benefits, operational characteristics, and responsiveness to NEM rates., Comment: 19 pages, 7 figures, 2 tables. arXiv admin note: text overlap with arXiv:2211.09360. Accepted for publication at the IEEE Transactions on Energy Markets, Policy and Regulation

Probabilistic time series forecasting predicts the conditional probability distributions of the time series at a future time given past realizations. Such techniques are critical in risk-based decision-making and planning under uncertainties. Existing approaches are primarily based on parametric or semi-parametric time-series models that are restrictive, difficult to validate, and challenging to adapt to varying conditions. This paper proposes a nonparametric method based on the classic notion of {\em innovations} pioneered by Norbert Wiener and Gopinath Kallianpur that causally transforms a nonparametric random process to an independent and identical uniformly distributed {\em innovations process}. We present a machine-learning architecture and a learning algorithm that circumvent two limitations of the original Wiener-Kallianpur innovations representation: (i) the need for known probability distributions of the time series and (ii) the existence of a causal decoder that reproduces the original time series from the innovations representation. We develop a deep-learning approach and a Monte Carlo sampling technique to obtain a generative model for the predicted conditional probability distribution of the time series based on a weak notion of Wiener-Kallianpur innovations representation. The efficacy of the proposed probabilistic forecasting technique is demonstrated on a variety of electricity price datasets, showing marked improvement over leading benchmarks of probabilistic forecasting techniques.

We consider the problem of behind-the-meter EV charging by a prosumer, co-optimized with rooftop solar, electric battery, and flexible consumptions such as water heaters and HVAC. Under the time-of-use net energy metering tariff with the stochastic solar production and random EV charging demand, a finite-horizon surplus-maximization problem is formulated. We show that a procrastination threshold policy that delays EV charging to the last possible moment is optimal when EV charging is co-optimized with flexible demand, and the policy thresholds can be computed easily offline. When battery storage is part of the co-optimization, it is shown that the net consumption of the prosumer is a two-threshold piecewise linear function of the behind-the-meter renewable generation under the optimal policy, and the procrastination threshold policy remains optimal, although the thresholds cannot be computed easily. We propose a simple myopic solution and demonstrate in simulations that the performance gap between the myopic policy and an oracle upper bound appears to be 0.5-7.5%.

We consider the problem of co-optimizing behind-the-meter (BTM) storage and flexible demands with BTM stochastic renewable generation. Under a generalized net energy metering (NEM) policy-NEM X, we show that the optimal co-optimization policy schedules the flexible demands based on a load priority list that defers less prioritized loads to times when the BTM generation is abundant. This gives rise to the notion of a net-zero zone, which we quantify under different distributed energy resources (DER) compositions. We highlight the special case of inflexible demands that results in a storage policy that minimizes the imports and exports from and to the grid. Comparative statics are provided on the optimal co-optimization policy. Simulations using real residential data show the surplus gains of various customers under different DER compositions., Comment: 8 pages, 4 figures, 4 tables

We design a coordination mechanism between a distribution system operator (DSO) and distributed energy resource aggregators (DERAs) participating directly in the wholesale electricity market. Aimed at ensuring system reliability while providing open access to DERAs, the coordination mechanism includes a forward auction that allocates access limits to aggregators based on aggregators' bids that represent their benefits of aggregation. The proposed coordination mechanism results in decoupled DSO and DERAs operations that satisfy the network constraints, independent of the stochasticity of the renewables, wholesale real-time locational marginal prices, and individual DERA's aggregation policy. Optimal bidding strategies by competitive aggregators are also derived. The efficiency of the coordination mechanism and the locational price distribution at buses of a radial distribution grid are demonstrated for a 141-bus radial network., Comment: 7 pages, 5 figures

We consider the co-optimization of flexible household consumption, electric vehicle charging, and behind-the-meter distributed energy resources under the net energy metering tariff. Using a stochastic dynamic programming formulation, we show that the solution to the dynamic programming co-optimization is a procrastination threshold policy that delays and minimizes electricity purchasing for EV charging in each time interval. The policy thresholds can be computed off-line, simplifying the continuous action space dynamic optimization to decoupled closed-form charging and consumption decisions. Empirical studies using renewable, consumption, and EV data demonstrate the benefits of co-optimization.

We propose a social welfare maximizing mechanism for an energy community that aggregates individual and shared community resources under a general net energy metering (NEM) policy. Referred to as Dynamic NEM, the proposed mechanism adopts the standard NEM tariff model and sets NEM prices dynamically based on the total shared renewables within the community. We show that Dynamic NEM guarantees a higher benefit to each community member than possible outside the community. We further show that Dynamic NEM aligns the individual member's incentive with that of the overall community; each member optimizing individual surplus under Dynamic NEM results in maximum community's social welfare. Dynamic NEM is also shown to satisfy the cost-causation principle. Empirical studies using real data on a hypothetical energy community demonstrate the benefits to community members and grid operators., Comment: 9 pages, 5 figures

We consider novelty detection in time series with unknown and nonparametric probability structures. A deep learning approach is proposed to causally extract an innovations sequence consisting of novelty samples statistically independent of all past samples of the time series. A novelty detection algorithm is developed for the online detection of novel changes in the probability structure in the innovations sequence. A minimax optimality under a Bayes risk measure is established for the proposed novelty detection method, and its robustness and efficacy are demonstrated in experiments using real and synthetic datasets.

Wholesale market participation of storage with state-of-charge (SoC) dependent bids results in a non-convex cost in a multi-interval economic dispatch, which requires a mixed-integer linear program in the market clearing. We show that the economic dispatch can be convexified to the standard linear program when the SoC-dependent bid satisfies the equal decremental-cost ratio (EDCR) condition. Such EDCR bids are shown to support individual rationalities of all market participants in both the day-ahead multi-interval economic dispatch under locational marginal pricing and the rolling-window look-ahead dispatch under temporal-locational marginal pricing in the real-time market. A numerical example is presented to demonstrate a higher profit margin with an SoC-dependent bid over that from an SoC-independent bid., Comment: 8 pages, 4 figures, 2022 58th Allerton conference on communication, control, and computing

State-of-charge (SoC) dependent bidding allows merchant storage participants to incorporate SoC-dependent operation and opportunity costs in a bid-based market clearing process. However, such a bid results in a non-convex cost function in the multi-interval economic dispatch and market clearing, limiting its implementation in practice. We show that a simple restriction on the bidding format removes the non-convexity, making the multi-interval dispatch of SoC-dependent bids a standard convex piece-wise linear program., Comment: 5 pages, 2 figures

The co-optimization of behind-the-meter distributed energy resources is considered for prosumers under the net energy metering tariff. The distributed energy resources considered include renewable generations, flexible demands, and battery energy storage systems. An energy management system co-optimizes the consumptions and battery storage based on locally available stochastic renewables by solving a stochastic dynamic program that maximizes the expected operation surplus. To circumvent the exponential complexity of the dynamic program solution, we propose a closed-form and linear computation complexity co-optimization algorithm based on a relaxation-projection approach to a constrained stochastic dynamic program. Sufficient conditions for optimality for the proposed solution are obtained. Numerical studies demonstrate orders of magnitude reduction of computation costs and significantly reduced optimality gap., Comment: 20 pages, 8 figures, 2 tables

The problem of the large-scale aggregation of the behind-the-meter demand and generation resources by a distributed-energy-resource aggregator (DERA) is considered. As a profit-seeking wholesale market participant, a DERA maximizes its profit while providing competitive services to its customers with higher consumer/prosumer surpluses than those offered by the distribution utilities or community choice aggregators. A constrained profit maximization program for aggregating behind-the-meter generation and consumption resources is formulated, from which payment functions for the behind-the-meter consumptions and generations are derived. Also obtained are DERA's bid and offer curves for its participation in the wholesale energy market and the optimal schedule of behind-the-meter resources. It is shown that the proposed DERA's aggregation model can achieve market efficiency equivalent to that when its customers participate individually directly in the wholesale market., Comment: 9 pages, 3 figures

Solar hosting capacity analysis (HCA) assesses the ability of a distribution network to host distributed solar generation without seriously violating distribution network constraints. In this paper, we consider risk-sensitive HCA that limits the risk of network constraint violations with a collection of scenarios of solar irradiance and nodal power demands, where risk is modeled via the conditional value at risk (CVaR) measure. First, we consider the question of maximizing aggregate installed solar capacities, subject to risk constraints and solve it as a second-order cone program (SOCP) with a standard conic relaxation of the feasible set with power flow equations. Second, we design an incremental algorithm to decide whether a configuration of solar installations has acceptable risk of constraint violations, modeled via CVaR. The algorithm circumvents explicit risk computation by incrementally constructing inner and outer polyhedral approximations of the set of acceptable solar installation configurations from prior such tests conducted. Our numerical examples study the impact of risk parameters, the number of scenarios and the scalability of our framework., Comment: 10 pages, 7 figures, 2 tables

Pricing storage operation in the real-time market under demand and generation stochasticities is considered. A scenario-based stochastic rolling-window dispatch model is formulated for the real-time market, consisting of conventional generators, utility-scale storage, and distributed energy resource aggregators. We show that uniform pricing mechanisms require discriminative out-of-the-market uplifts, making settlements under locational marginal pricing (LMP) discriminative. It is shown that the temporal locational marginal pricing (TLMP) that adds nonuniform shadow prices of ramping and state-of-charge to LMP removes the need for out-of-the-market uplifts. Truthful bidding incentives are also established for price-taking participants under TLMP. Revenue adequacy and uplifts are evaluated in numerical simulations., Comment: 8 pages, 4 figures, 2022 Power Systems Computation Conference (PSCC)

The rapid growth of the behind-the-meter (BTM) distributed generation has led to initiatives to reform the net energy metering (NEM) policies to address pressing concerns of rising electricity bills, fairness of cost allocation, and the long-term growth of distributed energy resources. This article presents an analytical framework for the optimal prosumer consumption decision using an inclusive NEM X tariff model that covers existing and proposed NEM tariff designs. The structure of the optimal consumption policy lends itself to near closed-form optimal solutions suitable for practical energy management systems that are responsive to stochastic BTM generation and dynamic pricing. The short and long-run performance of NEM and feed-in tariffs (FiT) are considered under a sequential rate-setting decision process. Also presented are numerical results that characterize social welfare distributions, cross-subsidies, and long-run solar adoption performance for selected NEM and FiT policy designs., Comment: 20 pages, 11 figures, 6 tables

We consider the problem of uniformity testing of Lipschitz continuous distributions with bounded support. The alternative hypothesis is a composite set of Lipschitz continuous distributions that are at least $\varepsilon$ away in $\ell_1$ distance from the uniform distribution. We propose a sequential test that adapts to the unknown distribution under the alternative hypothesis. Referred to as the Adaptive Binning Coincidence (ABC) test, the proposed strategy adapts in two ways. First, it partitions the set of alternative distributions into layers based on their distances to the uniform distribution. It then sequentially eliminates the alternative distributions layer by layer in decreasing distance to the uniform, and subsequently takes advantage of favorable situations of a distant alternative by exiting early. Second, it adapts, across layers of the alternative distributions, the resolution level of the discretization for computing the coincidence statistic. The farther away the layer is from the uniform, the coarser the discretization is needed for eliminating/exiting this layer. It thus exits both early in the detection process and quickly by using a lower resolution to take advantage of favorable alternative distributions. The ABC test builds on a novel sequential coincidence test for discrete distributions, which is of independent interest. We establish the sample complexity of the proposed tests as well as a lower bound.

In [1], a single-period co-optimization model of energy and reserve is considered to better illustrate the properties of the co-optimization model and the associated market mechanism. To make the discussion more general, in this paper, the single-period co-optimization model (II) in [1] will be extended into a one-shot multi-period co-optimization model, where the coupling of ramping and reserve in multi-period operations will be considered. Also, the multi-period pricing approach and settlement process associated with the multi-period co-optimization model will be discussed. In addition, in case study, the average system total costs of the proposed multi-period model and the multi-period traditional model are compared, and the effectiveness of some market properties in multi-period operation is tested., Comment: 2 figures

We introduce NEM X, an inclusive retail tariff model that captures features of existing net energy metering (NEM) policies. It is shown that the optimal prosumer decision has three modes: (a) the net-consuming mode where the prosumer consumes more than its behind-the-meter distributed energy resource (DER) production when the DER production is below a predetermined lower threshold, (b) the net-producing mode where the prosumer consumes less than its DER production when the DER production is above a predetermined upper threshold, and (c) the net-zero energy mode where the prosumer's consumption matches to its DER generation when its DER production is between the lower and upper thresholds. Both thresholds are obtained in closed-form. Next, we analyze the regulator's rate-setting process that determines NEM X parameters such as retail/sell rates, fixed charges, and price differentials in time-of-use tariffs' on and off-peak periods. A stochastic Ramsey pricing program that maximizes social welfare subject to the revenue break-even constraint for the regulated utility is formulated. Performance of several NEM X policies is evaluated using real and synthetic data to illuminate impacts of NEM policy designs on social welfare, cross-subsidies of prosumers by consumers, and payback time of DER investments that affect long-run DER adoptions., Comment: 14 pages, 6 figures, 3 tables

We consider some crucial problems related to the secure and reliable operation of power systems with high renewable penetrations: how much reserve should we procure, how should reserve resources distribute among different locations, and how should we price reserve and charge uncertainty sources. These issues have so far been largely addressed empirically. In this paper, we first develop a scenario-oriented energy-reserve co-optimization model, which directly connects reserve procurement with possible outages and load/renewable power fluctuations without the need for empirical reserve requirements. Accordingly, reserve can be optimally procured system-wide to handle all possible future uncertainties with the minimum expected system total cost. Based on the proposed model, marginal pricing approaches are developed for energy and reserve, respectively. Locational uniform pricing is established for energy, and the similar property is also established for the combination of reserve and re-dispatch. In addition, properties of cost recovery for generators and revenue adequacy for the system operator are also proven., Comment: 12 pages, 6 figures

An innovations sequence of a time series is a sequence of independent and identically distributed random variables with which the original time series has a causal representation. The innovation at a time is statistically independent of the history of the time series. As such, it represents the new information contained at present but not in the past. Because of its simple probability structure, an innovations sequence is the most efficient signature of the original. Unlike the principle or independent component analysis representations, an innovations sequence preserves not only the complete statistical properties but also the temporal order of the original time series. An long-standing open problem is to find a computationally tractable way to extract an innovations sequence of non-Gaussian processes. This paper presents a deep learning approach, referred to as Innovations Autoencoder (IAE), that extracts innovations sequences using a causal convolutional neural network. An application of IAE to the one-class anomalous sequence detection problem with unknown anomaly and anomaly-free models is also presented.

Time-synchronized state estimation for reconfigurable distribution networks is challenging because of limited real-time observability. This paper addresses this challenge by formulating a deep learning (DL)-based approach for topology identification (TI) and unbalanced three-phase distribution system state estimation (DSSE). Two deep neural networks (DNNs) are trained for time-synchronized DNN-based TI and DSSE, respectively, for systems that are incompletely observed by synchrophasor measurement devices (SMDs) in real-time. A data-driven approach for judicious SMD placement to facilitate reliable TI and DSSE is also provided. Robustness of the proposed methodology is demonstrated by considering non-Gaussian noise in the SMD measurements. A comparison of the DNN-based DSSE with more conventional approaches indicates that the DL-based approach gives better accuracy with smaller number of SMDs., Comment: 13 pages. arXiv admin note: substantial text overlap with arXiv:2011.04272

The problem of pricing utility-scale energy storage resources (ESRs) in the real-time electricity market is considered. Under a rolling-window dispatch model where the operator centrally dispatches generation and consumption under forecasting uncertainty, it is shown that almost all uniform pricing schemes, including the standard locational marginal pricing (LMP), result in lost opportunity costs that require out-of-the-market settlements. It is also shown that such settlements give rise to disincentives for generating firms and storage participants to bid truthfully, even when these market participants are rational price-takers in a competitive market. Temporal locational marginal pricing (TLMP) is proposed for ESRs as a generalization of LMP to an in-market discriminative form. TLMP is a sum of the system-wide energy price, LMP, and the individual state-of-charge price. It is shown that, under arbitrary forecasting errors, the rolling-window implementation of TLMP eliminates the lost opportunity costs and provides incentives to price-taking firms to bid truthfully with their marginal costs. Numerical examples show insights into the effects of uniform and non-uniform pricing mechanisms on dispatch following and truthful bidding incentives., Comment: 7 pages, 3 figures

We consider the classical problem of missing-mass estimation, which deals with estimating the total probability of unseen elements in a sample. The missing-mass estimation problem has various applications in machine learning, statistics, language processing, ecology, sensor networks, and others. The naive, constrained maximum likelihood (CML) estimator is inappropriate for this problem since it tends to overestimate the probability of the observed elements. Similarly, the conventional constrained Cramer-Rao bound (CCRB), which is a lower bound on the mean-squared-error (MSE) of unbiased estimators, does not provide a relevant bound on the performance for this problem. In this paper, we introduce a frequentist, non-Bayesian parametric model of the problem of missing-mass estimation. We introduce the concept of missing-mass unbiasedness by using the Lehmann unbiasedness definition. We derive a non-Bayesian CCRB-type lower bound on the missing-mass MSE (mmMSE), named the missing-mass CCRB (mmCCRB), based on the missing-mass unbiasedness. The missing-mass unbiasedness and the proposed mmCCRB can be used to evaluate the performance of existing estimators. Based on the new mmCCRB, we propose a new method to improve existing estimators by an iterative missing-mass Fisher scoring method. Finally, we demonstrate via numerical simulations that the proposed mmCCRB is a valid and informative lower bound on the mmMSE of state-of-the-art estimators for this problem: the CML, the Good-Turing, and Laplace estimators. We also show that the performance of the Laplace estimator is improved by using the new Fisher-scoring method.

Optimal scheduling of deferrable electrical loads can reshape the aggregated load profile to achieve higher operational efficiency and reliability. This paper studies deferrable load scheduling under demand charge that imposes a penalty on the peak consumption over a billing period. Such a terminal cost poses challenges in real-time dispatch when demand forecasts are inaccurate. A block model-predictive control approach is proposed by breaking demand charge into a sequence of stage costs. The problem of charging electric vehicles is used to illustrate the efficacy of the proposed approach. Numerical examples show that the block model-predictive control outperforms benchmark methods in various settings., Comment: 10 pages, 4 plots

A deep learning approach is proposed to detect data and system anomalies using high-resolution continuous point-on-wave (CPOW) or phasor measurements. Both the anomaly and anomaly-free measurement models are assumed to have unknown temporal dependencies and probability distributions. Historical training samples are assumed for the anomaly-free model, while no training samples are available for the anomaly measurements. By transforming the anomaly-free observations into uniform independent and identically distributed sequences via a generative adversarial network, the proposed approach deploys a uniformity test for anomaly detection at the sensor level. A distributed detection scheme that combines sensor level detections at the control center is also proposed that combines local detections to form more reliable detections. Numerical results demonstrate significant improvement over the state-of-the-art solutions for various bad-data cases using real and synthetic CPOW and PMU data sets., Comment: 10 pages, 7 figures

This paper studies the probabilistic hosting capacity analysis (PHCA) problem in distribution networks considering uncertainties from distributed energy resources (DERs) and residential loads. PHCA aims to compute the hosting capacity, which is defined as the maximal level of DERs that can be securely integrated into a distribution network while satisfying operational constraints with high probability. We formulate PHCA as a chance-constrained optimization problem, and model the uncertainties from DERs and loads using historical data. Due to non-convexities and a substantial number of historical scenarios being used, PHCA is often formulated as large-scale nonlinear optimization problem, thus computationally intractable to solve. To address the core computational challenges, we propose a fast and extensible framework to solve PHCA based on Bayesian Optimization (BayesOpt). Comparing with state-of-the-art algorithms such as interior point and active set, numerical results show that the proposed BayesOpt approach is able to find better solutions (25% higher hosting capacity) with 70% savings in computation time on average.

We propose a scenario-oriented approach for energy-reserve joint procurement and pricing for electricity market. In this model, without the empirical reserve requirements, reserve is procured according to all possible contingencies and load/renewable generation fluctuations with the minimum expected system total cost. The innovative locational marginal pricing approach for loads, generations and reserve and the associated settlement process are proposed. We show that payments from loads, payments to generators and congestion rent will reach their balance in the basecase as well as in all scenarios, so that revenue adequacy can be guaranteed for the system operator.

Time-synchronized state estimation is a challenge for distribution systems because of limited real-time observability. This paper addresses this challenge by formulating a deep learning (DL)-based approach to perform unbalanced three-phase distribution system state estimation (DSSE). Initially, a data-driven approach for judicious measurement selection to facilitate reliable state estimation is provided. Then, a deep neural network (DNN) is trained to perform DSSE for systems that are incompletely observed by synchrophasor measurement devices (SMDs). Robustness of the proposed methodology is demonstrated by considering realistic measurement error models for SMDs. A comparative study of the DNN-based DSSE with classical linear state estimation indicates that the DL-based approach gives better accuracy with a significantly smaller number of SMDs., Comment: 5 pages, 5 figures, 2021 IEEE Power and Energy Society General Meeting (PES GM)

In this paper, we develop a model to plan energy-efficient speed trajectories of electric trucks in real-time by taking into account the information of topography and traffic ahead of the vehicle. In this real time control model, a novel state-space model is first developed to capture vehicle speed, acceleration, and state of charge. We then formulate an energy minimization problem and solve it by an alternating direction method of multipliers (ADMM) method that exploits the structure of the problem. A model predictive control framework is then employed to deal with topographic and traffic uncertainties in real-time. An empirical study is conducted on the performance of the proposed eco-driving algorithm and its impact on battery degradation. The experimental results show that the energy consumption by using the developed method is reduced by up to 5.05%, and the battery life extended by as high as 35.35% compared to benchmarking solutions.

A data compression system capable of providing real-time streaming of high-resolution continuous point-on-wave (CPOW) and phasor measurement unit (PMU) measurements is proposed. Referred to as adaptive subband compression (ASBC), the proposed technique partitions the signal space into subbands and adaptively compresses subband signals based on each subband's active bandwidth. The proposed technique conforms to existing industry phasor measurement standards, making it suitable for streaming high-resolution CPOW and PMU data either in continuous or burst on-demand/event-triggered modes. Experiments on synthetic and real data show that ASBC reduces the CPOW sampling rates by several orders of magnitude for real-time streaming while maintaining the precision required by industry standards.

Tie-line scheduling in multi-area power systems in the US largely proceeds through a market-based mechanism called Coordinated Transaction Scheduling (CTS). We analyze this market mechanism through a game-theoretic lens. Our analysis characterizes the effect of market liquidity, market participants' forecasts about inter-area price spreads, transactions fees and coupling of CTS markets with up-to-congestion virtual transactions. Using real data, we empirically verify that CTS bidders can employ simple learning algorithms to discover Nash equilibria that support the conclusions drawn from equilibrium analysis.

Interconnection studies for distributed energy resources (DERs) can currently take months since they entail simulating a large number of power flow scenarios. If DERs are to be actively controlled, probabilistic hosting capacity analysis (PHCA) studies become more time-consuming since they require solving multiple optimal power flow (OPF) tasks. PHCA is expedited here by leveraging the powerful tool of multiparametric programming (MPP). Using an approximate grid model, optimal DER setpoints are decided by a quadratic program, which depends on analysis and uncertain parameters in a possibly nonlinear fashion. By reformulating this program, feasible and infeasible OPF instances alike are handled in a unified way to uniquely reveal the location, frequency, and severity of feeder constraint violations. The effect of voltage regulators is also captured by novel approximate models. Upon properly extending MPP to PHCA, we were able to find the exact minimizers for 518,400 OPF instances on the IEEE 123-bus feeder by solving only 6,905 of them, and 86,400 instances on a 1,160-bus feeder by solving only 2,111 instances. This accelerated PHCA by a factor of 10. Thus, a utility can promptly infer grid statistics using real-world data without a probabilistic characterization of uncertain parameters., Comment: submitted to IEEE Transactions on Smart Grid

The problem of detecting data anomaly is considered. Under the null hypothesis that models anomaly-free data, measurements are assumed to be from an unknown distribution with some authenticated historical samples. Under the composite alternative hypothesis, measurements are from an unknown distribution positive distance away from the distribution under the null hypothesis. No training data are available for the distribution of anomaly data. A semi-supervised deep learning technique based on an inverse generative adversary network is proposed., Comment: 5 pages, letter

Pricing multi-interval economic dispatch of electric power under operational uncertainty is considered in this two-part paper. Part I investigates dispatch-following incentives for generators under the locational marginal pricing (LMP) and temporal locational marginal pricing (TLMP) policies. Extending the theoretical results developed in Part I, Part II evaluates a broader set of performance measures under a general network model. For networks with power flow constraints, TLMP is shown to have an energy-congestion-ramping price decomposition. Under the one-shot dispatch and pricing model, this decomposition leads to a nonnegative merchandising surplus equal to the sum of congestion and ramping surpluses. It is also shown that, comparing with LMP, TLMP imposes a penalty on generators with limited ramping capabilities, thus giving incentives for generators to reveal their ramping limits truthfully and improve their ramping capacities. Several benchmark pricing mechanisms are evaluated under the rolling-window dispatch and pricing models. The performance measures considered are the level of out-of-the-market uplifts, the revenue adequacy of the system operator, consumer payment, generator profit, level of discriminative payment, and price volatility., Comment: 13 pages, 18 figures, transaction journal paper

Pricing multi-interval economic dispatch of electric power under operational uncertainty is considered in this two-part paper. Part I investigates dispatch-following incentives of profit-maximizing generators and shows that, under mild conditions, no uniform-pricing scheme for the rolling-window economic dispatch provides dispatch-following incentives that avoid discriminative out-of-the-market uplifts. A nonuniform pricing mechanism, referred to as the temporal locational marginal pricing (TLMP), is proposed. As an extension of the standard locational marginal pricing (LMP), TLMP takes into account both generation and ramping-induced opportunity costs. It eliminates the need for the out-of-the-market uplifts and guarantees full dispatch-following incentives regardless of the accuracy of the demand forecasts used in the dispatch. It is also shown that, under TLMP, a price-taking market participant has incentives to bid truthfully with its marginal cost of generation. Part II of the paper extends the theoretical results developed in Part I to more general network settings. It investigates a broader set of performance measures, including the incentives of the truthful revelation of ramping limits, revenue adequacy of the operator, consumer payments, generator profits, and price volatility under the rolling-window dispatch model with demand forecast errors., Comment: 14 pages, 3 figures, IEEE transactions on power systems

This paper develops a new method for voltage instability prediction using a recurrent neural network with long short-term memory. The method is aimed to be used as a supplementary warning system for system operators, capable of assessing whether the current state will cause voltage instability issues several minutes into the future. The proposed method use a long sequence-based network, where both real-time and historic data are used to enhance the classification accuracy. The network is trained and tested on the Nordic32 test system, wherecombinations of different operating conditions and contingency scenarios are generated using time-domain simulations. The method shows that almost all N-1 contingency test cases were predicted correctly, and N-1-1 contingency test cases were predicted with over 93 % accuracy only seconds after a disturbance. Further, the impact of sequence length is examined, showing that the proposed long sequenced-based method provides significantly better classification accuracy than both a feedforward neural network and a network using a shorter sequence., Comment: 8 pages

The problem of state estimation for unobservable distribution systems is considered. A deep learning approach to Bayesian state estimation is proposed for real-time applications. The proposed technique consists of distribution learning of stochastic power injection, a Monte Carlo technique for the training of a deep neural network for state estimation, and a Bayesian bad-data detection and filtering algorithm. Structural characteristics of the deep neural networks are investigated. Simulations illustrate the accuracy of Bayesian state estimation for unobservable systems and demonstrate the benefit of employing a deep neural network. Numerical results show the robustness of Bayesian state estimation against modeling and estimation errors and the presence of bad and missing data. Comparing with pseudo-measurement techniques, direct Bayesian state estimation via deep learning neural network outperforms existing benchmarks.