Search

We consider a new framework where a continuous, though bounded, random variable has unobserved bounds that vary over time. In the context of univariate time series, we look at the bounds as parameters of the distribution of the bounded random variable. We introduce an extended log-likelihood estimation and design algorithms to track the bound through online maximum likelihood estimation. Since the resulting optimization problem is not convex, we make use of recent theoretical results on stochastic quasiconvex optimization, to eventually derive an Online Normalized Gradient Descent algorithm. We illustrate and discuss the workings of our approach based on both simulation studies and a real-world wind power forecasting problem. [ABSTRACT FROM AUTHOR]

We propose and develop a new algorithm for trading wind energy in electricity markets, within an online learning and optimization framework. In particular, we combine a component-wise adaptive variant of the gradient descent algorithm with recent advances in the feature-driven newsvendor model. This results in an online offering approach capable of leveraging data-rich environments, while adapting to the nonstationary characteristics of energy generation and electricity markets, also with a minimal computational burden. The performance of our approach is analyzed based on several numerical experiments, showing both better adaptability to nonstationary uncertain parameters and significant economic gains. [ABSTRACT FROM AUTHOR]

We present a data-driven approach for probabilistic wind power forecasting based on conditional normalizing flow (CNF). In contrast with the existing, this approach is distribution-free (as for non-parametric and quantile-based approaches) and can directly yield continuous probability densities, hence avoiding quantile crossing. It relies on a base distribution and a set of bijective mappings. Both the shape parameters of the base distribution and the bijective mappings are approximated with neural networks. Spline-based conditional normalizing flow is considered owing to its non-affine characteristics. Over the training phase, the model sequentially maps input examples onto samples of base distribution, given the conditional contexts, where parameters are estimated through maximum likelihood. To issue probabilistic forecasts, one eventually maps samples of the base distribution into samples of a desired distribution. Case studies based on open datasets validate the effectiveness of the proposed model, and allows us to discuss its advantages and caveats with respect to the state of the art. [ABSTRACT FROM AUTHOR]

Energy forecasting has attracted enormous attention over the last few decades, with novel proposals related to the use of heterogeneous data sources, probabilistic forecasting, online learning, etc. A key aspect that emerged is that learning and forecasting may highly benefit from distributed data, though not only in the geographical sense. That is, various agents collect and own data that may be useful to others. In contrast to recent proposals that look into distributed and privacy-preserving learning (incentive-free), we explore here a framework called regression markets. There, agents aiming to improve their forecasts post a regression task, for which other agents may contribute by sharing their data for their features and get monetarily rewarded for it. The market design is for regression models that are linear in their parameters, and possibly separable, with estimation performed based on either batch or online learning. Both in-sample and out-of-sample aspects are considered, with markets for fitting models in-sample, and then for improving genuine forecasts out-of-sample. Such regression markets rely on recent concepts within interpretability of machine learning approaches and cooperative game theory, with Shapley additive explanations. Besides introducing the market design and proving its desirable properties, application results are shown based on simulation studies (to highlight the salient features of the proposal) and with real-world case studies. [ABSTRACT FROM AUTHOR]

Participants in electricity markets are becoming more proactive owing to the fast deployment of distributed energy resources (DERs) and the further development of demand-side management (DSM), which also boosts the emergence of Peer-to-Peer (P2P) market mechanisms. Moreover, the market is also required to operate in a real-time scheme in response to changes in generation and load to maintain power balance. It is highly desirable to propose and analyse novel approaches suitable for real-time P2P market mechanisms. These are challenging since most often involving a heavy computational burden, while the time available for negotiation in real-time is very short. Our core contribution is to design and analyse a novel asynchronous online optimization framework and related real-time P2P market negotiation mechanism, which can greatly reduce the computation and communication burden from two aspects. First, a novel online consensus alternating direction method of multipliers (ADMM) algorithm is proposed. It significantly reduces computation since only one iteration is performed for each agent at every time period. Second, the market operates in an asynchronous mode so that all agents can freely trade without waiting for idle or inactive neighboring agents. The sublinear regret upper bound is proved for our asynchronous online algorithm, which indicates that social welfare in the market can be maximized in the long run on average (over time). Simulations show that our algorithm enjoys good convergence performance, robustness, and fairness. [ABSTRACT FROM AUTHOR]

Electricity systems around the world are decarbonizing, driven by reductions in the cost of renewable energy and encouraged by supportive regulatory policy. Electricity market designs are increasingly being tested to ensure that the bulk power system can deliver reliable, cost-effective energy to all consumers. [ABSTRACT FROM AUTHOR]

In modern power systems, small distributed energy resources (DERs) are considered a valuable source of flexibility towards accommodating high penetration of Renewable Energy Sources (RES). In this paper we consider an economic dispatch problem for a community of DERs, where energy management decisions are made online and under uncertainty. We model multiple sources of uncertainty such as RES, wholesale electricity prices as well as the arrival times and energy needs of a set of Electric Vehicles. The economic dispatch problem is formulated as a multi-agent Markov Decision Process. The difficulties lie in the curse of dimensionality and in guaranteeing the satisfaction of constraints under uncertainty. A novel method, that combines duality theory and deep learning, is proposed to tackle these challenges. In particular, a Neural Network (NN) is trained to return the optimal dual variables of the economic dispatch problem. By training the NN on the dual problem instead of the primal, the number of output neurons is dramatically reduced, which enhances the performance and reliability of the NN. Finally, by treating the resulting dual variables as prices, each distributed agent can self-schedule, which guarantees the satisfaction of its constraints. As a result, our simulations show that the proposed scheme performs reliably and efficiently. [ABSTRACT FROM AUTHOR]

Current electricity and natural gas markets operate with deterministic description of uncertain supply, and in a temporally and sectorally decoupled way. This practice in energy systems is being challenged by the increasing integration of stochastic renewable energy sources. There is a growing need for exchanging operational flexibility among energy sectors, which requires to improve the sectoral coordination between electricity and natural gas markets. In addition, the dispatch of flexible units in both sectors needs to be made in a more uncertainty-aware manner, requiring to strengthen the temporal coordination between day-ahead and real-time energy markets. We explore the use of existing financial instruments in the form of virtual bidding (VB) as a market-based solution to enhance both sectoral and temporal coordination in energy markets. It is established in the literature that VB by purely financial players is able to enhance the temporal coordination between deterministic day-ahead and real-time markets. By developing various stochastic equilibrium and optimization models, we show that VB by physical players, i.e., gas-fired power plants, at the interface of power and natural gas systems is of great potential to improve not only the temporal coordination between deterministic day-ahead and real-time markets, but also the sectoral coordination between deterministic electricity and natural gas markets. We exploit a fully stochastic co-optimization model as an ideal benchmark, and numerically illustrate the benefits of VB for increasing the overall market efficiency in terms of reduced expected operational cost of the entire energy system. We eventually show that flexible resources in both electricity and natural gas markets are dispatched more efficiently in the day-ahead stage when VB exists. [ABSTRACT FROM AUTHOR]

Owing to the fast deployment of distributed energy resources (DERs) and the further development of demand-side management, small agents in electricity markets are becoming more proactive. This may boost the development of peer-to-peer (P2P) market mechanisms. Meanwhile, since actual load and power generation may substantially deviate from schedules obtained at the day-ahead (forward) market stage, it is needed to rapidly reschedule the trades among agents to maintain power balance through a real-time market mechanism, also in a P2P framework. However, it is technically challenging to develop and operate such P2P market mechanisms in real-time, since they most often involve a heavy computational burden (e.g., based on iterative distributed optimization approaches), while real-time trading demands fast calculation. Our core contribution is hence to describe and analyze a novel online optimization framework to enable the real-time P2P market. It relies on online social welfare maximization using a novel online consensus alternating direction method of multipliers (OC-ADMM) algorithm. The computational complexity is then heavily reduced since only one iteration is performed for each agent at every time step in order to satisfy real-time requirements. We derive a sublinear non-stationary regret upper bound for our algorithm, which implies that social welfare will be maximized in the long run. Simulations based on a number of case studies show that our algorithm has good convergence performance, tracking ability, and high computational efficiency. [ABSTRACT FROM AUTHOR]

Marginal Utility Functions (MUFs) encapsulate the prosumer willingness to trade energy with other market agents. In large-scale distributed optimization schemes scalability and convergence are crucial, and the assumption of MUFs is common. In this work, instead of assuming the shape and coefficients of those functions, a method is proposed to derive them based on the optimization of a prosumer's energy procurement problem. We formulate a rolling-horizon optimization problem that considers asset characteristics and network tariffs, and we utilize forecasts to capture the effects of uncertainty. We use this formulation to calculate the true prosumer MUF. A test case with real data is used to investigate the shape of these functions. Our results reveal that they present a non-linear shape under certain conditions, they cannot be a priori derived, and their form is sensitive to a variety of factors. This indicates that MUFs should be constructed based on an approach similar to the one proposed in this paper to more accurately capture the prosumer's true willingness to trade. A linearized version of these functions, which is computationally attractive and scalable, can be determined using a least-squares estimator that achieves the best fit to the prosumer's preferences under a linearity constraint. [ABSTRACT FROM AUTHOR]

The increased penetration of renewable energy sources into existing power systems induces challenges in supply-demand balancing. Demand-side flexibility is seen as an option to accommodate variability and limited predictability from renewable energy generation. Heat pumps at residential level, if well coordinated, can be one of those flexibility sources. The complexity involved is high though, since their coordinated operation combines control, population effects and the fact agents may actually not behave as rational decision-makers. We describe here a coordinated control framework that accounts for those aspects altogether. Decentralized model predictive control for large populations of heterogeneous agents is employed. As the cost to be minimized is affected by the population behaviour as a whole through the electricity price, the decentralized control is re-thought as a mean-field game. Existence and uniqueness of a Nash equilibrium are discussed while the Picard-Banach algorithm is used as a solution approach. It is extended to the case of bounded-rational agents. The impact on system dynamics of modelling agents as bounded rational is illustrated through numerical simulations. [ABSTRACT FROM AUTHOR]

Data exchange between multiple renewable energy power plant owners can lead to an improvement in forecast skill thanks to the spatio-temporal dependencies in time series data. However, owing to business competitive factors, these different owners might be unwilling to share their data. In order to tackle this privacy issue, this paper formulates a novel privacy-preserving framework that combines data transformation techniques with the alternating direction method of multipliers. This approach allows not only to estimate the model in a distributed fashion but also to protect data privacy, coefficients and covariance matrix. Besides, asynchronous communication between peers is addressed in the model fitting, and two different collaborative schemes are considered: centralized and peer-to-peer. The results for a solar energy dataset show that the proposed method is robust to privacy breaches and communication failures, and delivers a forecast skill comparable to a model without privacy protection. [ABSTRACT FROM AUTHOR]

In modern smart grids, the focus is increasingly shifted towards distributed energy resources and flexible electricity assets owned by prosumers. A system with high penetration of flexible prosumers, has a very large number of variables and constraints, while a lot of the information is local and non-observable. Decomposition methods and local problem solving is considered a promising approach for such settings, particularly when the implementation of a decomposition method features a market-based analogy, i.e. it can be implemented in a Transactive Energy fashion. In this paper we present an auction-theoretic scheme for a setting with non-convex prosumer models and resource constraints. The scheme is evaluated on a particular case study and its scalability and efficiency properties are tested and compared to an optimal benchmark solution. A game-theoretic analysis is made with respect to how an intelligent agent, that bids on behalf of a prosumer can try to strategize within the auction, in order to make itself better-off. Our simulations show that there is an alignment of incentives, i.e., when the prosumers try to strategize, they actually improve the auction's efficiency. [ABSTRACT FROM AUTHOR]

The large shares of wind power generation in electricity markets motivate higher levels of operating reserves. However, current reserve sizing practices fail to account for important topological aspects that might hinder their deployment, thus resulting in high operating costs. Zonal reserve procurement mitigates such inefficiencies, however, the way the zones are defined is still open to interpretation. This paper challenges the efficiency of predetermined zonal setups that neglect the location of stochastic power production in the system, as well as the availability, cost and accessibility of flexible generating units. To this end, we propose a novel reserve procurement approach, formulated as a two-stage stochastic bilevel model, in which the upper level identifies a number of contiguous reserve zones using dynamic grid partitioning and sets zonal requirements based on the total expected operating costs. Using two standard IEEE reliability test cases, we show how the efficient partitioning of reserve zones can reduce expected system cost and promote the integration of stochastic renewables. [ABSTRACT FROM AUTHOR]

The proliferation of distributed energy assets necessitates the provision of flexibility to efficiently operate modern distribution systems. In this article, we propose a flexibility market through which the DSO may acquire flexibility services from asset aggregators in order to maintain network voltages and currents within safe limits. A max-min fair formulation is proposed for the allocation of flexibility. Since the DSO is not aware of each aggregator’s local flexibility costs, we show that strategic misreporting can lead to severe loss of efficiency. Using mechanism design theory, we provide a mechanism that makes it a payoff-maximizing strategy for each aggregator to make truthful bids to the flexibility market. While typical truthful mechanisms only work when the objective is the maximization of Social Welfare, the proposed mechanism lets the DSO achieve incentive compatibility and optimality for the max-min fairness objective. [ABSTRACT FROM AUTHOR]

Although distribution grid customers are obliged to share their consumption data with distribution system operators (DSOs), a possible leakage of this data is often disregarded in operational routines of DSOs. This paper introduces a privacy-preserving optimal power flow (OPF) mechanism for distribution grids that secures customer privacy from unauthorised access to OPF solutions, e.g., current and voltage measurements. The mechanism is based on the framework of differential privacy that allows to control the participation risks of individuals in a dataset by applying a carefully calibrated noise to the output of a computation. Unlike existing private mechanisms, this mechanism does not apply the noise to the optimization parameters or its result. Instead, it optimizes OPF variables as affine functions of the random noise, which weakens the correlation between the grid loads and OPF variables. To ensure feasibility of the randomized OPF solution, the mechanism makes use of chance constraints enforced on the grid limits. The mechanism is further extended to control the optimality loss induced by the random noise, as well as the variance of OPF variables. The paper shows that the differentially private OPF solution does not leak customer loads up to specified parameters. [ABSTRACT FROM AUTHOR]

Large deployment of distribute energy resources and the increasing awareness of end-users towards their energy procurement are challenging current practices of electricity markets. A change of paradigm, from a top-down hierarchical approach to a more decentralized framework, has been recently researched, with market structures relying on multi-bilateral trades among market participants. In order to guarantee feasibility in power system operation, it is crucial to rethink the interaction with system operators and the way operational costs are shared in such decentralized markets. We propose here to include system operators, both at transmission and distribution level, as active actors of the market, accounting for power grid constraints and line losses. Moreover, to avoid market outcomes that discriminate agents for their geographical location, we analyze loss allocation policies and their impact on market outcomes and prices. [ABSTRACT FROM AUTHOR]

The rapid growth of distributed energy resources results in the proliferation of prosumers at the distribution level, who can both produce and consume energy. In order to harvest benefits of aggregation to obtain revenues in wholesale market and encourage direct interactions among prosumers and other flexible players at the same time, this paper proposes an aggregation framework for prosumers based on transactive energy, in which prosumers aggregate and trade in an independent, proactive and privacy-preserved manner without relying on a real central entity, while other flexible players could be benefited equally. The aggregation of prosumers as a whole behaves as a retailer that trades in wholesale market and towards end-users in retail market. Therefore, the problem is generalized as a bi-level multi-leader multi-follower game in which prosumers are leaders and end-users are followers. To realize the transactive energy based trading framework, an inner-outer iteration approach is proposed, where the outer iteration involves interactions among prosumers and end-users, and achieves final results including the retail price and optimal behaviors of all players. The inner iteration is conducted during each time of the outer iteration to solve end-users’ optimal consuming profiles. The whole iteration process involves the independent decision makings of players, and only relies on exchange of information of retail price and prosumers’ offers instead of players’ private information. Case studies are then carried out demonstrating the proposed transactive energy based aggregation framework for prosumers could decrease the retail price, increase revenues for both prosumers and end-users, and reduce the peak load. [ABSTRACT FROM AUTHOR]

This paper proposes a two-stage auction-based local market mechanism to allocate physical storage rights (PSRs). As a market product, PSRs are provided by a storage owner and enable the local market participants (including renewable producers, consumers and prosumers) to access the storage. That is, they can book storage in the form of PSRs and dispatch it at a given time aiming to maximize their utility function. The business options we examine to evaluate the position of storage in the market range from storage owner entirely participating in day-ahead (DA) and real-time (RT) markets as an inter-temporal arbitrager, to exclusively acting as a PSR provider in DA only – this way, the storage owner is fully paid upfront in DA. Considering the context above, we propose an equilibrium model where each player optimizes its operational objective. We prove that the equilibrium model can be substituted with an equivalent optimization formulation which clears the proposed market ensuring the same desirable market properties, such as efficiency and revenue adequacy. Results suggest that the certain revenues earned by the storage owner in DA when acting as a PSR provider is equal to its expected profit as a regular market participant, mitigating however its payoff uncertainty and resulting in the same economic return. [ABSTRACT FROM AUTHOR]

This article addresses a centralized generation expansion planning problem, accounting for both long- and short-term uncertainties. The long-term uncertainty (demand growth) is modeled via a set of scenarios, while the short-term uncertainty (wind power generation) is described by a family of probability distributions with the same first- and second-order moments obtained from historical data. The resulting model is a distributionally robust chance-constrained optimization problem, which selects the conventional generating units to be built among predefined discrete options. This model includes a detailed representation of unit commitment constraints. To achieve computational tractability, we use a tight relaxation approach to convexify unit commitment constraints and solve the model with linear decision rules, resulting in a mixed-integer second-order cone program. It is observed that the proposed model exhibits better out-of-sample performance in terms of total expected system cost and its standard deviation compared to a chance-constrained model that assumes a Gaussian distribution of short-term uncertainty. A similar observation is made when comparing the proposed model against a chance-constrained program that uses empirical renewable power generation data with unknown type of distribution, recasting as either a robust optimization or a stochastic program. [ABSTRACT FROM AUTHOR]

Wind power forecast evaluation is of key importance for forecast provider selection, forecast quality control, and model development. While forecasts are most often evaluated based on squared or absolute errors, these error measures do not always adequately reflect the loss functions and true expectations of the forecast user, neither do they provide enough information for the desired evaluation task. Over the last decade, research in forecast verification has intensified, and a number of verification frameworks and diagnostic tools have been proposed. However, the corresponding literature is generally very technical and most often dedicated to forecast model developers. This can make forecast users struggle to select the most appropriate verification tools for their application while not fully appraising subtleties related to their application and interpretation. This paper revisits the most common verification tools from a forecast user perspective and discusses their suitability for different application examples as well as evaluation setup design and significance of evaluation results. [ABSTRACT FROM AUTHOR]

Due to increasing penetration of stochastic renewable energy sources in electric power systems, the need for flexible resources especially from fast-start conventional generation units (e.g., combined cycle gas turbine plants) is growing. The fast-start conventional units are being operated more frequently in order to respond to the variability and uncertainty of stochastic generation. This raises two important technical questions: as it is common in the literature, is it still an appropriate simplification to ignore the operational unit commitment (UC) constraints of conventional units within the generation expansion planning optimization? And if not, which UC constraint impacts most the expansion planning outcomes? To answer these questions, this paper aims at measuring the planning inefficiency (i.e., the underestimation of need for new generation capacity) caused by ignoring each UC constraint. To this purpose, we develop a centralized network-constrained generation expansion planning model incorporating UC constraints. In particular, we model start-up and shut-down costs, minimum production level and hourly ramping limits of conventional units. Wind power production is considered as the only source of uncertainty, and is modeled through a set of scenarios. A two-stage stochastic programming tool is used, whose first stage determines the long-term expansion and short-term UC decisions over different hours of representative days, while the second stage models the real-time operation for accommodating imbalances arising from wind deviation under different scenarios. Since this problem is potentially hard to solve especially with a large number of representative days and scenarios, a multi-cut Benders' decomposition algorithm is implemented. The well-functioning of the proposed model and the impact of each UC constraint on planning outcomes are evaluated using an extensive numerical study. In our case studies, the exclusion of ramping constraints from planning optimization causes large error and is the most distorting simplification. [ABSTRACT FROM AUTHOR]

Accurate short‐term power forecasts are crucial for the reliable and efficient integration of wind energy in power systems and electricity markets. Typically, forecasts for hours to days ahead are based on the output of numerical weather prediction models, and with the advance of computing power, the spatial and temporal resolutions of these models have increased substantially. However, high‐resolution forecasts often exhibit spatial and/or temporal displacement errors, and when regarding typical average performance metrics, they often perform worse than smoother forecasts from lower‐resolution models. Recent computational advances have enabled the use of large‐eddy simulations (LESs) in the context of operational weather forecasting, yielding turbulence‐resolving weather forecasts with a spatial resolution of 100 m or finer and a temporal resolution of 30 seconds or less. This paper is a proof‐of‐concept study on the prospect of leveraging these ultra high‐resolution weather models for operational forecasting at Horns Rev I in Denmark. It is shown that temporal smoothing of the forecasts clearly improves their skill, even for the benchmark resolution forecast, although potentially valuable high‐frequency information is lost. Therefore, a statistical post‐processing approach is explored on the basis of smoothing and feature engineering from the high‐frequency signal. The results indicate that for wind farm forecasting, using information content from both the standard and LES resolution models improves the forecast accuracy, especially with a feature selection stage, compared with using the information content solely from either source. [ABSTRACT FROM AUTHOR]

A large number of mechanisms are proposed to manage potential problems in distribution networks caused by the participation of distributed energy resources (DERs) in the wholesale markets. In this paper, we first introduce a practical and straightforward mechanism, based on capacity limits, which avoids conflicts between the transmission system operator and the distribution system operators (DSOs). Using a large number of real electric vehicle (EV) commercial charging stations we then show how an EV aggregator can forecast the opportunity cost incurred by offering a mid-term capacity limit service to the DSO. This cost is computed based on the estimated profit that the aggregator could gain in the day-ahead and real-time markets. The proposed methodology guarantees robustness against evolving EV uncertainty, both in terms of service delivery and driving requirements. It also allows the use of a variety of time-series forecasting methods without forecasting electricity prices and EV scenarios. The results of our empirical analysis show the exponential increase of opportunity cost and the considerable increase of the prediction intervals as the capacity limit decreases. The produced offering curves can be used as an indication of the underutilization cost of DERs caused by the DSO’s limitations. [ABSTRACT FROM AUTHOR]

Wind power producers participating in today's electricity markets face significant variability in revenue streams, with potential high losses mostly due to wind's limited predictability and the intermittent nature of the generated electricity. In order to further expand wind power generation despite such challenges, it is important to maximize its market value and move decisively towards economically sustainable and financially viable asset management. In this paper, we introduce a decision‐making framework based on stochastic optimization that allows wind power producers to hedge their position in the market by trading physically settled options in futures markets in conjunction with their participation in the short‐term electricity markets. The proposed framework relies on a series of two‐stage stochastic optimization models that identify a combined trading strategy for wind power producers actively participating in both financial and day‐ahead electricity markets. The proposed models take into consideration penalties from potential deviations between day‐ahead market offers and real‐time operation and incorporates different preferences of risk aversion, enabling a trade‐off between the expected profit and its variability. Empirical analysis based on data from the Nordic region illustrates high efficiency of the stochastic model and reveals increased revenues for both risk neutral and risk averse wind producers opting for combined strategies. [ABSTRACT FROM AUTHOR]

While power system organization has evolved from a hierarchical structure to a more decentralized model, electricity markets are still not up to date with the ongoing transformation toward more consumer-centric economies. As information and communication technologies are broadly adopted, they allow prosumers to have a more proactive role in power system operation. This work introduces the concept of energy collectives, as a community-based electricity market structure. We find that when prosumers are allowed to share energy at community level, the overall electricity procurement for the community reflects prosumers’ preferences. We show that community members can be influenced by a supervisory third-party in charge of interfacing with the market and system operator and of guaranteeing the collective common agreements. We simulate a number of test cases and apply typical principles from analysis of communication networks and distributed systems to assess community fairness. [ABSTRACT FROM AUTHOR]

The increasing shares of stochastic renewables bring higher uncertainty in power system operation and underline the need for optimal utilization of flexibility. However, the European market structure that separates energy and reserve capacity trading is prone to inefficient utilization of flexible assets, such as the HVDC interconnections, since their capacity has to be ex-ante allocated between these services. Stochastic programming models that co-optimize day-ahead energy schedules with reserve procurement and dispatch, provide endogenously the optimal transmission allocation in terms of minimum expected system cost. However, this perfect temporal coordination of trading floors cannot be attained in practice under the existing market design. To this end, we propose a decision-support tool that enables an implicit temporal coupling of the different trading floors using as control parameters the inter-regional transmission capacity allocation between energy and reserves and the area reserve requirements. The proposed method is formulated as a stochastic bilevel program and cast as mixed-integer linear programming problem, which can be efficiently solved using a Benders decomposition approach that improves computational tractability. This model bears the anticipativity features of a transmission allocation model based on a pure stochastic programming formulation, while being compatible with the current market structure. Our analysis shows that the proposed mechanism reduces the expected system cost and thus can facilitate the large-scale integration of intermittent renewables. [ABSTRACT FROM AUTHOR]

A major restructuring of electricity markets takes place worldwide, pursuing maximum economic efficiency. In most modern electricity markets, including the widely adapted Locational Marginal Price (LMP) market, efficiency is only guaranteed under the assumption of perfect competition. Moreover, market design is heavily focused on deterministic conventional generation. Electricity markets, though, are vulnerable to strategic behaviors and challenged by the increased penetration of renewable energy generation. In this paper, we cope with the aforementioned bottlenecks by investigating the application of Vickrey–Clarke–Groves (VCG) auction in a two-stage stochastic electricity market. The VCG mechanism achieves incentive-compatibility by rewarding market participants for their contribution towards market efficiency, being attractive from both market operation and participants perspectives. Both traditional and VCG market-clearing approaches are explored and compared, investigating as well the impact of increasing wind power penetration. The main shortcoming of VCG, i.e., not ensuring revenue adequacy, is quantified in terms of market budget imbalance for various levels of wind power penetration. To this end, a novel ex-post budget redistribution scheme is proposed, which achieves to partially recover budget deficit. [ABSTRACT FROM AUTHOR]

The deployment of distributed energy resources, combined with a more proactive demand side management, is inducing a new paradigm in power system operation and electricity markets. Within a consumer-centric market framework, peer-to-peer (P2P) approaches have gained substantial interest. P2P markets rely on multibilateral negotiation among all agents to match supply and demand. These markets can yield a complete mapping of exchanges onto the grid; hence, allowing to rethink the sharing of costs related to the use of common infrastructure and services. We propose here to attribute such costs through exogenous network charges in several alternative ways, i.e., uniformly, based on the electrical distance between agents and by zones. This variety covers the main grid physical and regulatory configurations. Since attribution mechanisms are defined in an exogenous manner to affect each P2P trade, they eventually shift the market issue to cover the grid exploitation costs. It can even be used to release the stress on the grid when necessary. The interest of our approach is illustrated on a test case using the IEEE 39 bus test system, underlying the impact of attribution mechanisms on trades and grid usage. [ABSTRACT FROM AUTHOR]

With the sustained deployment of distributed generation capacities and the more proactive role of consumers, power systems and their operation are drifting away from a conventional top–down hierarchical structure. Electricity market structures, however, have not yet embraced that evolution. Respecting the high-dimensional, distributed and dynamic nature of modern power systems would translate to designing peer-to-peer markets or, at least, to using such an underlying decentralized structure to enable a bottom–up approach to future electricity markets. A peer-to-peer market structure based on a multi-bilateral economic dispatch (MBED) formulation is introduced, allowing for multi-bilateral trading with product differentiation, for instance based on consumer preferences. A relaxed consensus + innovation (RCI) approach is described to solve the MBED in fully decentralized manner. A set of realistic case studies and their analysis allow us to show that such peer-to-peer market structures can effectively yield market outcomes that are different from centralized market structures and optimal in terms of respecting consumers preferences while maximizing social welfare. Additionally, the RCI solving approach allows for a fully decentralized market clearing that converges with a negligible optimality gap, with a limited amount of information being shared. [ABSTRACT FROM AUTHOR]

This paper deals with the impact of linear approximations for the unknown nonconvex confidence region of chance-constrained ac optimal power flow problems. Such approximations are required for the formulation of tractable chance constraints. In this context, we introduce the first formulation of a chance-constrained second-order cone (SOC) OPF. The proposed formulation provides convergence guarantees due to its convexity, while it demonstrates high computational efficiency. Combined with an ac feasibility recovery, it is able to identify better solutions than chance-constrained nonconvex ac-OPF formulations. To the best of our knowledge, this paper is the first to perform a rigorous analysis of the ac feasibility recovery procedures for robust SOC-OPF problems. We identify the issues that arise from the linear approximations, and by using a reformulation of the quadratic chance constraints, we introduce new parameters able to reshape the approximation of the confidence region. We demonstrate our method on the IEEE 118-bus system. [ABSTRACT FROM AUTHOR]

Despite substantial improvement in the development of forecasting approaches, conditional and dynamic uncertainty estimates ought to be accommodated in decision-making in power system operation and market, in order to yield either cost-optimal decisions in expectation, or decision with probabilistic guarantees. The representation of uncertainty serves as an interface between forecasting and decision-making problems, with different approaches handling various objects and their parameterization as input. Following substantial developments based on scenario-based stochastic methods, robust and chance-constrained optimization approaches have gained increasing attention. These often rely on polyhedra as a representation of the convex envelope of uncertainty. In this paper, we aim to bridge the gap between the probabilistic forecasting literature and such optimization approaches by generating forecasts in the form of polyhedra with probabilistic guarantees. For that, we see polyhedra as parameterized objects under alternative definitions (under $L_1$ and $L_\infty$ norms), the parameters of which may be modeled and predicted. We additionally discuss assessing the predictive skill of such multivariate probabilistic forecasts. An application and related empirical investigation results allow us to verify probabilistic calibration and predictive skills of our polyhedra. [ABSTRACT FROM AUTHOR]

The ever-increasing number of wind farms has brought both challenges and opportunities in the development of wind power forecasting techniques to take advantage of interdependencies between tens or hundreds of spatially distributed wind farms, e.g., over a region. In this paper, a sparsity-controlled vector autoregressive (SC-VAR) model is introduced to obtain sparse model structures in a spatio-temporal wind power forecasting framework by reformulating the original VAR model into a constrained mixed integer nonlinear programming (MINLP) problem. It allows controlling the sparsity of the coefficient matrices in direct manner. However this original SC-VAR is difficult to implement due to its complicated constraints and the lack of guidelines for setting its parameters. To reduce the complexity of this MINLP and to make it possible to incorporate prior expert knowledge to benefit model building and forecasting, the original SC-VAR is modified and a correlation-constrained SC-VAR (CCSC-VAR) is proposed based on spatial correlation information about wind farms. Our approach is evaluated based on a case study of very-short-term forecasting for 25 wind farms in Denmark. Comparison is performed with a set of traditional local methods and spatio-temporal methods. The results obtained show the proposed CCSC-VAR has better overall performance than both the original SC-VAR and other benchmark methods, taking into account all evaluation indicators, including sparsity-control ability, sparsity, accuracy, and efficiency. [ABSTRACT FROM AUTHOR]

While substantial advances are observed in probabilistic forecasting for power system operation and electricity market applications, most approaches are still developed in a univariate framework. This prevents from informing about the interdependence structure among locations, lead times, and variables of interest. Such dependencies are key in a large share of operational problems involving renewable power generation and electricity prices for instance. The few methods that account for dependencies translate to sampling scenarios based on given marginals and dependence structures. However, for classes of decision-making problems based on robust, interval, and chance-constrained optimization, necessary inputs take the form of multivariate prediction regions rather than scenarios. The current literature is at a very primitive stage of characterizing multivariate prediction regions to be employed in these classes of optimization problems. To address this issue, we introduce a new class of multivariate forecasts, which form as multivariate ellipsoids for non-Gaussian variables. We propose a data-driven systematic framework to readily generate and evaluate ellipsoidal prediction regions, with predefined probability guarantees and minimum conservativeness. A skill score is proposed for quantitative assessment of the quality of prediction ellipsoids. A set of experiments is used to illustrate the discrimination ability of the proposed scoring rule for potential misspecification of ellipsoidal prediction regions. Application results based on three datasets with wind, photovoltaic power, and electricity prices allow us to assess the skill of the resulting ellipsoidal prediction regions, in terms of calibration, sharpness, and overall skill. [ABSTRACT FROM AUTHOR]

Two desirable properties of electricity market mechanisms include: 1) revenue adequacy for the market, and 2) cost recovery for all generators. Previously proposed stochastic market-clearing mechanisms satisfy both properties in expectation only, or satisfy one property by scenario and another in expectation. Consequently, market parties may perceive significant risks to participating in the market since they may lose money in one or more scenarios, and therefore be discouraged from offering in the market or perhaps even from investing. We develop a stochastic two-stage market-clearing model including day-ahead and real-time settlements with an energy-only pricing scheme that ensures both properties by scenario. However, this approach is cost-inefficient in general and may sacrifice other desirable market attributes. Undesirable consequences include: One group of participants will have to pay more to ensure that all other participants have their costs covered, and thus their prices will not be equilibrium supporting; and day-ahead and real-time prices are not arbitraged in expectation, although this can be fixed by allowing virtual bidders to arbitrage but at the potential cost of increased market inefficiency. Considering these pros and cons, we propose our model as an appropriate tool for market analysis, and not for clearing actual markets. Numerical results from case studies illustrate the benefits and costs of the proposed stochastic market design. [ABSTRACT FROM AUTHOR]

In contrast to existing frameworks for available transfer capacity (ATC) determination, we propose to define ATCs in an integrated and data-driven manner, optimizing for expected operational costs of the whole system to derive cost-optimal ATCs. These ATCs are purely financial parameters, separated from the physical ATCs based on security indices only typically used in zonal electricity markets today. Determining cost-optimal ATCs requires viewing ATCs as an endogenous market construct, and leads naturally to the definition of a market entity whose responsibility is to optimize ATCs. The optimization problem that this entity solves is a stochastic bilevel problem, which we decompose to yield a computationally tractable formulation. We show that cost-optimal ATCs depend nontrivially on the underlying network structure, and the problem of finding a set of cost-optimal ATCs is in general nonconvex. On a European-scale test system, cost-optimal ATCs achieve expected total costs midway between those for nonintegrated ATCs and a fully stochastic nodal setup. This benefit comes from qualitatively different ATCs compared to typical definitions, with ATCs which exceed the physical cross-border capacity by a factor of 2 or more, and ATCs which are zero between well-connected areas. Our results indicate that the perceived efficiency gap between zonal and nodal markets may be exaggerated if nonoptimal ATCs are used. [ABSTRACT FROM AUTHOR]

The proliferation of distributed energy resources entails efficient market mechanisms in distribution-level networks. This paper establishes a local energy market (LEM) framework, in which electricity and hydrogen are traded. Players in the LEM consist of renewable distributed generators (DGs), loads, hydrogen vehicles, and a hydrogen storage system (HSS) operated by an HSS agent. An iterative LEM clearing method is proposed based on the merit order principle. Players submit offers/bids with consideration of their own preferences and profiles according to the utility functions. The decentralized LEM clearing process not only avoids complex calculation induced by the centralized decision process, but also preserves players’ privacy. Case studies are conducted that demonstrate that the LEM promotes local integration of renewable energy, reduces peak demand, and improves players’ utilities. Sensitivity analysis is then implemented to discuss the influences on the LEM clearing results of capacities of DGs, loads, and the HSS, as well as price of hydrogen from the hydrogen station. [ABSTRACT FROM AUTHOR]

Increased emphasis is placed today on various types of demand response, motivated by the integration of renewable energy generation and efficiency improvements in electricity markets. Some advocated for the development of price-based approaches, where the conditional dynamic elasticity of final users is exploited in the power system, e.g., for system balancing. However, very few real-world experiments have been carried out and price-based demand response has consistently been found difficult to assess and quantify. It is our aim here to describe an approach to do so, as motivated by the large-scale EcoGrid EU experiment. In this project, 1900 houses were equipped with smart meters and other automation devices in order to adapt consumption to real-time electricity prices every 5 min, while monitoring it with the same resolution. Our approach first relies on the clustering of residential load observations that behave similarly within a given experiment. Then, a clinical testing approach, based on a test and a control group, is adapted to assess whether price-responsive loads were actually responsive or not. Interestingly, in the deployment phase of the project, the results show that houses could be deemed price-responsive on some test days, while results were inconclusive on some others. [ABSTRACT FROM PUBLISHER]

This paper presents a methodology to obtain the optimal trading strategies between the proactive distribution company (PDISCO), heterogeneous distributed generation owners (DGOs), and wholesale market in a real-time trading framework. In this framework, the PDISCO’s decisions cover the power procurements from DGOs and the transactions within the real-time market. A one-leader multi-follower-type bilevel model is proposed to embody the PDISCO-DGO gaming structure. The upper-level (UL) problem is to maximize the PDISCO’s profit, while the lower-level (LL) problem indicates the profit maximization per DGO. Since the UL problem is non-linear and non-convex and the LL problems are linear and convex, we reformulate the proposed model to a solvable mathematical program with equilibrium constraints by an equivalent primal-dual approach. The numerical results of the case studies show the effectiveness and scalability of the proposed model. [ABSTRACT FROM PUBLISHER]

With increased competition in wholesale electricity markets, the need for new decision-making tools for strategic producers has arisen. Optimal bidding strategies have traditionally been modeled as stochastic profit maximization problems. However, for producers with non-negligible market power, modeling the interactions with rival participants is fundamental. This can be achieved through equilibrium and hierarchical optimization models. The efficiency of these methods relies on the strategic producer's ability to model rival participants’ behavior and supply curve, but a substantial gap remains in the literature on modeling this uncertainty. In this study, we introduce a Bayesian inference approach to reveal the aggregate supply curve in a day-ahead electricity market. The proposed algorithm relies on Markov Chain Monte Carlo and sequential Monte Carlo methods. The major appeal of this approach is that it provides a complete model of the uncertainty of the aggregate supply curve, through an estimate of its posterior distribution. We show on a small case study that we are able to reveal accurately the aggregate supply curve with no prior information on rival participants. Finally, we show how this piece of information can be used by a price-maker producer in order to devise an optimal bidding strategy. [ABSTRACT FROM PUBLISHER]

Visualization is a crucial component of data analysis. It is always a good idea to plot the data before fitting models, making predictions, or drawing conclusions. As sensors of the electric grid are collecting large volumes of data from various sources, power industry professionals are facing the challenge of visualizing such data in a timely fashion. In this article, we demonstrate several data-visualization solutions for big energy data through three case studies involving smart-meter data, phasor measurement unit (PMU) data, and probabilistic forecasts, respectively. [ABSTRACT FROM PUBLISHER]

In this paper, we present a real-time trading framework for distribution networks where a rational aggregator is identified as a broker by contracting with individual demands and dealing with the distribution company. Demand response capability is characterized by the coexistence of elastic and inelastic demand components. A one-leader multi-follower bi-level model is proposed to derive the procurement strategies, i.e., the upper-level problem intends to maximize the profit of the proactive distribution company, while the lower-level expresses the profit maximization per rational aggregator. The proposed model is then transformed into a solvable mathematical program with equilibrium constraints through a primal-dual approach. A modified 33-bus distribution network is utilized to demonstrate the effectiveness of the proposed model. [ABSTRACT FROM PUBLISHER]

The recent increase in the deployment of renewable energy sources may affect the offering strategy of conventional producers, mainly in the balancing market. The topics of optimal offering strategy and self-scheduling of thermal units have been extensively addressed in the literature. The feasible operating region of such units can be modeled using a mixed-integer linear programming approach, and the trading problem as a linear programming problem. However, the existing models mostly assume a uniform pricing scheme in all market stages, while several European balancing markets (e.g., in Germany and Italy) are settled under a pay-as-bid pricing scheme. The existing tools for solving the trading problem in pay-as-bid electricity markets rely on nonlinear optimization models, which, combined with the unit commitment constraints, result in a mixed-integer nonlinear programming problem. In contrast, we provide a linear formulation for that trading problem. Then, we extend the proposed approach by formulating a two-stage stochastic problem for optimal offering in a two-settlement electricity market with a pay-as-bid pricing scheme at the balancing stage. The resulting model is mixed-integer and linear. The proposed model is tested on a realistic case study against a sequential offering approach, showing the capability of increasing profits in expectation. [ABSTRACT FROM AUTHOR]