Search

Operational decisions relying on predictive distributions of electricity prices can result in significantly higher profits compared to those based solely on point forecasts. However, the majority of models developed in both academic and industrial settings provide only point predictions. To address this, we examine three postprocessing methods for converting point forecasts of day-ahead electricity prices into probabilistic ones: Quantile Regression Averaging, Conformal Prediction, and the recently introduced Isotonic Distributional Regression. We find that while the latter demonstrates the most varied behavior, it contributes the most to the ensemble of the three predictive distributions, as measured by Shapley values. Remarkably, the performance of the combination is superior to that of state-of-the-art Distributional Deep Neural Networks over two 4.5-year test periods from the German and Spanish power markets, spanning the COVID pandemic and the war in Ukraine.

Probabilistic price forecasting has recently gained attention in power trading because decisions based on such predictions can yield significantly higher profits than those made with point forecasts alone. At the same time, methods are being developed to combine predictive distributions, since no model is perfect and averaging generally improves forecasting performance. In this article we address the question of whether using CRPS learning, a novel weighting technique minimizing the continuous ranked probability score (CRPS), leads to optimal decisions in day-ahead bidding. To this end, we conduct an empirical study using hourly day-ahead electricity prices from the German EPEX market. We find that increasing the diversity of an ensemble can have a positive impact on accuracy. At the same time, the higher computational cost of using CRPS learning compared to an equal-weighted aggregation of distributions is not offset by higher profits, despite significantly more accurate predictions., Comment: 12 pages, 7 figures, 2 tables. Submitted to Operations Research and Decisions

Throughout its history, Operational Research has evolved to include a variety of methods, models and algorithms that have been applied to a diverse and wide range of contexts. This encyclopedic article consists of two main sections: methods and applications. The first aims to summarise the up-to-date knowledge and provide an overview of the state-of-the-art methods and key developments in the various subdomains of the field. The second offers a wide-ranging list of areas where Operational Research has been applied. The article is meant to be read in a nonlinear fashion. It should be used as a point of reference or first-port-of-call for a diverse pool of readers: academics, researchers, students, and practitioners. The entries within the methods and applications sections are presented in alphabetical order. The authors dedicate this paper to the 2023 Turkey/Syria earthquake victims. We sincerely hope that advances in OR will play a role towards minimising the pain and suffering caused by this and future catastrophes.

We present a novel approach to probabilistic electricity price forecasting which utilizes distributional neural networks. The model structure is based on a deep neural network that contains a so-called probability layer. The network's output is a parametric distribution with 2 (normal) or 4 (Johnson's SU) parameters. In a forecasting study involving day-ahead electricity prices in the German market, our approach significantly outperforms state-of-the-art benchmarks, including LASSO-estimated regressions and deep neural networks combined with Quantile Regression Averaging. The obtained results not only emphasize the importance of higher moments when modeling volatile electricity prices, but also -- given that probabilistic forecasting is the essence of risk management -- provide important implications for managing portfolios in the power sector.

Forecasting electricity prices is a challenging task and an active area of research since the 1990s and the deregulation of the traditionally monopolistic and government-controlled power sectors. Although it aims at predicting both spot and forward prices, the vast majority of research is focused on short-term horizons which exhibit dynamics unlike in any other market. The reason is that power system stability calls for a constant balance between production and consumption, while being weather (both demand and supply) and business activity (demand only) dependent. The recent market innovations do not help in this respect. The rapid expansion of intermittent renewable energy sources is not offset by the costly increase of electricity storage capacities and modernization of the grid infrastructure. On the methodological side, this leads to three visible trends in electricity price forecasting research as of 2022. Firstly, there is a slow, but more noticeable with every year, tendency to consider not only point but also probabilistic (interval, density) or even path (also called ensemble) forecasts. Secondly, there is a clear shift from the relatively parsimonious econometric (or statistical) models towards more complex and harder to comprehend, but more versatile and eventually more accurate statistical/machine learning approaches. Thirdly, statistical error measures are nowadays regarded as only the first evaluation step. Since they may not necessarily reflect the economic value of reducing prediction errors, more and more often, they are complemented by case studies comparing profits from scheduling or trading strategies based on price forecasts obtained from different models., Comment: Forthcoming in the Oxford Research Encyclopedia of Economics and Finance (https://oxfordre.com/economics)

Electricity price forecasting (EPF) is a branch of forecasting on the interface of electrical engineering, statistics, computer science, and finance, which focuses on predicting prices in wholesale electricity markets for a whole spectrum of horizons. These range from a few minutes (real-time/intraday auctions and continuous trading), through days (day-ahead auctions), to weeks, months or even years (exchange and over-the-counter traded futures and forward contracts). Over the last 25 years, various methods and computational tools have been applied to intraday and day-ahead EPF. Until the early 2010s, the field was dominated by relatively small linear regression models and (artificial) neural networks, typically with no more than two dozen inputs. As time passed, more data and more computational power became available. The models grew larger to the extent where expert knowledge was no longer enough to manage the complex structures. This, in turn, led to the introduction of machine learning (ML) techniques in this rapidly developing and fascinating area. Here, we provide an overview of the main trends and EPF models as of 2022., Comment: Forthcoming in: IEEE Power & Energy Magazine, May/June 2022

We employ a recently proposed change-point detection algorithm, the Narrowest-Over-Threshold (NOT) method, to select subperiods of past observations that are similar to the currently recorded values. Then, contrarily to the traditional time series approach in which the most recent $\tau$ observations are taken as the calibration sample, we estimate autoregressive models only for data in these subperiods. We illustrate our approach using a challenging dataset - day-ahead electricity prices in the German EPEX SPOT market - and observe a significant improvement in forecasting accuracy compared to commonly used approaches, including the Autoregressive Hybrid Nearest Neighbors (ARHNN) method., Comment: Forthcoming in: Proceedings of the International Conference on Computational Science (ICCS) 2022, London, UK

We extend the neural basis expansion analysis (NBEATS) to incorporate exogenous factors. The resulting method, called NBEATSx, improves on a well performing deep learning model, extending its capabilities by including exogenous variables and allowing it to integrate multiple sources of useful information. To showcase the utility of the NBEATSx model, we conduct a comprehensive study of its application to electricity price forecasting (EPF) tasks across a broad range of years and markets. We observe state-of-the-art performance, significantly improving the forecast accuracy by nearly 20% over the original NBEATS model, and by up to 5% over other well established statistical and machine learning methods specialized for these tasks. Additionally, the proposed neural network has an interpretable configuration that can structurally decompose time series, visualizing the relative impact of trend and seasonal components and revealing the modeled processes' interactions with exogenous factors. To assist related work we made the code available in https://github.com/cchallu/nbeatsx., Comment: 30 pages, 7 figures, 4 tables

Recent advancements in the fields of artificial intelligence and machine learning methods resulted in a significant increase of their popularity in the literature, including electricity price forecasting. Said methods cover a very broad spectrum, from decision trees, through random forests to various artificial neural network models and hybrid approaches. In electricity price forecasting, neural networks are the most popular machine learning method as they provide a non-linear counterpart for well-tested linear regression models. Their application, however, is not straightforward, with multiple implementation factors to consider. One of such factors is the network's structure. This paper provides a comprehensive comparison of two most common structures when using the deep neural networks -- one that focuses on each hour of the day separately, and one that reflects the daily auction structure and models vectors of the prices. The results show a significant accuracy advantage of using the latter, confirmed on data from five distinct power exchanges.

While the field of electricity price forecasting has benefited from plenty of contributions in the last two decades, it arguably lacks a rigorous approach to evaluating new predictive algorithms. The latter are often compared using unique, not publicly available datasets and across too short and limited to one market test samples. The proposed new methods are rarely benchmarked against well established and well performing simpler models, the accuracy metrics are sometimes inadequate and testing the significance of differences in predictive performance is seldom conducted. Consequently, it is not clear which methods perform well nor what are the best practices when forecasting electricity prices. In this paper, we tackle these issues by performing a literature survey of state-of-the-art models, comparing state-of-the-art statistical and deep learning methods across multiple years and markets, and by putting forward a set of best practices. In addition, we make available the considered datasets, forecasts of the state-of-the-art models, and a specifically designed python toolbox, so that new algorithms can be rigorously evaluated in future studies.

We conduct an extensive empirical study on short-term electricity price forecasting (EPF) to address the long-standing question if the optimal model structure for EPF is univariate or multivariate. We provide evidence that despite a minor edge in predictive performance overall, the multivariate modeling framework does not uniformly outperform the univariate one across all 12 considered datasets, seasons of the year or hours of the day, and at times is outperformed by the latter. This is an indication that combining advanced structures or the corresponding forecasts from both modeling approaches can bring a further improvement in forecasting accuracy. We show that this indeed can be the case, even for a simple averaging scheme involving only two models. Finally, we also analyze variable selection for the best performing high-dimensional lasso-type models, thus provide guidelines to structuring better performing forecasting model designs.

A fundamental question related to innovation diffusion is how the social network structure influences the process. Empirical evidence regarding real-world influence networks is very limited. On the other hand, agent-based modeling literature reports different and at times seemingly contradictory results. In this paper we study innovation diffusion processes for a range of Watts-Strogatz networks in an attempt to shed more light on this problem. Using the so-called Sznajd model as the backbone of opinion dynamics, we find that the published results are in fact consistent and allow to predict the role of network topology in various situations. In particular, the diffusion of innovation is easier on more regular graphs, i.e. with a higher clustering coefficient. Moreover, in the case of uncertainty - which is particularly high for innovations connected to public health programs or ecological campaigns - a more clustered network will help the diffusion. On the other hand, when social influence is less important (i.e. in the case of perfect information), a shorter path will help the innovation to spread in the society and - as a result - the diffusion will be easiest on a random graph.

We develop a simple test for deviations from power law tails, which is based on the asymptotic properties of the empirical distribution function. We use this test to answer the question whether great natural disasters, financial crashes or electricity price spikes should be classified as dragon kings or 'only' as black swans.

The Heston model stands out from the class of stochastic volatility (SV) models mainly for two reasons. Firstly, the process for the volatility is non-negative and mean-reverting, which is what we observe in the markets. Secondly, there exists a fast and easily implemented semi-analytical solution for European options. In this article we adapt the original work of Heston (1993) to a foreign exchange (FX) setting. We discuss the computational aspects of using the semi-analytical formulas, performing Monte Carlo simulations, checking the Feller condition, and option pricing with FFT. In an empirical study we show that the smile of vanilla options can be reproduced by suitably calibrating three out of five model parameters., Comment: Chapter prepared for the 2nd edition of Statistical Tools for Finance and Insurance, P.Cizek, W.Haerdle, R.Weron (eds.), Springer-Verlag, forthcoming in 2011

In this paper we introduce two models of opinion dynamics in oligopoly markets and apply them to a situation, where a new entrant challenges two incumbents of the same size. The models differ in the way the two forces influencing consumer choice -- (local) social interactions and (global) advertising -- interact. We study the general behavior of the models using the Mean Field Approach and Monte Carlo simulations and calibrate the models to data from the Polish telecommunications market. For one of the models criticality is observed -- below a certain critical level of advertising the market approaches a lock-in situation, where one market leader dominates the market and all other brands disappear. Interestingly, for both models the best fits to real data are obtained for conformity level $p \in (0.3,0.4)$. This agrees very well with the conformity level found by Solomon Asch in his famous social experiment.

We analyze a 19-year time series of North American electric power transmission system blackouts. Contrary to previously reported results we find a fatter than exponential decay in the distribution of inter-occurrence times and evidence of seasonal dependence in the number of events. Our findings question the use of self-organized criticality, and in particular the sandpile model, as a paradigm of blackout dynamics in power transmission systems. Hopefully, though, they will provide guidelines to more accurate models for evaluation of blackout risk., Comment: 5 pages, 3 figures, to appear in: H. Takayasu (ed.), Proceedings of the 3rd Nikkei Econophysics Symposium, Springer-Tokyo, 2005/2006

The price of electricity is far more volatile than that of other commodities normally noted for extreme volatility. The possibility of extreme price movements increases the risk of trading in electricity markets. However, underlying the process of price returns is a strong mean-reverting mechanism. We study this feature of electricity returns by means of Hurst R/S analysis, Detrended Fluctuation Analysis and periodogram regression., Comment: 7 pages, 2 figures. To appear in "Empirical Science of Financial Fluctuations", Tokyo, Nov. 2000 (Springer Verlag 2001)

A major issue in financial economics is the behavior of asset returns over long horizons. Various estimators of long range dependence have been proposed. Even though some have known asymptotic properties, it is important to test their accuracy by using simulated series of different lengths. We test R/S analysis, Detrended Fluctuation Analysis and periodogram regression methods on samples drawn from Gaussian white noise. The DFA statistics turns out to be the unanimous winner. Unfortunately, no asymptotic distribution theory has been derived for this statistics so far. We were able, however, to construct empirical (i.e. approximate) confidence intervals for all three methods. The obtained values differ largely from heuristic values proposed by some authors for the R/S statistics and are very close to asymptotic values for the periodogram regression method., Comment: 16 pages, 11 figures New version: 14 pages (smaller fonts), 11 figures, new Section on applications

The price of electricity is far more volatile than that of other commodities normally noted for extreme volatility. Demand and supply are balanced on a knife-edge because electric power cannot be economically stored, end user demand is largely weather dependent, and the reliability of the grid is paramount. The possibility of extreme price movements increases the risk of trading in electricity markets. However, a number of standard financial tools cannot be readily applied to pricing and hedging electricity derivatives. In this paper we present arguments why this is the case.

In this paper we address the issue of modeling electricity loads and prices with diffusion processes. More specifically, we study models which belong to the class of generalized Ornstein-Uhlenbeck processes. After comparing properties of simulated paths with those of deseasonalized data from the California power market and performing out-of-sample forecasts we conclude that, despite certain advantages, the analyzed continuous-time processes are not adequate models of electricity load and price dynamics., Comment: To be published in Physica A (2001): Proceedings of the NATO ARW on Application of Physics in Economic Modelling, Prague, Feb. 8-10, 2001

Power-law tail behavior and the summation scheme of Levy-stable distributions is the basis for their frequent use as models when fat tails above a Gaussian distribution are observed. However, recent studies suggest that financial asset returns exhibit tail exponents well above the Levy-stable regime ($0<\alpha\le 2$). In this paper we illustrate that widely used tail index estimates (log-log linear regression and Hill) can give exponents well above the asymptotic limit for $\alpha$ close to 2, resulting in overestimation of the tail exponent in finite samples. The reported value of the tail exponent $\alpha$ around 3 may very well indicate a Levy-stable distribution with $\alpha\approx 1.8$., Comment: To be published in Int. J. Modern Physics C (2001) vol. 12 no. 2