Search

International audience; Short-term forecasting is key to the safe, reliable, and economic operation of modern power systems. As the majority of modern forecasting tools are purely data-driven, their performance relies heavily on the quality and availability of data. In this work, we examine forecasting when a subset of features used during model training becomes unavailable (deleted or missing) in an operational setting, a subject that has been largely overlooked by previous works. Several reasons could lead to feature deletion, including malicious data-integrity attacks, network latency, and equipment malfunctions, among others. We leverage tools from robust optimization and machine learning and formulate a linear regression model that is optimally resilient to the deletion of features at test time. The robust counterpart of the proposed model is a linear program whose size grows polynomially with the number of training observations and the number of features; we further provide a decomposition algorithm based on the alternating direction method of multipliers to deal with large problem instances that are typically found in energy forecasting applications. We further extend to the case of Probabilistic Forecasting by robustifying the standard linear quantile regression model. To validate empirically the proposed approach, we examine several prevalent forecasting practices in power systems, namely electricity prices, load, wind production, and solar production forecasting. We compare against regularized and randomization-based models and benchmark their performance for the case of feature deletion. The results show that the proposed solution successfully mitigates the adverse effects of missing features, leading to the lowest overall performance degradation. Further, it successfully hedges against the most adverse scenario of deleting the most important feature from the test set. The results persist both for point and probabilistic forecasts and across the different series. Overall, this work highlights the benefits of leveraging robust optimization and provides a new perspective on how to deal with feature uncertainty in energy forecasting applications.

International audience; Standard practice of decision-making in energy systems relies largely on complex modeling chains to address technical constraints and integrate numerous sources of uncertainty. The increased penetration of Renewable Energy Sources (RES) such as wind and photovoltaic plants adds complexity due to the weather dependency of their electricity production. Artificial Intelligence (AI) based tools have proven their efficiency in different applications in the energy sector ranging from forecasting to optimization and decision making. They permit to simplify modeling chains and to improve performance due to higher learning capabilities compared to state-of-the-art methods.However, decision-makers of the energy sector need to understand how decision-aid tools construct their outputs from the data. AI-based tools are often seen as black-box models and this penalizes their acceptability by end-users (traders, power system operators a.o.). The lack of interpretability of AI tools is a major challenge for the wider adoption of AI in the energy sector and a fundamental requirement to better support humans in the decision-aid process. Agents of energy systems expect very high levels of reliability for the various services they provide. As energy systems are impacted by multiple uncertainty sources (e.g. available power of RES plants, climate change, market conditions), developed AI tools should not only be performant on average situations but be able to guarantee robust solutions in the case of an extreme event.Therefore, our research focuses on understandable representations of data-driven decision-aid models for human operators in the energy sector. In order to enhance the interpretability of the AI models, a technique borrowed from the computer science domain is explored and further developed. Genetic programming and more precisely Symbolic Regression is used to derive a symbolic representation for the data-driven model that can take the form of a single equation. This equation results according to a specific reward function. The optimal solutions are selected naturally mimicking the biological theory of survival of the fittest.The main outcome is the production of symbolic representations of the AI models that require minimum changes when applied to different case studies. In this presentation a real-world use-case is considered, to demonstrate the added value of the proposed tools for decision making when trading the production of wind and solar power plants to the day-ahead market. An annual period of data is considered to train and test the proposed model. The typical modeling chain involves as many as 12 models for forecasting RES production and market quantities together with stochastic optimization to derive trading decisions. This complex chain is here replaced by a single AI-based model. Such simplification is a significant enhancement to the modeling chain interpretability and facilitates trust to the human decision-maker.This work is carried out in part in the frame of the European project Smart4RES supported by the H2020 Framework Program and in part in the frame the Marie-Curie COFUND project Ai4theSciences

International audience; The Horizon2020 Smart4RES project develops forecasting and optimization solutions for the operation of Renewable Energy Sources (RES) and their application in electricity market trading and grid management. As a follow-up of the Poster presented in 2021, this poster presents the latest solutions developed in Smart4RES that intend to cover a large spectrum of prediction horizons and spatial scales in order to maximize the interest for the wind power industry.At the scale of a wind farm, the fluctuations of wind conditions at very-short-term horizons (seconds to minutes ahead) impose significant variations in the structural load of the turbine and its power output. These variations are challenging for a precise control of wind turbines. The forecasting models developed by Smart4RES make use of new data sources unexploited by traditional models such as 2-beam and 4-beam nacelle-mounted LIDAR. Results obtained on an operating turbine equipped with LIDAR show significant improvement in the forecasting error of both structural load and active power output for the next 20 seconds ahead. At the horizon of the next minutes, the power output of a wind farm is impacted by weather variability but also by wake losses that may vary as a function of turbine curtailment following a system operator request. This is why Smart4RES proposes a dynamic Machine Learning (ML) prediction model based on Transfer Learning which provides adaptive forecast that beat state-of-art approaches including a similar ML model trained only in batch mode. Aggregations of renewable power plants are key players for renewable-based provision of services to the grid and optimized management of distribution grids. In this context, a coherent forecast over the entire hierarchy of the aggregation is essential in order to take decisions that are feasible considering local constraints in the various levels of the hierarchy. A main challenge in such hierarchies is to produce a forecast even if data is missing, which can occur frequently at different periods and levels in the hierarchy. Whereas existing approaches tend to discard periods with missing data, which can drastically reduce the amount of available data for training and the applicability of the forecasting model in real conditions, Smart4RES proposes an end-to-end learning approach that is able to derive coherent and precise hierarchical forecasts even in the presence of missing values at different levels of the hierarchy.

International audience; Spatially aggregating renewable power plants is beneficial when participating in electricity markets. In this context, a substantial number of features is available from various data sources. In machine learning, feature selection is common so as to relieve the curse of dimensionality and avoid overfitting. However, there is no guarantee that the selected features result in reliable forecasts and post-processing can therefore be valuable. In this study, we combine model agnostic feature selection with linear and nonlinear probabilistic forecast combination techniques. Moreover, the filters automatically compute the weights for our analog ensemble (AnEn) forecast model that does not require training. We verify our model chain by generating intra-day forecasts of the aggregated output of 60 wind turbines and 20 photovoltaic power plants using 831 input features in total. We show that automatic feature selection improves forecast accuracy while forecast combination improves reliability.

In this presentation we detail highlight results obtained from the research work within the European Horizon 2020 project Smart4RES (http://www.smart4res.eu). The project, which started in 2019 and runs until 2023, aims at a better modelling and forecasting of weather variables necessary to optimise the integration of weather-dependent renewable energy (RES) production (i.e. wind, solar, run-of-the-river hydro) into power systems and electricity markets. Smart4RES gathers experts from several disciplines, from meteorology and renewable generation to market- and grid-integration. It aims to contribute to the pathway towards energy systems with very high RES penetrations by 2030 and beyond, through thematic objectives including:Improvement of weather and RES forecasting, Streamlined extraction of optimal value through new forecasting products, data market places, and novel business models; New data-driven optimization and decision-aid tools for market and grid management applications; Validation of new models in living labs and assessment of forecasting value vs costly remedies to hedge uncertainties (i.e. storage). In this presentation we will focus on our results on models that permit to improve forecasting of weather variables with focus on extreme situations and also through innovative measuring settings (i.e. a network of sky cameras). Also results will be presented on the development of seamless approach able to couple outputs from different ensemble numerical weather prediction (NWP) models with different temporal resolutions. Advances on the contribution of ultra-high resolution NWPs based on Large Eddy Simulation will be presented with evaluation results on real case studies like the Rhodes island in Greece.When it comes to forecasting the power output of RES plants, mainly wind and solar, the focus is on improving predictability using multiple sources of data. The proposed modelling approaches aim to efficiently combine highly dimensionally input (various types of satellite images, numerical weather predictions, spatially distributed measurements etc.). A priority has been to propose models that permit to generate probabilistic forecasts for multiple time frames in a seamless way. Thus, the objective is not only to improve accuracy and uncertainty estimations, but also to simplify complex forecasting modelling chains for applications that use forecasts at different time frames (i.e. a virtual power plant - VPP- with or without storage that participates in multiple markets). Our results show that the proposed seamless models permit to reach these performance objectives. Results will be presented also on how these approaches can be extended to aggregations of RES plants which is relevant for forecasting VPP production.

International audience; Standard practices of decision-making in energy systems are dynamic, non-linear, complex, and chaotic processes in nature. Trading the power produced by solar photovoltaic (PV) plants in electricity markets is an important decisionmaking problem which receives increasing attention in the past few decades. The main objective of this paper is to build an interpretable data-driven decision aid model for the case study of a solar power plant with the objective to minimize imbalance costs and thus maximise the revenue, using Symbolic Regression (SR) through Genetic Programming. The use of SR in the experiments and analysis developed in this paper show numerous advantages. SR evolves linear combinations of nonlinear functions of the input variables. Three penalty metrics are introduced to enhance the interpretability of the final solutions. SR shows robust results, especially in the case study.

International audience; In this presentation we will show the research directions and innovative solutions developed in the European Horizon 2020 project Smart4RES (http://www.smart4res.eu) for better modelling and forecasting of weather variables necessary to optimise the integration of renewable energy (RES) production (i.e. wind, solar, run-of-the-river hydro) into power systems and electricity markets. Smart4RES started in November 2019 and gathers experts from several disciplines, from meteorology and renewable generation to market- and grid-integration. It aims to contribute to reach very high RES penetrations in power grids of 2030 and beyond, through thematic objectives including:• Improvement of weather forecasting with focus on variables that are important for RES applications,• Improvement of RES power forecasting and better estimation of uncertainties, • Streamlined extraction of optimal value through new forecasting products, data market places, and novel business models,• New data-driven optimization and decision-aid tools for market and grid management applications.• Validation of new models in living labs and assessment of forecasting value vs costly remedies to hedge uncertainties (i.e. storage). Smart4RES focuses both on improving forecasting models of weather (e.g. physical models, data assimilation, Large Eddy Simulation, enabling weather forecasts seamless) and RES production (e.g. seamless models, highly resolved predictions), and on addressing applications in power grids. Developments in the project have been formalized in Use Cases that cover a large range of time frames, technologies and geographical scales. For example, use-cases on power grids refer to the provision of ancillary services to the upper-level grid (e.g., balancing power) and the local grid (e.g., voltage control and congestion management), where the accurate forecasts of variable generation are key for accurate decision-making. A grid state forecasting will quantify dynamically the flexibility potential of RES in distribution grids. Collaborative forecasting investigates the improvement associated to local data sharing between distributed RES plants. This data sharing paves the way to a data market where agents exchange measurements, predictions or other types of valuable data. Lastly, data-driven approaches will streamline decision-making by simplifying the model chain of bidding RES production, storage dispatch or predictive management electricity grids. They will also provide interpretable hindsight to decision-makers by integrating the decisions of experts (human-in-the-loop) and will be tested in realistic laboratory conditions (software-in-the-loop).In this presentation we focus on the work done for improving modelling and forecasting of weather variables with accent to wind energy; i.e. through innovative measuring set-ups; through the development of seamless numerical weather prediction (NWP) approaches to be able to couple outputs of NWP models with different resolutions; through ultra-high resolution NWPs based on Large Eddy Simulation. We present results using data from real world test cases considered in the project. Finally, we assess how the new forecasting products may bring value to the applications.

International audience; The efficient integration of photovoltaic (PV) production in energy systems is conditioned by the capacity to anticipate its variability, that is, the capacity to provide accurate forecasts. From the classical forecasting methods in the state of the art dealing with a single power plant, the focus has moved in recent years to spatio-temporal approaches, where geographically dispersed data are used as input to improve forecasts of a site for the horizons up to 6 h ahead. These spatio-temporal approaches provide different performances according to the data sources available but the question of the impact of each source on the actual forecasting performance is still not evaluated. In this paper, we propose a flexible spatio-temporal model to generate PV production forecasts for horizons up to 6 h ahead and we use this model to evaluate the effect of different spatial and temporal data sources on the accuracy of the forecasts. The sources considered are measurements from neighboring PV plants, local meteorological stations, Numerical Weather Predictions, and satellite images. The evaluation of the performance is carried out using a real-world test case featuring a high number of 136 PV plants. The forecasting error has been evaluated for each data source using the Mean Absolute Error and Root Mean Square Error. The results show that neighboring PV plants help to achieve around 10% reduction in forecasting error for the first three hours, followed by satellite images which help to gain an additional 3% all over the horizons up to 6 h ahead. The NWP data show no improvement for horizons up to 6 h but is essential for greater horizons.

International audience; This paper presents the solutions on renewable energy forecasting proposed by the Horizon2020 Project Smart4RES. The ambition of the project is twofold: (1) increase substantially the performance of short-term forecasting models of Renewable Energy Sources (RES) production and associated weather forecasting and (2) optimize decisions subject to RES uncertainty in power systems and electricity markets. Developments are based on latest advances in meteorology and original use of data science (combination of multiple data sources, data-driven approaches for trading and grid management). Finally, solutions such as flexibility forecast of distributed resources and data markets are oriented towards value for power system stakeholders. Although the project covers a broad scope, in this paper we focus on a selection of use cases that concern the integration of renewables in distribution grids.

International audience; Precise probabilistic forecasting tools of wind power generation are essential for solving problems related to the impact of wind generation uncertainty on electrical systems. In this context, the International Conference on the European Energy Market EEM20 set up a competition aiming at forecasting wind power generation for the four Swedish bidding zones. Participants had to generate day-ahead quantile forecasts of the aggregated wind power production from Numerical Weather Predictions (NWPs) ensembles provided all over the country [1].

International audience; Variable renewable energy has a growing impact on electricity markets and power systems in many regions of the world. In this context, the 17th International Conference on the European Energy Market EEM20 set up a competition to develop probabilistic forecasting tools of wind production at a regional level. This paper proposes an adaptive approach for regional wind power forecasting. A physics-oriented pre-processing of the data delivers analog weather patterns and wind-power-related variables, then a k-means clustering of wind farms further reduces the dimension of the problem. The generated representative features feed a Quantile Regression Forests model that produces sharp and reliable predictions. As a result, our model won the competition with a relative improvement of the average pinball loss of 6.7% and 14.7%, compared to the teams ranked second and third respectively.

The aim of this paper is to present the objectives, research directions and first highlight results of the Smart4RES project, which was launched in November 2019, under the Horizon 2020 Framework Programme. Smart4RES is a research project that aims to bring substantial performance improvements to the whole model and value chain in renewable energy (RES) forecasting, with particular emphasis placed on optimizing synergies with storage and to support power system operation and participation in electricity markets. For that, it concentrates on a number of disruptive proposals to support ambitious objectives for the future of renewable energy forecasting. This is thought of in a context with steady increase in the quantity of data being collected and computational capabilities. And, this comes in combination with recent advances in data science and approaches to meteorological forecasting. Smart4RES concentrates on novel developments towards very high-resolution and dedicated weather forecasting solutions. It makes optimal use of varied and distributed sources of data e.g. remote sensing (sky imagers, satellites, etc), power and meteorological measurements, as well as high-resolution weather forecasts, to yield high-quality and seamless approaches to renewable energy forecasting. The project accommodates the fact that all these sources of data are distributed geographically and in terms of ownership, with current restrictions preventing sharing. Novel alternative approaches are to be developed and evaluated to reach optimal forecast accuracy in that context, including distributed and privacy-preserving learning and forecasting methods, as well as the advent of platform-enabled data-markets, with associated pricing strategies. Smart4RES places a strong emphasis on maximizing the value from the use of forecasts in applications through advanced decision making and optimization approaches. This also goes through approaches to streamline the definition of new forecasting products balancing the complexity of forecast information and the need of forecast users. Focus is on developing models for applications involving storage, the provision of ancillary services, as well as market participation.

The aggregation of multiple renewable plants located in distinct climate zones, using different energy sources, enables to reduce the production uncertainty when compared to the production of a single plant. Such aggregations, controlled by a Virtual Power Plant (VPP) system, are good candidates for the provision of ancillary services. Stochastic optimization models are available to optimize biddings on ancillary services and energy markets (see for instance [1]). These models require trajectories of the renewable VPP production that anticipate production uncertainty and reproduce correctly the temporal correlations observed in the production signal. This is particularly important in ancillary services markets, where a reserve bid must be guaranteed over a production duration or validity period during which power fluctuations are significant (e.g. lasting currently 24 hours on the European common market for Frequency Containment Reserve, with a foreseen evolution to 4 hours by July 2020 [2]). Production trajectories may be obtained by coupling probabilistic forecasts and a model of temporal dependencies between forecast horizons [3] and possibly spatial dependencies in the case of a multivariate forecast at the scale of a region or a portfolio [4]. In the case of a renewable VPP, the aggregated production is primarily of interest. In this work, we propose a methodology to generate trajectories of aggregated production from probabilistic forecasts obtained with decision-tree based models or neural networks. A copula models the dependency between forecast horizons and the space defined by the plants contained in the aggregation. The model is tested in a day-ahead forecasting configuration on a 54 MW VPP comprising 15 plants with 3 different energy sources (Photovoltaics, Wind, Hydro). The comparison of trajectories generated from a direct forecast of the aggregated production and from forecasts at lower levels of the aggregation shows that the latter solution reproduces with more accuracy the temporal variability of the aggregated production over the whole horizon range, especially when Photovoltaics dominates the production capacities in the aggregation (15 % improvement of the Variogram Score). [1]: Soares, T., & Pinson, P. (2017). Renewable energy sources offering flexibility through electricity markets. Technical University of Denmark.[2]: ENTSO-E. (2018). TSO’s proposal for the establishment of common and harmonised rules and processes for the exchange and procurement of Balancing Capacity for Frequency Containment Reserves (FCR) TSOs’ proposal for the establishment of common and harmonised rules and pro-c, (October), 1–9.[3]: Pinson, P., Madsen, H., Nielsen, H. A., Papaefthymiou, G., & Klöckl, B. (2009). From probabilistic forecasts to statistical scenarios of short-term wind power production. Wind Energy, 12(1), 51–62. [4]: Golestaneh, F., Gooi, H. B., & Pinson, P. (2016). Generation and evaluation of space–time trajectories of photovoltaic power. Applied Energy, 176, 80–91.

International audience; Electricity systems around the world are decarbonizing, driven by reductions in the cost of renewable energy and encouraged by supportive regulatory policy. Electricity market designs are increasingly being tested to ensure that the bulk power system can deliver reliable, cost-effective energy to all consumers.

Smart4RES is a European Horizon2020 project developing next generation solutions for renewable energy forecast- ing. This paper presents highlight results obtained during the first year of the project. Data science is used throughout the proposed solutions in order to process the large amount of heterogeneous data available to forecasters, and derive model-free approaches of forecasting and decision-aid tasks. This paper presents a series of solutions addressing relevant for Photovoltaics (PV) and storage applications. High-resolution Numerical Weather Predictions and regional solar irradiance forecasting provide detailed information on local weather conditions and their variability. PV power forecasting benefits from such new data sources, but also the proposed collaborative data exchange. Finally, data-driven methods simplify decision-making for trading in short-term markets and for grid management., This paper (presentation file) was presented at the 11th Solar & Storage Integration Workshop and published in the workshop's proceedings

International audience; In this presentation we will outline the research directions and innovative solutions developed in the European Horizon 2020 project Smart4RES (http://www.smart4res.eu) aiming at improving all main steps of the model and value chain of renewable energy forecasting. The aim is to optimise integration of renewable energy (RES) production (i.e. wind, solar, run-of-the-river hydro) into power systems and electricity markets. Smart4RES started in November 2019 and gathers experts from several disciplines, from meteorology and renewable generation to market- and grid-integration. It aims to contribute to reach very high RES penetrations in power grids of 2030 and beyond, through thematic objectives including:• Improvement of weather forecasting with focus on variables that are important for RES applications;• Improvement of RES power forecasting and better estimation of uncertainties; • Streamlined extraction of optimal value through new forecasting products, data market places, and novel business models;• New data-driven optimization and decision-aid tools for market and grid management applications;• Validation of new models in living labs and assessment of forecasting value vs costly remedies to hedge uncertainties (i.e. storage). Smart4RES focuses both on improving forecasting models of weather (e.g. physical models, data assimilation, Large Eddy Simulation, enabling weather forecasts seamless) and RES production (e.g. seamless models, highly resolved predictions), and on addressing applications in power grids. Developments in the project have been formalized in Use Cases that cover a large range of time frames, technologies and geographical scales. For example, use-cases on power grids refer to the provision of ancillary services to the upper-level grid (e.g., balancing power) and the local grid (e.g., voltage control and congestion management), where the accurate forecasts of variable generation are key for accurate decision-making. A grid state forecasting will quantify dynamically the flexibility potential of RES in distribution grids. Collaborative forecasting investigates the improvement associated to local data sharing between distributed RES plants. This data sharing paves the way to a data market where agents exchange measurements, predictions or other types of valuable data. Lastly, data-driven approaches will streamline decision-making by simplifying the model chain of bidding RES production, storage dispatch or predictive management electricity grids. They will also provide interpretable hindsight to decision-makers by integrating the decisions of experts (human-in-the-loop) and will be tested in realistic laboratory conditions (software-in-the-loop).In this presentation we focus on the work done for the development of seamless numerical weather prediction (NWP) approaches to be able to couple outputs of NWP models with different resolutions and on the contribution of ultra-high resolution NWPs based on Large Eddy Simulation. A second group of results will focus on the optimal use of probabilisitic RES production and market price forecasts for trading in energy and ancillary service markets. Results from a new paradigm will be presented, where artificial intelligence methods are used to simplify the complex model chain (forecasting/trading) by a single decision making model that derives directly the bidding decisions using the available data. Results are based on data from real world test cases considered in the project.

International audience; Standard practice of decision-making in energy systems relies largely on complex modeling chains to address technical constraints and integrate numerous sources of uncertainty. The increased penetration of Renewable Energy Sources (RES) such as wind and photovoltaic plants adds complexity due to the weather dependency of their electricity production. Artificial Intelligence (AI) based tools have proven their efficiency in different applications in the energy sector ranging from forecasting to optimization and decision making. They permit to simplify modeling chains and to improve performance due to higher learning capabilities compared to state-of-the-art methods.However, decision-makers of the energy sector need to understand how decision-aid tools construct their outputs from the data. AI-based tools are often seen as black-box models and this penalizes their acceptability by end-users (traders, power system operators a.o.). The lack of interpretability of AI tools is a major challenge for the wider adoption of AI in the energy sector and a fundamental requirement to better support humans in the decision-aid process. Agents of energy systems expect very high levels of reliability for the various services they provide. As energy systems are impacted by multiple uncertainty sources (e.g. available power of RES plants, climate change, market conditions), developed AI tools should not only be performant on average situations but be able to guarantee robust solutions in the case of an extreme event.Therefore, our research focuses on understandable representations of data-driven decision-aid models for human operators in the energy sector. In order to enhance the interpretability of the AI models, a technique borrowed from the computer science domain is explored and further developed. Genetic programming and more precisely Symbolic Regression is used to derive a symbolic representation for the data-driven model that can take the form of a single equation. This equation results according to a specific reward function. The optimal solutions are selected naturally mimicking the biological theory of survival of the fittest.The main outcome is the production of symbolic representations of the AI models that require minimum changes when applied to different case studies. In this presentation a real-world use-case is considered, to demonstrate the added value of the proposed tools for decision making when trading the production of wind and solar power plants to the day-ahead market. An annual period of data is considered to train and test the proposed model. The typical modeling chain involves as many as 12 models for forecasting RES production and market quantities together with stochastic optimization to derive trading decisions. This complex chain is here replaced by a single AI-based model. Such simplification is a significant enhancement to the modeling chain interpretability and facilitates trust to the human decision-maker.This work is carried out in part in the frame of the European project Smart4RES supported by the H2020 Framework Program and in part in the frame the Marie-Curie COFUND project Ai4theSciences