Search

This paper addresses the challenge of minimizing training time for the control of Heating, Ventilation, and Air-conditioning (HVAC) systems with online Reinforcement Learning (RL). This is done by developing a novel approach to Multi-Agent Reinforcement Learning (MARL) to HVAC systems. In this paper, the environment formed by the HVAC system is formulated as a Markov Game (MG) in a general sum setting. The MARL algorithm is designed in a decentralized structure, where only relevant states are shared between agents, and actions are shared in a sequence, which are sensible from a system's point of view. The simulation environment is a domestic house located in Denmark and designed to resemble an average house. The heat source in the house is an air-to-water heat pump, and the HVAC system is an Underfloor Heating system (UFH). The house is subjected to weather changes from a data set collected in Copenhagen in 2006, spanning the entire year except for June, July, and August, where heat is not required. It is shown that: (1) When comparing Single Agent Reinforcement Learning (SARL) and MARL, training time can be reduced by 70% for a four temperature-zone UFH system, (2) the agent can learn and generalize over seasons, (3) the cost of heating can be reduced by 19% or the equivalent to 750 kWh of electric energy per year for an average Danish domestic house compared to a traditional control method, and (4) oscillations in the room temperature can be reduced by 40% when comparing the RL control methods with a traditional control method. [ABSTRACT FROM AUTHOR]

Environmental comfort takes a central role in the well-being and health of people. In modern industrial, commercial, and residential buildings, passive energy sources (such as solar irradiance and heat exchangers) and heating, ventilation, and air conditioning (HVAC) systems are usually employed to achieve the required comfort. While passive strategies can effectively enhance the livability of indoor spaces with limited or no energy cost, active strategies based on HVAC machines are often preferred to have direct control over the environment. Commonly, the working parameters of such machines are manually tuned to a fixed set point during working hours or throughout the whole day, leading to inefficiencies in terms of comfort and energy consumption. Albeit effective, previous works that tackle the comfort–energy tradeoff are tailored to the specific environment under study (in terms of geometry, characteristics of the building, etc.) and thus cannot be applied on a large industrial scale. We address the problem from a different angle and propose an adaptive and practical solution for comfort optimisation. It does not require the intervention of expert personnel or any customisations around the environment while it implicitly analyses the influence of different agents (e.g., passive phenomena) on the monitored parameters. A convolutional neural network (CNN) predicts the long-term impact on thermal comfort and energy consumption of a range of possible actuation strategies for the HVAC system. The decision on the best HVAC settings is taken by choosing the combination of ON/OFF and set point (SP), which optimises thermal comfort and, at the same time, minimises energy consumption. We validate our solution in a real-world scenario and through software simulations, providing a performance comparison against the fixed set point strategy and a greedy approach. The evaluation results show that our solution achieves the desired thermal comfort while reducing the energy footprint by up to approximately 16% in a real environment. [ABSTRACT FROM AUTHOR]

Retailers and major consumers of electricity generally purchase an important percentage of their estimated electricity needs years ahead in the forward market. This long-term electricity procurement task consists of determining when to buy electricity so that the resulting energy cost is minimised, and the forecast consumption is covered. In this scientific article, the focus is set on a yearly base load product from the Belgian forward market, named calendar (CAL), which is tradable up to three years ahead of the delivery period. This research paper introduces a novel algorithm providing recommendations to either buy electricity now or wait for a future opportunity based on the history of CAL prices. This algorithm relies on deep learning forecasting techniques and on an indicator quantifying the deviation from a perfectly uniform reference procurement policy. On average, the proposed approach surpasses the benchmark procurement policies considered and achieves a reduction in costs of 1.65% with respect to the perfectly uniform reference procurement policy achieving the mean electricity price. Moreover, in addition to automating the complex electricity procurement task, this algorithm demonstrates more consistent results throughout the years. Eventually, the generality of the solution presented makes it well suited for solving other commodity procurement problems. [ABSTRACT FROM AUTHOR]

Because of the high penetration of renewable energies and the installation of new control devices, modern distribution networks are faced with voltage regulation challenges. Recently, the rapid development of artificial intelligence technology has introduced new solutions for optimal control problems with high dimensions and dynamics. In this paper, a deep reinforcement learning method is proposed to solve the two-timescale optimal voltage control problem. All control variables are assigned to different agents, and discrete variables are solved by a deep Q network (DQN) agent while the continuous variables are solved by a deep deterministic policy gradient (DDPG) agent. All agents are trained simultaneously with specially designed reward aiming at minimizing long-term average voltage deviation. Case study is executed on a modified IEEE-123 bus system, and the results demonstrate that the proposed algorithm has similar or even better performance than the model-based optimal control scheme and has high computational efficiency and competitive potential for online application. [ABSTRACT FROM AUTHOR]

Accurate electricity price forecasting has become a substantial requirement since the liberalization of the electricity markets. Due to the challenging nature of electricity prices, which includes high volatility, sharp price spikes and seasonality, various types of electricity price forecasting models still compete and cannot outperform each other consistently. Neural Networks have been successfully used in machine learning problems and Recurrent Neural Networks (RNNs) have been proposed to address time-dependent learning problems. In particular, Long Short Term Memory (LSTM) and Gated Recurrent Units (GRU) are tailor-made for time series price estimation. In this paper, we propose to use multi-layer Gated Recurrent Units as a new technique for electricity price forecasting. We have trained a variety of algorithms with three-year rolling window and compared the results with the RNNs. In our experiments, three-layered GRUs outperformed all other neural network structures and state-of-the-art statistical techniques in a statistically significant manner in the Turkish day-ahead market. [ABSTRACT FROM AUTHOR]