Search

Wake steering is a technique that optimizes the energy production of a wind farm by employing yaw control to misalign upstream turbines with the incoming wind direction. This work highlights the important dependence between wind direction variations and wake steering optimization. The problem is formalized over time as the succession of multiple steady-state yaw control problems interconnected by the rotational constraints of the turbines and the evolution of the wind. Then, this work proposes a reformulation of the yaw optimization problem of each time step by augmenting the objective function by a new heuristic based on a wind prediction. The heuristic acts as a penalization for the optimization, encouraging solutions that will guarantee future energy production. Finally, a synthetic sensitivity analysis of the wind direction variations and wake steering optimization is conducted. Because of the rotational constraints of the turbines, as the magnitude of the wind direction fluctuations increases, the importance of considering wind prediction in a steady-state optimization is empirically demonstrated. The heuristic proposed in this work greatly improves the performance of controllers and significantly reduces the complexity of the original sequential decision problem by decreasing the number of decision variables. [ABSTRACT FROM AUTHOR]

Mixed-Integer Linear Programs (MILPs) have many practical applications. Most modelling tools for MILPs fall in two broad categories. Indeed, tools such as algebraic modelling languages allow practitioners to compactly encode models using syntax close to mathematical notation but usually lack support for special structures, while other tools instead provide predefined components that can be easily assembled but modifying or adding new components is difficult. In this work, we present the inner workings of the Graph-Based Optimization Modelling Language (GBOML), an open-source modelling tool implemented in Python combining the strengths of both worlds. GBOML natively supports special structures that can be encoded by a hierarchical hypergraph, offers syntax close to mathematical notation and facilitates the modular construction and reuse of time-indexed models. We detail design choices enabling these features and show that they simplify problem encoding, lead to faster instance generation times and sometimes faster solve times. We benchmark the times taken by GBOML, JuMP, Plasmo, Pyomo and AMPL to generate instances of a structured MILP. We find that GBOML outperforms Plasmo and Pyomo, is tied with JuMP but is slower than AMPL. With parallel model generation, GBOML outperforms JuMP and closes the gap with AMPL. GBOML has the smallest memory footprint. [ABSTRACT FROM AUTHOR]

Classical reinforcement learning (RL) techniques are generally concerned with the design of decision-making policies driven by the maximisation of the expected outcome. Nevertheless, this approach does not take into consideration the potential risk associated with the actions taken, which may be critical in certain applications. To address that issue, the present research work introduces a novel methodology based on distributional RL to derive sequential decision-making policies that are sensitive to the risk, the latter being modelled by the tail of the return probability distribution. The core idea is to replace the Q function generally standing at the core of learning schemes in RL by another function, taking into account both the expected return and the risk. Named the risk-based utility function U, it can be extracted from the random return distribution Z naturally learnt by any distributional RL algorithm. This enables the spanning of the complete potential trade-off between risk minimisation and expected return maximisation, in contrast to fully risk-averse methodologies. Fundamentally, this research yields a truly practical and accessible solution for learning risk-sensitive policies with minimal modification to the distributional RL algorithm, with an emphasis on the interpretability of the resulting decision-making process. [ABSTRACT FROM AUTHOR]

Wind is an infinitely renewable energy source that is not evenly distributed in space and time. The interconnection of energy‐demanding and energy‐resourceful (yet remote) regions would help prevent energy scarcity in a world where fossil fuels are no longer used. Previous studies have shown that South Greenland and West Europe have complementary wind regimes. In particular, the southern tip of Greenland, Cape Farewell, has gained growing interest for wind farm development as it is one of the windiest places on Earth. In order to gain new insights about future wind speed variations over South Greenland, the Modèle Atmosphérique Régional (MAR), validated against in situ observations over the tundra where wind turbines are most likely to be installed, is used to build climate projections under the emission scenario SSP5‐8.5 by downscaling an ensemble of CMIP6 Earth System Models (ESMs). It appeared that between 1981 and 2100, the wind speed is projected to decrease by ~−0.8 m·s−1 at 100 m a.g.l. over the tundra surrounding Cape Farewell. This decrease is particularly marked in winter while in summer, a wind speed acceleration is projected along the ice sheet margins. An analysis of two‐dimensional wind speed changes at different vertical levels indicates that the winter decrease is likely due to a large‐scale circulation change while in summer, the katabatic winds flowing down the ice sheet are expected to increase due to an enhanced temperature contrast between the ice sheet and the surroundings. As for the mean annual maximum wind power a turbine can yield, a decrease of ~−178.1 W is projected at 100 m a.g.l. Again, the decrease is especially pronounced in winter. Considering the very high winter wind speeds occurring in South Greenland which can cut off wind turbines if too intense, the projected wind speed decrease might be beneficial for the establishment of wind farms near Cape Farewell. [ABSTRACT FROM AUTHOR]

Estimating the distance to objects is crucial for autonomous vehicles, but cost, weight or power constraints sometimes prevent the use of dedicated depth sensors. In this case, the distance has to be estimated from on-board mounted RGB cameras, which is a complex task especially for environments such as natural outdoor landscapes. In this paper, we present a new depth estimation method suitable for use in such landscapes. First, we establish a bijective relationship between depth and the visual parallax of two consecutive frames and show how to exploit it to perform motion-invariant pixel-wise depth estimation. Then, we detail our architecture which is based on a pyramidal convolutional neural network where each level refines an input parallax map estimate by using two customized cost volumes. We use these cost volumes to leverage the visual spatio-temporal constraints imposed by motion and make the network robust for varied scenes. We benchmarked our approach both in test and generalization modes on public datasets featuring synthetic camera trajectories recorded in a wide variety of outdoor scenes. Results show that our network outperforms the state of the art on these datasets, while also performing well on a standard depth estimation benchmark. [ABSTRACT FROM AUTHOR]

In the context of multistage stochastic optimization problems, we propose a hybrid strategy for generalizing to nonlinear decision rules, using machine learning, a finite data set of constrained vector-valued recourse [...]

The paper addresses the problem of decentralized optimization for a power system partitioned into several areas controlled by different transmission system operators (TSOs). The optimization variables are the settings for taps, generators' voltages and compensators', and the objective function is either based on the minimization of reactive power support, the minimization of active power losses, or a combination of both criteria. We suppose that each TSO assumes an external network equivalent for its neighboring areas and optimizes without concern for the neighboring systems' objectives its own optimization function. We study, in the context where every TSO adopts the same type of objective function, the performance of an iterative scheme, where every TSO refreshes at each iteration the parameters of its external network equivalents depending on its past internal observations, solves its local optimization problem, and then, applies its 'optimal actions' to the power system. In the context of voltage optimization, we find out that this decentralized control scheme can converge to nearly optimal global performance for relatively simple equivalents and simple procedures for fitting their parameters. Index Terms--Decentralized control, multi-TSO power system operation, network equivalent, reactive power scheduling, voltage management.

This paper compares reinforcement learning (RL) with model predictive control (MPC) in a unified framework and reports experimental results of their application to the synthesis of a controller for a nonlinear and deterministic electrical power oscillations damping problem. Both families of methods are based on the formulation of the control problem as a discrete-time optimal control problem. The considered MPC approach exploits an analytical model of the system dynamics and cost function and computes open-loop policies by applying an interior-point solver to a minimization problem in which the system dynamics are represented by equality constraints. The considered RL approach infers in a model-free way closed-loop policies from a set of system trajectories and instantaneous cost values by solving a sequence of batch-mode supervised learning problems. The results obtained provide insight into the pros and cons of the two approaches and show that RL may certainly be competitive with MPC even in contexts where a good deterministic system model is available. Index Terms--Approximate dynamic programming (ADP), electric power oscillations damping, fitted Q iteration, interior-point method (IPM), model predictive control (MPC), reinforcement learning (RL), tree-based supervised learning (SL).

This paper focuses on contingency filtering to accelerate the iterative solution of preventive security-constrained optimal power flow (PSCOPF) problems. To this end, we propose two novel filtering techniques relying on the comparison at an intermediate PSCOPF solution of post-contingency constraint violations among postulated contingencies. We assess these techniques by comparing them with severity index-based filtering schemes, on a 60- and a 118-bus system. Our results show that the proposed contingency filtering techniques lead to faster solution of the PSCOPF, while being more robust and meaningful, than severity-index based ones. Index Terms--Contingency filtering, optimal power flow, security-constrained optimal power flow, static security control.

The transmission network plays a key role in an oligopolistic electricity market. In fact, the capacity of a transmission network determines the degree to which the generators in different locations compete with others and could also greatly influence the strategic behaviors of market participants. In such an oligopolistic framework, different agents may have distinct and sometimes opposite interests in urging or hindering certain transmission expansions. Therefore, the regulatory authority, starting from the existing grid, faces the challenge of defining an optimal network upgrade to be used as benchmark for approval or rejection of a given transmission expansion. The aim of this paper is to define the concept of reference transmission network (RTN) from an economic point of view and to provide a tool for the RTN assessment in a deregulated framework where strategic behaviors are likely to appear. A general game-theoretic model for the RTN evaluation is presented, and the solution procedure is discussed. The strategic behavior of market agents in the spot market is modeled according to a Supply Function Equilibrium approach. The impact of transmission capacity expansion on market participants' strategic behavior is studied on a three-bus test network. The RTN is computed and compared with the optimal expansion found when perfect competition among power producers is assumed. Index Terms--Game theory, optimal network, strategic bidding, transmission planning.

In this paper, we explore how a computational approach to learning from interactions, called reinforcement learning (RL), can be applied to control power systems. We describe some challenges in power system control and discuss how some of those challenges could be met by using these RL methods. The difficulties associated with their application to control power systems are described and discussed as well as strategies that can be adopted to overcome them. Two reinforcement learning modes are considered: the online mode in which the interaction occurs with the real power system and the offline mode in which the interaction occurs with a simulation model of the real power system. We present two case studies made on a four-machine power system model. The first one concerns the design by means of RL algorithms used in offline mode of a dynamic brake controller. The second concerns RL methods used in online mode when applied to control a thyristor controlled series capacitor (TCSC) aimed to damp power system oscillations. Index Terms--Agent, optimal control, power system control, power system oscillations, reinforcement learning, transient stability.

We consider the joint design and control of discrete-time stochastic dynamical systems over a finite time horizon. We formulate the problem as a multi-step optimization problem under uncertainty seeking to identify a system design and a control policy that jointly maximize the expected sum of rewards collected over the time horizon considered. The transition function, the reward function and the policy are all parametrized, assumed known and differentiable with respect to their parameters. We then introduce a deep reinforcement learning algorithm combining policy gradient methods with model-based optimization techniques to solve this problem. In essence, our algorithm iteratively approximates the gradient of the expected return via Monte-Carlo sampling and automatic differentiation and takes projected gradient ascent steps in the space of environment and policy parameters. This algorithm is referred to as Direct Environment and Policy Search (DEPS). We assess the performance of our algorithm in three environments concerned with the design and control of a mass-spring-damper system, a small-scale off-grid power system and a drone, respectively. In addition, our algorithm is benchmarked against a state-of-the-art deep reinforcement learning algorithm used to tackle joint design and control problems. We show that DEPS performs at least as well or better in all three environments, consistently yielding solutions with higher returns in fewer iterations. Finally, solutions produced by our algorithm are also compared with solutions produced by an algorithm that does not jointly optimize environment and policy parameters, highlighting the fact that higher returns can be achieved when joint optimization is performed. [ABSTRACT FROM AUTHOR]

With modern video games frequently featuring sophisticated and realistic environments, the need for smart and comprehensive agents that understand the various aspects of complex environments is pressing. Since video game AI is often specifically designed for each game, video game AI tools currently focus on allowing video game developers to quickly and efficiently create specific AI. One issue with this approach is that it does not efficiently exploit the numerous similarities that exist between video games not only of the same genre, but of different genres too, resulting in a difficulty to handle the many aspects of a complex environment independently for each video game. Inspired by the human ability to detect analogies between games and apply similar behavior on a conceptual level, this paper suggests an approach based on the use of a unified conceptual framework to enable the development of conceptual AI which relies on conceptual views and actions to define basic yet reasonable and robust behavior. The approach is illustrated using two video games, Raven and StarCraft: Brood War., 1. Introduction Because artificial intelligence (AI) is a broad notion in video games, it is important to start by defining the scope of this work. A video game can be [...]

Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as gated recurrent units (GRU) and long short-term memory (LSTM). Standard gated cells share a layer internal state to store information at the network level, and long term memory is shaped by network-wide recurrent connection weights. Biological neurons on the other hand are capable of holding information at the cellular level for an arbitrary long amount of time through a process called bistability. Through bistability, cells can stabilize to different stable states depending on their own past state and inputs, which permits the durable storing of past information in neuron state. In this work, we take inspiration from biological neuron bistability to embed RNNs with long-lasting memory at the cellular level. This leads to the introduction of a new bistable biologically-inspired recurrent cell that is shown to strongly improves RNN performance on time-series which require very long memory, despite using only cellular connections (all recurrent connections are from neurons to themselves, i.e. a neuron state is not influenced by the state of other neurons). Furthermore, equipping this cell with recurrent neuromodulation permits to link them to standard GRU cells, taking a step towards the biological plausibility of GRU. With this link, this work paves the way for studying more complex and biologically plausible neuromodulation schemes as gating mechanisms in RNNs. [ABSTRACT FROM AUTHOR]

Koopman operator theory offers numerous techniques for analysis and control of complex systems. In particular, in this chapter we will argue that for the problem of convergence to an equilibrium, the Koopman operator can be used to take advantage of the geometric properties of controlled systems, thus making the optimal solutions more transparent, and easier to analyze and implement. The motivation for the study of the convergence problem comes from biological applications, where easy-to-implement and easy-to-analyze solutions are of particular value. At the moment, theoretical results have been developed for a class of nonlinear systems called monotone systems. However, the core ideas presented here can be applied heuristically to non-monotone systems. Furthermore, the convergence problem can serve as a building block for solving other control problems such as switching between stable equilibria or inducing oscillations. These applications are illustrated in biologically inspired numerical examples. [ABSTRACT FROM AUTHOR]

Animals excel at adapting their intentions, attention, and actions to the environment, making them remarkably efficient at interacting with a rich, unpredictable and ever-changing external world, a property that intelligent machines currently lack. Such an adaptation property relies heavily on cellular neuromodulation, the biological mechanism that dynamically controls intrinsic properties of neurons and their response to external stimuli in a context-dependent manner. In this paper, we take inspiration from cellular neuromodulation to construct a new deep neural network architecture that is specifically designed to learn adaptive behaviours. The network adaptation capabilities are tested on navigation benchmarks in a meta-reinforcement learning context and compared with state-of-the-art approaches. Results show that neuromodulation is capable of adapting an agent to different tasks and that neuromodulation-based approaches provide a promising way of improving adaptation of artificial systems. [ABSTRACT FROM AUTHOR]