Search

In this paper, a novel framework for designing and implementing a coordinated wide-area controller architecture for improved power system dynamic stability is presented and tested. The algorithm is an optimal wide-area system-centric controller and observer based on a hybrid reinforcement learning and temporal difference framework. It allows the system to deal with major concerns of wide-area monitoring problem: delays in signal transmission, the uncertainty of the communication network, and data traffic. The main advantage of this design is its ability to learn from the past using eligibility traces and predict the optimal trajectory of cost function through temporal difference method. The control algorithm is evolved from adaptive critic design (ACD) and performed online at a finite horizon through backward and forward view. The ACD controller's training and testing are implemented on the Innovative Integration Picolo card integrated to TMS320C28335 processor. Results on a real experimental test bed using a real power system feeder shows that this architecture provides better stability compared with conventional schemes. [ABSTRACT FROM AUTHOR]

Capability assessment plays a crucial role in the demonstration and construction of equipment. To improve the accuracy and stability of capability assessment, we study the neural network learning algorithms in the field of capability assessment and index sensitivity. Aiming at the problem of overfitting and parameter optimization in neural network learning, the paper proposes an improved machine learning algorithm—the Ensemble Learning Based on Policy Optimization Neural Networks (ELPONN) with the policy optimization and ensemble learning. This algorithm presents an optimized neural network learning algorithm through different strategies evolution, and builds an ensemble learning model of multi-intelligent algorithms to assess the capability and analyze the sensitivity of the indexes. Through the assessment of capabilities, the algorithm effectively avoids parameter optimization from entering the minimum point in performance to improve the accuracy of equipment capability assessment, which is significantly better than previous neural network assessment methods. The experimental results show that the mean relative error is 4.10%, which is better than BP, GABP, and early stopping. The ELPONN algorithm has better accuracy and stability performance, and meets the requirements of capability assessment. [ABSTRACT FROM AUTHOR]

Traditional distributed deep reinforcement learning (RL) commonly relies on exchanging the experience replay memory (RM) of each agent. Since the RM contains all state observations and action policy history, it may incur huge communication overhead while violating the privacy of each agent. Alternatively, this article presents a communication-efficient and privacy-preserving distributed RL framework, coined federated reinforcement distillation (FRD). In FRD, each agent exchanges its proxy experience RM (ProxRM), in which policies are locally averaged with respect to proxy states clustering actual states. To provide FRD design insights, we present ablation studies on the impact of ProxRM structures, neural network architectures, and communication intervals. Furthermore, we propose an improved version of FRD, coined mixup augmented FRD (MixFRD), in which ProxRM is interpolated using the mixup data augmentation algorithm. Simulations in a Cartpole environment validate the effectiveness of MixFRD in reducing the variance of mission completion time and communication cost, compared to the benchmark schemes, vanilla FRD, federated RL (FRL), and policy distillation. [ABSTRACT FROM AUTHOR]

Temporal difference (TD) learning (Sutton and Barto, 1998) has become a popular reinforcement learning technique in recent years. TD methods, relying on function approximators to generalize learning to novel situations, have had some experimental successes and have been shown to exhibit some desirable properties in theory, but the most basic algorithms have often been found slow in practice. This empirical result has motivated the development of many methods that speed up reinforcement learning by modifying a task for the learner or helping the learner better generalize to novel situations. This article focuses on generalizing across tasks, thereby speeding up learning, via a novel form of transfer using handcoded task relationships. We compare learning on a complex task with three function approximators, a cerebellar model arithmetic computer (CMAC), an artificial neural network (ANN), and a radial basis function (RBF), and empirically demonstrate that directly transferring the action-value function can lead to a dramatic speedup in learning with all three. Using transfer via inter-task mapping (TVITM), agents are able to learn one task and then markedly reduce the time it takes to learn a more complex task. Our algorithms are fully implemented and tested in the RoboCup soccer Keepaway domain. This article contains and extends material published in two conference papers (Taylor and Stone, 2005; Taylor et al., 2005). [ABSTRACT FROM AUTHOR]

Emergence of game strategy in multiagent systems is studied. Symbolic and subsymbolic (neural network) approaches are compared. Symbolic approach is represented by a backtrack algorithm with specified search depth, whereas the subsymbolic approach is represented by feedforward neural networks that are adapted by reinforcement temporal difference TD(λ) technique. As a test game, we use simplified checkers. The problem is studied in the framework of multiagent system, where each agent is endowed with a neural network used for a classification of checkers positions. Three different strategies are used. The first strategy corresponds to a single agent that repeatedly plays games against MinMax version of a backtrack search method. The second strategy corresponds to single agents that are repeatedly playing a megatournament, where each agent plays two different games with all other agents, one game with white pieces and the other game with black pieces. After finishing each game, both agents modify their neural networks by reinforcement learning. The third strategy is an evolutionary modification of the second one. When a megatournament is finished, each agent is evaluated by a fitness, which reflects its success in the given megatournament (more successful agents have greater fitness). It is demonstrated that all these approaches lead to a population of agents very successfully playing checkers against a backtrack algorithm with the search depth 3. [ABSTRACT FROM AUTHOR]

Short-term load forecasting (STLF) is very important for planning and operating power systems and markets. Various algorithms have been developed for STLF. However, numerous utilities still apply additional correction processes, which depend on experienced professionals. In this study, an STLF algorithm that uses a similar day selection method based on reinforcement learning is proposed to substitute the dependence on an expert's experience. The proposed algorithm consists of the selection of similar days, which is based on the reinforcement algorithm, and the STLF, which is based on an artificial neural network. The proposed similar day selection model based on the reinforcement learning algorithm is developed based on the Deep Q-Network technique, which is a value-based reinforcement learning algorithm. The proposed similar day selection model and load forecasting model are tested using the measured load and meteorological data for Korea. The proposed algorithm shows an improvement accuracy of load forecasting over previous algorithms. The proposed STLF algorithm is expected to improve the predictive accuracy of STLF because it can be applied in a complementary manner along with other load forecasting algorithms. [ABSTRACT FROM AUTHOR]