Search

In recent years, consortium blockchains have attracted considerable interest from the business communities and academia. To satisfy specific application requirements, appropriately configuring or adjusting the parameters is essential but challenging for developers when building the consortium blockchain. In this paper, the parameter configuration and adjustment for the consortium blockchain is transformed as a multi-objective optimization problem, and a novel explainable deep reinforcement learning (DRL) algorithm is proposed to solve the problem. On considering that existing DRL algorithms cannot be directly used in the consortium blockchain as they suffer from lacking of explainability, a causal model for configuring and adjusting the consortium blockchain's parameters is proposed and integrated into the DRL algorithm. The causal model can be used to derive causal explanations of the DRL algorithm to increase its trustworthiness. Furthermore, the causal model-based DRL (C-DRL) algorithm can perform causal inference before taking action to eliminate unreasonable exploration and improve the DRL algorithm's performance. The experimental results demonstrate the proposed algorithm provides the consortium blockchain with adaptive parameter configuration and adjustment to achieve sustainable high performance and security. Furthermore, the proposed algorithm increases the convergence speed of the vanilla algorithm by 49.3% and is more trustworthy than the vanilla DRL algorithm. • To achieve sustainable high performance and security, we transform the parameter configuration and adjustment for the consortium blockchain as a multi-objective optimization problem and provide a formal representation of the problem. • We propose a causal model of consortium blockchain's parameters using the structural causal model (SCM) formalism. • We propose a causal model-based DRL (C-DRL) algorithm to solve the multi-objective optimization problem. The C-DRL algorithm is equipped with the causal model of consortium blockchain's parameters. • The agent can use the causal model to derive the causal explanation for its action to provide trustworthiness. Furthermore, the agent with the causal model can utilize causal inference to eliminate unreasonable exploration and improve the convergence speed of the current DRL algorithm. • Extensive experiments are conducted to validate our proposal. The simulation results show that the agent running the C-DRL algorithm has more reasonable behavior than that running the model-free DRL algorithm. Moreover, the C-DRL algorithm converges faster than the model-free DRL algorithm. [ABSTRACT FROM AUTHOR]

Deep Reinforcement Learning systems are now a hot topic in Machine Learning for their effectiveness in many complex tasks, but their application in safety-critical domains (e.g., robot control or self-autonomous driving) remains dangerous without mechanism to detect and prevent risk situations. In Deep RL, such risk is mostly in the form of adversarial attacks, which introduce small perturbations to sensor inputs with the aim of changing the network-based decisions and thus cause catastrophic situations. In the light of these dangers, a promising line of research is that of providing these Deep RL algorithms with suitable defenses, especially when deploying in real environments. This paper suggests that this line of research could be greatly improved by the concepts from the existing research field of Safe Reinforcement Learning, which has been postulated as a family of RL algorithms capable of providing defenses against many forms of risks. However, the connections between Safe RL and the design of defenses against adversarial attacks in Deep RL remain largely unexplored. This paper seeks to explore precisely some of these connections. In particular, this paper proposes to reuse some of the concepts from existing Safe RL algorithms to create a novel and effective instance-based defense for the deployment stage of Deep RL policies. The proposed algorithm uses a risk function based on how far a state is from the state space known by the agent, that allows identifying and preventing adversarial situations. The success of the proposed defense has been evaluated in 4 Atari games. [ABSTRACT FROM AUTHOR]

Home automation is seen as a potential pillar of the smart city revolution that combines smart mobility, lifestyle and ecosystem governed by intelligent sensors connected to the internet. Households can save money and be more comfortable with automated appliances. The cost of electricity and user comfort are fundamentally contradictory, so they can be presented as a dynamic multi-objective optimization problem with fluctuating priorities for the customer to use various devices at different times. For this reason, this paper proposes an advanced Intelligent Home Energy Management (IHEM) approach based on reinforcement learning to achieve home demand response (DR) efficiency. The optimal formulation of the one-hour-ahead energy consumption scheduling problem is considered a Markov Decision Process (MDP) with discrete time steps. An efficient Neural Network (NN)-based approach with a Q-learning algorithm is developed to address this problem, enabling the IHEM system to achieve better cost-effective scheduling performance. The accurate data of electricity price and energy supplied by the Photovoltaic (PV) system are analyzed in sliding periods by machine learning for uncertainty prediction. Using the newly developed approach, which has the dual objective of minimizing the electricity bill, it is possible to obtain scheduling decisions for appliances and energy storage. The results show that the proposed optimization method reduces the monthly electricity costs by 20% compared to the Integer Linear Programming (ILP)-based HEMS method. • Artificial intelligence (AI) approach to monitor household energy consumption. • An efficient deep reinforcement learning algorithm to control activity recognition in smart homes. • Design and implementation of Intelligent home energy management system. • Converge promptly and considerably reduce operating expenses adopting AI-based approach. [ABSTRACT FROM AUTHOR]

This paper proposes a Deep Reinforcement Learning (DRL) algorithm that expands the Random Neural Network (RNN) Reinforcement Learning (RL) method to include the previous learnings entirely from previous rewards, rather than only the actual one. The Random Neural Network weighs are updated with the current reward and the previous values, including time and memory. This addition makes DRL slower to make decisions, although it also increases its performance in some experiments. Several configurations to introduce DRL, such as sampling rate and memory duration, are also proposed and analysed in this article. The proposed DRL algorithm is included in a decision process that predicts trends: upward, downward and equal market directions in addition to values. Experimental results based on market prices demonstrate that the addition of Deep Learning to the Reinforcement Learning algorithm increases its performance slightly in some experiments; however, it also increases its computational cost. In random environments such as the stock market, it is preferable to make decisions based on the previous values (short memory) rather than historical records (long memory). [ABSTRACT FROM AUTHOR]

When a complex large power system is in an emergency, the conventional discrete emergency control strategy implemented will cause excess load or derivative accidents like line power overload, thereby raising the operation risk. To overcome the problems of excessive/insufficient regulation and subsequent accidents in the above context, this paper proposes a wide area automatic generation control (WA-AGC) framework, which integrates the emergency control strategy and a performance-based frequency regulation market mechanism. According to the frequency status of the power system, WA-AGC divides the AGC into four intervals, emergency AGC (EAGC), conventional AGC (CAGC), AGC transition and optimal power flow (OPF). These four together realize a comprehensive optimization of frequency and system stability as well as economy Based on the above framework, a swarm agent exploration distributed multiple delayed deep policy gradient algorithm (SAE-MD3) is developed, which uses multiple explorers with different exploration strategies for distributed optimization. In addition, several technologies are introduced to prevent Q value overestimation and generate a more robust optimal AGC strategy. Afterward, the effectiveness and feasibility of WA-AGC are verified through the simulations of an IEEE-9 two-area system and an IEEE-118 two-area system. Compared to conventional AGC strategies, the WA-AGC algorithm reduces the constraint violation time of the power line by 92.06% and the power generation cost by 0.27% as well as improves the CPS1 index by 0.04%. • A novel AGC framework using WAMS data is proposed to substitute the-conventional AGC. • The WA-AGC framework can achieve the optimal dispatch for different objective. • A novel deep reinforcement learning algorithm with better robustness is introduced. • The WA-AGC solves the excess/lack of regulation problem and derivative accidents. • The WA-AGC considers the impact of emergency fault on the power grid. [ABSTRACT FROM AUTHOR]