Search

Summary: Microgrid is a new era in the power system and it has more scope of investigation on research. Due to an increase in demand and future expansion of the power system, analyzing the complexities of the network becomes a challenging task. Artificial intelligence plays a vital role in resolving such issues in a microgrid in various aspects. Owing to the rapid growth of periodical update in computational cost reduction, enhanced data analysis‐based algorithm artificial intelligence enters into new epoch Artificial Intelligence AI 2.0. Based on such approach, machine learning has been evolved as AI 2.0 initially. Now, it develops branches like deep learning, reinforcement learning, and a combination of both deep reinforcement learning algorithms. These algorithms are precise to attain higher priority in decision‐making under a complex network. This paper deals with numerous challenges of the above algorithm to state the significance of AI 2.0 and summarization of their application toward microgrid is useful to analyze the power system. [ABSTRACT FROM AUTHOR]

Due to the wide application of cognitive wireless network, the network structure is becoming more and more complex. It is difficult to establish the corresponding mathematical model to simulate the high complexity network environment. The algorithm based on recurrent neural network in deep reinforcement learning can effectively solve this problem. In addition, with the rise of deep learning in recent years, the combination of reinforcement learning and deep learning shows excellent ability in dealing with complex problems and data operation. This paper is aimed at studying dynamic spectrum allocation based on cyclic neural network. This paper briefly introduces MATLAB, sets up the network environment of algorithm simulation, then analyzes the overall performance of the improved genetic algorithm, and explores the influence of genetic algorithm-related parameters and network environment-related parameters on the performance of the algorithm. The results show the improved genetic algorithm. The network efficiency can be improved by about 2%, but the spectrum switching frequency can be reduced by 69%. When the number of primary users in the network is large, the network benefit of improving the genetic algorithm is superior to the other two algorithms. In addition, when the crossover probability is 0.6 and 0.1, the fitness value is higher than the crossover probability of 0.9 and 0.5; the interference of authorized users in the network initially has less impact on the secondary user. [ABSTRACT FROM AUTHOR]

Developing efficient motion policies for multiagents is a challenge in a decentralized dynamic situation, where each agent plans its own paths without knowing the policies of the other agents involved. This paper presents an efficient learning‐based motion planning method for multiagent systems. It adopts the framework of multiagent deep deterministic policy gradient (MADDPG) to directly map partially observed information to motion commands for multiple agents. To improve the efficiency of MADDPG in sample utilization, so as to train more brilliant agents that can adapt to more complex environments, a strategy named mixed experience (ME) is introduced to MADDPG, and this has led to our proposed ME‐MADDPG algorithm. The novel ME strategy can be embodied into three specific mechanisms: (1) an artificial potential field‐based sample generator to produce high‐quality samples in the early training stage; (2) a dynamic mixed sampling strategy to mix the training data from different sources with a variable proportion; (3) a delayed learning skill to stabilize the training of the multiple agents. A series of experiments have been conducted to verify the performance of the proposed ME‐MADDPG algorithm, and it has been demonstrated that, compared with MADDPG, the proposed algorithm can significantly improve the convergence speed and convergence effect in the training process, and it has also shown better efficiency and better adaptability in complex dynamic environments while it is used for multiagent motion planning applications. [ABSTRACT FROM AUTHOR]

To address problems of serious loss of details and low detection definition in the traditional human motion posture detection algorithm, a human motion posture detection algorithm using deep reinforcement learning is proposed. Firstly, the perception ability of deep learning is used to match human motion feature points to obtain human motion posture features. Secondly, normalize the human motion image, take the color histogram distribution of human motion posture as the antigen, search the region close to the motion posture in the image, and take its candidate region as the antibody. By calculating the affinity between the antigen and the antibody, the feature extraction of human motion posture is realized. Finally, using the training characteristics of deep learning network and reinforcement learning network, the change information of human motion posture is obtained, and the design of human motion posture detection algorithm is realized. The results show that when the image resolution is 384 × 256 px, the motion pose contour detection accuracy of this algorithm is 87%. When the image size is 30 MB, the recognition time of this method is only 0.8 s. When the number of iterations is 500, the capture rate of human motion posture details can reach 98.5%. This shows that the proposed algorithm can improve the definition of human motion posture contour, improve the posture detailed capture rate, reduce the loss of detail, and have better effect and performance. [ABSTRACT FROM AUTHOR]

Navigation technology enables indoor robots to arrive at their destinations safely. Generally, the varieties of the interior environment contribute to the difficulty of robotic navigation and hurt their performance. This paper proposes a transfer navigation algorithm and improves its generalization by leveraging deep reinforcement learning and a self-attention module. To simulate the unfurnished indoor environment, we build the virtual indoor navigation (VIN) environment to compare our model and its competitors. In the VIN environment, our method outperforms other algorithms by adapting to an unseen indoor environment. The code of the proposed model and the virtual indoor navigation environment will be released. [ABSTRACT FROM AUTHOR]

Film and television literature recommendation is an AI algorithm that recommends related content according to user preferences and records. The wide application in various APPs and websites provides users with great convenience. This article aims to study the Internet of Things and machine learning technology, combining deep learning, reinforcement learning, and recommendation algorithms, to achieve accurate recommendation of film and television literature. This paper proposes to use the ConvMF-KNN recommendation model to verify and analyze the four models of PMF, ConvM, ConvMF-word2vec, and ConvMF-KNN, respectively, on public datasets. Using the path information between vertices in bipartite graph and considering the degree of vertices, the similarity between items is calculated, and the neighbor item set of items is obtained. The experimental results show that the ConvMF-KNN model combined with the KNN idea effectively improves the recommendation accuracy. Compared with the accuracy of the PMF model on the MovieLens 100 k, MovieLens 1 M, and AIV datasets, the accuracy of the ConvMF model on the above three datasets is 5.26%, 6.31%, and 26.71%, respectively, an increase of 2.26%, 1.22%, and 7.96%. This model is of great significance. [ABSTRACT FROM AUTHOR]

Caching in device-to-device (D2D) networks is emerging a promising trend, which enables to reduce backhaul traffic. Moreover, social interaction among users influences the performance of overall system network. Therefore, it is crucial to consider social attributes in the D2D networks to develop a caching strategy to resolve the problem of unbalanced content distributed. In this paper, we consider two types of users according to their activeness, i.e., active users and inactive users. Inactive users assist active users cache contents during off-peak periods and provide the contents to the active users during peak periods to relieve the pressure of base station (BS). In addition, caching system model is divided into physical domain model and social domain model. In physical domain, the quality of communication links is judged by the delay between D2D users. In social domain, based on a real-world dataset, CiaoDVD, we calculate user similarity in three dimensions and obtain user trust by a trust topology to measure user relationships. Finally, in order to maximize the cache hit ratio, a joint action deep Q -networks (JADQN) framework is proposed to pair the active users with inactive users and distribute the contents to inactive users. Simulation results indicate that the proposed strategy improves the cache hit ratio by 42.9% and reduces the download delay by 48.8% compared with least frequency used (LFU) algorithm, which validates the effectiveness of our method. [ABSTRACT FROM AUTHOR]

The reinforcement learning algorithms based on policy gradient may fall into local optimal due to gradient disappearance during the update process, which in turn affects the exploration ability of the reinforcement learning agent. In order to solve the above problem, in this paper, the cross-entropy method (CEM) in evolution policy, maximum mean difference (MMD), and twin delayed deep deterministic policy gradient algorithm (TD3) are combined to propose a diversity evolutionary policy deep reinforcement learning (DEPRL) algorithm. By using the maximum mean discrepancy as a measure of the distance between different policies, some of the policies in the population maximize the distance between them and the previous generation of policies while maximizing the cumulative return during the gradient update. Furthermore, combining the cumulative returns and the distance between policies as the fitness of the population encourages more diversity in the offspring policies, which in turn can reduce the risk of falling into local optimal due to the disappearance of the gradient. The results in the MuJoCo test environment show that DEPRL has achieved excellent performance on continuous control tasks; especially in the Ant-v2 environment, the return of DEPRL ultimately achieved a nearly 20% improvement compared to TD3. [ABSTRACT FROM AUTHOR]

This paper proposes a DRL-based cache content update policy in the cache-enabled network to improve the cache hit ratio and reduce the average latency. In contrast to the existing policies, a more practical cache scenario is considered in this work, in which the content requests vary by both time and location. Considering the constraint of the limited cache capacity, the dynamic content update problem is modeled as a Markov decision process (MDP). Besides that, the deep Q-learning network (DQN) algorithm is utilised to solve the MDP problem. Specifically, the neural network is optimised to approximate the Q value where the training data are chosen from the experience replay memory. The DQN agent derives the optimal policy for the cache decision. Compared with the existing policies, the simulation results show that our proposed policy is 56%–64% improved in terms of the cache hit ratio and 56%–59% decreased in terms of the average latency. [ABSTRACT FROM AUTHOR]

Deep deterministic policy gradient algorithm operating over continuous space of actions has attracted great attention for reinforcement learning. However, the exploration strategy through dynamic programming within the Bayesian belief state space is rather inefficient even for simple systems. Another problem is the sequential and iterative training data with autonomous vehicles subject to the law of causality, which is against the i.i.d. (independent identically distributed) data assumption of the training samples. This usually results in failure of the standard bootstrap when learning an optimal policy. In this paper, we propose a framework of m-out-of-n bootstrapped and aggregated multiple deep deterministic policy gradient to accelerate the training process and increase the performance. Experiment results on the 2D robot arm game show that the reward gained by the aggregated policy is 10%–50% better than those gained by subpolicies. Experiment results on the open racing car simulator (TORCS) demonstrate that the new algorithm can learn successful control policies with less training time by 56.7%. Analysis on convergence is also given from the perspective of probability and statistics. These results verify that the proposed method outperforms the existing algorithms in both efficiency and performance. [ABSTRACT FROM AUTHOR]

In the past few years, the Internet has become more and more popular, and more and more people use the Internet. A lot of new software has been developed on the Internet, so the network traffic has increased rapidly. A company once clearly stated that, in the future, the network volume in our country will reach nearly 5 zb. In addition, the network is comprehensively unified, and the learning of the Internet is strengthened by the way of suggesting errors. It is also necessary to learn from the external environment to make the finite value of the network as good as possible. Reinforcement learning is about solving many difficult problems. Use reinforcement learning to promote and change this state, so that the advantages of the network can be fully developed. Regarding the environment and actions at the time as the basic mapping requirements, and the network technology strategy can be best developed. In order to solve the QoS optimization scheme of the current mainstream heuristic algorithm in the software-defined network scene, the software-defined network QoS optimization algorithm is proposed. First, the network resources and status information are unified into the network model, and then, the long- and short-term memory network is used to improve the algorithm's flow perception ability. Finally, based on the deep reinforcement learning algorithm, a dynamic traffic scheduling strategy that satisfies the QoS objective is constructed. Among them, QoS refers to the main functions of the entire network system, especially for some new users, but for many old users, this system can represent a certain application, for example, whether a certain software can get timely, whether it can work smoothly when processing videos, and whether it can be interrupted when making a voice call for improvement. [ABSTRACT FROM AUTHOR]

Existing investigations of opponent modeling and intention inferencing cannot make clear descriptions and practical explanations of the opponent's behaviors and intentions, which may inevitably limit the applicability of them. In this work, we propose a novel approach for opponent's policy explanation and intention inference based on the behavioral portrait of opponent. Specifically, we use the multiagent deep deterministic policy gradients (MADDPG) algorithm to train the agent and opponent in the competitive environment, and collect the behavioral data of opponent based on agent's observations. Then we perform pattern segmentation and extract the opponent's behavior events via Toeplitz inverse covariance‐based clustering (TICC) algorithm; hence the opponent's behavior data can be encoded into a knowledge graph, named opponent's behavior knowledge graph (OKG). Based on this, we built a question‐answer system (QA system) to query and match opponent historical information in OKG, so that the agent can obtain additional experience and gradually infer the intention of opponent with the episodes of iteration. We evaluate the proposed method on the competitive scenario in multiagent particle environment (MPE). Simulation results show that the agents are able to learn better policies with opponent portrait in competitive settings. [ABSTRACT FROM AUTHOR]

Through the recognition and analysis of human motion information, the actual motion state of human body can be obtained. However, the multifeature fusion of human behavior has limitations in recognition accuracy and robustness. Combined with deep reinforcement learning, multifeature fusion human behavior recognition is studied and we proposed a multifeature fusion human behavior recognition algorithm using deep reinforcement learning. Firstly, several typical human behavior data sets are selected as the research data in the benchmark data set. In the selected data sets, the behavior category contained in each video is the same behavior, and there are category tags. Secondly, the attention model is constructed. In the deep reinforcement learning network, the small sampling area is used as the model input. Finally, the corresponding position of the next visual area is estimated according to the time series information obtained after the input. The human behavior recognition algorithm based on deep reinforcement learning multifeature fusion is completed. The results show that the average accuracy of multifeature fusion of the algorithm is about 95%, the human behavior recognition effect is good, the identification accuracy rate is as high as about 98% and passed the camera movement impact performance test and the algorithm robustness, and the average time consumption of the algorithm is only 12.7 s, which shows that the algorithm has very broad application prospects. [ABSTRACT FROM AUTHOR]

Vanilla policy gradient methods suffer from high variance, leading to unstable policies during training, where the policy's performance fluctuates drastically between iterations. To address this issue, we analyze the policy optimization process of the navigation method based on deep reinforcement learning (DRL) that uses asynchronous gradient descent for optimization. A variant navigation (asynchronous proximal policy optimization navigation, appoNav) is presented that can guarantee the policy monotonic improvement during the process of policy optimization. Our experiments are tested in DeepMind Lab, and the experimental results show that the artificial agents with appoNav perform better than the compared algorithm. [ABSTRACT FROM AUTHOR]

Wall defect detection is an important function for autonomous decoration robots. Object detection methods based on deep neural networks require a large number of images with the handcrafted bounding box for training. Nonetheless, building large datasets manually is impractical, which is time-consuming and labor-intensive. In this work, we solve this issue to propose the low-shot wall defect detection algorithm using deep reinforcement learning (DRL) for autonomous decoration robots. Our algorithm first utilizes the attention proposal network (APN) to generate attention regions and applies AlexNet to extract the features of attention patches to further reduce computation. Finally, we train our method with deep reinforcement learning to learn the optimal detection policy. The experiments are implemented on a low-shot dataset in which images are collected from real decoration environments, and the experimental results show the proposed method can achieve fast convergence and learn the optimal detection policy for wall defect images. [ABSTRACT FROM AUTHOR]

With more businesses are running online, the scale of data centers is increasing dramatically. The task-scheduling operation with traditional heuristic algorithms is facing the challenges of uncertainty and complexity of the data center environment. It is urgent to use new technology to optimize the task scheduling to ensure the efficient task execution. This study aimed at building a new scheduling model with deep reinforcement learning algorithm, which integrated the task scheduling with resource-utilization optimization. The proposed scheduling model was trained, tested, and compared with classical scheduling algorithms on real data center datasets in experiments to show the effectiveness and efficiency. The experiment report showed that the proposed algorithm worked better than the compared classical algorithms in the key performance metrics: average delay time of tasks, task distribution in different delay time levels, and task congestion degree. [ABSTRACT FROM AUTHOR]

Controlling traffic signals to alleviate increasing traffic pressure is a concept that has received public attention for a long time. However, existing systems and methodologies for controlling traffic signals are insufficient for addressing the problem. To this end, we build a truly adaptive traffic signal control model in a traffic microsimulator, i.e., "Simulation of Urban Mobility" (SUMO), using the technology of modern deep reinforcement learning. The model is proposed based on a deep Q-network algorithm that precisely represents the elements associated with the problem: agents, environments, and actions. The real-time state of traffic, including the number of vehicles and the average speed, at one or more intersections is used as an input to the model. To reduce the average waiting time, the agents provide an optimal traffic signal phase and duration that should be implemented in both single-intersection cases and multi-intersection cases. The co-operation between agents enables the model to achieve an improvement in overall performance in a large road network. By testing with data sets pertaining to three different traffic conditions, we prove that the proposed model is better than other methods (e.g., Q-learning method, longest queue first method, and Webster fixed timing control method) for all cases. The proposed model reduces both the average waiting time and travel time, and it becomes more advantageous as the traffic environment becomes more complex. [ABSTRACT FROM AUTHOR]