Search

In the paper entitled "Spam Detection Approach for Secure Mobile Message Communication Using Machine Learning Algorithms", L. G. Jun et al. proposed the applications of the machine learning-based spam detection method for accurate detection. The data extraction process of the review showed that (1) latent Dirichlet allocation topic modeling is among the widely used machine learning algorithms in the selected studies and (2) precision and recall are amongst the most utilized evaluation methods for measuring the performance of these machine learning algorithms. Machine learning (ML) and cryptography have many things in common, for instance, the amount of data to be handled and large search spaces. [Extracted from the article]

Task assignment is a key issue in mobile crowdsensing (MCS). Previous task assignment methods were mainly static offline assignment. However, the MCS platform needs to process dynamically changing workers and tasks online in the actual assignment process. Hence, a reliable dynamic assignment strategy is crucial to improving the platform's efficiency. This paper proposes an MCS dynamic task assignment framework to solve the task maximization assignment problem with spatiotemporal properties. First, a single worker is modeled for the Markov decision process, and a deep reinforcement learning algorithm (DDQN) is used to perform offline learning on historical task data. Then, in the dynamic assignment process, we consider the impact of current decisions on future decisions. Use the maximum flow model to maximize the number of tasks completed in each period while maximizing the expected Q value of all workers to achieve the optimal global assignment. Experiments show that the strategy proposed in this paper has good performance compared with the baseline strategy under different conditions. [ABSTRACT FROM AUTHOR]

Summary: Microgrid is a new era in the power system and it has more scope of investigation on research. Due to an increase in demand and future expansion of the power system, analyzing the complexities of the network becomes a challenging task. Artificial intelligence plays a vital role in resolving such issues in a microgrid in various aspects. Owing to the rapid growth of periodical update in computational cost reduction, enhanced data analysis‐based algorithm artificial intelligence enters into new epoch Artificial Intelligence AI 2.0. Based on such approach, machine learning has been evolved as AI 2.0 initially. Now, it develops branches like deep learning, reinforcement learning, and a combination of both deep reinforcement learning algorithms. These algorithms are precise to attain higher priority in decision‐making under a complex network. This paper deals with numerous challenges of the above algorithm to state the significance of AI 2.0 and summarization of their application toward microgrid is useful to analyze the power system. [ABSTRACT FROM AUTHOR]

Recently, artificial intelligence (AI) technology has great attention in transportation systems, which led to the emergence of a new concept known as Internet of Vehicles (IoV). The IoV has been associated with the IoT revolution and has become an active field of research due to the great need, in addition to the increase in the various applications of vehicle communication. AI provides unique solutions to enhance the quality of services (QoS) and performance of IoV systems as well. In this paper, some concepts related to deep learning networks will be discussed as one of the uses of machine learning in IoV systems, in addition to studying the effect of neural networks (NNs) and their types, as well as deep learning mechanisms that help in processing large amounts of unclassified data. Moreover, this paper briefly discusses the classification and clustering approaches in predicative analysis and reviews their abilities to enhance the performance of IoV application systems. [ABSTRACT FROM AUTHOR]

With the increasing complexity of UAV application scenarios, the performance of a single UAV cannot meet the mission requirements. Many complex tasks need the cooperation of multiple UAVs. How to coordinate UAV resources becomes the key to mission completion. In this paper, a task model including multiple UAVs and unknown obstacles is constructed, and the model is transformed into a Markov decision process (MDP). In addition, considering the influence of strategies among UAVs, a multiagent reinforcement learning algorithm based on SAC algorithm and centralized training and decentralized execution framework, MA-SAC (Multi-Agent Soft Actor-Critic), is proposed to solve the MDP. Simulation results show that the algorithm can effectively deal with the task allocation problem of multiple UAVs in this scenario, and its performance is better than other multiagent reinforcement learning algorithms. [ABSTRACT FROM AUTHOR]

Diseases can have a huge impact on the quality of life of the human population. Humans have always been in the quest to find strategies to avoid diseases that are life-threatening or affect the quality of life of humans. Effective use of resources available to human to control different diseases has always been critical. Researchers are recently more interested to find AI-based solutions to control the human population from diseases due to the overwhelming popularity of deep learning. There are many supervised techniques that have always been applied for disease diagnosis. However, the main problem of supervised based solutions is the availability of data, which is not always possible or not always complete. For instance, we do not have enough data that shows the different states of humans and different states of environments, and how all different actions taken by humans or viruses have ultimately resulted in a disease that eventually takes the lives of humans. Therefore, there is a need to find unsupervised based solutions or some techniques that do not have a dependency on the underlying dataset. In this paper, we have explored the reinforcement learning approach. We have tried different reinforcement learning algorithms to research different solutions for the prevention of diseases in the simulation of the human population. We have explored different techniques for controlling the transmission of diseases and its effects on health in the human population simulated in an environment. Our algorithms have found out policies that are best for the human population to protect themselves from the transmission and infection of malaria. The paper concludes that deep learning-based algorithms such as Deep Deterministic Policy Gradient (DDPG) have outperformed traditional algorithms such as Q-Learning or SARSA. [ABSTRACT FROM AUTHOR]

Nowadays, finding the optimal route for vehicles through online vehicle path planning is one of the main problems that the logistics industry needs to solve. Due to the uncertainty of the transportation system, especially the last-mile delivery problem of small packages in uncertain logistics transportation, the calculation of logistics vehicle routing planning becomes more complex than before. Most of the existing solutions are less applied to new technologies such as machine learning, and most of them use a heuristic algorithm. This kind of solution not only needs to set a lot of constraints but also requires much calculation time in the logistics network with high demand density. To design the uncertain logistics transportation path with minimum time, this paper proposes a new optimization strategy based on deep reinforcement learning that converts the uncertain online logistics routing problems into vehicle path planning problems and designs an embedded pointer network for obtaining the optimal solution. Considering the long time to solve the neural network, it is unrealistic to train parameters through supervised data. This article uses an unsupervised method to train the parameters. Because the process of parameter training is offline, this strategy can avoid the high delay. Through the simulation part, it is not difficult to see that the strategy proposed in this paper will effectively solve the uncertain logistics scheduling problem under the limited computing time, and it is significantly better than other strategies. Compared with traditional mathematical procedures, the algorithm proposed in this paper can reduce the driving distance by 60.71%. In addition, this paper also studies the impact of some key parameters on the effect of the program. [ABSTRACT FROM AUTHOR]

Count-based exploration algorithms have shown to be effective in dealing with various deep reinforcement learning tasks. However, existing count-based exploration algorithms cannot work well in high-dimensional state space due to the complexity of state representation. In this paper, we propose a novel count-based exploration method, which can explore high-dimensional continuous state space and combine with any reinforcement learning algorithms. Specifically, by introducing the embedding network to encode the state space and to merge the states with similar key characteristics, we can compress the high-dimensional state space. By utilizing the state binary code to count the occurrence number of states, we generate additional rewards which can encourage the agent to explore the environment. Extensive experimental results on several commonly used environments show that our proposed method outperforms other strong baselines significantly. [ABSTRACT FROM AUTHOR]

People employ a variety of devices to achieve comfort in various aspects of their lives; for example, numerous types of air conditioners are used to maintain a pleasant indoor temperature. The wide applications of heating, ventilation, and air conditioning (HVAC) systems have significantly improved the comfort level of the indoor environment for commercial buildings. However, air conditioners have their own set of issues, the most significant of which is the increased energy usage. Besides the fact that HVAC is a recent move toward the indoor temperature adjustment, the energy cost that arose from the HVAC system is still high, which takes up over 40% of the total energy. How to develop an energy-efficient HVAC system has become one of the challenging topics that need further investigation. This paper addresses this problem with the aid of deep learning (DL) algorithms. Furthermore, this study proposes an energy-efficient controller and improvement schemes for HVAC systems based on deep reinforcement learning algorithms. To reduce the implementation cost of the proposed design, this study utilizes neural network pruning techniques to develop a low-complexity HVAC controller. The experimental results show that the algorithm proposed in this study not only retains a high energy-saving rate but also reduces the implementation cost of the HVAC control algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we investigate an unmanned aerial vehicle- (UAV-) enhanced mobile edge computing network (MUEMN), where multiple UAVs are deployed as aerial edge servers to provide computing services for ground moving equipment (GME). Each GME is trained to simulate movement by a Gauss-Markov random model in this MUEMN. Under the condition of limited energy cost, UAV dynamically plans its flight position according to the movement trend of GME. Our objective is to minimize the total energy consumption of GME by jointly optimizing the offloading decisions of GME and the flight positions of UAVs. More explicitly, we model the optimization problem as a Markov decision process and achieve real-time offloading decisions via deep reinforcement learning algorithm according to the dynamic system state, where the asynchronous advantage actor-critic (A3C) framework with asynchronous characteristics is leveraged to accelerate the learning process. Finally, numerical results confirm that our proposed A3C-based offloading strategy can effectively reduce the total of energy consumption of GME and ensure the continuous operation of the GME. [ABSTRACT FROM AUTHOR]

Deep reinforcement learning is the technology of artificial neural networks in the field of decision-making and control. The traditional model-free reinforcement learning algorithm requires a large amount of environment interactive data to iterate the algorithm. This model's performance also suffers due to low utilization of training data, while the model-based reinforcement learning (MBRL) algorithm improves the efficiency of the data, MBRL locks into low prediction accuracy. Although MBRL can utilize the additional data generated by the dynamic model, a system dynamics model with low prediction accuracy will provide low-quality data and affect the algorithm's final result. In this paper, based on the A3C (Asynchronous Advantage Actor-Critic) algorithm, an improved model-based deep reinforcement learning algorithm using a learning degree network (MBRL-LDN) is presented. By comparing the differences between the predicted states outputted by the proposed multidynamic model and the original predicted states, the learning degree of the system dynamics model is calculated. The learning degree represents the quality of the data generated by the dynamic model and is used to decide whether to continue to interact with the dynamic model during a particular episode. Thus, low-quality data will be discarded. The superiority of the proposed method is verified by conducting extensive contrast experiments. [ABSTRACT FROM AUTHOR]

Film and television literature recommendation is an AI algorithm that recommends related content according to user preferences and records. The wide application in various APPs and websites provides users with great convenience. This article aims to study the Internet of Things and machine learning technology, combining deep learning, reinforcement learning, and recommendation algorithms, to achieve accurate recommendation of film and television literature. This paper proposes to use the ConvMF-KNN recommendation model to verify and analyze the four models of PMF, ConvM, ConvMF-word2vec, and ConvMF-KNN, respectively, on public datasets. Using the path information between vertices in bipartite graph and considering the degree of vertices, the similarity between items is calculated, and the neighbor item set of items is obtained. The experimental results show that the ConvMF-KNN model combined with the KNN idea effectively improves the recommendation accuracy. Compared with the accuracy of the PMF model on the MovieLens 100 k, MovieLens 1 M, and AIV datasets, the accuracy of the ConvMF model on the above three datasets is 5.26%, 6.31%, and 26.71%, respectively, an increase of 2.26%, 1.22%, and 7.96%. This model is of great significance. [ABSTRACT FROM AUTHOR]

The reinforcement learning algorithms based on policy gradient may fall into local optimal due to gradient disappearance during the update process, which in turn affects the exploration ability of the reinforcement learning agent. In order to solve the above problem, in this paper, the cross-entropy method (CEM) in evolution policy, maximum mean difference (MMD), and twin delayed deep deterministic policy gradient algorithm (TD3) are combined to propose a diversity evolutionary policy deep reinforcement learning (DEPRL) algorithm. By using the maximum mean discrepancy as a measure of the distance between different policies, some of the policies in the population maximize the distance between them and the previous generation of policies while maximizing the cumulative return during the gradient update. Furthermore, combining the cumulative returns and the distance between policies as the fitness of the population encourages more diversity in the offspring policies, which in turn can reduce the risk of falling into local optimal due to the disappearance of the gradient. The results in the MuJoCo test environment show that DEPRL has achieved excellent performance on continuous control tasks; especially in the Ant-v2 environment, the return of DEPRL ultimately achieved a nearly 20% improvement compared to TD3. [ABSTRACT FROM AUTHOR]

We study a joint pricing and inventory control problem for perishables with positive lead time in a finite horizon periodic-review system. Unlike most studies considering a continuous density function of demand, in our paper the customer demand depends on the price of current period and arrives according to a homogeneous Poisson process. We consider both backlogging and lost-sales cases, and our goal is to find a simultaneously ordering and pricing policy to maximize the expected discounted profit over the planning horizon. When there is no fixed ordering cost involved, we design a deep reinforcement learning algorithm to obtain a near-optimal ordering policy and show that there are some monotonicity properties in the learned policy. We also show that our deep reinforcement learning algorithm achieves a better performance than tabular-based Q-learning algorithms. When a fixed ordering cost is involved, we show that our deep reinforcement learning algorithm is effective and efficient, under which the problem of "curse of dimension" is circumvented. [ABSTRACT FROM AUTHOR]

In a one-on-one air combat game, the opponent's maneuver strategy is usually not deterministic, which leads us to consider a variety of opponent's strategies when designing our maneuver strategy. In this paper, an alternate freeze game framework based on deep reinforcement learning is proposed to generate the maneuver strategy in an air combat pursuit. The maneuver strategy agents for aircraft guidance of both sides are designed in a flight level with fixed velocity and the one-on-one air combat scenario. Middleware which connects the agents and air combat simulation software is developed to provide a reinforcement learning environment for agent training. A reward shaping approach is used, by which the training speed is increased, and the performance of the generated trajectory is improved. Agents are trained by alternate freeze games with a deep reinforcement algorithm to deal with nonstationarity. A league system is adopted to avoid the red queen effect in the game where both sides implement adaptive strategies. Simulation results show that the proposed approach can be applied to maneuver guidance in air combat, and typical angle fight tactics can be learnt by the deep reinforcement learning agents. For the training of an opponent with the adaptive strategy, the winning rate can reach more than 50%, and the losing rate can be reduced to less than 15%. In a competition with all opponents, the winning rate of the strategic agent selected by the league system is more than 44%, and the probability of not losing is about 75%. [ABSTRACT FROM AUTHOR]

Summary: The new generation of artificial intelligence (AI), called AI 2.0, has recently become a research focus. Data‐driven AI 2.0 will accelerate the development of smart energy and electric power system (Smart EEPS). In AI 2.0, machine learning (ML) forms a typical representative algorithm category used to achieve predictions and judgments by analyzing and learning from massive amounts of historical and synthetic data to help people make optimal decisions. ML has preliminarily been applied to the Smart Grid (SG) and Energy Internet (EI) fields, which are important Smart EEPS representatives. AI 2.0, especially ML, is undergoing a critical period of rapid development worldwide and will play an essential role in Smart EEPS. In this context, this study, combined with the emerging SG and EI technologies, takes the typical representative of AI 2.0—ML—as the research objective and reviews its research status in the operation, optimization, control, dispatching, and management of SG and EI. The paper focuses on introducing and summarizing the mainstream uses of seven representative ML methods, including reinforcement learning, deep learning, transfer learning, parallel learning, hybrid learning, adversarial learning, and ensemble learning, in the SG and EI fields. In this survey, we begin with an introduction to these seven types of ML methods and then systematically review their applications in Smart EEPS. Finally, we discuss ML development under the big data thinking and offer a prospect for the future development of AI 2.0 and ML in Smart EEPS. We conduct this survey intended to arouse the interest and excitement of experts and scholars in the EEPS industry and to look ahead to efforts that jointly promote the rapid development of AI 2.0 in the Smart EEPS field. [ABSTRACT FROM AUTHOR]

Edge caching is a promising method to deal with the traffic explosion problem towards future network. In order to satisfy the demands of user requests, the contents can be proactively cached locally at the proximity to users (e.g., base stations or user device). Recently, some learning-based edge caching optimizations are discussed. However, most of the previous studies explore the influence of dynamic and constant expanding action and caching space, leading to unpracticality and low efficiency. In this paper, we study the edge caching optimization problem by utilizing the Double Deep Q-network (Double DQN) learning framework to maximize the hit rate of user requests. Firstly, we obtain the Device-to-Device (D2D) sharing model by considering both online and offline factors and then we formulate the optimization problem, which is proved as NP-hard. Then the edge caching replacement problem is derived by Markov decision process (MDP). Finally, an edge caching strategy based on Double DQN is proposed. The experimental results based on large-scale actual traces show the effectiveness of the proposed framework. [ABSTRACT FROM AUTHOR]

The Industrial Internet of Things has grown significantly in recent years. While implementing industrial digitalization, automation, and intelligence introduced a slew of cyber risks, the complex and varied industrial Internet of Things environment provided a new attack surface for network attackers. As a result, conventional intrusion detection technology cannot satisfy the network threat discovery requirements in today's Industrial Internet of Things environment. In this research, the authors have used reinforcement learning rather than supervised and unsupervised learning, because it could very well improve the decision-making ability of the learning process by integrating abstract thinking of complete understanding, using deep knowledge to perform simple and nonlinear transformations of large-scale original input data into higher-level abstract expressions, and using learning algorithm or learning based on feedback signals, in the lack of guiding knowledge, which is based on the trial-and-error learning model, from the interaction with the environment to find the best good solution. In this respect, this article presents a near-end strategy optimization method for the Industrial Internet of Things intrusion detection system based on a deep reinforcement learning algorithm. This method combines deep learning's observation capability with reinforcement learning's decision-making capability to enable efficient detection of different kinds of cyberassaults on the Industrial Internet of Things. In this manuscript, the DRL-IDS intrusion detection system is built on a feature selection method based on LightGBM, which efficiently selects the most attractive feature set from industrial Internet of Things data; when paired with deep learning algorithms, it effectively detects intrusions. To begin, the application is based on GBM's feature selection algorithm, which extracts the most compelling feature set from Industrial Internet of Things data; then, in conjunction with the deep learning algorithm, the hidden layer of the multilayer perception network is used as the shared network structure for the value network and strategic network in the PPO2 algorithm; and finally, the intrusion detection model is constructed using the PPO2 algorithm and ReLU (R). Numerous tests conducted on a publicly available data set of the Industrial Internet of Things demonstrate that the suggested intrusion detection system detects 99 percent of different kinds of network assaults on the Industrial Internet of Things. Additionally, the accuracy rate is 0.9%. The accuracy, precision, recall rate, F1 score, and other performance indicators are superior to those of the existing intrusion detection system, which is based on deep learning models such as LSTM, CNN, and RNN, as well as deep reinforcement learning models such as DDQN and DQN. [ABSTRACT FROM AUTHOR]

Investors are frequently concerned with the potential return from changes in a company's stock price. However, stock price fluctuations are frequently highly nonlinear and nonstationary, rendering them to be uncontrollable and the primary reason why the majority of investors earn low long-term returns. Historically, people have always simulated and predicted using classic econometric models and simple machine learning models. In recent years, an increasing amount of research has been conducted using more complex machine learning and deep learning methods to forecast stock prices, and their research reports also indicate that their prediction accuracy is gradually improving. While the prediction results and accuracy of these models improve over time, their adaptability in a volatile market environment is questioned. Highly optimized machine learning algorithms include the following: FNN and the RNN are incapable of predicting the stock price of random walks and their results are frequently not consistent with stock price movements. The purpose of this article is to increase the accuracy and speed of stock price volatility prediction by incorporating the PG method's deep reinforcement learning model. Finally, our tests demonstrate that the new algorithm's prediction accuracy and reward convergence speed are significantly higher than those of the traditional DRL algorithm. As a result, the new algorithm is more adaptable to fluctuating market conditions. [ABSTRACT FROM AUTHOR]

In the past few years, the Internet has become more and more popular, and more and more people use the Internet. A lot of new software has been developed on the Internet, so the network traffic has increased rapidly. A company once clearly stated that, in the future, the network volume in our country will reach nearly 5 zb. In addition, the network is comprehensively unified, and the learning of the Internet is strengthened by the way of suggesting errors. It is also necessary to learn from the external environment to make the finite value of the network as good as possible. Reinforcement learning is about solving many difficult problems. Use reinforcement learning to promote and change this state, so that the advantages of the network can be fully developed. Regarding the environment and actions at the time as the basic mapping requirements, and the network technology strategy can be best developed. In order to solve the QoS optimization scheme of the current mainstream heuristic algorithm in the software-defined network scene, the software-defined network QoS optimization algorithm is proposed. First, the network resources and status information are unified into the network model, and then, the long- and short-term memory network is used to improve the algorithm's flow perception ability. Finally, based on the deep reinforcement learning algorithm, a dynamic traffic scheduling strategy that satisfies the QoS objective is constructed. Among them, QoS refers to the main functions of the entire network system, especially for some new users, but for many old users, this system can represent a certain application, for example, whether a certain software can get timely, whether it can work smoothly when processing videos, and whether it can be interrupted when making a voice call for improvement. [ABSTRACT FROM AUTHOR]

With the development of satellite technology, space information networks (SINs) have been applied to various fields. SINs can provide more and more complex services and receive more and more tasks. The existing resource scheduling algorithms are difficult to play an efficient role in such a complex environment of resources and tasks. We propose a resource allocation scheme based on reinforcement learning. Firstly, according to the characteristics of the resources of SINs, we established the cloud computing architecture of SINs to manage the resources uniformly. Next, we adopt a variable granularity resources clustering algorithm based on fuzzy and hierarchical clustering algorithms. This algorithm can adaptively adjust the resource size and reduce the scheduling range. Finally, we model the resource scheduling process by a reinforcement learning algorithm to solve the joint resource scheduling problem. The simulation results show that the scheme can effectively reduce resources consumption, shorten the task execution time, and improve the resource utilization efficiency of SINs. [ABSTRACT FROM AUTHOR]

The purpose of stock market investment is to obtain more profits. In recent years, an increasing number of researchers have tried to implement stock trading based on machine learning. Facing the complex stock market, how to obtain effective information from multisource data and implement dynamic trading strategies is difficult. To solve these problems, this study proposes a new deep reinforcement learning model to implement stock trading, analyzes the stock market through stock data, technical indicators and candlestick charts, and learns dynamic trading strategies. Fusing the features of different data sources extracted by the deep neural network as the state of the stock market, the agent in reinforcement learning makes trading decisions on this basis. Experiments on the Chinese stock market dataset and the S&P 500 stock market dataset show that our trading strategy can obtain higher profits compared with other trading strategies. [ABSTRACT FROM AUTHOR]

Gas turbine is widely used because of its advantages of fast start and stop, no pollution, and high thermal efficiency. However, the working environment of high temperature, high pressure, and high speed makes the gas turbine prone to failure. The traditional gas path fault intelligent diagnosis scheme of the gas turbine has the problems of poor control effect and low scheduling accuracy. Experiment studies the application of neural network and reinforcement learning algorithm in gas path fault intelligent diagnosis of the gas turbine. The accurate control of fault diagnosis planning is realized from gas path fault diagnosis, daily maintenance, service condition monitoring, power utilization rate, and other aspects of the gas turbine. The reinforcement learning model can realize the intelligent diagnosis and record of gas path fault of the gas turbine, to achieve diversified analysis and intelligent diagnosis scheme. Through neural network algorithm and deep learning technology, the whole process monitoring of the gas turbine is realized, and the failure rate of the gas turbine in the working process is reduced. The experimental results show that, compared with the thermal fault diagnosis method and the fault diagnosis method of the electric percussion drill, using thermal imaging, the gas turbine gas path fault intelligent diagnosis model based on the reinforcement learning algorithm can complete the data information in the process of real-time data transmission. The quantified conversion and processing of the system has the advantages of higher control accuracy and faster response speed, which can effectively improve the diagnostic efficiency and accuracy. [ABSTRACT FROM AUTHOR]

With the intensification of market competition and the development of market globalization, the efficiency of supply chain management orders has become an important part of enterprise competition resources. The competition among enterprises is fierce. To achieve effective customer response quickly, the time for supply chain order management is minimized, and refine the order processing process. This article introduces the strategy research of supply chain management order based on a reinforcement learning algorithm. This article first combines the reinforcement learning algorithm and deep learning algorithm, using the optimal decision-making ability of reinforcement learning algorithm and deep learning algorithm. The combination of data perception and the optimal ability to analyze examine the data of the order process, order cycle, and order delivery process of the supply chain order management and give the optimal decision. The supply chain order management process conducts questionnaire surveys and seminars to understand the current process of supply chain order management and the problems derived from the analysis of data based on the deep learning algorithm. Finally, through the output of the optimal strategy of the reinforcement learning algorithm, the supply chain order management process was improved, and the satisfaction survey was conducted again. The survey showed that the satisfaction was improved, and the satisfaction reached more than 90%. [ABSTRACT FROM AUTHOR]

Summary: Integrating weather‐dependent renewable energy sources into the electricity system impose challenges on the power grid. Balancing services are needed, which can be provided by virtual power plants (VPP) that aggregate distributed energy resources (DER) to consume or produce electricity on demand. Electric vehicle (EV) fleets can use idle cars' batteries as combined storage to offer balancing services on smart electricity markets. However, there are risks associated with this business model extension. The fleet faces severe imbalance penalties if it cannot charge the offered amount of balancing energy due to the vehicles' unpredicted mobility demand. Ensuring the fleet can fulfill all market commitments risks denying profitable customer rentals. We study the design of a decision support system that estimates these risks, dynamically adjusts the composition of a VPP portfolio, and profitably places bids on multiple electricity markets simultaneously. Here we show that a reinforcement learning agent can optimize the VPP portfolio by learning from favorable market conditions and fleet demand uncertainties. In comparison to previous research, in which the bidding risks were unknown and fleets could only offer conservative amounts of balancing power to a single market, our proposed approach increases the amount of offered balancing power by 48% to 82% and achieves a charging cost reduction of the fleet by 25%. In experiments with real‐world carsharing data of 500 EVs, we found that mobility demand forecasting algorithms' accuracy is crucial for a successful bidding strategy. Moreover, we show that recent advancements in deep reinforcement learning decrease the convergence time and improve the results' robustness. Our results demonstrate how modern RL algorithms can be successfully used for fleet management, VPP optimization, and demand response in the smart grid. We anticipate that DER, such as EVs, will play an essential role in providing reliable backup power for the grid and formulate market design recommendations to allow easier access to these resources. [ABSTRACT FROM AUTHOR]

With more businesses are running online, the scale of data centers is increasing dramatically. The task-scheduling operation with traditional heuristic algorithms is facing the challenges of uncertainty and complexity of the data center environment. It is urgent to use new technology to optimize the task scheduling to ensure the efficient task execution. This study aimed at building a new scheduling model with deep reinforcement learning algorithm, which integrated the task scheduling with resource-utilization optimization. The proposed scheduling model was trained, tested, and compared with classical scheduling algorithms on real data center datasets in experiments to show the effectiveness and efficiency. The experiment report showed that the proposed algorithm worked better than the compared classical algorithms in the key performance metrics: average delay time of tasks, task distribution in different delay time levels, and task congestion degree. [ABSTRACT FROM AUTHOR]