Search

In order to solve the problems of low learning efficiency and slow convergence speed when mobile robot uses reinforcement learning method for path planning in complex environment, a reinforcement learning method based on each round path planning result is proposed. Firstly, the algorithm adds obstacle learning matrix to improve the success rate of path planning; and introduces heuristic reward to speed up the learning process by reducing the search space; then proposes a method of dynamically adjusting the exploration factor to balance the exploration and utilization in path planning, so as to further improve the performance of the algorithm. Finally, the simulation experiment in grid environment shows that compared with Q-learning algorithm, the improved algorithm not only shortens the average path length of the robot to reach the target position, but also speeds up the learning efficiency of the algorithm, so that the robot can find the optimal path more quickly. The code of EPRQL algorithm proposed in this paper has been published to GitHub: https://github.com/panpanpanguoguoqian/mypaper1.git. [ABSTRACT FROM AUTHOR]

In the order picking process of the warehouse center, considering the rapid increase in the volume of orders arriving at the picking center at the time of the promotional festival, a hybrid operation mode with multiple picking tables is used to meet the picking requirements of the huge number of real-time orders. Therefore, in this paper, a hybrid picking mode is proposed, taking into account both the idle degree of picking stations and their order item centers of gravity, and a new reinforcement learning algorithm embedding mechanism (PRL) with placeholder control is designed to solve the problem of a huge number of real-time item orders arriving at the picking center system on promotional holidays and in inconsistent quantities, and numerical simulations are performed for this algorithm. The experimental results show that the PRL algorithm in hybrid picking mode can handle a huge number of orders simultaneously and improve picking efficiency effectively. [ABSTRACT FROM AUTHOR]

Path planning of intelligent driving vehicles in emergencies is a hot research issue, this paper proposes a new method of the best path selection for the intelligent driving vehicles to solve this problem. Based on the prior knowledge applied reinforcement learning strategy and the searching- optimized A* algorithm, we designed a hybrid algorithm to help intelligent driving vehicles selecting the best path in the traffic network in emergencies including limited height, width, weight, accident, and traffic jam. Through simulation experiments and scene experiments, it is proved that the proposed algorithm has good stability, high efficiency, and practicability. [ABSTRACT FROM AUTHOR]

Based on the clinical states of the patient, dynamic treatment regime technology can provide various therapeutic methods, which is helpful for medical treatment policymaking. Reinforcement learning is an important approach for developing this technology. In order to implement the reinforcement learning algorithm efficiently, the computation of health data is usually outsourced to the untrustworthy cloud server. However, it may leak, falsify, or delete private health data. Encryption is a common method for solving this problem. But the cloud server is difficult to calculate encrypted health data. In this paper, based on Cheon et al.'s approximate homomorphic encryption scheme, we first propose secure computation protocols for implementing comparison, maximum, exponentiation, and division. Next, we design a homomorphic reciprocal of square root protocol firstly, which only needs one approximate computation. Based on the proposed secure computation protocols, we design a secure asynchronous advantage actor-critic reinforcement learning algorithm for the first time. Then, it is used to implement a secure treatment decision-making algorithm. Simulation results show that our secure computation protocols and algorithms are feasible. [ABSTRACT FROM AUTHOR]

The reinforcement learning algorithms based on policy gradient may fall into local optimal due to gradient disappearance during the update process, which in turn affects the exploration ability of the reinforcement learning agent. In order to solve the above problem, in this paper, the cross-entropy method (CEM) in evolution policy, maximum mean difference (MMD), and twin delayed deep deterministic policy gradient algorithm (TD3) are combined to propose a diversity evolutionary policy deep reinforcement learning (DEPRL) algorithm. By using the maximum mean discrepancy as a measure of the distance between different policies, some of the policies in the population maximize the distance between them and the previous generation of policies while maximizing the cumulative return during the gradient update. Furthermore, combining the cumulative returns and the distance between policies as the fitness of the population encourages more diversity in the offspring policies, which in turn can reduce the risk of falling into local optimal due to the disappearance of the gradient. The results in the MuJoCo test environment show that DEPRL has achieved excellent performance on continuous control tasks; especially in the Ant-v2 environment, the return of DEPRL ultimately achieved a nearly 20% improvement compared to TD3. [ABSTRACT FROM AUTHOR]

In reinforcement learning (RL), the exploration/exploitation (E/E) dilemma is a very crucial issue, which can be described as searching between the exploration of the environment to find more profitable actions, and the exploitation of the best empirical actions for the current state. We focus on the single trajectory RL problem where an agent is interacting with a partially unknown MDP over single trajectories, and try to deal with the E/E in this setting. Given the reward function, we try to find a good E/E strategy to address the MDPs under some MDP distribution. This is achieved by selecting the best strategy in mean over a potential MDP distribution from a large set of candidate strategies, which is done by exploiting single trajectories drawn from plenty of MDPs. In this paper, we mainly make the following contributions: (1) We discuss the strategy-selector algorithm based on formula set and polynomial function. (2) We provide the theoretical and experimental regret analysis of the learned strategy under an given MDP distribution. (3) We compare these methods with the 'state-of-the-art' Bayesian RL method experimentally. [ABSTRACT FROM AUTHOR]

A novel framework is proposed for integrating reconfigurable intelligent surfaces (RIS) in unmanned aerial vehicle (UAV) enabled wireless networks, where an RIS is deployed for enhancing the service quality of the UAV. Non-orthogonal multiple access (NOMA) technique is invoked to further improve the spectrum efficiency of the network, while mobile users (MUs) are considered as roaming continuously. The energy consumption minimizing problem is formulated by jointly designing the movement of the UAV, phase shifts of the RIS, power allocation policy from the UAV to MUs, as well as determining the dynamic decoding order. A decaying deep Q-network (D-DQN) based algorithm is proposed for tackling this pertinent problem. In the proposed D-DQN based algorithm, the central controller is selected as an agent for periodically observing the state of UAV-enabled wireless network and for carrying out actions to adapt to the dynamic environment. In contrast to the conventional DQN algorithm, the decaying learning rate is leveraged in the proposed D-DQN based algorithm for attaining a tradeoff between accelerating training speed and converging to the local optimal. Numerical results demonstrate that: 1) In contrast to the conventional Q-learning algorithm, which cannot converge when being adopted for solving the formulated problem, the proposed D-DQN based algorithm is capable of converging with minor constraints; 2) The energy dissipation of the UAV can be significantly reduced by integrating RISs in UAV-enabled wireless networks; 3) By designing the dynamic decoding order and power allocation policy, the RIS-NOMA case consumes 11.7% less energy than the RIS-OMA case. [ABSTRACT FROM AUTHOR]

Multi-agent reinforcement learning (MARL) has been successfully applied in many fields. In MARL, the policy evaluation problem is one of crucial problems. In order to solve this problem, distributed Temporal-Difference (TD) learning algorithm is one of the most popular methods in a cooperative manner. Despite its empirical success, however, the theory of the adaptive variant of distributed TD learning still remain limited. To fill this gap, we propose an adaptive distributed temporal-difference algorithm (referred to as MS - ADTD) under Markovian sampling over time-varying networks. Furthermore, we rigorously analyze the convergence of MS - ADTD , the theoretical results show that the local estimation can converge linearly to the optimal neighborhood. Meanwhile, the theoretical results are verified by simulation experiments. • This paper proposes a distributed adaptive TD learning algorithm under Markovian sampling. • It theoretically analyzes the non-asymptotic convergence performance of the proposed algorithm. • It verifies the performance of the proposed algorithm by experiments. [ABSTRACT FROM AUTHOR]

Machine learning research has been able to solve problems in multiple domains. Machine learning represents an open area of research for solving optimisation problems. The optimisation problems can be solved using a metaheuristic algorithm, which can find a solution in a reasonable amount of time. However, the time required to find an appropriate metaheuristic algorithm, that would have the convenient configurations to solve a set of optimisation problems properly presents a problem. The proposal described in this article contemplates an approach that automatically creates metaheuristic algorithms given a set of optimisation problems. These metaheuristic algorithms are created by modifying their logical structure via the execution of an evolutionary process. This process employs an extension of the reinforcement learning approach that considers multi-agents in their environment, and a learning agent composed of an analysis process and a process of modification of the algorithms. The approach succeeded in creating a metaheuristic algorithm that managed to solve different continuous domain optimisation problems from the experiments performed. The implications of this work are immediate because they describe a basis for the generation of metaheuristic algorithms in an online-evolution. [ABSTRACT FROM AUTHOR]

Deep Reinforcement Learning (DRL) is a class of Machine Learning (ML) that combines Deep Learning with Reinforcement Learning and provides a framework by which a system can learn from its previous actions in an environment to select its efforts in the future efficiently. DRL has been used in many application fields, including games, robots, networks, etc. for creating autonomous systems that improve themselves with experience. It is well acknowledged that DRL is well suited to solve optimization problems in distributed systems in general and network routing especially. Therefore, a novel query routing approach called Deep Reinforcement Learning based Route Selection (DRLRS) is proposed for unstructured P2P networks based on a Deep Q-Learning algorithm. The main objective of this approach is to achieve better retrieval effectiveness with reduced searching cost by less number of connected peers, exchanged messages, and reduced time. The simulation results shows a significantly improve searching a resource with compression to k-Random Walker and Directed BFS. Here, retrieval effectiveness, search cost in terms of connected peers, and average overhead are 1.28, 106, 149, respectively. [ABSTRACT FROM AUTHOR]

Robot grasping has become a very hot research field so that the requirements for robot operation are getting higher and higher. In previous research studies, the use of traditional target detection algorithms for grasping is often very inefficient, and this article is dedicated to improving the deep reinforcement learning algorithm to improve the grasping efficiency and solve the problem of robots dealing with the impact of unknown disturbances on grasping. Using the characteristic that deep reinforcement learning actively explores the unknown environment, a Gaussian parameter Deep Deterministic Policy Gradient (Gaussian-DDPG) algorithm based on the Importance-Weighted Autoencoder (IWAE) is proposed to realize the robot's autonomous learning of the grasping task. Traditional coordinate positioning methods and deep learning methods have poor grasping effects for disturbed situations (such as the movement of the target object). The IWAE algorithm is used to compress the high-dimensional information of the original visual input to the hidden space and pass it to the deep reinforcement learning network as part of the state value. Based on the classic DDPG algorithm, it smoothly adds Gaussian parameters to improve the exploratory nature of the algorithm, dynamically sets the robot grasping space parameters to adapt to the workspace of multiple scales, and finally, realizes the accurate grasping of the robot. Relying on the possible position information deviation of the visual information, the control of the grasping position by the manipulator torque information is further optimized to improve the grasping efficiency of disturbed objects. [ABSTRACT FROM AUTHOR]

Currently, industry is undergoing an exponential increase in binary-based combinatorial problems. In this regard, metaheuristics have been a common trend in the field in order to design approaches to successfully solve them. Thus, a well-known strategy includes the employment of continuous swarm-based algorithms transformed to perform in binary environments. In this work, we propose a hybrid approach that contains discrete smartly adapted population-based strategies to efficiently tackle binary-based problems. The proposed approach employs a reinforcement learning technique, known as SARSA (State–Action–Reward–State–Action), in order to utilize knowledge based on the run time. In order to test the viability and competitiveness of our proposal, we compare discrete state-of-the-art algorithms smartly assisted by SARSA. Finally, we illustrate interesting results where the proposed hybrid outperforms other approaches, thus, providing a novel option to tackle these types of problems in industry. [ABSTRACT FROM AUTHOR]