Search

Paper artifacts have unique cultural and historical values. However, over time, many paper artifacts appear with disease characteristics such as embrittlement and photoaging, losing the most fundamental function of the literature archive. The reinforcement handling of degraded paper artifacts is, therefore, a necessary measure to extend their service life, the key to which lies in the reinforcement and prevention of photoaging. This paper intended to use graphene oxide (GO) as a UV protective agent, carboxymethyl cellulose (CMC) as a reinforcement, and polyethyleneimine (PEI) as a modifier. In this work, the amino-modified graphene oxide carboxymethyl cellulose composite (CMC-aGO) was prepared by chemical modification, which was used as bifunctional paper protection material with anti-ultraviolet and reinforcement. It showed excellent performance in both tensile strength testing and UV resistance testing. The CMC-aGO raw material is low cost, colorless, transparent, simple to synthesize, convenient to operate, and is an excellent conservation material with dual functions of UV aging protection and paper reinforcement. [ABSTRACT FROM AUTHOR]

We study the quadratic cycle cover problem (QCCP), which aims to find a node-disjoint cycle cover in a directed graph with minimum interaction cost between successive arcs. We derive several semidefinite programming (SDP) relaxations and use facial reduction to make these strictly feasible. We investigate a nontrivial relationship between the transformation matrix used in the reduction and the structure of the graph, which is exploited in an efficient algorithm that constructs this matrix for any instance of the problem. To solve our relaxations, we propose an algorithm that incorporates an augmented Lagrangian method into a cutting-plane framework by utilizing Dykstra's projection algorithm. Our algorithm is suitable for solving SDP relaxations with a large number of cutting-planes. Computational results show that our SDP bounds and efficient cutting-plane algorithm outperform other QCCP bounding approaches from the literature. Finally, we provide several SDP-based upper bounding techniques, among which is a sequential Q-learning method that exploits a solution of our SDP relaxation within a reinforcement learning environment. Summary of Contribution: The quadratic cycle cover problem (QCCP) is the problem of finding a set of node-disjoint cycles covering all the nodes in a graph such that the total interaction cost between successive arcs is minimized. The QCCP has applications in many fields, among which are robotics, transportation, energy distribution networks, and automatic inspection. Besides this, the problem has a high theoretical relevance because of its close connection to the quadratic traveling salesman problem (QTSP). The QTSP has several applications, for example, in bioinformatics, and is considered to be among the most difficult combinatorial optimization problems nowadays. After removing the subtour elimination constraints, the QTSP boils down to the QCCP. Hence, an in-depth study of the QCCP also contributes to the construction of strong bounds for the QTSP. In this paper, we study the application of semidefinite programming (SDP) to obtain strong bounds for the QCCP. Our strongest SDP relaxation is very hard to solve by any SDP solver because of the large number of involved cutting-planes. Because of that, we propose a new approach in which an augmented Lagrangian method is incorporated into a cutting-plane framework by utilizing Dykstra's projection algorithm. We emphasize an efficient implementation of the method and perform an extensive computational study. This study shows that our method is able to handle a large number of cuts and that the resulting bounds are currently the best QCCP bounds in the literature. We also introduce several upper bounding techniques, among which is a distributed reinforcement learning algorithm that exploits our SDP relaxations. [ABSTRACT FROM AUTHOR]

Leveraging Rubik Tables, the authors designed a constant-factor optimal algorithm for stack rearrangement and multi-robot motion planning problems under extreme robot density. A general algorithm and several specialized algorithms are then introduced to solve the problem, constructing a product between the event model and a specification automaton. Toward that, the authors analyze closed-loop dynamics to identify conditions that lead to deadlock and, using duality theory and KKT conditions, derive geometric properties of robot configurations in deadlock. [Extracted from the article]

In order to improve the effect of modern music education, this paper applies the digital information technology of music resources to the construction of the music teaching system and derives two new types of semi-decision-making process reinforcement learning algorithms based on the Bellman optimality equation base on discrete time. Moreover, this paper uses the comparative research methods to obtain the Q-value learning curve of the incremental value iterative reinforcement learning algorithm based on the semi-Markov decision process and the incremental value iterative reinforcement learning algorithm based on the dichotomy to improve the fusion effect of music teaching resources. Finally, this paper combines the actual needs of modern music education to construct an intelligent music teaching model. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of real-time onboard routing for dynamic low earth orbit (LEO) satellite networks. It is difficult to apply general routing algorithms to dynamic LEO networks due to the frequent changes in satellite topology caused by the disconnection between moving satellites. Deep reinforcement learning (DRL) models trained by various dynamic networks can be considered. However, since the inference process with the DRL model requires too long a computation time due to multiple convolutional layer operations, it is not practical to apply to a real-time on-board computer (OBC) with limited computing resources. To solve the problem, this paper proposes a practical co-design method with heterogeneous processors to parallelize and accelerate a part of the multiple convolutional layer operations on a field-programmable gate array (FPGA). The proposed method was tested with a real heterogeneous processor-based OBC and showed that the proposed method was about 3.10 times faster than the conventional method while achieving the same routing results. [ABSTRACT FROM AUTHOR]

This paper aims to help students better grasp the required professional knowledge and core concepts. This paper presents a design method for a multi-layer knowledge base based on XML. According to learners' identity characteristics and learning behaviour, using the mathematical statistics method, the feature expression for the learning system is constructed. Multivariable linear regression theory establishes convergence constraints for accurate and deep mining. The average detection results of the collected samples are used for high-quality deep mining of user portraits in the learning system. This project intends to study the method of solving accurate confidence intervals for user portrait data in the education system. Excel and Access are used to complete the data collection of the teaching object and the construction of the database. A multi-mode interactive editing and processing method of user portrait information for education systems is studied in cloud computing. Finally, a learning system based on mathematical loading mode is proposed, and an object-oriented learning recommendation system is designed. The developed teaching software can enable students to get more teaching guidance when they acquire the required knowledge to improve students' learning effect effectively. [ABSTRACT FROM AUTHOR]

This article is an editorial that focuses on the topic of decision-making and planning for multi-agent systems. Multi-agent systems are used in various real-world applications, but there are challenges in coordinating the behaviors of multiple agents. The article discusses the difficulties in translating single-agent techniques to the multi-agent domain, including complex interaction dynamics, constrained communication, and heterogeneity. The objective of this research topic is to report on recent advances in multi-agent planning and decision-making and to address the existing challenges in this area. The article includes summaries of five papers that cover different aspects of multi-agent decision-making, such as emergent communication, motion planning, reinforcement learning, and cooperative navigation. These papers provide insights into the complexity of multi-agent systems and suggest potential areas for further research. [Extracted from the article]

This document is a guest editorial for a special issue on computational methods and artificial intelligence applications in low-carbon energy systems. The editorial highlights the urgent need for advanced computing and artificial intelligence in the clean energy transition to improve system reliability, economics, and sustainability. The special issue includes 19 original research articles covering topics such as energy forecasting, situational awareness, multi-energy system dispatch, and power system operation. The articles present state-of-the-art methods and techniques in these areas, including wind power forecasting, demand-side flexibility, fault diagnosis of photovoltaic strings, and energy management strategies. The authors express their gratitude to the participating authors and anonymous reviewers for their contributions to the special section. [Extracted from the article]

Name disambiguation is a key and also a very tough problem in many online systems such as social search and academic search. Despite considerable research, a critical issue that has not been systematically studied is disambiguation on the fly — to complete the disambiguation in the real-time. This is very challenging, as the disambiguation algorithm must be accurate, efficient, and error tolerance. In this paper, we propose a novel framework — CONNA — to train a matching component and a decision component jointly via reinforcement learning. The matching component is responsible for finding the top matched candidate for the given paper, and the decision component is responsible for deciding on assigning the top matched person or creating a new person. The two components are intertwined and can be bootstrapped via jointly training. Empirically, we evaluate CONNA on two name disambiguation datasets. Experimental results show that the proposed framework can achieve a 1.21-19.84 percent improvement on F1-score using joint training of the matching and the decision components. The proposed CONNA has been successfully deployed on AMiner — a large online academic search system. [ABSTRACT FROM AUTHOR]

In October 2021, the Secure Knowledge Management (SKM) Conference convened many of scholars and practitioners to discuss and share ideas about the use and development of AI for SKM and cybersecurity. Over the past half-decade, numerous federal funding agencies such as the National Science Foundation (NSF), National Academies of Science (NAS), and National Security and Technology Council (NSTC) have noted the significant role that Artificial Intelligence (AI)-enabled analytics techniques such as deep learning, machine learning, network science, generative models, reinforcement learning, text analytics, and other techniques could play for Secure Knowledge Management (SKM), and more broadly, cybersecurity. Papers in each theme had a component of AI that was based in one or more of the major themes of extant AI-enabled cybersecurity and SKM research summarized in the previous section. [Extracted from the article]

In the process of promoting energy green transformation, the optimization of regional integrated energy system faces many challenges such as cooperative management, energy saving and emission reduction, as well as uncertainty of new energy output. Therefore, this paper proposes a multi-agent game operation method of regional integrated energy system based on carbon emission flow. First, this paper establishes a carbon emission flow calculation model for each subject, and proposes a comprehensive tariff model based on the carbon emission flow, which discounts the carbon emissions from the power supply side to the power consumption side. Secondly, considering the interests of each subject, this paper establishes the decision- making model of each subject. And the new energy uncertainty, the cost of energy preference of prosumers, and the thermal inertia of buildings are considered in the decision model. Finally, the model is solved using differential evolution algorithm and solver. The case study verifies that the comprehensive electricity pricing model based on carbon emission flow developed in this paper can play a role in balancing economy and low carbon. [ABSTRACT FROM AUTHOR]

This document is an editorial for a special issue of the journal "Technologies" focused on data science and big data in various fields such as biology, physical science, and engineering. The editorial highlights the importance of analyzing large amounts of data generated by digital technologies and the need for data scientists to use artificial intelligence and machine learning to extract valuable knowledge. The special issue includes 12 papers covering topics such as machine learning techniques for customer churn prediction, agile program management in the U.S. Navy, deep learning for cybersecurity in Industry 5.0, self-directed learning during the COVID-19 era, decision tree-based neural networks for data classification, data-driven governance in technology companies, and more. The papers explore different approaches, models, and tools in the context of data science and big data. [Extracted from the article]

Social engineering attacks are considered one of themost hazardous cyberattacks in cybersecurity, as human vulnerabilities are often the weakest link in the entire network. Such vulnerabilities are becoming increasingly susceptible to network security risks. Addressing the social engineering attack defense problem has been the focus of many studies. However, two main challenges hinder its successful resolution. Firstly, the vulnerabilities in social engineering attacks are unique due to multistage attacks, leading to incorrect social engineering defense strategies. Secondly, social engineering attacks are real-time, and the defense strategy algorithms based on gaming or reinforcement learning are too complex to make rapid decisions. This paper proposes a multiattribute quantitative incentive method based on human vulnerability and an improved Q-learning (IQL) reinforcement learning method on human vulnerability attributes. The proposed algorithm aims to address the two main challenges in social engineering attack defense by using a multiattribute incentive method based on human vulnerability to determine the optimal defense strategy. Furthermore, the IQL reinforcement learning method facilitates rapid decision-making during real-time attacks. The experimental results demonstrate that the proposed algorithm outperforms the traditional Qlearning (QL) and deep Q-network (DQN) approaches in terms of time efficiency, taking 9.1% and 19.4% less time, respectively. Moreover, the proposed algorithm effectively addresses the non-uniformity of vulnerabilities in social engineering attacks and provides a reliable defense strategy based on human vulnerability attributes. This study contributes to advancing social engineering attack defense by introducing an effective and efficient method for addressing the vulnerabilities of human factors in the cybersecurity domain. [ABSTRACT FROM AUTHOR]

In the third paper, I Deep transfer learning: a novel glucose prediction framework for new subjects with Type 2 diabetes i , the authors designed a novel cross-subject glucose prediction framework by integrating instance-based and network-based deep transfer learning via segmented continuous glucose monitoring time series. Control and optimization Six papers have devoted to the computational intelligence-based decision-making and analysis of complex systems. Complex systems, which are composed of many interconnected and interactive functional parts, widely exist in the nature and human society. [Extracted from the article]

To achieve frequency stability and economic efficiency in isolated microgrids, grid operators face a trade-off between multiple performance indicators. This paper introduces a data-driven adaptive load frequency control (DD-ALFC) approach, where the load frequency controller is modeled as an agent that can balance different objectives autonomously. The paper also proposes a priority replay soft actor critic (PR-SAC) algorithm to implement the DD-ALFC method. The PR-SAC algorithm enhances the policy randomness by using entropy regularization and maximization, and improves the learning adaptability and generalization by using priority experience replay. The proposed DD-ALFC method based on the PR-SAC algorithm can achieve higher adaptability and robustness in complex microgrid environments with multiple performance indicators, and improve both the frequency control and the economic efficiency. The paper validates the effectiveness of the proposed method in the Zhuzhou Island microgrid. [ABSTRACT FROM AUTHOR]

Recommender systems are a key technology for many applications, such as e-commerce, streaming media, and social media. Traditional recommender systems rely on collaborative filtering or content-based filtering to make recommendations. However, these approaches have limitations, such as the cold start and the data sparsity problem. This survey paper presents an in-depth analysis of the paradigm shift from conventional recommender systems to generative pre-trained-transformers-(GPT)-based chatbots. We highlight recent developments that leverage the power of GPT to create interactive and personalized conversational agents. By exploring natural language processing (NLP) and deep learning techniques, we investigate how GPT models can better understand user preferences and provide context-aware recommendations. The paper further evaluates the advantages and limitations of GPT-based recommender systems, comparing their performance with traditional methods. Additionally, we discuss potential future directions, including the role of reinforcement learning in refining the personalization aspect of these systems. [ABSTRACT FROM AUTHOR]

This document discusses the rapid development of Artificial Intelligence (AI) and its bioinspired applications. It highlights the benefits of bioinspired AI, such as increased accuracy in image and speech processing, reduced cost and energy usage through edge devices, and enhanced bio-signal quality. However, it also acknowledges the challenges posed by improper AI utilization, such as the generation of fake news and security issues. The document calls for research papers on bioinspired AI applications to explore its potential and address these challenges. It includes examples of research papers that utilize deep reinforcement learning for robot task sequencing, propose a real-time multi-surveillance pedestrian target detection model, develop an intelligent breast mass classification approach, and introduce a bio-inspired object detection algorithm for remote sensing images. The document concludes by emphasizing the importance of biomimetic artificial intelligence in various fields and promoting further research in this area. [Extracted from the article]

The article discusses the telecommunication industry has evolved to provide services for new types of Internet of Things (IoT) devices in addition to basic mobile phone or internet services. Topics include the extended reality devices, sensors, and ground and aerial robots, to name a few. With the deployment of these new services, it is difficult for the wireless network to support ubiquitous connections with diverse quality-of-service (QoS) requirements.

To achieve efficient personalized production at an affordable cost, a modular manufacturing system (MMS) can be utilized. MMS enables restructuring of its configuration to accommodate product changes and is thus an efficient solution to reduce the costs involved in personalized production. A micro smart factory (MSF) is an MMS with heterogeneous production processes to enable personalized production. Similar to MMS, MSF also enables the restructuring of production configuration; additionally, it comprises cyber-physical production systems (CPPSs) that help achieve resilience. However, MSFs need to overcome performance hurdles with respect to production control. Therefore, this paper proposes a digital twin (DT) and reinforcement learning (RL)-based production control method. This method replaces the existing dispatching rule in the type and instance phases of the MSF. In this method, the RL policy network is learned and evaluated by coordination between DT and RL. The DT provides virtual event logs that include states, actions, and rewards to support learning. These virtual event logs are returned based on vertical integration with the MSF. As a result, the proposed method provides a resilient solution to the CPPS architectural framework and achieves appropriate actions to the dynamic situation of MSF. Additionally, applying DT with RL helps decide what-next/where-next in the production cycle. Moreover, the proposed concept can be extended to various manufacturing domains because the priority rule concept is frequently applied. [ABSTRACT FROM AUTHOR]

This paper presents a collaborative filtering algorithm based on reinforcement learning theory. Then, the personalized bank financial recommendation system for users is constructed in the massive data environment. Tags mimic different types of user interest points to build a representative personalized data set. The collaborative screening of bank financial products is realized using the simulation results and users' historical access records. The ranking calculation of related financial products is added to the general bank financial product recommendation system. This method can more accurately express the query results for a specific user. It is found that the collaborative filtering algorithm based on enhanced learning theory can improve the efficiency of collaborative screening of bank financial products. The best results can be obtained by combining the two organically. This paper proposes that the recommendation algorithm of reinforcement learning bank financial products based on user preference and collaborative filtering is feasible. [ABSTRACT FROM AUTHOR]

With the successful application of artificial intelligence technology in various fields, deep reinforcement learning (DRL) algorithms have applied in active corrective control in the power system to improve accuracy and efficiency. However, the "black-box" nature of deep reinforcement learning models reduces their reliability in practical applications, making it difficult for operators to comprehend the decision-making mechanism. process of these models, thus undermining their credibility. In this paper, a DRL model is constructed based on the Markov decision process (MDP) to effectively address active corrective control issues in a 36-bus system. Furthermore, a feature importance explainability method is proposed, validating that the proposed feature importance-based explainability method enhances the transparency and reliability of the DRL model for active corrective control. [ABSTRACT FROM AUTHOR]

Autonomous Underwater Vehicles (AUVs), as a member of the unmanned intelligent ocean vehicle group, can replace human beings to complete dangerous tasks in the ocean. It is of great significance to apply reinforcement learning (RL) to AUVs to realize intelligent control. This paper proposes an AUV obstacle avoidance framework based on event-triggered reinforcement learning. Firstly, an environment perception model is designed to judge the relative position relationship between the AUV and all unknown obstacles and known targets. Secondly, considering that the detection range of AUVs is limited, and the proposed method needs to deal with unknown static obstacles and unknown dynamic obstacles at the same time, two different event-triggered mechanisms are designed. Soft actor–critic (SAC) with a non-policy sampling method is used. Then, improved reinforcement learning and the event-triggered mechanism are combined in this paper. Finally, a simulation experiment of the obstacle avoidance task is carried out on the Gazebo simulation platform. Results show that the proposed method can obtain higher rewards and complete tasks successfully. At the same time, the trajectory and the distance between each obstacle confirm that the AUV can reach the target well while maintaining a safe distance from static and dynamic obstacles. [ABSTRACT FROM AUTHOR]

The three-dimensional (3D) path planning problem of an Unmanned Aerial Vehicle (UAV) considering the effect of environmental wind in a dense city is investigated in this paper. The mission of the UAV is to fly from its initial position to its destination while ensuring safe flight. The dense obstacle avoidance and the energy consumption in 3D space need to be considered during the mission, which are often ignored in common studies. To solve these problems, an improved Deep Reinforcement Learning (DRL) path planning algorithm based on Double Deep Q-Network (DDQN) is proposed in this paper. Among the algorithms, the random obstacle training method is first proposed to make the algorithm consider various flight scenarios more globally and comprehensively and improve the algorithm's robustness and adaptability. Then, the linear soft update strategy is employed to realize the smooth neural network parameter update, which enhances the stability and convergence of the training. In addition, the wind disturbances are integrated into the energy consumption model and reward function, which can effectively describe the wind disturbances during the UAV mission to achieve the minimum drag flight. To prevent the neural network from interfering with training failures, the meritocracy mechanism is proposed to enhance the algorithm's stability. The effectiveness and applicability of the proposed method are verified through simulation analysis and comparative studies. The UAV based on this algorithm has good autonomy and adaptability, which provides a new way to solve the UAV path planning problem in dense urban scenes. [ABSTRACT FROM AUTHOR]

Space layout design is a critical aspect of architectural design, influencing functionality and aesthetics. The inherent combinatorial nature of layout design poses challenges for traditional planning approaches; thus, it demands the exploration of novel methods. This paper presents a novel framework that leverages the potential of deep reinforcement learning (RL) algorithms to optimize space layouts. RL has demonstrated remarkable success in addressing complex decision-making problems, yet its application in the design process remains relatively unexplored. We argue that RL is particularly well-suited for the design process due to its ability to accommodate offline tasks and seamless integration with existing computer-aided design software, effectively acting as a simulator for design exploration. Framing space layout design as an RL problem and employing RL methods allows for the automated exploration of the expansive design space, thereby enhancing the discovery of innovative solutions. This paper also elucidates the synergy between the design process and the RL problem, which opens new avenues for exploring the potential of RL algorithms in design. We aim to foster experimentation and collaboration within the RL and architecture communities. To facilitate our research, we have developed SpaceLayoutGym , an environment specifically designed for space layout design tasks. SpaceLayoutGym serves as a customizable environment that encapsulates the essential elements of the layout design process within an RL framework. To showcase the effectiveness of SpaceLayoutGym and the capabilities of RL as an artificial space layout designer, we employ the Proximal Policy Optimization (PPO) algorithm to train the RL agent in selected design scenarios with both geometrical constraints and topological objectives. The study further extends to contrast the effectiveness of PPO agents with that of genetic algorithms, and also includes a comparative analysis with existing layouts. Our results demonstrate the potential of RL to optimize space layouts, offering a promising direction for the future of artificial intelligence-aided design. [ABSTRACT FROM AUTHOR]

The modular robotic arms can achieve desired performances in different scenarios through the combination of various modules, and concurrently hold the potential to exhibit geometric symmetry and uniform mass symmetry. Therefore, selecting the appropriate combination of modules is crucial for realizing the functions of the robotic arm and ensuring the elegance of the system. To this end, this paper proposes a double deep Q-network (DDQN)-based configuration design algorithm for modular robotic arms, which aims to find the optimal configuration under different tasks. First, a library of small modules of collaborative robotic arms consisting of multiple tandem robotic arms is constructed. These modules are described in a standard format that can be directly imported into the software for simulation, providing greater convenience and flexibility in the development of modular robotic arms. Subsequently, the DDQN design framework for module selection is established to obtain the optimal robotic arm configuration. The proposed method could deal with the overestimation problem in the traditional deep Q-network (DQN) method and improve the estimation accuracy of the value function for each module. In addition, the experience replay mechanism is improved based on the SumTree technique, which enables the algorithm to make effective use of historical experience and prevents the algorithm from falling into local optimal solutions. Finally, comparative experiments are carried out on the PyBullet simulation platform to verify the effectiveness and superiority of the configuration design method developed in the paper. The simulation results show that the proposed DDQN-based method with experience replay mechanism has higher search efficiency and accuracy compared to the traditional DQN scheme. [ABSTRACT FROM AUTHOR]

Addressing the formidable challenges posed by multiple jammers jamming multiple radars, which arise from spatial discretization, many degrees of freedom, numerous model input parameters, and the complexity of constraints, along with a multi-peaked objective function, this paper proposes a cooperative jamming resource allocation method, based on evolutionary reinforcement learning, that uses joint multi-domain information. Firstly, an adversarial scenario model is established, characterizing the interaction between multiple jammers and radars based on a multi-beam jammer model and a radar detection model. Subsequently, considering real-world scenarios, this paper analyzes the constraints and objective function involved in cooperative jamming resource allocation by multiple jammers. Finally, accounting for the impact of spatial, frequency, and energy domain information on jamming resource allocation, matrices representing spatial condition constraints, jamming beam allocation, and jamming power allocation are formulated to characterize the cooperative jamming resource allocation problem. Based on this foundation, the joint allocation of the jamming beam and jamming power is optimized under the constraints of jamming resources. Through simulation experiments, it was determined that, compared to the dung beetle optimizer (DBO) algorithm and the particle swarm optimization (PSO) algorithm, the proposed evolutionary reinforcement learning algorithm based on DBO and Q-Learning (DBO-QL) offers 3.03% and 6.25% improvements in terms of jamming benefit and 26.33% and 50.26% improvements in terms of optimization success rate, respectively. In terms of algorithm response time, the proposed hybrid DBO-QL algorithm has a response time of 0.11 s, which is 97.35% and 96.57% lower than the response times of the DBO and PSO algorithms, respectively. The results show that the method proposed in this paper has good convergence, stability, and timeliness. [ABSTRACT FROM AUTHOR]

The low carbon park islanded microgrid faces operational challenges due to the high variability and uncertainty of distributed renewable energy sources. These sources cause severe random disturbances that impair the frequency control performance and increase the regulation cost of the islanded microgrid, jeopardizing its safety and stability. This paper presents a data-driven intelligent load frequency control (DDI-LFC) method to address this problem. The method replaces the conventional LFC controller with an intelligent agent based on a deep reinforcement learning algorithm. To adapt to the complex islanded microgrid environment and achieve adaptive multi-objective optimal frequency control, this paper proposes the quantum-inspired maximum entropy actor-critic (QIS-MEAC) algorithm, which incorporates the quantum-inspired principle and the maximum entropy exploration strategy into the actor-critic algorithm. The algorithm transforms the experience into a quantum state and leverages the quantum features to improve the deep reinforcement learning's experience replay mechanism, enhancing the data efficiency and robustness of the algorithm and thus the quality of DDI-LFC. The validation on the Yongxing Island isolated microgrid model of China Southern Grid (CSG) demonstrates that the proposed method utilizes the frequency regulation potential of distributed generation, and reduces the frequency deviation and generation cost. [ABSTRACT FROM AUTHOR]

This paper considers the confrontation problem for two tank swarms of equal size and capability in a complex urban scenario. Based on the Unity platform (2022.3.20f1c1), the confrontation scenario is constructed featuring multiple crossing roads. Through the analysis of a substantial amount of biological data and wildlife videos regarding animal behavioral strategies during confrontations for hunting or food competition, two strategies are been utilized to design a novel bio-inspired intelligent swarm confrontation algorithm. The first one is the "fire concentration" strategy, which assigns a target for each tank in a way that the isolated opponent will be preferentially attacked with concentrated firepower. The second one is the "back and forth maneuver" strategy, which makes the tank tactically retreat after firing in order to avoid being hit when the shell is reloading. Two state-of-the-art swarm confrontation algorithms, namely the reinforcement learning algorithm and the assign nearest algorithm, are chosen as the opponents for the bio-inspired swarm confrontation algorithm proposed in this paper. Data of comprehensive confrontation tests show that the bio-inspired swarm confrontation algorithm has significant advantages over its opponents from the aspects of both win rate and efficiency. Moreover, we discuss how vital algorithm parameters would influence the performance indices. [ABSTRACT FROM AUTHOR]

In our modern society, there has been a noticeable increase in pollution due to the trend of post-use handling of items. This necessitates the adoption of recycling and remanufacturing processes, advocating for sustainable resource management. This paper aims to address the issue of disassembly line balancing. Existing disassembly methods largely rely on manual labor, raising concerns regarding safety and sustainability. This paper proposes a human–machine collaborative disassembly approach to enhance safety and optimize resource utilization, aligning with sustainable development goals. A mixed-integer programming model is established, considering various disassembly techniques for hazardous and delicate parts, with the objective of minimizing the total disassembly time. The CPLEX solver is employed to enhance model accuracy. An improvement is made to the Q-learning algorithm in reinforcement learning to tackle the bilateral disassembly line balancing problem in human–machine collaboration. This approach outperforms CPLEX in both solution efficiency and quality, especially for large-scale problems. A comparative analysis with the original Q-learning algorithm and SARSA algorithm validates the superiority of the proposed algorithm in terms of convergence speed and solution quality. [ABSTRACT FROM AUTHOR]

With the rapid development of renewable energy and the increasing maturity of energy storage technology, microgrids are quickly becoming popular worldwide. The stochastic scheduling problem of microgrids can increase operational costs and resource wastage. In order to reduce operational costs and optimize resource utilization efficiency, the real-time scheduling of microgrids becomes particularly important. After collecting extensive data, reinforcement learning (RL) can provide good strategies. However, it cannot make quick and rational decisions in different environments. As a method with generalization ability, meta-learning can compensate for this deficiency. Therefore, this paper introduces a microgrid scheduling strategy based on RL and meta-learning. This method can quickly adapt to different environments with a small amount of training data, enabling rapid energy scheduling policy generation in the early stages of microgrid operation. This paper first establishes a microgrid model, including components such as energy storage, load, and distributed generation (DG). Then, we use a meta-reinforcement learning framework to train the initial scheduling strategy, considering the various operational constraints of the microgrid. The experimental results show that the MAML-based RL strategy has advantages in improving energy utilization and reducing operational costs in the early stages of microgrid operation. This research provides a new intelligent solution for microgrids' efficient, stable, and economical operation in their initial stages. [ABSTRACT FROM AUTHOR]

Load variation, distributed power output uncertainty and multi‐microgrids network complexity have brought great difficulties to the frequency stability of the whole microgrid. To address this problem, this paper uses a multi‐agent deep reinforcement learning(DRL) algorithm to design the controllers to control the frequency of the multi‐microgrids. Firstly, a load frequency control (LFC) model for multi‐microgrids was built. Secondly, based on the centralized training and decentralized execution (CTDE) multi‐agent reinforcement learning (RL) framework, the multi‐agent soft actor‐critic (MASAC) algorithm was designed and applied to the multi‐microgrids model. The state space and action space of multi‐agent were established according to the frequency deviation of every sub‐microgrid and the output of each distributed power source. The reward function was then established according to the frequency deviation. The appropriate neural network and training parameters were selected to generate the interconnected microgrid controllers through multiple training of pre‐learning. Finally, the simulation study shows that the MASAC controller proposed in this paper can quickly maintain frequency stability when the system is disturbed. Sensitivity analysis shows that the MASAC controller can effectively cope with the uncertainty of the system parameters. [ABSTRACT FROM AUTHOR]

The work presented in this paper concerns a switching-based control formulation for multi-intersection and multiphase traffic light systems. A macroscopic traffic flow modeling approach is first presented, which is instrumental to the development of a model-based and switching-based optimization method for traffic signal operation, in the framework of adaptive dynamic programming (ADP). The main advantage of the switching-based formulation is its capability to determine both “when”’ to switch and “which” mode to switch on without the need to use the cycle-based average flow approximation typical of state-of-the-art formulations. In addition, the framework can handle different cycle times across intersections without the need for synchronization constraints and, moreover, minimum dwell-time constraints can be directly enforced to comply with minimum green/red times in each phase. The simulation experiments on a multi-intersection and multiphase traffic light systems are presented to show the effectiveness of the method. [ABSTRACT FROM AUTHOR]

The article reports that With the continued growth of IoT devices and their deployment, manually managing and connecting them is impractical and presents multiple challenges. To that end, Zero Touch Networks that rely on software-based modules instead of dedicated propriety hardware become a viable potential solution. The overall aim of zero-touch networks is for machines to learn how to become more autonomous so that we can delegate complex, mundane tasks to them.