Search

The container pre-marshalling problem (CPMP) aims to minimise the number of reshuffling moves, ultimately achieving an optimised stacking arrangement in each bay based on the priority of containers during the non-loading phase. Given the sequential decision nature, we formulated the CPMP as a Markov decision process (MDP) model to account for the specific state and action of the reshuffling process. To address the challenge that the relocated container may trigger a chain effect on the subsequent reshuffling moves, this paper develops an improved policy-based Monte Carlo tree search (P-MCTS) to solve the CPMP, where eight composite reshuffling rules and modified upper confidence bounds are employed in the selection phases, and a well-designed heuristic algorithm is utilised in the simulation phases. Meanwhile, considering the effectiveness of reinforcement learning methods for solving the MDP model, an improved Q-learning is proposed as the compared method. Numerical results show that the P-MCTS outperforms all compared methods in scenarios where all containers have different priorities and scenarios where containers can share the same priority. [ABSTRACT FROM AUTHOR]

This article presents a new type of hybrid model for Bayesian optimization (BO) adept at managing mixed variables, encompassing both quantitative (continuous and integer) and qualitative (categorical) types. Our proposed new hybrid models (named hybridM) merge the Monte Carlo Tree Search structure (MCTS) for categorical variables with Gaussian Processes (GP) for continuous ones. hybridM leverages the upper confidence bound tree search (UCTS) for MCTS strategy, showcasing the tree architecture’s integration into Bayesian optimization. Our innovations, including dynamic online kernel selection in the surrogate modeling phase and a unique UCTS search strategy, position our hybrid models as an advancement in mixed-variable surrogate models. Numerical experiments underscore the superiority of hybrid models, highlighting their potential in Bayesian optimization. Supplementary materials for this article are available online.

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

We present a reinforcement learning‐based heuristic for a two‐player interdiction game called the dynamic shortest path interdiction problem (DSPI). The DSPI involves an evader and an interdictor who take turns in the problem, with the interdictor selecting a set of arcs to attack and the evader choosing an arc to traverse at each step of the game. Our model employs the Monte Carlo tree search framework to learn a policy for the players using randomized roll‐outs. This policy is stored as an asymmetric game tree and can be further refined as the game unfolds. We leverage alpha–beta pruning and existing bounding schemes in the literature to prune suboptimal branches. Our numerical experiments demonstrate that the prescribed approach yields near‐optimal solutions in many cases and allows for flexibility in balancing solution quality and computational effort. [ABSTRACT FROM AUTHOR]

This paper improves the timeliness of satellite mission planning to cope with the rapid response to changes. In this paper, satellite mission planning is investigated. Firstly, the satellite dynamics model and mission planning model are established, and an improved Monte Carlo tree (Improved-MCTS) algorithm is proposed, which utilizes the Monte Carlo tree search in combination with the state uncertainty network (State-UN) to reduce the time of exploring the nodes (At the MCTS selection stage, the exploration of nodes specifically refers to the algorithm needing to decide whether to choose nodes that have already been visited (exploitation) or nodes that have not been visited yet (exploration)). The results show that this algorithm performs better in terms of profit (in this paper, the observation task is given a weight of 0–1, and each planned task will receive a profit; that is, a profit will be assigned at the initial moment) and convergence speed compared to the ant colony algorithm (ACO) and the asynchronous advantage actor critic (A3C). [ABSTRACT FROM AUTHOR]

Packing problems have been studied for a long time and have great applications in real-world scenarios. In recent times, with problems in the industrial world increasing in size, exact algorithms are often not a viable option and faster approaches are needed. We study Monte Carlo tree search, a random sampling algorithm that has gained great importance in literature in the last few years. We propose three approaches based on MCTS and its integration with metaheuristic algorithms or deep learning models to obtain approximated solutions to packing problems that are also interpretable by means of MCTS exploration and from which knowledge can be extracted. We focus on two-dimensional rectangle packing problems in our experimentation and use several well known benchmarks from literature to compare our solutions with existing approaches and offer a view on the potential uses for knowledge extraction from our method. We manage to match the quality of state-of-the-art methods, with improvements in time with respect to some of them and greater interpretability. [ABSTRACT FROM AUTHOR]

An automatic design optimization of a wireless power transfer system is performed using Monte Carlo tree search (MCTS). Several key factors, i.e., the compensation network, shapes and geometrical parameters of the coils are determined after searches, in order to achieve the high transfer efficiencies for coaxial aligned and misaligned cases. The improved Selection policy in MCTS, called first play urgency (FPU), is implemented for optimization. It proves to be a more promising technique for finding optimal solutions when the searching volume is limited, compared with the original MCTS with using the upper confidence bound applies to tree (UCB1). [ABSTRACT FROM AUTHOR]

In this article we try different algorithms, namely Nested Monte Carlo Search and Greedy Best First Search, on AstraZeneca's open source retrosynthetic tool : AiZynthFinder. We compare these algorithms to AiZynthFinder's base Monte Carlo Tree Search on a benchmark selected from the PubChem database and by Bayer's chemists. We show that both Nested Monte Carlo Search and Greedy Best First Search outperform AstraZeneca's Monte Carlo Tree Search, with a slight advantage for Nested Monte Carlo Search while experimenting on a playout heuristic. We also show how the search algorithms are bounded by the quality of the policy network, in order to improve our results the next step is to improve the policy network. [ABSTRACT FROM AUTHOR]

In this work, we are dedicated to multi-target active object tracking (AOT), where the goal is to achieve continuous tracking of targets through real-time control of camera. This form of active camera control can be applied to unmanned aerial vehicles (UAV), intelligent robots, and sports events. Our work is conducted in an environment featuring multiple cameras and targets, where our goal is to maximize target coverage. Contrasting with previous research, our work introduces additional degrees of freedom for the cameras, allowing them not only to rotate but also to move along boundary lines. In addition, we model the motion of target to predict the future position of the target in environment. With target's future position, we use Monte Carlo Tree Search (MCTS) method to find the optimal action of camera. Since the action space is large, we propose to leverage the action selection from multi-agent reinforcement learning (MARL) network to prune the search tree of Monte Carlo Tree Search method, so as to find the optimal action more efficiently. We establish a multi-target 2D environment to simulate several sports games, and experimental results demonstrate that our method can effectively improve the target coverage. The code is available at: http://github.com/HopeChanger/ActiveObjectTracking. [ABSTRACT FROM AUTHOR]

Abstract We propose a novel Deep Reinforcement Learning (DRL) architecture for sequential decision processes under uncertainty, as encountered in inspection and maintenance (I &M) planning. Unlike other DRL algorithms for (I &M) planning, the proposed +RQN architecture dispenses with computing the belief state and directly handles erroneous observations instead. We apply the algorithm to a basic I &M planning problem for a one-component system subject to deterioration. In addition, we investigate the performance of Monte Carlo tree search for the I &M problem and compare it to the +RQN. The comparison includes a statistical analysis of the two methods’ resulting policies, as well as their visualization in the belief space.

As the bias between the estimated value and the true value, overestimation is a basic problem in reinforcement learning, which leads to a lower total reward because of the incorrect action decisions. In order to reduce the impact of overestimation on reinforcement learning, we propose a double Actor-Critic learning system embedding improved Monte Carlo Tree Search (DAC-IMCTS). The proposed learning system consists of a reference module, a simulation module and an outcome module. The reference module and the simulation module are designed to compute the upper bound and lower bound of the expected reward of the agent, respectively. And the outcome module is developed to learn the agent's control policies. The reference module, constructed based on the Actor-Critic framework, provides an upper confidence bound of the expected reward. Different from the classic Actor-Critic learning system, we introduce a simulation module into the new learning system to estimate the lower confidence bound of the expected reward. We propose an improved MCTS in this module to sample the policy distribution more efficiency. Based on the lower and upper confidence bounds, we propose a confidence interval weighted estimation algorithm (CIWE) in the outcome module for generating the target expected reward. We then prove that the target expected reward generated by our method has zero expectation bias, which reduces the overestimation that exists in the classic Actor-Critic learning system. We evaluate our learning system on OpenAI Gym experimental tasks. The experimental results show that our proposed model and algorithm outperform the state-of-the-art learning systems. [ABSTRACT FROM AUTHOR]

The expansion of satellites in near-Earth and cislunar environments necessitates scalable and efficient space object tracking methodologies. Recent literature poses this problem as a sequential decision making problem for many observing agents, but decentralized decision making methods often require significant communication between agents. A two-stage approach is presented to ensure a convergent decision making methodology with limited lines of communication. First, a scheme is outlined for randomized, directed communication between agents. Guarantees on connectivity and communication times are illustrated. Additionally, communications of optimal action trajectories for another agent often lead to changes in locally optimal actions. With the assumption that such breakpoints occur as a power law in search time, asymptotic convergence toward a globally optimal consensus is demonstrated. The decision making methodology is then presented for a variety of space object sensor tasking problems in the local geostationary environment and the cislunar domain. The developed methodology is demonstrated to successfully maintain large catalogs of space objects in a manner robust to nonlinearities and failures in communication. [ABSTRACT FROM AUTHOR]

There is considerable interest in applying reinforcement learning (RL) to improve machine control across multiple industries, and the automotive industry is one of the prime examples. Monte Carlo Tree Search (MCTS) has emerged and proven powerful in decision-making games, even without understanding the rules. In this study, multibody system dynamics (MSD) control is first modeled as a Markov Decision Process and solved with Monte Carlo Tree Search. Based on randomized search space exploration, the MCTS framework builds a selective search tree by repeatedly applying a Monte Carlo rollout at each child node. However, without a library of available choices, deciding among the many possibilities for agent parameters can be intimidating. In addition, the MCTS poses a significant challenge for searching due to the large branching factor. This challenge is typically overcome by appropriate parameter design, search guiding, action reduction, parallelization, and early termination. To address these shortcomings, the overarching goal of this study is to provide needed insight into inverted pendulum controls via vanilla and modified MCTS agents, respectively. A series of reward functions are well-designed according to the control goal, which maps a specific distribution shape of reward bonus and guides the MCTS-based control to maintain the upright position. Numerical examples show that the reward-modified MCTS algorithms significantly improve the control performance and robustness of the default choice of a constant reward that constitutes the vanilla MCTS. The exponentially decaying reward functions perform better than the constant value or polynomial reward functions. Moreover, the exploitation vs. exploration trade-off and discount parameters are carefully tested. The study's results can guide the research of RL-based MSD users. [ABSTRACT FROM AUTHOR]

A static world assumption is often used when considering the simultaneous localization and mapping (SLAM) problem. In reality, especially when long-term autonomy is the objective, this is not a valid assumption. This paper studies a scenario where landmarks can occupy multiple discrete positions at different points in time, where each possible position is added to a multi-hypothesis map representation. A selector-mixture distribution is introduced and used in the observation model. Each landmark position hypothesis is associated with one component in the mixture. The landmark movements are modeled by a discrete Markov chain and the Monte Carlo tree search algorithm is suggested to be used as component selector. The non-static environment model is further incorporated into the factor graph formulation of the SLAM problem and is solved by iterating between estimating discrete variables with a component selector and optimizing continuous variables with an efficient state-of-the-art nonlinear least squares SLAM solver. The proposed non-static SLAM system is validated in numerical simulation and with a publicly available dataset by showing that a non-static environment can successfully be navigated. [ABSTRACT FROM AUTHOR]

Recent advances in cancer immunotherapy have highlighted the potential of neoantigen-based vaccines. However, the design of such vaccines is hindered by the possibility of weak binding affinity between the peptides and the patient's specific human leukocyte antigen (HLA) alleles, which may not elicit a robust adaptive immune response. Triggering cross-immunity by utilizing peptide mutations that have enhanced binding affinity to target HLA molecules, while preserving their homology with the original one, can be a promising avenue for neoantigen vaccine design. In this study, we introduced UltraMutate, a novel algorithm that combines Reinforcement Learning and Monte Carlo Tree Search, which identifies peptide mutations that not only exhibit enhanced binding affinities to target HLA molecules but also retains a high degree of homology with the original neoantigen. UltraMutate outperformed existing state-of-the-art methods in identifying affinity-enhancing mutations in an independent test set consisting of 3660 peptide–HLA pairs. UltraMutate further showed its applicability in the design of peptide vaccines for Human Papillomavirus and Human Cytomegalovirus, demonstrating its potential as a promising tool in the advancement of personalized immunotherapy. [ABSTRACT FROM AUTHOR]

We propose a novel Deep Reinforcement Learning (DRL) architecture for sequential decision processes under uncertainty, as encountered in inspection and maintenance (I &M) planning. Unlike other DRL algorithms for (I &M) planning, the proposed +RQN architecture dispenses with computing the belief state and directly handles erroneous observations instead. We apply the algorithm to a basic I &M planning problem for a one-component system subject to deterioration. In addition, we investigate the performance of Monte Carlo tree search for the I &M problem and compare it to the +RQN. The comparison includes a statistical analysis of the two methods' resulting policies, as well as their visualization in the belief space. [ABSTRACT FROM AUTHOR]

Human-like automatic deductive reasoning has always been one of the most challenging open problems in the interdisciplinary field of mathematics and artificial intelligence. This paper is the third in a series of our works. We built a neural-symbolic system, named FGeo-DRL, to automatically perform human-like geometric deductive reasoning. The neural part is an AI agent based on deep reinforcement learning, capable of autonomously learning problem-solving methods from the feedback of a formalized environment, without the need for human supervision. It leverages a pre-trained natural language model to establish a policy network for theorem selection and employ Monte Carlo Tree Search for heuristic exploration. The symbolic part is a reinforcement learning environment based on geometry formalization theory and FormalGeo, which models geometric problem solving (GPS) as a Markov Decision Process (MDP). In the formal symbolic system, the symmetry of plane geometric transformations ensures the uniqueness of geometric problems when converted into states. Finally, the known conditions and objectives of the problem form the state space, while the set of theorems forms the action space. Leveraging FGeo-DRL, we have achieved readable and verifiable automated solutions to geometric problems. Experiments conducted on the formalgeo7k dataset have achieved a problem-solving success rate of 86.40%. [ABSTRACT FROM AUTHOR]

Freeform nanostructures have the potential to support complex resonances and their interactions, which are crucial for achieving desired spectral responses. However, the design optimization of such structures is nontrivial and computationally intensive. Furthermore, the current "black box" design approaches for freeform nanostructures often neglect the underlying physics. Here, a hybrid data‐efficient neural optimizer for resonant nanostructures by combining a reinforcement learning algorithm and Powell's local optimization technique is presented. As a case study, silicon nanostructures with a highly‐saturated red color are designed and experimentally demonstrated. Specifically, color coordinates of (0.677, 0.304) in the International Commission on Illumination (CIE) chromaticity diagram – close to the ideal Schrödinger's red, with polarization independence, high reflectance (>85%), and a large viewing angle (i.e., up to ± 25°) is achieved. The remarkable performance is attributed to underlying generalized multipolar interferences within each nanostructure rather than the collective array effects. Based on that, pixel size down to ≈400 nm, corresponding to a printing resolution of 65000 pixels per inch is demonstrated. Moreover, the proposed design model requires only ≈300 iterations to effectively search a thirteen‐dimensional (13D) design space – an order of magnitude more efficient than the previously reported approaches. The work significantly extends the free‐form optical design toolbox for high‐performance flat‐optical components and metadevices. [ABSTRACT FROM AUTHOR]

The Stochastic Shortest Path (SSP) is a formalism widely used for modeling goal-oriented probabilistic planning problems. When dead ends, which are states from which goal states cannot be reached, are present in the problem and cannot be avoided, the standard criterion for solving SSPs is not well defined in these scenarios. Because of that, several alternate criteria for solving SSPs with unavoidable dead ends have been proposed in the literature. One of these criteria is GUBS (Goals with Utility-Based Semantics), a criterion that makes trade-offs between probability-to-goal and cost by combining goal prioritization with Expected Utility Theory. GUBS is a good choice for these problems because it is one of the only criteria that are known to maintain the ?-strong probability-to-goal priority property, a property that provides guarantees on how a decision criterion can choose policies without having to preprocess any specific SSP problem. Although there already exist two exact algorithms for solving GUBS, eGUBS-VI and eGUBS-AO*, both are offline and there is no algorithm for solving GUBS in an online manner. In this paper we propose UCT-GUBS, an online approximate algorithm based on UCT (a Monte Carlo tree search algorithm) that solves SSPs under the GUBS criterion. We provide an analysis of an empirical evaluation performed on two probabilistic planning domains (Triangle Tireworld and Navigation) to observe how the probability-to-goal and utility values of the resulting policies compare to the optimal values, and also how the time performance of UCT-GUBS compares to the ones of eGUBS-VI and eGUBS-AO*. Our conclusion is that, like other algorithms, the usage of UCT-GUBS has to be evaluated considering the application requirements and of the problem being solved. Depending on these factors, it can be a good alternative for obtaining policies in an online fashion while, for some problems, also being able to have better time performance than other algorithms

Quadratic Unconstrained Binary Optimization (QUBO) is a versatile approach used to represent a wide range of NP-hard Combinatorial Optimization (CO) problems through binary variables. The transformation of QUBO to an Ising Hamiltonian is recognized as an effective method for solving key optimization problems using quantum algorithms. Recently, PI-GNN, a generic framework, has been proposed to address CO problems over graphs based on QUBO with Hamiltonian loss function to train the underlying GNN architecture. Though PI-GNN is highly scalable, it exhibits a noticeable decrease in terms of the number of satisfied constraints with higher graph densities. In this paper, firstly, we identify the limitations and empirically present our strategy to improve PI-GNN’s performance. Secondly, we formulate and evaluate two strategies to integrate QUBO-Hamiltonian as the generic loss function in Reinforcement learning-based (RL) frameworks. The major contribution of our work lies in understanding the feasibility and quality of the QUBO-based generic reward function in an unsupervised RL setup in addressing graph-based CO problems. Empirically, through our empirical evaluation (Our implementation can be found in https://tinyurl.com/5apnymz7), we have observed up to 44% improvement in terms of the number of satisfied constraints over PI-GNN in a representative Max-Cut problem.

This study discusses various bandwidth allocation methods for network slicing and identifies their limitations. The heuristic-based methods were found to be insufficient in handling the randomness of varying channel quality and bandwidth requirements of each slice, while the AI-based methods were not sufficiently optimized. To address these limitations, this study proposes the MCTS-RA algorithm, which is based on Monte Carlo Tree Search methodology, to handle the randomness of each slice's channel quality and requested bandwidth and optimize the throughput. Additionally, MCTS-RA provides a generic performance index, i.e., MCTS-Level, to evaluate the performance gap of previous works to the optimal solution. The simulation results showed that MCTS-RA improved the overall throughput achieved by previous algorithms by around 10%. Furthermore, the results revealed that the MCTS-Level of previous algorithms was around 100–200, while the optimal solution's MCTS-Level was around 2000. [ABSTRACT FROM AUTHOR]

The advent of AlphaGo and its successors marked the beginning of a new paradigm in playing games using artificial intelligence. This was achieved by combining Monte Carlo tree search, a planning procedure, and deep learning. While the impact on the domain of games has been undeniable, it is less clear how useful similar approaches are in applications beyond games and how they need to be adapted from the original methodology. We perform a systematic literature review of peer-reviewed articles detailing the application of neural Monte Carlo tree search methods in domains other than games. Our goal is to systematically assess how such methods are structured in practice and if their success can be extended to other domains. We find applications in a variety of domains, many distinct ways of guiding the tree search using learned policy and value functions, and various training methods. Our review maps the current landscape of algorithms in the family of neural monte carlo tree search as they are applied to practical problems, which is a first step towards a more principled way of designing such algorithms for specific problems and their requirements. [ABSTRACT FROM AUTHOR]

With the advent of the big data era, the significance of data analysis has increasingly come to the forefront, showcasing its ability to uncover valuable insights from vast datasets, thereby enhancing the decision-making process for users. Nonetheless, the data analysis workflow faces three dominant challenges: high coupling in the analysis workflow, a plethora of interactive interfaces, and a time-intensive exploratory analysis process. To address these challenges, we introduce with this paper Navi, a data analysis system powered by natural language interaction. Navi embraces a modular design philosophy that abstracts three core functional modules from mainstream data analysis workflows: data querying, visualization generation, and visualization exploration. This approach effectively reduces the coupling of the system. Meanwhile, Navi leverages natural language as a unified interactive interface to seamlessly integrate various functional modules through a task scheduler, ensuring their effective collaboration. Moreover, in order to address the challenges of exponential search space and ambiguous user intent in visualization exploration, we propose an automated approach for visualization exploration based on Monte Carlo tree search. In addition, a pruning algorithm and a composite reward function, both incorporating visualization domain knowledge, are devised to enhance the search efficiency and result quality. Finally, we validate the effectiveness of Navi through both quantitative experiments and user studies. [ABSTRACT FROM AUTHOR]

Spatial search, and environmental monitoring are key technologies in robotics. These problems can be reformulated as maximal coverage problems with routing constraints, which are NP-hard problems. The generalized cost-benefit algorithm (GCB) can solve these problems with theoretical guarantees. To achieve better performance, evolutionary algorithms (EA) boost its performance via more samples. However, it is hard to know the terminal conditions of EA to outperform GCB. To solve these problems with theoretical guarantees and terminal conditions, in this research, the cross-entropy based Monte Carlo Tree Search algorithm (CE-MCTS) is proposed. It consists of three parts: the EA for sampling the branches, the upper confidence bound policy for selections, and the estimation of distribution algorithm for simulations. The experiments demonstrate that the CE-MCTS outperforms benchmark approaches (e.g., GCB, EAMC) in spatial search problems. [ABSTRACT FROM AUTHOR]

Abstract Freeform nanostructures have the potential to support complex resonances and their interactions, which are crucial for achieving desired spectral responses. However, the design optimization of such structures is nontrivial and computationally intensive. Furthermore, the current “black box” design approaches for freeform nanostructures often neglect the underlying physics. Here, a hybrid data‐efficient neural optimizer for resonant nanostructures by combining a reinforcement learning algorithm and Powell's local optimization technique is presented. As a case study, silicon nanostructures with a highly‐saturated red color are designed and experimentally demonstrated. Specifically, color coordinates of (0.677, 0.304) in the International Commission on Illumination (CIE) chromaticity diagram – close to the ideal Schrödinger's red, with polarization independence, high reflectance (>85%), and a large viewing angle (i.e., up to ± 25°) is achieved. The remarkable performance is attributed to underlying generalized multipolar interferences within each nanostructure rather than the collective array effects. Based on that, pixel size down to ≈400 nm, corresponding to a printing resolution of 65000 pixels per inch is demonstrated. Moreover, the proposed design model requires only ≈300 iterations to effectively search a thirteen‐dimensional (13D) design space – an order of magnitude more efficient than the previously reported approaches. The work significantly extends the free‐form optical design toolbox for high‐performance flat‐optical components and metadevices.

This paper improves the timeliness of satellite mission planning to cope with the rapid response to changes. In this paper, satellite mission planning is investigated. Firstly, the satellite dynamics model and mission planning model are established, and an improved Monte Carlo tree (Improved-MCTS) algorithm is proposed, which utilizes the Monte Carlo tree search in combination with the state uncertainty network (State-UN) to reduce the time of exploring the nodes (At the MCTS selection stage, the exploration of nodes specifically refers to the algorithm needing to decide whether to choose nodes that have already been visited (exploitation) or nodes that have not been visited yet (exploration)). The results show that this algorithm performs better in terms of profit (in this paper, the observation task is given a weight of 0–1, and each planned task will receive a profit; that is, a profit will be assigned at the initial moment) and convergence speed compared to the ant colony algorithm (ACO) and the asynchronous advantage actor critic (A3C).

In this paper, we describe the Factored Value MCTS Hybrid Cost-Max-Plus algorithm, a collection of decision-making algorithms (centralized, decentralized, and hybrid) for a multi-agent system in a collaborative setting that considers action costs. Our proposed algorithm is made up of two steps. In the first step, each agent searches for the best individual actions with the lowest cost using the Monte Carlo Tree Search (MCTS) algorithm. Each agent's most promising activities are chosen and presented to the team. The Hybrid Cost Max-Plus method is utilized for joint action selection in the second step. The Hybrid Cost Max-Plus algorithm improves the well-known centralized and distributed Max-Plus algorithm by incorporating the cost of actions in agent interactions. The Max-Plus algorithm employed the Coordination Graph framework, which exploits agent dependencies to decompose the global payoff function as the sum of local terms. In terms of the number of agents and their interactions, the suggested Factored Value MCTS-Hybrid Cost-Max-Plus method is online, anytime, distributed, and scalable. Our contribution competes with state-of-the-art methodologies and algorithms by leveraging the locality of agent interactions for planning and acting utilizing MCTS and Max-Plus algorithms. [ABSTRACT FROM AUTHOR]

Vehicle routing (VRP) and traveling salesman problems (TSP) are classical and interesting NP-hard routing combinatorial optimization (CO) with practical significance. While moving forward with artificial intelligence, researchers are paying more and more attention to applying machine learning to classical CO problems. However, traditional reinforcement learning faces challenges like reward sparsity and unstable training, so it is necessary to assist agents in finding high-quality routings in the initial model training stage to obtain more positive feedback. This paper proposes a novel Monte Carlo Tree Search (MCTS)-based two-stage multi-agent reinforcement learning training pipeline (MCRL) in which we also design a multifunctional reward function, improving efficiency, accuracy, and diversity to guide agents to learn the routings over graphs better. Besides, previous approaches are frequently too sluggish in runtime to be useful in contexts with sparsely connected networks and uncertain traffic. As an alternative, we design a model based on graph neural networks that can execute multi-agent routing in a sparsely connected graph with constantly changing traffic circumstances. Also, the agents are better equipped to collaborate online and adjust to changes thanks to our learned communication module. [ABSTRACT FROM AUTHOR]

With the escalating complexity of surface operations at large airports, the conflict risk for aircraft taxiing has correspondingly increased. Usually, the Air Traffic Controllers (ATCOs) generate route, speed and holding instructions to resolve conflicts. In this paper, we introduce a conflict resolution framework that incorporates prior knowledge by integrating a Multi-Layer Perceptron (MLP) neural network into the Monte Carlo Tree Search (MCTS) approach. The neural network is trained to learn the allocation strategy for waiting time extracted from actual aircraft taxiing trajectory data. Subsequently, the action probability distribution generated with the neural network is embedded into the MCTS algorithm as a heuristic evaluation function to guide the search process in finding the optimal conflict resolution strategy. Experimental results show that the average conflict resolution rate is 96.8% in different conflict scenarios, and the taxiing time required to resolve conflicts is reduced by an average of 42.77% compared to the taxiing time in actual airport surface operations. [ABSTRACT FROM AUTHOR]

A physics-based model for the prediction of wildfire propagation, which combines the cellular automata concept with virtual Lagrangian fire particles, is further developed to include fire extinction actions. Deposition of water and firebreaks are included in the formulation. The fire propagation model is then coupled with a Monte Carlo Tree Search (MCTS) algorithm to optimize the allocation of fire extinction actions. Starting from an ignited fire, and fixing the amount of resources available for firefighting, the model suggests which series of actions minimizes the loss of wildland. The model has been assessed and validated with model fires and then applied to a realistic scenario. MCTS optimization is found to autonomously outperform human intuition for medium-scale fires and to successfully enhance human decision-making capabilities for large-scale fires with the use of convolution-based terrain re-sampling. This study opens up new possibilities for the development of decision-making tools to assist the real-time allocation of firefighting resources as well as to support the design of preventive measures to preserve the environment and reduce the potential impact of wildfires. [ABSTRACT FROM AUTHOR]

The multi‐step retrosynthesis problem can be solved by a search algorithm, such as Monte Carlo tree search (MCTS). The performance of multistep retrosynthesis, as measured by a trade‐off in search time and route solvability, therefore depends on the hyperparameters of the search algorithm. In this paper, we demonstrated the effect of three MCTS hyperparameters (number of iterations, tree depth, and tree width) on metrics such as Linear integrated speed‐accuracy score (LISAS) and Inverse efficiency score which consider both route solvability and search time. This exploration was conducted by employing three data‐driven approaches, namely a systematic grid search, Bayesian optimization over an ensemble of molecules to obtain static MCTS hyperparameters, and a machine learning approach to dynamically predict optimal MCTS hyperparameters given an input target molecule. With the obtained results on the internal dataset, we demonstrated that it is possible to identify a hyperparameter set which outperforms the current AiZynthFinder default setting. It appeared optimal across a variety of target input molecules, both on proprietary and public datasets. The settings identified with the in‐house dataset reached a solvability of 93 % and median search time of 151 s for the in‐house dataset, and a 74 % solvability and 114 s for the ChEMBL dataset. These numbers can be compared to the current default settings which solved 85 % and 73 % during a median time of 110s and 84 s, for in‐house and ChEMBL, respectively. [ABSTRACT FROM AUTHOR]

Approximate computing (AxC) provides the scope for achieving disproportionate gains in a system's power, performance, and area (PPA) metrics by leveraging an application's inherent error-resilient behavior (BEHAV). Trading computational accuracy for performance gains makes AxC an attractive proposition for implementing computationally complex AI/ML-based applications on resource-constrained embedded systems. The growing diversity of application domains using AI/ML has also led to the increasing usage of FPGA-based embedded systems. However, implementing AxC for FPGAs has primarily been limited to the post-processing of ASIC-optimized approximate operators (AxOs). This approach usually involves selecting from a set of AxOs that have been optimized for a gate-based implementation in an ASIC. While such an approach does allow leveraging existing knowledge of ASIC-based AxO design, it limits the scope for considering the challenges and opportunities associated with FPGA's LUT-based computation structures. Similarly, the few works considering the LUT-based computing for AxO design use generic optimization approaches that do not allow integrating problem-specific prior knowledge--empirical and/or statistical. To this end, we propose a novel tree search-based approach to AxO synthesis for FPGAs. Specifically, we present a design methodology using Monte Carlo Tree Search (MCTS)-based search tree traversal that allows the designer to integrate statistical data, such as correlation, into the AxOs optimization. With the proposed methods, we report improvements over standard MCTS algorithm-based results as well as improved hypervolume for both operator-level and application-specific DSE, compared to state-of-the-art design methodologies. [ABSTRACT FROM AUTHOR]