Search

In the field of security, multi-objective security games (MOSGs) allow defenders to simultaneously protect targets from multiple heterogeneous attackers. MOSGs aim to simultaneously maximize all the heterogeneous payoffs, e.g., life, money, and crime rate, without merging heterogeneous attackers. In real-world scenarios, the number of heterogeneous attackers and targets to be protected may exceed the capability of most existing state-of-the-art methods, i.e., MOSGs are limited by the issue of scalability. To this end, this paper proposes a general framework called SDES based on many-objective evolutionary search to scale up MOSGs to large-scale targets and heterogeneous attackers. SDES consists of four consecutive key components, i.e., discretization, optimization, evaluation, and refinement. Specifically, SDES first discretizes the originally high-dimensional continuous solution space to the low-dimensional discrete one by the maximal indifference property in game theory. This property helps evolutionary algorithms (EAs) bypass the high-dimensional step function and ensure a well-convergent Pareto front. Then, a many-objective EA is used for optimization in the low-dimensional discrete solution space to obtain a well-spaced Pareto front. To evaluate solutions, SDES restores solutions back to the original space via greedily optimizing a novel divergence measurement. Finally, the refinement in SDES boosts the optimization performance with acceptable cost. Theoretically, we prove the optimization consistency and convergence of SDES. Experiment results show that SDES is the first linear-time MOSG algorithm for both large-scale attackers and targets. SDES is able to solve up to 20 attackers and 100 targets MOSG problems, while the state-of-the-art (SOTA) methods can only solve up to 8 attackers and 25 targets ones. Ablation study verifies the necessity of all components in SDES.

Pavement Distress Recognition (PDR) is an important step in pavement inspection and can be powered by image-based automation to expedite the process and reduce labor costs. Pavement images are often in high-resolution with a low ratio of distressed to non-distressed areas. Advanced approaches leverage these properties via dividing images into patches and explore discriminative features in the scale space. However, these approaches usually suffer from information loss during image resizing and low efficiency due to complex learning frameworks. In this paper, we propose a novel and efficient method for PDR. A light network named the Kernel Inversed Pyramidal Resizing Network (KIPRN) is introduced for image resizing, and can be flexibly plugged into the image classification network as a pre-network to exploit resolution and scale information. In KIPRN, pyramidal convolution and kernel inversed convolution are specifically designed to mine discriminative information across different feature granularities and scales. The mined information is passed along to the resized images to yield an informative image pyramid to assist the image classification network for PDR. We applied our method to three well-known Convolutional Neural Networks (CNNs), and conducted an evaluation on a large-scale pavement image dataset named CQU-BPDD. Extensive results demonstrate that KIPRN can generally improve the pavement distress recognition of these CNN models and show that the simple combination of KIPRN and EfficientNet-B3 significantly outperforms the state-of-the-art patch-based method in both performance and efficiency., Comment: Accepted By ICONIP as Oral paper

Wormhole propagation plays a very important role in the product enhancement of oil and gas reservoir. A new combined hybrid mixed finite element method is proposed to solve incompressible wormhole propagation problem with discontinuous Galerkin finite element procedure, in which, the new hybrid mixed finite element algorithm is established for pressure equation, while the discontinuous Galerkin finite element method is considered for concentration equation, and then the porosity function is computed straightly by the approximate value of the concentration. This new combined method can keep local mass balance, meantime it also keeps the boundedness of the porosity. The convergence of the proposed method is analyzed and the optimal error estimate is derived. Finally, numerical examples are presented to verify the validity of the algorithm and the correctness of the theoretical results.

A new hybrid mixed discontinuous Galerkin finite element (HMDGFE) method is constructed for incompressible miscible displacement problem. In this method, the hybrid mixed finite element (HMFE) procedure is considered to solve pressure and velocity equations, and a new hybrid mixed discontinuous Galerkin procedure is constructed to solve the concentration equation with upwind technique. Compared with other traditional discontinuous Galerkin methods, the new method can reach global systems with less unknowns and sparser stencils. The consistency and conservation of the method are analyzed, the stability and optimal error estimates are also derived by the new technique.

This paper addresses policy learning in non-stationary environments and games with continuous actions. Rather than the classical reward maximization mechanism, inspired by the ideas of follow-the-regularized-leader (FTRL) and mirror descent (MD) update, we propose a no-regret style reinforcement learning algorithm PORL for continuous action tasks. We prove that PORL has a last-iterate convergence guarantee, which is important for adversarial and cooperative games. Empirical studies show that, in stationary environments such as MuJoCo locomotion controlling tasks, PORL performs equally well as, if not better than, the soft actor-critic (SAC) algorithm; in non-stationary environments including dynamical environments, adversarial training, and competitive games, PORL is superior to SAC in both a better final policy performance and a more stable training process.

A good model for action-effect prediction, named environment model, is important to achieve sample-efficient decision-making policy learning in many domains like robot control, recommender systems, and patients' treatment selection. We can take unlimited trials with such a model to identify the appropriate actions so that the costs of queries in the real world can be saved. It requires the model to handle unseen data correctly, also called counterfactual data. However, standard data fitting techniques do not automatically achieve such generalization ability and commonly result in unreliable models. In this work, we introduce counterfactual-query risk minimization (CQRM) in model learning for generalizing to a counterfactual dataset queried by a specific target policy. Since the target policies can be various and unknown in policy learning, we propose an adversarial CQRM objective in which the model learns on counterfactual data queried by adversarial policies, and finally derive a tractable solution GALILEO. We also discover that adversarial CQRM is closely related to the adversarial model learning, explaining the effectiveness of the latter. We apply GALILEO in synthetic tasks and a real-world application. The results show that GALILEO makes accurate predictions on counterfactual data and thus significantly improves policies in real-world testing.

Recovering reward function from expert demonstrations is a fundamental problem in reinforcement learning. The recovered reward function captures the motivation of the expert. Agents can imitate experts by following these reward functions in their environment, which is known as apprentice learning. However, the agents may face environments different from the demonstrations, and therefore, desire transferable reward functions. Classical reward learning methods such as inverse reinforcement learning (IRL) or, equivalently, adversarial imitation learning (AIL), recover reward functions coupled with training dynamics, which are hard to be transferable. Previous dynamics-agnostic reward learning methods rely on assumptions such as that the reward function has to be state-only, restricting their applicability. In this work, we present a dynamics-agnostic discriminator-ensemble reward learning method (DARL) within the AIL framework, capable of learning both state-action and state-only reward functions. DARL achieves this by decoupling the reward function from training dynamics, employing a dynamics-agnostic discriminator on a latent space derived from the original state-action space. This latent space is optimized to minimize information on the dynamics. We moreover discover the policy-dependency issue of the AIL framework that reduces the transferability. DARL represents the reward function as an ensemble of discriminators during training to eliminate policy dependencies. Empirical studies on MuJoCo tasks with changed dynamics show that DARL better recovers the reward function and results in better imitation performance in transferred environments, handling both state-only and state-action reward scenarios.

Fictitious play with reinforcement learning is a general and effective framework for zero-sum games. However, using the current deep neural network models, the implementation of fictitious play faces crucial challenges. Neural network model training employs gradient descent approaches to update all connection weights, and thus is easy to forget the old opponents after training to beat the new opponents. Existing approaches often maintain a pool of historical policy models to avoid the forgetting. However, learning to beat a pool in stochastic games, i.e., a wide distribution over policy models, is either sample-consuming or insufficient to exploit all models with limited amount of samples. In this paper, we propose a learning process with neural fictitious play to alleviate the above issues. We train a single model as our policy model, which consists of sub-models and a selector. Everytime facing a new opponent, the model is expanded by adding a new sub-model, where only the new sub-model is updated instead of the whole model. At the same time, the selector is also updated to mix up the new sub-model with the previous ones at the state-level, so that the model is maintained as a behavior strategy instead of a wide distribution over policy models. Experiments on Kuhn poker, a grid-world Treasure Hunting game, and Mini-RTS environments show that the proposed approach alleviates the forgetting problem, and consequently improves the learning efficiency and the robustness of neural fictitious play.

We compare contributions from the triangle diagram and the $D\bar D^*$ bubble chain to the processes of $e^{+}e^{-}\rightarrow J/\psi\pi^{+}\pi^{-}$ and $e^{+}e^{-}\rightarrow (D\bar D^*)^\mp\pi^{\pm}$. By fitting the $J/\psi\pi$ maximum spectrum and the $D\bar D^*$ spectrum, we find that the triangle diagram cannot explain the new data from BESIII Collaboration at center of mass at 4.23GeV and 4.26GeV, simultaneously. On the contrary, the molecular assignment of $Z_c(3900)$ gives a much better description.

We present a complete study of the associated production of the $\eta_c$ meson with light hadrons in $e^+e^-$ collisions at the B-factory energy, which is demonstrated to be one of the best laboratories for testing the colour-octet (CO) mechanism. The colour-siglet contributions are evaluated up to $O(\alpha^2\alpha_s^3)$ while the CO ones are evaluated up to $O(\alpha^2\alpha_s^2)$. For the first time, the angular distribution of the $^1S_0^{[8]}$ production is studied at QCD next-to-leading order. We find that the $^1S_0^{[8]}$ channel dominate the total cross section, while the $^1P_1^{[8]}$ one exhibit its importance in the angular distribution, which turns out to be downward going with respect to $\mathrm{cos}\theta$. This can be considered as the most distinct signal for the CO mechanism.

A comprehensive study on the nature of the $Z_c(3900)$ resonant structure is carried out in this work. By constructing the pertinent effective Lagrangians and considering the important final-state-interaction effects, we first give a unified description to all the relevant experimental data available, including the $J/\psi\pi$ and $\pi\pi$ invariant mass distributions from the $e^+e^-\to J/\psi\pi\pi$ process, the $h_c\pi$ distribution from $e^+e^-\to h_c\pi\pi$ and also the $D\bar D^{*}$ spectrum in the $e^+e^-\to D\bar D^{*}\pi$ process. After fitting the unknown parameters to the previous data, we search the pole in the complex energy plane and find only one pole in the nearby energy region in different Riemann sheets. Therefore we conclude that $Z_c(3900)$ is of $D\bar D^*$ molecular nature, according to the pole counting rule method~[Nucl.~Phys.~A543, 632 (1992); Phys.~Rev.~D 35,~1633 (1987)]. We emphasize that the conclusion based upon the pole counting method is not trivial, since both the $D\bar D^{*}$ contact interactions and the explicit $Z_c$ exchanges are introduced in our analyses and they lead to the same conclusion., Comment: 21 pages, 9 figures. To match the published version in PRD. Additional discussion on the spectral density function is included