Search

The core issue of learning-to-rank (LTR) for document retrieval lies in finding an optimal ranking policy to meet the search intent of the user. The majority of proposed LTR approaches treat the ranking as a static process, employing a fixed ranking policy to immediately assign scores to documents. By contrast, ranking is not a static but an interactive process where the user continues interacting with the document retrieval system through information exchange such as search intent (e.g., rating or clicking for the retrieved items). We model the interactive ranking process (IRP), and propose an Attention-Based Interactive LTR model (AIRank) to constitute an intent-aware flexible ranking policy to gratify the user’s need. To enhance the ranking quality, the inherent relations among documents are procured by the self-attention method to contribute to an enriched user intent representation. Furthermore, we mend the policy gradient learning method to train the AIRank in the IRP. Experiments demonstrate the effectiveness of AIRank compared to the state-of-the-art methods in terms of normalized discounted cumulative gain and expected reciprocal rank. [ABSTRACT FROM AUTHOR]

Originating from model-free reinforcement learning (RL), many modern multi-agent reinforcement learning (MARL) algorithms are usually armed with the paradigm of Centralized Training with Decentralized Execution (CTDE) to mitigate the non-stationary problem and make the training process stable. However, these methods still suffer from sample inefficiency and slow training convergence as in the single-agent reinforcement learning setting. Many common methods aiming to tackle these problems utilize the experience buffer and parallel training mechanism to speed up the training process, which would cost more computing resources and may still underuse the sampled experiences. In this paper, we propose R egularization - A dapted A nderson A cceleration (RA3) for model-free, off-policy MARL algorithms. Under the CTDE paradigm, this specific RA3 approach treats the joint action-value function update as a fixed-point iteration task and speeds up the training process with the same amount of sampled experiences as in the baseline algorithms. Furthermore, our RA3 employs an adaptive regularization strategy related to Bellman residuals to stabilize the update process and enhance the training performance. Experimental results demonstrate that the improved learning speed and superior performance of our proposed method are significantly improved on the predator–prey game and the challenging StarCraft II micromanagement benchmark tasks. [ABSTRACT FROM AUTHOR]

Existing multi-agent reinforcement learning methods employ a paradigm of centralized training with decentralized execution (CTDE) to learn cooperative policy among agents via coordination. However, within continuous destruction conditions, the inclusion of information from dead agents significantly undermines the ability to effectively learn cooperative policies in multi-agent systems. In this paper, we first analyze the bias introduced by dead agents under the CTDE paradigm and how it affects cooperation among agents. Following this, we propose q-learning based downsizing adaptive policy (QDAP) framework for cooperative multi-agent reinforcement learning. QDAP actively discerns relevant values from dead agents and utilizes an innovative approach to convert historical trajectories into weighting factors, thereby aiding remaining active agents in learning more appropriate cooperative policies. Moreover, we extend our proposed framework into the CTDE paradigm, facilitating seamless adaptation with the methods of value decomposition. Experimental results demonstrate that QDAP significantly improves learning speed and achieves superior cooperation performance on challenging Starcraft II micromanagement benchmark tasks. [ABSTRACT FROM AUTHOR]