Search

Multiple-Choice Questions (MCQs) constitute a critical area of research in the study of Large Language Models (LLMs). Previous works have investigated the selection bias problem in MCQs within few-shot scenarios, in which the LLM's performance may be influenced by the presentation of answer choices, leaving the selection bias during Supervised Fine-Tuning (SFT) unexplored. In this paper, we reveal that selection bias persists in the SFT phase , primarily due to the LLM's inadequate Multiple Choice Symbol Binding (MCSB) ability. This limitation implies that the model struggles to associate the answer options with their corresponding symbols (e.g., A/B/C/D) effectively. To enhance the model's MCSB capability, we first incorporate option contents into the loss function and subsequently adjust the weights of the option symbols and contents, guiding the model to understand the option content of the current symbol. Based on this, we introduce an efficient SFT algorithm for MCQs, termed Point-wise Intelligent Feedback (PIF). PIF constructs negative instances by randomly combining the incorrect option contents with all candidate symbols, and proposes a point-wise loss to provide feedback on these negative samples into LLMs. Our experimental results demonstrate that PIF significantly reduces the model's selection bias by improving its MCSB capability. Remarkably, PIF exhibits a substantial enhancement in the accuracy for MCQs., Comment: Accept at ACL2024 Main

Recent strides in large language models (LLMs) have yielded remarkable performance, leveraging reinforcement learning from human feedback (RLHF) to significantly enhance generation and alignment capabilities. However, RLHF encounters numerous challenges, including the objective mismatch issue, leading to suboptimal performance in Natural Language Understanding (NLU) tasks. To address this limitation, we propose a novel Reinforcement Learning framework enhanced with Label-sensitive Reward (RLLR) to amplify the performance of LLMs in NLU tasks. By incorporating label-sensitive pairs into reinforcement learning, our method aims to adeptly capture nuanced label-sensitive semantic features during RL, thereby enhancing natural language understanding. Experiments conducted on five diverse foundation models across eight tasks showcase promising results. In comparison to Supervised Fine-tuning models (SFT), RLLR demonstrates an average performance improvement of 1.54%. Compared with RLHF models, the improvement averages at 0.69%. These results reveal the effectiveness of our method for LLMs in NLU tasks. Code and data available at: https://github.com/MagiaSN/ACL2024_RLLR., Comment: Accept at ACL2024 Main

Recently, the success of pre-training in text domain has been fully extended to vision, audio, and cross-modal scenarios. The proposed pre-training models of different modalities are showing a rising trend of homogeneity in their model structures, which brings the opportunity to implement different pre-training models within a uniform framework. In this paper, we present TencentPretrain, a toolkit supporting pre-training models of different modalities. The core feature of TencentPretrain is the modular design. The toolkit uniformly divides pre-training models into 5 components: embedding, encoder, target embedding, decoder, and target. As almost all of common modules are provided in each component, users can choose the desired modules from different components to build a complete pre-training model. The modular design enables users to efficiently reproduce existing pre-training models or build brand-new one. We test the toolkit on text, vision, and audio benchmarks and show that it can match the performance of the original implementations.

We study a model of two-player bargaining game in the shadow of a preventive trade war that examines why states deliberately maintain trade barriers in the age of globalization. Globalization can induce substantial power shifts between states, which makes the threat of a preventive trade war salient. In this situation, there may exist "healthy" levels of trade barriers that dampen the war incentives by reducing states' expected payoffs from such a war. Thus, we demonstrate that trade barriers can sometimes serve as breaks and cushions necessary to sustain inefficient yet peaceful economic cooperation between states. We assess the theoretical implications by examining the US-China trade relations since 1972.

Precise information is essential for making good policies, especially those regarding reform decisions. However, decision-makers may hesitate to gather such information if certain decisions could have negative impacts on their future careers. We model how decision-makers with career concerns may acquire policy-relevant information and carry out reform decisions when their policy discretion can be limited ex ante. Typically, decision-makers with career concerns have weaker incentives to acquire information compared to decision-makers without such concerns. In this context, we demonstrate that the public can encourage information acquisition by eliminating either the "moderate policy" or the status quo from decision-makers' discretion. We also analyze when reform decisions should be strategically delegated to decision-makers with or without career concerns.

I study how political bias and audience costs impose domestic institutional constraints that affect states' capacity to reach peaceful agreements during crises. With a mechanism design approach, I show that the existence of peaceful agreements hinges crucially on whether the resource being divided can appease two sides of the highest type (i.e. the maximum war capacity). The derivation has two major implications. On the one hand, if war must be averted, then political leaders are not incentivized by audience costs to communicate private information; they will pool on the strategy that induces the maximum bargaining gains. On the other hand, political bias matters for the scope of peace because it alters a state's expected war payoff.

In industrial applications like online advertising and recommendation systems, diverse and accurate user profiles can greatly help improve personalization. Deep learning is widely applied to mine expressive tags to users from their historical interactions in the system, e.g., click, conversion action in the advertising chain. The usual approach is to take a certain action as the objective, and introduce multiple independent Two-Tower models to predict the possibility of users' action on tags (known as CTR or CVR prediction). The predicted users' high probably attractive tags are to represent their preferences. However, the single-action models cannot learn complementarily and support effective training on data-sparse actions. Besides, limited by the lack of information fusion between the two towers, the model learns insufficiently to represent users' preferences on various tag \textbf{topics} well. This paper introduces a novel multi-task model called Mixture of Virtual-Kernel Experts (MVKE) to learn user preferences on various actions and topics unitedly. In MVKE, we propose a concept of Virtual-Kernel Expert, which focuses on modeling one particular facet of the user's preferences, and all of them learn coordinately. Besides, the gate-based structure used in MVKE builds an information fusion bridge between two towers, improving the model's capability and maintaining high efficiency. We apply the model in Tencent Advertising System, where both online and offline evaluations show that our method has a significant improvement compared with the existing ones and brings about an obvious lift to actual advertising revenue., Comment: 11 pages, accepted by SIGKDD'2022

We build a formal model that examines how different policymaking environments shape career-concerned officials' reform decisions and implementation. When career concerns are strong, officials will inefficiently initiate reforms to signal to the central government that they are congruent. To improve the quality of reform policymaking, the central government must hold officials accountable to policy outcomes. We demonstrate that the central government can exercise this accountability by requiring officials to publicize policy outcomes while maintain secrecy on implementation details. In this situation, officials can signal their congruence only through a desirable policy outcome, so they are highly motivated to carry out a reform well. We also demonstrate that the accountability on policy outcomes is infeasible under alternative policymaking environments. We apply the results to China's recent practice in decentralized reform policymaking.

Keyphrases are capable of providing semantic metadata characterizing documents and producing an overview of the content of a document. Since keyphrase extraction is able to facilitate the management, categorization, and retrieval of information, it has received much attention in recent years. There are three approaches to address keyphrase extraction: (i) traditional two-step ranking method, (ii) sequence labeling and (iii) generation using neural networks. Two-step ranking approach is based on feature engineering, which is labor intensive and domain dependent. Sequence labeling is not able to tackle overlapping phrases. Generation methods (i.e., Sequence-to-sequence neural network models) overcome those shortcomings, so they have been widely studied and gain state-of-the-art performance. However, generation methods can not utilize context information effectively. In this paper, we propose a novelty Span Keyphrase Extraction model that extracts span-based feature representation of keyphrase directly from all the content tokens. In this way, our model obtains representation for each keyphrase and further learns to capture the interaction between keyphrases in one document to get better ranking results. In addition, with the help of tokens, our model is able to extract overlapped keyphrases. Experimental results on the benchmark datasets show that our proposed model outperforms the existing methods by a large margin.