Search

Aligning language models (LMs) with human preferences has become a key area of research, enabling these models to meet diverse user needs better. Inspired by weak-to-strong generalization, where a strong LM fine-tuned on labels generated by a weaker model can consistently outperform its weak supervisor, we extend this idea to model alignment. In this work, we observe that the alignment behavior in weaker models can be effectively transferred to stronger models and even exhibit an amplification effect. Based on this insight, we propose a method called Weak-to-Strong Preference Optimization (WSPO), which achieves strong model alignment by learning the distribution differences before and after the alignment of the weak model. Experiments demonstrate that WSPO delivers outstanding performance, improving the win rate of Qwen2-7B-Instruct on Arena-Hard from 39.70 to 49.60, achieving a remarkable 47.04 length-controlled win rate on AlpacaEval 2, and scoring 7.33 on MT-bench. Our results suggest that using the weak model to elicit a strong model with a high alignment ability is feasible.

Topological magnetic materials are expected to show multiple transport responses because of their unusual bulk electronic topology in momentum space and topological spin texture in real space. However, such multiple topological properties-hosting materials are rare in nature. In this work, we reveal the coexistence of a large tunable anomalous Hall effect and topological magnetic skyrmions in a Weyl nodal ring ferromagnet Mn5Ge3, by using electrical transport and Lorentz transmission electronic microscope (TEM) measurements. It was found that the intrinsic anomalous Hall conductivity (AHC) can reach up to 979.7 S/cm with current along [120] and magnetic field along [001] of the Mn5Ge3 single crystals. Our theoretical calculations reveal that the large AHC is closely related with two Weyl nodal rings in band structure near the Fermi level and is strongly modified by the content of Ge. Moreover, our Lorentz-TEM images and micromagnetic simulation results, together with the sizable topological Hall effect clearly point to the robust formation of magnetic skyrmions over a wide temperature-magnetic field region. These results prove Mn5Ge3 as a rare magnetic topological nodal-line semimetal with great significance to explore novel multiple topological phenomena, which facilitates the development of spintronics., Comment: 38 pages, 22 figures

The advent of large language models (LLMs) has significantly advanced various fields, including natural language processing and automated dialogue systems. This paper explores the application of LLMs in psychological counseling, addressing the increasing demand for mental health services. We present a method for instruction tuning LLMs with specialized prompts to enhance their performance in providing empathetic, relevant, and supportive responses. Our approach involves developing a comprehensive dataset of counseling-specific prompts, refining them through feedback from professional counselors, and conducting rigorous evaluations using both automatic metrics and human assessments. The results demonstrate that our instruction-tuned model outperforms several baseline LLMs, highlighting its potential as a scalable and accessible tool for mental health support., Comment: 9 pages

There has been a significant focus on real topological systems that enjoy space-time inversion symmetry (PT ) and lack spin-orbit coupling. While the theoretical classification of the real topology has been established, more progress has yet to be made in the materials realization of such real topological systems in three dimensions (3D). To address this crucial issue, by selecting the carbon-based material candidates as targets, we perform high-throughput computing to inspect the real topology in the phonon spectrums of the 3D carbon allotropes in the Samara Carbon Allotrope Database (SACADA). Among 1192 kinds of 3D carbon allotropes, we find 65 real topological systems with a phononic real Chern insulating (PRCI) state, 2 real topological systems with a phononic real nodal line (PRNL) state, 10 real topological systems with a phononic real Dirac point (PRDP) state, and 8 real topological systems with a phononic real triple-point pair (PRTPP) state. This extremely expands the material candidates with real topology, especially for the gapless topological phonons. We exhibit the PRCI, PRNL, PRTPP, and PRDP states of 27-SG. 166-pcu-h, 1081-SG. 194- 4 2T13-CA, 52-SG. 141-gis, and 132-SG. 191-3,4T157 as illustrative examples, and explore the second-order boundary mode, i.e., phononic hinge mode. Among the four examples, the materials 1081-SG. 194-42T13-CA and 52-SG. 141-gis are so ideal that the PRNL and PRTPP in them are well separated from other bands, and the phononic hinge mode can be clearly observed. This study aims to broaden the understanding of 3D topological phonons, and emphasizes the potential of 3D carbon allotropes as a valuable framework for exploring the fascinating physics related to phononic hinge modes and phononic real topology.

While Large Language Models (LLMs) have shown remarkable abilities, they are hindered by significant resource consumption and considerable latency due to autoregressive processing. In this study, we introduce Adaptive N-gram Parallel Decoding (ANPD), an innovative and lossless approach that accelerates inference by allowing the simultaneous generation of multiple tokens. ANPD incorporates a two-stage approach: it begins with a rapid drafting phase that employs an N-gram module, which adapts based on the current interactive context, followed by a verification phase, during which the original LLM assesses and confirms the proposed tokens. Consequently, ANPD preserves the integrity of the LLM's original output while enhancing processing speed. We further leverage a multi-level architecture for the N-gram module to enhance the precision of the initial draft, consequently reducing inference latency. ANPD eliminates the need for retraining or extra GPU memory, making it an efficient and plug-and-play enhancement. In our experiments, models such as LLaMA and its fine-tuned variants have shown speed improvements up to 3.67x, validating the effectiveness of our proposed ANPD.

The orbital Hall effect in light materials has attracted considerable attention for developing novel orbitronic devices. Here we investigate the orbital torque efficiency and demonstrate the switching of the perpendicularly magnetized materials through the orbital Hall material (OHM), i.e., Zirconium (Zr). The orbital torque efficiency of approximately 0.78 is achieved in the Zr OHM with the perpendicularly magnetized [Co/Pt]3 sample, which significantly surpasses that of the perpendicularly magnetized CoFeB/Gd/CoFeB sample (approximately 0.04). Such notable difference is attributed to the different spin-orbit correlation strength between the [Co/Pt]3 sample and the CoFeB/Gd/CoFeB sample, which has been confirmed through the theoretical calculations. Furthermore, the full magnetization switching of the [Co/Pt]3 sample with a switching current density of approximately 2.6x106 A/cm2 has been realized through Zr, which even outperforms that of the W spin Hall material. Our finding provides a guideline to understand orbital torque efficiency and paves the way to develop energy-efficient orbitronic devices., Comment: 21 pages, 4 figures, submitted

Current language models decode text token by token according to probabilistic distribution, and determining the appropriate candidates for the next token is crucial to ensure generation quality. This study introduces adaptive decoding, a mechanism that dynamically empowers language models to ascertain a sensible candidate set during generation. Specifically, we introduce an entropy-based metric called confidence and conceptualize determining the optimal candidate set as a confidence-increasing process. The rationality of including a token in the candidate set is assessed by leveraging the increment of confidence. Experimental results reveal that our method balances diversity and coherence well. The human evaluation shows that our method can generate human-preferred text. Additionally, our method can potentially improve the reasoning ability of language models., Comment: ICML2024

The recent discoveries of surperisingly large anomalous Hall effect in chiral antiderromagnets have triggered extensive research efforts in various fields, ranging from topological condensed-matter physics to antiferromagnetic spintronics, and energy harvesting technology. However, such AHE-hosting antiferromagnetic materials are rare in nature. Herein, we demonstrate that Mn2.4Ga, a Fermi-level-tuned kagome antiferromagnet, has a large anomalous Hall conductivity of about 150 {\Omega}-1cm-1 at room temperature that surpasses the usual high values (i.e.,20-50 {\Omega}-1cm-1) observed so far in two outstanding kagome antiferromagnets, Mn3Sn and Mn3Ge. The spin triangular structure of Mn2.4Ga guarantees a nonzero Berry curvature while generates only a weak net moment in the kagome plane.Moreover, the anomalous Hall conductivity exhibits a sign reversal with the rotation of a small magnetic field, which can be ascribed to the field-controlled chirality of the spin triangular structure. Our theoretical calculation indicate that the large AHE in Mn2.4Ga originates from a significantly enhanced Berry curvature associated wiht the tuning of the Fermi level close to the Weyl points. These properties, together with the ability to manipulate moment orientations using a moderate external magnetic field, make Mn2.4Ga extremely exciting for future antiferromagnetic spintronics., Comment: This article has been accepted by the Advanced Functional Materials

Representing the Neural Radiance Field (NeRF) with the explicit voxel grid (EVG) is a promising direction for improving NeRFs. However, the EVG representation is not efficient for storage and transmission because of the terrific memory cost. Current methods for compressing EVG mainly inherit the methods designed for neural network compression, such as pruning and quantization, which do not take full advantage of the spatial correlation of voxels. Inspired by prosperous digital image compression techniques, this paper proposes SPC-NeRF, a novel framework applying spatial predictive coding in EVG compression. The proposed framework can remove spatial redundancy efficiently for better compression performance.Moreover, we model the bitrate and design a novel form of the loss function, where we can jointly optimize compression ratio and distortion to achieve higher coding efficiency. Extensive experiments demonstrate that our method can achieve 32% bit saving compared to the state-of-the-art method VQRF on multiple representative test datasets, with comparable training time.

In this study, we investigate the reliability of Large Language Models (LLMs) in professing human-like personality traits through responses to personality questionnaires. Our goal is to evaluate the consistency between LLMs' professed personality inclinations and their actual "behavior", examining the extent to which these models can emulate human-like personality patterns. Through a comprehensive analysis of LLM outputs against established human benchmarks, we seek to understand the cognition-action divergence in LLMs and propose hypotheses for the observed results based on psychological theories and metrics.

We are currently in an era of fierce competition among various large language models (LLMs) continuously pushing the boundaries of benchmark performance. However, genuinely assessing the capabilities of these LLMs has become a challenging and critical issue due to potential data contamination, and it wastes dozens of time and effort for researchers and engineers to download and try those contaminated models. To save our precious time, we propose a novel and useful method, Clean-Eval, which mitigates the issue of data contamination and evaluates the LLMs in a cleaner manner. Clean-Eval employs an LLM to paraphrase and back-translate the contaminated data into a candidate set, generating expressions with the same meaning but in different surface forms. A semantic detector is then used to filter the generated low-quality samples to narrow down this candidate set. The best candidate is finally selected from this set based on the BLEURT score. According to human assessment, this best candidate is semantically similar to the original contamination data but expressed differently. All candidates can form a new benchmark to evaluate the model. Our experiments illustrate that Clean-Eval substantially restores the actual evaluation results on contaminated LLMs under both few-shot learning and fine-tuning scenarios., Comment: NAACL2024(findings)

Facial expression recognition (FER) in the wild is a challenging task affected by the image quality and has attracted broad interest in computer vision. There is no research using feature fusion and ensemble strategy for FER simultaneously. Different from previous studies, this paper applies both internal feature fusion for a single model and feature fusion among multiple networks, as well as the ensemble strategy. This paper proposes one novel single model named R18+FAML, as well as one ensemble model named R18+FAML-FGA-T2V to improve the performance of the FER in the wild. Based on the structure of ResNet18 (R18), R18+FAML combines internal Feature fusion and three Attention blocks using Multiple Loss functions (FAML) to improve the diversity of the feature extraction. To improve the performance of R18+FAML, we propose a Feature fusion among networks based on the Genetic Algorithm (FGA), which can fuse the convolution kernels for feature extraction of multiple networks. On the basis of R18+FAML and FGA, we propose one ensemble strategy, i.e., the Top Two Voting (T2V) to support the classification of FER, which can consider more classification information comprehensively. Combining the above strategies, R18+FAML-FGA-T2V can focus on the main expression-aware areas. Extensive experiments demonstrate that our single model R18+FAML and the ensemble model R18+FAML-FGA-T2V achieve the accuracies of $\left( 90.32, 62.17, 65.83 \right)\%$ and $\left( 91.59, 63.27, 66.63 \right)\%$ on three challenging unbalanced FER datasets RAF-DB, AffectNet-8 and AffectNet-7 respectively, both outperforming the state-of-the-art results., Comment: 12 pages, 8 figures

The decoding algorithm is critical for open-ended text generation, transforming latent representations into coherent and meaningful outputs. This paper investigates the self-reinforcement effect in text generation and the effectiveness of a repetition penalty to mitigate it. However, determining the optimal repetition penalty value is challenging. To tackle this, we propose a forgetting mechanism that disregards distant tokens, reducing the burden of penalty selection. In addition, we introduce a length penalty to address overly short sentences caused by excessive penalties. Our penalty decoding approach incorporating three strategies helps resolve issues with sampling methods deviating from factual information. Experimental results demonstrate the efficacy of our approach in generating high-quality sentences resembling human output., Comment: Accepted by EMNLP2023