Search

Large Language Models (LLMs) have recently revolutionized the NLP field, while they still fall short in some specific down-stream tasks. In the work, we focus on utilizing LLMs to perform machine translation, where we observe that two patterns of errors frequently occur and drastically affect the translation quality: language mismatch and repetition. The work sets out to explore the potential for mitigating these two issues by leveraging model editing methods, e.g., by locating Feed-Forward Network (FFN) neurons or something that are responsible for the errors and deactivating them in the inference time. We find that directly applying such methods either limited effect on the targeted errors or has significant negative side-effect on the general translation quality, indicating that the located components may also be crucial for ensuring machine translation with LLMs on the rails. To this end, we propose to refine the located components by fetching the intersection of the locating results under different language settings, filtering out the aforementioned information that is irrelevant to targeted errors. The experiment results empirically demonstrate that our methods can effectively reduce the language mismatch and repetition ratios and meanwhile enhance or keep the general translation quality in most cases., Comment: 20 pages, EMNLP'2024 Main Conference

Quantum purity amplification (QPA) offers a novel approach to counteracting the pervasive noise that degrades quantum states. We present the optimal QPA protocol for general quantum systems against global depolarizing noise, which has remained unknown for two decades. We construct and prove the optimality of our protocol, which demonstrates improved fidelity scaling compared to the best-known methods. We explore the operational interpretation of the protocol and provide simple examples of how to compile it into efficient circuits for near-term experiments. Furthermore, we conduct numerical simulations to investigate the effectiveness of our protocol in the quantum simulation of Hamiltonian evolution, demonstrating its ability to enhance fidelity even under circuit-level noise. Our findings suggest that QPA could improve the performance of quantum information processing tasks, particularly in the context of Noisy Intermediate-Scale Quantum (NISQ) devices, where reducing the effect of noise with limited resources is critical., Comment: 7+15 pages, 4+5 figures, 0+2 tables. Comments are welcome!

Fine-tuning large language models (LLMs) can cause them to lose their general capabilities. However, the intrinsic mechanisms behind such forgetting remain unexplored. In this paper, we begin by examining this phenomenon by focusing on knowledge understanding and instruction following, with the latter identified as the main contributor to forgetting during fine-tuning. Consequently, we propose the Instruction Vector (IV) framework to capture model representations highly related to specific instruction-following capabilities, thereby making it possible to understand model-intrinsic forgetting. Through the analysis of IV dynamics pre and post-training, we suggest that fine-tuning mostly adds specialized reasoning patterns instead of erasing previous skills, which may appear as forgetting. Building on this insight, we develop IV-guided training, which aims to preserve original computation graph, thereby mitigating catastrophic forgetting. Empirical tests on three benchmarks confirm the efficacy of this new approach, supporting the relationship between IVs and forgetting. Our code will be made available soon.

Compositional generalization, representing the model's ability to generate text with new attribute combinations obtained by recombining single attributes from the training data, is a crucial property for multi-aspect controllable text generation (MCTG) methods. Nonetheless, a comprehensive compositional generalization evaluation benchmark of MCTG is still lacking. We propose CompMCTG, a benchmark encompassing diverse multi-aspect labeled datasets and a crafted three-dimensional evaluation protocol, to holistically evaluate the compositional generalization of MCTG approaches. We observe that existing MCTG works generally confront a noticeable performance drop in compositional testing. To mitigate this issue, we introduce Meta-MCTG, a training framework incorporating meta-learning, where we enable models to learn how to generalize by simulating compositional generalization scenarios in the training phase. We demonstrate the effectiveness of Meta-MCTG through achieving obvious improvement (by at most 3.64%) for compositional testing performance in 94.4% cases., Comment: Accepted to ACL 2024 (Main); 32 pages

LLMs have marked a revolutonary shift, yet they falter when faced with compositional reasoning tasks. Our research embarks on a quest to uncover the root causes of compositional reasoning failures of LLMs, uncovering that most of them stem from the improperly generated or leveraged implicit reasoning results. Inspired by our empirical findings, we resort to Logit Lens and an intervention experiment to dissect the inner hidden states of LLMs. This deep dive reveals that implicit reasoning results indeed surface within middle layers and play a causative role in shaping the final explicit reasoning results. Our exploration further locates multi-head self-attention (MHSA) modules within these layers, which emerge as the linchpins in accurate generation and leveraing of implicit reasoning results. Grounded on the above findings, we develop CREME, a lightweight method to patch errors in compositional reasoning via editing the located MHSA modules. Our empirical evidence stands testament to CREME's effectiveness, paving the way for autonomously and continuously enhancing compositional reasoning capabilities in language models., Comment: Accepted by ACL'2024 Findings

This paper considers the out-of-distribution (OOD) generalization problem under the setting that both style distribution shift and spurious features exist and domain labels are missing. This setting frequently arises in real-world applications and is underlooked because previous approaches mainly handle either of these two factors. The critical challenge is decoupling style and spurious features in the absence of domain labels. To address this challenge, we first propose a structural causal model (SCM) for the image generation process, which captures both style distribution shift and spurious features. The proposed SCM enables us to design a new framework called IRSS, which can gradually separate style distribution and spurious features from images by introducing adversarial neural networks and multi-environment optimization, thus achieving OOD generalization. Moreover, it does not require additional supervision (e.g., domain labels) other than the images and their corresponding labels. Experiments on benchmark datasets demonstrate that IRSS outperforms traditional OOD methods and solves the problem of Invariant risk minimization (IRM) degradation, enabling the extraction of invariant features under distribution shift., Comment: 10 pages, 12 figures

Metasurfaces, optics made from subwavelength-scale nanostructures, have been limited to millimeter-sizes by the scaling challenge of producing vast numbers of precisely engineered elements over a large area. In this study, we demonstrate an all-glass 100 mm diameter metasurface lens (metalens) comprising 18.7 billion nanostructures that operates in the visible spectrum with a fast f-number (f/1.5, NA=0.32) using deep-ultraviolet (DUV) projection lithography. Our work overcomes the exposure area constraints of lithography tools and demonstrates that large metasurfaces are commercially feasible. Additionally, we investigate the impact of various fabrication errors on the imaging quality of the metalens, several of which are unique to such large area metasurfaces. We demonstrate direct astronomical imaging of the Sun, the Moon, and emission nebulae at visible wavelengths and validate the robustness of such metasurfaces under extreme environmental thermal swings for space applications.

In recent years, Deep Reinforcement Learning (DRL) has emerged as a promising method for robot collision avoidance. However, such DRL models often come with limitations, such as adapting effectively to structured environments containing various pedestrians. In order to solve this difficulty, previous research has attempted a few approaches, including training an end-to-end solution by integrating a waypoint planner with DRL and developing a multimodal solution to mitigate the drawbacks of the DRL model. However, these approaches have encountered several issues, including slow training times, scalability challenges, and poor coordination among different models. To address these challenges, this paper introduces a novel approach called evolutionary curriculum training to tackle these challenges. The primary goal of evolutionary curriculum training is to evaluate the collision avoidance model's competency in various scenarios and create curricula to enhance its insufficient skills. The paper introduces an innovative evaluation technique to assess the DRL model's performance in navigating structured maps and avoiding dynamic obstacles. Additionally, an evolutionary training environment generates all the curriculum to improve the DRL model's inadequate skills tested in the previous evaluation. We benchmark the performance of our model across five structured environments to validate the hypothesis that this evolutionary training environment leads to a higher success rate and a lower average number of collisions. Further details and results at our project website., Comment: Robotics: Science and Systems

Despite the rising prevalence of neural sequence models, recent empirical evidences suggest their deficiency in compositional generalization. One of the current de-facto solutions to this problem is compositional data augmentation, aiming to incur additional compositional inductive bias. Nonetheless, the improvement offered by existing handcrafted augmentation strategies is limited when successful systematic generalization of neural sequence models requires multi-grained compositional bias (i.e., not limited to either lexical or structural biases only) or differentiation of training sequences in an imbalanced difficulty distribution. To address the two challenges, we first propose a novel compositional augmentation strategy dubbed \textbf{Span} \textbf{Sub}stitution (SpanSub) that enables multi-grained composition of substantial substructures in the whole training set. Over and above that, we introduce the \textbf{L}earning \textbf{to} \textbf{S}ubstitute \textbf{S}pan (L2S2) framework which empowers the learning of span substitution probabilities in SpanSub in an end-to-end manner by maximizing the loss of neural sequence models, so as to outweigh those challenging compositions with elusive concepts and novel surroundings. Our empirical results on three standard compositional generalization benchmarks, including SCAN, COGS and GeoQuery (with an improvement of at most 66.5\%, 10.3\%, 1.2\%, respectively), demonstrate the superiority of SpanSub, %the learning framework L2S2 and their combination., Comment: accepted by ACL 2023

We study the far-from-equilibrium dynamics of isolated two-dimensional Heisenberg antiferromagnets. We consider spin spiral initial conditions which imprint a position-dependent staggered-magnetization (or Neel order) in the two-dimensional lattice. Remarkably, we find a long-lived prethermal regime characterized by self-similar behavior of staggered magnetization fluctuations, although the system has no long-range order at finite energy and the staggered magnetization does not couple with conserved charges. Exploiting the separation of length scales introduced by the initial condition, we derive a simplified analytical model that allow us to compute the spatial-temporal scaling exponents and power-law distribution of the staggered magnetization fluctuations analytically, and find excellent agreement with numerical simulations using phase space methods. The scaling exponents are insensitive to details of the initial condition, in particular, no fine-tuning of energy is required to trigger the self-similar scaling regime. Compared with recent results on far-from-equilibrium universality on the Heisenberg ferromagnet, we find quantitatively distinct spatial-temporal scaling exponents, therefore suggesting that the same model with ferromagnetic and antiferromagnetic initial conditions can host different universal regimes. Our predictions are relevant to ultra-cold atoms simulators of Heisenberg magnets and driven antiferromagnetic insulators., Comment: 8+2 pages, 2+1 figures

We investigate a family of fault-tolerant quantum error correction schemes based on the concatenation of small error detection or error correction codes with the three-dimensional cluster state. We propose fault-tolerant state preparation and decoding schemes that effectively convert every circuit-level error into an erasure error, leveraging the cluster state's high threshold against such errors. We find a set of codes for which such a conversion is possible, and study their performance against the standard circuit-level depolarizing model. Our best performing scheme, which is based on a concatenation with a classical code, improves the threshold by $16.5\%$ and decreases the spacetime overhead by $32\%$ compared to the scheme without concatenation, with each scheme subject to a physical error rate of $10^{-3}$ and achieving a logical error rate of $10^{-6}$., Comment: 11+3 pages, 9+12 figures

We prove that the two-step backward differentiation formula (BDF2) method is stable on arbitrary time grids; while the variable-step BDF3 scheme is stable if almost all adjacent step ratios are less than 2.553. These results relax the severe mesh restrictions in the literature and provide a new understanding of variable-step BDF methods. Our main tools include the discrete orthogonal convolution kernels and an elliptic-type matrix norm., Comment: 20 pages, 2 figures, 3 tables

We introduce end-to-end metaoptics inverse design for multi-channel imaging: reconstruction of depth, spectral and polarization channels from a single-shot monochrome image. The proposed technique integrates a single-layer metasurface frontend with an efficient Tikhonov reconstruction backend, without any additional optics except a grayscale sensor. Our method yields multi-channel imaging by spontaneous demultiplexing: the metaoptics front-end separates different channels into distinct spatial domains whose locations on the sensor are optimally discovered by the inverse-design algorithm. We present large-area metasurface designs, compatible with standard lithography, for multi-spectral imaging, depth-spectral imaging, and ``all-in-one'' spectro-polarimetric-depth imaging with robust reconstruction performance ($\lesssim 10\%$ error with 1\% detector noise). In contrast to neural networks, our framework is physically interpretable and does not require large training sets. It can be used to reconstruct arbitrary three-dimensional scenes with full multi-wavelength spectra and polarization textures.

Meta-optics has achieved major breakthroughs in the past decade; however, conventional forward design faces challenges as functionality complexity and device size scale up. Inverse design aims at optimizing meta-optics design but has been currently limited by expensive brute-force numerical solvers to small devices, which are also difficult to realize experimentally. Here, we present a general inverse design framework for aperiodic large-scale complex meta-optics in three dimensions, which alleviates computational cost for both simulation and optimization via a fast-approximate solver and an adjoint method, respectively. Our framework naturally accounts for fabrication constraints via a surrogate model. In experiments, we demonstrate, for the first time, aberration-corrected metalenses working in the visible with high numerical aperture, poly-chromatic focusing, and large diameter up to centimeter scale. Such large-scale meta-optics opens a new paradigm for applications, and we demonstrate its potential for future virtual-reality platforms by using a meta-eyepiece and a laser back-illuminated micro-Liquid Crystal Display.

The phase-matching condition is a key aspect in nonlinear wavelength conversion processes, which requires the momenta of the photons involved in the processes to be conserved. Conventionally, nonlinear phase matching is achieved using either birefringent or periodically poled nonlinear crystals, which requires careful dispersion engineering and are usually narrowband. In recent years, metasurfaces consisting of densely packed arrays of optical antennas have been demonstrated to provide an effective optical momentum to bend light in arbitrary ways. Here, we demonstrate that gradient metasurface structures consisting of phased array antennas are able to circumvent the phase-matching requirement in on-chip nonlinear wavelength conversion. We experimentally demonstrate phase-matching-free second harmonic generation over many coherent lengths in thin film lithium niobate waveguides patterned with the gradient metasurfaces. Efficient second-harmonic generation (1660% W-1cm-2) in the metasurface-based devices was observed over a wide range of pump wavelengths (1580-1650 nm).