Search

Black-box finetuning is an emerging interface for adapting state-of-the-art language models to user needs. However, such access may also let malicious actors undermine model safety. To demonstrate the challenge of defending finetuning interfaces, we introduce covert malicious finetuning, a method to compromise model safety via finetuning while evading detection. Our method constructs a malicious dataset where every individual datapoint appears innocuous, but finetuning on the dataset teaches the model to respond to encoded harmful requests with encoded harmful responses. Applied to GPT-4, our method produces a finetuned model that acts on harmful instructions 99% of the time and avoids detection by defense mechanisms such as dataset inspection, safety evaluations, and input/output classifiers. Our findings question whether black-box finetuning access can be secured against sophisticated adversaries., Comment: 22 pages

With a view toward addressing the explosive growth in the computational demands of nuclear structure and reactions modeling, we develop a novel quantum algorithm for neutron-nucleus simulations with general potentials, which provides acceptable bound-state energies even in the presence of noise, through the noise-resilient training method. In particular, the algorithm can now solve for any band-diagonal to full Hamiltonian matrices, as needed to accommodate a general central potential. This includes exponential Gaussian-like potentials and ab initio inter-cluster potentials (optical potentials). The approach can also accommodate the complete form of the chiral effective-field-theory nucleon-nucleon potentials used in ab initio nuclear calculations. We make this potential available for three different qubit encodings, including the one-hot (OHE), binary (BE), and Gray encodings (GE), and we provide a comprehensive analysis of the number of Pauli terms and commuting sets involved. We find that the GE allows for an efficient scaling of the model-space size $N$ (or number of basis states used) and is more resource efficient not only for tridiagonal Hamiltonians, but also for band-diagonal Hamiltonians having bandwidth up to $N$. We introduce a new commutativity scheme called distance-grouped commutativity (DGC) and compare its performance with the well-known qubit-commutativity (QC) scheme. We lay out the explicit grouping of Pauli strings and the diagonalizing unitary under the DGC scheme, and we find that it outperforms the QC scheme, at the cost of a more complex diagonalizing unitary. Lastly, we provide first solutions of the neutron-alpha dynamics from quantum simulations suitable for NISQ processors, using an optical potential rooted in first principles, and a study of the bound-state physics in neutron-Carbon systems, along with a comparison of the efficacy of the OHE and GE., Comment: 38 pages, 13 tables, and 18 figures

Large language models trained for safety and harmlessness remain susceptible to adversarial misuse, as evidenced by the prevalence of "jailbreak" attacks on early releases of ChatGPT that elicit undesired behavior. Going beyond recognition of the issue, we investigate why such attacks succeed and how they can be created. We hypothesize two failure modes of safety training: competing objectives and mismatched generalization. Competing objectives arise when a model's capabilities and safety goals conflict, while mismatched generalization occurs when safety training fails to generalize to a domain for which capabilities exist. We use these failure modes to guide jailbreak design and then evaluate state-of-the-art models, including OpenAI's GPT-4 and Anthropic's Claude v1.3, against both existing and newly designed attacks. We find that vulnerabilities persist despite the extensive red-teaming and safety-training efforts behind these models. Notably, new attacks utilizing our failure modes succeed on every prompt in a collection of unsafe requests from the models' red-teaming evaluation sets and outperform existing ad hoc jailbreaks. Our analysis emphasizes the need for safety-capability parity -- that safety mechanisms should be as sophisticated as the underlying model -- and argues against the idea that scaling alone can resolve these safety failure modes., Comment

State-of-the-art federated learning methods can perform far worse than their centralized counterparts when clients have dissimilar data distributions. For neural networks, even when centralized SGD easily finds a solution that is simultaneously performant for all clients, current federated optimization methods fail to converge to a comparable solution. We show that this performance disparity can largely be attributed to optimization challenges presented by nonconvexity. Specifically, we find that the early layers of the network do learn useful features, but the final layers fail to make use of them. That is, federated optimization applied to this non-convex problem distorts the learning of the final layers. Leveraging this observation, we propose a Train-Convexify-Train (TCT) procedure to sidestep this issue: first, learn features using off-the-shelf methods (e.g., FedAvg); then, optimize a convexified problem obtained from the network's empirical neural tangent kernel approximation. Our technique yields accuracy improvements of up to +36% on FMNIST and +37% on CIFAR10 when clients have dissimilar data., Comment: Accepted at Neural Information Processing Systems (NeurIPS) 2022. V2 releases code

Of theories for why large-scale machine learning models generalize despite being vastly overparameterized, which of their assumptions are needed to capture the qualitative phenomena of generalization in the real world? On one hand, we find that most theoretical analyses fall short of capturing these qualitative phenomena even for kernel regression, when applied to kernels derived from large-scale neural networks (e.g., ResNet-50) and real data (e.g., CIFAR-100). On the other hand, we find that the classical GCV estimator (Craven and Wahba, 1978) accurately predicts generalization risk even in such overparameterized settings. To bolster this empirical finding, we prove that the GCV estimator converges to the generalization risk whenever a local random matrix law holds. Finally, we apply this random matrix theory lens to explain why pretrained representations generalize better as well as what factors govern scaling laws for kernel regression. Our findings suggest that random matrix theory, rather than just being a toy model, may be central to understanding the properties of neural representations in practice.

We propose a metric -- Projection Norm -- to predict a model's performance on out-of-distribution (OOD) data without access to ground truth labels. Projection Norm first uses model predictions to pseudo-label test samples and then trains a new model on the pseudo-labels. The more the new model's parameters differ from an in-distribution model, the greater the predicted OOD error. Empirically, our approach outperforms existing methods on both image and text classification tasks and across different network architectures. Theoretically, we connect our approach to a bound on the test error for overparameterized linear models. Furthermore, we find that Projection Norm is the only approach that achieves non-trivial detection performance on adversarial examples. Our code is available at https://github.com/yaodongyu/ProjNorm.

Large-scale, two-sided matching platforms must find market outcomes that align with user preferences while simultaneously learning these preferences from data. Classical notions of stability (Gale and Shapley, 1962; Shapley and Shubik, 1971) are unfortunately of limited value in the learning setting, given that preferences are inherently uncertain and destabilizing while they are being learned. To bridge this gap, we develop a framework and algorithms for learning stable market outcomes under uncertainty. Our primary setting is matching with transferable utilities, where the platform both matches agents and sets monetary transfers between them. We design an incentive-aware learning objective that captures the distance of a market outcome from equilibrium. Using this objective, we analyze the complexity of learning as a function of preference structure, casting learning as a stochastic multi-armed bandit problem. Algorithmically, we show that "optimism in the face of uncertainty," the principle underlying many bandit algorithms, applies to a primal-dual formulation of matching with transfers and leads to near-optimal regret bounds. Our work takes a first step toward elucidating when and how stable matchings arise in large, data-driven marketplaces., Comment: Accepted to the Journal of the ACM; conference version appeared at NeurIPS 2021

We study the design of a decentralized two-sided matching market in which agents' search is guided by the platform. There are finitely many agent types, each with (potentially random) preferences drawn from known type-specific distributions. Equipped with knowledge of these distributions, the platform guides the search process by determining the meeting rate between each pair of types from the two sides. Focusing on symmetric pairwise preferences in a continuum model, we first characterize the unique stationary equilibrium that arises given a feasible set of meeting rates. We then introduce the platform's optimal directed search problem, which involves optimizing meeting rates to maximize equilibrium social welfare. We first show that incentive issues arising from congestion and cannibalization make the design problem fairly intricate. Nonetheless, we develop an efficiently computable search design whose corresponding equilibrium achieves at least 1/4 the social welfare of the optimal design. In fact, our construction always recovers at least 1/4 the first-best social welfare, where agents' incentives are disregarded. Our directed search design is simple and easy-to-implement, as its corresponding bipartite graph consists of disjoint stars. Furthermore, our design implies the platform can substantially limit choice and yet induce an equilibrium with an approximately optimal welfare. Finally, we show that approximation is likely the best we can hope for by establishing that the problem of designing optimal directed search is NP-hard to even approximate beyond a certain constant factor.

We study the problem of improving the performance of online algorithms by incorporating machine-learned predictions. The goal is to design algorithms that are both consistent and robust, meaning that the algorithm performs well when predictions are accurate and maintains worst-case guarantees. Such algorithms have been studied in a recent line of works due to Lykouris and Vassilvitskii (ICML '18) and Purohit et al (NeurIPS '18). They provide robustness-consistency trade-offs for a variety of online problems. However, they leave open the question of whether these trade-offs are tight, i.e., to what extent to such trade-offs are necessary. In this paper, we provide the first set of non-trivial lower bounds for competitive analysis using machine-learned predictions. We focus on the classic problems of ski-rental and non-clairvoyant scheduling and provide optimal trade-offs in various settings., Comment: To appear at NeurIPS 2020

Kapton HN films, adopted worldwide due to their superior thermal durability (up to 400 {\deg}C), allow the high temperature sintering of nanoparticle based metal inks. By carefully selecting inks and Kapton substrates, outstanding thermal stability and anti-delaminating features are obtained in both aqueous and organic solutions and were applied to four novel devices: a solid state ion selective nitrate sensor, an ssDNA based mercury aptasensor, a low cost protein sensor, and a long lasting organic thin film transistor (OTFT). Many experimental studies on parameter combinations were conducted during the development of the above devices. The results showed that the ion selective nitrate sensor displayed a linear sensitivity range with a limit of detection of 2 ppm. The mercury sensor exhibited a linear correlation between the RCT values and the increasing concentrations of mercury. The protein printed circuit board (PCB) sensor provided a much simpler method of protein detection. Finally, the OTFT demonstrated a stable performance with mobility values for the linear and saturation regimes, and the threshold voltage. These devices have shown their value and reveal possibilities that could be pursued.

Lykouris and Vassilvitskii (ICML 2018) introduce a model of online caching with machine-learned advice, where each page request additionally comes with a prediction of when that page will next be requested. In this model, a natural goal is to design algorithms that (1) perform well when the advice is accurate and (2) remain robust in the worst case a la traditional competitive analysis. Lykouris and Vassilvitskii give such an algorithm by adapting the Marker algorithm to the learning-augmented setting. In a recent work, Rohatgi (SODA 2020) improves on their result with an approach also inspired by randomized marking. We continue the study of this problem, but with a somewhat different approach: We consider combining the BlindOracle algorithm, which just na\"ively follows the predictions, with an optimal competitive algorithm for online caching in a black-box manner. The resulting algorithm outperforms all existing approaches while being significantly simpler. Moreover, we show that combining BlindOracle with LRU is in fact optimal among deterministic algorithms for this problem.

We show that approximate near neighbor search in high dimensions can be solved in a Las Vegas fashion (i.e., without false negatives) for $\ell_p$ ($1\le p\le 2$) while matching the performance of optimal locality-sensitive hashing. Specifically, we construct a data-independent Las Vegas data structure with query time $O(dn^{\rho})$ and space usage $O(dn^{1+\rho})$ for $(r, c r)$-approximate near neighbors in $\mathbb{R}^{d}$ under the $\ell_p$ norm, where $\rho = 1/c^p + o(1)$. Furthermore, we give a Las Vegas locality-sensitive filter construction for the unit sphere that can be used with the data-dependent data structure of Andoni et al. (SODA 2017) to achieve optimal space-time tradeoffs in the data-dependent setting. For the symmetric case, this gives us a data-dependent Las Vegas data structure with query time $O(dn^{\rho})$ and space usage $O(dn^{1+\rho})$ for $(r, c r)$-approximate near neighbors in $\mathbb{R}^{d}$ under the $\ell_p$ norm, where $\rho = 1/(2c^p - 1) + o(1)$. Our data-independent construction improves on the recent Las Vegas data structure of Ahle (FOCS 2017) for $\ell_p$ when $1 < p\le 2$. Our data-dependent construction does even better for $\ell_p$ for all $p\in [1, 2]$ and is the first Las Vegas approximate near neighbors data structure to make use of data-dependent approaches. We also answer open questions of Indyk (SODA 2000), Pagh (SODA 2016), and Ahle by showing that for approximate near neighbors, Las Vegas data structures can match state-of-the-art Monte Carlo data structures in performance for both the data-independent and data-dependent settings and across space-time tradeoffs.

Coherent nanoscale photon sources are of paramount importance to achieving all-optical communication. Several nanolasers smaller than the diffraction limit have been theoretically proposed and experimentally demonstrated using plasmonic cavities to confine optical fields. Such compact cavities exhibit large Purcell factors, thereby enhancing spontaneous emission, which feeds into the lasing mode. However, most plasmonic nanolasers reported so far have employed resonant nanostructures and therefore had the lasing restricted to the proximity of the resonance wavelength. Here, we report on an approach based on gold nanorod hyperbolic metamaterials for lasing. Hyperbolic metamaterials provide broadband Purcell enhancement due to large photonic density of optical states, while also supporting surface plasmon modes to deliver optical feedback for lasing due to nonlocal effects in nanorod media. We experimentally demonstrate the advantage of hyperbolic metamaterials in achieving lasing action by its comparison with that obtained in a metamaterial with elliptic dispersion. The conclusions from the experimental results are supported with numerical simulations comparing the Purcell factors and surface plasmon modes for the metamaterials with different dispersions. We show that although the metamaterials of both types support lasing, emission with hyperbolic samples is about twice as strong with 35% lower threshold vs. the elliptic ones. Hence, hyperbolic metamaterials can serve as a convenient platform of choice for nanoscale coherent photon sources in a broad wavelength range.