Search

As AI systems increasingly make critical decisions, deceptive AI poses a significant challenge to trust and safety. We present Self-Other Overlap (SOO) fine-tuning, a promising approach in AI Safety that could substantially improve our ability to build honest artificial intelligence. Inspired by cognitive neuroscience research on empathy, SOO aims to align how AI models represent themselves and others. Our experiments on LLMs with 7B, 27B, and 78B parameters demonstrate SOO's efficacy: deceptive responses of Mistral-7B-Instruct-v0.2 dropped from 73.6% to 17.2% with no observed reduction in general task performance, while in Gemma-2-27b-it and CalmeRys-78B-Orpo-v0.1 deceptive responses were reduced from 100% to 9.3% and 2.7%, respectively, with a small impact on capabilities. In reinforcement learning scenarios, SOO-trained agents showed significantly reduced deceptive behavior. SOO's focus on contrastive self and other-referencing observations offers strong potential for generalization across AI architectures. While current applications focus on language models and simple RL environments, SOO could pave the way for more trustworthy AI in broader domains. Ethical implications and long-term effects warrant further investigation, but SOO represents a significant step forward in AI safety research., Comment: NeurIPS 2024 Safe Generative AI Workshop

Data privacy is a core tenet of responsible computing, and in the United States, differential privacy (DP) is the dominant technical operationalization of privacy-preserving data analysis. With this study, we qualitatively examine one class of DP mechanisms: private data synthesizers. To that end, we conducted semi-structured interviews with data experts: academics and practitioners who regularly work with data. Broadly, our findings suggest that quantitative DP benchmarks must be grounded in practitioner needs, while communication challenges persist. Participants expressed a need for context-aware DP solutions, focusing on parity between research outcomes on real and synthetic data. Our analysis led to three recommendations: (1) improve existing insufficient sanitized benchmarks; successful DP implementations require well-documented, partner-vetted use cases, (2) organizations using DP synthetic data should publish discipline-specific standards of evidence, and (3) tiered data access models could allow researchers to gradually access sensitive data based on demonstrated competence with high-privacy, low-fidelity synthetic data.

We prove that no eigenvalue of the clamped disk has multiplicity greater than four. This improves upon a previous bound. Exploiting a linear recursion formula of order two for cross-product Bessel functions in which the coefficients are non-rational functions satisfying a non-linear algebraic recursion, we show that higher multiplicity eigenvalues must be algebraic, in contradiction with the Siegel-Shidlovskii theory., Comment: 7 pages

Imbalanced learning occurs in classification settings where the distribution of class-labels is highly skewed in the training data, such as when predicting rare diseases or in fraud detection. This class imbalance presents a significant algorithmic challenge, which can be further exacerbated when privacy-preserving techniques such as differential privacy are applied to protect sensitive training data. Our work formalizes these challenges and provides a number of algorithmic solutions. We consider DP variants of pre-processing methods that privately augment the original dataset to reduce the class imbalance; these include oversampling, SMOTE, and private synthetic data generation. We also consider DP variants of in-processing techniques, which adjust the learning algorithm to account for the imbalance; these include model bagging, class-weighted empirical risk minimization and class-weighted deep learning. For each method, we either adapt an existing imbalanced learning technique to the private setting or demonstrate its incompatibility with differential privacy. Finally, we empirically evaluate these privacy-preserving imbalanced learning methods under various data and distributional settings. We find that private synthetic data methods perform well as a data pre-processing step, while class-weighted ERMs are an alternative in higher-dimensional settings where private synthetic data suffers from the curse of dimensionality., Comment: 14 pages

Fair predictive algorithms hinge on both equality and trust, yet inherent uncertainty in real-world data challenges our ability to make consistent, fair, and calibrated decisions. While fairly managing predictive error has been extensively explored, some recent work has begun to address the challenge of fairly accounting for irreducible prediction uncertainty. However, a clear taxonomy and well-specified objectives for integrating uncertainty into fairness remains undefined. We address this gap by introducing FairlyUncertain, an axiomatic benchmark for evaluating uncertainty estimates in fairness. Our benchmark posits that fair predictive uncertainty estimates should be consistent across learning pipelines and calibrated to observed randomness. Through extensive experiments on ten popular fairness datasets, our evaluation reveals: (1) A theoretically justified and simple method for estimating uncertainty in binary settings is more consistent and calibrated than prior work; (2) Abstaining from binary predictions, even with improved uncertainty estimates, reduces error but does not alleviate outcome imbalances between demographic groups; (3) Incorporating consistent and calibrated uncertainty estimates in regression tasks improves fairness without any explicit fairness interventions. Additionally, our benchmark package is designed to be extensible and open-source, to grow with the field. By providing a standardized framework for assessing the interplay between uncertainty and fairness, FairlyUncertain paves the way for more equitable and trustworthy machine learning practices.

We study the problem of approximately recovering a probability distribution given noisy measurements of its Chebyshev polynomial moments. We sharpen prior work, proving that accurate recovery in the Wasserstein distance is possible with more noise than previously known. As a main application, our result yields a simple "linear query" algorithm for constructing a differentially private synthetic data distribution with Wasserstein-1 error $\tilde{O}(1/n)$ based on a dataset of $n$ points in $[-1,1]$. This bound is optimal up to log factors and matches a recent breakthrough of Boedihardjo, Strohmer, and Vershynin [Probab. Theory. Rel., 2024], which uses a more complex "superregular random walk" method to beat an $O(1/\sqrt{n})$ accuracy barrier inherent to earlier approaches. We illustrate a second application of our new moment-based recovery bound in numerical linear algebra: by improving an approach of Braverman, Krishnan, and Musco [STOC 2022], our result yields a faster algorithm for estimating the spectral density of a symmetric matrix up to small error in the Wasserstein distance.

Stochastic diffusion is the noisy and uncertain process through which dynamics like epidemics, or agents like animal species, disperse over a larger area. Understanding these processes is becoming increasingly important as we attempt to better prepare for potential pandemics and as species ranges shift in response to climate change. Unfortunately, modeling of stochastic diffusion is mostly done through inaccurate deterministic tools that fail to capture the random nature of dispersal or else through expensive computational simulations. In particular, standard tools fail to fully capture the heterogeneity of the area over which this diffusion occurs. Rural areas with low population density require different epidemic models than urban areas; likewise, the edges of a species range require us to explicitly track low integer numbers of individuals rather than vague averages. In this work, we introduce a series of new tools called "mean-FLAME" models that track stochastic dispersion using approximate master equations that explicitly follow the probability distribution of an area of interest over all of its possible states, up to states that are active enough to be approximated using a mean-field model. In one limit, this approach is locally exact if we explicitly track enough states, and in the other limit collapses back to traditional deterministic models if we track no state explicitly. Applying this approach, we show how deterministic tools fail to capture the uncertainty around the speed of nonlinear dynamical processes. This is especially true for marginal areas that are close to unsuitable for diffusion, like the edge of a species range or epidemics in small populations. Capturing the uncertainty in such areas is key to producing accurate forecasts and guiding potential interventions., Comment: For supplemental appendix, see "Ancillary files" under "Access paper" on the right

Self-models have been a topic of great interest for decades in studies of human cognition and more recently in machine learning. Yet what benefits do self-models confer? Here we show that when artificial networks learn to predict their internal states as an auxiliary task, they change in a fundamental way. To better perform the self-model task, the network learns to make itself simpler, more regularized, more parameter-efficient, and therefore more amenable to being predictively modeled. To test the hypothesis of self-regularizing through self-modeling, we used a range of network architectures performing three classification tasks across two modalities. In all cases, adding self-modeling caused a significant reduction in network complexity. The reduction was observed in two ways. First, the distribution of weights was narrower when self-modeling was present. Second, a measure of network complexity, the real log canonical threshold (RLCT), was smaller when self-modeling was present. Not only were measures of complexity reduced, but the reduction became more pronounced as greater training weight was placed on the auxiliary task of self-modeling. These results strongly support the hypothesis that self-modeling is more than simply a network learning to predict itself. The learning has a restructuring effect, reducing complexity and increasing parameter efficiency. This self-regularization may help explain some of the benefits of self-models reported in recent machine learning literature, as well as the adaptive value of self-models to biological systems. In particular, these findings may shed light on the possible interaction between the ability to model oneself and the ability to be more easily modeled by others in a social or cooperative context.

We investigate the almost everywhere convergence of sequences of convolution operators given by probability measures $\mu_n$ on $\mathbb R$. If this sequence of operators constitutes an approximate identity on a particular class of functions $\mathcal F$, under what additional conditions do we have $\mu_n \ast f \to f$ a.e. for all $f \in \mathcal F$? We focus on the particular case of a sequence of contractions $C_{t_n}\mu$ of a single probability measure $\mu$, with $t_n \to 0$, so that that the sequence of operators is an approximate identity.

In this paper, we investigate the degree to which fine-tuning in Large Language Models (LLMs) effectively mitigates versus merely conceals undesirable behavior. Through the lens of semi-realistic role-playing exercises designed to elicit such behaviors, we explore the response dynamics of LLMs post fine-tuning interventions. Our methodology involves prompting models for Chain-of-Thought (CoT) reasoning and analyzing the coherence between the reasoning traces and the resultant outputs. Notably, we identify a pervasive phenomenon we term \emph{reason-based deception}, where models either stop producing reasoning traces or produce seemingly ethical reasoning traces that belie the unethical nature of their final outputs. We further examine the efficacy of response strategies (polite refusal versus explicit rebuttal) in curbing the occurrence of undesired behavior in subsequent outputs of multi-turn interactions. Our findings reveal that explicit rebuttals significantly outperform polite refusals in preventing the continuation of undesired outputs and nearly eliminate reason-based deception, challenging current practices in model fine-tuning. Accordingly, the two key contributions of this paper are (1) defining and studying reason-based deception, a new type of hidden behavior, and (2) demonstrating that rebuttals provide a more robust response model to harmful requests than refusals, thereby highlighting the need to reconsider the response strategies in fine-tuning approaches., Comment: ICLR 2024 AGI Workshop Poster

The rapid proliferation of generative AI has raised questions about the competitiveness of lower-parameter, locally tunable, open-weight models relative to high-parameter, API-guarded, closed-weight models in terms of performance, domain adaptation, cost, and generalization. Centering under-resourced yet risk-intolerant settings in government, research, and healthcare, we see for-profit closed-weight models as incompatible with requirements for transparency, privacy, adaptability, and standards of evidence. Yet the performance penalty in using open-weight models, especially in low-data and low-resource settings, is unclear. We assess the feasibility of using smaller, open-weight models to replace GPT-4-Turbo in zero-shot, few-shot, and fine-tuned regimes, assuming access to only a single, low-cost GPU. We assess value-sensitive issues around bias, privacy, and abstention on three additional tasks relevant to those topics. We find that with relatively low effort, very low absolute monetary cost, and relatively little data for fine-tuning, small open-weight models can achieve competitive performance in domain-adapted tasks without sacrificing generality. We then run experiments considering practical issues in bias, privacy, and hallucination risk, finding that open models offer several benefits over closed models. We intend this work as a case study in understanding the opportunity cost of reproducibility and transparency over for-profit state-of-the-art zero shot performance, finding this cost to be marginal under realistic settings., Comment: Accepted at the ACM Conference on Fairness, Accountability, and Transparency (FAccT) 2024

For an ergodic map $T$ and a non-constant, real-valued $f \in L^1$, the ergodic averages $\mathbb{A}_N f(x) = \frac{1} {N} \sum_{n=1}^N f(T^n x)$ converge a.e., but the convergence is never monotone. Depending on particular properties of the function $f$, the averages $\mathbb{A}_N f(x)$ may or may not actually fluctuate around the mean value infinitely often a.e. We will prove that a.e. fluctuation around the mean is the generic behavior. That is, for a fixed ergodic $T$, the generic non-constant $f\in L^1$ has the averages $\mathbb{A}_N f(x)$ fluctuating around the mean infinitely often for almost every $x$. We also consider fluctuation for other stochastic processes like subsequences of the ergodic averages, convolution operators, weighted averages, uniform distribution and martingales. We will show that in general, in these settings fluctuation around the limit infinitely often persists as the generic behavior., Comment: 29 pages

We review results of {\v{S}}ver\'{a}k, and Goldstein-Haj{\l}asz-Pakzad on how to show the continuity of functions in a critical Sobolev space with positive Jacobian. In the final chapter we expand on the theory of $VMO$ functions, showing a version of the change of variables theorem in this regularity, which generalizes the first step in {\v{S}}ver\'{a}k's proof to a co-dimension 1 case., Comment: This work was done as part of a master's program on Hebrew University of Jerusalem Israel under the supervision of Cy Maor

Differentially private (DP) mechanisms have been deployed in a variety of high-impact social settings (perhaps most notably by the U.S. Census). Since all DP mechanisms involve adding noise to results of statistical queries, they are expected to impact our ability to accurately analyze and learn from data, in effect trading off privacy with utility. Alarmingly, the impact of DP on utility can vary significantly among different sub-populations. A simple way to reduce this disparity is with stratification. First compute an independent private estimate for each group in the data set (which may be the intersection of several protected classes), then, to compute estimates of global statistics, appropriately recombine these group estimates. Our main observation is that naive stratification often yields high-accuracy estimates of population-level statistics, without the need for additional privacy budget. We support this observation theoretically and empirically. Our theoretical results center on the private mean estimation problem, while our empirical results center on extensive experiments on private data synthesis to demonstrate the effectiveness of stratification on a variety of private mechanisms. Overall, we argue that this straightforward approach provides a strong baseline against which future work on reducing utility disparities of DP mechanisms should be compared.

The open streets initiative "opens" streets to pedestrians and bicyclists by closing them to cars and trucks. The initiative, adopted by many cities across North America, increases community space in urban environments. But could open streets also make cities safer and less congested? We study this question by framing the choice of which streets to open as a reinforcement learning problem. In order to simulate the impact of opening streets, we first compare models for predicting vehicle collisions given network and temporal data. We find that a recurrent graph neural network, leveraging the graph structure and the short-term temporal dependence of the data, gives the best predictive performance. Then, with the ability to simulate collisions and traffic, we frame a reinforcement learning problem to find which streets to open. We compare the streets in the NYC Open Streets program to those proposed by a Q-learning algorithm. We find that the streets proposed by the Q-learning algorithm have reliably better outcomes, while streets in the program have similar outcomes to randomly selected streets. We present our work as a step toward principally choosing which streets to open for safer and less congested cities. All our code and data are available on Github., Comment: camera ready for AAAI 2024

Epidemic models study the spread of an undesired agent through a population, be it infectious diseases through a country, misinformation in online social media, or pests infesting a region. In combating these epidemics, we rely neither on global top-down interventions, nor solely on individual adaptations. Instead, interventions most commonly come from local institutions such as public health departments, moderation teams on social media platforms, or other forms of group governance. Classic models, which are often individual or agent-based, are ill-suited to capture local adaptations. We leverage recent development of institutional dynamics based on cultural group selection to study how groups can attempt local control of an epidemic by taking inspiration from the successes and failures of other groups. Incorporating these institutional changes into the epidemic dynamics reveals paradoxes: a higher transmission rate can result in smaller outbreaks and decreasing the speed of institutional adaptation generally reduces outbreak size. When groups perceive a contagion as more worrisome, they can invest in improved policies and, if they maintain these policies long enough to have impact, lead to a reduction in endemicity. By looking at the interplay between the speed of institutions and the transmission rate of the contagions, we find rich co-evolutionary dynamics that reflect the complexity of known biological and social contagions.

An invasive species of grass known as "buffelgrass" contributes to severe wildfires and biodiversity loss in the Southwest United States. We tackle the problem of predicting buffelgrass "green-ups" (i.e. readiness for herbicidal treatment). To make our predictions, we explore temporal, visual and multi-modal models that combine satellite sensing and deep learning. We find that all of our neural-based approaches improve over conventional buffelgrass green-up models, and discuss how neural model deployment promises significant resource savings.

We study the structure of the zero set of a nontrivial finite point charge electrical field $F = (X,Y)$ in the plane $\mathbb R^2$. We establish equations satisfied by the possible directions for the zero sets \{X = 0\} and $\{Y = 0\}$ separately, and we show that there are only finitely many possible asymptotic directions for both of these zero sets. We suspect that the set of asymptotic directions for \{X = 0\} and the set of asymptotic directions for $\{Y = 0\}$ are (essentially) distinct., Comment: This article is the second part of an earlier article by the same authors on zeros of electrical fields of points charges. See arXiv:2106.04706

The ``impossibility theorem'' -- which is considered foundational in algorithmic fairness literature -- asserts that there must be trade-offs between common notions of fairness and performance when fitting statistical models, except in two special cases: when the prevalence of the outcome being predicted is equal across groups, or when a perfectly accurate predictor is used. However, theory does not always translate to practice. In this work, we challenge the implications of the impossibility theorem in practical settings. First, we show analytically that, by slightly relaxing the impossibility theorem (to accommodate a \textit{practitioner's} perspective of fairness), it becomes possible to identify a large set of models that satisfy seemingly incompatible fairness constraints. Second, we demonstrate the existence of these models through extensive experiments on five real-world datasets. We conclude by offering tools and guidance for practitioners to understand when -- and to what degree -- fairness along multiple criteria can be achieved. For example, if one allows only a small margin-of-error between metrics, there exists a large set of models simultaneously satisfying \emph{False Negative Rate Parity}, \emph{False Positive Rate Parity}, and \emph{Positive Predictive Value Parity}, even when there is a moderate prevalence difference between groups. This work has an important implication for the community: achieving fairness along multiple metrics for multiple groups (and their intersections) is much more possible than was previously believed., Comment: 14 pages, 3 figures, 1 table