Search

Text data has become extremely valuable due to the emergence of machine learning algorithms that learn from it. A lot of high-quality text data generated in the real world is private and therefore cannot be shared or used freely due to privacy concerns. Generating synthetic replicas of private text data with a formal privacy guarantee, i.e., differential privacy (DP), offers a promising and scalable solution. However, existing methods necessitate DP finetuning of large language models (LLMs) on private data to generate DP synthetic data. This approach is not viable for proprietary LLMs (e.g., GPT-3.5) and also demands considerable computational resources for open-source LLMs. Lin et al. (2024) recently introduced the Private Evolution (PE) algorithm to generate DP synthetic images with only API access to diffusion models. In this work, we propose an augmented PE algorithm, named Aug-PE, that applies to the complex setting of text. We use API access to an LLM and generate DP synthetic text without any model training. We conduct comprehensive experiments on three benchmark datasets. Our results demonstrate that Aug-PE produces DP synthetic text that yields competitive utility with the SOTA DP finetuning baselines. This underscores the feasibility of relying solely on API access of LLMs to produce high-quality DP synthetic texts, thereby facilitating more accessible routes to privacy-preserving LLM applications. Our code and data are available at https://github.com/AI-secure/aug-pe., Comment: ICML'24 Spotlight

Service providers of large language model (LLM) applications collect user instructions in the wild and use them in further aligning LLMs with users' intentions. These instructions, which potentially contain sensitive information, are annotated by human workers in the process. This poses a new privacy risk not addressed by the typical private optimization. To this end, we propose using synthetic instructions to replace real instructions in data annotation and model fine-tuning. Formal differential privacy is guaranteed by generating those synthetic instructions using privately fine-tuned generators. Crucial in achieving the desired utility is our novel filtering algorithm that matches the distribution of the synthetic instructions to that of the real ones. In both supervised fine-tuning and reinforcement learning from human feedback, our extensive experiments demonstrate the high utility of the final set of synthetic instructions by showing comparable results to real instructions. In supervised fine-tuning, models trained with private synthetic instructions outperform leading open-source models such as Vicuna., Comment: ICML 2024. Code available at https://github.com/google-research/google-research/tree/master/dp_instructions

Text prediction models, when used in applications like email clients or word processors, must protect user data privacy and adhere to model size constraints. These constraints are crucial to meet memory and inference time requirements, as well as to reduce inference costs. Building small, fast, and private domain-specific language models is a thriving area of research. In this work, we show that a careful pre-training on a \emph{subset} of the public dataset that is guided by the private dataset is crucial to train small language models with differential privacy. On standard benchmarks, small models trained with our new framework achieve state-of-the-art performance. In addition to performance improvements, our results demonstrate that smaller models, through careful pre-training and private fine-tuning, can match the performance of much larger models that do not have access to private data. This underscores the potential of private learning for model compression and enhanced efficiency., Comment: Transactions on Machine Learning Research. Code available at https://github.com/dayu11/selective_pretraining_for_private_finetuning

In this article, we present a detailed review of current practices and state-of-the-art methodologies in the field of differential privacy (DP), with a focus of advancing DP's deployment in real-world applications. Key points and high-level contents of the article were originated from the discussions from "Differential Privacy (DP): Challenges Towards the Next Frontier," a workshop held in July 2022 with experts from industry, academia, and the public sector seeking answers to broad questions pertaining to privacy and its implications in the design of industry-grade systems. This article aims to provide a reference point for the algorithmic and design decisions within the realm of privacy, highlighting important challenges and potential research directions. Covering a wide spectrum of topics, this article delves into the infrastructure needs for designing private systems, methods for achieving better privacy/utility trade-offs, performing privacy attacks and auditing, as well as communicating privacy with broader audiences and stakeholders.

In the absence of time-reversal invariance, metals without inversion symmetry may exhibit nonreciprocal charge transport -- a magnetochiral anisotropy that manifests as unequal electrical resistance for opposite current flow directions. If superconductivity also sets in, the charge transmission may become dissipationless in one direction while remaining dissipative in the opposite, thereby realizing a superconducting diode. Through both direct-current and alternating-current measurements, we study the nonreciprocal effects in thin films of the noncentrosymmetric superconductor T$_d$-MoTe\textsubscript{2} with disorders. We observe nonreciprocal superconducting critical currents with a diode efficiency close to 20\%~, and a large magnetochiral anisotropy coefficient up to $\SI{5.9e8}{\per\tesla\per\ampere}$, under weak out-of-plane magnetic field in the millitesla range. The great enhancement of rectification efficiency under out-of-plane magnetic field is likely abscribed to the vortex ratchet effect, which naturally appears in the noncentrosymmetric superconductor with disorders. Intriguingly, unlike the finding in Rashba systems, the strongest in-plane nonreciprocal effect does not occur when the field is perpendicular to the current flow direction. We develop a phenomenological theory to demonstrate that this peculiar behavior can be attributed to the asymmetric structure of spin-orbit coupling in T$_d$-MoTe\textsubscript{2}. Our study highlights how the crystallographic symmetry critically impacts the nonreciprocal transport, and would further advance the research for designing the superconducting diode with the best performance., Comment: 7 pages, 5figures

Differentially private deep learning has recently witnessed advances in computational efficiency and privacy-utility trade-off. We explore whether further improvements along the two axes are possible and provide affirmative answers leveraging two instantiations of \emph{group-wise clipping}. To reduce the compute time overhead of private learning, we show that \emph{per-layer clipping}, where the gradient of each neural network layer is clipped separately, allows clipping to be performed in conjunction with backpropagation in differentially private optimization. This results in private learning that is as memory-efficient and almost as fast per training update as non-private learning for many workflows of interest. While per-layer clipping with constant thresholds tends to underperform standard flat clipping, per-layer clipping with adaptive thresholds matches or outperforms flat clipping under given training epoch constraints, hence attaining similar or better task performance within less wall time. To explore the limits of scaling (pretrained) models in differentially private deep learning, we privately fine-tune the 175 billion-parameter GPT-3. We bypass scaling challenges associated with clipping gradients that are distributed across multiple devices with \emph{per-device clipping} that clips the gradient of each model piece separately on its host device. Privately fine-tuning GPT-3 with per-device clipping achieves a task performance at $\epsilon=1$ better than what is attainable by non-privately fine-tuning the largest GPT-2 on a summarization task., Comment: 25 pages

Adversarial examples, which are usually generated for specific inputs with a specific model, are ubiquitous for neural networks. In this paper we unveil a surprising property of adversarial noises when they are put together, i.e., adversarial noises crafted by one-step gradient methods are linearly separable if equipped with the corresponding labels. We theoretically prove this property for a two-layer network with randomly initialized entries and the neural tangent kernel setup where the parameters are not far from initialization. The proof idea is to show the label information can be efficiently backpropagated to the input while keeping the linear separability. Our theory and experimental evidence further show that the linear classifier trained with the adversarial noises of the training data can well classify the adversarial noises of the test data, indicating that adversarial noises actually inject a distributional perturbation to the original data distribution. Furthermore, we empirically demonstrate that the adversarial noises may become less linearly separable when the above conditions are compromised while they are still much easier to classify than original features., Comment: 13 pages

We report magnetotransport measurements in the NbN/ magnetic topological insulator MnBi2Te4 (MBT)/ NbN junction at low temperature. At 10 mK, the nonlinear current-voltage characteristic of the junction shows a tunneling behavior, indicating the existence of interfacial potential barriers within the heterostructure. Under an out of plane perpendicular magnetic field, a transition from negative to positive magnetoresistance (MR) is found when increasing the bias voltage. A proximity-induced superconducting gap is estimated to be 0.1meV by a pair of differential resistance dips. Moreover, the induced gap is enhanced by gradually tuning the Fermi level toward the charge neutral point by a back gate voltage, which is ascribed to the increased transport contribution of the topological surface states in MBT. Intriguingly, the induced gap exhibits an anomalous magnetic field assisted enhancement, which may originate from the spin orbit coupling and magnetic order of MBT. Our results reveal the interplay between magnetism and superconductivity in MBT, paving the way for further studies on topological superconductivity and chiral Majorana edge modes in quantum anomalous Hall insulator/superconductor hybrid systems., Comment: 9 pages, 8 figures

Differentially private stochastic gradient descent (DP-SGD) is the workhorse algorithm for recent advances in private deep learning. It provides a single privacy guarantee to all datapoints in the dataset. We propose output-specific $(\varepsilon,\delta)$-DP to characterize privacy guarantees for individual examples when releasing models trained by DP-SGD. We also design an efficient algorithm to investigate individual privacy across a number of datasets. We find that most examples enjoy stronger privacy guarantees than the worst-case bound. We further discover that the training loss and the privacy parameter of an example are well-correlated. This implies groups that are underserved in terms of model utility simultaneously experience weaker privacy guarantees. For example, on CIFAR-10, the average $\varepsilon$ of the class with the lowest test accuracy is 44.2\% higher than that of the class with the highest accuracy., Comment: Add clarification about the applicability of Definition 4

We present detail thermodynamic and muon spin relaxation ($\mu$SR) studies of quantum spin liquid (QSL) candidate H$_3$LiIr$_2$O$_6$. In agreement with the low temperature thermodynamic evidence (\textit{e.g.} bulk magnetization and heat capacity) for the absence of magnetic transition, zero-field (ZF)-$\mu$SR measurements indicate the absence of static magnetic ordering or spin freezing down to our lowest temperature of 80~mK. Both ZF- and longitudinal-field (LF)-$\mu$SR measurements reveal persistent spin fluctuations at low temperatures. These results provide well-established evidence of a QSL state in H$_3$LiIr$_2$O$_6$. Furthermore, the observation of the time-field scaling behavior of $\mu$SR spectra $A(t)\sim A(t/H^{0.46})$, and the low temperature power-law specific heat coefficient $C/T \sim T^{-0.57}$, indicate the finite density of state in the form of $N(E) \sim E^{-0.5}$, in a good agreement with the disorder-induced states in the Kitaev spin liquid.

Availability attacks, which poison the training data with imperceptible perturbations, can make the data \emph{not exploitable} by machine learning algorithms so as to prevent unauthorized use of data. In this work, we investigate why these perturbations work in principle. We are the first to unveil an important population property of the perturbations of these attacks: they are almost \textbf{linearly separable} when assigned with the target labels of the corresponding samples, which hence can work as \emph{shortcuts} for the learning objective. We further verify that linear separability is indeed the workhorse for availability attacks. We synthesize linearly-separable perturbations as attacks and show that they are as powerful as the deliberately crafted attacks. Moreover, such synthetic perturbations are much easier to generate. For example, previous attacks need dozens of hours to generate perturbations for ImageNet while our algorithm only needs several seconds. Our finding also suggests that the \emph{shortcut learning} is more widely present than previously believed as deep models would rely on shortcuts even if they are of an imperceptible scale and mixed together with the normal features. Our source code is published at \url{https://github.com/dayu11/Availability-Attacks-Create-Shortcuts}., Comment: Published as a research track paper at KDD 2022

We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-scale pre-trained language models, which achieve the state-of-the-art privacy versus utility tradeoffs on many standard NLP tasks. We propose a meta-framework for this problem, inspired by the recent success of highly parameter-efficient methods for fine-tuning. Our experiments show that differentially private adaptations of these approaches outperform previous private algorithms in three important dimensions: utility, privacy, and the computational and memory cost of private training. On many commonly studied datasets, the utility of private models approaches that of non-private models. For example, on the MNLI dataset we achieve an accuracy of $87.8\%$ using RoBERTa-Large and $83.5\%$ using RoBERTa-Base with a privacy budget of $\epsilon = 6.7$. In comparison, absent privacy constraints, RoBERTa-Large achieves an accuracy of $90.2\%$. Our findings are similar for natural language generation tasks. Privately fine-tuning with DART, GPT-2-Small, GPT-2-Medium, GPT-2-Large, and GPT-2-XL achieve BLEU scores of 38.5, 42.0, 43.1, and 43.8 respectively (privacy budget of $\epsilon = 6.8,\delta=$ 1e-5) whereas the non-private baseline is $48.1$. All our experiments suggest that larger models are better suited for private fine-tuning: while they are well known to achieve superior accuracy non-privately, we find that they also better maintain their accuracy when privacy is introduced., Comment: ICLR 2022. Code available at https://github.com/huseyinatahaninan/Differentially-Private-Fine-tuning-of-Language-Models

We propose a reparametrization scheme to address the challenges of applying differentially private SGD on large neural networks, which are 1) the huge memory cost of storing individual gradients, 2) the added noise suffering notorious dimensional dependence. Specifically, we reparametrize each weight matrix with two \emph{gradient-carrier} matrices of small dimension and a \emph{residual weight} matrix. We argue that such reparametrization keeps the forward/backward process unchanged while enabling us to compute the projected gradient without computing the gradient itself. To learn with differential privacy, we design \emph{reparametrized gradient perturbation (RGP)} that perturbs the gradients on gradient-carrier matrices and reconstructs an update for the original weight from the noisy gradients. Importantly, we use historical updates to find the gradient-carrier matrices, whose optimality is rigorously justified under linear regression and empirically verified with deep learning tasks. RGP significantly reduces the memory cost and improves the utility. For example, we are the first able to apply differential privacy on the BERT model and achieve an average accuracy of $83.9\%$ on four downstream tasks with $\epsilon=8$, which is within $5\%$ loss compared to the non-private baseline but enjoys much lower privacy leakage risk., Comment: Published as a conference paper in International Conference on Machine Learning (ICML 2021). Source code available at https://github.com/dayu11/Differentially-Private-Deep-Learning

The privacy leakage of the model about the training data can be bounded in the differential privacy mechanism. However, for meaningful privacy parameters, a differentially private model degrades the utility drastically when the model comprises a large number of trainable parameters. In this paper, we propose an algorithm \emph{Gradient Embedding Perturbation (GEP)} towards training differentially private deep models with decent accuracy. Specifically, in each gradient descent step, GEP first projects individual private gradient into a non-sensitive anchor subspace, producing a low-dimensional gradient embedding and a small-norm residual gradient. Then, GEP perturbs the low-dimensional embedding and the residual gradient separately according to the privacy budget. Such a decomposition permits a small perturbation variance, which greatly helps to break the dimensional barrier of private learning. With GEP, we achieve decent accuracy with reasonable computational cost and modest privacy guarantee for deep models. Especially, with privacy bound $\epsilon=8$, we achieve $74.9\%$ test accuracy on CIFAR10 and $95.1\%$ test accuracy on SVHN, significantly improving over existing results., Comment: Published as a conference paper at ICLR 2021. Source code available at https://github.com/dayu11/Gradient-Embedding-Perturbation

It is observed in the literature that data augmentation can significantly mitigate membership inference (MI) attack. However, in this work, we challenge this observation by proposing new MI attacks to utilize the information of augmented data. MI attack is widely used to measure the model's information leakage of the training set. We establish the optimal membership inference when the model is trained with augmented data, which inspires us to formulate the MI attack as a set classification problem, i.e., classifying a set of augmented instances instead of a single data point, and design input permutation invariant features. Empirically, we demonstrate that the proposed approach universally outperforms original methods when the model is trained with data augmentation. Even further, we show that the proposed approach can achieve higher MI attack success rates on models trained with some data augmentation than the existing methods on models trained without data augmentation. Notably, we achieve a 70.1% MI attack success rate on CIFAR10 against a wide residual network while the previous best approach only attains 61.9%. This suggests the privacy risk of models trained with data augmentation could be largely underestimated., Comment: AAAI Conference on Artificial Intelligence (AAAI-21). Source code available at: https://github.com/dayu11/MI_with_DA

Characterized by bulk Dirac or Weyl cones and surface Fermi-arc states, topological semimetals have sparked enormous research interest in recent years. The nanostructures, with large surface-to-volume ratio and easy field-effect gating, provide ideal platforms to detect and manipulate the topological quantum states. Exotic physical properties originating from these topological states endow topological semimetals attractive for future topological electronics (topotronics). For example, the linear energy dispersion relation is promising for broadband infrared photodetectors, the spin-momentum locking nature of topological surface states is valuable for spintronics, and the topological superconductivity is highly desirable for fault-tolerant qubits. For real-life applications, topological semimetals in the form of nanostructures are necessary in terms of convenient fabrication and integration. Here, we review the recent progresses in topological semimetal nanostructures and start with the quantum transport properties. Then topological semimetal-based electronic devices are introduced. Finally, we discuss several important aspects that should receive great effort in the future, including controllable synthesis, manipulation of quantum states, topological field effect transistors, spintronic applications, and topological quantum computation., Comment: Review Article

The notion of topological phases has been extended to higher-order and has been generalized to different dimensions. As a paradigm, Cd3As2 is predicted to be a higher-order topological semimetal, possessing three-dimensional (3D) bulk Dirac fermions, two-dimensional (2D) Fermi arcs, and one-dimensional (1D) hinge states. These topological states have different characteristic length scales in electronic transport, allowing to distinguish their properties when changing sample size. Here, we report an anomalous dimensional reduction of supercurrent transport by increasing the size of Dirac semimetal Cd3As2-based Josephson junctions. An evolution of the supercurrent quantum interferences from a standard Fraunhofer pattern to a superconducting quantum interference device (SQUID)-like one is observed when the junction channel length is increased. The SQUID-like interference pattern indicates the supercurrent flowing through the 1D hinges. The identification of 1D hinge states should be valuable for deeper understanding the higher-order topological phase in a 3D Dirac semimetal.

Gradient perturbation, widely used for differentially private optimization, injects noise at every iterative update to guarantee differential privacy. Previous work first determines the noise level that can satisfy the privacy requirement and then analyzes the utility of noisy gradient updates as in the non-private case. In contrast, we explore how privacy noise affects optimization property. We show that for differentially private convex optimization, the utility guarantee of differentially private (stochastic) gradient descent is determined by an \emph{expected curvature} rather than the minimum curvature. The \emph{expected curvature}, which represents the average curvature over the optimization path, is usually much larger than the minimum curvature. By using the \emph{expected curvature}, we show that gradient perturbation can achieve a significantly improved utility guarantee that can theoretically justify the advantage of gradient perturbation over other perturbation methods. Finally, our extensive experiments suggest that gradient perturbation with the advanced composition method indeed outperforms other perturbation approaches by a large margin, matching our theoretical findings., Comment: International Joint Conference on Artificial Intelligence (IJCAI) 2020; 7 pages, 2 figures, and 4 tables