Search

As large language models (LLMs) become increasingly prevalent across many real-world applications, understanding and enhancing their robustness to adversarial attacks is of paramount importance. Existing methods for identifying adversarial prompts tend to focus on specific domains, lack diversity, or require extensive human annotations. To address these limitations, we present Rainbow Teaming, a novel black-box approach for producing a diverse collection of adversarial prompts. Rainbow Teaming casts adversarial prompt generation as a quality-diversity problem, and uses open-ended search to generate prompts that are both effective and diverse. Focusing on the safety domain, we use Rainbow Teaming to target various state-of-the-art LLMs, including the Llama 2 and Llama 3 models. Our approach reveals hundreds of effective adversarial prompts, with an attack success rate exceeding 90% across all tested models. Furthermore, we demonstrate that fine-tuning models with synthetic data generated by the Rainbow Teaming method significantly enhances their safety without sacrificing general performance or helpfulness. We additionally explore the versatility of Rainbow Teaming by applying it to question answering and cybersecurity, showcasing its potential to drive robust open-ended self-improvement in a wide range of applications.

Captum is a comprehensive library for model explainability in PyTorch, offering a range of methods from the interpretability literature to enhance users' understanding of PyTorch models. In this paper, we introduce new features in Captum that are specifically designed to analyze the behavior of generative language models. We provide an overview of the available functionalities and example applications of their potential for understanding learned associations within generative language models.

Neural networks employ spurious correlations in their predictions, resulting in decreased performance when these correlations do not hold. Recent works suggest fixing pretrained representations and training a classification head that does not use spurious features. We investigate how spurious features are represented in pretrained representations and explore strategies for removing information about spurious features. Considering the Waterbirds dataset and a few pretrained representations, we find that even with full knowledge of spurious features, their removal is not straightforward due to entangled representation. To address this, we propose a linear autoencoder training method to separate the representation into core, spurious, and other features. We propose two effective spurious feature removal approaches that are applied to the encoding and significantly improve classification performance measured by worst group accuracy.

People capture photos and videos to relive and share memories of personal significance. Recently, media montages (stories) have become a popular mode of sharing these memories due to their intuitive and powerful storytelling capabilities. However, creating such montages usually involves a lot of manual searches, clicks, and selections that are time-consuming and cumbersome, adversely affecting user experiences. To alleviate this, we propose task-oriented dialogs for montage creation as a novel interactive tool to seamlessly search, compile, and edit montages from a media collection. To the best of our knowledge, our work is the first to leverage multi-turn conversations for such a challenging application, extending the previous literature studying simple media retrieval tasks. We collect a new dataset C3 (Conversational Content Creation), comprising 10k dialogs conditioned on media montages simulated from a large media collection. We take a simulate-and-paraphrase approach to collect these dialogs to be both cost and time efficient, while drawing from natural language distribution. Our analysis and benchmarking of state-of-the-art language models showcase the multimodal challenges present in the dataset. Lastly, we present a real-world mobile demo application that shows the feasibility of the proposed work in real-world applications. Our code and data will be made publicly available., Comment: 8 pages, 6 figures, 2 tables

Despite their wide adoption, the underlying training and memorization dynamics of very large language models is not well understood. We empirically study exact memorization in causal and masked language modeling, across model sizes and throughout the training process. We measure the effects of dataset size, learning rate, and model size on memorization, finding that larger language models memorize training data faster across all settings. Surprisingly, we show that larger models can memorize a larger portion of the data before over-fitting and tend to forget less throughout the training process. We also analyze the memorization dynamics of different parts of speech and find that models memorize nouns and numbers first; we hypothesize and provide empirical evidence that nouns and numbers act as a unique identifier for memorizing individual training examples. Together, these findings present another piece of the broader puzzle of trying to understand what actually improves as models get bigger.

Streamer discharges are often operated in a repetitively pulsed mode and are therefore influenced by species left over from the previous discharge, especially free electrons and ions. We have investigated these effects by applying two consecutive positive high voltage pulses of 200-700 ns duration to a point plane gap in artificial air, pure nitrogen, other nitrogen-oxygen mixtures and pure argon at pressures between 67 and 533 mbar. The pulses had pulse-to-pulse intervals ({\Delta}t) between 200 ns and 40 ms. We imaged both discharges with two ICCD cameras and combined this to a compound image. We observe for values of {\Delta}t below 0.5-15 {\mu}s (at 133 mbar, depending on gas mixture) that during the second pulse the streamers continue the paths of the first-pulse streamers. We call the maximal time for which this continuation still occurs the continuation time. For N$_{2}$ - O$_{2}$ mixtures, this time has a maximum at an oxygen concentration of about 0.2%. According to our plasma chemical modelling this maximum is determined by the electron loss rate which has a minimum around this oxygen concentration. Depending on oxygen concentration the dominant recombining positive ion is N$_{4}^{+}$, O$_2^+$ or O$_{4}^{+}$ where O$_{2}^{+}$ dominates around 0.2% O$_{2}$ and recombines slowest. For increasing values of {\Delta}t we observe that after the continuation phase first no new streamers occur at all, then new streamers show up that avoid the entire pre-ionized region. Next we see new thin streamers that follow the edges of the old channels. For larger {\Delta}t (10-200 {\mu}s) the new streamers start to increase in size and move to the centre of the old channel. Finally, around millisecond timescales the new channels are completely independent of the old channels., Comment: 27 pages, 17 figures

The high order fluid model developed in the preceding paper is employed here to study the propagation of negative planar streamer fronts in pure nitrogen. The model consists of the balance equations for electron density, average electron velocity, average electron energy and average electron energy flux. These balance equations have been obtained as velocity moments of Boltzmann's equation and are here coupled to the Poisson equation for the space charge electric field. Here the results of simulations with the high order model, with a PIC/MC (Particle in cell/Monte Carlo) model and with the first order fluid model based on the hydrodynamic drift-diffusion approximation are presented and compared. The comparison with the MC model clearly validates our high order fluid model, thus supporting its correct theoretical derivation and numerical implementation. The results of the first order fluid model with local field approximation, as usually used for streamer discharges, show considerable deviations. Furthermore, we study the inaccuracies of simulation results caused by an inconsistent implementation of transport data into our high order fluid model. We also demonstrate the importance of the energy flux term in the high order model by comparing with results where this term is neglected. Finally, results with an approximation for the high order tensor in the energy flux equation is found to agree well with the PIC/MC results for reduced electric fields up to 1000 Townsend, as considered in this work., Comment: 26 pages, 11 figures

Streamer discharges pose basic problems in plasma physics, as they are very transient, far from equilibrium and have high ionization density gradients; they appear in diverse areas of science and technology. The present paper focuses on the derivation of a high order fluid model for streamers. Using momentum transfer theory, the fluid equations are obtained as velocity moments of the Boltzmann equation; they are closed in the local mean energy approximation and coupled to the Poisson equation for the space charge generated electric field. The high order tensor in the energy flux equation is approximated by the product of two lower order moments to close the system. The average collision frequencies for momentum and energy transfer in elastic and inelastic collisions for electrons in molecular nitrogen are calculated from a multi term Boltzmann equation solution. We then discuss, in particular, (1) the correct implementation of transport data in streamer models; (2) the accuracy of the two term approximation for solving Boltzmann's equation in the context of streamer studies; and (3) the evaluation of the mean-energy-dependent collision rates for electrons required as an input in the high order fluid model. In the second paper in this sequence, we will discuss the solutions of the high order fluid model for streamers, based on model and input data derived in the present paper., Comment: 31 pages, 6 figures