Search

This report presents a comparative analysis of open-source vulnerability scanners for conversational large language models (LLMs). As LLMs become integral to various applications, they also present potential attack surfaces, exposed to security risks such as information leakage and jailbreak attacks. Our study evaluates prominent scanners - Garak, Giskard, PyRIT, and CyberSecEval - that adapt red-teaming practices to expose these vulnerabilities. We detail the distinctive features and practical use of these scanners, outline unifying principles of their design and perform quantitative evaluations to compare them. These evaluations uncover significant reliability issues in detecting successful attacks, highlighting a fundamental gap for future development. Additionally, we contribute a preliminary labelled dataset, which serves as an initial step to bridge this gap. Based on the above, we provide strategic recommendations to assist organizations choose the most suitable scanner for their red-teaming needs, accounting for customizability, test suite comprehensiveness, and industry-specific use cases., Comment: 15 pages, 11 figures

Fine-tuning is arguably the most straightforward way to tailor a pre-trained model (e.g., a foundation model) to downstream applications, but it also comes with the risk of losing valuable knowledge the model had learned in pre-training. For example, fine-tuning a pre-trained classifier capable of recognizing a large number of classes to master a subset of classes at hand is shown to drastically degrade the model's accuracy in the other classes it had previously learned. As such, it is hard to further use the fine-tuned model when it encounters classes beyond the fine-tuning data. In this paper, we systematically dissect the issue, aiming to answer the fundamental question, "What has been damaged in the fine-tuned model?" To our surprise, we find that the fine-tuned model neither forgets the relationship among the other classes nor degrades the features to recognize these classes. Instead, the fine-tuned model often produces more discriminative features for these other classes, even if they were missing during fine-tuning! {What really hurts the accuracy is the discrepant logit scales between the fine-tuning classes and the other classes}, implying that a simple post-processing calibration would bring back the pre-trained model's capability and at the same time unveil the feature improvement over all classes. We conduct an extensive empirical study to demonstrate the robustness of our findings and provide preliminary explanations underlying them, suggesting new directions for future theoretical analysis. Our code is available at https://github.com/OSU-MLB/Fine-Tuning-Is-Fine-If-Calibrated., Comment: The paper has been accepted to NeurIPS 2024. The first three authors contribute equally

This study presents a dataset consisting of 268 retinal images from 179 individuals, including 133 left-eye and 135 right-eye images, collected from Natasha Eye Care and Research Institute in Pune, Maharashtra, India. The images were captured using a nonmydriatic Optical Coherence Tomography Angiography (OCTA) device, specifically the Optovue Avanti Edition machine as per the protocol mentioned in this paper. Two ophthalmologists then annotated the images. This dataset can be used by researchers and doctors to develop automated diagnostic tools for early detection of diabetic retinopathy (DR)., Comment: 11 pages

In the study of auditory attention, it has been revealed that there exists a robust correlation between attended speech and elicited neural responses, measurable through electroencephalography (EEG). Therefore, it is possible to use the attention information available within EEG signals to guide the extraction of the target speaker in a cocktail party computationally. In this paper, we present a neuro-guided speaker extraction model, i.e. NeuroSpex, using the EEG response of the listener as the sole auxiliary reference cue to extract attended speech from monaural speech mixtures. We propose a novel EEG signal encoder that captures the attention information. Additionally, we propose a cross-attention (CA) mechanism to enhance the speech feature representations, generating a speaker extraction mask. Experimental results on a publicly available dataset demonstrate that our proposed model outperforms two baseline models across various evaluation metrics.

Camera traps are important tools in animal ecology for biodiversity monitoring and conservation. However, their practical application is limited by issues such as poor generalization to new and unseen locations. Images are typically associated with diverse forms of context, which may exist in different modalities. In this work, we exploit the structured context linked to camera trap images to boost out-of-distribution generalization for species classification tasks in camera traps. For instance, a picture of a wild animal could be linked to details about the time and place it was captured, as well as structured biological knowledge about the animal species. While often overlooked by existing studies, incorporating such context offers several potential benefits for better image understanding, such as addressing data scarcity and enhancing generalization. However, effectively incorporating such heterogeneous context into the visual domain is a challenging problem. To address this, we propose a novel framework that transforms species classification as link prediction in a multimodal knowledge graph (KG). This framework enables the seamless integration of diverse multimodal contexts for visual recognition. We apply this framework for out-of-distribution species classification on the iWildCam2020-WILDS and Snapshot Mountain Zebra datasets and achieve competitive performance with state-of-the-art approaches. Furthermore, our framework enhances sample efficiency for recognizing under-represented species., Comment: 12 pages, 5 figures

Security-Enhanced Linux (SELinux) is a Linux kernel module that allows for a role-based access control (RBAC) mechanism. It provides a fine-grained security framework enabling system administrators to define security policies at the system and application level. Whilst SELinux offers robust security features through a customisable, powerful RBAC model, its manual policy management is prone to error, leaving the system vulnerable to accidental misconfigurations or loopholes. We present a tool to automate the conversion of SELinux policies into satisfiability modulo theories (SMT), enabling the verification of the intended security configurations using automated theorem proving. Our tool is capable of flagging common policy misconfigurations by asserting consistency between supplied RBAC policies and the intended specification by the user in SMT. RBAC policies are inherently complicated to verify entirely. We envision that the automated tool presented here can be further extended to identify an even broader range of policy misconfigurations, relieving the burden of managing convoluted policies on system administrators., Comment: 10 pages (excluding appendices), 2 figures, 3 appendices

We propose a learning problem involving adapting a pre-trained source model to the target domain for classifying all classes that appeared in the source data, using target data that covers only a partial label space. This problem is practical, as it is unrealistic for the target end-users to collect data for all classes prior to adaptation. However, it has received limited attention in the literature. To shed light on this issue, we construct benchmark datasets and conduct extensive experiments to uncover the inherent challenges. We found a dilemma -- on the one hand, adapting to the new target domain is important to claim better performance; on the other hand, we observe that preserving the classification accuracy of classes missing in the target adaptation data is highly challenging, let alone improving them. To tackle this, we identify two key directions: 1) disentangling domain gradients from classification gradients, and 2) preserving class relationships. We present several effective solutions that maintain the accuracy of the missing classes and enhance the overall performance, establishing solid baselines for holistic transfer of pre-trained models with partial target data., Comment: Accepted to NeurIPS 2023 main track

HISPEC is a new, high-resolution near-infrared spectrograph being designed for the W.M. Keck II telescope. By offering single-shot, R=100,000 between 0.98 - 2.5 um, HISPEC will enable spectroscopy of transiting and non-transiting exoplanets in close orbits, direct high-contrast detection and spectroscopy of spatially separated substellar companions, and exoplanet dynamical mass and orbit measurements using precision radial velocity monitoring calibrated with a suite of state-of-the-art absolute and relative wavelength references. MODHIS is the counterpart to HISPEC for the Thirty Meter Telescope and is being developed in parallel with similar scientific goals. In this proceeding, we provide a brief overview of the current design of both instruments, and the requirements for the two spectrographs as guided by the scientific goals for each. We then outline the current science case for HISPEC and MODHIS, with focuses on the science enabled for exoplanet discovery and characterization. We also provide updated sensitivity curves for both instruments, in terms of both signal-to-noise ratio and predicted radial velocity precision., Comment: 25 pages, 9 figures. To appear in the Proceedings of SPIE: Techniques and Instrumentation for Detection of Exoplanets XI, vol. 12680 (2023)

Building joint representations across images and text is an essential step for tasks such as Visual Question Answering and Video Question Answering. In this work, we find that the representations must not only jointly capture features from both modalities but should also be diverse for better generalization performance. To this end, we propose joint vision-language representation learning by diversifying the tokenization learning process, enabling tokens that are sufficiently disentangled from each other to be learned from both modalities. We observe that our approach outperforms the baseline models in a majority of settings and is competitive with state-of-the-art methods., Comment: Accepted to Transformers for Vision (T4V) workshop, CVPR 2023; 7 pages, 5 figures

Ayyer and Linusson studied correlations in the multispecies TASEP on a ring (Trans AMS, 2017) using a combinatorial analysis of the multiline queues construction defined by Ferrari and Martin (AOP, 2008). It is natural to explore whether an analogous application of appropriate multiline queues could give similar results for the partially asymmetric case. In this paper, we solve this problem of correlations of adjacent particles on the first two sites in the multispecies PASEP on a finite ring. We use the multiline processes defined by Martin (EJP, 2020), the dynamics of which also depend on the asymmetry parameter $q$, to compute the correlations.

KOSMOS is a low-resolution, long-slit, optical spectrograph that has been upgraded at the University of Washington for its move from Kitt Peak National Observatory's Mayall 4m telescope to the Apache Point Observatory's ARC 3.5m telescope. One of the additions to KOSMOS is a slitviewer, which requires the fabrication of reflective slits, as KOSMOS previously used matte slits machined via wire EDM. We explore a novel method of slit fabrication using nanofabrication methods and compare the slit edge roughness, width uniformity, and the resulting scattering of the new fabricated slits to the original slits. We find the kerf surface of the chemically-etched reflective silicon slits are generally smoother than the machined matte slits, with an upper limit average roughness of 0.42 $\pm$ 0.03 $\mu$m versus 1.06 $\pm$ 0.04 $\mu$m respectively. The etched slits have width standard deviations of 6 $\pm$ 3 $\mu$m versus 10 $\pm$ 6 $\mu$m, respectively. The scattering for the chemically-etched slits is higher than that of the machined slits, showing that the reflectivity is the major contributor to scattering, not the roughness. This scattering, however, can be effectively reduced to zero with proper background subtraction. As slit widths increase, scattering increases for both types of slits, as expected. Future work will consist of testing and comparing the throughput and spectrophotometric data quality of these nanofabricated slits to the machined slits with on-sky data, in addition to making the etched slits more robust against breakage and finalizing the slit manufacturing process., Comment: 36 pages, 12 figures, accepted by SPIE

Photo-rechargeable (solar) battery can be considered as an energy harvesting cum storage system, where it can charge the conventional metal-ion battery using light instead of electricity, without having other parasitic reactions. Here we demonstrate a two-electrode lithium ion solar battery with multifaceted TiS2-TiO2 hybrid sheets as cathode. Choice of TiS2-TiO2 electrode ensures the formation of a type II semiconductor heterostructure while the lateral heterostructure geometry ensures high mass/charge transfer and light interactions with the electrode. TiS2 has a higher lithium binding energy (1.6 eV) than TiO2 (1.03 eV), ensuring the possibilities of higher amount of Li ion insertion to TiS2 and hence the maximum recovery with the photocharging, as further confirmed by the experiments. Apart from the demonstration of solar solid-state batteries, the charging of lithium ion full cell with light indicates the formation of lithium intercalated graphite compounds, ensuring the charging of the battery without any other parasitic reactions at the electrolyte or electrode-electrolyte interfaces. Possible mechanisms proposed here for the charging and discharging processes of solar batteries, based on our experimental and theoretical results, indicate the potential of such systems in forthcoming era of renewable energies.