Search

Fine-tuning large language models (LLMs) on limited tabular data for classification tasks can lead to \textit{fine-tuning multiplicity}, where equally well-performing models make conflicting predictions on the same inputs due to variations in the training process (i.e., seed, random weight initialization, retraining on additional or deleted samples). This raises critical concerns about the robustness and reliability of Tabular LLMs, particularly when deployed for high-stakes decision-making, such as finance, hiring, education, healthcare, etc. This work formalizes the challenge of fine-tuning multiplicity in Tabular LLMs and proposes a novel metric to quantify the robustness of individual predictions without expensive model retraining. Our metric quantifies a prediction's stability by analyzing (sampling) the model's local behavior around the input in the embedding space. Interestingly, we show that sampling in the local neighborhood can be leveraged to provide probabilistic robustness guarantees against a broad class of fine-tuned models. By leveraging Bernstein's Inequality, we show that predictions with sufficiently high robustness (as defined by our measure) will remain consistent with high probability. We also provide empirical evaluation on real-world datasets to support our theoretical results. Our work highlights the importance of addressing fine-tuning instabilities to enable trustworthy deployment of LLMs in high-stakes and safety-critical applications.

Spurious patterns refer to a mathematical association between two or more variables in a dataset that are not causally related. However, this notion of spuriousness, which is usually introduced due to sampling biases in the dataset, has classically lacked a formal definition. To address this gap, this work presents the first information-theoretic formalization of spuriousness in a dataset (given a split of spurious and core features) using a mathematical framework called Partial Information Decomposition (PID). Specifically, we disentangle the joint information content that the spurious and core features share about another target variable (e.g., the prediction label) into distinct components, namely unique, redundant, and synergistic information. We propose the use of unique information, with roots in Blackwell Sufficiency, as a novel metric to formally quantify dataset spuriousness and derive its desirable properties. We empirically demonstrate how higher unique information in the spurious features in a dataset could lead a model into choosing the spurious features over the core features for inference, often having low worst-group-accuracy. We also propose a novel autoencoder-based estimator for computing unique information that is able to handle high-dimensional image data. Finally, we also show how this unique information in the spurious feature is reduced across several dataset-based spurious-pattern-mitigation techniques such as data reweighting and varying levels of background mixing, demonstrating a novel tradeoff between unique information (spuriousness) and worst-group-accuracy., Comment: Accepted at ICML 2024 Workshop on Data-centric Machine Learning Research (DMLR): Datasets for Foundation Models

Counterfactual explanations find ways of achieving a favorable model outcome with minimum input perturbation. However, counterfactual explanations can also be exploited to steal the model by strategically training a surrogate model to give similar predictions as the original (target) model. In this work, we investigate model extraction by specifically leveraging the fact that the counterfactual explanations also lie quite close to the decision boundary. We propose a novel strategy for model extraction that we call Counterfactual Clamping Attack (CCA) which trains a surrogate model using a unique loss function that treats counterfactuals differently than ordinary instances. Our approach also alleviates the related problem of decision boundary shift that arises in existing model extraction attacks which treat counterfactuals as ordinary instances. We also derive novel mathematical relationships between the error in model approximation and the number of queries using polytope theory. Experimental results demonstrate that our strategy provides improved fidelity between the target and surrogate model predictions on several real world datasets.

Let $\mathbf{W}\in\mathbb{C}^{n\times n}$ be a {\it single-spiked} Wishart matrix in the class $\mathbf{W}\sim \mathcal{CW}_n(m,\mathbf{I}_n+ \theta \mathbf{v}\mathbf{v}^\dagger) $ with $m\geq n$, where $\mathbf{I}_n$ is the $n\times n$ identity matrix, $\mathbf{v}\in\mathbb{C}^{n\times 1}$ is an arbitrary vector with unit Euclidean norm, $\theta\geq 0$ is a non-random parameter, and $(\cdot)^\dagger$ represents the conjugate-transpose operator. Let $\mathbf{u}_1$ and $\mathbf{u}_n$ denote the eigenvectors corresponding to the samllest and the largest eigenvalues of $\mathbf{W}$, respectively. This paper investigates the probability density function (p.d.f.) of the random quantity $Z_{\ell}^{(n)}=\left|\mathbf{v}^\dagger\mathbf{u}_\ell\right|^2\in(0,1)$ for $\ell=1,n$. In particular, we derive a finite dimensional closed-form p.d.f. for $Z_{1}^{(n)}$ which is amenable to asymptotic analysis as $m,n$ diverges with $m-n$ fixed. It turns out that, in this asymptotic regime, the scaled random variable $nZ_{1}^{(n)}$ converges in distribution to $\chi^2_2/2(1+\theta)$, where $\chi_2^2$ denotes a chi-squared random variable with two degrees of freedom. This reveals that $\mathbf{u}_1$ can be used to infer information about the spike. On the other hand, the finite dimensional p.d.f. of $Z_{n}^{(n)}$ is expressed as a double integral in which the integrand contains a determinant of a square matrix of dimension $(n-2)$. Although a simple solution to this double integral seems intractable, for special configurations of $n=2,3$, and $4$, we obtain closed-form expressions., Comment: Revised to add analyses on real and complex singular Wishart matrices

Let $\mathbf{X}\in\mathbb{C}^{n\times m}$ ($m\geq n$) be a random matrix with independent columns each distributed as complex multivariate Gaussian with zero mean and {\it single-spiked} covariance matrix $\mathbf{I}_n+ \eta \mathbf{u}\mathbf{u}^*$, where $\mathbf{I}_n$ is the $n\times n$ identity matrix, {\color{blue}$\mathbf{u}\in\mathbb{C}^{n\times 1}$} is an arbitrary vector with unit Euclidean norm, $\eta\geq 0$ is a non-random parameter, and $(\cdot)^*$ represents the conjugate-transpose. This paper investigates the distribution of the random quantity $\kappa_{\text{SC}}^2(\mathbf{X})=\sum_{k=1}^n \lambda_k/\lambda_1$, where {\color{blue}$0\le \lambda_1\le \lambda_2\le \ldots\leq \lambda_n<\infty$} are the ordered eigenvalues of $\mathbf{X}\mathbf{X}^*$ (i.e., single-spiked Wishart matrix). This random quantity is intimately related to the so called {\it scaled condition number} or the Demmel condition number (i.e., $\kappa_{\text{SC}}(\mathbf{X})$) and the minimum eigenvalue of the fixed trace Wishart-Laguerre ensemble (i.e., $\kappa_{\text{SC}}^{-2}(\mathbf{X})$). In particular, we use an orthogonal polynomial approach to derive an exact expression for the probability density function of $\kappa_{\text{SC}}^2(\mathbf{X})$ which is amenable to asymptotic analysis as matrix dimensions grow large. Our asymptotic results reveal that, as $m,n\to\infty$ such that $m-n$ is fixed and when $\eta$ scales on the order of $1/n$, $\kappa_{\text{SC}}^2(\mathbf{X})$ scales on the order of $n^3$. In this respect we establish simple closed-form expressions for the limiting distributions. {\color{blue}It turns out that, as $m,n\to\infty$ such that $n/m\to c\in(0,1)$, properly centered $\kappa_{\text{SC}}^{2}(\mathbf{X})$ fluctuates on the scale $m^{\frac{1}{3}}$}.