Search

Low-Rank Adaptation (LoRA) has become ubiquitous for efficiently fine-tuning foundation models. However, federated fine-tuning using LoRA is challenging due to suboptimal updates arising from traditional federated averaging of individual adapters. Existing solutions either incur prohibitively high communication cost that scales linearly with the number of clients or suffer from performance degradation due to limited expressivity. We introduce Federated Silver Bullet (Fed-SB), a novel approach for federated fine-tuning of LLMs using LoRA-SB, a recently proposed low-rank adaptation method. LoRA-SB optimally aligns the optimization trajectory with the ideal low-rank full fine-tuning projection by learning a small square matrix (R) between adapters B and A, keeping other components fixed. Direct averaging of R guarantees exact updates, substantially reducing communication cost, which remains independent of the number of clients, and enables scalability. Fed-SB achieves state-of-the-art performance across commonsense reasoning, arithmetic reasoning, and language inference tasks while reducing communication costs by up to 230x. In private settings, Fed-SB further improves performance by (1) reducing trainable parameters, thereby lowering the noise required for differential privacy and (2) avoiding noise amplification introduced by other methods. Overall, Fed-SB establishes a new Pareto frontier in the tradeoff between communication and performance, offering an efficient and scalable solution for both private and non-private federated fine-tuning. Our code is publicly available at https://github.com/CERT-Lab/fed-sb., Comment: Raghav Singhal and Kaustubh Ponkshe contributed equally to this work

In the case of Type-2 AGNs, estimating the mass of the black hole is challenging. Understanding how galaxies form and evolve requires considerable insight into the mass of black holes. This work compared different classical and quantum machine learning (QML) algorithms for black hole mass estimation, wherein the classical algorithms are Linear Regression, XGBoost Regression, Random Forest Regressor, Support Vector Regressor (SVR), Lasso Regression, Ridge Regression, Elastic Net Regression, Bayesian Regression, Decision Tree Regressor, Gradient Booster Regressor, Classical Neural Networks, Gated Recurrent Unit (GRU), LSTM, Deep Residual Networks (ResNets) and Transformer-Based Regression. On the other hand, quantum algorithms including Hybrid Quantum Neural Networks (QNN), Quantum Long Short-Term Memory (Q-LSTM), Sampler-QNN, Estimator-QNN, Variational Quantum Regressor (VQR), Quantum Linear Regression(Q-LR), QML with JAX optimization were also tested. The results revealed that classical algorithms gave better R^2, MAE, MSE, and RMSE results than the quantum models. Among the classical models, LSTM has the best result with an accuracy of 99.77%. Estimator-QNN has the highest accuracy for quantum algorithms with an MSE of 0.0124 and an accuracy of 99.75%. This study ascertains both the strengths and weaknesses of the classical and the quantum approaches. As far as our knowledge goes, this work could pave the way for the future application of quantum algorithms in astrophysical data analysis., Comment: 29 pages, 12 Figures, 6 Tables

Large Language Models (LLMs) demonstrate remarkable capabilities in question answering (QA), but metrics for assessing their reliance on memorization versus retrieval remain underdeveloped. Moreover, while finetuned models are state-of-the-art on closed-domain tasks, general-purpose models like GPT-4o exhibit strong zero-shot performance. This raises questions about the trade-offs between memorization, generalization, and retrieval. In this work, we analyze the extent to which multimodal retrieval-augmented VLMs memorize training data compared to baseline VLMs. Using the WebQA benchmark, we contrast finetuned models with baseline VLMs on multihop retrieval and question answering, examining the impact of finetuning on data memorization. To quantify memorization in end-to-end retrieval and QA systems, we propose several proxy metrics by investigating instances where QA succeeds despite retrieval failing. Our results reveal the extent to which finetuned models rely on memorization. In contrast, retrieval-augmented VLMs have lower memorization scores, at the cost of accuracy (72% vs 52% on WebQA test set). As such, our measures pose a challenge for future work to reconcile memorization and generalization in both Open-Domain QA and joint Retrieval-QA tasks.

The robustness of large language models (LLMs) against knowledge conflicts in unimodal question answering systems has been well studied. However, the effect of conflicts in information sources on vision language models (VLMs) in multimodal settings has not yet been explored. In this work, we propose \segsub, a framework that applies targeted perturbations to image sources to study and improve the robustness of VLMs against three different types of knowledge conflicts, namely parametric, source, and counterfactual conflicts. Contrary to prior findings that showed that LLMs are sensitive to parametric conflicts arising from textual perturbations, we find VLMs are largely robust to image perturbation. On the other hand, VLMs perform poorly on counterfactual examples (<30% accuracy) and fail to reason over source conflicts (<1% accuracy). We also find a link between hallucinations and image context, with GPT-4o prone to hallucination when presented with highly contextualized counterfactual examples. While challenges persist with source conflicts, finetuning models significantly improves reasoning over counterfactual samples. Our findings highlight the need for VLM training methodologies that enhance their reasoning capabilities, particularly in addressing complex knowledge conflicts between multimodal sources.

Energy-energy correlators (EECs) within high energy jets serve as a key experimentally accessible quantity to probe the scale and structure of the quark-gluon plasma (QGP) in relativistic heavy-ion collisions. The CMS Collaboration's first measurement of the modification to the EEC within single inclusive jets in Pb+Pb collisions relative to p+p collisions reveals a significant enhancement at small angles, which may arise from jet transverse momentum $p_T$ selection biases due to jet energy loss. We investigate the dependence of jet EECs on the flavor of the initiating parton. The EEC distribution of a gluon jet is broader and the peak of transition from perturbative to non-perturbative regime occurs at a larger angle than a quark jet. Such flavor dependence leads to the different EECs for $\gamma$-jets and single inclusive jets due to their different flavor composition. It is also responsible for a colliding energy dependence of EECs of single inclusive jets at fixed jet energy. We also investigate the impact of flavor composition variation on the $p_T$ dependence of the jet EEC. We further propose that a change in the gluon jet fraction in A+A collisions compared to p+p can also contribute to a non-negligible enhancement of the medium modified EEC at small angles. Using the \textsc{Jewel} model, we predict the reduction of the gluon jet fraction in A+A collisions and estimate its impact on the EEC.

India is a diverse society with unique challenges in developing AI systems, including linguistic diversity, oral traditions, data accessibility, and scalability. Existing foundation models are primarily trained on English, limiting their effectiveness for India's population. Indic languages comprise only 1 percent of Common Crawl corpora despite India representing 18 percent of the global population, leading to linguistic biases. Thousands of regional languages, dialects, and code mixing create additional representation challenges due to sparse training data. We introduce Krutrim LLM, a 2 trillion token multilingual model designed for India's linguistic landscape. It incorporates the largest known Indic dataset, mitigating data scarcity and ensuring balanced performance across dialects. Krutrim outperforms or matches state-of-the-art models on Indic benchmarks while maintaining competitive English performance. Despite being significantly smaller in training flops, Krutrim LLM matches or exceeds models like LLAMA-2 on 10 out of 16 tasks, with an average score of 0.57 versus 0.55. This evidences Krutrim's flexible multilingual fluency across diverse linguistic contexts. Krutrim is integrated with real-time search to improve factual accuracy in conversational AI applications. This enhances accessibility for over 1 billion users worldwide. Through intentional design choices addressing data imbalances, Krutrim LLM signifies meaningful progress in building ethical, globally representative AI models.

Assistive agents performing household tasks such as making the bed or cooking breakfast often compute and execute actions that accomplish one task at a time. However, efficiency can be improved by anticipating upcoming tasks and computing an action sequence that jointly achieves these tasks. State-of-the-art methods for task anticipation use data-driven deep networks and Large Language Models (LLMs), but they do so at the level of high-level tasks and/or require many training examples. Our framework leverages the generic knowledge of LLMs through a small number of prompts to perform high-level task anticipation, using the anticipated tasks as goals in a classical planning system to compute a sequence of finer-granularity actions that jointly achieve these goals. We ground and evaluate our framework's abilities in realistic scenarios in the VirtualHome environment and demonstrate a 31% reduction in execution time compared with a system that does not consider upcoming tasks., Comment: Accepted to IEEE International Conference on Robotics and Automation (ICRA) 2024

Quantum oscillations in magnetization or resistivity are a defining feature of metals subject to an external magnetic field. The phenomenon is generally not expected in insulators without a Fermi surface. The observations of quantum oscillations in Kondo insulating materials have provided a rare counterexample and attracted much theoretical interest. However, the magnetic oscillations in correlated insulators remain poorly understood. Here we report the observations of resistivity quantum oscillations in an excitonic insulator realized in Coulomb-coupled electron-hole double layers with gate-tunability that allows the phenomenon to be explored in a more controllable fashion than in bulk materials. When the cyclotron energy of the electrons or holes is tuned to be comparable to or larger than the exciton binding energy, recurring transitions between excitonic insulators and electron-hole decoupled quantum Hall states are observed. Compressibility measurements show an oscillatory exciton binding energy as a function of magnetic field and electron-hole pair density. Coulomb drag measurements further reveal the formation of excitons with finite angular momentum. Our results are qualitatively captured by mean-field theory calculations. The study demonstrates a new platform for studying quantum oscillations in correlated insulators.

Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities. In response, we introduce Humanity's Last Exam (HLE), a multi-modal benchmark at the frontier of human knowledge, designed to be the final closed-ended academic benchmark of its kind with broad subject coverage. HLE consists of 2,700 questions across dozens of subjects, including mathematics, humanities, and the natural sciences. HLE is developed globally by subject-matter experts and consists of multiple-choice and short-answer questions suitable for automated grading. Each question has a known solution that is unambiguous and easily verifiable, but cannot be quickly answered via internet retrieval. State-of-the-art LLMs demonstrate low accuracy and calibration on HLE, highlighting a significant gap between current LLM capabilities and the expert human frontier on closed-ended academic questions. To inform research and policymaking upon a clear understanding of model capabilities, we publicly release HLE at https://lastexam.ai., Comment: 27 pages, 6 figures

Diffusion models produce impressive results in modalities ranging from images and video to protein design and text. However, generating samples with user-specified properties remains a challenge. Recent research proposes fine-tuning models to maximize rewards that capture desired properties, but these methods require expensive training and are prone to mode collapse. In this work, we propose Feynman Kac (FK) steering, an inference-time framework for steering diffusion models with reward functions. FK steering works by sampling a system of multiple interacting diffusion processes, called particles, and resampling particles at intermediate steps based on scores computed using functions called potentials. Potentials are defined using rewards for intermediate states and are selected such that a high value indicates that the particle will yield a high-reward sample. We explore various choices of potentials, intermediate rewards, and samplers. We evaluate FK steering on text-to-image and text diffusion models. For steering text-to-image models with a human preference reward, we find that FK steering a 0.8B parameter model outperforms a 2.6B parameter fine-tuned model on prompt fidelity, with faster sampling and no training. For steering text diffusion models with rewards for text quality and specific text attributes, we find that FK steering generates lower perplexity, more linguistically acceptable outputs and enables gradient-free control of attributes like toxicity. Our results demonstrate that inference-time scaling and steering of diffusion models, even with off-the-shelf rewards, can provide significant sample quality gains and controllability benefits. Code is available at https://github.com/zacharyhorvitz/Fk-Diffusion-Steering .

Generative models have enabled easy creation and generation of images of all kinds given a single prompt. However, this has also raised ethical concerns about what is an actual piece of content created by humans or cameras compared to model-generated content like images or videos. Watermarking data generated by modern generative models is a popular method to provide information on the source of the content. The goal is for all generated images to conceal an invisible watermark, allowing for future detection or identification. The Stable Signature finetunes the decoder of Latent Diffusion Models such that a unique watermark is rooted in any image produced by the decoder. In this paper, we present a novel adversarial fine-tuning attack that disrupts the model's ability to embed the intended watermark, exposing a significant vulnerability in existing watermarking methods. To address this, we further propose a tamper-resistant fine-tuning algorithm inspired by methods developed for large language models, tailored to the specific requirements of watermarking in LDMs. Our findings emphasize the importance of anticipating and defending against potential vulnerabilities in generative systems.

The realization of graphene has provided a bench-top laboratory for quantum electrodynamics. The low-energy excitations of graphene are two-dimensional massless Dirac fermions with opposite chiralities at the $\pm$K valleys of the graphene Brillouin zone. It has been speculated that the electron-electron interactions in graphene could spontaneously break the chiral symmetry to induce a finite mass for Dirac fermions, in analogue to dynamical mass generation in elementary particles. The phenomenon is also known as the relativistic Mott transition and has not been observed in pristine graphene because the interaction strength is insufficient. Here, we report the realization of strongly correlated artificial graphene and the observation of the relativistic Mott transition in twisted WSe2 tetralayers. Using magneto transport, we show that the first $\Gamma$-valley moir\'e valence band mimics the low-energy graphene band structure. At half-band filling, the system exhibits hallmarks of massless Dirac fermions, including an anomalous Landau fan originated from a $\pi$-Berry phase and a square-root density dependence of the cyclotron mass. We tune the interaction across the semimetal-insulator transition by reducing the twist angle below about 2.7 degrees. The emergent insulator is compatible with an antiferromagnetic Mott insulator. Our results open the possibility of studying strongly correlated Dirac fermions in a condensed matter system.

Traditional greedy tokenization methods have been a critical step in Natural Language Processing (NLP), influencing how text is converted into tokens and directly impacting model performance. While subword tokenizers like Byte-Pair Encoding (BPE) are widely used, questions remain about their optimality across model scales and languages. In this work, we demonstrate through extensive experiments that an optimal BPE configuration significantly reduces token count compared to greedy segmentation, yielding improvements in token-saving percentages and performance benefits, particularly for smaller models. We evaluate tokenization performance across various intrinsic and extrinsic tasks, including generation and classification. Our findings suggest that compression-optimized tokenization strategies could provide substantial advantages for multilingual and low-resource language applications, highlighting a promising direction for further research and inclusive NLP., Comment: LoResLM @ COLING 2025

Evaluating large language models (LLMs) on their linguistic reasoning capabilities is an important task to understand the gaps in their skills that may surface during large-scale adoption. In this work, we investigate the abilities of such models to perform abstract multilingual reasoning through the lens of linguistic puzzles on extremely low-resource languages. As these translation tasks involve inductive and deductive reasoning from reference instances, we examine whether diverse auxiliary demonstrations can be automatically induced from seed exemplars, through analogical prompting. We employ a two-stage procedure, first generating analogical exemplars with a language model, and then applying them in-context along with provided target language exemplars. Our results on the modeLing dataset show that analogical prompting is effective in eliciting models' knowledge of language grammar similarities, boosting the performance of GPT-4o by as much as 8.1% and Llama-3.1-405B-Instruct by 5.9% over chain-of-thought approaches. These gains are attributable to the analogical demonstrations, both when self-generated as well as when produced by weaker multilingual models. Furthermore, we demonstrate that our method generalizes to other tasks present in Linguistics Olympiad competitions, achieving sizable improvements across all problem types and difficulty levels included in the LINGOLY dataset with GPT-4o. We also report several findings about interesting phenomena which drive linguistic reasoning performance, suggesting that such puzzles are a valuable benchmark for new reasoning methods.

In high energy physics, self-supervised learning (SSL) methods have the potential to aid in the creation of machine learning models without the need for labeled datasets for a variety of tasks, including those related to jets -- narrow sprays of particles produced by quarks and gluons in high energy particle collisions. This study introduces an approach to learning jet representations without hand-crafted augmentations using a jet-based joint embedding predictive architecture (J-JEPA), which aims to predict various physical targets from an informative context. As our method does not require hand-crafted augmentation like other common SSL techniques, J-JEPA avoids introducing biases that could harm downstream tasks. Since different tasks generally require invariance under different augmentations, this training without hand-crafted augmentation enables versatile applications, offering a pathway toward a cross-task foundation model. We finetune the representations learned by J-JEPA for jet tagging and benchmark them against task-specific representations., Comment: 5 pages, 2 figures. Accepted to Machine Learning for Physical Sciences NeurIPS 2024 workshop

Low-rank adapters have become standard for efficiently fine-tuning large language models (LLMs), but they often fall short of achieving the performance of full fine-tuning. We propose a method, LoRA Silver Bullet or LoRA-SB, that approximates full fine-tuning within low-rank subspaces using a carefully designed initialization strategy. We theoretically demonstrate that the architecture of LoRA-XS, which inserts a learnable (r x r) matrix between B and A while keeping other matrices fixed, provides the precise conditions needed for this approximation. We leverage its constrained update space to achieve optimal scaling for high-rank gradient updates while removing the need for hyperparameter tuning. We prove that our initialization offers an optimal low-rank approximation of the initial gradient and preserves update directions throughout training. Extensive experiments across mathematical reasoning, commonsense reasoning, and language understanding tasks demonstrate that our approach exceeds the performance of standard LoRA while using \textbf{27-90} times fewer learnable parameters, and comprehensively outperforms LoRA-XS. Our findings establish that it is possible to simulate full fine-tuning in low-rank subspaces, and achieve significant efficiency gains without sacrificing performance. Our code is publicly available at https://github.com/RaghavSinghal10/lora-sb., Comment: Kaustubh Ponkshe and Raghav Singhal contributed equally to this work

Measurements of jet substructure in ultra-relativistic heavy-ion collisions indicate that interactions with the quark-gluon plasma quench the jet showering process. Modern data-driven methods have shown promise in probing these modifications in the jet's hard substructure. In this Letter, we present a machine learning framework to identify quenched jets while accounting for pileup, uncorrelated soft particle background, and detector effects; a more experimentally realistic and challenging scenario than previously addressed. Our approach leverages an interpretable sequential attention-based mechanism that integrates representations of individual jet constituents alongside global jet observables as features. The framework sets a new benchmark for tagging quenched jets with reduced model dependence.

The Needle-in-a-haystack (NIAH) test is a general task used to assess language models' (LMs') abilities to recall particular information from long input context. This framework however does not provide a means of analyzing what factors, beyond context length, contribute to LMs' abilities or inabilities to separate and recall needles from their haystacks. To provide a systematic means of assessing what features contribute to LMs' NIAH capabilities, we developed a synthetic benchmark called DENIAHL (Data-oriented Evaluation of NIAH for LLM's). Our work expands on previous NIAH studies by ablating NIAH features beyond typical context length including data type, size, and patterns. We find stark differences between GPT-3.5 and LLaMA 2-7B's performance on DENIAHL, and drops in recall performance when features like item size are increased, and to some degree when data type is changed from numbers to letters. This has implications for increasingly large context models, demonstrating factors beyond item-number impact NIAH capabilities.

We present parameter sets corresponding to new underlying event tunes for the Herwig7.3 Monte Carlo event generator. The existing Herwig tunes are in good agreement with LHC data, however, they are not typically designed for center-of-mass energies below $\sqrt{s}=300$ GeV. The tunes presented in this study can describe mid-rapidity data collected at the nominal RHIC energy of $\sqrt{s }=200$ GeV, as well as higher center-of-mass energies utilized by experiments elsewhere, such as the LHC. The base "New Haven" tune is developed by fitting minimum-bias simulations of proton-proton collisions to mid-rapidity identified hadron and jet data from the STAR experiment. The "Nashville" tune includes a separate set of parameters developed by tuning to Tevatron proton-antiproton data at $\sqrt{s}=300$, $900$ and $1960$ GeV from CDF, and LHC proton-proton measurements from CMS at $\sqrt{s}=7$ TeV, in addition to the STAR measurements. Both new tunes demonstrate significant improvements over the recommended default tune currently included in the latest version of Herwig for minimum bias production. As such, we advocate using these tunes for future simulation studies at mid-rapidity by the experimental collaborations at RHIC (STAR and sPHENIX) and the LHC (ATLAS, ALICE, CMS).

This article surveys convolution-based models including convolutional neural networks (CNNs), Conformers, ResNets, and CRNNs-as speech signal processing models and provide their statistical backgrounds and speech recognition, speaker identification, emotion recognition, and speech enhancement applications. Through comparative training cost assessment, model size, accuracy and speed assessment, we compare the strengths and weaknesses of each model, identify potential errors and propose avenues for further research, emphasizing the central role it plays in advancing applications of speech technologies.

Video understanding has witnessed significant progress with recent video foundation models demonstrating strong performance owing to self-supervised pre-training objectives; Masked Autoencoders (MAE) being the design of choice. Nevertheless, the majority of prior works that leverage MAE pre-training have focused on relatively short video representations (16 / 32 frames in length) largely due to hardware memory and compute limitations that scale poorly with video length due to the dense memory-intensive self-attention decoding. One natural strategy to address these challenges is to subsample tokens to reconstruct during decoding (or decoder masking). In this work, we propose an effective strategy for prioritizing tokens which allows training on longer video sequences (128 frames) and gets better performance than, more typical, random and uniform masking strategies. The core of our approach is an adaptive decoder masking strategy that prioritizes the most important tokens and uses quantized tokens as reconstruction objectives. Our adaptive strategy leverages a powerful MAGVIT-based tokenizer that jointly learns the tokens and their priority. We validate our design choices through exhaustive ablations and observe improved performance of the resulting long-video (128 frames) encoders over short-video (32 frames) counterparts. With our long-video masked autoencoder (LVMAE) strategy, we surpass state-of-the-art on Diving48 by 3.9 points and EPIC-Kitchens-100 verb classification by 2.5 points while relying on a simple core architecture and video-only pre-training (unlike some of the prior works that require millions of labeled video-text pairs or specialized encoders)., Comment: 10.5 pages of main paper, 25 pages total, 4 figures and 10 tables. To appear in NeurIPS'24

We study scattering in Ising Field Theory (IFT) using matrix product states and the time-dependent variational principle. IFT is a one-parameter family of strongly coupled non-integrable quantum field theories in 1+1 dimensions, interpolating between massive free fermion theory and Zamolodchikov's integrable massive $E_8$ theory. Particles in IFT may scatter either elastically or inelastically. In the post-collision wavefunction, particle tracks from all final-state channels occur in superposition; processes of interest can be isolated by projecting the wavefunction onto definite particle sectors, or by evaluating energy density correlation functions. Using numerical simulations we determine the time delay of elastic scattering and the probability of inelastic particle production as a function of collision energy. We also study the mass and width of the lightest resonance near the $E_8$ point in detail. Close to both the free fermion and $E_8$ theories, our results for both elastic and inelastic scattering are in good agreement with expectations from form-factor perturbation theory. Using numerical computations to go beyond the regime accessible by perturbation theory, we find that the high energy behavior of the two-to-two particle scattering probability in IFT is consistent with a conjecture of Zamolodchikov. Our results demonstrate the efficacy of tensor-network methods for simulating the real-time dynamics of strongly coupled quantum field theories in 1+1 dimensions., Comment: 16 + 12 pages, many spacetime pictures of scattering processes

Reservoir computing (RC), is a class of computational methods such as Echo State Networks (ESN) and Liquid State Machines (LSM) describe a generic method to perform pattern recognition and temporal analysis with any non-linear system. This is enabled by Reservoir Computing being a shallow network model with only Input, Reservoir, and Readout layers where input and reservoir weights are not learned (only the readout layer is trained). LSM is a special case of Reservoir computing inspired by the organization of neurons in the brain and generally refers to spike-based Reservoir computing approaches. LSMs have been successfully used to showcase decent performance on some neuromorphic vision and speech datasets but a common problem associated with LSMs is that since the model is more-or-less fixed, the main way to improve the performance is by scaling up the Reservoir size, but that only gives diminishing rewards despite a tremendous increase in model size and computation. In this paper, we propose two approaches for effectively ensembling LSM models - Multi-Length Scale Reservoir Ensemble (MuLRE) and Temporal Excitation Partitioned Reservoir Ensemble (TEPRE) and benchmark them on Neuromorphic-MNIST (N-MNIST), Spiking Heidelberg Digits (SHD), and DVSGesture datasets, which are standard neuromorphic benchmarks. We achieve 98.1% test accuracy on N-MNIST with a 3600-neuron LSM model which is higher than any prior LSM-based approach and 77.8% test accuracy on the SHD dataset which is on par with a standard Recurrent Spiking Neural Network trained by Backprop Through Time (BPTT). We also propose receptive field-based input weights to the Reservoir to work alongside the Multi-Length Scale Reservoir ensemble model for vision tasks. Thus, we introduce effective means of scaling up the performance of LSM models and evaluate them against relevant neuromorphic benchmarks

Shifts in data distribution can substantially harm the performance of clinical AI models. Hence, various methods have been developed to detect the presence of such shifts at deployment time. However, root causes of dataset shifts are varied, and the choice of shift mitigation strategies is highly dependent on the precise type of shift encountered at test time. As such, detecting test-time dataset shift is not sufficient: precisely identifying which type of shift has occurred is critical. In this work, we propose the first unsupervised dataset shift identification framework, effectively distinguishing between prevalence shift (caused by a change in the label distribution), covariate shift (caused by a change in input characteristics) and mixed shifts (simultaneous prevalence and covariate shifts). We discuss the importance of self-supervised encoders for detecting subtle covariate shifts and propose a novel shift detector leveraging both self-supervised encoders and task model outputs for improved shift detection. We report promising results for the proposed shift identification framework across three different imaging modalities (chest radiography, digital mammography, and retinal fundus images) on five types of real-world dataset shifts, using four large publicly available datasets., Comment: Code available at https://github.com/biomedia-mira/shift_identification

We present the results of the "Fast Calorimeter Simulation Challenge 2022" - the CaloChallenge. We study state-of-the-art generative models on four calorimeter shower datasets of increasing dimensionality, ranging from a few hundred voxels to a few tens of thousand voxels. The 31 individual submissions span a wide range of current popular generative architectures, including Variational AutoEncoders (VAEs), Generative Adversarial Networks (GANs), Normalizing Flows, Diffusion models, and models based on Conditional Flow Matching. We compare all submissions in terms of quality of generated calorimeter showers, as well as shower generation time and model size. To assess the quality we use a broad range of different metrics including differences in 1-dimensional histograms of observables, KPD/FPD scores, AUCs of binary classifiers, and the log-posterior of a multiclass classifier. The results of the CaloChallenge provide the most complete and comprehensive survey of cutting-edge approaches to calorimeter fast simulation to date. In addition, our work provides a uniquely detailed perspective on the important problem of how to evaluate generative models. As such, the results presented here should be applicable for other domains that use generative AI and require fast and faithful generation of samples in a large phase space., Comment: 204 pages, 100+ figures, 30+ tables

This study proposes an advanced Federated Learning (FL) framework designed to enhance data privacy and security in IoT environments by integrating Decentralized Attribute-Based Encryption (DABE), Homomorphic Encryption (HE), Secure Multi-Party Computation (SMPC), and Blockchain technology. Unlike traditional FL, our framework enables secure, decentralized authentication and encryption directly on IoT devices using DABE, allowing sensitive data to remain locally encrypted. Homomorphic Encryption permits computations on encrypted data, and SMPC ensures privacy in collaborative computations, while Blockchain technology provides transparent, immutable record-keeping for all transactions and model updates. Local model weights are encrypted and transmitted to fog layers for aggregation using HE and SMPC, then iteratively refined by the central server using differential privacy to safeguard against data leakage. This secure, privacy-preserving FL framework delivers a robust solution for efficient model training and real-time analytics across distributed IoT devices, offering significant advancements in secure decentralized learning for IoT applications.

We study agents acting in an unknown environment where the agent's goal is to find a robust policy. We consider robust policies as policies that achieve high cumulative rewards for all possible environments. To this end, we consider agents minimizing the maximum regret over different environment parameters, leading to the study of minimax regret. This research focuses on deriving information-theoretic bounds for minimax regret in Markov Decision Processes (MDPs) with a finite time horizon. Building on concepts from supervised learning, such as minimum excess risk (MER) and minimax excess risk, we use recent bounds on the Bayesian regret to derive minimax regret bounds. Specifically, we establish minimax theorems and use bounds on the Bayesian regret to perform minimax regret analysis using these minimax theorems. Our contributions include defining a suitable minimax regret in the context of MDPs, finding information-theoretic bounds for it, and applying these bounds in various scenarios.

We present a quantum computational framework for SU(2) lattice gauge theory, leveraging continuous variables instead of discrete qubits to represent the infinite-dimensional Hilbert space of the gauge fields. We consider a ladder as well as a two-dimensional grid of plaquettes, detailing the use of gauge fixing to reduce the degrees of freedom and simplify the Hamiltonian. We demonstrate how the system dynamics, ground states, and energy gaps can be computed using the continuous-variable approach to quantum computing. Our results indicate that it is feasible to study non-Abelian gauge theories with continuous variables, providing new avenues for understanding the real-time dynamics of quantum field theories., Comment: 26 pages

Quantum Hall phases have recently emerged as a platform to investigate non-Hermitian topology in condensed-matter systems. This platform is particularly interesting due to its tunability, which allows to modify the properties and topology of the investigated non-Hermitian phases by tuning external parameters of the system such as the magnetic field. Here, we show the tunability of non-Hermitian topology chirality in a graphene heterostructure using a gate voltage. By changing the charge carrier density, we unveil some novel properties specific to different quantum Hall regimes. First, we find that the best quantization of the non-Hermitian topological invariant is interestingly obtained at very high filling factor rather than on well-quantized quantum Hall plateaus. This is of particular importance for the efficient operation of devices based on non-Hermitian topology. Moreover, we observe an additional non-Hermitian topological phase in the insulating nu=0 quantum Hall plateau, which survives at lower fields than the opening of the nu=0 gap, confirming a recent prediction of a disorder-induced trivial phase. Our results evidence graphene as a promising platform for the study of non-Hermitian physics and of emergent phases in such topological devices.

Low-Rank Adaptation (LoRA) is a popular technique for efficient fine-tuning of foundation models. However, applying LoRA in federated learning environments, where data is distributed across multiple clients, presents unique challenges. Existing methods rely on traditional federated averaging of LoRA adapters, resulting in inexact updates. To address this, we propose Federated Exact LoRA, or FedEx-LoRA, which adds a residual error term to the pretrained frozen weight matrix. Our approach achieves exact updates with minimal computational and communication overhead, preserving LoRA's efficiency. We evaluate the method on various models across arithmetic reasoning, commonsense reasoning, natural language understanding and natural language generation tasks, showing consistent performance gains over state-of-the-art methods across multiple settings. Through extensive analysis, we quantify that the deviations in updates from the ideal solution are significant, highlighting the need for exact aggregation. Our method's simplicity, efficiency, and broad applicability position it as a promising solution for accurate and effective federated fine-tuning of foundation models. Our code is publicly available at https://github.com/RaghavSinghal10/fedex-lora., Comment: Raghav Singhal and Kaustubh Ponkshe contributed equally to this work. A version of the paper presented at FITML @ NeurIPS 2024

Star formation rates (SFRs) are a crucial observational tracer of galaxy formation and evolution. Spectroscopy, which is expensive, is traditionally used to estimate SFRs. This study tests the possibility of inferring SFRs of large samples of galaxies from only photometric data, using state-of-the-art machine learning and deep learning algorithms. The dataset adopted in this work is the one collected by Delli Veneri et al. (2019): it includes photometric data of more than 27 million galaxies coming from the Sloan Digital Sky Survey Data Release 7 (SDSS-DR7). The algorithms we implemented and tested for comparing the performances include Linear Regression, Long Short-Term Memory (LSTM) networks, Support Vector Regression (SVR), Random Forest Regressor, Decision Tree Regressor, Gradient Boosting Regressor, and classical deep learning models. Our results mention that the Linear Regression model predicted an impressive accuracy of 98.97 percent as measured by the Mean Absolute Error (MAE), demonstrating that machine-learning approaches can be effective when it comes to photometric SFR estimation. Besides, the paper also reported the results for other intelligent algorithms, which predicted the SFRs, providing a detailed comparison of the performance of different machine learning algorithms in the photometric SFR estimation. This study not only shows the estimated SFR from photometric data is promising but also opens a door toward the application of machine learning and deep learning in astrophysics., Comment: 19 pages, 6 figures, and 2 tables

The research focuses on determining the metallicity ([Fe/H]) predicted in the solar twin stars by using various regression modeling techniques which are, Random Forest, Linear Regression, Decision Tree, Support Vector, and Gradient Boosting. The data set that is taken into account here includes Stellar parameters and chemical abundances derived from a high-accuracy abundance catalog of solar twins from the GALAH survey. To overcome the missing values, intensive preprocessing techniques involving, imputation are done. Each model will subjected to training using different critical observables, which include, Mean Squared Error(MSE), Mean Absolute Error(MAE), Root Mean Squared Error(RMSE), and R-squared. Modeling is done by using, different feature sets like temperature: effective temperature(Teff), surface gravity: log g of 14-chemical-abundances namely, (([Na/Fe], [Mg/Fe], [Al/Fe], [Si/Fe], [Ca/Fe], [Sc/Fe], [Ti/Fe], [Cr/Fe], [Mn/Fe], [Ni/Fe], [Cu/Fe], [Zn/Fe], [Y/Fe], [Ba/Fe])). The target variable considered is the metallicity ([Fe/H]). The findings indicate that the Random Forest model achieved the highest accuracy, with an MSE of 0.001628 and an R-squared value of 0.9266. The results highlight the efficacy of ensemble methods in handling complex datasets with high dimensionality. Additionally, this study underscores the importance of selecting appropriate regression models for astronomical data analysis, providing a foundation for future research in predicting stellar properties with machine learning techniques., Comment: 16 pages, 7 figure, and 2 tables

Blazars, a class of active galactic nuclei (AGN) powered by supermassive black holes, are known for their remarkable variability across multiple timescales and wavelengths. Despite significant advancements in our understanding of AGN central engines, thanks to both ground- and space-based telescopes, the details of the mechanisms driving this variability remain elusive. The primary objective of this study is to constrain the X-ray variability properties of the TeV blazar PKS 2155-304. We conducted a comprehensive X-ray spectral and timing analysis, focusing on both long-term and intra-day variability (IDV), using data from 22 epochs of {\it XMM-Newton} observations collected over 15 years (2000 to 2014). For the timing analysis, we estimated the fractional variability, variability amplitude, minimum variability timescales, flux distribution, and power spectral density. In the spectral analysis, we fitted the X-ray spectra using power-law, log-parabola, and broken power-law models to determine the best-fitting parameters. We observed moderate IDV in the majority of the light curves. Seven out of the 22 observations showed a clear bimodal flux distribution, indicating the presence of two distinct flux states. Our analysis revealed a variable power spectral slope. Most hardness ratio plots did not show significant variation with flux, except for two observations, where the hardness ratio changed considerably with flux. The fitted X-ray spectra favored the broken power law model for the majority of observations, indicating break in the spectral profiles. The findings of this work shed light on the IDV of blazars, providing insights into the non-thermal jet processes that drive the observed flux variations.

Deep multimodal learning has shown remarkable success by leveraging contrastive learning to capture explicit one-to-one relations across modalities. However, real-world data often exhibits shared relations beyond simple pairwise associations. We propose M3CoL, a Multimodal Mixup Contrastive Learning approach to capture nuanced shared relations inherent in multimodal data. Our key contribution is a Mixup-based contrastive loss that learns robust representations by aligning mixed samples from one modality with their corresponding samples from other modalities thereby capturing shared relations between them. For multimodal classification tasks, we introduce a framework that integrates a fusion module with unimodal prediction modules for auxiliary supervision during training, complemented by our proposed Mixup-based contrastive loss. Through extensive experiments on diverse datasets (N24News, ROSMAP, BRCA, and Food-101), we demonstrate that M3CoL effectively captures shared multimodal relations and generalizes across domains. It outperforms state-of-the-art methods on N24News, ROSMAP, and BRCA, while achieving comparable performance on Food-101. Our work highlights the significance of learning shared relations for robust multimodal learning, opening up promising avenues for future research. Our code is publicly available at https://github.com/RaghavSinghal10/M3CoL., Comment: RK and RS contributed equally to this work, 20 Pages, 8 Figures, 9 Tables. Another version of the paper accepted at NeurIPS 2024 Workshop on Unifying Representations in Neural Models (UniReps)

Spectral clustering has proven effective in grouping speech representations for speaker diarization tasks, although post-processing the affinity matrix remains difficult due to the need for careful tuning before constructing the Laplacian. In this study, we present a novel pruning algorithm to create a sparse affinity matrix called \emph{spectral clustering on p-neighborhood retained affinity matrix} (SC-pNA). Our method improves on node-specific fixed neighbor selection by allowing a variable number of neighbors, eliminating the need for external tuning data as the pruning parameters are derived directly from the affinity matrix. SC-pNA does so by identifying two clusters in every row of the initial affinity matrix, and retains only the top $p\%$ similarity scores from the cluster containing larger similarities. Spectral clustering is performed subsequently, with the number of clusters determined as the maximum eigengap. Experimental results on the challenging DIHARD-III dataset highlight the superiority of SC-pNA, which is also computationally more efficient than existing auto-tuning approaches., Comment: Submitted to ICASSP 2025

In this report, we describe the speaker diarization (SD) and language diarization (LD) systems developed by our team for the Second DISPLACE Challenge, 2024. Our contributions were dedicated to Track 1 for SD and Track 2 for LD in multilingual and multi-speaker scenarios. We investigated different speech enhancement techniques, voice activity detection (VAD) techniques, unsupervised domain categorization, and neural embedding extraction architectures. We also exploited the fusion of various embedding extraction models. We implemented our system with the open-source SpeechBrain toolkit. Our final submissions use spectral clustering for both the speaker and language diarization. We achieve about $7\%$ relative improvement over the challenge baseline in Track 1. We did not obtain improvement over the challenge baseline in Track 2.

The first-ever measurement of energy correlators within inclusive jets produced in heavy-ion collisions, revealed by the CMS Collaboration, shows a clear enhancement at large angles relative to the proton-proton (p-p) baseline. However, interpreting this enhancement is complicated due to selection bias from energy loss, which also distorts the energy correlator heavy-ion to p-p ratio in the hadronization region, hindering our understanding of parton/hadron dynamics in a colored medium. In this Letter, we introduce a new ratio of energy correlator observables that removes the leading effects of selection bias from the two-point energy correlator spectrum (E2C). Pythia and Herwig simulations show that the impact of selection bias in the E2C is reduced by an order of magnitude, while sensitivity to any other medium modifications is retained. This quantity can be obtained directly from the experimental measurements presented by CMS, as illustrated in the accompanying note., Comment: 5 pages, 2 figures, 1 page of supplemental material with 2 additional figures. Typo corrected in Eq. 13, results unchanged

The radial time-independent Schr\"odinger equation is solved for the diatomic molecules: H2, LiH, HCl, CO, VH, CrH, CuLi, TiC, NiC, and ScN using the recently developed Nikiforov-Uvarov Functional Analysis (NUFA) method. A modified Morse potential is considered and the Pekeris approximation is used to accommodate the centrifugal term. Accurate energy eigenvalues and eigenfunction solutions are obtained for vibrational ($\mathit{n}$) and rotational ($\ell$) states. For H2, LiH, HCl, and CO, excellent agreement is observed between present values and literature, provided that the Pekeris approximation remains valid. For other molecules, a collection of low and high-lying states not found in literature are reported. The NUFA method is a simple, general and accurate approach that may be applied to other interatomic potentials., Comment: 10 pages, 4 figures, 4 tables

In physics, accurately modeling large-scale phenomena such as core-collapse supernovae, (CCSN), and neutron star mergers are computationally challenging and require solving large sets of partial and ordinary differential equations. Traditional methods used widely in the scientific community are predominantly implicit, which are approximations that often require drastic simplifications and can be computationally inefficient. This thesis presents results on a new software suite titled "Fast Explicit Neutrino Networks" or "FENN", that introduces a suite of algebraically stabilized explicit methods known as explicit asymptotic for modeling Neutrino Electron Scattering, (NES), presenting a novel approach that combines the stability of traditional methods with enhanced computational efficiency. Initial results show that FENN can deliver accurate solutions for neutrino networks at improved computational speeds. This thesis further covers new results for scaled networks beyond the constraints of standard energy groupings, as well as the dynamics of neutrino interactions such as the scattering of various neutrino flavors -- electron neutrinos ($\nu_e$), electron anti-neutrinos ($\bar{\nu}_e$), and muon/tau neutrinos ($\nu_{\mu,\tau}$) as well as their anti-particles ($\bar{\nu}_{\mu,\tau}$) -- off electrons., Comment: Bachelor of Science Thesis, 106 Pages

This study introduces an innovative approach to analyzing unlabeled data in high-energy physics (HEP) through the application of self-supervised learning (SSL). Faced with the increasing computational cost of producing high-quality labeled simulation samples at the CERN LHC, we propose leveraging large volumes of unlabeled data to overcome the limitations of supervised learning methods, which heavily rely on detailed labeled simulations. By pretraining models on these vast, mostly untapped datasets, we aim to learn generic representations that can be finetuned with smaller quantities of labeled data. Our methodology employs contrastive learning with augmentations on jet datasets to teach the model to recognize common representations of jets, addressing the unique challenges of LHC physics. Building on the groundwork laid by previous studies, our work demonstrates the critical ability of SSL to utilize large-scale unlabeled data effectively. We showcase the scalability and effectiveness of our models by gradually increasing the size of the pretraining dataset and assessing the resultant performance enhancements. Our results, obtained from experiments on two datasets -- JetClass, representing unlabeled data, and Top Tagging, serving as labeled simulation data -- show significant improvements in data efficiency, computational efficiency, and overall performance. These findings suggest that SSL can greatly enhance the adaptability of ML models to the HEP domain. This work opens new avenues for the use of unlabeled data in HEP and contributes to a better understanding the potential of SSL for scientific discovery., Comment: ACAT 2024 Proceedings

Radiofrequency harvesting using ambient wireless energy could be used to reduce the carbon footprint of electronic devices. However, ambient radiofrequency energy is weak (less than -20 dBm), and thermodynamic limits and high-frequency parasitic impedance restrict the performance of state-of-the-art radiofrequency rectifiers. Nanoscale spin rectifiers based on magnetic tunnel junctions have recently demonstrated high sensitivity, but suffer from a low a.c.-to-d.c. conversion efficiency (less than 1%). Here, we report a sensitive spin rectifier rectenna that can harvest ambient radiofrequency signals between -62 and -20 dBm. We also develop an on-chip co-planar waveguide-based spin rectifier array with a large zero-bias sensitivity (around 34,500 mV/mW) and high efficiency (7.81%). Self-parametric excitation driven by voltage-controlled magnetic anisotropy is a key mechanism that contributes to the performance of the spin-rectifier array. We show that these spin rectifiers can wirelessly power a sensor at a radiofrequency power of -27 dBm.

Electrodes for decoding speech from electromyography (EMG) are typically placed on the face, requiring adhesives that are inconvenient and skin-irritating if used regularly. We explore a different device form factor, where dry electrodes are placed around the neck instead. 11-word, multi-speaker voiced EMG classifiers trained on data recorded with this device achieve 92.7% accuracy. Ablation studies reveal the importance of having more than two electrodes on the neck, and phonological analyses reveal similar classification confusions between neck-only and neck-and-face form factors. Finally, speech-EMG correlation experiments demonstrate a linear relationship between many EMG spectrogram frequency bins and self-supervised speech representation dimensions.

We study the Krylov state complexity of the Sachdev-Ye-Kitaev (SYK) model for $N \le 28$ Majorana fermions with $q$-body fermion interaction with $q=4,6,8$ for a range of sparse parameter $k$. Using the peak of the Krylov complexity as a probe, we find change in behavior as we vary $k$ for various $q$. We argue that this captures the change from holographic to non-holographic behavior in the sparse SYK-type models such that model is holographic for all $k \ge k_{\text{min}}$., Comment: v2: 18 pages, 8 figures. Added references and N=28 results. v1: 16 pages, 7 figures. Comments welcome!

Reward-based finetuning is crucial for aligning language policies with intended behaviors (e.g., creativity and safety). A key challenge is to develop steerable language models that trade-off multiple (conflicting) objectives in a flexible and efficient manner. This paper presents Conditional Language Policy (CLP), a general framework for finetuning language models on multiple objectives. Building on techniques from multi-task training and parameter-efficient finetuning, CLP learn steerable models that effectively trade-off conflicting objectives at inference time. Notably, this does not require training or maintaining multiple models to achieve different trade-offs between the objectives. Through extensive experiments and ablations on two summarization datasets, we show that CLP learns steerable language models that outperform and Pareto-dominate the existing approaches for multi-objective finetuning., Comment: 40 pages. Findings of EMNLP 2024

We present the Bio-Silicon Intelligence System (BSIS), an innovative hybrid platform that integrates biological neural networks with silicon-based computing. The BSIS, a Physics-Informed Hybrid Hierarchical Reinforcement Learning State Machine, employs carbon nanotube-coated electrodes to interface rat brains with computational systems, enabling high-fidelity neural interfacing and bidirectional communication through self-organizing systems in both biological and silicon forms. Our system leverages both analogue and digital AI theory, incorporating concepts from computational theory, chaos theory, dynamical systems theory, physics, and quantum mechanics. Additionally, the BSIS replicates the neuronal dynamics typical of intelligent brain tissue, employing nonlinear operations underlying learning and information storage. Neural signals are read through the FreeEEG32 board and BrainFlow software, then features are extracted and mapped to game actions by tracking feature changes in continuous data. Metadata is encoded into both analogue and digital brain stimulation signals at the microvolt level using our proprietary software and hardware. The system employs a dual signaling approach for training the rat brain, incorporating a reward solution and sound as well as human-inaudible distress sounds. This paper details the design, theory, functionality, and technical specifications of the BSIS, highlighting its interdisciplinary approach and advanced technological integration.