Search

Continual learning (CL) learns a sequence of tasks incrementally. This paper studies the challenging CL setting of class-incremental learning (CIL). CIL has two key challenges: catastrophic forgetting (CF) and inter-task class separation (ICS). Despite numerous proposed methods, these issues remain persistent obstacles. This paper proposes a novel CIL method, called Kernel Linear Discriminant Analysis (KLDA), that can effectively avoid CF and ICS problems. It leverages only the powerful features learned in a foundation model (FM). However, directly using these features proves suboptimal. To address this, KLDA incorporates the Radial Basis Function (RBF) kernel and its Random Fourier Features (RFF) to enhance the feature representations from the FM, leading to improved performance. When a new task arrives, KLDA computes only the mean for each class in the task and updates a shared covariance matrix for all learned classes based on the kernelized features. Classification is performed using Linear Discriminant Analysis. Our empirical evaluation using text and image classification datasets demonstrates that KLDA significantly outperforms baselines. Remarkably, without relying on replay data, KLDA achieves accuracy comparable to joint training of all classes, which is considered the upper bound for CIL performance. The KLDA code is available at https://github.com/salehmomeni/klda.

Existing continual learning (CL) methods mainly rely on fine-tuning or adapting large language models (LLMs). They still suffer from catastrophic forgetting (CF). Little work has been done to exploit in-context learning (ICL) to leverage the extensive knowledge within LLMs for CL without updating any parameters. However, incrementally learning each new task in ICL necessitates adding training examples from each class of the task to the prompt, which hampers scalability as the prompt length increases. This issue not only leads to excessively long prompts that exceed the input token limit of the underlying LLM but also degrades the model's performance due to the overextended context. To address this, we introduce InCA, a novel approach that integrates an external continual learner (ECL) with ICL to enable scalable CL without CF. The ECL is built incrementally to pre-select a small subset of likely classes for each test instance. By restricting the ICL prompt to only these selected classes, InCA prevents prompt lengths from becoming excessively long, while maintaining high performance. Experimental results demonstrate that InCA significantly outperforms existing CL baselines, achieving substantial performance gains.

In this paper, we reported our experiments with various strategies to improve code-mixed humour and sarcasm detection. We did all of our experiments for Hindi-English code-mixed scenario, as we have the linguistic expertise for the same. We experimented with three approaches, namely (i) native sample mixing, (ii) multi-task learning (MTL), and (iii) prompting very large multilingual language models (VMLMs). In native sample mixing, we added monolingual task samples in code-mixed training sets. In MTL learning, we relied on native and code-mixed samples of a semantically related task (hate detection in our case). Finally, in our third approach, we evaluated the efficacy of VMLMs via few-shot context prompting. Some interesting findings we got are (i) adding native samples improved humor (raising the F1-score up to 6.76%) and sarcasm (raising the F1-score up to 8.64%) detection, (ii) training MLMs in an MTL framework boosted performance for both humour (raising the F1-score up to 10.67%) and sarcasm (increment up to 12.35% in F1-score) detection, and (iii) prompting VMLMs couldn't outperform the other approaches. Finally, our ablation studies and error analysis discovered the cases where our model is yet to improve. We provided our code for reproducibility., Comment: 26 pages; under review

Dynamical Ising machines achieve accelerated solving of complex combinatorial optimization problems by remapping the convergence to the ground state of the classical spin networks to the evolution of specially constructed continuous dynamical systems. The main adapted principle of constructing such systems is based on requiring that, on the one hand, the system converges to a binary state and, on the other hand, the system's energy in such states mimics the classical Ising Hamiltonian. The emergence of binary-like states is regarded to be an indispensable feature of dynamical Ising machines as it establishes the relation between the machine's continuous terminal state and the inherently discrete solution of a combinatorial optimization problem. This is emphasized by problems where the unknown quantities are represented by spin complexes, for example, the graph coloring problem. In such cases, an imprecise mapping of the continuous states to spin configurations may lead to invalid solutions requiring intensive post-processing. In contrast to such an approach, we show that there exists a class of non-binary dynamical Ising machines without the incongruity between the continuous character of the machine's states and the discreteness of the spin states. We demonstrate this feature by applying such a machine to the problems of finding proper graph coloring, constructing Latin squares, and solving Sudoku puzzles. Thus, we demonstrate that the information characterizing discrete states can be unambiguously presented in essentially continuous dynamical systems. This opens new opportunities in the realization of scalable electronic accelerators of combinatorial optimization., Comment: 21 pages

We study user history modeling via Transformer encoders in deep learning recommendation models (DLRM). Such architectures can significantly improve recommendation quality, but usually incur high latency cost necessitating infrastructure upgrades or very small Transformer models. An important part of user history modeling is early fusion of the candidate item and various methods have been studied. We revisit early fusion and compare concatenation of the candidate to each history item against appending it to the end of the list as a separate item. Using the latter method, allows us to reformulate the recently proposed amortized history inference algorithm M-FALCON \cite{zhai2024actions} for the case of DLRM models. We show via experimental results that appending with cross-attention performs on par with concatenation and that amortization significantly reduces inference costs. We conclude with results from deploying this model on the LinkedIn Feed and Ads surfaces, where amortization reduces latency by 30\% compared to non-amortized inference., Comment: 5 pages, 3 figures, WWW 2025

We present SNOWS, a one-shot post-training pruning framework aimed at reducing the cost of vision network inference without retraining. Current leading one-shot pruning methods minimize layer-wise least squares reconstruction error which does not take into account deeper network representations. We propose to optimize a more global reconstruction objective. This objective accounts for nonlinear activations deep in the network to obtain a better proxy for the network loss. This nonlinear objective leads to a more challenging optimization problem -- we demonstrate it can be solved efficiently using a specialized second-order optimization framework. A key innovation of our framework is the use of Hessian-free optimization to compute exact Newton descent steps without needing to compute or store the full Hessian matrix. A distinct advantage of SNOWS is that it can be readily applied on top of any sparse mask derived from prior methods, readjusting their weights to exploit nonlinearities in deep feature representations. SNOWS obtains state-of-the-art results on various one-shot pruning benchmarks including residual networks and Vision Transformers (ViT/B-16 and ViT/L-16, 86m and 304m parameters respectively)., Comment: 10 pages excl. appendix

Robotic prostheses and exoskeletons can do wonders compared to their non-robotic counterpart. However, in a cost-soaring world where 1 in every 10 patients has access to normal medical prostheses, access to advanced ones is, unfortunately, extremely limited especially due to their high cost, a significant portion of which is contributed to by the diagnosis and controlling units. However, affordability is often not a major concern for developing such devices as with cost reduction, performance is also found to be deducted due to the cost vs. performance trade-off. Considering the gravity of such circumstances, the goal of this research was to propose an affordable wearable real-time gait diagnosis unit (GDU) aimed at robotic prostheses and exoskeletons. As a proof of concept, it has also developed the GDU prototype which leveraged TinyML to run two parallel quantized int8 models into an ESP32 NodeMCU development board (7.30 USD) to effectively classify five gait scenarios (idle, walk, run, hopping, and skip) and generate an anomaly score based on acceleration data received from two attached IMUs. The developed wearable gait diagnosis stand-alone unit could be fitted to any prosthesis or exoskeleton and could effectively classify the gait scenarios with an overall accuracy of 92% and provide anomaly scores within 95-96 ms with only 3 seconds of gait data in real-time.

This paper presents L-VITeX, a lightweight visual intuition system for terrain exploration designed for resource-constrained robots and swarms. L-VITeX aims to provide a hint of Regions of Interest (RoIs) without computationally expensive processing. By utilizing the Faster Objects, More Objects (FOMO) tinyML architecture, the system achieves high accuracy (>99%) in RoI detection while operating on minimal hardware resources (Peak RAM usage < 50 KB) with near real-time inference (<200 ms). The paper evaluates L-VITeX's performance across various terrains, including mountainous areas, underwater shipwreck debris regions, and Martian rocky surfaces. Additionally, it demonstrates the system's application in 3D mapping using a small mobile robot run by ESP32-Cam and Gaussian Splats (GS), showcasing its potential to enhance exploration efficiency and decision-making.

Lung diseases have become a prevalent problem throughout the United States, affecting over 34 million people. Accurate and timely diagnosis of the different types of lung diseases is critical, and Artificial Intelligence (AI) methods could speed up these processes. A dual-stage vision transformer is built throughout this research by integrating a Vision Transformer (ViT) and a Swin Transformer to classify 14 different lung diseases from X-ray scans of patients with these diseases. The proposed model achieved an accuracy of 92.06\% when making predictions on an unseen testing subset of the dataset after data preprocessing and training the neural network. The model showed promise for accurately classifying lung diseases and diagnosing patients who suffer from these harmful diseases., Comment: 3 pages, 3 figures, Applied to the IEEE MetroCon 2024 Conference

The exponential growth in computational power and accessibility has transformed the complexity and scale of bioinformatics research, necessitating standardized documentation for transparency, reproducibility, and regulatory compliance. The IEEE BioCompute Object (BCO) standard addresses this need but faces adoption challenges due to the overhead of creating compliant documentation, especially for legacy research. This paper presents a novel approach to automate the creation of BCOs from scientific papers using Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs). We describe the development of the BCO assistant tool that leverages RAG to extract relevant information from source papers and associated code repositories, addressing key challenges such as LLM hallucination and long-context understanding. The implementation incorporates optimized retrieval processes, including a two-pass retrieval with re-ranking, and employs carefully engineered prompts for each BCO domain. We discuss the tool's architecture, extensibility, and evaluation methods, including automated and manual assessment approaches. The BCO assistant demonstrates the potential to significantly reduce the time and effort required for retroactive documentation of bioinformatics research while maintaining compliance with the standard. This approach opens avenues for AI-assisted scientific documentation and knowledge extraction from publications thereby enhancing scientific reproducibility. The BCO assistant tool and documentation is available at https://biocompute-objects.github.io/bco-rag/., Comment: 21 pages, 8 figures

Establishing universal features of spoof-plasmonic systems beyond spectral properties is challenging due to the complexity of the specific physical realizations of spoof-plasmonic channels. We introduce a simple 1D scalar model reproducing the key properties of spoof-plasmonic channels and investigate manifestations of plasmonic-like features when only a few local resonances are present. We show that the channel between the source applied to the interior of the structure and the terminal ends (output ports) effectively comprises two subchannels. The activation of one of the subchannels depends on the spatial variation of the source, and therefore, the contribution of this subchannel in conventional systems is small if the source occupies a subwavelength region. We show that, in spoof-plasmonic structures, the activation of this subchannel can enhance significantly in the frequency region where the spoof-plasmonic effects are prominent. This demonstrates that even a few local scattering resonances may strongly impact the flow of wave-carried information., Comment: 9 pages, 4 figures

Coherent Diffraction Imaging (CDI) is an experimental technique to gain images of isolated structures by recording the light scattered off the sample. In principle, the sample density can be recovered from the scattered light field through a straightforward Fourier Transform operation. However, only the amplitude of the field is recorded, while the phase is lost during the measurement process and has to be retrieved by means of suitable, well-established, phase retrieval algorithms. In this work we present SPRING, an analysis framework tailored on X-ray Free Electron Laser (XFEL) diffraction data that implements the Memetic Phase Retrieval method to mitigate the shortcomings of conventional algorithms. We benchmark the approach on experimental data acquired in two experimental campaigns at SwissFEL and European XFEL. Imaging results on isolated nanostructures reveal unprecedented stability and resilience of the algorithm's behavior on the input parameters, as well as the capability of identifying the solution in conditions hardly treatable so far with conventional methods. A user-friendly implementation of SPRING is released as open-source software, aiming at being a reference tool for the coherent diffraction imaging community at XFEL and synchrotron facilities., Comment: 30 pages, 13 figures. Software released as open source. Authors list updated. Using revtex document class. New section on comparison with conventional approaches. New supplemental material

We investigate one and two texture zeros in the neutrino mass matrix using the latest oscillation data through a bottom-up approach. In this context, we begin by estimating the detailed features of each matrix element by varying the CP violating phases within $(0,\,2\pi)$, the lowest neutrino mass within $(0,\,1)\,{\rm eV}$ and the neutrino oscillation parameters such as three mixing angles and the two mass squared differences within $3\sigma$ of their central values. We find that for normal ordering, only $ee$, $e\mu$ and $e\tau$ elements of the mass matrix can vanish, whereas, for inverted ordering, five elements -- $e\mu$, $e\tau$, $\mu\mu$, $\mu\tau$ and $\tau\tau$ -- can vanish. For two texture zeros, only $(ee,\, e\mu = 0)$ and $(ee,\, e\tau =0)$ are allowed in case of normal ordering. For a particular vanishing element, we also estimate the range of the lowest neutrino mass and the CP violating phases. In particular, very interesting correlation among the CP violating phases and the lowest neutrino mass is obtained for each vanishing cases., Comment: 24 pages

Drag and diffusion coefficients of the Heavy Quarks (HQs), such as charm and bottom, are one of the prime tools for discerning the properties of the deconfined QCD medium created in the Heavy Ion Collisions experiments. The innate non-perturbative nature of the QCD medium renders it imperative to estimate the transport coefficients in that domain. The present work evaluates the drag and diffusion coefficients of the moving HQ interacting with the medium particles via two-body collisional and three-body radiative processes to the first order in opacity by employing Gribov mechanism for the non-perturbative regime. We proffer the latest results of the HQ transport coefficients computed for the non-perturbative and non-eikonal gluon radiation off the HQ. The calculations show significant increment of the transport coefficients with the increasing non-eikonality by juxtaposing the results with those of the pertubative and eikonal regions. We hope to shed fresh light towards explaining the experimental data on the nuclear modification facor, $R_{AA}$, the elliptic flow, $v_2$ of the HQ by advocating the importance of the non-eikonality of the gluon radiation off the HQ., Comment: 9 pages, 8 figures

We give a general existence and convergence result for interacting particle systems on locally finite graphs with possibly unbounded degrees or jump rates. We allow the local state space to be Polish, and the jumps at a site to affect the states of its neighbours. The two common assumptions on interacting particle systems are uniform bounds on degrees and jump rates. In this paper, we relax these assumptions and allow for vertices with high degrees or rapid jumps. We introduce new assumptions ensuring that such vertices are placed sufficiently apart from each other and hence the process does not explode. Our assumptions involve finitude of certain weighted connective constants on the $2$-step graph of the underlying graph and our proofs proceed by showing that these assumptions imply non-percolation of the Poisson graphical construction. For some random graph models, if the jump rates are bounded by powers of vertex degrees, we give readily verifiable sufficient conditions on the underlying graph itself, under which our assumptions hold almost surely. These conditions involve exponential growth of the average number of self-avoiding walks from each vertex and that of moments of the vertex degrees. Using these conditions, we show the existence of interacting particle systems with unbounded jump rates like sandpile models on random graphs which can lack uniform bounds on degrees almost surely, e.g., long-range percolations on quasi-transitive graphs, and geometric random graphs on Delone sets., Comment: 28 pages. Corrected and revised. Uniform lower bound assumption on jump rates removed. Examples on Delone sets added

In this paper, we present a framework for generic decoding of convolutional codes, which allows us to do cryptanalysis of code-based systems that use convolutional codes. We then apply this framework to information set decoding, study success probabilities and give tools to choose variables. Finally, we use this to attack two cryptosystems based on convolutional codes. In the first, our code recovered about 74% of errors in less than 10 hours each, and in the second case, we give experimental evidence that 80% of the errors can be recovered in times corresponding to about 60 bits of operational security, with some instances being significantly lower.

Large Language Models (LLMs) have excelled at language understanding and generating human-level text. However, even with supervised training and human alignment, these LLMs are susceptible to adversarial attacks where malicious users can prompt the model to generate undesirable text. LLMs also inherently encode potential biases that can cause various harmful effects during interactions. Bias evaluation metrics lack standards as well as consensus and existing methods often rely on human-generated templates and annotations which are expensive and labor intensive. In this work, we train models to automatically create adversarial prompts to elicit biased responses from target LLMs. We present LLM- based bias evaluation metrics and also analyze several existing automatic evaluation methods and metrics. We analyze the various nuances of model responses, identify the strengths and weaknesses of model families, and assess where evaluation methods fall short. We compare these metrics to human evaluation and validate that the LLM-as-a-Judge metric aligns with human judgement on bias in response generation., Comment: 6 pages paper content, 17 pages of appendix

Let $f$ be a real polynomial with irrational leading co-efficient. In this article, we derive distribution of $f(n)$ modulo one for all $n$ with at least three divisors and also we study distribution of $f(n)$ for all square-free $n$ with at least two prime factors. We study exponential sums when weighted by divisor functions and exponential sums over square free numbers. In particular, we are interested in evaluating \begin{align*} \sum_{n\leq N}\tau(n)e\left(f(n)\right) ~\text{and}~\sum_{n\leq N}\mu^2(n)e\left(f(n)\right), \end{align*} for some polynomial $f$, where $\tau$ is the divisor function and $\mu$ is the M\"{o}bius function. We get non-trivial estimates when the leading co-efficient $\alpha$ of $f$ belongs to the minor arc., Comment: 15 Pages, Comments are welcome

We present results from MeerKAT single-dish HI intensity maps, the final observations to be performed in L-band in the MeerKAT Large Area Synoptic Survey (MeerKLASS) campaign. The observations represent the deepest single-dish HI intensity maps to date, produced from 41 repeated scans over $236\,{\rm deg}^2$, providing 62 hours of observational data for each of the 64 dishes before flagging. By introducing an iterative self-calibration process, the estimated thermal noise of the reconstructed maps is limited to ${\sim}\,1.21\,$mK ($1.2\,\times$ the theoretical noise level). This thermal noise will be sub-dominant relative to the HI fluctuations on large scales ($k\,{\lesssim}\,0.15\,h\,\text{Mpc}^{-1}$), which demands upgrades to power spectrum analysis techniques, particularly for covariance estimation. In this work, we present the improved MeerKLASS analysis pipeline, validating it on both a suite of mock simulations and a small sample of overlapping spectroscopic galaxies from the Galaxy And Mass Assembly (GAMA) survey. Despite only overlapping with ${\sim}\,25\%$ of the MeerKLASS deep field, and a conservative approach to covariance estimation, we still obtain a ${>}\,4\,\sigma$ detection of the cross-power spectrum between the intensity maps and the 2269 galaxies at the narrow redshift range $0.39\,{<}\,z\,{<}\,0.46$. We briefly discuss the HI auto-power spectrum from this data, the detection of which will be the focus of follow-up work. For the first time with MeerKAT single-dish intensity maps, we also present evidence of HI emission from stacking the maps onto the positions of the GAMA galaxies., Comment: 29 pages, 27 figures. See Fig.14 and 15 for auto-HI and HI-galaxy cross power spectra. Fig 19 and 20 for stacking detections. Submitted to MNRAS

The redshifted $21$\,cm signal of neutral hydrogen can be used as a direct probe of the intergalactic medium during Cosmic Dawn\,(CD) and Epoch of Reionization\,(EoR). However, detecting this inherently weak signal has numerous challenges. The major ones include accurate foreground removal from low-frequency radio observations and systematics arising from instrumental effects. The Earth's ionosphere poses a major obstacle at these low radio frequencies. Thus, a systematic study of ionospheric effects on these sensitive low-frequency observations is critical, given that the construction of the Square Kilometre Array (SKA1-Low) is in full progress. We use the end-to-end pipeline, called {\sc 21cme2e}, to study the effect of time-varying ionospheric corruption on the $21$\,cm power spectrum recovery. We use two models: a) a catalogue-based model focused on source position shift due to the refractive effect of the ionosphere and b) a realistic ionospheric condition generated using Kolmogorov's turbulence model. We assess the effect of the imperfections thus generated on the extraction of \HI\ $21$\,cm signal power spectrum. Our study shows that beyond ``median ionospheric offset" ($\theta_{\text{MIO}} \lesssim 0.1''$), the $21$\,cm signal from the EoR is unaffected by residual ionospheric effects. Our study emphasizes the need for the development of efficient ionospheric calibration algorithms for the upcoming SKA1-Low observations to extract the \HI\ $21$\,cm power spectra from the CD/EoR., Comment: 28 pages, 9 figures, To be submitted to JCAP

Single-stranded RNA viruses co-assemble their capsid with the genome and variations in capsid structures can have significant functional relevance. In particular, viruses need to respond to a dehydrating environment to prevent genomic degradation and remain active upon rehydration. Theoretical work has predicted low-energy buckling transitions in icosahedral capsids which could protect the virus from further dehydration. However, there has been no direct experimental evidence, nor molecular mechanism, for such behaviour. Here we observe this transition using X-ray single particle imaging of MS2 bacteriophages after aerosolization. Using a combination of machine learning tools, we classify hundreds of thousands of single particle diffraction patterns to learn the structural landscape of the capsid morphology as a function of time spent in the aerosol phase. We found a previously unreported compact conformation as well as intermediate structures which suggest an incoherent buckling transition which does not preserve icosahedral symmetry. Finally, we propose a mechanism of this buckling, where a single 19-residue loop is destabilised, leading to the large observed morphology change. Our results provide experimental evidence for a mechanism by which viral capsids protect themselves from dehydration. In the process, these findings also demonstrate the power of single particle X-ray imaging and machine learning methods in studying biomolecular structural dynamics., Comment: 10 pages, 4 figures plus 9 pages supplementary information

High voltage (~2 kV) AlGaN-channel HEMTs were fabricated with 64% Aluminum composition in the channel. The average on-resistance was ~75 ohm. mm (~21 miliohm. cm^2) for LGD = 20 microns. Breakdown voltage reached >3 kV (tool limit) before passivation however it reduced to ~2 kV after SiN surface passivation and field plates. The apparent high breakdown voltage prior to passivation can possibly be attributed to the field plate effect of the charged trap states of the surface. The breakdown voltage and RON demonstrated a strong linear correlation in a scattered plot with ~50 measured transistors. In pulsed IV measurements with 100 microsecond pulse width and 40 V of off-state bias (tool limit), the dynamic RON increased by ~5% compared to DC RON and current collapse was <10%., Comment: 4 pages, 6 figures

Given a graph $G = (V, E)$ and a model of information flow on that network, a fundamental question is to understand whether all nodes have sufficient access to information generated at other nodes in the graph. If not, we can ask if a small set of interventions in the form of edge additions improve information access. Formally, the broadcast value of a network is defined to be the minimum over pairs $u,v \in V$ of the probability that an information cascade starting at $u$ reaches $v$. Having a high broadcast value ensures that every node has sufficient access to information spreading in a network, thus quantifying fairness of access. In this paper, we formally study the Broadcast Improvement problem: given $G$ and a parameter $k$, the goal is to find the best set of $k$ edges to add to $G$ in order to maximize the broadcast value of the resulting graph. We develop efficient approximation algorithms for this problem. If the optimal solution adds $k$ edges and achieves a broadcast of $\beta^*$, we develop algorithms that can (a) add $k$ edges and achieve a broadcast value roughly $(\beta^*)^4/16^k$, or (b) add $O(k\log n)$ edges and achieve a broadcast roughly $\beta^*$. We also provide other trade-offs that can be better depending on the parameter values. Our algorithms rely on novel probabilistic tools to reason about the existence of paths in edge-sampled graphs, and extend to a single-source variant of the problem, where we obtain analogous algorithmic results. We complement our results by proving that unless P = NP, any algorithm that adds $O(k)$ edges must lose significantly in the approximation of $\beta^*$, resolving an open question from prior work., Comment: Version 2 adds a new single-criteria approximation

The impressive performance of Large Language Models (LLMs) across various natural language processing tasks comes at the cost of vast computational resources and storage requirements. One-shot pruning techniques offer a way to alleviate these burdens by removing redundant weights without the need for retraining. Yet, the massive scale of LLMs often forces current pruning approaches to rely on heuristics instead of optimization-based techniques, potentially resulting in suboptimal compression. In this paper, we introduce ALPS, an optimization-based framework that tackles the pruning problem using the operator splitting technique and a preconditioned conjugate gradient-based post-processing step. Our approach incorporates novel techniques to accelerate and theoretically guarantee convergence while leveraging vectorization and GPU parallelism for efficiency. ALPS substantially outperforms state-of-the-art methods in terms of the pruning objective and perplexity reduction, particularly for highly sparse models. On the OPT-30B model with 70% sparsity, ALPS achieves a 13% reduction in test perplexity on the WikiText dataset and a 19% improvement in zero-shot benchmark performance compared to existing methods.

Hate detection has long been a challenging task for the NLP community. The task becomes complex in a code-mixed environment because the models must understand the context and the hate expressed through language alteration. Compared to the monolingual setup, we see much less work on code-mixed hate as large-scale annotated hate corpora are unavailable for the study. To overcome this bottleneck, we propose using native language hate samples (native language samples/ native samples hereafter). We hypothesise that in the era of multilingual language models (MLMs), hate in code-mixed settings can be detected by majorly relying on the native language samples. Even though the NLP literature reports the effectiveness of MLMs on hate detection in many cross-lingual settings, their extensive evaluation in a code-mixed scenario is yet to be done. This paper attempts to fill this gap through rigorous empirical experiments. We considered the Hindi-English code-mixed setup as a case study as we have the linguistic expertise for the same. Some of the interesting observations we got are: (i) adding native hate samples in the code-mixed training set, even in small quantity, improved the performance of MLMs for code-mixed hate detection, (ii) MLMs trained with native samples alone observed to be detecting code-mixed hate to a large extent, (iii) the visualisation of attention scores revealed that, when native samples were included in training, MLMs could better focus on the hate emitting words in the code-mixed context, and (iv) finally, when hate is subjective or sarcastic, naively mixing native samples doesn't help much to detect code-mixed hate. We will release the data and code repository to reproduce the reported results., Comment: Generated from XeLaTeX

Designing Reinforcement Learning (RL) solutions for real-life problems remains a significant challenge. A major area of concern is safety. "Shielding" is a popular technique to enforce safety in RL by turning user-defined safety specifications into safe agent behavior. However, these methods either suffer from extreme learning delays, demand extensive human effort in designing models and safe domains in the problem, or require pre-computation. In this paper, we propose a new permissibility-based framework to deal with safety and shield construction. Permissibility was originally designed for eliminating (non-permissible) actions that will not lead to an optimal solution to improve RL training efficiency. This paper shows that safety can be naturally incorporated into this framework, i.e. extending permissibility to include safety, and thereby we can achieve both safety and improved efficiency. Experimental evaluation using three standard RL applications shows the effectiveness of the approach., Comment: 9 pages, 3 figures

Let $q$ be a positive integral power of some prime $p$ and $\mathbb{F}_{q^m}$ be a finite field with $q^m$ elements for some $m \in \mathbb{N}$. Here we establish a sufficient condition for the existence of primitive normal pairs of the type $(\epsilon, f(\epsilon))$ in $\mathbb{F}_{q^m}$ over $\mathbb{F}_{q}$ with two prescribed traces, $Tr_{{\mathbb{F}_{q^m}}/{\mathbb{F}_q}}(\epsilon)=a$ and $Tr_{{\mathbb{F}_{q^m}}/{\mathbb{F}_q}}(f(\epsilon))=b$, where $f(x) \in \mathbb{F}_{q^m}(x)$ is a rational function with some restrictions and $a, b \in \mathbb{F}_q$. Furthermore, for $q=5^k$, $m \geq 9$ and rational functions with degree sum 4, we explicitly find at most 12 fields in which the desired pair may not exist.

Let $q, n, m \in \mathbb{N}$ be such that $q$ is a prime power and $a, b \in \mathbb{F}$. In this article we establish a sufficient condition for the existence of a primitive normal pair $(\alpha, f(\alpha)) \in \mathbb{F}_{q^m}$ over $\mathbb{F}$ with a prescribed primitive norm $a$ and a non-zero trace $b$ over $\mathbb{F}$ of $\alpha$, where $f(x) \in \mathbb{F}_{q^m}(x)$ is a rational function of degree sum $n$ with some minor restrictions. Furthermore, for $q=7^k$, $m \geq 7$ and rational functions with numerator and denominator being linear, we explicitly find at most 6 fields in which the desired pair may not exist., Comment: 16 pages

Artificial neural networks trained on large, expert-labelled datasets are considered state-of-the-art for a range of medical image recognition tasks. However, categorically labelled datasets are time-consuming to generate and constrain classification to a pre-defined, fixed set of classes. For neuroradiological applications in particular, this represents a barrier to clinical adoption. To address these challenges, we present a self-supervised text-vision framework that learns to detect clinically relevant abnormalities in brain MRI scans by directly leveraging the rich information contained in accompanying free-text neuroradiology reports. Our training approach consisted of two-steps. First, a dedicated neuroradiological language model - NeuroBERT - was trained to generate fixed-dimensional vector representations of neuroradiology reports (N = 50,523) via domain-specific self-supervised learning tasks. Next, convolutional neural networks (one per MRI sequence) learnt to map individual brain scans to their corresponding text vector representations by optimising a mean square error loss. Once trained, our text-vision framework can be used to detect abnormalities in unreported brain MRI examinations by scoring scans against suitable query sentences (e.g., 'there is an acute stroke', 'there is hydrocephalus' etc.), enabling a range of classification-based applications including automated triage. Potentially, our framework could also serve as a clinical decision support tool, not only by suggesting findings to radiologists and detecting errors in provisional reports, but also by retrieving and displaying examples of pathologies from historical examinations that could be relevant to the current case based on textual descriptors., Comment: Under Review

Efficient thermal management in high-power electronics cooling can be achieved using phase-change heat transfer devices, such as vapor chambers. Traditional vapor chambers use wicks to transport condensate for efficient thermal exchange and to prevent "dry-out" of the evaporator. However, wicks in vapor chambers present significant design challenges arising out of large pressure drops across the wicking material, which slows down condensate transport rates and increases the chances for dry-out. Thicker wicks add to overall thermal resistance, while deterring the development of thinner devices by limiting the total thickness of the vapor chamber. Wickless vapor chambers eliminate the use of metal wicks entirely, by incorporating complementary wettability-patterned flat plates on both the evaporator and the condenser side. Such surface modifications enhance fluid transport on the evaporator side, while allowing the chambers to be virtually as thin as imaginable, thereby permitting design of thermally efficient thin electronic cooling devices. While wick-free vapor chambers have been studied and efficient design strategies have been suggested, we delve into real-life applications of wick-free vapor chambers in forced air cooling of high-power electronics. An experimental setup is developed wherein two Si-based MOSFETs of TO-247-3 packaging having high conduction resistance, are connected in parallel and switched at 100 kHz, to emulate high frequency power electronics operations. A rectangular copper wick-free vapor chamber spreads heat laterally over a surface 13 times larger than the heating area. This chamber is cooled externally by a fan that circulates air at room temperature. The present experimental setup extends our previous work on wick-free vapor chambers, while demonstrating the effectiveness of low-cost air cooling in vapor-chamber enhanced high-power electronics applications., Comment: Presented at IEEE ITherm (Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems) 2023, Orlando FL. Corresponding author: cmm@uic.edu

The increasing computational demands of modern neural networks present deployment challenges on resource-constrained devices. Network pruning offers a solution to reduce model size and computational cost while maintaining performance. However, most current pruning methods focus primarily on improving sparsity by reducing the number of nonzero parameters, often neglecting other deployment costs such as inference time, which are closely related to the number of floating-point operations (FLOPs). In this paper, we propose FALCON, a novel combinatorial-optimization-based framework for network pruning that jointly takes into account model accuracy (fidelity), FLOPs, and sparsity constraints. A main building block of our approach is an integer linear program (ILP) that simultaneously handles FLOP and sparsity constraints. We present a novel algorithm to approximately solve the ILP. We propose a novel first-order method for our optimization framework which makes use of our ILP solver. Using problem structure (e.g., the low-rank structure of approx. Hessian), we can address instances with millions of parameters. Our experiments demonstrate that FALCON achieves superior accuracy compared to other pruning approaches within a fixed FLOP budget. For instance, for ResNet50 with 20% of the total FLOPs retained, our approach improves the accuracy by 48% relative to state-of-the-art. Furthermore, in gradual pruning settings with re-training between pruning steps, our framework outperforms existing pruning methods, emphasizing the significance of incorporating both FLOP and sparsity constraints for effective network pruning.

Structured pruning is a promising approach for reducing the inference costs of large vision and language models. By removing carefully chosen structures, e.g., neurons or attention heads, the improvements from this approach can be realized on standard deep learning hardware. In this work, we focus on structured pruning in the one-shot (post-training) setting, which does not require model retraining after pruning. We propose a novel combinatorial optimization framework for this problem, based on a layer-wise reconstruction objective and a careful reformulation that allows for scalable optimization. Moreover, we design a new local combinatorial optimization algorithm, which exploits low-rank updates for efficient local search. Our framework is time and memory-efficient and considerably improves upon state-of-the-art one-shot methods on vision models (e.g., ResNet50, MobileNet) and language models (e.g., OPT-1.3B -- OPT-30B). For language models, e.g., OPT-2.7B, OSSCAR can lead to $125\times$ lower test perplexity on WikiText with $2\times$ inference time speedup in comparison to the state-of-the-art ZipLM approach. Our framework is also $6\times$ -- $8\times$ faster. Notably, our work considers models with tens of billions of parameters, which is up to $100\times$ larger than what has been previously considered in the structured pruning literature.

We study the often overlooked phenomenon, first noted in \cite{breiman2001random}, that random forests appear to reduce bias compared to bagging. Motivated by an interesting paper by \cite{mentch2020randomization}, where the authors argue that random forests reduce effective degrees of freedom and only outperform bagging ensembles in low signal-to-noise ratio (SNR) settings, we explore how random forests can uncover patterns in the data missed by bagging. We empirically demonstrate that in the presence of such patterns, random forests reduce bias along with variance and increasingly outperform bagging ensembles when SNR is high. Our observations offer insights into the real-world success of random forests across a range of SNRs and enhance our understanding of the difference between random forests and bagging ensembles with respect to the randomization injected into each split. Our investigations also yield practical insights into the importance of tuning $mtry$ in random forests.

We present FAST, an optimization framework for fast additive segmentation. FAST segments piecewise constant shape functions for each feature in a dataset to produce transparent additive models. The framework leverages a novel optimization procedure to fit these models $\sim$2 orders of magnitude faster than existing state-of-the-art methods, such as explainable boosting machines \citep{nori2019interpretml}. We also develop new feature selection algorithms in the FAST framework to fit parsimonious models that perform well. Through experiments and case studies, we show that FAST improves the computational efficiency and interpretability of additive models.