Search

Research on school improvement has accumulated an extensive list of factors that facilitate turnarounds at underperforming schools. Given that contextual or resource constraints may limit the possibilities of putting all of these factors in place, an important question is what is necessary and sufficient to turn a school around. We use a qualitative comparative analysis (QCA) of 77 Swedish schools studied over 12 years to answer this question. Our core finding is that there is no "silver bullet" solution. There are, instead, several distinct combinations of factors that can enable a turnaround. The local school context is essential for which combinations of factors are necessary and sufficient for school turnaround. We discuss implications for research on school improvement and education policy.

Concept inventories are standardized assessments that evaluate student understanding of key concepts within academic disciplines. While prevalent across STEM fields, their development lags for advanced computer science topics like dynamic programming (DP) -- an algorithmic technique that poses significant conceptual challenges for undergraduates. To fill this gap, we developed and validated a Dynamic Programming Concept Inventory (DPCI). We detail the iterative process used to formulate multiple-choice questions targeting known student misconceptions about DP concepts identified through prior research studies. We discuss key decisions, tradeoffs, and challenges faced in crafting probing questions to subtly reveal these conceptual misunderstandings. We conducted a preliminary psychometric validation by administering the DPCI to 172 undergraduate CS students finding our questions to be of appropriate difficulty and effectively discriminating between differing levels of student understanding. Taken together, our validated DPCI will enable instructors to accurately assess student mastery of DP. Moreover, our approach for devising a concept inventory for an advanced theoretical computer science concept can guide future efforts to create assessments for other under-evaluated areas currently lacking coverage., Comment: Accepted to SIGCSE 2025

Linearity testing has been a focal problem in property testing of functions. We combine different known techniques and observations about linearity testing in order to resolve two recent versions of this task. First, we focus on the online manipulations model introduced by Kalemaj, Raskhodnikova and Varma (ITCS 2022 \& Theory of Computing 2023). In this model, up to $t$ data entries are adversarially manipulated after each query is answered. Ben-Eliezer, Kelman, Meir, and Raskhodnikova (ITCS 2024) showed an asymptotically optimal linearity tester that is resilient to $t$ manipulations per query, but their approach fails if $t$ is too large. We extend this result, showing an optimal tester for almost any possible value of $t$. First, we simplify their result when $t$ is small, and for larger values of $t$ we instead use sample-based testers, as defined by Goldreich and Ron (ACM Transactions on Computation Theory 2016). A key observation is that sample-based testing is resilient to online manipulations, but still achieves optimal query complexity for linearity when $t$ is large. We complement our result by showing that when $t$ is \emph{very} large, any reasonable property, and in particular linearity, cannot be tested at all. Second, we consider linearity over the reals with proximity parameter $\varepsilon$. Fleming and Yoshida (ITCS 2020) gave a tester using $O(1/\varepsilon\ \cdot log(1/\varepsilon))$ queries. We simplify their algorithms and modify the analysis accordingly, showing an optimal tester that only uses $O(1/\varepsilon)$ queries. This modification works for the low-degree testers presented in Arora, Bhattacharyya, Fleming, Kelman, and Yoshida (SODA 2023) as well, resulting in optimal testers for degree-$d$ polynomials, for any constant degree $d$., Comment: To appear at SOSA 2025

Spoken term detection (STD) is often hindered by reliance on frame-level features and the computationally intensive DTW-based template matching, limiting its practicality. To address these challenges, we propose a novel approach that encodes speech into discrete, speaker-agnostic semantic tokens. This facilitates fast retrieval using text-based search algorithms and effectively handles out-of-vocabulary terms. Our approach focuses on generating consistent token sequences across varying utterances of the same term. We also propose a bidirectional state space modeling within the Mamba encoder, trained in a self-supervised learning framework, to learn contextual frame-level features that are further encoded into discrete tokens. Our analysis shows that our speech tokens exhibit greater speaker invariance than those from existing tokenizers, making them more suitable for STD tasks. Empirical evaluation on LibriSpeech and TIMIT databases indicates that our method outperforms existing STD baselines while being more efficient., Comment: Submitted to ICASSP 2025

With the increased importance of autonomous navigation systems has come an increasing need to protect the safety of Vulnerable Road Users (VRUs) such as pedestrians. Predicting pedestrian intent is one such challenging task, where prior work predicts the binary cross/no-cross intention with a fusion of visual and motion features. However, there has been no effort so far to hedge such predictions with human-understandable reasons. We address this issue by introducing a novel problem setting of exploring the intuitive reasoning behind a pedestrian's intent. In particular, we show that predicting the 'WHY' can be very useful in understanding the 'WHAT'. To this end, we propose a novel, reason-enriched PIE++ dataset consisting of multi-label textual explanations/reasons for pedestrian intent. We also introduce a novel multi-task learning framework called MINDREAD, which leverages a cross-modal representation learning framework for predicting pedestrian intent as well as the reason behind the intent. Our comprehensive experiments show significant improvement of 5.6% and 7% in accuracy and F1-score for the task of intent prediction on the PIE++ dataset using MINDREAD. We also achieved a 4.4% improvement in accuracy on a commonly used JAAD dataset. Extensive evaluation using quantitative/qualitative metrics and user studies shows the effectiveness of our approach.

Machine unlearning algorithms are increasingly important as legal concerns arise around the provenance of training data, but verifying the success of unlearning is often difficult. Provable guarantees for unlearning are often limited to supervised learning settings. In this paper, we provide the first theoretical guarantees for unlearning in the pre-training and fine-tuning paradigm by studying topic models, simple bag-of-words language models that can be adapted to solve downstream tasks like retrieval and classification. First, we design a provably effective unlearning algorithm for topic models that incurs a computational overhead independent of the size of the original dataset. Our analysis additionally quantifies the deletion capacity of the model -- i.e., the number of examples that can be unlearned without incurring a significant cost in model performance. Finally, we formally extend our analyses to account for adaptation to a given downstream task. In particular, we design an efficient algorithm to perform unlearning after fine-tuning the topic model via a linear head. Notably, we show that it is easier to unlearn pre-training data from models that have been fine-tuned to a particular task, and one can unlearn this data without modifying the base model.

This paper presents a multi-cloud networking architecture built on zero trust principles and micro-segmentation to provide secure connectivity with authentication, authorization, and encryption in transit. The proposed design includes the multi-cloud network to support a wide range of applications and workload use cases, compute resources including containers, virtual machines, and cloud-native services, including IaaS (Infrastructure as a Service (IaaS), PaaS (Platform as a service). Furthermore, open-source tools provide flexibility, agility, and independence from locking to one vendor technology. The paper provides a secure architecture with micro-segmentation and follows zero trust principles to solve multi-fold security and operational challenges., Comment: 8 pages, 6 figures

This manuscript presents the Quantum Finite Element Method (Q-FEM) developed for use in noisy intermediate-scale quantum (NISQ) computers, and employs the variational quantum linear solver (VQLS) algorithm. The proposed method leverages the classical FEM procedure to perform the unitary decomposition of the stiffness matrix and employs generator functions to design explicit quantum circuits corresponding to the unitaries. Q-FEM keeps the structure of the finite element discretization intact allowing for the use of variable element lengths and material coefficients in FEM discretization. The proposed method is tested on a steady-state heat equation discretized using linear and quadratic shape functions. Numerical verification studies demonstrate that Q-FEM is effective in converging to the correct solution for a variety of problems and model discretizations, including with different element lengths, variable coefficients, and different boundary conditions. The formalism developed herein is general and can be extended to problems with higher dimensions. However, numerical examples also demonstrate that the number of parameters for the variational ansatz scale exponentially with the number of qubits to increase the odds of convergence, and deterioration of system conditioning with problem size results in barren plateaus, and hence convergence difficulties.

We introduce LLMStinger, a novel approach that leverages Large Language Models (LLMs) to automatically generate adversarial suffixes for jailbreak attacks. Unlike traditional methods, which require complex prompt engineering or white-box access, LLMStinger uses a reinforcement learning (RL) loop to fine-tune an attacker LLM, generating new suffixes based on existing attacks for harmful questions from the HarmBench benchmark. Our method significantly outperforms existing red-teaming approaches (we compared against 15 of the latest methods), achieving a +57.2% improvement in Attack Success Rate (ASR) on LLaMA2-7B-chat and a +50.3% ASR increase on Claude 2, both models known for their extensive safety measures. Additionally, we achieved a 94.97% ASR on GPT-3.5 and 99.4% on Gemma-2B-it, demonstrating the robustness and adaptability of LLMStinger across open and closed-source models., Comment: Accepted at AAAI 2025

This monograph presents a theoretical background and a broad introduction to the Min-Max Framework for Majorization-Minimization (MM4MM), an algorithmic methodology for solving minimization problems by formulating them as min-max problems and then employing majorization-minimization. The monograph lays out the mathematical basis of the approach used to reformulate a minimization problem as a min-max problem. With the prerequisites covered, including multiple illustrations of the formulations for convex and non-convex functions, this work serves as a guide for developing MM4MM-based algorithms for solving non-convex optimization problems in various areas of signal processing. As special cases, we discuss using the majorization-minimization technique to solve min-max problems encountered in signal processing applications and min-max problems formulated using the Lagrangian. Lastly, we present detailed examples of using MM4MM in ten signal processing applications such as phase retrieval, source localization, independent vector analysis, beamforming and optimal sensor placement in wireless sensor networks. The devised MM4MM algorithms are free of hyper-parameters and enjoy the advantages inherited from the use of the majorization-minimization technique such as monotonicity., Comment: 84 pages, no figures, published in Foundations and Trends in Signal Processing: Vol. 18: No. 4, pp 310-389. http://dx.doi.org/10.1561/2000000129

We present the first set of galaxy scaling relations derived from kinematic models of the Widefield ASKAP L-band Legacy All-sky Blind surveY (WALLABY) pilot phase observations. Combining the results of the first and second pilot data releases, there are 236 available kinematic models. We develop a framework for robustly measuring HI disk structural properties from these kinematic models; applicable to the full WALLABY survey. Utilizing this framework, we obtained the HI size, a measure of the rotational velocity, and angular momentum for 148 galaxies. These comprise the largest sample of galaxy properties from an untargetted, uniformly observed and modelled HI survey to date. We study the neutral atomic Hydrogen (HI) size-mass, size-velocity, mass-velocity, and angular momentum-mass scaling relations. We calculate the slope, intercept, and scatter for these scaling relations and find that they are similar to those obtained from other HI surveys. We also obtain stellar masses for 92 of the 148 robustly measured galaxies using multiband photometry through the Dark Energy Sky Instrument Legacy Imaging Survey Data Release-10 images. We use a subset of 61 of these galaxies that have consistent optical and kinematic inclinations to examine the stellar and baryonic Tully Fisher relations, the gas fraction-disk stability and gas fraction-baryonic mass relations. These measurements and relations demonstrate the unprecedented resource that WALLABY will represent for resolved galaxy scaling relations in HI., Comment: 23 pages, 8 figures, Table 1 data available for download with package, accepted to ApJ

Shadow removal and segmentation remain challenging tasks in computer vision, particularly in complex real world scenarios. This study presents a novel approach that enhances the ShadowFormer model by incorporating Masked Autoencoder (MAE) priors and Fast Fourier Convolution (FFC) blocks, leading to significantly faster convergence and improved performance. We introduce key innovations: (1) integration of MAE priors trained on Places2 dataset for better context understanding, (2) adoption of Haar wavelet features for enhanced edge detection and multiscale analysis, and (3) implementation of a modified SAM Adapter for robust shadow segmentation. Extensive experiments on the challenging DESOBA dataset demonstrate that our approach achieves state of the art results, with notable improvements in both convergence speed and shadow removal quality.

Multimodal foundation models, such as Gemini and ChatGPT, have revolutionized human-machine interactions by seamlessly integrating various forms of data. Developing a universal spoken language model that comprehends a wide range of natural language instructions is critical for bridging communication gaps and facilitating more intuitive interactions. However, the absence of a comprehensive evaluation benchmark poses a significant challenge. We present Dynamic-SUPERB Phase-2, an open and evolving benchmark for the comprehensive evaluation of instruction-based universal speech models. Building upon the first generation, this second version incorporates 125 new tasks contributed collaboratively by the global research community, expanding the benchmark to a total of 180 tasks, making it the largest benchmark for speech and audio evaluation. While the first generation of Dynamic-SUPERB was limited to classification tasks, Dynamic-SUPERB Phase-2 broadens its evaluation capabilities by introducing a wide array of novel and diverse tasks, including regression and sequence generation, across speech, music, and environmental audio. Evaluation results indicate that none of the models performed well universally. SALMONN-13B excelled in English ASR, while WavLLM demonstrated high accuracy in emotion recognition, but current models still require further innovations to handle a broader range of tasks. We will soon open-source all task data and the evaluation pipeline.

In particle physics, homogeneous calorimeters are used to measure the energy of particles as they interact with the detector material. Although not as precise as trackers or muon detectors, these calorimeters provide valuable insights into the properties of particles by analyzing their energy deposition patterns. Recent advances in material science, notably in nanomaterial scintillators with tunable emission bandwidths, have led to the proposal of the chromatic calorimetry concept. This proposed concept aims to track electromagnetic or hadronic shower progression within a module, enhancing particle identification and energy resolution by layering scintillators with different emission wavelengths. The idea is to use the emission spectra of the inorganic scintillators to reconstruct the shower progression. Our study validates this proposed concept using inorganic scintillators strategically stacked by decreasing emission wavelength. Using electrons and pions with up to 100 GeV, we achieved analytical discrimination and longitudinal shower measurement. This proof of concept underscores chromatic calorimetry's potential for broader applications.

Photo-control of correlated phases is central to advancing and manipulating novel functional properties of quantum materials. Here, we explore microwave enhancement of superconductivity in flat bands through generation of nonequilibrium quasiparticles at subgap frequencies. In conventional superconductors, it is known to occur via radiation absorption determined by fermi velocity, which however is small in flat bands resulting in quenched quasiparticle excitations. Strikingly, in contrast to the conventional paradigm we show a non-vanishing microwave absorption in flat band systems enabled by Bloch quantum geometry leading to superconducting gap enhancement, underscoring the band-geometric origin of nonequilibrium flat band superconductivity. Specifically, we demonstrate this in twisted bilayer graphene, a promising candidate material, and find significant gap enhancement near critical temperature. This work highlights that the nonequilibrium dynamics of materials with non-trivial flat bands as a promising area for future experimental and theoretical investigation.

We study learning in a dynamically evolving environment modeled as a Markov game between a learner and a strategic opponent that can adapt to the learner's strategies. While most existing works in Markov games focus on external regret as the learning objective, external regret becomes inadequate when the adversaries are adaptive. In this work, we focus on \emph{policy regret} -- a counterfactual notion that aims to compete with the return that would have been attained if the learner had followed the best fixed sequence of policy, in hindsight. We show that if the opponent has unbounded memory or if it is non-stationary, then sample-efficient learning is not possible. For memory-bounded and stationary, we show that learning is still statistically hard if the set of feasible strategies for the learner is exponentially large. To guarantee learnability, we introduce a new notion of \emph{consistent} adaptive adversaries, wherein, the adversary responds similarly to similar strategies of the learner. We provide algorithms that achieve $\sqrt{T}$ policy regret against memory-bounded, stationary, and consistent adversaries., Comment: NeurIPS'24

The synthesis of heavy elements in supernovae is affected by low-energy (n,p) and (p,n) reactions on unstable nuclei, yet experimental data on such reaction rates are scarce. The SECAR (SEparator for CApture Reactions) recoil separator at FRIB (Facility for Rare Isotope Beams) was originally designed to measure astrophysical reactions that change the mass of a nucleus significantly. We used a novel approach that integrates machine learning with ion-optical simulations to find an ion-optical solution for the separator that enables the measurement of (p,n) reactions, despite the reaction leaving the mass of the nucleus nearly unchanged. A new measurement of the $^{58}$Fe(p,n)$^{58}$Co reaction in inverse kinematics with a 3.66$\pm$0.12 MeV/nucleon $^{58}$Fe beam (corresponding to 3.69$\pm$0.12 MeV proton energy in normal kinematics) yielded a cross-section of 20.3$\pm$6.3 mb and served as a benchmark for the new technique demonstrating its effectiveness in achieving the required performance criteria. This novel approach marks a significant advancement in experimental nuclear astrophysics, as it paves the way for studying astrophysically important (p,n) reactions on unstable nuclei produced at FRIB.

Large-scale deployment of large language models (LLMs) in various applications, such as chatbots and virtual assistants, requires LLMs to be culturally sensitive to the user to ensure inclusivity. Culture has been widely studied in psychology and anthropology, and there has been a recent surge in research on making LLMs more culturally inclusive in LLMs that goes beyond multilinguality and builds on findings from psychology and anthropology. In this paper, we survey efforts towards incorporating cultural awareness into text-based and multimodal LLMs. We start by defining cultural awareness in LLMs, taking the definitions of culture from anthropology and psychology as a point of departure. We then examine methodologies adopted for creating cross-cultural datasets, strategies for cultural inclusion in downstream tasks, and methodologies that have been used for benchmarking cultural awareness in LLMs. Further, we discuss the ethical implications of cultural alignment, the role of Human-Computer Interaction in driving cultural inclusion in LLMs, and the role of cultural alignment in driving social science research. We finally provide pointers to future research based on our findings about gaps in the literature.

The challenge of mapping AI architectures to GPU hardware is creating a critical bottleneck in AI progress. Despite substantial efforts, hand-written custom kernels fail to meet their theoretical performance thresholds, even on well-established operations like linear attention. The diverse hardware capabilities of GPUs might suggest that we need a wide variety of techniques to achieve high performance. However, our work explores whether a small number of key abstractions can drastically simplify the process. We present ThunderKittens (TK), a framework for writing performant AI kernels while remaining easy to use and maintain. Our abstractions map to the three levels of the GPU hierarchy: (1) at the warp-level, we provide 16x16 matrix tiles as basic data structures and PyTorch-like parallel compute operations over tiles, (2) at the thread-block level, we provide a template for overlapping asynchronous operations across parallel warps, and (3) at the grid-level, we provide support to help hide the block launch and tear-down, and memory costs. We show the value of TK by providing kernels that match or outperform prior kernels for a range of AI operations. We match CuBLAS and FlashAttention-3 on GEMM and attention inference performance and outperform the strongest baselines by $10-40\%$ on attention backwards, $8\times$ on state space models, and $14\times$ on linear attention.

In this study, we explore an ensemble-based approach to improve classification accuracy in complex image datasets. Utilizing a Convolutional Block Attention Module (CBAM) alongside a Deep Neural Network (DNN) we leverage the unique feature-extraction capabilities of each model to enhance the overall accuracy. Additional models, such as Random Forest, XGBoost, Support Vector Machine (SVM), and K-Nearest Neighbors (KNN), are introduced to further diversify the predictive power of our ensemble. By leveraging these methods, the proposed approach provides robust feature discrimination and improved classification results. Experimental evaluations demonstrate that the ensemble achieves higher accuracy and robustness across challenging and imbalanced classes, showing significant promise for broader applications in computer vision tasks.

GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.

The study of the X-ray emission from massive binaries constitutes a relevant approach to investigate shock physics. The case of short period binaries may turn out to be quite challenging, especially in very asymmetric systems where the primary wind may overwhelm that of the secondary in the wind interaction. Our objective consists in providing an observational diagnostic of the X-ray behaviour of HD93205, that is a very good candidate to investigate these aspects. We analysed 31 epochs of XMM-Newton X-ray data spanning about two decades to investigate its spectral and timing behaviour. The X-ray spectrum is very soft along the full orbit, with a luminosity exclusively from the wind interaction region in the range of 2.3 -- 5.4\,$\times$10$^{32}$\,erg\,s$^{-1}$. The light curve peaks close to periastron, with a rather wide pre-periastron low-state coincident with the secondary's body hiding a part of the X-ray emitting region close to its surface. We determined a variability time scale of 6.0807\,$\pm$\,0.0013\,d, in full agreement with the orbital period. Making use of a one-dimensional approach to deal with mutual radiative effects, our results point to a very likely hybrid wind interaction, with a wind-photosphere occurring along most of the orbit, while a brief episode of wind-wind interaction may still develop close to apastron. Beside mutual radiative effects, the radiative nature of the shock that leads to some additional pre-shock obliquitity of the primary wind flow certainly explains the very soft emission. HD93205 constitutes a relevant target to investigate shock physics in short period, asymmetric massive binary systems, where various mutual radiative effects and radiative shocks concur to display an instructive soft X-ray behaviour. HD93205 should be considered as a valid, though challenging target for future three-dimensional modelling initiatives., Comment: 17 pages, 15 figures, accepted for publication in Astronomy & Astrophysics

In-context learning (ICL) is a powerful technique for getting language models to perform complex tasks with no training updates. Prior work has established strong correlations between the number of in-context examples provided and the accuracy of the model's predictions. In this paper, we seek to explain this correlation by showing that ICL approximates a Bayesian learner. This perspective gives rise to a family of novel Bayesian scaling laws for ICL. In experiments with \mbox{GPT-2} models of different sizes, our scaling laws exceed or match existing scaling laws in accuracy while also offering interpretable terms for task priors, learning efficiency, and per-example probabilities. To illustrate the analytic power that such interpretable scaling laws provide, we report on controlled synthetic dataset experiments designed to inform real-world studies of safety alignment. In our experimental protocol, we use SFT to suppress an unwanted existing model capability and then use ICL to try to bring that capability back (many-shot jailbreaking). We then experiment on real-world instruction-tuned LLMs using capabilities benchmarks as well as a new many-shot jailbreaking dataset. In all cases, Bayesian scaling laws accurately predict the conditions under which ICL will cause the suppressed behavior to reemerge, which sheds light on the ineffectiveness of post-training at increasing LLM safety., Comment: 10 pages main text, 26 pages total

Spherical proportional counters (SPCs) are gaseous particle detectors sensitive to single ionization electrons in their target media, with large detector volumes and low background rates. The $\mbox{NEWS-G}$ collaboration employs this technology to search for low-mass dark matter, having previously performed searches with detectors at the Laboratoire Souterrain de Modane (LSM), including a recent campaign with a 135 cm diameter SPC filled with methane. While in situ calibrations of the detector response were carried out at the LSM, measurements of the mean ionization yield and fluctuations of methane gas in SPCs were performed using a 30 cm diameter detector. The results of multiple measurements taken at different operating voltages are presented. A UV laser system was used to measure the mean gas gain of the SPC, along with $\mathrm{^{37}Ar}$ and aluminum-fluorescence calibration sources. These measurements will inform the energy response model of future operating detectors., Comment: 11 pages, 8 figures

Let $a_1,\ldots,a_m$ be nonzero integers, $c \in \mathbb Z$ and $r \ge 2$. The Rado number for the equation \[ \sum_{i=1}^m a_ix_i = c \] in $r$ colours is the least positive integer $N$ such that any $r$-colouring of the integers in the interval $[1,N]$ admits a monochromatic solution to the given equation. We introduce the concept of $t$-distributability of sets of positive integers, and determine exact values whenever possible, and upper and lower bounds otherwise, for the Rado numbers when the set $\{a_1,\ldots,a_{m-1}\}$ is $2$-distributable or $3$-distributable, $a_m=-1$, and $r=2$. This generalizes previous works by several authors., Comment: One table of summary, 25 referenecs, 17 pages

Reasoning and linguistic skills form the cornerstone of human intelligence, facilitating problem-solving and decision-making. Recent advances in Large Language Models (LLMs) have led to impressive linguistic capabilities and emergent reasoning behaviors, fueling widespread adoption across application domains. However, LLMs still struggle with complex reasoning tasks, highlighting their systemic limitations. In this work, we focus on evaluating whether LLMs have the requisite representations to reason using two foundational relationships: "equivalence" and "inheritance". We introduce novel tasks and benchmarks spanning six languages and observe that current SOTA LLMs often produce conflicting answers to the same questions across languages in 17.3-57.5% of cases and violate inheritance constraints in up to 37.2% cases. To enhance consistency across languages, we propose novel "Compositional Representations" where tokens are represented as composition of equivalent tokens across languages, with resulting conflict reduction (up to -4.7%) indicating benefits of shared LLM representations.

Recent works show we can linearize large language models (LLMs) -- swapping the quadratic attentions of popular Transformer-based LLMs with subquadratic analogs, such as linear attention -- avoiding the expensive pretraining costs. However, linearizing LLMs often significantly degrades model quality, still requires training over billions of tokens, and remains limited to smaller 1.3B to 7B LLMs. We thus propose Low-rank Linear Conversion via Attention Transfer (LoLCATs), a simple two-step method that improves LLM linearizing quality with orders of magnitudes less memory and compute. We base these steps on two findings. First, we can replace an LLM's softmax attentions with closely-approximating linear attentions, simply by training the linear attentions to match their softmax counterparts with an output MSE loss ("attention transfer"). Then, this enables adjusting for approximation errors and recovering LLM quality simply with low-rank adaptation (LoRA). LoLCATs significantly improves linearizing quality, training efficiency, and scalability. We significantly reduce the linearizing quality gap and produce state-of-the-art subquadratic LLMs from Llama 3 8B and Mistral 7B v0.1, leading to 20+ points of improvement on 5-shot MMLU. Furthermore, LoLCATs does so with only 0.2% of past methods' model parameters and 0.4% of their training tokens. Finally, we apply LoLCATs to create the first linearized 70B and 405B LLMs (50x larger than prior work). When compared with prior approaches under the same compute budgets, LoLCATs significantly improves linearizing quality, closing the gap between linearized and original Llama 3.1 70B and 405B LLMs by 77.8% and 78.1% on 5-shot MMLU., Comment: 47 pages, 20 figures, 18 tables, preprint

Lie symmetry analysis has been applied to the extended Boiti-Leon-Manna-Pempinelli (eBLMP) equation. This system illustrates the exchange of information between two waves with distinct dispersion characteristics. The optimal system of the corresponding Lie algebra has been constructed. The equation considered has been reduced into a simpler form for the computation of analytical solutions. The novelty of this research is the optimal system of subalgebras in one dimension using the adjoint action approach. To analyze and understand the eBLMP more clearly, graphs have been plotted. We have also found conservation laws, Comment: 21 pages, 18 figures

Galaxies like the Milky Way are surrounded by complex populations of satellites at all stages of tidal disruption. In this paper, we present a dynamical study of the disrupting satellite galaxies in the Auriga simulations that are orbiting 28 distinct Milky Way-mass hosts across three resolutions. We find that the satellite galaxy populations are highly disrupted. The majority of satellites that remain fully intact at present day were accreted recently without experiencing more than one pericentre ($n_{\rm peri} \lesssim 1$) and have large apocentres ($r_{\rm apo} \gtrsim 200$ kpc) and pericentres ($r_{\rm peri} \gtrsim 50$ kpc). The remaining satellites have experienced significant tidal disruption and, given full knowledge of the system, would be classified as stellar streams. We find stellar streams in Auriga across the range of pericentres and apocentres of the known Milky Way dwarf galaxy streams and, interestingly, overlapping significantly with the Milky Way intact satellite population. We find no significant change in satellite orbital distributions across resolution. However, we do see substantial halo-to-halo variance of $(r_\text{peri}, r_\text{apo})$ distributions across host galaxies, as well as a dependence of satellite orbits on host halo mass - systems disrupt at larger pericentres and apocentres in more massive hosts. Our results suggest that either cosmological simulations (including, but not limited to, Auriga) are disrupting satellites far too readily, or that the Milky Way's satellites are more disrupted than current imaging surveys have revealed. Future observing facilities and careful mock observations of these systems will be key to revealing the nature of this apparent discrepancy., Comment: 16 pages, 10 figures, 1 table, submitted to MNRAS

There have been key advancements to building universal approximators for multi-goal collections of reinforcement learning value functions -- key elements in estimating long-term returns of states in a parameterized manner. We extend this to hierarchical reinforcement learning, using the options framework, by introducing hierarchical universal value function approximators (H-UVFAs). This allows us to leverage the added benefits of scaling, planning, and generalization expected in temporal abstraction settings. We develop supervised and reinforcement learning methods for learning embeddings of the states, goals, options, and actions in the two hierarchical value functions: $Q(s, g, o; \theta)$ and $Q(s, g, o, a; \theta)$. Finally we demonstrate generalization of the HUVFAs and show they outperform corresponding UVFAs., Comment: 13 pages, 11 figures, 3 appendices. Currently under review

Direct Preference Optimization (DPO) and its variants are increasingly used for aligning language models with human preferences. Although these methods are designed to teach a model to generate preferred responses more frequently relative to dispreferred responses, prior work has observed that the likelihood of preferred responses often decreases during training. The current work sheds light on the causes and implications of this counter-intuitive phenomenon, which we term likelihood displacement. We demonstrate that likelihood displacement can be catastrophic, shifting probability mass from preferred responses to responses with an opposite meaning. As a simple example, training a model to prefer $\texttt{No}$ over $\texttt{Never}$ can sharply increase the probability of $\texttt{Yes}$. Moreover, when aligning the model to refuse unsafe prompts, we show that such displacement can unintentionally lead to unalignment, by shifting probability mass from preferred refusal responses to harmful responses (e.g., reducing the refusal rate of Llama-3-8B-Instruct from 74.4% to 33.4%). We theoretically characterize that likelihood displacement is driven by preferences that induce similar embeddings, as measured by a centered hidden embedding similarity (CHES) score. Empirically, the CHES score enables identifying which training samples contribute most to likelihood displacement in a given dataset. Filtering out these samples effectively mitigated unintentional unalignment in our experiments. More broadly, our results highlight the importance of curating data with sufficiently distinct preferences, for which we believe the CHES score may prove valuable., Comment: Code available at https://github.com/princeton-nlp/unintentional-unalignment

Consider the scenario where a human cleans a table and a robot observing the scene is instructed with the task "Remove the cloth using which I wiped the table". Instruction following with temporal reasoning requires the robot to identify the relevant past object interaction, ground the object of interest in the present scene, and execute the task according to the human's instruction. Directly grounding utterances referencing past interactions to grounded objects is challenging due to the multi-hop nature of references to past interactions and large space of object groundings in a video stream observing the robot's workspace. Our key insight is to factor the temporal reasoning task as (i) estimating the video interval associated with event reference, (ii) performing spatial reasoning over the interaction frames to infer the intended object (iii) semantically track the object's location till the current scene to enable future robot interactions. Our approach leverages existing large pre-trained models (which possess inherent generalization capabilities) and combines them appropriately for temporal grounding tasks. Evaluation on a video-language corpus acquired with a robot manipulator displaying rich temporal interactions in spatially-complex scenes displays an average accuracy of 70.10%. The dataset, code, and videos are available at https://reail-iitdelhi.github.io/temporalreasoning.github.io/ .

Two-dimensional (2D) semiconductors are likely to dominate next-generation electronics due to their advantages in compactness and low power consumption. However, challenges such as high contact resistance and inefficient doping hinder their applicability. Here, we investigate workfunction-mediated charge transfer (modulation doping) as a pathway for achieving high-performance p-type 2D transistors. Focusing on type-III band alignment, we explore the doping capabilities of 27 candidate materials, including transition metal oxides, oxyhalides, and {\alpha}-RuCl3, on channel materials such as transition metal dichalcogenides (TMDs) and group-III nitrides. Our extensive first-principles density functional theory (DFT) reveal p-type doping capabilities of high electron affinity materials, including {\alpha}-RuCl3, MoO3, and V2O5. We predict significant reductions in contact resistance and enhanced channel mobility through efficient hole transfer without introducing detrimental defects. We analyze transistor geometries and identify promising material combinations beyond the current focus on WSe2 doping, suggesting new avenues for hBN, AlN, GaN, and MoS2. This comprehensive investigation provides a roadmap for developing high-performance p-type monolayer transistors toward the realization of 2D electronics.

Albedo is one of the important characteristics of hot Jupiter exoplanets. However, albedo constraints have been obtained for very few exoplanets. In this work, we present the TESS Phase Curve observations of WASP-18b, WASP-19b, WASP-121b, WASP-43b, WASP-17b, and WASP-77b, all JWST targets for atmospheric characterization and constrain their occultation depth as well as geometric albedo (A$_g$). We use a grid of self-consistent model atmospheres to constrain the metallicity, C/O ratio, and heat re-distribution for these six targets by fitting to their HST and/or Spitzer observations and also compute the thermal contribution to total occultation depth in the TESS bandpass. We report the first value of TESS occultation depth for WASP-17b ($151_{-66}^{+83}$) and updated value for WASP-77Ab ($94_{-62}^{+53}$). We find self-consistent models constrain high values of thermal contribution to total occultation compared to Planck models. We find very low A$_g$ values for WASP-18b (< 0.089), WASP-19b (< 0.022), WASP-121b ($0.0^{+0.055}_{-0.104}$), WASP-77Ab ($0.017^{+0.126}_{-0.147}$) and significantly higher value for WASP-43b ($0.109^{+0.086}_{-0.088}$) and WASP-17b ($0.401^{+0.526}_{-0.307}$). We find WASP-17b lies in the ideal spot of low gravity and low equilibrium temperature, conducive for cloud formation, leading to high A$_g$. With the best-fit models, we constrain low heat re-distribution for all planets, with WASP-18b having the least. We also constrain sub-solar metallicity for all planets except WASP-17b and WASP-19b. We find a highly sub-solar C/O ratio for WASP-77Ab and WASP-43b, solar for WASP-18b, and super-solar for WASP-121b. The best-fit $P$-$T$ profiles show thermal inversion for WASP-18b and WASP-121b and none for WASP-77b and WASP-43b, which is in agreement with previous works., Comment: 12 pages and 12 Figures. Accepted for Publication in MNRAS

Achieving low duty cycle operation in low-power wireless networks in urban environments is complicated by the complex and variable dynamics of external interference and fading. We explore the use of reinforcement learning for achieving low power consumption for the task of optimal selection of channels. The learning relies on a hybrid of passive channel sampling for dealing with external interference and active channel sampling for dealing with fading. Our solution, Passive-Active Multi-armed bandit for LoRa (PAMLR, pronounced "Pamela"), balances the two types of samples to achieve energy-efficient channel selection: active channel measurements are tuned to an appropriately low level to update noise thresholds, and to compensate passive channel measurements are tuned to an appropriately high level for selecting the top-most channels from channel exploration using the noise thresholds. The rates of both types of samples are adapted in response to channel dynamics. Based on extensive testing in multiple environments in different cities, we validate that PAMLR can maintain excellent communication quality, as demonstrated by a low SNR regret compared to the optimal channel allocation policy, while substantially minimizing the energy cost associated with channel measurements., Comment: BuildSys 23

Links are a fundamental part of information networks, turning isolated pieces of knowledge into a network of information that is much richer than the sum of its parts. However, adding a new link to the network is not trivial: it requires not only the identification of a suitable pair of source and target entities but also the understanding of the content of the source to locate a suitable position for the link in the text. The latter problem has not been addressed effectively, particularly in the absence of text spans in the source that could serve as anchors to insert a link to the target entity. To bridge this gap, we introduce and operationalize the task of entity insertion in information networks. Focusing on the case of Wikipedia, we empirically show that this problem is, both, relevant and challenging for editors. We compile a benchmark dataset in 105 languages and develop a framework for entity insertion called LocEI (Localized Entity Insertion) and its multilingual variant XLocEI. We show that XLocEI outperforms all baseline models (including state-of-the-art prompt-based ranking with LLMs such as GPT-4) and that it can be applied in a zero-shot manner on languages not seen during training with minimal performance drop. These findings are important for applying entity insertion models in practice, e.g., to support editors in adding links across the more than 300 language versions of Wikipedia., Comment: EMNLP 2024; 24 pages; 62 figures

$\Lambda$CDM cosmology predicts the hierarchical formation of galaxies which build up mass by merger events and accreting smaller systems. The stellar halo of the Milky Way has proven to be useful a tool for tracing this accretion history. However, most of this work has focused on the outer halo where dynamical times are large and the dynamical properties of accreted systems are preserved. In this work, we investigate the inner galaxy regime, where dynamical times are relatively small and systems are generally completely phase-mixed. Using the FIRE-2 and Auriga cosmological zoom-in simulation suites of Milky Way-mass galaxies, we find the stellar density profiles along the minor axis (perpendicular to the galactic disk) within the NFW scale radii (R$\approx$15 kpc) are best described as an exponential disk with scale height <0.3 kpc and a power law component with slope $\alpha\approx$-4. The stellar density amplitude and slope for the power law component is not significantly correlated with metrics of the galaxy's accretion history. Instead, we find the stellar profiles strongly correlate with the dark matter profile. Across simulation suites, the galaxies studied in this work have a stellar to dark matter mass ratio that decreases as $1/r^2$ along the minor axis., Comment: 29 pages, 13 figures, 1 table, Submitted to ApJ, comments welcome!

Time synchronization is a critical component in network operation and management, and it is also required by Ultra-Reliable, Low-Latency Communications (URLLC) in next-generation wireless systems such as those of 5G, 6G, and Open RAN. In this context, we design and implement AraSync as an end-to-end time synchronization system in the ARA wireless living lab to enable advanced wireless experiments and applications involving stringent time constraints. We make use of Precision Time Protocol (PTP) at different levels to achieve synchronization accuracy in the order of nanoseconds. Along with fiber networks, AraSync enables time synchronization across the AraHaul wireless x-haul network consisting of long-range, high-capacity mmWave and microwave links. In this paper, we present the detailed design and implementation of AraSync, including its hardware and software components and the PTP network topology. Further, we experimentally characterize the performance of AraSync from spatial and temporal dimensions. Our measurement and analysis of the clock offset and mean path delay show the impact of the wireless channel and weather conditions on the PTP synchronization accuracy., Comment: 8 pages, 10 figures, accepted in ACM WiNTECH 2024 (The 18th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & Characterization 2024)

We characterise quantitative semi-uniform stability for $C_0$-semigroups arising from port-Hamiltonian systems, complementing recent works on exponential and strong stability. With the result, we present a simple universal example class of port-Hamiltonian $C_0$-semigroups exhibiting arbitrary decay rates slower than $t^{-1/2}$. The latter is based on results from the theory of Diophantine approximation as the decay rates will be strongly related to approximation properties of irrational numbers by rationals given through cut-offs of continued fraction expansions., Comment: 22 pages

Intent detection is a critical component of task-oriented dialogue systems (TODS) which enables the identification of suitable actions to address user utterances at each dialog turn. Traditional approaches relied on computationally efficient supervised sentence transformer encoder models, which require substantial training data and struggle with out-of-scope (OOS) detection. The emergence of generative large language models (LLMs) with intrinsic world knowledge presents new opportunities to address these challenges. In this work, we adapt 7 SOTA LLMs using adaptive in-context learning and chain-of-thought prompting for intent detection, and compare their performance with contrastively fine-tuned sentence transformer (SetFit) models to highlight prediction quality and latency tradeoff. We propose a hybrid system using uncertainty based routing strategy to combine the two approaches that along with negative data augmentation results in achieving the best of both worlds ( i.e. within 2% of native LLM accuracy with 50% less latency). To better understand LLM OOS detection capabilities, we perform controlled experiments revealing that this capability is significantly influenced by the scope of intent labels and the size of the label space. We also introduce a two-step approach utilizing internal LLM representations, demonstrating empirical gains in OOS detection accuracy and F1-score by >5% for the Mistral-7B model., Comment: Accepted at EMNLP 2024 Industry Track

Let $X$ be a $G$-space. In this paper, we introduce the notion of sectional category with respect to $G$. As a result, we obtain $G$-homotopy invariants: the LS category with respect to $G$, the sequential topological complexity with respect to $G$ (which is same as the weak sequential equivariant topological complexity $\mathrm{TC}_{k,G}^w(X)$ in the sense of Farber and Oprea), and the strong sequential topological complexity with respect to $G$, denoted by $\mathrm{cat}_G^{\#}(X)$, $\mathrm{TC}_{k,G}^{\#}(X)$, and $\mathrm{TC}_{k,G}^{\#,*}(X)$, respectively. We explore several relationships among these invariants and well-known ones, such as the LS category, the sequential (equivariant) topological complexity, and the sequential strong equivariant topological complexity. In one of our main results, we give an additive upper bound for $\mathrm{TC}_k(E)$ for a fibre bundle $F \hookrightarrow E \to B$ with structure group $G$ in terms of certain motion planning covers of the base $B$ and the invariant $\mathrm{TC}_{k,G}^{\#,*}(F)$ or $\mathrm{cat}_{G^k}^{\#}(F^k)$, where the fibre $F$ is viewed as a $G$-space. As applications of these results, we give bounds on the sequential topological complexity of generalized projective product spaces and mapping tori., Comment: 27 pages. Title has been changed. Proposition 2.35 has been improved by removing assumptions on fixed points, following suggestions from Prof. Oprea. Additionally, Section 6 has been added, containing applications to mapping tori. Any comments are most welcome

As large language models (LLMs) become increasingly advanced, their ability to exhibit compositional generalization -- the capacity to combine learned skills in novel ways not encountered during training -- has garnered significant attention. This type of generalization, particularly in scenarios beyond training data, is also of great interest in the study of AI safety and alignment. A recent study introduced the SKILL-MIX evaluation, where models are tasked with composing a short paragraph demonstrating the use of a specified $k$-tuple of language skills. While small models struggled with composing even with $k=3$, larger models like GPT-4 performed reasonably well with $k=5$ and $6$. In this paper, we employ a setup akin to SKILL-MIX to evaluate the capacity of smaller models to learn compositional generalization from examples. Utilizing a diverse set of language skills -- including rhetorical, literary, reasoning, theory of mind, and common sense -- GPT-4 was used to generate text samples that exhibit random subsets of $k$ skills. Subsequent fine-tuning of 7B and 13B parameter models on these combined skill texts, for increasing values of $k$, revealed the following findings: (1) Training on combinations of $k=2$ and $3$ skills results in noticeable improvements in the ability to compose texts with $k=4$ and $5$ skills, despite models never having seen such examples during training. (2) When skill categories are split into training and held-out groups, models significantly improve at composing texts with held-out skills during testing despite having only seen training skills during fine-tuning, illustrating the efficacy of the training approach even with previously unseen skills. This study also suggests that incorporating skill-rich (potentially synthetic) text into training can substantially enhance the compositional capabilities of models., Comment: Accepted to NeurIPS 2024