Search

Self-consistency decoding enhances LLMs' performance on reasoning tasks by sampling diverse reasoning paths and selecting the most frequent answer. However, it is computationally expensive, as sampling many of these (lengthy) paths is required to increase the chances that the correct answer emerges as the most frequent one. To address this, we introduce Confidence-Informed Self-Consistency (CISC). CISC performs a weighted majority vote based on confidence scores obtained directly from the model. By prioritizing high-confidence paths, it can identify the correct answer with a significantly smaller sample size. When tested on nine models and four datasets, CISC outperforms self-consistency in nearly all configurations, reducing the required number of reasoning paths by over 40% on average. In addition, we introduce the notion of within-question confidence evaluation, after showing that standard evaluation methods are poor predictors of success in distinguishing correct and incorrect answers to the same question. In fact, the most calibrated confidence method proved to be the least effective for CISC. Lastly, beyond these practical implications, our results and analyses show that LLMs can effectively judge the correctness of their own outputs, contributing to the ongoing debate on this topic.

Online Continual Learning (OCL) is a critical area in machine learning, focusing on enabling models to adapt to evolving data streams in real-time while addressing challenges such as catastrophic forgetting and the stability-plasticity trade-off. This study conducts the first comprehensive Systematic Literature Review (SLR) on OCL, analyzing 81 approaches, extracting over 1,000 features (specific tasks addressed by these approaches), and identifying more than 500 components (sub-models within approaches, including algorithms and tools). We also review 83 datasets spanning applications like image classification, object detection, and multimodal vision-language tasks. Our findings highlight key challenges, including reducing computational overhead, developing domain-agnostic solutions, and improving scalability in resource-constrained environments. Furthermore, we identify promising directions for future research, such as leveraging self-supervised learning for multimodal and sequential data, designing adaptive memory mechanisms that integrate sparse retrieval and generative replay, and creating efficient frameworks for real-world applications with noisy or evolving task boundaries. By providing a rigorous and structured synthesis of the current state of OCL, this review offers a valuable resource for advancing this field and addressing its critical challenges and opportunities. The complete SLR methodology steps and extracted data are publicly available through the provided link: https://github.com/kiyan-rezaee/ Systematic-Literature-Review-on-Online-Continual-Learning

The all-optical control of magnetization at room temperature broadens the scope of applications of spin degrees-of-freedom in data storage, spintronics, and quantum computing. Topological magnetic spin structures, such as skyrmions, are of particular interest due to their particle-like properties, small size and inherent stability. Controlling skyrmion states without strong magnetic fields or large current densities would create new possibilities for their application. In this work, we utilize femtosecond optical pulses to alter the helicity of the spin configuration in dipolar skyrmions formed in the kagome magnet Fe$_3$Sn$_2$ in the absence of an external magnetic field and at room temperature. In situ Lorentz transmission electron microscopy is used to visualize the stochastic, light-induced switching process of chiral N\'eel caps, while the internal Bloch component of the dipolar skyrmions remain unchanged. In addition to this switching process, we observe the interconversion between type I skyrmionic and type II bubble configurations depending on the external magnetic field and illumination conditions. To corroborate the spin states and the light-induced magnetization dynamics, micromagnetic modelling and simulations of the resulting electron phase shift maps are conducted to elucidate the spin rearrangement induced by individual femtosecond optical pulses., Comment: J.T. Weber and A. Kov\'acs contributed equally to this work

This white paper discusses the role of large-scale AI in the telecommunications industry, with a specific focus on the potential of generative AI to revolutionize network functions and user experiences, especially in the context of 6G systems. It highlights the development and deployment of Large Telecom Models (LTMs), which are tailored AI models designed to address the complex challenges faced by modern telecom networks. The paper covers a wide range of topics, from the architecture and deployment strategies of LTMs to their applications in network management, resource allocation, and optimization. It also explores the regulatory, ethical, and standardization considerations for LTMs, offering insights into their future integration into telecom infrastructure. The goal is to provide a comprehensive roadmap for the adoption of LTMs to enhance scalability, performance, and user-centric innovation in telecom networks.

Diffusion models exhibit impressive scalability in robotic task learning, yet they struggle to adapt to novel, highly dynamic environments. This limitation primarily stems from their constrained replanning ability: they either operate at a low frequency due to a time-consuming iterative sampling process, or are unable to adapt to unforeseen feedback in case of rapid replanning. To address these challenges, we propose RA-DP, a novel diffusion policy framework with training-free high-frequency replanning ability that solves the above limitations in adapting to unforeseen dynamic environments. Specifically, our method integrates guidance signals which are often easily obtained in the new environment during the diffusion sampling process, and utilizes a novel action queue mechanism to generate replanned actions at every denoising step without retraining, thus forming a complete training-free framework for robot motion adaptation in unseen environments. Extensive evaluations have been conducted in both well-recognized simulation benchmarks and real robot tasks. Results show that RA-DP outperforms the state-of-the-art diffusion-based methods in terms of replanning frequency and success rate. Moreover, we show that our framework is theoretically compatible with any training-free guidance signal.

Large-scale battery energy storage systems (BESS) have found ever-increasing use across industry and society to accelerate clean energy transition and improve energy supply reliability and resilience. However, their optimal power management poses significant challenges: the underlying high-dimensional nonlinear nonconvex optimization lacks computational tractability in real-world implementation, and the uncertainty of the exogenous power demand makes exact optimization difficult. This paper presents a new solution framework to address these bottlenecks. The solution pivots on introducing power-sharing ratios to specify each cell's power quota from the output power demand. To find the optimal power-sharing ratios, we formulate a nonlinear model predictive control (NMPC) problem to achieve power-loss-minimizing BESS operation while complying with safety, cell balancing, and power supply-demand constraints. We then propose a parameterized control policy for the power-sharing ratios, which utilizes only three parameters, to reduce the computational demand in solving the NMPC problem. This policy parameterization allows us to translate the NMPC problem into a Bayesian inference problem for the sake of 1) computational tractability, and 2) overcoming the nonconvexity of the optimization problem. We leverage the ensemble Kalman inversion technique to solve the parameter estimation problem. Concurrently, a low-level control loop is developed to seamlessly integrate our proposed approach with the BESS to ensure practical implementation. This low-level controller receives the optimal power-sharing ratios, generates output power references for the cells, and maintains a balance between power supply and demand despite uncertainty in output power. We conduct extensive simulations and experiments on a 20-cell prototype to validate the proposed approach.

Long-term monitoring and exploration of extreme environments, such as underwater storage facilities, is costly, labor-intensive, and hazardous. Automating this process with low-cost, collaborative robots can greatly improve efficiency. These robots capture images from different positions, which must be processed simultaneously to create a spatio-temporal model of the facility. In this paper, we propose a novel approach that integrates data simulation, a multi-modal deep learning network for coordinate prediction, and image reassembly to address the challenges posed by environmental disturbances causing drift and rotation in the robots' positions and orientations. Our approach enhances the precision of alignment in noisy environments by integrating visual information from snapshots, global positional context from masks, and noisy coordinates. We validate our method through extensive experiments using synthetic data that simulate real-world robotic operations in underwater settings. The results demonstrate very high coordinate prediction accuracy and plausible image assembly, indicating the real-world applicability of our approach. The assembled images provide clear and coherent views of the underwater environment for effective monitoring and inspection, showcasing the potential for broader use in extreme settings, further contributing to improved safety, efficiency, and cost reduction in hazardous field monitoring. Code is available on https://github.com/ChrisChen1023/Micro-Robot-Swarm., Comment: Accepted by ICRA 2025

The R2D2 Deep Neural Network (DNN) series was recently introduced for image formation in radio interferometry. It can be understood as a learned version of CLEAN, whose minor cycles are substituted with DNNs. We revisit R2D2 on the grounds of series convergence, training methodology, and DNN architecture, improving its robustness in terms of generalisability beyond training conditions, capability to deliver high data fidelity, and epistemic uncertainty. Firstly, while still focusing on telescope-specific training, we enhance the learning process by randomising Fourier sampling integration times, incorporating multi-scan multi-noise configurations, and varying imaging settings, including pixel resolution and visibility-weighting scheme. Secondly, we introduce a convergence criterion whereby the reconstruction process stops when the data residual is compatible with noise, rather than simply using all available DNNs. This not only increases the reconstruction efficiency by reducing its computational cost, but also refines training by pruning out the data/image pairs for which optimal data fidelity is reached before training the next DNN. Thirdly, we substitute R2D2's early U-Net DNN with a novel architecture (U-WDSR) combining U-Net and WDSR, which leverages wide activation, dense connections, weight normalisation, and low-rank convolution to improve feature reuse and reconstruction precision. As previously, R2D2 was trained for monochromatic intensity imaging with the Very Large Array (VLA) at fixed $512 \times 512$ image size. Simulations on a wide range of inverse problems and a case study on real data reveal that the new R2D2 model consistently outperforms its earlier version in image reconstruction quality, data fidelity, and epistemic uncertainty., Comment: 17 pages, 6 figures

Recent advancements in cognitive science and multi-round reasoning techniques for Large Language Models (LLMs) suggest that iterative thinking processes improve problem-solving performance in complex tasks. Inspired by this, approaches like Chain-of-Thought, debating, and self-refinement have been applied to auto-regressive LLMs, achieving significant successes in tasks such as mathematical reasoning, commonsense reasoning, and multi-hop question answering. Despite these successes, the theoretical basis for how multi-round reasoning enhances problem-solving abilities remains underexplored. In this work, we investigate the approximation, learnability, and generalization properties of multi-round auto-regressive models. We show that Transformers with finite context windows are universal approximators for steps of Turing-computable functions and can approximate any Turing-computable sequence-to-sequence function through multi-round reasoning. We extend PAC learning to sequence generation and demonstrate that multi-round generation is learnable even when the sequence length exceeds the model's context window. Finally, we examine how generalization error propagates across rounds, and show how the aforementioned approaches can help constrain this error, ensuring outputs stay within an expectation boundary. This work sheds light on the systemic theoretical foundations of multi-round sequence learning and reasoning, emphasizing its role in inference complexity.

Transformers have revolutionized the field of machine learning. In particular, they can be used to solve complex algorithmic problems, including graph-based tasks. In such algorithmic tasks a key question is what is the minimal size of a transformer that can implement a task. Recent work has begun to explore this problem for graph-based tasks, showing that for sub-linear embedding dimension (i.e., model width) logarithmic depth suffices. However, an open question, which we address here, is what happens if width is allowed to grow linearly. Here we analyze this setting, and provide the surprising result that with linear width, constant depth suffices for solving a host of graph-based problems. This suggests that a moderate increase in width can allow much shallower models, which are advantageous in terms of inference time. For other problems, we show that quadratic width is required. Our results demonstrate the complex and intriguing landscape of transformer implementations of graph-based algorithms. We support our theoretical results with empirical evaluations.

This paper presents a comprehensive software stack architecture for integrating quantum computing (QC) capabilities with High-Performance Computing (HPC) environments. While quantum computers show promise as specialized accelerators for scientific computing, their effective integration with classical HPC systems presents significant technical challenges. We propose a hardware-agnostic software framework that supports both current noisy intermediate-scale quantum devices and future fault-tolerant quantum computers, while maintaining compatibility with existing HPC workflows. The architecture includes a quantum gateway interface, standardized APIs for resource management, and robust scheduling mechanisms to handle both simultaneous and interleaved quantum-classical workloads. Key innovations include: (1) a unified resource management system that efficiently coordinates quantum and classical resources, (2) a flexible quantum programming interface that abstracts hardware-specific details, (3) A Quantum Platform Manager API that simplifies the integration of various quantum hardware systems, and (4) a comprehensive tool chain for quantum circuit optimization and execution. We demonstrate our architecture through implementation of quantum-classical algorithms, including the variational quantum linear solver, showcasing the framework's ability to handle complex hybrid workflows while maximizing resource utilization. This work provides a foundational blueprint for integrating QC capabilities into existing HPC infrastructures, addressing critical challenges in resource management, job scheduling, and efficient data movement between classical and quantum resources., Comment: 14 pages, 14 figures

In continual time series analysis using neural networks, catastrophic forgetting (CF) of previously learned models when training on new data domains has always been a significant challenge. This problem is especially challenging in vehicle estimation and control, where new information is sequentially introduced to the model. Unfortunately, existing work on continual learning has not sufficiently addressed the adverse effects of catastrophic forgetting in time series analysis, particularly in multivariate output environments. In this paper, we present EM-ReSeleCT (Efficient Multivariate Representative Selection for Continual Learning in Time Series Tasks), an enhanced approach designed to handle continual learning in multivariate environments. Our approach strategically selects representative subsets from old and historical data and incorporates memory-based continual learning techniques with an improved optimization algorithm to adapt the pre-trained model on new information while preserving previously acquired information. Additionally, we develop a sequence-to-sequence transformer model (autoregressive model) specifically designed for vehicle state estimation. Moreover, we propose an uncertainty quantification framework using conformal prediction to assess the sensitivity of the memory size and to showcase the robustness of the proposed method. Experimental results from tests on an electric Equinox vehicle highlight the superiority of our method in continually learning new information while retaining prior knowledge, outperforming state-of-the-art continual learning methods. Furthermore, EM-ReSeleCT significantly reduces training time, a critical advantage in continual learning applications.

Video object segmentation is an emerging technology that is well-suited for real-time surgical video segmentation, offering valuable clinical assistance in the operating room by ensuring consistent frame tracking. However, its adoption is limited by the need for manual intervention to select the tracked object, making it impractical in surgical settings. In this work, we tackle this challenge with an innovative solution: using previously annotated frames from other patients as the tracking frames. We find that this unconventional approach can match or even surpass the performance of using patients' own tracking frames, enabling more autonomous and efficient AI-assisted surgical workflows. Furthermore, we analyze the benefits and limitations of this approach, highlighting its potential to enhance segmentation accuracy while reducing the need for manual input. Our findings provide insights into key factors influencing performance, offering a foundation for future research on optimizing cross-patient frame selection for real-time surgical video analysis.

At least since Francis Bacon, the slogan 'knowledge is power' has been used to capture the relationship between decision-making at a group level and information. We know that being able to shape the informational environment for a group is a way to shape their decisions; it is essentially a way to make decisions for them. This paper focuses on strategies that are intentionally, by design, impactful on the decision-making capacities of groups, effectively shaping their ability to take advantage of information in their environment. Among these, the best known are political rhetoric, propaganda, and misinformation. The phenomenon this paper brings out from these is a relatively new strategy, which we call slopaganda. According to The Guardian, News Corp Australia is currently churning out 3000 'local' generative AI (GAI) stories each week. In the coming years, such 'generative AI slop' will present multiple knowledge-related (epistemic) challenges. We draw on contemporary research in cognitive science and artificial intelligence to diagnose the problem of slopaganda, describe some recent troubling cases, then suggest several interventions that may help to counter slopaganda., Comment: 19 pages, pre-print Filosofiska Notiser

Cuscuton Gravity is characterized as a scalar field that can be added to general relativity without introducing any new dynamical degrees of freedom on a cosmological background. Yet, it modifies gravity such that spacetime singularities can be avoided. This has led to the Cuscuton bounce, a nonsingular cosmology that has been shown to be linearly stable, which is a rare feat. Upon introducing mechanisms known to generate a near-scale-invariant power spectrum of isocurvature perturbations in the prebounce contracting phase, we perform an extensive linear analysis of all scalar perturbations as they evolve through the Cuscuton bounce, both analytically and numerically. Then, after deriving the third-order perturbed action for our theory, we compare the magnitude of its terms (on shell) to those in the second-order action. We show that perturbativity is maintained in the infrared throughout the evolution, including through the bounce. In the ultraviolet, we show that a hierarchy of scales is maintained, with the strong coupling scale well above the relevant background energy scale at all times. We reconfirm these results by computing the three-point functions in various limits and demonstrate that the models do not have any strong coupling problems and furthermore that there is negligible non-Gaussianities on observable scales. Consequently, the primary potential source of observable non-Gaussianities may only arise from the conversion of isocurvature perturbations to curvature perturbations. The whole scenario is thus a robust, stable, weakly coupled nonsingular cosmological model, consistent with observations., Comment: 66 pages, 6 figures

As quantum computing technology advances, the complexity of quantum algorithms increases, necessitating a shift from low-level circuit descriptions to high-level programming paradigms. This paper addresses the challenges of developing a compilation algorithm that optimizes memory management and scales well for bigger, more complex circuits. Our approach models the high-level quantum code as a control flow graph and presents a workflow that searches for a topological sort that maximizes opportunities for qubit reuse. Various heuristics for qubit reuse strategies handle the trade-off between circuit width and depth. We also explore scalability issues in large circuits, suggesting methods to mitigate compilation bottlenecks. By analyzing the structure of the circuit, we are able to identify sub-problems that can be solved separately, without a significant effect on circuit quality, while reducing runtime significantly. This method lays the groundwork for future advancements in quantum programming and compiler optimization by incorporating scalability into quantum memory management.

Accurate prediction of fracture toughness under complex loading conditions, like mixed mode I/II, is essential for reliable failure assessment. This paper aims to develop a machine learning framework for predicting fracture toughness and crack initiation angles by directly utilizing stress, strain, or displacement distributions represented by selected nodes as input features. Validation is conducted using experimental data across various mode mixities and specimen geometries for brittle materials. Among stress, strain, and displacement fields, it is shown that the stress-based features, when paired with Multilayer Perceptron models, achieve high predictive accuracy with R2 scores exceeding 0.86 for fracture load predictions and 0.94 for angle predictions. A comparison with the Theory of Critical Distances (Generalized Maximum Tangential Stress) demonstrates the high accuracy of the framework. Furthermore, the impact of input parameter selections is studied, and it is demonstrated that advanced feature selection algorithms enable the framework to handle different ranges and densities of the representing field. The framework's performance was further validated for datasets with a limited number of data points and restricted mode mixities, where it maintained high accuracy. The proposed framework is computationally efficient and practical, and it operates without any supplementary post-processing steps, such as stress intensity factor calculations.

We present Dur360BEV, a novel spherical camera autonomous driving dataset equipped with a high-resolution 128-channel 3D LiDAR and a RTK-refined GNSS/INS system, along with a benchmark architecture designed to generate Bird-Eye-View (BEV) maps using only a single spherical camera. This dataset and benchmark address the challenges of BEV generation in autonomous driving, particularly by reducing hardware complexity through the use of a single 360-degree camera instead of multiple perspective cameras. Within our benchmark architecture, we propose a novel spherical-image-to-BEV module that leverages spherical imagery and a refined sampling strategy to project features from 2D to 3D. Our approach also includes an innovative application of focal loss, specifically adapted to address the extreme class imbalance often encountered in BEV segmentation tasks, that demonstrates improved segmentation performance on the Dur360BEV dataset. The results show that our benchmark not only simplifies the sensor setup but also achieves competitive performance.

Safe and reliable operation of lithium-ion battery packs depends on effective fault diagnosis. However, model-based approaches often encounter two major challenges: high computational complexity and extensive sensor requirements. To address these bottlenecks, this paper introduces a novel approach that harnesses the structural properties of battery packs, including cell uniformity and the sparsity of fault occurrences. We integrate this approach into a Moving Horizon Estimation (MHE) framework and estimate fault signals such as internal and external short circuits and faults in voltage and current sensors. To mitigate computational demands, we propose a hierarchical solution to the MHE problem. The proposed solution breaks up the pack-level MHE problem into smaller problems and solves them efficiently. Finally, we perform extensive simulations across various battery pack configurations and fault types to demonstrate the effectiveness of the proposed approach. The results highlight that the proposed approach simultaneously reduces the computational demands and sensor requirements of fault diagnosis.

In 2006, Ihara defined and systematically studied a generalization of the Euler-Mascheroni constant for all global fields, named the Euler-Kronecker constants. This paper examines their distribution across geometric quadratic extensions of a rational global function field, via the values of logarithmic derivatives of Dirichlet $L$-functions at 1. Using a probabilistic model, we show that the values converge to a limiting distribution with a smooth, positive density function, as the genii of quadratic fields approach infinity. We then prove a discrepancy theorem for the convergence of the frequency of these values, and obtain information about the proportion of the small values. Finally, we prove omega results on the extreme values. Our theorems imply new distribution results on the stable Taguchi heights and logarithmic Weil heights of rank 2 Drinfeld modules with CM., Comment: 30 pages

Sparse and feature SLAM methods provide robust camera pose estimation. However, they often fail to capture the level of detail required for inspection and scene awareness tasks. Conversely, dense SLAM approaches generate richer scene reconstructions but impose a prohibitive computational load to create 3D maps. We present a novel distributed volumetric mapping framework designated as CRADMap that addresses these issues by extending the state-of-the-art (SOTA) ORBSLAM3 [1] system with the COVINS [2] on the backend for global optimization. Our pipeline for volumetric reconstruction fuses dense keyframes at a centralized server via 5G connectivity, aggregating geometry, and occupancy information from multiple autonomous mobile robots (AMRs) without overtaxing onboard resources. This enables each AMR to independently perform mapping while the backend constructs high-fidelity 3D maps in real time. To overcome the limitation of standard visual nodes we automate a 4D mmWave radar, standalone from CRADMap, to test its capabilities for making extra maps of the hidden metallic object(s) in a cluttered environment. Experimental results Section-IV confirm that our framework yields globally consistent volumetric reconstructions and seamlessly supports applied distributed mapping in complex indoor environments., Comment: 7 pages, 5 figures, IEEE

Large Language Models (LLMs) effectively leverage common-sense knowledge for general reasoning, yet they struggle with personalized reasoning when tasked with interpreting multifactor personal data. This limitation restricts their applicability in domains that require context-aware decision-making tailored to individuals. This paper introduces Personalized Causal Graph Reasoning as an agentic framework that enhances LLM reasoning by incorporating personal causal graphs derived from data of individuals. These graphs provide a foundation that guides the LLM's reasoning process. We evaluate it on a case study on nutrient-oriented dietary recommendations, which requires personal reasoning due to the implicit unique dietary effects. We propose a counterfactual evaluation to estimate the efficiency of LLM-recommended foods for glucose management. Results demonstrate that the proposed method efficiently provides personalized dietary recommendations to reduce average glucose iAUC across three time windows, which outperforms the previous approach. LLM-as-a-judge evaluation results indicate that our proposed method enhances personalization in the reasoning process.

Background and Purpose: Glioma segmentation is crucial for clinical decisions and treatment planning. Uncertainty quantification methods, including conformal prediction (CP), can enhance segmentation models reliability. This study aims to use CP in glioma segmentation. Methods: We used the UCSF and UPenn glioma datasets, with the UCSF dataset split into training (70%), validation (10%), calibration (10%), and test (10%) sets, and the UPenn dataset divided into external calibration (30%) and external test (70%) sets. A UNet model was trained, and its optimal threshold was set to 0.5 using prediction normalization. To apply CP, the conformal threshold was selected based on the internal/external calibration nonconformity score, and CP was subsequently applied to the internal/external test sets, with coverage reported for all. We defined the uncertainty ratio (UR) and assessed its correlation with the Dice score coefficient (DSC). Additionally, we categorized cases into certain and uncertain groups based on UR and compared their DSC. We also evaluate the correlation between UR and DSC of the BraTS fusion model segmentation (BFMS), and compare DSC in the certain and uncertain subgroups. Results: The base model achieved a DSC of 0.8628 and 0.8257 on the internal and external test sets, respectively. The CP coverage was 0.9982 for the internal test set and 0.9977 for the external test set. Statistical analysis showed a significant negative correlation between UR and DSC for test sets (p<0.001). UR was also linked to significantly lower DSCs in the BFMS (p<0.001). Additionally, certain cases had significantly higher DSCs than uncertain cases in test sets and the BFMS (p<0.001). Conclusion: CP effectively quantifies uncertainty in glioma segmentation. Using CONSeg improves the reliability of segmentation models and enhances human-computer interaction., Comment: 15 pages, 2 figures, 4 tables, 9 supplementary figures

Real-time video segmentation is a promising feature for AI-assisted surgery, providing intraoperative guidance by identifying surgical tools and anatomical structures. However, deploying state-of-the-art segmentation models, such as SAM2, in real-time settings is computationally demanding, which makes it essential to balance frame rate and segmentation performance. In this study, we investigate the impact of frame rate on zero-shot surgical video segmentation, evaluating SAM2's effectiveness across multiple frame sampling rates for cholecystectomy procedures. Surprisingly, our findings indicate that in conventional evaluation settings, frame rates as low as a single frame per second can outperform 25 FPS, as fewer frames smooth out segmentation inconsistencies. However, when assessed in a real-time streaming scenario, higher frame rates yield superior temporal coherence and stability, particularly for dynamic objects such as surgical graspers. Finally, we investigate human perception of real-time surgical video segmentation among professionals who work closely with such data and find that respondents consistently prefer high FPS segmentation mask overlays, reinforcing the importance of real-time evaluation in AI-assisted surgery.

In multidimensional NMR spectroscopy, practical resolution is defined as the ability to distinguish and accurately determine signal positions against a background of overlapping peaks, thermal noise, and spectral artifacts. In the pursuit of ultimate resolution, we introduce Peak Probability Presentations ($P^3$)- a statistical spectral representation that assigns a probability to each spectral point, indicating the likelihood of a peak maximum occurring at that location. The mapping between the spectrum and $P^3$ is achieved using MR-Ai, a physics-inspired deep learning neural network architecture, designed to handle multidimensional NMR spectra. Furthermore, we demonstrate that MR-Ai enables coprocessing of multiple spectra, facilitating direct information exchange between datasets. This feature significantly enhances spectral quality, particularly in cases of highly sparse sampling. Performance of MR-Ai and high value of the $P^3$ are demonstrated on the synthetic data and spectra of Tau, MATL1, Calmodulin, and several other proteins.

This work aims to provide both privacy and utility within a split learning framework while considering both forward attribute inference and backward reconstruction attacks. To address this, a novel approach has been proposed, which makes use of class activation maps and autoencoders as a plug-in strategy aiming to increase the user's privacy and destabilize an adversary. The proposed approach is compared with a dimensionality-reduction-based plug-in strategy, which makes use of principal component analysis to transform the feature map onto a lower-dimensional feature space. Our work shows that our proposed autoencoder-based approach is preferred as it can provide protection at an earlier split position over the tested architectures in our setting, and, hence, better utility for resource-constrained devices in edge-cloud collaborative inference (EC) systems., Comment: 15 pages, 6 figures

This paper presents the development and application of a Large Language Model Retrieval-Augmented Generation (LLM-RAG) system tailored for nanotechnology research. The system leverages the capabilities of a sophisticated language model to serve as an intelligent research assistant, enhancing the efficiency and comprehensiveness of literature reviews in the nanotechnology domain. Central to this LLM-RAG system is its advanced query backend retrieval mechanism, which integrates data from multiple reputable sources. The system retrieves relevant literature by utilizing Google Scholar's advanced search, and scraping open-access papers from Elsevier, Springer Nature, and ACS Publications. This multifaceted approach ensures a broad and diverse collection of up-to-date scholarly articles and papers. The proposed system demonstrates significant potential in aiding researchers by providing a streamlined, accurate, and exhaustive literature retrieval process, thereby accelerating research advancements in nanotechnology. The effectiveness of the LLM-RAG system is validated through rigorous testing, illustrating its capability to significantly reduce the time and effort required for comprehensive literature reviews, while maintaining high accuracy, query relevance and outperforming standard, publicly available LLMS., Comment: 61 pages, 3 figures

Let $R=K[x_1,\ldots, x_n]$ be the polynomial ring in $n$ variables over a field $K$ and $I$ be monomial ideal of $R$. In this paper, we show that if $I$ is a generic monomial ideal, then $R/I$ is pretty clean if and only if $R/I$ is sequentially Cohen-Macaulay. Furthermore, we prove that this equivalence remains unchanged for some special monomial ideals. Moreover, we provide an example that disproves the conjecture raised in \cite[p. 123]{S1} regarding generic monomial ideals., Comment: 7 pages. Comments welcome

Quantum computing hardware is advancing at a rapid pace, yet the lack of high-level programming abstractions remains a serious bottleneck in the development of new applications. Widely used frameworks still rely on gate-level circuit descriptions, causing the algorithm's functional intent to become lost in low-level implementation details, and hindering flexibility and reuse. While various high-level quantum programming languages have emerged in recent years - offering a significant step toward higher abstraction - many still lack support for classical-like expression syntax, and native constructs for useful quantum algorithmic idioms. This paper presents Qmod, a high-level quantum programming language designed to capture algorithmic intent in natural terms while delegating implementation decisions to automation. Qmod introduces quantum numeric variables and expressions, including digital fixed-point arithmetic tuned for compact representations and optimal resource usage. Beyond digital encoding, Qmod also supports non-digital expression modes - phase and amplitude encoding - frequently exploited by quantum algorithms to achieve computational advantages. We describe the language's constructs, demonstrate practical usage examples, and outline future work on evaluating Qmod across a broader set of use cases., Comment: 9 pages

Mode division multiplexing (MDM) systems leveraging spatial modes carrying orbital angular momentum (OAM) present a promising approach to enhance communication capacity in free-space and fiber-optic networks. Efficient detection of OAM modes is critical for their practical implementation. Spiral phase plates (SPPs) are low-cost optical elements with simple structures, capable of generating OAM waves with high conversion efficiency and precision. This makes the inverse-SPP a promising candidate for the detection of OAM modes. In this study, the performance of SPPs as OAM detectors is thoroughly analyzed. Key parameters, including efficiency, crosstalk, and signal-to-interference ratio, are examined through analytical and numerical methods. Results demonstrate that an inverse-SPP, combined with an optimized propagation length and aperture size, enables effective and precise OAM detection operation., Comment: 15 pages, 9 figures, 1 table

The goal of graph inference is to design algorithms for learning properties of a hidden graph using queries to an oracle that returns information about the graph. Graph reconstruction, verification, and property testing are all types of graph inference. In this work, we study graph inference using an oracle that returns the effective resistance (ER) between a pair of vertices. Effective resistance is a distance originating from the study of electrical circuits with many applications. However, ER has received little attention from a graph inference perspective. Indeed, although it is known that an $n$-vertex graph can be uniquely reconstructed from all $\binom{n}{2}$ possible ER queries, little else is known. We address this gap with several new results, including: 1. $O(n)$-query algorithms for testing whether a graph is a tree; deciding whether two graphs are equal assuming one is a subgraph of the other; and testing whether a given vertex (or edge) is a cut vertex (or cut edge). 2. Property testing algorithms, including for testing whether a graph is vertex- or edge-biconnected. We also give a reduction to adapt property testing results from the bounded-degree model to our ER query model. This yields ER-query-based algorithms for testing $k$-connectivity, bipartiteness, planarity, and containment of a fixed subgraph. 3. Graph reconstruction algorithms, including an algorithm for reconstructing a graph from a low-width tree decomposition; a $\Theta(k^2)$-query, polynomial-time algorithm for recovering the adjacency matrix $A$ of a hidden graph, given $A$ with $k$ of its entries deleted; and a $k$-query, exponential-time algorithm for the same task. We also compare the power of ER queries and shortest path queries, which are closely related but better studied. Interestingly, we show that the two query models are incomparable in power.

G-Protein Coupled Receptors (GPCRs) are integral to numerous physiological processes and are the target of approximately one-third of FDA-approved therapeutics. Despite their significance, only a limited subset of GPCRs has been successfully targeted, primarily due to challenges in accurately modeling their structures. AlphaFold, a state-of-the-art deep learning model, has demonstrated remarkable capability in predicting protein structures with high accuracy. This study conducts an evaluation of AlphaFold performance in predicting GPCR structures and their conformational states by comparing its predictions to experimentally determined structures using metrics such as average deformation between alpha carbon atoms and the Helix 3 - Helix 6 (H3-H6) distance. Our analysis reveals that both AlphaFold 2 (AF2) and AlphaFold 3 (AF3) produce more accurate predictions for GPCRs in inactive conformations, with lower activity levels correlating with smaller deformations. Conversely, higher activity levels are associated with increased variability in AlphaFold performance due to difficulties with accurately predicting conformational changes upon GPCR activation and ligand binding. Additionally, AlphaFold performance varies across different GPCR classes, influenced by the availability and quality of training data as well as the structural complexity and diversity of the receptors. These findings demonstrate the potential of AlphaFold in advancing drug discovery efforts, while also highlighting the necessity for continued refinement to enhance predictive accuracy for active conformations.

In large language models (LLMs), certain neurons can store distinct pieces of knowledge learned during pretraining. While knowledge typically appears as a combination of relations and entities, it remains unclear whether some neurons focus on a relation itself -- independent of any entity. We hypothesize such neurons detect a relation in the input text and guide generation involving such a relation. To investigate this, we study the Llama-2 family on a chosen set of relations with a statistics-based method. Our experiments demonstrate the existence of relation-specific neurons. We measure the effect of selectively deactivating candidate neurons specific to relation $r$ on the LLM's ability to handle (1) facts whose relation is $r$ and (2) facts whose relation is a different relation $r' \neq r$. With respect to their capacity for encoding relation information, we give evidence for the following three properties of relation-specific neurons. $\textbf{(i) Neuron cumulativity.}$ The neurons for $r$ present a cumulative effect so that deactivating a larger portion of them results in the degradation of more facts in $r$. $\textbf{(ii) Neuron versatility.}$ Neurons can be shared across multiple closely related as well as less related relations. Some relation neurons transfer across languages. $\textbf{(iii) Neuron interference.}$ Deactivating neurons specific to one relation can improve LLM generation performance for facts of other relations. We will make our code publicly available at https://github.com/cisnlp/relation-specific-neurons., Comment: preprint

The development of high-entropy alloys (HEAs) has marked a paradigm shift in alloy design, moving away from traditional methods that prioritize a dominant base metal enhanced by minor elements. HEAs instead incorporate multiple alloying elements with no single dominant component, broadening the scope of alloy design. This shift has led to the creation of diverse alloys with high entropy (AHEs) families, including high-entropy steels, superalloys, and intermetallics, each highlighting the need to consider additional factors such as stacking fault energy (SFE), lattice misfit, and anti-phase boundary energy (APBE) due to their significant influence on microstructure and performance. Leveraging multiple elements in alloying opens up promising possibilities for developing new alloys from multi-component scrap and electronic waste, reducing reliance on critical metals and emphasizing the need for advanced data generation techniques. With the vast possibilities offered by these multi-component feedstocks, modelling and Artificial Intelligence based tools are essential to efficiently explore and optimize new alloys, supporting sustainable progress in metallurgy. These advancements call for a reimagined alloy design framework, emphasizing robust data acquisition, alternative design parameters, and advanced computational tools over traditional composition-focused methodologies.

For practical superconducting quantum processors, orders of magnitude improvement in coherence is required, motivating efforts to optimize hardware design and explore new materials. Among the latter, the coherence of superconducting transmon qubits has been shown to improve by forming the qubit capacitor pads from $\alpha$-tantalum, avoiding the meta-stable $\beta$-phase that forms when depositing tantalum at room temperature, and has been previously identified to be a source of microwave losses. In this work, we show lumped element resonators containing $\beta$-phase tantalum in the form of inclusions near the metal-substrate interface with internal quality factors ($Q_\text{i}$) up to $(5.0 \pm 2.5) \times 10^6$ in the single photon regime. They outperform resonators with no sign of the $\beta$-phase in x-ray diffraction and thermal quasi-particle loss. Our results indicate that small concentrations of $\beta$-phase can be beneficial, enhancing critical magnetic fields and potentially, for improving coherence in tantalum based superconducting circuits.

We present RMA-Mamba, a novel architecture that advances the capabilities of vision state space models through a specialized reverse mamba attention module (RMA). The key innovation lies in RMA-Mamba's ability to capture long-range dependencies while maintaining precise local feature representation through its hierarchical processing pipeline. By integrating Vision Mamba (VMamba)'s efficient sequence modeling with RMA's targeted feature refinement, our architecture achieves superior feature learning across multiple scales. This dual-mechanism approach enables robust handling of complex morphological patterns while maintaining computational efficiency. We demonstrate RMA-Mamba's effectiveness in the challenging domain of pathological liver segmentation (from both CT and MRI), where traditional segmentation approaches often fail due to tissue variations. When evaluated on a newly introduced cirrhotic liver dataset (CirrMRI600+) of T2-weighted MRI scans, RMA-Mamba achieves the state-of-the-art performance with a Dice coefficient of 92.08%, mean IoU of 87.36%, and recall of 92.96%. The architecture's generalizability is further validated on the cancerous liver segmentation from CT scans (LiTS: Liver Tumor Segmentation dataset), yielding a Dice score of 92.9% and mIoU of 88.99%. Our code is available for public: https://github.com/JunZengz/RMAMamba., Comment: 8 pages, 3 figures

We present an end-to-end deep learning framework for automated liver cirrhosis stage estimation from multi-sequence MRI. Cirrhosis is the severe scarring (fibrosis) of the liver and a common endpoint of various chronic liver diseases. Early diagnosis is vital to prevent complications such as decompensation and cancer, which significantly decreases life expectancy. However, diagnosing cirrhosis in its early stages is challenging, and patients often present with life-threatening complications. Our approach integrates multi-scale feature learning with sequence-specific attention mechanisms to capture subtle tissue variations across cirrhosis progression stages. Using CirrMRI600+, a large-scale publicly available dataset of 628 high-resolution MRI scans from 339 patients, we demonstrate state-of-the-art performance in three-stage cirrhosis classification. Our best model achieves 72.8% accuracy on T1W and 63.8% on T2W sequences, significantly outperforming traditional radiomics-based approaches. Through extensive ablation studies, we show that our architecture effectively learns stage-specific imaging biomarkers. We establish new benchmarks for automated cirrhosis staging and provide insights for developing clinically applicable deep learning systems. The source code will be available at https://github.com/JunZengz/CirrhosisStage., Comment: 8 pages, 1 figure

Nanomechanical waveguides offer a multitude of applications in quantum and classical technologies. Here, we design, fabricate, and characterize a compact silicon single-mode phononic waveguide actuated by a thin-film lithium niobate piezoelectric element. Our device directly transduces between microwave frequency photons and phonons propagating in the silicon waveguide, providing a route for coupling to superconducting circuits. We probe the device at millikelvin temperatures through a superconducting microwave resonant matching cavity to reveal harmonics of the silicon waveguide and extract a piezoelectric coupling rate $g/2\pi= 1.1$ megahertz and a mechanical coupling rate $f/2\pi=5$ megahertz. Through time-domain measurements of the silicon mechanical modes, we observe energy relaxation timescales of $T_{1,\text{in}} \approx 500$ microseconds, pure dephasing timescales of $T_\phi \approx {60}$ microseconds and dephasing dynamics that indicate the presence of an underlying frequency noise process with a non-uniform spectral distribution. We measure phase noise cross-correlations between silicon mechanical modes and observe detuning-dependent positively-correlated frequency fluctuations. Our measurements provide valuable insights into the dynamics and decoherence characteristics of hybrid piezoelectric-silicon acoustic devices, and suggest approaches for mitigating and circumventing noise processes for emerging quantum acoustic systems., Comment: 13 pages, 4 main figures, 3 appendix figures

As the number of electric vehicles (EVs) continues to grow, the demand for charging stations is also increasing, leading to challenges such as long wait times and insufficient infrastructure. High-precision forecasting of EV charging demand is crucial for efficient station management, to address some of these challenges. This paper presents an approach to predict the charging demand at 15-minute intervals for the day ahead using a multivariate long short-term memory (LSTM) network with an attention mechanism. Additionally, the model leverages explainable AI techniques to evaluate the influence of various factors on the predictions, including weather conditions, day of the week, month, and any holiday. SHapley Additive exPlanations (SHAP) are used to quantify the contribution of each feature to the final forecast, providing deeper insights into how these factors affect prediction accuracy. As a result, the framework offers enhanced decision-making for infrastructure planning. The efficacy of the proposed method is demonstrated by simulations using the test data collected from the EV charging stations at California State University, Long Beach., Comment: Accepted to be presented in the 2025 American Control Conference (ACC)

Despite strong performance on vision-language tasks, Multimodal Large Language Models (MLLMs) struggle with mathematical problem-solving, with both open-source and state-of-the-art models falling short of human performance on visual-math benchmarks. To systematically examine visual-mathematical reasoning in MLLMs, we (1) evaluate their understanding of geometric primitives, (2) test multi-step reasoning, and (3) explore a potential solution to improve visual reasoning capabilities. Our findings reveal fundamental shortcomings in shape recognition, with top models achieving under 50% accuracy in identifying regular polygons. We analyze these failures through the lens of dual-process theory and show that MLLMs rely on System 1 (intuitive, memorized associations) rather than System 2 (deliberate reasoning). Consequently, MLLMs fail to count the sides of both familiar and novel shapes, suggesting they have neither learned the concept of sides nor effectively process visual inputs. Finally, we propose Visually Cued Chain-of-Thought (VC-CoT) prompting, which enhances multi-step mathematical reasoning by explicitly referencing visual annotations in diagrams, boosting GPT-4o's accuracy on an irregular polygon side-counting task from 7% to 93%. Our findings suggest that System 2 reasoning in MLLMs remains an open problem, and visually-guided prompting is essential for successfully engaging visual reasoning. Code available at: https://github.com/rsinghlab/Shape-Blind.

Predicting seizure freedom is essential for tailoring epilepsy treatment. But accurate prediction remains challenging with traditional methods, especially with diverse patient populations. This study developed a deep learning-based graph neural network (GNN) model to predict seizure freedom from stereo electroencephalography (sEEG) data in patients with refractory epilepsy. We utilized high-quality sEEG data from 15 pediatric patients to train a deep learning model that can accurately predict seizure freedom outcomes and advance understanding of brain connectivity at the seizure onset zone. Our model integrates local and global connectivity using graph convolutions with multi-scale attention mechanisms to capture connections between difficult-to-study regions such as the thalamus and motor regions. The model achieved an accuracy of 92.4% in binary class analysis, 86.6% in patient-wise analysis, and 81.4% in multi-class analysis. Node and edge-level feature analysis highlighted the anterior cingulate and frontal pole regions as key contributors to seizure freedom outcomes. The nodes identified by our model were also more likely to coincide with seizure onset zones. Our findings underscore the potential of new connectivity-based deep learning models such as GNNs for enhancing the prediction of seizure freedom, predicting seizure onset zones, connectivity analysis of the brain during seizure, as well as informing AI-assisted personalized epilepsy treatment planning.

Electro-optics, the tuning of optical properties of materials with electric fields, is key to a multitude of quantum and classical photonics applications. However, a major obstacle preventing many emerging use cases is inefficient modulation in cryogenic environments, as traditional tuning mechanisms degrade at low temperatures. Guided by the connection between phase transitions and nonlinearity, we identify the quantum paraelectric perovskite SrTiO$_3$ (STO) as the strongest cryogenic electro-optic photonic material. As a result of the unique quantum paraelectric phase of STO, we demonstrate a dynamically tunable linear Pockels coefficient ($r_{33}$) exceeding 500 pm/V at $T=5$ K, and study its full temperature and bias dependence. We also measure an enhanced piezo-electric coefficient ($d_{33}$) above 90 pC/N. Both of these coefficients exceed all previously reported values for cryogenic materials, including lithium niobate ($r_{33}\approx24$ pm/V) and barium titanate ($r_{42}\approx170$ pm/V). Furthermore, by tuning STO towards \textit{quantum criticality} with oxygen isotope substitution we more than double the optical and piezo-electric nonlinearities, demonstrating a linear Pockels coefficient above 1100 pm/V. Our results probe the link between quantum phase transitions, dielectric susceptibility, and optical nonlinearities, unlocking opportunities in cryogenic optical and mechanical systems, and provide a framework for discovering new nonlinear materials.

Despite the impressive performance of multilingual large language models (mLLMs) in various natural language processing tasks, their ability to understand procedural texts, particularly those with culture-specific content, remains largely unexplored. Texts describing cultural procedures, including rituals, traditional craftsmanship, and social etiquette, require an inherent understanding of cultural context, presenting a significant challenge for mLLMs. In this work, we introduce CAPTex, a benchmark designed to evaluate mLLMs' ability to process and reason about culturally diverse procedural texts across multiple languages using various methodologies to assess their performance. Our findings indicate that (1) mLLMs face difficulties with culturally contextualized procedural texts, showing notable performance declines in low-resource languages, (2) model performance fluctuates across cultural domains, with some areas presenting greater difficulties, and (3) language models exhibit better performance on multiple-choice tasks within conversational frameworks compared to direct questioning. These results underscore the current limitations of mLLMs in handling culturally nuanced procedural texts and highlight the need for culturally aware benchmarks like CAPTex to enhance their adaptability and comprehension across diverse linguistic and cultural landscapes.

Motion planning is a fundamental problem in autonomous driving and perhaps the most challenging to comprehensively evaluate because of the associated risks and expenses of real-world deployment. Therefore, simulations play an important role in efficient development of planning algorithms. To be effective, simulations must be accurate and realistic, both in terms of dynamics and behavior modeling, and also highly customizable in order to accommodate a broad spectrum of research frameworks. In this paper, we introduce SMARTS 2.0, the second generation of our motion planning simulator which, in addition to being highly optimized for large-scale simulation, provides many new features, such as realistic map integration, vehicle-to-vehicle (V2V) communication, traffic and pedestrian simulation, and a broad variety of sensor models. Moreover, we present a novel benchmark suite for evaluating planning algorithms in various highly challenging scenarios, including interactive driving, such as turning at intersections, and adaptive driving, in which the task is to closely follow a lead vehicle without any explicit knowledge of its intention. Each scenario is characterized by a variety of traffic patterns and road structures. We further propose a series of common and task-specific metrics to effectively evaluate the performance of the planning algorithms. At the end, we evaluate common motion planning algorithms using the proposed benchmark and highlight the challenges the proposed scenarios impose. The new SMARTS 2.0 features and the benchmark are publicly available at github.com/huawei-noah/SMARTS., Comment: 8 figures, 3 tables

Image tokenization has enabled major advances in autoregressive image generation by providing compressed, discrete representations that are more efficient to process than raw pixels. While traditional approaches use 2D grid tokenization, recent methods like TiTok have shown that 1D tokenization can achieve high generation quality by eliminating grid redundancies. However, these methods typically use a fixed number of tokens and thus cannot adapt to an image's inherent complexity. We introduce FlexTok, a tokenizer that projects 2D images into variable-length, ordered 1D token sequences. For example, a 256x256 image can be resampled into anywhere from 1 to 256 discrete tokens, hierarchically and semantically compressing its information. By training a rectified flow model as the decoder and using nested dropout, FlexTok produces plausible reconstructions regardless of the chosen token sequence length. We evaluate our approach in an autoregressive generation setting using a simple GPT-style Transformer. On ImageNet, this approach achieves an FID<2 across 8 to 128 tokens, outperforming TiTok and matching state-of-the-art methods with far fewer tokens. We further extend the model to support to text-conditioned image generation and examine how FlexTok relates to traditional 2D tokenization. A key finding is that FlexTok enables next-token prediction to describe images in a coarse-to-fine "visual vocabulary", and that the number of tokens to generate depends on the complexity of the generation task., Comment: Project page at https://flextok.epfl.ch/

Message-Passing Graph Neural Networks (GNNs) are known to have limited expressive power, due to their message passing structure. One mechanism for circumventing this limitation is to add unique node identifiers (IDs), which break the symmetries that underlie the expressivity limitation. In this work, we highlight a key limitation of the ID framework, and propose an approach for addressing it. We begin by observing that the final output of the GNN should clearly not depend on the specific IDs used. We then show that in practice this does not hold, and thus the learned network does not possess this desired structural property. Such invariance to node IDs may be enforced in several ways, and we discuss their theoretical properties. We then propose a novel regularization method that effectively enforces ID invariance to the network. Extensive evaluations on both real-world and synthetic tasks demonstrate that our approach significantly improves ID invariance and, in turn, often boosts generalization performance., Comment: arXiv admin note: text overlap with arXiv:2411.02271

With the increasing prevalence of recorded human speech, spoken language understanding (SLU) is essential for its efficient processing. In order to process the speech, it is commonly transcribed using automatic speech recognition technology. This speech-to-text transition introduces errors into the transcripts, which subsequently propagate to downstream NLP tasks, such as dialogue summarization. While it is known that transcript noise affects downstream tasks, a systematic approach to analyzing its effects across different noise severities and types has not been addressed. We propose a configurable framework for assessing task models in diverse noisy settings, and for examining the impact of transcript-cleaning techniques. The framework facilitates the investigation of task model behavior, which can in turn support the development of effective SLU solutions. We exemplify the utility of our framework on three SLU tasks and four task models, offering insights regarding the effect of transcript noise on tasks in general and models in particular. For instance, we find that task models can tolerate a certain level of noise, and are affected differently by the types of errors in the transcript.

Test-time compute scaling has emerged as a new axis along which to improve model accuracy, where additional computation is used at inference time to allow the model to think longer for more challenging problems. One promising approach for test-time compute scaling is search against a process reward model, where a model generates multiple potential candidates at each step of the search, and these partial trajectories are then scored by a separate reward model in order to guide the search process. The diversity of trajectories in the tree search process affects the accuracy of the search, since increasing diversity promotes more exploration. However, this diversity comes at a cost, as divergent trajectories have less KV sharing, which means they consume more memory and slow down the search process. Previous search methods either do not perform sufficient exploration, or else explore diverse trajectories but have high latency. We address this challenge by proposing Efficient Tree Search (ETS), which promotes KV sharing by pruning redundant trajectories while maintaining necessary diverse trajectories. ETS incorporates a linear programming cost model to promote KV cache sharing by penalizing the number of nodes retained, while incorporating a semantic coverage term into the cost model to ensure that we retain trajectories which are semantically different. We demonstrate how ETS can achieve 1.8$\times$ reduction in average KV cache size during the search process, leading to 1.4$\times$ increased throughput relative to prior state-of-the-art methods, with minimal accuracy degradation and without requiring any custom kernel implementation. Code is available at: https://github.com/SqueezeAILab/ETS., Comment: 11 pages

Sleep is an essential component of human physiology, contributing significantly to overall health and quality of life. Accurate sleep staging and disorder detection are crucial for assessing sleep quality. Studies in the literature have proposed PSG-based approaches and machine-learning methods utilizing single-modality signals. However, existing methods often lack multimodal, multilabel frameworks and address sleep stages and disorders classification separately. In this paper, we propose a 1D-Vision Transformer for simultaneous classification of sleep stages and sleep disorders. Our method exploits the sleep disorders' correlation with specific sleep stage patterns and performs a simultaneous identification of a sleep stage and sleep disorder. The model is trained and tested using multimodal-multilabel sensory data (including photoplethysmogram, respiratory flow, and respiratory effort signals). The proposed method shows an overall accuracy (cohen's Kappa) of 78% (0.66) for five-stage sleep classification and 74% (0.58) for sleep apnea classification. Moreover, we analyzed the encoder attention weights to clarify our models' predictions and investigate the influence different features have on the models' outputs. The result shows that identified patterns, such as respiratory troughs and peaks, make a higher contribution to the final classification process.

We know from prior work that LLMs encode social biases, and that this manifests in clinical tasks. In this work we adopt tools from mechanistic interpretability to unveil sociodemographic representations and biases within LLMs in the context of healthcare. Specifically, we ask: Can we identify activations within LLMs that encode sociodemographic information (e.g., gender, race)? We find that gender information is highly localized in middle MLP layers and can be reliably manipulated at inference time via patching. Such interventions can surgically alter generated clinical vignettes for specific conditions, and also influence downstream clinical predictions which correlate with gender, e.g., patient risk of depression. We find that representation of patient race is somewhat more distributed, but can also be intervened upon, to a degree. To our knowledge, this is the first application of mechanistic interpretability methods to LLMs for healthcare.