Search

Masked Diffusion Language Models (MDLMs) have recently emerged as a strong class of generative models, paralleling state-of-the-art (SOTA) autoregressive (AR) performance across natural language modeling domains. While there have been advances in AR as well as both latent and discrete diffusion-based approaches for protein sequence design, masked diffusion language modeling with protein language models (pLMs) is unexplored. In this work, we introduce MeMDLM, an MDLM tailored for membrane protein design, harnessing the SOTA pLM ESM-2 to de novo generate realistic membrane proteins for downstream experimental applications. Our evaluations demonstrate that MeMDLM-generated proteins exceed AR-based methods by generating sequences with greater transmembrane (TM) character. We further apply our design framework to scaffold soluble and TM motifs in sequences, demonstrating that MeMDLM-reconstructed sequences achieve greater biological similarity to their original counterparts compared to SOTA inpainting methods. Finally, we show that MeMDLM captures physicochemical membrane protein properties with similar fidelity as SOTA pLMs, paving the way for experimental applications. In total, our pipeline motivates future exploration of MDLM-based pLMs for protein design.

Large Language Models (LLMs) have made significant strides in text generation and comprehension, with recent advancements extending into multimodal LLMs that integrate visual and audio inputs. However, these models continue to struggle with fine-grained, cross-modal temporal understanding, particularly when correlating events across audio and video streams. We address these challenges with two key contributions: a new dataset and model, called OCTAV and OMCAT respectively. OCTAV (Omni Context and Temporal Audio Video) is a novel dataset designed to capture event transitions across audio and video. Second, OMCAT (Omni Context Aware Transformer) is a powerful model that leverages RoTE (Rotary Time Embeddings), an innovative extension of RoPE, to enhance temporal grounding and computational efficiency in time-anchored tasks. Through a robust three-stage training pipeline-feature alignment, instruction tuning, and OCTAV-specific training-OMCAT excels in cross-modal temporal understanding. Our model demonstrates state-of-the-art performance on Audio-Visual Question Answering (AVQA) tasks and the OCTAV benchmark, showcasing significant gains in temporal reasoning and cross-modal alignment, as validated through comprehensive experiments and ablation studies. Our dataset and code will be made publicly available. The link to our demo page is https://om-cat.github.io., Comment: Demo page: https://om-cat.github.io

This study is the first to explore the application of a time-series foundation model for VaR estimation. Foundation models, pre-trained on vast and varied datasets, can be used in a zero-shot setting with relatively minimal data or further improved through finetuning. We compare the performance of Google's model, called TimesFM, against conventional parametric and non-parametric models, including GARCH, Generalized Autoregressive Score (GAS), and empirical quantile estimates, using daily returns from the S\&P 100 index and its constituents over 19 years. Our backtesting results indicate that, in terms of the actual-over-expected ratio, the fine-tuned TimesFM model consistently outperforms traditional methods. Regarding the quantile score loss function, it achieves performance comparable to the best econometric approach, the GAS model. Overall, the foundation model is either the best or among the top performers in forecasting VaR across the 0.01, 0.025, 0.05, and 0.1 VaR levels. We also found that fine-tuning significantly improves the results, and the model should not be used in zero-shot settings. Overall, foundation models can provide completely alternative approaches to traditional econometric methods, yet there are challenges to be tackled.

In this article, we define two new versions of integral and Frobenius closures of ideals which incorporate an auxiliary ideal and a real parameter. These additional ingredients are common in adjusting old definitions of ideal closures in order to generalize them to pairs, with an eye towards further applications in algebraic geometry. In the case of tight closure, similar generalizations exist due to N. Hara and K.I. Yoshida, as well as A. Vraciu. We study their basic properties and give computationally effective calculations of the adjusted tight, Frobenius, and integral closures in the case of affine semigroup rings in terms of the convex geometry of the associated exponent sets., Comment: 25 pages, comments welcome!

In this study, we investigate the performance of few-shot classification models across different domains, specifically natural images and histopathological images. We first train several few-shot classification models on natural images and evaluate their performance on histopathological images. Subsequently, we train the same models on histopathological images and compare their performance. We incorporated four histopathology datasets and one natural images dataset and assessed performance across 5-way 1-shot, 5-way 5-shot, and 5-way 10-shot scenarios using a selection of state-of-the-art classification techniques. Our experimental results reveal insights into the transferability and generalization capabilities of few-shot classification models between diverse image domains. We analyze the strengths and limitations of these models in adapting to new domains and provide recommendations for optimizing their performance in cross-domain scenarios. This research contributes to advancing our understanding of few-shot learning in the context of image classification across diverse domains.

Rapid search and rescue is critical to maximizing survival rates following natural disasters. However, these efforts are challenged by the need to search large disaster zones, lack of reliability in the communications infrastructure, and a priori unknown numbers of objects of interest (OOIs), such as injured survivors. Aerial robots are increasingly being deployed for search and rescue due to their high mobility, but there remains a gap in deploying multi-robot autonomous aerial systems for methodical search of large environments. Prior works have relied on preprogrammed paths from human operators or are evaluated only in simulation. We bridge these gaps in the state of the art by developing and demonstrating a decentralized active search system, which biases its trajectories to take additional views of uncertain OOIs. The methodology leverages stochasticity for rapid coverage in communication denied scenarios. When communications are available, robots share poses, goals, and OOI information to accelerate the rate of search. Extensive simulations and hardware experiments in Bloomingdale, OH, are conducted to validate the approach. The results demonstrate the active search approach outperforms greedy coverage-based planning in communication-denied scenarios while maintaining comparable performance in communication-enabled scenarios.

Model predictive control (MPC) is a powerful control technique for online optimization using system model-based predictions over a finite time horizon. However, the computational cost MPC requires can be prohibitive in resource-constrained computer systems. This paper presents a fuzzy controller synthesis framework guided by MPC. In the proposed framework, training data is obtained from MPC closed-loop simulations and is used to optimize a low computational complexity controller to emulate the response of MPC. In particular, autoregressive moving average (ARMA) controllers are trained using data obtained from MPC closed-loop simulations, such that each ARMA controller emulates the response of the MPC controller under particular desired conditions. Using a Takagi-Sugeno (T-S) fuzzy system, the responses of all the trained ARMA controllers are then weighted depending on the measured system conditions, resulting in the Fuzzy-Autoregressive Moving Average (F-ARMA) controller. The effectiveness of the trained F-ARMA controllers is illustrated via numerical examples., Comment: 8 pages, 8 figures, submitted to the American Control Conference 2025

Audio deepfake detection is crucial to combat the malicious use of AI-synthesized speech. Among many efforts undertaken by the community, the ASVspoof challenge has become one of the benchmarks to evaluate the generalizability and robustness of detection models. In this paper, we present Reality Defender's submission to the ASVspoof5 challenge, highlighting a novel pretraining strategy which significantly improves generalizability while maintaining low computational cost during training. Our system SLIM learns the style-linguistics dependency embeddings from various types of bonafide speech using self-supervised contrastive learning. The learned embeddings help to discriminate spoof from bonafide speech by focusing on the relationship between the style and linguistics aspects. We evaluated our system on ASVspoof5, ASV2019, and In-the-wild. Our submission achieved minDCF of 0.1499 and EER of 5.5% on ASVspoof5 Track 1, and EER of 7.4% and 10.8% on ASV2019 and In-the-wild respectively., Comment: Accepted into ASVspoof5 workshop

Assuming the existence of a Landau-Siegel zero, we establish an explicit Deuring-Heilbronn zero repulsion phenomenon for Dirichlet $L$-functions modulo $q$. Our estimate is uniform in the entire critical strip, and improves over the previous best known explicit estimate due to Thorner and Zaman., Comment: Minor correction to constant $c_4$ in main results

Knowledge distillation leverages a teacher model to improve the training of a student model. A persistent challenge is that a better teacher does not always yield a better student, to which a common mitigation is to use additional supervision from several ``intermediate'' teachers. One empirically validated variant of this principle is progressive distillation, where the student learns from successive intermediate checkpoints of the teacher. Using sparse parity as a sandbox, we identify an implicit curriculum as one mechanism through which progressive distillation accelerates the student's learning. This curriculum is available only through the intermediate checkpoints but not the final converged one, and imparts both empirical acceleration and a provable sample complexity benefit to the student. We then extend our investigation to Transformers trained on probabilistic context-free grammars (PCFGs) and real-world pre-training datasets (Wikipedia and Books). Through probing the teacher model, we identify an analogous implicit curriculum where the model progressively learns features that capture longer context. Our theoretical and empirical findings on sparse parity, complemented by empirical observations on more complex tasks, highlight the benefit of progressive distillation via implicit curriculum across setups.

This paper proves Strichartz estimates for the Schrodinger Equation with a potential term and white noise dispersion in dimension $1$. We also explore dispersive estimates using previous results in the field.

We study the classical contest design problem of allocating a budget across different prizes to maximize effort in a finite type-space environment. For any contest, we characterize the unique symmetric equilibrium. In this equilibrium, different agent types mix over contiguous intervals so that more efficient agents always exert greater effort than less efficient agents. We then solve for the expected equilibrium effort, investigate the effect of increasing competition under linear costs, and identify conditions under which this effect persists under general costs. As a result, we find that the winner-takes-all contest is optimal under linear and concave costs. Lastly, we obtain an equilibrium convergence result for the continuum type-space, and since the finite type-space encompasses the complete information environment as a special case, our analysis offers a unified approach to studying contests in these classical environments., Comment: 32 pages, 1 figure

This paper demonstrates the sim-to-real capabilities of retrospective cost optimization-based adaptive control for multirotor stabilization and trajectory-tracking problems. First, a continuous-time version of the widely used discrete-time retrospective control adaptive control algorithm is developed. Next, a computationally inexpensive 12-degree-of-freedom model of a multirotor is used to learn the control system in a simulation environment with a single trajectory. Finally, the performance of the learned controller is verified in a complex and realistic multirotor model in simulation and with a physical quadcopter in a waypoint command and a helical trajectory command.

Player Experience Modelling (PEM) is the study of AI techniques applied to modelling a player's experience within a video game. PEM development can be labour-intensive, requiring expert hand-authoring or specialized data collection. In this work, we propose a novel PEM development approach, approximating player experience from gameplay video. We evaluate this approach predicting affect in the game Angry Birds via a human subject study. We validate that our PEM can strongly correlate with self-reported and sensor measures of affect, demonstrating the potential of this approach., Comment: 9 pages, 3 figures, AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment

This paper presents a model-free, learning-based adaptive controller for the density tracking problem in a two-level Lindblad-Gorini-Kossakowski-Sudarshan (LGKS) quantum system. The adaptive controller is based on the continuous-time retrospective cost adaptive control. To preserve the geometric properties of the quantum system, an adaptive PID controller driven and optimized by Ulhmann's fidelity is used. The proposed controller is validated in simulation for a low and a high-entropy density-tracking problem.

We present a novel dataset aimed at advancing danger analysis and assessment by addressing the challenge of quantifying danger in video content and identifying how human-like a Large Language Model (LLM) evaluator is for the same. This is achieved by compiling a collection of 100 YouTube videos featuring various events. Each video is annotated by human participants who provided danger ratings on a scale from 0 (no danger to humans) to 10 (life-threatening), with precise timestamps indicating moments of heightened danger. Additionally, we leverage LLMs to independently assess the danger levels in these videos using video summaries. We introduce Mean Squared Error (MSE) scores for multimodal meta-evaluation of the alignment between human and LLM danger assessments. Our dataset not only contributes a new resource for danger assessment in video content but also demonstrates the potential of LLMs in achieving human-like evaluations., Comment: Preprint

In this paper, we give closed form expressions of the Frobenius number for the triple of the $2$-step star numbers $an(n-2) + 1$ for an integer $a \geq 4$. These numbers have been studied from different aspects for some $a$'s. These numbers can also be considered as variations of the well known star numbers of the form $6n(n-1) + 1$. We also give closed form expressions of the Sylvester number (genus) for the triple of the $2$-step star numbers.

The application of neural networks in modeling dynamic systems has become prominent due to their ability to estimate complex nonlinear functions. Despite their effectiveness, neural networks face challenges in long-term predictions, where the prediction error diverges over time, thus degrading their accuracy. This paper presents a neural filter to enhance the accuracy of long-term state predictions of neural network-based models of dynamic systems. Motivated by the extended Kalman filter, the neural filter combines the neural network state predictions with the measurements from the physical system to improve the estimated state's accuracy. The neural filter's improvements in prediction accuracy are demonstrated through applications to four nonlinear dynamical systems. Numerical experiments show that the neural filter significantly improves prediction accuracy and bounds the state estimate covariance, outperforming the neural network predictions.

The paper develops a singularity-free backstepping-based adaptive control for stabilizing and tracking the trajectory of a bicopter system. In the bicopter system, the inertial parameters parameterize the input map. Since the classical adaptive backstepping technique requires the inversion of the input map, which contains the estimate of parameter estimates, the stability of the closed-loop system cannot be guaranteed due to the inversion of parameter estimates. This paper proposes a novel technique to circumvent the inversion of parameter estimates in the control law. The resulting controller requires only the sign of the unknown parameters. The proposed controller is validated in simulation for a smooth and nonsmooth trajectory-tracking problem., Comment: arXiv admin note: text overlap with arXiv:2402.03709

In pursuit of a full-fledged theory of quantum gravity, operational approaches offer insights into quantum-gravitational effects produced by quantum superposition of different spacetimes not diffeomorphic to one another. Recent work applies this approach to superpose cylindrically identified Minkowski spacetimes (i.e. periodic boundary conditions) with different characteristic circumferences, where a two-level detector coupled to a quantum field residing in the spacetime exhibits resonance peaks in response at certain values of the superposed length ratios. Here, we extend this analysis to a superposition of cylindrically identified Rindler spacetimes, considering a two-level detector constantly accelerated in the direction orthogonal to the compact dimension. Similarly to previous work, we find resonance peaks in the detector response at rational ratios of the superposed compactified lengths, which we observe to be accentuated by the acceleration of the detector. Furthermore, for the first time we confirm the detailed balance condition due to acceleration in a superposition of spacetimes, commensurate with the Unruh effect in a single spacetime state. The resonant structure of detector response in the presence of event horizons, for the first time observed in 3+1 dimensions, may offer clues to the nature of black hole entropy in the full theory of quantum gravity., Comment: 8 + 12 pages, 7 + 2 figures

The escalating global mortality and morbidity rates associated with gastrointestinal (GI) bleeding, compounded by the complexities and limitations of traditional endoscopic methods, underscore the urgent need for a critical review of current methodologies used for addressing this condition. With an estimated 300,000 annual deaths worldwide, the demand for innovative diagnostic and therapeutic strategies is paramount. The introduction of Video Capsule Endoscopy (VCE) has marked a significant advancement, offering a comprehensive, non-invasive visualization of the digestive tract that is pivotal for detecting bleeding sources unattainable by traditional methods. Despite its benefits, the efficacy of VCE is hindered by diagnostic challenges, including time-consuming analysis and susceptibility to human error. This backdrop sets the stage for exploring Machine Learning (ML) applications in automating GI bleeding detection within capsule endoscopy, aiming to enhance diagnostic accuracy, reduce manual labor, and improve patient outcomes. Through an exhaustive analysis of 113 papers published between 2008 and 2023, this review assesses the current state of ML methodologies in bleeding detection, highlighting their effectiveness, challenges, and prospective directions. It contributes an in-depth examination of AI techniques in VCE frame analysis, offering insights into open-source datasets, mathematical performance metrics, and technique categorization. The paper sets a foundation for future research to overcome existing challenges, advancing gastrointestinal diagnostics through interdisciplinary collaboration and innovation in ML applications.

The use of high-dimensional systems for quantum communication opens interesting perspectives, such as increased information capacity and noise resilience. In this context, it is crucial to certify that a given quantum channel can reliably transmit high-dimensional quantum information. Here we develop efficient methods for the characterization of high-dimensional quantum channels. We first present a notion of dimensionality of quantum channels, and develop efficient certification methods for this quantity. We consider a simple prepare-and-measure setup, and provide witnesses for both a fully and a partially trusted scenario. In turn we apply these methods to a photonic experiment and certify dimensionalities up to 59 for a commercial graded-index multi-mode optical fiber. Moreover, we present extensive numerical simulations of the experiment, providing an accurate noise model for the fiber and exploring the potential of more sophisticated witnesses. Our work demonstrates the efficient characterization of high-dimensional quantum channels, a key ingredient for future quantum communication technologies.

The current standard for detecting human epidermal growth factor receptor 2 (HER2) status in breast cancer patients relies on HER2 amplification, identified through fluorescence in situ hybridization (FISH) or immunohistochemistry (IHC). However, hematoxylin and eosin (H\&E) tumor stains are more widely available, and accurately predicting HER2 status using H\&E could reduce costs and expedite treatment selection. Deep Learning algorithms for H&E have shown effectiveness in predicting various cancer features and clinical outcomes, including moderate success in HER2 status prediction. In this work, we employed a customized weak supervision classification technique combined with MoCo-v2 contrastive learning to predict HER2 status. We trained our pipeline on 182 publicly available H&E Whole Slide Images (WSIs) from The Cancer Genome Atlas (TCGA), for which annotations by the pathology team at Yale School of Medicine are publicly available. Our pipeline achieved an Area Under the Curve (AUC) of 0.85 across four different test folds. Additionally, we tested our model on 44 H&E slides from the TCGA-BRCA dataset, which had an HER2 score of 2+ and included corresponding HER2 status and FISH test results. These cases are considered equivocal for IHC, requiring an expensive FISH test on their IHC slides for disambiguation. Our pipeline demonstrated an AUC of 0.81 on these challenging H&E slides. Reducing the need for FISH test can have significant implications in cancer treatment equity for underserved populations.

Computer-based analysis of Wireless Capsule Endoscopy (WCE) is crucial. However, a medically annotated WCE dataset for training and evaluation of automatic classification, detection, and segmentation of bleeding and non-bleeding frames is currently lacking. The present work focused on development of a medically annotated WCE dataset called WCEbleedGen for automatic classification, detection, and segmentation of bleeding and non-bleeding frames. It comprises 2,618 WCE bleeding and non-bleeding frames which were collected from various internet resources and existing WCE datasets. A comprehensive benchmarking and evaluation of the developed dataset was done using nine classification-based, three detection-based, and three segmentation-based deep learning models. The dataset is of high-quality, is class-balanced and contains single and multiple bleeding sites. Overall, our standard benchmark results show that Visual Geometric Group (VGG) 19, You Only Look Once version 8 nano (YOLOv8n), and Link network (Linknet) performed best in automatic classification, detection, and segmentation-based evaluations, respectively. Automatic bleeding diagnosis is crucial for WCE video interpretations. This diverse dataset will aid in developing of real-time, multi-task learning-based innovative solutions for automatic bleeding diagnosis in WCE. The dataset and code are publicly available at https://zenodo.org/records/10156571 and https://github.com/misahub2023/Benchmarking-Codes-of-the-WCEBleedGen-dataset.

Deliberation involves participants exchanging knowledge, arguments, and perspectives and has been shown to be effective at addressing polarization. The Stanford Online Deliberation Platform facilitates large-scale deliberations. It enables video-based online discussions on a structured agenda for small groups without requiring human moderators. This paper's data comes from various deliberation events, including one conducted in collaboration with Meta in 32 countries, and another with 38 post-secondary institutions in the US. Estimating the quality of contributions in a conversation is crucial for assessing feature and intervention impacts. Traditionally, this is done by human annotators, which is time-consuming and costly. We use a large language model (LLM) alongside eight human annotators to rate contributions based on justification, novelty, expansion of the conversation, and potential for further expansion, with scores ranging from 1 to 5. Annotators also provide brief justifications for their ratings. Using the average rating from other human annotators as the ground truth, we find the model outperforms individual human annotators. While pairs of human annotators outperform the model in rating justification and groups of three outperform it on all four metrics, the model remains competitive. We illustrate the usefulness of the automated quality rating by assessing the effect of nudges on the quality of deliberation. We first observe that individual nudges after prolonged inactivity are highly effective, increasing the likelihood of the individual requesting to speak in the next 30 seconds by 65%. Using our automated quality estimation, we show that the quality ratings for statements prompted by nudging are similar to those made without nudging, signifying that nudging leads to more ideas being generated in the conversation without losing overall quality.

Introduction: Electrical impedance spectroscopy (EIS) has recently developed as a novel diagnostic device for screening and evaluating cervical dysplasia, prostate cancer, breast cancer and basal cell carcinoma. The current study aimed to validate and evaluate bioimpedance as a diagnostic tool for tobacco-induced oral lesions. Methodology: The study comprised 50 OSCC and OPMD tissue specimens for in-vitro study and 320 subjects for in vivo study. Bioimpedance device prepared and calibrated. EIS measurements were done for the habit and control groups and were compared. Results: The impedance value in the control group was significantly higher compared to the OPMD and OSCC groups. Diagnosis based on BIS measurements has a sensitivity of 95.9% and a specificity of 86.7%. Conclusion: Bioimpedance device can help in decision-making for differentiating OPMD and OSCC cases and their management, especially in primary healthcare settings. Keywords: Impedance, Cancer, Diagnosis, Device, Community

This paper develops an agent-based automated fact-checking approach for detecting misinformation. We demonstrate that combining a powerful LLM agent, which does not have access to the internet for searches, with an online web search agent yields better results than when each tool is used independently. Our approach is robust across multiple models, outperforming alternatives and increasing the macro F1 of misinformation detection by as much as 20 percent compared to LLMs without search. We also conduct extensive analyses on the sources our system leverages and their biases, decisions in the construction of the system like the search tool and the knowledge base, the type of evidence needed and its impact on the results, and other parts of the overall process. By combining strong performance with in-depth understanding, we hope to provide building blocks for future search-enabled misinformation mitigation systems., Comment: 1 main figure, 8 tables, 10 pages, 12 figures in Appendix, 7 tables in Appendix GitHub URL: https://github.com/ComplexData-MILA/webretrieval

In this survey, we summarize recent developments in research fueled by the growing adoption of automated bidding strategies in online advertising. We explore the challenges and opportunities that have arisen as markets embrace this autobidding and cover a range of topics in this area, including bidding algorithms, equilibrium analysis and efficiency of common auction formats, and optimal auction design.

The growth of weeds poses a significant challenge to agricultural productivity, necessitating efficient and accurate weed detection and management strategies. The combination of multispectral imaging and drone technology has emerged as a promising approach to tackle this issue, enabling rapid and cost-effective monitoring of large agricultural fields. This systematic review surveys and evaluates the state-of-the-art in machine learning interventions for weed detection that utilize multispectral images captured by unmanned aerial vehicles. The study describes the various models used for training, features extracted from multi-spectral data, their efficiency and effect on the results, the performance analysis parameters, and also the current challenges faced by researchers in this domain. The review was conducted in accordance with the PRISMA guidelines. Three sources were used to obtain the relevant material, and the screening and data extraction were done on the COVIDENCE platform. The search string resulted in 600 papers from all sources. The review also provides insights into potential research directions and opportunities for further advancements in the field. These insights would serve as a valuable guide for researchers, agricultural scientists, and practitioners in developing precise and sustainable weed management strategies to enhance agricultural productivity and minimize ecological impact.

To address the data scarcity associated with 3D assets, 2D-lifting techniques such as Score Distillation Sampling (SDS) have become a widely adopted practice in text-to-3D generation pipelines. However, the diffusion models used in these techniques are prone to viewpoint bias and thus lead to geometric inconsistencies such as the Janus problem. To counter this, we introduce MT3D, a text-to-3D generative model that leverages a high-fidelity 3D object to overcome viewpoint bias and explicitly infuse geometric understanding into the generation pipeline. Firstly, we employ depth maps derived from a high-quality 3D model as control signals to guarantee that the generated 2D images preserve the fundamental shape and structure, thereby reducing the inherent viewpoint bias. Next, we utilize deep geometric moments to ensure geometric consistency in the 3D representation explicitly. By incorporating geometric details from a 3D asset, MT3D enables the creation of diverse and geometrically consistent objects, thereby improving the quality and usability of our 3D representations., Comment: 9 pages, 8 figures

The increasing adoption of Deep Neural Networks (DNNs) has led to their application in many challenging scientific visualization tasks. While advanced DNNs offer impressive generalization capabilities, understanding factors such as model prediction quality, robustness, and uncertainty is crucial. These insights can enable domain scientists to make informed decisions about their data. However, DNNs inherently lack ability to estimate prediction uncertainty, necessitating new research to construct robust uncertainty-aware visualization techniques tailored for various visualization tasks. In this work, we propose uncertainty-aware implicit neural representations to model scalar field data sets effectively and comprehensively study the efficacy and benefits of estimated uncertainty information for volume visualization tasks. We evaluate the effectiveness of two principled deep uncertainty estimation techniques: (1) Deep Ensemble and (2) Monte Carlo Dropout (MCDropout). These techniques enable uncertainty-informed volume visualization in scalar field data sets. Our extensive exploration across multiple data sets demonstrates that uncertainty-aware models produce informative volume visualization results. Moreover, integrating prediction uncertainty enhances the trustworthiness of our DNN model, making it suitable for robustly analyzing and visualizing real-world scientific volumetric data sets., Comment: To appear in IEEE Workshop on Uncertainty Visualization in conjunction with IEEE VIS 2024, Florida, USA

Deep learning models used for medical image classification tasks are often constrained by the limited amount of training data along with severe class imbalance. Despite these problems, models should be explainable to enable human trust in the models' decisions to ensure wider adoption in high-risk situations. In this paper, we propose PRECISe, an explainable-by-design model meticulously constructed to concurrently address all three challenges. Evaluation on 2 imbalanced medical image datasets reveals that PRECISe outperforms the current state-of-the-art methods on data efficient generalization to minority classes, achieving an accuracy of ~87% in detecting pneumonia in chest x-rays upon training on <60 images only. Additionally, a case study is presented to highlight the model's ability to produce easily interpretable predictions, reinforcing its practical utility and reliability for medical imaging tasks.

We present the Capsule Vision 2024 Challenge: Multi-Class Abnormality Classification for Video Capsule Endoscopy. It is being virtually organized by the Research Center for Medical Image Analysis and Artificial Intelligence (MIAAI), Department of Medicine, Danube Private University, Krems, Austria and Medical Imaging and Signal Analysis Hub (MISAHUB) in collaboration with the 9th International Conference on Computer Vision & Image Processing (CVIP 2024) being organized by the Indian Institute of Information Technology, Design and Manufacturing (IIITDM) Kancheepuram, Chennai, India. This document describes the overview of the challenge, its registration and rules, submission format, and the description of the utilized datasets., Comment: 6 pages

Autonomous lander missions on extraterrestrial bodies need to sample granular materials while coping with domain shifts, even when sampling strategies are extensively tuned on Earth. To tackle this challenge, this paper studies the few-shot scooping problem and proposes a vision-based adaptive scooping strategy that uses the deep kernel Gaussian process method trained with a novel meta-training strategy to learn online from very limited experience on out-of-distribution target terrains. Our Deep Kernel Calibration with Maximal Deployment Gaps (kCMD) strategy explicitly trains a deep kernel model to adapt to large domain shifts by creating simulated maximal deployment gaps from an offline training dataset and training models to overcome these deployment gaps during training. Employed in a Bayesian Optimization sequential decision-making framework, the proposed method allows the robot to perform high-quality scooping actions on out-of-distribution terrains after a few attempts, significantly outperforming non-adaptive methods proposed in the excavation literature as well as other state-of-the-art meta-learning methods. The proposed method also demonstrates zero-shot transfer capability, successfully adapting to the NASA OWLAT platform, which serves as a state-of-the-art simulator for potential future planetary missions. These results demonstrate the potential of training deep models with simulated deployment gaps for more generalizable meta-learning in high-capacity models. Furthermore, they highlight the promise of our method in autonomous lander sampling missions by enabling landers to overcome the deployment gap between Earth and extraterrestrial bodies., Comment: arXiv admin note: substantial text overlap with arXiv:2303.02893

Effective fraud detection and analysis of government-issued identity documents, such as passports, driver's licenses, and identity cards, are essential in thwarting identity theft and bolstering security on online platforms. The training of accurate fraud detection and analysis tools depends on the availability of extensive identity document datasets. However, current publicly available benchmark datasets for identity document analysis, including MIDV-500, MIDV-2020, and FMIDV, fall short in several respects: they offer a limited number of samples, cover insufficient varieties of fraud patterns, and seldom include alterations in critical personal identifying fields like portrait images, limiting their utility in training models capable of detecting realistic frauds while preserving privacy. In response to these shortcomings, our research introduces a new benchmark dataset, IDNet, designed to advance privacy-preserving fraud detection efforts. The IDNet dataset comprises 837,060 images of synthetically generated identity documents, totaling approximately 490 gigabytes, categorized into 20 types from $10$ U.S. states and 10 European countries. We evaluate the utility and present use cases of the dataset, illustrating how it can aid in training privacy-preserving fraud detection methods, facilitating the generation of camera and video capturing of identity documents, and testing schema unification and other identity document management functionalities., Comment: 40 pages

Microservice architectures are revolutionizing both small businesses and large corporations, igniting a new era of innovation with their exceptional advantages in maintainability, reusability, and scalability. However, these benefits come with significant security challenges, as the increased complexity of service interactions, expanded attack surfaces, and intricate dependency management introduce a new array of cybersecurity vulnerabilities. While security concerns are mounting, there is a lack of comprehensive research that integrates a review of existing knowledge with empirical analysis of microservice vulnerabilities. This study aims to fill this gap by gathering, analyzing, and synthesizing existing literature on security vulnerabilities associated with microservice architectures. Through a thorough examination of 62 studies, we identify, analyze, and report 126 security vulnerabilities inherent in microservice architectures. This comprehensive analysis enables us to (i) propose a taxonomy that categorizes microservice vulnerabilities based on the distinctive features of microservice architectures; (ii) conduct an empirical analysis by performing vulnerability scans on four diverse microservice benchmark applications using three different scanning tools to validate our taxonomy; and (iii) map our taxonomy vulnerabilities with empirically identified vulnerabilities, providing an in-depth vulnerability analysis at microservice, application, and scanning tool levels. Our study offers crucial guidelines for practitioners and researchers to advance both the state-of-the-practice and the state-of-the-art in securing microservice architectures.

In online learning, the ability to provide quick and accurate feedback to learners is crucial. In skill-based learning, learners need to understand the underlying concepts and mechanisms of a skill to be able to apply it effectively. While videos are a common tool in online learning, they cannot comprehend or assess the skills being taught. Additionally, while Generative AI methods are effective in searching and retrieving answers from a text corpus, it remains unclear whether these methods exhibit any true understanding. This limits their ability to provide explanations of skills or help with problem-solving. This paper proposes a novel approach that merges Cognitive AI and Generative AI to address these challenges. We employ a structured knowledge representation, the TMK (Task-Method-Knowledge) model, to encode skills taught in an online Knowledge-based AI course. Leveraging techniques such as Large Language Models, Chain-of-Thought, and Iterative Refinement, we outline a framework for generating reasoned explanations in response to learners' questions about skills., Comment: 9 pages, 6 figures, 1 table

The Virtual Experimental Research Assistant (VERA) is an inquiry-based learning environment that empowers a learner to build conceptual models of complex ecological systems and experiment with agent-based simulations of the models. This study investigates the convergence of cognitive AI and generative AI for self-explanation in interactive AI agents such as VERA. From a cognitive AI viewpoint, we endow VERA with a functional model of its own design, knowledge, and reasoning represented in the Task--Method--Knowledge (TMK) language. From the perspective of generative AI, we use ChatGPT, LangChain, and Chain-of-Thought to answer user questions based on the VERA TMK model. Thus, we combine cognitive and generative AI to generate explanations about how VERA works and produces its answers. The preliminary evaluation of the generation of explanations in VERA on a bank of 66 questions derived from earlier work appears promising., Comment: 10 pages, 2 figures, 2 tables, 1 appendix, HEXED Workshop @EDM July 2024

We routinely perform procedures (such as cooking) that include a set of atomic steps. Often, inadvertent omission or misordering of a single step can lead to serious consequences, especially for those experiencing cognitive challenges such as dementia. This paper introduces PrISM-Observer, a smartwatch-based, context-aware, real-time intervention system designed to support daily tasks by preventing errors. Unlike traditional systems that require users to seek out information, the agent observes user actions and intervenes proactively. This capability is enabled by the agent's ability to continuously update its belief in the user's behavior in real-time through multimodal sensing and forecast optimal intervention moments and methods. We first validated the steps-tracking performance of our framework through evaluations across three datasets with different complexities. Then, we implemented a real-time agent system using a smartwatch and conducted a user study in a cooking task scenario. The system generated helpful interventions, and we gained positive feedback from the participants. The general applicability of PrISM-Observer to daily tasks promises broad applications, for instance, including support for users requiring more involved interventions, such as people with dementia or post-surgical patients., Comment: conditionally accepted to ACM UIST 2024

We consider the setting where a user with sensitive features wishes to obtain a recommendation from a server in a differentially private fashion. We propose a ``multi-selection'' architecture where the server can send back multiple recommendations and the user chooses one from these that matches best with their private features. When the user feature is one-dimensional -- on an infinite line -- and the accuracy measure is defined w.r.t some increasing function $\mathfrak{h}(.)$ of the distance on the line, we precisely characterize the optimal mechanism that satisfies differential privacy. The specification of the optimal mechanism includes both the distribution of the noise that the user adds to its private value, and the algorithm used by the server to determine the set of results to send back as a response and further show that Laplace is an optimal noise distribution. We further show that this optimal mechanism results in an error that is inversely proportional to the number of results returned when the function $\mathfrak{h}(.)$ is the identity function.

Jill Watson, a virtual teaching assistant powered by LLMs, answers student questions and engages them in extended conversations on courseware provided by the instructors. In this paper, we analyze student interactions with Jill across multiple courses and colleges, focusing on the types and complexity of student questions based on Bloom's Revised Taxonomy and tool usage patterns. We find that, by supporting a wide range of cognitive demands, Jill encourages students to engage in sophisticated, higher-order cognitive questions. However, the frequency of usage varies significantly across deployments, and the types of questions asked depend on course-specific contexts. These findings pave the way for future work on AI-driven educational tools tailored to individual learning styles and course structure, potentially enhancing both the teaching and learning experience in classrooms., Comment: Accepted in the Seventeenth International Conference on Educational Data Mining (EDM) Workshop: Leveraging LLMs for Next Generation Educational Technologies, July 2024

In observational studies of discrimination, the most common statistical approaches consider either the rate at which decisions are made (benchmark tests) or the success rate of those decisions (outcome tests). Both tests, however, have well-known statistical limitations, sometimes suggesting discrimination even when there is none. Despite the fallibility of the benchmark and outcome tests individually, here we prove a surprisingly strong statistical guarantee: under a common non-parametric assumption, at least one of the two tests must be correct; consequently, when both tests agree, they are guaranteed to yield correct conclusions. We present empirical evidence that the underlying assumption holds approximately in several important domains, including lending, education, and criminal justice -- and that our hybrid test is robust to the moderate violations of the assumption that we observe in practice. Applying this approach to 2.8 million police stops across California, we find evidence of widespread racial discrimination.

Vision transformers (ViT) have made substantial progress for classification tasks in computer vision. Recently, Gong et. al. '21, introduced attention-based modeling for several audio tasks. However, relatively unexplored is the use of a ViT for audio spoof detection task. We bridge this gap and introduce ViTs for this task. A vanilla baseline built on fine-tuning the SSAST (Gong et. al. '22) audio ViT model achieves sub-optimal equal error rates (EERs). To improve performance, we propose a novel attention-based contrastive learning framework (SSAST-CL) that uses cross-attention to aid the representation learning. Experiments show that our framework successfully disentangles the bonafide and spoof classes and helps learn better classifiers for the task. With appropriate data augmentations policy, a model trained on our framework achieves competitive performance on the ASVSpoof 2021 challenge. We provide comparisons and ablation studies to justify our claim., Comment: Proc. INTERSPEECH 2023

Sample Return Capsules (SRCs) entering Earth's atmosphere at hypervelocity from interplanetary space are a valuable resource for studying meteor phenomena. The 24 September 2023 arrival of the OSIRIS-REx (Origins, Spectral Interpretation, Resource Identification, and Security-Regolith Explorer) SRC provided an unprecedented chance for geophysical observations of a well-characterized source with known parameters, including timing and trajectory. A collaborative effort involving researchers from 16 institutions executed a carefully planned geophysical observational campaign at strategically chosen locations, deploying over 400 ground-based sensors encompassing infrasound, seismic, distributed acoustic sensing (DAS), and GPS technologies. Additionally, balloons equipped with infrasound sensors were launched to capture signals at higher altitudes. This campaign (the largest of its kind so far) yielded a wealth of invaluable data anticipated to fuel scientific inquiry for years to come. The success of the observational campaign is evidenced by the near-universal detection of signals across instruments, both proximal and distal. This paper presents a comprehensive overview of the collective scientific effort, field deployment, and preliminary findings. The early findings have the potential to inform future space missions and terrestrial campaigns, contributing to our understanding of meteoroid interactions with planetary atmospheres. Furthermore, the dataset collected during this campaign will improve entry and propagation models as well as augment the study of atmospheric dynamics and shock phenomena generated by meteoroids and similar sources., Comment: 87 pages, 14 figures

In this paper, we assess the scientific promise and technology feasibility of distributed instruments for planetary science. A distributed instrument is an instrument designed to collect spatially and temporally correlated data from multiple networked, geographically distributed point sensors. Distributed instruments are ubiquitous in Earth science, where they are routinely employed for weather and climate science, seismic studies and resource prospecting, and detection of industrial emissions. However, to date, their adoption in planetary surface science has been minimal. It is natural to ask whether this lack of adoption is driven by low potential to address high-priority questions in planetary science; immature technology; or both. To address this question, we survey high-priority planetary science questions that are uniquely well-suited to distributed instruments. We identify four areas of research where distributed instruments hold promise to unlock answers that are largely inaccessible to monolithic sensors, namely, weather and climate studies of Mars; localization of seismic events on rocky and icy bodies; localization of trace gas emissions, primarily on Mars; and magnetometry studies of internal composition. Next, we survey enabling technologies for distributed sensors and assess their maturity. We identify sensor placement (including descent and landing on planetary surfaces), power, and instrument autonomy as three key areas requiring further investment to enable future distributed instruments. Overall, this work shows that distributed instruments hold great promise for planetary science, and paves the way for follow-on studies of future distributed instruments for Solar System in-situ science.

This paper addresses a significant gap in Autonomous Cyber Operations (ACO) literature: the absence of effective edge-blocking ACO strategies in dynamic, real-world networks. It specifically targets the cybersecurity vulnerabilities of organizational Active Directory (AD) systems. Unlike the existing literature on edge-blocking defenses which considers AD systems as static entities, our study counters this by recognizing their dynamic nature and developing advanced edge-blocking defenses through a Stackelberg game model between attacker and defender. We devise a Reinforcement Learning (RL)-based attack strategy and an RL-assisted Evolutionary Diversity Optimization-based defense strategy, where the attacker and defender improve each other strategy via parallel gameplay. To address the computational challenges of training attacker-defender strategies on numerous dynamic AD graphs, we propose an RL Training Facilitator that prunes environments and neural networks to eliminate irrelevant elements, enabling efficient and scalable training for large graphs. We extensively train the attacker strategy, as a sophisticated attacker model is essential for a robust defense. Our empirical results successfully demonstrate that our proposed approach enhances defender's proficiency in hardening dynamic AD graphs while ensuring scalability for large-scale AD., Comment: The manuscript has been accepted as full paper at European Symposium on Research in Computer Security (ESORICS) 2024

Recent approaches representing 3D objects and scenes using Gaussian splats show increased rendering speed across a variety of platforms and devices. While rendering such representations is indeed extremely efficient, storing and transmitting them is often prohibitively expensive. To represent large-scale scenes, one often needs to store millions of 3D Gaussians, occupying gigabytes of disk space. This poses a very practical limitation, prohibiting widespread adoption.Several solutions have been proposed to strike a balance between disk size and rendering quality, noticeably reducing the visual quality. In this work, we propose a new representation that dramatically reduces the hard drive footprint while featuring similar or improved quality when compared to the standard 3D Gaussian splats. When compared to other compact solutions, ours offers higher quality renderings with significantly reduced storage, being able to efficiently run on a mobile device in real-time. Our key observation is that nearby points in the scene can share similar representations. Hence, only a small ratio of 3D points needs to be stored. We introduce an approach to identify such points which are called parent points. The discarded points called children points along with attributes can be efficiently predicted by tiny MLPs., Comment: Project Page: https://plumpuddings.github.io/LPGS//

We study how to subvert large language models (LLMs) from following prompt-specified rules. We model rule-following as inference in propositional Horn logic, a mathematical system in which rules have the form ``if $P$ and $Q$, then $R$'' for some propositions $P$, $Q$, and $R$. We prove that although LLMs can faithfully follow such rules, maliciously crafted prompts can mislead even idealized, theoretically constructed models. Empirically, we find that the reasoning behavior of LLMs aligns with that of our theoretical constructions, and popular attack algorithms find adversarial prompts with characteristics predicted by our theory. Our logic-based framework provides a novel perspective for mechanistically understanding the behavior of LLMs in rule-based settings such as jailbreak attacks.