Search

Reconstructing brain sources is a fundamental challenge in neuroscience, crucial for understanding brain function and dysfunction. Electroencephalography (EEG) signals have a high temporal resolution. However, identifying the correct spatial location of brain sources from these signals remains difficult due to the ill-posed structure of the problem. Traditional methods predominantly rely on manually crafted priors, missing the flexibility of data-driven learning, while recent deep learning approaches focus on end-to-end learning, typically using the physical information of the forward model only for generating training data. We propose the novel hybrid method 3D-PIUNet for EEG source localization that effectively integrates the strengths of traditional and deep learning techniques. 3D-PIUNet starts from an initial physics-informed estimate by using the pseudo inverse to map from measurements to source space. Secondly, by viewing the brain as a 3D volume, we use a 3D convolutional U-Net to capture spatial dependencies and refine the solution according to the learned data prior. Training the model relies on simulated pseudo-realistic brain source data, covering different source distributions. Trained on this data, our model significantly improves spatial accuracy, demonstrating superior performance over both traditional and end-to-end data-driven methods. Additionally, we validate our findings with real EEG data from a visual task, where 3D-PIUNet successfully identifies the visual cortex and reconstructs the expected temporal behavior, thereby showcasing its practical applicability., Comment: Under Review in IEEE Transactions on Medical Imaging

This paper introduces a framework for an indoor autonomous mobility system that can perform patient transfers and materials handling. Unlike traditional systems that rely on onboard perception sensors, the proposed approach leverages a global perception and localization (PL) through Infrastructure Sensor Nodes (ISNs) and cloud computing technology. Using the global PL, an integrated Model Predictive Control (MPC)-based local planning and tracking controller augmented with Artificial Potential Field (APF) is developed, enabling reliable and efficient motion planning and obstacle avoidance ability while tracking predefined reference motions. Simulation results demonstrate the effectiveness of the proposed MPC controller in smoothly navigating around both static and dynamic obstacles. The proposed system has the potential to extend to intelligent connected autonomous vehicles, such as electric or cargo transport vehicles with four-wheel independent drive/steering (4WID-4WIS) configurations.

In this paper, we propose an accurate and robust perception module for Autonomous Vehicles (AVs) for drivable space extraction. Perception is crucial in autonomous driving, where many deep learning-based methods, while accurate on benchmark datasets, fail to generalize effectively, especially in diverse and unpredictable environments. Our work introduces a robust easy-to-generalize perception module that leverages LiDAR, camera, and HD map data fusion to deliver a safe and reliable drivable space in all weather conditions. We present an adaptive ground removal and curb detection method integrated with HD map data for enhanced obstacle detection reliability. Additionally, we propose an adaptive DBSCAN clustering algorithm optimized for precipitation noise, and a cost-effective LiDAR-camera frustum association that is resilient to calibration discrepancies. Our comprehensive drivable space representation incorporates all perception data, ensuring compatibility with vehicle dimensions and road regulations. This approach not only improves generalization and efficiency, but also significantly enhances safety in autonomous vehicle operations. Our approach is tested on a real dataset and its reliability is verified during the daily (including harsh snowy weather) operation of our autonomous shuttle, WATonoBus

Federated learning is a prominent distributed learning paradigm that incorporates collaboration among diverse clients, promotes data locality, and thus ensures privacy. These clients have their own technological, cultural, and other biases in the process of data generation. However, the present standard often ignores this bias/heterogeneity, perpetuating bias against certain groups rather than mitigating it. In response to this concern, we propose an equitable clustering-based framework where the clients are categorized/clustered based on how similar they are to each other. We propose a unique way to construct the similarity matrix that uses activation vectors. Furthermore, we propose a client weighing mechanism to ensure that each cluster receives equal importance and establish $O(1/\sqrt{K})$ rate of convergence to reach an $\epsilon-$stationary solution. We assess the effectiveness of our proposed strategy against common baselines, demonstrating its efficacy in terms of reducing the bias existing amongst various client clusters and consequently ameliorating algorithmic bias against specific groups., Comment: 28 pages

Seamlessly integrating stimulated Brillouin scattering (SBS) in a low-loss and mature photonic integration platform remains a complicated task. Virtually all current approaches fall short in simultaneously achieving strong SBS, low losses, and technological scalability. In this work we incorporate stong SBS into a standard silicon nitride platform by a simple deposition of a tellurium oxide layer, a commonly used material for acousto-optic modulators. In these heterogeneously integrated waveguides, we harness novel SBS interactions actuated by surface acoustic waves (SAWs) leading to more than two orders of magnitude gain enhancement. Three novel applications are demonstrated in this platform: (i) a silicon nitride Brillouin amplifier with 5 dB net optical gain, (ii) a compact intermodal stimulated Brillouin laser (SBL) capable of high purity radio frequency (RF) signal generation with 7 Hz intrinsic linewidth, and (iii) a widely tunable microwave photonic notch filter with ultra-narrow linewidth of 2.2 MHz enabled by Brillouin induced opacity. These advancements can unlock an array of new RF and optical technologies to be directly integrated in silicon nitride.

Attributing answers to source documents is an approach used to enhance the verifiability of a model's output in retrieval augmented generation (RAG). Prior work has mainly focused on improving and evaluating the attribution quality of large language models (LLMs) in RAG, but this may come at the expense of inducing biases in the attribution of answers. We define and examine two aspects in the evaluation of LLMs in RAG pipelines, namely attribution sensitivity and bias with respect to authorship information. We explicitly inform an LLM about the authors of source documents, instruct it to attribute its answers, and analyze (i) how sensitive the LLM's output is to the author of source documents, and (ii) whether the LLM exhibits a bias towards human-written or AI-generated source documents. We design an experimental setup in which we use counterfactual evaluation to study three LLMs in terms of their attribution sensitivity and bias in RAG pipelines. Our results show that adding authorship information to source documents can significantly change the attribution quality of LLMs by 3% to 18%. Moreover, we show that LLMs can have an attribution bias towards explicit human authorship, which can serve as a competing hypothesis for findings of prior work that shows that LLM-generated content may be preferred over human-written contents. Our findings indicate that metadata of source documents can influence LLMs' trust, and how they attribute their answers. Furthermore, our research highlights attribution bias and sensitivity as a novel aspect of brittleness in LLMs.

The large hadron collider precision measurement of $h_{125}$ properties suggests that a misaligned two Higgs doublet model type 1 is preferred among other types of the model. In this analysis, a lepton collider potential for charged Higgs boson observation is evaluated and compared with HL-LHC expectations. We focus on $H^{\pm} \to W^{\pm}h_{125}$ decay which is exclusively activated in misalignment scenario. An event analysis including fast detector simulation is performed and it is shown that some parts of the parameter space which are out of HL-LHC reach can be probed by a high energy lepton collider with signal significance exceeding $2\sigma$ and $5\sigma$ for exclusion or discovery. The region with highest sensitivity is in the middle range charged Higgs boson mass, i.e., $350 < m_{H^{\pm}} <450$ GeV and $\tan \beta \gtrsim4$ at non-zero $\cos(\beta-\alpha)$ values., Comment: 14 pages, 11 figures

Distributed stochastic non-convex optimization problems have recently received attention due to the growing interest of signal processing, computer vision, and natural language processing communities in applications deployed over distributed learning systems (e.g., federated learning). We study the setting where the data is distributed across the nodes of a time-varying directed network, a topology suitable for modeling dynamic networks experiencing communication delays and straggler effects. The network nodes, which can access only their local objectives and query a stochastic first-order oracle to obtain gradient estimates, collaborate to minimize a global objective function by exchanging messages with their neighbors. We propose an algorithm, novel to this setting, that leverages stochastic gradient descent with momentum and gradient tracking to solve distributed non-convex optimization problems over time-varying networks. To analyze the algorithm, we tackle the challenges that arise when analyzing dynamic network systems which communicate gradient acceleration components. We prove that the algorithm's oracle complexity is $\mathcal{O}(1/\epsilon^{1.5})$, and that under Polyak-$\L$ojasiewicz condition the algorithm converges linearly to a steady error state. The proposed scheme is tested on several learning tasks: a non-convex logistic regression experiment on the MNIST dataset, an image classification task on the CIFAR-10 dataset, and an NLP classification test on the IMDB dataset. We further present numerical simulations with an objective that satisfies the PL condition. The results demonstrate superior performance of the proposed framework compared to the existing related methods., Comment: This work has been accepted at IEEE Transactions on Automatic Control

The rapid development of automated vehicles (AVs) promises to revolutionize transportation by enhancing safety and efficiency. However, ensuring their reliability in diverse real-world conditions remains a significant challenge, particularly due to rare and unexpected situations known as edge cases. Although numerous approaches exist for detecting edge cases, there is a notable lack of a comprehensive survey that systematically reviews these techniques. This paper fills this gap by presenting a practical, hierarchical review and systematic classification of edge case detection and assessment methodologies. Our classification is structured on two levels: first, categorizing detection approaches according to AV modules, including perception-related and trajectory-related edge cases; and second, based on underlying methodologies and theories guiding these techniques. We extend this taxonomy by introducing a new class called "knowledge-driven" approaches, which is largely overlooked in the literature. Additionally, we review the techniques and metrics for the evaluation of edge case detection methods and identified edge cases. To our knowledge, this is the first survey to comprehensively cover edge case detection methods across all AV subsystems, discuss knowledge-driven edge cases, and explore evaluation techniques for detection methods. This structured and multi-faceted analysis aims to facilitate targeted research and modular testing of AVs. Moreover, by identifying the strengths and weaknesses of various approaches and discussing the challenges and future directions, this survey intends to assist AV developers, researchers, and policymakers in enhancing the safety and reliability of automated driving (AD) systems through effective edge case detection., Comment: Preprint submitted to IEEE Transactions on Intelligent Transportation Systems

We study the evolution of a newly formed magnetized neutron-star (NS) as a power source of gamma-ray bursts (GRBs) in the light of both gravitational wave (GW) and electromagnetic (EM) radiations. The compressible and incompressible fluids are employed in order to model the secular evolution of Maclaurian spheroids. It is shown that the GW and EM light curves evolve as a function of eccentricity and rotational frequency with time. We find that the light curve characteristics crucially depend on NS parameters such as magnitude and structure of magnetic field, ellipticity and the equation of state (EOS) of the fluid. The presence of X-ray flares, whose origins are not yet well understood, can be captured in our model regarding some specific nuclear EOSs. Our model allowing us to explain flares that occur within the wide range of $ 10$ to $10^4$ seconds and the peak luminosity in the order of $10^{46}$ - $10^{51}$ $\rm \text{erg}/s$ by using a reasonable set of parameters such as magnetic field strength around $10^{14}-10^{16}$ Gauss, the quadrupole to dipole ratio of magnetic field up to 500. By applying our model to a sample of GRB X-ray flares observed by Swift/XRT, we try to constraint the crucial parameters of a deformed magnetar via MCMC fitting method. Our analysis shows that ongoing and upcoming joint multi-messenger detections can be used to understand the nature of GRB's central engine and its evolution at the early times of the burst formation.

Path planning is an essential component of autonomous driving. A global planner is responsible for the high-level planning. It basically performs a shortest-path search on a known map, thereby defining waypoints used to control the local (low-level) planner. Local planning is a runtime verification method which is repeatedly run on the vehicle itself in real-time, so as to find the optimal short-horizon path which leads to the desired waypoint in a way which is both efficient and safe. The challenge is that the local planner has to take into account repeatedly incoming updates about the information available of the environment. In addition, it performs a complex task, as it has to take into account a large variety of requirements, originating from the necessity of collision avoidance with obstacles, respecting traffic rules, sticking to regulatory requirements, and lastly to reach the next waypoint efficiently. In this paper, we describe a logic-based specification mechanism which fulfills all these requirements., Comment: 10 pages, 5 Figures, 1 Table, Accepted as a short paper at 27th Brazilian Symposium on Formal Methods (SBMF 2024)

Since neural networks can make wrong predictions even with high confidence, monitoring their behavior at runtime is important, especially in safety-critical domains like autonomous driving. In this paper, we combine ideas from previous monitoring approaches based on observing the activation values of hidden neurons. In particular, we combine the Gaussian-based approach, which observes whether the current value of each monitored neuron is similar to typical values observed during training, and the Outside-the-Box monitor, which creates clusters of the acceptable activation values, and, thus, considers the correlations of the neurons' values. Our experiments evaluate the achieved improvement.

Intrinsic motivation, inspired by the psychology of developmental learning in infants, stimulates exploration in agents without relying solely on sparse external rewards. Existing methods in reinforcement learning like Random Network Distillation (RND) face significant limitations, including (1) relying on raw visual inputs, leading to a lack of meaningful representations, (2) the inability to build a robust latent space, (3) poor target network initialization and (4) rapid degradation of intrinsic rewards. In this paper, we introduce Pre-trained Network Distillation (PreND), a novel approach to enhance intrinsic motivation in reinforcement learning (RL) by improving upon the widely used prediction-based method, RND. PreND addresses these challenges by incorporating pre-trained representation models into both the target and predictor networks, resulting in more meaningful and stable intrinsic rewards, while enhancing the representation learned by the model. We also tried simple but effective variants of the predictor network optimization by controlling the learning rate. Through experiments on the Atari domain, we demonstrate that PreND significantly outperforms RND, offering a more robust intrinsic motivation signal that leads to better exploration, improving overall performance and sample efficiency. This research highlights the importance of target and predictor networks representation in prediction-based intrinsic motivation, setting a new direction for improving RL agents' learning efficiency in sparse reward environments., Comment: 8 pages, 4 figures

We develop a simple two-step algorithm for enclosing Chebyshev expansions whose cost is linear in terms of the polynomial degree. The algorithm first transforms the expansion from Chebyshev to the Laurent basis and then applies the interval Horner method. It outperforms the existing eigenvalue-based methods if the degree is high or the evaluation point is close to the boundaries of the domain.

Accurate imputation is essential for the reliability and success of downstream tasks. Recently, diffusion models have attracted great attention in this field. However, these models neglect the latent distribution in a lower-dimensional space derived from the observed data, which limits the generative capacity of the diffusion model. Additionally, dealing with the original missing data without labels becomes particularly problematic. To address these issues, we propose the Latent Space Score-Based Diffusion Model (LSSDM) for probabilistic multivariate time series imputation. Observed values are projected onto low-dimensional latent space and coarse values of the missing data are reconstructed without knowing their ground truth values by this unsupervised learning approach. Finally, the reconstructed values are fed into a conditional diffusion model to obtain the precise imputed values of the time series. In this way, LSSDM not only possesses the power to identify the latent distribution but also seamlessly integrates the diffusion model to obtain the high-fidelity imputed values and assess the uncertainty of the dataset. Experimental results demonstrate that LSSDM achieves superior imputation performance while also providing a better explanation and uncertainty analysis of the imputation mechanism. The website of the code is \textit{https://github.com/gorgen2020/LSSDM\_imputation}., Comment: 5 pages, conference

Collaborative learning is an important tool to train multiple clients more effectively by enabling communication among clients. Identifying helpful clients, however, presents challenging and often introduces significant overhead. In this paper, we model client-selection and model-training as two interconnected optimization problems, proposing a novel bilevel optimization problem for collaborative learning. We introduce CoBo, a scalable and elastic, SGD-type alternating optimization algorithm that efficiently addresses these problem with theoretical convergence guarantees. Empirically, CoBo achieves superior performance, surpassing popular personalization algorithms by 9.3% in accuracy on a task with high heterogeneity, involving datasets distributed among 80 clients.

For many materials, Raman spectra are intricately structured and provide valuable information about compositional stoichiometry and crystal quality. Here we use density-functional theory calculations, mass approximation, and the Raman intensity weighted $\Gamma$-point density of state approach to analyze Raman scattering and vibrational modes in zincblende, wurtzite, and hexagonal BX (X = N, P, and As) structures. The influence of crystal structure and boron isotope disorder on Raman line shapes is examined. Our results demonstrate that long-range Coulomb interactions significantly influence the evolution of Raman spectra in cubic and wurtzite BN compounds. With the evolution of the compositional rate from $^{11}$B to $^{10}$B, a shift toward higher frequencies, as well as the maximum broadening and asymmetry of the Raman peaks, is expected around the 1:1 ratio. The calculated results are in excellent agreement with the available experimental data. This study serves as a guide for understanding how crystal symmetry and isotope disorder affect phonons in BX compounds, which are relevant to quantum single-photon emitters, heat management, and crystal quality assessments.

We study the maximum weight matching problem in the random-order semi-streaming model and in the robust communication model. Unlike many other sublinear models, in these two frameworks, there is a large gap between the guarantees of the best known algorithms for the unweighted and weighted versions of the problem. In the random-order semi-streaming setting, the edges of an $n$-vertex graph arrive in a stream in a random order. The goal is to compute an approximate maximum weight matching with a single pass over the stream using $O(n\text{ polylog } n)$ space. Our main result is a $(2/3-\epsilon)$-approximation algorithm for maximum weight matching in random-order streams, using space $O(n \log n \log R)$, where $R$ is the ratio between the heaviest and the lightest edge in the graph. Our result nearly matches the best known unweighted $(2/3+\epsilon_0)$-approximation (where $\epsilon_0 \sim 10^{-14}$ is a small constant) achieved by Assadi and Behnezhad [ICALP 2021], and significantly improves upon previous weighted results. Our techniques also extend to the related robust communication model, in which the edges of a graph are partitioned randomly between Alice and Bob. Alice sends a single message of size $O(n\text{ polylog }n)$ to Bob, who must compute an approximate maximum weight matching. We achieve a $(5/6-\epsilon)$-approximation using $O(n \log n \log R)$ words of communication, matching the results of Azarmehr and Behnezhad [ICALP 2023] for unweighted graphs.

How can Transformers model and learn enumerative geometry? What is a robust procedure for using Transformers in abductive knowledge discovery within a mathematician-machine collaboration? In this work, we introduce a new paradigm in computational enumerative geometry in analyzing the $\psi$-class intersection numbers on the moduli space of curves. By formulating the enumerative problem as a continuous optimization task, we develop a Transformer-based model for computing $\psi$-class intersection numbers based on the underlying quantum Airy structure. For a finite range of genera, our model is capable of regressing intersection numbers that span an extremely wide range of values, from $10^{-45}$ to $10^{45}$. To provide a proper inductive bias for capturing the recursive behavior of intersection numbers, we propose a new activation function, Dynamic Range Activator (DRA). Moreover, given the severe heteroscedasticity of $\psi$-class intersections and the required precision, we quantify the uncertainty of the predictions using Conformal Prediction with a dynamic sliding window that is aware of the number of marked points. Next, we go beyond merely computing intersection numbers and explore the enumerative "world-model" of the Transformers. Through a series of causal inference and correlational interpretability analyses, we demonstrate that Transformers are actually modeling Virasoro constraints in a purely data-driven manner. Additionally, we provide evidence for the comprehension of several values appearing in the large genus asymptotic of $\psi$-class intersection numbers through abductive hypothesis testing.

In a conventional Federated Learning framework, client selection for training typically involves the random sampling of a subset of clients in each iteration. However, this random selection often leads to disparate performance among clients, raising concerns regarding fairness, particularly in applications where equitable outcomes are crucial, such as in medical or financial machine learning tasks. This disparity typically becomes more pronounced with the advent of performance-centric client sampling techniques. This paper introduces two novel methods, namely SUBTRUNC and UNIONFL, designed to address the limitations of random client selection. Both approaches utilize submodular function maximization to achieve more balanced models. By modifying the facility location problem, they aim to mitigate the fairness concerns associated with random selection. SUBTRUNC leverages client loss information to diversify solutions, while UNIONFL relies on historical client selection data to ensure a more equitable performance of the final model. Moreover, these algorithms are accompanied by robust theoretical guarantees regarding convergence under reasonable assumptions. The efficacy of these methods is demonstrated through extensive evaluations across heterogeneous scenarios, revealing significant improvements in fairness as measured by a client dissimilarity metric., Comment: 13 pages

Diverse critical data, such as location information and driving patterns, can be collected by IoT devices in vehicular networks to improve driving experiences and road safety. However, drivers are often reluctant to share their data due to privacy concerns. The Federated Vehicular Network (FVN) is a promising technology that tackles these concerns by transmitting model parameters instead of raw data, thereby protecting the privacy of drivers. Nevertheless, the performance of Federated Learning (FL) in a vehicular network depends on the joining ratio, which is restricted by the limited available wireless resources. To address these challenges, this paper proposes to apply Non-Orthogonal Multiple Access (NOMA) to improve the joining ratio in a FVN. Specifically, a vehicle selection and transmission power control algorithm is developed to exploit the power domain differences in the received signal to ensure the maximum number of vehicles capable of joining the FVN. Our simulation results demonstrate that the proposed NOMA-based strategy increases the joining ratio and significantly enhances the performance of the FVN., Comment: The paper is accepted by IEEE Globecom 2024

In this paper, we study the problem of distributed transmission control and video streaming optimization for unmanned aerial vehicles (UAVs) operating in unlicensed spectrum bands. We develop a rigorous cross-layer analysis framework that jointly considers three inter-dependent factors: (i) in-band interference introduced by ground/aerial nodes at the physical (PHY) layer, (ii) limited-size queues with delay-constrained packet arrival at the medium access control (MAC) layer, and (iii) video encoding rate at the application layer. First, we formulate an optimization problem to maximize the throughput by optimizing the fading threshold. To this end, we jointly analyze the queue-related packet loss probabilities (i.e., buffer overflow and time threshold event) as well as the outage probability due to the low signal-to-interference-plus-noise ratio (SINR). We introduce the Distributed Transmission Control (DTC) algorithm that maximizes the throughput by adjusting transmission policies to balance the trade-offs between packet drop from queues vs. transmission errors due to low SINRs. Second, we incorporate the video distortion model to develop distributed peak signal-to-noise ratio (PSNR) optimization for video streaming. The formulated optimization incorporates two cross-layer parameters, specifically the fading threshold and video encoding rate. To tackle this problem, we develop the Joint Distributed Video Transmission and Encoder Control (JDVT-EC) algorithm that enhances the PSNR for all nodes by fine-tuning transmission policies and video encoding rates to balance the trade-offs between packet loss and lossy video compression distortions. Through extensive numerical analysis, we thoroughly examine the proposed algorithms and demonstrate that they are able to find the optimal transmission policies and video encoding rates under various scenarios., Comment: Submitted to IEEE Transactions on Communications

In the last decade, an impressive increase in software adaptions has led to a surge in log data production, making manual log analysis impractical and establishing the necessity for automated methods. Conversely, most automated analysis tools include a component designed to separate log templates from their parameters, commonly referred to as a "log parser". This paper aims to introduce a new fast and accurate log parser, named "Tipping". Tipping combines rule-based tokenizers, interdependency token graphs, strongly connected components, and various techniques to ensure rapid, scalable, and precise log parsing. Furthermore, Tipping is parallelized and capable of running on multiple processing cores with close to linear efficiency. We evaluated Tipping against other state-of-the-art log parsers in terms of accuracy, performance, and the downstream task of anomaly detection. Accordingly, we found that Tipping outperformed existing methods in accuracy and performance in our evaluations. More in-depth, Tipping can parse 11 million lines of logs in less than 20 seconds on a laptop machine. Furthermore, we re-implemented a parallelized version of the past IpLom algorithm to demonstrate the effect of parallel processing, and it became the second-fastest parser. As logs keep growing in volume and complexity, the software engineering community needs to ensure automated log analysis tools keep up with the demand, being capable of efficiently handling massive volumes of logs with high accuracy. Tipping's robustness, versatility, efficiency, and scalability make it a viable tool for the modern automated log analysis task.

Due to their inherent capabilities of capturing non-local dependencies, Transformer neural networks have quickly been established as the paradigmatic architecture for large language models and image processing. Next to these traditional applications, machine learning methods have also been demonstrated to be versatile tools in the analysis of image-like data of quantum phases of matter, e.g. given snapshots of many-body wave functions obtained in ultracold atom experiments. While local correlation structures in image-like data of physical systems can reliably be detected, identifying phases of matter characterized by global, non-local structures with interpretable machine learning methods remains a challenge. Here, we introduce the correlator Transformer (CoTra), which classifies different phases of matter while at the same time yielding full interpretability in terms of physical correlation functions. The network's underlying structure is a tailored attention mechanism, which learns efficient ways to weigh local and non-local correlations for a successful classification. We demonstrate the versatility of the CoTra by detecting local order in the Heisenberg antiferromagnet, and show that local gauge constraints in one- and two-dimensional lattice gauge theories can be identified. Furthermore, we establish that the CoTra reliably detects non-local structures in images of correlated fermions in momentum space (Cooper pairs) and that it can distinguish percolating from non-percolating images.

In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We also train the 2B and 9B models with knowledge distillation (Hinton et al., 2015) instead of next token prediction. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community.

Handwriting Verification is a critical in document forensics. Deep learning based approaches often face skepticism from forensic document examiners due to their lack of explainability and reliance on extensive training data and handcrafted features. This paper explores using Vision Language Models (VLMs), such as OpenAI's GPT-4o and Google's PaliGemma, to address these challenges. By leveraging their Visual Question Answering capabilities and 0-shot Chain-of-Thought (CoT) reasoning, our goal is to provide clear, human-understandable explanations for model decisions. Our experiments on the CEDAR handwriting dataset demonstrate that VLMs offer enhanced interpretability, reduce the need for large training datasets, and adapt better to diverse handwriting styles. However, results show that the CNN-based ResNet-18 architecture outperforms the 0-shot CoT prompt engineering approach with GPT-4o (Accuracy: 70%) and supervised fine-tuned PaliGemma (Accuracy: 71%), achieving an accuracy of 84% on the CEDAR AND dataset. These findings highlight the potential of VLMs in generating human-interpretable decisions while underscoring the need for further advancements to match the performance of specialized deep learning models., Comment: 4 Pages, 1 Figure, 1 Table, Accepted as Short paper at Irish Machine Vision and Image Processing (IMVIP) Conference

Abstention Ability (AA) is a critical aspect of Large Language Model (LLM) reliability, referring to an LLM's capability to withhold responses when uncertain or lacking a definitive answer, without compromising performance. Although previous studies have attempted to improve AA, they lack a standardised evaluation method and remain unsuitable for black-box models where token prediction probabilities are inaccessible. This makes comparative analysis challenging, especially for state-of-the-art closed-source commercial LLMs. This paper bridges this gap by introducing a black-box evaluation approach and a new dataset, Abstain-QA, crafted to rigorously assess AA across varied question types (answerable and unanswerable), domains (well-represented and under-represented), and task types (fact centric and reasoning). We also propose a new confusion matrix, the ''Answerable-Unanswerable Confusion Matrix'' (AUCM) which serves as the basis for evaluating AA, by offering a structured and precise approach for assessment. Finally, we explore the impact of three prompting strategies-Strict Prompting, Verbal Confidence Thresholding, and Chain-of-Thought (CoT)-on improving AA. Our results indicate that even powerful models like GPT-4, Mixtral 8x22b encounter difficulties with abstention; however, strategic approaches such as Strict prompting and CoT can enhance this capability., Comment: 8 pages (excluding limitations, references and appendix) and 5 figures

Liver zonation is a fundamental characteristic of hepatocyte spatial heterogeneity, which is challenging to recapitulate in traditional cell cultures. This study presents a novel microfluidic device designed to induce zonation in liver cell cultures by establishing an oxygen gradient using standard laboratory gases. The device consists of two layers; a bottom layer containing a gas channel network that delivers high and low oxygenated gases to create three distinct zones within the cell culture chamber in the layer above. Computational simulations and ratiometric oxygen sensing were employed to validate the oxygen gradient, demonstrating that stable oxygen levels were achieved within two hours. Liver zonation was confirmed using immunofluorescence staining, which showed zonated albumin production in HepG2 cells directly correlating with oxygen levels and mimicking in-vivo zonation behavior. This user-friendly device supports studies on liver zonation and related metabolic disease mechanisms in vitro. It can also be utilized for experiments that necessitate precise gas concentration gradients, such as hypoxia-related research areas focused on angiogenesis and cancer development.

Reachability analysis is a popular method to give safety guarantees for stochastic cyber-physical systems (SCPSs) that takes in a symbolic description of the system dynamics and uses set-propagation methods to compute an overapproximation of the set of reachable states over a bounded time horizon. In this paper, we investigate the problem of performing reachability analysis for an SCPS that does not have a symbolic description of the dynamics, but instead is described using a digital twin model that can be simulated to generate system trajectories. An important challenge is that the simulator implicitly models a probability distribution over the set of trajectories of the SCPS; however, it is typical to have a sim2real gap, i.e., the actual distribution of the trajectories in a deployment setting may be shifted from the distribution assumed by the simulator. We thus propose a statistical reachability analysis technique that, given a user-provided threshold $1-\epsilon$, provides a set that guarantees that any reachable state during deployment lies in this set with probability not smaller than this threshold. Our method is based on three main steps: (1) learning a deterministic surrogate model from sampled trajectories, (2) conducting reachability analysis over the surrogate model, and (3) employing {\em robust conformal inference} using an additional set of sampled trajectories to quantify the surrogate model's distribution shift with respect to the deployed SCPS. To counter conservatism in reachable sets, we propose a novel method to train surrogate models that minimizes a quantile loss term (instead of the usual mean squared loss), and a new method that provides tighter guarantees using conformal inference using a normalized surrogate error. We demonstrate the effectiveness of our technique on various case studies.

For the purpose of causal inference we employ a stochastic model of the data generating process, utilizing individual propensity probabilities for the treatment, and also individual and counterfactual prognosis probabilities for the outcome. We assume a generalized version of the stable unit treatment value assumption, but we do not assume any version of strongly ignorable treatment assignment. Instead of conducting a sensitivity analysis, we utilize the principle of maximum entropy to estimate the distribution of causal effects. We develop a principled middle-way between extreme explanations of the observed data: we do not conclude that an observed association is wholly spurious, and we do not conclude that it is wholly causal. Rather, our conclusions are tempered and we conclude that the association is part spurious and part causal. In an example application we apply our methodology to analyze an observed association between marijuana use and hard drug use.

We present a sequential version of the multilinear Nystr\"om algorithm which is suitable for the low-rank Tucker approximation of tensors given in a streaming format. Accessing the tensor $\mathcal{A}$ exclusively through random sketches of the original data, the algorithm effectively leverages structures in $\mathcal{A}$, such as low-rankness, and linear combinations. We present a deterministic analysis of the algorithm and demonstrate its superior speed and efficiency in numerical experiments including an application in video processing.

Is explainability a false promise? This debate has emerged from the insufficient evidence that explanations aid people in situations they are introduced for. More human-centered, application-grounded evaluations of explanations are needed to settle this. Yet, with no established guidelines for such studies in NLP, researchers accustomed to standardized proxy evaluations must discover appropriate measurements, tasks, datasets, and sensible models for human-AI teams in their studies. To help with this, we first review fitting existing metrics. We then establish requirements for datasets to be suitable for application-grounded evaluations. Among over 50 datasets available for explainability research in NLP, we find that 4 meet our criteria. By finetuning Flan-T5-3B, we demonstrate the importance of reassessing the state of the art to form and study human-AI teams. Finally, we present the exemplar studies of human-AI decision-making for one of the identified suitable tasks -- verifying the correctness of a legal claim given a contract., Comment: 9 pages main, 7 pages references, 32 pages appendix

We study a multi-task decision-making problem for 360 video processing in a wireless multi-user virtual reality (VR) system that includes an edge computing unit (ECU) to deliver 360 videos to VR users and offer computing assistance for decoding/rendering of video frames. However, this comes at the expense of increased data volume and required bandwidth. To balance this trade-off, we formulate a constrained quality of experience (QoE) maximization problem in which the rebuffering time and quality variation between video frames are bounded by user and video requirements. To solve the formulated multi-user QoE maximization, we leverage deep reinforcement learning (DRL) for multi-task rate adaptation and computation distribution (MTRC). The proposed MTRC approach does not rely on any predefined assumption about the environment and relies on video playback statistics (i.e., past throughput, decoding time, transmission time, etc.), video information, and the resulting performance to adjust the video bitrate and computation distribution. We train MTRC with real-world wireless network traces and 360 video datasets to obtain evaluation results in terms of the average QoE, peak signal-to-noise ratio (PSNR), rebuffering time, and quality variation. Our results indicate that the MTRC improves the users' QoE compared to state-of-the-art rate adaptation algorithm. Specifically, we show a 5.97 dB to 6.44 dB improvement in PSNR, a 1.66X to 4.23X improvement in rebuffering time, and a 4.21 dB to 4.35 dB improvement in quality variation., Comment: 2024 IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR)

Non-convex functional constrained optimization problems have gained substantial attention in machine learning and signal processing. This paper develops a new primal-dual algorithm for solving this class of problems. The algorithm is based on a novel form of the Lagrangian function, termed {\em Proximal-Perturbed Augmented Lagrangian}, which enables us to develop an efficient and simple first-order algorithm that converges to a stationary solution under mild conditions. Our method has several key features of differentiation over existing augmented Lagrangian-based methods: (i) it is a single-loop algorithm that does not require the continuous adjustment of the penalty parameter to infinity; (ii) it can achieves an improved iteration complexity of $\widetilde{\mathcal{O}}(1/\epsilon^2)$ or at least ${\mathcal{O}}(1/\epsilon^{2/q})$ with $q \in (2/3,1)$ for computing an $\epsilon$-approximate stationary solution, compared to the best-known complexity of $\mathcal{O}(1/\epsilon^3)$; and (iii) it effectively handles functional constraints for feasibility guarantees with fixed parameters, without imposing boundedness assumptions on the dual iterates and the penalty parameters. We validate the effectiveness of our method through numerical experiments on popular non-convex problems.

In many complex systems, the interactions between objects span multiple aspects. Multiplex networks are accurate paradigms to model such systems, where each edge is associated with a type. A key graph mining primitive is extracting dense subgraphs, and this has led to interesting notions such as K-cores, known as building blocks of complex networks. Despite recent attempts to extend the notion of core to multiplex networks, existing studies suffer from a subset of the following limitations: They 1) force all nodes to exhibit their high degree in the same set of relation types while in multiplex networks some connection types can be noisy for some nodes, 2) either require high computational cost or miss the complex information of multiplex networks, and 3) assume the same importance for all relation types. We introduce S-core, a novel and unifying family of dense structures in multiplex networks that uses a function S(.) to summarize the degree vector of each node. We then discuss how one can choose a proper S(.) from the data. To demonstrate the usefulness of S-cores, we focus on finding the densest subgraph as well as modeling user engagement in multiplex networks. We present a new density measure in multiplex networks and discuss its advantages over existing density measures. We show that the problem of finding the densest subgraph in multiplex networks is NP-hard and design an efficient approximation algorithm based on S-cores. Finally, we present a new mathematical model of user engagement in the presence of different relation types. Our experiments shows the efficiency and effectiveness of our algorithms and supports the proposed mathematical model of user engagement.

In this paper, we propose a data-driven framework for collaborative wideband spectrum sensing and scheduling for networked unmanned aerial vehicles (UAVs), which act as the secondary users (SUs) to opportunistically utilize detected "spectrum holes". Our overall framework consists of three main stages. Firstly, in the model training stage, we explore dataset generation in a multi-cell environment and training a machine learning (ML) model using the federated learning (FL) architecture. Unlike the existing studies on FL for wireless that presume datasets are readily available for training, we propose a novel architecture that directly integrates wireless dataset generation, which involves capturing I/Q samples from over-the-air signals in a multi-cell environment, into the FL training process. Secondly, in the collaborative spectrum inference stage, we propose a collaborative spectrum fusion strategy that is compatible with the unmanned aircraft system traffic management (UTM) ecosystem. Finally, in the spectrum scheduling stage, we leverage reinforcement learning (RL) solutions to dynamically allocate the detected spectrum holes to the secondary users. To evaluate the proposed methods, we establish a comprehensive simulation framework that generates a near-realistic synthetic dataset using MATLAB LTE toolbox by incorporating base-station~(BS) locations in a chosen area of interest, performing ray-tracing, and emulating the primary users channel usage in terms of I/Q samples. This evaluation methodology provides a flexible framework to generate large spectrum datasets that could be used for developing ML/AI-based spectrum management solutions for aerial devices., Comment: This is a preprint version submitted to IEEE Transactions on Machine learning in Communications and Networking. arXiv admin note: text overlap with arXiv:2308.05036

NGC 5128 (Cen A) is the nearest giant elliptical galaxy and one of the brightest extragalactic radio sources in the sky, boasting a prominent dust lane and jets emanating from its nuclear supermassive black hole. In this paper, we construct the star formation history (SFH) of two small fields in the halo of NGC 5128: a northeastern field (Field 1) at a projected distance of $\sim 18.8$ kpc from the center, and a southern field (Field 2) $\sim 9.9$ kpc from the center. Our method is based on identifying long period variable (LPV) stars that trace their sibling stellar population and hence historical star formation due to their high luminosity and strong variability; we identified 395 LPVs in Field 1 and 671 LPVs in Field 2. Even though the two fields are $\sim 28$ kpc apart on opposite sides from the center, they show similar SFHs. In Field 1, star formation rates (SFRs) increased significantly around $t\sim 800$ Myr and $t\sim 3.8$ Gyr; and in Field 2, SFRs increased considerably around $t\sim 800$ Myr, $t\sim 3.8$ Gyr, and $t\sim 6.3$ Gyr, where $t$ is look--back time. The increase in SFR $\sim 800$ Myr ago agrees with previous suggestions that the galaxy experienced a merger around that time. The SFH reconstructed from LPVs supports a scenario in which multiple episodes of nuclear activity lead to episodic jet-induced star formation. While there is no catalog of LPVs for the central part of NGC 5128, applying our method to the outer regions (for the first time in a galaxy outside the Local Group) has enabled us to put constraints on the complex evolution of this cornerstone galaxy., Comment: Accepted for publication in ApJ

The rapid advancement of 5G networks and the upcoming transition to 6G necessitate the use of the Open Radio Access Network (O-RAN) architecture to enable greater flexibility, interoperability, and innovation. This shift towards 6G and O-RAN requires the development of advanced simulation tools for testing, analyzing, and optimizing Radio Access Network (RAN) operations. This need becomes critical due to the complex dynamics of mobility management inherent in the 6G vision and next-generation networks. These networks anticipate advanced handover methods for mobile users, UAVs, IoT devices, and beyond. Addressing this gap, this paper introduces RANFusion: a robust RAN simulator specifically created to explore a variety of handover scenarios and to test and balance resources between users. This tool enables precise simulations for refining handover strategies within RAN and O-RAN environments, thereby ensuring optimal performance and reliability in these advanced network infrastructures., Comment: 7 pages, 5 Figure, 2 table

Motor Imagery (MI) is gaining traction in both rehabilitation and sports settings, but its immediate influence on human postural control is not yet clearly understood. The focus of this study is to examine the effects of MI on the dynamics of the Center of Pressure (COP), a crucial metric for evaluating postural stability. In the experiment, thirty healthy young adults participated in four different scenarios: normal standing with both open and closed eyes, and kinesthetic motor imagery focused on mediolateral (ML) and anteroposterior (AP) sway movements. A mathematical model was developed to characterize the nonlinear dynamics of the COP and to assess the impact of MI on these dynamics. Our results show a statistically significant increase (p-value<0.05) in variables such as COP path length and Long-Range Correlation (LRC) during MI compared to the closed-eye and normal standing conditions. These observations align well with psycho-neuromuscular theory, which suggests that imagining a specific movement activates neural pathways, consequently affecting postural control. This study presents compelling evidence that motor imagery not only has a quantifiable impact on COP dynamics but also that changes in the Center of Pressure (COP) are directionally consistent with the imagined movements. This finding holds significant implications for the field of rehabilitation science, suggesting that motor imagery could be strategically utilized to induce targeted postural adjustments. Nonetheless, additional research is required to fully understand the complex mechanisms that underlie this relationship and to corroborate these results across a more diverse set of populations.

We present SSL-HV: Self-Supervised Learning approaches applied to the task of Handwriting Verification. This task involves determining whether a given pair of handwritten images originate from the same or different writer distribution. We have compared the performance of multiple generative, contrastive SSL approaches against handcrafted feature extractors and supervised learning on CEDAR AND dataset. We show that ResNet based Variational Auto-Encoder (VAE) outperforms other generative approaches achieving 76.3% accuracy, while ResNet-18 fine-tuned using Variance-Invariance-Covariance Regularization (VICReg) outperforms other contrastive approaches achieving 78% accuracy. Using a pre-trained VAE and VICReg for the downstream task of writer verification we observed a relative improvement in accuracy of 6.7% and 9% over ResNet-18 supervised baseline with 10% writer labels., Comment: 8 pages, 2 figures, 2 tables, Accepted at Irish Machine Vision and Image Processing Conference 2024

We study the trajectory of iterations and the convergence rates of the Expectation-Maximization (EM) algorithm for two-component Mixed Linear Regression (2MLR). The fundamental goal of MLR is to learn the regression models from unlabeled observations. The EM algorithm finds extensive applications in solving the mixture of linear regressions. Recent results have established the super-linear convergence of EM for 2MLR in the noiseless and high SNR settings under some assumptions and its global convergence rate with random initialization has been affirmed. However, the exponent of convergence has not been theoretically estimated and the geometric properties of the trajectory of EM iterations are not well-understood. In this paper, first, using Bessel functions we provide explicit closed-form expressions for the EM updates under all SNR regimes. Then, in the noiseless setting, we completely characterize the behavior of EM iterations by deriving a recurrence relation at the population level and notably show that all the iterations lie on a certain cycloid. Based on this new trajectory-based analysis, we exhibit the theoretical estimate for the exponent of super-linear convergence and further improve the statistical error bound at the finite-sample level. Our analysis provides a new framework for studying the behavior of EM for Mixed Linear Regression., Comment: This paper was accepted by the 41st International Conference on Machine Learning (ICML 2024). The code for numerical experiments is available at https://github.com/dassein/cycloid_em_mlr

Multiple observations now suggest that the hydrogen reionization may have ended well below redshift six. While there has previously been no conclusive proof of extended neutral islands in the $z < 6$ intergalactic medium, it is possible that such islands give rise to the giant Ly$\alpha$ absorption troughs seen in the spectra of high-redshift quasars. Here we present evidence that the deepest and longest-known Ly$\alpha$ trough at $z < 6$, towards ULAS J0148+0600 (J0148), is associated with damping wing absorption. The evidence comes from a window of strong Ly$\alpha$ transmission at the edge of the J0148 proximity zone. We show that the relatively smooth profile of this transmission window is highly unlikely to arise from resonant absorption alone, but is consistent with the presence of a damping wing. We further argue that the damping wing is unlikely to arise from a compact source due to the lack of associated metal lines, and is more likely to arise from an extended neutral island associated with the giant Ly$\alpha$ trough. We investigate the physical conditions that may give rise to the strong transmission window, and speculate that it may signal an usually deep void, nearby ionizing sources, and/or the recent passage of an ionization front., Comment: 17 pages, 18 figures; resubmitted to MNRAS after addressing referee comments

We present a series of results, including local characterizations of $(\lambda,m+n)$-Einstein metrics in the context of warped product Einstein spaces. Using these local properties, we restate already known global characterizations of $(\lambda,m+n)$-Einstein manifolds from He, Petersen and Wylie.