Search

Training data attribution (TDA) methods aim to attribute model outputs back to specific training examples, and the application of these methods to large language model (LLM) outputs could significantly advance model transparency and data curation. However, it has been challenging to date to apply these methods to the full scale of LLM pretraining. In this paper, we refine existing gradient-based methods to work effectively at scale, allowing us to retrieve influential examples for an 8B-parameter language model from a pretraining corpus of over 160B tokens with no need for subsampling or pre-filtering. Our method combines several techniques, including optimizer state correction, a task-specific Hessian approximation, and normalized encodings, which we find to be critical for performance at scale. In quantitative evaluations on a fact tracing task, our method performs best at identifying examples that influence model predictions, but classical, model-agnostic retrieval methods such as BM25 still perform better at finding passages which explicitly contain relevant facts. These results demonstrate a misalignment between factual attribution and causal influence. With increasing model size and training tokens, we find that influence more closely aligns with attribution. Finally, we examine different types of examples identified as influential by our method, finding that while many directly entail a particular fact, others support the same output by reinforcing priors on relation types, common entities, and names.

The first detection of the quasiperiodic ultrafast outflow in the ASASSN-20qc system was reported by Pasham et al. (2024). The outflow is revealed in the soft X-ray spectra as an absorption feature, which is periodically enhanced every $ \sim 8.3$ days. The periodic nature of the ultrafast outflow is best explained by an orbiting massive perturber, most likely an intermediate-mass black hole (IMBH), that is inclined with respect to the accretion flow around the primary supermassive black hole (SMBH). In this way, the orbiting body pushes the disc gas into the outflow funnel, where it is accelerated by the ordered magnetic field (Sukov\'a et al. 2021). Quasiperiodic ultrafast outflows (QPOuts) are thus a novel phenomenon that can help reveal new extreme-/intemediate-mass ratio inspiral (EMRI/IMRI) candidates that otherwise do not exhibit significant periodic patterns in the continuum flux density., Comment: 10 pages, 4 figures, submitted to the Proceedings of the IAU; based on the talk presented at IAUS 389 (XXXII IAU GA) in Cape Town, South Africa

The Wide Field Monitor (WFM) is one of the three instruments on the Spectroscopic Time-Resolving Observatory for Broadband Energy X-rays (STROBE-X) mission, which was proposed in response to the NASA 2023 call for a probe class mission. The WFM is a coded-mask camera system that would be the most scientifically capable wide-angle monitor ever flown. The field of view covers one third of the sky, to 50 percent mask coding, and the energy sensitivity is 2 to 50 keV. The WFM is designed to identify new X-ray transients and to capture spectral and timing changes in known sources with data of unprecedented quality. Science applications cover diverse classes, in including X-ray bursts that coincide with gravitational wave detections, gamma ray bursts and their transition from prompt emission to afterglow, subluminous GRBs that may signal shock breakout in supernovae, state transitions in accreting compact objects and their jets, bright flares in fast X-ray transients, accretion onset in transitional pulsars, and coronal flares from many types of active stars., Comment: 19 pages, 5 figures, accepted for publication in JATIS

We consider the discrete time infinite horizon average reward restless markovian bandit (RMAB) problem. We propose a \emph{model predictive control} based non-stationary policy with a rolling computational horizon $\tau$. At each time-slot, this policy solves a $\tau$ horizon linear program whose first control value is kept as a control for the RMAB. Our solution requires minimal assumptions and quantifies the loss in optimality in terms of $\tau$ and the number of arms, $N$. We show that its sub-optimality gap is $O(1/\sqrt{N})$ in general, and $\exp(-\Omega(N))$ under a local-stability condition. Our proof is based on a framework from dynamic control known as \emph{dissipativity}. Our solution easy to implement and performs very well in practice when compared to the state of the art. Further, both our solution and our proof methodology can easily be generalized to more general constrained MDP settings and should thus, be of great interest to the burgeoning RMAB community., Comment: Under review

The Ingot WFS was designed to overcome some of the challenges present in classical wavefront sensors when they deal with sodium LGSs. This innovative sensor works by sensing the full 3D volume of the elongated LGS and is suitable for use in very large telescopes. A test bench has been assembled at the INAF - Osservatorio Astronomico di Padova laboratories to test and characterize the functioning of the Ingot WFS. In this work, we summarize the main results of the tests performed on a new search algorithm. Then, we move towards a more accurate simulation of the sodium LGS by replicating real time-varying sodium layer profiles. The study of their impact on the ingot pupil signals is described in this work., Comment: 17 pages, 17 figures, Proceedings of SPIE 2024

Many giant stars are magnetically active, which causes rotational variability, chromospheric emission lines, and X-ray emission. Large outbursts in these emission features can set limits on the magnetic field strength and thus constrain the mechanism of the underlying dynamo. HD~251108 is a Li-rich active K-type giant. We find a rotational period of 21.3~d with color changes and additional long-term photometric variability. Both can be explained with very stable stellar spots. We followed the decay phase of a superflare for 28 days with NICER and from the ground. We track the flare decay in unprecedented detail in several coronal temperature components. With a peak flux around $10^{34}$~erg~s$^{-1}$ (0.5-4.0~keV) and an exponential decay time of 2.2~days in the early decay phase, this is one of the strongest flares ever observed; yet it follows trends established from samples of smaller flares, for example for the relations between H$\alpha$ and X-ray flux, indicating that the physical process that powers the flare emission is consistent over a large range of flare energies. We estimate a flare loop length about 2-4 times the stellar radius. No evidence is seen for abundance changes during the flare., Comment: submitted to ApJ, one electronic figures and data will be available with the journal publication. The version on arXiv contains a static image of that figure

We consider self-organization and memory formation of a sheared amorphous solid subject to a random shear strain protocol confined to a strain range $\pm \varepsilon_{\rm max}$. We show that, as in the case of applied cyclic shear, the response of the driven system retains a memory of the strain range that can subsequently be retrieved. Our findings suggest that self-organization and memory formation of disordered materials can emerge under more general conditions, such as a system interacting with its fluctuating environment., Comment: 8 pages, 3 figures, including Appendix

We examine various possibilities for the pairing mechanisms in the recently discovered bilayer-nickelate superconductor within the Bardeen-Cooper-Schrieffer framework. Unlike earlier studies, where only a pure $d$-wave or sign-changing $s$-wave superconductivity instability was investigated, our study explores the possibilities of mixed-state superconducting instability such as the one involving both $d$- and sign-changing $s$-waves. While assuming that the superconductivity arises because of the magnetic correlations, we examine the nature of the superconducting gap function associated density of states with various possible magnetic correlation wavevectors arising out as a result of multiple pockets owing to the multiple orbitals and bilayer splitting. We also explore the effect of differences in the nature of Fermi surfaces suggested by various studies., Comment: 6 pages, 11 figures

The mechanical behavior of disordered materials such as dense suspensions, glasses or granular materials depends on their thermal and mechanical past. Here we report the memory behavior of a quenched mesoscopic elasto-plastic (QMEP) model. After prior oscillatory training, a simple read-out protocol gives access to both the training protocol's amplitude and the last shear direction. The memory of direction emerges from the development of a mechanical polarization during training. The analysis of sample-to-sample fluctuations gives direct access to the irreversibility transition. Despite the quadrupolar nature of the elastic interactions in amorphous solids, a behavior close to Return Point Memory (RPM) is observed. The quasi RPM property is used to build a simple Preisach-like model of directional memory., Comment: 13 pages, 6 figures + SI with 4 figures

Without adhering to any specific model, we have presented 4 X 4 quark mixing matrix as an extension of the 3 X 3 PDG parametrization of the CKM matrix. Using unitarity constraints as well as the hierarchy among the elements of the 3 X 3 CKM matrix, we have found the hierarchy among the 4th row and 4th column elements of the 4 X 4 quark mixing matrix. Further, for the fourth generation case, we have explicitly found the 9 independent rephasing invariant parameters J_4X4. Also, using phenomenological estimates of the 4th row and 4th column elements, we have numerically evaluated these 9 parameters., Comment: 16 pages

We investigate the low rank matrix completion problem in an online setting with ${M}$ users, ${N}$ items, ${T}$ rounds, and an unknown rank-$r$ reward matrix ${R}\in \mathbb{R}^{{M}\times {N}}$. This problem has been well-studied in the literature and has several applications in practice. In each round, we recommend ${S}$ carefully chosen distinct items to every user and observe noisy rewards. In the regime where ${M},{N} >> {T}$, we propose two distinct computationally efficient algorithms for recommending items to users and analyze them under the benign \emph{hott items} assumption.1) First, for ${S}=1$, under additional incoherence/smoothness assumptions on ${R}$, we propose the phased algorithm \textsc{PhasedClusterElim}. Our algorithm obtains a near-optimal per-user regret of $\tilde{O}({N}{M}^{-1}(\Delta^{-1}+\Delta_{{hott}}^{-2}))$ where $\Delta_{{hott}},\Delta$ are problem-dependent gap parameters with $\Delta_{{hott}} >> \Delta$ almost always. 2) Second, we consider a simplified setting with ${S}=r$ where we make significantly milder assumptions on ${R}$. Here, we introduce another phased algorithm, \textsc{DeterminantElim}, to derive a regret guarantee of $\widetilde{O}({N}{M}^{-1/r}\Delta_{det}^{-1}))$ where $\Delta_{{det}}$ is another problem-dependent gap. Both algorithms crucially use collaboration among users to jointly eliminate sub-optimal items for groups of users successively in phases, but with distinctive and novel approaches., Comment: Appeared at the Forty-first International Conference on Machine Learning, 2024

Applying Reinforcement Learning (RL) to Restless Multi-Arm Bandits (RMABs) offers a promising avenue for addressing allocation problems with resource constraints and temporal dynamics. However, classic RMAB models largely overlook the challenges of (systematic) data errors - a common occurrence in real-world scenarios due to factors like varying data collection protocols and intentional noise for differential privacy. We demonstrate that conventional RL algorithms used to train RMABs can struggle to perform well in such settings. To solve this problem, we propose the first communication learning approach in RMABs, where we study which arms, when involved in communication, are most effective in mitigating the influence of such systematic data errors. In our setup, the arms receive Q-function parameters from similar arms as messages to guide behavioral policies, steering Q-function updates. We learn communication strategies by considering the joint utility of messages across all pairs of arms and using a Q-network architecture that decomposes the joint utility. Both theoretical and empirical evidence validate the effectiveness of our method in significantly improving RMAB performance across diverse problems.

The hierarchy amongst the CKM matrix elements, highlighted recently by Luo and Xing, has been rigorously revisited using the PDG parameterization incorporating unitarity constraints. Further, we have explored the evaluation of the CP violating parameter \epsilon_k for the 9 possible unitarity ensured equivalent independent parametrizations of CKM matrix. Interestingly, we find that not all of these reproduce the value of parameter \epsilon_k, this being presumably due to the hierarchical nature of the CKM matrix elements. This situation is echoing similar conclusions regarding the evaluation of Jarlskog's rephasing invariant parameter J through unitarity ensured equivalent possibilities., Comment: 13 pages. arXiv admin note: text overlap with arXiv:2310.11152

Self-consistency (Wang et al., 2023) suggests that the most consistent answer obtained through large language models (LLMs) is more likely to be correct. In this paper, we challenge this argument and propose a nuanced correction. Our observations indicate that consistent answers derived through more computation i.e. longer reasoning texts, rather than simply the most consistent answer across all outputs, are more likely to be correct. This is predominantly because we demonstrate that LLMs can autonomously produce chain-of-thought (CoT) style reasoning with no custom prompts merely while generating longer responses, which lead to consistent predictions that are more accurate. In the zero-shot setting, by sampling Mixtral-8x7B model multiple times and considering longer responses, we achieve 86% of its self-consistency performance obtained through zero-shot CoT prompting on the GSM8K and MultiArith datasets. Finally, we demonstrate that the probability of LLMs generating a longer response is quite low, highlighting the need for decoding strategies conditioned on output length.

We address the challenge of online convex optimization where the objective function's gradient exhibits sparsity, indicating that only a small number of dimensions possess non-zero gradients. Our aim is to leverage this sparsity to obtain useful estimates of the objective function's gradient even when the only information available is a limited number of function samples. Our motivation stems from distributed queueing systems like microservices-based applications, characterized by request-response workloads. Here, each request type proceeds through a sequence of microservices to produce a response, and the resource allocation across the collection of microservices is controlled to balance end-to-end latency with resource costs. While the number of microservices is substantial, the latency function primarily reacts to resource changes in a few, rendering the gradient sparse. Our proposed method, CONGO (Compressive Online Gradient Optimization), combines simultaneous perturbation with compressive sensing to estimate gradients. We establish analytical bounds on the requisite number of compressive sensing samples per iteration to maintain bounded bias of gradient estimates, ensuring sub-linear regret. By exploiting sparsity, we reduce the samples required per iteration to match the gradient's sparsity, rather than the problem's original dimensionality. Numerical experiments and real-world microservices benchmarks demonstrate CONGO's superiority over multiple stochastic gradient descent approaches, as it quickly converges to performance comparable to policies pre-trained with workload awareness., Comment: 28 pages, 7 figures

In this study, we explore the accelerated expansion of the universe within the framework of modified $f(Q)$ gravity. The investigation focus on the role of bulk viscosity in understanding the universe's accelerated expansion. Specifically, a bulk viscous matter-dominated cosmological model is considered, with the bulk viscosity coefficient expressed as $\zeta= \zeta_0 \rho H^{-1} + \zeta_1 H $. We consider the power law $f(Q)$ function $f(Q)=\alpha Q^n $, where $\alpha$ and $n$ are arbitrary constants and derive the analytical solutions for the field equations corresponding to a flat FLRW metric. Subsequently, we used the combined Cosmic Chronometers (CC)+Pantheon+SH0ES sample to estimate the free parameters of the obtained analytic solution. We conduct Bayesian statistical analysis to estimate the posterior probability by employing the likelihood function and the MCMC random sampling technique, along with the AIC and BIC statistical assessment criteria. In addition, we explore the evolutionary behavior of significant cosmological parameters. The effective equation of state (EOS) parameter predicts the accelerating behavior of the cosmic expansion phase. Further, by the statefinder and $Om(z)$ diagnostic test, we found that our viscous model favors quintessence-type behavior and can successfully describe the late-time scenario., Comment: General Relativity and Gravitation published version

In this expository note, we explore the possibility of the existence of Kirby move of type 1 for contact surgery diagrams. In particular, we give the necessary conditions on a contact surgery diagram to become a potential candidate for contact Kirby move of type 1. We observe that there is a collection of contact positive integral surgery diagrams on Legendrian unknots satisfying those conditions., Comment: 10 pages, 8 figures

An image editing model should be able to perform diverse edits, ranging from object replacement, changing attributes or style, to performing actions or movement, which require many forms of reasoning. Current general instruction-guided editing models have significant shortcomings with action and reasoning-centric edits. Object, attribute or stylistic changes can be learned from visually static datasets. On the other hand, high-quality data for action and reasoning-centric edits is scarce and has to come from entirely different sources that cover e.g. physical dynamics, temporality and spatial reasoning. To this end, we meticulously curate the AURORA Dataset (Action-Reasoning-Object-Attribute), a collection of high-quality training data, human-annotated and curated from videos and simulation engines. We focus on a key aspect of quality training data: triplets (source image, prompt, target image) contain a single meaningful visual change described by the prompt, i.e., truly minimal changes between source and target images. To demonstrate the value of our dataset, we evaluate an AURORA-finetuned model on a new expert-curated benchmark (AURORA-Bench) covering 8 diverse editing tasks. Our model significantly outperforms previous editing models as judged by human raters. For automatic evaluations, we find important flaws in previous metrics and caution their use for semantically hard editing tasks. Instead, we propose a new automatic metric that focuses on discriminative understanding. We hope that our efforts : (1) curating a quality training dataset and an evaluation benchmark, (2) developing critical evaluations, and (3) releasing a state-of-the-art model, will fuel further progress on general image editing., Comment: NeurIPS 2024 (Dataset & Benchmarks)

Large language models (LLMs) have recently shown tremendous promise in serving as the backbone to agentic systems, as demonstrated by their performance in multi-faceted, challenging benchmarks like SWE-Bench and Agent-Bench. However, to realize the true potential of LLMs as autonomous agents, they must learn to identify, call, and interact with external tools and application program interfaces (APIs) to complete complex tasks. These tasks together are termed function calling. Endowing LLMs with function calling abilities leads to a myriad of advantages, such as access to current and domain-specific information in databases and knowledge sources, and the ability to outsource tasks that can be reliably performed by tools, e.g., a Python interpreter or calculator. While there has been significant progress in function calling with LLMs, there is still a dearth of open models that perform on par with proprietary LLMs like GPT, Claude, and Gemini. Therefore, in this work, we introduce the GRANITE-20B-FUNCTIONCALLING model under an Apache 2.0 license. The model is trained using a multi-task training approach on seven fundamental tasks encompassed in function calling, those being Nested Function Calling, Function Chaining, Parallel Functions, Function Name Detection, Parameter-Value Pair Detection, Next-Best Function, and Response Generation. We present a comprehensive evaluation on multiple out-of-domain datasets comparing GRANITE-20B-FUNCTIONCALLING to more than 15 other best proprietary and open models. GRANITE-20B-FUNCTIONCALLING provides the best performance among all open models on the Berkeley Function Calling Leaderboard and fourth overall. As a result of the diverse tasks and datasets used for training our model, we show that GRANITE-20B-FUNCTIONCALLING has better generalizability on multiple tasks in seven different evaluation datasets.

The tidal disruption event (TDE) AT2018dyk/ASASSN-18UL showed a rapid dimming event 500 days after discovery, followed by a re-brightening roughly 700 days later. It has been hypothesized that this behavior results from a repeating partial TDE (rpTDE), such that prompt dimmings/shutoffs are coincident with the return of the star to pericenter and rebrightenings generated by the renewed supply of tidally stripped debris. This model predicted that the emission should shut off again around August of 2023. We report AT2018fyk's continued X-ray and UV monitoring, which shows an X-ray (UV) drop in flux by a factor of 10 (5) over a span of two months, starting 14 Aug 2023. This sudden change can be interpreted as the second emission shutoff, which 1) strengthens the rpTDE scenario for AT2018fyk, 2) allows us to constrain the orbital period to a more precise value of 1306$\pm$47 days, and 3) establishes that X-ray and UV/optical emission track the fallback rate onto this SMBH -- an often-made assumption that otherwise lacks observational verification -- and therefore the UV/optical lightcurve is powered predominantly by processes tied to X-rays. The second cutoff implies that another rebrightening should happen between May-Aug 2025, and if the star survived the second encounter, a third shutoff is predicted to occur between Jan-July 2027. Finally, low-level accretion from the less bound debris tail (which is completely unbound/does not contribute to accretion in a non-repeating TDE) can result in a faint X-ray plateau that could be detectable until the next rebrightening., Comment: To be published in ApJ Letters

Automating the annotation of scanned documents is challenging, requiring a balance between computational efficiency and accuracy. DocParseNet addresses this by combining deep learning and multi-modal learning to process both text and visual data. This model goes beyond traditional OCR and semantic segmentation, capturing the interplay between text and images to preserve contextual nuances in complex document structures. Our evaluations show that DocParseNet significantly outperforms conventional models, achieving mIoU scores of 49.12 on validation and 49.78 on the test set. This reflects a 58% accuracy improvement over state-of-the-art baseline models and an 18% gain compared to the UNext baseline. Remarkably, DocParseNet achieves these results with only 2.8 million parameters, reducing the model size by approximately 25 times and speeding up training by 5 times compared to other models. These metrics, coupled with a computational efficiency of 0.039 TFLOPs (BS=1), highlight DocParseNet's high performance in document annotation. The model's adaptability and scalability make it well-suited for real-world corporate document processing applications. The code is available at https://github.com/ahmad-shirazi/DocParseNet

Novel dataset creation for all multi-object tracking, crowd-counting, and industrial-based videos is arduous and time-consuming when faced with a unique class that densely populates a video sequence. We propose a time efficient method called POPCat that exploits the multi-target and temporal features of video data to produce a semi-supervised pipeline for segmentation or box-based video annotation. The method retains the accuracy level associated with human level annotation while generating a large volume of semi-supervised annotations for greater generalization. The method capitalizes on temporal features through the use of a particle tracker to expand the domain of human-provided target points. This is done through the use of a particle tracker to reassociate the initial points to a set of images that follow the labeled frame. A YOLO model is then trained with this generated data, and then rapidly infers on the target video. Evaluations are conducted on GMOT-40, AnimalTrack, and Visdrone-2019 benchmarks. These multi-target video tracking/detection sets contain multiple similar-looking targets, camera movements, and other features that would commonly be seen in "wild" situations. We specifically choose these difficult datasets to demonstrate the efficacy of the pipeline and for comparison purposes. The method applied on GMOT-40, AnimalTrack, and Visdrone shows a margin of improvement on recall/mAP50/mAP over the best results by a value of 24.5%/9.6%/4.8%, -/43.1%/27.8%, and 7.5%/9.4%/7.5% where metrics were collected., Comment: 10 pages, 5 figures, Accepted in "Conference on Robots and Vision 2024"

We present an approach to build Large Language Model (LLM) based slot-filling system to perform Dialogue State Tracking in conversational assistants serving across a wide variety of industry-grade applications. Key requirements of this system include: 1) usage of smaller-sized models to meet low latency requirements and to enable convenient and cost-effective cloud and customer premise deployments, and 2) zero-shot capabilities to serve across a wide variety of domains, slot types and conversational scenarios. We adopt a fine-tuning approach where a pre-trained LLM is fine-tuned into a slot-filling model using task specific data. The fine-tuning data is prepared carefully to cover a wide variety of slot-filling task scenarios that the model is expected to face across various domains. We give details of the data preparation and model building process. We also give a detailed analysis of the results of our experimental evaluations. Results show that our prescribed approach for slot-filling model building has resulted in 6.9% relative improvement of F1 metric over the best baseline on a realistic benchmark, while at the same time reducing the latency by 57%. More over, the data we prepared has helped improve F1 on an average by 4.2% relative across various slot-types.

Some electromagnetic outbursts from the nuclei of distant galaxies have been found to repeat on months-to-years timescales, and each of these sources can putatively arise from the accretion flares generated through the repeated tidal stripping of a star on a bound orbit about a supermassive black hole (SMBH), i.e., a repeating partial tidal disruption event (rpTDE). Here we test the rpTDE model through analytical estimates and hydrodynamical simulations of the interaction between a range of stars, which differ from one another in mass and age, and an SMBH. We show that higher-mass ($\gtrsim 1 M_{\odot}$), evolved stars can survive many ($\gtrsim 10-100$) encounters with an SMBH while simultaneously losing $few \times 0.01 M_{\odot}$, resulting in accretion flares that are approximately evenly spaced in time with nearly the same amplitude, quantitatively reproducing ASASSN-14ko. We also show that the energy imparted to the star via tides can lead to a change in its orbital period that is comparable to the observed decay in the recurrence time of ASASSN-14ko's flares, $\dot{P}\simeq-0.0026$. Contrarily, lower-mass and less-evolved stars lose progressively more mass and produce brighter accretion flares on subsequent encounters for the same pericenter distances, leading to the rapid destruction of the star and cessation of flares. Such systems cannot reproduce ASASSN-14ko-like transients, but are promising candidates for recreating events such as AT2020vdq, which displayed a second and much brighter outburst compared to the first. Our results imply that the lightcurves of repeating transients are tightly coupled with stellar type., Comment: 24 pages, 18 figures, 2 appendices, Accepted to ApJ on July 31, 2024

Langevin Dynamics is a Stochastic Differential Equation (SDE) central to sampling and generative modeling and is implemented via time discretization. Langevin Monte Carlo (LMC), based on the Euler-Maruyama discretization, is the simplest and most studied algorithm. LMC can suffer from slow convergence - requiring a large number of steps of small step-size to obtain good quality samples. This becomes stark in the case of diffusion models where a large number of steps gives the best samples, but the quality degrades rapidly with smaller number of steps. Randomized Midpoint Method has been recently proposed as a better discretization of Langevin dynamics for sampling from strongly log-concave distributions. However, important applications such as diffusion models involve non-log concave densities and contain time varying drift. We propose its variant, the Poisson Midpoint Method, which approximates a small step-size LMC with large step-sizes. We prove that this can obtain a quadratic speed up of LMC under very weak assumptions. We apply our method to diffusion models for image generation and show that it maintains the quality of DDPM with 1000 neural network calls with just 50-80 neural network calls and outperforms ODE based methods with similar compute., Comment: "One often meets his destiny on the road he takes to avoid it" - Master Oogway. My destiny seems to be to write triangle inequalities for the rest of my life

We introduce the Glauber Generative Model (GGM), a new class of discrete diffusion models, to obtain new samples from a distribution given samples from a discrete space. GGM deploys a discrete Markov chain called the heat bath dynamics (or the Glauber dynamics) to denoise a sequence of noisy tokens to a sample from a joint distribution of discrete tokens. Our novel conceptual framework provides an exact reduction of the task of learning the denoising Markov chain to solving a class of binary classification tasks. More specifically, the model learns to classify a given token in a noisy sequence as signal or noise. In contrast, prior works on discrete diffusion models either solve regression problems to learn importance ratios, or minimize loss functions given by variational approximations. We apply GGM to language modeling and image generation, where images are discretized using image tokenizers like VQGANs. We show that it outperforms existing discrete diffusion models in language generation, and demonstrates strong performance for image generation without using dataset-specific image tokenizers. We also show that our model is capable of performing well in zero-shot control settings like text and image infilling.

We present the tidal disruption event (TDE) AT2022lri, hosted in a nearby ($\approx\!144$ Mpc) quiescent galaxy with a low-mass massive black hole ($10^4\,M_\odot < M_{\rm BH} < 10^6\,M_\odot$). AT2022lri belongs to the TDE-H+He subtype. More than 1 Ms of X-ray data were collected with NICER, Swift, and XMM-Newton from 187 d to 672 d after peak. The X-ray luminosity gradually declined from $1.5\times 10^{44}\,{\rm erg\,s^{-1}}$ to $1.5\times 10^{43}\,{\rm erg\,s^{-1}}$ and remains much above the UV and optical luminosity, consistent with a super-Eddington accretion flow viewed face-on. Sporadic strong X-ray dips atop a long-term decline are observed, with variability timescale of $\approx\!0.5$ hr--1 d and amplitude of $\approx\!2$--8. When fitted with simple continuum models, the X-ray spectrum is dominated by a thermal disk component with inner temperature going from $\sim\! 146$ eV to $\sim\! 86$ eV. However, there are residual features that peak around 1 keV, which, in some cases, cannot be reproduced by a single broad emission line. We analyzed a subset of time-resolved spectra with two physically motivated models describing either a scenario where ionized absorbers contribute extra absorption and emission lines or where disk reflection plays an important role. Both models provide good and statistically comparable fits, show that the X-ray dips are correlated with drops in the inner disk temperature, and require the existence of sub-relativistic (0.1--0.3$c$) ionized outflows. We propose that the disk temperature fluctuation stems from episodic drops of the mass accretion rate triggered by magnetic instabilities or/and wobbling of the inner accretion disk along the black hole's spin axis., Comment: 35 pages, 20 figures, submitted

Autonomous driving systems require a quick and robust perception of the nearby environment to carry out their routines effectively. With the aim to avoid collisions and drive safely, autonomous driving systems rely heavily on object detection. However, 2D object detections alone are insufficient; more information, such as relative velocity and distance, is required for safer planning. Monocular 3D object detectors try to solve this problem by directly predicting 3D bounding boxes and object velocities given a camera image. Recent research estimates time-to-contact in a per-pixel manner and suggests that it is more effective measure than velocity and depth combined. However, per-pixel time-to-contact requires object detection to serve its purpose effectively and hence increases overall computational requirements as two different models need to run. To address this issue, we propose per-object time-to-contact estimation by extending object detection models to additionally predict the time-to-contact attribute for each object. We compare our proposed approach with existing time-to-contact methods and provide benchmarking results on well-known datasets. Our proposed approach achieves higher precision compared to prior art while using a single image., Comment: 9 pages, 4 figures

This study focuses on discerning the role of structural parameters on the bifurcation characteristics and the underlying synchronization mechanism in an aeroelastic system undergoing nonlinear stall behaviour. To that end, wind tunnel experiments are performed on a NACA 0012 airfoil capable of undergoing bending (plunging) and torsional (pitching) oscillations under scenarios involving nonlinear aerodynamic loads, i.e., dynamic stall conditions. Flow conditions under both deterministic/sterile flows and fluctuating/stochastic flows are fostered. The structure possesses continuous or polynomial-type stiffness nonlinearities, and therefore, is an aeroelastic experiment involving both structural and aerodynamic nonlinearities. We discern the bifurcation routes for a range of key structural parameters such as frequency ratio, static imbalance, and the extent of structural nonlinearity. In addition to interesting and atypical routes to stall-induced instabilities, we systematically demonstrate the role of modal interactions - via a synchronization analysis - over the manifestation of these instabilities. To the best of the authors' knowledge, this is perhaps the first study to document the role of multiple structural parameters on a stall-induced aeroelastic system, and in turn, cast the physical mechanism behind these dynamical transitions from the vantage of synchronization.

Uncertainty estimation is a significant issue for current large language models (LLMs) that are generally poorly calibrated and over-confident, especially with reinforcement learning from human feedback (RLHF). Unlike humans, whose decisions and confidences not only stem from intrinsic beliefs but can also be adjusted through daily observations, existing calibration methods for LLMs focus on estimating or eliciting individual confidence without taking full advantage of the "Collective Wisdom": the interaction among multiple LLMs that can collectively improve both accuracy and calibration. In this work, we propose Collaborative Calibration, a post-hoc training-free calibration strategy that leverages the collaborative and expressive capabilities of multiple tool-augmented LLM agents in a simulated group deliberation process. We demonstrate the effectiveness of Collaborative Calibration on generative QA tasks across various domains, showing its potential in harnessing the rationalization of collectively calibrated confidence assessments and improving the reliability of model predictions., Comment: Accepted at ICLR 2024 Workshop on Reliable and Responsible Foundation Models

Extreme-mass-ratio inspirals (EMRIs) are one of the primary targets for the recently adopted millihertz gravitational-wave (GW) observatory LISA. Some previous studies have argued that a fraction of all EMRIs form in matter-rich environments, and can potentially explain the dozens of soft X-ray band ($\sim 10^{-1} \rm keV$), low-frequency ($\sim 0.1$ mHz) periodic phenomena known as quasi-periodic eruptions (QPEs) and quasi-periodic oscillations (QPOs). Here, using a representative EMRI population retrofitted with cutoffs on LISA-band SNRs and luminosity distances to account for the sensitivity of current instruments, we estimate the mean frequency band in which QPEs and QPOs originating from detectable LISA EMRIs may be emitting an X-ray signal ``today'' (i.e., in 2024) to be $0.46 \pm 0.22$ mHz. We also model the well-known QPO source, RE J1034+396, which falls in this frequency band, as an EMRI assuming its primary black hole mass to be $10^6-10^7 M_\odot$. Through a prior-predictive analysis, we estimate the orbiting compact object's mass to be $46^{+ 10}_{-40} M_\odot$ and the source's LISA-band SNR as $\approx 14$, highlighting it as a candidate multi-messenger EMRI target. We also highlight the role of current and near-future X-ray and UV observatories in enabling multi-messenger observations of EMRIs in conjunction with LISA, and conclude with a discussion of caveats of the current analysis, such as the exclusion of eccentricity and inclination from the model, and the measurability of sub-solar mass compact object EMRIs., Comment: (Before-proofs-accepted) 15 + 1 pages, 10 + 1 figures

The application of natural language processing models to PDF documents is pivotal for various business applications yet the challenge of training models for this purpose persists in businesses due to specific hurdles. These include the complexity of working with PDF formats that necessitate parsing text and layout information for curating training data and the lack of privacy-preserving annotation tools. This paper introduces DOCMASTER, a unified platform designed for annotating PDF documents, model training, and inference, tailored to document question-answering. The annotation interface enables users to input questions and highlight text spans within the PDF file as answers, saving layout information and text spans accordingly. Furthermore, DOCMASTER supports both state-of-the-art layout-aware and text models for comprehensive training purposes. Importantly, as annotations, training, and inference occur on-device, it also safeguards privacy. The platform has been instrumental in driving several research prototypes concerning document analysis such as the AI assistant utilized by University of California San Diego's (UCSD) International Services and Engagement Office (ISEO) for processing a substantial volume of PDF documents.

It is believed that the core of a neutron star can be host to various novel phases of matter, from nucleon superfluid phase to exotic high baryon density quantum chromodynamics (QCD) phases. Different observational signals for such phase transitions have been discussed in the literature. Here, we point out a unique phenomenon associated with phase transition to a superfluid phase, which may be the nucleon superfluid phase or a phase like the CFL phase, allowing for superfluid vortices. In any superfluid phase transition, a random network of vortices forms via the so-called Kibble-Zurek mechanism, which eventually mostly decays away, finally leaving primarily vortices arising from the initial angular momentum of the core. This transient, random vortex network can have a non-zero net angular momentum for the superfluid component, which will generally be oriented in an arbitrary direction. This is in contrast to the final vortices, which arise from initial rotation and hence have the initial angular momentum of the neutron star. The angular momentum of the random vortex network is balanced by an equal and opposite angular momentum in the normal fluid due to the conservation of angular momentum, thereby imparting an arbitrarily oriented angular momentum component to the outer shell of the neutron star. This will affect the pulse timing and pulse profile of a pulsar. These changes in the pulses will decay away in a characteristic manner such that the random vortex network decays, obeying specific scaling laws leading to universal features for the detection of superfluid transitions occurring in a pulsar core., Comment: 9 pages, no figures

In this article, we shall discuss the fermionic oscillators in the context of thermodynamic and statistical framework. Further, we shall also provide a generalization and physical realization of Schwinger algebra for a system of pair of oscillators of general types, i.e. bosonic-bosonic, fermionic-fermionic, bosonic-fermionic. We shall also discuss a few applications of the results obtained., Comment: 7 pages, 0 figures

A major limitation of the weakly compressible approaches to simulate incompressible flows is the appearance of artificial acoustic waves that introduce a large mass conservation error and lead to spurious oscillations in the force coefficients. In this work, we propose a non-homogeneous anisotropic bulk viscosity term to effectively damp the acoustic waves. By implementing this term in a computational framework based on the recently proposed general pressure equation, we demonstrate that the non-homogeneous and anisotropic nature of the term makes it significantly more effective than the isotropic homogeneous version widely used in the literature. Moreover, it is computationally more efficient than the pressure (or mass) diffusion term, which is an alternative mechanism used to suppress acoustic waves. We simulate a range of benchmark problems to comprehensively investigate the performance of the bulk viscosity on the effective suppression of acoustic waves, mass conservation error, order of convergence of the solver, and computational efficiency. The proposed form of the bulk viscosity enables fairly accurate modelling of the initial transients of unsteady simulations, which is highly challenging for weakly compressible approaches, and to the best of our knowledge, existing approaches can't provide an accurate prediction of such transients.