Search

We study infinite-horizon average-reward reinforcement learning (RL) for Lipschitz MDPs and develop an algorithm ZoRL that discretizes the state-action space adaptively and zooms into promising regions of the state-action space. We show that its regret can be bounded as $\mathcal{\tilde{O}}\big(T^{1 - d_{\text{eff.}}^{-1}}\big)$, where $d_{\text{eff.}} = 2d_\mathcal{S} + d_z + 3$, $d_\mathcal{S}$ is the dimension of the state space, and $d_z$ is the zooming dimension. $d_z$ is a problem-dependent quantity, which allows us to conclude that if MDP is benign, then its regret will be small. We note that the existing notion of zooming dimension for average reward RL is defined in terms of policy coverings, and hence it can be huge when the policy class is rich even though the underlying MDP is simple, so that the regret upper bound is nearly $O(T)$. The zooming dimension proposed in the current work is bounded above by $d$, the dimension of the state-action space, and hence is truly adaptive, i.e., shows how to capture adaptivity gains for infinite-horizon average-reward RL. ZoRL outperforms other state-of-the-art algorithms in experiments; thereby demonstrating the gains arising due to adaptivity.

Let $Hom^0(\Gamma,G)$ be the path-connected component of the identity representation of the variety of representations of a finitely generated nilpotent group $\Gamma$ into a connected reductive complex affine algebraic group $G$. With the formulas given by Florentino, Lawton and Silva, we provide explicit partition type formulas for the mixed Hodge polynomials of character varieties $Hom^0(\Gamma,G)// G$ when $G=Sp_{2n}$ and $G=SO_{n}$., Comment: A mistake in the previous version of the article is corrected (see Fact 2.2) and this version generalizes to finitely generated nilpotent groups of positive abelian rank

The addition of polymers fundamentally alters the dynamics of turbulent flows in a way that defies Kolmogorov predictions. However, we now present a formalism that reconciles our understanding of polymeric turbulence with the classical Kolmogorov phenomenology. This is achieved by relying on an appropriate form of the K\'{a}rm\'{a}n-Howarth-Monin-Hill relation, which motivates the definition of extended velocity increments and the associated structure functions, by accounting for the influence of the polymers on the flow. We show, both analytically and numerically, that the ${\rm p}$th-order extended structure functions exhibit a power-law behaviour in the elasto-inertial range of scales, with exponents deviating from the analytically predicted value of ${\rm p}/3$. These deviations are readily accounted for by considering local averages of the total dissipation, rather than global averages, in analogy with the refined similarity hypotheses of Kolmogorov for classical Newtonian turbulence. We also demonstrate the scale-invariance of multiplier statistics of extended velocity increments, whose distributions collapse well for a wide range of scales.

In this work, we explore the decoding of mental imagery from subjects using their fMRI measurements. In order to achieve this decoding, we first created a mapping between a subject's fMRI signals elicited by the videos the subjects watched. This mapping associates the high dimensional fMRI activation states with visual imagery. Next, we prompted the subjects textually, primarily with emotion labels which had no direct reference to visual objects. Then to decode visual imagery that may have been in a person's mind's eye, we align a latent representation of these fMRI measurements with a corresponding video-fMRI based on textual labels given to the videos themselves. This alignment has the effect of overlapping the video fMRI embedding with the text-prompted fMRI embedding, thus allowing us to use our fMRI-to-video mapping to decode. Additionally, we enhance an existing fMRI dataset, initially consisting of data from five subjects, by including recordings from three more subjects gathered by our team. We demonstrate the efficacy of our model on this augmented dataset both in accurately creating a mapping, as well as in plausibly decoding mental imagery.

Neuroscience employs diverse neuroimaging techniques, each offering distinct insights into brain activity, from electrophysiological recordings such as EEG, which have high temporal resolution, to hemodynamic modalities such as fMRI, which have increased spatial precision. However, integrating these heterogeneous data sources remains a challenge, which limits a comprehensive understanding of brain function. We present the Spatiotemporal Alignment of Multimodal Brain Activity (SAMBA) framework, which bridges the spatial and temporal resolution gaps across modalities by learning a unified latent space free of modality-specific biases. SAMBA introduces a novel attention-based wavelet decomposition for spectral filtering of electrophysiological recordings, graph attention networks to model functional connectivity between functional brain units, and recurrent layers to capture temporal autocorrelations in brain signal. We show that the training of SAMBA, aside from achieving translation, also learns a rich representation of brain information processing. We showcase this classify external stimuli driving brain activity from the representation learned in hidden layers of SAMBA, paving the way for broad downstream applications in neuroscience research and clinical contexts.

The advent of transformer-based architectures and large language models (LLMs) have significantly advanced the performance of natural language processing (NLP) models. Since these LLMs are trained on huge corpuses of data from the web and other sources, there has been a major concern about harmful prejudices that may potentially be transferred from the data. In many applications, these pre-trained LLMs are fine-tuned on task specific datasets, which can further contribute to biases. This paper studies the extent of biases absorbed by LLMs during pre-training as well as task-specific behaviour after fine-tuning. We found that controlled interventions on pre-trained LLMs, prior to fine-tuning, have minimal effect on lowering biases in classifiers. However, the biases present in domain-specific datasets play a much bigger role, and hence mitigating them at this stage has a bigger impact. While pre-training does matter, but after the model has been pre-trained, even slight changes to co-occurrence rates in the fine-tuning dataset has a significant effect on the bias of the model., Comment: arXiv admin note: This work has been withdrawn by arXiv administrators due to inappropriate text reuse from external sources

Recent work in behavioral testing for natural language processing (NLP) models, such as Checklist, is inspired by related paradigms in software engineering testing. They allow evaluation of general linguistic capabilities and domain understanding, hence can help evaluate conceptual soundness and identify model weaknesses. However, a major challenge is the creation of test cases. The current packages rely on semi-automated approach using manual development which requires domain expertise and can be time consuming. This paper introduces an automated approach to develop test cases by exploiting the power of large language models and statistical techniques. It clusters the text representations to carefully construct meaningful groups and then apply prompting techniques to automatically generate Minimal Functionality Tests (MFT). The well-known Amazon Reviews corpus is used to demonstrate our approach. We analyze the behavioral test profiles across four different classification algorithms and discuss the limitations and strengths of those models.

Advances in engineering mesoscopic quantum devices have led to new material platforms where electronic transport can be achieved on foldable structures. In this respect, we study quantum phases and their transitions on a Kirigami structure, a Japanese craft form, where its individual building blocks are topologically non-trivial. In particular, we find that by mechanically deforming the Kirigami structure one can engineer topological phase transitions in the system. Using a multipronged approach, we show that the physics of the system can be captured within a transfer matrix formalism akin to Chalker-Coddington networks, where the junctions describe scattering between effective non-Hermitian one-dimensional edge channels. We further show that the nature of the Kirigami structure can affect the critical folding angles where the topological phase transitions occur. Our study shows the rich interplay between topological quantum phenomena and structural configuration of an underlying Kirigami network reflecting its potential as an intriguing platform., Comment: 6+5 pages, 4+8 figures

A principled approach to cyclicality and intransitivity in cardinal paired comparison data is developed within the framework of graphical linear models. Fundamental to our developments is a detailed understanding and study of the parameter space which accommodates cyclicality and intransitivity. In particular, the relationships between the reduced, completely transitive model, the full, not necessarily transitive model, and all manner of intermediate models are explored for both complete and incomplete paired comparison graphs. It is shown that identifying cyclicality and intransitivity reduces to a model selection problem and a new method for model selection employing geometrical insights, unique to the problem at hand, is proposed. The large sample properties of the estimators as well as guarantees on the selected model are provided. It is thus shown that in large samples all cyclicalities and intransitivities can be identified. The method is exemplified using simulations and the analysis of an illustrative example., Comment: 60 pages, 2 figures, 8 tables

The dynamics of fibres, modelled as a sequence of inertial beads linked via elastic springs, in turbulent flows is dictated by a non-trivial interplay of their inertia and elasticity. Such elastic, inertial fibres preferentially sample a three-dimensional turbulent flow in a manner qualitatively similar to that in two-dimensions [Singh et al., Phys. Rev. E 101, 053105 (2020)]. Inertia and elasticity have competing effects on fibre dynamics: Inertia drives fibres away from vortices while elasticity tends to trap them inside. However, both these effects are reversed at large values. A large inertia makes the fibres sample the flow more uniformly while a very large elasticity facilitates the sampling of straining regions. This complex sampling behaviour is further corroborated by quantifying the chaotic nature of sampled flow regions. This is achieved by evaluating the maximal Lagrangian Lyapunov Exponents associated with the flow along fibre trajectories.

This paper establishes an interesting connection between the family of CMC surfaces of revolution in $\mathbb E_1^3$ and some specific families of elliptic curves. As a consequence of this connection, we show in the class of spacelike CMC surfaces of revolution in the $\mathbb E_1^3$, only spacelike cylinders and standard hyperboloids are algebraic. We also show that a similar connection exists between CMC surfaces of revolution in $\mathbb E^3$ and elliptic curves. Further, we use this to reestablish the fact that the only CMC algebraic surfaces of revolution in $\mathbb E^3$ are spheres and right circular cylinders., Comment: 10 pages, typos are fixed. Comments are welcome

We study infinite-horizon average-reward reinforcement learning (RL) for Lipschitz MDPs and develop an algorithm PZRL that discretizes the state-action space adaptively and zooms in to promising regions of the "policy space" which seems to yield high average rewards. We show that the regret of PZRL can be bounded as $\tilde{\mathcal{O}}\big(T^{1 - d_{\text{eff.}}^{-1}}\big)$, where $d_{\text{eff.}}= 2d_\mathcal{S} + d^\Phi_z+2$, $d_\mathcal{S}$ is the dimension of the state space, and $d^\Phi_z$ is the zooming dimension. $d^\Phi_z$ is a problem-dependent quantity that depends not only on the underlying MDP but also the class of policies $\Phi$ used by the agent, which allows us to conclude that if the agent apriori knows that optimal policy belongs to a low-complexity class (that has small $d^\Phi_z$), then its regret will be small. The current work shows how to capture adaptivity gains for infinite-horizon average-reward RL in terms of $d^\Phi_z$. We note that the preexisting notions of zooming dimension are adept at handling only the episodic RL case since zooming dimension approaches covering dimension of state-action space as $T\to\infty$ and hence do not yield any possible adaptivity gains. Several experiments are conducted to evaluate the performance of PZRL. PZRL outperforms other state-of-the-art algorithms; this clearly demonstrates the gains arising due to adaptivity., Comment: 22 pages, 2 figures

Perturbing the arrangements of pegs on a static Galton board can result in non-trivial stationary distributions, which in the continuum limit correspond to departure from regular gaussian behavior. Two such distributions are obtained. Further, the distributions generated for a dynamic galton board under external forcing in a general direction are obtained by solution of the corresponding stochastic differential equations. Exact cumulant generating functions for the distribution are presented for forcing in one dimension. An approximate expression, correct to first order in the forcing amplitude, is presented for the case of two dimensions. Both cases show nontrivial departures from the static gaussian solution., Comment: 10 pages

Radio Frequency Knock Out resonant slow extraction is a standard method for extracting stored particle beams from synchrotrons by transverse excitation. Classically, the beam is excited with an RF field comprising a frequency band around one of the betatron sidebands. This article demonstrates that the third-integer resonance commonly used for the slow extraction induces nonlinear motion, resulting in the appearance of additional sidebands of higher order at multiples of the betatron tune. Measured and simulated beam spectra are presented, revealing these sidebands and the beam's response to being excited at first and second order sidebands. The feasibility of using a second order sideband for the purpose of slow extraction is demonstrated. This results in a significant improvement in the temporal structure (spill quality) of the extracted beam, but at the cost of higher excitation power requirements. This is observed both experimentally and in tracking simulations. The mechanism behind the observed improvement is explained using beam dynamics simulations.

The advent of Large Language Models (LLMs) has revolutionized various domains of artificial intelligence, including the realm of software engineering. In this research, we evaluate the efficacy of pre-trained LLMs in replicating the tasks traditionally performed by developers in response to code review comments. We provide code contexts to five popular LLMs and obtain the suggested code-changes (patch sets) derived from real-world code-review comments. The performance of each model is meticulously assessed by comparing their generated patch sets against the historical data of human-generated patch-sets from the same repositories. This comparative analysis aims to determine the accuracy, relevance, and depth of the LLMs' feedback, thereby evaluating their readiness to support developers in responding to code-review comments. Novelty: This particular research area is still immature requiring a substantial amount of studies yet to be done. No prior research has compared the performance of existing Large Language Models (LLMs) in code-review comments. This in-progress study assesses current LLMs in code review and paves the way for future advancements in automated code quality assurance, reducing context-switching overhead due to interruptions from code change requests., Comment: 2 pages

In this work, we consider a wireless networked control system (WNCS) consisting of a plant, a battery-operated sensor, a controller, and an actuator. The battery in the sensor harvests energy from the environment. The sensor then uses this energy for packet transmissions. There are two types of wireless communication channels, (i) sensor--controller channel (also called uplink channel), and (ii) controller--actuator channel (also called downlink channel). The controller is \emph{half-duplex}, and this prevents it from simultaneously receiving an update from the sensor, and also transmitting a control packet to the actuator. Though frequent transmissions via uplink channel improve controller's estimate of the plant state, but this also reduces the timely control of the plant. Hence, in order to strike a balance between these two, we consider the problem of designing an optimal scheduling policy that minimizes the expected cumulative infinite horizon discounted cost, where the instantaneous cost is equal to the square of the plant state. At each time $t$, the scheduler at the sensor has to decide whether it should activate the uplink channel, or downlink. We pose this dynamic optimization problem as a Markov decision process (MDP), in which the state at time $t$ is composed of (i) the plant state $x(t)$, (ii) the age of the data packet available at the controller, denoted by $\tau(t)$, (iii) a binary variable $y(t)$ which indicates the availability of a control packet at the controller, and (iv) the energy level of the battery at the sensor $b(t)$. We show that there exists an optimal scheduling policy that exhibits a threshold structure, meaning that for each time $t$, if there is a control packet available with the controller, then the sensor activates the downlink channel in case $|x(t)|$ exceeds a threshold $x\ust(\tau(t),b(t))$.

Instruction-level error injection analyses aim to find instructions where errors often lead to unacceptable outcomes like Silent Data Corruptions (SDCs). These analyses require significant time, which is especially problematic if developers wish to regularly analyze software that evolves over time. We present FastFlip, a combination of empirical error injection and symbolic SDC propagation analyses that enables fast, compositional error injection analysis of evolving programs. FastFlip calculates how SDCs propagate across program sections and correctly accounts for unexpected side effects that can occur due to errors. Using FastFlip, we analyze five benchmarks, plus two modified versions of each benchmark. FastFlip speeds up the analysis of incrementally modified programs by $3.2\times$ (geomean). FastFlip selects a set of instructions to protect against SDCs that minimizes the runtime cost of protection while protecting against a developer-specified target fraction of all SDC-causing errors.

We design scheduling policies that minimize a risk-sensitive cost criterion for a remote estimation setup. Since risk-sensitive cost objective takes into account not just the mean value of the cost, but also higher order moments of its probability distribution, the resulting policy is robust to changes in the underlying system's parameters. The setup consists of a sensor that observes a discrete-time autoregressive Markov process, and at each time $t$ decides whether or not to transmit its observations to a remote estimator using an unreliable wireless communication channel after encoding these observations into data packets. We model the communication channel as a Gilbert-Elliott channel \cite{10384144}. Sensor probes the channel \cite{laourine2010betting} and hence knows the channel state at each time $t$ before making scheduling decision. The scheduler has to minimize the expected value of the exponential of the finite horizon cumulative cost that is sum of the following two quantities (i) the cumulative transmission power consumed, (ii) the cumulative squared estimator error. We pose this dynamic optimization problem as a Markov decision process (MDP), in which the system state at time $t$ is composed of (i) the instantaneous error $\Delta(t):= x(t)-a\hat{x}(t-1)$, where $x(t),\hat{x}(t-1)$ are the system state and the estimate at time $t,t-1$ respectively, and (ii) the channel state $c(t)$. We show that there exists an optimal policy that has a threshold structure, i.e., at each time $t$, for each possible channel state $c$, there is a threshold $\D\ust(c)$ such that if the current channel state is $c$, then it transmits only when the error $\D(t)$ exceeds $\D\ust(c)$.