Search

A/B tests, also known as randomized controlled experiments (RCTs), are the gold standard for evaluating the impact of new policies, products, or decisions. However, these tests can be costly in terms of time and resources, potentially exposing users, customers, or other test subjects (units) to inferior options. This paper explores practical considerations in applying methodologies inspired by "synthetic control" as an alternative to traditional A/B testing in settings with very large numbers of units, involving up to hundreds of millions of units, which is common in modern applications such as e-commerce and ride-sharing platforms. This method is particularly valuable in settings where the treatment affects only a subset of units, leaving many units unaffected. In these scenarios, synthetic control methods leverage data from unaffected units to estimate counterfactual outcomes for treated units. After the treatment is implemented, these estimates can be compared to actual outcomes to measure the treatment effect. A key challenge in creating accurate counterfactual outcomes is interpolation bias, a well-documented phenomenon that occurs when control units differ significantly from treated units. To address this, we propose a two-phase approach: first using nearest neighbor matching based on unit covariates to select similar control units, then applying supervised learning methods suitable for high-dimensional data to estimate counterfactual outcomes. Testing using six large-scale experiments demonstrates that this approach successfully improves estimate accuracy. However, our analysis reveals that machine learning bias -- which arises from methods that trade off bias for variance reduction -- can impact results and affect conclusions about treatment effects. We document this bias in large-scale experimental settings and propose effective de-biasing techniques to address this challenge., Comment: 15 pages, 2 figures, 6 tables

Integrating power electronics with batteries can offer many advantages, including load sharing and balancing with parallel connectivity. However, parallel batteries with differing voltages and power profiles can cause large circulating currents and uncontrolled energy transfers, risking system instability. To overcome these challenges, we propose a novel modular reconfigurable topology for AC batteries with coupled inductors between adjacent submodules. This approach dynamically manages energy distribution between energy and power modules. Under normal conditions, energy modules operate in series or parallel configurations to deliver stable voltage and current, ensuring efficient power delivery. During heavy loads, such as during electric-vehicle acceleration, power modules seamlessly meet additional power demands beyond the energy modules' capacity. The use of mutual inductors reduces inductance along the load path for faster response times while providing sufficient inductance to regulate circulating currents and enable efficient energy transfer among modules. The system's AC load further complicates control, necessitating a simple yet effective feedforward-feedback control strategy to maintain satisfactory performance., Comment: 15 pages, 36 figures

Single-file diffusion (SFD) is a key mechanism underlying transport phenomena in confined physical and biological systems. In a typical SFD process, microscopic particles are restricted to moving in a narrow channel where they cannot pass one another, resulting in constrained motion and anomalous diffusion at long times. In this study, we use Brownian dynamics simulations and analytical theory to investigate the SFD of active Brownian particles (ABPs) -- a minimal model of active colloids. Using a combination of scaling relations and heuristic arguments, we derive an accurate analytical expression for a tagged ABP's mean square displacement (MSD). We find the MSD exhibits ballistic behavior at short times, which can quantitatively be related to the reduced kinetic temperature of the single-file ABP system. We also find that while the characteristic subdiffusive scaling of SFD $[\langle (\Delta x)^2 \rangle \sim t^{1/2}]$ is preserved at long times, self-propulsion introduces significant modifications to the 1D-mobility, which can be directly related to the constant P\'eclet (Pe) compressibility. Furthermore, we demonstrate that the generalized 1D-mobility, initially proposed by Kollmann for equilibrium systems [Phys. Rev. Lett. $\textbf{90}$, 180602 (2003)], can be extended to active systems with minimal modification. These findings have important implications for tuning particle transport at the microscale and provide a basis for understanding active matter in geometries with highly restricted motion., Comment: 12 page main text + 4 page SI. Comments Welcome!

Accurate estimation of treatment effects is essential for decision-making across various scientific fields. This task, however, becomes challenging in areas like social sciences and online marketplaces, where treating one experimental unit can influence outcomes for others through direct or indirect interactions. Such interference can lead to biased treatment effect estimates, particularly when the structure of these interactions is unknown. We address this challenge by introducing a new class of estimators based on causal message-passing, specifically designed for settings with pervasive, unknown interference. Our estimator draws on information from the sample mean and variance of unit outcomes and treatments over time, enabling efficient use of observed data to estimate the evolution of the system state. Concretely, we construct non-linear features from the moments of unit outcomes and treatments and then learn a function that maps these features to future mean and variance of unit outcomes. This allows for the estimation of the treatment effect over time. Extensive simulations across multiple domains, using synthetic and real network data, demonstrate the efficacy of our approach in estimating total treatment effect dynamics, even in cases where interference exhibits non-monotonic behavior in the probability of treatment.

In this paper, we investigate some properties of symbolic powers and symbolic Rees algebras of binomial edge ideals associated with some classes of block graphs. First, it is shown that symbolic powers of binomial edge ideals of pendant cliques graphs coincide with the ordinary powers. Furthermore, we see that binomial edge ideals of a generalization of these graphs are symbolic $F$-split. Consequently, net-free generalized caterpillar graphs are also a class of block graphs with symbolic $F$-split binomial edge ideals. Finally, it turns out that symbolic Rees algebras of binomial edge ideals associated with these two classes, namely pendant cliques graphs and net-free generalized caterpillar graphs, are strongly $F$-regular.

AI model alignment is crucial due to inadvertent biases in training data and the underspecified machine learning pipeline, where models with excellent test metrics may not meet end-user requirements. While post-training alignment via human feedback shows promise, these methods are often limited to generative AI settings where humans can interpret and provide feedback on model outputs. In traditional non-generative settings with numerical or categorical outputs, detecting misalignment through single-sample outputs remains challenging, and enforcing alignment during training requires repeating costly training processes. In this paper we consider an alternative strategy. We propose interpreting model alignment through property testing, defining an aligned model $f$ as one belonging to a subset $\mathcal{P}$ of functions that exhibit specific desired behaviors. We focus on post-processing a pre-trained model $f$ to better align with $\mathcal{P}$ using conformal risk control. Specifically, we develop a general procedure for converting queries for testing a given property $\mathcal{P}$ to a collection of loss functions suitable for use in a conformal risk control algorithm. We prove a probabilistic guarantee that the resulting conformal interval around $f$ contains a function approximately satisfying $\mathcal{P}$. We exhibit applications of our methodology on a collection of supervised learning datasets for (shape-constrained) properties such as monotonicity and concavity. The general procedure is flexible and can be applied to a wide range of desired properties. Finally, we prove that pre-trained models will always require alignment techniques even as model sizes or training data increase, as long as the training data contains even small biases.

A longstanding goal in colloidal active matter is to understand how gradients in fuel concentration influence the motion of phoretic Janus particles. Here, we present a theoretical description of the motion of a spherical phoretic Janus particle in the presence of a radial gradient of the chemical solute driving self-propulsion. Radial gradients are a geometry relevant to many scenarios in active matter systems and naturally arise due to the presence of a point source or sink of fuel. We derive an analytical solution for the Janus particle's velocity and quantify the influence of the radial concentration gradient on the particle's trajectory. Compared to a phoretic Janus particle in a linear gradient in fuel concentration, we uncover a much richer set of dynamical behaviors, including circular orbits and trapped stationary states. We identify the ratio of the phoretic mobilities between the two domains of the Janus particle as a central quantity in tuning their dynamics. Our results provide a path for developing novel protocols for tuning the dynamics of phoretic Janus particles and mixing fluid at the microscale. In addition, this work suggests a method for quantifying the surface properties of phoretic Janus particles, which have proven challenging to probe experimentally., Comment: 25 page main text + 13 page SI (single column). For associated simulation videos, see http://www.dropbox.com/scl/fo/i9btwirg89zrj7isrhogi/ADB3u9T4rKJknWFaAmk_pVA?rlkey=t3zcgz2rp7lv53asqgt6gd9tp&dl=0

A growing trend involves integrating human knowledge into learning frameworks, leveraging subtle human feedback to refine AI models. Despite these advances, no comprehensive theoretical framework describing the specific conditions under which human comparisons improve the traditional supervised fine-tuning process has been developed. To bridge this gap, this paper studies the effective use of human comparisons to address limitations arising from noisy data and high-dimensional models. We propose a two-stage "Supervised Fine Tuning+Human Comparison" (SFT+HC) framework connecting machine learning with human feedback through a probabilistic bisection approach. The two-stage framework first learns low-dimensional representations from noisy-labeled data via an SFT procedure, and then uses human comparisons to improve the model alignment. To examine the efficacy of the alignment phase, we introduce a novel concept termed the "label-noise-to-comparison-accuracy" (LNCA) ratio. This paper theoretically identifies the conditions under which the "SFT+HC" framework outperforms pure SFT approach, leveraging this ratio to highlight the advantage of incorporating human evaluators in reducing sample complexity. We validate that the proposed conditions for the LNCA ratio are met in a case study conducted via an Amazon Mechanical Turk experiment.

Although the security testing of Web systems can be automated by generating crafted inputs, solutions to automate the test oracle, i.e., vulnerability detection, remain difficult to apply in practice. Specifically, though previous work has demonstrated the potential of metamorphic testing, security failures can be determined by metamorphic relations that turn valid inputs into malicious inputs, metamorphic relations are typically executed on a large set of inputs, which is time-consuming and thus makes metamorphic testing impractical. We propose AIM, an approach that automatically selects inputs to reduce testing costs while preserving vulnerability detection capabilities. AIM includes a clustering-based black-box approach, to identify similar inputs based on their security properties. It also relies on a novel genetic algorithm to efficiently select diverse inputs while minimizing their total cost. Further, it contains a problem-reduction component to reduce the search space and speed up the minimization process. We evaluated the effectiveness of AIM on two well-known Web systems, Jenkins and Joomla, with documented vulnerabilities. We compared AIM's results with four baselines involving standard search approaches. Overall, AIM reduced metamorphic testing time by 84% for Jenkins and 82% for Joomla, while preserving the same level of vulnerability detection. Furthermore, AIM significantly outperformed all the considered baselines regarding vulnerability coverage.

In this paper, we investigate whether the symbolic and ordinary powers of a binomial edge ideal $J_{G}$ are equal. We show that the equality $J_{G}^{t}=J_{G}^{(t)}$ holds for every $t \geq 1$ when $|Ass(J_{G})|=2$. Moreover, if $G$ is a caterpillar tree, then one has the same equality. Finally, we characterize the generalized caterpillar graphs which the equality of symbolic and ordinary powers of $J_{G}$ occurs.

This work presents a new approach to solve the sparse linear regression problem, i.e., to determine a k-sparse vector w in R^d that minimizes the cost ||y - Aw||^2_2. In contrast to the existing methods, our proposed approach splits this k-sparse vector into two parts -- (a) a column stochastic binary matrix V, and (b) a vector x in R^k. Here, the binary matrix V encodes the location of the k non-zero entries in w. Equivalently, it encodes the subset of k columns in the matrix A that map w to y. We demonstrate that this enables modeling several non-trivial application-specific structural constraints on w as constraints on V. The vector x comprises of the actual non-zero values in w. We use Maximum Entropy Principle (MEP) to solve the resulting optimization problem. In particular, we ascribe a probability distribution to the set of all feasible binary matrices V, and iteratively determine this distribution and the vector x such that the associated Shannon entropy gets minimized, and the regression cost attains a pre-specified value. The resulting algorithm employs homotopy from the convex entropy function to the non-convex cost function to avoid poor local minimum. We demonstrate the efficacy and flexibility of our proposed approach in incorporating a variety of practical constraints, that are otherwise difficult to model using the existing benchmark methods.

Randomized experiments are a powerful methodology for data-driven evaluation of decisions or interventions. Yet, their validity may be undermined by network interference. This occurs when the treatment of one unit impacts not only its outcome but also that of connected units, biasing traditional treatment effect estimations. Our study introduces a new framework to accommodate complex and unknown network interference, moving beyond specialized models in the existing literature. Our framework, termed causal message-passing, is grounded in high-dimensional approximate message passing methodology. It is tailored for multi-period experiments and is particularly effective in settings with many units and prevalent network interference. The framework models causal effects as a dynamic process where a treated unit's impact propagates through the network via neighboring units until equilibrium is reached. This approach allows us to approximate the dynamics of potential outcomes over time, enabling the extraction of valuable information before treatment effects reach equilibrium. Utilizing causal message-passing, we introduce a practical algorithm to estimate the total treatment effect, defined as the impact observed when all units are treated compared to the scenario where no unit receives treatment. We demonstrate the effectiveness of this approach across five numerical scenarios, each characterized by a distinct interference structure.

In this paper, we investigate when symbolic and ordinary powers of the parity binomial edge ideal of a graph fail to be equal. It turns out that if $\mathcal{I}_{G}$ is the parity binomial edge ideal of a graph $G$, then in each of the following cases the symbolic power $\mathcal{I}_{G}^{(t)}$ and the ordinary power $\mathcal{I}_{G}^t$ are not equal for some $t$: (i) the clique number of $G$ is greater than 3; (ii) $G$ has a net; or (iii) $G$ has a PT as an induced subgraph.

In this paper, we investigate which classes of monomial ideals have a quasi-additive property of homological shift ideals. More precisely, for a monomial ideal $I$ we are interested to find out whether $HS_{i+j}(I)\subseteq HS_i(HS_j(I))$. It turns out that $\mathbf{c}$-bounded principal Borel ideals as well as polymatroidal ideals satisfying strong exchange property, and polymatroidal ideals generated in degree two have this quasi-additive property. For squarefree Borel ideals, we even have equality. Besides, the inclusion holds for every equigenerated Borel ideal and polymatroidal ideal when $j=1$.

The pursuit problem is a historical issue of the application of mathematics in physics, which has been discussed for centuries since the time of Leonardo Da Vinci, and its applications are wide ranging from military and industrial to recreational, but its place of interest is nowhere but nature and inspiration from the way of migration of birds and hunting of archer fish. The pursuit problem involves one or more pursuers trying to catch a target that is moving in a certain direction. In this article, we delve into two modes of movement: movement on a straight line and movement on a curve. Our primary focus is on the latter. Within the context of movement on a straight line, we explore two methods and compare their respective results. Furthermore, we investigate the movement of two particles chasing each other and extend these findings to N particles that are chasing each other in pairs. By leveraging these two modes of movement, we present a novel relationship for two-particle and N-particle systems in pursuit. Lastly, we analyze the movement of moths around a lamp and evaluate their motion in relation to two-particle and N-particle systems in pursuit. The results of this analysis are carefully examined.

This paper is motivated by recent research in the $d$-dimensional stochastic linear bandit literature, which has revealed an unsettling discrepancy: algorithms like Thompson sampling and Greedy demonstrate promising empirical performance, yet this contrasts with their pessimistic theoretical regret bounds. The challenge arises from the fact that while these algorithms may perform poorly in certain problem instances, they generally excel in typical instances. To address this, we propose a new data-driven technique that tracks the geometric properties of the uncertainty ellipsoid around the main problem parameter. This methodology enables us to formulate an instance-dependent frequentist regret bound, which incorporates the geometric information, for a broad class of base algorithms, including Greedy, OFUL, and Thompson sampling. This result allows us to identify and ``course-correct" problem instances in which the base algorithms perform poorly. The course-corrected algorithms achieve the minimax optimal regret of order $\tilde{\mathcal{O}}(d\sqrt{T})$ for a $T$-period decision-making scenario, effectively maintaining the desirable attributes of the base algorithms, including their empirical efficacy. We present simulation results to validate our findings using synthetic and real data.

The industrial drying process consumes approximately 12% of the total energy used in manufacturing, with the potential for a 40% reduction in energy usage through improved process controls and the development of new drying technologies. To achieve cost-efficient and high-performing drying, multiple drying technologies can be combined in a modular fashion with optimal sequencing and control parameters for each. This paper presents a mathematical formulation of this optimization problem and proposes a framework based on the Maximum Entropy Principle (MEP) to simultaneously solve for both optimal values of control parameters and optimal sequence. The proposed algorithm addresses the combinatorial optimization problem with a non-convex cost function riddled with multiple poor local minima. Simulation results on drying distillers dried grain (DDG) products show up to 12% improvement in energy consumption compared to the most efficient single-stage drying process. The proposed algorithm converges to local minima and is designed heuristically to reach the global minimum.

Multi-armed bandit (MAB) algorithms are efficient approaches to reduce the opportunity cost of online experimentation and are used by companies to find the best product from periodically refreshed product catalogs. However, these algorithms face the so-called cold-start at the onset of the experiment due to a lack of knowledge of customer preferences for new products, requiring an initial data collection phase known as the burn-in period. During this period, standard MAB algorithms operate like randomized experiments, incurring large burn-in costs which scale with the large number of products. We attempt to reduce the burn-in by identifying that many products can be cast into two-sided products, and then naturally model the rewards of the products with a matrix, whose rows and columns represent the two sides respectively. Next, we design two-phase bandit algorithms that first use subsampling and low-rank matrix estimation to obtain a substantially smaller targeted set of products and then apply a UCB procedure on the target products to find the best one. We theoretically show that the proposed algorithms lower costs and expedite the experiment in cases when there is limited experimentation time along with a large product set. Our analysis also reveals three regimes of long, short, and ultra-short horizon experiments, depending on dimensions of the matrix. Empirical evidence from both synthetic data and a real-world dataset on music streaming services validates this superior performance.

The combination of photonic integrated circuits and free-space meta-optics has the ability to unclasp technological knots that require advanced light manipulation due their conjoined ability to guide and shape electromagnetic waves. The need for large scale access and component interchangeability is essential for rapid prototyping of optical systems. Such capability represents a functional challenge in terms of fabrication and alignment of compound photonic platform. Here, we report a multi-functional interface that demonstrates the capabilities of a flexible and interchangeable combination of a photonic integrated circuit to a free-space coupling chip with different designs of low-loss meta-optics at a wavelength of 780 nm. We show that robustness and fidelity of the designed optical functions can be achieved without prior precise characterization of the free-space input nor stringent alignment between the photonic integrated chip and the meta-optics chip. A diffraction limited spot of approximately 3 micron for a hyperboloid metalens of numerical aperture 0.15 was achieved despite an input Gaussian elliptical deformation of up to 35% and misalignments of the components of up to 20 micron. A holographic display with a peak signal-to-noise ratio of more than 10 compared to its ground truth is also reported using this platform making this work the first interface to shape photonic integrated modes into free space using different diffractive optical functions.

Security testing aims at verifying that the software meets its security properties. In modern Web systems, however, this often entails the verification of the outputs generated when exercising the system with a very large set of inputs. Full automation is thus required to lower costs and increase the effectiveness of security testing. Unfortunately, to achieve such automation, in addition to strategies for automatically deriving test inputs, we need to address the oracle problem, which refers to the challenge, given an input for a system, of distinguishing correct from incorrect behavior. In this paper, we propose Metamorphic Security Testing for Web-interactions (MST-wi), a metamorphic testing approach that integrates test input generation strategies inspired by mutational fuzzing and alleviates the oracle problem in security testing. It enables engineers to specify metamorphic relations (MRs) that capture many security properties of Web systems. To facilitate the specification of such MRs, we provide a domain-specific language accompanied by an Eclipse editor. MST-wi automatically collects the input data and transforms the MRs into executable Java code to automatically perform security testing. It automatically tests Web systems to detect vulnerabilities based on the relations and collected data. We provide a catalog of 76 system-agnostic MRs to automate security testing in Web systems. It covers 39% of the OWASP security testing activities not automated by state-of-the-art techniques; further, our MRs can automatically discover 102 different types of vulnerabilities, which correspond to 45% of the vulnerabilities due to violations of security design principles according to the MITRE CWE database. We also define guidelines that enable test engineers to improve the testability of the system under test with respect to our approach., Comment: arXiv admin note: text overlap with arXiv:1912.05278

Diffusion processes that evolve according to linear stochastic differential equations are an important family of continuous-time dynamic decision-making models. Optimal policies are well-studied for them, under full certainty about the drift matrices. However, little is known about data-driven control of diffusion processes with uncertain drift matrices as conventional discrete-time analysis techniques are not applicable. In addition, while the task can be viewed as a reinforcement learning problem involving exploration and exploitation trade-off, ensuring system stability is a fundamental component of designing optimal policies. We establish that the popular Thompson sampling algorithm learns optimal actions fast, incurring only a square-root of time regret, and also stabilizes the system in a short time period. To the best of our knowledge, this is the first such result for Thompson sampling in a diffusion process control problem. We validate our theoretical results through empirical simulations with real parameter matrices from two settings of airplane and blood glucose control. Moreover, we observe that Thompson sampling significantly improves (worst-case) regret, compared to the state-of-the-art algorithms, suggesting Thompson sampling explores in a more guarded fashion. Our theoretical analysis involves characterization of a certain optimality manifold that ties the local geometry of the drift parameters to the optimal control of the diffusion process. We expect this technique to be of broader interest.