Search

The digital twin approach has gained recognition as a promising solution to the challenges faced by the Architecture, Engineering, Construction, Operations, and Management (AECOM) industries. However, its broader application across AECOM sectors remains limited. One significant obstacle is that traditional digital twins rely on deterministic models, which require deterministic input parameters. This limits their accuracy, as they do not account for the substantial uncertainties inherent in AECOM projects. These uncertainties are particularly pronounced in geotechnical design and construction. To address this challenge, we propose a Probabilistic Digital Twin (PDT) framework that extends traditional digital twin methodologies by incorporating uncertainties, and is tailored to the requirements of geotechnical design and construction. The PDT framework provides a structured approach to integrating all sources of uncertainty, including aleatoric, data, model, and prediction uncertainties, and propagates them throughout the entire modeling process. To ensure that site-specific conditions are accurately reflected as additional information is obtained, the PDT leverages Bayesian methods for model updating. The effectiveness of the probabilistic digital twin framework is showcased through an application to a highway foundation construction project, demonstrating its potential to improve decision-making and project outcomes in the face of significant uncertainties.

Recovering high-level type information in binaries is a key task in reverse engineering and binary analysis. Binaries contain very little explicit type information. The structure of binary code is incredibly flexible allowing for ad-hoc subtyping and polymorphism. Prior work has shown that precise type inference on binary code requires expressive subtyping and polymorphism. Implementations of these type system features in a binary type inference algorithm have thus-far been too inefficient to achieve widespread adoption. Recent advances in traditional type inference have achieved simple and efficient principal type inference in an ML like language with subtyping and polymorphism through the framework of algebraic subtyping. BinSub, a new binary type inference algorithm, recognizes the connection between algebraic subtyping and the type system features required to analyze binaries effectively. Using this connection, BinSub achieves simple, precise, and efficient binary type inference. We show that BinSub maintains a similar precision to prior work, while achieving a 63x improvement in average runtime for 1568 functions. We also present a formalization of BinSub and show that BinSub's type system maintains the expressiveness of prior work., Comment: To appear in SAS 2024

We revisit the question of whether the strong law of large numbers (SLLN) holds uniformly in a rich family of distributions, culminating in a distribution-uniform generalization of the Marcinkiewicz-Zygmund SLLN. These results can be viewed as extensions of Chung's distribution-uniform SLLN to random variables with uniformly integrable $q^\text{th}$ absolute central moments for $0 < q < 2$. Furthermore, we show that uniform integrability of the $q^\text{th}$ moment is both sufficient and necessary for the SLLN to hold uniformly at the Marcinkiewicz-Zygmund rate of $n^{1/q - 1}$. These proofs centrally rely on novel distribution-uniform analogues of some familiar almost sure convergence results including the Khintchine-Kolmogorov convergence theorem, Kolmogorov's three-series theorem, a stochastic generalization of Kronecker's lemma, and the Borel-Cantelli lemmas. We also consider the non-identically distributed case., Comment: 32 pages

Are asymptotic confidence sequences and anytime $p$-values uniformly valid for a nontrivial class of distributions $\mathcal{P}$? We give a positive answer to this question by deriving distribution-uniform anytime-valid inference procedures. Historically, anytime-valid methods -- including confidence sequences, anytime $p$-values, and sequential hypothesis tests that enable inference at stopping times -- have been justified nonasymptotically. Nevertheless, asymptotic procedures such as those based on the central limit theorem occupy an important part of statistical toolbox due to their simplicity, universality, and weak assumptions. While recent work has derived asymptotic analogues of anytime-valid methods with the aforementioned benefits, these were not shown to be $\mathcal{P}$-uniform, meaning that their asymptotics are not uniformly valid in a class of distributions $\mathcal{P}$. Indeed, the anytime-valid inference literature currently has no central limit theory to draw from that is both uniform in $\mathcal{P}$ and in the sample size $n$. This paper fills that gap by deriving a novel $\mathcal{P}$-uniform strong Gaussian approximation theorem. We apply some of these results to obtain an anytime-valid test of conditional independence without the Model-X assumption, as well as a $\mathcal{P}$-uniform law of the iterated logarithm.

Cosine similarity is an established similarity metric for computing associations on vectors, and it is commonly used to identify related samples from biological perturbational data. The distribution of cosine similarity changes with the covariance of the data, and this in turn affects the statistical power to identify related signals. The relationship between the mean and covariance of the distribution of the data and the distribution of cosine similarity is poorly understood. In this work, we derive the asymptotic moments of cosine similarity as a function of the data and identify the criteria of the data covariance matrix that minimize the variance of cosine similarity. We find that the variance of cosine similarity is minimized when the eigenvalues of the covariance matrix are equal for centered data. One immediate application of this work is characterizing the null distribution of cosine similarity over a dataset with non-zero covariance structure. Furthermore, this result can be used to optimize over a set of transformations or representations on a dataset to maximize power, recall, or other discriminative metrics, with direct application to noisy biological data. While we consider the specific biological domain of perturbational data analysis, our result has potential application for any use of cosine similarity or Pearson's correlation on data with covariance structure., Comment: 30 pages, 4 figures

A/B tests are the gold standard for evaluating digital experiences on the web. However, traditional "fixed-horizon" statistical methods are often incompatible with the needs of modern industry practitioners as they do not permit continuous monitoring of experiments. Frequent evaluation of fixed-horizon tests ("peeking") leads to inflated type-I error and can result in erroneous conclusions. We have released an experimentation service on the Adobe Experience Platform based on anytime-valid confidence sequences, allowing for continuous monitoring of the A/B test and data-dependent stopping. We demonstrate how we adapted and deployed asymptotic confidence sequences in a full featured A/B testing platform, describe how sample size calculations can be performed, and how alternate test statistics like "lift" can be analyzed. On both simulated data and thousands of real experiments, we show the desirable properties of using anytime-valid methods instead of traditional approaches., Comment: 15 pages, 12 figures. Expanded version of ACM Web Conference Proceedings paper

Hybrid lead halide perovskites (LHPs) are a class of semiconductor with novel properties that are distinctively governed by structural fluctuations. Diffraction experiments sensitive to average, long-range order reveal a cubic structure in the device-relevant, high-temperature phase. Local probes find additional short-range order with lower symmetry that may govern the structure-function relationships of LHPs. However, the dimensionality, participating atoms, and dynamics of this short-range order are unresolved, impeding our understanding of technologically relevant properties including long carrier lifetimes and facile halide migration. Here, we determine the true structure of two prototypical hybrid LHPs, CH$_3$NH$_3$PbI$_3$ and CH$_3$NH$_3$PbBr$_3$, using a combination of single-crystal X-ray and neutron diffuse scattering, neutron inelastic spectroscopy, and molecular dynamics simulations. The remarkable collective dynamics we found are not suggested by previous studies and consist of a network of local two-dimensional, circular pancake-like regions of dynamically tilting lead halide octahedra (lower symmetry) that induce longer range intermolecular correlations within the CH$_3$NH$_3^+$ sublattice. The dynamic local structure can introduce transient ferroelectric or antiferroelectric domains that increase charge carrier lifetimes, and strongly affect the halide migration, a poorly understood degradation mechanism. Our approach of co-analyzing single-crystal X-ray and neutron diffuse scattering data with MD simulations will provide unparalleled insights into the structure of hybrid materials and materials with engineered disorder.

An accurate force calculation with the Poisson-Boltzmann equation is challenging, as it requires the electric field on the molecular surface. Here, we present a calculation of the electric field on the solute-solvent interface that is exact for piece-wise linear variations of the potential and analyze four different alternatives to compute the force using a boundary element method. We performed a verification exercise for two cases: the isolated and two interacting molecules. Our results suggest that the boundary element method outperforms the finite difference method, as the latter needs a much finer mesh than in solvation energy calculations to get acceptable accuracy in the force, whereas the same surface mesh than a standard energy calculation is appropriate for the boundary element method. Among the four evaluated alternatives of force calculation, we saw that the most accurate one is based on the Maxwell stress tensor. However, for a realistic application, like the barnase-barstar complex, the approach based on variations of the energy functional, which is less accurate, gives equivalent results. This analysis is useful towards using the Poisson-Boltzmann equation for force calculations in applications where high accuracy is key, for example, to feed molecular dynamics models or to enable the study of the interaction between large molecular structures, like viruses adsorbed onto substrates., Comment: 33 pages

Contextual bandit algorithms are ubiquitous tools for active sequential experimentation in healthcare and the tech industry. They involve online learning algorithms that adaptively learn policies over time to map observed contexts $X_t$ to actions $A_t$ in an attempt to maximize stochastic rewards $R_t$. This adaptivity raises interesting but hard statistical inference questions, especially counterfactual ones: for example, it is often of interest to estimate the properties of a hypothetical policy that is different from the logging policy that was used to collect the data -- a problem known as ``off-policy evaluation'' (OPE). Using modern martingale techniques, we present a comprehensive framework for OPE inference that relax unnecessary conditions made in some past works, significantly improving on them both theoretically and empirically. Importantly, our methods can be employed while the original experiment is still running (that is, not necessarily post-hoc), when the logging policy may be itself changing (due to learning), and even if the context distributions are a highly dependent time-series (such as if they are drifting over time). More concretely, we derive confidence sequences for various functionals of interest in OPE. These include doubly robust ones for time-varying off-policy mean reward values, but also confidence bands for the entire cumulative distribution function of the off-policy reward distribution. All of our methods (a) are valid at arbitrary stopping times (b) only make nonparametric assumptions, (c) do not require importance weights to be uniformly bounded and if they are, we do not need to know these bounds, and (d) adapt to the empirical variance of our estimators. In summary, our methods enable anytime-valid off-policy inference using adaptively collected contextual bandit data., Comment: 43 pages, 6 figures

This work derives methods for performing nonparametric, nonasymptotic statistical inference for population means under the constraint of local differential privacy (LDP). Given bounded observations $(X_1, \dots, X_n)$ with mean $\mu^\star$ that are privatized into $(Z_1, \dots, Z_n)$, we present confidence intervals (CI) and time-uniform confidence sequences (CS) for $\mu^\star$ when only given access to the privatized data. To achieve this, we study a nonparametric and sequentially interactive generalization of Warner's famous ``randomized response'' mechanism, satisfying LDP for arbitrary bounded random variables, and then provide CIs and CSs for their means given access to the resulting privatized observations. For example, our results yield private analogues of Hoeffding's inequality in both fixed-time and time-uniform regimes. We extend these Hoeffding-type CSs to capture time-varying (non-stationary) means, and conclude by illustrating how these methods can be used to conduct private online A/B tests., Comment: 50 pages, 7 figures, to appear in the 2023 International Conference on Machine Learning with an Oral Presentation

Accurately determining the outcome of an election is a complex task with many potential sources of error, ranging from software glitches in voting machines to procedural lapses to outright fraud. Risk-limiting audits (RLA) are statistically principled "incremental" hand counts that provide statistical assurance that reported outcomes accurately reflect the validly cast votes. We present a suite of tools for conducting RLAs using confidence sequences -- sequences of confidence sets which uniformly capture an electoral parameter of interest from the start of an audit to the point of an exhaustive recount with high probability. Adopting the SHANGRLA framework, we design nonnegative martingales which yield computationally and statistically efficient confidence sequences and RLAs for a wide variety of election types., Comment: 19 pages, 8 figures

dentifying associations among biological variables is a major challenge in modern quantitative biological research, particularly given the systemic and statistical noise endemic to biological systems. Drug sensitivity data has proven to be a particularly challenging field for identifying associations to inform patient treatment. To address this, we introduce two semi-parametric variations on the commonly used concordance index: the robust concordance index and the kernelized concordance index (rCI, kCI), which incorporate measurements about the noise distribution from the data. We demonstrate that common statistical tests applied to the concordance index and its variations fail to control for false positives, and introduce efficient implementations to compute p-values using adaptive permutation testing. We then evaluate the statistical power of these coefficients under simulation and compare with Pearson and Spearman correlation coefficients. Finally, we evaluate the various statistics in matching drugs across pharmacogenomic datasets. We observe that the rCI and kCI are better powered than the concordance index in simulation and show some improvement on real data. Surprisingly, we observe that the Pearson correlation was the most robust to measurement noise among the different metrics.

Confidence intervals based on the central limit theorem (CLT) are a cornerstone of classical statistics. Despite being only asymptotically valid, they are ubiquitous because they permit statistical inference under weak assumptions and can often be applied to problems even when nonasymptotic inference is impossible. This paper introduces time-uniform analogues of such asymptotic confidence intervals, adding to the literature on confidence sequences (CS) -- sequences of confidence intervals that are uniformly valid over time -- which provide valid inference at arbitrary stopping times and incur no penalties for "peeking" at the data, unlike classical confidence intervals which require the sample size to be fixed in advance. Existing CSs in the literature are nonasymptotic, enjoying finite-sample guarantees but not the aforementioned broad applicability of asymptotic confidence intervals. This work provides a definition for "asymptotic CSs" and a general recipe for deriving them. Asymptotic CSs forgo nonasymptotic validity for CLT-like versatility and (asymptotic) time-uniform guarantees. While the CLT approximates the distribution of a sample average by that of a Gaussian for a fixed sample size, we use strong invariance principles (stemming from the seminal 1960s work of Strassen) to uniformly approximate the entire sample average process by an implicit Gaussian process. As an illustration, we derive asymptotic CSs for the average treatment effect in observational studies (for which nonasymptotic bounds are essentially impossible to derive even in the fixed-time regime) as well as randomized experiments, enabling causal inference in sequential environments., Comment: 69 pages, 10 figures

This paper derives confidence intervals (CI) and time-uniform confidence sequences (CS) for the classical problem of estimating an unknown mean from bounded observations. We present a general approach for deriving concentration bounds, that can be seen as a generalization and improvement of the celebrated Chernoff method. At its heart, it is based on a class of composite nonnegative martingales, with strong connections to testing by betting and the method of mixtures. We show how to extend these ideas to sampling without replacement, another heavily studied problem. In all cases, our bounds are adaptive to the unknown variance, and empirically vastly outperform existing approaches based on Hoeffding or empirical Bernstein inequalities and their recent supermartingale generalizations. In short, we establish a new state-of-the-art for four fundamental problems: CSs and CIs for bounded means, when sampling with and without replacement., Comment: 70 pages, 20 figures; Python implementation: https://github.com/gostevehoward/confseq

Many practical tasks involve sampling sequentially without replacement (WoR) from a finite population of size $N$, in an attempt to estimate some parameter $\theta^\star$. Accurately quantifying uncertainty throughout this process is a nontrivial task, but is necessary because it often determines when we stop collecting samples and confidently report a result. We present a suite of tools for designing confidence sequences (CS) for $\theta^\star$. A CS is a sequence of confidence sets $(C_n)_{n=1}^N$, that shrink in size, and all contain $\theta^\star$ simultaneously with high probability. We present a generic approach to constructing a frequentist CS using Bayesian tools, based on the fact that the ratio of a prior to the posterior at the ground truth is a martingale. We then present Hoeffding- and empirical-Bernstein-type time-uniform CSs and fixed-time confidence intervals for sampling WoR, which improve on previous bounds in the literature and explicitly quantify the benefit of WoR sampling.

We investigate the hypothesis of dynamic tetragonal domains occuring in cubic CH$_3$NH$_3$PbI$_3$ using high-resolution neutron spectroscopy to study a fully deuterated single crystal. The $\textit{R}$-point scattering above the 327.5 K tetragonal-cubic phase transition is always resolution-limited in energy and therefore inconsistent with dynamic domain predictions. This behavior is instead consistent with the central peak phenomenon observed in other perovskites. The scattering may originate from small, static, tetragonal-phase domains nucleating about crystal defects, and the temperature dependence demonstrates the transition is first-order.

Recently several European National Measurement Institutes have established traceable calibration methods for dynamic mechanical quantities, e.g., dynamic force, torque and pressure. However, the use in industry and elsewhere of dynamic calibration information provided on certificates is not straightforward. Typically it is necessary to employ deconvolution techniques to obtain estimates of measurands, and the deconvolution method itself and the associated algorithms are sources of uncertainty that must be included in uncertainty budgets. There is a need for practical guidance for end users on how to use the newly-available dynamic calibration information. To this end we set out an approach to the evaluation of uncertainties associated with dynamic measurements that we believe covers the most relevant cases. The methods have been embodied in publicly-available software and we show how they can be used to tackle some example problems. We believe that the methods lead to more reliable estimates of the relevant measurands and their associated uncertainties., Comment: 21 pages, 10 figures

Hybrid organic-inorganic perovskites (HOIPs) have become an important class of semiconductors for solar cells and other optoelectronic applications. Electron-phonon coupling plays a critical role in all optoelectronic devices, and although the lattice dynamics and phonon frequencies of HOIPs have been well studied, little attention has been given to phonon lifetimes. We report the first high-precision measurements of acoustic phonon lifetimes in the hybrid perovskite methylammonium lead iodide (MAPI), using inelastic neutron spectroscopy to provide high energy resolution and fully deuterated single crystals to reduce incoherent scattering from hydrogen. Our measurements reveal extremely short lifetimes on the order of picoseconds, corresponding to nanometer mean free paths and demonstrating that acoustic phonons are unable to dissipate heat efficiently. Lattice-dynamics calculations using ab-initio third-order perturbation theory indicate that the short lifetimes stem from strong three-phonon interactions and a high density of low-energy optical phonon modes related to the degrees of freedom of the organic cation. Such short lifetimes have significant implications for electron-phonon coupling in MAPI and other HOIPs, with direct impacts on optoelectronic devices both in the cooling of hot carriers and in the transport and recombination of band edge carriers. These findings illustrate a fundamental difference between HOIPs and conventional photovoltaic semiconductors and demonstrate the importance of understanding lattice dynamics in the effort to develop metal halide perovskite optoelectronic devices., Comment: 31 pages, 14 figures