Search

Score-based models, trained with denoising score matching, are remarkably effective in generating high dimensional data. However, the high variance of their training objective hinders optimisation. We attempt to reduce it with a control variate, derived via a $k$-th order Taylor expansion on the training objective and its gradient. We prove an equivalence between the two and demonstrate empirically the effectiveness of our approach on a low dimensional problem setting; and study its effect on larger problems., Comment: 14 pages, ICML Structured Probabilistic Inference & Generative Modeling 2024

This study explores the age-old quest to construct a geometric model of a quantum particle. While static classical particle models have largely been dismissed, the focus has now shifted to intricate dynamic models that hold the promise of reconciling general relativity with quantum mechanics. We propose that matter particles can be described as radiation confined within dynamically curved spacetime regions, without the need for quantization of space and time, and using standard field equations and natural Planck units. Specifically, we investigate a cyclic or oscillating radiation-dominated micro cosmos undergoing repeated bouncing. Our methodology employs integration, with carefully defined initial conditions. The results include several observable properties characteristic of quantum particles. We calculate the total mass, revealing a compelling inverse proportionality between mass and radius identical with the de Broglie relationship. Applying this model to protons, we discover a profound and surprisingly simple relationship between the proton's radius and mass expressed in Planck units. This enables a definition of the proton radius that aligns remarkably well with the 2018 CODATA value. Furthermore, our analysis demonstrates that the radial density profile of the proton (or nucleon), averaged over a cycle time, increases toward the center. The problem of embedding the micro cosmos within a background spacetime is also described. These results underscore the relevance of general relativity in the domain of nuclear physics. Moreover, the model offers a fresh perspective that can stimulate new ideas in the ongoing quest to unify general relativity with quantum physics., Comment: 18 pages, 5 figures

In this work, we address the problem of assessing and constructing feedback for early-stage writing automatically using machine learning. Early-stage writing is typically vastly different from conventional writing due to phonetic spelling and lack of proper grammar, punctuation, spacing etc. Consequently, early-stage writing is highly non-trivial to analyze using common linguistic metrics. We propose to use sequence-to-sequence models for "translating" early-stage writing by students into "conventional" writing, which allows the translated text to be analyzed using linguistic metrics. Furthermore, we propose a novel robust likelihood to mitigate the effect of noise in the dataset. We investigate the proposed methods using a set of numerical experiments and demonstrate that the conventional text can be predicted with high accuracy., Comment: 9 pages, 1 figure, 1 table, to be published in the proceedings of the Northern Lights Deep Learning Conference 2024

The NASA's Double-Asteroid Redirection Test (DART) was a unique planetary defence and technology test mission, the first of its kind. The main spacecraft of the DART mission impacted the target asteroid Dimorphos, a small moon orbiting asteroid (65803) Didymos, on 2022 September 26. The impact brought up a mass of ejecta which, together with the direct momentum transfer from the collision, caused an orbital period change of 33 +/- 1 minutes, as measured by ground-based observations. We report here the outcome of the optical monitoring campaign of the Didymos system from the Danish 1.54 m telescope at La Silla around the time of impact. The observations contributed to the determination of the changes in the orbital parameters of the Didymos-Dimorphos system, as reported by arXiv:2303.02077, but in this paper we focus on the ejecta produced by the DART impact. We present photometric measurements from which we remove the contribution from the Didymos-Dimorphos system using a H-G photometric model. Using two photometric apertures we determine the fading rate of the ejecta to be 0.115 +/- 0.003 mag/d (in a 2" aperture) and 0.086 +/- 0.003 mag/d (5") over the first week post-impact. After about 8 days post-impact we note the fading slows down to 0.057 +/- 0.003 mag/d (2" aperture) and 0.068 +/- 0.002 mag/d (5"). We include deep-stacked images of the system to illustrate the ejecta evolution during the first 18 days, noting the emergence of dust tails formed from ejecta pushed in the anti-solar direction, and measuring the extent of the particles ejected sunward to be at least 4000 km., Comment: 20 pages, 6 figures. Accepted for publication in The Planetary Science Journal

In geospatial planning, it is often essential to represent objects in a vectorized format, as this format easily translates to downstream tasks such as web development, graphics, or design. While these problems are frequently addressed using semantic segmentation, which requires additional post-processing to vectorize objects in a non-trivial way, we present an Image-to-Sequence model that allows for direct shape inference and is ready for vector-based workflows out of the box. We demonstrate the model's performance in various ways, including perturbations to the image input that correspond to variations or artifacts commonly encountered in remote sensing applications. Our model outperforms prior works when using ground truth bounding boxes (one object per image), achieving the lowest maximum tangent angle error., Comment: ICLR 2023 Workshop on Machine Learning in Remote Sensing

We present a novel pipeline for learning the conditional distribution of a building roof mesh given pixels from an aerial image, under the assumption that roof geometry follows a set of regular patterns. Unlike alternative methods that require multiple images of the same object, our approach enables estimating 3D roof meshes using only a single image for predictions. The approach employs the PolyGen, a deep generative transformer architecture for 3D meshes. We apply this model in a new domain and investigate the sensitivity of the image resolution. We propose a novel metric to evaluate the performance of the inferred meshes, and our results show that the model is robust even at lower resolutions, while qualitatively producing realistic representations for out-of-distribution samples., Comment: Copyright 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Bayesian optimization (BO) is a popular method for black-box optimization, which relies on uncertainty as part of its decision-making process when deciding which experiment to perform next. However, not much work has addressed the effect of uncertainty on the performance of the BO algorithm and to what extent calibrated uncertainties improve the ability to find the global optimum. In this work, we provide an extensive study of the relationship between the BO performance (regret) and uncertainty calibration for popular surrogate models and compare them across both synthetic and real-world experiments. Our results confirm that Gaussian Processes are strong surrogate models and that they tend to outperform other popular models. Our results further show a positive association between calibration error and regret, but interestingly, this association disappears when we control for the type of model in the analysis. We also studied the effect of re-calibration and demonstrate that it generally does not lead to improved regret. Finally, we provide theoretical justification for why uncertainty calibration might be difficult to combine with BO due to the small sample sizes commonly used., Comment: 14 pages, 4 figures, 2 tables

The body of research on classification of solar panel arrays from aerial imagery is increasing, yet there are still not many public benchmark datasets. This paper introduces two novel benchmark datasets for classifying and localizing solar panel arrays in Denmark: A human annotated dataset for classification and segmentation, as well as a classification dataset acquired using self-reported data from the Danish national building registry. We explore the performance of prior works on the new benchmark dataset, and present results after fine-tuning models using a similar approach as recent works. Furthermore, we train models of newer architectures and provide benchmark baselines to our datasets in several scenarios. We believe the release of these datasets may improve future research in both local and global geospatial domains for identifying and mapping of solar panel arrays from aerial imagery. The data is accessible at https://osf.io/aj539/., Comment: 7 pages, 2 figures, to access the dataset, see https://osf.io/aj539/

Black-box variational inference (BBVI) now sees widespread use in machine learning and statistics as a fast yet flexible alternative to Markov chain Monte Carlo methods for approximate Bayesian inference. However, stochastic optimization methods for BBVI remain unreliable and require substantial expertise and hand-tuning to apply effectively. In this paper, we propose Robust and Automated Black-box VI (RABVI), a framework for improving the reliability of BBVI optimization. RABVI is based on rigorously justified automation techniques, includes just a small number of intuitive tuning parameters, and detects inaccurate estimates of the optimal variational approximation. RABVI adaptively decreases the learning rate by detecting convergence of the fixed--learning-rate iterates, then estimates the symmetrized Kullback--Leibler (KL) divergence between the current variational approximation and the optimal one. It also employs a novel optimization termination criterion that enables the user to balance desired accuracy against computational cost by comparing (i) the predicted relative decrease in the symmetrized KL divergence if a smaller learning were used and (ii) the predicted computation required to converge with the smaller learning rate. We validate the robustness and accuracy of RABVI through carefully designed simulation studies and on a diverse set of real-world model and data examples.

We report the first unambiguous detection and mass measurement of an isolated stellar-mass black hole (BH). We used the Hubble Space Telescope (HST) to carry out precise astrometry of the source star of the long-duration (t_E~270 days), high-magnification microlensing event MOA-2011-BLG-191/OGLE-2011-BLG-0462 (hereafter designated as MOA-11-191/OGLE-11-462), in the direction of the Galactic bulge. HST imaging, conducted at eight epochs over an interval of six years, reveals a clear relativistic astrometric deflection of the background star's apparent position. Ground-based photometry of MOA-11-191/OGLE-11-462 shows a parallactic signature of the effect of the Earth's motion on the microlensing light curve. Combining the HST astrometry with the ground-based light curve and the derived parallax, we obtain a lens mass of 7.1 +/- 1.3 Msun and a distance of 1.58 +/- 0.18 kpc. We show that the lens emits no detectable light, which, along with having a mass higher than is possible for a white dwarf or neutron star, confirms its BH nature. Our analysis also provides an absolute proper motion for the BH. The proper motion is offset from the mean motion of Galactic-disk stars at similar distances by an amount corresponding to a transverse space velocity of ~45 km/s, suggesting that the BH received a 'natal kick' from its supernova explosion. Previous mass determinations for stellar-mass BHs have come from radial-velocity measurements of Galactic X-ray binaries, and from gravitational radiation emitted by merging BHs in binary systems in external galaxies. Our mass measurement is the first for an isolated stellar-mass BH using any technique., Comment: 37 pages, Published in ApJ

Antigen test kits have been used extensively as a screening tool during the worldwide pandemic of coronavirus (SARS-CoV-2). While it is generally expected that taking samples for analysis with PCR testing gives more reliable results than using antigen test kits, the overall sensitivity and specificity of the two protocols in the field have not yet been estimated without assuming that the PCR test constitutes a gold standard. We use latent class models to estimate the in situ performance of both PCR and antigen testing, using data from the Danish national registries. The results are based on 240,000 paired tests results sub-selected from the 55 million test results that were obtained in Denmark during the period from February 2021 until June 2021. We found that the specificity of both tests is very high in our data sample (>99.7%), while the sensitivity of PCR sampling was estimated to be 95.7% (95% CI: 92.8-98.4%) and that of the antigen test kits used in Denmark over the study period was estimated at 53.8% (95% CI: 49.8-57.9%). Our findings can be used as supplementary information for consideration when implementing serial testing strategies that employ a confirmatory PCR sample following a positive result from an antigen test kit, such as the policy used in Denmark. We note that while this strategy reduces the number of false positives associated with antigen test screening, it also increases the false negatives. We demonstrate that the balance of trading false positives for false negatives only favours the use of serial testing when the expected true prevalence is low. Our results contain substantial uncertainty in the estimates for sensitivity due to the relatively small number of positive test results over this period: validation of our findings in a population with higher prevalence would therefore be highly relevant for future work.

As the rate of data collection continues to grow rapidly, developing visualization tools that scale to immense data sets is a serious and ever-increasing challenge. Existing approaches generally seek to decouple storage and visualization systems, performing just-in-time data reduction to transparently avoid overloading the visualizer. We present a new architecture in which the visualizer and data store are tightly coupled. Unlike systems that read raw data from storage, the performance of our system scales linearly with the size of the final visualization, essentially independent of the size of the data. Thus, it scales to massive data sets while supporting interactive performance (sub-100 ms query latency). This enables a new class of visualization clients that automatically manage data, quickly and transparently requesting data from the underlying database without requiring the user to explicitly initiate queries. It lays a groundwork for supporting truly interactive exploration of big data and opens new directions for research on scalable information visualization systems., Comment: 14 pages; Originally published in May 2018 as a technical report in the UC Berkeley EECS Technical Report Series (see https://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-85.html)

Current black-box variational inference (BBVI) methods require the user to make numerous design choices -- such as the selection of variational objective and approximating family -- yet there is little principled guidance on how to do so. We develop a conceptual framework and set of experimental tools to understand the effects of these choices, which we leverage to propose best practices for maximizing posterior approximation accuracy. Our approach is based on studying the pre-asymptotic tail behavior of the density ratios between the joint distribution and the variational approximation, then exploiting insights and tools from the importance sampling literature. Our framework and supporting experiments help to distinguish between the behavior of BBVI methods for approximating low-dimensional versus moderate-to-high-dimensional posteriors. In the latter case, we show that mass-covering variational objectives are difficult to optimize and do not improve accuracy, but flexible variational families can improve accuracy and the effectiveness of importance sampling -- at the cost of additional optimization challenges. Therefore, for moderate-to-high-dimensional posteriors we recommend using the (mode-seeking) exclusive KL divergence since it is the easiest to optimize, and improving the variational family or using model parameter transformations to make the posterior and optimal variational approximation more similar. On the other hand, in low-dimensional settings, we show that heavy-tailed variational families and mass-covering divergences are effective and can increase the chances that the approximation can be improved by importance sampling.

Recent work reported the discovery of a gamma-ray burst (GRB) associated with the galaxy GN-z11 at $z\sim 11$. The extreme improbability of the transient source being a GRB in the very early Universe requires robust elimination of all plausible alternative hypotheses. We identify numerous examples of similar transient signals in separate archival MOSFIRE observations and argue that Solar system objects -- natural or artificial -- are a far more probable explanation for these phenomena. An appendix has been added in response to additional points raised in Jiang et al. (2021), which do not change the conclusion.

We consider the problem of fitting variational posterior approximations using stochastic optimization methods. The performance of these approximations depends on (1) how well the variational family matches the true posterior distribution,(2) the choice of divergence, and (3) the optimization of the variational objective. We show that even in the best-case scenario when the exact posterior belongs to the assumed variational family, common stochastic optimization methods lead to poor variational approximations if the problem dimension is moderately large. We also demonstrate that these methods are not robust across diverse model types. Motivated by these findings, we develop a more robust and accurate stochastic optimization framework by viewing the underlying optimization algorithm as producing a Markov chain. Our approach is theoretically motivated and includes a diagnostic for convergence and a novel stopping rule, both of which are robust to noisy evaluations of the objective function. We show empirically that the proposed framework works well on a diverse set of models: it can automatically detect stochastic optimization failure or inaccurate variational approximation

We formulate approximate Bayesian inference in non-conjugate temporal and spatio-temporal Gaussian process models as a simple parameter update rule applied during Kalman smoothing. This viewpoint encompasses most inference schemes, including expectation propagation (EP), the classical (Extended, Unscented, etc.) Kalman smoothers, and variational inference. We provide a unifying perspective on these algorithms, showing how replacing the power EP moment matching step with linearisation recovers the classical smoothers. EP provides some benefits over the traditional methods via introduction of the so-called cavity distribution, and we combine these benefits with the computational efficiency of linearisation, providing extensive empirical analysis demonstrating the efficacy of various algorithms under this unifying framework. We provide a fast implementation of all methods in JAX., Comment: Accepted to International Conference on Machine Learning (ICML) 2020

Gaussian processes are powerful non-parametric probabilistic models for stochastic functions. However, the direct implementation entails a complexity that is computationally intractable when the number of observations is large, especially when estimated with fully Bayesian methods such as Markov chain Monte Carlo. In this paper, we focus on a low-rank approximate Bayesian Gaussian processes, based on a basis function approximation via Laplace eigenfunctions for stationary covariance functions. The main contribution of this paper is a detailed analysis of the performance, and practical recommendations for how to select the number of basis functions and the boundary factor. Intuitive visualizations and recommendations, make it easier for users to improve approximation accuracy and computational performance. We also propose diagnostics for checking that the number of basis functions and the boundary factor are adequate given the data. The approach is simple and exhibits an attractive computational complexity due to its linear structure, and it is easy to implement in probabilistic programming frameworks. Several illustrative examples of the performance and applicability of the method in the probabilistic programming language Stan are presented together with the underlying Stan model code., Comment: 27 pages, 18 figures

Most research in Bayesian optimization (BO) has focused on \emph{direct feedback} scenarios, where one has access to exact values of some expensive-to-evaluate objective. This direction has been mainly driven by the use of BO in machine learning hyper-parameter configuration problems. However, in domains such as modelling human preferences, A/B tests, or recommender systems, there is a need for methods that can replace direct feedback with \emph{preferential feedback}, obtained via rankings or pairwise comparisons. In this work, we present preferential batch Bayesian optimization (PBBO), a new framework that allows finding the optimum of a latent function of interest, given any type of parallel preferential feedback for a group of two or more points. We do so by using a Gaussian process model with a likelihood specially designed to enable parallel and efficient data collection mechanisms, which are key in modern machine learning. We show how the acquisitions developed under this framework generalize and augment previous approaches in Bayesian optimization, expanding the use of these techniques to a wider range of domains. An extensive simulation study shows the benefits of this approach, both with simulated functions and four real data sets., Comment: 6 pages + 7 pages in supplementary material

Recently, new methods for model assessment, based on subsampling and posterior approximations, have been proposed for scaling leave-one-out cross-validation (LOO) to large datasets. Although these methods work well for estimating predictive performance for individual models, they are less powerful in model comparison. We propose an efficient method for estimating differences in predictive performance by combining fast approximate LOO surrogates with exact LOO subsampling using the difference estimator and supply proofs with regards to scaling characteristics. The resulting approach can be orders of magnitude more efficient than previous approaches, as well as being better suited to model comparison.

Microfading Spectrometry (MFS) is a method for assessing light sensitivity color (spectral) variations of cultural heritage objects. The MFS technique provides measurements of the surface under study, where each point of the surface gives rise to a time-series that represents potential spectral (color) changes due to sunlight exposition over time. Color fading is expected to be non-decreasing as a function of time and stabilize eventually. These properties can be expressed in terms of the partial derivatives of the functions. We propose a spatio-temporal model that takes this information into account by jointly modeling the spatio-temporal process and its derivative process using Gaussian processes (GPs). We fitted the proposed model to MFS data collected from the surface of prehistoric rock art paintings. A multivariate covariance function in a GP allows modeling trichromatic image color variables jointly with spatial distances and time points variables as inputs to evaluate the covariance structure of the data. We demonstrated that the colorimetric variables are useful for predicting the color fading time-series for new unobserved spatial locations. Furthermore, constraining the model using derivative sign observations for monotonicity was shown to be beneficial in terms of both predictive performance and application-specific interpretability., Comment: arXiv admin note: substantial text overlap with arXiv:1910.12575

For nonlinear supervised learning models, assessing the importance of predictor variables or their interactions is not straightforward because it can vary in the domain of the variables. Importance can be assessed locally with sensitivity analysis using general methods that rely on the model's predictions or their derivatives. In this work, we extend derivative based sensitivity analysis to a Bayesian setting by differentiating the R\'enyi divergence of a model's predictive distribution. By utilising the predictive distribution instead of a point prediction, the model uncertainty is taken into account in a principled way. Our empirical results on simulated and real data sets demonstrate accurate and reliable identification of important variables and interaction effects compared to alternative methods.