Search

A new golden age in astronomy is upon us, dominated by data. Large astronomical surveys are broadcasting unprecedented rates of information, demanding machine learning as a critical component in modern scientific pipelines to handle the deluge of data. The upcoming Legacy Survey of Space and Time (LSST) of the Vera C. Rubin Observatory will raise the big-data bar for time-domain astronomy, with an expected 10 million alerts per-night, and generating many petabytes of data over the lifetime of the survey. Fast and efficient classification algorithms that can operate in real-time, yet robustly and accurately, are needed for time-critical events where additional resources can be sought for follow-up analyses. In order to handle such data, state-of-the-art deep learning architectures coupled with tools that leverage modern hardware accelerators are essential. We showcase how the use of modern deep compression methods can achieve a $18\times$ reduction in model size, whilst preserving classification performance. We also show that in addition to the deep compression techniques, careful choice of file formats can improve inference latency, and thereby throughput of alerts, on the order of $8\times$ for local processing, and $5\times$ in a live production setting. To test this in a live setting, we deploy this optimised version of the original time-series transformer, t2, into the community alert broking system of FINK on real Zwicky Transient Facility (ZTF) alert data, and compare throughput performance with other science modules that exist in FINK. The results shown herein emphasise the time-series transformer's suitability for real-time classification at LSST scale, and beyond, and introduce deep model compression as a fundamental tool for improving deploy-ability and scalable inference of deep learning models for transient classification., Comment: 16 pages, 11 figures

Cosmic strings are linear topological defects that may have been produced during symmetry-breaking phase transitions in the very early Universe. In an expanding Universe the existence of causally separate regions prevents such symmetries from being broken uniformly, with a network of cosmic string inevitably forming as a result. To faithfully generate observables of such processes requires computationally expensive numerical simulations, which prohibits many types of analyses. We propose a technique to instead rapidly emulate observables, thus circumventing simulation. Emulation is a form of generative modelling, often built upon a machine learning backbone. End-to-end emulation often fails due to high dimensionality and insufficient training data. Consequently, it is common to instead emulate a latent representation from which observables may readily be synthesised. Wavelet phase harmonics are an excellent latent representations for cosmological fields, both as a summary statistic and for emulation, since they do not require training and are highly sensitive to non-Gaussian information. Leveraging wavelet phase harmonics as a latent representation, we develop techniques to emulate string induced CMB anisotropies over a 7.2 degree field of view, with sub-arcminute resolution, in under a minute on a single GPU. Beyond generating high fidelity emulations, we provide a technique to ensure these observables are distributed correctly, providing a more representative ensemble of samples. The statistics of our emulations are commensurate with those calculated on comprehensive Nambu-Goto simulations. Our findings indicate these fast emulation approaches may be suitable for wide use in, e.g., simulation based inference pipelines. We make our code available to the community so that researchers may rapidly emulate cosmic string induced CMB anisotropies for their own analysis.

Inspired by recent interest in geometric deep learning, this work generalises the recently developed Slepian scale-discretised wavelets on the sphere to Riemannian manifolds. Through the sifting convolution, one may define translations and, thus, convolutions on manifolds - which are otherwise not well-defined in general. Slepian wavelets are constructed on a region of a manifold and are therefore suited to problems where data only exists in a particular region. The Slepian functions, on which Slepian wavelets are built, are the basis functions of the Slepian spatial-spectral concentration problem on the manifold. A tiling of the Slepian harmonic line with smoothly decreasing generating functions defines the scale-discretised wavelets; allowing one to probe spatially localised, scale-dependent features of a signal. By discretising manifolds as graphs, the Slepian functions and wavelets of a triangular mesh are presented. Through a wavelet transform, the wavelet coefficients of a field defined on the mesh are found and used in a straightforward thresholding denoising scheme., Comment: 12 pages, 12 figures

No existing spherical convolutional neural network (CNN) framework is both computationally scalable and rotationally equivariant. Continuous approaches capture rotational equivariance but are often prohibitively computationally demanding. Discrete approaches offer more favorable computational performance but at the cost of equivariance. We develop a hybrid discrete-continuous (DISCO) group convolution that is simultaneously equivariant and computationally scalable to high-resolution. While our framework can be applied to any compact group, we specialize to the sphere. Our DISCO spherical convolutions exhibit $\text{SO}(3)$ rotational equivariance, where $\text{SO}(n)$ is the special orthogonal group representing rotations in $n$-dimensions. When restricting rotations of the convolution to the quotient space $\text{SO}(3)/\text{SO}(2)$ for further computational enhancements, we recover a form of asymptotic $\text{SO}(3)$ rotational equivariance. Through a sparse tensor implementation we achieve linear scaling in number of pixels on the sphere for both computational cost and memory usage. For 4k spherical images we realize a saving of $10^9$ in computational cost and $10^4$ in memory usage when compared to the most efficient alternative equivariant spherical convolution. We apply the DISCO spherical CNN framework to a number of benchmark dense-prediction problems on the sphere, such as semantic segmentation and depth estimation, on all of which we achieve the state-of-the-art performance., Comment: 19 pages, 7 figures, accepted by ICLR 2023

Inverse problems defined naturally on the sphere are becoming increasingly of interest. In this article we provide a general framework for evaluation of inverse problems on the sphere, with a strong emphasis on flexibility and scalability. We consider flexibility with respect to the prior selection (regularization), the problem definition - specifically the problem formulation (constrained/unconstrained) and problem setting (analysis/synthesis) - and optimization adopted to solve the problem. We discuss and quantify the trade-offs between problem formulation and setting. Crucially, we consider the Bayesian interpretation of the unconstrained problem which, combined with recent developments in probability density theory, permits rapid, statistically principled uncertainty quantification (UQ) in the spherical setting. Linearity is exploited to significantly increase the computational efficiency of such UQ techniques, which in some cases are shown to permit analytic solutions. We showcase this reconstruction framework and UQ techniques on a variety of spherical inverse problems. The code discussed throughout is provided under a GNU general public license, in both C++ and Python.

We resurrect the infamous harmonic mean estimator for computing the marginal likelihood (Bayesian evidence) and solve its problematic large variance. The marginal likelihood is a key component of Bayesian model selection since it is required to evaluate model posterior probabilities; however, its computation is challenging. The original harmonic mean estimator, first proposed in 1994 by Newton and Raftery, involves computing the harmonic mean of the likelihood given samples from the posterior. It was immediately realised that the original estimator can fail catastrophically since its variance can become very large and may not be finite. A number of variants of the harmonic mean estimator have been proposed to address this issue although none have proven fully satisfactory. We present the learnt harmonic mean estimator, a variant of the original estimator that solves its large variance problem. This is achieved by interpreting the harmonic mean estimator as importance sampling and introducing a new target distribution. The new target distribution is learned to approximate the optimal but inaccessible target, while minimising the variance of the resulting estimator. Since the estimator requires samples of the posterior only it is agnostic to the strategy used to generate posterior samples. We validate the estimator on a variety of numerical experiments, including a number of pathological examples where the original harmonic mean estimator fails catastrophically. In all cases our learnt harmonic mean estimator is shown to be highly accurate. The estimator is computationally scalable and can be applied to problems of dimension $\mathcal{O}(10^3)$ and beyond. Code implementing the learnt harmonic mean estimator is made publicly available., Comment: 37 pages, 8 figures, code available at https://github.com/astro-informatics/harmonic

Future surveys such as the Legacy Survey of Space and Time (LSST) of the Vera C. Rubin Observatory will observe an order of magnitude more astrophysical transient events than any previous survey before. With this deluge of photometric data, it will be impossible for all such events to be classified by humans alone. Recent efforts have sought to leverage machine learning methods to tackle the challenge of astronomical transient classification, with ever improving success. Transformers are a recently developed deep learning architecture, first proposed for natural language processing, that have shown a great deal of recent success. In this work we develop a new transformer architecture, which uses multi-head self attention at its core, for general multi-variate time-series data. Furthermore, the proposed time-series transformer architecture supports the inclusion of an arbitrary number of additional features, while also offering interpretability. We apply the time-series transformer to the task of photometric classification, minimising the reliance of expert domain knowledge for feature selection, while achieving results comparable to state-of-the-art photometric classification methods. We achieve a logarithmic-loss of 0.507 on imbalanced data in a representative setting using data from the Photometric LSST Astronomical Time-Series Classification Challenge (PLAsTiCC). Moreover, we achieve a micro-averaged receiver operating characteristic area under curve of 0.98 and micro-averaged precision-recall area under curve of 0.87., Comment: Revised argument in section 5.2 and 5.3, and minor change to title, results unchanged. 15 pages, 12 figures

Many problems across computer vision and the natural sciences require the analysis of spherical data, for which representations may be learned efficiently by encoding equivariance to rotational symmetries. We present a generalized spherical CNN framework that encompasses various existing approaches and allows them to be leveraged alongside each other. The only existing non-linear spherical CNN layer that is strictly equivariant has complexity $\mathcal{O}(C^2L^5)$, where $C$ is a measure of representational capacity and $L$ the spherical harmonic bandlimit. Such a high computational cost often prohibits the use of strictly equivariant spherical CNNs. We develop two new strictly equivariant layers with reduced complexity $\mathcal{O}(CL^4)$ and $\mathcal{O}(CL^3 \log L)$, making larger, more expressive models computationally feasible. Moreover, we adopt efficient sampling theory to achieve further computational savings. We show that these developments allow the construction of more expressive hybrid models that achieve state-of-the-art accuracy and parameter efficiency on spherical benchmark problems., Comment: 20 pages, 4 figures, accepted by ICLR, code at https://www.kagenova.com/products/fourpiAI/

We present a new algorithm to perform wide-field radio interferometric image reconstruction, with exact non-coplanar correction, that scales to big-data. This algorithm allows us to image 2 billion visibilities on 50 nodes of a computing cluster for a 25 by 25 degree field of view, in a little over an hour. We build on the recently developed distributed $w$-stacking $w$-projection hybrid algorithm, extending it to include a new distributed degridding algorithm that balances the computational load of the $w$-projection gridding kernels. The implementation of our algorithm is made publicly available in the PURIFY software package. Wide-field image reconstruction for data sets of this size cannot be performed effectively using the allocated computational resources without computational load balancing, demonstrating that our algorithms are critical for next-generation wide-field radio interferometers., Comment: 4 pages, 1 figure

This paper presents novel single and multi-shell sampling schemes for diffusion MRI. In diffusion MRI, it is paramount that the number of samples is as small as possible in order that scan times are practical in a clinical setting. The proposed schemes use an efficient number of measurements in that the number of samples is equal to the degrees of freedom in the orthonormal bases used for reconstruction. Novel reconstruction algorithms based on smaller subsystems of linear equations, as compared to the standard regularized least-squares method, are developed for both single and multi-shells sampling schemes. The smaller matrices used in these novel reconstruction algorithms are designed to be well-conditioned, leading to improved reconstruction accuracy. Accurate and robust reconstruction is also achieved through incorporation of regularization into the novel reconstruction algorithms and using a Rician or non-central Chi noise model. We quantitatively validate our single and multi-shell schemes against standard least-squares reconstruction methods to show that they enable more accurate reconstruction when the number of samples is equal to the degrees of freedom in the basis. Human brain data is also used to qualitatively evaluate reconstruction, Comment: 19 pages, with 6 pages supplementary material attached at the end, submitted to PLOS ONE

For the representation of spin-$s$ band-limited functions on the sphere, we propose a sampling scheme with optimal number of samples equal to the number of degrees of freedom of the function in harmonic space. In comparison to the existing sampling designs, which require ${\sim}2L^2$ samples for the representation of spin-$s$ functions band-limited at $L$, the proposed scheme requires $N_o=L^2-s^2$ samples for the accurate computation of the spin-$s$ spherical harmonic transform~($s$-SHT). For the proposed sampling scheme, we also develop a method to compute the $s$-SHT. We place the samples in our design scheme such that the matrices involved in the computation of $s$-SHT are well-conditioned. We also present a multi-pass $s$-SHT to improve the accuracy of the transform. We also show the proposed sampling design exhibits superior geometrical properties compared to existing equiangular and Gauss-Legendre sampling schemes, and enables accurate computation of the $s$-SHT corroborated through numerical experiments., Comment: 5 pages, 2 figures

We summarise the construction of exact axisymmetric scale-discretised wavelets on the sphere and on the ball. The wavelet transform on the ball relies on a novel 3D harmonic transform called the Fourier-Laguerre transform which combines the spherical harmonic transform with damped Laguerre polynomials on the radial half-line. The resulting wavelets, called flaglets, extract scale-dependent, spatially localised features in three-dimensions while treating the tangential and radial structures separately. Both the Fourier-Laguerre and the flaglet transforms are theoretically exact thanks to a novel sampling theorem on the ball. Our implementation of these methods is publicly available and achieves floating-point accuracy when applied to band-limited signals., Comment: 1 page, 1 figure, Proceedings of International BASP Frontiers Workshop 2013. Codes are publicly available at http://www.s2let.org and http://www.flaglets.org

We study the impact of the spread spectrum effect in radio interferometry on the quality of image reconstruction. This spread spectrum effect will be induced by the wide field-of-view of forthcoming radio interferometric telescopes. The resulting chirp modulation improves the quality of reconstructed interferometric images by increasing the incoherence of the measurement and sparsity dictionaries. We extend previous studies of this effect to consider the more realistic setting where the chirp modulation varies for each visibility measurement made by the telescope. In these first preliminary results, we show that for this setting the quality of reconstruction improves significantly over the case without chirp modulation and achieves almost the reconstruction quality of the case of maximal, constant chirp modulation., Comment: 1 page, 1 figure, Proceedings of the Biomedical and Astronomical Signal Processing Frontiers (BASP) workshop 2013

In the last decade diffusion MRI has become a powerful tool to non-invasively study white-matter integrity in the brain. Recently many research groups have focused their attention on multi-shell spherical acquisitions with the aim of effectively mapping the diffusion signal with a lower number of q-space samples, hence enabling a crucial reduction of acquisition time. One of the quantities commonly studied in this context is the so-called orientation distribution function (ODF). In this setting, the spherical harmonic (SH) transform has gained a great deal of popularity thanks to its ability to perform convolution operations efficiently and accurately, such as the Funk-Radon transform notably required for ODF computation from q-space data. However, if the q-space signal is described with an unsuitable angular resolution at any b-value probed, aliasing (or interpolation) artifacts are unavoidably created. So far this aspect has been tackled empirically and, to our knowledge, no study has addressed this problem in a quantitative approach. The aim of the present work is to study more theoretically the efficiency of multi-shell spherical sampling in diffusion MRI, in order to gain understanding in HYDI-like approaches, possibly paving the way to further optimization strategies., Comment: 1 page, 2 figures, 19th Annual Meeting of International Society for Magnetic Resonance in Medicine

A sampling theorem on the sphere has been developed recently, requiring half as many samples as alternative equiangular sampling theorems on the sphere. A reduction by a factor of two in the number of samples required to represent a band-limited signal on the sphere exactly has important implications for compressed sensing, both in terms of the dimensionality and sparsity of signals. We illustrate the impact of this property with an inpainting problem on the sphere, where we show the superior reconstruction performance when adopting the new sampling theorem compared to the alternative., Comment: 1 page, 2 figures, Signal Processing with Adaptive Sparse Structured Representations (SPARS) 2011