Search

We present the MULTIMODAL UNIVERSE, a large-scale multimodal dataset of scientific astronomical data, compiled specifically to facilitate machine learning research. Overall, the MULTIMODAL UNIVERSE contains hundreds of millions of astronomical observations, constituting 100\,TB of multi-channel and hyper-spectral images, spectra, multivariate time series, as well as a wide variety of associated scientific measurements and "metadata". In addition, we include a range of benchmark tasks representative of standard practices for machine learning methods in astrophysics. This massive dataset will enable the development of large multi-modal models specifically targeted towards scientific applications. All codes used to compile the MULTIMODAL UNIVERSE and a description of how to access the data is available at https://github.com/MultimodalUniverse/MultimodalUniverse, Comment: Accepted at NeurIPS Datasets and Benchmarks track

Feature-based methods are commonly used to explain model predictions, but these methods often implicitly assume that interpretable features are readily available. However, this is often not the case for high-dimensional data, and it can be hard even for domain experts to mathematically specify which features are important. Can we instead automatically extract collections or groups of features that are aligned with expert knowledge? To address this gap, we present FIX (Features Interpretable to eXperts), a benchmark for measuring how well a collection of features aligns with expert knowledge. In collaboration with domain experts, we propose FIXScore, a unified expert alignment measure applicable to diverse real-world settings across cosmology, psychology, and medicine domains in vision, language and time series data modalities. With FIXScore, we find that popular feature-based explanation methods have poor alignment with expert-specified knowledge, highlighting the need for new methods that can better identify features interpretable to experts.

The revolutionary discovery of dark energy and accelerating cosmic expansion was made with just 42 type Ia supernovae (SNe Ia) in 1999. Since then, large synoptic surveys, e.g., Dark Energy Survey (DES), have observed thousands more SNe Ia and the upcoming Rubin Legacy Survey of Space and Time (LSST) and Roman Space Telescope promise to deliver millions in the next decade. This unprecedented data volume can be used to test concordance cosmology. However, extracting a pure SN Ia sample with accurate redshifts for such a large dataset will be a challenge. Spectroscopic classification will not be possible for the vast majority of discovered objects, and only 25% will have spectroscopic redshifts. This thesis presents a series of observational and methodological studies designed to address the questions associated with this new era of photometric SN Ia cosmology. First, we present a machine learning (ML) method for SN photometric classification, SCONE. Photometric classification enables SNe with no spectroscopic information to be categorized, a critical step for cosmological analysis. SCONE achieves 99+% accuracy distinguishing simulated SNe Ia from non-Ia SNe, and is a part of DES, LSST, and Roman analysis pipelines. We also show that SCONE can classify 6 SN types with 75% accuracy on the night of initial discovery, comparable to results in the literature for full-phase SNe. Next, we study current methods for estimating SN Ia redshifts and propose an ML alternative that uses SN photometry alone to extract redshift information. Photo-zSNthesis is a host galaxy-independent redshift estimator accurate to within 2% across the redshift range of LSST, a first in the literature. Finally, we focus on ML robustness and demonstrate a general method for improving robustness that achieves new state-of-the-art results on astronomical object classification, wildlife identification, and tumor detection., Comment: PhD Dissertation

Models trained on a labeled source domain (e.g., labeled images from wildlife camera traps) often generalize poorly when deployed on an out-of-distribution (OOD) target domain (e.g., images from new camera trap locations). In the domain adaptation setting where unlabeled target data is available, self-supervised pretraining (e.g., masked autoencoding or contrastive learning) is a promising method to mitigate this performance drop. Pretraining improves OOD error when the generic data augmentations used (e.g., masking or cropping) connect the source and target domains, which may be far apart in the input space. In this paper, we show on real-world tasks that standard fine-tuning after pretraining does not consistently improve OOD error over simply training from scratch on labeled source data. To better leverage pretraining for distribution shifts, we propose Connect Later: after pretraining with generic augmentations, fine-tune with targeted augmentations designed with knowledge of the distribution shift. Pretraining learns good representations within the source and target domains, while targeted augmentations connect the domains better during fine-tuning. Connect Later improves average OOD error over standard fine-tuning and supervised learning with targeted augmentations on 4 real-world datasets: Connect Later achieves the state-of-the-art on astronomical time-series classification (AstroClassification) by 2.5%, wildlife species identification (iWildCam-WILDS) with ResNet-50 by 0.9%, and tumor identification (Camelyon17-WILDS) with DenseNet121 by 1.1%; as well as best performance on a new dataset for astronomical time-series redshift prediction (Redshifts) by 0.03 RMSE (11% relative). Code and datasets are available at https://github.com/helenqu/connect-later., Comment: ICML 2024

Feature attributions explain machine learning predictions by assigning importance scores to input features. While faithful attributions accurately reflect feature contributions to the model's prediction, unfaithful ones can lead to misleading interpretations, making them unreliable in high-stake domains. The challenge of unfaithfulness of post-hoc attributions led to the development of self-explaining models. However, self-explaining models often trade-off performance for interpretability. In this work, we develop Sum-of-Parts (SOP), a new framework that transforms any differentiable model into a self-explaining model whose predictions can be attributed to groups of features. The SOP framework leverages pretrained deep learning models with custom attention modules to learn useful feature groups end-to-end without direct supervision. With these capabilities, SOP achieves highest performance while also scoring high with respect to faithfulness metrics on both ImageNet and CosmoGrid. We validate the usefulness of the groups learned by SOP through their high purity, strong human distinction ability, and practical utility in scientific discovery. In a case study, we show how SOP assists cosmologists in uncovering new insights about galaxy formation.

The deep learning architecture associated with ChatGPT and related generative AI products is known as transformers. Initially applied to Natural Language Processing, transformers and the self-attention mechanism they exploit have gained widespread interest across the natural sciences. The goal of this pedagogical and informal review is to introduce transformers to scientists. The review includes the mathematics underlying the attention mechanism, a description of the original transformer architecture, and a section on applications to time series and imaging data in astronomy. We include a Frequently Asked Questions section for readers who are curious about generative AI or interested in getting started with transformers for their research problem., Comment: 17 pages, 5 figures

Upcoming photometric surveys will discover tens of thousands of Type Ia supernovae (SNe Ia), vastly outpacing the capacity of our spectroscopic resources. In order to maximize the science return of these observations in the absence of spectroscopic information, we must accurately extract key parameters, such as SN redshifts, with photometric information alone. We present Photo-zSNthesis, a convolutional neural network-based method for predicting full redshift probability distributions from multi-band supernova lightcurves, tested on both simulated Sloan Digital Sky Survey (SDSS) and Vera C. Rubin Legacy Survey of Space and Time (LSST) data as well as observed SDSS SNe. We show major improvements over predictions from existing methods on both simulations and real observations as well as minimal redshift-dependent bias, which is a challenge due to selection effects, e.g. Malmquist bias. Specifically, we show a 61x improvement in prediction bias on PLAsTiCC simulations and 5x improvement on real SDSS data compared to results from a widely used photometric redshift estimator, LCFIT+Z. The PDFs produced by this method are well-constrained and will maximize the cosmological constraining power of photometric SNe Ia samples., Comment: Accepted by ApJ

One of the brightest objects in the universe, supernovae (SNe) are powerful explosions marking the end of a star's lifetime. Supernova (SN) type is defined by spectroscopic emission lines, but obtaining spectroscopy is often logistically unfeasible. Thus, the ability to identify SNe by type using time-series image data alone is crucial, especially in light of the increasing breadth and depth of upcoming telescopes. We present a convolutional neural network method for fast supernova time-series classification, with observed brightness data smoothed in both the wavelength and time directions with Gaussian process regression. We apply this method to full duration and truncated SN time-series, to simulate retrospective as well as real-time classification performance. Retrospective classification is used to differentiate cosmologically useful Type Ia SNe from other SN types, and this method achieves >99% accuracy on this task. We are also able to differentiate between 6 SN types with 60% accuracy given only two nights of data and 98% accuracy retrospectively., Comment: Accepted at the ICML 2022 Workshop on Machine Learning for Astrophysics

We present a study of the potential for Convolutional Neural Networks (CNNs) to enable separation of astrophysical transients from image artifacts, a task known as "real-bogus" classification without requiring a template subtracted (or difference) image which requires a computationally expensive process to generate, involving image matching on small spatial scales in large volumes of data. Using data from the Dark Energy Survey, we explore the use of CNNs to (1) automate the "real-bogus" classification, (2) reduce the computational costs of transient discovery. We compare the efficiency of two CNNs with similar architectures, one that uses "image triplets" (templates, search, and difference image) and one that takes as input the template and search only. We measure the decrease in efficiency associated with the loss of information in input finding that the testing accuracy is reduced from 96% to 91.1%. We further investigate how the latter model learns the required information from the template and search by exploring the saliency maps. Our work (1) confirms that CNNs are excellent models for "real-bogus" classification that rely exclusively on the imaging data and require no feature engineering task; (2) demonstrates that high-accuracy (> 90%) models can be built without the need to construct difference images, but some accuracy is lost. Since once trained, neural networks can generate predictions at minimal computational costs, we argue that future implementations of this methodology could dramatically reduce the computational costs in the detection of transients in synoptic surveys like Rubin Observatory's Legacy Survey of Space and Time by bypassing the Difference Image Analysis entirely.

We present constraints on cosmological parameters from the Pantheon+ analysis of 1701 light curves of 1550 distinct Type Ia supernovae (SNe Ia) ranging in redshift from $z=0.001$ to 2.26. This work features an increased sample size, increased redshift span, and improved treatment of systematic uncertainties in comparison to the original Pantheon analysis and results in a factor of two improvement in cosmological constraining power. For a Flat$\Lambda$CDM model, we find $\Omega_M=0.334\pm0.018$ from SNe Ia alone. For a Flat$w_0$CDM model, we measure $w_0=-0.90\pm0.14$ from SNe Ia alone, H$_0=73.5\pm1.1$ km s$^{-1}$ Mpc$^{-1}$ when including the Cepheid host distances and covariance (SH0ES), and $w_0=-0.978^{+0.024}_{-0.031}$ when combining the SN likelihood with constraints from the cosmic microwave background (CMB) and baryon acoustic oscillations (BAO); both $w_0$ values are consistent with a cosmological constant. We also present the most precise measurements to date on the evolution of dark energy in a Flat$w_0w_a$CDM universe, and measure $w_a=-0.1^{+0.9}_{-2.0}$ from Pantheon+ alone, H$_0=73.3\pm1.1$ km s$^{-1}$ Mpc$^{-1}$ when including SH0ES, and $w_a=-0.65^{+0.28}_{-0.32}$ when combining Pantheon+ with CMB and BAO data. Finally, we find that systematic uncertainties in the use of SNe Ia along the distance ladder comprise less than one third of the total uncertainty in the measurement of H$_0$ and cannot explain the present "Hubble tension" between local measurements and early-Universe predictions from the cosmological model., Comment: 34 Pages, 16 Figures, 7 Tables. Published in ApJ. Comments welcome. Papers and data release here: https://pantheonplussh0es.github.io

We present here a re-calibration of the photometric systems used in the Pantheon+ sample of Type Ia supernovae (SNe Ia) including those used for the SH0ES distance-ladder measurement of H$_0$. We utilize the large and uniform sky coverage of the public Pan-STARRS stellar photometry catalog to cross-calibrate against tertiary standards released by individual SN Ia surveys. The most significant updates over the `SuperCal' cross-calibration used for the previous Pantheon and SH0ES analyses are: 1) expansion of the number of photometric systems (now 25) and filters (now 105), 2) solving for all filter offsets in all systems simultaneously in order to produce a calibration uncertainty covariance matrix that can be used in cosmological-model constraints, and 3) accounting for the change in the fundamental flux calibration of the HST CALSPEC standards from previous versions on the order of $1.5\%$ over a $\Delta \lambda$ of 4000~\AA. The re-calibration of samples used for light-curve fitting has historically been decoupled from the retraining of the light-curve model. Here, we are able to retrain the SALT2 model using this new calibration and find the change in the model coupled with the change to the calibration of the light-curves themselves causes a net distance modulus change ($d\mu/dz$) of 0.04 mag over the redshift range $0

In this work, we present classification results on early supernova lightcurves from SCONE, a photometric classifier that uses convolutional neural networks to categorize supernovae (SNe) by type using lightcurve data. SCONE is able to identify SN types from lightcurves at any stage, from the night of initial alert to the end of their lifetimes. Simulated LSST SNe lightcurves were truncated at 0, 5, 15, 25, and 50 days after the trigger date and used to train Gaussian processes in wavelength and time space to produce wavelength-time heatmaps. SCONE uses these heatmaps to perform 6-way classification between SN types Ia, II, Ibc, Ia-91bg, Iax, and SLSN-I. SCONE is able to perform classification with or without redshift, but we show that incorporating redshift information improves performance at each epoch. SCONE achieved 75% overall accuracy at the date of trigger (60% without redshift), and 89% accuracy 50 days after trigger (82% without redshift). SCONE was also tested on bright subsets of SNe (r<20 mag) and produced 91% accuracy at the date of trigger (83% without redshift) and 95% 5 days after trigger (94.7% without redshift). SCONE is the first application of convolutional neural networks to the early-time photometric transient classification problem. All of the data processing and model code developed for this paper can be found in the SCONE software package located at github.com/helenqu/scone., Comment: submitted to AJ, code available at https://github.com/helenqu/scone

We present a novel method of classifying Type Ia supernovae using convolutional neural networks, a neural network framework typically used for image recognition. Our model is trained on photometric information only, eliminating the need for accurate redshift data. Photometric data is pre-processed via 2D Gaussian process regression into two-dimensional images created from flux values at each location in wavelength-time space. These "flux heatmaps" of each supernova detection, along with "uncertainty heatmaps" of the Gaussian process uncertainty, constitute the dataset for our model. This preprocessing step not only smooths over irregular sampling rates between filters but also allows SCONE to be independent of the filter set on which it was trained. Our model has achieved impressive performance without redshift on the in-distribution SNIa classification problem: $99.73 \pm 0.26$% test accuracy with no over/underfitting on a subset of supernovae from PLAsTiCC's unblinded test dataset. We have also achieved $98.18 \pm 0.3$% test accuracy performing 6-way classification of supernovae by type. The out-of-distribution performance does not fully match the in-distribution results, suggesting that the detailed characteristics of the training sample in comparison to the test sample have a big impact on the performance. We discuss the implication and directions for future work. All of the data processing and model code developed for this paper can be found in the SCONE software package located at github.com/helenqu/scone., Comment: Accepted to AJ, all code available at https://github.com/helenqu/scone