1. Anomaly Detection and Approximate Similarity Searches of Transients in Real-time Data Streams
- Author
-
Aleo, P. D., Engel, A. W., Narayan, G., Angus, C. R., Malanchev, K., Auchettl, K., Baldassare, V. F., Berres, A., de Boer, T. J. L., Boyd, B. M., Chambers, K. C., Davis, K. W., Esquivel, N., Farias, D., Foley, R. J., Gagliano, A., Gall, C., Gao, H., Gomez, S., Grayling, M., Jones, D. O., Lin, C. -C., Magnier, E. A., Mandel, K. S., Matheson, T., Raimundo, S. I., Shah, V. G., Soraisam, M. D., de Soto, K. M., Vicencio, S., Villar, V. A., and Wainscoat, R. J.
- Subjects
Astrophysics - High Energy Astrophysical Phenomena ,Astrophysics - Instrumentation and Methods for Astrophysics - Abstract
We present LAISS (Lightcurve Anomaly Identification and Similarity Search), an automated pipeline to detect anomalous astrophysical transients in real-time data streams. We deploy our anomaly detection model on the nightly ZTF Alert Stream via the ANTARES broker, identifying a manageable $\sim$1-5 candidates per night for expert vetting and coordinating follow-up observations. Our method leverages statistical light-curve and contextual host-galaxy features within a random forest classifier, tagging transients of rare classes (spectroscopic anomalies), of uncommon host-galaxy environments (contextual anomalies), and of peculiar or interaction-powered phenomena (behavioral anomalies). Moreover, we demonstrate the power of a low-latency ($\sim$ms) approximate similarity search method to find transient analogs with similar light-curve evolution and host-galaxy environments. We use analogs for data-driven discovery, characterization, (re-)classification, and imputation in retrospective and real-time searches. To date we have identified $\sim$50 previously known and previously missed rare transients from real-time and retrospective searches, including but not limited to: SLSNe, TDEs, SNe IIn, SNe IIb, SNe Ia-CSM, SNe Ia-91bg-like, SNe Ib, SNe Ic, SNe Ic-BL, and M31 novae. Lastly, we report the discovery of 325 total transients, all observed between 2018-2021 and absent from public catalogs ($\sim$1% of all ZTF Astronomical Transient reports to the Transient Name Server through 2021). These methods enable a systematic approach to finding the "needle in the haystack" in large-volume data streams. Because of its integration with the ANTARES broker, LAISS is built to detect exciting transients in Rubin data., Comment: 44 pages (68 pages with Appendix), 15 figures, accepted to ApJ
- Published
- 2024