Search

The precise characterization and mitigation of systematic effects is one of the biggest roadblocks impeding the detection of the fluctuations of cosmological 21cm signals. Missing data in radio cosmological experiments, often due to radio frequency interference (RFI), poses a particular challenge to power spectrum analysis as it could lead to the ringing of bright foreground modes in Fourier space, heavily contaminating the cosmological signals. Here we show that the problem of missing data becomes even more arduous in the presence of systematic effects. Using a realistic numerical simulation, we demonstrate that partially flagged data combined with systematic effects can introduce significant foreground ringing. We show that such an effect can be mitigated through inpainting the missing data. We present a rigorous statistical framework that incorporates the process of inpainting missing data into a quadratic estimator of the 21cm power spectrum. Under this framework, the uncertainties associated with our inpainting method and its impact on power spectrum statistics can be understood. These results are applied to the latest Phase II observations taken by the Hydrogen Epoch of Reionization Array, forming a crucial component in power spectrum analyses as we move toward detecting 21cm signals in the ever more noisy RFI environment., Comment: 25 pages, 11 figures, submitted to ApJ

Radio interferometers targeting the 21cm brightness temperature fluctuations at high redshift are subject to systematic effects that operate over a range of different timescales. These can be isolated by designing appropriate Fourier filters that operate in fringe-rate (FR) space, the Fourier pair of local sidereal time (LST). Applications of FR filtering include separating effects that are correlated with the rotating sky vs. those relative to the ground, down-weighting emission in the primary beam sidelobes, and suppressing noise. FR filtering causes the noise contributions to the visibility data to become correlated in time however, making interpretation of subsequent averaging and error estimation steps more subtle. In this paper, we describe fringe rate filters that are implemented using discrete prolate spheroidal sequences, and designed for two different purposes -- beam sidelobe/horizon suppression (the `mainlobe' filter), and ground-locked systematics removal (the `notch' filter). We apply these to simulated data, and study how their properties affect visibilities and power spectra generated from the simulations. Included is an introduction to fringe-rate filtering and a demonstration of fringe-rate filters applied to simple situations to aid understanding., Comment: 21 pages, 18 figures, submitted to Monthly Notices of the Royal Astronomical Society

This paper presents the design and deployment of the Hydrogen Epoch of Reionization Array (HERA) phase II system. HERA is designed as a staged experiment targeting 21 cm emission measurements of the Epoch of Reionization. First results from the phase I array are published as of early 2022, and deployment of the phase II system is nearing completion. We describe the design of the phase II system and discuss progress on commissioning and future upgrades. As HERA is a designated Square Kilometer Array (SKA) pathfinder instrument, we also show a number of "case studies" that investigate systematics seen while commissioning the phase II system, which may be of use in the design and operation of future arrays. Common pathologies are likely to manifest in similar ways across instruments, and many of these sources of contamination can be mitigated once the source is identified.

Detection of the faint 21 cm line emission from the Cosmic Dawn and Epoch of Reionisation will require not only exquisite control over instrumental calibration and systematics to achieve the necessary dynamic range of observations but also validation of analysis techniques to demonstrate their statistical properties and signal loss characteristics. A key ingredient in achieving this is the ability to perform high-fidelity simulations of the kinds of data that are produced by the large, many-element, radio interferometric arrays that have been purpose-built for these studies. The large scale of these arrays presents a computational challenge, as one must simulate a detailed sky and instrumental model across many hundreds of frequency channels, thousands of time samples, and tens of thousands of baselines for arrays with hundreds of antennas. In this paper, we present a fast matrix-based method for simulating radio interferometric measurements (visibilities) at the necessary scale. We achieve this through judicious use of primary beam interpolation, fast approximations for coordinate transforms, and a vectorised outer product to expand per-antenna quantities to per-baseline visibilities, coupled with standard parallelisation techniques. We validate the results of this method, implemented in the publicly-available matvis code, against a high-precision reference simulator, and explore its computational scaling on a variety of problems., Comment: 25 pages, 20 figures, submitted to RAS Techniques and Instruments, matvis is publicly available at https://github.com/HERA-Team/matvis

Observations with radio arrays that target the 21-cm signal originating from the early Universe suffer from a variety of systematic effects. An important class of these are reflections and spurious couplings between antennas. We apply a Hamiltonian Monte Carlo sampler to the modelling and mitigation of these systematics in simulated Hydrogen Epoch of Reionisation Array (HERA) data. This method allows us to form statistical uncertainty estimates for both our models and the recovered visibilities, which is an important ingredient in establishing robust upper limits on the Epoch of Reionisation (EoR) power spectrum. In cases where the noise is large compared to the EoR signal, this approach can constrain the systematics well enough to mitigate them down to the noise level for both systematics studied. Where the noise is smaller than the EoR, our modelling can mitigate the majority of the reflections with there being only a minor level of residual systematics, while cross-coupling sees essentially complete mitigation. Our approach performs similarly to existing filtering/fitting techniques used in the HERA pipeline, but with the added benefit of rigorously propagating uncertainties. In all cases it does not significantly attenuate the underlying signal., Comment: 19 pages, 14 figures, submitted to MNRAS

The key to detecting neutral hydrogen during the epoch of reionization (EoR) is to separate the cosmological signal from the dominating foreground radiation. We developed direct optimal mapping (DOM) to map interferometric visibilities; it contains only linear operations, with full knowledge of point spread functions from visibilities to images. Here, we demonstrate a fast Fourier transform-based image power spectrum and its window functions computed from the DOM images. We use noiseless simulation, based on the Hydrogen Epoch of Reionization Array Phase I configuration, to study the image power spectrum properties. The window functions show $<10^{-11}$ of the integrated power leaks from the foreground-dominated region into the EoR window; the 2D and 1D power spectra also verify the separation between the foregrounds and the EoR., Comment: Published in ApJ

We present deep upper limits from the 2014 Murchison Widefield Array (MWA) Phase I observing season, with a particular emphasis on identifying the spectral fingerprints of extremely faint radio frequency interference (RFI) contamination in the 21~cm power spectra (PS). After meticulous RFI excision involving a combination of the \textsc{SSINS} RFI flagger and a series of PS-based jackknife tests, our lowest upper limit on the Epoch of Reionization (EoR) 21~cm PS signal is $\Delta^2 \leq 1.61\cdot10^4 \text{ mK}^2$ at $k=0.258\text{ h Mpc}^{-1}$ at a redshift of 7.1 using 14.7 hours of data. By leveraging our understanding of how even fainter RFI is likely to contaminate the EoR PS, we are able to identify ultra-faint RFI signals in the cylindrical PS. Surprisingly this signature is most obvious in PS formed with less than an hour of data, but is potentially subdominant to other systematics in multiple-hour integrations. Since the total RFI budget in a PS detection is quite strict, this nontrivial integration behavior suggests a need to more realistically model coherently integrated ultra-faint RFI in PS measurements so that its potential contribution to a future detection can be diagnosed., Comment: Update acknowledgements, author metadata, and attach journal doi

Radio interferometers aiming to measure the power spectrum of the redshifted 21 cm line during the Epoch of Reionisation (EoR) need to achieve an unprecedented dynamic range to separate the weak signal from overwhelming foreground emissions. Calibration inaccuracies can compromise the sensitivity of these measurements to the effect that a detection of the EoR is precluded. An alternative to standard analysis techniques makes use of the closure phase, which allows one to bypass antenna-based direction-independent calibration. Similarly to standard approaches, we use a delay spectrum technique to search for the EoR signal. Using 94 nights of data observed with Phase I of the Hydrogen Epoch of Reionization Array (HERA), we place approximate constraints on the 21 cm power spectrum at $z=7.7$. We find at 95% confidence that the 21 cm EoR brightness temperature is $\le$(372)$^2$ "pseudo" mK$^2$ at 1.14 "pseudo" $h$ Mpc$^{-1}$, where the "pseudo" emphasises that these limits are to be interpreted as approximations to the actual distance scales and brightness temperatures. Using a fiducial EoR model, we demonstrate the feasibility of detecting the EoR with the full array. Compared to standard methods, the closure phase processing is relatively simple, thereby providing an important independent check on results derived using visibility intensities, or related., Comment: 16 pages, 14 figures, accepted for publication by MNRAS

Through a very careful analysis Kolopanis and collaborators identified a negative power spectrum (PS) systematic. The 21 cm cosmology community has assumed that any observational systematics would add power, as negative PS are non-physical. In addition to the mystery of their origin, negative PS systematics raise the spectre of artificially lowering upper limits on the 21 cm PS. It appears that the source of the negative PS systematics is a subtle interaction between choices in how the PS estimate is calculated and baseline-dependent systematic power. In this paper we present a statistical model of baseline dependent systematics to explore how negative PS systematics can appear and their statistical characteristics. This leads us to recommendations on when and how to consider negative PS systematics when reporting observational 21 cm cosmology upper limits., Comment: Updated to match accepted paper in MNRAS

Radio Frequency Interference (RFI) is one of the systematic challenges preventing 21cm interferometric instruments from detecting the Epoch of Reionization. To mitigate the effects of RFI on data analysis pipelines, numerous inpaint techniques have been developed to restore RFI corrupted data. We examine the qualitative and quantitative errors introduced into the visibilities and power spectrum due to inpainting. We perform our analysis on simulated data as well as real data from the Hydrogen Epoch of Reionization Array (HERA) Phase 1 upper limits. We also introduce a convolutional neural network that capable of inpainting RFI corrupted data in interferometric instruments. We train our network on simulated data and show that our network is capable at inpainting real data without requiring to be retrained. We find that techniques that incorporate high wavenumbers in delay space in their modeling are best suited for inpainting over narrowband RFI. We also show that with our fiducial parameters Discrete Prolate Spheroidal Sequences (DPSS) and CLEAN provide the best performance for intermittent ``narrowband'' RFI while Gaussian Progress Regression (GPR) and Least Squares Spectral Analysis (LSSA) provide the best performance for larger RFI gaps. However we caution that these qualitative conclusions are sensitive to the chosen hyperparameters of each inpainting technique. We find these results to be consistent in both simulated and real visibilities. We show that all inpainting techniques reliably reproduce foreground dominated modes in the power spectrum. Since the inpainting techniques should not be capable of reproducing noise realizations, we find that the largest errors occur in the noise dominated delay modes. We show that in the future, as the noise level of the data comes down, CLEAN and DPSS are most capable of reproducing the fine frequency structure in the visibilities of HERA data., Comment: 21 pages, 13 figures

We report the most sensitive upper limits to date on the 21 cm epoch of reionization power spectrum using 94 nights of observing with Phase I of the Hydrogen Epoch of Reionization Array (HERA). Using similar analysis techniques as in previously reported limits (HERA Collaboration 2022a), we find at 95% confidence that $\Delta^2(k = 0.34$ $h$ Mpc$^{-1}$) $\leq 457$ mK$^2$ at $z = 7.9$ and that $\Delta^2 (k = 0.36$ $h$ Mpc$^{-1}) \leq 3,496$ mK$^2$ at $z = 10.4$, an improvement by a factor of 2.1 and 2.6 respectively. These limits are mostly consistent with thermal noise over a wide range of $k$ after our data quality cuts, despite performing a relatively conservative analysis designed to minimize signal loss. Our results are validated with both statistical tests on the data and end-to-end pipeline simulations. We also report updated constraints on the astrophysics of reionization and the cosmic dawn. Using multiple independent modeling and inference techniques previously employed by HERA Collaboration (2022b), we find that the intergalactic medium must have been heated above the adiabatic cooling limit at least as early as $z = 10.4$, ruling out a broad set of so-called "cold reionization" scenarios. If this heating is due to high-mass X-ray binaries during the cosmic dawn, as is generally believed, our result's 99% credible interval excludes the local relationship between soft X-ray luminosity and star formation and thus requires heating driven by evolved low-metallicity stars., Comment: 57 pages, 37 figures. Updated to match the accepted ApJ version. Corresponding author: Joshua S. Dillon

Combining the visibilities measured by an interferometer to form a cosmological power spectrum is a complicated process. In a delay-based analysis, the mapping between instrumental and cosmological space is not a one-to-one relation. Instead, neighbouring modes contribute to the power measured at one point, with their respective contributions encoded in the window functions. To better understand the power measured by an interferometer, we assess the impact of instrument characteristics and analysis choices on these window functions. Focusing on the Hydrogen Epoch of Reionization Array (HERA) as a case study, we find that long-baseline observations correspond to enhanced low-k tails of the window functions, which facilitate foreground leakage, whilst an informed choice of bandwidth and frequency taper can reduce said tails. With simple test cases and realistic simulations, we show that, apart from tracing mode mixing, the window functions help accurately reconstruct the power spectrum estimator of simulated visibilities. The window functions depend strongly on the beam chromaticity, and less on its spatial structure - a Gaussian approximation, ignoring side lobes, is sufficient. Finally, we investigate the potential of asymmetric window functions, down-weighting the contribution of low-k power to avoid foreground leakage. The window functions presented here correspond to the latest HERA upper limits for the full Phase I data. They allow an accurate reconstruction of the power spectrum measured by the instrument and will be used in future analyses to confront theoretical models and data directly in cylindrical space., Comment: 18 pages, 19 figures, accepted for publication in MNRAS

The Multi-frequency Angular Power Spectrum (MAPS) is an alternative to spherically-averaged power spectra, and computes local fluctuations in the angular power spectrum without need for line-of-sight spectral transform. To test different approaches to MAPS and treatment of the foreground contamination, and compare with the spherically-averaged power spectrum, and the single-frequency angular power spectrum. We apply the MAPS to 110~hours of data in $z=6.2-7.5$ obtained for the Murchison Widefield Array Epoch of Reionisation experiment to compute the statistical power of 21~cm brightness temperature fluctuations. In the presence of bright foregrounds, a filter is applied to remove large-scale modes prior to MAPS application, significantly reducing MAPS power due to systematics. The MAPS shows a contrast of 10$^2$--10$^3$ to a simulated 21~cm cosmological signal for spectral separations of 0--4~MHz after application of the filter, reflecting results for the spherically-averaged power spectrum. The single-frequency angular power spectrum is also computed. At $z=7.5$ and $l=200$, we find an angular power of 53~mK$^2$, exceeding a simulated cosmological signal power by a factor of one thousand. Residual spectral structure, inherent to the calibrated data, and not spectral leakage from large-scale modes, is the dominant source of systematic power bias. The single-frequency angular power spectrum yields slightly poorer results compared with the spherically-averaged power spectrum, having applied a spectral filter to reduce foregrounds. Exploration of other filters may improve this result, along with consideration of wider bandwidths., Comment: 9 pages, 9 figures; accepted for publication in Astronomy & Astrophysics

Motivated by the desire for wide-field images with well-defined statistical properties for 21cm cosmology, we implement an optimal mapping pipeline that computes a maximum likelihood estimator for the sky using the interferometric measurement equation. We demonstrate this direct optimal mapping with data from the Hydrogen Epoch of Reionization (HERA) Phase I observations. After validating the pipeline with simulated data, we develop a maximum likelihood figure-of-merit for comparing four sky models at 166MHz with a bandwidth of 100kHz. The HERA data agree with the GLEAM catalogs to <10%. After subtracting the GLEAM point sources, the HERA data discriminate between the different continuum sky models, providing most support for the model of Byrne et al. 2021. We report the computation cost for mapping the HERA Phase I data and project the computation for the HERA 320-antenna data; both are feasible with a modern server. The algorithm is broadly applicable to other interferometers and is valid for wide-field and non-coplanar arrays., Comment: 16 pages, 10 figures, 2 tables, published on ApJ

We describe a new polarized imaging pipeline implemented in the FHD software package. The pipeline is based on the optimal mapmaking imaging approach and performs horizon-to-horizon image reconstruction in all polarization modes. We discuss the formalism behind the pipeline's polarized analysis, describing equivalent representations of the polarized beam response, or Jones matrix. We show that, for arrays where antennas have uniform polarization alignments, defining a non-orthogonal instrumental polarization basis enables accurate and efficient image reconstruction. Finally, we present a new calibration approach that leverages widefield effects to perform fully-polarized calibration. This analysis pipeline underlies the analysis of Murchison Widefield Array (MWA) data in Byrne et al. (2022, MNRAS, 510, 2011)., Comment: Published in PASA. The updated version includes an example of image reconstruction of a simulated polarized source

We explore how chromatic RFI flags affect 21-cm power spectrum measurements. We particularly study flags that are coarser than the analysis resolution. We find that such RFI flags produce excess power in the EoR window in much the same way as residual RFI. We use Fast Holographic Deconvolution (FHD) simulations to explain this as a result of chromatic disruptions in the interferometric sampling function of the array. We also use these simulations in conjunction with Error Propagated Power Spectrum with InterLeaved Observed Noise ($\varepsilon$ppsilon) to show that without modifying current flagging strategies or implementing extremely accurate and complete foreground subtraction, 21-cm EoR experiments will fail to make a significant detection. As a mitigation strategy, we find that circumventing the chromatic structure altogether by flagging the entire analysis band when RFI is detected is simple to implement and highly successful. This demands a detection strategy with a low false positive rate in order to prevent excessive data loss., Comment: Including revisions from peer review

We present a framework for identifying and flagging malfunctioning antennas in large radio interferometers. We outline two distinct categories of metrics designed to detect outliers along known failure modes of large arrays: cross-correlation metrics, based on all antenna pairs, and auto-correlation metrics, based solely on individual antennas. We define and motivate the statistical framework for all metrics used, and present tailored visualizations that aid us in clearly identifying new and existing systematics. We implement these techniques using data from 105 antennas in the Hydrogen Epoch of Reionization Array (HERA) as a case study. Finally, we provide a detailed algorithm for implementing these metrics as flagging tools on real data sets., Comment: 31 pages, 17 figures

We report upper-limits on the Epoch of Reionization (EoR) 21 cm power spectrum at redshifts 7.9 and 10.4 with 18 nights of data ($\sim36$ hours of integration) from Phase I of the Hydrogen Epoch of Reionization Array (HERA). The Phase I data show evidence for systematics that can be largely suppressed with systematic models down to a dynamic range of $\sim10^9$ with respect to the peak foreground power. This yields a 95% confidence upper limit on the 21 cm power spectrum of $\Delta^2_{21} \le (30.76)^2\ {\rm mK}^2$ at $k=0.192\ h\ {\rm Mpc}^{-1}$ at $z=7.9$, and also $\Delta^2_{21} \le (95.74)^2\ {\rm mK}^2$ at $k=0.256\ h\ {\rm Mpc}^{-1}$ at $z=10.4$. At $z=7.9$, these limits are the most sensitive to-date by over an order of magnitude. While we find evidence for residual systematics at low line-of-sight Fourier $k_\parallel$ modes, at high $k_\parallel$ modes we find our data to be largely consistent with thermal noise, an indicator that the system could benefit from deeper integrations. The observed systematics could be due to radio frequency interference, cable sub-reflections, or residual instrumental cross-coupling, and warrant further study. This analysis emphasizes algorithms that have minimal inherent signal loss, although we do perform a careful accounting in a companion paper of the small forms of loss or bias associated with the pipeline. Overall, these results are a promising first step in the development of a tuned, instrument-specific analysis pipeline for HERA, particularly as Phase II construction is completed en route to reaching the full sensitivity of the experiment., Comment: Accepted to ApJ. https://reionization.org/science/public-data-release-1/

We present a broadband map of polarized diffuse emission at 167-198 MHz developed from data from the Murchison Widefield Array (MWA). The map is designed to improve visibility simulation and precision calibration for 21 cm Epoch of Reionization (EoR) experiments. It covers a large swath - 11,000 sq. deg. - of the Southern Hemisphere sky in all four Stokes parameters and captures emission on angular scales of 1 to 9 degrees. The band-averaged diffuse structure is predominantly unpolarized but has significant linearly polarized structure near RA = 0 h. We evaluate the accuracy of the map by combining it with the GLEAM catalog and simulating an observation from the MWA, demonstrating that the accuracy of the short baselines (6.1-50 wavelengths) now approaches the accuracy of the longer baselines typically used for EoR calibration. We discuss how to use the map for visibility simulation for a variety of interferometric arrays. The map has potential to improve calibration accuracy for experiments such as the Hydrogen Epoch of Reionization Array (HERA) and the forthcoming Square Kilometre Array (SKA) as well as the MWA., Comment: Accepted for publication in MNRAS

Precision calibration poses challenges to experiments probing the redshifted 21-cm signal of neutral hydrogen from the Cosmic Dawn and Epoch of Reionization (z~30-6). In both interferometric and global signal experiments, systematic calibration is the leading source of error. Though many aspects of calibration have been studied, the overlap between the two types of instruments has received less attention. We investigate the sky based calibration of total power measurements with a HERA dish and an EDGES style antenna to understand the role of auto-correlations in the calibration of an interferometer and the role of sky in calibrating a total power instrument. Using simulations we study various scenarios such as time variable gain, incomplete sky calibration model, and primary beam model. We find that temporal gain drifts, sky model incompleteness, and beam inaccuracies cause biases in the receiver gain amplitude and the receiver temperature estimates. In some cases, these biases mix spectral structure between beam and sky resulting in spectrally variable gain errors. Applying the calibration method to the HERA and EDGES data, we find good agreement with calibration via the more standard methods. Although instrumental gains are consistent with beam and sky errors similar in scale to those simulated, the receiver temperatures show significant deviations from expected values. While we show that it is possible to partially mitigate biases due to model inaccuracies by incorporating a time-dependent gain model in calibration, the resulting errors on calibration products are larger and more correlated. Completely addressing these biases will require more accurate sky and primary beam models., Comment: 16 pages, 13 figures, 1 table; accepted for publication in MNRAS main journal

We describe the validation of the HERA Phase I software pipeline by a series of modular tests, building up to an end-to-end simulation. The philosophy of this approach is to validate the software and algorithms used in the Phase I upper limit analysis on wholly synthetic data satisfying the assumptions of that analysis, not addressing whether the actual data meet these assumptions. We discuss the organization of this validation approach, the specific modular tests performed, and the construction of the end-to-end simulations. We explicitly discuss the limitations in scope of the current simulation effort. With mock visibility data generated from a known analytic power spectrum and a wide range of realistic instrumental effects and foregrounds, we demonstrate that the current pipeline produces power spectrum estimates that are consistent with known analytic inputs to within thermal noise levels (at the 2 sigma level) for k > 0.2 h/Mpc for both bands and fields considered. Our input spectrum is intentionally amplified to enable a strong `detection' at k ~0.2 h/Mpc -- at the level of ~25 sigma -- with foregrounds dominating on larger scales, and thermal noise dominating at smaller scales. Our pipeline is able to detect this amplified input signal after suppressing foregrounds with a dynamic range (foreground to noise ratio) of > 10^7. Our validation test suite uncovered several sources of scale-independent signal loss throughout the pipeline, whose amplitude is well-characterized and accounted for in the final estimates. We conclude with a discussion of the steps required for the next round of data analysis., Comment: 32 pages, 20 figures. Submitted to the Astrophysical Journal

As current- and next-generation astronomical instruments come online, they will generate an unprecedented deluge of data. Analyzing these data in real time presents unique conceptual and computational challenges, and their long-term storage and archiving is scientifically essential for generating reliable, reproducible results. We present here the real-time processing (RTP) system for the Hydrogen Epoch of Reionization Array (HERA), a radio interferometer endeavoring to provide the first detection of the highly redshifted 21 cm signal from Cosmic Dawn and the Epoch of Reionization by an interferometer. The RTP system consists of analysis routines run on raw data shortly after they are acquired, such as calibration and detection of radio-frequency interference (RFI) events. RTP works closely with the Librarian, the HERA data storage and transfer manager which automatically ingests data and transfers copies to other clusters for post-processing analysis. Both the RTP system and the Librarian are public and open source software, which allows for them to be modified for use in other scientific collaborations. When fully constructed, HERA is projected to generate over 50 terabytes (TB) of data each night, and the RTP system enables the successful scientific analysis of these data., Comment: 15 pages, 3 figures, published in Astronomy and Computing

Precise measurements of the 21 cm power spectrum are crucial for understanding the physical processes of hydrogen reionization. Currently, this probe is being pursued by low-frequency radio interferometer arrays. As these experiments come closer to making a first detection of the signal, error estimation will play an increasingly important role in setting robust measurements. Using the delay power spectrum approach, we have produced a critical examination of different ways that one can estimate error bars on the power spectrum. We do this through a synthesis of analytic work, simulations of toy models, and tests on small amounts of real data. We find that, although computed independently, the different error bar methodologies are in good agreement with each other in the noise-dominated regime of the power spectrum. For our preferred methodology, the predicted probability distribution function is consistent with the empirical noise power distributions from both simulated and real data. This diagnosis is mainly in support of the forthcoming HERA upper limit, and also is expected to be more generally applicable., Comment: 35 Pages, 9 Figures, 4 Tables. Replaced with accepted ApJ version; some clarifying text added in response to referee comments with no changes to results

Precise instrumental calibration is of crucial importance to 21-cm cosmology experiments. The Murchison Widefield Array's (MWA) Phase II compact configuration offers us opportunities for both redundant calibration and sky-based calibration algorithms; using the two in tandem is a potential approach to mitigate calibration errors caused by inaccurate sky models. The MWA Epoch of Reionization (EoR) experiment targets three patches of the sky (dubbed EoR0, EoR1, and EoR2) with deep observations. Previous work in \cite{Li_2018} and \cite{Wenyang_2019} studied the effect of tandem calibration on the EoR0 field and found that it yielded no significant improvement in the power spectrum over sky-based calibration alone. In this work, we apply similar techniques to the EoR1 field and find a distinct result: the improvements in the power spectrum from tandem calibration are significant. To understand this result, we analyze both the calibration solutions themselves and the effects on the power spectrum over three nights of EoR1 observations. We conclude that the presence of the bright radio galaxy Fornax A in EoR1 degrades the performance of sky-based calibration, which in turn enables redundant calibration to have a larger impact. These results suggest that redundant calibration can indeed mitigate some level of model-incompleteness error., Comment: 20 pages, 11 figures, accepted for publication in PASA

The central challenge in 21~cm cosmology is isolating the cosmological signal from bright foregrounds. Many separation techniques rely on the accurate knowledge of the sky and the instrumental response, including the antenna primary beam. For drift-scan telescopes such as the Hydrogen Epoch of Reionization Array \citep[HERA, ][]{DeBoer2017} that do not move, primary beam characterization is particularly challenging because standard beam-calibration routines do not apply \citep{Cornwell2005} and current techniques require accurate source catalogs at the telescope resolution. We present an extension of the method from \citet{Pober2012} where they use beam symmetries to create a network of overlapping source tracks that break the degeneracy between source flux density and beam response and allow their simultaneous estimation. We fit the beam response of our instrument using early HERA observations and find that our results agree well with electromagnetic simulations down to a -20~dB level in power relative to peak gain for sources with high signal-to-noise ratio. In addition, we construct a source catalog with 90 sources down to a flux density of 1.4~Jy at 151~MHz., Comment: 22 pages, 22 figures, accepted for publication in ApJ

Characterizing the epoch of reionization (EoR) at $z\gtrsim 6$ via the redshifted 21 cm line of neutral Hydrogen (HI) is critical to modern astrophysics and cosmology, and thus a key science goal of many current and planned low-frequency radio telescopes. The primary challenge to detecting this signal is the overwhelmingly bright foreground emission at these frequencies, placing stringent requirements on the knowledge of the instruments and inaccuracies in analyses. Results from these experiments have largely been limited not by thermal sensitivity but by systematics, particularly caused by the inability to calibrate the instrument to high accuracy. The interferometric bispectrum phase is immune to antenna-based calibration and errors therein, and presents an independent alternative to detect the EoR HI fluctuations while largely avoiding calibration systematics. Here, we provide a demonstration of this technique on a subset of data from the Hydrogen Epoch of Reionization Array (HERA) to place approximate constraints on the brightness temperature of the intergalactic medium (IGM). From this limited data, at $z=7.7$ we infer "$1\sigma$" upper limits on the IGM brightness temperature to be $\le 316$ "pseudo" mK at $\kappa_\parallel=0.33$ "pseudo" $h$ Mpc$^{-1}$ (data-limited) and $\le 1000$ "pseudo" mK at $\kappa_\parallel=0.875$ "pseudo" $h$ Mpc$^{-1}$ (noise-limited). The "pseudo" units denote only an approximate and not an exact correspondence to the actual distance scales and brightness temperatures. By propagating models in parallel to the data analysis, we confirm that the dynamic range required to separate the cosmic HI signal from the foregrounds is similar to that in standard approaches, and the power spectrum of the bispectrum phase is still data-limited (at $\gtrsim 10^6$ dynamic range) indicating scope for further improvement in sensitivity as the array build-out continues., Comment: 22 pages, 12 figures (including sub-figures). Published in PhRvD. Abstract may be slightly abridged compared to the actual manuscript due to length limitations on arXiv

Calibration precision is currently a limiting systematic in 21 cm cosmology experiments. While there are innumerable calibration approaches, most can be categorized as either `sky-based,' relying on an extremely accurate model of astronomical foreground emission, or `redundant,' requiring a precisely regular array with near-identical antenna response patterns. Both of these classes of calibration are inflexible to the realities of interferometric measurement. In practice, errors in the foreground model, antenna position offsets, and beam response inhomogeneities degrade calibration performance and contaminate the cosmological signal. Here we show that sky-based and redundant calibration can be unified into a highly general and physically motivated calibration framework based on a Bayesian statistical formalism. Our new framework includes sky and redundant calibration as special cases but can additionally support relaxing the rigid assumptions implicit in those approaches. Furthermore, we present novel calibration techniques such as redundant calibration for arrays with no redundant baselines, representing an alternative calibration method for imaging arrays such as the MWA Phase I. These new calibration approaches could mitigate systematics and reduce calibration error, thereby improving the precision of cosmological measurements., Comment: Accepted for publication in MNRAS. This updated version has a section exploring the effects of the unified calibration framework on simulated data and comparing unified calibration to sky-based and redundant calibration. It also has an expanded discussion of calibration biases

We quantify the effect of radio frequency interference (RFI) on measurements of the 21-cm power spectrum during the Epoch of Reionization (EoR). Specifically, we investigate how the frequency structure of RFI source emission generates contamination in higher-order wave modes that is much more problematic than smooth-spectrum foreground sources. Using a relatively optimistic EoR model, we find that even a single relatively dim RFI source can overwhelm the EoR power spectrum signal of $\sim10\text{ mK}^2$ for modes $0.1 \text{ }h\text{ Mpc}^{-1} < k < 2 \text{ }h\text{ Mpc}^{-1}$. If total apparent RFI flux density in the final power spectrum integration is kept below 1 mJy, an EoR signal resembling this optimistic model should be detectable for modes $k < 0.9\text{ }h\text{ Mpc}^{-1}$, given no other systematic contaminants and an error tolerance as high as 10%. More pessimistic models will be more restrictive. These results emphasize the need for highly effective RFI mitigation strategies for telescopes used to search for the EoR.

The key challenge in the observation of the redshifted 21-cm signal from cosmic reionization is its separation from the much brighter foreground emission. Such separation relies on the different spectral properties of the two components, although, in real life, the foreground intrinsic spectrum is often corrupted by the instrumental response, inducing systematic effects that can further jeopardize the measurement of the 21-cm signal. In this paper, we use Gaussian Process Regression to model both foreground emission and instrumental systematics in $\sim 2$ hours of data from the Hydrogen Epoch of Reionization Array. We find that a simple co-variance model with three components matches the data well, giving a residual power spectrum with white noise properties. These consist of an "intrinsic" and instrumentally corrupted component with a coherence-scale of 20 MHz and 2.4 MHz respectively (dominating the line of sight power spectrum over scales $k_{\parallel} \le 0.2$ h cMpc$^{-1}$) and a baseline dependent periodic signal with a period of $\sim 1$ MHz (dominating over $k_{\parallel} \sim 0.4 - 0.8$h cMpc$^{-1}$) which should be distinguishable from the 21-cm EoR signal whose typical coherence-scales is $\sim 0.8$ MHz., Comment: 15 pages, 15 figures, 1 table, Accepted to MNRAS

In 21 cm cosmology, precision calibration is key to the separation of the neutral hydrogen signal from very bright but spectrally-smooth astrophysical foregrounds. The Hydrogen Epoch of Reionization Array (HERA), an interferometer specialized for 21 cm cosmology and now under construction in South Africa, was designed to be largely calibrated using the self-consistency of repeated measurements of the same interferometric modes. This technique, known as "redundant-baseline calibration" resolves most of the internal degrees of freedom in the calibration problem. It assumes, however, on antenna elements with identical primary beams placed precisely on a redundant grid. In this work, we review the detailed implementation of the algorithms enabling redundant-baseline calibration and report results with HERA data. We quantify the effects of real-world non-redundancy and how they compare to the idealized scenario in which redundant measurements differ only in their noise realizations. Finally, we study how non-redundancy can produce spurious temporal structure in our calibration solutions--both in data and in simulations--and present strategies for mitigating that structure., Comment: 24 Pages, 19 Figures. Updated to match the accepted MNRAS version

We discuss absolute calibration strategies for Phase I of the Hydrogen Epoch of Reionization Array (HERA), which aims to measure the cosmological 21 cm signal from the Epoch of Reionization (EoR). HERA is a drift-scan array with a 10 degree wide field of view, meaning bright, well-characterized point source transits are scarce. This, combined with HERA's redundant sampling of the uv plane and the modest angular resolution of the Phase I instrument, make traditional sky-based and self-calibration techniques difficult to implement with high dynamic range. Nonetheless, in this work we demonstrate calibration for HERA using point source catalogues and electromagnetic simulations of its primary beam. We show that unmodeled diffuse flux and instrumental contaminants can corrupt the gain solutions, and present a gain smoothing approach for mitigating their impact on the 21 cm power spectrum. We also demonstrate a hybrid sky and redundant calibration scheme and compare it to pure sky-based calibration, showing only a marginal improvement to the gain solutions at intermediate delay scales. Our work suggests that the HERA Phase I system can be well-calibrated for a foreground-avoidance power spectrum estimator by applying direction-independent gains with a small set of degrees of freedom across the frequency and time axes., Comment: Accepted to ApJ

We present a study of internal reflection and cross coupling systematics in Phase I of the Hydrogen Epoch of Reionization Array (HERA). In a companion paper, we outlined the mathematical formalism for such systematics and presented algorithms for modeling and removing them from the data. In this work, we apply these techniques to data from HERA's first observing season as a method demonstration. The data show evidence for systematics that, without removal, would hinder a detection of the 21 cm power spectrum for the targeted EoR line-of-sight modes in the range 0.2 < k_parallel < 0.5\ h^-1 Mpc. After systematic removal, we find we can recover these modes in the power spectrum down to the integrated noise-floor of a nightly observation, achieving a dynamic range in the EoR window of 10^-6 in power (mK^2 units) with respect to the bright galactic foreground signal. In the absence of other systematics and assuming the systematic suppression demonstrated here continues to lower noise levels, our results suggest that fully-integrated HERA Phase I may have the capacity to set competitive upper limits on the 21 cm power spectrum. For future observing seasons, HERA will have upgraded analog and digital hardware to better control these systematics in the field., Comment: Submitted to ApJ

The detection of the Epoch of Reionization (EoR) delay power spectrum using a "foreground avoidance method" highly depends on the instrument chromaticity. The systematic effects induced by the radio-telescope spread the foreground signal in the delay domain, which contaminates the EoR window theoretically observable. Applied to the Hydrogen Epoch of Reionization Array (HERA), this paper combines detailed electromagnetic and electrical simulations in order to model the chromatic effects of the instrument, and quantify its frequency and time responses. In particular, the effects of the analogue receiver, transmission cables, and mutual coupling are included. These simulations are able to accurately predict the intensity of the reflections occurring in the 150-m cable which links the antenna to the back-end. They also show that electromagnetic waves can propagate from one dish to another one through large sections of the array due to mutual coupling. The simulated system time response is attenuated by a factor $10^{4}$ after a characteristic delay which depends on the size of the array and on the antenna position. Ultimately, the system response is attenuated by a factor $10^{5}$ after 1400 ns because of the reflections in the cable, which corresponds to characterizable ${k_\parallel}$-modes above 0.7 $h\;\rm{Mpc}^{-1}$ at 150 MHz. Thus, this new study shows that the detection of the EoR signal with HERA Phase I will be more challenging than expected. On the other hand, it improves our understanding of the telescope, which is essential to mitigate the instrument chromaticity., Comment: 13 pages, 17 figures - Submitted to MNRAS - 2nd revision

We introduce a new pipeline for analyzing and mitigating radio frequency interference (RFI), which we call Sky-Subtracted Incoherent Noise Spectra (SSINS). SSINS is designed to identify and remove faint RFI below the single baseline thermal noise by employing a frequency-matched detection algorithm on baseline-averaged amplitudes of time-differenced visibilities. We demonstrate the capabilities of SSINS using the Murchison Widefield Array (MWA) in Western Australia. We successfully image aircraft flying over the array via digital television (DTV) reflection detected using SSINS and summarize an RFI occupancy survey of MWA Epoch of Reionization data. We describe how to use SSINS with new data using a documented, publicly available implementation with comprehensive usage tutorials., Comment: Publication Draft

Radio Frequency Interference (RFI) is an ever-present limiting factor among radio telescopes even in the most remote observing locations. When looking to retain the maximum amount of sensitivity and reduce contamination for Epoch of Reionization studies, the identification and removal of RFI is especially important. In addition to improved RFI identification, we must also take into account computational efficiency of the RFI-Identification algorithm as radio interferometer arrays such as the Hydrogen Epoch of Reionization Array grow larger in number of receivers. To address this, we present a Deep Fully Convolutional Neural Network (DFCN) that is comprehensive in its use of interferometric data, where both amplitude and phase information are used jointly for identifying RFI. We train the network using simulated HERA visibilities containing mock RFI, yielding a known "ground truth" dataset for evaluating the accuracy of various RFI algorithms. Evaluation of the DFCN model is performed on observations from the 67 dish build-out, HERA-67, and achieves a data throughput of 1.6$\times 10^{5}$ HERA time-ordered 1024 channeled visibilities per hour per GPU. We determine that relative to an amplitude only network including visibility phase adds important adjacent time-frequency context which increases discrimination between RFI and Non-RFI. The inclusion of phase when predicting achieves a Recall of 0.81, Precision of 0.58, and $F_{2}$ score of 0.75 as applied to our HERA-67 observations., Comment: 11 pages, 7 figures

Precise instrument calibration is critical to the success of 21 cm Cosmology experiments. Unmitigated errors in calibration contaminate the Epoch of Reionization (EoR) signal, precluding a detection. Barry et al. 2016 characterizes one class of inherent errors that emerge from calibrating to an incomplete sky model, however it has been unclear if errors in the sky model affect the calibration of redundant arrays. In this paper, we show that redundant calibration is vulnerable to errors from sky model incompleteness even in the limit of perfect antenna positioning and identical beams. These errors are at a level that can overwhelm the EoR signal and prevent a detection. Finally, we suggest error mitigation strategies with implications for the Hydrogen Epoch of Reionization Array (HERA) and the Square Kilometre Array (SKA).

21 cm power spectrum observations have the potential to revolutionize our understanding of the Epoch of Reionization and Dark Energy, but require extraordinarily precise data analysis methods to separate the cosmological signal from the astrophysical and instrumental contaminants. This analysis challenge has led to a diversity of proposed analyses, including delay spectra, imaging power spectra, m-mode analysis, and numerous others. This diversity of approach is a strength, but has also led to confusion within the community about whether insights gleaned by one group are applicable to teams working in different analysis frameworks. In this paper we show that all existing analysis proposals can be classified into two distinct families based on whether they estimate the power spectrum of the measured or reconstructed sky. This subtle difference in the statistical question posed largely determines the susceptibility of the analyses to foreground emission and calibration errors, and ultimately the science different analyses can pursue. In this paper we detail the origin of the two analysis families, categorize the analyses being actively developed, and explore their relative sensitivities to foreground contamination and calibration errors., Comment: Accepted for publication in MNRAS

New massively redundant low frequency arrays allow for a novel investigation of closure relations in interferometry. We employ commissioning data from the Hydrogen Epoch of Reionization Array to investigate closure quantities in this densely packed grid array of 14m antennas operating at 100 MHz to 200 MHz. We investigate techniques that utilize closure phase spectra for redundant triads to estimate departures from redundancy for redundant baseline visibilities. We find a median absolute deviation from redundancy in closure phase across the observed frequency range of about 4.5deg. This value translates into a non-redundancy per visibility phase of about 2.6deg, using prototype electronics. The median absolute deviations from redundancy decrease with longer baselines. We show that closure phase spectra can be used to identify ill-behaved antennas in the array, independent of calibration. We investigate the temporal behavior of closure spectra. The Allan variance increases after a one minute stride time, due to passage of the sky through the primary beam of the transit telescope. However, the closure spectra repeat to well within the noise per measurement at corresponding local sidereal times (LST) from day to day. In future papers in this series we will develop the technique of using closure phase spectra in the search for the HI 21cm signal from cosmic reionization., Comment: 32 pages. 11 figures. Accepted to Radio Science

Foreground power dominates the measurements of interferometers that seek a statistical detection of highly-redshifted HI emission from the Epoch of Reionization (EoR). The chromaticity of the instrument creates a boundary in the Fourier transform of frequency (proportional to $k_\parallel$) between spectrally smooth emission, characteristic of the strong synchrotron foreground (the "wedge"), and the spectrally structured emission from HI in the EoR (the "EoR window"). Faraday rotation can inject spectral structure into otherwise smooth polarized foreground emission, which through instrument effects or miscalibration could possibly pollute the EoR window. Using data from the HERA 19-element commissioning array, we investigate the polarization response of this new instrument in the power spectrum domain. We perform a simple image-based calibration based on the unpolarized diffuse emission of the Global Sky Model, and show that it achieves qualitative redundancy between the nominally-redundant baselines of the array and reasonable amplitude accuracy. We construct power spectra of all fully polarized coherencies in all pseudo-Stokes parameters. We compare to simulations based on an unpolarized diffuse sky model and detailed electromagnetic simulations of the dish and feed, confirming that in Stokes I, the calibration does not add significant spectral structure beyond the expected level. Further, this calibration is stable over the 8 days of observations considered. Excess power is seen in the power spectra of the linear polarization Stokes parameters which is not easily attributable to leakage via the primary beam, and results from some combination of residual calibration errors and actual polarized emission. Stokes V is found to be highly discrepant from the expectation of zero power, strongly pointing to the need for more accurate polarized calibration., Comment: 21 pages, 12 figures, submitted to ApJ

The experimental efforts to detect the redshifted 21 cm signal from the Epoch of Reionization (EoR) are limited predominantly by the chromatic instrumental systematic effect. The delay spectrum methodology for 21 cm power spectrum measurements brought new attention to the critical impact of an antenna's chromaticity on the viability of making this measurement. This methodology established a straightforward relationship between time-domain response of an instrument and the power spectrum modes accessible to a 21 cm EoR experiment. We examine the performance of a prototype of the Hydrogen Epoch of Reionization Array (HERA) array element that is currently observing in Karoo desert, South Africa. We present a mathematical framework to derive the beam integrated frequency response of a HERA prototype element in reception from the return loss measurements between 100-200 MHz and determined the extent of additional foreground contamination in the delay space. The measurement reveals excess spectral structures in comparison to the simulation studies of the HERA element. Combined with the HERA data analysis pipeline that incorporates inverse covariance weighting in optimal quadratic estimation of power spectrum, we find that in spite of its departure from the simulated response, HERA prototype element satisfies the necessary criteria posed by the foreground attenuation limits and potentially can measure the power spectrum at spatial modes as low as $k_{\parallel} > 0.1h$~Mpc$^{-1}$. The work highlights a straightforward method for directly measuring an instrument response and assessing its impact on 21 cm EoR power spectrum measurements for future experiments that will use reflector-type antenna., Comment: 13 pages, 10 figures, Submitted to APJ

Low radio frequency solar observations using the Murchison Widefield Array have recently revealed the presence of numerous weak, short-lived and narrow-band emission features, even during moderately quiet solar conditions. These non-thermal features occur at rates of many thousands per hour in the 30.72 MHz observing bandwidth, and hence, necessarily require an automated approach for their detection and characterization. Here, we employ continuous wavelet transform using a mother Ricker wavelet for feature detection from the dynamic spectrum. We establish the efficacy of this approach and present the first statistically robust characterization of the properties of these features. In particular, we examine distributions of their peak flux densities, spectral spans, temporal spans and peak frequencies. We can reliably detect features weaker than 1 SFU, making them, to the best of our knowledge, the weakest bursts reported in literature. The distribution of their peak flux densities follows a power law with an index of -2.23 in the 12-155 SFU range, implying that they can provide an energetically significant contribution to coronal and chromospheric heating. These features typically last for 1-2 seconds and possess bandwidths of about 4-5 MHz. Their occurrence rate remains fairly flat in the 140-210 MHz frequency range. At the time resolution of the data, they appear as stationary bursts, exhibiting no perceptible frequency drift. These features also appear to ride on a broadband background continuum, hinting at the likelihood of them being weak type-I bursts., Comment: 15 pages, 14 figures, Accepted for publication in ApJ

Using the Murchison Widefield Array (MWA), the low-frequency Square Kilometre Array (SKA1 LOW) precursor located in Western Australia, we have completed the GaLactic and Extragalactic All-sky MWA (GLEAM) survey, and present the resulting extragalactic catalogue, utilising the first year of observations. The catalogue covers 24,831 square degrees, over declinations south of $+30^\circ$ and Galactic latitudes outside $10^\circ$ of the Galactic plane, excluding some areas such as the Magellanic Clouds. It contains 307,455 radio sources with 20 separate flux density measurements across 72--231MHz, selected from a time- and frequency- integrated image centred at 200MHz, with a resolution of $\approx 2$'. Over the catalogued region, we estimate that the catalogue is 90% complete at 170mJy, and 50% complete at 55mJy, and large areas are complete at even lower flux density levels. Its reliability is 99.97% above the detection threshold of $5\sigma$, which itself is typically 50mJy. These observations constitute the widest fractional bandwidth and largest sky area survey at radio frequencies to date, and calibrate the low frequency flux density scale of the southern sky to better than 10%. This paper presents details of the flagging, imaging, mosaicking, and source extraction/characterisation, as well as estimates of the completeness and reliability. All source measurements and images are available online (http://www.mwatelescope.org/science/gleam-survey). This is the first in a series of publications describing the GLEAM survey results., Comment: 30 pages, 18 figures, 6 tables, published in Monthly Notices of the Royal Astronomical Society

The Detection of redshifted 21 cm emission from the epoch of reionization (EoR) is a challenging task owing to strong foregrounds that dominate the signal. In this paper, we propose a general method, based on the delay spectrum approach, to extract HI power spectra that is applicable to tracking observations using an imaging radio interferometer (Delay Spectrum with Imaging Arrays (DSIA)). Our method is based on modelling the HI signal taking into account the impact of wide field effects such as the $w$-term which are then used as appropriate weights in cross-correlating the measured visibilities. Our method is applicable to any radio interferometer that tracks a phase center and could be utilized for arrays such as MWA, LOFAR, GMRT, PAPER and HERA. In the literature the delay spectrum approach has been implemented for near-redundant baselines using drift scan observations. In this paper we explore the scheme for non-redundant tracking arrays, and this is the first application of delay spectrum methodology to such data to extract the HI signal. We analyze 3 hours of MWA tracking data on the EoR1 field. We present both 2-dimensional ($k_\parallel,k_\perp$) and 1-dimensional (k) power spectra from the analysis. Our results are in agreement with the findings of other pipelines developed to analyse the MWA EoR data., Comment: 17 pages, 10 figures, accepted for publication in The Astrophysical Journal

The Hydrogen Epoch of Reionization Array (HERA) is a staged experiment to measure 21 cm emission from the primordial intergalactic medium (IGM) throughout cosmic reionization ($z=6-12$), and to explore earlier epochs of our Cosmic Dawn ($z\sim30$). During these epochs, early stars and black holes heated and ionized the IGM, introducing fluctuations in 21 cm emission. HERA is designed to characterize the evolution of the 21 cm power spectrum to constrain the timing and morphology of reionization, the properties of the first galaxies, the evolution of large-scale structure, and the early sources of heating. The full HERA instrument will be a 350-element interferometer in South Africa consisting of 14-m parabolic dishes observing from 50 to 250 MHz. Currently, 19 dishes have been deployed on site and the next 18 are under construction. HERA has been designated as an SKA Precursor instrument. In this paper, we summarize HERA's scientific context and provide forecasts for its key science results. After reviewing the current state of the art in foreground mitigation, we use the delay-spectrum technique to motivate high-level performance requirements for the HERA instrument. Next, we present the HERA instrument design, along with the subsystem specifications that ensure that HERA meets its performance requirements. Finally, we summarize the schedule and status of the project. We conclude by suggesting that, given the realities of foreground contamination, current-generation 21 cm instruments are approaching their sensitivity limits. HERA is designed to bring both the sensitivity and the precision to deliver its primary science on the basis of proven foreground filtering techniques, while developing new subtraction techniques to unlock new capabilities. The result will be a major step toward realizing the widely recognized scientific potential of 21 cm cosmology., Comment: 26 pages, 24 figures, 2 tables

We present the E-field Parallel Imaging Calibration (EPICal) algorithm, which addresses the need for a fast calibration method for direct imaging radio astronomy correlators. Direct imaging involves a spatial fast Fourier transform of antenna signals, alleviating an $\mathcal{O}(N_{\mathrm{ant}}^2)$ computational bottleneck typical in radio correlators, and yielding a more gentle $\mathcal{O}(N_g \log_2 N_g)$ scaling, where $N_{\mathrm{ant}}$ is the number of antennas in the array and $N_g$ is the number of grid points in the imaging analysis. This can save orders of magnitude in computation cost for next generation arrays consisting of hundreds or thousands of antennas. However, because antenna signals are mixed in the imaging correlator without creating visibilities, gain correction must be applied prior to imaging, rather than on visibilities post-correlation. We develop the EPICal algorithm to form gain solutions quickly and without ever forming visibilities. This method scales as the number of antennas, and produces results comparable to those from visibilities. We use simulations to demonstrate the EPICal technique and study the noise properties of our gain solutions, showing they are similar to visibility based solutions in realistic situations. By applying EPICal to two seconds of Long Wavelength Array data we achieve a 65% dynamic range improvement compared to uncalibrated images, showing this algorithm is a promising solution for next generation instruments., Comment: 13 pages, 11 figures, Accepted to MNRAS