Search

Strongly coupled data assimilation (SCDA), such as using atmospheric observations to update ocean analyses, is critical for properly initializing Earth System models to predict subseasonal to decadal timescales. We show that a Kalman filter with a linear emulator of the coupled dynamics can be used to efficiently assimilate observations with SCDA. A linear inverse model (LIM), trained on 25 years of Climate Forecast System Reanalysis gridded data, is used to assimilate observations daily during an independent 7‐year period. SCDA sea‐surface temperature (SST) analysis errors are reduced over 20% in global‐mean mean‐squared error relative to a control experiment where only SST observations are assimilated with an SST LIM. The analysis improvements enhance forecast skill for leads of at least 50 days. In contrast, extratropical Northern Hemisphere 2 m air temperature forecast errors increase for coupled data assimilation in these experiments, despite reduction during the training period. Plain Language Summary: Using observations to consistently initialize a forecast is very difficult with coupled Earth system models due to their enormous computational demand. Here we show that a simplified model can be used to improve forecasts through this initialization process. In particular, we show that this approach allows observations of the atmosphere to be used to estimate sea‐surface temperature (SST) and to improve forecasts of SST when compared to an approach that uses only observations of SST. This suggests the potential to improve SST forecasts one to 2 months in the future using such an approach. Key Points: We provide proof‐of‐concept results that low‐dimensional Earth System emulators are useful for testing coupled data assimilation approachesStrongly coupled data assimilation improves analysis and forecast skill on subseasonal timescales relative to the weakly coupled case [ABSTRACT FROM AUTHOR]

Linear transformations are widely used in data assimilation for covariance modeling, for reducing dimensionality (such as averaging dense observations to form "superobs"), and for managing sampling error in ensemble data assimilation. Here we describe a linear transformation that is optimal in the sense that, in the transformed space, the state variables and observations have uncorrelated errors, and a diagonal gain matrix in the update step. We conjecture, and provide numerical evidence, that the transformation is the best possible to precede covariance localization in an ensemble Kalman filter. A central feature of this transformation in the update step are scalars, which we term canonical observation operators (COOs), that relate pairs of transformed observations and state variables and rank-order those pairs by their influence in the update. We show for an idealized problem that sample-based estimates of the COOs, in conjunction with covariance localization for the sample covariance, can approximate well the true values, but a practical implementation of the transformation for high-dimensional applications remains a subject for future research. The COOs also completely describe important properties of the update step, such as observation-state mutual information, signal-tonoise and degrees of freedom for signal, and so give new insights, including relations among reduced-rank approximations to variational schemes, particle-filter weight degeneracy, and the local ensemble transform Kalman filter. Plain Language Summary Good estimates of the state of the Earth system rely on combining the latest observations with recent forecasts through a technique called data assimilation. To some degree, there is freedom to choose which variables are used in data assimilation and, before performing assimilation, to combine and transform the observations. Careful choices of variables and transformations of the observations not only have practical benefits, but also, by greatly simplifying the mathematical formulation of data assimilation, yield insights into how new observations influence Earth-system state estimates and common measures of the information provided by observations. [ABSTRACT FROM AUTHOR]

Arctic sea ice decline in recent decades has been dramatic; however, few long-term records of Arctic sea ice exist to put such a decline in context. Here we employ an ensemble Kalman filter data assimilation approach to reconstruct Arctic sea ice concentration over the last two millennia by assimilating temperature-sensitive proxy records with ensembles drawn from last millennium climate model simulations. We first test the efficacy of this method using pseudoproxy experiments. Results show good agreement between the target and reconstructed total Arctic sea ice extent (R2 value and coefficient of efficiency values of 0.51 and 0.47 for perfect model experiments and 0.43 and 0.43 for imperfect model experiments). Imperfect model experiments indicate that the reconstructions inherit some bias from the model prior. We assimilate 487 temperature-sensitive proxy records with two climate model simulations to produce two gridded reconstructions of Arctic sea ice over the last two millennia. These reconstructions show good agreement with satellite observations between 1979 and 1999 CE for total Arctic sea ice extent with an R2 value and coefficient of efficiency of about 0.60 and 0.50, respectively, for both models. Regional quantities derived from these reconstructions show encouraging similarities with independent reconstructions and sea ice sensitive proxy records from the Barents Sea, Baffin Bay, and East Greenland Sea. The reconstructions show a positive trend in Arctic sea ice extent between around 750 and 1820 CE, and increases during years with large volcanic eruptions that persist for about 5 years. Trend analysis of total Arctic sea ice extent reveals that for time periods longer than 30 years, the satellite era decline in total Arctic sea ice extent is unprecedented over the last millennium. Significance Statement: Areal coverage of Arctic sea ice is a critical aspect of the climate system that has been changing rapidly in recent decades. Prior to the advent of satellite observations, sparse observations of Arctic sea ice make it difficult to put the current changes in context. Here we reconstruct annual averages of Arctic sea ice coverage for the last two millennia by combining temperature-sensitive proxy records (i.e., ice cores, tree rings, and corals) with climate model simulations using a statistical technique called data assimilation. We find large interannual changes in Arctic sea ice coverage prior to 1850 that are associated with volcanic eruptions, with a steady rise in Arctic sea ice coverage between 750 and 1820 CE. The satellite-period loss of sea ice has no analog during the last millennium. [ABSTRACT FROM AUTHOR]

Arctic sea‐ice extent (SIE) has declined drastically in recent decades, yet its evolution prior to the satellite era is highly uncertain. Studies using SIE observations find little variability prior to the 1970s; however, these reconstructions are based on limited data, especially prior to the 1950s. We use ensemble Kalman filter data assimilation of surface air temperature observations with Last Millennium climate model simulations to create a fully gridded Arctic sea‐ice concentration reconstruction from 1850 to 2018 and investigate the evolution of Arctic SIE during this period. We find a decline of ∼1.25×106 km2 during the early 20th‐century warming (1910–1940). The 25‐year trends during this period are ∼33–38% smaller than the satellite era (1979–2018) but almost twice as large as previous estimates. Additionally, we find that variability of SIE on decadal timescales prior to the satellite era is ∼40% greater than previously estimated. Plain Language Summary: Arctic sea ice is an important part of the climate system, serving as the interface between the ocean–atmosphere system. Arctic sea ice has undergone a rapid decline in recent decades, prompting the question of whether there have been changes of similar magnitude in the past. To answer such questions, a long record of sea ice is necessary, but spatially and temporally complete satellite observations are only available starting in 1979. Previous studies combining sea‐ice observations from various sources during the Instrumental Era (1850–2014) found little variability in sea‐ice extent prior to the satellite era, but data availability is limited prior to the 1950s. Here, we create an independent estimate of Arctic sea ice from 1850 to 2018 using a data assimilation approach that blends more abundant temperature observations with data from climate models. Our results show substantial loss of sea ice between 1910 and 1940, with a trend that is about ∼33–38% less than what has been observed in satellite observations. These results reinforce previous findings not only that the current trend is unprecedented in duration since 1850 but also that sea‐ice variability prior to 1979 is ∼40% larger than previously estimated. Key Points: Data assimilation is a skillful technique for reconstructing Arctic sea‐ice extent during the satellite eraReconstructed sea ice shows large declines in total Arctic sea‐ice extent during the early 20th‐century warming (1910–1940)Trends in total Arctic sea‐ice extent during the satellite era are ∼33–38% greater than during the early 20th‐century warming [ABSTRACT FROM AUTHOR]

Predicting when and where individual convective storms will develop remains an elusive challenge. Previous studies have suggested that surface observations can capture convective-scale features relevant to the convective initiation (CI) process, and new surface observing platforms such as crowdsourcing could significantly increase surface observation density in the near future. Here, a series of observing system simulation experiments (OSSEs) are performed to determine the required density of surface observations necessary to constrain storm-scale forecasts of CI. Ensemble simulations of an environment where CI occurs are cycled hourly using the CM1 model while assimilating synthetic surface observations at varying densities. Skillful and reliable storm-scale forecasts of CI are produced when surface observations of at least 4-km-and particularly with 1-km-density are assimilated, but only for forecasts initiated within 1 h of CI. Time scales of forecast improvement in surface variables suggest that hourly cycling is at the upper limit for CI forecast improvement. In addition, the structure of the assimilation increments, ensemble calibration in these experiments, and challenges of convective-scale assimilation are discussed. [ABSTRACT FROM AUTHOR]

A central issue for understanding past climates involves the use of sparse time-integrated data to recover the physical properties of the coupled climate system. This issue is explored in a simple model of the midlatitude climate system that has attributes consistent with the observed climate. A quasigeostrophic (QG) model thermally coupled to a slab ocean is used to approximate midlatitude coupled variability, and a variant of the ensemble Kalman filter is used to assimilate time-averaged observations. The dependence of reconstruction skill on coupling and thermal inertia is explored. Results from this model are compared with those for an even simpler two-variable linear stochastic model of midlatitude air-sea interaction, for which the assimilation problem can be solved semianalytically. Results for the QG model show that skill decreases as the length of time over which observations are averaged increases in both the atmosphere and ocean when normalized against the time-averaged climatological variance. Skill in the ocean increases with slab depth, as expected from thermal inertia arguments, but skill in the atmosphere decreases. An explanation of this counterintuitive result derives from an analytical expression for the forecast error covariance in the two-variable stochastic model, which shows that the ratio of noise to total error increases with slab ocean depth. Essentially, noise becomes trapped in the atmosphere by a thermally stiffer ocean, which dominates the decrease in initial condition error owing to improved skill in the ocean. Increasing coupling strength in the QG model yields higher skill in the atmosphere and lower skill in the ocean, as the atmosphere accesses the longer ocean memory and the ocean accesses more atmospheric high-frequency 'noise.' The two-variable stochastic model fails to capture this effect, showing decreasing skill in both the atmosphere and ocean for increased coupling strength, due to an increase in the ratio of noise to the forecast error variance. Implications for the potential for data assimilation to improve climate reconstructions are discussed. [ABSTRACT FROM AUTHOR]

Previous research suggests that an ensemble Kalman filter (EnKF) data assimilation and modeling system can produce accurate atmospheric analyses and forecasts at 30--50-km grid spacing. This study examines the ability of a mesoscale EnKF system using multiscale (36/12 km) Weather Research and Forecasting (WRF) model simulations to produce high-resolution, accurate, regional surface analyses, and 6-h forecasts. This study takes place over the complex terrain of the Pacific Northwest, where the small-scale features of the near-surface flow field make the region particularly attractive for testing an EnKF and its flow-dependent background error covariances. A variety of EnKF experiments are performed over a 5-week period to test the impact of decreasing the grid spacing from 36 to 12 km and to evaluate new approaches for dealing with representativeness error, lack of surface background variance, and low-level bias. All verification in this study is performed with independent, unassimilated observations. Significant surface analysis and 6-h forecast improvements are found when EnKF grid spacing is reduced from 36 to 12 km. Forecast improvements appear to be a consequence of increased resolution during model integration, whereas analysis improvements also benefit from high-resolution ensemble covariances during data assimilation. On the 12-km domain, additional analysis improvements are found by reducing observation error variance in order to address representativeness error. Removing model surface biases prior to assimilation significantly enhances the analysis. Inflating surface wind and temperature background error variance has large impacts on analyses, but only produces small improvements in analysis RMS errors. Both surface and upper-air 6-h forecasts are nearly unchanged in the 12-km experiments. Last, 12-km WRF EnKF surface analyses and 6-h forecasts are shown to generally outperform those of the Global Forecast System (GFS), North American Model (NAM), and the Rapid Update Cycle (RUC) by about 10%%--30%%, although these improvements do not extend above the surface. Based on these results, future improvements in multiscale EnKF are suggested. [ABSTRACT FROM AUTHOR]

The problem of reconstructing past climates from a sparse network of noisy time-averaged observations is considered with a novel ensemble Kalman filter approach. Results for a sparse network of 100 idealized observations for a quasi-geostrophic model of a jet interacting with a mountain reveal that, for a wide range of observation averaging times, analysis errors are reduced by about 50% relative to the control case without assimilation. Results are robust to changes to observational error, the number of observations, and an imperfect model. Specifically, analysis errors are reduced relative to the control case for observations having errors up to three times the climatological variance for a fixed 100-station network, and for networks consisting of ten or more stations when observational errors are fixed at one-third the climatological variance. In the limit of small numbers of observations, station location becomes critically important, motivating an optimally determined network. A network of fifteen optimally determined observations reduces analysis errors by 30% relative to the control, as compared to 50% for a randomly chosen network of 100 observations. [ABSTRACT FROM AUTHOR]

An ensemble Kalman filter based on the Weather Research and Forecasting (WRF) model is used to generate ensemble analyses and forecasts for the extratropical transition (ET) events associated with Typhoons Tokage (2004) and Nabi (2005). Ensemble sensitivity analysis is then used to evaluate the relationship between forecast errors and initial condition errors at the onset of transition, and to objectively determine the observations having the largest impact on forecasts of these storms. Observations from rawinsondes, surface stations, aircraft, cloud winds, and cyclone best-track position are assimilated every 6 h for a period before, during, and after transition. Ensemble forecasts initialized at the onset of transition exhibit skill similar to the operational Global Forecast System (GFS) forecast and to a WRF forecast initialized from the GFS analysis. WRF ensemble forecasts of Tokage (Nabi) are characterized by relatively large (small) ensemble variance and greater (smaller) sensitivity to the initial conditions. In both cases, the 48-h forecast of cyclone minimum SLP and the RMS forecast error in SLP are most sensitive to the tropical cyclone position and to midlatitude troughs that interact with the tropical cyclone during ET. Diagnostic perturbations added to the initial conditions based on ensemble sensitivity reduce the error in the storm minimum SLP forecast by 50%. Observation impact calculations indicate that assimilating approximately 40 observations in regions of greatest initial condition sensitivity produces a large, statistically significant impact on the 48-h cyclone minimum SLP forecast. For the Tokage forecast, assimilating the single highest impact observation, an upper-tropospheric zonal wind observation from a Mongolian rawinsonde, yields 48-h forecast perturbations in excess of 10 hPa and 60 m in SLP and 500-hPa height, respectively. [ABSTRACT FROM AUTHOR]

An ensemble Kalman filter (EnKF) based on the Weather Research and Forecasting model is applied to generate ensemble analyses and forecasts of Hurricane Katrina (2005) and the surrounding area every 6 h over the lifetime of the storm on a nested domain. Analyses are derived from assimilating conventional in situ observations, reconnaissance dropsondes, including data taken during the Hurricane Rainband and Intensity Exchange Experiment (RAINEX), and tropical cyclone position estimates. Observation assimilation at individual times consistently reduces errors in tropical cyclone position, but not necessarily in intensity; however, withholding observations leads to significantly larger errors in both quantities. Analysis increments for observations near the tropical cyclone are dominated by changes in vortex position, and these increments increase the asymmetric structure of the storm. Data denial experiments indicate that dropsondes deployed in the synoptic environment provide minimal benefit to the outer domain; however, dropsondes deployed within the tropical cyclone lead to significant reductions in position and intensity errors on the inner domain. Specifically, errors in the inner domain ensemble-mean 6-h forecasts of minimum pressure are 70% larger when dropsonde data is not assimilated. Precipitation fields are qualitatively similar to Tropical Rainfall Measuring Mission (TRMM) satellite estimates, although model values are double the values of the satellite estimate. Moreover, the spinup period and initial imbalance in EnKF-initialized WRF forecasts is less than starting the model from a GFS analysis. Ensemble-mean 48-h forecasts initialized with EnKF analyses have track and intensity errors that are 50% smaller than GFS and NHC official forecasts. [ABSTRACT FROM AUTHOR]

The 2-yr performance of a pseudo-operational (real time) limited-area ensemble Kalman filter (EnKF) based on the Weather Research and Forecasting Model is described. This system assimilates conventional observations from surface stations, rawinsondes, the Aircraft Communications Addressing and Reporting System (ACARS), and cloud motion vectors every 6 h on a domain that includes the eastern North Pacific Ocean and western North America. Ensemble forecasts from this system and deterministic output from operational numerical weather prediction models during this same period are verified against rawinsonde and surface observation data. Relative to operational forecasts, the forecast from the ensemble-mean analysis has slightly larger errors in wind and temperature but smaller errors in moisture, even though satellite radiances are not assimilated by the EnKF. Time-averaged correlations indicate that assimilating ACARS and cloud wind data with flow-dependent error statistics provides corrections to the moisture field in the absence of direct observations of that field. Comparison with a control experiment in which a deterministic forecast is cycled without observation assimilation indicates that the skill in the EnKF’s forecasts results from assimilating observations and not from lateral boundary conditions or the model formulation. Furthermore, the ensemble variance is generally in good agreement with the ensemble-mean error and the spread increases monotonically with forecast hour. [ABSTRACT FROM AUTHOR]

Balance dynamics are proposed in a probabilistic framework, assuming that the state variables and the master, or control, variables are random variables described by continuous probability density functions. Balance inversion, defined as recovering the state variables from the control variables, is achieved through Bayes’ theorem. Balance dynamics are defined by the propagation of the joint probability of the state and control variables through the Liouville equation. Assuming Gaussian statistics, balance inversion reduces to linear regression of the state variables onto the control variables, and assuming linear dynamics, balance dynamics reduces to a Kalman filter subject to perfect observations given by the control variables. Example solutions are given for an elliptical vortex in shallow water having unity Rossby and Froude numbers, which produce an outward-propagating pulse of inertia–gravity wave activity. Applying balance inversion to the potential vorticity reveals that, because potential vorticity and divergence share well-defined patterns of covariability, the inertia–gravity wave field is recovered in addition to the vortical field. Solutions for a probabilistic balance dynamics model applied to the elliptical vortex reveal smaller errors (“imbalance”) for height control compared to potential vorticity control. Important attributes of the probabilistic balance theory include quantification of the concept of balance manifold “fuzziness,” and clear state-independent definitions of balance and imbalance in terms of the range of the probabilistic inversion operators. Moreover, the theory provides a generalization of the notion of balance that may prove useful for problems involving moist physics, chemistry, and tropical circulations. [ABSTRACT FROM AUTHOR]

The sensitivity of forecasts to observations is evaluated using an ensemble approach with data drawn from a pseudo-operational ensemble Kalman filter. For Gaussian statistics and a forecast metric defined as a scalar function of the forecast variables, the effect of observations on the forecast metric is quantified by changes in the metric mean and variance. For a single observation, expressions for these changes involve a product of scalar quantities, which can be rapidly evaluated for large numbers of observations. This technique is applied to determining climatological forecast sensitivity and predicting the impact of observations on sea level pressure and precipitation forecast metrics. The climatological 24-h forecast sensitivity of the average pressure over western Washington State shows a region of maximum sensitivity to the west of the region, which tilts gently westward with height. The accuracy of ensemble sensitivity predictions is tested by withholding a single buoy pressure observation from this region and comparing this perturbed forecast with the control case where the buoy is assimilated. For 30 cases, there is excellent agreement between these forecast differences and the ensemble predictions, as measured by the forecast metric. This agreement decreases for increasing numbers of observations. Nevertheless, by using statistical confidence tests to address sampling error, the impact of thousands of observations on forecast-metric variance is shown to be well estimated by a subset of the O(100) most significant observations. [ABSTRACT FROM AUTHOR]

The sensitivity of numerical weather forecasts to small changes in initial conditions is estimated using ensemble samples of analysis and forecast errors. Ensemble sensitivity is defined here by linear regression of analysis errors onto a given forecast metric. It is shown that ensemble sensitivity is proportional to the projection of the analysis-error covariance onto the adjoint-sensitivity field. Furthermore, the ensemble-sensitivity approach proposed here involves a small calculation that is easy to implement. Ensemble- and adjoint-based sensitivity fields are compared for a representative wintertime flow pattern near the west coast of North America for a 90-member ensemble of independent initial conditions derived from an ensemble Kalman filter. The forecast metric is taken for simplicity to be the 24-h forecast of sea level pressure at a single point in western Washington State. Results show that adjoint and ensemble sensitivities are very different in terms of location, scale, and magnitude. Adjoint-sensitivity fields reveal mesoscale lower-tropospheric structures that tilt strongly upshear, whereas ensemble-sensitivity fields emphasize synoptic-scale features that tilt modestly throughout the troposphere and are associated with significant weather features at the initial time. Optimal locations for targeting can easily be determined from ensemble sensitivity, and results indicate that the primary targeting locations are located away from regions of greatest adjoint and ensemble sensitivity. It is shown that this method of targeting is similar to previous ensemble-based methods that estimate forecast-error variance reduction, but easily allows for the application of statistical confidence measures to deal with sampling error. [ABSTRACT FROM AUTHOR]

Ensemble Kalman filter (EnKF) data assimilation experiments are conducted on a limited-area domain over the Pacific Northwest region of the United States, using the Weather Research and Forecasting model. Idealized surface pressure, radiosoundings, and aircraft observations are assimilated every 6 h for a 7-day period in January 2004. The objectives here are to study the performance of the filter in constraining analysis errors with a relatively inhomogeneous, sparse-observation network and to explore the potential for such a network to serve as the basis for a real-time EnKF system dedicated to the Pacific Northwest region of the United States. When only a single observation type is assimilated, results show that the ensemble-mean analysis error and ensemble spread (standard deviation) are significantly reduced compared to a control ensemble without assimilation for both observed and unobserved variables. Analysis errors are smaller than background errors over nearly the entire domain when averaged over the 7-day period. Moreover, comparisons of background errors and observation increments at each assimilation step suggest that the flow-dependent filter corrections are accurate in both scale and amplitude. An illustrative example concerns a misspecified mesoscale 500-hPa short-wave trough moving along the British Columbia coast, which is corrected by surface pressure observations alone. The relative impact of each observation type upon different variables and vertical levels is also discussed. [ABSTRACT FROM AUTHOR]

One aspect of implementing a limited-area ensemble Kalman filter (EnKF) involves the specification of a suitable ensemble of lateral boundary conditions. Two classes of methods to populate a boundary condition ensemble are proposed. In the first class, the ensemble of boundary conditions is provided by an EnKF on a larger domain and is approximately a random draw from the probability distribution function for the forecast (or analysis) on the limited-area domain boundary. The second class perturbs around a deterministic estimate of the state using assumed spatial and temporal covariance relationships. Methods in the second class are relatively flexible and easy to implement. Experiments that test the utility of these methods are performed for both an idealized low-dimensional model and limited-area simulations using the Weather Research and Forecasting (WRF) model; all experiments employ simulated observations under the perfect model assumption. The performance of the ensemble boundary condition methods is assessed by comparing the results of each experiment against a control “global” EnKF that extends beyond the limited-area domain. For all methods tested, results show that errors for the limited-area EnKF are larger near the lateral boundaries than those from a control EnKF, but decay inside the limited-area domain so that errors there are comparable to the control case. The relatively larger errors near the boundaries in the limited-area EnKF originate from not assimilating observations outside the limited-area domain and, in the second class of methods, from deficiencies in boundary spatial and temporal covariances. Overall, these experiments suggest that for observation densities typical in numerical weather prediction models, ensemble boundary conditions can be specified in the absence of a global ensemble without significant penalty in the domain interior by perturbing around an ensemble mean. [ABSTRACT FROM AUTHOR]