The importance of state transformations when using the ensemble Kalman filter for unsaturated flow modeling: Dealing with strong nonlinearities

The ensemble Kalman filter is, due to its computational efficiency, becoming more and more popular as a method for estimating both model states and parameters in hydrologic modeling, also for nonlinear state propagation models. In the ensemble Kalman filter the calculation of the error correlations, and hence the filter update, is done based on the ensemble of model evaluations and can therefore be strongly influenced by a few ensemble members with extreme values. With nonlinear state propagation models, extreme values can be a common phenomenon that can be, especially if there are nonlinearities between the observed variable and the modeled states, problematic during the filter update. An illustrative example of this problem is shown using an unsaturated flow model where the modeled states are pressure heads and observations are water content. It is demonstrated that the ensemble Kalman filter can in this case yield a deterioration of state predictions. We discuss the normal score transform and the transform with the retention function applied to the model states in order to mitigate this problem. It is shown that both transforms improve the estimation of the model states and parameters.