1. Demystifying Fisher Information: What Observation Data Reveal about Our Models
- Author
-
Judith Schenk, William Navidi, and Eileen P. Poeter
- Subjects
010504 meteorology & atmospheric sciences ,0208 environmental biotechnology ,Uncertainty ,02 engineering and technology ,Models, Theoretical ,Information theory ,01 natural sciences ,Model complexity ,020801 environmental engineering ,symbols.namesake ,Jacobian matrix and determinant ,symbols ,Errors-in-variables models ,Applied mathematics ,Entropy (information theory) ,Boundary value problem ,Computers in Earth Sciences ,Observation data ,Fisher information ,Groundwater ,0105 earth and related environmental sciences ,Water Science and Technology ,Mathematics - Abstract
Information theory is the basis for understanding how information is transmitted as observations. Observation data can be used to compare uncertainty on parameter estimates and predictions between models. Jacobian Information (JI) is quantified as the determinant of the weighted Jacobian (sensitivity) matrix. Fisher Information (FI) is quantified as the determinant of the weighted FI matrix. FI measures the relative disorder of a model (entropy) in a set of models. One-dimensional models are used to demonstrate the relationship between JI and FI, and the resulting uncertainty on estimated parameter values and model predictions for increasing model complexity, different model structures, different boundary conditions, and over-fitted models. Greater model complexity results in increased JI accompanied by an increase in parameter and prediction uncertainty. FI generally increases with increasing model complexity unless model error is large. Models with lower FI have a higher level of disorder (increase in entropy) which results in greater uncertainty of parameter estimates and model predictions. A constant-head boundary constrains the heads in the area near the boundary, reducing sensitivity of simulated equivalents to estimated parameters. JI and FI are lower for this boundary condition as compared to a constant-outflow boundary in which the heads in the area of the boundary can adjust freely. Complex, over-fitted models, in which the structure of the model is not supported by the observation dataset, result in lower JI and FI because there is insufficient information to estimate all parameters in the model.
- Published
- 2018