Back to Search Start Over

Robust imputation method with context-aware voting ensemble model for management of water-quality data.

Authors :
Choi, Junhyuk
Lim, Kyoung Jae
Ji, Bongjun
Source :
Water Research. Sep2023, Vol. 243, pN.PAG-N.PAG. 1p.
Publication Year :
2023

Abstract

• Identified factors affecting imputation accuracy and examined effects on various methods. • Developed a robust imputation model using dynamic weighting based on missingness scenarios. • Experiments on real-world water-quality data show superior accuracy of proposed method. • Proposed method's robustness highlighted by outperforming baselines in various scenarios. Water-quality monitoring and management are crucial for ensuring the safety and sustainability of water resources. However, missing data is a frequent problem in water-quality datasets, which can result in biased results in hydrological modeling and data analysis. While classic statistical methods and emerging machine/deep learning methods have been applied for imputing missing values, most existing studies perform well in specific missing scenarios, but not in universal scenarios. Therefore, existing imputation methods often fail to robustly impute missing values across various scenarios. To address the problem, we propose an imputation method that uses a context-aware voting-ensemble model to dynamically select optimal weights to integrate various imputation models across different missingness scenarios. For first identify the attributes of missingness scenarios that influence imputation accuracy. Then after introducing missing values in collected data according to the missingness scenarios, we measure the accuracy of various imputation models across the missingness scenarios. Weights of imputation models are optimized by estimating non-linear functions with regression model that can capture relationships between missingness scenarios and imputation accuracies of models. The final imputed value of the ensemble model for a missing scenario can be determined by multiplying each imputation model's weight by its imputed value, then summing the products. The method inherits the advantages of state-of-art imputation models, including the ability to learn long-term dependencies in time series, as well as the flexibility of using a dynamic weighting strategy to process various missingness scenarios. To validate the superiority of our method, we evaluate on real-world water-quality data from a river in South Korea. The proposed method achieves higher accuracy and lower variation of imputed values than baseline models across various missingness scenarios. Furthermore, we showed the applicability of our method to various hydrological environment by validating our method on industrial water quality dataset. This study highlights the potential value of the ensemble model with dynamic weighting in robust imputation of water-quality data. [Display omitted] [ABSTRACT FROM AUTHOR]

Details

Language :
English
ISSN :
00431354
Volume :
243
Database :
Academic Search Index
Journal :
Water Research
Publication Type :
Academic Journal
Accession number :
171339686
Full Text :
https://doi.org/10.1016/j.watres.2023.120369