Search

With the ultimate goal of developing models that involve the use of environmental variables, a GIS-based application is being developed that is circumscribed to the region of Galicia (Spain). Ten-minute data of six meteorological variables were collected from 150 stations of the MeteoGalicia network over a period of 18 years, but the time series data are not complete. In order to estimate missing rainfall data, four imputation methods were evaluated in this study: missForest, MICE, Amelia II, and inverse distance weighting (IDW). Crossvalidation results show that the precipitation is out of phase in the different stations due to their geographical locations, and the imputation can be improved with a displacement of the time series; on the other hand, the missForest method provided better results in the imputation of this meteorological variable than the MICE, Amelia, or IDW. [ABSTRACT FROM AUTHOR]

In this paper, we study the problem of extracting trends from time series data involving missing values. In particular, we investigate a general class of procedures that impute the missing data and then extract trends using seasonal-trend decomposition based on loess (STL), where loess stands for locally weighted smoothing, a popular tool for describing the regression relationship between two variables by a smooth curve. We refer to them as the imputation-STL procedures. Two results are obtained in this paper. First, we settle a theoretical issue, namely the connection between imputation error and the overall error from estimating the trend. Specifically, we derive the bounds for the overall error in terms of the imputation error. This subsequently facilitates the error analysis of any imputation-STL procedure and justifies its use in practice. Second, we investigate loess-STL, a particular imputation-STL procedure with the imputation also being performed using loess. Through both theoretical arguments and simulation results, we show that loess-STL has the capacity of handling a high proportion of missing data and providing reliable trend estimates if the underlying trend is smooth and the missing data are dispersed over the time series. In addition to mathematical derivations and simulation study, we apply our loess-STL procedure to profile radiosonde records of upper air temperature at 22 Antarctic research stations covering the past 50 years. For purpose of illustration, we present in this paper only the results for Novolazaravskaja station which has temperature records with more than 8.4% dispersed missing values at 8 pressure levels from October/1969 to March/2011. [ABSTRACT FROM AUTHOR]

In recent years, the development of computer technology has promoted the informatization and intelligentization of hospital management systems and thus produced a large amount of medical data. These medical data are valuable resources for research. We can obtain inducers and unknown symptoms that can help discover diseases and make earlier diagnoses. Hypertensive disorder in pregnancy (HDP) is a common obstetric complication in pregnant women, which has severe adverse effects on the life safety of pregnant women and fetuses. However, the early and mid-term symptoms of HDP are not obvious, and there is no effective solution for it except for terminating the pregnancy. Therefore, detecting and preventing HDP is of great importance. This study aims at the preprocessing of pregnancy examination data, which serves as a part of HDP prediction. We found that the problem of missing data has a large impact on HDP prediction. Unlike general data, pregnancy examination data have high dimension and a high missing rate, are in a time series, and often have many non-linear relations. Current methods are not able to process the data effectively. To this end, we propose an improved bi-LSTM-based missing value imputation approach. It combines traditional machine learning and bidirectional LSTM to deal with missing data of pregnancy examination data. Our missing value imputation method obtains a good effect and improves the accuracy of the later prediction of HDP using examination data. [ABSTRACT FROM AUTHOR]

Accurate estimation of precipitation patterns is essential for the modeling of hydrological systems and for the planning and management of water resources. However, rainfall time series, as obtained from traditional rain gauges, are frequently corrupted by missing values that might hinder frequency analysis, hydrological and environmental modeling, and meteorological drought monitoring. In this paper, we evaluated three techniques for filling missing values at daily and monthly time scales, namely, simple linear regression, multiple linear regression, and the direct imputation of satellite retrievals from the Global Precipitation Measurement (GPM) mission, in rainfall gauging stations located in the Brazilian midwestern region. Our results indicated that, despite the relatively low predictive skills of the models at the daily scale, the satellite retrievals provided moderately more accurate estimates, with better representations of the temporal dynamics of the dry and wet states and of the largest observed rainfall events in most testing sites in comparison to the statistical models. At the monthly scale, the performance of the three methods was similar, but the regression-based models were unable to reproduce the seasonal characteristics of the precipitation records, which, at least to some extent, were circumvented by the satellite products. As such, the satellite retrievals might comprise a useful alternative for dealing with missing values in rainfall time series, especially in those regions with complex spatial precipitation patterns. [ABSTRACT FROM AUTHOR]

The problem addressed by dictionary learning (DL) is the representation of data as a sparse linear combination of columns of a matrix called dictionary. Both the dictionary and the sparse representations are learned from the data. We show how DL can be employed in the imputation of multivariate time series. We use a structured dictionary, which is comprised of one block for each time series and a common block for all the time series. The size of each block and the sparsity level of the representation are selected by using information theoretic criteria. The objective function used in learning is designed to minimize either the sum of the squared errors or the sum of the magnitudes of the errors. We propose dimensionality reduction techniques for the case of high-dimensional time series. For demonstrating how the new algorithms can be used in practical applications, we conduct a large set of experiments on five real-life data sets. The missing data (MD) are simulated according to various scenarios where both the percentage of MD and the length of the sequences of MD are considered. This allows us to identify the situations in which the novel DL-based methods are superior to the existing methods. [ABSTRACT FROM AUTHOR]

A great challenge for intelligent transportation systems (ITS) is missing traffic data. Traffic data are input from various transportation applications. In the past few decades, several methods for traffic temporal data imputation have been proposed. A key issue is that temporal information collected by neighbor detectors can make traffic missing data imputation more accurate. This review analyzes traffic temporal data imputation methods. Research methods, missing patterns, assumptions, imputation styles, application conditions, limitations, and public datasets are reviewed. Then, five representative methods are tested under different missing patterns and missing ratios. California performance measurement system (PeMS) data including traffic volume and speed are selected to conduct the test. Probabilistic principal component analysis performs the best under the most conditions. [ABSTRACT FROM AUTHOR]

Missing observations in time series will distort the data characteristics, change the dataset expectations, high-order distances, and other statistics, and increase the difficulty of data analysis. Therefore, data imputation needs to be performed first. Generally, data imputation methods include statistical imputation, regression imputation, multiple imputation, and imputation based on machine learning methods. However, these methods currently have problems such as insufficient utilization of time characteristics, low imputation efficiency, and poor performance under high missing rates. In response to these problems, we propose the informer-WGAN, a network model based on adversarial training and a self-attention mechanism. With the help of the discriminator network and the random missing rate training method, the informer-WGAN can efficiently solve the problem of multidimensional time series imputation. According to the experimental results under different missing rates, the informer-WGAN model achieves better imputation results than the original informer on two datasets. Our model also shows excellent performance on time series imputation of the key parameters of a spacecraft control moment gyroscope (CMG). [ABSTRACT FROM AUTHOR]

The data obtained from air quality monitoring stations, which are used to carry out studies using data mining techniques, present the problem of missing values. This paper describes a research work on missing data imputation. Among the most common methods, the method that best imputes values to the available data set is analysed. It uses an algorithm that randomly replaces all known values in a dataset once with imputed values and compares them with the actual known values, forming several subsets. Data from seven stations in the Silesian region (Poland) were analyzed for hourly concentrations of four pollutants: nitrogen dioxide (NO2), nitrogen oxides (NOx), particles of 10 μm or less (PM10) and sulphur dioxide (SO2) for five years. Imputations were performed using linear imputation (LI), predictive mean matching (PMM), random forest (RF), k-nearest neighbours (k-NN) and imputation by Kalman smoothing on structural time series (Kalman) methods and performance evaluations were performed. Once the comparison method was validated, it was determine that, in general, Kalman structural smoothing and the linear imputation methods best fitted the imputed values to the data pattern. It was observed that each imputation method behaves in an analogous way for the different stations The variables with the best results are NO2 and SO2. The UMI method is the worst imputer for missing values in the data sets. [ABSTRACT FROM AUTHOR]

Independent component analysis (ICA) is one of the most effective approaches in extracting independent signals from a global navigation satellite system (GNSS) regional station network. However, ICA requires the involved time series to be complete, thereby the missing data of incomplete time series should be interpolated beforehand. In this contribution, a modified ICA is proposed, by which the missing data are first recovered based on the reversible property between the original time series and decomposed principal components, then the complete time series are further processed with FastICA. To evaluate the performance of the modified ICA for extracting independent components, 24 regional GNSS network stations located in North China from 2011 to 2019 were selected. After the trend, annual and semiannual terms were removed from the GNSS time series, the first two independent components captured 17.42, 18.44 and 17.38% of the total energy for the North, East and Up coordinate components, more than those derived by the iterative ICA that accounted for 16.21%, 17.72% and 16.93%, respectively. Therefore, modified ICA can extract more independent signals than iterative ICA. Subsequently, selecting the 7 stations with less missing data from the network, we repeatedly process the time series after randomly deleting parts of the data and compute the root mean square error (RMSE) from the differences of reconstructed signals before and after deleting data. All RMSEs of modified ICA are smaller than those of iterative ICA, indicating that modified ICA can extract more exact signals than iterative ICA. [ABSTRACT FROM AUTHOR]

Airborne pollen monitoring datasets sometimes exhibit gaps, even very long, either because of maintenance or because of a lack of expert personnel. Despite the numerous imputation techniques available, not all of them effectively include the spatial relations of the data since the assumption of missing-at-random is made. However, there are several techniques in geostatistics that overcome this limitation such as the inverse distance weighting and Gaussian processes or kriging. In this paper, a new method is proposed that utilizes convolutional neural networks. This method not only shows a competitive advantage in terms of accuracy when compared to the aforementioned techniques by improving the error by 5% on average, but also reduces execution training times by 90% when compared to a Gaussian process. To show the advantages of the proposal, 10%, 20%, and 30% of the data points are removed in the time series of a Poaceae pollen observation station in the region of Madrid, and the airborne concentrations from the remaining available stations in the network are used to impute the data removed. Even though the improvements in terms of accuracy are not significantly large, even if consistent, the gain in computational time and the flexibility of the proposed convolutional neural network allow field experts to adapt and extend the solution, for instance including meteorological variables, with the potential decrease of the errors reported in this paper. [ABSTRACT FROM AUTHOR]