Search

The presence of missing components in incomplete instances precludes a kernel-based model from incorporating partially observed components of incomplete instances and computing kernels, including Gaussian kernels that are extensively used in machine learning modeling and applications. Existing methods with Gaussian kernels to handle incomplete data, however, are based on independence among variables. In this study, we propose a new method, the expected Gaussian kernel with correlated variables, that estimates the Gaussian kernel with incomplete data, by considering the correlation among variables. In the proposed method, the squared distance between two instance vectors is modeled with the sum of the correlated squared unit-dimensional distances between the instances, and the Gaussian kernel with missing values is obtained by estimating the expected Gaussian kernel function under the probability distribution for the squared distance between the vectors. The proposed method is evaluated on synthetic data and real-life data from benchmarks and a case from a multi-pattern photolithographic process for wafer fabrication in semiconductor manufacturing. The experimental results show the improvement by the proposed method in the estimation of Gaussian kernels with incomplete data of correlated variables. [ABSTRACT FROM AUTHOR]

Multiple imputation is a practical approach in analyzing incomplete data, with multiple imputation by chained equations (MICE) being popularly used. MICE specifies a conditional distribution for each variable to be imputed, but estimating it is inherently a high-dimensional problem for large-scale data. Existing approaches propose to utilize regularized regression models, such as lasso. However, the estimation of them occurs iteratively across all incomplete variables, leading to a considerable increase in computational burden, as demonstrated in our simulation study. To overcome this computational bottleneck, we propose a novel method that estimates the conditional independence structure among variables before the imputation procedure. We extract such information from an undirected graphical model, leveraging the graphical lasso method based on the inverse probability weighting estimator. Our simulation study verifies the proposed method is way faster against the existing methods, while still maintaining comparable imputation performance. [ABSTRACT FROM AUTHOR]

For multi-view data in reality, part of its elements may be missing because of human or machine error. Incomplete multi-view clustering (IMC) clusters the incomplete multi-view data according to the characters of various views of the instances. Recently, IMC has attracted much attention and many related methods have been proposed. However, the existing approaches still need to be developed and innovated in the following aspects: (1) current methods only consider the differences of different views, while the different influences of instances, as well as distinguishes between missing values and completed values are ignored. (2) The updating scheme for weighting matrix in adaptive weighted algorithms usually relies on an optimization sub-problem, whose optimal solution may not be easy to achieve. (3) The adaptive weighted subspace algorithms that can recover the incomplete data are anchor types. The randomness of the anchor matrix may cause unreliability. To tackle these limitations, we propose an adaptive weighted self-representation (AWSR) subspace method for IMC. The AWSR method tunes the weighting matrix adaptively in accordance with the views of different instances and the recovery process of the missing values. The low rank and smoothness constraints on the representation matrix make the subspace reveal the underlying features of the dataset accurately. We also analyze the convergence property of the block coordinate method for our optimization model theoretically. Numerical performance on five real-world data shows that the AWSR method is effective and delivers superior results when compared to other eight widely-used approaches considering the clustering accuracy (ACC), normalized mutual information (NMI) and Purity. [ABSTRACT FROM AUTHOR]

Technological advances in information-processing capacity have enabled integrated analyses (multi-omics) of different omics data types, improving target discovery and clinical diagnosis. This study proposes novel artificial intelligence (AI) learning strategies for incomplete datasets, common in omics research. The model comprises (1) a multi-omics generative model based on a variational auto-encoder that learns tumor genetic patterns based on different omics data types and (2) an expanded classification model that predicts cancer phenotypes. Padding was applied to replace missing data with virtual data. The embedding data generated by the model accurately classified cancer phenotypes, addressing the class imbalance issue (weighted F1 score: cancer type > 0.95, primary site > 0.92, sample type > 0.97). The classification performance was maintained in the absence of omics data, and the virtual data resembled actual omics data (cosine similarity mRNA gene expression > 0.96, mRNA isoform expression > 0.95, DNA methylation > 0.96). Meanwhile, in the presence of omics data, high-quality, non-existent omics data were generated (cosine similarity mRNA gene expression: 0.9702, mRNA isoform expression: 0.9546, DNA methylation: 0.9687). This model can effectively classify cancer phenotypes based on incomplete omics data with data sparsity robustness, generating omics data through deep learning and enabling precision medicine. [ABSTRACT FROM AUTHOR]

Objectives: Evaluate the impact of missing data on observed racial disparities in the likelihood of an incarceration sentence, given that complete case analysis in the common analytic approach used in criminological research. Methods: Using a simulation study with data based on cases sentenced in the Court of Common Pleas in Pennsylvania between 2010 and 2019, we assess the differences in the likelihood of incarceration between similarly situated White and Black defendants based on varying sample sizes and patterns of missing data. Results: Complete case analysis (CCA) of incomplete data can fail to provide unbiased estimates of the race effect, even with less than 10% of cases missing. The degree of bias introduced depends on the amount, pattern, assumptions, and treatment of missing data. Multiple imputation provides an established, valid methodology for the unbiased estimation of race effects when data are missing at random, and this holds across sample sizes and number of imputations. Conclusions: The existence and magnitude of race effects on the likelihood of an incarceration sentence can vary greatly based on the degree, pattern, assumptions, and treatment of missing data. Limitations include that missing data mechanisms cannot be truly known outside of a data simulation. Future sentencing research should prioritize the identification, treatment, and reporting of missing data prior to isolating race effects, in line with calls from the field for more open science practices. Sensitivity analyses should also be prioritized. [ABSTRACT FROM AUTHOR]

Missing data are a pervasive problem in data analysis. Three common methods for addressing the problem are (a) complete-case analysis, where only units that are complete on the variables in an analysis are included; (b) weighting, where the complete cases are weighted by the inverse of an estimate of the probability of being complete; and (c) multiple imputation (MI), where missing values of the variables in the analysis are imputed as draws from their predictive distribution under an implicit or explicit statistical model, the imputation process is repeated to create multiple filled-in data sets, and analysis is carried out using simple MI combining rules. This article provides a non-technical discussion of the strengths and weakness of these approaches, and when each of the methods might be adopted over the others. The methods are illustrated on data from the Youth Cohort (Time) Series (YCS) for England, Wales and Scotland, 1984–2002. [ABSTRACT FROM AUTHOR]

Abstract A method for skyline query of multidimensional incomplete data based on a classification tree has been proposed to address the problem of a large amount of useless data in existing skyline queries with multidimensional incomplete data, which leads to low query efficiency and algorithm performance. This method consists of two main parts. The first part is the proposed incomplete data weighted classification tree algorithm. In the first part, an incomplete data weighted classification tree is proposed, and the incomplete data set is classified using this tree. The data classified in the first part serves as the basis for the second step of the query. The second part proposes a skyline query algorithm for multidimensional incomplete data. The concept of optimal virtual points has been recently introduced, effectively reducing the number of comparisons of a large amount of data, thereby improving the query efficiency for incomplete data. Theoretical research and experimental analysis have shown that the proposed method can perform skyline queries for multidimensional incomplete data well, with high query efficiency and accuracy of the algorithm.

Abstract Background Binary outcomes are likely the most common in randomized controlled trials, but ordinal outcomes can also be of interest. For example, rather than simply collecting data on diseased versus healthy study subjects, investigators may collect information on the severity of disease, with no disease, mild, moderate, and severe disease as possible levels of the outcome. While some investigators may be interested in all levels of the ordinal variable, others may combine levels that are not of particular interest. Therefore, when research synthesizers subsequently conduct a network meta-analysis on a network of trials for which an ordinal outcome was measured, they may encounter a network in which outcome categorization varies across trials. Methods The standard method for network meta-analysis for an ordinal outcome based on a multinomial generalized linear model is not designed to accommodate the multiple outcome categorizations that might occur across trials. In this paper, we propose a network meta-analysis model for an ordinal outcome that allows for multiple categorizations. The proposed model incorporates the partial information provided by trials that combine levels through modification of the multinomial likelihoods of the affected arms, allowing for all available data to be considered in estimation of the comparative effect parameters. A Bayesian fixed effect model is used throughout, where the ordinality of the outcome is accounted for through the use of the adjacent-categories logit link. Results We illustrate the method by analyzing a real network of trials on the use of antibiotics aimed at preventing liver abscesses in beef cattle and explore properties of the estimates of the comparative effect parameters through simulation. We find that even with the categorization of the levels varying across trials, the magnitudes of the biases are relatively small and that under a large sample size, the root mean square errors become small as well. Conclusions Our proposed method to conduct a network meta-analysis for an ordinal outcome when the categorization of the outcome varies across trials, which utilizes the adjacent-categories logit link, performs well in estimation. Because the method considers all available data in a single estimation, it will be particularly useful to research synthesizers when the network of interest has only a limited number of trials for each categorization of the outcome.

Abstract Integrated step-selection analyses (iSSAs) are versatile and powerful frameworks for studying habitat and movement preferences of tracked animals. iSSAs utilize integrated step-selection functions (iSSFs) to model movements in discrete time, and thus, require animal location data that are regularly spaced in time. However, many real-world datasets are incomplete due to tracking devices failing to locate an individual at one or more scheduled times, leading to slight irregularities in the duration between consecutive animal locations. To address this issue, researchers typically only consider bursts of regular data (i.e., sequences of locations that are equally spaced in time), thereby reducing the number of observations used to model movement and habitat selection. We reassess this practice and explore four alternative approaches that account for temporal irregularity resulting from missing data. Using a simulation study, we compare these alternatives to a baseline approach where temporal irregularity is ignored and demonstrate the potential improvements in model performance that can be gained by leveraging these additional data. We also showcase these benefits using a case study on a spotted hyena (Crocuta crocuta).

Effect modification plays a central role in stratified medicine, of which the goal is often to find biomarker profiles that identify individuals who benefit from a treatment of interest. We consider the problem of causal inference regarding the effect modifying role of a biomarker, which is only available for some individuals in an observational study. We develop inverse probability weighted mean score estimating functions with one weight to account for confounding and a second weight for the missing data process. An iterative approach is described for solving the equations in the spirit of the expectation-maximization algorithm, and large sample properties of the resulting estimator are developed. Simulation studies are conducted to compare the proposed method with a doubly weighted complete case analysis and a propensity score weighted multiple imputation approach. An application to a study of the effect of a biologic therapy on inflammation in a rheumatology cohort is given for illustration. [ABSTRACT FROM AUTHOR]

Theoretically, the maximum likelihood estimator has the sandwich-type asymptotic variance-covariance matrix under model misspecification. Its empirical estimator, that is called the robust variance estimator, is consistent. Thus, the estimator is asymptotically valid even under model misspecification. In practice, the robust variance estimator is used for computation of standard errors in longitudinal data analysis. Recently, Golden et al. (2019 Econometrics, 7, 1-27) showed that the maximum likelihood estimator retains a sandwich-type asymptotic variance-covariance matrix in the presence of missing data even when the missing-data mechanism is missing not at random. Although they revealed the asymptotic validity of the robust variance estimator in the simultaneous presence of both model misspecification and missing data, its finite-sample performance did not be investigated. In this article, we evaluated the finite-sample performance via simulation studies and clarify its small-sample problems. In addition, we illustrated the robust variance estimator using longitudinal CD4 count data from a randomized double-blind study. [ABSTRACT FROM AUTHOR]

A networked time series (NETS) is a family of time series on a given graph, one for each node. It has a wide range of applications from intelligent transportation to environment monitoring to smart grid management. An important task in such applications is to predict the future values of a NETS based on its historical values and the underlying graph. Most existing methods require complete data for training. However, in real-world scenarios, it is not uncommon to have missing data due to sensor malfunction, incomplete sensing coverage, and so on. In this article, we study the problem of NETS prediction with incomplete data. We propose networked time series Imputation Generative Adversarial Network (NETS-ImpGAN), a novel deep learning framework that can be trained on incomplete data with missing values in both history and future. Furthermore, we propose Graph Temporal Attention Networks, which incorporate the attention mechanism to capture both inter-time series and temporal correlations. We conduct extensive experiments on four real-world datasets under different missing patterns and missing rates. The experimental results show that NETS-ImpGAN outperforms existing methods, reducing the Mean Absolute Error by up to 25%. [ABSTRACT FROM AUTHOR]

Electrical impedance tomography (EIT) is a technique to obtain conductivity maps from electrical voltage measurements in a region of interest. In this work, we discuss the proposal we submitted to the Kuopio Tomography Challenge 2023, whose aim was to reconstruct and segment EIT images obtained from limited data, after electrode disconnection. Our proposal, denoted 01A, consisted of an initial reconstruction using the smoothness prior and post-processing steps, including denoising and deblurring with a convolutional neural network (CNN), as a way to integrate deep learning and inverse problems. The score was calculated using the structural similarity index in 21 test cases. While the score of the reconstruction using only smoothness prior was 9.69, the score of 01A was 12.75. Also, we developed an improved proposal, denoted 01A+, using hyperparameter optimization and its score was 13.30. We obtained better results using 01A and 01A+ than using the original smoothness prior, but the proposals lacked consistency when more electrodes were disconnected and when the targets were too different from the CNN training data. Even so, 01A obtained second place at KTC2023, representing a way to remove artifacts from electrode disconnection in EIT reconstructions. [ABSTRACT FROM AUTHOR]

Background: Binary outcomes are likely the most common in randomized controlled trials, but ordinal outcomes can also be of interest. For example, rather than simply collecting data on diseased versus healthy study subjects, investigators may collect information on the severity of disease, with no disease, mild, moderate, and severe disease as possible levels of the outcome. While some investigators may be interested in all levels of the ordinal variable, others may combine levels that are not of particular interest. Therefore, when research synthesizers subsequently conduct a network meta-analysis on a network of trials for which an ordinal outcome was measured, they may encounter a network in which outcome categorization varies across trials. Methods: The standard method for network meta-analysis for an ordinal outcome based on a multinomial generalized linear model is not designed to accommodate the multiple outcome categorizations that might occur across trials. In this paper, we propose a network meta-analysis model for an ordinal outcome that allows for multiple categorizations. The proposed model incorporates the partial information provided by trials that combine levels through modification of the multinomial likelihoods of the affected arms, allowing for all available data to be considered in estimation of the comparative effect parameters. A Bayesian fixed effect model is used throughout, where the ordinality of the outcome is accounted for through the use of the adjacent-categories logit link. Results: We illustrate the method by analyzing a real network of trials on the use of antibiotics aimed at preventing liver abscesses in beef cattle and explore properties of the estimates of the comparative effect parameters through simulation. We find that even with the categorization of the levels varying across trials, the magnitudes of the biases are relatively small and that under a large sample size, the root mean square errors become small as well. Conclusions: Our proposed method to conduct a network meta-analysis for an ordinal outcome when the categorization of the outcome varies across trials, which utilizes the adjacent-categories logit link, performs well in estimation. Because the method considers all available data in a single estimation, it will be particularly useful to research synthesizers when the network of interest has only a limited number of trials for each categorization of the outcome. [ABSTRACT FROM AUTHOR]

A method for skyline query of multidimensional incomplete data based on a classification tree has been proposed to address the problem of a large amount of useless data in existing skyline queries with multidimensional incomplete data, which leads to low query efficiency and algorithm performance. This method consists of two main parts. The first part is the proposed incomplete data weighted classification tree algorithm. In the first part, an incomplete data weighted classification tree is proposed, and the incomplete data set is classified using this tree. The data classified in the first part serves as the basis for the second step of the query. The second part proposes a skyline query algorithm for multidimensional incomplete data. The concept of optimal virtual points has been recently introduced, effectively reducing the number of comparisons of a large amount of data, thereby improving the query efficiency for incomplete data. Theoretical research and experimental analysis have shown that the proposed method can perform skyline queries for multidimensional incomplete data well, with high query efficiency and accuracy of the algorithm. [ABSTRACT FROM AUTHOR]

Integrated step-selection analyses (iSSAs) are versatile and powerful frameworks for studying habitat and movement preferences of tracked animals. iSSAs utilize integrated step-selection functions (iSSFs) to model movements in discrete time, and thus, require animal location data that are regularly spaced in time. However, many real-world datasets are incomplete due to tracking devices failing to locate an individual at one or more scheduled times, leading to slight irregularities in the duration between consecutive animal locations. To address this issue, researchers typically only consider bursts of regular data (i.e., sequences of locations that are equally spaced in time), thereby reducing the number of observations used to model movement and habitat selection. We reassess this practice and explore four alternative approaches that account for temporal irregularity resulting from missing data. Using a simulation study, we compare these alternatives to a baseline approach where temporal irregularity is ignored and demonstrate the potential improvements in model performance that can be gained by leveraging these additional data. We also showcase these benefits using a case study on a spotted hyena (Crocuta crocuta). [ABSTRACT FROM AUTHOR]

When are inferences (whether Direct-Likelihood, Bayesian, or Frequentist) obtained from partial data valid? This article answers this question by offering a new asymptotic theory about inference with missing data that is more general than existing theories. It proves that as the sample size increases and the extent of missingness decreases, the average-loglikelihood function generated by partial data and that ignores the missingness mechanism will converge in probability to that which would have been generated by complete data; and if the data are Missing at Random, this convergence depends only on sample size. Thus, inferences from partial data, such as posterior modes, confidence intervals, likelihood ratios, test statistics, and indeed, all quantities or features derived from the partial-data loglikelihood function, will be consistently estimated. Additionally, the missing data mechanism has asymptotically no effect on parameter estimation and hypothesis testing if the data are Missing at Random. This adds to previous research which has only proved the consistency and asymptotic normality of the posterior mode. Practical implications are discussed, and the theory is illustrated through simulation using a previous study of International Human Rights Law. [ABSTRACT FROM AUTHOR]

Function dependency (FD) discovery methods typically focus on identifying all results that satisfy the syntax features of function dependencies. In the case of incomplete data, it often results in a significant number of established but meaningless FD. In response to the issue of discovering invalid FD, this paper proposed a method for incomplete data FD discovery based on correlation analysis. It constructed a probability distribution with attributes containing missing values using a probability graph model, captured the relationships between attributes through correlation analysis, avoided enumerating all possibilities, to discover FD with statistical significance. The experimental results show that the proposed method can more accurately pinpoint meaningful FD, more accurately, with an average F1-score improvement of 1.5 times compared to state-of-the-art FD discovery methods. [ABSTRACT FROM AUTHOR]

Assessing soil liquefaction potential is a crucial consideration in the seismic design of structures and their seismic stability. The complex nonlinear behavior of the liquefiable soils and the non-deterministic nature of earthquakes make the liquefaction analysis vague. Accordingly, researchers have progressively focused on employing machine learning and mathematical algorithms to address the complexities and uncertainties of evaluating soil liquefaction potential. This paper investigates the performance of fuzzy c-means clustering of incomplete data for assessing liquefaction potential based on cone penetration test (CPT) and shear wave velocity (Vs) field data. The research was conducted using two approaches: (1) whole data strategy; (2) partial distance strategy. The used database contains 786 CPT and 846 Vs records, with specified liquefaction conditions in past earthquake events. We compared the effectiveness and success of this method with traditional deterministic and probabilistic liquefaction evaluation approaches. It was found that the fuzzy c-means clustering model had a comparable predictive ability with other methods and would be reliable when assessing the liquefaction possibility. [ABSTRACT FROM AUTHOR]

A frequent concern in analyzing incomplete multivariate measurements in mixed categorical and quantitative scales is whether missing completely at random (MCAR) is an appropriate model. Realized MCAR refers to constancy of conditional probability at realized missing data patterns and differs from always MCAR. We develop a scalable approach for diagnostics of realized MCAR in mixed-type data for which existing methods are lacking. We interestingly establish that the null framework may hold under the broader condition of observed at random (OAR) under component independence and the method cannot detect departure in the direction of OAR under independence but may do so under dependence. We demonstrate that the proposed method is easy to implement and scalable. In the special case of non-mixed type data, we face computational difficulties with existing methods whereas the proposed approach performs superiorly. The proposed approach is applied to analyze incomplete mixed-type data from the ARCAD metastatic colorectal cancer database. [ABSTRACT FROM AUTHOR]

Characteristic viscosity is a key indicator of the quality of polyester melts, whose accurate prediction can help to identify potential quality problems of polyester melts in advance, adjust the process parameters in time and reduce enterprise losses. Considering the data incompleteness, data time series and high dimensional redundancy of the polyester melt production process, a method is proposed to predict the characteristic viscosity of polyester melt under incomplete data. A missing data generative adversarial nets (MDGAN) with a convolutional neural network discriminator and an attention long short-term memory neural network generator is designed to address the data incompleteness problem caused by the extreme production environment of polyester melts, and the missing data is filled by the adversarial generation mechanism. The extreme gradient boosting-bidirectional gated recurrent unit (XGBoost-BiGRU) is designed to predict the viscosity of polyester melts based on high dimensional redundancy and temporal characteristics prediction. The actual data test results of a polyester fiber manufacturer in Zhejiang show that the filling accuracy of the MDGAN algorithm at different missing rate data sets is better than that of data filling algorithms such as KNN,RF,MICE,and GAIN. The XGBoost-BiGRU characteristic viscosity prediction method has significant advantages over STL-GPR, CAGRU, BiGRU. In combination of MDGAN characteristic viscosity prediction, the method proposed can effectively solve the problem of predicting the characteristic viscosity of polyester melts under incomplete data.

Copyright of Journal of Shanghai Jiao Tong University (1006-2467) is the property of Journal of Shanghai Jiao Tong University Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Statistical methods have been well developed for comparing the predictive values of two binary diagnostic tests under a paired design. However, existing methods do not make allowance for incomplete data. Although maximum likelihood based method can be used to deal with incomplete data, it requires iterative algorithm for implementation. A simple and easily implemented statistical method is therefore needed. Simple methods exist for comparing two sensitivities or specificities with incomplete data but such simple methods are not available for comparing two predictive values with incomplete data. In this paper, we propose two simple methods for comparing two predictive values with incomplete data. The test statistics derived by these two methods are simple to compute, only involving some minor modification of the existing weighted generalized score statistics with complete data. Simulation results demonstrate that the proposed methods are more efficient than the ad-hoc method that only uses the subjects wit complete data. As an illustration, the proposed methods are applied to an observational study comparing two non-invasive methods in detecting endometriosis. [ABSTRACT FROM AUTHOR]

Single-case experimental designs (SCEDs) have grown in popularity in the fields such as education, psychology, medicine, and rehabilitation. Although SCEDs are valid experimental designs for determining evidence-based practices, they encounter some challenges in analyses of data. One of these challenges, missing data, is likely to be occurred frequently in SCEDs research due to repeated measurements over time. Since missing data is a critical factor that can weaken the validity and generalizability of a study, it is important to determine the characteristics of missing data in SCEDs, which are especially conducted with a small number of participants. In this regard, this study aimed to describe missing data features in SCEDs studies in detail. To accomplish this goal, 465 published SCEDs studies within the recent 5 years in six journals were included in the investigation. The overall results showed that the prevalence of missing data among SCEDs articles in at least one phase, as at least one data point, was approximately 30%. In addition, the results indicated that the missing data rates were above 10% within most studies where missing data occurred. Although missing data is so common in SCEDs research, only a handful of studies (5%) have handled missing data; however, their methods are traditional. In analyzing SCEDs data, several methods are proposed considering missing data ratios in the literature. Therefore, missing data rates determined in this study results can shed light on the analyses of SCEDs data with proper methods by improving the validity and generalizability of study results. [ABSTRACT FROM AUTHOR]

AbstractProfile monitoring is a widely used tool in quality control. The rapid development of sensor technology has created unprecedented opportunities for multi-channel profile data collection, which motivates the modeling and transfer learning for multi-profile data. The Multi-output Gaussian Process (MGP) is often used for multi-profile data, due to its flexible modeling capability and elegant mathematical properties. However, two practical concerns limit the broader application of MGP for transfer learning and monitoring of multi-profile data: high computational complexity and data incompleteness. In this article, we propose a Variational Inference (VI)-based MGP framework to facilitate transfer learning and profile monitoring using incomplete profile data. The proposed framework features a specially designed convolutional structure for constructing an explicit covariance relationship between the inducing variables in VI and the MGP in multi-profile data. This structure inspires a comprehensive solution to both computational complexity and data incompleteness in modeling multi-profile data, which facilities the transfer learning for profile monitoring. Various numerical studies and one real case study are conducted to demonstrate and compare the transfer learning and monitoring performance of the proposed method. The results show the method can achieve superior monitoring performance while maintain a very low level of computational complexity. [ABSTRACT FROM AUTHOR]

Addressing missing values is a persistent challenge in the field of data mining. The presence of incomplete data can significantly compromise the overall data quality. Consequently, it is crucial to handle incomplete data efficiently. This paper presents a novel approach for imputing missing values that incorporates Kernelized Fuzzy C-Means (KFCM) clustering and proposes a method termed LIKFCM, which combines its benefits with Linear Interpolation (LI). The proposed LIKFCM's performance is assessed through a comparison against nine state-of-the-art imputation techniques (mean, median, LI, EMI, KNNI, KMI, FKMI, LIFCM, and LIPFCM) across ten widely used real-world datasets from the UCI repository with six combinations of missing ratios to assess the efficacy of the proposed imputation method. From the experimental results, it is evident that our proposed method outperforms the existing imputation methods with significant improvements in terms of RMSE & MAE for these datasets. Additionally, experiments examining the effect of missing values validate the robustness of the proposed approach by handling different missing ratios. The performance validation of the proposed approach against other state-of-the-art imputation methods has been conducted utilizing a Kendall's W statistical test, involving a comparison of their mean ranks across different missing ratios. The outcomes indicate that LIKFCM has outperformed other imputation methods, attaining the highest rank in terms of different evaluation criteria. [ABSTRACT FROM AUTHOR]

The integrity of data is crucial for the majority of existing data analysis methods. However, incomplete and unbalanced datasets from the collection and organization process will affect analysis accuracy. Existing interpolation algorithms often overlook feature importance, resulting in either cumbersome processes or underutilization of data information. This study introduces an interpolation method based on clustering algorithm focused on improving the accuracy and efficiency of missing data processing in datasets. First, in this paper, we clarify the problem to be solved about data interpolation, and we consider the importance of the information brought by the data itself to the interpolation, so we propose a scheme that combines clustering and interpolation. We propose a new method that uses the Lp norm as a similarity measure in the K-means clustering algorithm, and introduce a controllable weighting formula based on the current data segmentation. Methodologically, the clustering and interpolation are synchronized by iteratively updating the variable optimization cost function. The experimental results demonstrate significant improvements of the proposed interpolation algorithm over traditional techniques, particularly in tasks such as data labeling and classification within real datasets for clustering and classification.

Sleep abnormalities can have severe health consequences. Automated sleep staging, i.e. labelling the sequence of sleep stages from the patient’s physiological recordings, could simplify the diagnostic process. Previous work on automated sleep staging has achieved great results, mainly relying on the EEG signal. However, often multiple sources of information are available beyond EEG. This can be particularly beneficial when the EEG recordings are noisy or even missing completely. In this paper, we propose CoRe-Sleep, a Coordinated Representation multimodal fusion network that is particularly focused on improving the robustness of signal analysis on imperfect data. We demonstrate how appropriately handling multimodal information can be the key to achieving such robustness. CoRe-Sleep tolerates noisy or missing modalities segments, allowing training on incomplete data. Additionally, it shows state-of-the-art performance when testing on both multimodal and unimodal data using a single model on SHHS-1, the largest publicly available study that includes sleep stage labels. The results indicate that training the model on multimodal data does positively influence performance when tested on unimodal data. This work aims at bridging the gap between automated analysis tools and their clinical utility.

In this paper, we analyze the length‐biased and partly interval‐censored data, whose challenges primarily come from biased sampling and interfere induced by interval censoring. Unlike existing methods that focus on low‐dimensional data and assume the covariates to be precisely measured, sometimes researchers may encounter high‐dimensional data subject to measurement error, which are ubiquitous in applications and make estimation unreliable. To address those challenges, we explore a valid inference method for handling high‐dimensional length‐biased and interval‐censored survival data with measurement error in covariates under the accelerated failure time model. We primarily employ the SIMEX method to correct for measurement error effects and propose the boosting procedure to do variable selection and estimation. The proposed method is able to handle the case that the dimension of covariates is larger than the sample size and enjoys appealing features that the distributions of the covariates are left unspecified. [ABSTRACT FROM AUTHOR]

In modeling missing data, the missing data latent variable of the confirmatory factor model accounts for systematic variation associated with missing data so that replacement of what is missing is not required. This study aimed at extending the modeling missing data approach to tetrachoric correlations as input and at exploring the consequences of switching between models with free and fixed factor loadings. In a simulation study, confirmatory factor analysis (CFA) models with and without a missing data latent variable were used for investigating the structure of data with and without missing data. In addition, the numbers of columns of data sets with missing data and the amount of missing data were varied. The root mean square error of approximation (RMSEA) results revealed that an additional missing data latent variable recovered the degree-of-model fit characterizing complete data when tetrachoric correlations served as input while comparative fit index (CFI) results showed overestimation of this degree-of-model fit. Whereas the results for fixed factor loadings were in line with the assumptions of modeling missing data, the other results showed only partial agreement. Therefore, modeling missing data with fixed factor loadings is recommended. [ABSTRACT FROM AUTHOR]

The increasing computerisation of medical services has highlighted inconsistencies in the way in which patients' historic medical data were recorded. Differences in process and practice between medical services and facilities have led to many incomplete and inaccurate medical histories being recorded. To create a single point of truth going forward, it is necessary to correct these inconsistencies. A common way to do this has been to use imputation techniques to predict missing data values based on the known values in the data set. In this paper, we propose a neighborhood similarity measure-based imputation technique and analyze its achieved prediction accuracy in comparison with a number of traditional imputation methods using both an incomplete anonymized diabetes medical data set and a number of simulated data sets as the sources of our data. The aim is to determine whether any improvement could be made in the accuracy of predicting a diabetes diagnosis using the known outcomes of the diabetes patients' data set. The obtained results have proven the effectiveness of our proposed approach compared to other state-of-the-art single-pass imputation techniques. [ABSTRACT FROM AUTHOR]

Computed tomography (CT) from full-angle data is of great importance to medical imaging and other areas. However, restrictions during data acquisition, such as measuring the target in a limited angular range, can lead to poorer quality results due to the increased difficulty of the image reconstruction problem. For challenging situations like this, traditional techniques may not be able to produce CT images with enough quality. In this study, we propose software for reconstruction of limited-angle CT images using a combination of deep image prior (DIP) and matching pursuit (MMP). Evaluation of our proposal was performed on the dataset provided by the Helsinki Tomographic Challenge 2022 (HTC 2022). Our results indicate that, by using MMP as a regularizer for the problem, the quality of DIP image reconstructions can be improved. Furthermore, we obtained images with a minimum Matthews correlation coefficient of 0.936 in five of seven difficulty levels of the HTC 2022, showing that it is a promising technique for this application. [ABSTRACT FROM AUTHOR]

With continuous progress in science and technology, a large amount of data are produced in all fields of the world at anytime and anywhere. These data are unmarked and lack marking information, while manual marking is time-consuming and laborious. Herein, this paper introduces a distributed semi-supervised labeling framework. This framework addresses the issue of missing data by proposing an attribute-filling method based on subspace learning. Furthermore, this paper presents a distributed semi-supervised learning strategy that trains sub-models (private models) within each sub-system. Finally, this paper develops a distributed graph convolutional neural network fusion technique with enhanced interpretability grounded on the attention mechanism. This paper assigns weights of importance to the edges of each layer in the graph neural network based on sub-models and public data, thereby enabling distributed and interpretable graph convolutional attention. Extensive experimentation using public datasets demonstrates the superiority of the proposed scheme over other state-of-the-art baselines, achieving a reduction in loss of 50% compared to the original approach. [ABSTRACT FROM AUTHOR]

Multistage sequential decision-making occurs in many real-world applications such as healthcare diagnosis and treatment. One concrete example is when the doctors need to decide to collect which kind of information from subjects so as to make the good medical decision cost-effectively. In this paper, an active learning-based method is developed to model the doctors' decision-making process that actively collects necessary information from each subject in a sequential manner. The effectiveness of the proposed model, especially its two-stage version, is validated on both simulation studies and a case study of common bile duct stone evaluation for pediatric patients. [ABSTRACT FROM AUTHOR]

Incomplete data are unavoidable and pose serious complications in statistical analyses of epidemiological research. However, many researchers still ignore these complications and proceed with complete case analysis. In this paper, we propose modern procedures to handle incomplete spatially correlated data. We illustrate these procedures with an application to spatially-referenced HIV data from Ghana. The main goal was to identify significant risk factors of HIV prevalence in Ghana via a Bayesian spatial hierarchical modeling approach. Results revealed substantial differences in parameter estimates and highest posterior density intervals widths obtained using our approach and that of complete case analysis. Based on these findings, our approach provides more accurate parameter estimates, reduces the chances of committing a type I error, and has higher predictive efficiency. In addition, results from these findings identified higher-risk regions that require targeted interventions by government to help reduce the spread of HIV in Ghana. [ABSTRACT FROM AUTHOR]