Search

Semi-supervised learning aims to learn prediction models from both labeled and unlabeled samples. There has been extensive research in this area. Among existing work, generative mixture models with Expectation-Maximization (EM) is a popular method due to clear statistical properties. However, existing literature on EM-based semi-supervised learning largely focuses on unstructured prediction, assuming that samples are independent and identically distributed. Studies on EM-based semi-supervised approach in structured prediction is limited. This paper aims to fill the gap through a comparative study between unstructured and structured methods in EM-based semi-supervised learning. Specifically, we compare their theoretical properties and find that both methods can be considered as a generalization of self-training with soft class assignment of unlabeled samples, but the structured method additionally considers structural constraint in soft class assignment. We conducted a case study on real-world flood mapping datasets to compare the two methods. Results show that structured EM is more robust to class confusion caused by noise and obstacles in features in the context of the flood mapping application.

We propose a ParametRIc MAnifold Learning (PRIMAL) algorithm for Gaussian mixtures models (GMM), assuming that GMMs lie on or near to a manifold of probability distributions that is generated from a low-dimensional hierarchical latent space through parametric mappings. Inspired by principal component analysis (PCA), the generative processes for priors, means and covariance matrices are modeled by their respective latent space and parametric mapping. Then, the dependencies between latent spaces are captured by a hierarchical latent space by a linear or kernelized mapping. The function parameters and hierarchical latent space are learned by minimizing the reconstruction error between ground-truth GMMs and manifold-generated GMMs, measured by Kullback-Leibler Divergence (KLD). Variational approximation is employed to handle the intractable KLD between GMMs and a variational EM algorithm is derived to optimize the objective function. Experiments on synthetic data, flow cytometry analysis, eye-fixation analysis and topic models show that PRIMAL learns a continuous and interpretable manifold of GMM distributions and achieves a minimum reconstruction error.

In recent years, with the rapid development of deep learning, super-resolution methods based on convolutional neural networks (CNNs) have made great progress. However, the parameters and the required consumption of computing resources of these methods are also increasing to the point that such methods are difficult to implement on devices with low computing power. To address this issue, we propose a lightweight single image super-resolution network with an expectation-maximization attention mechanism (EMASRN) for better balancing performance and applicability. Specifically, a progressive multi-scale feature extraction block (PMSFE) is proposed to extract feature maps of different sizes. Furthermore, we propose an HR-size expectation-maximization attention block (HREMAB) that directly captures the long-range dependencies of HR-size feature maps. We also utilize a feedback network to feed the high-level features of each generation into the next generationb’s shallow network. Compared with the existing lightweight single image super-resolution (SISR) methods, our EMASRN reduces the number of parameters by almost one-third. The experimental results demonstrate the superiority of our EMASRN over state-of-the-art lightweight SISR methods in terms of both quantitative metrics and visual quality. The source code can be downloaded at https://github.com/xyzhu1/EMASRN.

Free-to-play has become one of the most popular monetization models, and as a consequence game developers need to get the players to purchase in the game instead of getting players to buy the game. Game analytics and player monetization prediction are important parts in estimating the profitability of a free-to-play game. In this paper, we concentrate on predicting the fraction of monetizing players among all players. Our method is based on a survival analysis mixture cure model, and can be applied to unlabeled data collected from any free-to-play game. We formulate a statistical model and use the Expectation Maximization algorithm to solve the latent monetization percentage and the monetization rate. The original method is modified by using Bayesian inference, and the results of the versions are compared. The method can be applied as a preliminary profitability study in situations where there is no extensive historical game data available, such as game and business development scenarios that need to utilize real time analytics.

This article proposes a new denoising method for hyperspectral images (HSIs) corrupted by mixtures (in a statistical sense) of stripe noise, Gaussian noise, and impulsive noise. The proposed method has three distinctive features: 1) it exploits the intrinsic characteristics of HSIs, namely, low-rank and self-similarity; 2) the observation noise is assumed to be additive and modeled by a mixture of Gaussian (MoG) densities; 3) the inference is performed with an expectation maximization (EM) algorithm, which, in addition to the clean HSI, also estimates the mixture parameters (posterior probability of each mode and variances). Comparisons of the proposed method with state-of-the-art algorithms provide experimental evidence of the effectiveness of the proposed denoising algorithm. A MATLAB demo of this work will be available at https://github.com/TaiXiangJiang for the sake of reproducibility.

In order to improve the effect of e-government work, it is necessary to analyze its influencing factors. E-government is affected by many social factors, so it is necessary to combine intelligent models to improve the effect of factor analysis. This paper combines the essence of e-government influencing factor data to improve the machine learning algorithm and uses the EM algorithm to derive the parameter estimation formula of the data in the case of missing data to improve the accuracy of data analysis. Moreover, this article combines the structure of the e-government system to build the main structure of the intelligent analysis model of the influence factors of e-government. According to the key influencing factor model of e-government adoption and multi-dimensional research on technology, organization and implementation, this paper puts forward the model and promotion mode of the e-government system based on cloud computing and conducts a simulation. From the simulation results, the effect of the model proposed in this paper is more significant.

Image binarization of uneven lighted images, using thresholding techniques, is still a challenging task. Adaptive thresholding methods are the widely adopted approaches for binarization of uneven lighting images. However, the efficacy of these adaptive thresholding methods is highly sensitive to the criteria function used for measuring the bimodal property of the gray level distribution of a local region. In this paper, we propose Gaussian Mixture Model (GMM) which is based on adaptive thresholding for binarizing uneven lighting images. The proposed GMM based criteria function efficiently partitioning the uneven light images into bimodal and unimodal subimages with low uneven light effect. At first, the bimodal subimages are binarized using Otsu’s thresholding approach, followed by unimodal subimages being thresholded using the bilinear interpolation of neighbouring thresholds of bimodal subimages. Next a fast Expectation Maximization(EM) algorithm is developed to reduce the computational complexity of the GMM. Experimental results on different uneven light images demonstrate that the proposed adaptive thresholding outperforms the other considered methods with an avg. misclassification error of 1.68 % and an average computation time of 1.50 seconds. The computational time can be further reduced by a specially purposed hardware and parallel processing of each subimages for real time applications.

The most widely-used data for civilian Space Situational Awareness are the Two-Line Element sets (TLEs) provided by the USSTRATCOM. TLEs play critical roles in space monitoring and analysis activities, and are also used in many applications requiring position information of space objects. However, TLEs suffer from uneven data quality, and it is usually necessary to perform outlier or anomaly detections before utilizing these ephemerides. There are many such procedures available in the published literature, and thresholds are usually obtained through simulations or from experiences, which lacks theoretical soundness and thus is inflexible to use. A new filter based on the Expectation Maximization (EM) algorithm is proposed to ease some restrictions, in which outlier thresholds are determined based on the variances estimated in the polynomial regression and prediction, which makes the new filter theoretically sound and more flexible to use. The new filter can be applied to detect outliers in TLEs of objects in any orbital regions, under various space environments, and enduring different operations such as orbit maneuvers. Multiple experiments are presented to demonstrate the effectiveness of the proposed filter to detect outliers of different magnitudes in TLEs. Simulated outliers in the mean motion and eccentricity with the relative magnitude ≥ 5 × 10 - 3 are almost 100% detected, 74% – 98% of outliers in the inclination with the relative magnitude = 0.01 are detected, and 37% – 68% of those with the relative magnitude 5 × 10 - 3 detected. In addition, sequences can be more accurately identified, which help detect outliers in different sequences, making the proposed filter more applicable to complex situations.

Multi-instance learning (MIL) has been extensively applied to various real tasks involving objects with bags of instances, such as in drugs and images. Previous studies on MIL assume that data are entirely complete. However, in many real tasks, the instance is fragmentary. In this article, we present probably the first study on multi-instance classification with fragmentary data. In our proposed framework, called fragmentary multi-instance classification (FIC), the fragmentary data are completed and the multi-instance classifier is learned jointly. To facilitate the integration between the completion and classifier learning, FIC establishes the weighting mechanism to measure the importance levels of different instances. To validate the compatibility of our framework, four typical MIL methods, including multi-instance support vector machine (MI-SVM), expectation maximization diverse density (EM-DD), citation-K nearest neighbors (Citation-KNNs), and MIL with discriminative bag mapping (MILDM), are embedded into the framework to obtain the corresponding FIC versions. As an illustration, an efficient solving algorithm is developed to address the problem for MI-SVM, together with the proof of convergence behavior. The experimental results on various types of real-world datasets demonstrate the effectiveness.

A convenient, accurate and well-known way toward any supervised task is using Nearest Neighbor approach or its variants. However, there has been little attempt toward improving interpretability by human and providing a classical optimization of Synthetic Reduced Nearest Neighbor. To tackle these issues, this paper provides a novel optimization of Synthetic Reduced Nearest Neighbor based on Expectation Maximization (EM-SRNN). Reduced Nearest Neighbor model consists of a subset of the samples from the trainset that has similar accuracy to Nearest Neighbor. Synthetic Reduced Nearest Neighbor relaxes the model to learn K synthetic samples (or prototypes/centroids) in the space of dataset. Therefore, inspired by EM algorithm for K-means, we propose a novel optimization based on EM algorithm to learn EM-SRNN by iterating over the centroids of the model and assignment of training samples to the centroids. The first step consists of optimizing the position of each centroid based on the assignment of the samples to the centroid and the second step consists finding optimal assignments and labels of the centroids. The EM-SRNN is interpretable since the centroids exist inside the space of training samples. Additionally, the centroids represent the multiple modalities (or sub-clusters) of the classes that are interpretable by human. These properties make this type of interpretability unique, hence, making this model suitable for various studies that are related to interpretability by human, such as image processing and epidemiological studies. In this paper, analytical aspects of problem are explored and it is shown that computational complexity of proposed optimization is linear over size of the trainset. Finally, EM-SRNN shows superior or similar performance when compared with several other interpretable and similar state-of-the-art models, such as trees and kernel SVMs.

To model count data with excess zeros, ones and twos, for the first time we introduce a so-called zero-one-two-inflated Poisson (ZOTIP) distribution, including the zero-inflated Poisson (ZIP) and t...

Channel modeling is a critical issue when designing or evaluating the performance of reconfigurable intelligent surface (RIS)-assisted communications. Inspired by the promising potential of learning-based methods for characterizing the radio environment, we present a general approach to model the RIS end-to-end equivalent channel using the unsupervised expectation-maximization (EM) learning algorithm. We show that an EM-based approximation through a simple mixture of two Nakagami- ${m}$ distributions suffices to accurately approximate the equivalent channel, while allowing for the incorporation of crucial aspects into RIS’s channel modeling such as beamforming, spatial channel correlation, phase-shift errors, arbitrary fading conditions, and coexistence of direct and RIS channels. Based on the proposed analytical framework, we evaluate the outage probability under different settings of RIS’s channel features and confirm the superiority of this approach compared to recent results in the literature.

Dealing with the different artifacts in medical images is necessary to perform several tasks, in particular segmentation and registration. We introduce in this paper a novel method for bias field correction in Magnetic Resonance Images (MRI). Using the segmentation results obtained by a modified Expectation Maximization clustering, the bias field is fitted as an hyper-surface in a 4D hyper-space. Then, it is corrected based on the fact that voxels belonging to the same tissue should have the same intensity in the whole image. So, after a quick and coarse unsupervised voxel labeling by clustering by parts is performed, the bias field is computed at the voxels that have been reliably labeled as belonging to one of the tissues of interest. For the less reliably labeled voxels the bias field is interpolated using an hyper-surface, estimated by a 4D lagrangian interpolation. In order to evaluate the efficiency of the proposed method, we compare the MRI segmen- tation results with and without bias field correction, and we use also the coefficient of variations within the MRI volume.Segmentation results, and the coefficient of variations results were significantly enhanced after bias field correction by the proposed method. Furthermore, the proposed method is considered fast compared to other methods, that are often time consuming du to their iterative nature.

This article proposes unsupervised truth-finding algorithms that combine consideration of multi-modal content features with analysis of propagation patterns to evaluate the veracity of observations in social sensing applications. A key social sensing challenge is to develop effective algorithms for estimating both the reliability of sources and the veracity of their observations without prior knowledge. In contrast to prior solutions that use labeled examples to learn content features that are correlated with veracity, our approach is entirely unsupervised. Hence, given no prior training data, we jointly learn the importance of different content features together with the veracity of observations using propagation patterns as an indicator of perceived content reliability. A novel penalized expectation maximization (PEM) algorithm is proposed to improve the quality of estimation results for observations bolstered by multiple features. In addition, we develop a constrained expectation maximum likelihood with multiple features (CEM-MultiF) that introduces a novel constraint to boost the probability of correctness of some claims. Finally, we evaluate the performance of the proposed algorithms, called EM-Multi, CEM-Multi and PEM-MultiF, respectively, on real-world data sets collected from Twitter. The evaluation results demonstrate that the proposed algorithms outperform the existing fact-finding approaches, and offer tunable knobs for controlling robustness/performance trade-offs in the presence of malicious sources.

Summary Clustering with variable selection is a challenging yet critical task for modern small-n-large-p data. Existing methods based on sparse Gaussian mixture models or sparse $K$-means provide solutions to continuous data. With the prevalence of RNA-seq technology and lack of count data modeling for clustering, the current practice is to normalize count expression data into continuous measures and apply existing models with a Gaussian assumption. In this article, we develop a negative binomial mixture model with lasso or fused lasso gene regularization to cluster samples (small $n$) with high-dimensional gene features (large $p$). A modified EM algorithm and Bayesian information criterion are used for inference and determining tuning parameters. The method is compared with existing methods using extensive simulations and two real transcriptomic applications in rat brain and breast cancer studies. The result shows the superior performance of the proposed count data model in clustering accuracy, feature selection, and biological interpretation in pathways.

Phase Modulation on the Hypersphere (PMH) is a power efficient modulation scheme for the load-modulated multiple-input multiple-output (MIMO) transmitters with central power amplifiers (CPA). However, it is difficult to obtain the precise channel state information (CSI), and the traditional optimal maximum likelihood (ML) detection scheme incurs high complexity which increases exponentially with the number of transmitting antennas and the number of bits carried per antenna in the PMH modulation. To detect the PMH signals without knowing the prior CSI, we first propose a signal detection scheme, termed as the hypersphere clustering scheme based on the expectation maximization (EM) algorithm with maximum likelihood detection (HEM-ML). By leveraging machine learning, the proposed detection scheme can accurately obtain information of the channel from a few of the received symbols with little resource cost and achieve comparable detection results as that of the optimal ML detector. To further reduce the computational complexity in the ML detection in HEM-ML, we also propose the second signal detection scheme, termed as the hypersphere clustering scheme based on the EM algorithm with KD-tree detection (HEM-KD). The CSI obtained from the EM algorithm is used to build a spatial KD-tree receiver codebook and the signal detection problem can be transformed into a nearest neighbor search (NNS) problem. The detection complexity of HEM-KD is significantly reduced without any detection performance loss as compared to HEM-ML. Extensive simulation results verify the effectiveness of our proposed detection schemes.

The implementation of emission-computed tomography (ECT), including positron emission tomography and single-photon emission-computed tomography, has been an important research topic in recent years and is of significant and practical importance. However, the slow rate of convergence and the computational complexity have severely impeded the efficient implementation of iterative reconstruction. By combining the maximum-likelihood expectation maximization (MLEM) iteratively along with the Beltrami filter, this paper proposes a new approach to reformulate the MLEM algorithm. Beltrami filtering is applied to an image obtained using the MLEM algorithm for each iteration. The role of Beltrami filtering is to remove mainly out-of-focus slice blurs, which are artifacts present in most existing images. To improve the quality of an image reconstructed using MLEM, the Beltrami filter employs similar structures, which in turn reduce the number of errors in the reconstructed image. Numerical image reconstruction tomography experiments have demonstrated the performance capability of the proposed algorithm in terms of an increase in signal-to-noise ratio (SNR) and the recovery of fine details that can be hidden in the data. The SNR and visual inspections of the reconstructed images are significantly improved compared to those of a standard MLEM. We conclude that the proposed algorithm provides an edge-preserving image reconstruction and substantially suppress noise and edge artifacts.

Database security is pertinent to every organisation with the onset of increased traffic over large networks especially the internet and increase in usage of cloud based transactions and interactions. Greater exposure of organisations to the cloud implies greater risks for the organisational as well as user data. In this paper, we propose a novel approach towards database intrusion detection systems (DIDS) based on Expectation maximization Clustering and Sequential Pattern Mining (EMSPM). This approach unlike any other does not have records and assumes a predetermined policy to be maintained in an organisational database and can operate seamlessly on databases that follow Role Based Access Control as well as on those which do not conform to any such access control and restrictions. This is achieved by focusing on pre-existing logs for the database and using the Expectation maximization clustering algorithm to allot role profiles according to the database user’s activities. These clusters and patterns are then processed into an algorithm that prevents generation of unwanted rules followed by prevention of malicious transactions. Assessment into the accuracy of EMSPM over sets of synthetically generated transactions yielded propitious results with accuracies over 93%.

Images are often degraded during the data acquisition process. The degradation may involve blurring, information loss due to sampling, and various sources of noise. The purpose of image restoration...

Intrusion detection is the investigation process of information about the system activities or its data to detect any malicious behavior or unauthorized activity. Most of the IDS implement K-means clustering technique due to its linear complexity and fast computing ability. Nonetheless, it is Naïve use of the mean data value for the cluster core that presents a major drawback. The chances of two circular clusters having different radius and centering at the same mean will occur. This condition cannot be addressed by the K-means algorithm because the mean value of the various clusters is very similar together. However, if the clusters are not spherical, it fails. To overcome this issue, a new integrated hybrid model by integrating expectation maximizing (EM) clustering using a Gaussian mixture model (GMM) and naïve Bays classifier have been proposed. In this model, GMM give more flexibility than K-Means in terms of cluster covariance. Also, they use probabilities function and soft clustering, that’s why they can have multiple cluster for a single data. In GMM, we can define the cluster form in GMM by two parameters: the mean and the standard deviation. This means that by using these two parameters, the cluster can take any kind of elliptical shape. EM-GMM will be used to cluster data based on data activity into the corresponding category.

Position and Orientation system (POS), a loosely integrated inertial navigation system (INS) and global positioning system (GPS), can provide high-accuracy motion information for the airborne remote sensing loads, which plays a crucial role in airborne remote sensing imaging. However, the airborne POS often suffers from the harsh environment, such as aircraft maneuver mode and other external disturbance, which will lead to measurement noise unknown and further affects the accuracy of motion parameters. In this paper, an adaptive central difference Kalman filter method based on expectation maximization algorithm is proposed, which can estimate measurement noise adaptively and further improve the performance of POS. A flight experiment is conducted and the results show that the proposed method achieves higher-accuracy motion information by compared with the traditional CDKF method and covariance matching.

The development of physical functioning after a caesura in an aged population is still widely unexplored. Analysis of this topic would need to model the longitudinal trajectories of physical functioning and simultaneously take terminal events (deaths) into account. Separate analysis of both results in biased estimates, since it neglects the inherent connection between the two outcomes. Thus, this type of data generating process is best modelled jointly. To facilitate this several software applications were made available. They differ in model formulation, estimation technique (likelihood-based, Bayesian inference, statistical boosting) and a comparison of the different approaches is necessary to identify their capabilities and limitations. Therefore, we compared the performance of the packages JM, joineRML, JMbayes and JMboost of the R software environment with respect to estimation accuracy, variable selection properties and prediction precision. With these findings we then illustrate the topic of physical functioning after a caesura with data from the German ageing survey (DEAS). The results suggest that in smaller data sets and theory driven modelling likelihood-based methods (expectation maximation, JM, joineRML) or Bayesian inference (JMbayes) are preferable, whereas statistical boosting (JMboost) is a better choice with high-dimensional data and data exploration settings.

Obtaining fast and accurate access to disaster information for underground mines is key in order to enact safe and effective emergency rescue measures. This study proposes a method to remotely evaluate the damage status in underground mines after a mine accident by using a tracer gas. This discussion includes the laws of migration, mixing and separating the tracer gas in the airway, and the superposition of the tracer gas concentration curve conforms to the Gaussian Mixture Model (GMM), which can be decomposed by the expectation-maximization (EM) algorithm. Based on the EM algorithm, the tracer gas method can assess the location and severity of the airway damage. The results show that the error between the calculated damage location and the actual damage location is very small. In addition, the area ratio of the concentration curve after the decomposition by the EM algorithms is highly correlated with the severity of the airway damage. This study demonstrates that the tracer gases can be used to remotely evaluate the location and severity of a mine accident, which is critical in order to perform successful rescue and emergency management.

Ekti, Ali Rıza (Balikesir Author), With the recent developments on opening the terahertz (THz) spectrum for experimental purposes by the Federal Communications Commission, transceivers operating in the range of 0.1THz-10THz, which are known as THz bands, will enable ultra-high throughput wireless communications. However, actual implementation of the high-speed and high reliability THz band communication systems should start with providing extensive knowledge in regards to the propagation channel characteristics. Considering the huge bandwidth and the rapid changes in the characteristics of THz wireless channels, ray tracing and one-shot statistical modeling are not adequate to define an accurate channel model. In this work, we propose Gamma mixture based channel modeling for the THz band via the expectation-maximization (EM) algorithm. First, maximum likelihood estimation (MLE) is applied to characterize the Gamma mixture model parameters, and then EM algorithm is used to compute MLEs of the unknown parameters of the measurement data. The accuracy of the proposed model is investigated by using the Weighted relative mean difference (WMRD) error metrics, Kullback-Leibler (KL)-divergence, and Kolmogorov-Smirnov (KS) test to show the difference between the proposed model and the actual probability density functions (PDFs) that are obtained via the designed test environment. To efficiently evaluate the performance of the proposed method in more realistic scenarios, all the analysis is done by examining measurement data from a measurement campaign in the 240 GHz to 300 GHz frequency range, using a well-isolated anechoic chamber. According to WMRD error metrics, KL-divergence, and KS test results, PDFs generated by the mixture of Gamma distributions fit to the actual histogram of the measurement data. It is shown that instead of taking pseudo-average characteristics of sub-bands in the wide band, using the mixture models allows for determining channel parameters more precisely.

Survival data including potentially cured subjects are common in clinical studies and mixture cure rate models are often used for analysis. The non-cured probabilities are often predicted by non-parametric, high-dimensional, or even unstructured (e.g. image) predictors, which is a challenging task for traditional nonparametric methods such as spline and local kernel. We propose to use the neural network to model the nonparametric or unstructured predictors’ effect in cure rate models and retain the proportional hazards structure due to its explanatory ability. We estimate the parameters by Expectation–Maximization algorithm. Estimators are showed to be consistent. Simulation studies show good performance in both prediction and estimation. Finally, we analyze Open Access Series of Imaging Studies data to illustrate the practical use of our methods.

Recently, most existing human pose estimation methods fuse multi-stage convolutional modules to learn a shared feature representation. In this paper, we propose a expectation–maximization (EM) mapping-based network to learn specific related body parts for human pose estimation, named EMposeNet. It maps specific feature of related parts from the original fully shared feature space. From the perspective of multi-task learning, we can regard the task of human pose estimation as a homogeneous multi-task learning. Sharing features among related tasks can result in a more compact model and better generalization ability. However, sharing features for those unrelated or weakly related tasks will deteriorate the estimation performance. Our proposed method aims at performing EM algorithm to learn the related body part, where the predicted keypoint heatmap is potentially more accurate and spatially more precise. We conduct extensive experiments on two benchmark datasets, including the MSCOCO keypoint detection dataset and the MPII human pose dataset, to empirically demonstrate the validity of the proposed method. The results on such two benchmark datasets show that the proposed approach achieves a competitive performance.

A new approach to the analysis of heterogeneous categorical sequences is proposed. The first-order Markov model is employed in a finite mixture setting with initial state and transition probabilities being expressed as functions of time. The expectation–maximization algorithm approach to parameter estimation is implemented in the presence of positive equivalence constraints that determine which observations must be placed in the same class in the solution. The proposed model is applied to a dataset from the British Household Panel Survey to evaluate the association between the education background and life outcomes of study participants. The analysis of the survey data reveals many interesting relationships between the level of education and major life events.

Background Deep learning (DL)-based image quality improvement is a novel technique based on convolutional neural networks. The aim of this study was to compare the clinical value of 18F-fluorodeoxyglucose positron emission tomography (18F-FDG PET) images obtained with the DL method with those obtained using a Gaussian filter. Methods Fifty patients with a mean age of 64.4 (range, 19–88) years who underwent 18F-FDG PET/CT between April 2019 and May 2019 were included in the study. PET images were obtained with the DL method in addition to conventional images reconstructed with three-dimensional time of flight-ordered subset expectation maximization and filtered with a Gaussian filter as a baseline for comparison. The reconstructed images were reviewed by two nuclear medicine physicians and scored from 1 (poor) to 5 (excellent) for tumor delineation, overall image quality, and image noise. For the semi-quantitative analysis, standardized uptake values in tumors and healthy tissues were compared between images obtained using the DL method and those obtained with a Gaussian filter. Results Images acquired using the DL method scored significantly higher for tumor delineation, overall image quality, and image noise compared to baseline (P P Conclusions Deep learning method improves the quality of PET images.

In recent years, the research of image quality assessment (IQA) based on deep learning, especially convolutional neural network (CNN), has made rapid development. The most widely used way to build a CNN-based IQA model is to divide image into patches and conduct a patch-based training. However, this method has a critical defect that the local ground-truth for each patch is not available. This defect leads to a sub-optimal result. To address this issue, we propose a novel deep blind IQA algorithm under the multiple instance regression (MIR) framework. Specifically, we assume each instance (patch) has a certain probability to be the prime instance of the bag (image), which is responsible for the bag label. Then the global quality score of the bag can be computed by the weighted summation of the local quality scores of the instances, where the weights are the probabilities of the instances to be the prime one. To simplify the training procedure, we propose an EM-like algorithm, called conditional EM algorithm, to train the deep MIR IQA model. Experimental results show that the proposed deep MIR IQA algorithm performs better than the traditional deep blind IQA algorithm. Moreover, the proposed algorithm can be used as a unified framework to improve the performance of any patch-based deep IQA models.

We present an online expectation maximization (EM) based algorithm for the problem of model parameter estimation and hypothesis testing based on observations from a network of heterogeneous sensors. The sensors’ measurements are assumed to be correlated and copula theory is used to model this correlation. Moreover, it is assumed that the statistical model for the sensors’ data is not completely known. We first develop the batch-mode EM to estimate the parameters of the underlying distributions of the sensors’ measurements and to detect the hypothesis. Next, we develop an online EM-based method that processes the data on a sample-by-sample basis. Numerical results are presented for both simulation data and two different real-world datasets. These results demonstrate the efficacy of the proposed method showing significant improvements for both parameter estimation and hypothesis testing compared to the method that ignores the correlation in sensors’ measurements. Hypothesis testing results from the real-world data are also compared with several unsupervised and supervised learning methods. It is shown that the proposed methods outperform the unsupervised and even some supervised methods.

[18F]NaF PET imaging is a useful tool for measuring regional bone metabolism. However, due to tracer in urine, [18F]NaF PET images of the hip reconstructed using filtered back projection (FBP) frequently show streaking artifacts in slices through the bladder leading to noisy time-activity curves unsuitable for quantification. This study compares differences between quantitative outcomes at the hip derived from images reconstructed using the FBP and ordered-subset expectation maximization (OSEM) methods. Dynamic [18F]NaF PET data at the hip for four postmenopausal women were reconstructed using FBP and nine variations of the OSEM algorithm (all combinations of 1, 5, 15 iterations and 10, 15, 21 subsets). Seven volumes of interest were placed in the hip. Bone metabolism was measured using standardized uptake values, Patlak analysis (Ki-PAT) and Hawkins model Ki-4k. Percentage differences between the standardized uptake values and Ki values from FBP and OSEM images were assessed. OSEM images appeared visually smoother and without the streaking artifacts seen with FBP. However, due to loss of counts, they failed to recover the quantitative values in VOIs close to the bladder, including the femoral head and femoral neck. This was consistent for all quantification methods. Volumes of interest farther from the bladder or larger and receiving greater counts showed good convergence with 5 iterations and 21 subsets. For VOIs close to the bladder, including the femoral neck and femoral head, 15 iterations and 10, 15 or 21 subsets were not enough to obtain OSEM images suitable for measuring bone metabolism and showed no improvement compared to FBP.

The prediction of customers' churn is a challenging task in different industrial sectors, in which the motor insurance industry is one of the well-known industries. Due to the incessant upgradation done in the insurance policies, the retention process of customers plays a significant role for the concern. The main objective of this study is to predict the behaviors of the customers and to classify the churners and non-churners at an earlier stage. The Motor Insurance sector dataset consists of 20,000 records with 37 attributes collected from the machine learning industry. The missing values of the records are analyzed and explored via Expectation Maximization algorithm that categorizes the collected data based on the policy renewals. Then, the behavior of the customers are also investigated, so as to ease the construction process training classifiers. With the help of Naive bayes algorithm, the behaviors of the customers on the upgraded policies are examined. Depending on the dependency rate of each variable, a hybrid GWO-KELM algorithm is introduced to classify the churners and non-churners by exploring the optimal feature analysis. Experimental results have proved the efficiency of the hybrid algorithm in terms of 95% prediction accuracy; 97% precision; 91% recall & 94% F-score.

Deep Learning and Machine Learning models have recently been shown to be effective in many real world applications. While these models achieve increasingly better predictive performance, their structures have also become much more complex. A common and difficult problem for complex models is overfitting. Regularization is used to penalize the complexity of the model in order to avoid overfitting. However, in most learning frameworks, regularization function is usually set with some hyper-parameters where the best setting is difficult to find. In this paper, we propose an adaptive regularization method, as part of a large end-to-end healthcare data analytics software stack, which effectively addresses the above difficulty. First, we propose a general adaptive regularization method based on Gaussian Mixture (GM) to learn the best regularization function according to the observed parameters. Second, we develop an effective update algorithm which integrates Expectation Maximization (EM) with Stochastic Gradient Descent (SGD). Third, we design a lazy update and sparse update algorithm to reduce the computational cost by 4x and 20x, respectively. The overall regularization framework is fast, adaptive, and easy-to-use. We validate the effectiveness of our regularization method through an extensive experimental study over 14 standard benchmark datasets and three kinds of deep learning/machine learning models. The results illustrate that our proposed adaptive regularization method achieves significant improvement over state-of-the-art regularization methods.

Hi-C experiments have become very popular for studying the 3D genome structure in recent years. Identification of long-range chromosomal interaction, i.e., peak detection, is crucial for Hi-C data analysis. But it remains a challenging task due to the inherent high dimensionality, sparsity and the over-dispersion of the Hi-C count data matrix. We propose EBHiC, an empirical Bayes approach for peak detection from Hi-C data. The proposed framework provides flexible over-dispersion modeling by explicitly including the “true” interaction intensities as latent variables. To implement the proposed peak identification method (via the empirical Bayes test), we estimate the overall distributions of the observed counts semiparametrically using a Smoothed Expectation Maximization algorithm, and the empirical null based on the zero assumption. We conducted extensive simulations to validate and evaluate the performance of our proposed approach and applied it to real datasets. Our results suggest that EBHiC can identify better peaks in terms of accuracy, biological interpretability, and the consistency across biological replicates. The source code is available on Github (https://github.com/QiZhangStat/EBHiC).

Cognitive diagnostic models (CDMs) have arisen as advanced psychometric models in the past few decades for assessments that intend to measure students’ mastery of a set of attributes. Recently, quite a few studies attempted to extend CDMs to longitudinal versions, and they all tended to model transition probabilities from non-mastery to mastery or vice versa for each attribute separately, with an exception of a few studies (e.g., Chen et al. 2018; Madison & Bradshaw 2018). However, these pioneering works have not taken into consideration the attribute relationships and the ever-changing attributes in a learning period. In this paper, we consider a profile-level latent transition CDM (TCDM hereafter), which can not only identify transition probabilities across the same attributes over time, but also the transition pathways across different attributes. Two versions of the penalized expectation-maximization (PEM) algorithms are proposed to shrink the probabilities associated with impermissible transition pathways to 0 and, thereby, help explore attribute relationships in a longitudinal setting. Simulation results reveal that PEM with group penalty holds great promise for identifying learning trajectories.

Due to the uncontrollable growth of data generation in various networks, rapid clustering of massive datasets is seriously demanded in order to reveal the hidden structure of data as well as discovering the relations among samples. Among the clustering approaches, density-based clustering methods showed an acceptable processing speed to encounter with big data. However, they have some fixed parameters, which are not certainly optimized for all parts of the feature space. Moreover, the complexity of these clustering methods is highly dependent on the number of samples. In this paper, we have deployed Fisher expectation maximization (FEM) to adaptively divide the feature space into some subspaces, where no cluster is shared between the adjacent subspaces. Afterward, we applied density-based spatial clustering of applications with noise (DBSCAN) to each partition yielding to decrease the computational complexity on each thread as well as better learning of its parameters on each subspace. The performance of the proposed method was assessed over three big-size and ten middle-size datasets. The achieved results implied the superiority of the proposed method to OPTICS, Den Clue and DBSCAN methods in terms of clustering accuracy (purity) and processing time.

We investigate a latent variable model for multinomial classification inspired by recent capsule architectures for visual object recognition (Sabour, Frosst, & Hinton, 2017 ). Capsule architectures use vectors of hidden unit activities to encode the pose of visual objects in an image, and they use the lengths of these vectors to encode the probabilities that objects are present. Probabilities from different capsules can also be propagated through deep multilayer networks to model the part-whole relationships of more complex objects. Notwithstanding the promise of these networks, there still remains much to understand about capsules as primitive computing elements in their own right. In this letter, we study the problem of capsule regression—a higher-dimensional analog of logistic, probit, and softmax regression in which class probabilities are derived from vectors of competing magnitude. To start, we propose a simple capsule architecture for multinomial classification: the architecture has one capsule per class, and each capsule uses a weight matrix to compute the vector of hidden unit activities for patterns it seeks to recognize. Next, we show how to model these hidden unit activities as latent variables, and we use a squashing nonlinearity to convert their magnitudes as vectors into normalized probabilities for multinomial classification. When different capsules compete to recognize the same pattern, the squashing nonlinearity induces nongaussian terms in the posterior distribution over their latent variables. Nevertheless, we show that exact inference remains tractable and use an expectation-maximization procedure to derive least-squares updates for each capsule's weight matrix. We also present experimental results to demonstrate how these ideas work in practice.

Resting-state functional magnetic resonance imaging (R-fMRI) applications can entail a higher temporal-sampling rate that trades off spatial resolution, thereby challenging effective scientific studies. To enable higher spatial resolution, current schemes speedup per-timeframe scanning by reconstruction from simultaneous multislice (SMS) magnetic resonance imaging (MRI) with k-space undersampling (sometimes temporal undersampling), while using prior models on the signal. We propose a novel algorithmic framework to reconstruct R-fMRI (SMS with controlled aliasing) that has, both, k-space undersampling and temporal undersampling. We propose a coupled spatiotemporal sparse model, incorporating (i) a robust spatially-regularized temporal-dictionary prior and (ii) a spatiotemporal wavelet prior, which we fit efficiently using variational Bayesian expectation maximization with nested minorization (VBEMNM). We show that our framework has the potential to enable higher spatial resolution without increasing scan time in R-fMRI that has inherently weak signals and is therefore prone to large physiological fluctuations, acquisition noise, and imaging artifacts. Qualitative and quantitative evaluation on retrospectively undersampled brain R-fMRI shows that estimates of resting-state networks from our framework and the boost in temporal stability given by our framework compares favourably to existing methods for R-fMRI reconstruction.

Automatic speech recognition (ASR) technologies enable humans to communicate with computers. Isolated word recognition (IWR) is an important part of many known ASR systems. Minimizing the word error rate in cases of incremental learning is a unique challenge for developing an on-line ASR system. This paper focuses on on-line IWR using a recursive hidden Markov model (HMM) multivariate parameter estimation algorithm. The maximum likelihood method was used to estimate the unknown parameters of the model, and an algorithm for the adapted recursive EM algorithm for HMMs parameter estimation was derived. The resulting recursive EM algorithm is unique among its counterparts because of state transition probabilities calculation. It obtains more accurate parameter estimates compared to other algorithms of this type. In our experiment, the algorithm was implemented and adapted to several datasets for IWR. Thus, the recognition rate and algorithm convergence results are discussed in this work.

Positron emission tomography (PET) is an advanced medical imaging technique widely used in various clinical applications, such as tumor detection and neurologic disorders. Reducing the radiotracer dose is desirable in PET imaging because it decreases the patient’s radiation exposure. However, reducing the dose can also increase noise, affecting the image quality. Therefore, an advanced image reconstruction algorithm based on low-dose PET data is needed to improve the quality of the reconstructed image. In this article, we propose the use of a direct PET image reconstruction network (DPIR-Net) using an improved Wasserstein generative adversarial network (WGAN) framework to enhance image quality. This article provides two main findings. First, our network uses sinogram data as input and outputs high-quality PET images direct, resulting in shorter reconstruction times compared with traditional model-based reconstruction networks. Second, we combine perceptual loss, mean square error, and the Wasserstein distance as the loss function, which effectively solves the problem of excessive smoothness and loss of detailed information in traditional network image reconstruction. We performed a comparative study using maximum-likelihood expectation maximization (MLEM) with a post-Gaussian filter, a total variation (TV)-norm regularization, a nonlocal means (NLMs) denoising method, a neural network denoising method, a traditional deep learning PET reconstruction network, and our proposed DPIR-Net method and evaluated the proposed method using both human and mouse data. The mouse data were obtained from a small animal PET prototype system developed by our laboratory. The quantitative and qualitative results show that our proposed method outperformed the conventional methods.

Reinforcement learning has been widely applied for sequential decision making problems in various fields of the real world, including recommendation, e-learning, etc. The features of multi-policy, latent mixture environments and offline learning implied by many real applications bring a new challenge for reinforcement learning. To this challenge, the paper proposes a reinforcement learning approach called offline multi-policy gradient for latent mixture environments. The proposed method uses an objective of expected return of trajectory with respect to the joint distribution of trajectory and model, and adopts a multi-policy searching algorithm to find the optimal policies based on expectation maximization. We also prove that the off-policy technique of importance sampling and advantage function can be used by offline multi-policy learning with fixed historical trajectories. The effectiveness of our approach is demonstrated by the experiments on both synthetic and real datasets.

The integration of spectral and spatial information is crucial in remotely sensed hyperspectral image classification. Some available approaches extract spatial features before classification, while other techniques include spatial information as a spatial regularizer. Due to the model complexity, these methods generally assume that a given pixel is only dependent on itself and its neighboring pixels. However, it is known that there are strong interactions among the pixels in different areas of the image, and as a result, this general assumption can limit the classifier performance. In this work, we develop a new semisupervised discriminative random field (DRF) technique for spatial-spectral classification of hyperspectral images, which constructs the complete model using the whole image. Furthermore, to cope with the unlabeled samples in the model, a new expectation maximization-based algorithm is introduced to optimize the learning of DRF parameters, where the missing labels of these samples serve as the latent variable. As compared to other conventional learning approaches such as maximum a posteriori marginal labeling, the proposed approach can obtain better optimization that leads to improved classification performance. Our experimental results, conducted using real hyperspectral datasets, indicate that the newly developed DRF algorithm exhibits very good performance.

Multi-release is critical for modern open source software product in order to satisfy more customer requirements. Masked data, a kind of missing data, is the system failure data when the exact cause of the failures might be unknown. That is, the cause of the system failures may be any one of the objects. However, due to the influence of the test strategy in real project, the cause of the system failures may be a subset of the system objects, not any one of the objects. In this paper, the mathematical description of general masked data is presented based on the traditional masked data. Furthermore, a novel multi-release open source software (OSS) reliability model based on general masked data is proposed. Different from traditional multi-release OSS reliability model, the proposed approach is based on additive model with general masked data other than change point model. And then, the maximum likelihood estimation (MLE) process of the model parameters is derived in detail, and expectation maximization (EM) algorithm is used to solve the extremely complicated problem of the log-likelihood function. Finally, two data sets from real open source software project are applied to the proposed approach, and the results show that the proposed reliability model is useful and powerful.

Land cover is of great significance for the study of global ecological environmental change. Multitemporal land cover can help us to understand the change process of the regional environment and formulate corresponding protection policies. For single-period image classification, the spatial-temporal information is often ignored, and the classification accuracy is difficult to improve. In this paper, an iterative hidden Markov model (STHMM) is proposed to optimize the multitemporal classification, in which a stacked autoencoding classifier is used to calculate the initial class probability, and the extended random walker-based spatial optimization technique is adopted to optimize the class probability. Finally, the hidden Markov model with expectation maximization is built by exploiting postprocessing temporal optimization. Experimental results show that the proposed method can outperform other classification techniques, and the spatial-temporal hidden Markov model proposed in this paper exhibits more stable and reliable performance and can be widely used in multitemporal classification.

The probabilistic distribution of wind speed is critical information required in evaluating wind resources, designing wind farms, and mitigating the possible risks in wind power expansion. Various distributions have been used in the research literature to estimate wind speed distribution. In this paper, we propose a flexible family of mixture distributions, whose elements are convex linear combinations of the skew-t-normal Birnbaum-Saunders distributions and is suitable for modelling heavy-tailed data with a heterogeneous population. These distributions are then used to estimate the wind speed distribution. The performance of the proposed family has been compared with five mixture models of the already used distributions, using wind speed data collected at nine stations across Ontario, Canada. The results indicate that mixture models generally provide a better fit than unimodal distributions, according to the model selection criterion. Based on the obtained results, the proposed family provides highly flexible models at all selected stations. It functions better than the other considered distributions at seven of the stations, whereas it is ranked second in the remaining two stations. The proposed family can be utilized in a widespread manner to describe the wind speed in Canada, as well as other regions with similar features.