Search

The economic operation of the integrated energy system faces the problems of coupling between energy production and conversion equipment in the system and the imbalance of various energy demands. Therefore, taking system safety as the constraint and minimum economic cost as the objective function, including fuel cost, operation, and maintenance cost, this paper proposes the operation dispatching model of the integrated energy system based on hydrogen fuel cell (HFC), including HFC, photovoltaic, wind turbine, electric boiler, electric chiller, absorption chiller, electric energy storage, and thermal energy storage equipment. On this basis, a distributionally robust optimization (DRO) model is introduced to deal with the uncertainty of wind power and photovoltaic output. In the distributionally robust optimization model, Kullback–Leibler (KL) divergence is used to construct an ambiguity set, which is mainly used to describe the prediction errors of renewable energy output. Finally, the DRO economic dispatching model of the HFC integrated energy system (HFC-IES) is established. Besides, based on the same load scenario, the economic benefits of hybrid energy storage equipment are discussed. The dispatching results show that, compared with the scenario of only electric energy storage and only thermal energy storage, the economic cost of the scenario of hybrid electric and thermal storage can be reduced by 3.92% and 7.55% respectively, and the use of energy supply equipment can be reduced, and the stability of the energy storage equipment can be improved.

As the proportion of renewable energy gradually increases, it brings challenges to the stable operation of the combined heat and power (CHP) system. As an important flexible resource, energy storage (ES) has attracted more and more attention. However, the profit of energy storage can't make up for the investment and operation cost, and there is a lack of measurement system for multiple values, which seriously hinders the development of energy storage industry. Based on this, this paper makes a quantitative analysis on the system value of multiple energy storage in CHP. Firstly, the uncertain output of renewable energy is characterized by Kullback–Leibler (KL) divergence, and a two-level dispatching model is constructed based on the distributionally robust optimization method, so as to study the optimal operation strategy of the ES-CHP system. Secondly, based on the system value theory, this paper analyzes the system value of multiple energy storage, including internal value and external value, and constructs the value quantitative model, respectively. Finally, in a typical ES-CHP system, the system value of multiple energy storage is quantified. The effectiveness of the two-level model constructed in this paper can be seen from the simulation results, and the influence of different electricity prices on the system value of multiple energy storage is further analyzed. [ABSTRACT FROM AUTHOR]

For the existing Closed Set Recognition (CSR) methods mistakenly identify unknown jamming signals as a known class, a Conditional Gaussian Encoder (CG-Encoder) for 1-dimensional signal Open Set Recognition (OSR) is designed. The network retains the original form of the signal as much as possible and deep neural network is used to extract useful information. CG-Encoder adopts residual network structure and a new Kullback-Leibler (KL) divergence is defined. In the training phase, the known classes are approximated to different Gaussian distributions in the latent space and the discrimination between classes is increased to improve the recognition performance of the known classes. In the testing phase, a specific and effective OSR algorithm flow is designed. Simulation experiments are carried out on 9 jamming types. The results show that the CSR and OSR performance of CG-Encoder is better than that of the other three kinds of network structures. When the openness is the maximum, the open set average accuracy of CG-Encoder is more than 70%, which is about 30% higher than the worst algorithm, and about 20% higher than the better one. When the openness is the minimum, the average accuracy of OSR is more than 95%.

• The "bilateral reduction" effect of Rényi entropy is theoretically studied and proved. • Relative entropy is extended by adding order α to generalized relative entropy. • Generalized relative entropy satisfies six sparse criteria. • Conversion between complexity measures and sparsity measures is realized. • New sparse measure cluster is proposed to enrich machine condition monitoring. Complexity measures and sparsity measures are capable of detecting and characterizing underlying dynamic changes in machine condition monitoring. Previous studies have developed Shannon entropy and Kullback-Leibler (KL) divergence and their generalization, namely Rényi entropy and Rényi divergence, but their theoretical properties as a function of sparsity and their transitions from complexity measures to sparsity measures are not fully explored and discussed. This paper continues conducting theoretical and experimental investigations on Rényi entropy and Rényi divergence and exploring their theoretical properties and transitions to enrich the domain of complexity and sparsity measures and their applications to machine condition monitoring. Specifically, this paper theoretically proves and verifies that Rényi entropy has a similar "bilateral reduction" effect exhibited by other complexity measures including correlation dimension, approximate entropy, fuzzy entropy and sample entropy. Inspired by relative entropy, which is a special case of KL divergence and a sparsity measure, and the relationship between KL divergence and Rényi divergence, in this paper, we attempt to generalize relative entropy by adding an order of α , to convert a series of complexity measures from Rényi entropy to a series of sparsity measures, which answers how to realize transitions from complexity measures to sparsity measures. More importantly, this paper is helpful to provide a guideline for correct uses of Rényi entropy, Rényi divergence, and generalized relative entropy for the quantification of impulsive transients caused by machine faults for machine condition monitoring. [ABSTRACT FROM AUTHOR]

Radiometric normalization is an essential preprocessing step for almost all remote sensing applications such as change detection, image mosaic, and 3-D reconstruction. This article proposes a novel radiometric normalizing method based on spatiotemporal filtering using a reference moderate resolution imaging spectroradiometer (MODIS) product. This differs from traditional relative radiometric normalization (RRN) methods in two folds: first, the number of reference images is more than one, which introduces more complexities than RRN with a single reference image; second, the resolution of MODIS product is significantly lower, thus requiring the algorithms to accommodate scale differences. To address, our approach extends the traditional spatiotemporal filtering method with per image bias that represents both internal (e.g., sensor characteristics) and external (e.g., atmosphere and topography) against the reference data. In addition, we use the Kullback-Leibler divergence metric to statistically determine the resemblance degree between the temporal images for weighting. We applied our proposed method to normalize Landsat Operational Land Imager, Enhanced Thematic Mapper Plus +, and Sentinel MSI using MODIS Nadir BRDF-adjusted reflectance product, covering two study areas of 30 × 15 km2 and 32 × 52 km2, respectively, and we show a notable radiometric consistency over both temporal and spatial dimension after the processing through three comparative experiments with state-of-the-art methods. 1) 3–7% improvement in the contexts of transfer learning, which favors only images with consistent radiometric properties and 2) Mosaic results using our processed images show no apparent seamlines as compared with images processed by other methods.

Land cover classification plays a crucial role in land resource monitoring and planning. Recently, deep learning-based methods are becoming the dominating method for precise land cover mapping. However, the large-scale application of them is deeply hindered by the domain shift between different images, which is easily caused by illumination, climate, regional divergence, and so on. With the aim to cope with the problem of domain shift, many domain adaptation (DA) methods have been provided and great achievements have been made, especially the newborn adversarial DA, which usually contains a generator and a discriminator. Among these methods, the pixel-level methods are of high memory consumption, whereas feature-level methods are found hard to decode the structured information for semantic segmentation tasks due to the lack of low-dimensional information. Therefore, we propose an adversarial domain adaptation framework with Kullback–Leibler constraint (KL-ADDA) for remote sensing land cover classification. A state-of-the-art (SOTA) semantic segmentation network is utilized as the generator, which directly outputs the segmentation results to the discriminator to retain more low-level information. Besides, a Kullback–Leibler (KL)-divergence is calculated to improve the discriminative ability of the discriminator and thus enhance the generator’s performance. Experiments on the international society for photogrammetry and remote sensing (ISPRS) data set and two simulated target data sets have shown the effectiveness of KL-ADDA for DA. [ABSTRACT FROM AUTHOR]

As for Generative Adversarial Networks (GANs), its interpretability may be closely related to optimization objective functions, that is, information metrics play important roles in networks training and data generation. In terms of original GAN, the objective function based on Kullback-Leibler (KL) divergence has limitations on the performance of data training and generation. Therefore, it is significant to investigate objective functions for the optimization in GANs to bring gains on the efficiency of network learning from the perspective of metrics. In this paper, the objective function with exponential form, referred from the Message Importance Measure (MIM), is adapted to replace that with logarithm form in the optimization for adversarial networks. This approach named MIM-based GAN, may provide more hidden information in terms of interpretability on training process and probability events generation. Specifically, we first analyze the intrinsic relationship between the proposed approach and other classical GANs. Moreover, compared with the original GAN, LSGAN and WGAN, we discuss its advantages on training performance in theory including sensitivity, convergence rate and so on. In addition, we do simulations on the datasets to confirm why the MIM-based GAN achieves state-of-the-art performance on training process and data generation. [ABSTRACT FROM AUTHOR]

The frequent switching operations and triggering events of automation devices are poorly documented in distribution management systems, leading to challenging work regarding the timely detection of system state changes. Additionally, the limited measurements in distribution networks and the randomness of plug-and-play devices make obtaining real-time observations of system states difficult. Furthermore, distributed generation (DG) output uncertainties exacerbate the situation. Thus, we propose a dynamic topology awareness (DTA) approach to make judgments regarding operational states and identify all possible topology changes (including normal operations, islanding and outages) in the absence of real-time DG measurements. The method consists of two steps. The first step applies parallel Gaussian mixture model-based probabilistic state estimators (GMM-PSEs) of subareas and extended subareas to calculate the probability distributions of border bus voltages with full consideration of DG uncertainties. The second step utilizes the Kullback-Leibler (KL) divergence measure to evaluate the difference between the two probability distributions estimated by subareas and extended subareas, thereby obtaining an indicator (average KL divergence) for identifying possible topologies. The effectiveness of the proposed method is examined on the IEEE 33-bus and IEEE 123-bus test cases. It is computationally efficient and accurate when identifying all possible topologies under limited measurements and DG uncertainties. [ABSTRACT FROM AUTHOR]

It is of great practical importance to compare and combine data from different studies in order to carry out appropriate and more powerful statistical inference. We propose a partition based measure to quantify the compatibility of two datasets using their respective posterior distributions. We further propose an information gain measure to quantify the information increase (or decrease) in combining two datasets. These measures are well calibrated and efficient computational algorithms are provided for their calculations. We use examples in a benchmark dose toxicology study, a six cities pollution data and a melanoma clinical trial to illustrate how these two measures are useful in combining current data with historical data and missing data. [ABSTRACT FROM AUTHOR]

Bitumen in the oil sands industry is separated from sand using a water-based gravity separation process in a primary separation vessel (PSV). The interface between the froth and the middlings layer is an important parameter to control for optimal operation of the PSV unit. In this paper, a method using computer vision based on convolutional neural networks (CNNs) and Kalman filter (KF) is designed for the detection of the interface in PSV, with the CNN estimating both the interface level and the image quality. The proposed method consists of two parts: an offline and an online stage, wherein the parameters of CNN and KF are trained in the offline stage using the available data. The algorithm is made robust to any new type of occlusions, not present in the training dataset, in the online stage. Experimental results demonstrate that the proposed method is accurate and robust to different abnormalities in the process. [ABSTRACT FROM AUTHOR]

This article proposes a variational Bayesian (VB) inference based on the nonsynchronous array measurement (NAM) (VB-NAM) method in order to obtain the fast and robust localization of low-frequency acoustic sources. To enlarge the aperture size compared with the prototype array, the NAM is performed to measure the acoustic pressures with low-frequency based on the forward power propagation model. The implementation of the NAM can be reformulated into a cross-spectral matrix (CSM) completion problem. Then, to solve the inverse problem of the NAM power propagation model, the VB inference based on the Student-t priors and Kullback–Leibler (KL) divergence optimization is proposed. The advantages of the proposed VB-NAM benefit from the optimization of matrix inversion and adaptive estimation of regularization parameters. The contribution of the adaptive parameter evaluation is to reduce the impact of multiple interferences (such as additive noise and the matrix completion error) in NAM. Finally, both simulations at 800 Hz and experimental results at 1000 Hz are presented to show the validation of the proposed VB-NAM method, even under the anisotropic Gaussian noise conditions. Algorithm performance and iteration process are analyzed to demonstrate the efficiency and robustness. [ABSTRACT FROM AUTHOR]

In this paper, we concern the uplink of a massive single-input multiple-output enabled ultra-reliable low-latency communication system, in which a single-antenna transmitter aims to timely and reliably send data to a receiver equipped with a large number of antennas over Rayleigh fading channels. For such a scenario, to eliminate the considerable overhead caused by channel estimation, we adopt a noncoherent maximum-likelihood (ML) receiver, which is known to be optimal in terms of average symbol-error rate for equiprobable discrete input signals. We propose a two-dimensional noncoherent constellation design framework to enhance the reliability of the considered system. Specifically, our design principle is to maximize the minimum Kullback-Leibler divergence between the conditional distributions induced by different transmitted signals under average power constraint for any given transmission rate. The resulting optimization problem is shown to be a challenging mixed discrete-continuous problem. We manage to solve the problem by deliberately designing optimal bit allocation and optimal constellation structure as a function of signal-to-noise ratios. We then unveil that the proposed constellation can facilitate efficient ML detection with low computational complexity. Finally, simulation results illustrate that the proposed scheme has a superior error performance than conventional training-based schemes and existing energy detection designs. [ABSTRACT FROM AUTHOR]

In the context of recent deep clustering studies, discriminative models dominate the literature and report the most competitive performances. These models learn a deep discriminative neural network classifier in which the labels are latent. Typically, they use multinomial logistic regression posteriors and parameter regularization, as is very common in supervised learning. It is generally acknowledged that discriminative objective functions (e.g., those based on the mutual information or the KL divergence) are more flexible than generative approaches (e.g., K-means) in the sense that they make fewer assumptions about the data distributions and, typically, yield much better unsupervised deep learning results. On the surface, several recent discriminative models may seem unrelated to K-means. This study shows that these models are, in fact, equivalent to K-means under mild conditions and common posterior models and parameter regularization. We prove that, for the commonly used logistic regression posteriors, maximizing the L2 regularized mutual information via an approximate alternating direction method (ADM) is equivalent to minimizing a soft and regularized K-means loss. Our theoretical analysis not only connects directly several recent state-of-the-art discriminative models to K-means, but also leads to a new soft and regularized deep K-means algorithm, which yields competitive performance on several image clustering benchmarks. [ABSTRACT FROM AUTHOR]

Fault diagnosis gains increasing attention for its ability to enhance process safety and efficiency. This brief proposes a maximized ratio divergence analysis (MRDA) approach for fault diagnosis, which maximizes the pairwise ratio Kullback–Leibler (KL) divergence between each pair of classes during the dimensionality reduction step. In addition, an iterative algorithm based on deflation techniques is put forward for learning the loading vectors of MRDA. The proposed MRDA-based approach allows for improved power of fault diagnosis because of the following advantages over classical monitoring methods. First, MRDA maximizes the pairwise ratio divergence between each pair of classes, which directly leads to enhanced classification performance in the low-dimensional space. Moreover, MRDA is less likely to be dominated by “outlier” classes, since its objective is an average of ratio divergence, thereby facilitating the proposed method to be beneficial to the classification of imbalanced faulty classes. The effectiveness of the MRDA-based approach for fault diagnosis is verified by the Tennessee Eastman process (TEP). [ABSTRACT FROM AUTHOR]

With the ever-growing amount of online data and information, recommender systems are becoming overwhelmingly popular as an adequate approach for overcoming the challenge of information overload. Artificial Intelligence (AI) and Deep Learning (DL) have accumulated significant interest in many research areas, and recommender systems are one of them. In this paper, a Graph Convolutional Neural Network (GCNN)-based approach was used for online product recommendation. Graph-based methods have undergone substantial consideration for several recommendation tasks, with effective results. However, handling the computational complexities and training large datasets remain a challenge for such a model. Even though they are useful, the excessive measure of the model's boundaries obstructs their applications in real-world recommender frameworks to a great extent. The recursive way of generating neighbor node embeddings for each node in the graph makes it more challenging to train a deep and large GCNN model. Therefore, we propose a model that incorporates measures of similarity between two different nodes, and these similarity measures help us to sample the neighbors beforehand. We estimate the similarity based on their interaction probability distribution with other nodes. We use KL divergence on different probability distributions to find the distance between them. This way, we set a threshold criterion for neighbor selection and generate other clusters. These clusters are then converted to subgraphs and are used as input for the proposed GCNN model. This approach simplifies the task of neighbor sampling for GCNN, and hence, we can observe a significant improvement in the computational complexity of the GCNN model. Finally, we compared the results with those for the previously proposed OpGCN model, basic GCNN model, and other traditional approaches such as collaborative filtering and probabilistic matrix factorization. The experiments showed that the complexity and computational time were decreased by estimating the similarity among nodes and sampling the nodes before training. [ABSTRACT FROM AUTHOR]

Timely and effective detection of potential incipient faults in satellites plays an important role in improving their availability and extending their service life. In this paper, the problem of detecting incipient faults using projection vector (PV) and Kullback-Leibler (KL) divergence is studied in the context of detecting incipient faults in satellites. Under the assumption that the variables obey a multidimensional Gaussian distribution and using KL divergence to detect incipient faults, this paper models the optimum PV for detecting incipient faults as an optimization problem. It proves that the PVs obtained by principal component analysis (PCA) are not necessarily the optimum PV for detecting incipient faults. It then compares the on-line probability density function (PDF) with the reference PDF for detecting incipient faults on the local optimum PV. A numerical example and a real satellite fault case were used to assess the validity and superiority of the method proposed in this paper over conventional methods. Since the method takes into account the characteristics of the actual incipient faults, it is more adaptable to various possible incipient faults. Fault detection rates of three simulated faults and the real satellite fault are 98%, 84%, 93% and 92%, respectively. [ABSTRACT FROM AUTHOR]

Since data are collected from a range of sources via different techniques, multiview clustering has become an emerging technique for unsupervised data classification. However, most existing soft multiview clustering methods only consider the pairwise correlations and ignore high-order correlations among multiple views. To integrate more comprehensive information from different views, this article innovates a fuzzy clustering model using the low-rank tensor to address the multiview data clustering problem. Our method first conducts a standard fuzzy clustering on different views of the data separately. Then, the obtained soft partition results are aggregated as the new data to be handled by a Kullback–Leibler (KL) divergence-based fuzzy model with low-rank tensor constraints. The KL divergence function, which replaces the traditional minimized Euclidean distance, can enhance the robustness of the model. More importantly, we formulate fuzzy partition matrices of different views as a third-order tensor. So, a low-rank tensor is introduced as a norm constraint in the KL divergence-based fuzzy clustering to obtain dexterously high-order correlations of different views. The minimization of the final model is convex and we present an efficient augmented Lagrangian alternating direction method to handle this problem. Specially, the global membership is derived by using tensor factorization. The efficiency and superiority of the proposed approach are demonstrated by the comparison with state-of-the-art multiview clustering algorithms on many multiple-view data sets. [ABSTRACT FROM AUTHOR]

In a wireless channel, shadow fading along with multipath effect causes random fluctuations of the received signal strength at the receiver. Several composite fading distributions are available for modeling the randomness. Among them, Lognormal-based models are unable to give a closed-form expression of the composite distribution, but Gamma-based model overcomes the problem faced by the former one. However, the Gamma-based model comes with a complicated mathematical function. To get rid of that, mixture of Lognormal (MoLN) distribution is used in this article. Maximum-likelihood approach is used for parameters estimation with the help of the expectation maximization algorithm. Simulation results have been provided in order to show the accuracy of the proposed mixture model. The first-order statistics of both the envelope and signal-to-noise ratio of a wireless channel are represented by a simple mathematical form. Several performance metrics, used in a wireless system, are expressed in a closed form for the MoLN distribution. The fitness of MoLN distribution is tested. Quantitatively, by measuring metrics like mean square error and Kullback–Leibler divergence, it has been shown that MoLN distribution is a suitable candidate to represent the composite wireless channel accurately. [ABSTRACT FROM AUTHOR]

Signals obtained from impact-echo techniques can be used to detect and classify the defects in damaged materials. The defects change the wave propagation between the impact and the sensors producing particular spectrum elements, which define the feature vector. We propose a hierarchical clustering method that models the feature vector as a mixture of Gaussians (MoG) for every class and then merges different clusters using as a distance measure the symmetric Kullback–Leibler (KL) divergence. Since there is no closed-form solution to the KL divergence between MoGs, some approximations are introduced. We apply the hierarchical clustering algorithms to the signals obtained from real specimens made of aluminum alloy. The samples are classified into four classes according to the state: homogeneous (no defect), one hole, one crack, and multiple defects. We compare the performance of different approximations and discuss the dendrograms that are obtained. Similar kinds of defects are clustered first, and more importantly, the high-level hierarchy is able to distinguish between the defective and nondefective materials. [ABSTRACT FROM AUTHOR]

The scale parameter (SP), to control the sizes of objects, is of great significance in multiscale segmentation which is a prerequisite and foundational step for object-based change detection (OBCD). However, the appropriate SP is not readily apparent and the majority of the existing OBCD algorithms obtain the SPs by empirical or subjective trial-and-error ways that may lead to dissatisfactory accuracy and be time-consuming. To address this issue, an automatic approach for optimal segmentation scale selection for OBCD is proposed in this letter. First, a changed fuzziness image for bitemporal images was generated. Second, multiscale segmentation was implemented in a series of candidate scales, and the merging relationships between adjacent scales were built. Then, mapping the segments to the previous fuzziness image, a statistical metric describing the homogeneity of objects based on the Kullback–Leibler divergence was defined, the increments of the metric between merged objects and their child objects were calculated and weighted to identify the optimal SPs. For performance evaluation, Dempter–Shafer (DS) evidence fusion was utilized in the scales selected by the proposed approach in comparison with other state of art or empirical ones. The experimental results employing GF-1, Google Earth, and aerial images demonstrated the superiority and effectiveness of the SPs identified by the proposed approach in the OBCD task. [ABSTRACT FROM AUTHOR]

A loss function measures the discrepancy between the true values and their estimated fits, for a given instance of data. In classification problems, a loss function is said to be proper if a minimizer of the expected loss is the true underlying probability. We show that for binary classification, the divergence associated with smooth, proper, and convex loss functions is upper bounded by the Kullback-Leibler (KL) divergence, to within a normalization constant. This implies that by minimizing the logarithmic loss associated with the KL divergence, we minimize an upper bound to any choice of loss from this set. As such the logarithmic loss is universal in the sense of providing performance guarantees with respect to a broad class of accuracy measures. Importantly, this notion of universality is not problem-specific, enabling its use in diverse applications, including predictive modeling, data clustering and sample complexity analysis. Generalizations to arbitary finite alphabets are also developed. The derived inequalities extend several well-known $f$ -divergence results. [ABSTRACT FROM AUTHOR]

Abstract Background Ultrasound imaging is safer than other imaging modalities, because it is noninvasive and nonradiative. Speckle noise degrades the quality of ultrasound images and has negative effects on visual perception and diagnostic operations. Methods In this paper, a nonlocal total variation (NLTV) method for ultrasonic speckle reduction is proposed. A spatiogram similarity measurement is introduced for the similarity calculation between image patches. It is based on symmetric Kullback-Leibler (KL) divergence and signal-dependent speckle model for log-compressed ultrasound images. Each patch is regarded as a spatiogram, and the spatial distribution of each bin of the spatiogram is regarded as a weighted Gamma distribution. The similarity between the corresponding bins of the two spatiograms is computed by the symmetric KL divergence. The Split-Bregman fast algorithm is then used to solve the adapted NLTV object function. Kolmogorov-Smirnov (KS) test is performed on synthetic noisy images and real ultrasound images. Results We validate our method on synthetic noisy images and clinical ultrasound images. Three measures are adopted for the quantitative evaluation of the despeckling performance: the signal-to-noise ratio (SNR), structural similarity index (SSIM), and natural image quality evaluator (NIQE). For synthetic noisy images, when the noise level increases, the proposed algorithm achieves slightly higher SNRS than that of the other two algorithms, and the SSIMS yielded by the proposed algorithm is obviously higher than that of the other two algorithms. For liver, IVUS and 3DUS images, the NIQE values are 8.25, 6.42 and 9.01, all of which are higher than that of the other two algorithms. Conclusions The results of the experiments over synthetic and real ultrasound images demonstrate that the proposed method outperforms current state-of-the-art despeckling methods with respect to speckle reduction and tissue texture preservation.

Detection of faults at the incipient stage is critical to improving the availability and continuity of satellite services. The application of a local optimum projection vector and the Kullback–Leibler (KL) divergence can improve the detection rate of incipient faults. However, this suffers from the problem of high time complexity. We propose decomposing the KL divergence in the original optimization model and applying the property of the generalized Rayleigh quotient to reduce time complexity. Additionally, we establish two distribution models for subfunctions F1(w) and F3(w) to detect the slight anomalous behavior of the mean and covariance. The effectiveness of the proposed method was verified through a numerical simulation case and a real satellite fault case. The results demonstrate the advantages of low computational complexity and high sensitivity to incipient faults.

There exist numerous proofs of Bell's theorem, stating that quantum mechanics is incompatible with local realistic theories of nature. Here the strength of such nonlocality proofs is defined in terms of the amount of evidence against local realism provided by the corresponding experiments. Statistical considerations show that the amount of evidence should be measured by the Kull-back-Leibler (KL) or relative entropy divergence. The statistical strength of the following proofs is determined: Bell's original proof and Peres' optimized variant of it, and proofs by Clauser, Horne, Shimony, and Holt (CHSH), Hardy, Mermin, and Greenberger, Horne, and Zeilinger (GHZ). The GHZ proof. is at least four and a half times stronger than all other proofs, while of the two-party proofs, the one of CHSH is the strongest.

Probabilistic approaches are a promising solution to the image retrieval problem that, when compared to standard retrieval methods, can lead to a significant gain in retrieval accuracy. However, this occurs at the cost of a significant increase in computational complexity. In fact, closed-form solutions for probabilistic retrieval are currently available only for simple probabilistic models such as the Gaussian or the histogram. We analyze the case of mixture densities and exploit the asymptotic equivalence between likelihood and Kullback-Leibler (KL) divergence to derive solutions for these models. In particular, 1) we show that the divergence can be computed exactly for vector quantizers (VQs) and 2) has an approximate solution for Gauss mixtures (GMs) that, in high-dimensional feature spaces, introduces no significant degradation of the resulting similarity judgments. In both cases, the new solutions have closed-form and computational complexity equivalent to that of standard retrieval approaches.

With the ever-growing amount of online data and information, recommender systems are becoming overwhelmingly popular as an adequate approach for overcoming the challenge of information overload. Artificial Intelligence (AI) and Deep Learning (DL) have accumulated significant interest in many research areas, and recommender systems are one of them. In this paper, a Graph Convolutional Neural Network (GCNN)-based approach was used for online product recommendation. Graph-based methods have undergone substantial consideration for several recommendation tasks, with effective results. However, handling the computational complexities and training large datasets remain a challenge for such a model. Even though they are useful, the excessive measure of the model’s boundaries obstructs their applications in real-world recommender frameworks to a great extent. The recursive way of generating neighbor node embeddings for each node in the graph makes it more challenging to train a deep and large GCNN model. Therefore, we propose a model that incorporates measures of similarity between two different nodes, and these similarity measures help us to sample the neighbors beforehand. We estimate the similarity based on their interaction probability distribution with other nodes. We use KL divergence on different probability distributions to find the distance between them. This way, we set a threshold criterion for neighbor selection and generate other clusters. These clusters are then converted to subgraphs and are used as input for the proposed GCNN model. This approach simplifies the task of neighbor sampling for GCNN, and hence, we can observe a significant improvement in the computational complexity of the GCNN model. Finally, we compared the results with those for the previously proposed OpGCN model, basic GCNN model, and other traditional approaches such as collaborative filtering and probabilistic matrix factorization. The experiments showed that the complexity and computational time were decreased by estimating the similarity among nodes and sampling the nodes before training.

A fundamental challenge of mobile sensor networks is automated active reconfiguration of sensors in response to environmental stimuli in order to maximize their total sensing quality (or minimize their total sensing cost) of events occurring over an environment. In this paper, given an event distribution over a convex environment, we consider mobile isotropic sensors with adjustable sensing range and propose a new family of provably correct reactive coverage control algorithms for both continuous- and discrete-time sensor dynamics. The proposed coverage control algorithms constantly (re)configure sensor positions and sensing ranges in order to minimize a statistical distance, in particular, an $f$ -divergence, between the event distribution over the environment and the overall event detection probability of sensors. We show that the standard Voronoi-based coverage control law of homogeneous mobile sensor networks is a special case of our framework where the event detection probability of each sensor has a Gaussian form, the statistical distance is set to be the Kullback–Leibler (KL) divergence and sensor allocation is performed based on Voronoi diagrams. To increase the practicality of our framework, we also present its integration with a Voronoi-based collision avoidance strategy for disk-shaped sensor bodies and its extension to differential drive sensor dynamics, while retaining the stability properties. [ABSTRACT FROM AUTHOR]

Data sparsity is a widespread problem of collaborative filtering (CF) recommendation algorithms. However, some common CF methods cannot adequately utilize all user rating information; they are only able to use a small part of the rating data, depending on the co-rated items, which leads to low prediction accuracy. To alleviate this problem, a novel K-medoids clustering recommendation algorithm based on probability distribution for CF is proposed. The proposed scheme makes full use of all rating information based on Kullback–Leibler (KL) divergence from the perspective of item rating probability distribution, and distinguishes different items efficiently when selecting the cluster centers. Meanwhile, the distance model breaks the symmetric mode of classic geometric distance methods (such as Euclidean distance) and considers the effects of different rating numbers between items to emphasize their asymmetric relationship. Experimental results on different datasets show that the proposed clustering algorithm outperforms other compared methods in various evaluation metrics; this approach enhances the prediction accuracy and effectively deals with the sparsity problem. • An improved Kullback–Leibler (KL) divergence is introduced to calculate item similarity. • The optimum center of our algorithm is obtained by maximizing the contribution sum of distance. • An asymmetric mode is used to emphasize the asymmetric relationship between items. • Results show that our scheme improves the effectiveness of recommendation systems. [ABSTRACT FROM AUTHOR]

In ground-based remote sensing cloud image observation, images with the highest possible resolution are captured to obtain sufficient information about clouds. However, when features are extracted and classification is performed on the basis of the original images, a high-resolution probably means a high (or even more, unacceptable) computation cost. In practical application, a simple and commonly adopted method is to appropriately resize the original image to a version with a decreased resolution. An inevitable problem is whether useful information is lost in this resizing operation. This paper demonstrates that information loss is inevitable and poor classification results may be obtained from the analysis of local binary pattern (LBP) histogram features. However, this problem has been always neglected in previous studies, and the original image is arbitrarily resized without any criterion. In particular, the histogram features based on LBPs actually reflect the distribution of features. Thus, a criterion based on the Kullback–Leibler divergence between LBP histograms from the original and resized images and a penalty term imposed on the resolution are proposed to select the resolution of the resized image. The optimal resolution of the resized image can be selected by minimizing this criterion. Furthermore, experiments based on three ground-based remote sensing cloud image data sets with different original resolutions validate this criterion by analyzing the LBP histogram features. [ABSTRACT FROM AUTHOR]

Timely and effective detection of potential incipient faults in satellites plays an important role in improving their availability and extending their service life. In this paper, the problem of detecting incipient faults using projection vector (PV) and Kullback-Leibler (KL) divergence is studied in the context of detecting incipient faults in satellites. Under the assumption that the variables obey a multidimensional Gaussian distribution and using KL divergence to detect incipient faults, this paper models the optimum PV for detecting incipient faults as an optimization problem. It proves that the PVs obtained by principal component analysis (PCA) are not necessarily the optimum PV for detecting incipient faults. It then compares the on-line probability density function (PDF) with the reference PDF for detecting incipient faults on the local optimum PV. A numerical example and a real satellite fault case were used to assess the validity and superiority of the method proposed in this paper over conventional methods. Since the method takes into account the characteristics of the actual incipient faults, it is more adaptable to various possible incipient faults. Fault detection rates of three simulated faults and the real satellite fault are 98%, 84%, 93% and 92%, respectively.

This paper uses Bayesian robust new hidden Markov modeling (BRNHMM) for bearing fault detection and diagnosis based on its acoustic emission signal. A variational Bayesian approach is used that simultaneously approximates the distribution over the hidden states and parameters with simpler distribution hence using Bayesian inference for the estimation of the posterior HMM hyperparameters. This allows for online detection as small data sets can be used. Also, the Kullback-Leibler (KL) divergence is effectively used to access the divergence of the probability function of the BRNHMM, to find its lower bound approximation and by applying a linear transform to the maximum output probability parameter generation (MOPPG). The training set result obtained from BRNHMM is then compared to the result from artificial neural network (ANN) fault detection for same complex system of low speed and varying load conditions which are difficult from a diagnostic perspective, as found in rolling mills. [ABSTRACT FROM AUTHOR]

This paper presents an improved incipient fault detection method based on Kullback-Leibler (KL) divergence under multivariate statistical analysis frame. Different from the traditional multivariate fault detection methods, this methodology can detect slight anomalous behaviors by comparing the online probability density function (PDF) online with the reference PDF obtained from large scale off-line data set. In the principal and residual subspaces obtained via PCA, a symmetric evaluation function is defined for both single variate and multivariate cases. The uniform form of probability distribution and fault detection thresholds associated with all eigenvalues are given. In addition, the robust performance is analyzed with respect to a wide range of Signal to Noise Ratio (SNR). Case studies are conducted with three types of incipient faults on a numerical example; combining with two nonlinear projections, the proposed scheme is successfully used for incipient fault detection in non-Gaussian electrical drive system. The results can demonstrate the superiority of the proposed method than several other methods. [ABSTRACT FROM AUTHOR]

Wedding mobile phone sensor technology and human spatial behaviour has great potential. The ubiquity of Global Positioning Systems (GPS) technology has made gathering data about human mobility simpler, more precise, and with higher fidelity, providing minute-by-minute records of the locations of cohorts from dozens of participants. While this data provides a strong basis for Geographic Information Science research, it also constitutes an invasion of the participants’ privacy and can provide more information than researchers require to answer their questions. As an ethical and practical consideration, researchers should gather only as much data as they need. In this paper, we take three weeks of GPS traces from over a hundred student participants in mobile phone-based tracking studies and show that fewer than 14 days of data is necessary to establish complete activity spaces. We define ‘complete’ as the point at which marginal information gains become negligible according to a pairwise temporal analysis of the Kullback-Leibler (KL) divergence of the spatial (bivariate) histogram through time. For the fixed level of information difference, observable in the data, impacts due to individual variability, population composition, and spatial resolution are evident. However, all populations at each level of resolution examined in the paper demonstrated convergence to low divergence levels occurred within a matter of days, and to negligible information gain in less than two weeks. The methods described in the paper represent a novel metric useful to understand the interaction between measurements and information in human mobility. [ABSTRACT FROM AUTHOR]

In this paper, the standard hard C-means (HCM) clustering approach to image segmentation is modified by incorporating weighted membership Kullback-Leibler (KL) divergence and local data information into the HCM objective function. The membership KL divergence, used for fuzzification, measures the proximity between each cluster membership function of a pixel and the locally-smoothed value of the membership in the pixel vicinity. The fuzzification weight is a function of the pixel to cluster-centers distances. The used pixel to a cluster-center distance is composed of the original pixel data distance plus a fraction of the distance generated from the locally-smoothed pixel data. It is shown that the obtained membership function of a pixel is proportional to the locally-smoothed membership function of this pixel multiplied by an exponentially distributed function of the minus pixel distance relative to the minimum distance provided by the nearest cluster-center to the pixel. Therefore, since incorporating the locally-smoothed membership and data information in addition to the relative distance, which is more tolerant to additive noise than the absolute distance, the proposed algorithm has a threefold noise-handling process. The presented algorithm, named local data and membership KL divergence based fuzzy C-means (LDMKLFCM), is tested by synthetic and real-world noisy images and its results are compared with those of several FCM-based clustering algorithms. [ABSTRACT FROM AUTHOR]

A novel optimal power allocation algorithm for radar network systems is proposed for Low Probability of Intercept (LPI) technology in modern electronic warfare. The algorithm is based on the LPI optimization. First, the Schleher intercept factor for a radar network is derived, and then the Schleher intercept factor is minimized by optimizing the transmission power allocation among netted radars in the network to guarantee target-tracking performance. Furthermore, the Nonlinear Programming Genetic Algorithm (NPGA) is used to solve the resulting nonconvex, nonlinear, and constrained optimization problem. Numerical simulation results show the effectiveness of the proposed algorithm.

In this paper, a fuzzy clustering technique for image segmentation is developed by incorporating a hybrid of local spatial membership and data information into the conventional hard C-means (HCM) algorithm. This incorporation is a threefold procedure. (1) The membership function of a pixel is spatially smoothed in the pixel vicinity. (2) The Kullback-Leibler (KL) divergence between the pixel membership function and the smoothed one is added to the HCM objective function for fuzzification. (3) The resulting fuzzified HCM is regularized by adding a weighted HCM-like function where the original pixel data are replaced by locally smoothed ones. Thereby the weight is proportional to the residual of the locally smoothed membership. This residual decreases when many pixels existing in the pixel vicinity belong to the same cluster. Thus, the weighted distance decreases, allowing the pixel membership to follow the dominant membership in the pixel vicinity. The simulation results of segmenting synthetic, medical and media images have shown that the proposed algorithm provides better performance compared to several previously developed algorithms. For example, in a synthetic image, with added white Gaussian noise having a variance of 0.3, the proposed algorithm provides accuracy, sensitivity and specificity of 92%, 84% and 94.7% respectively, while the algorithm with the closest results provides 81.9% of accuracy, 62.2% of sensitivity and 86.8% of specificity. In addition, the proposed algorithm shows the capability to identify the number of clusters. [ABSTRACT FROM AUTHOR]

This paper proposes a robust similarity measure for two attributed scattering center (ASC) sets and applies it to synthetic aperture radar (SAR) automatic target recognition (ATR). The extraction uncertainty of an individual ASC is modeled by an adaptive Gaussian distribution according to its attributes. Then the distance between two individual ASCs is defined as the Kullback–Leibler (KL) divergence between two Gaussian distributions which model the uncertainties of those two ASCs. The proposed distance measure can better exploit the inner discrepancy between individual ASCs compared with the Euclid distance or Mahalanobis distance. Based on the proposed distance measure, a cost matrix which contains the costs of false and missing ASCs is built and the Hungarian algorithm is employed to build a one-to-one correspondence between two ASC sets. A threshold method is carried out to further evaluate the Hungarian assignment. Afterwards, a robust similarity measure is designed to evaluate the similarity between the two ASC sets which comprehensively considers the influences of the missing and false ASCs as well as the disproportionate contributions by different ASCs. Finally, the target type is determined by the similarities between the testing image and various types of template targets. Experimental results on the moving and stationary target acquisition and recognition (MSTAR) dataset verify the validity and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

A new algorithm to partial discharges (PD) location in transformer windings by means of the discrete wavelets transform (DWT) and the Kullback–Leibler (KL) divergence is presented. When the insulation system of transformers has considerable damage, high levels of PD appear. A PD analysis will help to prevent and to avoid catastrophic faults in the transformer insulation system, and is also useful to quantify the damage level into it. However, it is a complex task, because PD signals have a small magnitude and are presented during some microseconds. In this paper, a lumped parameter model RLC is used to model the transformer winding and to obtain PD reference signals. The DWT is used to process those signals. PD location is finally estimated by means of the minimum entropy concept that minimizes the Kullback–Leibler (KL) divergence between PD signals (test and reference signals). Different time durations and amplitudes to the PD signals were considered. The results of computer simulation confirm the accuracy of the proposed algorithm, with an error less than 5%. Also, the algorithm is validated in a distribution transformer winding. [ABSTRACT FROM AUTHOR]

The main problem of ecological data modeling is their interpretation and its correct understanding. This problem cannot be solved solely by a big data collection. To sufficiently understand ecosystems we need to know how these processes behave and how they respond to internal and external factors. Similarly, we need to know the behavior of processes that are involved in the climate system and the biosphere of the earth. In order to characterize precisely the behavior of individual elements and ecosystems we need to use deterministic, stochastic and chaotic behavior. Unfortunately, the chaotic part of systems is typically completely ignored in almost all approaches. Ignoring of chaotical part leads to many biased outcomes. To overcome this gap we model chaotic system behavior by random iterated function system which provides a generic guideline for such data management. This also allows to replicate a complexity and chaos of ecosystem. [ABSTRACT FROM AUTHOR]

The toxicity and efficacy of more than 30 anticancer agents present very high variations, depending on the dosing time. Therefore, the biologists studying the circadian rhythm require a very precise method for estimating the periodic component (PC) vector of chronobiological signals. Moreover, in recent developments, not only the dominant period or the PC vector present a crucial interest but also their stability or variability. In cancer treatment experiments, the recorded signals corresponding to different phases of treatment are short, from 7 days for the synchronization segment to 2 or 3 days for the after-treatment segment. When studying the stability of the dominant period, we have to consider very short length signals relative to the prior knowledge of the dominant period, placed in the circadian domain. The classical approaches, based on Fourier transform (FT) methods are inefficient (i.e., lack of precision) considering the particularities of the data (i.e., the short length). Another particularity of the signals considered in such experiments is the level of noise: such signals are very noisy and establishing the periodic components that are associated with the biological phenomena and distinguishing them from the ones associated with the noise are difficult tasks. In this paper, we propose a new method for the estimation of the PC vector of biomedical signals, using the biological prior informations and considering a model that accounts for the noise. The experiments developed in cancer treatment context are recording signals expressing a limited number of periods. This is a prior information that can be translated as the sparsity of the PC vector. The proposed method considers the PC vector estimation as an Inverse Problem (IP) using the general Bayesian inference in order to infer the unknown of our model, i.e. the PC vector but also the hyperparameters (i.e the variances). The sparsity prior information is modeled using a sparsity enforcing prior law. In this paper, we propose a Student's t distribution, viewed as the marginal distribution of a bivariate normal-inverse gamma distribution. We build a general infinite Gaussian scale mixture (IGSM) hierarchical model where we assign prior distributions also for the hyperparameters. The expression of the joint posterior law of the unknown PC vector and hyperparameters is obtained via Bayes rule, and then, the unknowns are estimated via joint maximum a posteriori (JMAP) or posterior mean (PM). For the PM estimator, the expression of the posterior distribution is approximated by a separable one, via variational Bayesian approximation (VBA), using the Kullback-Leibler (KL) divergence. For the PM estimation, two possibilities are considered: an approximation with a partially separable distribution and an approximation with a fully separable one. Both resulting algorithms corresponding to the PM estimation and the one corresponding to the JMAP estimation are iterative algorithms. The algorithms are presented in detail and are compared with the ones corresponding to the Gaussian model. We examine the convergency of the algorithms and give simulation results to compare their performances. Finally, we show simulation results on synthetic and real data in cancer treatment applications. The real data considered in this paper examines the rest-activity patterns of KI/KI Per2::luc mouse, aged 10 weeks, singly housed in RealTime Biolumicorder (RT-BIO). [ABSTRACT FROM AUTHOR]

Unlike existing studies focused on the causal relationship between electricity consumption and economic growth at the macro level, this paper uses monthly data from January 2006 to December 2015 and applies the correlation coefficient, as well as Kullback-Leibler (KL) divergence, to study the time difference relationship between sectoral electricity consumption and economic growth. The empirical results draw some main findings as follows: First, the time difference relationships show diversity at the sector level but will form a kind of overall characteristic between economic growth and total electricity consumption. Secondly, not all sectors have a remarkable correlation between sectoral electricity consumption and economic growth as only part of them have reasonable values to describe the time difference relationship. Thirdly, the results present both diversity and aggregation at the industry level, while lagging sectors mainly concentrate in the manufacturing industry. The relationship between sectoral electricity consumption and economic growth can be further explored and described from a new perspective based on the results. Further, the trend of economic development and sectoral electricity consumption can be predicted to help policy-makers formulate proper policies.

• A DRO model based on Kullback-Leibler (KL) divergence is constructed. • Energy sharing and profit allocation are considered in dispatching. • By introducing energy sharing, the operation cost of the community can be reduced. • Through profit reallocation, the enthusiasm of community cooperation can be improved. Community energy consumption accounts for a large proportion of total social energy consumption. Therefore, community integrated energy systems have emerged to satisfy these demands. However, limited resource allocation of the integrated energy system in a single community is possible because of poor anti-interference and insufficient photovoltaic absorption. A distributionally robust optimal dispatching model was proposed for the joint operation of multiple community integrated energy systems, and the energy sharing and joint demand response among communities were considered. Kullback–Leibler (KL) divergence was used to describe the ambiguity set of photovoltaic output probability distribution. When the photovoltaic output prediction error obeyed the extreme probability distribution of ambiguity set, the minimum operation cost of multi-community was solved. Furthermore, to ensure successful alliance of multi-community cooperative alliance, a profit allocation mechanism based on the improved Shapley value was proposed to reallocate the excess return fairly. Finally, a typical multi community integrated energy system was simulated and analysed. The results revealed that energy sharing and joint demand response improve the economy of multi community operation, and the coordination of system robustness and economy of the model was realised. The proposed profit allocation mechanism can effectively ensure multi community cooperation. [ABSTRACT FROM AUTHOR]