Search

• The "bilateral reduction" effect of Rényi entropy is theoretically studied and proved. • Relative entropy is extended by adding order α to generalized relative entropy. • Generalized relative entropy satisfies six sparse criteria. • Conversion between complexity measures and sparsity measures is realized. • New sparse measure cluster is proposed to enrich machine condition monitoring. Complexity measures and sparsity measures are capable of detecting and characterizing underlying dynamic changes in machine condition monitoring. Previous studies have developed Shannon entropy and Kullback-Leibler (KL) divergence and their generalization, namely Rényi entropy and Rényi divergence, but their theoretical properties as a function of sparsity and their transitions from complexity measures to sparsity measures are not fully explored and discussed. This paper continues conducting theoretical and experimental investigations on Rényi entropy and Rényi divergence and exploring their theoretical properties and transitions to enrich the domain of complexity and sparsity measures and their applications to machine condition monitoring. Specifically, this paper theoretically proves and verifies that Rényi entropy has a similar "bilateral reduction" effect exhibited by other complexity measures including correlation dimension, approximate entropy, fuzzy entropy and sample entropy. Inspired by relative entropy, which is a special case of KL divergence and a sparsity measure, and the relationship between KL divergence and Rényi divergence, in this paper, we attempt to generalize relative entropy by adding an order of α , to convert a series of complexity measures from Rényi entropy to a series of sparsity measures, which answers how to realize transitions from complexity measures to sparsity measures. More importantly, this paper is helpful to provide a guideline for correct uses of Rényi entropy, Rényi divergence, and generalized relative entropy for the quantification of impulsive transients caused by machine faults for machine condition monitoring. [ABSTRACT FROM AUTHOR]

It is of great practical importance to compare and combine data from different studies in order to carry out appropriate and more powerful statistical inference. We propose a partition based measure to quantify the compatibility of two datasets using their respective posterior distributions. We further propose an information gain measure to quantify the information increase (or decrease) in combining two datasets. These measures are well calibrated and efficient computational algorithms are provided for their calculations. We use examples in a benchmark dose toxicology study, a six cities pollution data and a melanoma clinical trial to illustrate how these two measures are useful in combining current data with historical data and missing data. [ABSTRACT FROM AUTHOR]

In the context of recent deep clustering studies, discriminative models dominate the literature and report the most competitive performances. These models learn a deep discriminative neural network classifier in which the labels are latent. Typically, they use multinomial logistic regression posteriors and parameter regularization, as is very common in supervised learning. It is generally acknowledged that discriminative objective functions (e.g., those based on the mutual information or the KL divergence) are more flexible than generative approaches (e.g., K-means) in the sense that they make fewer assumptions about the data distributions and, typically, yield much better unsupervised deep learning results. On the surface, several recent discriminative models may seem unrelated to K-means. This study shows that these models are, in fact, equivalent to K-means under mild conditions and common posterior models and parameter regularization. We prove that, for the commonly used logistic regression posteriors, maximizing the L2 regularized mutual information via an approximate alternating direction method (ADM) is equivalent to minimizing a soft and regularized K-means loss. Our theoretical analysis not only connects directly several recent state-of-the-art discriminative models to K-means, but also leads to a new soft and regularized deep K-means algorithm, which yields competitive performance on several image clustering benchmarks. [ABSTRACT FROM AUTHOR]

A new algorithm to partial discharges (PD) location in transformer windings by means of the discrete wavelets transform (DWT) and the Kullback–Leibler (KL) divergence is presented. When the insulation system of transformers has considerable damage, high levels of PD appear. A PD analysis will help to prevent and to avoid catastrophic faults in the transformer insulation system, and is also useful to quantify the damage level into it. However, it is a complex task, because PD signals have a small magnitude and are presented during some microseconds. In this paper, a lumped parameter model RLC is used to model the transformer winding and to obtain PD reference signals. The DWT is used to process those signals. PD location is finally estimated by means of the minimum entropy concept that minimizes the Kullback–Leibler (KL) divergence between PD signals (test and reference signals). Different time durations and amplitudes to the PD signals were considered. The results of computer simulation confirm the accuracy of the proposed algorithm, with an error less than 5%. Also, the algorithm is validated in a distribution transformer winding. [ABSTRACT FROM AUTHOR]

The main problem of ecological data modeling is their interpretation and its correct understanding. This problem cannot be solved solely by a big data collection. To sufficiently understand ecosystems we need to know how these processes behave and how they respond to internal and external factors. Similarly, we need to know the behavior of processes that are involved in the climate system and the biosphere of the earth. In order to characterize precisely the behavior of individual elements and ecosystems we need to use deterministic, stochastic and chaotic behavior. Unfortunately, the chaotic part of systems is typically completely ignored in almost all approaches. Ignoring of chaotical part leads to many biased outcomes. To overcome this gap we model chaotic system behavior by random iterated function system which provides a generic guideline for such data management. This also allows to replicate a complexity and chaos of ecosystem. [ABSTRACT FROM AUTHOR]

We concentrate on the paradigm question how much stochasticity and how much chaos are present in the methane emission model. In particular we analyze the residua from the process of methane emissions from wetlands in the sedge-grass marsh, in South Bohemia, Czech Republic. Relation to entropy and a specific version of Kullback–Leibler divergence will be given. A graphical tool to assess the amount of entropy in the system is developed and illustrated on real data from the sedge-grass marsh methane emission. [ABSTRACT FROM AUTHOR]