Search

Two-dimensional fuzzy entropy, dispersion entropy, and their multiscale extensions (MFuzzyEn2D and MDispEn2D, respectively) have shown promising results for image classifications. However, these results rely on the selection of key parameters that may largely influence the entropy values obtained. Yet, the optimal choice for these parameters has not been studied thoroughly. We propose a study on the impact of these parameters in image classification. For this purpose, the entropy-based algorithms are applied to a variety of images from different datasets, each containing multiple image classes. Several parameter combinations are used to obtain the entropy values. These entropy values are then applied to a range of machine learning classifiers and the algorithm parameters are analyzed based on the classification results. By using specific parameters, we show that both MFuzzyEn2D and MDispEn2D approach state-of-the-art in terms of image classification for multiple image types. They lead to an average maximum accuracy of more than 95% for all the datasets tested. Moreover, MFuzzyEn2D results in a better classification performance than that extracted by MDispEn2D as a majority. Furthermore, the choice of classifier does not have a significant impact on the classification of the extracted features by both entropy algorithms. The results open new perspectives for these entropy-based measures in textural analysis.

The leader–follower structure is widely used in unmanned aerial vehicle formation. This paper adopts the proportional-integral-derivative (PID) and the linear quadratic regulator controllers to construct the leader–follower formation. Tuning the PID controllers is generally empirical; hence, various surrogate models have been introduced to identify more refined parameters with relatively lower cost. However, the construction of surrogate models faces the problem that the singular points may affect the accuracy, such that the global surrogate models may be invalid. Thus, to tune controllers quickly and accurately, the regional surrogate model technique (RSMT), based on analyzing the regional information entropy, is proposed. The proposed RSMT cooperates only with the successful samples to mitigate the effect of singular points along with a classifier screening failed samples. Implementing the RSMT with various kinds of surrogate models, this study evaluates the Pareto fronts of the original simulation model and the RSMT to compare their effectiveness. The results show that the RSMT can accurately reconstruct the simulation model. Compared with the global surrogate models, the RSMT reduces the run time of tuning PID controllers by one order of magnitude, and it improves the accuracy of surrogate models by dozens of orders of magnitude.

We describe a classifier made of an ensemble of decision trees, designed using information theory concepts. In contrast to algorithms C4.5 or ID3, the tree is built from the leaves instead of the root. Each tree is made of nodes trained independently of the others, to minimize a local cost function (information bottleneck). The trained tree outputs the estimated probabilities of the classes given the input datum, and the outputs of many trees are combined to decide the class. We show that the system is able to provide results comparable to those of the tree classifier in terms of accuracy, while it shows many advantages in terms of modularity, reduced complexity, and memory requirements.

The dynamics of brain area influenced by focal epilepsy can be studied using focal and non-focal electroencephalogram (EEG) signals. This paper presents a new method to detect focal and non-focal EEG signals based on an integrated index, termed the focal and non-focal index (FNFI), developed using discrete wavelet transform (DWT) and entropy features. The DWT decomposes the EEG signals up to six levels, and various entropy measures are computed from approximate and detail coefficients of sub-band signals. The computed entropy measures are average wavelet, permutation, fuzzy and phase entropies. The proposed FNFI developed using permutation, fuzzy and Shannon wavelet entropies is able to clearly discriminate focal and non-focal EEG signals using a single number. Furthermore, these entropy measures are ranked using different techniques, namely the Bhattacharyya space algorithm, Student’s t-test, the Wilcoxon test, the receiver operating characteristic (ROC) and entropy. These ranked features are fed to various classifiers, namely k-nearest neighbour (KNN), probabilistic neural network (PNN), fuzzy classifier and least squares support vector machine (LS-SVM), for automated classification of focal and non-focal EEG signals using the minimum number of features. The identification of the focal EEG signals can be helpful to locate the epileptogenic focus.

Nowadays, sustained development of different social media can be observed worldwide. One of the relevant research domains intensively explored recently is analysis of social communities existing in social media as well as prediction of their future evolution taking into account collected historical evolution chains. These evolution chains proposed in the paper contain group states in the previous time frames and its historical transitions that were identified using one out of two methods: Stable Group Changes Identification (SGCI) and Group Evolution Discovery (GED). Based on the observed evolution chains of various length, structural network features are extracted, validated and selected as well as used to learn classification models. The experimental studies were performed on three real datasets with different profile: DBLP, Facebook and Polish blogosphere. The process of group prediction was analysed with respect to different classifiers as well as various descriptive feature sets extracted from evolution chains of different length. The results revealed that, in general, the longer evolution chains the better predictive abilities of the classification models. However, chains of length 3 to 7 enabled the GED-based method to almost reach its maximum possible prediction quality. For SGCI, this value was at the level of 3–5 last periods.

The brain is a complex structure made up of interconnected neurons, and its electrical activities can be evaluated using electroencephalogram (EEG) signals. The characteristics of the brain area affected by partial epilepsy can be studied using focal and non-focal EEG signals. In this work, a method for the classification of focal and non-focal EEG signals is presented using entropy measures. These entropy measures can be useful in assessing the nonlinear interrelation and complexity of focal and non-focal EEG signals. These EEG signals are first decomposed using the empirical mode decomposition (EMD) method to extract intrinsic mode functions (IMFs). The entropy features, namely, average Shannon entropy (ShEnAvg), average Renyi’s entropy (RenEnAvg ), average approximate entropy (ApEnAvg), average sample entropy (SpEnAvg) and average phase entropies (S1Avg and S2Avg), are computed from different IMFs of focal and non-focal EEG signals. These entropies are used as the input feature set for the least squares support vector machine (LS-SVM) classifier to classify into focal and non-focal EEG signals. Experimental results show that our proposed method is able to differentiate the focal and non-focal EEG signals with an average classification accuracy of 87% correct.

In this work, we analyze the cross-entropy function, widely used in classifiers both as a performance measure and as an optimization objective. We contextualize cross-entropy in the light of Bayesian decision theory, the formal probabilistic framework for making decisions, and we thoroughly analyze its motivation, meaning and interpretation from an information-theoretical point of view. In this sense, this article presents several contributions: First, we explicitly analyze the contribution to cross-entropy of (i) prior knowledge; and (ii) the value of the features in the form of a likelihood ratio. Second, we introduce a decomposition of cross-entropy into two components: discrimination and calibration. This decomposition enables the measurement of different performance aspects of a classifier in a more precise way; and justifies previously reported strategies to obtain reliable probabilities by means of the calibration of the output of a discriminating classifier. Third, we give different information-theoretical interpretations of cross-entropy, which can be useful in different application scenarios, and which are related to the concept of reference probabilities. Fourth, we present an analysis tool, the Empirical Cross-Entropy (ECE) plot, a compact representation of cross-entropy and its aforementioned decomposition. We show the power of ECE plots, as compared to other classical performance representations, in two diverse experimental examples: a speaker verification system, and a forensic case where some glass findings are present. [ABSTRACT FROM AUTHOR]

In the present work, an automated method to diagnose Congestive Heart Failure (CHF) using Heart Rate Variability (HRV) signals is proposed. This method is based on Flexible Analytic Wavelet Transform (FAWT), which decomposes the HRV signals into different sub-band signals. Further, Accumulated Fuzzy Entropy (AFEnt) and Accumulated Permutation Entropy (APEnt) are computed over cumulative sums of these sub-band signals. This provides complexity analysis using fuzzy and permutation entropies at different frequency scales. We have extracted 20 features from these signals obtained at different frequency scales of HRV signals. The Bhattacharyya ranking method is used to rank the extracted features from the HRV signals of three different lengths (500, 1000 and 2000 samples). These ranked features are fed to the Least Squares Support Vector Machine (LS-SVM) classifier. Our proposed system has obtained a sensitivity of 98.07%, specificity of 98.33% and accuracy of 98.21% for the 500-sample length of HRV signals. Our system yielded a sensitivity of 97.95%, specificity of 98.07% and accuracy of 98.01% for HRV signals of a length of 1000 samples and a sensitivity of 97.76%, specificity of 97.67% and accuracy of 97.71% for signals corresponding to the 2000-sample length of HRV signals. Our automated system can aid clinicians in the accurate detection of CHF using HRV signals. It can be installed in hospitals, polyclinics and remote villages where there is no access to cardiologists. [ABSTRACT FROM AUTHOR]

In this work, we analyze the cross-entropy function, widely used in classifiers both as a performance measure and as an optimization objective. We contextualize cross-entropy in the light of Bayesian decision theory, the formal probabilistic framework for making decisions, and we thoroughly analyze its motivation, meaning and interpretation from an information-theoretical point of view. In this sense, this article presents several contributions: First, we explicitly analyze the contribution to cross-entropy of (i) prior knowledge; and (ii) the value of the features in the form of a likelihood ratio. Second, we introduce a decomposition of cross-entropy into two components: discrimination and calibration. This decomposition enables the measurement of different performance aspects of a classifier in a more precise way; and justifies previously reported strategies to obtain reliable probabilities by means of the calibration of the output of a discriminating classifier. Third, we give different information-theoretical interpretations of cross-entropy, which can be useful in different application scenarios, and which are related to the concept of reference probabilities. Fourth, we present an analysis tool, the Empirical Cross-Entropy (ECE) plot, a compact representation of cross-entropy and its aforementioned decomposition. We show the power of ECE plots, as compared to other classical performance representations, in two diverse experimental examples: a speaker verification system, and a forensic case where some glass findings are present.

In the present work, an automated method to diagnose Congestive Heart Failure (CHF) using Heart Rate Variability (HRV) signals is proposed. This method is based on Flexible Analytic Wavelet Transform (FAWT), which decomposes the HRV signals into different sub-band signals. Further, Accumulated Fuzzy Entropy (AFEnt) and Accumulated Permutation Entropy (APEnt) are computed over cumulative sums of these sub-band signals. This provides complexity analysis using fuzzy and permutation entropies at different frequency scales. We have extracted 20 features from these signals obtained at different frequency scales of HRV signals. The Bhattacharyya ranking method is used to rank the extracted features from the HRV signals of three different lengths (500, 1000 and 2000 samples). These ranked features are fed to the Least Squares Support Vector Machine (LS-SVM) classifier. Our proposed system has obtained a sensitivity of 98.07%, specificity of 98.33% and accuracy of 98.21% for the 500-sample length of HRV signals. Our system yielded a sensitivity of 97.95%, specificity of 98.07% and accuracy of 98.01% for HRV signals of a length of 1000 samples and a sensitivity of 97.76%, specificity of 97.67% and accuracy of 97.71% for signals corresponding to the 2000-sample length of HRV signals. Our automated system can aid clinicians in the accurate detection of CHF using HRV signals. It can be installed in hospitals, polyclinics and remote villages where there is no access to cardiologists.

The dynamics of brain area influenced by focal epilepsy can be studied using focal and non-focal electroencephalogram (EEG) signals. This paper presents a new method to detect focal and non-focal EEG signals based on an integrated index, termed the focal and non-focal index (FNFI), developed using discrete wavelet transform (DWT) and entropy features. The DWT decomposes the EEG signals up to six levels, and various entropy measures are computed from approximate and detail coefficients of sub-band signals. The computed entropy measures are average wavelet, permutation, fuzzy and phase entropies. The proposed FNFI developed using permutation, fuzzy and Shannon wavelet entropies is able to clearly discriminate focal and non-focal EEG signals using a single number. Furthermore, these entropy measures are ranked using different techniques, namely the Bhattacharyya space algorithm, Student's t-test, the Wilcoxon test, the receiver operating characteristic (ROC) and entropy. These ranked features are fed to various classifiers, namely k-nearest neighbour (KNN), probabilistic neural network (PNN), fuzzy classifier and least squares support vector machine (LS-SVM), for automated classification of focal and non-focal EEG signals using the minimum number of features. The identification of the focal EEG signals can be helpful to locate the epileptogenic focus. [ABSTRACT FROM AUTHOR]

Nowadays, sustained development of different social media can be observed worldwide. One of the relevant research domains intensively explored recently is analysis of social communities existing in social media as well as prediction of their future evolution taking into account collected historical evolution chains. These evolution chains proposed in the paper contain group states in the previous time frames and its historical transitions that were identified using one out of two methods: Stable Group Changes Identification (SGCI) and Group Evolution Discovery (GED). Based on the observed evolution chains of various length, structural network features are extracted, validated and selected as well as used to learn classification models. The experimental studies were performed on three real datasets with different profile: DBLP, Facebook and Polish blogosphere. The process of group prediction was analysed with respect to different classifiers as well as various descriptive feature sets extracted from evolution chains of different length. The results revealed that, in general, the longer evolution chains the better predictive abilities of the classification models. However, chains of length 3 to 7 enabled the GED-based method to almost reach its maximum possible prediction quality. For SGCI, this value was at the level of 3-5 last periods. [ABSTRACT FROM AUTHOR]

The brain is a complex structure made up of interconnected neurons, and its electrical activities can be evaluated using electroencephalogram (EEG) signals. The characteristics of the brain area affected by partial epilepsy can be studied using focal and non-focal EEG signals. In this work, a method for the classification of focal and non-focal EEG signals is presented using entropy measures. These entropy measures can be useful in assessing the nonlinear interrelation and complexity of focal and non-focal EEG signals. These EEG signals are first decomposed using the empirical mode decomposition (EMD) method to extract intrinsic mode functions (IMFs). The entropy features, namely, average Shannon entropy (ShEnAvg), average Renyi's entropy (RenEnAvg), average approximate entropy (ApEnAvg), average sample entropy (SpEnAvg) and average phase entropies (S1Avg and S2Avg), are computed from different IMFs of focal and non-focal EEG signals. These entropies are used as the input feature set for the least squares support vector machine (LS-SVM) classifier to classify into focal and non-focal EEG signals. Experimental results show that our proposed method is able to differentiate the focal and non-focal EEG signals with an average classification accuracy of 87% correct. [ABSTRACT FROM AUTHOR]

Two-dimensional fuzzy entropy, dispersion entropy, and their multiscale extensions ( MFuzzyEn 2 D and MDispEn 2 D , respectively) have shown promising results for image classifications. However, these results rely on the selection of key parameters that may largely influence the entropy values obtained. Yet, the optimal choice for these parameters has not been studied thoroughly. We propose a study on the impact of these parameters in image classification. For this purpose, the entropy-based algorithms are applied to a variety of images from different datasets, each containing multiple image classes. Several parameter combinations are used to obtain the entropy values. These entropy values are then applied to a range of machine learning classifiers and the algorithm parameters are analyzed based on the classification results. By using specific parameters, we show that both MFuzzyEn 2 D and MDispEn 2 D approach state-of-the-art in terms of image classification for multiple image types. They lead to an average maximum accuracy of more than 95% for all the datasets tested. Moreover, MFuzzyEn 2 D results in a better classification performance than that extracted by MDispEn 2 D as a majority. Furthermore, the choice of classifier does not have a significant impact on the classification of the extracted features by both entropy algorithms. The results open new perspectives for these entropy-based measures in textural analysis. [ABSTRACT FROM AUTHOR]

The leader–follower structure is widely used in unmanned aerial vehicle formation. This paper adopts the proportional-integral-derivative (PID) and the linear quadratic regulator controllers to construct the leader–follower formation. Tuning the PID controllers is generally empirical; hence, various surrogate models have been introduced to identify more refined parameters with relatively lower cost. However, the construction of surrogate models faces the problem that the singular points may affect the accuracy, such that the global surrogate models may be invalid. Thus, to tune controllers quickly and accurately, the regional surrogate model technique (RSMT), based on analyzing the regional information entropy, is proposed. The proposed RSMT cooperates only with the successful samples to mitigate the effect of singular points along with a classifier screening failed samples. Implementing the RSMT with various kinds of surrogate models, this study evaluates the Pareto fronts of the original simulation model and the RSMT to compare their effectiveness. The results show that the RSMT can accurately reconstruct the simulation model. Compared with the global surrogate models, the RSMT reduces the run time of tuning PID controllers by one order of magnitude, and it improves the accuracy of surrogate models by dozens of orders of magnitude. [ABSTRACT FROM AUTHOR]

We describe a classifier made of an ensemble of decision trees, designed using information theory concepts. In contrast to algorithms C4.5 or ID3, the tree is built from the leaves instead of the root. Each tree is made of nodes trained independently of the others, to minimize a local cost function (information bottleneck). The trained tree outputs the estimated probabilities of the classes given the input datum, and the outputs of many trees are combined to decide the class. We show that the system is able to provide results comparable to those of the tree classifier in terms of accuracy, while it shows many advantages in terms of modularity, reduced complexity, and memory requirements. [ABSTRACT FROM AUTHOR]