Search

Based on Shannon's communication theory, in the present paper, we provide the theoretical background to finding an objective measurement—the text-entropy—that can describe the quality of digital natural language documents handled with word processors. The text-entropy can be calculated from the formatting, correction, and modification entropy, and based on these values, we are able to tell how correct or how erroneous digital text-based documents are. To present how the theory can be applied to real-world texts, for the present study, three erroneous MS Word documents were selected. With these examples, we can demonstrate how to build their correcting, formatting, and modification algorithms, to calculate the time spent on modification and the entropy of the completed tasks, in both the original erroneous and the corrected documents. In general, it was found that using and modifying properly edited and formatted digital texts requires less or an equal number of knowledge-items. In information theory, it means that less data must be put on the communication channel than in the case of erroneous documents. The analysis also revealed that in the corrected documents not only the quantity of the data is less, but the quality of the data (knowledge pieces) is higher. As the consequence of these two findings, it is proven that the modification time of erroneous documents is severalfold of the correct ones, even in the case of minimal first level actions. It is also proven that to avoid the repetition of the time- and resource-consuming actions, we must correct the documents before their modification. [ABSTRACT FROM AUTHOR]

This paper presents a method for the transparent, robust, and highly capacitive watermarking of video signals using an information mapper. The proposed architecture is based on the use of deep neural networks to embed the watermark in the luminance channel in the YUV color space. An information mapper was used to enable the transformation of a multi-bit binary signature of varying capacitance reflecting the entropy measure of the system into a watermark embedded in the signal frame. To confirm the effectiveness of the method, tests were carried out for video frames with a resolution of 256 × 256 pixels, with a watermark capacity of 4 to 16,384 bits. Transparency metrics (SSIM and PSNR) and a robustness metric—the bit error rate (BER)—were used to assess the performance of the algorithms. [ABSTRACT FROM AUTHOR]

In modern industry, due to the poor working environment and the complex working conditions of mechanical equipment, the characteristics of the impact signals caused by faults are often submerged in strong background signals and noises. Therefore, it is difficult to effectivelyextract the fault features. In this paper, a fault feature extraction method based on improved VMD multi-scale dispersion entropy and TVD-CYCBD is proposed. First, the marine predator algorithm (MPA) is used to optimize the modal components and penalty factors in VMD. Second, the optimized VMD is used to model and decompose the fault signal, and then the optimal signal components are filtered according to the combined weight index criteria. Third, TVD is used to denoise the optimal signal components. Finally, CYCBD filters the de-noised signal and then envelope demodulation analysis is carried out. Through the simulation signal experiment and the actual fault signal experiment, the results verified that multiple frequency doubling peaks can be seen from the envelope spectrum, and there is little interference near the peak, which shows the good performance of the method. [ABSTRACT FROM AUTHOR]

Today's world faces a serious public health problem with cancer. One type of cancer that begins in the breast and spreads to other body areas is breast cancer (BC). Breast cancer is one of the most prevalent cancers that claim the lives of women. It is also becoming clearer that most cases of breast cancer are already advanced when they are brought to the doctor's attention by the patient. The patient may have the evident lesion removed, but the seeds have reached an advanced stage of development or the body's ability to resist them has weakened considerably, rendering them ineffective. Although it is still much more common in more developed nations, it is also quickly spreading to less developed countries. The motivation behind this study is to use an ensemble method for the prediction of BC, as an ensemble model aims to automatically manage the strengths and weaknesses of each of its separate models, resulting in the best decision being made overall. The main objective of this paper is to predict and classify breast cancer using Adaboost ensemble techniques. The weighted entropy is computed for the target column. Taking each attribute's weights results in the weighted entropy. Each class's likelihood is represented by the weights. The amount of information gained increases with a decrease in entropy. Both individual and homogeneous ensemble classifiers, created by mixing Adaboost with different single classifiers, have been used in this work. In order to deal with the class imbalance issue as well as noise, the synthetic minority over-sampling technique (SMOTE) was used as part of the data mining pre-processing. The suggested approach uses a decision tree (DT) and naive Bayes (NB), with Adaboost ensemble techniques. The experimental findings shown 97.95% accuracy for prediction using the Adaboost-random forest classifier. [ABSTRACT FROM AUTHOR]

As the core equipment of the high-pressure diaphragm pump, the working conditions of the check valve are complicated, and the vibration signal generated during operation displays non-stationary and nonlinear characteristics. In order to accurately describe the non-linear dynamics of the check valve, the smoothing prior analysis (SPA) method is used to decompose the vibration signal of the check valve, obtain the tendency term and fluctuation term components, and calculate the frequency-domain fuzzy entropy (FFE) of the component signals. Using FFE to characterize the operating state of the check valve, the paper proposes a kernel extreme-learning machine (KELM) function norm regularization method, which is used to construct a structurally constrained kernel extreme-learning machine (SC-KELM) fault-diagnosis model. Experiments demonstrate that the frequency-domain fuzzy entropy can accurately characterize the operation state of check valve, and the improvement of the generalization of the SC-KELM check valve fault model improves the recognition accuracy of the check-valve fault-diagnosis model, with an accuracy rate of 96.67%. [ABSTRACT FROM AUTHOR]

Estimation of Rényi entropy is of fundamental importance to many applications in cryptography, statistical inference, and machine learning. This paper aims to improve the existing estimators with regard to: (a) the sample size, (b) the estimator adaptiveness, and (c) the simplicity of the analyses. The contribution is a novel analysis of the generalized "birthday paradox" collision estimator. The analysis is simpler than in prior works, gives clear formulas, and strengthens existing bounds. The improved bounds are used to develop an adaptive estimation technique that outperforms previous methods, particularly in regimes of low or moderate entropy. Last but not least, to demonstrate that the developed techniques are of broader interest, a number of applications concerning theoretical and practical properties of "birthday estimators" are discussed. [ABSTRACT FROM AUTHOR]

Entropy is a great tool in thermodynamics and statistical physics. This Special Issue is devoted to the mathematical background and physical motivation behind using non-additive entropy formulas, which are, in most cases, still group entropies, defined as expectation values, and incorporate, in one way or another, nontrivial correlations in the investigated systems and processes. The concept of entropy has been changing swiftly since C. Tsallis and, before him, a series of mathematicians and informatics experts proposed various generalizations of Boltzmann-Gibbs entropy. [Extracted from the article]

In a classic paper, Morduchow and Libby use an analytic solution for the profile of a Navier-Stokes shock to show that the equilibrium thermodynamic entropy has a maximum inside the shock. There is no general nonequilibrium thermodynamic formulation of entropy; the extension of equilibrium theory to nonequililbrium processes is usually made through the assumption of local thermodynamic equilibrium (LTE). However, gas kinetic theory provides a perfectly general formulation of a nonequilibrium entropy in terms of the probability distribution function (PDF) solutions of the Boltzmann equation. In this paper I will evaluate the Boltzmann entropy for the PDF that underlies the Navier-Stokes equations and also for the PDF of the Mott-Smith shock solution. I will show that both monotonically increase in the shock. I will propose a new nonequilibrium thermodynamic entropy and show that it is also monotone and closely approximates the Boltzmann entropy. [ABSTRACT FROM AUTHOR]

The present age, which can be called the Information Age, has a core technology constituted by bits transported by photons. Both concepts, bit and photon, originated in the past century: the concept of photon was introduced by Planck in 1900 when he advanced the solution of the blackbody spectrum, and bit is a term first used by Shannon in 1948 when he introduced the theorems that founded information theory. The connection between Planck and Shannon is not immediately apparent; nor is it obvious that they derived their basic results from the concept of entropy. Examination of other important scientists can shed light on Planck's and Shannon's work in these respects. Darwin and Fowler, who in 1922 published a couple of papers where they reinterpreted Planck's results, pointed out the centrality of the partition function to statistical mechanics and thermodynamics. The same roots have been more recently reconsidered by Jaynes, who extended the considerations advanced by Darwin and Fowler to information theory. This paper investigates how the concept of entropy was propagated in the past century in order to show how a simple intuition, born in the 1824 during the first industrial revolution in the mind of the young French engineer Carnot, is literally still enlightening the fourth industrial revolution and probably will continue to do so in the coming century. [ABSTRACT FROM AUTHOR]

Given two graphs G and H, the mapping of f : V (G) → V (H) is called a graph homomorphism from G to H if it maps the adjacent vertices of G to the adjacent vertices of H. For the graph G, a subset of vertices is called a dissociation set of G if it induces a subgraph of G containing no paths of order three, i.e., a subgraph of a maximum degree, which is at most one. Graph homomorphisms and dissociation sets are two generalizations of the concept of independent sets. In this paper, by utilizing an entropy approach, we provide upper bounds on the number of graph homomorphisms from the bipartite graph G to the graph H and the number of dissociation sets in a bipartite graph G. [ABSTRACT FROM AUTHOR]

As one of the entropy-based methods, the k-Star algorithm benefits from information theory in computing the distances between data instances during the classification task. k-Star is a machine learning method with a high classification performance and strong generalization ability. Nevertheless, as a standard supervised learning method, it performs learning only from labeled data. This paper proposes an improved method, called Semi-Supervised k-Star (SSS), which makes efficient predictions by considering unlabeled data in addition to labeled data. Moreover, it introduces a novel semi-supervised learning approach, called holo-training, against self-training. It has the advantage of enabling a powerful and robust model of data by combining multiple classifiers and using an entropy measure. The results of extensive experimental studies showed that the proposed holo-training approach outperformed the self-training approach on 13 out of the 18 datasets. Furthermore, the proposed SSS method achieved higher accuracy (95.25%) than the state-of-the-art semi-supervised methods (90.01%) on average. The significance of the experimental results was validated by using both the Binomial Sign test and the Friedman test. [ABSTRACT FROM AUTHOR]

Slope Entropy (SlpEn) is a very recently proposed entropy calculation method. It is based on the differences between consecutive values in a time series and two new input thresholds to assign a symbol to each resulting difference interval. As the histogram normalisation value, SlpEn uses the actual number of unique patterns found instead of the theoretically expected value. This maximises the information captured by the method but, as a consequence, SlpEn results do not usually fall within the classical [ 0 , 1 ] interval. Although this interval is not necessary at all for time series classification purposes, it is a convenient and common reference framework when entropy analyses take place. This paper describes a method to keep SlpEn results within this interval, and improves the interpretability and comparability of this measure in a similar way as for other methods. It is based on a max–min normalisation scheme, described in two steps. First, an analytic normalisation is proposed using known but very conservative bounds. Afterwards, these bounds are refined using heuristics about the behaviour of the number of patterns found in deterministic and random time series. The results confirm the suitability of the approach proposed, using a mixture of the two methods. [ABSTRACT FROM AUTHOR]

This paper focuses on the ultimate limit theory of image compression. It proves that for an image source, there exists a coding method with shapes that can achieve the entropy rate under a certain condition where the shape-pixel ratio in the encoder/decoder is O (1 / log t) . Based on the new finding, an image coding framework with shapes is proposed and proved to be asymptotically optimal for stationary and ergodic processes. Moreover, the condition O (1 / log t) of shape-pixel ratio in the encoder/decoder has been confirmed in the image database MNIST, which illustrates the soft compression with shape coding is a near-optimal scheme for lossless compression of images. [ABSTRACT FROM AUTHOR]

To derive a latent trait (for instance ability) in a computer adaptive testing (CAT) framework, the obtained results from a model must have a direct relationship to the examinees' response to a set of items presented. The set of items is previously calibrated to decide which item to present to the examinee in the next evaluation question. Some useful models are more naturally based on conditional probability in order to involve previously obtained hits/misses. In this paper, we integrate an experimental part, obtaining the information related to the examinee's academic performance, with a theoretical contribution of maximum entropy. Some academic performance index functions are built to support the experimental part and then explain under what conditions one can use constrained prior distributions. Additionally, we highlight that heuristic prior distributions might not properly work in all likely cases, and when to use personalized prior distributions instead. Finally, the inclusion of the performance index functions, arising from current experimental studies and historical records, are integrated into a theoretical part based on entropy maximization and its relationship with a CAT process. [ABSTRACT FROM AUTHOR]

This editorial explains the scope of the special issue and provides a thematic introduction to the contributed papers. [ABSTRACT FROM AUTHOR]

The (weighted) partial maximum satisfiability ((W)PMS) problem is an important generalization of the classic problem of propositional (Boolean) satisfiability with a wide range of real-world applications. In this paper, we propose an initialization and a diversification strategy to improve local search for the (W)PMS problem. Our initialization strategy is based on a novel definition of variables' structural entropy, and it aims to generate a solution that is close to a high-quality feasible one. Then, our diversification strategy picks a variable in two possible ways, depending on a parameter: continuing to pick variables with the best benefits or focusing on a clause with the greatest penalty and then selecting variables probabilistically. Based on these strategies, we developed a local search solver dubbed ImSATLike , as well as a hybrid solver ImSATLike-TT , and experimental results on (weighted) partial MaxSAT instances in recent MaxSAT Evaluations show that they outperform or have nearly the same performances as state-of-the-art local search and hybrid competitors, respectively, in general. Furthermore, we carried out experiments to confirm the individual impacts of each proposed strategy. [ABSTRACT FROM AUTHOR]

With the continuous construction of transportation infrastructure, intersection nodes have been increasing rapidly, bringing growing numbers of tunnel- and exit-adjacent sections (TEAS) in mountain expressways in China. With the complex variation in the surrounding environment, drivers always face congestion and confusion on tunnel and the exit connecting sections (TECS) without adequate length, meanwhile excessively long TECS create detours. To better provide a sustainable design strategy for TEAS, based on a certain section of expressway in Shaanxi, China, this paper establishes a theoretical calculation model through analysis. The characteristics of traffic flow and drivers' light adaptation at tunnel exit are obtained through data collection and driving tests, and the length requirements of the tunnel and exit connecting sections (TECS) are discussed. A VISSIM microscopic simulation model is also built under various design schemes and entropy-based multi-attribute decision making (EBMADM) is used to objectively calculate the weights of the four selected evaluation indexes. Then, the design schemes of the TECS with different lengths have been comprehensively evaluated. The results show the match between the evaluation results of EBMADM with theoretical calculations under existing traffic conditions, which proves the rationality of EBMADM in such problems. For more cases, the results of the EBMADM evaluation show a positive correlation between the length of TECS for the best performing design scheme with traffic volume and diverging ratio. [ABSTRACT FROM AUTHOR]

Several classical problems in symbolic dynamics concern the characterization of the simplex of measures of maximal entropy. For subshifts of finite type in higher dimensions, methods of statistical mechanics are ideal for dealing with these problems. R. Burton and J. Steif developed a strategy to construct examples of strongly irreducible subshifts of finite type admitting several measures of maximal entropy. This strategy exploits a correspondence between equilibrium statistical mechanics and symbolic dynamics—a correspondence which was later formalized by O. Häggström. In this paper, we revisit and discuss this correspondence with the aim of presenting a simplified version of it and present some applications of rigorous results concerning the Potts model and the six-vertex model to symbolic dynamics, illustrating in this way the possibilities of this correspondence. [ABSTRACT FROM AUTHOR]

Mathematical generalizations of the additive Boltzmann–Gibbs–Shannon entropy formula have been numerous since the 1960s. In this paper we seek an interpretation of the Rényi and Tsallis q-entropy formulas single parameter in terms of physical properties of a finite capacity heat-bath and fluctuations of temperature. Ideal gases of non-interacting particles are used as a demonstrating example. [ABSTRACT FROM AUTHOR]

Entanglement entropy (EE) is a quantitative measure of the effective degrees of freedom and the correlation between the sub-systems of a physical system. Using the replica trick, we can obtain the EE by evaluating the entanglement Renyi entropy (ERE). The ERE is a q-analogue of the EE and expressed by the q replicated partition function. In the semi-classical approximation, it is apparently easy to calculate the EE because the classical action represents the partition function by the saddle point approximation and we do not need to perform the path integral for the evaluation of the partition function. In previous studies, it has been assumed that only the minimal-valued saddle point contributes to the EE. In this paper, we propose that all the saddle points contribute comparably but not necessarily equally to the EE by dealing carefully with the semi-classical limit and then the q → 1 limit. For example, we numerically evaluate the ERE of two disjoint intervals for the large c Liouville field theory with q ∼ 1 . We exploit the BPZ equation with the four twist operators, whose solution is given by the Heun function. We determine the ERE by tuning the behavior of the Heun function such that it becomes consistent with the geometry of the replica manifold. We find the same two saddle points as previous studies for q ∼ 1 in the above system. Then, we provide the ERE for the large but finite c and the q ∼ 1 in case that all the saddle points contribute comparably to the ERE. In particular, the ERE is the summation of these two saddle points by the same weight, due to the symmetry of the system. Based on this work, it shall be of interest to reconsider EE in other semi-classical physical systems with multiple saddle points. [ABSTRACT FROM AUTHOR]

As a measure of complexity, information entropy is frequently used to categorize time series, such as machinery failure diagnostics, biological signal identification, etc., and is thought of as a characteristic of dynamic systems. Many entropies, however, are ineffective for multivariate scenarios due to correlations. In this paper, we propose a local structure entropy (LSE) based on the idea of a recurrence network. Given certain tolerance and scales, LSE values can distinguish multivariate chaotic sequences between stochastic signals. Three financial market indices are used to evaluate the proposed LSE. The results show that the L S E F S T E 100 and L S E S & P 500 are higher than L S E S Z I , which indicates that the European and American stock markets are more sophisticated than the Chinese stock market. Additionally, using decision trees as the classifiers, LSE is employed to detect bearing faults. LSE performs higher on recognition accuracy when compared to permutation entropy. [ABSTRACT FROM AUTHOR]

The dynamic of music is an important factor to arouse emotional experience, but current research mainly uses short-term artificial stimulus materials, which cannot effectively awaken complex emotions and reflect their dynamic brain response. In this paper, we used three long-term stimulus materials with many dynamic emotions inside: the "Waltz No. 2" containing pleasure and excitement, the "No. 14 Couplets" containing excitement, briskness, and nervousness, and the first movement of "Symphony No. 5 in C minor" containing passion, relaxation, cheerfulness, and nervousness. Approximate entropy (ApEn) and sample entropy (SampEn) were applied to extract the non-linear features of electroencephalogram (EEG) signals under long-term dynamic stimulation, and the K-Nearest Neighbor (KNN) method was used to recognize emotions. Further, a supervised feature vector dimensionality reduction method was proposed. Firstly, the optimal channel set for each subject was obtained by using a particle swarm optimization (PSO) algorithm, and then the number of times to select each channel in the optimal channel set of all subjects was counted. If the number was greater than or equal to the threshold, it was a common channel suitable for all subjects. The recognition results based on the optimal channel set demonstrated that each accuracy of two categories of emotions based on "Waltz No. 2" and three categories of emotions based on "No. 14 Couplets" was generally above 80%, respectively, and the recognition accuracy of four categories based on the first movement of "Symphony No. 5 in C minor" was about 70%. The recognition accuracy based on the common channel set was about 10% lower than that based on the optimal channel set, but not much different from that based on the whole channel set. This result suggested that the common channel could basically reflect the universal features of the whole subjects while realizing feature dimension reduction. The common channels were mainly distributed in the frontal lobe, central region, parietal lobe, occipital lobe, and temporal lobe. The channel number distributed in the frontal lobe was greater than the ones in other regions, indicating that the frontal lobe was the main emotional response region. Brain region topographic map based on the common channel set showed that there were differences in entropy intensity between different brain regions of the same emotion and the same brain region of different emotions. The number of times to select each channel in the optimal channel set of all 30 subjects showed that the principal component channels representing five brain regions were Fp1/F3 in the frontal lobe, CP5 in the central region, Pz in the parietal lobe, O2 in the occipital lobe, and T8 in the temporal lobe, respectively. [ABSTRACT FROM AUTHOR]

An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the "black-box" nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrated on reservoir interpretability, the perspective of reservoir modeling is relatively single, and the relationship between reservoir richness and reservoir projection capacity has not been effectively established. To tackle this problem, a novel reservoir interpretability framework based on permutation entropy (PE) theory is proposed in this paper. In structure, this framework consists of reservoir state extraction, PE modeling, and PE analysis. Based on these, the instantaneous reservoir states and neuronal time-varying states are extracted, which are followed by phase space reconstruction, sorting, and entropy calculation. Firstly, the obtained instantaneous state entropy (ISE) and global state entropy (GSE) can measure reservoir richness for interpreting good reservoir projection capacity. On the other hand, the multiscale complexity–entropy analysis of global and neuron-level reservoir states is performed to reveal more detailed dynamics. Finally, the relationships between ESN performance and reservoir dynamic are investigated via Pearson correlation, considering different prediction steps and time scales. Experimental evaluations on several benchmarks and real-world datasets demonstrate the effectiveness and superiority of the proposed reservoir interpretability framework. [ABSTRACT FROM AUTHOR]

Optimal transport is a mathematical tool that has been a widely used to measure the distance between two probability distributions. To mitigate the cubic computational complexity of the vanilla formulation of the optimal transport problem, regularized optimal transport has received attention in recent years, which is a convex program to minimize the linear transport cost with an added convex regularizer. Sinkhorn optimal transport is the most prominent one regularized with negative Shannon entropy, leading to densely supported solutions, which are often undesirable in light of the interpretability of transport plans. In this paper, we report that a deformed entropy designed by q-algebra, a popular generalization of the standard algebra studied in Tsallis statistical mechanics, makes optimal transport solutions supported sparsely. This entropy with a deformation parameter q interpolates the negative Shannon entropy ( q = 1 ) and the squared 2-norm ( q = 0 ), and the solution becomes more sparse as q tends to zero. Our theoretical analysis reveals that a larger q leads to a faster convergence when optimized with the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm. In summary, the deformation induces a trade-off between the sparsity and convergence speed. [ABSTRACT FROM AUTHOR]

In this paper, "Classical Model of Quons", by G. Kaniadakis and A. M Scarfone [[1]] by using a kinetic interaction principle an evolution equation describing quons statistics is proposed by properly generalizing the inclusion/exclusion principle of standard boson and fermion statistics. In the paper "A Note on the Entropy Force in Kinetic Theory and Black Holes" by R. A. Treumann and W. Baumjohann [[4]] is derived a kinetic equations of a large system of particles including a collective integral term to the Klimontovich equation for the evolution of the signle-particle distribution function. Here, the novelty is to endow great importance for the entropy balance equation in updating the state variables, thus the numerical compatibility with the second law of thermodynamics is ensured. [Extracted from the article]

Ransomware is a malicious class of software that utilises encryption to implement an attack on system availability. The target's data remains encrypted and is held captive by the attacker until a ransom demand is met. A common approach used by many crypto-ransomware detection techniques is to monitor file system activity and attempt to identify encrypted files being written to disk, often using a file's entropy as an indicator of encryption. However, often in the description of these techniques, little or no discussion is made as to why a particular entropy calculation technique is selected or any justification given as to why one technique is selected over the alternatives. The Shannon method of entropy calculation is the most commonly-used technique when it comes to file encryption identification in crypto-ransomware detection techniques. Overall, correctly encrypted data should be indistinguishable from random data, so apart from the standard mathematical entropy calculations such as Chi-Square ( χ 2 ), Shannon Entropy and Serial Correlation, the test suites used to validate the output from pseudo-random number generators would also be suited to perform this analysis. The hypothesis being that there is a fundamental difference between different entropy methods and that the best methods may be used to better detect ransomware encrypted files. The paper compares the accuracy of 53 distinct tests in being able to differentiate between encrypted data and other file types. The testing is broken down into two phases, the first phase is used to identify potential candidate tests, and a second phase where these candidates are thoroughly evaluated. To ensure that the tests were sufficiently robust, the NapierOne dataset is used. This dataset contains thousands of examples of the most commonly used file types, as well as examples of files that have been encrypted by crypto-ransomware. During the second phase of testing, 11 candidate entropy calculation techniques were tested against more than 270,000 individual files—resulting in nearly three million separate calculations. The overall accuracy of each of the individual test's ability to differentiate between files encrypted using crypto-ransomware and other file types is then evaluated and each test is compared using this metric in an attempt to identify the entropy method most suited for encrypted file identification. An investigation was also undertaken to determine if a hybrid approach, where the results of multiple tests are combined, to discover if an improvement in accuracy could be achieved. [ABSTRACT FROM AUTHOR]

Word processing is one of the most popular digital activities. Despite its popularity, it is haunted by false assumptions, misconceptions, and ineffective and inefficient practices leading to erroneous digital text-based documents. The focus of the present paper is automated numbering and distinguishing between manual and automated numbering. In general, one bit of information on the GUI—the position of the cursor—is enough to tell whether a numbering is manual or automated. To decide how much information must be put on the channel—the teaching–learning process—in order to reach end-users, we designed and implemented a method that includes the analysis of teaching, learning, tutorial, and testing sources, the collection and analysis of Word documents shared on the internet or in closed groups, the testing of grade 7–10 students' knowledge in automated numbering, and calculating the entropy of automated numbering. The combination of the test results and the semantics of the automated numbering was used to measure the entropy of automated numbering. It was found that to transfer one bit of information on the GUI, at least three bits of information must be transferred during the teaching–learning process. Furthermore, it was revealed that the information connected to numbering is not the pure use of tools, but the semantics of this feature put into a real-world context. [ABSTRACT FROM AUTHOR]

Crowd evacuation has gained increasing attention due to its importance in the day-to-day management of public areas. During an emergency evacuation, there are a variety of factors that need to be considered when designing a practical evacuation model. For example, relatives tend to move together or look for each other. These behaviors undoubtedly aggravate the chaos degree of evacuating crowds and make evacuations hard to model. In this paper, we propose an entropy-based combined behavior model to better analyze the influence of these behaviors on the evacuation process. Specifically, we utilize the Boltzmann entropy to quantitatively denote the degree of chaos in the crowd. The evacuation behavior of heterogeneous people is simulated through a series of behavior rules. Moreover, we devise a velocity adjustment method to ensure the evacuees follow a more orderly direction. Extensive simulation results demonstrate the effectiveness of the proposed evacuation model and provide useful insights into the design of practical evacuation strategies. [ABSTRACT FROM AUTHOR]

Many time series entropy calculation methods have been proposed in the last few years. They are mainly used as numerical features for signal classification in any scientific field where data series are involved. We recently proposed a new method, Slope Entropy (SlpEn), based on the relative frequency of differences between consecutive samples of a time series, thresholded using two input parameters, γ and δ. In principle, δ was proposed to account for differences in the vicinity of the 0 region (namely, ties) and, therefore, was usually set at small values such as 0.001. However, there is no study that really quantifies the role of this parameter using this default or other configurations, despite the good SlpEn results so far. The present paper addresses this issue, removing δ from the SlpEn calculation to assess its real influence on classification performance, or optimising its value by means of a grid search in order to find out if other values beyond the 0.001 value provide significant time series classification accuracy gains. Although the inclusion of this parameter does improve classification accuracy according to experimental results, gains of 5 % at most probably do not support the additional effort required. Therefore, SlpEn simplification could be seen as a real alternative. [ABSTRACT FROM AUTHOR]

The uncertainty of information is an important issue that must be faced when dealing with decision-making problems. Randomness and fuzziness are the two most common types of uncertainty. In this paper, we propose a multicriteria group decision-making method based on intuitionistic normal cloud and cloud distance entropy. First, the backward cloud generation algorithm for intuitionistic normal clouds is designed to transform the intuitionistic fuzzy decision information given by all experts into an intuitionistic normal cloud matrix to avoid the loss and distortion of information. Second, the distance measurement of the cloud model is introduced into the information entropy theory, and the concept of cloud distance entropy is proposed. Then, the distance measurement for intuitionistic normal clouds based on numerical features is defined and its properties are discussed, based on which the criterion weight determination method under intuitionistic normal cloud information is proposed. In addition, the VIKOR method, which integrates group utility and individual regret, is extended to the intuitionistic normal cloud environment, and thus the ranking results of the alternatives are obtained. Finally, the effectiveness and practicality of the proposed method are demonstrated by two numerical examples. [ABSTRACT FROM AUTHOR]

Considering both biological and non-biological polygonal shape organizations, in this paper we introduce a quantitative method which is able to determine informational entropy as spatial differences between heterogeneity of internal areas from simulation and experimental samples. According to these data (i.e., heterogeneity), we are able to establish levels of informational entropy using statistical insights of spatial orders using discrete and continuous values. Given a particular state of entropy, we establish levels of information as a novel approach which can unveil general principles of biological organization. Thirty-five geometric aggregates are tested (biological, non-biological, and polygonal simulations) in order to obtain the theoretical and experimental results of their spatial heterogeneity. Geometrical aggregates (meshes) include a spectrum of organizations ranging from cell meshes to ecological patterns. Experimental results for discrete entropy using a bin width of 0.5 show that a particular range of informational entropy (0.08 to 0.27 bits) is intrinsically associated with low rates of heterogeneity, which indicates a high degree of uncertainty in finding non-homogeneous configurations. In contrast, differential entropy (continuous) results reflect negative entropy within a particular range (−0.4 to −0.9) for all bin widths. We conclude that the differential entropy of geometrical organizations is an important source of neglected information in biological systems. [ABSTRACT FROM AUTHOR]

In this paper we recall, extend and compute some information measures for the concomitants of the generalized order statistics (GOS) from the Farlie–Gumbel–Morgenstern (FGM) family. We focus on two types of information measures: some related to Shannon entropy, and some related to Tsallis entropy. Among the information measures considered are residual and past entropies which are important in a reliability context. [ABSTRACT FROM AUTHOR]

In this paper, we explore a new approach in which understanding is interpreted as a set representation. We prove that understanding/representation, finding the appropriate coordination of data, is equivalent to finding the minimum of the representational entropy. For the control of the search for the correct representation, we propose a loss function as a combination of the representational entropy, type one and type two errors. Computational complexity estimates are presented for the process of understanding and using the representation found. [ABSTRACT FROM AUTHOR]

Photovoltaic (PV) system diagnosis is a growing research domain likewise solar energy's ongoing significant expansion. Indeed, efficient Fault Detection and Diagnosis (FDD) tools are crucial to guarantee reliability, avoid premature aging and improve the profitability of PV plants. In this paper, an on-line diagnosis method using the PV plant electrical output is presented. This entirely signal-based method combines variational mode decomposition (VMD) and multiscale dispersion entropy (MDE) for the purpose of detecting and isolating faults in a real grid-connected PV plant. The present method seeks a low-cost design, an ease of implementation and a low computation cost. Taking into account the innovation of applying these techniques to PV FDD, the VMD and MDE procedures as well as parameters identification are carefully detailed. The proposed FFD approach performance is assessed on a real rooftop PV plant with experimentally induced faults, and the first results reveal the MDE approach has good suitability for PV plants diagnosis. [ABSTRACT FROM AUTHOR]

Emotion recognition based on electroencephalography (EEG) has attracted high interest in fields such as health care, user experience evaluation, and human–computer interaction (HCI), as it plays an important role in human daily life. Although various approaches have been proposed to detect emotion states in previous studies, there is still a need to further study the dynamic changes of EEG in different emotions to detect emotion states accurately. Entropy-based features have been proved to be effective in mining the complexity information in EEG in many areas. However, different entropy features vary in revealing the implicit information of EEG. To improve system reliability, in this paper, we propose a framework for EEG-based cross-subject emotion recognition using fused entropy features and a Bidirectional Long Short-term Memory (BiLSTM) network. Features including approximate entropy (AE), fuzzy entropy (FE), Rényi entropy (RE), differential entropy (DE), and multi-scale entropy (MSE) are first calculated to study dynamic emotional information. Then, we train a BiLSTM classifier with the inputs of entropy features to identify different emotions. Our results show that MSE of EEG is more efficient than other single-entropy features in recognizing emotions. The performance of BiLSTM is further improved with an accuracy of 70.05% using fused entropy features compared with that of single-type feature. [ABSTRACT FROM AUTHOR]

An alternate measure of uncertainty, termed the fractional generalized cumulative residual entropy, has been introduced in the literature. In this paper, we first investigate some variability properties this measure has and then establish its connection to other dispersion measures. Moreover, we prove under sufficient conditions that this measure preserves the location-independent riskier order. We then elaborate on the fractional survival functional entropy of coherent and mixed systems' lifetime in the case that the component lifetimes are dependent and they have identical distributions. Finally, we give some bounds and illustrate the usefulness of the given bounds. [ABSTRACT FROM AUTHOR]

Properties of data distributions can be assessed at both global and local scales. At a highly localized scale, a fundamental measure is the local intrinsic dimensionality (LID), which assesses growth rates of the cumulative distribution function within a restricted neighborhood and characterizes properties of the geometry of a local neighborhood. In this paper, we explore the connection of LID to other well known measures for complexity assessment and comparison, namely, entropy and statistical distances or divergences. In an asymptotic context, we develop analytical new expressions for these quantities in terms of LID. This reveals the fundamental nature of LID as a building block for characterizing and comparing data distributions, opening the door to new methods for distributional analysis at a local scale. [ABSTRACT FROM AUTHOR]

The energy loss of the vertical axial flow pump device increases due to the unstable internal flow, which reduces the efficiency of the pump device and increases its energy consumption of the pump device. The research results of the flow loss characteristics of the total internal conduit are still unclear. Therefore, to show the internal energy loss mechanism of the axial flow pump, this paper used the entropy production method to calculate the energy loss of the total conduit of the pump device to clarify the internal energy loss mechanism of the pump device. The results show that the energy loss of the impeller is the largest under various flow conditions, accounting for more than 40% of the total energy loss of the pump device. The variation trend of the volume average entropy production and the energy loss is similar under various flow coefficients (KQ). The volume average entropy production rate (EPR) and the energy loss decrease first and then increase with the increase of flow, the minimum volume average entropy production is 378,000 W/m3 at KQ = 0.52, and the area average EPR of the impeller increases gradually with the increase of flow. Under various flow coefficient KQ, the energy loss of campaniform inlet conduit is the smallest, accounting for less than 1% of the total energy loss. Its maximum value is 63.58 W. The energy loss of the guide vane and elbow increases with the increase of flow coefficient KQ, and the maximum ratio of energy loss to the total energy loss of the pump device is 29% and 21%, respectively, at small flow condition KQ = 0.38. The energy loss of straight outlet conduit reduces first and then increases with the increase of flow coefficient KQ. When flow coefficient KQ = 0.62, it accounts for 27% of the total energy loss of the pump device, but its area average entropy production rate (EPR) and volume average entropy production rate (EPR) are small. The main entropy production loss in the pump device is dominated by entropy production by turbulent dissipation (EPTD), and the proportion of entropy production by direct dissipation (EPDD) is the smallest. [ABSTRACT FROM AUTHOR]

In this paper, we present the concept of the logical entropy of order m, logical mutual information, and the logical entropy for information sources. We found upper and lower bounds for the logical entropy of a random variable by using convex functions. We show that the logical entropy of the joint distributions X 1 and X 2 is always less than the sum of the logical entropy of the variables X 1 and X 2 . We define the logical Shannon entropy and logical metric permutation entropy to an information system and examine the properties of this kind of entropy. Finally, we examine the amount of the logical metric entropy and permutation logical entropy for maps. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of frequency stability prediction (FSP) following active power disturbances in power systems by proposing a vision transformer (ViT) method that predicts frequency stability in real time. The core idea of the FSP approach employing the ViT is to use the time-series data of power system operations as ViT inputs to perform FSP accurately and quickly so that operators can decide frequency control actions, minimizing the losses caused by incidents. Additionally, due to the high-dimensional and redundant input data of the power system and the O(N2) computational complexity of the transformer, feature selection based on copula entropy (CE) is used to construct image-like data with fixed dimensions from power system operation data and remove redundant information. Moreover, no previous FSP study has taken safety margins into consideration, which may threaten the secure operation of power systems. Therefore, a frequency security index (FSI) is used to form the sample labels, which are categorized as "insecurity", "relative security", and "absolute security". Finally, various case studies are carried out on a modified New England 39-bus system and a modified ACTIVSg500 system for projected 0% to 40% nonsynchronous system penetration levels. The simulation results demonstrate that the proposed method achieves state-of-the-art (SOTA) performance on normal, noisy, and incomplete datasets in comparison with eight machine-learning methods. [ABSTRACT FROM AUTHOR]

Multi-source information fusion is widely used because of its similarity to practical engineering situations. With the development of science and technology, the sources of information collected under engineering projects and scientific research are more diverse. To extract helpful information from multi-source information, in this paper, we propose a multi-source information fusion method based on the Dempster-Shafer (DS) evidence theory with the negation of reconstructed basic probability assignments (nrBPA). To determine the initial basic probability assignment (BPA), the Gaussian distribution BPA functions with padding terms are used. After that, nrBPAs are determined by two processes, reassigning the high blur degree BPA and transforming them into the form of negation. In addition, evidence of preliminary fusion is obtained using the entropy weight method based on the improved belief entropy of nrBPAs. The final fusion results are calculated from the preliminary fused evidence through the Dempster's combination rule. In the experimental section, the UCI iris data set and the wine data set are used for validating the arithmetic processes of the proposed method. In the comparative analysis, the effectiveness of the BPA determination using a padded Gaussian function is verified by discussing the classification task with the iris data set. Subsequently, the comparison with other methods using the cross-validation method proves that the proposed method is robust. Notably, the classification accuracy of the iris data set using the proposed method can reach an accuracy of 97.04%, which is higher than many other methods. [ABSTRACT FROM AUTHOR]

Many similarity measure algorithms of nodes in weighted graph data have been proposed by employing the degree of nodes in recent years. Despite these algorithms obtaining great results, there may be still some limitations. For instance, the strength of nodes is ignored. Aiming at this issue, the relative entropy of the distance distribution based similarity measure of nodes is proposed in this paper. At first, the structural weights of nodes are given by integrating their degree and strength. Next, the distance between any two nodes is calculated with the help of their structural weights and the Euclidean distance formula to further obtain the distance distribution of each node. After that, the probability distribution of nodes is constructed by normalizing their distance distributions. Thus, the relative entropy can be applied to measure the difference between the probability distributions of the top d important nodes and all nodes in graph data. Finally, the similarity of two nodes can be measured in terms of this above-mentioned difference calculated by relative entropy. Experimental results demonstrate that the algorithm proposed by considering the strength of node in the relative entropy has great advantages in the most similar node mining and link prediction. [ABSTRACT FROM AUTHOR]

In the fault monitoring of rolling bearings, there is always loud noise, leading to poor signal stationariness. How to accurately and efficiently identify the fault type of rolling bearings is a challenge. Based on multivariate multiscale sample entropy (mvMSE), this paper introduces the refined composite mvMSE (RCmvMSE) into the fault extraction of the rolling bearing. A rolling bearing fault-diagnosis method based on stacked auto encoder and RCmvMSE (SDAE-RCmvMSE) is proposed. In the actual environment, the fault-diagnosis method use the multichannel vibration signals of the bearing as the input of stacked denoising autoencoders (SDAEs) to filter the noise of the vibration signals. The features of denoise signals are extracted by RCmvMSE and the rolling bearing operation-state diagnosis is completed with a support-vector machine (SVM) model. The results show that in the original test data, the accuracy rates of SDAE-RCmvMSE, RCmvMSE, and commonplace features of vibration signals combined with SVM (CFVS-SVM) methods are 99.5%, 100%, and 96% respectively. In the data with noise, the accuracy rates of RCmvMSE and CFVS-SVM are 97.75% and 93.08%, respectively, but the accuracy of SDAE-RCmvMSE is still 100%. [ABSTRACT FROM AUTHOR]

Music has become a common adjunctive treatment for Alzheimer's disease (AD) in recent years. Because Alzheimer's disease can be classified into different degrees of dementia according to its severity (mild, moderate, severe), this study is to investigate whether there are differences in brain response to music stimulation in AD patients with different degrees of dementia. Seventeen patients with mild-to-moderate dementia, sixteen patients with severe dementia, and sixteen healthy elderly participants were selected as experimental subjects. The nonlinear characteristics of electroencephalogram (EEG) signals were extracted from 64-channel EEG signals acquired before, during, and after music stimulation. The results showed the following. (1) At the temporal level, both at the whole brain area and sub-brain area levels, the EEG responses of the mild-to-moderate patients showed statistical differences from those of the severe patients (p < 0.05). The nonlinear characteristics during music stimulus, including permutation entropy (PmEn), sample entropy (SampEn), and Lempel–Ziv complexity (LZC), were significantly higher in both mild-to-moderate patients and healthy controls compared to pre-stimulation, while it was significantly lower in severe patients. (2) At the spatial level, the EEG responses of the mild-to-moderate patients and the severe patients showed statistical differences (p < 0.05), showing that as the degree of dementia progressed, fewer pairs of EEG characteristic showed significant differences among brain regions under music stimulation. In this paper, we found that AD patients with different degrees of dementia had different EEG responses to music stimulation. Our study provides a possible explanation for this discrepancy in terms of the pathological progression of AD and music cognitive hierarchy theory. Our study has adjunctive implications for clinical music therapy in AD., potentially allowing for more targeted treatment. Meanwhile, the variations in the brains of Alzheimer's patients in response to music stimulation might be a model for investigating the neural mechanism of music perception. [ABSTRACT FROM AUTHOR]

This paper provides a review of different contributions dedicated thus far to entropy generation analysis (EGA) in turbulent combustion systems. We account for various parametric studies that include wall boundedness, flow operating conditions, combustion regimes, fuels/alternative fuels and application geometries. Special attention is paid to experimental and numerical modeling works along with selected applications. First, the difficulties of performing comprehensive experiments that may support the understanding of entropy generation phenomena are outlined. Together with practical applications, the lumped approach to calculate the total entropy generation rate is presented. Apart from direct numerical simulation, numerical modeling approaches are described within the continuum formulation in the framework of non-equilibrium thermodynamics. Considering the entropy transport equations in both Reynolds-averaged Navier–Stokes and large eddy simulation modeling, different modeling degrees of the entropy production terms are presented and discussed. Finally, exemplary investigations and validation cases going from generic or/and canonical configurations to practical configurations, such as internal combustion engines, gas turbines and power plants, are reported. Thereby, the areas for future research in the development of EGA for enabling efficient combustion systems are highlighted. Since EGA is known as a promising tool for optimization of combustion systems, this aspect is highlighted in this work. [ABSTRACT FROM AUTHOR]

In this paper, a novel Double Intuitionistic Fuzzy Synthetic Measure (DIFSM), based on intuitionistic fuzzy values for handling multi-criteria decision-making problems used to rank alternatives, is presented. In the studies, intuitionistic fuzzy sets (IFSs) represented uncertain, imprecise information or human judgment. The intuitionistic fuzzy sets can also reflect the approval, rejection, and hesitation of decision-makers. The degrees of satisfiability and non-satisfiability and uncertainty of each alternative with respect to a set of criteria are described by membership functions, non-membership functions, and hesitancy indexes, respectively. The aggregation algorithm DIFSM is inspired by Hellwig's method based on two reference points: ideal point (pattern) and anti-ideal point (anti-pattern), measuring distances between the alternative and ideal point and distance between the ideal and anti-ideal point. The proposed methods take into consideration the entropy-based weights of criteria. An illustrative example is given to demonstrate the practicality and effectiveness of the proposed approach. Additionally, the comparative analysis results, using the DIFSM and the Intuitionistic Fuzzy TOPSIS-based framework, are presented. [ABSTRACT FROM AUTHOR]

The input–output (IO) network is the quantitative description of an IO-based economy in which nodes represent industries and edges connecting nodes represent the economic connection between industries. Robustness refers to the ability of tolerating perturbations that might affect the system's functional body. There is both practical and theoretical significance to explore the robustness of the IO network for economic development. In this paper, we probe the robustness of the Chinese IO network based on the relative entropy of the probability distribution of network parameters (node degree, strongest path betweenness, downstream closeness and upstream closeness) under random node or edge failure and intentional node or edge attack. It is found that the Chinese IO network shows relatively weak robustness when it is under intentional attack, but relatively strong robustness when it is under random failure. Our experiment also verifies the applicability and effectiveness of the relative entropy model in measuring the robustness of the IO network. [ABSTRACT FROM AUTHOR]

The fractional generalized cumulative residual entropy (FGCRE) has been introduced recently as a novel uncertainty measure which can be compared with the fractional Shannon entropy. Various properties of the FGCRE have been studied in the literature. In this paper, further results for this measure are obtained. The results include new representations of the FGCRE and a derivation of some bounds for it. We conduct a number of stochastic comparisons using this measure and detect the connections it has with some well-known stochastic orders and other reliability measures. We also show that the FGCRE is the Bayesian risk of a mean residual lifetime (MRL) under a suitable prior distribution function. A normalized version of the FGCRE is considered and its properties and connections with the Lorenz curve ordering are studied. The dynamic version of the measure is considered in the context of the residual lifetime and appropriate aging paths. [ABSTRACT FROM AUTHOR]

In this paper, the fractional cumulative entropy is considered to get its further properties and also its developments to dynamic cases. The measure is used to characterize a family of symmetric distributions and also another location family of distributions. The links between the fractional cumulative entropy and the classical differential entropy and some reliability quantities are also unveiled. In addition, the connection the measure has with the standard deviation is also found. We provide some examples to establish the variability property of this measure. [ABSTRACT FROM AUTHOR]

The complex nature of the interlacement of economic actors is quite evident at the level of the Stock market, where any company may actually interact with the other companies buying and selling their shares. In this respect, the companies populating a Stock market, along with their connections, can be effectively modeled through a directed network, where the nodes represent the companies, and the links indicate the ownership. This paper deals with this theme and discusses the concentration of a market. A cross-shareholding matrix is considered, along with two key factors: the node out-degree distribution which represents the diversification of investments in terms of the number of involved companies, and the node in-degree distribution which reports the integration of a company due to the sales of its own shares to other companies. While diversification is widely explored in the literature, integration is most present in literature on contagions. This paper captures such quantities of interest in the two frameworks and studies the stochastic dependence of diversification and integration through a copula approach. We adopt entropies as measures for assessing the concentration in the market. The main question is to assess the dependence structure leading to a better description of the data or to market polarization (minimal entropy) or market fairness (maximal entropy). In so doing, we derive information on the way in which the in- and out-degrees should be connected in order to shape the market. The question is of interest to regulators bodies, as witnessed by specific alert threshold published on the US mergers guidelines for limiting the possibility of acquisitions and the prevalence of a single company on the market. Indeed, all countries and the EU have also rules or guidelines in order to limit concentrations, in a country or across borders, respectively. The calibration of copulas and model parameters on the basis of real data serves as an illustrative application of the theoretical proposal. [ABSTRACT FROM AUTHOR]