Search

This paper presents a novel method for finding features in the analysis of variable distributions stemming from time series. We apply the methodology to the case of submitted and accepted papers in peer-reviewed journals. We provide a comparative study of editorial decisions for papers submitted to two peer-reviewed journals: the Journal of the Serbian Chemical Society (JSCS) and this MDPI Entropy journal. We cover three recent years for which the fate of submitted papers—about 600 papers to JSCS and 2500 to Entropy—is completely determined. Instead of comparing the number distributions of these papers as a function of time with respect to a uniform distribution, we analyze the relevant probabilities, from which we derive the information entropy. It is argued that such probabilities are indeed more relevant for authors than the actual number of submissions. We tie this entropy analysis to the so called diversity of the variable distributions. Furthermore, we emphasize the correspondence between the entropy and the diversity with inequality measures, like the Herfindahl-Hirschman index and the Theil index, itself being in the class of entropy measures; the Gini coefficient which also measures the diversity in ranking is calculated for further discussion. In this sample, the seasonal aspects of the peer review process are outlined. It is found that the use of such indices, non linear transformations of the data distributions, allow us to distinguish features and evolutions of the peer review process as a function of time as well as comparing the non-uniformity of distributions. Furthermore, t- and z-statistical tests are applied in order to measure the significance (p-level) of the findings, that is, whether papers are more likely to be accepted if they are submitted during a few specific months or during a particular "season"; the predictability strength depends on the journal. [ABSTRACT FROM AUTHOR]

Slope Entropy (SlpEn) is a novel method recently proposed in the field of time series entropy estimation. In addition to the well-known embedded dimension parameter, m, used in other methods, it applies two additional thresholds, denoted as δ and γ , to derive a symbolic representation of a data subsequence. The original paper introducing SlpEn provided some guidelines for recommended specific values of these two parameters, which have been successfully followed in subsequent studies. However, a deeper understanding of the role of these thresholds is necessary to explore the potential for further SlpEn optimisations. Some works have already addressed the role of δ , but in this paper, we extend this investigation to include the role of γ and explore the impact of using an asymmetric scheme to select threshold values. We conduct a comparative analysis between the standard SlpEn method as initially proposed and an optimised version obtained through a grid search to maximise signal classification performance based on SlpEn. The results confirm that the optimised version achieves higher time series classification accuracy, albeit at the cost of significantly increased computational complexity. [ABSTRACT FROM AUTHOR]

B Transductive Feature Selection Using Clustering-Based Sample Entropy for Temperature Prediction in Weather Forecasting b In addition, by taking into account the local structure of the data, we proposed a clustering-based sample entropy methodology. In this approach, depending on the clustering information of the training data and the membership values of the test point to the clusters, the samples have different impact on the sample entropy. [Extracted from the article]

The Special Issue of Entropy titled "Recent Advances in Statistical Theory and Applications" explores various challenges and methodologies related to the analysis of complex data. The issue covers topics such as clustering, change point inference, multiple sample tests, generalized linear models, machine learning, stochastic processes, and time series data. The articles propose innovative approaches to address these challenges and provide valuable insights for researchers in these fields. The collection aims to offer beneficial information and ideas to readers conducting their own research. [Extracted from the article]

Marine background noise (MBN) is the background noise of the marine environment, which can be used to invert the parameters of the marine environment. However, due to the complexity of the marine environment, it is difficult to extract the features of the MBN. In this paper, we study the feature extraction method of MBN based on nonlinear dynamics features, where the nonlinear dynamical features include two main categories: entropy and Lempel–Ziv complexity (LZC). We have performed single feature and multiple feature comparative experiments on feature extraction based on entropy and LZC, respectively: for entropy-based feature extraction experiments, we compared feature extraction methods based on dispersion entropy (DE), permutation entropy (PE), fuzzy entropy (FE), and sample entropy (SE); for LZC-based feature extraction experiments, we compared feature extraction methods based on LZC, dispersion LZC (DLZC) and permutation LZC (PLZC), and dispersion entropy-based LZC (DELZC). The simulation experiments prove that all kinds of nonlinear dynamics features can effectively detect the change of time series complexity, and the actual experimental results show that regardless of the entropy-based feature extraction method or LZC-based feature extraction method, they both present better feature extraction performance for MBN. [ABSTRACT FROM AUTHOR]

Slope Entropy (SlpEn) is a very recently proposed entropy calculation method. It is based on the differences between consecutive values in a time series and two new input thresholds to assign a symbol to each resulting difference interval. As the histogram normalisation value, SlpEn uses the actual number of unique patterns found instead of the theoretically expected value. This maximises the information captured by the method but, as a consequence, SlpEn results do not usually fall within the classical [ 0 , 1 ] interval. Although this interval is not necessary at all for time series classification purposes, it is a convenient and common reference framework when entropy analyses take place. This paper describes a method to keep SlpEn results within this interval, and improves the interpretability and comparability of this measure in a similar way as for other methods. It is based on a max–min normalisation scheme, described in two steps. First, an analytic normalisation is proposed using known but very conservative bounds. Afterwards, these bounds are refined using heuristics about the behaviour of the number of patterns found in deterministic and random time series. The results confirm the suitability of the approach proposed, using a mixture of the two methods. [ABSTRACT FROM AUTHOR]

As a measure of complexity, information entropy is frequently used to categorize time series, such as machinery failure diagnostics, biological signal identification, etc., and is thought of as a characteristic of dynamic systems. Many entropies, however, are ineffective for multivariate scenarios due to correlations. In this paper, we propose a local structure entropy (LSE) based on the idea of a recurrence network. Given certain tolerance and scales, LSE values can distinguish multivariate chaotic sequences between stochastic signals. Three financial market indices are used to evaluate the proposed LSE. The results show that the L S E F S T E 100 and L S E S & P 500 are higher than L S E S Z I , which indicates that the European and American stock markets are more sophisticated than the Chinese stock market. Additionally, using decision trees as the classifiers, LSE is employed to detect bearing faults. LSE performs higher on recognition accuracy when compared to permutation entropy. [ABSTRACT FROM AUTHOR]

An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the "black-box" nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrated on reservoir interpretability, the perspective of reservoir modeling is relatively single, and the relationship between reservoir richness and reservoir projection capacity has not been effectively established. To tackle this problem, a novel reservoir interpretability framework based on permutation entropy (PE) theory is proposed in this paper. In structure, this framework consists of reservoir state extraction, PE modeling, and PE analysis. Based on these, the instantaneous reservoir states and neuronal time-varying states are extracted, which are followed by phase space reconstruction, sorting, and entropy calculation. Firstly, the obtained instantaneous state entropy (ISE) and global state entropy (GSE) can measure reservoir richness for interpreting good reservoir projection capacity. On the other hand, the multiscale complexity–entropy analysis of global and neuron-level reservoir states is performed to reveal more detailed dynamics. Finally, the relationships between ESN performance and reservoir dynamic are investigated via Pearson correlation, considering different prediction steps and time scales. Experimental evaluations on several benchmarks and real-world datasets demonstrate the effectiveness and superiority of the proposed reservoir interpretability framework. [ABSTRACT FROM AUTHOR]

In the last decade permutation entropy (PE) has become a popular tool to analyze the degree of randomness within a time series. In typical applications, changes in the dynamics of a source are inferred by observing changes of PE computed on different time series generated by that source. However, most works neglect the crucial question related to the statistical significance of these changes. The main reason probably lies in the difficulty of assessing, out of a single time series, not only the PE value, but also its uncertainty. In this paper we propose a method to overcome this issue by using generation of surrogate time series. The analysis conducted on both synthetic and experimental time series shows the reliability of the approach, which can be promptly implemented by means of widely available numerical tools. The method is computationally affordable for a broad range of users. [ABSTRACT FROM AUTHOR]

Sample Entropy is the most popular definition of entropy and is widely used as a measure of the regularity/complexity of a time series. On the other hand, it is a computationally expensive method which may require a large amount of time when used in long series or with a large number of signals. The computationally intensive part is the similarity check between points in m dimensional space. In this paper, we propose new algorithms or extend already proposed ones, aiming to compute Sample Entropy quickly. All algorithms return exactly the same value for Sample Entropy, and no approximation techniques are used. We compare and evaluate them using cardiac inter-beat (RR) time series. We investigate three algorithms. The first one is an extension of the kd-trees algorithm, customized for Sample Entropy. The second one is an extension of an algorithm initially proposed for Approximate Entropy, again customized for Sample Entropy, but also improved to present even faster results. The last one is a completely new algorithm, presenting the fastest execution times for specific values of m, r, time series length, and signal characteristics. These algorithms are compared with the straightforward implementation, directly resulting from the definition of Sample Entropy, in order to give a clear image of the speedups achieved. All algorithms assume the classical approach to the metric, in which the maximum norm is used. The key idea of the two last suggested algorithms is to avoid unnecessary comparisons by detecting them early. We use the term unnecessary to refer to those comparisons for which we know a priori that they will fail at the similarity check. The number of avoided comparisons is proved to be very large, resulting in an analogous large reduction of execution time, making them the fastest algorithms available today for the computation of Sample Entropy. [ABSTRACT FROM AUTHOR]

In this paper, the influence of four key issues on wavelet-based analysis of hydrologic series' complexity under multi-temporal scales, including the choice of mother wavelet, noise, estimation of probability density function and trend of series data, was first studied. Then, the complexities of several representative hydrologic series data were quantified and described, based on which the performances of four wavelet-based entropy measures used commonly, namely continuous wavelet entropy (CWE), continuous wavelet relative entropy (CWRE), discrete wavelet entropy (DWE) and discrete wavelet relative entropy (DWRE) respectively, were compared and discussed. Finally, according to the analytic results of various examples, some understanding and conclusions about the calculation of wavelet-based entropy values gained in this study have been summarized, and the corresponding suggestions have also been proposed, based on which the analytic results of complexity of hydrologic series data can be improved. [ABSTRACT FROM AUTHOR]

By exploiting the statistical analysis method, human dynamics provides new insights to the research of human behavior. In this paper, we analyze the characteristics of the computer operating behavior through a modified multiscale entropy algorithm with both the interval time series and the number series of individuals' operating behavior been investigated. We also discuss the activity of individuals' behavior from the three groups denoted as the retiree group, the student group and the worker group based on the nature of their jobs. We find that the operating behavior of the retiree group exhibits more complex dynamics than the other two groups and further present a reasonable explanation for this phenomenon. Our findings offer new insights for the further understanding of individual behavior at different time scales. [ABSTRACT FROM AUTHOR]

In addition to their importance in statistical thermodynamics, probabilistic entropy measurements are crucial for understanding and analyzing complex systems, with diverse applications in time series and one-dimensional profiles. However, extending these methods to two- and three-dimensional data still requires further development. In this study, we present a new method for classifying spatiotemporal processes based on entropy measurements. To test and validate the method, we selected five classes of similar processes related to the evolution of random patterns: (i) white noise; (ii) red noise; (iii) weak turbulence from reaction to diffusion; (iv) hydrodynamic fully developed turbulence; and (v) plasma turbulence from MHD. Considering seven possible ways to measure entropy from a matrix, we present the method as a parameter space composed of the two best separating measures of the five selected classes. The results highlight better combined performance of Shannon permutation entropy ( S H p ) and a new approach based on Tsallis Spectral Permutation Entropy ( S q s ). Notably, our observations reveal the segregation of reaction terms in this S H p × S q s space, a result that identifies specific sectors for each class of dynamic process, and it can be used to train machine learning models for the automatic classification of complex spatiotemporal patterns. [ABSTRACT FROM AUTHOR]

Several measures of volatility have been developed in order to quantify the degree of uncertainty of an energy price series, which include historical volatility and price velocities, among others. This paper suggests using the permutation entropy, topological entropy and the modified permutation entropy as alternatives to measure volatility in energy markets. Simulated data show that these measures are more appropriate to quantify the uncertainty associated to a time series than those based on the standard deviation or other measures of dispersion. Finally, the proposed method is applied to some typical electricity markets: Nord Pool, Ontario, Omel and four Australian markets. [ABSTRACT FROM AUTHOR]

The classical information-theoretic measures such as the entropy and the mutual information (MI) are widely applicable to many areas in science and engineering. Csiszar generalized the entropy and the MI by using the convex functions. Recently, we proposed the grid occupancy (GO) and the quasientropy (QE) as measures of independence. The QE explicitly includes a convex function in its definition, while the expectation of GO is a subclass of QE. In this paper, we study the effect of different convex functions on GO, QE, and Csiszar's generalized mutual information (GMI). A quality factor (QF) is proposed to quantify the sharpness of their minima. Using the QF, it is shown that these measures can have sharper minima than the classical MI. Besides, a recursive algorithm for computing GMI, which is a generalization of Fraser and Swinney's algorithm for computing MI, is proposed. Moreover, we apply GO, QE, and GMI to chaotic time series analysis. It is shown that these measures are good criteria for determining the optimum delay in strange attractor reconstruction. [ABSTRACT FROM AUTHOR]

In this paper, the complexities of various noises, which are quantified by wavelet energy entropy (WEE) and differential coefficient of WEE (D(WEE)), were first analyzed and their uncertainties then estimated and described using confidence intervals. Then, quantitative criterion for judging the WEE and D(WEE) difference between noisy series and noise was put forward, based on which the decomposition level (DL) choice method in wavelet threshold de-noising proposed in 2010 by Sang et al. was improved. Finally, analytical results from examples verified the performance of the improved method, and also demonstrated its much wider applicability; moreover, the DL chosen using it is more reliable because of the fact that uncertainty is taken into consideration. [ABSTRACT FROM AUTHOR]

On the capital market, price movements of stock corporations can be observed independent of overall market developments as a result of company-specific news, which suggests the occurrence of a sudden risk event. In recent years, numerous concepts from statistical physics have been transferred to econometrics to model these effects and other issues, e.g., in socioeconomics. Like other studies, we extend the approaches based on the "buy" and "sell" positions of agents (investors' stance) with a third "hold" position. We develop the corresponding theory within the framework of the microcanonical and canonical ensembles for an ideal agent system and apply it to a capital market example. We thereby design a procedure to estimate the required model parameters from time series on the capital market. The aim is the appropriate modeling and the one-step-ahead assessment of the effect of a sudden risk event. From a one-step-ahead performance comparison with selected benchmark approaches, we infer that the model is well-specified and the model parameters are well determined. [ABSTRACT FROM AUTHOR]

Rotary machines often exhibit nonlinear behavior due to factors such as nonlinear stiffness, damping, friction, coupling effects, and defects. Consequently, their vibration signals display nonlinear characteristics. Entropy techniques prove to be effective in detecting these nonlinear dynamic characteristics. Recently, an approach called fuzzy dispersion entropy (DE–FDE) was introduced to quantify the uncertainty of time series. FDE, rooted in dispersion patterns and fuzzy set theory, addresses the sensitivity of DE to its parameters. However, FDE does not adequately account for the presence of multiple time scales inherent in signals. To address this limitation, the concept of multiscale fuzzy dispersion entropy (MFDE) was developed to capture the dynamical variability of time series across various scales of complexity. Compared to multiscale DE (MDE), MFDE exhibits reduced sensitivity to noise and higher stability. In order to enhance the stability of MFDE, we propose a refined composite MFDE (RCMFDE). In comparison with MFDE, MDE, and RCMDE, RCMFDE's performance is assessed using synthetic signals and three real bearing datasets. The results consistently demonstrate the superiority of RCMFDE in detecting various patterns within synthetic and real bearing fault data. Importantly, classifiers built upon RCMFDE achieve notably high accuracy values for bearing fault diagnosis applications, outperforming classifiers based on refined composite multiscale dispersion and sample entropy methods. [ABSTRACT FROM AUTHOR]

Fever is a readily measurable physiological response that has been used in medicine for centuries. However, the information provided has been greatly limited by a plain thresholding approach, overlooking the additional information provided by temporal variations and temperature values below such threshold that are also representative of the subject status. In this paper, we propose to utilize continuous body temperature time series of patients that developed a fever, in order to apply a method capable of diagnosing the specific underlying fever cause only by means of a pattern relative frequency analysis. This analysis was based on a recently proposed measure, Slope Entropy, applied to a variety of records coming from dengue and malaria patients, among other fever diseases. After an input parameter customization, a classification analysis of malaria and dengue records took place, quantified by the Matthews Correlation Coefficient. This classification yielded a high accuracy, with more than 90% of the records correctly labelled in some cases, demonstrating the feasibility of the approach proposed. This approach, after further studies, or combined with more measures such as Sample Entropy, is certainly very promising in becoming an early diagnosis tool based solely on body temperature temporal patterns, which is of great interest in the current Covid-19 pandemic scenario. [ABSTRACT FROM AUTHOR]

Approximate Entropy and especially Sample Entropy are recently frequently used algorithms for calculating the measure of complexity of a time series. A lesser known fact is that there are also accelerated modifications of these two algorithms, namely Fast Approximate Entropy and Fast Sample Entropy. All these algorithms are effectively implemented in the R software package TSEntropies. This paper contains not only an explanation of all these algorithms, but also the principle of their acceleration. Furthermore, the paper contains a description of the functions of this software package and their parameters, as well as simple examples of using this software package to calculate these measures of complexity of an artificial time series and the time series of a complex real-world system represented by the course of supercomputer infrastructure power consumption. These time series were also used to test the speed of this package and to compare its speed with another R package pracma. The results show that TSEntropies is up to 100 times faster than pracma and another important result is that the computational times of the new Fast Approximate Entropy and Fast Sample Entropy algorithms are up to 500 times lower than the computational times of their original versions. At the very end of this paper, the possible use of this software package TSEntropies is proposed. [ABSTRACT FROM AUTHOR]

Network security requires real-time monitoring of network traffic in order to detect new and unexpected attacks. Attack detection methods based on deep packet inspection are time consuming and costly, due to their high computational demands. This paper proposes a fast, lightweight method to distinguish different attack types observed in an IP darkspace monitor. The method is based on entropy measures of traffic-flow features and machine learning techniques. The explored data belongs to a portion of the Internet background radiation from a large IP darkspace, i.e., real traffic captures that exclusively contain unsolicited traffic, ongoing attacks, attack preparation activities and attack aftermaths. Results from an in-depth traffic analysis based on packet headers and content are used as a reference to label data and to evaluate the quality of the entropy-based classification. Full IP darkspace traffic captures from a three-week observation period in April, 2012, are used to compare the entropy-based classification with the in-depth traffic analysis. Results show that several traffic types present a high correlation to the respective traffic-flow entropy signals and can even fit polynomial regression models. Therefore, sudden changes in traffic types caused by new attacks or attack preparation activities can be identified based on entropy variations. [ABSTRACT FROM AUTHOR]

In this paper we illustrate the potential of ordinal-patterns-based methods for analysis of real-world data and, especially, of electroencephalogram (EEG) data. We apply already known (empirical permutation entropy, ordinal pattern distributions) and new (empirical conditional entropy of ordinal patterns, robust to noise empirical permutation entropy) methods for measuring complexity, segmentation and classification of time series. [ABSTRACT FROM AUTHOR]

Malaria, a disease with major health and socio-economic impacts, is driven by multiple factors, including a complex interaction with various climatic variables. In this paper, five methods developed for inferring causal relations between dynamic processes based on the information encapsulated in time series are applied on cases previously studied in literature by means of statistical methods. The causality detection techniques investigated in the paper are: a version of the kernel Granger causality, transfer entropy, recurrence plot, causal decomposition and complex networks. The methods provide coherent results giving a quite good confidence in the conclusions. [ABSTRACT FROM AUTHOR]

Modeling financial time series at different time scales is still an open challenge. The choice of a suitable indicator quantifying the distance between the model and the data is therefore of fundamental importance for selecting models. In this paper, we propose a multiscale model selection method based on the Jensen-Shannon distance in order to select the model that is able to better reproduce the distribution of price changes at different time scales. Specifically, we consider the problem of modeling the ultra high frequency dynamics of an asset with a large tick-to-price ratio. We study the price process at different time scales and compute the Jensen-Shannon distance between the original dataset and different models, showing that the coupling between spread and returns is important to model return distribution at different time scales of observation, ranging from the scale of single transactions to the daily time scale. [ABSTRACT FROM AUTHOR]

Permutation entropy, introduced by Bandt and Pompe, is a conceptually simple and well-interpretable measure of time series complexity. In this paper, we propose efficient methods for computing it and related ordinal-patterns-based characteristics. The methods are based on precomputing values of successive ordinal patterns of order d, considering the fact that they are "overlapped" in d points, and on precomputing successive values of the permutation entropy related to "overlapping" successive time-windows. The proposed methods allow for measurement of the complexity of very large datasets in real-time. [ABSTRACT FROM AUTHOR]

Surface electromyography (sEMG) is a valuable technique that helps provide functional and structural information about the electric activity of muscles. As sEMG measures output of complex living systems characterized by multiscale and nonlinear behaviors, Multiscale Permutation Entropy (MPE) is a suitable tool for capturing useful information from the ordinal patterns of sEMG time series. In a previous work, a theoretical comparison in terms of bias and variance of two MPE variants—namely, the refined composite MPE (rcMPE) and the refined composite downsampling (rcDPE), was addressed. In the current paper, we assess the superiority of rcDPE over MPE and rcMPE, when applied to real sEMG signals. Moreover, we demonstrate the capacity of rcDPE in quantifying fatigue levels by using sEMG data recorded during a fatiguing exercise. The processing of four consecutive temporal segments, during biceps brachii exercise maintained at 70% of maximal voluntary contraction until exhaustion, shows that the 10th-scale of rcDPE was capable of better differentiation of the fatigue segments. This scale actually brings the raw sEMG data, initially sampled at 10 kHz, to the specific 0–500 Hz sEMG spectral band of interest, which finally reveals the inner complexity of the data. This study promotes good practices in the use of MPE complexity measures on real data. [ABSTRACT FROM AUTHOR]

This paper presents a novel framework for the complexity analysis of rainfall, runoff, and runoff coefficient (RC) time series using multiscale entropy (MSE). The MSE analysis of RC time series was used to investigate changes in the complexity of rainfallrunoff processes due to human activities. Firstly, a coarse graining process was applied to a time series. The sample entropy was then computed for each coarse-grained time series, and plotted as a function of the scale factor. The proposed method was tested in a case study of daily rainfall and runoff data for the upstream Wu-Tu watershed. Results show that the entropy measures of rainfall time series are higher than those of runoff time series at all scale factors. The entropy measures of the RC time series are between the entropy measures of the rainfall and runoff time series at various scale factors. Results also show that the entropy values of rainfall, runoff, and RC time series increase as scale factors increase. The changes in the complexity of RC time series indicate the changes of rainfallrunoff relations due to human activities and provide a reference for the selection of rainfall-runoff models that are capable of dealing with great complexity and take into account of obvious self-similarity can be suggested to the modeling of rainfall-runoff processes. Moreover, the robustness of the MSE results were tested to confirm that MSE analysis is consistent and the same results when removing 25% data, making this approach suitable for the complexity analysis of rainfall, runoff, and RC time series. [ABSTRACT FROM AUTHOR]

this paper, the energy distributions of various noises following normal, log-normal and Pearson-III distributions are first described quantitatively using the wavelet energy entropy (WEE), and the results are compared and discussed. Then, on the basis of these analytic results, a method for use in choosing the decomposition level (DL) in wavelet threshold de-noising (WTD) is put forward. Finally, the performance of the proposed method is verified by analysis of both synthetic and observed series. Analytic results indicate that the proposed method is easy to operate and suitable for various signals. Moreover, contrary to traditional white noise testing which depends on "autocorrelations", the proposed method uses energy distributions to distinguish real signals and noise in noisy series, therefore the chosen DL is reliable, and the WTD results of time series can be improved. [ABSTRACT FROM AUTHOR]

Time delays are a signature of many physical systems, including the brain, and considerably shape their dynamics; moreover, they play a key role in consciousness, as postulated by the temporo-spatial theory of consciousness (TTC). However, they are often not known a priori and need to be estimated from time series. In this study, we propose the use of permutation entropy (PE) to estimate time delays from neural time series as a more robust alternative to the widely used autocorrelation window (ACW). In the first part, we demonstrate the validity of this approach on synthetic neural data, and we show its resistance to regimes of nonstationarity in time series. Mirroring yet another example of comparable behavior between different nonlinear systems, permutation entropy–time delay estimation (PE-TD) is also able to measure intrinsic neural timescales (INTs) (temporal windows of neural activity at rest) from hd-EEG human data; additionally, this replication extends to the abnormal prolongation of INT values in disorders of consciousness (DoCs). Surprisingly, the correlation between ACW-0 and PE-TD decreases in a state-dependent manner when consciousness is lost, hinting at potential different regimes of nonstationarity and nonlinearity in conscious/unconscious states, consistent with many current theoretical frameworks on consciousness. In summary, we demonstrate the validity of PE-TD as a tool to extract relevant time scales from neural data; furthermore, given the divergence between ACW and PE-TD specific to DoC subjects, we hint at its potential use for the characterization of conscious states. [ABSTRACT FROM AUTHOR]

Entropy indicates irregularity or randomness of a dynamic system. Over the decades, entropy calculated at different scales of the system through subsampling or coarse graining has been used as a surrogate measure of system complexity. One popular multi-scale entropy analysis is the multi-scale sample entropy (MSE), which calculates entropy through the sample entropy (SampEn) formula at each time scale. SampEn is defined by the "logarithmic likelihood" that a small section (within a window of a length m) of the data "matches" with other sections will still "match" the others if the section window length increases by one. "Match" is defined by a threshold of r times standard deviation of the entire time series. A problem of current MSE algorithm is that SampEn calculations at different scales are based on the same matching threshold defined by the original time series but data standard deviation actually changes with the subsampling scales. Using a fixed threshold will automatically introduce systematic bias to the calculation results. The purpose of this paper is to mathematically present this systematic bias and to provide methods for correcting it. Our work will help the large MSE user community avoiding introducing the bias to their multi-scale SampEn calculation results. [ABSTRACT FROM AUTHOR]

Within the paper, the problem of globalisation during financial crises is analysed. The research is based on the Forex exchange rates. In the analysis, the power law classification scheme (PLCS) is used. The study shows that during crises cross-correlations increase resulting in significant growth of cliques, and also the ranks of nodes on the converging time series network are growing. This suggests that the crises expose the globalisation processes, which can be verified by the proposed analysis. [ABSTRACT FROM AUTHOR]

Nonlinear markers of coupling strength are often utilized to typify cardiorespiratory and cerebrovascular regulations. The computation of these indices requires techniques describing nonlinear interactions between respiration (R) and heart period (HP) and between mean arterial pressure (MAP) and mean cerebral blood velocity (MCBv). We compared two model-free methods for the assessment of dynamic HP–R and MCBv–MAP interactions, namely the cross-sample entropy (CSampEn) and k-nearest-neighbor cross-unpredictability (KNNCUP). Comparison was carried out first over simulations generated by linear and nonlinear unidirectional causal, bidirectional linear causal, and lag-zero linear noncausal models, and then over experimental data acquired from 19 subjects at supine rest during spontaneous breathing and controlled respiration at 10, 15, and 20 b r e a t h s · m i n u t e − 1 as well as from 13 subjects at supine rest and during 60° head-up tilt. Linear markers were computed for comparison. We found that: (i) over simulations, CSampEn and KNNCUP exhibit different abilities in evaluating coupling strength; (ii) KNNCUP is more reliable than CSampEn when interactions occur according to a causal structure, while performances are similar in noncausal models; (iii) in healthy subjects, KNNCUP is more powerful in characterizing cardiorespiratory and cerebrovascular variability interactions than CSampEn and linear markers. We recommend KNNCUP for quantifying cardiorespiratory and cerebrovascular coupling. [ABSTRACT FROM AUTHOR]

Financial time series have a fractal nature that poses challenges for their dynamical characterization. The Dow Jones Industrial Average (DJIA) is one of the most influential financial indices, and due to its importance, it is adopted as a test bed for this study. The paper explores an alternative strategy to the standard time analysis, by joining the multidimensional scaling (MDS) computational tool and the concepts of distance, entropy, fractal dimension, and fractional calculus. First, several distances are considered to measure the similarities between objects under study and to yield proper input information to the MDS. Then, the MDS constructs a representation based on the similarity of the objects, where time can be viewed as a parametric variable. The resulting plots show a complex structure that is further analyzed with the Shannon entropy and fractal dimension. In a final step, a deeper and more detailed assessment is achieved by associating the concepts of fractional calculus and entropy. Indeed, the fractional-order entropy highlights the results obtained by the other tools, namely that the DJIA fractal nature is visible at different time scales with a fractional order memory that permeates the time series. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new cross-sample entropy, namely the composite multiscale partial cross-sample entropy (CMPCSE), for quantifying the intrinsic similarity of two time series affected by common external factors. First, in order to test the validity of CMPCSE, we apply it to three sets of artificial data. Experimental results show that CMPCSE can accurately measure the intrinsic cross-sample entropy of two simultaneously recorded time series by removing the effects from the third time series. Then CMPCSE is employed to investigate the partial cross-sample entropy of Shanghai securities composite index (SSEC) and Shenzhen Stock Exchange Component Index (SZSE) by eliminating the effect of Hang Seng Index (HSI). Compared with the composite multiscale cross-sample entropy, the results obtained by CMPCSE show that SSEC and SZSE have stronger similarity. We believe that CMPCSE is an effective tool to study intrinsic similarity of two time series. [ABSTRACT FROM AUTHOR]

Measures of predictability in physiological signals using entropy measures have been widely applied in many areas of research. Multiscale entropy expresses different levels of either approximate entropy or sample entropy by means of multiple factors for generating multiple time series, enabling the capture of more useful information than using a scalar value produced by the two entropy methods. This paper presents the use of different time shifts on various intervals of time series to discover different entropy patterns of the time series. Examples and experimental results using white noise, 1/ f noise, photoplethysmography, and electromyography signals suggest the validity and better performance of the proposed time-shift multiscale entropy analysis of physiological signals than the multiscale entropy. [ABSTRACT FROM AUTHOR]

Despite its widely tested and proven usefulness, there is still room for improvement in the basic permutation entropy (PE) algorithm, as several subsequent studies have demonstrated in recent years. Some of these new methods try to address the well-known PE weaknesses, such as its focus only on ordinal and not on amplitude information, and the possible detrimental impact of equal values found in subsequences. Other new methods address less specific weaknesses, such as the PE results' dependence on input parameter values, a common problem found in many entropy calculation methods. The lack of discriminating power among classes in some cases is also a generic problem when entropy measures are used for data series classification. This last problem is the one specifically addressed in the present study. Toward that purpose, the classification performance of the standard PE method was first assessed by conducting several time series classification tests over a varied and diverse set of data. Then, this performance was reassessed using a new Shannon Entropy normalisation scheme proposed in this paper: divide the relative frequencies in PE by the number of different ordinal patterns actually found in the time series, instead of by the theoretically expected number. According to the classification accuracy obtained, this last approach exhibited a higher class discriminating power. It was capable of finding significant differences in six out of seven experimental datasets—whereas the standard PE method only did in four—and it also had better classification accuracy. It can be concluded that using the additional information provided by the number of forbidden/found patterns, it is possible to achieve a higher discriminating power than using the classical PE normalisation method. The resulting algorithm is also very similar to that of PE and very easy to implement. [ABSTRACT FROM AUTHOR]

We analyzed a dynamic duopoly game where players adopt specific preferences. These preferences are derived from Cobb–Douglas utility function with the assumption that they depend on past choices. For this paper, we investigated two possible cases for the suggested game. The first case considers only focusing on the action done by one player. This action reduces the game's map to a one-dimensional map, which is the logistic map. Using analytical and numerical simulation, the stability of fixed points of this map is studied. In the second case, we focus on the actions applied by both players. The fixed points, in this case, are calculated, and their stability is discussed. The conditions of stability are provided in terms of the game's parameters. Numerical simulation is carried out to give local and global investigations of the chaotic behavior of the game's map. In addition, we use a statistical measure, such as entropy, to get more evidences on the regularity and predictability of time series associated with this case. [ABSTRACT FROM AUTHOR]

Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an information transfer measure used to quantify the statistical coherence between events (time series). Later, it was related to causality, even if they are not the same. There are only few papers reporting applications of causality or TE in neural networks. Our contribution is an information-theoretical method for analyzing information transfer between the nodes of feedforward neural networks. The information transfer is measured by the TE of feedback neural connections. Intuitively, TE measures the relevance of a connection in the network and the feedback amplifies this connection. We introduce a backpropagation type training algorithm that uses TE feedback connections to improve its performance. [ABSTRACT FROM AUTHOR]

Recently, the concept of evaluating an unusually large learning effort of an adaptive system to detect novelties in the observed data was introduced. The present paper introduces a new measure of the learning effort of an adaptive system. The proposed method also uses adaptable parameters. Instead of a multi-scale enhanced approach, the generalized Pareto distribution is employed to estimate the probability of unusual updates, as well as for detecting novelties. This measure was successfully tested in various scenarios with (i) synthetic data, (ii) real time series datasets, and multiple adaptive filters and learning algorithms. The results of these experiments are presented. [ABSTRACT FROM AUTHOR]

Entropy is a key concept in the characterization of uncertainty for any given signal, and its extensions such as Spectral Entropy and Permutation Entropy. They have been used to measure the complexity of time series. However, these measures are subject to the discretization employed to study the states of the system, and identifying the relationship between complexity measures and the expected performance of the four selected forecasting methods that participate in the M4 Competition. This relationship allows the decision, in advance, of which algorithm is adequate. Therefore, in this paper, we found the relationships between entropy-based complexity framework and the forecasting error of four selected methods (Smyl, Theta, ARIMA, and ETS). Moreover, we present a framework extension based on the Emergence, Self-Organization, and Complexity paradigm. The experimentation with both synthetic and M4 Competition time series show that the feature space induced by complexities, visually constrains the forecasting method performance to specific regions; where the logarithm of its metric error is poorer, the Complexity based on the emergence and self-organization is maximal. [ABSTRACT FROM AUTHOR]

Most of time series deriving from complex systems in real life is non-stationary, where the data distribution would be influenced by various internal/external factors such that the contexts are persistently changing. Therefore, the concept drift detection of time series has practical significance. In this paper, a novel method called online entropy-based time domain feature extraction (ETFE) for concept drift detection is proposed. Firstly, the empirical mode decomposition based on extrema symmetric extension is used to decompose time series, where features in various time scales can be adaptively extracted. Meanwhile, the end point effect caused by traditional empirical mode decomposition can be avoided. Secondly, by using the entropy calculation, the time-domain features are coarse-grained to quantify the structure and complexity of the time series, among which six kinds of entropy are used for discussion. Finally, a statistical process control method based on generalized likelihood ratio is used to monitor the change of the entropy, which can effectively track the mean and amplitude of the time series. Therefore, the early alarm of concept drift can be given. Synthetic data sets and neonatal electroencephalogram (EEG) recordings with seizures annotations data sets are used to validate the effectiveness and accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

Entropy applications in hydrometric network design problems have been extensively studied in the most recent decade. Although many studies have successfully found the optimal networks, there have been assumptions which could not be logically integrated into their methodology. One of the major assumptions is the uncertainty that can arise from data processing, such as time series simulation for the potential stations, and the necessary data quantization in entropy calculations. This paper introduces a methodology called ensemble-based dual entropy and multiobjective optimization (EnDEMO), which considers uncertainty from the ensemble generation of the input data. The suggested methodology was applied to design hydrometric networks in the Nelson-Churchill River Basin in central Canada. First, the current network was evaluated by transinformation analysis. Then, the optimal networks were explored using the traditional deterministic network design method and the newly proposed ensemble-based method. Result comparison showed that the most frequently selected stations by EnDEMO were fewer and appeared more reliable for practical use. The maps of station selection frequency from both DEMO and EnDEMO allowed us to identify preferential locations for additional stations; however, EnDEMO provided a more robust outcome than the traditional approach. [ABSTRACT FROM AUTHOR]

Entropy is re-examined as a quantification of ignorance in the predictability of a one dimensional continuous phenomenon. Although traditional estimators for entropy have been widely utilized in this context, we show that both the thermodynamic and Shannon's theory of entropy are fundamentally discrete, and that the limiting process used to define differential entropy suffers from similar problems to those encountered in thermodynamics. In contrast, we consider a sampled data set to be observations of microstates (unmeasurable in thermodynamics and nonexistent in Shannon's discrete theory), meaning, in this context, it is the macrostates of the underlying phenomenon that are unknown. To obtain a particular coarse-grained model we define macrostates using quantiles of the sample and define an ignorance density distribution based on the distances between quantiles. The geometric partition entropy is then just the Shannon entropy of this finite distribution. Our measure is more consistent and informative than histogram-binning, especially when applied to complex distributions and those with extreme outliers or under limited sampling. Its computational efficiency and avoidance of negative values can also make it preferable to geometric estimators such as k-nearest neighbors. We suggest applications that are unique to this estimator and illustrate its general utility through an application to time series in the approximation of an ergodic symbolic dynamics from limited observations. [ABSTRACT FROM AUTHOR]

Ordinal pattern-based approaches have great potential to capture intrinsic structures of dynamical systems, and therefore, they continue to be developed in various research fields. Among these, the permutation entropy (PE), defined as the Shannon entropy of ordinal probabilities, is an attractive time series complexity measure. Several multiscale variants (MPE) have been proposed in order to bring out hidden structures at different time scales. Multiscaling is achieved by combining linear or nonlinear preprocessing with PE calculation. However, the impact of such a preprocessing on the PE values is not fully characterized. In a previous study, we have theoretically decoupled the contribution of specific signal models to the PE values from that induced by the inner correlations of linear preprocessing filters. A variety of linear filters such as the autoregressive moving average (ARMA), Butterworth, and Chebyshev were tested. The current work is an extension to nonlinear preprocessing and especially to data-driven signal decomposition-based MPE. The empirical mode decomposition, variational mode decomposition, singular spectrum analysis-based decomposition and empirical wavelet transform are considered. We identify possible pitfalls in the interpretation of PE values induced by these nonlinear preprocessing, and hence, we contribute to improving the PE interpretation. The simulated dataset of representative processes such as white Gaussian noise, fractional Gaussian processes, ARMA models and synthetic sEMG signals as well as real-life sEMG signals are tested. [ABSTRACT FROM AUTHOR]

This study addresses the problem of learning a summary causal graph on time series with potentially different sampling rates. To do so, we first propose a new causal temporal mutual information measure for time series. We then show how this measure relates to an entropy reduction principle that can be seen as a special case of the probability raising principle. We finally combine these two ingredients in PC-like and FCI-like algorithms to construct the summary causal graph. There algorithm are evaluated on several datasets, which shows both their efficacy and efficiency. [ABSTRACT FROM AUTHOR]

This study examined the association between pressure injuries and complexity of abdominal temperature measured in residents of a nursing facility. The temperature served as a proxy measure for skin thermoregulation. Refined multiscale sample entropy and bubble entropy were used to measure the irregularity of the temperature time series measured over two days at 1-min intervals. Robust summary measures were derived for the multiscale entropies and used in predictive models for pressure injuries that were built with adaptive lasso regression and neural networks. Both types of entropies were lower in the group of participants with pressure injuries ( n = 11 ) relative to the group of non-injured participants ( n = 15 ). This was generally true at the longer temporal scales, with the effect peaking at scale τ = 22 min for sample entropy and τ = 23 min for bubble entropy. Predictive models for pressure injury on the basis of refined multiscale sample entropy and bubble entropy yielded 96% accuracy, outperforming predictions based on any single measure of entropy. Combining entropy measures with a widely used risk assessment score led to the best prediction accuracy. Complexity of the abdominal temperature series could therefore serve as an indicator of risk of pressure injury. [ABSTRACT FROM AUTHOR]

Making predictions on the dynamics of time series of a system is a very interesting topic. A fundamental prerequisite of this work is to evaluate the predictability of the system over a wide range of time. In this paper, we propose an information-theoretic tool, multiscale entropy difference (MED), to evaluate the predictability of nonlinear financial time series on multiple time scales. We discuss the predictability of the isolated system and open systems, respectively. Evidence from the analysis of the logistic map, Hénon map, and the Lorenz system manifests that the MED method is accurate, robust, and has a wide range of applications. We apply the new method to five-minute high-frequency data and the daily data of Chinese stock markets. Results show that the logarithmic change of stock price (logarithmic return) has a lower possibility of being predicted than the volatility. The logarithmic change of trading volume contributes significantly to the prediction of the logarithmic change of stock price on multiple time scales. The daily data are found to have a larger possibility of being predicted than the five-minute high-frequency data. This indicates that the arbitrage opportunity exists in the Chinese stock markets, which thus cannot be approximated by the effective market hypothesis (EMH). [ABSTRACT FROM AUTHOR]

This paper presents a new formula for the entropy of a distribution, that is conceived having in mind the Liouville fractional derivative. For illustrating the new concept, the proposed definition is applied to the Dow Jones Industrial Average. Moreover, the Jensen-Shannon divergence is also generalized and its variation with the fractional order is tested for the time series. [ABSTRACT FROM AUTHOR]

In this paper, we simultaneously use two different scales in the analysis of ordinal patterns to measure the complexity of the dynamics of heartbeat time series. Rényi entropy and weighted Rényi entropy are the entropy-like measures proposed in the multiscale analysis in which, with the new scheme, four parameters are involved. First, the influence of the variation of the new parameters in the entropy values is analyzed when different groups of subjects (with cardiac diseases or healthy) are considered. Secondly, we exploit the introduction of multiscale analysis in order to detect differences between the groups. [ABSTRACT FROM AUTHOR]