Search

Affective computing is one of the key technologies to achieve advanced brain-machine interfacing. It is increasingly concerning research orientation in the field of artificial intelligence. Emotion recognition is closely related to affective computing. Although emotion recognition based on electroencephalogram (EEG) has attracted more and more attention at home and abroad, subject-independent emotion recognition still faces enormous challenges. We proposed a subject-independent emotion recognition algorithm based on dynamic empirical convolutional neural network (DECNN) in view of the challenges. Combining the advantages of empirical mode decomposition (EMD) and differential entropy (DE), we proposed a dynamic differential entropy (DDE) algorithm to extract the features of EEG signals. After that, the extracted DDE features were classified by convolutional neural networks (CNN). Finally, the proposed algorithm is verified on SJTU Emotion EEG Dataset (SEED). In addition, we discuss the brain area closely related to emotion and design the best profile of electrode placements to reduce the calculation and complexity. Experimental results show that the accuracy of this algorithm is 3.53 percent higher than that of the state-of-the-art emotion recognition methods. What's more, we studied the key electrodes for EEG emotion recognition, which is of guiding significance for the development of wearable EEG devices.

In this article, we propose nonparametric kernel type estimators for the Shannon differential entropy for length-biased censored data. The asymptotic properties of the estimators are established under specific regularity conditions. The performance of the estimators is examined through simulated observations and is compared using mean squared errors for various sample sizes. The applicability of the estimators is demonstrated using real data.

The task of estimating the parameters of the Pareto distribution, first of all, of an indicator of this distribution for a given sample, is relevant. This article establishes that for this estimate, it is sufficient to know the product of the sample elements. It is proved that this product is a sufficient statistic for the Pareto distribution parameter. On the basis of the maximum likelihood method the distribution degree indicator is estimated. It is proved that this estimate is biased, and a formula eliminating the bias is justified. For the product of the sample elements considered as a random variable the distribution function and probability density are found; mathematical expectation, higher moments, and differential entropy are calculated. The corresponding graphs are built. In addition, it is noted that any function of this product is a sufficient statistic, in particular, the geometric mean. For the geometric mean also considered as a random variable, the distribution function, probability density, and the mathematical expectation are found; the higher moments, and the differential entropy are also calculated, and the corresponding graphs are plotted. In addition, it is proved that the geometric mean of the sample is a more convenient sufficient statistic from a practical point of view than the product of the sample elements. Also, on the basis of the Rao–Blackwell–Kolmogorov theorem, effective estimates of the Pareto distribution parameter are constructed. In conclusion, as an example, the technique developed here is applied to the exponential distribution. In this case, both the sum and the arithmetic mean of the sample can be used as sufficient statistics.

In this paper, we propose nonparametric kernel estimators for Shannon differential entropy function under length-biased sampling. Asymptotic properties of the estimators are established under suita...

This article addresses the estimation of differential entropy of the Weibull distribution based on different non-informative prior distributions. We first derive non-informative priors using formal rules, such as Jeffreys prior and maximal data information prior based on Fisher information and entropy, respectively. We also develop reference prior and probability matching prior for the differential entropy. Next, we investigate the effects of these priors in the Bayes estimates of the differential entropy of the Weibull distribution based on complete sample. The Bayes estimates are computed using random-walk Metropolis-Hastings algorithms. We use Monte Carlo simulation to evaluate the performance of the Bayes estimates of differential entropy in terms of bias and root mean-squared error, while highest posterior density credible intervals are assessed in terms of average width and coverage probabilities. Finally, a real data set has been analyzed for illustrative purposes.

In this article an information-based method for the selection of expansion coefficients of functions in a Hilbert basis is presented. An information-based measure, namely Entropic NID (ENID), is presented; the optimal separation point between more informative coefficients and less informative ones is selected by evaluating the information contribution of two competing sets of expansion coefficients. A consistent numerical scheme is given to approximate ENID and the numerical error is studied. Numerical simulations are provided to test the behaviour of ENID in different wavelet bases, as well as to perform comparative studies.

This paper introduces the bivariate extension of the amoroso distribution and its density function is expressed in terms of hyper-geometric function. The standard amoroso distribution, cumulative distribution functions, conditional distributions, and its moments are also derived. The Product moments, Co-variance, correlations, and Shannon’s differential entropy are also shown. Moreover, the generating functions such as moment, Cumulant, Characteristic functions are expressed in Fox-wright function, and the Survival, hazard, and Cumulative hazard functions are also computed. The special cases of the bivariate amoroso distribution are also discussed and nearly 780 bivariate mixtures of distributions can be derived. Finally, the two-dimensional probability surfaces are visualized for the selected special cases and we also showed the estimation of parameters by the method of maximum likelihood approach, and the constrained maximum likelihood approach is also computed by using Non-linear Programming with a numerical application

In this paper, we present a review of Shannon and differential entropy rate estimation techniques. Entropy rate, which measures the average information gain from a stochastic process, is a measure of uncertainty and complexity of a stochastic process. We discuss the estimation of entropy rate from empirical data, and review both parametric and non-parametric techniques. We look at many different assumptions on properties of the processes for parametric processes, in particular focussing on Markov and Gaussian assumptions. Non-parametric estimation relies on limit theorems which involve the entropy rate from observations, and to discuss these, we introduce some theory and the practical implementations of estimators of this type.

We consider an Ito stochastic differential equation and study the asymptotic behaviors of stationary measures of the corresponding Fokker–Planck equation in the vicinity of a deterministic attractor or a repeller. By establishing global measure estimates in domains of interest, we show Gaussian-like behaviors of these measures in the basins of the attractor and the repeller. Not only do our results quantify the concentration results of stationary measures obtained in Huang et al. (2018), but also they are valid near a usual attractor (resp. repeller) instead of a strong attractor (resp. repeller) assumed in Huang et al. (2018). Our approach in conducting the measure estimates is based on a new idea of constructing a sequence of Lyapunov functions (anti-Lyapunov functions) in the basin of attraction (resp. expansion) of the attractor (resp. repeller). As applications of these measure estimates, we also derive upper bounds for the differential entropy and establish certain entropy-dimension inequalities.

Voice activity detection (VAD) is essential for multiple microphone arrays processing, in which massive potential devices, such as microphone devices for far-field voice-based interaction in smart home environments, will be activated when sound sources appear. Therefore, the VAD can save a lot of computing resources in massive microphone arrays processing for the sparsity in sound source activity. However, it may not be feasible to obtain an accurate VAD in harsh environments, such as far-field, time-varying noise field. In this paper, the long-term speech information (LTSI) and the log-energy are modeled for deriving a more accurate VAD. First, the LTSI can be obtained by measuring the differential entropy of long-term smoothed noisy signal spectrum. Then, the LTSI is used to get labeled data for the initialization of a Gaussian mixture model (GMM), which is used to fit the log-energy distribution of noise and (noisy) speech. Finally, combining the LTSI and the GMM parameters of noise and speech distribution, this paper derives an adaptive threshold, which represents a reasonable boundary between noise and speech. Experimental results show that our VAD method has a remarkable improvement for a massive microphone network.

Intrinsic dimensional analysis and differential entropy estimation are closely related to the point that they can be jointly estimated as two complementary outputs of a single approach. In this brief letter, it is shown that, given a noisy signal, intrinsic dimension and differential entropy, jointly analyzed through visual approach, can be used to estimate noise variance directly from noisy signal samples. Preliminary results corroborating this idea are presented through experiments with chaotic (Lorenz chaotic model) and almost periodic (human vocalization) signals, including comparisons with results from methods in the concerned literature.

Di Crescenzo and Longobardi [Di Crescenzo and Longobardi, On cumulative entropies, J. Statist. Plann. Inference 139 2009, 12, 4072–4087] proposed the cumulative entropy (CE) as an alternative to the differential entropy. They presented an estimator of CE using empirical approach. In this paper, we consider a risk measure based on CE and compare it with the standard deviation and the Gini mean difference for some distributions. We also make empirical comparisons of these measures using samples from stock market in members of the Organization for Economic Co-operation and Development (OECD) countries.

In this short note, we revisit the partial monotonicity of the differential entropy, and point out some errors in Shangari and Chen (2012) and Gupta and Bajaj (2013). The partial monotonicity of the Shannon entropy is also established.

Differential entropy can be negative, while discrete entropy is always non-negative. This article shows that negative entropy is a significant flaw when entropy is used as a sensitivity measure in global sensitivity analysis. Global sensitivity analysis based on differential entropy cannot have negative entropy, just as Sobol sensitivity analysis does not have negative variance. Entropy is similar to variance but does not have the same properties. An alternative sensitivity measure based on the approximation of the differential entropy using dome-shaped functionals with non-negative values is proposed in the article. Case studies have shown that new sensitivity measures lead to a rational structure of sensitivity indices with a significantly lower proportion of higher-order sensitivity indices compared to other types of distributional sensitivity analysis. In terms of the concept of sensitivity analysis, a decrease in variance to zero means a transition from the differential to discrete entropy. The form of this transition is an open question, which can be studied using other scientific disciplines. The search for new functionals for distributional sensitivity analysis is not closed, and other suitable sensitivity measures may be found. Diferenciální entropie může být záporná, zatímco diskrétní entropie je vždy nezáporná. Tento článek ukazuje, že záporná entropie je významnou chybou, když je entropie použita jako citlivostní míra v globální citlivostní analýze. Globální citlivostní analýza založená na diferenciální entropii nemůže mít negativní entropii, stejně jako Sobolova citlivostní analýza nemá negativní rozptyl. Entropie je podobná rozptylu, ale nemá stejné vlastnosti. V článku je navrženo alternativní měření citlivosti založené na aproximaci diferenciální entropie pomocí kopulovitých funkcionálů s nezápornými hodnotami. Případové studie ukázaly, že nová měření citlivosti vedou k racionální struktuře indexů citlivosti s výrazně nižším podílem indexů citlivosti vyššího řádu ve srovnání s jinými typy distribučních citlivostních analýz. Z hlediska konceptu citlivostní analýzy znamená pokles rozptylu na nulu přechod od diferenciální k diskrétní entropii. Forma tohoto přechodu je otevřenou otázkou, kterou lze studovat pomocí jiných vědních oborů. Hledání nových funkcionálů pro distribuční analýzu citlivosti není uzavřeno a lze nalézt další vhodná opatření pro citlivost.

We study the increase in per-sample differential entropy rate of random sequences and processes after being passed through a non minimum-phase (NMP) discrete-time, linear time-invariant (LTI) filter G. For LTI discrete-time filters and random processes, it has long been established by Theorem 14 in Shannon’s seminal paper that this entropy gain, (G), equals the integral of log|G|. In this note, we first show that Shannon’s Theorem 14 does not hold in general. Then, we prove that, when comparing the input differential entropy to that of the entire (longer) output of G, the entropy gain equals (G). We show that the entropy gain between equal-length input and output sequences is upper bounded by (G) and arises if and only if there exists an output additive disturbance with finite differential entropy (no matter how small) or a random initial state. Unlike what happens with linear maps, the entropy gain in this case depends on the distribution of all the signals involved. We illustrate some of the consequences of these results by presenting their implications in three different problems. Specifically: conditions for equality in an information inequality of importance in networked control problems, extending to a much broader class of sources the existing results on the rate-distortion function for non-stationary Gaussian sources, and an observation on the capacity of auto-regressive Gaussian channels with feedback.

International audience; This paper is devoted to the simulation of the three-phase flow model [20], in order to account for immiscible components. The whole model is first recalled, and the main properties of the closed set are given, with particular focus on the Riemann problem associated with the convective subset that contains non-conservative terms, and also on the relaxation process. The model is hyperbolic, far from resonance occurrence, and a physically relevant entropy inequality holds for smooth solutions of the whole system. Owing to the uniqueness of jump conditions, specific solutions of the one-dimensional Riemann problem can be built, and these are useful (and mandatory) for the verification procedure. The fractional step method proposed herein complies with the continuous entropy inequality, and implicit schemes that are considered to account for relaxation terms take their roots on the true relaxation process. Once verification tests have been achieved, focus is given on the simulation of the experimental setup [8, 9], in order to simulate a cloud of droplets that is hit by an incoming gas shock-wave. Finally, the study of a three-phase flow setup involving thermal effects is presented, it is based on the KROTOS experiment [25] which focuses on vapour explosion simulation.

The second paper of this series presents two robust entropy stable shock-capturing methods for discontinuous Galerkin spectral element (DGSEM) discretizations of the compressible magneto-hydrodynamics (MHD) equations. Specifically, we use the resistive GLM-MHD equations, which include a divergence cleaning mechanism that is based on a generalized Lagrange multiplier (GLM). For the continuous entropy analysis to hold, and due to the divergence-free constraint on the magnetic field, the GLM-MHD system requires the use of non-conservative terms, which need special treatment. Hennemann et al. (2020) [25] recently presented an entropy stable shock-capturing strategy for DGSEM discretizations of the Euler equations that blends the DGSEM scheme with a subcell first-order finite volume (FV) method. Our first contribution is the extension of the method of Hennemann et al. to systems with non-conservative terms, such as the GLM-MHD equations. In our approach, the advective and non-conservative terms of the equations are discretized with a hybrid FV/DGSEM scheme, whereas the visco-resistive terms are discretized only with the high-order DGSEM method. We prove that the extended method is semi-discretely entropy stable on three-dimensional unstructured curvilinear meshes. Our second contribution is the derivation and analysis of a second entropy stable shock-capturing method that provides enhanced resolution by using a subcell reconstruction procedure that is carefully built to ensure entropy stability. We provide a numerical verification of the properties of the hybrid FV/DGSEM schemes on curvilinear meshes and show their robustness and accuracy with common benchmark cases, such as the Orszag-Tang vortex and the GEM (Geospace Environmental Modeling) reconnection challenge. Finally, we simulate a space physics application: the interaction of Jupiter's magnetic field with the plasma torus generated by the moon Io.

Objective prior distributions represent an important tool that allows one to have the advantages of using a Bayesian framework even when information about the parameters of a model is not available. The usual objective approaches work off the chosen statistical model and in the majority of cases the resulting prior is improper, which can pose limitations to a practical implementation, even when the complexity of the model is moderate. In this paper we propose to take a novel look at the construction of objective prior distributions, where the connection with a chosen sampling distribution model is removed. We explore the notion of defining objective prior distributions which allow one to have some degree of flexibility, in particular in exhibiting some desirable features, such as being proper, or log-concave, convex etc. The basic tool we use are proper scoring rules and the main result is a class of objective prior distributions that can be employed in scenarios where the usual model based priors fail, such as mixture models and model selection via Bayes factors. In addition, we show that the proposed class of priors is the result of minimising the information it contains, providing solid interpretation to the method.

This paper develops entropy stable (ES) adaptive moving mesh schemes for the 2D and 3D special relativistic hydrodynamic (RHD) equations. They are built on the ES finite volume approximation of the RHD equations in curvilinear coordinates, the discrete geometric conservation laws, and the mesh adaptation implemented by iteratively solving the Euler-Lagrange equations of the mesh adaption functional in the computational domain with suitably chosen monitor functions. First, a sufficient condition is proved for the two-point entropy conservative (EC) flux, by mimicking the derivation of the continuous entropy identity in curvilinear coordinates and using the discrete geometric conservation laws given by the conservative metrics method. Based on such sufficient condition, the EC fluxes for the RHD equations in curvilinear coordinates are derived and the second-order accurate semi-discrete EC schemes are developed to satisfy the entropy identity for the given convex entropy pair. Next, the semi-discrete ES schemes satisfying the entropy inequality are proposed by adding a suitable dissipation term to the EC scheme and utilizing linear reconstruction with the minmod limiter in the scaled entropy variables in order to suppress the numerical oscillations of the above EC scheme. Then, the semi-discrete ES schemes are integrated in time by using the second-order strong stability preserving explicit Runge-Kutta schemes. Finally, several numerical results show that our 2D and 3D ES adaptive moving mesh schemes effectively capture the localized structures, such as sharp transitions or discontinuities, and are more efficient than their counterparts on uniform mesh., 35 pages, 11 figures, 3 tables

A suitable replacement model for random lifetimes is extended to the context of past lifetimes. At a fixed time u an item is planned to be replaced by another one having the same age but a different lifetime distribution. We investigate the past lifetime of this system, given that at a larger time t the system is found to be failed. Subsequently, we perform some stochastic comparisons between the random lifetimes of the single items and the doubly truncated random variable that describes the system lifetime. Moreover, we consider the relative ratio of improvement evaluated at x &isin, ( u , t ) , which is finalized to measure the goodness of the replacement procedure. The characterization and the properties of the differential entropy of the system lifetime are also discussed. Finally, an example of application to the firing activity of a stochastic neuronal model is provided.

A shift-dependent information measure is favorable to handle in some specific applied contexts such as mathematical neurobiology and survival analysis. For this reason, the weighted differential entropy has been introduced in the literature. In accordance with this measure, we propose the weighted generalized cumulative residual entropy as well. Despite existing apparent similarities between these measures, however, there are quite substantial and subtle differences between them because of their different metrics. In this paper, particularly, we show that the proposed measure is equivalent to the generalized cumulative residual entropy of the cumulative weighted random variable. Thus, we first provide expressions for the variance and the new measure in terms of the weighted mean residual life function and then elaborate on some characteristics of such measures, including equivalent expressions, stochastic comparisons, bounds, and connection with the excess wealth transform. Finally, we also illustrate some applications of interest in system reliability with reference to shock models and random minima.

We study a variant of the many-help one hypothesis testing against independence problem in which the source, not necessarily Gaussian, has finite differential entropy and the observation noises under the null hypothesis are Gaussian. Under the criterion that stipulates minimization of the Type II error exponent subject to a (constant) bound on the Type I error rate, we derive an upper bound on the exponent-rates function. The bound is shown to mirror a corresponding explicit lower bound, except that the lower bound involves the source power (variance) whereas the upper bound has the source entropy power. Part of the utility of the established bound is for investigating asymptotic exponent/rates and losses incurred by distributed detection as function of the number of observations.

We study the continuity property of the generalized entropy as a functional of probability distribution, defined with an action space and a loss function. Upper and lower bounds for the entropy difference of two distributions are derived in terms of several commonly used f-divergences, the Wasserstein distance, and a distance that depends on the action space and the loss function. Among these bounds, we notice a connection between the continuity of Shannon/differential entropy in the distribution in KL divergence and the continuity of the Renyi entropy in the entropy order. For information-theoretic applications, we derive several new mutual information upper bounds based on the entropy difference bounds. The general results may find broader applications in estimation theory, statistical learning theory, and control theory.

There are several ways to measure the compressibility of a random measure; they include general approaches, such as using the rate-distortion curve, as well as more specific notions, such as the Renyi information dimension (RID), and dimensional-rate bias (DRB). The RID parameter indicates the concentration of the measure around lower-dimensional subsets of the space while the DRB parameter specifies the compressibility of the distribution over these lower-dimensional subsets. While the evaluation of such compressibility parameters is well-studied for continuous and discrete measures (e.g., the DRB is closely related to the entropy and differential entropy in discrete and continuous cases, respectively), the case of discrete-continuous measures is quite subtle. In this paper, we focus on a class of multi-dimensional random measures that have singularities on affine lower-dimensional subsets. This class of distributions naturally arises when considering linear transformation of component-wise independent discrete-continuous random variables. Here, we evaluate the RID and DRB for such probability measures. We further provide an upper-bound for the RID of multi-dimensional random measures that are obtained by Lipschitz functions of component-wise independent discrete-continuous random variables (X). The upper-bound is shown to be achievable when the Lipschitz function is AX, where A satisfies SPARK(A) = rank(A) + 1 (e.g., Vandermonde matrices). When considering discrete-domain moving-average processes with non-Gaussian excitation noise, the above results allow us to evaluate the block-average RID and DRB, as well as to determine a relationship between these parameters and other existing compressibility measures.

This work is focused on the entropy analysis of a semi-discrete nodal discontinuous Galerkin spectral element method (DGSEM) on moving meshes for hyperbolic conservation laws. The DGSEM is constructed with a local tensor-product Lagrange-polynomial basis computed from Legendre-Gauss-Lobatto points. Furthermore, the collocation of interpolation and quadrature nodes is used in the spatial discretization. This approach leads to discrete derivative approximations in space that are summation-by-parts (SBP) operators. On a static mesh, the SBP property and suitable two-point flux functions, which satisfy the entropy condition from Tadmor, allow to mimic results from the continuous entropy analysis, if it is ensured that properties such as positivity preservation (of the water height, density or pressure) are satisfied on the discrete level. In this paper, Tadmor’s condition is extended to the moving mesh framework. We show that the volume terms in the semi-discrete moving mesh DGSEM do not contribute to the discrete entropy evolution when a two-point flux function that satisfies the moving mesh entropy condition is applied in the split form DG framework. The discrete entropy behavior then depends solely on the interface contributions and on the domain boundary contribution. The interface contributions are directly controlled by proper choice of the numerical element interface fluxes. If an entropy conserving two-point flux is chosen, the interface contributions vanish. To increase the robustness of the discretization we use so-called entropy stable two-point fluxes at the interfaces that are guaranteed entropy dissipative and thus give a bound on the interface contributions in the discrete entropy balance. The remaining boundary condition contributions depend on the type of the considered boundary condition. E.g. for periodic boundary conditions that are of entropy conserving type, our methodology with the entropy conserving interface fluxes is fully entropy conservative and with the entropy stable interface fluxes is guaranteed entropy stable. The presented proof does not require any exactness of quadrature in the spatial integrals of the variational forms. As it is the case for static meshes, these results rely on the assumption that additional properties like positivity preservation are satisfied on the discrete level. Besides the entropy stability, the time discretization of the moving mesh DGSEM will be investigated and it will be proven that the moving mesh DGSEM satisfies the free stream preservation property for an arbitrary s-stage Runge–Kutta method, when periodic boundary conditions are used. The theoretical properties of the moving mesh DGSEM will be validated by numerical experiments for the compressible Euler equations with periodic boundary conditions., H2020 European Research Council, Projekt DEAL

The aim of present paper is to obtain Shannon type inequalities using the extended version of Jensen’s inequality in time scales settings. The concept of differential entropy of a continuous random variable on time scales is introduced, and its bounds for some particular distributions are also estimated.

In this paper, we propose nonparametric kernel estimators for Shannon differential entropy function under length-biased sampling. Asymptotic properties of the estimators are established under suita...

Several generalizations of Shannon entropy have been introduced in the literature. One of such a measure is the cumulative residual Tsallis entropy (CRTE) which can be viewed as an alternative dispersion measure. In this paper, we obtain some further results for such a measure, in relation with the cumulative residual Tsallis entropy and with the variance of random variables. Specifically, we present new equivalent expressions, bounds, normalized CRTE, connection to the differential entropy and excess wealth transform and stochastic comparisons involving such measures. Besides, the dynamic version of such under consideration measure is elaborated and some monotonicity results are also given. Finally, we consider the problem of estimating the CRTE by means of the empirical CRTE. In this regard, we use two different empirical estimators of cumulative distribution function to estimate CRTE. Then, we study practical results of the second estimator in blind image quality assessment.

The problem of assessing tightness of the interdependence between random vectors of different dimensionality is considered. These random vectors can obey arbitrary multidimensional continuous distribution laws. An analytical expression is derived for the coefficient of tightness of the interdependence between random vectors. It is expressed in terms of the coefficients of determination of conditional regressions between the components of random vectors. For the case of Gaussian random vectors, a simpler formula is obtained, expressed through the determinants of each of the random vectors and determinant of their association. It is shown that the introduced coefficient meets all the basic requirements imposed on the degree of tightness of the interdependence between random vectors. This approach is more preferable compared to the method of canonical correlations providing determination of the actual tightness of the interdependence between random vectors. Moreover, it can also be used in case of non-linear correlation dependence between the components of random vectors. The measure thus introduced is rather simply interpretable and can be applied in practice to real data samplings. Examples of calculating the tightness of the interdependence between Gaussian random vectors of different dimensionality are given.

We propose a mathematical model of the entropy estimator in the Linux random number generator. First, we construct a probability model for random event times in entropy sources, and then precisely derive probability distributions for the first, second, and third time differences. Second, we obtain the probability distribution for the minimum of absolute values of these differences, which is used for the estimated entropy in the Linux system. Moreover, we provide several simulations that display the accuracy of our results for various parameters.

We consider the single-cell multi-user multiple-input multiple-output uplink with several single antenna transmitters/users and one base station (BS) with $N$ antennas in the $N\rightarrow \infty $ regime. The BS antennas are evenly distributed to $n$ admissible locations throughout the cell. First, we show that a reliable (per-user) rate of $O(\log n)$ is achievable through the optimal locational optimization of BS antennas. We also prove that an $O(\log n)$ rate is the best possible. Therefore, in contrast to a centralized or circular deployment, where the achievable rate is at most a constant, the rate with a general deployment can grow logarithmically with $n$ , resulting in a certain form of “macro-multiplexing gain.” Second, using tools from high-resolution quantization theory, we present an accurate heuristic formula for the best achievable rate given any $n$ and any user density function. According to our formula, the dependence of the optimal rate on the user density function $f$ is curiously only through the differential entropy of $f$ . In fact, the optimal rate decreases linearly with the differential entropy, and the worst-case scenario is a uniform user density. We also describe a modified gradient ascent procedure for the numerical optimization of antenna locations. Simulations confirm our analytical findings.

Recent studies found that many channels are affected by additive noise that is impulsive in nature and is best explained by heavy-tailed symmetric alpha-stable distributions. Dealing with impulsive noise environments comes with an added complexity with respect to the standard Gaussian environment: the alpha-stable probability density functions have an infinite second moment and the "nice" Hilbert space structure of the space of random variables having a finite second moment is lost along with its tools and methodologies. This is indeed the case in estimation theory where classical tools to quantify performance of an estimator are tightly related to the assumption of finite variance variables. In alpha-stable environments, expressions such as the mean square error and the Cramer-Rao bound are hence problematic. In this work, we tackle the parameter estimation problem in impulsive noise environments and develop novel tools that are tailored to the alpha-stable and heavy-tailed noise environments, tools that coincide with the standard ones adopted in the Gaussian setup, namely a generalized "power" measure and a generalized Fisher information. We generalize known information inequalities commonly used in the Gaussian context: the de Bruijn's identity, the data processing inequality, the Fisher information inequality, the isoperimetric inequality for entropies and the Cramer-Rao bound. Additionally, we derive upper bounds on the differential entropy of independent sums having a stable component. Finally, the new "power" measure is used to shed some light on the additive alpha-stable noise channel capacity in a setup that generalizes the linear average power constrained AWGN channel. Our theoretical findings are paralleled with numerical evaluations of various quantities and bounds using developed {\em Matlab} packages., 42 pages, 5 figures, submitted to the IEEE transactions on information theory for peer review

Recently, Sati and Gupta (2015) have introduced a generalized cumulative residual entropy based on the non-additive Tsallis entropy. The cumulative residual entropy, introduced by Rao et al. (2004) is a generalized measure of uncertainty which is applied in reliability and image alignment and non-additive measures of entropy. This entropy finds justifications in many physical, biological and chemical phenomena. In this paper, we derive the weighted form of this measure and call it Weighted Cumulative Residual Tsallis Entropy (WCRTE). Being a “length-biased” shift-dependent information measure, WCRTE is related to the differential information in which higher weight is assigned to large values of observed random variables. Based on the dynamic version of this new information measure, we propose ageing classes and it is shown that it can uniquely determine the survival function and Rayleigh distribution. Several properties, including linear transformations, bounds and related results to stochastic orders are obtained for these measures. Also, we identify classes of distributions in which some well-known distributions are maximum dynamic version of WCRTE. The empirical WCRTE is finally proposed to estimate the new information measure.

In this study, the Hunt equation for computing suspension concentration in sediment-laden flows is derived using Shannon entropy theory. Considering the inverse of the void ratio as a random variable and using principle of maximum entropy, probability density function and cumulative distribution function of suspension concentration is derived. A new and more general cumulative distribution function for the flow domain is proposed which includes several specific other models of CDF reported in literature. This general form of cumulative distribution function also helps to derive the Rouse equation. The entropy based approach helps to estimate model parameters using suspension data of sediment concentration which shows the advantage of using entropy theory. Finally model parameters in the entropy based model are also expressed as functions of the Rouse number to establish a link between the parameters of the deterministic and probabilistic approaches.

Shannon entropy plays an important role in measuring the expected uncertainty contained in the probability density function about the predictability of an outcome of a random variable. However, in certain systems, Shannon entropy may not be appropriate, where some generalized versions of it are only suitable. One such generalization is due to Boekee and Lubee [1] , called R-norm entropy. Recently, Nanda and Das [2] studied the R-norm entropy and its divergence measure in the context of used items, useful in reliability modelling. In the present article, we further study R-norm entropy and divergence in the context of weighted models. We also extend these measures to the conditionally specified and conditional survival models, and studied their properties.

In this article, we propose high order, semi-discrete, entropy stable, finite difference schemes for Ten-Moment Gaussian Closure equations. The crucial components of these schemes are an entropy conservative flux and suitable high order entropy dissipative operators to ensure entropy stability. We design two numerical fluxes, one is approximately entropy conservative, and another is entropy conservative flux. For the construction of appropriate entropy dissipative operators, we also derive entropy scaled right eigenvectors. This is used for sign preserving reconstruction of scaled entropy variables, which results in second and third order entropy stable schemes. We also extend these schemes to a plasma flow model with source term. Several numerical results are presented for homogeneous and non-homogeneous cases to demonstrate stability and performance of these schemes.

Partial entropy is a device to measure how much of entropy of an uncertain random variable belongs to uncertain variables. However, partial entropy may fail to measure the uncertainty of an uncertain random variable. In this paper, for refining this problem, a definition of partial triangular entropy is presented. In addition, several properties of partial triangular entropy are obtained. Moreover, partial triangular entropy of an uncertain random variable is applied to mean-variance portfolio selection.

We propose a statistical monitoring method to detect the increase of entropy in categorical data. First, we propose a distribution estimation method to approximate the probability distribution of t...

We present a series of closed-form upper bounds of the differential entropy of univariate continuous distributions based on the maximum entropy principle. We apply those bounds to Gaussian mixture models, and study their tightness properties.

To overcome the drawbacks of a shift-independent information measure, Di Crescenzo and Longobardi (2006) proposed weighted differential entropy. Motivated by this, the cumulative residual entropy can be generalized to weighted cumulative residual entropy. In this communication, we consider the weighted cumulative residual entropy and its dynamic version. These measures are useful in neurology and reliability engineering. Several results including bounds, inequalities, effects of linear transformations are obtained. Further, we propose two new classes of distributions and study closure and reversed closure properties. Finally, dynamic weighted cumulative residual entropy of residual lifetime of a series system is studied.

In this paper, a type of entropy of action of a semi-group on a measure space is introduced. Then we apply this concept to specific semi-groups to extract metric entropy of a dynamical system as a special case.

In this paper a kernel estimator of the differential entropy of the mark distribution of a homogeneous Poisson marked point process is proposed. The marks have an absolutely continuous distribution on a compact Riemannian manifold without boundary. We investigate L2 and the almost surely consistency of this estimator as well as its asymptotic normality.

Two approaches to the entropy management of Gaussian stochastic system are considered. The first approach is scalar and implements the concept of "growth points". In this case the problem of maximizing (increasing) or minimizing (decreasing) the system entropy is solved. The second approach is the vector management, allowing to ensure effective changing of the entropy of two-dimensional vector, the components of which are randomness and self-organization entropies. For vector control an optimization problem on the conditional extremum is formulated. This problem can be solved using penalty methods. It is shown that the vector management of entropy for a number of cases has advantages compared to the scalar management. Examples of entropy models of real stochastic systems are provided.

Based on methodologies of variational calculus and differential entropy, we propose in this work a non-parametric model that provides a robust estimation of reliability to be used in a RCM scheme. A Weibull analysis is presented first in a case study within the usual RCM schemes. If the sample of data is severly reduced the weibull analysis lost preciscion, impacting in the RCM scheme. To solve this limitation, a maximum entropy aproach is propoused.Differential entropy has been shown as a solid tool to model the response of a random variable when reduced sample size information is available. We take advantage of the formalism of the variational calculus to express a functional that obeys the Euler-Lagrange equations and supported by the Kolmogorov axioms, we extract a generalized non-parametric probability density. By subjecting this density to appropriate boundary conditions in terms of the first moments of the generalized probability density.

While Shannon&rsquo, s differential entropy adequately quantifies a dimensioned random variable&rsquo, s information deficit under a given measurement system, the same cannot be said of differential weighted entropy in its existing formulation. We develop weighted and residual weighted entropies of a dimensioned quantity from their discrete summation origins, exploring the relationship between their absolute and differential forms, and thus derive a &ldquo, differentialized&rdquo, absolute entropy based on a chosen &ldquo, working granularity&rdquo, consistent with Buckingham&rsquo, s &Pi, theorem. We apply this formulation to three common continuous distributions: exponential, Gaussian, and gamma and consider policies for optimizing the working granularity.

In this work we analyze the entropic properties of the Euler equations when the system is closed with the assumption of a polytropic gas. In this case, the pressure solely depends upon the density of the fluid and the energy equation is not necessary anymore as the mass conservation and momentum conservation then form a closed system. Further, the total energy acts as a convex mathematical entropy function for the polytropic Euler equations. The polytropic equation of state gives the pressure as a scaled power law of the density in terms of the adiabatic index $$\gamma $$ γ . As such, there are important limiting cases contained within the polytropic model like the isothermal Euler equations ($$\gamma {=}1$$ γ = 1 ) and the shallow water equations ($$\gamma {=}2$$ γ = 2 ). We first mimic the continuous entropy analysis on the discrete level in a finite volume context to get special numerical flux functions. Next, these numerical fluxes are incorporated into a particular discontinuous Galerkin (DG) spectral element framework where derivatives are approximated with summation-by-parts operators. This guarantees a high-order accurate DG numerical approximation to the polytropic Euler equations that is also consistent to its auxiliary total energy behavior. Numerical examples are provided to verify the theoretical derivations, i.e., the entropic properties of the high order DG scheme.

In many rare disease Phase II clinical trials, two objectives are of interest to an investigator: maximising the statistical power and maximising the number of patients responding to the treatment. These two objectives are competing, therefore, clinical trial designs offering a balance between them are needed. Recently, it was argued that response-adaptive designs such as families of multi-arm bandit (MAB) methods could provide the means for achieving this balance. Furthermore, response-adaptive designs based on a concept of context-dependent (weighted) information criteria were recently proposed with a focus on Shannon's differential entropy. The information-theoretic designs based on the weighted Renyi, Tsallis and Fisher informations are also proposed. Due to built-in parameters of these novel designs, the balance between the statistical power and the number of patients that respond to the treatment can be tuned explicitly. The asymptotic properties of these measures are studied in order to construct intuitive criteria for arm selection. A comprehensive simulation study shows that using the exact criteria over asymptotic ones or using information measures with more parameters, namely Renyi and Tsallis entropies, brings no sufficient gain in terms of the power or proportion of patients allocated to superior treatments. The proposed designs based on information-theoretical criteria are compared to several alternative approaches. For example, via tuning of the built-in parameter, one can find designs with power comparable to the fixed equal randomisation's but a greater number of patients responded in the trials.