34 results on '"Differential entropy"'
Search Results
2. University of Messina Researchers Update Understanding of Engineering (A Novel Quality Assessment Method for the Clinical Reproduction of Orthodontic Attachments Based on Differential Entropy).
- Subjects
DIFFERENTIAL entropy ,RESEARCH personnel ,ENGINEERING ,ORTHODONTIC appliances ,DENTAL impressions - Abstract
Researchers from the University of Messina in Italy have developed a new technique for reproducing attachments during orthodontic treatment with clear aligners. The technique involves the use of a pre-drilled template and a high-viscosity composite. The researchers used digital scans to compare the clinical outcomes of the experimental technique with the standard technique, and found that the experimental technique offered good alignment in attachment placement. The researchers also developed a quality assessment method called CorAl, which allows for the drawing of quality maps that show areas with the highest deviation. [Extracted from the article]
- Published
- 2024
3. Yunnan Normal University Researchers Describe Advances in Signal Processing (EEG emotion recognition based on differential entropy feature matrix through 2D-CNN-LSTM network).
- Subjects
EMOTION recognition ,SIGNAL processing ,DIFFERENTIAL entropy ,RESEARCH personnel ,ELECTROENCEPHALOGRAPHY ,RECOGNITION (Psychology) - Abstract
Researchers from Yunnan Normal University have developed a new method for emotion recognition using electroencephalography (EEG) signals. Traditional methods of EEG-based emotion recognition do not consider the spatial correlation between electrodes, so the researchers proposed a method that combines a differential entropy feature matrix (DEFM) and a 2D-CNN-LSTM network. The DEFM captures the spatiotemporal correlation of the EEG signals, and the 2D-CNN-LSTM accurately identifies the emotional categories. The method achieved high classification accuracy in experiments and has potential applications in emotion classification and recognition based on EEG signals. [Extracted from the article]
- Published
- 2024
4. Privacy-preserving point-of-interest recommendation based on geographical and social influence.
- Author
-
Huo, Yongfeng, Chen, Bilian, Tang, Jing, and Zeng, Yifeng
- Subjects
- *
SOCIAL influence , *RECOMMENDER systems , *PRIVACY , *DIFFERENTIAL entropy , *SOCIAL status , *ALGORITHMS - Abstract
• We present an effective and adjustable privacy-preserving POI recommender system. • We provide fuzzy position and social relation techniques to preserve user's privacy. • The privacy-preserving property of the proposed methods is formally proved. • The proposals get both private and effective performance via empirically evaluation. We investigate a privacy-preserving problem for point-of-interest (POI) recommendation system for rapidly growing location-based social networks (LBSNs). The LBSN-based recommendation algorithms usually consider three factors: user similarity, social influence between friends and geographical influence in. The LBSN-based recommendation system first needs to collect relevant information of users and then provide them with potentially interesting contents. However, sensitive information of users may be leaked when the recommendation is provided. In this article, we focus on preventing user's privacy from disclosure upon geographical location and friend relationship factors. We propose a geographical location privacy-preserving algorithm (GLP) that achieves 〈 r , h 〉 -privacy and present a friend relationship privacy-preserving algorithm (FRP) through adding Laplacian distributed noise for fusing the user trusts. Subsequently, we integrate the GLP and FRP algorithms into a general recommendation system and build a privacy-preserving recommendation system. The novel system enjoys the privacy guarantee under the metric differential entropy through theoretical analysis. Experimental results demonstrate a good trade-off between privacy and accuracy of the proposed recommendation system. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
5. An automatic and parameter-free information-based method for sparse representation in wavelet bases.
- Author
-
Bruni, V., Della Cioppa, L., and Vitulano, D.
- Subjects
- *
DIFFERENTIAL entropy , *HILBERT functions , *INFORMATION measurement , *ABSOLUTE value , *THERMAL expansion - Abstract
In this article an information-based method for the selection of expansion coefficients of functions in a Hilbert basis is presented. An information-based measure, namely Entropic NID (ENID), is presented; the optimal separation point between more informative coefficients and less informative ones is selected by evaluating the information contribution of two competing sets of expansion coefficients. A consistent numerical scheme is given to approximate ENID and the numerical error is studied. Numerical simulations are provided to test the behaviour of ENID in different wavelet bases, as well as to perform comparative studies. • Sparse representation is achieved in wavelet bases. • Absolute value of differential entropy of a function is a measure of complexity. • An information measure based on Normalized Information Distance is proposed. • Optimum coefficients set is related via experiments to Vitanyi's NCD. [ABSTRACT FROM AUTHOR]
- Published
- 2020
- Full Text
- View/download PDF
6. Reports from Shandong University Highlight Recent Findings in Engineering (Eeg-based Familiar and Unfamiliar Face Classification Using Filter-bank Differential Entropy Features).
- Subjects
DIFFERENTIAL entropy ,RECOGNITION (Psychology) ,FACE perception ,ENGINEERING - Abstract
A recent study conducted at Shandong University in Jinan, China, explores the neural mechanism and electroencephalography (EEG) features involved in face recognition. The researchers propose a new method for classifying familiar and unfamiliar faces based on EEG signals. They use a filter-bank strategy to segment and filter the EEG signals, and employ the support vector machine (SVM) with Gaussian kernels as a classifier. The study demonstrates the feasibility of developing an efficient and interpretable brain-computer interface for EEG-based face recognition. [Extracted from the article]
- Published
- 2024
7. Rate-Cost Tradeoffs in Control.
- Author
-
Kostina, Victoria and Hassibi, Babak
- Subjects
- *
DIFFERENTIAL entropy , *STOCHASTIC systems , *VECTOR quantization , *DYNAMIC programming , *DATA compression , *STOCHASTIC control theory , *COST functions - Abstract
Consider a control problem with a communication channel connecting the observer of a linear stochastic system to the controller. The goal of the controller is to minimize a quadratic cost function in the state variables and control signal, known as the linear quadratic regulator (LQR). We study the fundamental tradeoff between the communication rate $r$ b/s and the expected cost $b$. We obtain a lower bound on a certain rate-cost function, which quantifies the minimum directed mutual information between the channel input and output that is compatible with a target LQR cost. The rate-cost function has operational significance in multiple scenarios of interest: among others, it allows us to lower-bound the minimum communication rate for fixed and variable length quantization, and for control over noisy channels. We derive an explicit lower bound to the rate-cost function, which applies to the vector, non-Gaussian, and partially observed systems, thereby extending and generalizing an earlier explicit expression for the scalar Gaussian system, due to Tatikonda et al. [S. Tatikonda, A. Sahai, and S. Mitter, “Stochastic linear control over a communication channel,” IEEE Trans. Autom. Control, vol. 49, no. 9, pp. 1549–1561, Sep. 2004.]. The bound applies as long as the differential entropy of the system noise is not $-\infty$. It can be closely approached by a simple lattice quantization scheme that only quantizes the innovation, that is, the difference between the controller's belief about the current state and the true state. Via a separation principle between control and communication, similar results hold for causal lossy compression of additive noise Markov sources. Apart from standard dynamic programming arguments, our technical approach leverages the Shannon lower bound, develops new estimates for data compression with coding memory, and uses some recent results on high resolution variable-length vector quantization to prove that the new converse bounds are tight. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF
8. Long-term speech information based threshold for voice activity detection in massive microphone network.
- Author
-
Zhu, Mengyao, Wu, Xiukun, Lu, Zhihua, Wang, Tao, and Zhu, Xiaoqiang
- Subjects
- *
MICROPHONES , *GAUSSIAN mixture models , *MICROPHONE arrays , *DIFFERENTIAL entropy , *HOME automation , *SPEECH - Abstract
Voice activity detection (VAD) is essential for multiple microphone arrays processing, in which massive potential devices, such as microphone devices for far-field voice-based interaction in smart home environments, will be activated when sound sources appear. Therefore, the VAD can save a lot of computing resources in massive microphone arrays processing for the sparsity in sound source activity. However, it may not be feasible to obtain an accurate VAD in harsh environments, such as far-field, time-varying noise field. In this paper, the long-term speech information (LTSI) and the log-energy are modeled for deriving a more accurate VAD. First, the LTSI can be obtained by measuring the differential entropy of long-term smoothed noisy signal spectrum. Then, the LTSI is used to get labeled data for the initialization of a Gaussian mixture model (GMM), which is used to fit the log-energy distribution of noise and (noisy) speech. Finally, combining the LTSI and the GMM parameters of noise and speech distribution, this paper derives an adaptive threshold, which represents a reasonable boundary between noise and speech. Experimental results show that our VAD method has a remarkable improvement for a massive microphone network. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF
9. Long-range memory, distributional variation and randomness of bitcoin volatility.
- Author
-
Lahmiri, Salim, Bekiros, Stelios, and Salvi, Antonio
- Subjects
- *
BITCOIN , *MARKET volatility , *TIME series analysis , *SKEWNESS (Probability theory) , *DIFFERENTIAL entropy , *MATHEMATICAL models - Abstract
We investigate the nonlinear patterns of volatility in seven Bitcoin markets. In particular, we explore the fractional long-range dependence in conjunction with the potential inherent stochasticity of volatility time series under four diverse distributional assumptions, i.e., Normal, Student- t , Generalized Error (GED), and t -Skewed distribution. Our empirical findings signify the existence of long-range memory in Bitcoin market volatility, irrespectively of distributional inference. The same applies to entropy measurement, which indicates a high degree of randomness in the estimated series. As Bitcoin markets are highly disordered and risky, they cannot be considered suitable for hedging purposes. Our results provide strong evidence against the efficient market hypothesis. [ABSTRACT FROM AUTHOR]
- Published
- 2018
- Full Text
- View/download PDF
10. Study Results from Shanghai University in the Area of Computer Science Published (Robust zero-watermarking scheme using DT CWT and improved differential entropy for color medical images).
- Subjects
COMPUTER science ,DIFFERENTIAL entropy ,SCIENTIFIC computing ,DIAGNOSTIC imaging ,SCIENCE publishing - Abstract
Keywords: Computer Science; Computers; Legal Issues EN Computer Science Computers Legal Issues 7124 7124 1 09/19/23 20230922 NES 230922 2023 SEP 22 (NewsRx) -- By a News Reporter-Staff News Editor at Health & Medicine Week -- Data detailed on computer science have been presented. Keywords for this news article include: Shanghai University, Shanghai, People's Republic of China, Asia, Legal Issues, Computer Science. [Extracted from the article]
- Published
- 2023
11. Distribution spread and location metrics using entropic separation.
- Author
-
Bowden, Roger J.
- Subjects
- *
DISTRIBUTION (Probability theory) , *PARTITIONS (Mathematics) , *ROBUST statistics , *DIFFERENTIAL entropy , *BINARY number system - Abstract
Distribution spread as a separation width is formalised using partition entropy, resulting in a metric for dispersion with associated end points. The binary framework resolves interpretive and other limitations of the traditional differential entropy metric, and is robust to distributional shape. [ABSTRACT FROM AUTHOR]
- Published
- 2017
- Full Text
- View/download PDF
12. Performance Analysis of Gradient-Based Nash Seeking Algorithms Under Quantization.
- Author
-
Nekouei, Ehsan, Nair, Girish N., and Alpcan, Tansu
- Subjects
- *
NASH equilibrium , *PERFORMANCE evaluation , *SIGNAL quantization , *MATHEMATICAL bounds , *PROBABILITY theory - Abstract
This paper investigates the impact of quantized inter-agent communications on the asymptotic and transient behavior of gradient-based Nash-seeking algorithms in non-cooperative games. Using the information-theoretic notion of entropy power, we establish a universal lower bound on the asymptotic rate of exponential mean-square convergence to the Nash equilibrium (NE). This bound depends on the inter-agent data rate and the local behavior of the agents' utility functions, and is independent of the quantizer structure. Next, we study transient performance and derive an upper bound on the average time required to settle inside a specified ball around the NE, under uniform quantization. Furthermore, we establish an upper bound on the probability that agents' actions lie outside this ball, and show that this bound decays double-exponentially with time. Finally, we propose an adaptive quantization scheme that allows the gradient algorithm to converge to the NE despite quantized inter-agent communications. [ABSTRACT FROM AUTHOR]
- Published
- 2016
- Full Text
- View/download PDF
13. Deng entropy.
- Author
-
Deng, Yong
- Subjects
- *
ENTROPY (Information theory) , *ERGODIC theory , *EVIDENCE-based management , *INFORMATION theory , *DIFFERENTIAL entropy - Abstract
Dempster Shafer evidence theory has been widely used in many applications due to its advantages to handle uncertainty. However, how to measure uncertainty in evidence theory is still an open issue. The main contribution of this paper is that a new entropy, named as Deng entropy, is presented to measure the uncertainty of a basic probability assignment (BPA). Deng entropy is the generalization of Shannon entropy since the value of Deng entropy is identical to that of Shannon entropy when the BPA defines a probability measure. Numerical examples are illustrated to show the efficiency of Deng entropy. [ABSTRACT FROM AUTHOR]
- Published
- 2016
- Full Text
- View/download PDF
14. Tribological Behavior of AlCoCrFeNi(Ti0.5) High Entropy Alloys under Oil and MACs Lubrication.
- Author
-
Yu, Yuan, Wang, Jun, Li, Jinshan, Yang, Jun, Kou, Hongchao, and Liu, Weimin
- Subjects
TRIBOLOGY ,ALLOYS ,DIFFERENTIAL entropy ,OIL mist lubrication ,HEAT treatment ,DENDRITIC crystals ,WEAR resistance ,DELAMINATION of composite materials - Abstract
The tribological properties of AlCoCrFeNi and AlCoCrFeNiTi 0.5 high entropy alloys under gear oil and multiply alkylated cyclopentanes (MACs) lubrication condition have been studied. The equiaxed crystal structure of AlCoCrFeNi alloy is obtained after heat-treatment. The AlCoCrFeNiTi 0.5 alloy keeps dendrite structure. Under the gear oil with good lubrication action, AlCoCrFeNiTi 0.5 alloy preserves better tribological properties than AlCoCrFeNi alloy. The delamination and crack behaviors tend to occur in the grain boundary of AlCoCrFeNi alloy and along the interdendrite region of AlCoCrFeNiTi 0.5 alloy. Under the MACs with relatively poor lubrication action, the applied load slightly influences the wear behavior of AlCoCrFeNi alloy, but seriously impacts the wear mechanism of AlCoCrFeNiTi 0.5 alloy. Compared with AlCoCrFeNi alloy, AlCoCrFeNiTi 0.5 alloy keeps better wear-resistance at low applied load of 100 N, but preserves less wear-resistance at high applied load of 200 N. [ABSTRACT FROM AUTHOR]
- Published
- 2016
- Full Text
- View/download PDF
15. Robust voice activity detection based on weighted average of long-term quadratic Renyi and differential entropies.
- Author
-
Muralishankar, R., Ghosh, Debayan, and Gurugopinath, Sanjeev
- Subjects
- *
DIFFERENTIAL entropy , *QUADRATIC differentials , *RENYI'S entropy , *AUTOMATIC speech recognition , *LIKELIHOOD ratio tests , *SIGNAL sampling - Abstract
We propose two novel voice activity detection (VAD) algorithms by employing i) the Rényi entropy estimate, and ii) a weighted combination of Rényi and differential entropies, at each frequency bin of power spectral estimates, by considering overlapping frames in the received signal samples. The Bartlett-Welch method is employed to estimate the power spectrum from a suitably chosen number of past frames in the frequency domain. Exploiting the known fact that a long-term averaging enhances the performance of VAD, we first propose a novel frequency domain long-term quadratic Rényi entropy (FLQRE) feature, which is extracted by summing the estimated Rényi entropy values across the frequency bins over each signal frame. Later, we propose a weighted combination of the frequency domain long-term differential entropy (FLDE) and FLQRE feature for VAD, termed as a weighted quadratic and differential entropy (WQDE) feature. We evaluate the performance of the proposed FLQRE and WQDE VAD schemes, by considering twelve types of noises from NOISEX-92 at five different SNR values, artificially added to speech samples from SWITCHBOARD and TIMIT corpora. We present an extensive performance comparison study to establish the utility of our proposed VAD algorithms over state-of-the-art short- and long-term VAD techniques such as ITU-T G.729B, likelihood ratio test, long-term signal variability, and long-term spectral flatness measure-based algorithms. We show that the proposed algorithms yield the best average accuracy and noise-hit rate under the SWITCHBOARD corpus, and yield a comparable performance under the TIMIT corpus. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
16. Dynamics of non-stationary nonlinear processes that follow the maximum of differential entropy principle.
- Author
-
Fradkov, Alexander L. and Shalymov, Dmitry S.
- Subjects
- *
NONLINEAR differential equations , *DIFFERENTIAL entropy , *MAXIMUM entropy method , *PROBABILITY density function , *ENERGY conservation , *STOCHASTIC convergence - Abstract
Dynamics of non-stationary nonlinear processes that follow the maximum entropy principle (MaxEnt) for continuous probability distributions is considered. A set of equations describing the dynamics of probability density function (pdf) under the mass conservation and the energy conservation constraints is derived. The asymptotic convergence of evolving pdf and convergence of differential entropy are examined. It is shown that for pdfs with compact carrier the limit pdf is unique and coincides with the pdf obtained from Jaynes’s MaxEnt principle. [ABSTRACT FROM AUTHOR]
- Published
- 2015
- Full Text
- View/download PDF
17. Non-parametric entropy estimators based on simple linear regression.
- Author
-
Hino, Hideitsu, Koshijima, Kensuke, and Murata, Noboru
- Subjects
- *
NONPARAMETRIC estimation , *DIFFERENTIAL entropy , *REGRESSION analysis , *DENSITY functional theory , *PROBABILITY theory , *PROBABILITY density function - Abstract
Estimators for differential entropy are proposed. The estimators are based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Simple linear regression is utilized to estimate the values of density function and its second derivative at a point. After estimating the values of the probability density function at each of the given sample points, by taking the empirical average of the negative logarithm of the density estimates, two entropy estimators are derived. Other entropy estimators which directly estimate entropy by linear regression, are also proposed. The proposed four estimators are shown to perform well through numerical experiments for various probability distributions. [ABSTRACT FROM AUTHOR]
- Published
- 2015
- Full Text
- View/download PDF
18. Rényi entropy and divergence for VARFIMA processes based on characteristic and impulse response functions.
- Author
-
Contreras-Reyes, Javier E.
- Subjects
- *
RENYI'S entropy , *IMPULSE response , *CHARACTERISTIC functions , *DIFFERENTIAL entropy , *INFORMATION measurement , *DISTRIBUTED parameter systems , *MOVING average process - Abstract
Rényi entropy based on characteristic function has been used as an information measure contained in wide-sense and real stationary vector autoregressive and moving average (VARMA) processes. These classes of processes have been extended by fractionally integrated VARMA (VARFIMA) ones, composed of a VARMA process, a vector of fractional differencing parameters, and independent and identically distributed multivariate normal random errors. Such processes have often been used to explicitly account for persistence to incorporate long-term correlations into multivariate data. The purpose of this paper is to extend Rényi entropy from VARMA to VARFIMA processes, addressing long-memory behavior of time series by adding a fractional differencing parameter. The characteristic function of the process can be derived directly from the asymptotic form of the impulse response function using the Wold representation. Then, assuming multivariate Gaussian white noise with known fractional differencing, autoregressive and moving average matrix parameters, the differential and Rényi entropies and Kullback–Leibler and Rényi divergences were obtained by evaluating the variance-covariance matrix identified with VARFIMA process distribution. The influences of the fractional differencing parameters on the Rényi entropy increment were analyzed, as were comparisons between VARFIMA processes using the Kullback–Leibler and Rényi divergences. Finally, numerical examples and an application to U.S. daily temperature time series are presented. • Rényi entropy has been used as a measure of information contained in VARFIMA processes. • VARFIMA processes account for persistence to incorporate the long-term correlations in the multivariate data. • We extend the Rényi entropy from VARMA to VARFIMA processes by adding a fractional differencing parameter. • Characteristic function is derived directly from the asymptotic form of the impulse response function. • Numerical examples and an application to U.S. daily temperature time series are presented. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
19. Entanglement entropy for a Maxwell field: Numerical calculation on a two-dimensional lattice.
- Author
-
Casini, Horatio and Huerta, Marina
- Subjects
- *
MAXWELL equations , *DIFFERENTIAL entropy , *LATTICE theory , *GAUSSIAN processes , *COMMUTATORS (Operator theory) - Abstract
We study entanglement entropy (EE) for a Maxwell field in (2+1) dimensions. We do numerical calculations in two-dimensional lattices. This gives a concrete example of the general results of our recent work [1] on entropy for lattice gauge fields using an algebraic approach. To evaluate the entropies we extend the standard calculation methods for the entropy of Gaussian states in canonical commutation algebras to the more general case of algebras with center and arbitrary numerical commutators. We find that while the entropy depends on the details of the algebra choice, mutual information has a well defined continuum limit as predicted in [1]. We study several universal terms for the entropy of the Maxwell field and compare with the case of a massless scalar field. We find some interesting new phenomena: an "evanescent" logarithmically divergent term in the entropy with topological coefficient which does not have any correspondence with ultraviolet entanglement in the universal quantities, and a nonstandard way in which strong subadditivity is realized. Based on the results of our calculations we propose a generalization of strong subadditivity for the entropy on some algebras that are not in tensor product. [ABSTRACT FROM AUTHOR]
- Published
- 2014
- Full Text
- View/download PDF
20. Negentropy as a function of cumulants.
- Author
-
Withers, Christopher S. and Nadarajah, Saralees
- Subjects
- *
CUMULANTS , *DIFFERENTIAL entropy , *MATHEMATICAL functions , *AUTOCORRELATION (Statistics) , *DISTRIBUTION (Probability theory) - Abstract
Abstract: Suppose f is a density that is close to that of a standard normal, . For the first time, exact expressions are given for the negentropy, the differential entropy, and related quantities such as for in terms of cumulants. Several advantages of our expressions over known ones are pointed out. [Copyright &y& Elsevier]
- Published
- 2014
- Full Text
- View/download PDF
21. Bulk curves from boundary data in holography.
- Author
-
Balasubramanian, Vijay, Chowdhury, Boran D., Czech, Bartlomiej, de Boer, Jan, and Heller, Michal P.
- Subjects
- *
HOLOGRAPHY , *SPACETIME , *FIELD theory (Physics) , *DIFFERENTIAL entropy , *RENORMALIZATION group , *MINKOWSKI space - Abstract
We embed spherical Rindler space--a geometry with a spherical hole in its center--in asymptotically anti- de Sitter (AdS) spacetime and show that it carries a gravitational entropy proportional to the area of the hole. Spherical AdS-Rindler space is holographically dual to an ultraviolet sector of the boundary field theory given by restriction to a strip of finite duration in time. Because measurements have finite durations, local observers in the field theory can only access information about bounded spatial regions. We propose a notion of differential entropy that captures uncertainty about the state of a system left by the collection of local, finite-time observables. For two-dimensional conformai field theories we use holography and the strong subadditivity of entanglement to propose a formula for differential entropy and show that it precisely reproduces the areas of circular holes in AdS3. Extending the notion to field theories on strips with variable durations in time, we show more generally that differential entropy computes the areas of all closed, inhomogeneous curves on a spatial slice of AdS3. We discuss the extension to higher-dimensional field theories, the relation of differential entropy to entanglement between scales, and some implications for the emergence of space from the renormalization group flow of entangled field theories. [ABSTRACT FROM AUTHOR]
- Published
- 2014
- Full Text
- View/download PDF
22. The memory center.
- Author
-
Spurek, P. and Tabor, J.
- Subjects
- *
COMPUTER storage devices , *BINARY codes , *PROBLEM solving , *RANDOM variables , *CLUSTER analysis (Statistics) , *COMPUTER networks - Abstract
Abstract: As we know, the length of binary code of a point (with accuracy h >0) is approximately . We will consider the problem where we should translate the origin a of the coordinate system so that the mean amount of bits needed to code a randomly chosen element from a realization of a random variable X is minimal. In other words, we want to find such that attains minimum. We show that under reasonable assumptions the choice of a does not depend on h asymptotically. Consequently, we reduce the problem to finding the minimum of function where f is the density distribution of the random variable X. Moreover, we provide constructive approach for determining a. [Copyright &y& Elsevier]
- Published
- 2013
- Full Text
- View/download PDF
23. Compressive system identification: Sequential methods and entropy bounds
- Author
-
Carmi, Avishy Y.
- Subjects
- *
COMPRESSED sensing , *ENTROPY , *SEQUENTIAL analysis , *KALMAN filtering , *COEFFICIENTS (Statistics) , *AUTOREGRESSION (Statistics) , *ESTIMATION theory - Abstract
Abstract: In the first part of this work, a novel Kalman filtering-based method is introduced for estimating the coefficients of sparse, or more broadly, compressible autoregressive models using fewer observations than normally required. By virtue of its (unscented) Kalman filter mechanism, the derived method essentially addresses the main difficulties attributed to the underlying estimation problem. In particular, it facilitates sequential processing of observations and is shown to attain a good recovery performance, particularly under substantial deviations from ideal conditions, those which are assumed to hold true by the theory of compressive sensing. In the remaining part of this paper we derive a few information-theoretic bounds pertaining to the problem at hand. The obtained bounds establish the relation between the complexity of the autoregressive process and the attainable estimation accuracy through the use of a novel measure of complexity. This measure is used in this work as a substitute to the generally incomputable restricted isometric property. [Copyright &y& Elsevier]
- Published
- 2013
- Full Text
- View/download PDF
24. Parity projection of QCD sum rules for the nucleon.
- Author
-
Ohtani, Keisuke, Gubler, Philipp, and Oka, Makoto
- Subjects
- *
QUANTUM chromodynamics , *PARTICLES (Nuclear physics) , *DIFFERENTIAL entropy , *GAUSSIAN distribution , *KERNEL functions - Abstract
The nucleon and its negative-parity excited states are examined in a maximum entropy method analysis of QCD sum rules. First, we rederive the parity projected sum rules for baryons using "old-fashioned" correlation functions. Doing this, the method is generalized so that higher order operator product expansion terms can be calculated unambiguously. We then apply this approach to the nucleon channel taking into account all known first order as corrections to the Wilson coefficients of the operator product expansion. As these corrections have turned out to be large, we suppress them by using a phase-rotated Gaussian kernel. Simultaneously, this phase rotation strongly suppresses the continuum contribution and improves the convergence of the operator product expansion. The resulting sum rule has the interesting feature that it is dominated by the term containing the chiral condensate of dimension 3. Analyzing this sum rule by the maximum entropy method, we are able to extract information of both the positive- and negative-parity states. [ABSTRACT FROM AUTHOR]
- Published
- 2013
- Full Text
- View/download PDF
25. On pressure and corner boundary conditions with two lattice Boltzmann construction approaches
- Author
-
Machado, Raúl
- Subjects
- *
BOUNDARY value problems , *LATTICE Boltzmann methods , *POISEUILLE flow , *DISTRIBUTION (Probability theory) , *INTERPOLATION , *COLLISIONS (Physics) , *DIFFERENTIAL entropy , *INCOMPRESSIBLE flow - Abstract
Abstract: A pressure (density) and a corner no-slip boundary condition formulation are introduced for the two-dimensional lattice Boltzmann method and numerically tested for the Hagen–Poiseuille flow case in this work. Both formulations are derived independently of the equilibrium distribution function model and no bounce-back rule is needed. The corners are free of interpolations and are implemented with collisions. The Hermite-based and the “entropy”-based lattice Boltzmann constructions are reviewed, described in some details and compared. The best two models from each approach are chosen. An indirect substitution of the boundary values in the equilibrium distribution function is further expanded to its “entropic” counterpart. For the straight walls, two existing boundary conditions with first and second order of accuracy are selected. The numerical results demonstrate that the presented boundary conditions are capable to preserve up to the second order of accuracy, depending on the straight walls. Based on the theoretical and numerical results, the so called Hermite construction is the recommended approach for isothermal incompressible flows. [Copyright &y& Elsevier]
- Published
- 2012
- Full Text
- View/download PDF
26. Complex Independent Component Analysis by Entropy Bound Minimization.
- Author
-
Xi-Lin Li and Adalı, Tülay
- Subjects
- *
ENTROPY , *RANDOM variables , *ARTIFICIAL neural networks , *ESTIMATION theory , *COMBINATORICS - Abstract
We first present a new (differential) entropy estimator for complex random variables by approximating the entropy estimate using a numerically computed maximum entropy bound. The associated maximum entropy distributions belong to the class of weighted linear combinations and elliptical distributions, and together, they provide a rich array of bivariate distributions for density matching. Next, we introduce a new complex independent component analysis (ICA) algorithm, complex ICA by entropy-bound minimization (complex ICA-EBM), using this new entropy estimator and a line search optimization procedure. We present simulation results to demonstrate the superior separation performance and computational efficiency of complex ICA-EBM in separation of complex sources that come from a wide range of bivariate distributions. [ABSTRACT FROM AUTHOR]
- Published
- 2010
- Full Text
- View/download PDF
27. Feedback Control in the Presence of Noisy Channels: "Bode-Like" Fundamental Limitations of Performance.
- Author
-
Martins, Nuno C. and Dahleh, Munther A.
- Subjects
- *
CONSERVATION laws (Mathematics) , *FLUX (Metallurgy) , *ENTROPY , *UNITS of time , *INFORMATION measurement , *MATHEMATICAL inequalities , *PARAMETER estimation , *ERGODIC theory , *THERMODYNAMICS - Abstract
This paper addresses fundamental limitations of feedback using information theoretic conservation laws and flux arguments. The paper has two parts. In the first part, we derive a conservation law dictating that causal feedback cannot reduce the differential entropy inserted in the loop by external sources. An interpretation of this result is that the total randomness induced by disturbances, as measured by differential entropy, cannot be reduced by causal feedback; it can only be re-allocated in time or in frequency (if well defined). Under asymptotic stationarity assumptions, this result has a spectral representation which constitutes an extension of Bode's inequality for arbitrary feedback. Our proofs make clear the role of causality, as well as how stability assumptions impact the final result. In the second part, we derive an inequality unveiling that the feedback loop must be able to convey information originating from two independent sources: 1) initial states of the physical plant; 2) exogenous disturbance signals. By using such principle, we construct a variety of information rate (information flux) inequalities. Furthermore, we derive a universal performance bound which is parameterized solely by the feedback capacity and the parameters of the plant. The latter is a new fundamental limitation, which is different from Bode's classical result, indicating that finite feedback capacity brings a new type of performance bound. [ABSTRACT FROM AUTHOR]
- Published
- 2008
- Full Text
- View/download PDF
28. On the estimation of random unobserved signals by maximization of target likelihoods and its application to blind timing and phase recovery
- Author
-
Míguez, Joaquín and Bugallo, Mónica F.
- Subjects
- *
STOCHASTIC processes , *DIGITAL signal processing , *SIGNAL processing , *INFORMATION measurement - Abstract
Abstract: Many important problems in signal processing can be reduced to the adequate selection of the parameters of a (possibly nonlinear) filter in order to obtain an output signal that complies with some desired properties. In this work, we analyze a novel criterion for selecting filter parameters that relies on the ability to characterize the desired filter output in terms of a target probability density function (pdf). This target pdf can be handled as a likelihood function to be maximized, thus we refer to the new criterion as maximum target-likelihood (MTL). We present a very general signal model where the MTL criterion can be applied and derive necessary and sufficient conditions for asymptotic convergence of the method. The relationship and differences between MTL and standard maximum likelihood (ML), minimum Kullback–Leibler divergence (MKLD), and minimum entropy (ME) methods are explored. Finally, as an example, we apply the novel criterion to the problem of blind timing and phase recovery in a digital transmission system and show that the resulting algorithm is competitive with existing non-data-aided ML-based algorithms. [Copyright &y& Elsevier]
- Published
- 2005
- Full Text
- View/download PDF
29. Response adaptive designs for Phase II trials with binary endpoint based on context-dependent information measures.
- Author
-
Kasianova, Ksenia, Kelbert, Mark, and Mozgunov, Pavel
- Subjects
- *
INFORMATION measurement , *DIFFERENTIAL entropy , *EXPERIMENTAL design , *STATISTICAL power analysis , *FISHER information - Abstract
In many rare disease Phase II clinical trials, two objectives are of interest to an investigator: maximising the statistical power and maximising the number of patients responding to the treatment. These two objectives are competing, therefore, clinical trial designs offering a balance between them are needed. Recently, it was argued that response-adaptive designs such as families of multi-arm bandit (MAB) methods could provide the means for achieving this balance. Furthermore, response-adaptive designs based on a concept of context-dependent (weighted) information criteria were recently proposed with a focus on Shannon's differential entropy. The information-theoretic designs based on the weighted Renyi, Tsallis and Fisher informations are also proposed. Due to built-in parameters of these novel designs, the balance between the statistical power and the number of patients that respond to the treatment can be tuned explicitly. The asymptotic properties of these measures are studied in order to construct intuitive criteria for arm selection. A comprehensive simulation study shows that using the exact criteria over asymptotic ones or using information measures with more parameters, namely Renyi and Tsallis entropies, brings no sufficient gain in terms of the power or proportion of patients allocated to superior treatments. The proposed designs based on information-theoretical criteria are compared to several alternative approaches. For example, via tuning of the built-in parameter, one can find designs with power comparable to the fixed equal randomisation's but a greater number of patients responded in the trials. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
30. Information entropy in AdS/QCD: Mass spectroscopy of isovector mesons.
- Author
-
da Rocha, R.
- Subjects
- *
MASS spectrometry , *QUANTUM chromodynamics , *VECTOR mesons , *ENTROPY , *DIFFERENTIAL entropy , *MESONS , *ADVERTISING - Abstract
The mass spectra of isovector Υ, ψ, ϕ, and ω meson resonances are investigated, in the AdS/QCD and information entropy setups. The differential configurational entropy is employed to obtain the mass spectra of radial S-wave resonances, with higher excitation levels, in each one of these meson families, whose respective first undisclosed states are discussed and matched up to candidates in the Particle Data Group. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
31. Entropic force between two horizons of a charged Gauss-Bonnet black hole in de Sitter spacetime.
- Author
-
Xiong-Ying Guo, Ying Gao, Huai-Fan Li, and Ren Zhao
- Subjects
- *
SPACETIME , *HORIZON , *DIFFERENTIAL entropy , *DIFFERENTIAL equations , *BLACK holes , *SCHWARZSCHILD black holes , *ENTROPY - Abstract
The basic equations of the thermodynamic system give the relationship between the internal energy, entropy and volume of two neighboring equilibrium states. By using the functional relationship between the state parameters in the basic equation, we give the differential equation satisfied by the entropy of spacetime. We can obtain the entropy by solving the differential equation. This entropy is the sum of entropy corresponding to the two event horizons and the interaction term. The interaction term is the ratio between the location of the black hole horizon and the cosmological horizon. The entropic force, which is strikingly similar to the Lennard-Jones force between particles, varies with the ratio of the two event horizons. The discovery of this phenomenon makes us realize that the entropic force between the two horizons may be one of the candidates to promote the expansion of the universe. [ABSTRACT FROM AUTHOR]
- Published
- 2020
- Full Text
- View/download PDF
32. Configuration entropy for quarkonium in a finite density plasma.
- Author
-
Braga, Nelson R. F. and da Mata, Rodrigo
- Subjects
- *
PLASMA density , *ENTROPY , *DEGREES of freedom , *CHARMONIUM , *DIFFERENTIAL entropy , *VECTOR mesons , *CHEMICAL potential - Abstract
In recent years, many examples appeared in the literature where the configuration entropy (CE), introduced by Gleiser and Stamatopoulos, plays the role of an indicator of stability of physical systems. It was observed that, comparing states of the same system, the lower is the value of the CE, the more stable is the state. In this work, we investigate the behavior of the differential configuration entropy (DCE), that is appropriate for systems with continuous degrees of freedom, in a new context. We consider quasistates of quarkonium (a vector meson made of a heavy quark antiquark pair) inside a plasma at finite density. It is known that the density increases the dissociation effect for quasiparticles inside a plasma. So, increasing the density of a thermal medium corresponds to reducing the stability of the quasiparticles. In order to investigate how this situation is translated in the configuration entropy context, we use a recently developed holographic anti-de Sitter/QCD model for heavy vector mesons. The quasinormal modes describing the quasistates are obtained and the corresponding DCE is calculated. We find, for bottomonium and charmonium 1S quasi-states, that the DCE increases with the quark density, or quark chemical potential, of the medium. This result shows that the DCE works again as an indicator of stability, represented in this case by the dissociation effect associated with the density. [ABSTRACT FROM AUTHOR]
- Published
- 2020
- Full Text
- View/download PDF
33. Fractional cumulative residual entropy.
- Author
-
Xiong, Hui, Shang, Pengjian, and Zhang, Yali
- Subjects
- *
CENTRAL limit theorem , *ENTROPY (Information theory) , *DIFFERENTIAL entropy , *INFORMATION measurement , *STATISTICAL sampling , *CUSUM technique - Abstract
• We generate the CRE to fractional order, named fractional CRE. • Some properties of the fractional CRE are presented. • The validity of fractional CRE is supported by numerical simulation and practical application. • The fractional CRE has inherited the merits of both CRE and fractional entropy. Recently, cumulative residual entropy was proposed as an alternative measure of information to Shannon entropy. In this work, we generate the cumulative residual entropy (CRE) to the case of fractional order, named fractional CRE. Some properties of the new quantity are presented. The connections of fractional CRE to the CRE and classic differential entropy are studied. Besides, we show that the proposed information measure can be estimated by the empirical fractional CRE of sample data. A central limit theorem for the empirical fractional CRE for random samples from the exponential distribution is derived. Its property of stability is also discussed. Finally, simulations on logistic map and application in financial data are given to support the validity of fractional CRE. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF
34. Entanglement entropy in inhomogeneous quenches in AdS3/CFT2.
- Author
-
De Jonckheere, Tim and Lindgren, Jonathan
- Subjects
- *
DIFFERENTIAL entropy , *HOLOGRAPHY , *SCHWARZSCHILD black holes - Abstract
We compute entanglement entropy and differential entropy in inhomogeneous holographic quenches in AdS3/CFT2. The quenches are arbitrarily inhomogeneous and modeled by an infalling shell of massless nonrotating matter where the final state is not dual to a static black hole but rather to a black hole with time-dependent stress-energy tensor modes. We study the entanglement entropy of an interval and differential entropy of a family of intervals analytically when the inhomogeneities have a perturbative amplitude and numerically for nonperturbative inhomogeneities. While we are in principle able to study these quantities for any inhomogeneities, we discuss two concrete examples: an oscillatory quench and a bilocal quench. Both cases display saturation towards a steady state but do not fully thermalize. Depending on the location and size of the interval, the entanglement entropy displays a variety of interesting phenomena such as plateau phases, bumps, and discontinuities in its first derivative with respect to time. [ABSTRACT FROM AUTHOR]
- Published
- 2018
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.