Search

A new information theoretic condition is presented for reconstructing a discrete random variable X based on the knowledge of a set of discrete functions of X. The reconstruction condition is derived from Shannon's 1953 lattice theory with two entropic metrics of Shannon and Rajski. Because such a theoretical material is relatively unknown and appears quite dispersed in different references, we first provide a synthetic description (with complete proofs) of its concepts, such as total, common, and complementary information. The definitions and properties of the two entropic metrics are also fully detailed and shown to be compatible with the lattice structure. A new geometric interpretation of such a lattice structure is then investigated, which leads to a necessary (and sometimes sufficient) condition for reconstructing the discrete random variable X given a set { X 1 , ... , X n } of elements in the lattice generated by X. Intuitively, the components X 1 , ... , X n of the original source of information X should not be globally "too far away" from X in the entropic distance in order that X is reconstructable. In other words, these components should not overall have too low of a dependence on X; otherwise, reconstruction is impossible. These geometric considerations constitute a starting point for a possible novel "perfect reconstruction theory", which needs to be further investigated and improved along these lines. Finally, this condition is illustrated in five specific examples of perfect reconstruction problems: the reconstruction of a symmetric random variable from the knowledge of its sign and absolute value, the reconstruction of a word from a set of linear combinations, the reconstruction of an integer from its prime signature (fundamental theorem of arithmetic) and from its remainders modulo a set of coprime integers (Chinese remainder theorem), and the reconstruction of the sorting permutation of a list from a minimal set of pairwise comparisons. [ABSTRACT FROM AUTHOR]

Using majorization theory via "Robin Hood" elementary operations, optimal lower and upper bounds are derived on Rényi and guessing entropies with respect to either error probability (yielding reverse-Fano and Fano inequalities) or total variation distance to the uniform (yielding reverse-Pinsker and Pinsker inequalities). This gives a general picture of how the notion of randomness can be measured in many areas of computer science. [ABSTRACT FROM AUTHOR]

A finite metric space is called here distance degree regular if its distance degree sequence is the same for every vertex. A notion of designs in such spaces is introduced that generalizes that of designs in Q-polynomial distance-regular graphs. An approximation of their cumulative distribution function, based on the notion of Christoffel function in approximation theory is given. As an application we derive limit laws on the weight distributions of binary orthogonal arrays of strength going to infinity. An analogous result for combinatorial designs of strength going to infinity is given. [ABSTRACT FROM AUTHOR]

Trajectories in human aimed movements are inherently variable. Using the concept of positional variance profiles, such trajectories are shown to be decomposable into two phases: In a first phase, the variance of the limb position over many trajectories increases rapidly; in a second phase, it then decreases steadily. A new theoretical model, where the aiming task is seen as a Shannon-like communication problem, is developed to describe the second phase: Information is transmitted from a "source" (determined by the position at the end of the first phase) to a "destination" (the movement's end-point) over a "channel" perturbed by Gaussian noise, with the presence of a noiseless feedback link. Information-theoretic considerations show that the positional variance decreases exponentially with a rate equal to the channel capacity C. Two existing datasets for simple pointing tasks are re-analyzed and observations on real data confirm our model. The first phase has constant duration, and C is found constant across instructions and task parameters, which thus characterizes the participant's performance. Our model provides a clear understanding of the speed-accuracy tradeoff in aimed movements: Since the participant's capacity is fixed, a higher prescribed accuracy necessarily requires a longer second phase resulting in an increased overall movement time. The well-known Fitts' law is also recovered using this approach. [ABSTRACT FROM AUTHOR]

Yet another simple proof of the entropy power inequality is given, which avoids both the integration over a path of Gaussian perturbation and the use of Young’s inequality with sharp constant or Rényi entropies. The proof is based on a simple change of variables, is formally identical in one and several dimensions, and easily settles the equality case. [ABSTRACT FROM AUTHOR]

While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, up to now Shannon's entropy power inequality (EPI) is an exception: Existing information theoretic proofs of the EPI hinge on representations of differential entropy using either Fisher information or minimum mean-square error (MMSE), which are derived from de Bruijn's identity. In this paper, we first present an unified view of these proofs, showing that they share two essential ingredients: 1) a data processing argument applied to a covariance-preserving linear transformation; 2) an integration over a path of a continuous Gaussian perturbation. Using these ingredients, we develop a new and brief proof of the EPI through a mutual information inequality, which replaces Stam and Blachman's Fisher information inequality (FII) and an inequality for MMSE by Guo, Shamai, and Verdú used in earlier proofs. The result has the advantage of being very simple in that it relies only on the basic properties of mutual information. These ideas are then generalized to various extended versions of the EPI: Zamir and Feder's generalized EPI for linear transformations of the random variables, Takano and Johnson's EPI for dependent variables, Liu and Viswanath's covariance-constrained EPI, and Costa's concavity inequality for the entropy power. [ABSTRACT FROM AUTHOR]

Following a recent proof of Shannon's entropy power inequality (EPI), a comprehensive framework for deriving various EPIs for the Rényi entropy is presented that uses transport arguments from normal densities and a change of variable by rotation. Simple arguments are given to recover the previously known Rényi EPIs and derive new ones, by unifying a multiplicative form with constant c and a modification with exponent α of previous works. In particular, for log-concave densities, we obtain a simple transportation proof of a sharp varentropy bound. [ABSTRACT FROM AUTHOR]

In a theoretical context of side-channel attacks, optimal bounds between success rate, guessing entropy and statistical distance are derived with a simple majorization (Schur-concavity) argument. They are further theoretically refined for different versions of the classical Hamming weight leakage model, in particular assuming a priori equiprobable secret keys and additive white Gaussian measurement noise. Closed-form expressions and numerical computation are given. A study of the impact of the choice of the substitution box with respect to side-channel resistance reveals that its nonlinearity tends to homogenize the expressivity of success rate, guessing entropy and statistical distance. The intriguing approximate relation between guessing entropy and success rate G E = 1 / S R is observed in the case of 8-bit bytes and low noise. The exact relation between guessing entropy, statistical distance and alphabet size G E = M + 1 2 − M 2 S D for deterministic leakages and equiprobable keys is proved. [ABSTRACT FROM AUTHOR]

The bounds presented at CHES 2017 based on Massey's guessing entropy represent the most scalable side-channel security evaluation method to date. In this paper, we present an improvement of this method, by determining the asymptotically optimal Massey-like inequality and then further refining it for finite support distributions. The impact of these results is highlighted for side-channel attack evaluations, demonstrating the improvements over the CHES 2017 bounds. [ABSTRACT FROM AUTHOR]

Fitts’ law is often employed as a predictive model for human movement, especially in the field of human-computer interaction. Models with an assumed Gaussian error structure are usually adequate when applied to data collected from controlled studies. However, observational data (often referred to as data gathered ‘in the wild’) typically display noticeable positive skewness relative to a mean trend as users do not routinely try to minimize their task completion time. As such, the exponentially modified Gaussian (EMG) regression model has been applied to aimed movements data. However, it is also of interest to reasonably characterize those regions where a user likely was not trying to minimize their task completion time. In this article, we propose a novel model with a two-component mixture structure—one Gaussian and one exponential—on the errors to identify such a region. An expectation-conditional-maximization (ECM) algorithm is developed for estimation of such a model and some properties of the algorithm are established. The efficacy of the proposed model, as well as its ability to inform model-based clustering, are addressed in this work through extensive simulations and an insightful analysis of a human aiming performance study. [ABSTRACT FROM AUTHOR]

Presents a study which observes an image coding technique based on an L norm criterion and exploiting statistical properties of the reconstruction error. Identity of the classes of preprocessing used to quantize, encode and reconstruct the original image; Details on the statistical properties of the reconstruction error.

Rotated and cyclic Q-delayed (RCQD) quadrature amplitude modulation provides signal space diversity and thus improves system performance over fading channels. However, previously published RCQD solutions were designed without fully considering the high demodulation complexity which prohibits wider applications. In this paper, a complete solution is proposed to reduce complexity for both the modulator and the demodulator. This solution uses a series of rotation angles ${\alpha =}$ arctan($1/{\sqrt {M}}$) which bring many interesting properties to the RCQD signals. A simplified sphere demapping algorithm is derived for fading channels with and without erasure events. In contrast to the sphere-decoder used for multiple input multiple output detection, the radius of the proposed sphere-demapper involves an exact amount of constellation points, thereby ensuring to perform the soft demapping operation successfully. Moreover, when either the in-phase (${I}$) or the quadrature component (${Q}$) is erased, the proposed demapping algorithm performs as well as the full-complexity max-log algorithm, with a reduced complexity. Compared to the solution currently used in DVB-T2, the proposed method reduces tremendously the computational complexity while still achieving similar performance over fading channels and even better performance over fading erasure channels. [ABSTRACT FROM AUTHOR]