Search

In recent years, solutions to various hypothesis testing problems in the asymptotic setting have been proposed using the results from large deviation theory. Such tests are optimal in terms of appropriately defined error exponents. For the practitioner, however, error probabilities in the finite sample size setting are more important. In this paper, we show how results on weak convergence of the test statistic can be used to obtain better approximations for the error probabilities in the finite sample size setting. While this technique is popular among statisticians for common tests, we demonstrate its applicability for several recently proposed asymptotically optimal tests, including tests for robust goodness of fit, homogeneity tests, outlier hypothesis testing, and graphical model estimation. [ABSTRACT FROM PUBLISHER]

We propose two coding schemes for the two-receiver discrete memoryless broadcast channel (BC) with rate-limited feedback from one or both receivers. They improve over the no feedback capacity region for a large class of channels, including the class of strictly essentially less-noisy BCs that we introduce in this paper. Examples of strictly essentially less-noisy BCs are the binary symmetric BC or the binary erasure BC with unequal crossover or erasure probabilities at the two receivers. When the feedback rates are sufficiently large, our schemes recover all previously known capacity results for discrete memoryless BCs with feedback. In both our schemes, we let the receivers feedback quantization messages about their receive signals. In the first scheme, the transmitter simply relays the quantization information obtained from Receiver 1 to Receiver 2, and vice versa. This provides each receiver with a second observation of the input signal and can thus improve its decoding performance unless the BC is physically degraded. Moreover, each receiver uses its knowledge of the quantization message describing its own outputs so as to attain the same performance as if this message had not been transmitted at all. In our second scheme, the transmitter first reconstructs and processes the quantized output signals, and then sends the outcome as a common update information to both receivers. A special case of our second scheme also applies to memoryless BCs without feedback but with strictly causal state-information at the transmitter and causal state-information at the receivers. It recovers all previous achievable regions also for this setup with state-information. [ABSTRACT FROM AUTHOR]

Opportunistic detection rules (ODRs) are variants of fixed-sample-size detection rules in which the statistician is allowed to make an early decision on the alternative hypothesis opportunistically based on the sequentially observed samples. From a sequential decision perspective, ODRs are also mixtures of one-sided and truncated sequential detection rules. Several results regarding ODRs are established in this paper. In the finite regime, the maximum sample size is modeled either as a fixed finite number, or a geometric random variable with a fixed finite mean. For both cases, the corresponding Bayesian formulations are investigated. The former case is a slight variation of the well-known finite-length sequential hypothesis testing procedure in the literature, whereas the latter case is new, for which the Bayesian optimal ODR is shown to be a sequence of likelihood ratio threshold tests with two different thresholds. A running threshold, which is determined by solving a stationary state equation, is used when future samples are still available, and a terminal threshold (simply the ratio between the priors scaled by costs) is used when the statistician reaches the final sample and, thus, has to make a decision immediately. In the asymptotic regime, the tradeoff among the exponents of the (false alarm and miss) error probabilities and the normalized expected stopping time under the alternative hypothesis is completely characterized and proved to be tight, via an information-theoretic argument. Within the tradeoff region, one noteworthy fact is that the performance of the Stein-Chernoff lemma is attainable by ODRs. [ABSTRACT FROM AUTHOR]

In this paper, the multiple access channel (MAC) with combined cooperation and partial cribbing is studied, and its capacity region is characterized. Cooperation means that each of the two encoders sends a message to the other via a rate-limited link prior to transmission, while partial cribbing means that each of the two encoders obtains a deterministic function of the other encoder’s output with or without delay. Prior work in this field dealt separately with cooperation and partial cribbing, but by combining these two methods, we can achieve significantly higher rates. Surprisingly, the capacity region of the MAC with combined cooperation and partial cribbing can be expressed using only one auxiliary random variable (RV) similar to the capacity regions of the MAC with cooperation and with partially cribbing encoders. The reason is that in an optimal coding scheme, the encoders use both cooperation and partial cribbing to generate a common message between the encoders. Furthermore, the Gaussian MAC with combined one-sided cooperation and quantized cribbing is studied. For this model, an achievability scheme is given. This scheme shows how many cooperation or quantization bits are required to practically achieve the capacity region of the Gaussian MAC with full message cooperation or perfect cribbing. To ratify the main results, two additional models are studied. In both models, only one auxiliary RV is needed. The first is a rate distortion dual setting for the MAC with degraded message set and combined cooperation and cribbing. The second is a state-dependent MAC with cooperation, where the state is known at a partially cribbing encoder and at the decoder. However, there are cases where more than one auxiliary RV is needed, e.g., when the cooperation and the cribbing are not used for the same purposes. The MAC with an action-dependent state is presented, where the action is based on the cooperation but not on the cribbing. Therefore, in this case, more than one auxiliary RV is needed. As a result, when the common information shared by the two encoders is used unevenly by the users in the channel, more than one auxiliary RV is needed to express the capacity region. [ABSTRACT FROM AUTHOR]

In 1973, Gallager proved that the random-coding bound is exponentially tight for the random code ensemble at all rates, even below expurgation. This result explained that the random-coding exponent does not achieve the expurgation exponent due to the properties of the random ensemble, irrespective of the utilized bounding technique. It has been conjectured that this same behavior holds true for a random ensemble of linear codes. This conjecture is proved in this paper. In addition, it is shown that this property extends to Poltyrev’s random-coding exponent for a random ensemble of lattices. [ABSTRACT FROM PUBLISHER]

Separable codes were defined by Cheng and Miao in 2011, motivated by applications to the identification of pirates in a multimedia setting. Combinatorially, \overline t -separable codes lie somewhere between t -frameproof and (t-1) -frameproof codes: all t -frameproof codes are (t-1) -frameproof. Results for frameproof codes show that (when q is large) there are q codewords, and that no q -ary n can have more than approximately codewords. This paper provides improved probabilistic existence results for t\geq 3 . More precisely, for all t\geq 3 and all n\geq 3 , there exists a constant \kappa (depending only on t and n ), such that there exists a codewords for all sufficiently large integers q . This shows, in particular, that the upper bound [derived from the bound on (t-1) -frameproof codes] on the number of codewords in a t=2 . Results due to Gao and Ge show that a q -separable code of length n can contain at most \frac {3}{2}q^{2\lceil n/3\rceil }-\frac {1}{2}q^{\lceil n/3\rceil } codewords, and that codes with at least \kappa q^{2n/3} codewords exist. Thus, optimal \overline 2 -separable codes behave neither like two-frameproof nor one-frameproof codes. This paper also observes that the bound of Gao and Ge can be strengthened to show that the number of codewords of a q -ary \overline 2 -separable code of length $n$ is at most q^{\lceil 2n/3\rceil }+\tfrac {1}{2}q^{\lfloor n/3\rfloor }(q^{\lfloor n/3\rfloor }-1) . [ABSTRACT FROM PUBLISHER]

In this paper, we study the capacity of the diamond channel. We focus on the special case where the channel between the source node and the two relay nodes are two separate links with finite capacities and the link from the two relay nodes to the destination node is a Gaussian multiple access channel. We call this model the Gaussian multiple access diamond channel. We first propose an upper bound on the capacity. This upper bound is a single-letterization of an $n$ -letter upper bound proposed by Traskov and Kramer, and is tighter than the cut-set bound. As for the lower bound, we propose an achievability scheme based on sending correlated codes through the multiple access channel with superposition structure. We then specialize this achievable rate to the Gaussian multiple access diamond channel. Noting the similarity between the upper and lower bounds, we provide sufficient and necessary conditions that a Gaussian multiple access diamond channel has to satisfy such that the proposed upper and lower bounds meet. Thus, for a Gaussian multiple access diamond channel that satisfies these conditions, we have found its capacity. [ABSTRACT FROM PUBLISHER]

This paper investigates the maximal channel coding rate achievable at a given blocklength $ n$ and error probability \epsilon $ , when the codewords are subjected to a long-term (i.e., averaged-over-all-codeword) power constraint. The second-order term in the large-n expansion of the maximal channel coding rate is characterized both for additive white Gaussian noise (AWGN) channels and for quasi-static fading channels with perfect channel state information available at both the transmitter and the receiver. It is shown that in both the cases, the second-order term is proportional to ({n ^{-1}\ln n })^{1/2} . For the quasi-static fading case, this second-order term is achieved by truncated channel inversion, namely, by concatenating a dispersion-optimal code for an AWGN channel subject to a short-term power constraint, with a power controller that inverts the channel whenever the fading gain is above a certain threshold. Easy-to-evaluate approximations of the maximal channel coding rate are developed for both the AWGN and the quasi-static fading case. [ABSTRACT FROM AUTHOR]

This paper studies an information-theoretic one-shot variable-length secret key agreement problem with public discussion. Let X and Y be jointly distributed random variables, each taking values in some measurable space. Alice and Bob observe X and Y respectively, can communicate interactively through a public noiseless channel, and want to agree on a key length and a key that is approximately uniformly distributed over all bit sequences with the agreed key length. The public discussion is observed by an eavesdropper, Eve. The key should be approximately independent of the public discussion, conditional on the key length. We show that the optimal expected key length is close to the mutual information I(X;Y) within a logarithmic gap. Moreover, an upper bound and a lower bound on the optimal expected key length can be written down in terms of I(X;Y) only. This means that the optimal one-shot performance is always within a small gap of the optimal asymptotic performance regardless of the distribution of the pair (X,Y). This one-shot result may find applications in situations where the components of an i.i.d. pair source (Xn,Yn) are observed sequentially and the key is output bit by bit, or in situations where the random source is not an i.i.d. or ergodic process. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of secret key generation with one-way communication through both a rate-limited public channel and a rate-limited secure channels where the public channel is from Alice to Bob and Eve and the secure channel is from Alice to Bob. In this model, we do not pose any constraints on the sources, i.e. Bob is not degraded to or less noisy than Eve. We obtain the optimal secret key rate in this problem, both for the discrete memoryless sources and vector Gaussian sources. The vector Gaussian characterization is derived by suitably applying the enhancement argument, and proving a new extremal inequality. The extremal inequality can be seen as coupling of two extremal inequalities, which are related to the degraded compound MIMO Gaussian broadcast channel, and the vector generalization of Costa’s entropy power inequality, accordingly. [ABSTRACT FROM AUTHOR]

In this paper, we propose CodedSketch, as a distributed straggler-resistant scheme to compute an approximation of the multiplication of two massive matrices. The objective is to reduce the recovery threshold, defined as the total number of worker nodes that the master node needs to wait for to be able to recover the final result. To exploit the fact that only an approximated result is required, in reducing the recovery threshold, some sorts of pre-compression are required. However, compression inherently involves some randomness that would lose the structure of the matrices. On the other hand, considering the structure of the matrices is crucial to reduce the recovery threshold. In CodedSketch, we use count–sketch, as a hash-based compression scheme, on the rows of the first and columns of the second matrix, and a structured polynomial code on the columns of the first and rows of the second matrix. This arrangement allows us to exploit the gain of both in reducing the recovery threshold. To increase the accuracy of computation, multiple independent count–sketches are needed. This independency allows us to theoretically characterize the accuracy of the result and establish the recovery threshold achieved by the proposed scheme. To guarantee the independency of resulting count–sketches in the output, while keeping its cost on the recovery threshold minimum, we use another layer of structured codes. The proposed scheme provides an upper-bound on the recovery threshold as a function of the required accuracy of computation and the probability that the required accuracy can be violated. In addition, it provides an upper-bound on the recovery threshold for the case that the result of the multiplication is sparse, and the exact result is required. [ABSTRACT FROM AUTHOR]

This paper studies the fundamental limits of the minimum average length of lossless and lossy variable-length compression, allowing a nonzero error probability \epsilon , for lossless compression. We give nonasymptotic bounds on the minimum average length in terms of Erokhin’s rate-distortion function and we use those bounds to obtain a Gaussian approximation on the speed of approach to the limit, which is quite accurate for all but small blocklengths: (1 - \epsilon ) k H(\mathsf S) - {({({k V(\mathsf S)}/{2 \pi })})}^{1/2} \mathop {\rm exp}[- ({( Q^{-1}\left ({\epsilon }\right ))^{2}}/{2})] , where Q^{-1}\left ({\cdot }\right )$ is the functional inverse of the standard Gaussian complementary cumulative distribution function, and $V(\mathsf S)$ is the source dispersion. A nonzero error probability thus not only reduces the asymptotically achievable rate by a factor of $1 - \epsilon $ , but this asymptotic limit is approached from below, i.e., larger source dispersions and shorter blocklengths are beneficial. Variable-length lossy compression under an excess distortion constraint is shown to exhibit similar properties. [ABSTRACT FROM PUBLISHER]

This paper studies a decentralized wireless network of asynchronous transmitter–receiver pairs with burst transmissions. Each receiver learns about the number of active users, channel coefficients, and mutual delays based on locally available measurements. The estimates for the mutual delays are not perfect, however, they are reliable enough to guarantee successful decoding. Two signalling schemes are addressed, namely, randomized masking (RM) and reduced cycle transmission (RCT). Under RM, the $n$ symbols of a codeword are generated according to a Bernoulli–Gaussian distribution with activity factor $0<\theta \leq 1$ . This is in contrast to RCT where each codeword consists of $\lfloor \theta n\rfloor $ Gaussian symbols followed by $n-\lfloor \theta n\rfloor $ zeros. Assuming the transmitters are unaware of the number of users, channel coefficients, and mutual delays, the probability of outage under RM is considerably lower compared with RCT if the signal-to-noise ratio (SNR) is sufficiently large. A generalized RCT scheme is also examined where the $n-\lfloor \theta n\rfloor $ zero symbols are not necessarily located at the end of a codeword. In the asymptote of large SNR, the outage probability becomes vanishingly small under RM, however, it is bounded away from zero for generalized RCT regardless of the value of SNR. [ABSTRACT FROM PUBLISHER]

We consider the basic bidirectional relaying problem, in which two users in a wireless network wish to exchange messages through an intermediate relay node. In the compute-and-forward strategy, the relay computes a function of the two messages using the naturally occurring sum of symbols simultaneously transmitted by user nodes in a Gaussian multiple-access channel (MAC), and the computed function value is forwarded to the user nodes in an ensuing broadcast phase. In this paper, we study the problem under an additional security constraint, which requires that each user’s message be kept secure from the relay. We consider two types of security constraints: 1) perfect secrecy, in which the MAC channel output seen by the relay is independent of each user’s message and 2) strong secrecy, which is a form of asymptotic independence. We propose a coding scheme based on nested lattices, the main feature of which is that given a pair of nested lattices that satisfy certain goodness properties, we can explicitly specify probability distributions for randomization at the encoders to achieve the desired security criteria. In particular, our coding scheme guarantees perfect or strong secrecy even in the absence of channel noise. The noise in the channel only affects reliability of computation at the relay, and for Gaussian noise, we derive achievable rates for reliable and secure computation. We also present an application of our methods to the multihop line network in which a source needs to transmit messages to a destination through a series of intermediate relays. [ABSTRACT FROM AUTHOR]

Guyon (Guyon J. Limit theorems for bifurcating Markov chains. Application to the detection of cellular aging. Ann Appl Probab, 2007, 17: 1538-1569) introduced an important model for homogeneous bifurcating Markov chains indexed by a binary tree taking values in general state space and studied their limit theorems. The results were applied to detect cellular aging. In this paper, we define a discrete form of nonhomogeneous bifurcating Markov chains indexed by a binary tree and discuss the equivalent properties for them. The strong law of large numbers and the entropy ergodic theorem are studied for these Markov chains with finite state space. In contrast to previous work, we use a new approach to prove the main results of this paper. [ABSTRACT FROM AUTHOR]

This paper studies sequential methods for recovery of sparse signals in high dimensions. When compared with fixed sample size procedures, in the sparse setting, sequential methods can result in a large reduction in the number of samples needed for reliable signal support recovery. Starting with a lower bound, we show any coordinate-wise sequential sampling procedure fails in the high dimensional limit provided the average number of measurements per dimension is less then \log (s)/D(P0||P1) , where s is the level of sparsity and D(P_{0}||P_{1}) is the Kullback–Leibler divergence between the underlying distributions. A series of sequential probability ratio tests, which require complete knowledge of the underlying distributions is shown to achieve this bound. Motivated by real-world experiments and recent work in adaptive sensing, we introduce a simple procedure termed sequential thresholding, which can be implemented when the underlying testing problem satisfies a monotone likelihood ratio assumption. Sequential thresholding guarantees exact support recovery provided the average number of measurements per dimension grows faster than \log (s)/D(P0||P1) , achieving the lower bound. For comparison, we show any nonsequential procedure fails provided the number of measurements grows at a rate less than \log (n)/D(P_{1}||P_{0})$ , where $n$ is the total dimension of the problem. [ABSTRACT FROM AUTHOR]

We consider a cooperative two-user multiaccess channel in which the transmission is controlled by a random state. Both encoders transmit a common message and, one of the encoders also transmits an individual message. We study the capacity region of this communication model for different degrees of availability of the states at the encoders, causally or strictly causally. In the case in which the states are revealed causally to both encoders but not to the decoder we find an explicit characterization of the capacity region in the discrete memoryless case. In the case in which the states are revealed only strictly causally to both encoders, we establish inner and outer bounds on the capacity region. The outer bound is nontrivial, and has a relatively simple form. It has the advantage of incorporating only one auxiliary random variable. In particular, it suggests that there is none, or at best only little, to gain from having the encoder that transmits both messages also sending an individual description of the state to the receiver, in addition to the compressed version that is sent cooperatively with the other encoder. We then introduce a class of cooperative multiaccess channels with states known strictly causally at both encoders for which the inner and outer bounds agree, and so we characterize the capacity region for this class. In this class of channels, the state can be obtained as a deterministic function of the channel inputs and output. We also study the model in which the states are revealed, strictly causally, in an asymmetric manner, to only one encoder. Throughout this paper, we discuss a number of examples, and compute the capacity region of some of these examples. The results shed more light on the utility of delayed channel state information for increasing the capacity region of state-dependent cooperative multiaccess channels, and tie with recent progress in this framework. [ABSTRACT FROM AUTHOR]

In this paper, we revisit the structure of infeasibility results in network information theory, based on a notion of information state. We also discuss ideas for generalizing a known outer bound for lossless transmission of independent sources over a network to one of lossy transmission of dependent sources over the same network. To concretely demonstrate this, we apply our ideas and prove new results for lossy transmission of dependent sources by generalizing: 1) the cut-set bound; 2) the best known outer bound on the capacity region of a general broadcast channel; and 3) the outer bound part of the result of Maric, Yates, and Kramer on strong interference channels with a common message. [ABSTRACT FROM AUTHOR]

This paper investigates the downlink of a cloud radio access network (C-RAN) in which a central processor communicates with two mobile users through two base stations (BSs). The BSs act as relay nodes and cooperate with each other through error-free rate-limited links. We develop and analyze two coding schemes for this scenario. The first coding scheme modifies the Liu–Kang scheme (to make it amenable to a rigorous analysis) and extends it to introduce common codewords and to apply for downlink C-RAN with BS-to-BS cooperation. This first coding scheme enables arbitrary correlation among the auxiliary codewords that are recovered by the BSs. We show that this scheme improves over previous schemes for various instances of Gaussian C-RAN channels. In particular, in many scenarios, the scheme can better exploit the possibility of BS-to-BS cooperation than other schemes. The second coding scheme extends the distributed decode and forward (DDF) scheme by means of Gray–Wyner compression and by exploiting the cooperation links between BSs. In addition and as a separate extension, we provide an improved capacity approximation for the DDF strategy for the capacity of a general $N$ -BS $L$ -user C-RAN model in the memoryless Gaussian case. [ABSTRACT FROM AUTHOR]

High-order cumulants and high-order spectra play an important role in the theory of stationary processes. For nonlinear processes, it is quite challenging to develop an asymptotic theory for their estimators. This paper presents a systematic asymptotic theory for estimators of high-order moments for a general class of stationary processes, using the framework of functional dependence measures. In particular, we prove the asymptotic normality of the estimators and establish a uniform convergence rate. We also provide a sufficient condition for the summability of high-order cumulants. Based on the latter, we prove consistency and asymptotic normality of the third-order spectra or bispectrum estimators under mild and easily verifiable conditions. [ABSTRACT FROM AUTHOR]

Rivest and Shamir introduced a write-once memory (WOM), a model of storage devices whose storage elements have restrictions on state transitions, and they presented some coding methods to reuse a WOM. An interesting question about a WOM is how efficiently we can reuse it with the best coding method, and as an answer to the question, Fu and Han Vinck determined the capacity of Fiat and Shamir’s generalized WOMs. In this paper, we extend their results, introducing write-constrained memories (WCMs) that consider state transition costs, and determining the capacity of WCMs under a certain type of cost constraints. [ABSTRACT FROM AUTHOR]

This paper addresses the correspondence between linear inequalities for Shannon entropy and differential entropy for sums of independent group-valued random variables. We show that any balanced (with the sum of coefficients being zero) linear inequality for Shannon entropy holds if and only if its differential entropy counterpart also holds; moreover, any linear inequality for differential entropy must be balanced. In particular, our result shows that recently proved differential entropy inequalities by Kontoyiannis and Madiman can be deduced from their discrete counterparts due to Tao in a unified manner. Generalizations to certain abelian groups are also obtained. Our proof of extending inequalities for Shannon entropy to differential entropy relies on a result of Rényi which relates the Shannon entropy of a finely discretized random variable to its differential entropy and also helps in establishing that the entropy of the sum of quantized random variables is asymptotically equal to that of the quantized sum; the converse uses the asymptotics of the differential entropy of convolutions with weak additive noise. [ABSTRACT FROM PUBLISHER]

A measure of correlation is said to have the tensorization property if it does not change when computed for i.i.d. copies. More precisely, a measure of correlation between two random variables X, Y denoted by \rho (X, Y) , has the tensorization property if \rho (X^n, Y^n)=\rho (X, Y) where (X^n, Y^n) denotes $n$ i.i.d. copies of (X, Y)$ . Two well-known examples of such measures are the maximal correlation and the hypercontractivity ribbon (HC ribbon). We show that the maximal correlation and the HC ribbon are special cases of the new notion of \Phi $ -ribbons, defined in this paper for a class of convex functions \Phi $ . \Phi $ -ribbon reduces to the HC ribbon and the maximal correlation for special choices of \Phi $ , and is a measure of correlation with the tensorization property. We show that the \Phi $ -ribbon also characterizes the recently introduced and introduce an equivalent characterization of this ribbon. [ABSTRACT FROM AUTHOR]

It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$ , with equality if and only if $f$ is one-to-one. In this paper, we give tight bounds on $H(f(X))$ , when the function $f$ is not one-to-one, and we illustrate a few scenarios, where this matters. As an intermediate step toward our main result, we derive a lower bound on the entropy of a probability distribution, when only a bound on the ratio between the maximal and minimal probabilities is known. The lower bound improves on previous results in the literature, and it could find applications outside the present scenario. [ABSTRACT FROM AUTHOR]

We consider the problem of mean-square estimation of the state of a discrete time dynamical system having additive non-Gaussian noise. We assume that the noise has the structure that at each time instant, it is a projection of a fixed high-dimensional noise vector with a log-concave density. We derive conditions which guarantee that, as the dimension of this noise vector grows large, the optimal estimator of the problem with Gaussian noise becomes near-optimal for the problem with non-Gaussian noise. The results are derived by first showing an asymptotic Gaussian lower bound on the minimum error which holds even for nonlinear systems. The bound is asymptotically tight for linear systems. For nonlinear systems, this bound is tight provided and the noise has a strongly log-concave density with some additional structure. These results imply that the estimate obtained by employing the Gaussian estimator on the non-Gaussian problem satisfies an approximate orthogonality principle, and the difference between this estimate and the optimal estimate vanishes strongly in L^2 . For linear systems with high-dimensional log-concave noise of this structure, we get that the Kalman filter serves as a near-optimal estimator. A key ingredient in the proofs is a recent central limit theorem of Eldan and Klartag. The minimum mean-square error is in general neither weakly upper semicontinuous nor weakly lower semicontinuous. Our results show that in the setting we consider, it is weakly continuous.

We then consider mean-square estimation over a noisy network, where the problem is to jointly design strategies for sensing and estimation to minimize a composite quadratic cost. These problems do not have a static information structure, whereby the previously derived results do not apply. We consider two different cases—when the estimation part of the problem corresponds to estimating the original source sensed by the sensor, and when this part corresponds to the estimation of the action of the sensor. In the former case, we show that results analogous to the above continue to hold. The latter case corresponds to a variant of Witsenhausen’s problem, wherein we show only partial results. [ABSTRACT FROM AUTHOR]

Usually, it is very difficult to determine the exact distribution for a test statistic. In this paper, asymptotic distributions of locally most powerful invariant test for independence of complex Gaussian vectors are developed. In particular, its cumulative distribution function (CDF) under the null hypothesis is approximated by a function of chi-squared CDFs. Moreover, the CDF corresponding to the non-null distribution is expressed in terms of non-central chi-squared CDFs for close hypothesis, and Gaussian CDF as well as its derivatives for far hypothesis. The results turn out to be very accurate in terms of fitting their empirical counterparts. Closed-form expression for the detection threshold is also provided. Numerical results are presented to validate our theoretical findings. [ABSTRACT FROM PUBLISHER]

We revisit the problem of asymmetric binary hypothesis testing against a composite alternative hypothesis. We introduce a general framework to treat such problems when the alternative hypothesis adheres to certain axioms. In this case, we find the threshold rate, the optimal error and strong converse exponents (at large deviations from the threshold), and the second order asymptotics (at small deviations from the threshold). We apply our results to find the operational interpretations of various Rényi information measures. In case the alternative hypothesis is comprised of bipartite product distributions, we find that the optimal error and strong converse exponents are determined by the variations of Rényi mutual information. In case the alternative hypothesis consists of tripartite distributions satisfying the Markov property, we find that the optimal exponents are determined by the variations of Rényi conditional mutual information. In either case, the relevant notion of Rényi mutual information depends on the precise choice of the alternative hypothesis. As such, this paper also strengthens the view that different definitions of Rényi mutual information, conditional entropy, and conditional mutual information are adequate depending on the context in which the measures are used. [ABSTRACT FROM AUTHOR]

In this paper, we propose an open-loop unequal-error-protection querying policy based on superposition coding for the noisy 20 questions problem. In this problem, a player wishes to successively refine an estimate of the value of a continuous random variable by posing binary queries and receiving noisy responses. When the queries are designed non-adaptively as a single block and the noisy responses are modeled as the output of a binary symmetric channel, the 20 questions problem can be mapped to an equivalent problem of channel coding with unequal error protection (UEP). A new non-adaptive querying strategy based on UEP superposition coding is introduced, whose estimation error decreases with an exponential rate of convergence that is significantly better than that of the UEP repetition coding introduced by Variani et al. (2015). With the proposed querying strategy, the rate of exponential decrease in the number of queries matches the rate of a closed-loop adaptive scheme, where queries are sequentially designed with the benefit of feedback. Furthermore, the achievable error exponent is significantly better than that of random block codes employing equal error protection. [ABSTRACT FROM AUTHOR]

The entropy of a random variable is well-known to equal the exponential growth rate of the volumes of its typical sets. In this paper, we show that for any log-concave random variable X , the sequence of the \lfloor n\theta \rfloor ^{\text {th}} intrinsic volumes of the typical sets of X in dimensions n \geq 1 grows exponentially with a well-defined rate. We denote this rate by h_{X}(\theta ) , and call it the \theta ^{\text {th}} intrinsic entropy of X . We show that h_{X}(\theta ) is a continuous function of \theta $ over the range [0, 1], thereby providing a smooth interpolation between the values 0 and $h(X)$ at the endpoints 0 and 1, respectively. [ABSTRACT FROM PUBLISHER]

Let X and Y be dependent random variables. This paper considers the problem of designing a scalar quantizer for Y to maximize the mutual information between the quantizer’s output and X, and develops fundamental properties and bounds for this form of quantization, which is connected to the log-loss distortion criterion. The main focus is the regime of low ${I}({X};{Y})$ , where it is shown that, if X is binary, a constant fraction of the mutual information can always be preserved using $\mathcal {O}(\log (1/{I}({X};{Y})))$ quantization levels, and there exist distributions for which this many quantization levels are necessary. Furthermore, for larger finite alphabets $2 < |\mathcal {X}| < \infty $ , it is established that an $\eta $ -fraction of the mutual information can be preserved using roughly $(\log (| \mathcal {X} | /{I}({X};{Y})))^{\eta \cdot (|\mathcal {X}| - 1)}$ quantization levels. [ABSTRACT FROM AUTHOR]

This paper provides a precise error analysis for the maximum likelihood estimate $\hat {\textit a}_{\text {ML}}({\textit u}_{1}^{\textit n})$ of the parameter ${\textit a}$ given samples ${\textit u}_{1}^{\textit n} = ({\textit u}_{1}, \ldots, {\textit u}_{\textit n})'$ drawn from a nonstationary Gauss-Markov process ${\textit U}_{\textit i} = {\textit a} {\textit U}_{\textit i-1} + {\textit Z}_{\textit i},\,\,{\textit i}\geq 1$ , where ${\textit U}_{0} = 0$ , ${\textit a}> 1$ , and ${\textit Z}_{\textit i}$ ’s are independent Gaussian random variables with zero mean and variance $\sigma ^{2}$. We show a tight nonasymptotic exponentially decaying bound on the tail probability of the estimation error. Unlike previous works, our bound is tight already for a sample size of the order of hundreds. We apply the new estimation bound to find the dispersion for lossy compression of nonstationary Gauss-Markov sources. We show that the dispersion is given by the same integral formula that we derived previously for the asymptotically stationary Gauss-Markov sources, i.e., $|{\textit a}| < 1$. New ideas in the nonstationary case include separately bounding the maximum eigenvalue (which scales exponentially) and the other eigenvalues (which are bounded by constants that depend only on a) of the covariance matrix of the source sequence, and new techniques in the derivation of our estimation error bound. [ABSTRACT FROM AUTHOR]

In many information-theoretic channel coding problems, adding an input cost constraint to the operational setup amounts to restricting the optimization domain in the capacity formula. This paper shows that, in contrast to common belief, such a simple modification does not hold for the cost-constrained (CC) wiretap channel (WTC). The secrecy-capacity of the discrete memoryless (DM) WTC without cost constraints is described by a single auxiliary random variable. For the CC DM-WTC, however, we show that two auxiliaries are necessary to achieve capacity. Specifically, we first derive the secrecy-capacity formula, proving the direct part via superposition coding. Then, we provide an example of a CC DM-WTC whose secrecy-capacity cannot be achieved using a single auxiliary. This establishes the fundamental role of superposition coding over CC WTCs. [ABSTRACT FROM AUTHOR]

In group testing, simple binary-output tests are designed to identify a small number $t$ of defective items that are present in a large population of $N$ items. Each test takes as input a group of items and produces a binary output indicating whether the group is free of the defective items or contains one or more of them. In this paper, we study a relaxation of the combinatorial group testing problem. A matrix is called $(t,\epsilon )$ -disjunct if it gives rise to a nonadaptive group testing scheme with the property of identifying a uniformly random $t$ -set of defective subjects out of a population of size $N$ with false positive probability of an item at most $\epsilon $ . We establish a new connection between $(t,\epsilon )$ -disjunct matrices and error correcting codes based on the dual distance of the codes and derive estimates of the parameters of codes that give rise to such schemes. Our methods rely on the moments of the distance distribution of codes and inequalities for moments of sums of independent random variables. We also provide a new connection between group testing schemes and combinatorial designs. [ABSTRACT FROM PUBLISHER]

The problem of characterizing the fundamental tradeoff between storage and repair bandwidth of exact-repair regenerating codes against a passive eavesdropper is studied. The eavesdropper is assumed to be capable of observing the data stored in a fixed number of nodes and the data involved in the repair of these nodes. In this paper, the tradeoff for regenerating codes with small parameters is characterized, and then, the results are extended to some general settings. [ABSTRACT FROM PUBLISHER]

In this paper, we consider a cache aided network in which each user is assumed to have individual caches, while upon users’ requests, an update message is sent through a common link to all users. First, we formulate a general information theoretic setting that represents the database as a discrete memoryless source, and the users’ requests as side information that is available everywhere except at the cache encoder. The decoders’ objective is to recover a function of the source and the side information. By viewing cache aided networks in terms of a general distributed source coding problem and through information theoretic arguments, we present inner and outer bounds on the fundamental tradeoff of cache memory size and update rate. Then, we specialize our general inner and outer bounds to a specific model of content delivery networks: file selection networks, in which the database is a collection of independent equal-size files and each user requests one of the files independently. For file selection networks, we provide an outer bound and two inner bounds (for centralized and decentralized caching strategies). For the case when the user request information is uniformly distributed, we characterize the rate versus cache size tradeoff to within a multiplicative gap of 4. By further extending our arguments to the framework of Maddah-Ali and Niesen, we also establish a new outer bound and two new inner bounds in which it is shown to recover the centralized and decentralized strategies, previously established by Maddah-Ali and Niesen. Finally, in terms of rate versus cache size tradeoff, we improve the previous multiplicative gap of 72 to 4.7 for the average case with uniform requests. [ABSTRACT FROM PUBLISHER]

A factor-graph representation of quantum-mechanical probabilities (involving any number of measurements) is proposed. Unlike standard statistical models, the proposed representation uses auxiliary variables (state variables) that are not random variables. All joint probability distributions are marginals of some complex-valued function $q$ , and it is demonstrated how the basic concepts of quantum mechanics relate to factorizations and marginals of $q$ . [ABSTRACT FROM AUTHOR]

We consider the problem of detecting abrupt parameter changes in a stochastic regression with unknown noise distribution. The process changes at some unknown point of time. Under general conditions on the regression function and unknown distributions of observations before and after the disruption, the paper develops a nonparametric cumulative sum procedure (CUSUM). Unlike likelihood-based CUSUM algorithms, constructed mostly on log-likelihood ratio statistics, we use a special system of basic statistics in Page’s procedure. By applying a sequential sampling scheme, which measures time in terms of accumulated Kullback–Leibler (K-L) divergence, we come to a system of statistics with the martingale properties close to those of the log-likelihood ratios. The proposed approach suggests also an alternative performance criterion in the analysis of the procedure by replacing the expected detection delay by the corresponding K-L divergence. We show that, under the false alarm probability constraint, the nonparametric CUSUM rule is optimal in the sense that it ensures the logarithmic asymptotic for the detection delay. [ABSTRACT FROM AUTHOR]

An important subclass of secure multiparty computation is secure sampling: two parties output samples of a pair of jointly distributed random variables such that neither party learns more about the other party's output than what its own output reveals. The parties make use of a setup—correlated random variables with a different distribution—as well as unlimited noiseless communication. An upperbound on the rate of producing samples of a desired distribution from a given setup is presented. The region of tension developed in this paper measures how well the dependence between a pair of random variables can be resolved by a piece of common information. The bounds on rate are a consequence of a monotonicity property; a protocol between two parties can only lower the tension between their views. Connections are drawn between the region of tension and the notion of common information. A generalization of the Gács-Körner common information, called the assisted common information, which takes into account almost common information ignored by Gács-Körner common information is defined. The region of tension is shown to be related to the rate regions of both the assisted common information and the Gray-Wyner systems (and, a fortiori, Wyner's common information). [ABSTRACT FROM PUBLISHER]

We prove that channel combining and splitting via Arikan's polarization transformation improves Gallager's reliability function E0 for binary input channels. In this sense, polarization creates E0 . This observation gives yet another justification as to why the polar transform yields capacity achieving and low complexity codes: the improvement in E0 translates to an improvement in complexity-error-probability trade-off. In analyzing polar codes, one examines auxiliary random processes that follow the evolution of information measures as an underlying communication channel undergoes a sequence of transformations. The conclusion of this paper shows that the E0 process associated to such an analysis is a submartingale. [ABSTRACT FROM AUTHOR]

We provide a refinement of the sphere-packing bound for constant composition codes over asymmetric discrete memoryless channels that improves the subexponential factor in front of the exponent. The order of our subexponential factor is \Omega (N^-0.5(1+\epsilon+\rhoR^{\ast)}) for any \epsilon>0, where \rhoR^{\ast} is the left derivative of the sphere-packing exponent at rate R and N is the blocklength. [ABSTRACT FROM AUTHOR]

Causal video coding is a coding paradigm where video source frames X1, X2,\ldots, XN are encoded in a frame-by-frame manner, the encoder for each frame can use all previous source frames and all previous encoded frames, and the corresponding decoder can use only all previous encoded frames. In the special case where the encoder for each frame Xk is further restricted to enlist help only from all previous encoded frames, causal video coding is reduced to predictive video coding, which all MPEG-series and H-series video coding standards proposed so far are based upon. In this paper, we compare the rate distortion performance of causal video coding with that of predictive video coding from an information theoretic perspective by modeling each frame Xk itself as a source Xk=\{Xk(i)\}i=1^{\infty}. Let R^\astc(D1,\ldots,DN) (R^\astp(D1,\ldots,DN), respectively) denote the minimum total rate required to achieve a given distortion level D1,\ldots,DN in causal video coding (predictive video coding, respectively). We first show that like R^\astc(D1,\ldots, DN), for jointly stationary and totally ergodic sources X1, X2,\ldots, XN, Rp^{\ast}(D1,\ldots,DN) is equal to the infimum of the nth order total rate distortion function Rp,n(D1,\ldots,DN) over all n, where Rp,n(D1,\ldots,DN) itself is given by the minimum of an information quantity over a set of auxiliary random variables. We then prove that if the jointly stationary and totally ergodic sources X1,\ldots, XN form a (first-order) Markov chain, we have R^\astp(D1,\ldots,DN)=R^\astc(D1,\ldots,DN). However, this is not true in general if X1,\ldots, XN do not form a (first-order) Markov chain. Specifically, we demonstrate that for independent and identically distributed vector source (X1,\ldots, XN), if X1,\ldots, XN do not form a (first-order) Markov chain, then under some conditions on source frames and distortion, R^\astc(D1,\ldots, DN) is strictly less than Rp^{\ast}(D1,\ldots, DN) in general. Our techniques allow us to compare Rp^{\ast}(D1,\ldots, DN) with R^\astc(D1,\ldots, DN) even when the single-letter characterization of R^\astp(D1,\ldots, DN), if any, is unknown. [ABSTRACT FROM AUTHOR]

This paper considers a binary channel with deletions and insertions, where each input bit is transformed in one of the following ways: it is deleted with probability d, or an extra bit is added after it with probability i, or it is transmitted unmodified with probability 1-d-i. A computable lower bound on the capacity of this channel is derived. The transformation of the input sequence by the channel may be viewed in terms of runs as follows: some runs of the input sequence get shorter/longer, some runs get deleted, and some new runs are added. It is difficult for the decoder to synchronize the channel output sequence to the transmitted codeword mainly due to deleted runs and new inserted runs. The main idea is a mutual information decomposition in terms of the rate achieved by a suboptimal decoder that determines the positions of the deleted and inserted runs in addition to decoding the transmitted codeword. The mutual information between the channel input and output sequences is expressed as the sum of the rate achieved by this decoder and the rate loss due to its suboptimality. Obtaining computable lower bounds on each of these quantities yields a lower bound on the capacity. The bounds proposed in this paper provide the first characterization of achievable rates for channels with general insertions, and for channels with both deletions and insertions. For the special case of the deletion channel, the proposed bound improves on the previous best lower bound for deletion probabilities up to 0.3. [ABSTRACT FROM AUTHOR]

The objective of a genome-wide association study (GWAS) is to associate subsequences of individuals’ genomes to the observable characteristics called phenotypes (e.g., high blood pressure). Motivated by the GWAS problem, in this paper we introduce the information-theoretic problem of associated subsequence retrieval, where a dataset of N (possibly high-dimensional) sequences of length G, and their corresponding observable (binary) characteristics is given. The sequences are chosen independently and uniformly at random from $\mathcal {X}^{\text {G}}$ , where $\mathcal {X}$ is a finite alphabet. The observable (binary) characteristic is only related to a specific unknown subsequence of length $L$ of the sequences, called associated subsequence. For each sequence, if the associated subsequence of it belongs to a universal finite set, then it is more likely to display the observable characteristic (i.e., it is more likely that the observable characteristic is one). The goal is to retrieve the associated subsequence using a dataset of N sequences and their observable characteristics. We demonstrate that as the parameters N, G, and L grow, a threshold effect appears in the curve of probability of error versus the rate which is defined as ${{\it\text { Gh}}(\text {L}/\text {G})}/{\text {N}}$ , where $\text {h}(\cdot)$ is the binary entropy function. This effect allows us to define the capacity of associated subsequence retrieval. We develop an achievable scheme and a matching converse for this problem, and thus characterize its capacity in two scenarios: the zero-error-rate and the $\epsilon $ -error-rate. [ABSTRACT FROM AUTHOR]

The support recovery problem consists of determining a sparse subset of variables that is relevant in generating a set of observations. In this paper, we study the support recovery problem in the phase retrieval model consisting of noisy phaseless measurements, which arises in a diverse range of settings such as optical detection, X-ray crystallography, electron microscopy, and coherent diffractive imaging. Our focus is on information-theoretic fundamental limits under an approximate recovery criterion, considering both discrete and Gaussian models for the sparse non-zero entries, along with Gaussian measurement matrices. In both cases, our bounds provide sharp thresholds with near-matching constant factors in several scaling regimes on the sparsity and signal-to-noise ratio. As a key step towards obtaining these results, we develop new concentration bounds for the conditional information content of log-concave random variables, which may be of independent interest. [ABSTRACT FROM AUTHOR]

The classical problem of supervised learning is to infer an accurate estimate of a target variable $Y$ from a measured variable $X$ using a set of labeled training samples. Motivated by the increasingly distributed nature of data and decision making, this paper considers a variation of this classical problem in which the inference is distributed between two nodes, e.g., a mobile device and a cloud, with a rate constraint on the communication between them. The mobile device observes $X$ and sends a description $M$ of $X$ to the cloud, which computes an estimate $\hat {Y}$ of $Y$. We follow the recent minimax learning approach to study this inference problem and show that it corresponds to a one-shot minimax noisy lossy source coding problem. We then establish information theoretic bounds on the risk-rate Lagrangian cost, leading to a general method for designing a near-optimal descriptor-estimator pair. A key ingredient in the proof of our result is a refined version of the strong functional representation lemma previously used to establish several one-shot source coding theorems. Our results show that a naive estimate-compress scheme for rate-constrained inference is not optimal in general. When the distribution of $(X,Y)$ is known and the error is measured by the logarithmic loss, our bounds on the risk-rate Lagrangian cost provide a new one-shot operational interpretation of the information bottleneck. We also demonstrate a way to bound the excess risk of the descriptor-estimator pair obtained by our method. [ABSTRACT FROM AUTHOR]

We consider the recursive estimation of the probability density function of continuous random variables from a strongly mixing random sample. We revisit here earlier research on this subject by considering a more general class of recursive estimators, including the usual ones. We derive the quadratic mean error of the considered class of estimators. Moreover, we establish a central limit theorem by using Lindeberg’s method resulting in a simplification of the existing assumptions on the sequence of smooth parameters and the mixing coefficient. This is the main contribution of this paper. Finally, the feasibility of the proposed estimator is illustrated throughout an empirical study. [ABSTRACT FROM AUTHOR]

Algebraic network information theory is an emerging facet of network information theory, studying the achievable rates of random code ensembles that have algebraic structure, such as random linear codes. A distinguishing feature is that linear combinations of codewords can sometimes be decoded more efficiently than codewords themselves. The present work further develops this framework by studying the simultaneous decoding of multiple messages. Specifically, consider a receiver in a multi-user network that wishes to decode several messages. Simultaneous joint typicality decoding is one of the most powerful techniques for determining the fundamental limits at which reliable decoding is possible. This technique has historically been used in conjunction with random i.i.d. codebooks to establish achievable rate regions for networks. Recently, it has been shown that, in certain scenarios, nested linear codebooks in conjunction with “single-user” or sequential decoding can yield better achievable rates. For instance, the compute–forward problem examines the scenario of recovering $L \le K$ linear combinations of transmitted codewords over a $K$ -user multiple-access channel (MAC), and it is well established that linear codebooks can yield higher rates. This paper develops bounds for simultaneous joint typicality decoding used in conjunction with nested linear codebooks, and applies them to obtain a larger achievable region for compute–forward over a $K$ -user discrete memoryless MAC. The key technical challenge is that competing codeword tuples that are linearly dependent on the true codeword tuple introduce statistical dependencies, which requires careful partitioning of the associated error events. [ABSTRACT FROM AUTHOR]

In the distributed function computation problem, dichotomy theorems, initiated by Han-Kobayashi, seek to classify functions by whether the rate regions for function computation improve on the Slepian-Wolf regions or not. In this paper, we develop a general approach to derive converse bounds on the distributed function computation problem. By using this approach, we recover the sufficiency part, i.e. the conditions such that the Slepian-Wolf regions become optimal, of the known dichotomy theorems in the two-terminal distributed computing. Furthermore, we derive an improved sufficient condition on the dichotomy theorem in the multiterminal distributed computing for the class of i.i.d. sources with the positivity condition. Finally, we derive the matching sufficient and necessary condition on the dichotomy theorem in the multiterminal distributed computing for the class of smooth sources. [ABSTRACT FROM AUTHOR]

Proving or disproving an information inequality is a crucial step in establishing the converse results in coding theorems. However, an information inequality involving more than a few random variables is difficult to be proved or disproved manually. In 1997, Yeung developed a framework that uses linear programming for verifying linear information inequalities. Under the framework, this paper considers a few other problems that can be solved by using Lagrange duality and convex approximation. We will demonstrate how linear programming can be used to find an analytic proof of an information inequality or an analytic counterexample to disprove it if the inequality is not true in general. The way to automatically find a shortest proof or a smallest counterexample is explored. When a given information inequality cannot be proved, the sufficient conditions for a counterexample to disprove the information inequality are found by linear programming. Lastly, we propose a scalable algorithmic framework based on the alternating direction method of multipliers to accelerate solving a multitude of user-specific problems whose overall computational cost can be amortized with the number of users, and present its publicly-available software implementation for large-scale problems. [ABSTRACT FROM AUTHOR]

The capacity-achieving input distribution of the discrete-time, additive white Gaussian noise (AWGN) channel with an amplitude constraint is discrete and seems difficult to characterize explicitly. A dual capacity expression is used to derive analytic capacity upper bounds for scalar and vector AWGN channels. The scalar bound improves on McKellips’ bound and is within 0.1 bit of capacity for all signal-to-noise ratios (SNRs). The 2-D bound is within 0.15 bits of capacity provably up to 4.5 dB; numerical evidence suggests a similar gap for all SNRs. As the SNR tends to infinity, these bounds are accurate and match with a volume-based lower bound. For the 2-D complex case, an analytic lower bound is derived by using a concentric constellation and is shown to be within 1 bit of capacity. [ABSTRACT FROM PUBLISHER]