Search

Recently, $\varepsilon $ -almost dual universal2 hash functions have been proposed as a new and wider class of hash functions. Using this class of hash functions, several efficient hash functions were proposed. This paper evaluates the security performance when we apply this kind of hash functions. We evaluate the security in several kinds of setting based on the L_{1} distinguishability criterion and the modified mutual information criterion. The obtained evaluation is based on smoothing of Rényi entropy of order 2 and/or min entropy. We clarify the difference between these two methods. [ABSTRACT FROM AUTHOR]

In this paper, we investigate problems of communication over physically degraded, state-dependent broadcast channels (BCs) with cooperating decoders. Two different setups are considered, and their capacity regions are characterized. First, we study a setting in which one decoder can use a finite capacity link to send the other decoder information regarding the messages or the channel states. In this scenario, we analyze two cases: one, where noncausal state information, is available to the encoder and the strong decoder, and the other, where state information, is available only to the encoder in a causal manner. Second, we examine a setting in which the cooperation between the decoders is limited to taking place before the outputs of the channel are given. In this case, one decoder, which is informed of the state sequence noncausally, can cooperate only to send the other decoder rate-limited information about the state sequence. The proofs of the capacity regions introduce a new idea of coding for channels with cooperation between different users, where we exploit the link between the decoders for multiple binnings. Finally, we discuss the optimality of using rate-splitting techniques when coding for cooperative BCs. In particular, we show that rate splitting is not necessarily optimal when coding for cooperative BCs by solving an example in which our method of coding outperforms rate splitting. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of generating arbitrary three-party correlations from a combination of public and secret correlations. Two parties—called Alice and Bob—share perfectly correlated bits that are secret from a collaborating third party, Charlie. At the same time, all three parties have access to a separate source of correlated bits, and their goal is to convert these two resources into multiple copies of some given tripartite distribution \mathbb P(XYZ) . We obtain a single-letter characterization of the tradeoff between public and private bits that are needed to achieve this task. The rate of private bits is shown to generalize Wyner’s classic notion of common information held between a pair of random variables. The problem we consider can be contrasted fruitfully with the task of secrecy formation, in which \mathbb P(XYZ) is generated using public communication and local randomness but with Charlie functioning as an adversary instead of a collaborator. We describe in detail the differences between the collaborative and adversarial scenarios. [ABSTRACT FROM AUTHOR]

We propose two coding schemes for the two-receiver discrete memoryless broadcast channel (BC) with rate-limited feedback from one or both receivers. They improve over the no feedback capacity region for a large class of channels, including the class of strictly essentially less-noisy BCs that we introduce in this paper. Examples of strictly essentially less-noisy BCs are the binary symmetric BC or the binary erasure BC with unequal crossover or erasure probabilities at the two receivers. When the feedback rates are sufficiently large, our schemes recover all previously known capacity results for discrete memoryless BCs with feedback. In both our schemes, we let the receivers feedback quantization messages about their receive signals. In the first scheme, the transmitter simply relays the quantization information obtained from Receiver 1 to Receiver 2, and vice versa. This provides each receiver with a second observation of the input signal and can thus improve its decoding performance unless the BC is physically degraded. Moreover, each receiver uses its knowledge of the quantization message describing its own outputs so as to attain the same performance as if this message had not been transmitted at all. In our second scheme, the transmitter first reconstructs and processes the quantized output signals, and then sends the outcome as a common update information to both receivers. A special case of our second scheme also applies to memoryless BCs without feedback but with strictly causal state-information at the transmitter and causal state-information at the receivers. It recovers all previous achievable regions also for this setup with state-information. [ABSTRACT FROM AUTHOR]

Opportunistic detection rules (ODRs) are variants of fixed-sample-size detection rules in which the statistician is allowed to make an early decision on the alternative hypothesis opportunistically based on the sequentially observed samples. From a sequential decision perspective, ODRs are also mixtures of one-sided and truncated sequential detection rules. Several results regarding ODRs are established in this paper. In the finite regime, the maximum sample size is modeled either as a fixed finite number, or a geometric random variable with a fixed finite mean. For both cases, the corresponding Bayesian formulations are investigated. The former case is a slight variation of the well-known finite-length sequential hypothesis testing procedure in the literature, whereas the latter case is new, for which the Bayesian optimal ODR is shown to be a sequence of likelihood ratio threshold tests with two different thresholds. A running threshold, which is determined by solving a stationary state equation, is used when future samples are still available, and a terminal threshold (simply the ratio between the priors scaled by costs) is used when the statistician reaches the final sample and, thus, has to make a decision immediately. In the asymptotic regime, the tradeoff among the exponents of the (false alarm and miss) error probabilities and the normalized expected stopping time under the alternative hypothesis is completely characterized and proved to be tight, via an information-theoretic argument. Within the tradeoff region, one noteworthy fact is that the performance of the Stein-Chernoff lemma is attainable by ODRs. [ABSTRACT FROM AUTHOR]

Explicit characterization of the capacity region of communication networks is a long-standing problem. While it is known that network coding can outperform routing and replication, the set of feasible rates is not known in general. Characterizing the network coding capacity region requires the determination of the set of all entropic vectors. Furthermore, computing the explicitly known linear programming bound is infeasible in practice due to an exponential growth in complexity as a function of network size. This paper focuses on the fundamental problems of characterization and computation of outer bounds for multi-source multi-sink networks. Starting from the known local functional dependence induced by the communication network, we introduce the notion of irreducible sets, which characterize implied functional dependence. We provide recursions for the computation of all maximal irreducible sets. These sets act as information-theoretic bottlenecks, and provide an easily computable outer bound for networks with correlated sources. We extend the notion of irreducible sets (and resulting outer bound) for networks with independent sources. We compare our bounds with existing bounds in the literature. We find that our new bounds are the best among the known graph theoretic bounds for networks with correlated sources and for networks with independent sources. [ABSTRACT FROM AUTHOR]

This paper shows that the capacity region of the continuous-time Poisson broadcast channel is achieved via superposition coding for most channel parameter values. Interestingly, the channel in some subset of these parameter values does not belong to any of the existing classes of broadcast channels for which superposition coding is optimal (e.g., degraded, less noisy, and more capable). In particular, we introduce the notion of effectively less noisy broadcast channel and show that it implies less noisy but is not in general implied by more capable. For the rest of the channel parameter values, we show that there is a gap between Marton’s inner bound and the UV outer bound. [ABSTRACT FROM AUTHOR]

Let $A$ be an n \times n$ matrix, X$ be an n \times p$ matrix, and Y = AX$ . A challenging and important problem in data analysis, motivated by dictionary learning and other practical problems, is to recover both A $ and X$ , given Y$ . Under normal circumstances, it is clear that this problem is underdetermined. However, in the case, when X$ is sparse and random, Spielman et al. showed that one can recover both A and they conjectured that p \ge C n \log n$ suffices. The bound n \log n$ is sharp for an obvious information theoretical reason. In this paper, we show that suffices, matching the conjectural bound up to a polylogarithmic factor. The core of our proof is a theorem concerning l_{1} concentration of random matrices, which is of independent interest. Our proof of the concentration result is based on two ideas. The first is an economical way to apply the union bound. The second is a refined version of Bernstein’s concentration inequality for the sum of independent variables. Both have nothing to do with random matrices and are applicable in general settings. [ABSTRACT FROM AUTHOR]

Random forests are ensemble methods which grow trees as base learners and combine their predictions by averaging. Random forests are known for their good practical performance, particularly in high-dimensional settings. On the theoretical side, several studies highlight the potentially fruitful connection between the random forests and the kernel methods. In this paper, we work out this connection in detail. In particular, we show that by slightly modifying their definition, random forests can be rewritten as kernel methods (called KeRF for kernel based on random forests) which are more interpretable and easier to analyze. Explicit expressions of KeRF estimates for some specific random forest models are given, together with upper bounds on their rate of consistency. We also show empirically that the KeRF estimates compare favourably to the random forest estimates. [ABSTRACT FROM AUTHOR]

We consider the transmission of a memoryless bivariate Gaussian source over a two-user additive Gaussian multiple-access channel with unidirectional conferencing encoders. Here, prior to each transmission block, Encoder 1, which observes the first source component, is allowed to communicate with Encoder 2, which observes the second source component, via a unidirectional noise-free bit-pipe of given capacity. The main results of this paper are sufficient conditions and a necessary condition for the achievability of a distortion pair expressed as a function of the channel \sf SNR and of the source correlation. The main sufficient condition is obtained by an extension of the vector-quantizer scheme suggested by Lapidoth–Tinguely, for the case without conferencing, to the case with unidirectional conference. In the high- \sf SNR regime, and when the capacity of the conference channel is unlimited, these necessary and sufficient conditions are shown to agree. We evaluate the precise high- \sf SNR asymptotics for a subset of distortion pairs when the capacity of the conference channel is unlimited in which case we show that a separation-based scheme attains these optimal distortion pairs. However, with symmetric average-power constraints and fixed conferencing capacity, at high- \sf SNR , the latter separation-based scheme is shown to be suboptimal. [ABSTRACT FROM AUTHOR]

Wyner’s common information was originally defined for a pair of dependent discrete random variables. Its significance is largely reflected in, and also confined to, several existing interpretations in various source coding problems. This paper attempts to expand its practical significance by providing a new operational interpretation. In the context of the Gray–Wyner network, it is established that Wyner’s common information has a new lossy source coding interpretation. Specifically, it is established that, under suitable conditions, Wyner’s common information equals to the smallest common message rate when the total rate is arbitrarily close to the rate distortion function with joint decoding for the Gray–Wyner network. A surprising observation is that such equality holds independent of the values of distortion constraints as long as the distortions are within some distortion region. The new lossy source coding interpretation provides the first meaningful justification for defining Wyner’s common information for continuous random variables and the result can also be extended to that of multiple variables. Examples are given for characterizing the rate distortion region for the Gray–Wyner lossy source coding problem and for identifying conditions under which Wyner’s common information equals that of the smallest common rate. As a by-product, the explicit expression for the common information between a pair of Gaussian random variables is obtained. [ABSTRACT FROM AUTHOR]

In this paper, the multiple access channel (MAC) with combined cooperation and partial cribbing is studied, and its capacity region is characterized. Cooperation means that each of the two encoders sends a message to the other via a rate-limited link prior to transmission, while partial cribbing means that each of the two encoders obtains a deterministic function of the other encoder’s output with or without delay. Prior work in this field dealt separately with cooperation and partial cribbing, but by combining these two methods, we can achieve significantly higher rates. Surprisingly, the capacity region of the MAC with combined cooperation and partial cribbing can be expressed using only one auxiliary random variable (RV) similar to the capacity regions of the MAC with cooperation and with partially cribbing encoders. The reason is that in an optimal coding scheme, the encoders use both cooperation and partial cribbing to generate a common message between the encoders. Furthermore, the Gaussian MAC with combined one-sided cooperation and quantized cribbing is studied. For this model, an achievability scheme is given. This scheme shows how many cooperation or quantization bits are required to practically achieve the capacity region of the Gaussian MAC with full message cooperation or perfect cribbing. To ratify the main results, two additional models are studied. In both models, only one auxiliary RV is needed. The first is a rate distortion dual setting for the MAC with degraded message set and combined cooperation and cribbing. The second is a state-dependent MAC with cooperation, where the state is known at a partially cribbing encoder and at the decoder. However, there are cases where more than one auxiliary RV is needed, e.g., when the cooperation and the cribbing are not used for the same purposes. The MAC with an action-dependent state is presented, where the action is based on the cooperation but not on the cribbing. Therefore, in this case, more than one auxiliary RV is needed. As a result, when the common information shared by the two encoders is used unevenly by the users in the channel, more than one auxiliary RV is needed to express the capacity region. [ABSTRACT FROM AUTHOR]

In 1973, Gallager proved that the random-coding bound is exponentially tight for the random code ensemble at all rates, even below expurgation. This result explained that the random-coding exponent does not achieve the expurgation exponent due to the properties of the random ensemble, irrespective of the utilized bounding technique. It has been conjectured that this same behavior holds true for a random ensemble of linear codes. This conjecture is proved in this paper. In addition, it is shown that this property extends to Poltyrev’s random-coding exponent for a random ensemble of lattices. [ABSTRACT FROM PUBLISHER]

This paper introduces the concept of random context representations for the transition probabilities of a finite-alphabet stochastic process. Processes with these representations generalize context tree processes (also known as variable length Markov chains), and are proved to coincide with processes whose transition probabilities are almost surely continuous functions of the (infinite) past. This is similar to a classical result by Kalikow about continuous transition probabilities. Existence and uniqueness of a minimal random context representation are shown, in the sense that there exists a unique representation that looks into the past as little as possible in order to determine the next symbol. Both this representation and the transition probabilities can be consistently estimated from data, and some finite sample adaptivity properties are also obtained (including an oracle inequality). In particular, the estimator achieves minimax performance, up to logarithmic factors, for the class of binary renewal processes whose arrival distributions have bounded moments of order $2+\gamma $ . [ABSTRACT FROM AUTHOR]

In this paper, we study the capacity of the diamond channel. We focus on the special case where the channel between the source node and the two relay nodes are two separate links with finite capacities and the link from the two relay nodes to the destination node is a Gaussian multiple access channel. We call this model the Gaussian multiple access diamond channel. We first propose an upper bound on the capacity. This upper bound is a single-letterization of an $n$ -letter upper bound proposed by Traskov and Kramer, and is tighter than the cut-set bound. As for the lower bound, we propose an achievability scheme based on sending correlated codes through the multiple access channel with superposition structure. We then specialize this achievable rate to the Gaussian multiple access diamond channel. Noting the similarity between the upper and lower bounds, we provide sufficient and necessary conditions that a Gaussian multiple access diamond channel has to satisfy such that the proposed upper and lower bounds meet. Thus, for a Gaussian multiple access diamond channel that satisfies these conditions, we have found its capacity. [ABSTRACT FROM PUBLISHER]

Mrs. Gerber’s Lemma lower bounds the entropy at the output of a binary symmetric channel in terms of the entropy of the input process. In this paper, we lower bound the output entropy via a different measure of input uncertainty, pertaining to the minimum mean squared error prediction cost of the input process. We show that in many cases our bound is tighter than the one obtained from Mrs. Gerber’s Lemma. As an application, we evaluate the bound for binary hidden Markov processes, and obtain new estimates for the entropy rate. [ABSTRACT FROM PUBLISHER]

Let X be an arbitrary continuous random variable and Z be an independent Gaussian random variable with zero mean and unit variance. For t>0 , Costa proved that is concave in t , where the proof hinged on the first and second order derivatives of h(X+\sqrt tZ) . In particular, these two derivatives are signed, i.e., ( \partial / \partial t)h(X+\sqrt tZ) \geq 0 and ( \partial ^2/ \partial t^2)h(X+\sqrt tZ) \leq 0 . In this paper, we show that the third order derivative of h(X+\sqrt tZ) is nonnegative, which implies that the Fisher information J(X+\sqrt tZ) is convex in t . We further show that the fourth order derivative of _{\vphantom {\{}} the first four derivatives, we make two conjectures on h(X+\sqrt tZ) : the first is that ({\partial ^{n}}/{\partial t^{n}}) h(X+\sqrt {t}Z)$ is nonnegative in t$ if n$ is odd, and nonpositive otherwise; the second is that is convex in t$ . The first conjecture can be rephrased in the context of completely monotone functions: t$ . The history of the first conjecture may date back to a problem in mathematical physics studied by McKean in 1966. Apart from these results, we provide a geometrical interpretation to the covariance-preserving transformation and study the concavity of , revealing its connection with Costa’s entropy power inequality. [ABSTRACT FROM PUBLISHER]

Hilberg’s conjecture about natural language states that the mutual information between two adjacent long blocks of text grows like a power of the block length. The exponent in this statement can be upper bounded using the pointwise mutual information estimate computed for a carefully chosen code. The bound is the better, the lower the compression rate is, but there is a requirement that the code be universal. So as to improve a received upper bound for Hilberg’s exponent, in this paper, we introduce two novel universal codes, called the plain switch distribution and the preadapted switch distribution. Generally speaking, switch distributions are certain mixtures of adaptive Markov chains of varying orders with some additional communication to avoid the so-called catch-up phenomenon. The advantage of these distributions is that they both achieve a low compression rate and are guaranteed to be universal. Using the switch distributions, we obtain that a sample of a text in English is non-Markovian with Hilberg’s exponent being ≤0.83, which improves over the previous bound ≤0.94 obtained using the Lempel–Ziv code. [ABSTRACT FROM PUBLISHER]

This paper investigates the maximal channel coding rate achievable at a given blocklength $ n$ and error probability \epsilon $ , when the codewords are subjected to a long-term (i.e., averaged-over-all-codeword) power constraint. The second-order term in the large-n expansion of the maximal channel coding rate is characterized both for additive white Gaussian noise (AWGN) channels and for quasi-static fading channels with perfect channel state information available at both the transmitter and the receiver. It is shown that in both the cases, the second-order term is proportional to ({n ^{-1}\ln n })^{1/2} . For the quasi-static fading case, this second-order term is achieved by truncated channel inversion, namely, by concatenating a dispersion-optimal code for an AWGN channel subject to a short-term power constraint, with a power controller that inverts the channel whenever the fading gain is above a certain threshold. Easy-to-evaluate approximations of the maximal channel coding rate are developed for both the AWGN and the quasi-static fading case. [ABSTRACT FROM AUTHOR]

We consider the problem of channel state amplification in a Gaussian channel with additive Gaussian channel states, where the encoder observes noncausally a noisy version of these states. A complete characterization is provided for the minimum reconstruction distortion under a transmitter power constraint, and it is shown that a simple analog scheme with power control is optimal. More precisely, if the power available to the encoder is below certain threshold, the analog scheme using full power is optimal, however, when the power available to the encoder is above that threshold, analog transmission using only a fixed amount of the available power is optimal. Furthermore, the problem of simultaneous message transmission and Gaussian state amplification with noisy observations is studied, for which an inner bound and two nontrivial outer bounds to the optimal tradeoff between the transmission rate and the state reconstruction distortion are provided. The coding scheme underlying the inner bound combines analog signaling and Gelfand-Pinsker coding, where the latter deviates from the operating point of Costa’s dirty paper coding. The first outer bound is obtained by extending the channel decomposition technique, while the second outer bound requires a strategic analysis of the covariance matrix of the relevant random variables. [ABSTRACT FROM AUTHOR]

In 1733, Georges-Louis Leclerc, Comte de Buffon in France, set the ground of geometric probability theory by defining an enlightening problem: what is the probability that a needle thrown randomly on a ground made of equispaced parallel strips lies on two of them? In this paper, we show that the solution to this problem, and its generalization to N dimensions, allows us to discover a quantized form of the Johnson-Lindenstrauss (JL) lemma, i.e., one that combines a linear dimensionality reduction procedure with a uniform quantization of precision \delta >0 . In particular, given a finite set \mathcal S \subset \mathbb R ^N of S points and a distortion level \epsilon >0 , as soon as M > M_{0} = O(\epsilon ^{-2} \log S) , we can (randomly) construct a mapping from ( \mathcal S, \ell _{2}) to ( \delta \mathbb Z^M, \ell 1) that approximately preserves the pairwise distances between the points of \mathcal S . Interestingly, compared with the common JL lemma, the mapping is quasi-isometric and we observe both an additive and a multiplicative distortions on the embedded distances. These two distortions, however, decay as O((\log S/M)^1/2) when M increases. Moreover, for coarse quantization, i.e., for high \delta compared with the set radius, the distortion is mainly additive, while for small \delta we tend to a Lipschitz isometric embedding. Finally, we prove the existence of a nearly quasi-isometric embedding of into ( \delta \mathbb Z^{M}, \ell _{2}) . This one involves a non-linear distortion of the \ell _{2}$ -distance in \mathcal S$ that vanishes for distant points in this set. Noticeably, the additive distortion in this case is slower, and decays as O((\log S/M)^{1/4}) . [ABSTRACT FROM PUBLISHER]

Non-locality is the phenomenon of observing strong correlations among the outcomes of local measurements of a multipartite physical system. No-signaling boxes are the abstract objects for studying non-locality, and wirings are local operations on the space of no-signaling boxes. This means that, no matter how non-local the nature is, the set of physical non-local correlations must be closed under wirings. Then, one approach to identify the non-locality of nature is to characterize the closed sets of non-local correlations. Although non-trivial examples of wirings of no-signaling boxes are known, there is no systematic way to study wirings. In particular, given a set of no-signaling boxes, we do not know a general method to prove that it is closed under wirings. In this paper, we propose the first general method to construct such closed sets of non-local correlations. We show that a well-known measure of correlation, called maximal correlation, when appropriately defined for non-local correlations, is monotonically decreasing under wirings. This establishes a conjecture about the impossibility of simulating isotropic boxes from each other, implying the existence of a continuum of closed sets of non-local boxes under wirings. To prove our main result, we introduce some mathematical tools that may be of independent interest: we define a notion of maximal correlation ribbon as a generalization of maximal correlation, and provide a connection between it and a known object called hypercontractivity ribbon; we show that these two ribbons are monotone under wirings too. [ABSTRACT FROM AUTHOR]

We consider two secret key generation problems under a pairwise independent network model, and propose low complexity key generation schemes in a framework that connects our problems to network flow problems in graphs. Our schemes have two components: 1) local key generation and 2) global key propagation. In the local key generation, we use point-to-point source coding with side information to establish pairwise keys, from which we construct a graph with the capacity of each edge being the key rate of the corresponding point-to-point local key. In the global key propagation, depending on the particular problem, secret keys are delivered to users in the network using various network flow algorithms. In particular, in the first problem in which one is required to generate a group key for a group of users in the network, we propose a network coding-based global key propagation approach. This approach has a low complexity and has a better performance than the existing approach. In the second problem, in which one is required to generate multiple keys simultaneously for different pairs of users, we propose a multicommodity flow-based global key propagation approach. We show that the proposed approach is optimal for the case of generating two keys. For the general case of generating more than two keys, we show that the sum rate of the proposed scheme is larger than an upper bound characterized in this paper divided by a constant. [ABSTRACT FROM AUTHOR]

This paper studies a Shannon-theoretic version of the generalized distribution preserving quantization problem where a stationary and memoryless source is encoded subject to a distortion constraint and the additional requirement that the reproduction also be stationary and memoryless with a given distribution. The encoder and decoder are stochastic and assumed to have access to independent common randomness. Recent work has characterized the minimum achievable coding rate at a given distortion level when unlimited common randomness is available. Here, we consider the general case where the available common randomness may be rate limited. Our main result completely characterizes the set of achievable coding and common randomness rate pairs at any distortion level, thereby providing the optimal tradeoff between these two rate quantities. We also consider two variations of this problem where we investigate the effect of relaxing the strict output distribution constraint and the role of private randomness used by the decoder on the rate region. Our results have strong connections with Cuff’s recent work on distributed channel synthesis. In particular, our achievability proof combines a coupling argument with the approach developed by Cuff, where instead of explicitly constructing the encoder–decoder pair, a joint distribution is constructed from which a desired encoder–decoder pair is established. We show, however, that for our problem, the separated solution of first finding an optimal channel and then synthesizing this channel results in a suboptimal rate region. [ABSTRACT FROM AUTHOR]

In this paper, we extend the notion of Cauchy–Schwarz divergence to point processes and establish that the Cauchy–Schwarz divergence between the probability densities of two Poisson point processes is half the squared L^2 -distance between their intensity functions. Extension of this result to mixtures of Poisson point processes and, in the case where the intensity functions are Gaussian mixtures, closed form expressions for the Cauchy–Schwarz divergence are presented. Our result also implies that the Bhattacharyya distance between the probability distributions of two Poisson point processes is equal to the square of the Hellinger distance between their intensity measures. We illustrate the result via a sensor management application where the system states are modeled as point processes. [ABSTRACT FROM PUBLISHER]

This paper studies the fundamental limits of the minimum average length of lossless and lossy variable-length compression, allowing a nonzero error probability \epsilon , for lossless compression. We give nonasymptotic bounds on the minimum average length in terms of Erokhin’s rate-distortion function and we use those bounds to obtain a Gaussian approximation on the speed of approach to the limit, which is quite accurate for all but small blocklengths: (1 - \epsilon ) k H(\mathsf S) - {({({k V(\mathsf S)}/{2 \pi })})}^{1/2} \mathop {\rm exp}[- ({( Q^{-1}\left ({\epsilon }\right ))^{2}}/{2})] , where Q^{-1}\left ({\cdot }\right )$ is the functional inverse of the standard Gaussian complementary cumulative distribution function, and $V(\mathsf S)$ is the source dispersion. A nonzero error probability thus not only reduces the asymptotically achievable rate by a factor of $1 - \epsilon $ , but this asymptotic limit is approached from below, i.e., larger source dispersions and shorter blocklengths are beneficial. Variable-length lossy compression under an excess distortion constraint is shown to exhibit similar properties. [ABSTRACT FROM PUBLISHER]

This paper studies a decentralized wireless network of asynchronous transmitter–receiver pairs with burst transmissions. Each receiver learns about the number of active users, channel coefficients, and mutual delays based on locally available measurements. The estimates for the mutual delays are not perfect, however, they are reliable enough to guarantee successful decoding. Two signalling schemes are addressed, namely, randomized masking (RM) and reduced cycle transmission (RCT). Under RM, the $n$ symbols of a codeword are generated according to a Bernoulli–Gaussian distribution with activity factor $0<\theta \leq 1$ . This is in contrast to RCT where each codeword consists of $\lfloor \theta n\rfloor $ Gaussian symbols followed by $n-\lfloor \theta n\rfloor $ zeros. Assuming the transmitters are unaware of the number of users, channel coefficients, and mutual delays, the probability of outage under RM is considerably lower compared with RCT if the signal-to-noise ratio (SNR) is sufficiently large. A generalized RCT scheme is also examined where the $n-\lfloor \theta n\rfloor $ zero symbols are not necessarily located at the end of a codeword. In the asymptote of large SNR, the outage probability becomes vanishingly small under RM, however, it is bounded away from zero for generalized RCT regardless of the value of SNR. [ABSTRACT FROM PUBLISHER]

This paper shows that the Maddah-Ali–Tse (MAT) scheme, which achieves the symmetric capacity of two example broadcast channels with strictly causal state information at the transmitter, is a simple special case of the Shayevitz–Wigger (SW) scheme for the broadcast channel with generalized feedback, which involves block Markov coding, Gray–Wyner compression, superposition coding, and Marton coding. Focusing on the class of symmetric broadcast channels with state, we derive an expression for the maximum achievable symmetric rate using the SW scheme. We show that the MAT results for the two-receiver case can be recovered by evaluating this expression for the special case in which superposition coding and Marton coding are not used. We then introduce a new broadcast channel example that shares many features of the MAT examples. We show that another special case of our maximum symmetric rate expression in which superposition coding is also used attains a higher symmetric rate than the MAT scheme. The symmetric capacity of this new example is not known, however. [ABSTRACT FROM PUBLISHER]

The $H$ -transforms are integral transforms that involve Fox’s $H$ -functions as kernels. A large variety of integral transforms can be put into particular forms of the $H$ -transform since $H$ -functions subsume most of the known special functions including Meijer’s $G$ -functions. In this paper, we embody the $H$ -transform theory into a unifying framework for modeling and analysis in wireless communication. First, we systematize the use of elementary identities and properties of the $H$ -transform by introducing operations on parameter sequences of $H$ -functions. We then put forth $H$ -fading and degree-2 irregular $H$ -fading to model radio propagation under composite, specular, and/or inhomogeneous conditions. The $H$ -fading describes composite effects of multipath fading and shadowing as a single $H$ -variate, including most of typical models such as Rayleigh, Nakagami- $m$ , Weibull, $ \alpha - \mu $ , $N\ast $ Nakagami- $m$ , (generalized) $K$ -fading, and Weibull/gamma fading as its special cases. As a new class of $H$ -variates (called the degree- $ \zeta $ irregular $H$ -variate), the degree-2 irregular $H$ -fading characterizes specular and/or inhomogeneous radio propagation in which the multipath component consists of a strong specularly reflected or line-of-sight (LOS) wave as well as unequal-power or correlated in-phase and quadrature scattered waves. This fading includes a variety of typical models such as Rician, Nakagami- $q$ , $ \kappa - \mu $ , $ \eta - \mu $ , Rician/LOS gamma, and $ \kappa - \mu $ /LOS gamma fading as its special cases. Finally, we develop a unifying $H$ -transform analysis for the amount of fading, error probability, channel capacity, and error exponent in wireless communication using the new systematic language of transcendental $H$ -functions. By virtue of two essential operations—called Mellin and convolution operations—involved in the Mellin transform and Mellin convolution of two $H$ -functions, the $H$ -transforms for these performance measures culminate in $H$ -functions. Using the algebraic asymptotic expansions of the $H$ -transform, we further analyze the error probability and capacity at high and low signal-to-noise ratios in a unified fashion. [ABSTRACT FROM PUBLISHER]

In a network, a node is said to incur a delay if its encoding of each transmitted symbol involves only its received symbols obtained before the time slot in which the transmitted symbol is sent (hence the transmitted symbol sent in a time slot cannot depend on the received symbol obtained in the same time slot). A node is said to incur no delay if its received symbol obtained in a time slot is available for encoding its transmitted symbol sent in the same time slot. Under the classical model, every node in a discrete memoryless network (DMN) incurs a unit delay, and the capacity region of the DMN satisfies the well-known cut-set outer bound. In this paper, we propose a generalized model for the DMN where some nodes may incur no delay. Under our generalized model, we obtain a new cut-set outer bound, which is proved to be tight for some two-node DMN and is shown to subsume an existing cut-set bound for the causal relay network. In addition, we establish under the generalized model another cut-set outer bound on the positive-delay region—the set of achievable rate tuples under the constraint that every node incurs a delay. We use the cut-set bound on the positive-delay region to show that for some two-node DMN under the generalized model, the positive-delay region is strictly smaller than the capacity region. [ABSTRACT FROM PUBLISHER]

Random key graphs form a class of random intersection graphs that are naturally induced by the random key predistribution scheme of Eschenauer and Gligor for securing wireless sensor network (WSN) communications. Random key graphs have received much attention recently, owing in part to their wide applicability in various domains, including recommender systems, social networks, secure sensor networks, clustering and classification analysis, and cryptanalysis to name a few. In this paper, we study connectivity properties of random key graphs in the presence of unreliable links. Unreliability of graph links is captured by independent Bernoulli random variables, rendering them to be on or off independently from each other. The resulting model is an intersection of a random key graph and an Erdős–Rényi graph, and is expected to be useful in capturing various real-world networks; e.g., with secure WSN applications in mind, link unreliability can be attributed to harsh environmental conditions severely impairing transmissions. We present conditions on how to scale this model’s parameters so that: 1) the minimum node degree in the graph is at least $k$ and 2) the graph is $k$ -connected, both with high probability as the number of nodes becomes large. The results are given in the form of zero-one laws with critical thresholds identified and shown to coincide for both graph properties. These findings improve the previous results by Rybarczyk on $k$ -connectivity of random key graphs (with reliable links), as well as the zero-one laws by Yağan on one-connectivity of random key graphs with unreliable links. [ABSTRACT FROM PUBLISHER]

This paper studies variable-length (VL) source coding of general sources with side-information. Novel one-shot coding bounds for Slepian–Wolf (SW) coding, which give nonasymptotic tradeoff between the error probability and the codeword length of VL-SW coding, are established. One-shot results are applied to asymptotic analysis, and a general formula for the optimal coding rate achievable by weakly lossless VL-SW coding (i.e., VL-SW coding with vanishing error probability) is derived. Our general formula reveals how the encoder side-information and/or VL coding improve the optimal coding rate in the general setting. In addition, it is shown that if the encoder side-information is useless in weakly lossless VL coding then it is also useless even in the case where the error probability may be positive asymptotically. [ABSTRACT FROM AUTHOR]

In this paper, we consider the fluctuation of mutual information statistics of a multiple input multiple output channel communication systems without assuming that the entries of the channel matrix have zero pseudovariance. To this end, we also establish a central limit theorem of the linear spectral statistics for sample covariance matrices under general moment conditions by removing the restrictions imposed on the second moment and fourth moment on the matrix entries in Bai and Silverstein (2004). [ABSTRACT FROM AUTHOR]

We consider the basic bidirectional relaying problem, in which two users in a wireless network wish to exchange messages through an intermediate relay node. In the compute-and-forward strategy, the relay computes a function of the two messages using the naturally occurring sum of symbols simultaneously transmitted by user nodes in a Gaussian multiple-access channel (MAC), and the computed function value is forwarded to the user nodes in an ensuing broadcast phase. In this paper, we study the problem under an additional security constraint, which requires that each user’s message be kept secure from the relay. We consider two types of security constraints: 1) perfect secrecy, in which the MAC channel output seen by the relay is independent of each user’s message and 2) strong secrecy, which is a form of asymptotic independence. We propose a coding scheme based on nested lattices, the main feature of which is that given a pair of nested lattices that satisfy certain goodness properties, we can explicitly specify probability distributions for randomization at the encoders to achieve the desired security criteria. In particular, our coding scheme guarantees perfect or strong secrecy even in the absence of channel noise. The noise in the channel only affects reliability of computation at the relay, and for Gaussian noise, we derive achievable rates for reliable and secure computation. We also present an application of our methods to the multihop line network in which a source needs to transmit messages to a destination through a series of intermediate relays. [ABSTRACT FROM AUTHOR]

This paper shows the strong converse and the dispersion of memoryless channels with cost constraints and performs a refined analysis of the third-order term in the asymptotic expansion of the maximum achievable channel coding rate, showing that it is equal to (1/ 2)((\log n)/n) in most cases of interest. The analysis is based on a nonasymptotic converse bound expressed in terms of the distribution of a random variable termed the $\mathsf b$ -tilted information density, which plays a role similar to that of the $\mathsf d$ -tilted information in lossy source coding. We also analyze the fundamental limits of lossy joint-source-channel coding over channels with cost constraints. [ABSTRACT FROM AUTHOR]

This paper considers relay assisted transmission for multiuser networks when the relay has no access to the codebooks used by the transmitters. The relay is called oblivious for this reason. Of particular interest is the generalized compress-and-forward (GCF) strategy, where the destinations jointly decode the compression indices and the transmitted messages, and their optimality in this setting. The relay-to-destination links are assumed to be out-of-band with finite capacity. Two models are investigated: 1) the multiple access relay channel (MARC) and 2) the interference relay channel (IFRC). For the MARC with an oblivious relay, a new outerbound is derived and it is shown to be tight by means of achievability of the capacity region using GCF scheme. For the IFRC with an oblivious relay, a new strong interference condition is established, under which the capacity region is found by deriving a new outerbound and showing that it is achievable using GCF scheme. The result is further extended to establish the capacity region of M-user MARC with an oblivious relay, and multicast networks containing $M$ sources and $K$ destinations with an oblivious relay. [ABSTRACT FROM PUBLISHER]

In coding schemes for the wire-tap channel or for broadcast channels with confidential messages, it is well-known that the sender needs to use stochastic encoding to avoid information about the transmitted confidential message from being leaked to an eavesdropper. In this paper, we investigate the tradeoff between the rate of random numbers needed to realize the stochastic encoding and the rates of common, private, and confidential messages. For the direct theorem, we use the superposition coding scheme for the wire-tap channel, recently proposed by Chia and El Gamal, and its strong security is proved. The matching converse theorem is also established. Our result clarifies that a combination of ordinary stochastic encoding and channel prefixing by channel simulation is suboptimal. [ABSTRACT FROM AUTHOR]

Information theory provides principled ways to analyze different inference and learning problems, such as hypothesis testing, clustering, dimensionality reduction, classification, and so forth. However, the use of information theoretic quantities as test statistics, that is, as quantities obtained from empirical data, poses a challenging estimation problem that often leads to strong simplifications, such as Gaussian models, or the use of plug in density estimators that are restricted to certain representation of the data. In this paper, a framework to nonparametrically obtain measures of entropy directly from data using operators in reproducing kernel Hilbert spaces defined by infinitely divisible kernels is presented. The entropy functionals, which bear resemblance with quantum entropies, are defined on positive definite matrices and satisfy similar axioms to those of Renyi’s definition of entropy. Convergence of the proposed estimators follows from concentration results on the difference between the ordered spectrum of the Gram matrices and the integral operators associated to the population quantities. In this way, capitalizing on both the axiomatic definition of entropy and on the representation power of positive definite kernels, the proposed measure of entropy avoids the estimation of the probability distribution underlying the data. Moreover, estimators of kernel-based conditional entropy and mutual information are also defined. Numerical experiments on independence tests compare favorably with state-of-the-art. [ABSTRACT FROM PUBLISHER]

This paper studies sequential methods for recovery of sparse signals in high dimensions. When compared with fixed sample size procedures, in the sparse setting, sequential methods can result in a large reduction in the number of samples needed for reliable signal support recovery. Starting with a lower bound, we show any coordinate-wise sequential sampling procedure fails in the high dimensional limit provided the average number of measurements per dimension is less then \log (s)/D(P0||P1) , where s is the level of sparsity and D(P_{0}||P_{1}) is the Kullback–Leibler divergence between the underlying distributions. A series of sequential probability ratio tests, which require complete knowledge of the underlying distributions is shown to achieve this bound. Motivated by real-world experiments and recent work in adaptive sensing, we introduce a simple procedure termed sequential thresholding, which can be implemented when the underlying testing problem satisfies a monotone likelihood ratio assumption. Sequential thresholding guarantees exact support recovery provided the average number of measurements per dimension grows faster than \log (s)/D(P0||P1) , achieving the lower bound. For comparison, we show any nonsequential procedure fails provided the number of measurements grows at a rate less than \log (n)/D(P_{1}||P_{0})$ , where $n$ is the total dimension of the problem. [ABSTRACT FROM AUTHOR]

The Rényi entropy of general order unifies the well-known Shannon entropy with several other entropy notions, like the min-entropy or collision entropy. In contrast to the Shannon entropy, there seems to be no commonly accepted definition for the conditional Rényi entropy: several versions have been proposed and used in the literature. In this paper, we reconsider the definition for the conditional Rényi entropy of general order as proposed by Arimoto in the seventies. We show that this particular notion satisfies several natural properties. In particular, we show that it satisfies monotonicity under conditioning, meaning that conditioning can only reduce the entropy, and (a weak form of) chain rule, which implies that the decrease in entropy due to conditioning is bounded by the number of bits one conditions on. None of the other suggestions for the conditional Rényi entropy satisfies both these properties. Finally, we show a natural interpretation of the conditional Rényi entropy in terms of (unconditional) Rényi divergence, and we show consistency with a recently proposed notion of conditional Rényi entropy in the quantum setting. [ABSTRACT FROM PUBLISHER]

Information theoretical inequalities have strong ties with polymatroids and their representability. A polymatroid is entropic if its rank function is given by the Shannon entropy of the subsets of some discrete random variables. The book is a special iterated adhesive extension of a polymatroid with the property that entropic polymatroids have \(n\) -page book extensions over an arbitrary spine. We prove that every polymatroid has an \(n\) -page book extension over a single element and over an all-but-one-element spine. Consequently, for polymatroids on four elements, only book extensions over a two-element spine should be considered. Matúš proved that the Zhang–Yeung inequalities characterize polymatroids on four elements which have such a two-page book extension. The \(n\) -page book inequalities, defined in this paper, are conjectured to characterize polymatroids on four elements which have \(n\) -page book extensions over a two-element spine. We prove that the condition is necessary; consequently, every book inequality is an information inequality on four random variables. Using computer-aided multiobjective optimization, the sufficiency of the condition is verified up to nine-page book extensions. [ABSTRACT FROM PUBLISHER]

We consider a cooperative two-user multiaccess channel in which the transmission is controlled by a random state. Both encoders transmit a common message and, one of the encoders also transmits an individual message. We study the capacity region of this communication model for different degrees of availability of the states at the encoders, causally or strictly causally. In the case in which the states are revealed causally to both encoders but not to the decoder we find an explicit characterization of the capacity region in the discrete memoryless case. In the case in which the states are revealed only strictly causally to both encoders, we establish inner and outer bounds on the capacity region. The outer bound is nontrivial, and has a relatively simple form. It has the advantage of incorporating only one auxiliary random variable. In particular, it suggests that there is none, or at best only little, to gain from having the encoder that transmits both messages also sending an individual description of the state to the receiver, in addition to the compressed version that is sent cooperatively with the other encoder. We then introduce a class of cooperative multiaccess channels with states known strictly causally at both encoders for which the inner and outer bounds agree, and so we characterize the capacity region for this class. In this class of channels, the state can be obtained as a deterministic function of the channel inputs and output. We also study the model in which the states are revealed, strictly causally, in an asymmetric manner, to only one encoder. Throughout this paper, we discuss a number of examples, and compute the capacity region of some of these examples. The results shed more light on the utility of delayed channel state information for increasing the capacity region of state-dependent cooperative multiaccess channels, and tie with recent progress in this framework. [ABSTRACT FROM AUTHOR]

We show how to efficiently simulate the sending of a single message \(M\) to a receiver who has partial information about the message, so that the expected number of bits communicated in the simulation is close to the amount of additional information that the message reveals to the receiver. This is a generalization and strengthening of the Slepian–Wolf theorem, which shows how to carry out such a simulation with low amortized communication in the case that \(M\) is a deterministic function of \(X\) . A caveat is that our simulation is interactive. As a consequence, we prove that the internal information cost (namely the information revealed to the parties) involved in computing any relation or function using a two party interactive protocol is exactly equal to the amortized communication complexity of computing independent copies of the same relation or function. We also show that the only way to prove a strong direct sum theorem for randomized communication complexity is by solving a particular variant of the pointer jumping problem that we define. This paper implies that a strong direct sum theorem for communication complexity holds if and only if efficient compression of communication protocols is possible. In particular, together with our result, a recent result of Ganor, Kol, and Raz implies that the strongest version of direct sum for randomized communication complexity is false. [ABSTRACT FROM AUTHOR]

In this paper, we revisit the structure of infeasibility results in network information theory, based on a notion of information state. We also discuss ideas for generalizing a known outer bound for lossless transmission of independent sources over a network to one of lossy transmission of dependent sources over the same network. To concretely demonstrate this, we apply our ideas and prove new results for lossy transmission of dependent sources by generalizing: 1) the cut-set bound; 2) the best known outer bound on the capacity region of a general broadcast channel; and 3) the outer bound part of the result of Maric, Yates, and Kramer on strong interference channels with a common message. [ABSTRACT FROM AUTHOR]

This paper studies the impact of interactive fusion on detection performance in tandem fusion networks with conditionally independent observations. Within the Neyman–Pearson framework, two distinct regimes are considered: 1) the fixed sample size test and 2) the large sample test. For the former, it is established that interactive distributed detection may strictly outperform the one-way tandem fusion structure. However, for the large sample regime, it is shown that interactive fusion has no improvement on the asymptotic performance characterized by the Kullback–Leibler distance compared with the simple one-way tandem fusion. The results are then extended to interactive fusion systems where the fusion center and the sensor may undergo multiple steps of memoryless interactions or that involve multiple peripheral sensors, as well as to interactive fusion with soft sensor outputs. [ABSTRACT FROM AUTHOR]

We study a two-transmitter two-receiver network where Receiver 1 can only hear the transmitted signal of Transmitter 1. Transmitter 1 has two messages, one of which is intended for Receiver 1 while both are intended for Receiver 2. Transmitter 2 has one message which is intended for Receiver 2. We call this channel model the Z channel with degraded message sets. For networks with the element of distributed encoding, it has been shown that when the multiple access link between the two encoders and the decoder satisfies the conditions proposed by Liu and Goldsmith, capacity results can be obtained for a variety of problems. In this paper, we generalize these conditions and show that for a larger class of multiple access links, the capacity region of the Z channel with degraded message sets can be characterized despite the presence of distributed encoding. [ABSTRACT FROM AUTHOR]

The two-user compound Gaussian MIMO broadcast channel models the situation where each user has a finite set of possible realizations. The transmitter sends two messages, one for each user, such that each user must be able to decode its message regardless of the actual realization. This channel also models a broadcast channel (BC) with two groups of users and two messages, with one message intended for each group of users. Weingarten et al. established the capacity region for the degraded case where the realizations/users exhibit a degradedness order. The degradedness order is defined through an additional realization/user where the realizations/users from one set are degraded with respect to him and where he is degraded with respect to the realizations/users from the other set. To show that Gaussian inputs attain the capacity region, they proved a new extremal inequality and employed the use of the channel enhancement technique as well. In this paper, we extend the result to the \(N\) -user degraded compound Gaussian MIMO BC, where the \(N\) users exhibit a degradedness order similar to the two-user case. We first prove a generalization of the extremal inequality considered by Weingarten et al.; instead of considering the difference between the weighted sum of two sets of conditional differential entropies, we consider the summation of \(N-1\) sets of such differences, where the conditioning random variables of the \(N-1\) sets form a Markov chain. Our proof relies on the Gaussian perturbation approach, the necessary KKT conditions as well as a data processing inequality. Finally, we specialize the generalized extremal inequality to characterize the capacity region of the \(N\) -user degraded compound Gaussian MIMO BC. By making appropriate use of the necessary KKT conditions, we are able to do away with the use of the channel enhancement technique that was employed in the proof of the capacity region of the two-user case. [ABSTRACT FROM AUTHOR]

This paper addresses a two-user Gaussian multiple access channel (MAC) under peak power constraints at the transmitters. It is shown that generating the code-books of both users according to discrete distributions with a finite number of mass points achieves the largest weighted sum-rate in the network. This verifies that any point on the boundary of the capacity region of a two-user MAC under peak power constraints at both transmitters is achieved by discrete distributions with a finite number of mass points. Although the capacity-achieving distributions are not necessarily unique, it is verified that only discrete distributions with a finite number of mass points can achieve a point on the boundary of the capacity region. It is shown that there exist an infinite number of sum-rate-optimal points on the boundary of the capacity region. In contrast to the Gaussian MAC with average power constraints, we verify that time division (TD) cannot achieve any of the sum-rate-optimal points in the Gaussian MAC with peak power constraints. Using the so-called I-MMSE identity of Guo et al., the largest achievable sum-rate by orthogonal code division (OCD) is characterized where it is shown that Walsh–Hadamard spreading codes of length 2 are optimal. In the symmetric case where the peak power constraints at both transmitters are identical, we verify that OCD can achieve a sum-rate that is strictly larger than the highest sum-rate achieved by TD. Finally, it is demonstrated that there are values for the maximum peak power at the transmitters such that OCD can not achieve any of the sum-rate-optimal points on the boundary of the capacity region. [ABSTRACT FROM AUTHOR]

The problem of simultaneously generating a secret key (SK) and private key (PK) pair among three terminals via public discussion is investigated. In this problem, each terminal observes a component of correlated sources. All three terminals are required to generate the common SK to be concealed from an eavesdropper that has access to the public discussion, while two designated terminals are required to generate an extra PK to be concealed from both the eavesdropper and the remaining terminal. An outer bound on the SK-PK capacity region was established by Ye and Narayan, and was shown to be achievable for a special case. In this paper, the SK-PK capacity region is established in general by developing schemes to achieve the outer bound for the remaining two cases. The main technique lies in the novel design of a random binning-joint decoding scheme that achieves the existing outer bound. [ABSTRACT FROM AUTHOR]

A computable expression for Heegard and Berger’s rate-distortion function has eluded information theory for nearly three decades. Heegard and Berger’s single-letter achievability bound is well known to be optimal for physically degraded side information; however, it is not known whether the bound is optimal for arbitrarily correlated side information (general discrete memoryless sources). In this paper, we consider a new setup where the side information at one receiver is conditionally less noisy than that at the other. The new setup includes degraded side information as a special case, and it is motivated by the literature on degraded and less noisy broadcast channels. Our key contribution is a converse proving the optimality of Heegard and Berger’s achievability bound in a new setting, where the side information is conditionally less noisy and one distortion function is deterministic. The less noisy setup is also generalized to two different successive-refinement problems. [ABSTRACT FROM AUTHOR]