Search

In multi-source multi-terminal key-dissemination, here called ``key-cast,'' introduced by the authors in [ITW2022], network nodes hold independent random bits, and one seeks a communication scheme that allows all terminal nodes to share a secret key K. The work at hand addresses positive (albeit, arbitrarily small) rate key-cast under the security requirement that no single non-terminal network node can gain information about the shared key K; this scenario is useful in cryptographic settings. Specifically, key-dissemination protocols based on secure multicast network coding are designed. The analysis presented yields two combinatorial characterizations. In each, we assume a network in which an eavesdropper may access any individual network node. The first characterization captures all networks that support positive-rate secure multicast; computing the secure-multicast capacity in the setting studied is a known open problem. The second characterizes all networks that support positive-rate secure key-cast.

Motivated by the design of low-complexity low-power coding solutions for the Gaussian relay channel, this work presents an upper bound on the minimum energy-per-bit achievable on the Gaussian relay channel using rank-1 linear relaying. Our study addresses high-dimensional relay codes and presents bounds that outperform prior known bounds using 2-dimensional schemes. A novelty of our analysis ties the optimization problem at hand to the solution of a certain differential equation which, in turn, leads to a low energy-per-bit achievable scheme., Comment: Submitted to the IEEE International Symposium on Information Theory, 2024. 13 pages, 1 figure

The multiple access channel (MAC) capacity with feedback is considered under feedback models designed to tease out which factors contribute to the MAC feedback capacity benefit. Comparing the capacity of a MAC with ``perfect'' feedback, which causally delivers to the transmitters the true channel output, to that of a MAC with ``independent'' feedback, which causally delivers to the transmitters an independent instance of that same channel output, allows separation of effects like cooperation from alternative feedback benefits such as knowledge of the channel instance. Proving that the Cover-Leung (CL) achievability bound, which is known to be loose for some channels, is achievable also under (shared or distinct) independent feedback at the transmitters shows that the CL bound does not require transmitter knowledge of the channel instance. Proving that each transmitter's maximal rate under independent feedback exceeds that under perfect feedback highlights the potential power of an independent look at the channel output., Comment: 9 pages, to appear at the International Symposium on Information Theory 2023

The multicast key-dissemination problem over noiseless networks, introduced by Langberg and Effros [ITW 2022], here called the ``key-cast'' problem, captures the task of disseminating a shared secret random key to a set of terminals over a given network. Unlike traditional communication, where messages must be delivered from source to destination(s) unchanged, key-cast is more flexible since key-cast need not require source reconstruction at destination nodes. For example, the distributed keys can be mixtures of sources from which the sources themselves may be unrecoverable. The work at hand considers key dissemination in the single-source, multiple-multicast network coding setting, i.e., the ``multiple key-cast'' problem. Here, distinct keys are to be simultaneously transmitted from a single source node to multiple terminal sets, one shared random key per multicast set. Scenarios include the secure setting, in which only the source and intended destinations gain information about a given key; and the non-secure setting in which the only requirement is that the knowledge of one key does not reveal information about another. In both settings, we present combinatorial conditions for key dissemination and design corresponding multiple key-cast schemes. In addition, we compare the multiple key-cast rate with and without the restriction of source reconstruction, the former corresponding to traditional forms of communication; key-cast achieves a strict advantage in rate when source reconstruction is relaxed.

Towards the informed design of large-scale distributed data-gathering architectures under real-world assumptions such as nonzero communication delays and unknown environment dynamics, this paper considers the effects of allowing feedback communication from the central processor to external sensors. Using simple but representative state-estimation examples, we investigate fundamental tradeoffs between the mean-squared error (MSE) of the central processor's estimate of the environment state, and the total power expenditure per sensor under more conventional architectures without feedback (INformation) versus those with broadcast feedback (OUTformation). The primary advantage of enabling feedback is that each sensor's understanding of the central processor's estimate improves, which enables each sensor to determine when and what parts of its current observations to transmit. We use theory to demonstrate conditions in which OUTformation maintains the same MSE as INformation with less power expended on average, and conditions in which OUTformation obtains less MSE than INformation at additional power cost. These performance tradeoffs are also considered under settings where environments undergo less variation, and sensors implement random backoff times to prevent transmission collisions. Our results are supported via numerical studies, which show that the properties derived in theory still hold even when some of the simplifying assumptions are removed., Comment: Accepted to 61st IEEE Conference on Decision and Control (CDC 2022)

This paper studies the third-order characteristic of nonsingular discrete memoryless channels and the Gaussian channel with a maximal-power constraint. The third-order term in our expansions employs a new quantity here called the channel skewness, which affects the approximation accuracy more significantly as the error probability decreases. For the Gaussian channel, evaluating Shannon's 1959 random coding bound and Vazquez-Vilar's 2021 meta-converse bound in the central limit theorem (CLT) regime enables exact computation of the channel skewness. For discrete memoryless channels, this work generalizes Moulin's 2017 bounds on the asymptotic expansion of the maximum achievable message set size for nonsingular channels from the CLT regime to include the moderate deviations (MD) regime, thereby refining Altu\u{g} and Wagner's 2014 MD result. For an example binary symmetric channel and most practically important $(n, \epsilon)$ pairs, including $n \in [100, 500]$ and $\epsilon \in [10^{-10}, 10^{-1}]$, an approximation up to the channel skewness is the most accurate among several expansions in the literature. A derivation of the third-order term in the type-II error exponent of binary hypothesis testing in the MD regime is also included; the resulting third-order term is similar to the channel skewness., Comment: Presented at ISIT 2022. Published in IEEE Information Theory Transactions, Volume: 70, Issue: 9, September 2024

For a multi-source multi-terminal noiseless network, the key-dissemination problem involves the task of multicasting a secret key K from the network sources to its terminals. As in secure multicast network-coding, in the key-dissemination problem the source nodes have access to independent randomness and, as the network is noiseless, the resulting key K is a function of the sources' information. However, different from traditional forms of multicast, in key-dissemination the key K need not consist of source messages, but rather may be any function of the information generated at the sources, as long as it is shared by all terminals. Allowing the shared key K to be a mixture of source information grants a flexibility to the communication process which gives rise to the potential of increased key-rates when compared to traditional secure multicast. The multicast key-capacity is the supremum of achievable key-rates, subject to the security requirement that the shared key is not revealed to an eavesdropper with predefined eavesdropping capabilities. The key-dissemination problem (termed also, secret key-agreement) has seen significant studies over the past decades in memoryless network structures. In this work, we initiate the study of key-dissemination in the context of noiseless networks, i.e., network coding. In this context, we study similarities and differences between traditional secure-multicast and the more lenient task of key-dissemination.

This work addresses the cooperation facilitator (CF) model, in which network nodes coordinate through a rate limited communication device. For independent multiple-access channel (MAC) encoders, the CF model is known to show significant rate benefits, even when the rate of cooperation is negligible. Specifically, the benefit in MAC sum-rate, as a function of the cooperation rate $C_{CF}$, sometimes has an infinite slope at $C_{CF}=0$. This work studies the question of whether cooperation through a CF can yield similar infinite-slope benefits when applied to internal network encoders in which dependence among MAC transmitters can be established without the help of the CF. Towards this end, this work studies the CF model when applied to relay nodes of a single-source, single-terminal, diamond network consisting of a broadcast channel followed by a MAC. In the relay channel with orthogonal receiver components, careful generalization of the partial-decode-forward/compress-forward lower bound to the CF model yields sufficient conditions for an infinite-slope benefit. Additional results include derivation of a family of diamond networks for which the infinite-slope rate-benefit derives directly from the properties of the corresponding MAC component when studied in isolation., Comment: Full version of a paper submitted to ISIT 2022. 9 pages, 3 figures

This paper investigates variable-length stop-feedback codes for memoryless channels in point-to-point, multiple access, and random access communication scenarios. The proposed codes employ $L$ decoding times $n_1, n_2, \dots, n_L$ for the point-to-point and multiple access channels and $KL + 1$ decoding times for the random access channel with at most $K$ active transmitters. In the point-to-point and multiple access channels, the decoder uses the observed channel outputs to decide whether to decode at each of the allowed decoding times $n_1, \dots, n_L$, at each time telling the encoder whether or not to stop transmitting using a single bit of feedback. In the random access scenario, the decoder estimates the number of active transmitters at time $n_0$ and then chooses among decoding times $n_{k, 1}, \dots, n_{k, L}$ if it believes that there are $k$ active transmitters. In all cases, the choice of allowed decoding times is part of the code design; given fixed value $L$, allowed decoding times are chosen to minimize the expected decoding time for a given codebook size and target average error probability. The number $L$ in each scenario is assumed to be constant even when the blocklength is allowed to grow; the resulting code therefore requires only sparse feedback. The central results are asymptotic approximations of achievable rates as a function of the error probability, the expected decoding time, and the number of decoding times. A converse for variable-length stop-feedback codes with uniformly-spaced decoding times is included for the point-to-point channel., Comment: 29 Pages. Presented at ISIT 2021. Accepted for publication at IEEE Transactions on Information Theory Dec. 2023

The work at hand presents a finite-blocklength analysis of the multiple access channel (MAC) sum-rate under the cooperation facilitator (CF) model. The CF model, in which independent encoders coordinate through an intermediary node, is known to show significant rate benefits, even when the rate of cooperation is limited. We continue this line of study for cooperation rates which are sub-linear in the blocklength $n$. Roughly speaking, our results show that if the facilitator transmits $\log{K}$ bits, there is a sum-rate benefit of order $\sqrt{\log{K}/n}$. This result extends across a wide range of $K$: even a single bit of cooperation is shown to provide a sum-rate benefit of order $1/\sqrt{n}$., Comment: Full version of a paper submitted to ISIT 2021

The traditional notion of capacity studied in the context of memoryless network communication builds on the concept of block-codes and requires that, for sufficiently large blocklength n, all receiver nodes simultaneously decode their required information after n channel uses. In this work, we generalize the traditional capacity region by exploring communication rates achievable when some receivers are required to decode their information before others, at different predetermined times; referred here as the "time-rate" region. Through a reduction to the standard notion of capacity, we present an inner-bound on the time-rate region. The time-rate region has been previously studied and characterized for the memoryless broadcast channel (with a sole common message) under the name "static broadcasting".

The edge-removal problem asks whether the removal of a $\lambda$-capacity edge from a given network can decrease the communication rate between source-terminal pairs by more than $\lambda$. In this short manuscript, we prove that for undirected networks, removing a $\lambda$-capacity edge decreases the rate by $O(\lambda)$. Through previously known reductive arguments, here newly applied to undirected networks, our result implies that the zero-error capacity region of an undirected network equals its vanishing-error capacity region. Whether it is possible to prove similar results for directed networks remains an open question.

This paper applies error-exponent and dispersion-style analyses to derive finite-blocklength achievability bounds for low-density parity-check (LDPC) codes over the point-to-point channel (PPC) and multiple access channel (MAC). The error-exponent analysis applies Gallager's error exponent to bound achievable symmetrical and asymmetrical rates in the MAC. The dispersion-style analysis begins with a generalization of the random coding union (RCU) bound from random code ensembles with i.i.d. codewords to random code ensembles in which codewords may be statistically dependent; this generalization is useful since the codewords of random linear codes such as random LDPC codes are dependent. Application of the RCU bound yields improved finite-blocklength error bounds and asymptotic achievability results for i.i.d. random codes and new finite-blocklength error bounds and achievability results for LDPC codes. For discrete, memoryless channels, these results show that LDPC codes achieve first- and second-order performance that is optimal for the PPC and identical to the best-prior results for the MAC., Comment: 32 pages, 2 figures

Group network codes are a generalization of linear codes that have seen several studies over the last decade. When studying network codes, operations performed at internal network nodes called local encoding functions, are of significant interest. While local encoding functions of linear codes are well understood (and of operational significance), no similar operational definition exists for group network codes. To bridge this gap, we study the connections between group network codes and a family of codes called Coordinate-Wise-Linear (CWL) codes. CWL codes generalize linear codes and, in addition, can be defined locally (i.e., operationally). In this work, we study the connection between CWL codes and group codes from both a local and global encoding perspective. We show that Abelian group codes can be expressed as CWL codes and, as a result, they inherit an operational definition.

This paper presents finite-blocklength achievability bounds for the Gaussian multiple access channel (MAC) and random access channel (RAC) under average-error and maximal-power constraints. Using random codewords uniformly distributed on a sphere and a maximum likelihood decoder, the derived MAC bound on each transmitter's rate matches the MolavianJazi-Laneman bound (2015) in its first- and second-order terms, improving the remaining terms to $\frac12\frac{\log n}{n}+O \left(\frac 1 n \right)$ bits per channel use. The result then extends to a RAC model in which neither the encoders nor the decoder knows which of $K$ possible transmitters are active. In the proposed rateless coding strategy, decoding occurs at a time $n_t$ that depends on the decoder's estimate $t$ of the number of active transmitters $k$. Single-bit feedback from the decoder to all encoders at each potential decoding time $n_i$, $i \leq t$, informs the encoders when to stop transmitting. For this RAC model, the proposed code achieves the same first-, second-, and third-order performance as the best known result for the Gaussian MAC in operation., Comment: 27 pages, IEEE Transactions on Information Theory, ISIT 2020

In communication networks, cooperative strategies are coding schemes where network nodes work together to improve network performance metrics such as the total rate delivered across the network. This work studies encoder cooperation in the setting of a discrete multiple access channel (MAC) with two encoders and a single decoder. A network node, here called the cooperation facilitator (CF), that is connected to both encoders via rate-limited links, enables the cooperation strategy. Previous work by the authors presents two classes of MACs: (i) one class where the average-error sum-capacity has an infinite derivative in the limit where CF output link capacities approach zero, and (ii) a second class of MACs where the maximal-error sum-capacity is not continuous at the point where the output link capacities of the CF equal zero. This work contrasts the power of the CF in the maximal- and average-error cases, showing that a constant number of bits communicated over the CF output link can yield a positive gain in the maximal-error sum-capacity, while a far greater number of bits, even numbers that grow sublinearly in the blocklength, can never yield a non-negligible gain in the average-error sum-capacity., Comment: 30 pages, 1 figure. This article supersedes arXiv:1607.07024 and arXiv:1801.03655

It is common in the study of secure multicast network coding in the presence of an eavesdropper that has access to $z$ network links, to assume that the source node is the only node that generates random keys. In this setting, the secure multicast rate is well understood. Computing the secure multicast rate, or even the secure unicast rate, in the more general setting in which all network nodes may generate (independent) random keys is known to be as difficult as computing the (non-secure) capacity of multiple-unicast network coding instances --- a well known open problem. This work treats an intermediate model of secure unicast in which only one node can generate random keys, however that node need not be the source node. The secure communication rate for this setting is characterized again with an eavesdropper that has access to $z$ network links.

The edge removal problem studies the loss in network coding rates that results when a network communication edge is removed from a given network. It is known, for example, that in networks restricted to linear coding schemes and networks restricted to Abelian group codes, removing an edge e* with capacity Re* reduces the achievable rate on each source by no more than Re*. In this work, we seek to uncover larger families of encoding functions for which the edge removal statement holds. We take a local perspective: instead of requiring that all network encoding functions satisfy certain restrictions (e.g., linearity), we limit only the function carried on the removed edge e*. Our central results give sufficient conditions on the function carried by edge e* in the code used to achieve a particular rate vector under which we can demonstrate the achievability of a related rate vector once e* is removed.

Smart meters enable improvements in electricity distribution system efficiency at some cost in customer privacy. Users with home batteries can mitigate this privacy loss by applying charging policies that mask their underlying energy use. A battery charging policy is proposed and shown to provide universal privacy guarantees subject to a constraint on energy cost. The guarantee bounds our strategy's maximal information leakage from the user to the utility provider under general stochastic models of user energy consumption. The policy construction adapts coding strategies for non-probabilistic permuting channels to this privacy problem.

This work studies point-to-point, multiple access, and random access lossless source coding in the finite-blocklength regime. In each scenario, a random coding technique is developed and used to analyze third-order coding performance. Asymptotic results include a third-order characterization of the Slepian-Wolf rate region with an improved converse that relies on a connection to composite hypothesis testing. For dependent sources, the result implies that the independent encoders used by Slepian-Wolf codes can achieve the same third-order-optimal performance as a single joint encoder. The concept of random access source coding is introduced to generalize multiple access (Slepian-Wolf) source coding to the case where encoders decide independently whether or not to participate and the set of participating encoders is unknown {\em a priori} to both the encoders and the decoder. The proposed random access source coding strategy employs rateless coding with scheduled feedback. A random coding argument proves the existence of a single deterministic code of this structure that simultaneously achieves the third-order-optimal Slepian-Wolf performance for each possible active encoder set., Comment: 35 pages, 10 figures

As an attempt to bridge the gap between the probabilistic world of classical information theory and the combinatorial world of zero-error information theory, this paper studies the performance of randomly generated codebooks over discrete memoryless channels under a zero-error list-decoding constraint. This study allows the application of tools from one area to the other. Furthermore, it leads to an information-theoretic formulation of the birthday problem, which is concerned with the probability that in a given population, a fixed number of people have the same birthday. Due to the lack of a closed-form expression for this probability when the distribution of birthdays is not uniform, the resulting expression is not simple to analyze; in the information-theoretic formulation, however, the asymptotic behavior of this probability can be characterized exactly for all distributions., Comment: Extended version of paper presented at ISIT 2017 in Aachen

Consider a random access communication scenario over a channel whose operation is defined for any number of possible transmitters. As in the model recently introduced by Polyanskiy for the Multiple Access Channel (MAC) with a fixed, known number of transmitters, the channel is assumed to be invariant to permutations on its inputs, and all active transmitters employ identical encoders. Unlike the Polyanskiy model, in the proposed scenario, neither the transmitters nor the receiver knows which transmitters are active. We refer to this agnostic communication setup as the Random Access Channel (RAC). Scheduled feedback of a finite number of bits is used to synchronize the transmitters. The decoder is tasked with determining from the channel output the number of active transmitters, $k$, and their messages but not which transmitter sent which message. The decoding procedure occurs at a time $n_t$ depending on the decoder's estimate, $t$, of the number of active transmitters, $k$, thereby achieving a rate that varies with the number of active transmitters. Single-bit feedback at each time $n_i, i \leq t$, enables all transmitters to determine the end of one coding epoch and the start of the next. The central result of this work demonstrates the achievability on a RAC of performance that is first-order optimal for the MAC in operation during each coding epoch. While prior multiple access schemes for a fixed number of transmitters require $2^k - 1$ simultaneous threshold rules, the proposed scheme uses a single threshold rule and achieves the same dispersion., Comment: Presented at ISIT 2018. IEEE Transactions on Information Theory

In communication networks, cooperative strategies are coding schemes where network nodes work together to improve network performance metrics such as sum-rate. This work studies encoder cooperation in the setting of a discrete multiple access channel with two encoders and a single decoder. A node in the network that is connected to both encoders via rate-limited links, referred to as the cooperation facilitator (CF), enables the cooperation strategy. Previously, the authors presented a class of multiple access channels where the average-error sum-capacity has an infinite derivative in the limit where CF output link capacities approach zero. The authors also demonstrated that for some channels, the maximal-error sum-capacity is not continuous at the point where the output link capacities of the CF equal zero. This work shows that the the average-error sum-capacity is continuous when CF output link capacities converge to zero; that is, the infinite derivative of the average-error sum-capacity is not a result of its discontinuity as in the maximal-error case., Comment: 20 pages, 1 figure. To be submitted to ISIT '18

In many communication networks, the availability of channel state information at various nodes provides an opportunity for network nodes to work together, or "cooperate." This work studies the benefit of cooperation in the multiple access channel with a cooperation facilitator, distributed state information at the encoders, and full state information available at the decoder. Under various causality constraints, sufficient conditions are obtained such that encoder cooperation through the facilitator results in a gain in sum-capacity that has infinite slope in the information rate shared with the encoders. This result extends the prior work of the authors on cooperation in networks where none of the nodes have access to state information., Comment: Extended version of paper presented at ISIT 2017 in Aachen. 20 pages, 1 figure

In this work we study the capacity advantage achieved by adding a single bit of communication -- not a link of capacity 1 but a single bit over all time -- to a memoryless network. Specifically, we present a memoryless network in which adding a single bit of communication strictly increases the capacity region., Comment: 6 pages

In this paper, we study the effect of a single link on the capacity of a network of error-free bit pipes. More precisely, we study the change in network capacity that results when we remove a single link of capacity $\delta$. In a recent result, we proved that if all the sources are directly available to a single super-source node, then removing a link of capacity $\delta$ cannot change the capacity region of the network by more than $\delta$ in each dimension. In this paper, we extend this result to the case of multi-source, multi-sink networks for some special network topologies., Comment: Originally presented at ITA 2011 in San Diego, CA. The arXiv version contains an updated proof of Theorem 1

Cooperation strategies allow communication devices to work together to improve network capacity. Consider a network consisting of a $k$-user multiple access channel (MAC) and a node that is connected to all $k$ encoders via rate-limited bidirectional links, referred to as the "cooperation facilitator" (CF). Define the cooperation benefit as the sum-capacity gain resulting from the communication between the encoders and the CF and the cooperation rate as the total rate the CF shares with the encoders. This work demonstrates the existence of a class of $k$-user MACs where the ratio of the cooperation benefit to cooperation rate tends to infinity as the cooperation rate tends to zero. Examples of channels in this class include the binary erasure MAC for $k=2$ and the $k$-user Gaussian MAC for any $k\geq 2$., Comment: 46 pages, 4 figures. Submitted to the IEEE Transactions on Information Theory. This article supersedes arXiv:1504.04432 and arXiv:1508.03349

In network cooperation strategies, nodes work together with the aim of increasing transmission rates or reliability. This paper demonstrates that enabling cooperation between the transmitters of a two-user multiple access channel, via a cooperation facilitator that has access to both messages, always results in a network whose maximal- and average-error sum-capacities are the same---even when those capacities differ in the absence of cooperation and the information shared with the encoders is negligible. From this result, it follows that if a multiple access channel with no transmitter cooperation has different maximal- and average-error sum-capacities, then the maximal-error sum-capacity of the network consisting of this channel and a cooperation facilitator is not continuous with respect to the output edge capacities of the facilitator. This shows that there exist networks where sharing even a negligible number of bits per channel use with the encoders yields a non-negligible benefit., Comment: 27 pages, 3 figures. Submitted to the IEEE Transactions on Information Theory

The multivariate covering lemma states that given a collection of $k$ codebooks, each of sufficiently large cardinality and independently generated according to one of the marginals of a joint distribution, one can always choose one codeword from each codebook such that the resulting $k$-tuple of codewords is jointly typical with respect to the joint distribution. We give a proof of this lemma for weakly typical sets. This allows achievability proofs that rely on the covering lemma to go through for continuous channels (e.g., Gaussian) without the need for quantization. The covering lemma and its converse are widely used in information theory, including in rate-distortion theory and in achievability results for multi-user channels., Comment: 10 pages

Information theory is rapidly approaching its 70th birthday. What are promising future directions for research in information theory? Where will information theory be having the most impact in 10-20 years? What new and emerging areas are ripe for the most impact, of the sort that information theory has had on the telecommunications industry over the last 60 years? How should the IEEE Information Theory Society promote high-risk new research directions and broaden the reach of information theory, while continuing to be true to its ideals and insisting on the intellectual rigor that makes its breakthroughs so powerful? These are some of the questions that an ad hoc committee (composed of the present authors) explored over the past two years. We have discussed and debated these questions, and solicited detailed inputs from experts in fields including genomics, biology, economics, and neuroscience. This report is the result of these discussions.

In a cooperative coding scheme, network nodes work together to achieve higher transmission rates. To obtain a better understanding of cooperation, we consider a model in which two transmitters send rate-limited descriptions of their messages to a "cooperation facilitator", a node that sends back rate-limited descriptions of the pair to each transmitter. This model includes the conferencing encoders model and a prior model from the current authors as special cases. We show that except for a special class of multiple access channels, the gain in sum-capacity resulting from cooperation under this model is quite large. Adding a cooperation facilitator to any such channel results in a network that does not satisfy the edge removal property. An important special case is the Gaussian multiple access channel, for which we explicitly characterize the sum-rate cooperation gain., Comment: 15 pages, 3 figures. To be presented at ISIT 2015

In this paper, we propose a new cooperation model for discrete memoryless multiple access channels. Unlike in prior cooperation models (e.g., conferencing encoders), where the transmitters cooperate directly, in this model the transmitters cooperate through a larger network. We show that under this indirect cooperation model, there exist channels for which the increase in sum-capacity resulting from cooperation is significantly larger than the rate shared by the transmitters to establish the cooperation. This result contrasts both with results on the benefit of cooperation under prior models and results in the network coding literature, where attempts to find examples in which similar small network modifications yield large capacity benefits have to date been unsuccessful., Comment: Extended version of a paper submitted to ISIT 2014 (10 pages, 3 figures)

Traditional studies of multi-source, multi-terminal interference channels typically allow a vanishing probability of error in communication. Motivated by the study of network coding, this work addresses the task of quantifying the loss in rate when insisting on zero error communication in the context of interference channels.

We consider multi-resolution streaming in fully-connected peer-to-peer networks, where transmission rates are constrained by arbitrarily specified upload capacities of the source and peers. We fully characterize the capacity region of rate vectors achievable with arbitrary coding, where an achievable rate vector describes a vector of throughputs of the different resolutions that can be supported by the network. We then prove that all rate vectors in the capacity region can be achieved using pure routing strategies. This shows that coding has no capacity advantage over routing in this scenario.

We show that the network coding and index coding problems are equivalent. This equivalence holds in the general setting which includes linear and non-linear codes. Specifically, we present an efficient reduction that maps a network coding instance to an index coding one while preserving feasibility. Previous connections were restricted to the linear case.

We consider networks of noisy degraded wiretap channels in the presence of an eavesdropper. For the case where the eavesdropper can wiretap at most one channel at a time, we show that the secrecy capacity region, for a broad class of channels and any given network topology and communication demands, is equivalent to that of a corresponding network where each noisy wiretap channel is replaced by a noiseless wiretap channel. Thus in this case there is a separation between wiretap channel coding on each channel and secure network coding on the resulting noiseless network. We show with an example that such separation does not hold when the eavesdropper can access multiple channels at the same time, for which case we provide upper and lower bounding noiseless networks., Comment: 8 pages, presented at the 50th Annual Allerton Conference on Communication, Control and Computing 2012

We consider the delay of network coding compared to routing with retransmissions in packet erasure networks with probabilistic erasures. We investigate the sub-linear term in the block delay required for unicasting $n$ packets and show that there is an unbounded gap between network coding and routing. In particular, we show that delay benefit of network coding scales at least as $\sqrt{n}$. Our analysis of the delay function for the routing strategy involves a major technical challenge of computing the expectation of the maximum of two negative binomial random variables. This problem has been studied previously and we derive the first exact characterization which may be of independent interest. We also use a martingale bounded differences argument to show that the actual coding delay is tightly concentrated around its expectation., Comment: Submitted to IEEE Transactions on Information Theory. arXiv admin note: substantial text overlap with arXiv:0910.3975

In this paper we prove the separation of source-network coding and channel coding in wireline networks. For the purposes of this work, a wireline network is any network of independent, memoryless, point-to-point, finite-alphabet channels used to transmit dependent sources either losslessly or subject to a distortion constraint. In deriving this result, we also prove that in a general memoryless network with dependent sources, lossless and zero-distortion reconstruction are equivalent provided that the conditional entropy of each source given the other sources is non-zero. Furthermore, we extend the separation result to the case of continuous-alphabet, point-to-point channels such as additive white Gaussian noise (AWGN) channels., Comment: Submitted to the IEEE Transactions on Information Theory

In this work we study zero vs. epsilon-error capacity in network coding instances. For multicast network coding it is well known that all rates that can be delivered with arbitrarily small error probability can also be delivered with zero error probability; that is, the epsilon-error multicast capacity region and zero-error multicast capacity region are identical. For general network coding instances in which all sources originate at the same source node, Chan and Grant recently showed [ISIT 2010] that, again, epsilon-error communication has no rate advantage over zero-error communication. We start by revisiting the setting of co-located sources, where we present an alternative proof to that given by Chan and Grant. While the new proof is based on similar core ideas, our constructive strategy complements the previous argument.We then extend our results to the setting of index coding, which is a special and representative form of network coding that encapsulates the "source coding with side information" problem. Finally, we consider the "edge removal" problem (recently studied by Jalali, Effros, and Ho in [Allerton 2010] and [ITA 2011]) that aims to quantify the loss in capacity associated with removing a single edge from a given network. Using our proof for co-located sources, we tie the "zero vs. epsilon-error" problem in general network coding instances with the "edge removal" problem. Loosely speaking, we show that the two problem are equivalent.

In this paper, we demonstrate some applications of compressive sensing over networks. We make a connection between compressive sensing and traditional information theoretic techniques in source coding and channel coding. Our results provide an explicit trade-off between the rate and the decoding complexity. The key difference of compressive sensing and traditional information theoretic approaches is at their decoding side. Although optimal decoders to recover the original signal, compressed by source coding have high complexity, the compressive sensing decoder is a linear or convex optimization. First, we investigate applications of compressive sensing on distributed compression of correlated sources. Here, by using compressive sensing, we propose a compression scheme for a family of correlated sources with a modularized decoder, providing a trade-off between the compression rate and the decoding complexity. We call this scheme Sparse Distributed Compression. We use this compression scheme for a general multicast network with correlated sources. Here, we first decode some of the sources by a network decoding technique and then, we use a compressive sensing decoder to obtain the whole sources. Then, we investigate applications of compressive sensing on channel coding. We propose a coding scheme that combines compressive sensing and random channel coding for a high-SNR point-to-point Gaussian channel. We call this scheme Sparse Channel Coding. We propose a modularized decoder providing a trade-off between the capacity loss and the decoding complexity. At the receiver side, first, we use a compressive sensing decoder on a noisy signal to obtain a noisy estimate of the original signal and then, we apply a traditional channel coding decoder to find the original signal.

This work considers the multiple-access multicast error-correction scenario over a packetized network with $z$ malicious edge adversaries. The network has min-cut $m$ and packets of length $\ell$, and each sink demands all information from the set of sources $\sources$. The capacity region is characterized for both a "side-channel" model (where sources and sinks share some random bits that are secret from the adversary) and an "omniscient" adversarial model (where no limitations on the adversary's knowledge are assumed). In the "side-channel" adversarial model, the use of a secret channel allows higher rates to be achieved compared to the "omniscient" adversarial model, and a polynomial-complexity capacity-achieving code is provided. For the "omniscient" adversarial model, two capacity-achieving constructions are given: the first is based on random subspace code design and has complexity exponential in $\ell m$, while the second uses a novel multiple-field-extension technique and has $O(\ell m^{|\sources|})$ complexity, which is polynomial in the network size. Our code constructions are "end-to-end" in that all nodes except the sources and sinks are oblivious to the adversaries and may simply implement predesigned linear network codes (random or otherwise). Also, the sources act independently without knowledge of the data from other sources., Comment: This journal paper is a continuation of the conference paper written by S. Vyetrenko, T. Ho, M. Effros, J. Kliewer, E Erez, "Rate regions for coherent and noncoherent multisource network error correction," IEEE ISIT, Jun. 2009 and the conference paper written by H. Yao, T. K Dikaliotis, S. Jaggi, and T. Ho, "Multi-source operator channels: Efficient capacity-achieving codes," IEEE ITW, Dublin 2010

This paper studies the capacity of single-source single-sink noiseless networks under adversarial or arbitrary errors on no more than z edges. Unlike prior papers, which assume equal capacities on all links, arbitrary link capacities are considered. Results include new upper bounds, network error correction coding strategies, and examples of network families where our bounds are tight. An example is provided of a network where the capacity is 50% greater than the best rate that can be achieved with linear coding. While coding at the source and sink suffices in networks with equal link capacities, in networks with unequal link capacities, it is shown that intermediate nodes may have to do coding, nonlinear error detection, or error correction in order to achieve the network error correction capacity.

One major open problem in network coding is to characterize the capacity region of a general multi-source multi-demand network. There are some existing computational tools for bounding the capacity of general networks, but their computational complexity grows very quickly with the size of the network. This motivates us to propose a new hierarchical approach which finds upper and lower bounding networks of smaller size for a given network. This approach sequentially replaces components of the network with simpler structures, i.e., with fewer links or nodes, so that the resulting network is more amenable to computational analysis and its capacity provides an upper or lower bound on the capacity of the original network. The accuracy of the resulting bounds can be bounded as a function of the link capacities. Surprisingly, we are able to simplify some families of network structures without any loss in accuracy., Comment: 8 pages, 12 figures, 48th Annual Allerton Conference on Communication, Control, and Computing, 2010

A family of equivalence tools for bounding network capacities is introduced. Part I treats networks of point-to-point channels. The main result is roughly as follows. Given a network of noisy, independent, memoryless point-to-point channels, a collection of communication demands can be met on the given network if and only if it can be met on another network where each noisy channel is replaced by a noiseless bit pipe with throughput equal to the noisy channel capacity. This result was known previously for the case of a single-source multicast demand. The result given here treats general demands -- including, for example, multiple unicast demands -- and applies even when the achievable rate region for the corresponding demands is unknown in the noiseless network. In part II, definitions of upper and lower bounding channel models for general channels are introduced. By these definitions, a collection of communication demands can be met on a network of independent channels if it can be met on a network where each channel is replaced by its lower bounding model andonly if it can be met on a network where each channel is replaced by its upper bounding model. This work derives general conditions under which a network of noiseless bit pipes is an upper or lower bounding model for a multiterminal channel. Example upper and lower bounding models for broadcast, multiple access, and interference channels are given. It is then shown that bounding the difference between the upper and lower bounding models for a given channel yields bounds on the accuracy of network capacity bounds derived using those models. By bounding the capacity of a network of independent noisy channels by the network coding capacity of a network of noiseless bit pipes, this approach represents one step towards the goal of building computational tools for bounding network capacities., Comment: 91 pages, 18 figures. Submitted to the IEEE Transactions on Information Theory on April 14, 2010. Draft 2

This paper proves the separation between source-network coding and channel coding in networks of noisy, discrete, memoryless channels. We show that the set of achievable distortion matrices in delivering a family of dependent sources across such a network equals the set of achievable distortion matrices for delivering the same sources across a distinct network which is built by replacing each channel by a noiseless, point-to-point bit-pipe of the corresponding capacity. Thus a code that applies source-network coding across links that are made almost lossless through the application of independent channel coding across each link asymptotically achieves the optimal performance across the network as a whole., Comment: 5 pages, to appear in the proceedings of 2010 IEEE International Symposium on Information Theory (ISIT)

We focus on a particular form of network coding, reverse carpooling, in a wireless network where the potentially coded transmitted messages are to be decoded immediately upon reception. The network is fixed and known, and the system performance is measured in terms of the number of wireless broadcasts required to meet multiple unicast demands. Motivated by the structure of the coding scheme, we formulate the problem as a linear program by introducing a flow variable for each triple of connected nodes. This allows us to have a formulation polynomial in the number of nodes. Using dual decomposition and projected subgradient method, we present a decentralized algorithm to obtain optimal routing schemes in presence of coding opportunities. We show that the primal sub-problem can be expressed as a shortest path problem on an \emph{edge-graph}, and the proposed algorithm requires each node to exchange information only with its neighbors., Comment: submitted to CISS 2010

Polar codes have attracted much recent attention as the first codes with low computational complexity that provably achieve optimal rate-regions for a large class of information-theoretic problems. One significant drawback, however, is that for current constructions the probability of error decays sub-exponentially in the block-length (more detailed designs improve the probability of error at the cost of significantly increased computational complexity \cite{KorUS09}). In this work we show how the the classical idea of code concatenation -- using "short" polar codes as inner codes and a "high-rate" Reed-Solomon code as the outer code -- results in substantially improved performance. In particular, code concatenation with a careful choice of parameters boosts the rate of decay of the probability of error to almost exponential in the block-length with essentially no loss in computational complexity. We demonstrate such performance improvements for three sets of information-theoretic problems -- a classical point-to-point channel coding problem, a class of multiple-input multiple output channel coding problems, and some network source coding problems.

We consider the problem of zero error source coding with limited feedback when side information is present at the receiver. First, we derive an achievable rate region for arbitrary joint distributions on the source and the side information. When all source pairs of source and side information symbols are observable with non-zero probability, we show that this characterization gives the entire rate region. Next, we demonstrate a class of sources for which asymptotically zero feedback suffices to achieve zero-error coding at the rate promised by the Slepian-Wolf bound for asymptotically lossless coding. Finally, we illustrate these results with the aid of three simple examples.