Search

Posterior matching uses variable-length encoding of the message controlled by noiseless feedback of the received symbols to achieve high rates for short average blocklengths. Traditionally, the feedback of a received symbol occurs before the next symbol is transmitted. The transmitter optimizes the next symbol transmission with full knowledge of every past received symbol. To move posterior matching closer to practical communication, this paper seeks to constrain how often feedback can be sent back to the transmitter. We focus on reducing the frequency of the feedback while still maintaining the high rates that posterior matching achieves with feedback after every symbol. As it turns out, the frequency of the feedback can be reduced significantly with no noticeable reduction in rate.

Motivated by the need to communicate short control messages in 5G and beyond, this paper carefully designs codes for cyclic redundancy check (CRC)-aided list decoding of tail-biting convolutional codes (TBCCs) and polar codes. Both codes send a 32-bit message using an 11-bit CRC and 512 transmitted bits. We aim to provide a careful, fair comparison of the error performance and decoding complexity of polar and TBCC techniques for a specific case. Specifically, a TBCC is designed to match the rate of a (512, 43) polar code, and optimal 11-bit CRCs for both codes are designed. The paper examines the distance spectra of the polar and TBCC codes, illuminating the different distance structures for the two code types. We consider both adaptive and non-adaptive CRC-aided list decoding schemes. For polar codes, an adaptive decoder must start with a larger list size to avoid an error floor. For rate-32/512 codes with an 11-bit CRC, the optimized CRC-TBCC design achieves a lower total failure rate than the optimized CRC-polar design. Simulations showed that the optimized CRC-TBCC design achieved significantly higher throughput than the optimized CRC-polar design, so that the TBCC solution achieved a lower total failure rate while requiring less computational complexity., Comment: First revision submitted to IEEE Transactions on Communications

An algorithm is proposed to encode low-density parity-check (LDPC) codes into codewords with a non-uniform distribution. This enables power-efficient signalling for asymmetric channels. We show gains of 0.9 dB for additive white Gaussian noise (AWGN) channels with on-off keying modulation using 5G LDPC codes., Comment: submitted to ISTC 2023

In this paper, we present new achievability bounds on the maximal achievable rate of variable-length stop-feedback (VLSF) codes operating over a binary erasure channel (BEC) at a fixed message size $M = 2^k$. We provide new bounds for VLSF codes with zero error, infinite decoding times and with nonzero error, finite decoding times. Both new achievability bounds are proved by constructing a new VLSF code that employs systematic transmission of the first $k$ bits followed by random linear fountain parity bits decoded with a rank decoder. For VLSF codes with infinite decoding times, our new bound outperforms the state-of-the-art result for BEC by Devassy \emph{et al.} in 2016. We also give a negative answer to the open question Devassy \emph{et al.} put forward on whether the $23.4\%$ backoff to capacity at $k = 3$ is fundamental. For VLSF codes with finite decoding times, numerical evaluations show that the achievable rate for VLSF codes with a moderate number of decoding times closely approaches that for VLSF codes with infinite decoding times., Comment: 7 pages, double column, 4 figures; comments are welcome! changes in v2: corrected 2 typos in v1. arXiv admin note: substantial text overlap with arXiv:2205.15399

In this paper, we are interested in the performance of a variable-length stop-feedback (VLSF) code with m optimal decoding times for the binary-input additive white Gaussian noise channel. We first develop tight approximations to the tail probability of length-n cumulative information density. Building on the work of Yavas et al., for a given information density threshold, we formulate the integer program of minimizing the upper bound on average blocklength over all decoding times subject to the average error probability, minimum gap and integer constraints. Eventually, minimization of locally optimal upper bounds over all thresholds yields the globally minimum upper bound and the above method is called the two-step minimization. Relaxing to allow positive real-valued decoding times activates the gap constraint. We develop gap-constrained sequential differential optimization (SDO) procedure to find the optimal, gap-constrained, real-valued decoding times. In the error regime of practical interest, Polyanskiy's scheme of stopping at zero does not help. In this region, the achievability bounds estimated by the two-step minimization and gap-constrained SDO show that Polyanskiy’s achievability bound for VLSF codes can be approached with a small number of decoding times.

This paper focuses on the numerical evaluation of the maximal achievable rate of variable-length stop-feedback (VLSF) codes with m decoding times at a given message size and error probability for binary-input additive white Gaussian noise channel, binary symmetric channel, and binary erasure channel (BEC). Leveraging the Edgeworth and Petrov expansions, we develop tight approximations to the tail probability of length-n cumulative information density that are accurate for any blocklength n. We reduce Yavas et al.'s non-asymptotic achievability bound on VLSF codes with m decoding times to an integer program of minimizing the upper bound on the average blocklength subject to the average error probability, minimum gap, and integer constraints. We develop two distinct methods to solve this program. Numerical evaluations show that Polyanskiy's achievability bound for VLSF codes, which assumes m = ∞, can be approached with a relatively small m in all of the three channels. For BEC, we consider systematic transmission followed by random linear fountain coding. This allows us to obtain a new achievability bound stronger than a previously known bound and new VLSF codes whose rate further outperforms Polyanskiy's bound.

Recently, rate-1/n zero-terminated (ZT) and tail-biting (TB) convolutional codes (CCs) with cyclic redundancy check (CRC)-aided list decoding have been shown to closely approach the random-coding union (RCU) bound for short blocklengths. This paper designs CRC polynomials for rate- (n-1)/n ZT and TB CCs with short blocklengths. This paper considers both standard rate-(n-1)/n CC polynomials and rate- (n-1)/n designs resulting from puncturing a rate-1/2 code. The CRC polynomials are chosen to maximize the minimum distance d_min and minimize the number of nearest neighbors A_(d_min) . For the standard rate-(n-1)/n codes, utilization of the dual trellis proposed by Yamada et al. lowers the complexity of CRC-aided serial list Viterbi decoding (SLVD). CRC-aided SLVD of the TBCCs closely approaches the RCU bound at a blocklength of 128. This paper compares the FER performance (gap to the RCU bound) and complexity of the CRC-aided standard and punctured ZTCCs and TBCCs. This paper also explores the complexity-performance trade-off for three TBCC decoders: a single-trellis approach, a multi-trellis approach, and a modified single-trellis approach with pre-processing using the wrap around Viterbi algorithm., Comment: arXiv admin note: substantial text overlap with arXiv:2111.07929

We extend earlier work on the design of convolutional code-specific CRC codes to $Q$-ary alphabets, with an eye toward $Q$-ary orthogonal signaling. Starting with distance-spectrum optimal, zero-terminated, $Q$-ary convolutional codes, we design $Q$-ary CRC codes so that the CRC/convolutional concatenation is distance-spectrum optimal. The $Q$-ary code symbols are mapped to a $Q$-ary orthogonal signal set and sent over an AWGN channel with noncoherent reception. We focus on $Q = 4$, rate-1/2 convolutional codes in our designs. The random coding union bound and normal approximation are used in earlier works as benchmarks for performance for distance-spectrum optimal convolutional codes. We derive a saddlepoint approximation of the random coding union bound for the coded noncoherent signaling channel, as well as a normal approximation for this channel, and compare the performance of our codes to these limits. Our best design is within $0.6$ dB of the RCU bound at a frame error rate of $10^{-4}$., Comment: GLOBECOM 2022 camera-ready version

In this paper, we are interested in the performance of a variable-length stop-feedback (VLSF) code with $m$ optimal decoding times for the binary-input additive white Gaussian noise channel. We first develop tight approximations on the tail probability of length-$n$ cumulative information density. Building on the work of Yavas \emph{et al.}, for a given information density threshold, we formulate the integer program of minimizing the upper bound on average blocklength over all decoding times subject to the average error probability, minimum gap and integer constraints. Eventually, minimization of locally minimum upper bounds over all thresholds will yield the globally minimum upper bound and this is called the two-step minimization. For the integer program, we present a greedy algorithm that yields possibly suboptimal integer decoding times. By allowing a positive real-valued decoding time, we develop the gap-constrained sequential differential optimization (SDO) procedure that sequentially produces the optimal, real-valued decoding times. We identify the error regime in which Polyanskiy's scheme of stopping at zero does not improve the achievability bound. In this error regime, the two-step minimization with the gap-constrained SDO shows that a finite $m$ suffices to attain Polyanskiy's bound for VLSF codes with $m = \infty$., Comment: 8 pages, 4 figures; fixed some errors in v1 and added some new results; a short version of this preprint was submitted to ISIT 2022

This paper explores list decoding of convolutional and polar codes for short messages such as those found in the 5G physical broadcast channel. A cyclic redundancy check (CRC) is used to select a codeword from a list of likely codewords. One example in the 5G standard encodes a 32-bit message with a 24-bit CRC and a 512-bit polar code with additional bits added by repetition to achieve a very low rate of 32/864. This paper shows that optimizing the CRC length improves the $E_b/N_0$ performance of this polar code, where $E_b/N_0$ is the ratio of the energy per data bit to the noise power spectral density. Furthermore, even better $E_b/N_0$ performance is achieved by replacing the polar code with a tail-biting convolutional code (TBCC) with a distance-spectrum-optimal (DSO) CRC. This paper identifies the optimal CRC length to minimize the frame error rate (FER) of a rate-1/5 TBCC at a specific value of $E_b/N_0$. We also show that this optimized TBCC/CRC can attain the same excellent $E_b/N_0$ performance with the very low rate of 32/864 of the 5G polar code, where the low rate is achieved through repetition. We show that the proposed TBCC/CRC concatenated code outperforms the PBCH polar code described in the 5G standard both in terms of FER and decoding run time. We also explore the tradeoff between undetected error rate and erasure rate as the CRC size varies., Comment: 6 pages, 8 figures; this preprint is accepted for publication at the 2022 IEEE International Conference on Communications (ICC); camera-ready version to be updated

© 2018 IEEE. This paper develops an efficient method to compute the binomial channel capacity and applies it to the molecular channel. The binomial channel (with parameter n) takes the success probability for a Bernoulli trial as input and produces the number of successes in trials as output. The input alphabet is the unit interval and the output alphabet is the set of integers from zero to n. Despite the fact that the input alphabet is uncountably infinite the capacity-achieving input distributions turn out to have a small finite support that evolves gracefully as n increases. The ellipsoid algorithm was previously used to compute the binomial channel capacity, but convergence is rather slow even with a well-chosen initial condition. The Dynamic Assignment Blahut-Arimoto (DAB) algorithm starts with the capacity-achieving mass point locations for the n-1 case and exploits Csiszàr's Min-Max Capacity Theorem to check convergence and adjust mass point locations to achieve a much faster convergence rate, unlocking the potential for the capacity and corresponding input distribution to be computed for larger values of n.

Incremental redundancy with ACK/NACK feedback produces a variable-length stop-feedback (VLSF) code constrained to have $m$ decoding times, with an ACK/NACK feedback to the transmitter at each decoding time. This paper focuses on the numerical evaluation of the maximal achievable rate of random VLSF codes as a function of $m$ for the binary-input additive white Gaussian noise channel, binary symmetric channel, and binary erasure channel (BEC). Leveraging Edgeworth and Petrov expansions, we develop tight approximations to the tail probability of length-$n$ cumulative information density that are accurate for any blocklength $n$. We reduce Yavas et al.'s non-asymptotic achievability bound on VLSF codes with $m$ decoding times to an integer program of minimizing the upper bound on the average blocklength subject to the average error probability, minimum gap, and integer constraints. We develop two distinct methods to solve this program. Numerical evaluations show that Polyanskiy's achievability bound for VLSF codes, which assumes $m = \infty$, can be approached with a small $m$ for all three channels. For BEC, we consider systematic transmission followed by random linear fountain coding. This allows us to obtain a new achievability bound stronger than a previous bound and new VLSF codes whose rate further outperforms Polyanskiy's bound., Comment: 19 pages, 12 figures; submitted to IEEE Transactions on Communications. An earlier version of this work was published at ISIT 2022. Comments are welcome!

In this paper, we consider the problem of variable-length coding over the class of memoryless binary asymmetric channels (BACs) with noiseless feedback, including the binary symmetric channel (BSC) as a special case. In 2012, Naghshvar et al. introduced an encoding scheme, which we refer to as the small-enough-difference (SED) encoder, which asymptotically achieves both capacity and Burnashev's optimal error exponent for symmetric binary-input channels. Building on the work of Naghshvar et al., this paper extends the SED encoding scheme to the class of BACs and develops a non-asymptotic upper bound on the average blocklength that is shown to achieve both capacity and the optimal error exponent. For the specific case of the BSC, we develop an additional non-asymptotic bound using a two-phase analysis that leverages both a submartingale synthesis and a Markov chain time of first passage analysis. For the BSC with capacity $1/2$, both new achievability bounds exceed the achievability bound of Polyanskiy et al. for a system limited to stop-feedback codes., Comment: 18 pages; submitted to IEEE Transactions on Information Theory

This paper applies probabilistic amplitude shaping (PAS) to a cyclic redundancy check (CRC) aided trellis coded modulation (TCM) to achieve the short-blocklength random coding union (RCU) bound. In the transmitter, the equally likely message bits are first encoded by distribution matcher to generate amplitude symbols with the desired distribution. The binary representations of the distribution matcher outputs are then encoded by a CRC. Finally, the CRC-encoded bits are encoded and modulated by Ungerboeck's TCM scheme, which consists of a $\frac{k_0}{k_0+1}$ systematic tail-biting convolutional code and a mapping function that maps coded bits to channel signals with capacity-achieving distribution. This paper proves that, for the proposed transmitter, the CRC bits have uniform distribution and that the channel signals have symmetric distribution. In the receiver, the serial list Viterbi decoding (S-LVD) is used to estimate the information bits. Simulation results show that, for the proposed CRC-TCM-PAS system with 87 input bits and 65-67 8-AM coded output symbols, the decoding performance under additive white Gaussian noise channel achieves the RCU bound with properly designed CRC and convolutional codes.

Recently, rate-$1/\omega$ zero-terminated and tail-biting convolutional codes (ZTCCs and TBCCs) with cyclic-redundancy-check (CRC)-aided list decoding have been shown to closely approach the random-coding union (RCU) bound for short blocklengths. This paper designs CRCs for rate-$(\omega-1)/\omega$ CCs with short blocklengths, considering both the ZT and TB cases. The CRC design seeks to optimize the frame error rate (FER) performance of the code resulting from the concatenation of the CRC and the CC. Utilization of the dual trellis proposed by Yamada \emph{et al.} lowers the complexity of CRC-aided serial list Viterbi decoding (SLVD) of ZTCCs and TBCCs. CRC-aided SLVD of the TBCCs closely approaches the RCU bound at a blocklength of $128$., Comment: 6 pages; submitted to 2022 IEEE International Conference on Communications (ICC 2022)

© 2018 IEEE. This paper develops an efficient method to compute the binomial channel capacity and applies it to the molecular channel. The binomial channel (with parameter n) takes the success probability for a Bernoulli trial as input and produces the number of successes in trials as output. The input alphabet is the unit interval and the output alphabet is the set of integers from zero to n. Despite the fact that the input alphabet is uncountably infinite the capacity-achieving input distributions turn out to have a small finite support that evolves gracefully as n increases. The ellipsoid algorithm was previously used to compute the binomial channel capacity, but convergence is rather slow even with a well-chosen initial condition. The Dynamic Assignment Blahut-Arimoto (DAB) algorithm starts with the capacity-achieving mass point locations for the n-1 case and exploits Csiszàr's Min-Max Capacity Theorem to check convergence and adjust mass point locations to achieve a much faster convergence rate, unlocking the potential for the capacity and corresponding input distribution to be computed for larger values of n.

© 2015 IEEE. This work introduces the particle-intensity channel (PIC) as a new model for molecular communication systems that includes imperfections at both transmitter and receiver and provides a new characterization of the capacity limits as well as properties of the optimal (capacity-achieving) input distributions for such channels. In the PIC, the transmitter encodes information, in symbols of a given duration, based on the probability of particle release, and the receiver detects and decodes the message based on the number of particles detected during the symbol interval. In this channel, the transmitter may be unable to control precisely the probability of particle release, and the receiver may not detect all the particles that arrive. We model this channel using a generalization of the binomial channel and show that the capacity-achieving input distribution for this channel always has mass points at probabilities of particle release of zero and one. To find the capacity-achieving input distributions, we develop a novel and efficient algorithm we call dynamic assignment Blahut-Arimoto (DAB). For diffusive particle transport, we also derive the conditions under which the input with two mass points is capacity-achieving.

Traditional channel coding with feedback constructs and transmits a codeword only after all message bits are available at the transmitter. This paper joins Guo & Kostina and Lalitha et. al. in developing approaches for causal (or progressive) encoding, where the transmitter may begin transmitting codeword symbols as soon as the first message bit arrives. Building on the work of Horstein, Shayevitz and Feder, and Naghshvar et. al., this paper extends our previous computationally efficient systematic algorithm for traditional posterior matching to produce a four-phase encoder that progressively encodes using only the message bits causally available. Systematic codes work well with posterior matching on a channel with feedback, and they provide an immediate benefit when causal encoding is employed instead of traditional encoding. Our algorithm captures additional gains in the interesting region where the transmission rate mu is higher than the rate lambda at which message bits become available. In this region, transmission of additional symbols beyond systematic bits, before a traditional encoder would have begun transmission, further improves performance

A status updating system is considered in which a variable length code is used to transmit messages to a receiver over a noisy channel. The goal is to optimize the codewords lengths such that successfully-decoded messages are timely. That is, such that the age-of-information (AoI) at the receiver is minimized. A hybrid ARQ (HARQ) scheme is employed, in which variable-length incremental redundancy (IR) bits are added to the originally-transmitted codeword until decoding is successful. With each decoding attempt, a non-zero processing delay is incurred. The optimal codewords lengths are analytically derived utilizing a sequential differential optimization (SDO) framework. The framework is general in that it only requires knowledge of an analytical expression of the positive feedback (ACK) probability as a function of the codeword length., Comment: To appear in the IEEE Conference on Computer Communication (INFOCOM) Age of Information Workshop 2021

Increased data rates and very low-latency requirements place strict constraints on thecomputational complexity of channel decoders in the new 5G communications standard. Practicallow-density parity-check (LDPC) decoder implementations use message-passing decoding with finiteprecision, which becomes coarse as complexity is more severely constrained. In turn, performance degradesas the precision becomes more coarse. Recently, the information bottleneck (IB) method was used to designmutual-information-maximizing mappings that replace conventional finite-precision node computations. Asa result, the exchanged messages in the IB approach can be represented with a very small number of bits.5G LDPC codes have the so-called protograph-based raptor-like (PBRL) structure which offers inherentrate-compatibility and excellent performance. This paper extends the IB principle to the flexible class ofPBRL LDPC codes as standardized in 5G. The extensions include IB decoder design for puncturing andrate-compatibility. In contrast to existing IB decoder design techniques, the proposed decoder can be usedfor a large range of code rates with a static set of optimized mappings. The proposed construction approachis evaluated for a typical range of code rates and bit resolutions ranging from 3 bit to 5 bit. Frame errorrate simulations show that the proposed scheme always outperforms min-sum decoding algorithms andoperates close to double-precision sum-product belief propagation decoding. Furthermore, alternatives tothe lookup table implementations of the mutual-information-maximizing mappings are investigated.

In this paper, the Dynamic-Assignment Blahut-Arimoto (DAB) algorithm identifies finite-support probability mass functions (PMFs) with small cardinality that achieve capacity for amplitude-constrained (AC) Additive White Gaussian Noise (AWGN) Channels, or approach capacity to within less than 1% for power-constrained (PC) AWGN Channels. While a continuous Gaussian PDF is well-known to be a theoretical capacity-achieving distribution for the PC-AWGN channel, DAB identifies PMFs with small-cardinality that are, for practical purposes, indistinguishable in performance. We extend the results of Ozarow and Wyner that require a constellation cardinality of $2^{C+1}$ to approach capacity C to within the shaping loss. PMF's found by DAB approach capacity with essentially no shaping loss with constellation cardinality of $2^{C+1.2}$. For AC-AWGN channels, DAB characterizes the evolution of minimum-cardinality finite-support capacity-achieving PMFs., Comment: 6 pages, five figures, submitted to Globecom 2020

This work introduces the particle-intensity channel (PIC) as a model for molecular communication systems and characterizes the capacity limits as well as properties of the optimal (capacity-achieving) input distributions for such channels. In the PIC, the transmitter encodes information, in symbols of a given duration, based on the probability of particle release, and the receiver detects and decodes the message based on the number of particles detected during the symbol interval. In this channel, the transmitter may be unable to control precisely the probability of particle release, and the receiver may not detect all the particles that arrive. We model this channel using a generalization of the binomial channel and show that the capacity-achieving input distribution for this channel always has mass points at probabilities of particle release of zero and one. To find the capacity-achieving input distributions, we develop an efficient algorithm we call dynamic assignment Blahut-Arimoto (DAB). For diffusive particle transport, we also derive the conditions under which the input with two mass points is capacity-achieving., Comment: arXiv admin note: text overlap with arXiv:1705.08040

In this paper, we propose a finite-precision decoding method that features the three steps of Reconstruction, Computation, and Quantization (RCQ). Unlike Mutual-Information-Maximization Quantized Belief Propagation (MIM-QBP), RCQ can approximate either belief propagation or Min-Sum decoding. One problem faced by MIM-QBP decoder is that it cannot work well when the fraction of degree-2 variable nodes is large. However, sometimes a large fraction of degree-2 variable nodes is necessary for a fast encoding structure, as seen in the IEEE 802.11 standard and the DVB-S2 standard. In contrast, the proposed RCQ decoder may be applied to any off-the-shelf LDPC code, including those with a large fraction of degree-2 variable nodes.Our simulations show that a 4-bit Min-Sum RCQ decoder delivers frame error rate (FER) performance around 0.1dB of full-precision belief propagation (BP) for the IEEE 802.11 standard LDPC code in the low SNR region.The RCQ decoder actually outperforms full-precision BP in the high SNR region because it overcomes elementary trapping sets that create an error floor under BP decoding. This paper also introduces Hierarchical Dynamic Quantization (HDQ) to design the non-uniform quantizers required by RCQ decoders. HDQ is a low-complexity design technique that is slightly sub-optimal. Simulation results comparing HDQ and an optimal quantizer on the symmetric binary-input memoryless additive white Gaussian noise channel show a loss in mutual information between these two quantizers of less than $10^{-6}$ bits, which is negligible for practical applications.

Cyclic redundancy check (CRC) codes combined with convolutional codes yield a powerful concatenated code that can be efficiently decoded using list decoding. To help design such systems, this paper presents an efficient algorithm for identifying the distance-spectrum-optimal (DSO) CRC polynomial for a given tail-biting convolutional code (TBCC) when the target undetected error rate (UER) is small. Lou et al. found that the DSO CRC design for a given zero-terminated convolutional code under low UER is equivalent to maximizing the undetected minimum distance (the minimum distance of the concatenated code). This paper applies the same principle to design the DSO CRC for a given TBCC under low target UER. Our algorithm is based on partitioning the tail-biting trellis into several disjoint sets of tail-biting paths that are closed under cyclic shifts. This paper shows that the tail-biting path in each set can be constructed by concatenating the irreducible error events (IEEs) and circularly shifting the resultant path. This motivates an efficient collection algorithm that aims at gathering IEEs, and a search algorithm that reconstructs the full list of error events with bounded distance of interest, which can be used to find the DSO CRC. Simulation results show that DSO CRCs can significantly outperform suboptimal CRCs in the low UER regime., Comment: ISIT 2020 camera-ready version

Building on the work of Horstein, Shayevitz and Feder, and Naghshvar \emph{et al.}, this paper presents algorithms for low-complexity sequential transmission of a $k$-bit message over the binary symmetric channel (BSC) with full, noiseless feedback. To lower complexity, this paper shows that the initial $k$ binary transmissions can be sent before any feedback is required and groups messages with equal posteriors to reduce the number of posterior updates from exponential in $k$ to linear in $k$. Simulation results demonstrate that achievable rates for this full, noiseless feedback system approach capacity rapidly as a function of average blocklength, faster than known finite-blocklength lower bounds on achievable rate with noiseless active feedback and significantly faster than finite-blocklength lower bounds for a stop feedback system., Comment: ISIT 2020 camera-ready version

The new 5G communications standard increases data rates and supports low-latency communication that places constraints on the computational complexity of channel decoders. 5G low-density parity-check (LDPC) codes have the so-called protograph-based raptor-like (PBRL) structure which offers inherent rate-compatibility and excellent performance. Practical LDPC decoder implementations use message-passing decoding with finite precision, which becomes coarse as complexity is more severely constrained. Performance degrades as the precision becomes more coarse. Recently, the information bottleneck (IB) method was used to design mutual-information-maximizing lookup tables that replace conventional finite-precision node computations. Additionally, the IB approach exchanges messages represented by integers with very small bit width. This paper extends the IB principle to the flexible class of PBRL LDPC codes as standardized in 5G. The extensions includes puncturing and rate-compatible IB decoder design. As an example of the new approach, a 4-bit information bottleneck decoder is evaluated for PBRL LDPC codes over a typical range of rates. Bit error rate simulations show that the proposed scheme outperforms offset min-sum decoding algorithms and operates within 0.2 dB of double-precision sum-product belief propagation decoding.

In this paper, we consider the problem of sequential transmission over the binary symmetric channel (BSC) with full, noiseless feedback. Naghshvar et al. proposed a one-phase encoding scheme, for which we refer to as the small-enough difference (SED) encoder, which can achieve capacity and Burnashev's optimal error exponent for symmetric binary-input channels. They also provided a non-asymptotic upper bound on the average blocklength, which implies an achievability bound on rates. However, their achievability bound is loose compared to the simulated performance of SED encoder, and even lies beneath Polyanskiy's achievability bound of a system limited to stop feedback. This paper significantly tightens the achievability bound by using a Markovian analysis that leverages both the submartingale and Markov properties of the transmitted message. Our new non-asymptotic lower bound on achievable rate lies above Polyanskiy's bound and is close to the actual performance of the SED encoder over the BSC., Comment: This work has been accepted for publication at ISIT 2020. This is the complete version of the camera-ready ISIT paper

This paper studies the joint design of optimal convolutional codes (CCs) and CRC codes when serial list Viterbi algorithm (S-LVA) is employed in order to achieve the target frame error rate (FER). We first analyze the S-LVA performance with respect to SNR and list size, repsectively, and prove the convergence of the expected number of decoding attempts when SNR goes to the extreme. We then propose the coded channel capacity as the criterion to jointly design optimal CC-CRC pair and optimal list size and show that the optimal list size of S-LVA is always the cardinality of all possible CCs. With the maximum list size, we choose the design metric of optimal CC-CRC pair as the SNR gap to random coding union (RCU) bound and the optimal CC-CRC pair is the one that achieves a target SNR gap with the least complexity. Finally, we show that a weaker CC with a strong optimal CRC code could be as powerful as a strong CC with no CRC code., Comment: Portions of this work will be presented at the 2018 IEEE Global Communications Conference, Abu Dhabi, UAE

Let $X^n$ be a uniformly distributed $n$-dimensional binary vector, and $Y^n$ be the result of passing $X^n$ through a binary symmetric channel (BSC) with crossover probability $\alpha$. A recent conjecture postulated by Courtade and Kumar states that for any Boolean function $f:\{0,1\}^n\to\{0,1\}$, $I(f(X^n);Y^n)\le 1-H(\alpha)$. Although the conjecture has been proved to be true in the dimension-free high noise regime by Samorodnitsky, here we present a calculus-based approach to show a dimension-dependent result by examining the second derivative of $H(\alpha)-H(f(X^n)|Y^n)$ at $\alpha=1/2$. Along the way, we show that the dictator function is the most informative function in the high noise regime., Comment: 17 pages; 1 figure; the short version has been accepted to ISIT 2019; corrected multiple typos in the previous version

As sensing and instrumentation play an increasingly important role in systems controlled over wired and wireless networks, the need to better understand delay-sensitive communication becomes a prime issue. Along these lines, this article studies the operation of data links that employ incremental redundancy as a practical means to protect information from the effects of unreliable channels. Specifically, this work extends a powerful methodology termed sequential differential optimization to choose near-optimal block sizes for hybrid ARQ over erasure channels. In doing so, an interesting connection between random coding and well-known constants in number theory is established. Furthermore, results show that the impact of the coding strategy adopted and the propensity of the channel to erase symbols naturally decouple when analyzing throughput. Overall, block size selection is motivated by normal approximations on the probability of decoding success at every stage of the incremental transmission process. This novel perspective, which rigorously bridges hybrid ARQ and coding, offers a pragmatic means to select code rates and blocklengths for incremental redundancy., Comment: 7 pages, 2 figures; A shorter version of this article will appear in the proceedings of ISIT 2018

Protograph-based Raptor-like low-density parity-check codes (PBRL codes) are a recently proposed family of easily encodable and decodable rate-compatible LDPC (RC-LDPC) codes. These codes have an excellent iterative decoding threshold and performance across all design rates. PBRL codes designed thus far, for both long and short block-lengths, have been based on optimizing the iterative decoding threshold of the protograph of the RC code family at various design rates. In this work, we propose a design method to obtain better quasi-cyclic (QC) RC-LDPC codes with PBRL structure for short block-lengths (of a few hundred bits). We achieve this by maximizing an upper bound on the minimum distance of any QC-LDPC code that can be obtained from the protograph of a PBRL ensemble. The obtained codes outperform the original PBRL codes at short block-lengths by significantly improving the error floor behavior at all design rates. Furthermore, we identify a reduction in complexity of the design procedure, facilitated by the general structure of a PBRL ensemble., Comment: Longer version of a submission to 2017 IEEE International Symposium on Information Theory

The read channel of a Flash memory cell degrades after repetitive program and erase (P/E) operations. This degradation is often modeled as a function of the number of P/E cycles. In contrast, this paper models the degradation as a function of the cumulative effect of the charge written and erased from the cell. Based on this modeling approach, this paper dynamically allocates voltage using lower-voltage write thresholds at the beginning of the device lifetime and increasing the thresholds as needed to maintain the mutual information of the read channel in the face of degradation. The paper introduces the technique in an idealized setting and then removes ideal assumptions about channel knowledge and available voltage resolution to conclude with a practical scheme with performance close to that of the idealized setting., Comment: 13 pages, 11 figures. Accepted by IEEE Transactions on Communications

This paper presents a general approach for optimizing the number of symbols in increments (packets of incremental redundancy) in a feedback communication system with a limited number of increments. This approach is based on a tight normal approximation on the rate for successful decoding. Applying this approach to a variety of feedback systems using non-binary (NB) low-density parity-check (LDPC) codes shows that greater than 90% of capacity can be achieved with average blocklengths fewer than 500 transmitted bits. One result is that the performance with ten increments closely approaches the performance with an infinite number of increments. The paper focuses on binary- input additive-white Gaussian noise (BI-AWGN) channels but also demonstrates that the normal approximation works well on examples of fading channels as well as high-SNR AWGN channels that require larger QAM constellations. The paper explores both variable-length feedback codes with termination (VLFT) and the more practical variable length feedback (VLF) codes without termination that require no assumption of noiseless transmitter confirmation. For VLF we consider both a two-phase scheme and CRC-based scheme.

A message composed of packets is transmitted using erasure and channel coding over a fading channel with no feedback. For this scenario, the paper explores the trade-off between the redundancies allocated to the packet-level erasure code and the channel code, along with an objective of a low probability of failure to recover the message. To this end, we consider a fading model that we term proportional-diversity block fading (PD block fading). For a fixed overall code rate and transmit power, we formulate an optimization problem to numerically find the optimal channel-coding rate (and thus the optimal erasure-coding rate) that minimizes the probability of failure for various approximations of the problem. Furthermore, an interpretation of the results from an incremental redundancy point of view shows how rate-compatibility affects the possible trajectories of the failure probability as a function of the overall code rate. Our numerical results suggest that an optimal, rateless, hybrid coding scheme for a single-user wireless system over the PD block-fading channel should have the rate of the erasure code approach one., Comment: 6 pages, 4 figures

Cyclic redundancy check (CRC) codes check if a codeword is correctly received. This paper presents an algorithm to design CRC codes that are optimized for the code-specific error behavior of a specified feedforward convolutional code. The algorithm utilizes two distinct approaches to computing undetected error probability of a CRC code used with a specific convolutional code. The first approach enumerates the error patterns of the convolutional code and tests if each of them is detectable. The second approach reduces complexity significantly by exploiting the equivalence of the undetected error probability to the frame error rate of an equivalent catastrophic convolutional code. The error events of the equivalent convolutional code are exactly the undetectable errors for the original concatenation of CRC and convolutional codes. This simplifies the computation because error patterns do not need to be individually checked for detectability. As an example, we optimize CRC codes for a commonly used 64-state convolutional code for information length k=1024 demonstrating significant reduction in undetected error probability compared to the existing CRC codes with the same degrees. For a fixed target undetected error probability, the optimized CRC codes typically require 2 fewer bits., Comment: 12 pages, 8 figures, journal paper

We consider the problem of constructing $(3,L)$ quasi-cyclic low-density parity-check (LDPC) codes from complete protographs. A complete protograph is a small bipartite graph with two disjoint vertex sets such that every vertex in the variable-node set is connected to every vertex in the check-node set by a unique edge. This paper analyzes the required lifting factor for achieving girths of six or eight in the resulting quasi-cyclic codes with constraints on lifting. The required lifting factors provide lower bounds on the block-length of such codes., Comment: 6 pages, 2 figures, 5-page version to appear in the Proceedings of 2015 IEEE International Symposium on Information Theory. Update 1 - 05/29/2015 - Minor changes and added a reference

Current generation Flash devices experience significant read-channel degradation from damage to the oxide layer during program and erase operations. Information about the read-channel degradation drives advanced signal processing methods in Flash to mitigate its effect. In this context, channel estimation must be ongoing since channel degradation evolves over time and as a function of the number of program/erase (P/E) cycles. This paper proposes a framework for ongoing model-based channel estimation using limited channel measurements (reads). This paper uses a channel model characterizing degradation resulting from retention time and the amount of charge programmed and erased. For channel histogram measurements, bin selection to achieve approximately equal-probability bins yields a good approximation to the original distribution using only ten bins (i.e. nine reads). With the channel model and binning strategy in place, this paper explores candidate numerical least squares algorithms and ultimately demonstrates the effectiveness of the Levenberg-Marquardt algorithm which provides both speed and accuracy., Comment: 6 pages, 8 figures, Submitted to the IEEE International Communications Conference (ICC) 2015

This paper presents a variable-length decision-feedback scheme that uses tail-biting convolutional codes and the tail-biting Reliability-Output Viterbi Algoritm (ROVA). Comparing with recent results in finite-blocklength information theory, simulation results for both the BSC and the AWGN channel show that the decision-feedback scheme using ROVA can surpass the random-coding lower bound on throughput for feedback codes at average blocklengths less than 100 symbols. This paper explores ROVA-based decision feedback both with decoding after every symbol and with decoding limited to a small number of increments. The performance of the reliability-based stopping rule with the ROVA is compared to retransmission decisions based on CRCs. For short blocklengths where the latency overhead of the CRC bits is severe, the ROVA-based approach delivers superior rates., Comment: Accepted for publication to IEEE Transactions on Communications. 15 single-spaced, double-column pages; 8 figures; 3 tables

We propose an approach where each user independently seeks to minimize the amount of time that they occupy the channel. Essentially, we seek to minimize the number of transmitted symbols required to communicate a packet assuming variable-length coding with feedback. Users send a pilot sequence to estimate the channel quality and decide whether to proceed with a transmission or wait for the next opportunity. Thus a user may choose to leave the channel even though it has already gained access, in order to increase the network throughput and also save its own energy resources. This paper optimizes the number of pilots and the channel identification threshold to minimize the total number of transmitted symbols (including pilots) required to communicate the packet. We prove a sufficient condition for the optimal pilot length and the channel identification threshold. This optimal parameter pair is solved numerically and the reduction in channel occupancy is shown for various channel settings., Comment: 6 pages, 3 figures, conference

This paper proposes a class of rate-compatible LDPC codes, called protograph-based Raptor-like (PBRL) codes. The construction is focused on binary codes for BI-AWGN channels. As with the Raptor codes, additional parity bits are produced by exclusive-OR operations on the precoded bits, providing extensive rate compatibility. Unlike Raptor codes, the structure of each additional parity bit in the protograph is explicitly designed through density evolution. The construction method provides low iterative decoding thresholds and the lifted codes result in excellent error rate performance for long-blocklength PBRL codes. For short-blocklength PBRL codes the protograph design and lifting must avoid undesired graphical structures such as trapping sets and absorbing sets while also seeking to minimize the density evolution threshold. Simulation results are shown in information block sizes of $k=192$, $16368$ and $16384$. Comparing at the same information block size of $k=16368$ bits, the PBRL codes outperform the best known standardized code, the AR4JA codes in the waterfall region. The PBRL codes also perform comparably to DVB-S2 codes even though the DVB-S2 codes use LDPC codes with longer blocklengths and are concatenated with outer BCH codes., Comment: 12 pages, 9 figures

The read channel in Flash memory systems degrades over time because the Fowler-Nordheim tunneling used to apply charge to the floating gate eventually compromises the integrity of the cell because of tunnel oxide degradation. While degradation is commonly measured in the number of program/erase cycles experienced by a cell, the degradation is proportional to the number of electrons forced into the floating gate and later released by the erasing process. By managing the amount of charge written to the floating gate to maintain a constant read-channel mutual information, Flash lifetime can be extended. This paper proposes an overall system approach based on information theory to extend the lifetime of a flash memory device. Using the instantaneous storage capacity of a noisy flash memory channel, our approach allocates the read voltage of flash cell dynamically as it wears out gradually over time. A practical estimation of the instantaneous capacity is also proposed based on soft information via multiple reads of the memory cells., Comment: 5 pages. 5 figures

We present extensions to Raghavan and Baum's reliability-output Viterbi algorithm (ROVA) to accommodate tail-biting convolutional codes. These tail-biting reliability-output algorithms compute the exact word-error probability of the decoded codeword after first calculating the posterior probability of the decoded tail-biting codeword's starting state. One approach employs a state-estimation algorithm that selects the maximum a posteriori state based on the posterior distribution of the starting states. Another approach is an approximation to the exact tail-biting ROVA that estimates the word-error probability. A comparison of the computational complexity of each approach is discussed in detail. The presented reliability-output algorithms apply to both feedforward and feedback tail-biting convolutional encoders. These tail-biting reliability-output algorithms are suitable for use in reliability-based retransmission schemes with short blocklengths, in which terminated convolutional codes would introduce rate loss., Comment: Submitted to IEEE Transaction on Communications. 10 pages, 8 figures, 2 tables

This paper explores feedback systems using incremental redundancy (IR) with noiseless transmitter confirmation (NTC). For IR-NTC systems based on {\em finite-length} codes (with blocklength $N$) and decoding attempts only at {\em certain specified decoding times}, this paper presents the asymptotic expansion achieved by random coding, provides rate-compatible sphere-packing (RCSP) performance approximations, and presents simulation results of tail-biting convolutional codes. The information-theoretic analysis shows that values of $N$ relatively close to the expected latency yield the same random-coding achievability expansion as with $N = \infty$. However, the penalty introduced in the expansion by limiting decoding times is linear in the interval between decoding times. For binary symmetric channels, the RCSP approximation provides an efficiently-computed approximation of performance that shows excellent agreement with a family of rate-compatible, tail-biting convolutional codes in the short-latency regime. For the additive white Gaussian noise channel, bounded-distance decoding simplifies the computation of the marginal RCSP approximation and produces similar results as analysis based on maximum-likelihood decoding for latencies greater than 200. The efficiency of the marginal RCSP approximation facilitates optimization of the lengths of incremental transmissions when the number of incremental transmissions is constrained to be small or the length of the incremental transmissions is constrained to be uniform after the first transmission. Finally, an RCSP-based decoding error trajectory is introduced that provides target error rates for the design of rate-compatible code families for use in feedback communication systems., Comment: 23 pages, 15 figures

This paper presents a reliability-based decoding scheme for variable-length coding with feedback and demonstrates via simulation that it can achieve higher rates than Polyanskiy et al.'s random coding lower bound for variable-length feedback (VLF) coding on both the BSC and AWGN channel. The proposed scheme uses the reliability output Viterbi algorithm (ROVA) to compute the word error probability after each decoding attempt, which is compared against a target error threshold and used as a stopping criterion to terminate transmission. The only feedback required is a single bit for each decoding attempt, informing the transmitter whether the ROVA-computed word-error probability is sufficiently low. Furthermore, the ROVA determines whether transmission/decoding may be terminated without the need for a rate-reducing CRC., Comment: To be published at the 2013 IEEE International Symposium on Information Theory, Istanbul, Turkey. 5 pages, 2 figures

Theoretical analysis has long indicated that feedback improves the error exponent but not the capacity of single-user memoryless channels. Recently Polyanskiy et al. studied the benefit of variable-length feedback with termination (VLFT) codes in the non-asymptotic regime. In that work, achievability is based on an infinite length random code and decoding is attempted at every symbol. The coding rate backoff from capacity due to channel dispersion is greatly reduced with feedback, allowing capacity to be approached with surprisingly small expected latency. This paper is mainly concerned with VLFT codes based on finite-length codes and decoding attempts only at certain specified decoding times. The penalties of using a finite block-length $N$ and a sequence of specified decoding times are studied. This paper shows that properly scaling $N$ with the expected latency can achieve the same performance up to constant terms as with $N = \infty$. The penalty introduced by periodic decoding times is a linear term of the interval between decoding times and hence the performance approaches capacity as the expected latency grows if the interval between decoding times grows sub-linearly with the expected latency., Comment: 8 pages. A shorten version is submitted to ISIT 2013

Recent results by Chen et al. and Polyanskiy et al. explore using feedback to approach capacity with short blocklengths. This paper explores Chernoff bounding techniques to extend the rate-compatible sphere-packing (RCSP) analysis proposed by Chen et al. to scenarios involving numerous retransmissions and different step sizes in each incremental retransmission. Williamson et al. employ exact RCSP computations for up to six transmissions. However, exact RCSP computation with more than six retransmissions becomes unwieldy because of joint error probabilities involving numerous chi-squared distributions. This paper explores Chernoff approaches for upper and lower bounds to provide support for computations involving more than six transmissions. We present two versions of upper and lower bounds for the two-transmission case. One of the versions is extended to the general case of $m$ transmissions where $m \geq 1$. Computing the general bounds requires minimization of exponential functions with the auxiliary parameters, but is less complex and more stable than multiple rounds of numerical integration. These bounds also provide a good estimate of the expected throughput and expected latency, which are useful for optimization purposes., Comment: Submitted to 2012 Information Theory Workshop

Consider a connected network of n nodes that all wish to recover k desired packets. Each node begins with a subset of the desired packets and exchanges coded packets with its neighbors. This paper provides necessary and sufficient conditions which characterize the set of all transmission schemes that permit every node to ultimately learn (recover) all k packets. When the network satisfies certain regularity conditions and packets are randomly distributed, this paper provides tight concentration results on the number of transmissions required to achieve universal recovery. For the case of a fully connected network, a polynomial-time algorithm for computing an optimal transmission scheme is derived. An application to secrecy generation is discussed., Comment: 49 pages, 6 figures, submitted to Transactions on Information Theory

Recent work by Polyanskiy et al. and Chen et al. has excited new interest in using feedback to approach capacity with low latency. Polyanskiy showed that feedback identifying the first symbol at which decoding is successful allows capacity to be approached with surprisingly low latency. This paper uses Chen's rate-compatible sphere-packing (RCSP) analysis to study what happens when symbols must be transmitted in packets, as with a traditional hybrid ARQ system, and limited to relatively few (six or fewer) incremental transmissions. Numerical optimizations find the series of progressively growing cumulative block lengths that enable RCSP to approach capacity with the minimum possible latency. RCSP analysis shows that five incremental transmissions are sufficient to achieve 92% of capacity with an average block length of fewer than 101 symbols on the AWGN channel with SNR of 2.0 dB. The RCSP analysis provides a decoding error trajectory that specifies the decoding error rate for each cumulative block length. Though RCSP is an idealization, an example tail-biting convolutional code matches the RCSP decoding error trajectory and achieves 91% of capacity with an average block length of 102 symbols on the AWGN channel with SNR of 2.0 dB. We also show how RCSP analysis can be used in cases where packets have deadlines associated with them (leading to an outage probability)., Comment: To be published at the 2012 IEEE International Symposium on Information Theory, Cambridge, MA, USA. Updated to incorporate reviewers' comments and add new figures

This paper considers the problem of transferring a file from one source node to multiple receivers in a peer-to-peer (P2P) network. The objective is to minimize the weighted sum download time (WSDT) for the one-to-many file transfer. Previous work has shown that, given an order at which the receivers finish downloading, the minimum WSD can be solved in polynomial time by convex optimization, and can be achieved by linear network coding, assuming that node uplinks are the only bottleneck in the network. This paper, however, considers heterogeneous peers with both uplink and downlink bandwidth constraints specified. The static scenario is a file-transfer scheme in which the network resource allocation remains static until all receivers finish downloading. This paper first shows that the static scenario may be optimized in polynomial time by convex optimization, and the associated optimal static WSD can be achieved by linear network coding. This paper then presented a lower bound to the minimum WSDT that is easily computed and turns out to be tight across a wide range of parameterizations of the problem. This paper also proposes a static routing-based scheme and a static rateless-coding-based scheme which have almost-optimal empirical performances. The dynamic scenario is a file-transfer scheme which can re-allocate the network resource during the file transfer. This paper proposes a dynamic rateless-coding-based scheme, which provides significantly smaller WSDT than the optimal static scenario does., Comment: 67 pages, 21 figures

We consider a communication network with a single source that has a set of messages and two terminals where each terminal is interested in an arbitrary subset of messages at the source. A tight capacity region for this problem is demonstrated. We show by a simple graph-theoretic procedure that any such problem can be solved by performing network coding on the subset of messages that are requested by both the terminals and that routing is sufficient for transferring the remaining messages., Comment: 4 pages, appeared at the Canadian Workshop on Information Theory (CWIT), 2005