Search

The permanent of a non-negative square matrix can be well approximated by finding the minimum of the Bethe free energy functions associated with some suitably defined factor graph; the resulting approximation to the permanent is called the Bethe permanent. Vontobel gave a combinatorial characterization of the Bethe permanent via degree-$M$ Bethe permanents, which are based on degree-$M$ covers of the underlying factor graph. In this paper, we prove a degree-$M$-Bethe-permanent-based lower bound on the permanent of a non-negative matrix, which solves a conjecture proposed by Vontobel in [IEEE Trans. Inf. Theory, Mar. 2013]. We also prove a degree-$M$-Bethe-permanent-based upper bound on the permanent of a non-negative matrix. In the limit $M \to \infty$, these lower and upper bounds yield known Bethe-permanent-based lower and upper bounds on the permanent of a non-negative matrix. Moreover, we prove similar results for an approximation to the permanent known as the (scaled) Sinkhorn permanent., Comment: to appear in the IEEE Transactions on Information Theory

The permanent of a non-negative matrix appears naturally in many information processing scenarios. Because of the intractability of the permanent beyond small matrices, various approximation techniques have been developed in the past. In this paper, we study the Bethe approximation of the permanent and add to the body of literature showing that this approximation is very well behaved in many respects. Our main technical tool are topological double covers of the normal factor graph whose partition function equals the permanent of interest, along with a transformation of these double covers., Comment: 17 pages, 4 figures

Quantum mass functions (QMFs), which are tightly related to decoherence functionals, were introduced by Loeliger and Vontobel [IEEE Trans. Inf. Theory, 2017, 2020] as a generalization of probability mass functions toward modeling quantum information processing setups in terms of factor graphs. Simple quantum mass functions (SQMFs) are a special class of QMFs that do not explicitly model classical random variables. Nevertheless, classical random variables appear implicitly in an SQMF if some marginals of the SQMF satisfy some conditions; variables of the SQMF corresponding to these "emerging" random variables are called classicable variables. Of particular interest are jointly classicable variables. In this paper we initiate the characterization of the set of marginals given by the collection of jointly classicable variables of a graphical model and compare them with other concepts associated with graphical models like the sets of realizable marginals and the local marginal polytope. In order to further characterize this set of marginals given by the collection of jointly classicable variables, we generalize the CHSH inequality based on the Pearson correlation coefficients, and thereby prove a conjecture proposed by Pozsgay et al. A crucial feature of this inequality is its nonlinearity, which poses difficulties in the proof.

We consider reliable and secure communication over intersymbol interference wiretap channels (ISI-WTCs). In particular, we first derive an achievable secure rate for ISI-WTCs without imposing any constraints on the input distribution. Afterwards, we focus on the setup where the input distribution of the ISI-WTC is constrained to be a time-invariant finite-order Markov chain. Optimizing the parameters of this Markov chain toward maximizing the achievable secure rates is a computationally intractable problem in general, and so, toward finding a local maximum, we propose an iterative algorithm that at every iteration replaces the secure rate function with a suitable~surrogate function whose maximum can be found efficiently. Although the secure rates achieved in the unconstrained setup are potentially larger than the secure rates achieved in the constrained setup, the latter setup has the advantage of leading to efficient algorithms for estimating and optimizing the achievable secure rates, and also has the benefit of being the basis of efficient coding schemes., Comment: 16 pages, 8 figures, 1 table

We consider sparse superposition codes (SPARCs) over complex AWGN channels. Such codes can be efficiently decoded by an approximate message passing (AMP) decoder, whose performance can be predicted via so-called state evolution in the large-system limit. In this paper, we mainly focus on how to use concatenation of SPARCs and cyclic redundancy check (CRC) codes on the encoding side and use list decoding on the decoding side to improve the finite-length performance of the AMP decoder for SPARCs over complex AWGN channels. Simulation results show that such a concatenated coding scheme works much better than SPARCs with the original AMP decoder and results in a steep waterfall-like behavior in the bit-error rate performance curves. Furthermore, we apply our proposed concatenated coding scheme to spatially coupled SPARCs. Besides that, we also introduce a novel class of design matrices, i.e., matrices that describe the encoding process, based on circulant matrices derived from Frank or from Milewski sequences. This class of design matrices has comparable encoding and decoding computational complexity as well as very close performance with the commonly-used class of design matrices based on discrete Fourier transform (DFT) matrices, but gives us more degrees of freedom when designing SPARCs for various applications., Comment: 12 pages, 8 figures, revised

In previous work, we have shown that pseudocodewords can be used to characterize the behavior of decoders not only for classical codes but also for quantum stabilizer codes. With the insights obtained from this pseudocodewords-based analysis, we have also introduced a two-stage decoder based on pseudocodewords for quantum cycle codes that leads to improved decoding performance. In this paper, we consider quantum (stabilizer) color codes and propose a two-stage decoder that is a generalization of the pseudocodeword-based decoder for quantum cycle codes. Our decoder has only local or error-weight-dependent operations of low computational complexity and better decoding performance compared with previous decoding approaches for these types of codes., Comment: Submitted to ITW 2020

It has been shown that graph-cover pseudocodewords can be used to characterize the behavior of sum-product algorithm (SPA) decoding of classical codes. In this paper, we leverage and adapt these results to analyze SPA decoding of quantum stabilizer codes. We use the obtained insights to formulate modifications to the SPA that overcome some of its weaknesses., Comment: This is the extended version of a paper that appears in the proceedings of ISIT 2019 in Paris July 7-12, 2019

We consider the scenario of classical communication over a finite-dimensional quantum channel with memory using a separable-state input ensemble and local output measurements. We propose algorithms for estimating the information rate of such communication setups, along with algorithms for bounding the information rate based on so-called auxiliary channels. Some of the algorithms are extensions of their counterparts for (classical) finite-state-machine channels. Notably, we discuss suitable graphical models for doing the relevant computations. Moreover, the auxiliary channels are learned in a data-driven approach; i.e., only input/output sequences of the true channel are needed, but not the channel model of the true channel., Comment: This work has been submitted to the IEEE Transactions on Information Theory for possible publication

In prior work, we have shown how the basic concepts and terms of quantum mechanics relate to factorizations and marginals of complex-valued quantum mass functions, which are generalizations of joint probability mass functions. In this paper, using quantum mass functions, we discuss the realization of measurements in terms of unitary interactions and marginalizations. It follows that classical measurement results strictly belong to local models, i.e., marginals of more detailed models. Classical variables that are created by marginalization do not exist in the unmarginalized model, and different marginalizations may yield incompatible classical variables. These observations are illustrated by the Frauchiger-Renner paradox, which is analyzed (and resolved) in terms of quantum mass functions. Throughout, the paper uses factor graphs to represent quantum systems/models with multiple measurements at different points in time., Comment: Submitted

Coded computation is an emerging research area that leverages concepts from erasure coding to mitigate the effect of stragglers (slow nodes) in distributed computation clusters, especially for matrix computation problems. In this work, we present a class of distributed matrix-vector multiplication schemes that are based on codes in the Rosenbloom-Tsfasman metric and universally decodable matrices. Our schemes take into account the inherent computation order within a worker node. In particular, they allow us to effectively leverage partial computations performed by stragglers (a feature that many prior works lack). An additional main contribution of our work is a companion matrix-based embedding of these codes that allows us to obtain sparse and numerically stable schemes for the problem at hand. Experimental results confirm the effectiveness of our techniques., Comment: 6 pages, 1 figure

Some of the most interesting quantities associated with a factor graph are its marginals and its partition sum. For factor graphs \emph{without cycles} and moderate message update complexities, the sum-product algorithm (SPA) can be used to efficiently compute these quantities exactly. Moreover, for various classes of factor graphs \emph{with cycles}, the SPA has been successfully applied to efficiently compute good approximations to these quantities. Note that in the case of factor graphs with cycles, the local functions are usually non-negative real-valued functions. In this paper we introduce a class of factor graphs, called double-edge factor graphs (DE-FGs), which allow local functions to be complex-valued and only require them, in some suitable sense, to be positive semi-definite. We discuss various properties of the SPA when running it on DE-FGs and we show promising numerical results for various example DE-FGs, some of which have connections to quantum information processing., Comment: Submitted

We consider the problem of transmitting classical information over a time-invariant channel with memory. A popular class of time-invariant channels with memory are finite-state-machine channels, where a \emph{classical} state evolves over time and governs the relationship between the classical input and the classical output of the channel. For such channels, various techniques have been developed for estimating and bounding the information rate. In this paper we consider a class of time-invariant channels where a \emph{quantum} state evolves over time and governs the relationship between the classical input and the classical output of the channel. We propose algorithms for estimating and bounding the information rate of such channels. In particular, we discuss suitable graphical models for doing the relevant computations., Comment: This is an extended version of a paper that appears in Proc. 2017 IEEE International Symposium on Information Theory, Aachen, Germany, June 2017

We study the size (or volume) of balls in the metric space of permutations, $S_n$, under the infinity metric. We focus on the regime of balls with radius $r = \rho \cdot (n\!-\!1)$, $\rho \in [0,1]$, i.e., a radius that is a constant fraction of the maximum possible distance. We provide new lower bounds on the size of such balls. These new lower bounds reduce the asymptotic gap to the known upper bounds to at most $0.029$ bits per symbol. Additionally, they imply an improved ball-packing bound for error-correcting codes, and an improved upper bound on the size of optimal covering codes., Comment: To appear in IEEE Transactions on Information Theory

Many quantities of interest in communications, signal processing, artificial intelligence, and other areas can be expressed as the partition sum of some factor graph. Although the exact calculation of the partition sum is in many cases intractable, it can often be approximated rather well by the Bethe partition sum. In earlier work, we have shown that graph covers are a useful tool for expressing and analyzing the Bethe approximation. In this paper, we present a novel technique for analyzing double covers, a technique which ultimately leads to a deeper understanding of the Bethe approximation., Comment: Proceedings of the 2016 International Conference on Signal Processing and Communications, Bangalore, India, June 12-15, 2016

Algebraic topology studies topological spaces with the help of tools from abstract algebra. The main focus of this paper is to show that many concepts from algebraic topology can be conveniently expressed in terms of (normal) factor graphs. As an application, we give an alternative proof of a classical duality result of Kramers and Wannier, which expresses the partition function of the two-dimensional Ising model at a low temperature in terms of the partition function of the two-dimensional Ising model at a high temperature. Moreover, we discuss analogous results for the three-dimensional Ising model and the Potts model.

A factor-graph representation of quantum-mechanical probabilities (involving any number of measurements) is proposed. Unlike standard statistical models, the proposed representation uses auxiliary variables (state variables) that are not random variables. All joint probability distributions are marginals of some complex-valued function $q$, and it is demonstrated how the basic concepts of quantum mechanics relate to factorizations and marginals of $q$., Comment: To appear in IEEE Transactions on Information Theory, 2017

We design low-complexity error correction coding schemes for channels that introduce different types of errors and erasures: on the one hand, the proposed schemes can successfully deal with symbol errors and erasures, and, on the other hand, they can also successfully handle phased burst errors and erasures., Comment: To appear in IEEE Transactions on Information Theory, 2014

Linear Programming (LP) decoding of Low-Density Parity-Check (LDPC) codes has attracted much attention in the research community in the past few years. LP decoding has been derived for binary and nonbinary linear codes. However, the most important problem with LP decoding for both binary and nonbinary linear codes is that the complexity of standard LP solvers such as the simplex algorithm remains prohibitively large for codes of moderate to large block length. To address this problem, two low-complexity LP (LCLP) decoding algorithms for binary linear codes have been proposed by Vontobel and Koetter, henceforth called the basic LCLP decoding algorithm and the subgradient LCLP decoding algorithm. In this paper, we generalize these LCLP decoding algorithms to nonbinary linear codes. The computational complexity per iteration of the proposed nonbinary LCLP decoding algorithms scales linearly with the block length of the code. A modified BCJR algorithm for efficient check-node calculations in the nonbinary basic LCLP decoding algorithm is also proposed, which has complexity linear in the check node degree. Several simulation results are presented for nonbinary LDPC codes defined over Z_4, GF(4), and GF(8) using quaternary phase-shift keying and 8-phase-shift keying, respectively, over the AWGN channel. It is shown that for some group-structured LDPC codes, the error-correcting performance of the nonbinary LCLP decoding algorithms is similar to or better than that of the min-sum decoding algorithm., Comment: To appear in IEEE Transactions on Communications, 2013

A factor-graph representation of quantum-mechanical probabilities is proposed. Unlike standard statistical models, the proposed representation uses auxiliary variables (state variables) that are not random variables., Comment: Proc. IEEE International Symposium on Information Theory (ISIT), Cambridge, MA, July 1-6, 2012

It has recently been observed that the permanent of a non-negative square matrix, i.e., of a square matrix containing only non-negative real entries, can very well be approximated by solving a certain Bethe free energy function minimization problem with the help of the sum-product algorithm. We call the resulting approximation of the permanent the Bethe permanent. In this paper we give reasons why this approach to approximating the permanent works well. Namely, we show that the Bethe free energy function is convex and that the sum-product algorithm finds its minimum efficiently. We then discuss the fact that the permanent is lower bounded by the Bethe permanent, and we comment on potential upper bounds on the permanent based on the Bethe permanent. We also present a combinatorial characterization of the Bethe permanent in terms of permanents of so-called lifted versions of the matrix under consideration. Moreover, we comment on possibilities to modify the Bethe permanent so that it approximates the permanent even better, and we conclude the paper with some observations and conjectures about permanent-based pseudo-codewords and permanent-based kernels., Comment: Accepted for IEEE Transactions on Information Theory. Manuscript received July 21, 2011; date of current version October 20, 2012. Changes compared to v1: see v2. Changes compared to v2: changed t to \tau in Section IV in order to distinguish it from iteration number t in Section V. Fixed some typos

Inspired by some new advances on normal factor graphs (NFGs), we introduce NFGs as a simple and intuitive diagrammatic approach towards encoding some concepts from linear algebra. We illustrate with examples the workings of such an approach and settle a conjecture of Peterson on the Pfaffian., Comment: ISIT2011, Saint-Petersburg, Russia

One of the most common types of functions in mathematics, physics, and engineering is a sum of products, sometimes called a partition function. After "normalization," a sum of products has a natural graphical representation, called a normal factor graph (NFG), in which vertices represent factors, edges represent internal variables, and half-edges represent the external variables of the partition function. In physics, so-called trace diagrams share similar features. We believe that the conceptual framework of representing sums of products as partition functions of NFGs is an important and intuitive paradigm that, surprisingly, does not seem to have been introduced explicitly in the previous factor graph literature. Of particular interest are NFG modifications that leave the partition function invariant. A simple subclass of such NFG modifications offers a unifying view of the Fourier transform, tree-based reparameterization, loop calculus, and the Legendre transform., Comment: 8 pages, 17 figures. To be presented at 2011 Information Theory and Applications Workshop, La Jolla, CA, February 7, 2011

We present a mathematical connection between channel coding and compressed sensing. In particular, we link, on the one hand, \emph{channel coding linear programming decoding (CC-LPD)}, which is a well-known relaxation o maximum-likelihood channel decoding for binary linear codes, and, on the other hand, \emph{compressed sensing linear programming decoding (CS-LPD)}, also known as basis pursuit, which is a widely used linear programming relaxation for the problem of finding the sparsest solution of an under-determined system of linear equations. More specifically, we establis a tight connection between CS-LPD based on a zero-one measurement matrix over the reals and CC-LPD of the binary linear channel code that is obtained by viewing this measurement matrix as a binary parity-check matrix. This connection allows the translation of performance guarantees from one setup to the other. The main message of this paper is that parity-check matrices of "good" channel codes can be used as provably "good" measurement matrices under basis pursuit. In particular, we provide the first deterministic construction of compressed sensing measurement matrices with an order-optimal number of rows using high-girth low-density parity-check (LDPC) codes constructed by Gallager., Comment: To appear, IEEE Transactions on Information Theory, 2012

We present a combinatorial characterization of the Bethe entropy function of a factor graph, such a characterization being in contrast to the original, analytical, definition of this function. We achieve this combinatorial characterization by counting valid configurations in finite graph covers of the factor graph. Analogously, we give a combinatorial characterization of the Bethe partition function, whose original definition was also of an analytical nature. As we point out, our approach has similarities to the replica method, but also stark differences. The above findings are a natural backdrop for introducing a decoder for graph-based codes that we will call symbolwise graph-cover decoding, a decoder that extends our earlier work on blockwise graph-cover decoding. Both graph-cover decoders are theoretical tools that help towards a better understanding of message-passing iterative decoding, namely blockwise graph-cover decoding links max-product (min-sum) algorithm decoding with linear programming decoding, and symbolwise graph-cover decoding links sum-product algorithm decoding with Bethe free energy function minimization at temperature one. In contrast to the Gibbs entropy function, which is a concave function, the Bethe entropy function is in general not concave everywhere. In particular, we show that every code picked from an ensemble of regular low-density parity-check codes with minimum Hamming distance growing (with high probability) linearly with the block length has a Bethe entropy function that is convex in certain regions of its domain., Comment: Submitted to IEEE Trans. Inf. Theory, Nov. 20, 2010; rev. Sep. 22, 2012; current version, Oct. 9, 2012. Main changes from v1 to v2: new example (Example 34), new lemma (Lemma 35), changed some notation, changed the domain of the Gibbs free energy function and related functions, reordered some sections/appendices, fixed some typos, improved the background discussion, added some new references

Low-density parity-check (LDPC) convolutional codes are capable of achieving excellent performance with low encoding and decoding complexity. In this paper we discuss several graph-cover-based methods for deriving families of time-invariant and time-varying LDPC convolutional codes from LDPC block codes and show how earlier proposed LDPC convolutional code constructions can be presented within this framework. Some of the constructed convolutional codes significantly outperform the underlying LDPC block codes. We investigate some possible reasons for this "convolutional gain," and we also discuss the --- mostly moderate --- decoder cost increase that is incurred by going from LDPC block to LDPC convolutional codes., Comment: Submitted to IEEE Transactions on Information Theory, April 2010; revised August 2010, revised November 2010 (essentially final version). (Besides many small changes, the first and second revised versions contain corrected entries in Tables I and II.)

This is a tale of two linear programming decoders, namely channel coding linear programming decoding (CC-LPD) and compressed sensing linear programming decoding (CS-LPD). So far, they have evolved quite independently. The aim of the present paper is to show that there is a tight connection between, on the one hand, CS-LPD based on a zero-one measurement matrix over the reals and, on the other hand, CC-LPD of the binary linear code that is obtained by viewing this measurement matrix as a binary parity-check matrix. This connection allows one to translate performance guarantees from one setup to the other., Comment: Appeared in the Proceedings of the 47th Allerton Conference on Communications, Control, and Computing, Allerton House, Monticello, Illinois, USA, Sep. 30 - Oct. 2, 2009. This version of the paper contains all the proofs that were omitted in the official version due to space limitations

The linear-programming decoding performance of a binary linear code crucially depends on the structure of the fundamental cone of the parity-check matrix that describes the code. Towards a better understanding of fundamental cones and the vectors therein, we introduce the notion of absdet-pseudo-codewords and perm-pseudo-codewords: we give the definitions, we discuss some simple examples, and we list some of their properties.

Quasi-cyclic (QC) low-density parity-check (LDPC) codes are an important instance of proto-graph-based LDPC codes. In this paper we present upper bounds on the minimum Hamming distance of QC LDPC codes and study how these upper bounds depend on graph structure parameters (like variable degrees, check node degrees, girth) of the Tanner graph and of the underlying proto-graph. Moreover, for several classes of proto-graphs we present explicit QC LDPC code constructions that achieve (or come close to) the respective minimum Hamming distance upper bounds. Because of the tight algebraic connection between QC codes and convolutional codes, we can state similar results for the free Hamming distance of convolutional codes. In fact, some QC code statements are established by first proving the corresponding convolutional code statements and then using a result by Tanner that says that the minimum Hamming distance of a QC code is upper bounded by the free Hamming distance of the convolutional code that is obtained by "unwrapping" the QC code., Comment: To appear in IEEE Transactions on Information Theory. Changes compared to v1: some convolutional code results have been added; some incompleteness issues with some of the proofs have been corrected; a typo in one of the parity-check matrices has been corrected (i.e., an entry of H"(x) in Example 28 of v1 needs to be changed so that d_min=56 as written there, cf. \hat H(x) in Example 29 of v2)

Let n be an even positive integer and F be the field \GF(2). A word in F^n is called balanced if its Hamming weight is n/2. A subset C \subseteq F^n$ is called a balancing set if for every word y \in F^n there is a word x \in C such that y + x is balanced. It is shown that most linear subspaces of F^n of dimension slightly larger than 3/2\log_2(n) are balancing sets. A generalization of this result to linear subspaces that are "almost balancing" is also presented. On the other hand, it is shown that the problem of deciding whether a given set of vectors in F^n spans a balancing set, is NP-hard. An application of linear balancing sets is presented for designing efficient error-correcting coding schemes in which the codewords are balanced., Comment: The abstract of this paper appeared in the proc. of 2009 International Symposium on Information Theory

A generalization of the Reiger bound is presented for the list decoding of burst errors. It is then shown that Reed-Solomon codes attain this bound., Comment: Submitted to IEEE Transactions on Information Theory, August 19, 2008

Quantum error-correction codes (QECCs) are a vital ingredient of quantum computation and communication systems. In that context it is highly desirable to design QECCs that can be represented by graphical models which possess a structure that enables efficient and close-to-optimal iterative decoding. In this paper we focus on stabilizer QECCs, a class of QECCs whose construction is rendered non-trivial by the fact that the stabilizer label code, a code that is associated with a stabilizer QECC, has to satisfy a certain self-orthogonality condition. In order to design graphical models of stabilizer label codes that satisfy this condition, we extend a duality result for Forney-style factor graphs (FFGs) to the stabilizer label code framework. This allows us to formulate a simple FFG design rule for constructing stabilizer label codes, a design rule that unifies several earlier stabilizer label code constructions., Comment: Proceedings 5th International Symposium on Turbo Codes and Related Topics, Lausanne, Switzerland, September 1-5, 2008

Interior-point algorithms constitute a very interesting class of algorithms for solving linear-programming problems. In this paper we study efficient implementations of such algorithms for solving the linear program that appears in the linear-programming decoder formulation., Comment: Essentially the paper that appeared in Proc. 2008 Information Theory and Applications Workshop, UC San Diego, CA, USA, January 27 -- February 1, 2008

We consider the problem of minimizing upper bounds and maximizing lower bounds on information rates of stationary and ergodic discrete-time channels with memory. The channels we consider can have a finite number of states, such as partial response channels, or they can have an infinite state-space, such as time-varying fading channels. We optimize recently-proposed information rate bounds for such channels, which make use of auxiliary finite-state machine channels (FSMCs). Our main contribution in this paper is to provide iterative expectation-maximization (EM) type algorithms to optimize the parameters of the auxiliary FSMC to tighten these bounds. We provide an explicit, iterative algorithm that improves the upper bound at each iteration. We also provide an effective method for iteratively optimizing the lower bound. To demonstrate the effectiveness of our algorithms, we provide several examples of partial response and fading channels, where the proposed optimization techniques significantly tighten the initial upper and lower bounds. Finally, we compare our results with an improved variation of the \emph{simplex} local optimization algorithm, called \emph{Soblex}. This comparison shows that our proposed algorithms are superior to the Soblex method, both in terms of robustness in finding the tightest bounds and in computational efficiency. Interestingly, from a channel coding/decoding perspective, optimizing the lower bound is related to increasing the achievable mismatched information rate, i.e., the information rate of a communication system where the decoder at the receiver is matched to the auxiliary channel, and not to the original channel., Comment: Submitted to IEEE Transactions on Information Theory, November 24, 2007

Message-passing iterative decoders for low-density parity-check (LDPC) block codes are known to be subject to decoding failures due to so-called pseudo-codewords. These failures can cause the large signal-to-noise ratio performance of message-passing iterative decoding to be worse than that predicted by the maximum-likelihood decoding union bound. In this paper we address the pseudo-codeword problem from the convolutional-code perspective. In particular, we compare the performance of LDPC convolutional codes with that of their ``wrapped'' quasi-cyclic block versions and we show that the minimum pseudo-weight of an LDPC convolutional code is at least as large as the minimum pseudo-weight of an underlying quasi-cyclic code. This result, which parallels a well-known relationship between the minimum Hamming weight of convolutional codes and the minimum Hamming weight of their quasi-cyclic counterparts, is due to the fact that every pseudo-codeword in the convolutional code induces a pseudo-codeword in the block code with pseudo-weight no larger than that of the convolutional code's pseudo-codeword. This difference in the weight spectra leads to improved performance at low-to-moderate signal-to-noise ratios for the convolutional code, a conclusion supported by simulation results., Comment: 26 pages, 6 figures, 2 tables

In order to understand the performance of a code under maximum-likelihood (ML) decoding, one studies the codewords, in particular the minimal codewords, and their Hamming weights. In the context of linear programming (LP) decoding, one's attention needs to be shifted to the pseudo-codewords, in particular to the minimal pseudo-codewords, and their pseudo-weights. In this paper we investigate some families of codes that have good properties under LP decoding, namely certain families of low-density parity-check (LDPC) codes that are derived from projective and Euclidean planes: we study the structure of their minimal pseudo-codewords and give lower bounds on their pseudo-weight., Comment: Submitted to IEEE Transactions on Information Theory, February 25, 2006

Whereas many results are known about thresholds for ensembles of low-density parity-check codes under message-passing iterative decoding, this is not the case for linear programming decoding. Towards closing this knowledge gap, this paper presents some bounds on the thresholds of low-density parity-check code ensembles under linear programming decoding.

We consider linear-programming (LP) decoding of low-density parity-check (LDPC) codes. While it is clear that one can use any general-purpose LP solver to solve the LP that appears in the decoding problem, we argue in this paper that the LP at hand is equipped with a lot of structure that one should take advantage of. Towards this goal, we study the dual LP and show how coordinate-ascent methods lead to very simple update rules that are tightly connected to the min-sum algorithm. Moreover, replacing minima in the formula of the dual LP with soft-minima one obtains update rules that are tightly connected to the sum-product algorithm. This shows that LP solvers with complexity similar to the min-sum algorithm and the sum-product algorithm are feasible. Finally, we also discuss some sub-gradient-based methods.

In his thesis, Wiberg showed the existence of thresholds for families of regular low-density parity-check codes under min-sum algorithm decoding. He also derived analytic bounds on these thresholds. In this paper, we formulate similar results for linear programming decoding of regular low-density parity-check codes.

The notion of universally decodable matrices (UDMs) was recently introduced by Tavildar and Viswanath while studying slow fading channels. It turns out that the problem of constructing UDMs is tightly connected to the problem of constructing maximum distance separable (MDS) codes. In this paper, we first study the properties of UDMs in general and then we discuss an explicit construction of a class of UDMs, a construction which can be seen as an extension of Reed-Solomon codes. In fact, we show that this extension is, in a sense to be made more precise later on, unique. Moreover, the structure of this class of UDMs allows us to answer some open conjectures by Tavildar, Viswanath, and Doshi in the positive, and it also allows us to formulate an efficient decoding algorithm for this class of UDMs. It turns out that our construction yields a coding scheme that is essentially equivalent to a class of codes that was proposed by Rosenbloom and Tsfasman. Moreover, we point out connections to so-called repeated-root cyclic codes., Comment: Journal version of earlier conference proceedings paper. Submitted on April 27, 2006

Universally decodable matrices (UDMs) can be used for coding purposes when transmitting over slow fading channels. These matrices are parameterized by positive integers $L$ and $N$ and a prime power $q$. The main result of this paper is that the simple condition $L \leq q+1$ is both necessary and sufficient for $(L,N,q)$-UDMs to exist. The existence proof is constructive and yields a coding scheme that is equivalent to a class of codes that was proposed by Rosenbloom and Tsfasman. Our work resolves an open problem posed recently in the literature.

The goal of the present paper is the derivation of a framework for the finite-length analysis of message-passing iterative decoding of low-density parity-check codes. To this end we introduce the concept of graph-cover decoding. Whereas in maximum-likelihood decoding all codewords in a code are competing to be the best explanation of the received vector, under graph-cover decoding all codewords in all finite covers of a Tanner graph representation of the code are competing to be the best explanation. We are interested in graph-cover decoding because it is a theoretical tool that can be used to show connections between linear programming decoding and message-passing iterative decoding. Namely, on the one hand it turns out that graph-cover decoding is essentially equivalent to linear programming decoding. On the other hand, because iterative, locally operating decoding algorithms like message-passing iterative decoding cannot distinguish the underlying Tanner graph from any covering graph, graph-cover decoding can serve as a model to explain the behavior of message-passing iterative decoding. Understanding the behavior of graph-cover decoding is tantamount to understanding the so-called fundamental polytope. Therefore, we give some characterizations of this polytope and explain its relation to earlier concepts that were introduced to understand the behavior of message-passing iterative decoding for finite-length codes., Comment: Submitted to IEEE Transactions on Information Theory, December 2005

We would like to better understand the fundamental cone of Tanner graphs derived from finite projective planes. Towards this goal, we discuss bounds on the AWGNC and BSC pseudo-weight of minimal pseudo-codewords of such Tanner graphs, on one hand, and study the structure of minimal pseudo-codewords, on the other., Comment: This paper is a slightly reformulated version of the paper that appeared in the proceedings of the 43rd Allerton Conference on Communications, Control, and Computing, Allerton House, Monticello, Illinois, USA, Sept. 28-30, 2005

Universally decodable matrices can be used for coding purposes when transmitting over slow fading channels. These matrices are parameterized by positive integers $L$ and $n$ and a prime power $q$. Based on Pascal's triangle we give an explicit construction of universally decodable matrices for any non-zero integers $L$ and $n$ and any prime power $q$ where $L \leq q+1$. This is the largest set of possible parameter values since for any list of universally decodable matrices the value $L$ is upper bounded by $q+1$, except for the trivial case $n = 1$. For the proof of our construction we use properties of Hasse derivatives, and it turns out that our construction has connections to Reed-Solomon codes, Reed-Muller codes, and so-called repeated-root cyclic codes. Additionally, we show how universally decodable matrices can be modified so that they remain universally decodable matrices.

An important property of high-performance, low complexity codes is the existence of highly efficient algorithms for their decoding. Many of the most efficient, recent graph-based algorithms, e.g. message passing algorithms and decoding based on linear programming, crucially depend on the efficient representation of a code in a graphical model. In order to understand the performance of these algorithms, we argue for the characterization of codes in terms of a so called fundamental cone in Euclidean space which is a function of a given parity check matrix of a code, rather than of the code itself. We give a number of properties of this fundamental cone derived from its connection to unramified covers of the graphical models on which the decoding algorithms operate. For the class of cycle codes, these developments naturally lead to a characterization of the fundamental polytope as the Newton polytope of the Hashimoto edge zeta function of the underlying graph., Comment: Submitted, August 2005

Consider data transmission over a binary-input additive white Gaussian noise channel using a binary low-density parity-check code. We ask the following question: Given a decoder that takes log-likelihood ratios as input, does it help to modify the log-likelihood ratios before decoding? If we use an optimal decoder then it is clear that modifying the log-likelihoods cannot possibly help the decoder's performance, and so the answer is "no." However, for a suboptimal decoder like the linear programming decoder, the answer might be "yes": In this paper we prove that for certain interesting classes of low-density parity-check codes and large enough SNRs, it is advantageous to truncate the log-likelihood ratios before passing them to the linear programming decoder., Comment: To appear in Proc. 2005 IEEE International Symposium on Information Theory, Adelaide, Australia, September 4-9, 2005

In order to understand the performance of a code under maximum-likelihood (ML) decoding, it is crucial to know the minimal codewords. In the context of linear programming (LP) decoding, it turns out to be necessary to know the minimal pseudo-codewords. This paper studies the minimal codewords and minimal pseudo-codewords of some families of codes derived from projective and Euclidean planes. Although our numerical results are only for codes of very modest length, they suggest that these code families exhibit an interesting property. Namely, all minimal pseudo-codewords that are not multiples of a minimal codeword have an AWGNC pseudo-weight that is strictly larger than the minimum Hamming weight of the code. This observation has positive consequences not only for LP decoding but also for iterative decoding., Comment: To appear in Proc. 2005 IEEE International Symposium on Information Theory, Adelaide, Australia, September 4-9, 2005

Cycle codes are a special case of low-density parity-check (LDPC) codes and as such can be decoded using an iterative message-passing decoding algorithm on the associated Tanner graph. The existence of pseudo-codewords is known to cause the decoding algorithm to fail in certain instances. In this paper, we draw a connection between pseudo-codewords of cycle codes and the so-called edge zeta function of the associated normal graph and show how the Newton polyhedron of the zeta function equals the fundamental cone of the code, which plays a crucial role in characterizing the performance of iterative decoding algorithms., Comment: Presented at Information Theory Workshop (ITW), San Antonio, TX, 2004