Search

This article introduces a novel non-orthogonal multiple access (NOMA) scheme for coordinated uplink channels. The scheme builds on the recently proposed sparse-regression low-density parity-check (SR-LDPC) code, and extends the underlying notions to scenarios with many concurrent users. The resulting scheme, called Multi-User SR-LDPC (MU-SR-LDPC) coding, consists of each user transmitting its own SR-LDPC codeword using a unique sensing matrix in conjunction with a characteristic outer LDPC code. To recover the sent information, the decoder jointly processes the received signals using a low-complexity and highly-parallelizable AMP-BP algorithm. At finite blocklengths (FBL), MU-SR-LDPC codes are shown to achieve a target BER at a higher spectral efficiency than baseline orthogonal multiple access (OMA) and non-orthogonal multiple access (NOMA) schemes with similar computational complexity. Furthermore, MU-SR-LDPC codes are shown to match the performance of maximum a posteriori (MAP) decoding in certain regimes. For certain blocklengths and a sufficiently high number of users, MU-SR-LDPC codes may achieve a higher spectral efficiency than the approximate FBL capacity of the effective single-user Gaussian channel seen by each user in a comparable OMA scheme. Results are supported by numerical simulations., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Novel sparse regression LDPC (SR-LDPC) codes exhibit excellent performance over additive white Gaussian noise (AWGN) channels in part due to their natural provision of shaping gains. Though SR-LDPC-like codes have been considered within the context of single-user error correction and massive random access, they are yet to be examined as candidates for coordinated multi-user communication scenarios. This article explores this gap in the literature and demonstrates that SR-LDPC codes, when combined with coded demixing techniques, offer a new framework for efficient non-orthogonal multiple access (NOMA) in the context of coordinated multi-user communication channels. The ensuing communication scheme is referred to as MU-SR-LDPC coding. Empirical evidence suggests that, for a fixed SNR, MU-SR-LDPC coding can achieve a target bit error rate (BER) at a higher sum rate than orthogonal multiple access (OMA) techniques such as time division multiple access (TDMA) and frequency division multiple access (FDMA). Importantly, MU-SR-LDPC codes enable a pragmatic solution path for user-centric cell-free communication systems with (local) joint decoding. Results are supported by numerical simulations., Comment: Submitted to ISIT 2024

This article introduces a novel concatenated coding scheme called sparse regression LDPC (SR-LDPC) codes. An SR-LDPC code consists of an outer non-binary LDPC code and an inner sparse regression code (SPARC) whose respective field size and section sizes are equal. For such codes, an efficient decoding algorithm is proposed based on approximate message passing (AMP) that dynamically shares soft information between inner and outer decoders. This dynamic exchange of information is facilitated by a denoiser that runs belief propagation (BP) on the factor graph of the outer LDPC code within each AMP iteration. It is shown that this denoiser falls within the class of non-separable pseudo-Lipschitz denoising functions and thus that state evolution holds for the proposed AMP-BP algorithm. Leveraging the rich structure of SR-LDPC codes, this article proposes an efficient low-dimensional approximate state evolution recursion that can be used for efficient hyperparameter tuning, thus paving the way for future work on optimal code design. Finally, numerical simulations demonstrate that SR-LDPC codes outperform contemporary codes over the AWGN channel for parameters of practical interest. SR-LDPC codes are shown to be viable means to obtain shaping gains over the AWGN channel., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. arXiv admin note: substantial text overlap with arXiv:2301.01899

Cell-Free Massive MIMO systems aim to expand the coverage area of wireless networks by replacing a single high-performance Access Point (AP) with multiple small, distributed APs connected to a Central Processing Unit (CPU) through a fronthaul. Another novel wireless approach, known as the unsourced random access (URA) paradigm, enables a large number of devices to communicate concurrently on the uplink. We consider a quasi-static Rayleigh fading channel paired to a scalable cell-free system, wherein a small number of receive antennas in the distributed APs serve devices equipped with a single antenna each. The goal of the study is to extend previous URA results to more realistic channels by examining the performance of a scalable cell-free system. To achieve this goal, we construct a coding scheme that adapts the URA paradigm to various cell-free scenarios. Empirical evidence suggests that using a cell-free architecture can improve the performance of a URA system, especially when taking into account large-scale attenuation and fading., Comment: Changed the channel model

We explore a scheme that enables the training of a deep neural network in a Federated Learning configuration over an additive white Gaussian noise channel. The goal is to create a low complexity, linear compression strategy, called PolarAir, that reduces the size of the gradient at the user side to lower the number of channel uses needed to transmit it. The suggested approach belongs to the family of compressed sensing techniques, yet it constructs the sensing matrix and the recovery procedure using multiple access techniques. Simulations show that it can reduce the number of channel uses by ~30% when compared to conveying the gradient without compression. The main advantage of the proposed scheme over other schemes in the literature is its low time complexity. We also investigate the behavior of gradient updates and the performance of PolarAir throughout the training process to obtain insight on how best to construct this compression scheme based on compressed sensing.

Belief propagation applied to iterative decoding and sparse recovery through approximate message passing (AMP) are two research areas that have seen monumental progress in recent decades. Inspired by these advances, this article introduces sparse regression LDPC codes and their decoding. Sparse regression codes (SPARCs) are a class of error correcting codes that build on ideas from compressed sensing and can be decoded using AMP. In certain settings, SPARCs are known to achieve capacity; yet, their performance suffers at finite block lengths. Likewise, LDPC codes can be decoded efficiently using belief propagation and can also be capacity achieving. This article introduces a novel concatenated coding structure that combines an LDPC outer code with a SPARC-inspired inner code. Efficient decoding for such a code can be achieved using AMP with a denoiser that performs belief propagation on the factor graph of the outer LDPC code. The proposed framework exhibits performance improvements over SPARCs and standard LDPC codes for finite block lengths and results in a steep waterfall in error performance, a phenomenon not observed in uncoded SPARCs. Findings are supported by numerical results.

Unsourced random access (URA) has emerged as a candidate paradigm for massive machine-type communication (MTC) in next-generation wireless networks. While many excellent uplink schemes have been developed for URA, these schemes do not specify a mechanism for providing feedback regarding whether a user's message was successfully decoded. While this may be acceptable in some MTC scenarios, the lack of feedback is inadequate for applications that demand a high level of reliability. However, the problem of providing feedback to active users is complicated by the fact that the base station does not know the identities of the active users. In this paper, a novel downlink beamforming scheme called HashBeam is presented that enables the base station to provide feedback to the active users within URA, despite not knowing their identities. The key idea of this scheme is that the users' channels and hashes of their messages may be used as proxies for their true identities. The proposed scheme may be adapted to any number of antennas at the base station and it is shown that the required number of channel uses is linear in the number of users to acknowledge. The idea of using channel gains in conjunction with user hashes as discriminating attributes of active users is novel and expands the design space of URA schemes.

We introduce two generalizations to the paradigm of using Random Khatri-Rao Product (RKRP) codes for distributed matrix multiplication. We first introduce a class of codes called Sparse Random Khatri-Rao Product (SRKRP) codes which have sparse generator matrices. SRKRP codes result in lower encoding, computation and communication costs than RKRP codes when the input matrices are sparse, while they exhibit similar numerical stability to other state of the art schemes. We empirically study the relationship between the probability of the generator matrix (restricted to the set of non-stragglers) of a randomly chosen SRKRP code being rank deficient and various parameters of the coding scheme including the degree of sparsity of the generator matrix and the number of non-stragglers. Secondly, we show that if the master node can perform a very small number of matrix product computations in addition to the computations performed by the workers, the failure probability can be substantially improved.

Unsourced random access (URA) is a recently proposed multiple access paradigm tailored to the uplink channel of machine-type communication networks. By exploiting a strong connection between URA and compressed sensing, the massive multiple access problem may be cast as a compressed sensing (CS) problem, albeit one in exceedingly large dimensions. To efficiently handle the dimensionality of the problem, coded compressed sensing (CCS) has emerged as a pragmatic signal processing tool that, when applied to URA, offers good performance at low complexity. While CCS is effective at recovering a signal that is sparse with respect to a single basis, it is unable to jointly recover signals that are sparse with respect to separate bases. In this article, the CCS framework is extended to the demixing setting, yielding a novel technique called coded demixing. A generalized framework for coded demixing is presented and a low-complexity recovery algorithm based on approximate message passing (AMP) is developed. Coded demixing is applied to heterogeneous multi-class URA networks and traditional single-class networks. Its performance is analyzed and numerical simulations are presented to highlight the benefits of coded demixing., Comment: 1053-587X Copyright 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information

Group testing has recently attracted significant attention from the research community due to its applications in diagnostic virology. An instance of the group testing problem includes a ground set of individuals which includes a small subset of infected individuals. The group testing procedure consists of a number of tests, such that each test indicates whether or not a given subset of individuals includes one or more infected individuals. The goal of the group testing procedure is to identify the subset of infected individuals with the minimum number of tests. Motivated by practical scenarios, such as testing for viral diseases, this paper focuses on the following group testing settings: (i) the group testing procedure is noisy, i.e., the outcome of the group testing procedure can be flipped with a certain probability; (ii) there is a certain amount of side information on the distribution of the infected individuals available to the group testing algorithm. The paper makes the following contributions. First, we propose a probabilistic model, referred to as an interaction model, that captures the side information about the probability distribution of the infected individuals. Next, we present a decoding scheme, based on the belief propagation, that leverages the interaction model to improve the decoding accuracy. Our results indicate that the proposed algorithm achieves higher success probability and lower false-negative and false-positive rates when compared to the traditional belief propagation especially in the high noise regime.

This article considers the massive MIMO unsourced random access problem on a quasi-static Rayleigh fading channel. Given a fixed message length and a prescribed number of channel uses, the objective is to construct a coding scheme that minimizes the energy-per-bit subject to a fixed probability of error. The proposed scheme differs from other state-of-the-art schemes in that it blends activity detection, single-user coding, pilot-aided and temporary decisions-aided iterative channel estimation and decoding, minimum-mean squared error (MMSE) estimation, and successive interference cancellation (SIC). We show that an appropriate combination of these ideas can substantially outperform state-of-the-art coding schemes when the number of active users is more than 100, making this the best performing scheme known for this regime.

Unsourced random access (URA) has emerged as a pragmatic framework for next-generation distributed sensor networks. Within URA, concatenated coding structures are often employed to ensure that the central base station can accurately recover the set of sent codewords during a given transmission period. Many URA algorithms employ independent inner and outer decoders, which can help reduce computational complexity at the expense of a decay in performance. In this article, an enhanced decoding algorithm is presented for a concatenated coding structure consisting of a wide range of inner codes and an outer tree-based code. It is shown that this algorithmic enhancement has the potential to simultaneously improve error performance and decrease the computational complexity of the decoder. This enhanced decoding algorithm is applied to two existing URA algorithms and the performance benefits of the algorithm are characterized. Findings are supported by numerical simulations., Comment: Submitted to MDPI Sensors

This paper considers the noisy group testing problem where among a large population of items some are defective. The goal is to identify all defective items by testing groups of items, with the minimum possible number of tests. The focus of this work is on the practical settings with a limited number of items rather than the asymptotic regime. In the current literature, belief propagation has been shown to be effective in recovering defective items from the test results. In this work, we adopt two variants of the belief propagation algorithm for the noisy group testing problem. These algorithms have been used successfully in the decoding of low-density parity-check codes. We perform an experimental study and using extensive simulations we show that these algorithms achieve higher success probability, lower false-negative, and false-positive rates compared to the traditional belief propagation algorithm. For instance, our results show that the proposed algorithms can reduce the false-negative rate by about $50\%$ (or more) when compared to the traditional BP algorithm, under the combinatorial model. Moreover, under the probabilistic model, this reduction in the false-negative rate increases to about $80\%$ for the tested cases.

This article presents a novel enhancement to the random spreading based coding scheme developed by Pradhan et al.\ for the unsourced multiple access channel. The original coding scheme features a polar outer code in conjunction with a successive cancellation list decoder (SCLD) and a hard-input soft-output MMSE estimator. In contrast, the proposed scheme employs a soft-input soft-output MMSE estimator for multi-user detection. This is accomplished by replacing the SCLD based polar code with an LDPC code amenable to belief propagation decoding. This novel framework is leveraged to successfully pass pertinent soft information between the MMSE estimator and the outer code. LDPC codes are carefully designed using density evolution techniques to match the iterative process. This enhanced architecture exhibits significant performance improvements and represents the state-of-the-art over a wide range of system parameters.

Unsourced random access (URA) is an increasingly popular communication paradigm attuned to machine driven data transfers in \textit{Internet-of-Things} (IoT) networks. In a typical URA setting, a small subset of active devices within a very large population wish to transmit short messages to a central base station. Originally defined for base stations equipped with a single antenna, the URA paradigm has recently been extended to practical scenarios involving base stations with a massive number of antennas by Fengler et al. The proposed concatenated coding scheme therein utilizes a non-Bayesian sparse recovery algorithm coupled with the tree code introduced by Amalladinne et al. in the context of coded compressed sensing. Currently, the existing MIMO implementation uses these two components in isolation. This article introduces an enhanced successive cancellation list decoding style scheme that facilitates dynamic interactions between the sparse recovery algorithm and the tree decoder. This modification can reduce the search space of the AD algorithm drastically; and it results in significant improvements both in terms of error performance and computational complexity. Simulation results reveal that, for a system with 100 active users, the proposed decoder reduces the number of required antennas at the base station by 23\% to achieve a performance akin to the scheme by Fengler et al., Comment: Submitted to Asilomar 2021

Unsourced random access is a novel communication paradigm designed for handling a large number of uncoordinated users that sporadically transmit very short messages. Under this model, coded compressed sensing (CCS) has emerged as a low-complexity scheme that exhibits good error performance. Yet, one of the challenges faced by CCS pertains to disentangling a large number of codewords present on a single factor graph. To mitigate this issue, this article introduces a modified CCS scheme whereby active devices stochastically partition themselves into groups that utilize separate sampling matrices with low cross-coherence for message transmission. At the receiver, ideas from the field of compressed demixing are employed for support recovery, and separate factor graphs are created for message disambiguation in each cluster. This reduces the number of active users on a factor graph, which improves performance significantly in typical scenarios. Indeed, coded demixing reduces the probability of error as the number of groups increases, up to a point. Findings are supported with numerical simulations., Comment: Submitted to IEEE-SPAWC 2021

This article seeks to advance coded compressed sensing (CCS) as a practical scheme for unsourced random access. The original CCS algorithm features a concatenated structure where an inner code is tasked with support recovery, and an outer tree code conducts message disambiguation. Recently, a link between CCS and sparse regression codes was established, leading to the application of approximate message passing (AMP) to CCS. This connection was subsequently strengthened by integrating AMP and belief propagation on the outer code through a dynamic denoiser. Along these lines, this work shows how block diagonal sensing matrices akin to those used in traditional CCS, together with the aforementioned dynamic denoiser, form an effective means to get good performance at low-complexity. This novel architecture can be used to scale this scheme to dimensions that were previously impractical. Findings are supported by numerical simulations.

High-throughput molecular profiling technologies have produced high-dimensional multi-omics data, enabling systematic understanding of living systems at the genome scale. Studying molecular interactions across different data types helps reveal signal transduction mechanisms across different classes of molecules. In this paper, we develop a novel Bayesian representation learning method that infers the relational interactions across multi-omics data types. Our method, Bayesian Relational Learning (BayReL) for multi-omics data integration, takes advantage of a priori known relationships among the same class of molecules, modeled as a graph at each corresponding view, to learn view-specific latent variables as well as a multi-partite graph that encodes the interactions across views. Our experiments on several real-world datasets demonstrate enhanced performance of BayReL in inferring meaningful interactions compared to existing baselines.

Sparse regression codes with approximate message passing (AMP) decoding have gained much attention in recent times. The concepts underlying this coding scheme extend to unsourced random access with coded compressed sensing (CCS), as first demonstrated by Fengler, Jung, and Caire. Specifically, their approach employs a concatenated coding framework with an inner AMP decoder followed by an outer tree decoder. In their original implementation, these two components work independently of each other, with the tree decoder acting on the static output of the AMP decoder. This article introduces a novel framework where the inner AMP decoder and the outer tree decoder operate in tandem, dynamically passing information back and forth to take full advantage of the underlying CCS structure. This scheme necessitates the redesign of the tree code as to enable belief propagation in a computationally tractable manner. The enhanced architecture exhibits significant performance benefits over a range of system parameters. The error performance of the proposed scheme can be accurately predicted through a set of equations, known as state evolution of AMP. These findings are supported both analytically and through numerical methods.

We propose two-stage adaptive pooling schemes, 2-STAP and 2-STAMP, for detecting COVID-19 using real-time reverse transcription quantitative polymerase chain reaction (RT-qPCR) test kits. Similar to the Tapestry scheme of Ghosh et al., the proposed schemes leverage soft information from the RT-qPCR process about the total viral load in the pool. This is in contrast to conventional group testing schemes where the measurements are Boolean. The proposed schemes provide higher testing throughput than the popularly used Dorfman's scheme. They also provide higher testing throughput, sensitivity and specificity than the state-of-the-art non-adaptive Tapestry scheme. The number of pipetting operations is lower than state-of-the-art non-adaptive pooling schemes, and is higher than that for the Dorfman's scheme. The proposed schemes can work with substantially smaller group sizes than non-adaptive schemes and are simple to describe. Monte-Carlo simulations using the statistical model in the work of Ghosh et al. (Tapestry) show that 10 infected people in a population of size 961 can be identified with 70.86 tests on the average with a sensitivity of 99.50% and specificity of 99.62. This is 13.5x, 4.24x, and 1.3x the testing throughput of individual testing, Dorfman's testing, and the Tapestry scheme, respectively., Comment: 23 pages, 3 figures, 4 tables

COVID-19 has resulted in a global health crisis that may become even more acute over the upcoming months. One of the main reasons behind the current rapid growth of COVID-19 in the U.S. population is the limited availability of testing kits and the relatively-high cost of screening tests. In this draft, we demonstrate the effectiveness of group testing (pooling) ideas to accelerate testing for COVID-19. This draft is semi-tutorial in nature and is written for a broad audience with interest in mathematical formulations relevant to COVID-19 testing. Therefore, ideas are presented through illustrative examples rather than through purely theoretical formulations. The focus is also on pools of size less than 64 such as what is practical with current RT-PCR technology., Comment: 16 pages, 10 figures, 4 tables

Sparse regression codes with approximate message passing (AMP) decoding have gained much attention in recent times. The concepts underlying this coding scheme extend to unsourced access with coded compressed sensing (CCS), as first pointed out by Fengler, Jung, and Caire. More specifically, their approach uses a concatenated coding framework with an inner AMP decoder followed by an outer tree decoder. In the original implementation, these two components work independently of each other, with the tree decoder acting on the static output of the AMP decoder. This article introduces a novel framework where the inner AMP decoder and the outer tree decoder operate in tandem, dynamically passing information back and forth to take full advantage of the underlying CCS structure. The enhanced architecture exhibits significant performance benefit over a range of system parameters. Simulation results are provided to demonstrate the performance benefit offered by the proposed access scheme over existing schemes in the literature., Comment: Submitted o ISIT2020

This article presents a novel transmission scheme for the unsourced, uncoordinated Gaussian multiple access problem. The proposed scheme leverages notions from single-user coding, random spreading, minimum-mean squared error (MMSE) estimation, and successive interference cancellation. Specifically, every message is split into two parts: the first fragment serves as the argument to an injective function that determines which spreading sequence should be employed, whereas the second component of the message is encoded using a polar code. The latter coded bits are then spread using the sequence determined during the first step. The ensuing signal is transmitted through a Gaussian multiple-access channel (GMAC). On the receiver side, active sequences are detected using a correlation-based energy detector, thereby simultaneously recovering individual signature sequences and their generating information bits in the form of preimages of the sequence selection function. Using the set of detected active spreading sequences, an MMSE estimator is employed to produce log-likelihood ratios (LLRs) for the second part of the messages corresponding to these detected users. The LLRs associated with each detected user are then passed to a list decoder of the polar code, which performs single-user decoding to decode the second portion of the message. This decoding operation proceeds iteratively by subtracting the interference due to the successfully decoded messages from the received signal, and repeating the above steps on the residual signal. At this stage, the proposed algorithm outperforms alternate existing low-complexity schemes when the number of active uses is below 225.

Coded compressed sensing is an algorithmic framework tailored to sparse recovery in very large dimensional spaces. This framework is originally envisioned for the unsourced multiple access channel, a wireless paradigm attuned to machine-type communications. Coded compressed sensing uses a divide-and-conquer approach to break the sparse recovery task into sub-components whose dimensions are amenable to conventional compressed sensing solvers. The recovered fragments are then stitched together using a low complexity decoder. This article introduces an enhanced decoding algorithm for coded compressed sensing where fragment recovery and the stitching process are executed in tandem, passing information between them. This novel scheme leads to gains in performance and a significant reduction in computational complexity. This algorithmic opportunity stems from the realization that the parity structure inherent to coded compressed sensing can be used to dynamically restrict the search space of the subsequent recovery algorithm., Comment: Submitted to ICASSP2020

This paper considers the problem of Quantitative Group Testing (QGT) where there are some defective items among a large population of $N$ items. We consider the scenario in which each item is defective with probability $K/N$, independently from the other items. In the QGT problem, the goal is to identify all or a sufficiently large fraction of the defective items by testing groups of items, with the minimum possible number of tests. In particular, the outcome of each test is a non-negative integer which indicates the number of defective items in the tested group. In this work, we propose a non-adaptive QGT scheme for the underlying randomized model for defective items, which utilizes sparse graph codes over irregular bipartite graphs with optimized degree profiles on the left nodes of the graph as well as binary $t$-error-correcting BCH codes. We show that in the sub-linear regime, i.e., when the ratio $K/N$ vanishes as $N$ grows unbounded, the proposed scheme with ${m=c(t,d)K(t\log (\frac{\ell N}{c(t,d)K}+1)+1)}$ tests can identify all the defective items with probability approaching $1$, where $d$ and $\ell$ are the maximum and average left degree, respectively, and $c(t,d)$ depends only on $t$ and $d$ (and does not depend on $K$ and $N$). For any $t\leq 4$, the testing and recovery algorithms of the proposed scheme have the computational complexity of $\mathcal{O}(N\log \frac{N}{K})$ and $\mathcal{O}(K\log \frac{N}{K})$, respectively. The proposed scheme outperforms two recently proposed non-adaptive QGT schemes for the sub-linear regime, including our scheme based on regular bipartite graphs and the scheme of Gebhard et al., in terms of the number of tests required to identify all defective items with high probability., Comment: 7 pages; This work was presented at the 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton'19), Monticello, Illinois, USA, Sept 2019

Representation learning over graph structured data has been mostly studied in static graph settings while efforts for modeling dynamic graphs are still scant. In this paper, we develop a novel hierarchical variational model that introduces additional latent random variables to jointly model the hidden states of a graph recurrent neural network (GRNN) to capture both topology and node attribute changes in dynamic graphs. We argue that the use of high-level latent random variables in this variational GRNN (VGRNN) can better capture potential variability observed in dynamic graphs as well as the uncertainty of node latent representation. With semi-implicit variational inference developed for this new VGRNN architecture (SI-VGRNN), we show that flexible non-Gaussian latent representations can further help dynamic graph analytic tasks. Our experiments with multiple real-world dynamic graph datasets demonstrate that SI-VGRNN and VGRNN consistently outperform the existing baseline and state-of-the-art methods by a significant margin in dynamic link prediction., Comment: Accepted to Neural Information Processing Systems (NeurIPS2019)

Semi-implicit graph variational auto-encoder (SIG-VAE) is proposed to expand the flexibility of variational graph auto-encoders (VGAE) to model graph data. SIG-VAE employs a hierarchical variational framework to enable neighboring node sharing for better generative modeling of graph dependency structure, together with a Bernoulli-Poisson link decoder. Not only does this hierarchical construction provide a more flexible generative graph model to better capture real-world graph properties, but also does SIG-VAE naturally lead to semi-implicit hierarchical variational inference that allows faithful modeling of implicit posteriors of given graph data, which may exhibit heavy tails, multiple modes, skewness, and rich dependency structures. Compared to VGAE, the derived graph latent representations by SIG-VAE are more interpretable, due to more expressive generative model and more faithful inference enabled by the flexible semi-implicit construction. Extensive experiments with a variety of graph data show that SIG-VAE significantly outperforms state-of-the-art methods on several different graph analytic tasks., Comment: Accepted to Advances in Neural Information Processing Systems (NeurIPS 2019)

We propose two coding schemes for distributed matrix multiplication in the presence of stragglers. These coding schemes are adaptations of LT codes and Raptor codes to distributed matrix multiplication and are termed \emph{factored LT (FLT) codes} and \emph{factored Raptor (FR) codes}. Empirically, we show that FLT codes have near-optimal recovery thresholds when the number of worker nodes is very large, and that FR codes have excellent recovery thresholds while the number of worker nodes is moderately large. FLT and FR codes have better recovery thresholds when compared to Product codes and they are expected to have better numerical stability when compared to Polynomial codes, while they can also be decoded with a low-complexity decoding algorithm.

We propose a class of codes called random Khatri-Rao-Product (RKRP) codes for distributed matrix multiplication in the presence of stragglers. The main advantage of the proposed codes is that decoding of RKRP codes is highly numerically stable in comparison to decoding of Polynomial codes and decoding of the recently proposed OrthoPoly codes. We show that RKRP codes are maximum distance separable with probability 1. The communication cost and encoding complexity for RKRP codes are identical to that of OrthoPoly codes and Polynomial codes and the average decoding complexity of RKRP codes is lower than that of OrthoPoly codes. Numerical results show that the average relative $L_2$-norm of the reconstruction error for RKRP codes is substantially better than that of OrthoPoly codes., Comment: 16 pages, 4 figures, submitted to 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton 2019)

In this work, we leverage advances in sparse coding techniques to reduce the number of trainable parameters in a fully connected neural network. While most of the works in literature impose $\ell_1$ regularization, DropOut or DropConnect techniques to induce sparsity, our scheme considers feature importance as a criterion to allocate the trainable parameters (resources) efficiently in the network. Even though sparsity is ensured, $\ell_1$ regularization requires training on all the resources in a deep neural network. The DropOut/DropConnect techniques reduce the number of trainable parameters in the training stage by dropping a random collection of neurons/edges in the hidden layers. However, both these techniques do not pay heed to the underlying structure in the data when dropping the neurons/edges. Moreover, these frameworks require a storage space equivalent to the number of parameters in a fully connected neural network. We address the above issues with a more structured architecture inspired from spatially-coupled sparse constructions. The proposed architecture is shown to have a performance akin to a conventional fully connected neural network with dropouts, and yet achieving a $94\%$ reduction in the training parameters. Extensive simulations are presented and the performance of the proposed scheme is compared against traditional neural network architectures.

This article introduces a novel communication paradigm for the unsourced, uncoordinated Gaussian multiple access problem. The major components of the envisioned framework are as follows. The encoded bits of every message are partitioned into two groups. The first portion is transmitted using a compressive sensing scheme, whereas the second set of bits is conveyed using a multi-user coding scheme. The compressive sensing portion is key in sidestepping some of the challenges posed by the unsourced aspect of the problem. The information afforded by the compressive sensing is employed to create a sparse random multi-access graph conducive to joint decoding. This construction leverages the lessons learned from traditional IDMA into creating low-complexity schemes for the unsourced setting and its inherent randomness. Under joint message-passing decoding, the proposed scheme offers comparable performance to existing low-complexity alternatives. Findings are supported by numerical simulations.

We show that polynomial codes (and some related codes) used for distributed matrix multiplication are interleaved Reed-Solomon codes and, hence, can be collaboratively decoded. We consider a fault tolerant setup where $t$ worker nodes return erroneous values. For an additive random Gaussian error model, we show that for all $t < N-K$, errors can be corrected with probability 1. Further, numerical results show that in the presence of additive errors, when $L$ Reed-Solomon codes are collaboratively decoded, the numerical stability in recovering the error locator polynomial improves with increasing $L$.

This paper considers the problem of Quantitative Group Testing (QGT). Consider a set of $N$ items among which $K$ items are defective. The QGT problem is to identify (all or a sufficiently large fraction of) the defective items, where the result of a test reveals the number of defective items in the tested group. In this work, we propose a non-adaptive QGT algorithm using sparse graph codes over bi-regular bipartite graphs with left-degree $\ell$ and right degree $r$ and binary $t$-error-correcting BCH codes. The proposed scheme provides exact recovery with probabilistic guarantee, i.e. recovers all the defective items with high probability. In particular, we show that for the sub-linear regime where $\frac{K}{N}$ vanishes as $K,N\rightarrow\infty$, the proposed algorithm requires at most ${m=c(t)K\left(t\log_2\left(\frac{\ell N}{c(t)K}+1\right)+1\right)+1}$ tests to recover all the defective items with probability approaching one as ${K,N\rightarrow\infty}$, where $c(t)$ depends only on $t$. The results of our theoretical analysis reveal that the minimum number of required tests is achieved by $t=2$. The encoding and decoding of the proposed algorithm for any $t\leq 4$ have the computational complexity of $\mathcal{O}(K\log^2 \frac{N}{K})$ and $\mathcal{O}(K\log \frac{N}{K})$, respectively. Our simulation results also show that the proposed algorithm significantly outperforms a non-adaptive semi-quantitative group testing algorithm recently proposed by Abdalla \emph{et al.} in terms of the required number of tests for identifying all the defective items with high probability.

The neighbor discovery paradigm finds wide application in Internet of Things networks, where the number of active devices is orders of magnitude smaller than the total device population. Designing low-complexity schemes for asynchronous neighbor discovery has recently gained significant attention from the research community. Concurrently, a divide-and-conquer framework, referred to as coupled compressive sensing, has been introduced for the synchronous massive random access channel. This work adapts this novel algorithm to the problem of asynchronous neighbor discovery with unknown transmission delays. Simulation results suggest that the proposed scheme requires much fewer transmissions to achieve a performance level akin to that of state-of-the-art techniques.

This article introduces a novel communication scheme, termed coded compressed sensing, for unsourced multiple-access communication. The proposed divide-and-conquer approach leverages recent advances in compressed sensing and forward error correction to produce a novel uncoordinated access paradigm, along with a computationally efficient decoding algorithm. Within this framework, every active device partitions its data into several sub-blocks and, subsequently, adds redundancy using a systematic linear block code. Compressed sensing techniques are then employed to recover sub-blocks up to a permutation of their order, and the original messages are obtained by stitching fragments together using a tree-based algorithm. The error probability and computational complexity of this access paradigm are characterized. An optimization framework, which exploits the tradeoff between performance and computational complexity, is developed to assign parity-check bits to each sub-block. In addition, two emblematic parity bit allocation strategies are examined and their performances are analyzed in the limit as the number of active users and their corresponding payloads tend to infinity. The number of channel uses needed and the computational complexity associated with these allocation strategies are established for various scaling regimes. Numerical results demonstrate that coded compressed sensing outperforms other existing practical access strategies over a range of operational scenarios., Comment: Submitted to IEEE Transactions on Information Theory

Stationarity is a key assumption in many statistical models for random processes. With recent developments in the field of graph signal processing, the conventional notion of wide-sense stationarity has been extended to random processes defined on the vertices of graphs. It has been shown that well-known spectral graph kernel methods assume that the underlying random process over a graph is stationary. While many approaches have been proposed, both in machine learning and signal processing literature, to model stationary random processes over graphs, they are too restrictive to characterize real-world datasets as most of them are non-stationary processes. In this paper, to well-characterize a non-stationary process over graph, we propose a novel model and a computationally efficient algorithm that partitions a large graph into disjoint clusters such that the process is stationary on each of the clusters but independent across clusters. We evaluate our model for traffic prediction on a large-scale dataset of fine-grained highway travel times in the Dallas--Fort Worth area. The accuracy of our method is very close to the state-of-the-art graph based deep learning methods while the computational complexity of our model is substantially smaller.

We consider the problem of querying a string (or, a database) of length $N$ bits to determine all the locations where a substring (query) of length $M$ appears either exactly or is within a Hamming distance of $K$ from the query. We assume that sketches of the original signal can be computed off line and stored. Using the sparse Fourier transform computation based approach introduced by Pawar and Ramchandran, we show that all such matches can be determined with high probability in sub-linear time. Specifically, if the query length $M = O(N^\mu)$ and the number of matches $L=O(N^\lambda)$, we show that for $\lambda < 1-\mu$ all the matching positions can be determined with a probability that approaches 1 as $N \rightarrow \infty$ for $K \leq \frac{1}{6}M$. More importantly our scheme has a worst-case computational complexity that is only $O\left(\max\{N^{1-\mu}\log^2 N, N^{\mu+\lambda}\log N \}\right)$, which means we can recover all the matching positions in {\it sub-linear} time for $\lambda<1-\mu$. This is a substantial improvement over the best known computational complexity of $O\left(N^{1-0.359 \mu} \right)$ for recovering one matching position by Andoni {\em et al.} \cite{andoni2013shift}. Further, the number of Fourier transform coefficients that need to be computed, stored and accessed, i.e., the sketching complexity of this algorithm is only $O\left(N^{1-\mu}\log N\right)$. Several extensions of the main theme are also discussed.

We consider the problem of non-adaptive group testing of $N$ items out of which $K$ or less items are known to be defective. We propose a testing scheme based on left-and-right-regular sparse-graph codes and a simple iterative decoder. We show that for any arbitrarily small $\epsilon>0$ our scheme requires only $m=c_\epsilon K\log \frac{c_1N}{K}$ tests to recover $(1-\epsilon)$ fraction of the defective items with high probability (w.h.p) i.e., with probability approaching $1$ asymptotically in $N$ and $K$, where the value of constants $c_\epsilon$ and $\ell$ are a function of the desired error floor $\epsilon$ and constant $c_1=\frac{\ell}{c_\epsilon}$ (observed to be approximately equal to 1 for various values of $\epsilon$). More importantly the iterative decoding algorithm has a sub-linear computational complexity of $\mathcal{O}(K\log \frac{N}{K})$ which is known to be optimal. Also for $m=c_2 K\log K\log \frac{N}{K}$ tests our scheme recovers the \textit{whole} set of defective items w.h.p. These results are valid for both noiseless and noisy versions of the problem as long as the number of defective items scale sub-linearly with the total number of items, i.e., $K=o(N)$. The simulation results validate the theoretical results by showing a substantial improvement in the number of tests required when compared to the testing scheme based on left-regular sparse-graphs., Comment: Part of this work is submitted to IEEE International Symposium on Information Theory 2017

We exploit the redundancy of the language-based source to help polar decoding. By judging the validity of decoded words in the decoded sequence with the help of a dictionary, the polar list decoder constantly detects erroneous paths after every few bits are decoded. This path-pruning technique based on joint decoding has advantages over stand-alone polar list decoding in that most decoding errors in early stages are corrected. In order to facilitate the joint decoding, we first propose a construction of dynamic dictionary using a trie and show an efficient way to trace the dictionary during decoding. Then we propose a joint decoding scheme of polar codes taking into account both information from the channel and the source. The proposed scheme has the same decoding complexity as the list decoding of polar codes. A list-size adaptive joint decoding is further implemented to largely reduce the decoding complexity. We conclude by simulation that the joint decoding schemes outperform stand-alone polar codes with CRC-aided successive cancellation list decoding by over 0.6 dB., Comment: Single column, 20 pages, 8 figures, to be submitted to ISIT 2016

A novel construction of lattices is proposed. This construction can be thought of as a special class of Construction A from codes over finite rings that can be represented as the Cartesian product of $L$ linear codes over $\mathbb{F}_{p_1},\ldots,\mathbb{F}_{p_L}$, respectively, and hence is referred to as Construction $\pi_A$. The existence of a sequence of such lattices that is good for channel coding (i.e., Poltyrev-limit achieving) under multistage decoding is shown. A new family of multilevel nested lattice codes based on Construction $\pi_A$ lattices is proposed and its achievable rate for the additive white Gaussian channel is analyzed. A generalization named Construction $\pi_D$ is also investigated which subsumes Construction A with codes over prime fields, Construction D, and Construction $\pi_A$ as special cases., Comment: 26 pages, 11 figures. arXiv admin note: text overlap with arXiv:1401.2228

We consider the compute-and-forward paradigm with limited feedback. Without feedback, compute-and-forward is typically realized with lattice codes over the ring of integers, the ring of Gaussian integers, or the ring of Eisenstein integers, which are all principal ideal domains (PID). A novel scheme called adaptive compute-and-forward is proposed to exploit the limited feedback about the channel state by working with the best ring of imaginary quadratic integers. This is enabled by generalizing the famous Construction A from PID to other rings of imaginary quadratic integers which may not form PID and by showing such the construction can produce good lattices for coding in the sense of Poltyrev and for MSE quantization. Simulation results show that by adaptively choosing the best ring among the considered ones according to the limited feedback, the proposed adaptive compute-and-forward provides a better performance than that provided by the conventional compute-and-forward scheme which works over Gaussian or Eisenstein integers solely., Comment: 21 pages, 5 figures

In this paper, using compute-and-forward as an example, we provide an overview of constructions of lattices from codes that possess the right algebraic structures for harnessing interference. This includes Construction A, Construction D, and Construction $\pi_A$ (previously called product construction) recently proposed by the authors. We then discuss two generalizations where the first one is a general construction of lattices named Construction $\pi_D$ subsuming the above three constructions as special cases and the second one is to go beyond principal ideal domains and build lattices over algebraic integers.

In this paper, we consider the use of lattice codes over Eisenstein integers for implementing a compute-and-forward protocol in wireless networks when channel state information is not available at the transmitter. We extend the compute-and-forward paradigm of Nazer and Gastpar to decoding Eisenstein integer combinations of transmitted messages at relays by proving the existence of a sequence of pairs of nested lattices over Eisenstein integers in which the coarse lattice is good for covering and the fine lattice can achieve the Poltyrev limit. Using this result, we show that both the outage performance and error-correcting performance of nested lattice codebooks over Eisenstein integers surpasses lattice codebooks over integers considered by Nazer and Gastpar with no additional computational complexity., Comment: 27 pages, 6 figures, submitted to IEEE Trans. Inf. Theory

A novel construction of lattices is proposed. This construction can be thought of as Construction A with codes that can be represented as the Cartesian product of $L$ linear codes over $\mathbb{F}_{p_1},\ldots,\mathbb{F}_{p_L}$, respectively; hence, is referred to as the product construction. The existence of a sequence of such lattices that are good for quantization and Poltyrev-good under multistage decoding is shown. This family of lattices is then used to generate a sequence of nested lattice codes which allows one to achieve the same computation rate of Nazer and Gastpar for compute-and-forward under multistage decoding, which is referred to as lattice-based multistage compute-and-forward. Motivated by the proposed lattice codes, two families of signal constellations are then proposed for the separation-based compute-and-forward framework proposed by Tunali \textit{et al.} together with a multilevel coding/multistage decoding scheme tailored specifically for these constellations. This scheme is termed separation-based multistage compute-and-forward and is shown having a complexity of the channel coding dominated by the greatest common divisor of the constellation size (may not be a prime number) instead of the constellation size itself., Comment: 45 pages, 22 figures

Communication in the presence of bounded timing asynchronism which is known to the receiver but cannot be easily compensated is studied. Examples of such situations include point-to-point communication over inter-symbol interference (ISI) channels and asynchronous wireless networks. In these scenarios, although the receiver may know all the delays, it is often not be an easy task for the receiver to compensate the delays as the signals are mixed together. A novel framework called interleave/deinterleave transform (IDT) is proposed to deal with this problem. It is shown that the IDT allows one to design the delays so that quasi-cyclic (QC) codes with a proper shifting constraint can be used accordingly. When used in conjunction with QC codes, IDT provides significantly better performance than existing schemes relying solely on cyclic codes. Two instances of asynchronous physical-layer network coding, namely the integer-forcing equalization for ISI channels and asynchronous compute-and-forward, are then studied. For integer-forcing equalization, the proposed scheme provides improved performance over using cyclic codes. For asynchronous compute-and-forward, the proposed scheme shows that there is no loss in the achievable information due to delays which are integer multiples of the symbol duration. Further, the proposed approach shows that delays introduced by the channel can sometimes be exploited to obtain higher information rates than those obtainable in the synchronous case. The proposed IDT can be thought of as a generalization of the interleaving/deinterleaving idea proposed by Wang et al. which allows the use of QC codes thereby substantially increasing the design space., Comment: 21 pages, 15 figures

A variety of low-density parity-check (LDPC) ensembles have now been observed to approach capacity with message-passing decoding. However, all of them use soft (i.e., non-binary) messages and a posteriori probability (APP) decoding of their component codes. In this paper, we show that one can approach capacity at high rates using iterative hard-decision decoding (HDD) of generalized product codes. Specifically, a class of spatially-coupled GLDPC codes with BCH component codes is considered, and it is observed that, in the high-rate regime, they can approach capacity under the proposed iterative HDD. These codes can be seen as generalized product codes and are closely related to braided block codes. An iterative HDD algorithm is proposed that enables one to analyze the performance of these codes via density evolution (DE)., Comment: 22 pages, this version accepted to the IEEE Transactions on Information Theory

We consider a noisy Slepian-Wolf problem where two correlated sources are separately encoded (using codes of fixed rate) and transmitted over two independent binary memoryless symmetric channels. The capacity of each channel is characterized by a single parameter which is not known at the transmitter. The goal is to design systems that retain near-optimal performance without channel knowledge at the transmitter. It was conjectured that it may be hard to design codes that perform well for symmetric channel conditions. In this work, we present a provable capacity-achieving sequence of LDGM ensembles for the erasure Slepian-Wolf problem with symmetric channel conditions. We also introduce a staggered structure which enables codes optimized for single user channels to perform well for symmetric channel conditions. We provide a generic framework for analyzing the performance of joint iterative decoding, using density evolution. Using differential evolution, we design punctured systematic LDPC codes to maximize the region of achievable channel conditions. The resulting codes are then staggered to further increase the region of achievable parameters. The main contribution of this paper is to demonstrate that properly designed irregular LDPC codes can perform well simultaneously over a wide range of channel parameters., Comment: Submitted to IEEE Transactions on Communications. arXiv admin note: substantial text overlap with arXiv:1007.0931

We consider transmission of two independent and separately encoded sources over a two-user binary-input Gaussian multiple-access channel. The channel gains are assumed to be unknown at the transmitter and the goal is to design an encoder-decoder pair that achieves reliable communication for all channel gains where this is theoretically possible. We call such a system \emph{universal} with respect to the channel gains. Kudekar et al. recently showed that terminated low-density parity-check convolutional codes (a.k.a. spatially-coupled low-density parity-check ensembles) have belief-propagation thresholds that approach their maximum a-posteriori thresholds. This was proven for binary erasure channels and shown empirically for binary memoryless symmetric channels. It was conjectured that the principle of spatial coupling is very general and the phenomenon of threshold saturation applies to a very broad class of graphical models. In this work, we derive an area theorem for the joint decoder and empirically show that threshold saturation occurs for this problem. As a result, we demonstrate near-universal performance for this problem using the proposed spatially-coupled coding system., Comment: 8 pages, to appear in proceedings of Allerton 2011

Recently, it has been observed that terminated low-density-parity-check (LDPC) convolutional codes (or spatially-coupled codes) appear to approach capacity universally across the class of binary memoryless channels. This is facilitated by the "threshold saturation" effect whereby the belief-propagation (BP) threshold of the spatially-coupled ensemble is boosted to the maximum a-posteriori (MAP) threshold of the underlying constituent ensemble. In this paper, we consider the universality of spatially-coupled codes over intersymbol-interference (ISI) channels under joint iterative decoding. More specifically, we empirically show that threshold saturation also occurs for the considered problem. This can be observed by first identifying the EXIT curve for erasure noise and the GEXIT curve for general noise that naturally obey the general area theorem. From these curves, the corresponding MAP and the BP thresholds are then numerically obtained. With the fact that regular LDPC codes can achieve the symmetric information rate (SIR) under MAP decoding, spatially-coupled codes with joint iterative decoding can universally approach the SIR of ISI channels. For the dicode erasure channel, Kudekar and Kasai recently reported very similar results based on EXIT-like curves., Comment: 30 pages, 10 figures