Search

This paper presents a unifying framework to construct low-density parity-check (LDPC) codes with associated Tanner graphs of desired girth. Towards this goal, we highlight the role that a certain square matrix that appears in the product of the parity-check matrix with its transpose has in the construction of codes with graphs of desired girth and further explore it in order to generate the set of necessary and sufficient conditions for a Tanner graph to have a given girth between 6 and 12. For each such girth, we present algorithms to construct codes of the desired girth and we show how to use them to compute the minimum necessary value of the lifting factor. For girth larger than 12, we show how to use multi-step graph lifting methods to deterministically modify codes in order to increase their girth. We also give a new perspective on LDPC protograph-based parity-check matrices by viewing them as rows of a parity-check matrix equal to the sum of certain permutation matrices and obtain an important connection between all protographs and those with variable nodes of degree 2. We also show that the results and methodology that we develop for the all-one protograph can be used and adapted to analyze the girth of the Tanner graph of any parity-check matrix and demonstrate how this can be done using a well-known irregular, multi-edge protograph specified by the NASA Consultative Committee for Space Data Systems (CCSDS). Throughout the paper, we exemplify our theoretical results with constructions of LDPC codes with Tanner graphs of any girth between 6 and 14 and give sufficient conditions for a multi-step lifted parity-check matrix to have girth between 14 and 22. [ABSTRACT FROM AUTHOR]

This paper describes new, simple, recursive methods of construction for orientable sequences, i.e. periodic binary sequences in which any $n$ -tuple occurs at most once in a period in either direction. As has been previously described, such sequences have potential applications in automatic position-location systems, where the sequence is encoded onto a surface and a reader needs only examine $n$ consecutive encoded bits to determine its location and orientation on the surface. The only previously described method of construction (due to Dai et al.) is somewhat complex, whereas the new techniques are simple to both describe and implement. The methods of construction cover both the standard ‘infinite periodic’ case, and also the aperiodic, finite sequence, case. Both the new methods build on the Lempel homomorphism, first introduced as a means of recursively generating de Bruijn sequences. [ABSTRACT FROM AUTHOR]

In this paper, we introduce the concept of biprojectivity in studying the cryptographically important almost perfect nonlinear (APN) functions. Although several known families of biprojective APN functions exist in the literature, the concept itself has not been explicitly observed and studied in detail. We give a survey of known APN families and functions that fall into the biprojective setting, then provide a method for finding such families and functions. We finally give two new infinite families of biprojective APN functions ${\mathsf {CCZ}}$ -inequivalent to any known APN function and study their properties. [ABSTRACT FROM AUTHOR]

Previous algorithms for decoding AG codes up to the designed distance all assumed existence of an extra rational place on the base algebraic curve. The place is used to solve the decoding problem by linear algebra over the base field of the curve. The rationality of the place is essential, and therefore AG codes supported by all rational places on the curve are excluded from the domain of applicability of the decoding algorithms. This paper presents a decoding algorithm for those AG codes using an extra place of higher degree. Hence finally all AG codes, as Goppa defined 40 years ago, are equipped with a fast decoding algorithm. [ABSTRACT FROM AUTHOR]

The researches of Golay complementary sequences (GCSs) over 16 and 64 quadrature amplitude modulation (QAM) from 2001 to 2008 were generalized to $4^{q} $ -QAM GCSs of length $2^{m}$ by Li (the generalized cases I-III for $q\ge 2$) in 2010 and Liu et al. (the generalized cases IV-V for $q\ge 3$) in 2013. Those sequences are presented by the combination of the quaternary standard GCSs and compatible offsets. By providing new compatible offsets based on the factorization of the integer $q$ , we propose two new constructions of $4^{q} $ -QAM GCSs, which have the generalized cases I-V as special cases. The numbers of the proposed GCSs are equal to the product of the number of the quaternary standard GCSs and the number of the compatible offsets. Denote the number of prime factors of $q$ counted with multiplicity by $\Omega (q)$. The number of new offsets in our first construction is lower bounded by a polynomial of $m$ with degree $\Omega (q)$ , while the numbers of offsets in the generalized cases I-III and IV-V are a linear polynomial and a quadratic polynomial of $m$ , respectively. If $q$ has a prime factor larger than 2, the number of new offsets in our second construction is lower bounded by a polynomial of $m$ with degree $\Omega (q)+1$. As an example, the new offsets in our two constructions for $q=6$ , whose number is bounded by a cubic polynomial, is also given. The proof in this paper implies that all the mentioned GCSs over QAM can be regarded as projections of Golay complementary arrays of size $2\times 2\times \cdots \times 2$. [ABSTRACT FROM AUTHOR]

We consider the fundamental problem of matching a template to a signal. We do so by M-estimation, which encompasses procedures that are robust to gross errors (i.e., outliers). Using standard results from empirical process theory, we derive the convergence rate and the asymptotic distribution of the M-estimator under relatively mild assumptions. We also discuss the optimality of the estimator, both in finite samples in the minimax sense and in the large-sample limit in terms of local minimaxity and relative efficiency. Although most of the paper is dedicated to the study of the basic shift model in the context of a random design, we consider many extensions towards the end of the paper, including more flexible templates, fixed designs, the agnostic setting, and more. [ABSTRACT FROM AUTHOR]

This paper proposes a new algorithm, named Householder Dice (HD), for simulating dynamics on dense random matrix ensembles with rotational invariance. Examples include the Gaussian ensemble, the Haar-distributed random orthogonal ensemble, and their complex-valued counterparts. A “direct” approach to the simulation, where one first generates a dense $n \times n$ matrix from the ensemble, requires at least $\mathcal {O}(n^{2})$ resource in space and time. The HD algorithm overcomes this $\mathcal {O}(n^{2})$ bottleneck by using the principle of deferred decisions: rather than fixing the entire random matrix in advance, it lets the randomness unfold with the dynamics. At the heart of this matrix-free algorithm is an adaptive and recursive construction of (random) Householder reflectors. These orthogonal transformations exploit the group symmetry of the matrix ensembles, while simultaneously maintaining the statistical correlations induced by the dynamics. The memory and computation costs of the HD algorithm are $\mathcal {O}(nT)$ and $\mathcal {O}(nT^{2})$ , respectively, with $T$ being the number of iterations. When $T \ll n$ , which is nearly always the case in practice, the new algorithm leads to significant reductions in runtime and memory footprint. Numerical results demonstrate the promise of the HD algorithm as a new computational tool in the study of high-dimensional random systems. [ABSTRACT FROM AUTHOR]

The paper deals with linear information inequalities valid for entropy functions induced by discrete random variables. Specifically, the so-called conditional Ingleton inequalities are in the center of interest: these are valid under conditional independence assumptions on the inducing random variables. We discuss five inequalities of this particular type, four of which has appeared earlier in the literature. Besides the proof of the new fifth inequality, simpler proofs of (some of) former inequalities are presented. These five information inequalities are used to characterize all conditional independence structures induced by four discrete random variables. [ABSTRACT FROM AUTHOR]

We study in this paper lower bounds for the generalization error of models derived from multi-layer neural networks, in the regime where the size of the layers is commensurate with the number of samples in the training data. We derive explicit generalization lower bounds for general biased estimators, in the cases of two-layered networks. For linear activation function, the bound is asymptotically tight. In the nonlinear case, we provide a comparison of our bounds with an empirical study of the stochastic gradient descent algorithm. In addition, we derive bounds for unbiased estimators, which show that the latter have unacceptable performance for truly nonlinear networks. The analysis uses elements from the theory of large random matrices. [ABSTRACT FROM AUTHOR]

Let $q\geq 2$ be a prime power. In this paper, we study the distribution of the minimum distance (in the Hamming metric) of a random linear code of dimension $k$ in $\mathbb {F}_{q}^{n}$. We provide quantitative estimates showing that the distribution function of the minimum distance is close (superpolynomially in $n$) to the cumulative distribution function of the minimum of $(q^{k}-1)/(q-1)$ independent binomial random variables with parameters $\frac {1}{q}$ and $n$. The latter, in turn, converges to a Gumbel distribution at integer points when $\frac {k}{n}$ converges to a fixed number in (0, 1). Our result confirms in a strong sense that apart from identification of the weights of proportional codewords, the probabilistic dependencies introduced by the linear structure of the random code, produce a negligible effect on the minimum code weight. As a corollary of the main result, we obtain an improvement of the Gilbert–Varshamov bound for $2< q< 49$. [ABSTRACT FROM AUTHOR]

This paper studies the problem of recovering the hidden vertex correspondence between two edge-correlated random graphs. We focus on the Gaussian model where the two graphs are complete graphs with correlated Gaussian weights and the Erdős-Rényi model where the two graphs are subsampled from a common parent Erdős-Rényi graph ${\mathcal {G}}(n,p)$. For dense Erdős-Rényi graphs with $p=n^{-o(1)}$ , we prove that there exists a sharp threshold, above which one can correctly match all but a vanishing fraction of vertices and below which correctly matching any positive fraction is impossible, a phenomenon known as the “all-or-nothing” phase transition. Even more strikingly, in the Gaussian setting, above the threshold all vertices can be exactly matched with high probability. In contrast, for sparse Erdős-Rényi graphs with $p=n^{-\Theta (1)}$ , we show that the all-or-nothing phenomenon no longer holds and we determine the thresholds up to a constant factor. Along the way, we also derive the sharp threshold for exact recovery, sharpening the existing results in Erdős-Rényi graphs. The proof of the negative results builds upon a tight characterization of the mutual information based on the truncated second-moment computation and an “area theorem” that relates the mutual information to the integral of the reconstruction error. The positive results follows from a tight analysis of the maximum likelihood estimator that takes into account the cycle structure of the induced permutation on the edges. [ABSTRACT FROM AUTHOR]

Let $n$ be a positive integer and $p$ a prime. The power mapping $x^{p^{n}-3}$ over ${\mathbb {F}}_{p^{n}}$ has desirable differential properties, and its differential spectra for $p=2,\,3$ have been determined. In this paper, for any odd prime $p$ , by investigating certain quadratic character sums and some equations over ${\mathbb {F}}_{p^{n}}$ , we determine the differential spectrum of $x^{p^{n}-3}$ with a unified approach. The obtained result shows that for any given odd prime $p$ , the differential spectrum can be expressed explicitly in terms of $n$. Compared with previous results, a special elliptic curve over ${\mathbb {F}}_{p}$ plays an important role in our computation for the general case $p \ge 5$. [ABSTRACT FROM AUTHOR]

Sum-rank metric codes have recently attracted the attention of many researchers, due to their relevance in several applications. Mathematically, the sum-rank metric is a natural generalization of both the Hamming metric and the rank metric. In this paper, we provide an Anticode Bound for the sum-rank metric, which extends the corresponding Hamming and rank-metric Anticode bounds. We classify then optimal anticodes, i.e., codes attaining the sum-rank metric Anticode Bound. We use these optimal anticodes to define generalized sum-rank weights and we study their main properties. In particular, we prove that the generalized weights of an MSRD code are determined by its parameters. As an application, in the Appendix we explain how generalized weights measure information leakage in multishot network coding. [ABSTRACT FROM AUTHOR]

Reed-Muller codes are error-correcting codes used in many areas related to coding theory, such as electrical engineering and computer science. The binary $r^{th}$ -order Reed-Muller code $RM(r,n)$ can be viewed as the set of all $n$ -variable Boolean functions of algebraic degree at most $r$. Despite the intense work on these codes, many problems are known to be hard (notably, determining their covering radius) and remain open to this day. Fourteen years ago, Carlet and Mesnager improved in [IEEE Transactions on Information Theory, “Improving the Upper Bounds on the Covering Radii of Binary Reed-Muller Codes”, 53(1), 2007] the upper bound on the covering radius of the Reed-Muller code of order 2, and they deduced improved upper bounds on the covering radii of the Reed-Muller codes of higher orders. Until 2021, these upper bounds remained the best ones in the literature. The Reed-Muller code $RM(n-3,n)$ , which corresponds to the dual of the Reed-Muller code $RM(2,n)$ , has attracted much attention. One of the main reasons is that it is precisely the code that has been considered to get the upper bounds derived by Carlet and Mesnager. Those upper bounds have been obtained thanks to the characterization of the codewords of the Reed-Muller code, whose Hamming weights are strictly less than 2.5 times the minimum distance $2^{n-r}$ due to Kasami, Tokura, and Azumi. Despite their impressive work in the seventieth, a more refined study and profound description of those codewords of ${RM}({n-3},{n})$ whose Hamming weight equals 16, and especially 18, seem necessary, as it could help us significantly in improving the covering radius of Reed-Muller codes. In this paper, we push further the known results on the Reed-Muller codes by focusing on the Reed-Muller code ${RM}({n-3},{n})$. We provide a classification of the codewords of weights 16 and 18 of the Reed-Muller code $RM(n-3,n)$. Our algebraic descriptions allow us to count the number of such codewords and to enumerate all of them explicitly. [ABSTRACT FROM AUTHOR]

As a special subclass of cyclic codes, BCH codes have wide applications in communication and storage systems. A BCH code of length $n$ over $\mathbb {F}_{q}$ is always relative to an $n$ -th primitive root of unity $\beta $ in an extension field of $\mathbb {F}_{q}$ , and is called a dually-BCH code if its dual is also a BCH code relative to the same $\beta $. The question as to whether a BCH code is a dually-BCH code is in general very hard to answer. In this paper, an answer to this question for primitive narrow-sense BCH codes and projective narrow-sense ternary BCH codes is given. Sufficient and necessary conditions in terms of the designed distances $\delta $ will be presented to ensure that these BCH codes are dually-BCH codes. In addition, the parameters of the primitive narrow-sense BCH codes and their dual codes are investigated. Some lower bounds on minimum distances of the dual codes of primitive and projective narrow-sense BCH codes are developed. Especially for binary primitive narrow-sense BCH codes, the new bounds on the minimum distances of the dual codes improve the classical Sidel’nikov bound, and are also better than the Carlitz and Uchiyama bound for large designed distances $\delta $. The question as to what subclasses of cyclic codes are BCH codes is also answered to some extent. As a byproduct, the parameters of some subclasses of cyclic codes are also investigated. [ABSTRACT FROM AUTHOR]

For a continuous-time additive white Gaussian noise (AWGN) channel with possible feedback, it has been shown that as sampling gets infinitesimally fine, the mutual information of the associative discrete-time channels converges to that of the original continuous-time channel. We give in this paper more quantitative strengthenings of this result, which, among other implications, characterize how over-sampling approaches the true mutual information of a continuous-time Gaussian channel with bandwidth limit. The assumptions in our results are relatively mild. In particular, for the non-feedback case, compared to the Shannon-Nyquist sampling theorem, a widely used tool to connect continuous-time Gaussian channels to their discrete-time counterparts that requires the band-limitedness of the channel input, our results only require some integrability conditions on the power spectral density function of the input. [ABSTRACT FROM AUTHOR]

The Binary Iterative Hard Thresholding (BIHT) algorithm is a popular reconstruction method for one-bit compressed sensing due to its simplicity and fast empirical convergence. Despite considerable research on this algorithm, a theoretical understanding of the corresponding approximation error and convergence rate still remains an open problem. This paper shows that the normalized version of BIHT (NBIHT) achieves an approximation error rate optimal up to logarithmic factors. More precisely, using $m$ one-bit measurements of an $s$ -sparse vector $x$ , we prove that the approximation error of NBIHT is of order $O \left ({\frac{1 }{ m }}\right)$ up to logarithmic factors, which matches the information-theoretic lower bound $\Omega \left ({\frac{1 }{ m }}\right)$ proved by Jacques, Laska, Boufounos, and Baraniuk in 2013. To our knowledge, this is the first theoretical analysis of a BIHT-type algorithm that explains the optimal rate of error decay empirically observed in the literature. This also makes NBIHT the first provable computationally-efficient one-bit compressed sensing algorithm that breaks the inverse square-root error decay rate $O \left ({\frac{1 }{ m^{1/2} }}\right)\vphantom {{\left ({\frac{1 }{ m^{1/2} }}\right)}^{'}}$. [ABSTRACT FROM AUTHOR]

This paper proposes a new method of constructing compact fully-connected Quasi-Cyclic Low Density Parity Check (QC-LDPC) codes with girth $g$ = 8, 10, and 12. The originality of the proposed method is to impose constraints on the exponent matrix P to reduce the search space drastically. For a targeted lifting degree of $N$ , the first step of the method is to sieve the integer ring $\mathbb {Z}_{N}$ to make a particular sub-group with specific properties to construct the second column of P (the first column being filled with zeros). The remaining columns of P are determined recursively as multiples of the second column by adapting the sequentially multiplied column (SMC) method whereby a controlled greedy search is applied at each step. The codes constructed with the proposed semi-algebraic method show lengths that can be significantly shorter than their best counterparts in the literature. [ABSTRACT FROM AUTHOR]

This paper considers “ $\delta $ -almost Reed–Muller codes”, i.e., linear codes spanned by evaluations of all but a $\delta $ fraction of monomials of degree at most $d$. It is shown that for any $\delta > 0$ and any $\varepsilon >0$ , there exists a family of $\delta $ -almost Reed–Muller codes of constant rate that correct $1/2- \varepsilon $ fraction of random errors with high probability. For exact Reed–Muller codes, the analogous result is not known and represents a weaker version of the longstanding conjecture that Reed–Muller codes achieve capacity for random errors (Abbe-Shpilka-Wigderson STOC ’15). Our proof is based on the recent polarization result for Reed–Muller codes, combined with a combinatorial approach to establishing inequalities between the Reed–Muller code entropies. [ABSTRACT FROM AUTHOR]

A complete characterization for the restricted isometry constant (RIC) bounds on $\delta _{{{ tk}}}$ for all $ {t}>0$ is an important problem on recovery of sparse signals with prior support information via weighted $\ell _{{p}}$ -minimization ($0 < {p} \leqslant 1$). In this paper, new bounds on the restricted isometry constants $\delta _{{{ tk}}}$ ($0 < {t} < \frac {4}{3}{d}$), where $d$ is a key constant determined by prior support information, are established to guarantee the sparse signal recovery via the weighted $\ell _{{p}}$ minimization in both noiseless and noisy settings. This result fills a vacancy on $\delta _{{{ tk}}}$ with $0 < {t} < \frac {4}{3}{d}$ , compared with previous works on $\delta _{{{ tk}}}$ (${t} \geqslant \frac {4}3{d}$). We show that, when the accuracy of prior support estimate is at least 50%, the new recovery condition in terms of $\delta _{{{ tk}}}$ ($0 < {t} < \frac {4}{3}{d}$) via weighted $\ell _{1}$ minimization is weaker than the condition required by classical $\ell _{1}$ minimization without weighting. Our weighted $\ell _{1}$ minimization gives better recovery error bounds in noisy setting. Similarly, the new recovery condition in terms of $\delta _{{{ tk}}}$ ($0 < {t} < \frac {4}{3}{d}$) is extended to weighted $\ell _{{p}}$ ($0 < {p} < 1$) minimization, and it is also weaker than the condition obtained by standard non-convex $\ell _{{p}}$ ($0 < {p} < 1$) minimization without weighting. Numerical illustrations are provided to demonstrate our new theoretical results. [ABSTRACT FROM AUTHOR]

The $r$ -parallel set of a set $A \subseteq {\mathbb {R}} ^{d}$ is the set of all points whose distance from $A$ is less than $r$. In this paper, we show that the surface area of an $r$ -parallel set in ${\mathbb {R}}^{d}$ with volume at most $V$ is upper-bounded by $e^{\Theta (d)}V/r$ , whereas its Gaussian surface area is upper-bounded by $\max (e^{\Theta (d)}, e^{\Theta (d)}/r)$. We also derive a reverse form of the Brunn-Minkowski inequality for $r$ -parallel sets, and as an aside a reverse entropy power inequality for Gaussian-smoothed random variables. We apply our results to two problems in theoretical machine learning: (1) bounding the computational complexity of learning $r$ -parallel sets under a Gaussian distribution; and (2) bounding the sample complexity of estimating robust risk, which is a notion of risk in the adversarial machine learning literature that is analogous to the Bayes risk in hypothesis testing. [ABSTRACT FROM AUTHOR]

In a prior paper (Ellingsen et al., 2020), two of us, along with P. Ellingsen, P. Felke, and A. Tkachenko, defined a new (output) multiplicative differential and the corresponding $c$ -differential uniformity, which has the potential of extending differential cryptanalysis. Here, we continue the work by looking at some APN functions through the mentioned concept and showing that their $c$ -differential uniformity increases significantly in some cases. [ABSTRACT FROM AUTHOR]

We present an improved algorithm for quasi-properly learning convex polyhedra in the realizable PAC setting from data with a margin. Our learning algorithm constructs a consistent polyhedron as an intersection of about $t \log t$ halfspaces with constant-size margins in time polynomial in $t$ (where $t$ is the number of halfspaces forming an optimal polyhedron). We also identify distinct generalizations of the notion of margin from hyperplanes to polyhedra and investigate how they relate geometrically; this result may have ramifications beyond the learning setting. [ABSTRACT FROM AUTHOR]

We consider the high-dimensional linear regression problem, where the algorithmic goal is to efficiently infer an unknown feature vector $\beta ^{*}\in \mathbb {R}^{p}$ from its linear measurements, using a small number $n$ of samples. Unlike most of the literature, we make no sparsity assumption on $\beta ^{*}$ , but instead adopt a different regularization: In the noiseless setting, we assume $\beta ^{*}$ consists of entries, which are either rational numbers with a common denominator $Q\in \mathbb {Z}^{+}$ (referred to as $Q-$ rationality); or irrational numbers taking values in a rationally independent set of bounded cardinality, known to learner; collectively called as the mixed-range assumption. Using a novel combination of the Partial Sum of Least Squares (PSLQ) integer relation detection, and the Lenstra-Lenstra-Lovász (LLL) lattice basis reduction algorithms, we propose a polynomial-time algorithm which provably recovers a $\beta ^{*}\in \mathbb {R}^{p}$ enjoying the mixed-range assumption, from its linear measurements $Y=X\beta ^{*}\in \mathbb {R}^{n}$ for a large class of distributions for the random entries of $X$ , even with one measurement ($n=1$). In the noisy setting, we propose a polynomial-time, lattice-based algorithm, which recovers a $\beta ^{*}\in \mathbb {R}^{p}$ enjoying the $Q-$ rationality property, from its noisy measurements $Y=X\beta ^{*}+W\in \mathbb {R}^{n}$ , even from a single sample ($n=1$). We further establish that for large $Q$ , and normal noise, this algorithm tolerates information-theoretically optimal level of noise. We then apply these ideas to develop a polynomial-time, single-sample algorithm for the phase retrieval problem. Our methods address the single-sample ($n=1$) regime, where the sparsity-based methods such as the Least Absolute Shrinkage and Selection Operator (LASSO) and the Basis Pursuit are known to fail. Furthermore, our results also reveal algorithmic connections between the high-dimensional linear regression problem, and the integer relation detection, randomized subset-sum, and shortest vector problems. [ABSTRACT FROM AUTHOR]

In this manuscript we analyse generalised port-based teleportation (PBT) schemes, allowing for transmitting more than one unknown quantum state (or a composite quantum state) in one go, where the state ends up in several ports at Bob’s side. We investigate the efficiency of our scheme discussing both deterministic and probabilistic case, where parties share maximally entangled states. It turns out that the new scheme gives better performance than various variants of the optimal PBT protocol used for the same task. All the results are presented in group-theoretic manner depending on such quantities like dimensions and multiplicities of irreducible representations in the Schur-Weyl duality. The presented analysis was possible by considering the algebra of permutation operators acting on $n$ systems distorted by the action of partial transposition acting on more than one subsystem. Considering its action on the $n-$ fold tensor product of the Hilbert space with finite dimension, we present construction of the respective irreducible matrix representations, which are in fact matrix irreducible representations of the Walled Brauer Algebra. I turns out that the introduced formalism, and symmetries beneath it, appears in many aspects of theoretical physics and mathematics - theory of anti ferromagnetism, aspects of gravity theory or in the problem of designing quantum circuits for special task like for example inverting an unknown unitary. [ABSTRACT FROM AUTHOR]

We prove that for $n$ sufficiently large, all perfect 3-hash linear codes in $\mathbb {F}_{3}^{n}$ must have dimension at most $\left ({\frac {1}{4}-\epsilon }\right)n$ for some absolute constant $\epsilon > 0$. [ABSTRACT FROM AUTHOR]

Deep generative models have emerged as a powerful class of priors for signals in various inverse problems such as compressed sensing, phase retrieval and super-resolution. In this work, we consider the compressed sensing problem and assume the unknown signal to lie in the range of some pre-trained generative model. A popular approach for signal recovery is via gradient descent in the low-dimensional latent space. While gradient descent has achieved good empirical performance, its theoretical behavior is not well understood. We introduce the use of stochastic gradient Langevin dynamics (SGLD) for compressed sensing with a generative prior. Under mild assumptions on the generative model, we prove the convergence of SGLD to the true signal. We also demonstrate competitive empirical performance to standard gradient descent. [ABSTRACT FROM AUTHOR]

Given a probability measure $\mu $ over $\mathbb {R}^{n}$ , it is often useful to approximate it by the convex combination of a small number of probability measures, such that each component is close to a product measure. Recently, Ronen Eldan used a stochastic localization argument to prove a general decomposition result of this type. In Eldan’s theorem, the ‘number of components’ is characterized by the entropy of the mixture, and ‘closeness to product’ is characterized by the covariance matrix of each component. We present an elementary proof of Eldan’s theorem which makes use of an information theory (or estimation theory) interpretation. The proof is analogous to the one of an earlier decomposition result known as the ‘pinning lemma.’ [ABSTRACT FROM AUTHOR]

In this note we introduce a new family of entropy powers which are related to generalized entropies, called Sharma-Mittal entropies, and we prove their concavity along diffusion processes generated by $L^{2}$ -Wasserstein gradient flows of corresponding entropy functionals. This result extends the result of Savaré and Toscani (2014) on the concavity of Rényi entropy powers and reveals a connection to Rényi entropy power inequalities by Bobkov and Marsiglietti (2017). [ABSTRACT FROM AUTHOR]

Model complexity plays an essential role in its selection, namely, by choosing a model that fits the data and is also succinct. Two-part codes and the minimum description length have been successful in delivering procedures to single out the best models, avoiding overfitting. In this work, we pursue this approach and complement it by performing further assumptions in the parameter space. Concretely, we assume that the parameter space is a smooth manifold, and by using tools of Riemannian geometry, we derive a sharper expression than the standard one given by the stochastic complexity, where the scalar curvature of the Fisher information metric plays a dominant role. Furthermore, we compute a sharper approximation to the capacity for exponential families and apply our results to derive optimal dimensional reduction in the context of principal component analysis. [ABSTRACT FROM AUTHOR]

This work identifies information-theoretic quantities that are closely related to the required list size on average for successive cancellation list (SCL) decoding to implement maximum-likelihood decoding over general binary memoryless symmetric (BMS) channels. It also provides upper and lower bounds for these quantities that can be computed efficiently for very long codes. For the binary erasure channel (BEC), we provide a simple method to estimate the mean accurately via density evolution. The analysis shows how to modify, e.g., Reed-Muller codes, to improve the performance when practical list sizes, e.g., $L\in {[{8, 1024}]}$ , are adopted. Exemplary constructions with block lengths $N\in \{128,512\}$ outperform polar codes of 5G over the binary-input additive white Gaussian noise channel. It is further shown that there is a concentration around the mean of the logarithm of the required list size for sufficiently large block lengths, over discrete-output BMS channels. We provide the probability mass functions (p.m.f.s) of this logarithm, over the BEC, for a sequence of the modified RM codes with an increasing block length via simulations, which illustrate that the p.m.f.s concentrate around the estimated mean. [ABSTRACT FROM AUTHOR]

Many known networks have structure of affiliation networks, where each of $n$ network nodes (actors) selects an attribute set from a given collection of $m$ attributes, and two nodes (actors) establish adjacency relation whenever they share a common attribute. We study the behavior of a random walk on such networks. For this purpose, we use a common model of such networks, a random intersection graph. We establish the cover time of the simple random walk on the binomial random intersection graph ${\mathcal{ G}}(n,m,p)$ at the connectivity threshold and above it. We consider the range of $(n,m,p)$ , where the typical attribute is shared by a (stochastically) bounded number of actors. [ABSTRACT FROM AUTHOR]

A fundamental challenge in learning is the presence of nonlinear redundancies and dependencies in the data. To address this, we propose a Fourier-based approach to characterize feature redundancies, in unsupervised learning, and feature-label dependencies, in the supervised variant of the problem. We first develop a novel Fourier expansion for functions (more generally stochastic mappings) of correlated binary random variables. This is a generalization of the standard Fourier expansion on the Boolean cube beyond product probability spaces. As an important application of this analysis, we investigate learning with feature subset selection. In the unsupervised variant of this problem, we characterize feature redundancies via the Shannon entropy and group the features into sufficiently informative and redundant. Then, we make a connection to the proposed Fourier expansion and derive an upper bound on the joint entropy. Based on that, we propose a measure to quantify feature redundancies and present an unsupervised learning algorithm. We test our method on various real-world and synthetic datasets and demonstrate improvements on conventional unsupervised feature selection techniques. Then, we investigate the supervised feature subset selection and reformulate it in the Fourier domain. Bridging the Bayesian error rate with the Fourier coefficients, we demonstrate that the Fourier expansion provides a powerful tool to characterize nonlinear feature-label dependencies. Further, we introduce a computationally efficient measure for selecting relevant features. Via a theoretical analysis, we show that our proposed measure finds provably asymptotically optimal feature subsets. Lastly, we present an algorithm based on this measure and via numerical experiments demonstrate its improvements on various supervised feature selection algorithms. [ABSTRACT FROM AUTHOR]

Graph covers and the Bethe free energy (BFE) have been useful theoretical tools for producing lower bounds on a variety of counting problems in graphical models, including the permanent and the ferromagnetic Ising model. Here, we investigate weighted homomorphism counting problems over bipartite graphs that are related to a conjecture of Sidorenko. We show that the BFE does yield a lower bound in a variety of natural settings, and when it does yield a lower bound, it necessarily improves upon the lower bound conjectured by Sidorenko. Conversely, we show that there exist bipartite graphs for which the BFE does not yield a lower bound on the homomorphism number. Finally, we use the characterizations developed as part of this work to provide a simple proof of Sidorenko’s conjecture in a number of special cases. [ABSTRACT FROM AUTHOR]

We develop a new upper bound on the capacity of the relay channel that is tighter than previously known upper bounds. This upper bound is proved using traditional weak converse techniques involving mutual information inequalities and Gallager-type explicit identification of auxiliary random variables. We show that the new upper bound is strictly tighter than all previous bounds for the Gaussian relay channel with non-zero channel gains. When specialized to the relay channel with orthogonal receiver components, the bound resolves a conjecture by Kim on a class of deterministic relay channels. When further specialized to the class of product-form relay channels with orthogonal receiver components, the bound resolves a generalized version of Cover’s relay channel problem, recovers the recent upper bound for the Gaussian case by Wu et al., and improves upon the recent bounds for the binary symmetric case by Wu et al. and Barnes et al., which were obtained using non-traditional geometric proof techniques. For the special class of a relay channel with orthogonal receiver components, we develop another upper bound on the capacity which utilizes an auxiliary receiver and show that it is strictly tighter than the bound by Tandon and Ulukus. Finally, we show through the Gaussian relay channel with i.i.d. relay output sequence that the bound with the auxiliary receiver can be strictly tighter than our main bound. [ABSTRACT FROM AUTHOR]

Additive codes have become an increasingly important topic in algebraic coding theory due to their applications in quantum error-correction and quantum computing. Linear Complementary Dual (LCD) codes play an important role for improving the security of information against certain attacks. Motivated by these facts, we define additive complementary dual codes (ACD for short) over a finite abelian group in terms of an arbitrary duality on the ambient space and examine their properties. We show that the best minimum weight of ACD codes is always greater than or equal to the best minimum weight of LCD codes of the same size and that this inequality is often strict. We give some matrix constructions for quaternary ACD codes from a given quaternary ACD code and also from a given binary self-orthogonal code. Moreover, we construct an algorithm to determine if a given quaternary additive code is an ACD code with respect to all possible symmetric dualities. We also determine the largest minimum distance of quaternary ACD codes for lengths $n \leq 10$. The obtained codes are either optimal or near optimal according to Bierbrauer et al +. (2009). [ABSTRACT FROM AUTHOR]

We show that in pool-based active classification without assumptions on the underlying distribution, if the learner is given the power to abstain from some predictions by paying the price marginally smaller than the average loss 1/2 of a random guess, exponential savings in the number of label requests are possible whenever they are possible in the corresponding realizable problem. We extend this result to provide a necessary and sufficient condition for exponential savings in pool-based active classification under the model misspecification. [ABSTRACT FROM AUTHOR]

We study an information analogue of infinitely divisible probability distributions, where the i.i.d. sum is replaced by the joint distribution of an i.i.d. sequence. A random variable $X$ is called informationally infinitely divisible if, for any $n\ge 1$ , there exists an i.i.d. sequence of random variables $Z_{1},\ldots,Z_{n}$ that contains the same information as $X$ , i.e., there exists an injective function $f$ such that $X=f(Z_{1},\ldots,Z_{n})$. While there does not exist such informationally infinitely divisible discrete random variable, we show that any discrete random variable $X$ can be divided into arbitrarily many identical pieces with a multiplicative penalty to the entropy, that is, if we remove the injectivity requirement on $f$ , then there exists i.i.d. $Z_{1},\ldots,Z_{n}$ and $f$ satisfying $X=f(Z_{1},\ldots,Z_{n})$ , and the entropy satisfies $H(X)/n\le H(Z_{1})\le 1.59H(X)/n+2.43$ bits. Furthermore, we study the case where $X=(Y_{1},\ldots,Y_{m})$ is itself an i.i.d. sequence, $m\ge 2$ , for which the multiplicative gap 1.59 can be replaced by $1+5\sqrt {(\log m)/m}$. This means that as $m$ increases, $(Y_{1},\ldots,Y_{m})$ becomes closer to being spectral infinitely divisible in a uniform manner. This can be regarded as an information analogue of Kolmogorov’s uniform theorem. Applications of our result include independent component analysis and distributed storage with a secrecy constraint. [ABSTRACT FROM AUTHOR]

In the Gaussian sequence model $Y= \theta _{0} + \varepsilon $ in $\mathbb {R}^{n}$ , we study the fundamental limit of statistical estimation when the signal $\theta _{0}$ belongs to a class $\Theta _{n}(d,d_{0},k)$ of (generalized) splines with free knots located at equally spaced design points. Here $d$ is the degree of the spline, $d_{0}$ is the order of differentiability at each inner knot, and $k$ is the maximal number of pieces. We show that, given any integer $d\geq 0$ and $d_{0}\in \{-1,0,\ldots,d-1\}$ , the minimax rate of estimation over $\Theta _{n}(d,d_{0},k)$ exhibits the following phase transition: $\begin{aligned} \inf _{ \widetilde {\theta }}\sup _{\theta \in \Theta _{n}(d,d_{0}, k)} \mathbb {E}_\theta \lVert \widetilde {\theta } - \theta \rVert _{}^{2} \asymp _{d} \begin{cases} k\log \log (16n/k), & 2\leq k\leq k_{0},\\ k\log (en/k), & k \geq k_{0}+1. \end{cases} \end{aligned}$ The transition boundary $k_{0}$ , which takes the form $\left \lfloor{ (d+1)/(d-d_{0}) }\right \rfloor + 1$ , demonstrates the critical role of the regularity parameter $d_{0}$ in the separation between a faster $\log \log (16n)$ and a slower $\log (en)$ rate. We further show that, once encouraging an additional ‘ $d$ -monotonicity’ shape constraint (including monotonicity for $d = 0$ and convexity for $d=1$), the above phase transition is removed and the faster $k\log \log (16n/k)$ rate can be achieved for all $k$. These results provide theoretical support for developing $\ell _{0}$ -penalized (shape-constrained) spline regression procedures as useful alternatives to $\ell _{1}$ - and $\ell _{2}$ -penalized ones. [ABSTRACT FROM AUTHOR]

A new converse bound is presented for the two-user multiple-access channel under the average probability of error constraint. This bound shows that for most channels of interest, the second-order coding rate—that is, the difference between the best achievable rates and the asymptotic capacity region as a function of blocklength $n$ with fixed probability of error—is $O(1/\sqrt {n})$ bits per channel use. The principal tool behind this converse proof is a new measure of dependence between two random variables called wringing dependence, as it is inspired by Ahlswede’s wringing technique. The $O(1/\sqrt {n})$ gap is shown to hold for any channel satisfying certain regularity conditions, which includes all discrete-memoryless channels and the Gaussian multiple-access channel. Exact upper bounds as a function of the probability of error are proved for the coefficient in the $O(1/\sqrt {n})$ term, although for most channels they do not match existing achievable bounds. [ABSTRACT FROM AUTHOR]

The Hadamard Extension $\mathbb H({\mathrm {m}})$ of an $n \times k$ matrix m is the collection of all Hadamard products of subsets of its rows. This construction is essential for source identification (parameter estimation) of a mixture of $k$ product distributions over $n$ binary random variables. A necessary requirement for such identification is that $\mathbb H({\mathrm {m}})$ have full column rank; conversely, identification is possible if apart from each row there exist two disjoint sets of rows of m, each of whose extension has full column rank. It is necessary therefore to understand when $\mathbb H({\mathrm {m}})$ has full column rank; we provide two results in this direction. The first is that if $\mathbb H({\mathrm {m}})$ has full column rank then there exists a set of at most $k-1$ rows of m, whose extension already has full column rank. The second is a Hall-type condition on the values in the rows of m, that suffices to ensure full column rank of $\mathbb H({\mathrm {m}})$. [ABSTRACT FROM AUTHOR]

We propose novel randomized optimization methods for high-dimensional convex problems based on restrictions of variables to random subspaces. We consider oblivious and data-adaptive subspaces and study their approximation properties via convex duality and Fenchel conjugates. A suitable adaptive subspace can be generated by sampling a correlated random matrix whose second order statistics mirror the input data. We illustrate that the adaptive strategy can significantly outperform the standard oblivious sampling method, which is widely used in the recent literature. We show that the relative error of the randomized approximations can be tightly characterized in terms of the spectrum of the data matrix and Gaussian width of the dual tangent cone at optimum. We develop lower bounds for both optimization and statistical error measures based on concentration of measure and Fano’s inequality. We then present the consequences of our theory with data matrices of varying spectral decay profiles. Experimental results show that the proposed approach enables significant speed ups in a wide variety of machine learning and optimization problems including logistic regression, kernel classification with random convolution layers and shallow neural networks with rectified linear units. [ABSTRACT FROM AUTHOR]

We study the entanglement dynamics of quantum many-body systems and prove the following: (I) For any geometrically local Hamiltonian on a lattice, starting from a random product state the entanglement entropy is bounded away from the maximum entropy at all times with high probability. (II) In a spin-glass model with random all-to-all interactions, starting from any product state the average entanglement entropy is bounded away from the maximum entropy at all times. We also extend these results to any unitary evolution with charge conservation and to the Sachdev-Ye-Kitaev model. Our results highlight the difference between the entanglement generated by (chaotic) Hamiltonian dynamics and that of random states, for the latter is nearly maximal. [ABSTRACT FROM AUTHOR]

New asymptotic upper bounds are presented on the rate of sequences of locally repairable codes (LRCs) with a prescribed relative minimum distance and locality over a finite field $F$. The bounds apply to LRCs in which the recovery functions are linear; in particular, the bounds apply to linear LRCs over $F$. The new bounds are shown to improve on previously published results, especially when the repair groups are disjoint, namely, they form a partition of the set of coordinates. [ABSTRACT FROM AUTHOR]

In this work, we present linear exact repair schemes for one or two erasures in decreasing monomial-Cartesian codes (DM-CC), a family of codes which provides a framework for polar codes. In the case of two erasures, the positions of the erasures should satisfy a certain restriction. We present families of augmented Reed-Muller (ARM) and augmented Cartesian codes (ACar) which are families of evaluation codes obtained by strategically adding vectors to Reed-Muller and Cartesian codes, respectively. We develop repair schemes for one or two erasures for these families of augmented codes. Unlike the repair scheme for two erasures of DM-CC, the repair scheme for two erasures for the augmented codes has no restrictions on the positions of the erasures. When the dimension and base field are fixed, we give examples where ARM and ACar codes provide a lower bandwidth (resp., bitwidth) in comparison with Reed-Solomon (resp., Hermitian) codes. When the length and base field are fixed, we give examples where ACar codes provide a lower bandwidth in comparison with ARM. Finally, we analyze the asymptotic behavior when the augmented codes achieve the maximum rate. [ABSTRACT FROM AUTHOR]

An additive noise channel is considered, in which the distribution of the noise is nonparametric and unknown. The problem of learning encoders and decoders based on noise samples is considered. For uncoded communication systems, the problem of choosing a codebook and possibly also a generalized minimal distance decoder (which is parameterized by a covariance matrix) is addressed. High probability generalization bounds for the error probability loss function, as well as for a hinge-type surrogate loss function are provided. A stochastic-gradient based alternating-minimization algorithm for the latter loss function is proposed. In addition, a Gibbs-based algorithm that gradually expurgates an initial codebook from codewords in order to obtain a smaller codebook with improved error probability is proposed, and bounds on its average empirical error and generalization error, as well as a high probability generalization bound, are stated. Various experiments demonstrate the performance of the proposed algorithms. For coded systems, the problem of maximizing the mutual information between the input and the output with respect to the input distribution is addressed, and uniform convergence bounds for two different classes of input distributions are obtained. [ABSTRACT FROM AUTHOR]

Point processes in time have a wide range of applications that include the claims arrival process in insurance or the analysis of queues in operations research. Due to advances in technology, such samples of point processes are increasingly encountered. A key object of interest is the local intensity function. It has a straightforward interpretation that allows to understand and explore point process data. We consider functional approaches for point processes, where one has a sample of repeated realizations of the point process. This situation is inherently connected with Cox processes, where the intensity functions of the replications are modeled as random functions. Here we study a situation where one records covariates for each replication of the process, such as the daily temperature for bike rentals. For modeling point processes as responses with vector covariates as predictors we propose a novel regression approach for the intensity function that is intrinsically nonparametric. While the intensity function of a point process that is only observed once on a fixed domain cannot be identified, we show how covariates and repeated observations of the process can be utilized to make consistent estimation possible, and we also derive asymptotic rates of convergence without invoking parametric assumptions. [ABSTRACT FROM AUTHOR]

A family of error-correcting codes is list-decodable from error fraction $p$ if, for every code in the family, the number of codewords in any Hamming ball of fractional radius $p$ is less than some integer $L$. It is said to be list-recoverable for input list size $\ell $ if for every sufficiently large subset of at least $L$ codewords, there is a coordinate where the codewords take more than $\ell $ values. In this work, we study the list size of random linear codes for both list-decoding and list-recovery as the rate approaches capacity. We show the following claims hold with high probability over the choice of the code (below $q$ is the alphabet size, and $ \varepsilon > 0$ is the gap to capacity). (1) A random linear code of rate $1 - \log _{q}(\ell) - \varepsilon $ requires list size $L \ge \ell ^{\Omega (1/ \varepsilon)}$ for list-recovery from input list size $\ell $. (2) A random linear code of rate $1 - h_{q}(p) - \varepsilon $ requires list size $L \ge \left \lfloor{ {h_{q}(p)/ \varepsilon +0.99}}\right \rfloor $ for list-decoding from error fraction $p$. (3) A random binary linear code of rate $1 - h_{2}(p) - \varepsilon $ is list-decodable from average error fraction $p$ with list size with $L \leq \left \lfloor{ {h_{2}(p)/ \varepsilon }}\right \rfloor + 2$. Our lower bounds follow by exhibiting an explicit subset of codewords so that this subset—or some symbol-wise permutation of it—lies in a random linear code with high probability. Our upper bound follows by strengthening a result of (Li, Wootters, 2018). [ABSTRACT FROM AUTHOR]

We revisit the problem of the estimation of the differential entropy $H(f)$ of a random vector $X$ in $R^{d}$ with density $f$ , assuming that $H(f)$ exists and is finite. In this note, we study the consistency of the popular nearest neighbor estimate $H_{n}$ of Kozachenko and Leonenko. Without any smoothness condition we show that the estimate is consistent ($E\{|H_{n} - H(f)|\} \to 0$ as $n \to \infty $) if and only if ${\mathbb E}\{ \log (\| X \| + 1)\} < \infty $. Furthermore, if $X$ has compact support, then $H_{n} \to H(f)$ almost surely. [ABSTRACT FROM AUTHOR]

This note corrects a few errors in the proof of the main result of the article “Provable Low Rank Phase Retrieval.” The result itself has no change. This article introduced an alternating minimization solution, called AltMinLowRaP, for solving the Low Rank Phase Retrieval (LRPR) problem: recover an $n \times q$ matrix ${{X}^{*}}$ of rank $r$ from ${y}_{k}:= |{A}_{k}'{x}^{*}_{k}|, k=1,2, {\dots }, q$ when the measurement matrices $ {A}_{k}$ are mutually independent. Here $ {y}_{k}$ is an $m$ length vector, $ {A}_{k}$ is an $n \times m$ matrix, and $'$ denotes transpose. [ABSTRACT FROM AUTHOR]