Search

This paper develops a communication-efficient algorithm to solve the stochastic optimization problem defined over a distributed network, aiming at reducing the burdensome communication in applications such as distributed machine learning.Different from the existing works based on quantization and sparsification, we introduce a communication-censoring technique to reduce the transmissions of variables, which leads to our communication-Censored distributed Stochastic Gradient Descent (CSGD) algorithm. Specifically, in CSGD, the latest mini-batch stochastic gradient at a worker will be transmitted to the server if and only if it is sufficiently informative. When the latest gradient is not available, the stale one will be reused at the server. To implement this communication-censoring strategy, the batch-size is increasing in order to alleviate the effect of stochastic gradient noise. Theoretically, CSGD enjoys the same order of convergence rate as that of SGD, but effectively reduces communication. Numerical experiments demonstrate the sizable communication saving of CSGD.

In this paper we introduce an inductive method to study $\mathrm{OI}$-modules presented in finite degrees, where $\mathrm{OI}$ is a skeleton of the category of finitely totally ordered sets and strictly increasing maps. As an application, we obtain an explicit upper bound for the Castelnuovo-Mumford regularity of $\mathrm{OI}$-modules., Comment: Minor revisions added

Polar codes with memory (PCM) are proposed in this paper: a pair of consecutive code blocks containing a controlled number of mutual information bits. The shared mutual information bits of the succeeded block can help the failed block to recover. The underlying polar codes can employ any decoding scheme such as the successive cancellation (SC) decoding (PCM-SC), the belief propagation (BP) decoding (PCM-BP), and the successive cancellation list (SCL) decoding (PCM-SCL). The analysis shows that the packet error rate (PER) of PCM decreases to the order of PER squared while maintaining the same complexity as the underlying polar codes. Simulation results indicate that for PCM-SC, the PER is comparable to (less than 0.3 dB) the stand-alone SCL decoding with two lists for the block length $N=256$. The PER of PCM-SCL with $L$ lists can match that of the stand-alone SCL decoding with $2L$ lists. Two hardware decoders for PCM are also implemented: the in-serial (IS) decoder and the low-latency interleaved (LLI) decoder. For $N=256$, synthesis results show that in the worst case, the latency of the PCM LLI decoder is only $16.1\%$ of the adaptive SCL decoder with $L=2$, while the throughput is improved by 13 times compared to it., Comment: 12 pages, 14 figures, submitted to TC

In this paper, we are concerned with polymer models based on $\alpha$-stable processes, where $\alpha\in (\frac{d}{2},d\wedge 2)$ and $d$ stands for dimension. They are attached with a delta potential at the origin and the associated Gibbs measures are parametrized by a constant $\gamma$ playing the role of inverse temperature. Phase transition exhibits with critical value $\gamma_{cr}=0$. Our first object is to formulate the associated Dirichlet form of the canonical Markov process $X^{(\gamma)}$ induced by the Gibbs measure for a globular state $\gamma>0$ or the critical state $\gamma=0$. Approach of Dirichlet forms also leads to deeper descriptions of probabilistic counterparts of globular and critical states. Furthermore, we will characterize the behaviour of polymer near the critical point from probabilistic viewpoint by showing that $X^{(\gamma)}$ is convergent to $X^{(0)}$ as $\gamma\downarrow 0$ in a certain meaning.

We show in this paper that representations of a finite product of categories satisfying certain combinatorial conditions have finite Castelnuovo-Mumford regularity if and only if they are presented in finite degrees, and hence the category consisting of them is abelian. These results apply to examples such as the categories $\mathrm{FI}^m$ and $\mathrm{FI}_G^m$., Comment: A small change to extend the main result

For a given polar code construction, the existing literature on puncturing for polar codes focuses in finding the optimal puncturing pattern, and then re-selecting the information set. This paper devotes itself to find the optimal puncturing pattern when the information set is fixed. Puncturing the coded bits corresponding to the worst quality bit channels, called the worst quality puncturing (WQP), is proposed, which is analyzed to minimize the bit channel quality loss at the punctured positions. Simulation results show that WQP outperforms the best existing puncturing schemes when the information set is fixed., Comment: Polar codes, puncture, quasi-uniform puncturing,worst quality puncturing

Similar to existing codes, puncturing and shortening are two general ways to obtain an arbitrary code length and code rate for polar codes. When some of the coded bits are punctured or shortened, it is equivalent to a situation in which the underlying channels of the polar codes are different. Therefore, the quality of bit channels with puncturing or shortening differ from the original qualities, which can greatly affect the construction of polar codes. In this paper, a general construction of polar codes is studied in two aspects: 1) the theoretical foundation of the construction; and 2) the hardware implementation of polar codes encoders. In contrast to the original identical and independent binary-input, memoryless, symmetric (BMS) channels, these underlying BMS channels can be different. For binary erasure channel (BEC) channels, recursive equations can be employed assuming independent BMS channels. For all other channel types, the proposed general construction of polar codes is based on the existing Tal-Vardy's procedure. The symmetric property and the degradation relationship are shown to be preserved under the general setting, rendering the possibility of a modification of Tal-Vardy's procedure. Simulation results clearly show improved error performance with re-ordering using the proposed new procedures. In terms of hardware, a novel pruned folded encoder architecture is proposed which saves the computation for the beginning frozen bits. Implementation results show the pruned encoder achieve 28% throughput improvement., Comment: Polar Codes,General Construction,Puncture,Shorten