Search

Reed-Solomon (RS) codes are widely used to correct errors in storage systems. Finding the error locator polynomial is one of the key steps in the error correction procedure of RS codes. Modular Approach (MA) is an effective algorithm for solving the Welch-Berlekamp (WB) key-equation problem to find the error locator polynomial that needs $2t$ steps, where $t$ is the error correction capability. In this paper, we first present a new MA algorithm that only requires $2e$ steps and then propose two fast decoding algorithms for RS codes based on our MA algorithm, where $e$ is the number of errors and $e\leq t$. We propose Improved-Frequency Domain Modular Approach (I-FDMA) algorithm that needs $2e$ steps to solve the error locator polynomial and present our first decoding algorithm based on the I-FDMA algorithm. We show that, compared with the existing methods based on MA algorithms, our I-FDMA algorithm can effectively reduce the decoding complexity of RS codes when $e

Nowadays, large language models (LLMs) have been integrated with conventional recommendation models to improve recommendation performance. However, while most of the existing works have focused on improving the model performance, the privacy issue has only received comparatively less attention. In this paper, we review recent advancements in privacy within LLM-based recommendation, categorizing them into privacy attacks and protection mechanisms. Additionally, we highlight several challenges and propose future directions for the community to address these critical problems.

Maximum distance separable (MDS) codes facilitate the achievement of elevated levels of fault tolerance in storage systems while incurring minimal redundancy overhead. Reed-Solomon (RS) codes are typical MDS codes with the sub-packetization level being one, however, they require large repair bandwidth defined as the total amount of symbols downloaded from other surviving nodes during single-node failure/repair. In this paper, we present the {\em set transformation}, which can transform any MDS code into set transformed code such that (i) the sub-packetization level is flexible and ranges from 2 to $(n-k)^{\lfloor\frac{n}{n-k}\rfloor}$ in which $n$ is the number of nodes and $k$ is the number of data nodes, (ii) the new code is MDS code, (iii) the new code has lower repair bandwidth for any single-node failure. We show that our set transformed codes have both lower repair bandwidth and lower field size than the existing related MDS array codes, such as elastic transformed codes \cite{10228984}. Specifically, our set transformed codes have $2\%-6.6\%$ repair bandwidth reduction compared with elastic transformed codes \cite{10228984} for the evaluated typical parameters.

Reed-Solomon (RS) codes are constructed over a finite field that have been widely employed in storage and communication systems. Many fast encoding/decoding algorithms such as fast Fourier transform (FFT) and modular approach are designed for RS codes to reduce the encoding/decoding complexity defined as the number of XORs involved in the encoding/decoding procedure. In this paper, we present the construction of RS codes over the cyclic polynomial ring $ \mathbb{F}_2[x]/(1+x+\ldots+x^{p-1})$ and show that our codes are maximum distance separable (MDS) codes. Moreover, we propose the FFT and modular approach over the ring that can be employed in our codes for encoding/decoding complexity reduction. We show that our codes have 17.9\% encoding complexity reduction and 7.5\% decoding complexity reduction compared with RS codes over finite field, for $(n,k)=(2048,1984)$.

Large language models (LLMs) have demonstrated outstanding performance across various tasks, yet they still exhibit limitations such as hallucination, unfaithful reasoning, and toxic content. One potential approach to mitigate these issues is learning from human or external feedback (e.g. tools). In this paper, we introduce an intrinsic self-correct reasoning framework for LLMs that eliminates the need for human feedback, external tools, and handcraft prompts. The proposed framework, based on a multi-step reasoning paradigm \textbf{Le}arning from \textbf{Co}rrectness (\textsc{LeCo}), improves reasoning performance without needing to learn from errors. This paradigm prioritizes learning from correct reasoning steps, and a unique method to measure confidence for each reasoning step based on generation logits. Experimental results across various multi-step reasoning tasks demonstrate the effectiveness of the framework in improving reasoning performance with reduced token consumption., Comment: Accepted to COLM 2024

Gradient compression has surfaced as a key technique to address the challenge of communication efficiency in distributed learning. In distributed deep learning, however, it is observed that gradient distributions are heavy-tailed, with outliers significantly influencing the design of compression strategies. Existing parameter quantization methods experience performance degradation when this heavy-tailed feature is ignored. In this paper, we introduce a novel compression scheme specifically engineered for heavy-tailed gradients, which effectively combines gradient truncation with quantization. This scheme is adeptly implemented within a communication-limited distributed Stochastic Gradient Descent (SGD) framework. We consider a general family of heavy-tail gradients that follow a power-law distribution, we aim to minimize the error resulting from quantization, thereby determining optimal values for two critical parameters: the truncation threshold and the quantization density. We provide a theoretical analysis on the convergence error bound under both uniform and non-uniform quantization scenarios. Comparative experiments with other benchmarks demonstrate the effectiveness of our proposed method in managing the heavy-tailed gradients in a distributed learning environment., Comment: arXiv admin note: substantial text overlap with arXiv:2402.01160

This paper introduces a privacy-preserving distributed learning framework via private-key homomorphic encryption. Thanks to the randomness of the quantization of gradients, our learning with error (LWE) based encryption can eliminate the error terms, thus avoiding the issue of error expansion in conventional LWE-based homomorphic encryption. The proposed system allows a large number of learning participants to engage in neural network-based deep learning collaboratively over an honest-but-curious server, while ensuring the cryptographic security of participants' uploaded gradients.

Maximum distance separable (MDS) codes have the optimal trade-off between storage efficiency and fault tolerance, which are widely used in distributed storage systems. As typical non-MDS codes, simple regenerating codes (SRCs) can achieve both smaller repair bandwidth and smaller repair locality than traditional MDS codes in repairing single-node erasure. In this paper, we propose {\em generalized simple regenerating codes} (GSRCs) that can support much more parameters than that of SRCs. We show that there is a trade-off between sub-packetization and fault tolerance in our GSRCs, and SRCs achieve a special point of the trade-off of GSRCs. We show that the fault tolerance of our GSRCs increases when the sub-packetization increases linearly. We also show that our GSRCs can locally repair any singe-symbol erasure and any single-node erasure, and the repair bandwidth of our GSRCs is smaller than that of the existing related codes.

Partial maximum distance separable (PMDS) codes are a kind of erasure codes where the nodes are divided into multiple groups with each forming an MDS code with a smaller code length, thus they allow repairing a failed node with only a few helper nodes and can correct all erasure patterns that are information-theoretically correctable. However, the repair of a failed node of PMDS codes still requires a large amount of communication if the group size is large. Recently, PMDS array codes with each local code being an MSR code were introduced to reduce the repair bandwidth further. However, they require extensive rebuilding access and unavoidably a significant sub packetization level. In this paper, we first propose two constructions of PMDS array codes with two global parities that have smaller sub-packetization levels and much smaller finite fields than the existing one. One construction can support an arbitrary number of local parities and has $(1+\epsilon)$-optimal repair bandwidth (i.e., $(1+\epsilon)$ times the optimal repair bandwidth), while the other one is limited to two local parities but has significantly smaller rebuilding access and its sub packetization level is only $2$. In addition, we present a construction of PMDS array code with three global parities, which has a smaller sub-packetization level as well as $(1+\epsilon)$-optimal repair bandwidth, the required finite field is significantly smaller than existing ones., Comment: Accepted for publication in the IEEE Transactions on Information Theory

As a special class of array codes, $(n,k,m)$ piggybacking codes are MDS codes (i.e., any $k$ out of $n$ nodes can retrieve all data symbols) that can achieve low repair bandwidth for single-node failure with low sub-packetization $m$. In this paper, we propose two new piggybacking codes that have lower repair bandwidth than the existing piggybacking codes given the same parameters. Our first piggybacking codes can support flexible sub-packetization $m$ with $2\leq m\leq n-k$, where $n - k > 3$. We show that our first piggybacking codes have lower repair bandwidth for any single-node failure than the existing piggybacking codes when $n - k = 8,9$, $m = 6$ and $30\leq k \leq 100$. Moreover, we propose second piggybacking codes such that the sub-packetization is a multiple of the number of parity nodes (i.e., $(n-k)|m$), by jointly designing the piggyback function for data node repair and transformation function for parity node repair. We show that the proposed second piggybacking codes have lowest repair bandwidth for any single-node failure among all the existing piggybacking codes for the evaluated parameters $k/n = 0.75, 0.8, 0.9$ and $n-k\geq 4$.

Generalized Expanded-Blaum-Roth (GEBR) codes [1] are designed for large-scale distributed storage systems that have larger recoverability for single-symbol failures, multi-column failures and multi-row failures, compared with locally recoverable codes (LRC). GEBR codes encode an $\alpha\times k$ information array into a $p\tau\times (k+r)$ array such that lines of slope $i$ with $0\leq i\leq r-1$ have even parity and each column contains $p\tau-\alpha$ local parity symbols, where $p$ is an odd prime and $k+r\leq p\tau$. Necessary and sufficient conditions for GEBR codes to be $(n,k)$ recoverable (i.e., any $k$ out of $n=k+r$ columns can retrieve all information symbols) are given in [2] for $\alpha=(p-1)\tau$. However, the $(n,k)$ recoverable condition of GEBR codes is unknown when $\alpha<(p-1)\tau$. In this paper, we present the $(n,k)$ recoverable condition for GEBR codes for $\alpha< (p-1)\tau$. In addition, we present a sufficient condition for enabling GEBR codes to recover some erased lines of any slope $i$ ($0\leq i\leq p\tau-1$) for any parameter $r$ when $\tau$ is a power of $p$. Moreover, we present the construction of Generalized Blaum-Roth (GBR) codes that encode an $\alpha\times k$ information array into an $\alpha\times (k+r)$ array. We show that GBR codes share the same MDS condition as the $(n,k)$ recoverable condition of GEBR codes, and we also present a sufficient condition for GBR codes to recover some erased lines of any slope $i$ ($0\leq i\leq \alpha-1$).

Piggybacking codes are a special class of MDS array codes that can achieve small repair bandwidth with small sub-packetization by first creating some instances of an $(n,k)$ MDS code, such as a Reed-Solomon (RS) code, and then designing the piggyback function. In this paper, we propose a new piggybacking coding design which designs the piggyback function over some instances of both $(n,k)$ MDS code and $(n,k')$ MDS code, when $k\geq k'$. We show that our new piggybacking design can significantly reduce the repair bandwidth for single-node failures. When $k=k'$, we design piggybacking code that is MDS code and we show that the designed code has lower repair bandwidth for single-node failures than all existing piggybacking codes when the number of parity node $r=n-k\geq8$ and the sub-packetization $\alpha

A maximum distance separable (MDS) array code is composed of $m\times (k+r)$ arrays such that any $k$ out of $k+r$ columns suffice to retrieve all the information symbols. Expanded-Blaum-Roth (EBR) codes and Expanded-Independent-Parity (EIP) codes are two classes of MDS array codes that can repair any one symbol in a column by locally accessing some other symbols within the column, where the number of symbols $m$ in a column is a prime number. By generalizing the constructions of EBR and EIP codes, we propose new MDS array codes, such that any one symbol can be locally recovered and the number of symbols in a column can be not only a prime number but also a power of an odd prime number. Also, we present an efficient encoding/decoding method for the proposed generalized EBR (GEBR) and generalized EIP (GEIP) codes based on the LU factorization of a Vandermonde matrix. We show that the proposed decoding method has less computational complexity than existing methods. Furthermore, we show that the proposed GEBR codes have both a larger minimum symbol distance and a larger recovery ability of erased lines for some parameters when compared to EBR codes. We show that EBR codes can recover any $r$ erased lines of a slope for any parameter $r$, which was an open problem in [2].

In this paper, we consider the update bandwidth in distributed storage systems~(DSSs). The update bandwidth, which measures the transmission efficiency of the update process in DSSs, is defined as the total amount of data symbols transferred in the network when the data symbols stored in a node are updated. This paper contains the following contributions. First, we establish the closed-form expression of the minimum update bandwidth attainable by irregular array codes. Second, after defining a class of irregular array codes, called Minimum Update Bandwidth~(MUB) codes, which achieve the minimum update bandwidth of irregular array codes, we determine the smallest code redundancy attainable by MUB codes. Third, the code parameters, with which the minimum code redundancy of irregular array codes and the smallest code redundancy of MUB codes can be equal, are identified, which allows us to define MR-MUB codes as a class of irregular array codes that simultaneously achieve the minimum code redundancy and the minimum update bandwidth. Fourth, we introduce explicit code constructions of MR-MUB codes and MUB codes with the smallest code redundancy. Fifth, we establish a lower bound of the update complexity of MR-MUB codes, which can be used to prove that the minimum update complexity of irregular array codes may not be achieved by MR-MUB codes. Last, we construct a class of $(n = k + 2, k)$ vertical maximum-distance separable (MDS) array codes that can achieve all of the minimum code redundancy, the minimum update bandwidth and the optimal repair bandwidth of irregular array codes.

A novel implementation of a special class of Galois ring, in which the multiplication can be realized by a cyclic convolution, is applied to the construction of network codes. The primitive operations involved are byte-wise shifts and integer additions modulo a power of 2. Both of them can be executed efficiently in microprocessors. An illustration of how to apply this idea to array code is given at the end of the paper., Comment: Accepted for presentation in ISIT2020

An $(n,k)$ maximum distance separable (MDS) code has optimal repair access if the minimum number of symbols accessed from $d$ surviving nodes is achieved, where $k+1\le d\le n-1$. Existing results show that the sub-packetization $\alpha$ of an $(n,k,d)$ high code rate (i.e., $k/n>0.5$) MDS code with optimal repair access is at least $(d-k+1)^{\lceil\frac{n}{d-k+1}\rceil}$. In this paper, we propose a class of multi-layer transformed MDS codes such that the sub-packetization is $(d-k+1)^{\lceil\frac{n}{(d-k+1)\eta}\rceil}$, where $\eta=\lfloor\frac{n-k-1}{d-k}\rfloor$, and the repair access is optimal for any single node. We show that the sub-packetization of the proposed multi-layer transformed MDS codes is strictly less than the existing known lower bound when $\eta=\lfloor\frac{n-k-1}{d-k}\rfloor>1$, achieving by restricting the choice of $d$ specific helper nodes in repairing a failed node. We further propose multi-layer transformed EVENODD codes that have optimal repair access for any single node and lower sub-packetization than the existing binary MDS array codes with optimal repair access for any single node. With our multi-layer transformation, we can design new MDS codes that have the properties of low computational complexity, optimal repair access for any single node, and relatively small sub-packetization, all of which are critical for maintaining the reliability of distributed storage systems.

Consider a binary maximum distance separable (MDS) array code composed of an $m\times (k+r)$ array of bits with $k$ information columns and $r$ parity columns, such that any $k$ out of $k+r$ columns suffice to reconstruct the $k$ information columns. Our goal is to provide {\em optimal repair access} for binary MDS array codes, meaning that the bandwidth triggered to repair any single failed information or parity column is minimized. In this paper, we propose a generic transformation framework for binary MDS array codes, using EVENODD codes as a motivating example, to support optimal repair access for $k+1\le d \le k+r-1$, where $d$ denotes the number of non-failed columns that are connected for repair; note that when $d

In this paper, we study the delay-constrained input-queued switch where each packet has a deadline and it will expire if it is not delivered before its deadline. Such new scenario is motivated by the proliferation of real-time applications in multimedia communication systems, tactile Internet, networked controlled systems, and cyber-physical systems. The delay-constrained input-queued switch is completely different from the well-understood delay-unconstrained one and thus poses new challenges. We focus on three fundamental problems centering around the performance metric of timely throughput: (i) how to characterize the capacity region? (ii) how to design a feasibility/throughput-optimal scheduling policy? and (iii) how to design a network-utility-maximization scheduling policy? We use three different approaches to solve these three fundamental problems. The first approach is based on Markov Decision Process (MDP) theory, which can solve all three problems. However, it suffers from the curse of dimensionality. The second approach breaks the curse of dimensionality by exploiting the combinatorial features of the problem. It gives a new capacity region characterization with only a polynomial number of linear constraints. The third approach is based on the framework of Lyapunov optimization, where we design a polynomial-time maximum-weight T-disjoint-matching scheduling policy which is proved to be feasibility/throughput-optimal. Our three approaches apply to the frame-synchronized traffic pattern but our MDP-based approach can be extended to more general traffic patterns.

Array codes have been widely employed in storage systems, such as Redundant Arrays of Inexpensive Disks (RAID). The row-diagonal parity (RDP) codes and EVENODD codes are two popular double-parity array codes. As the capacity of hard disks increases, better fault tolerance by using array codes with three or more parity disks is needed. Although many extensions of RDP codes and EVENODD codes have been proposed, the high decoding complexity is the main drawback of them. In this paper, we present a new construction for all families of EVENODD codes and RDP codes, and propose a unified form of them. Under this unified form, RDP codes can be treated as shortened codes of EVENODD codes. Moreover, an efficient decoding algorithm based on an LU factorization of Vandermonde matrix is proposed when the number of continuous surviving parity columns is no less than the number of erased information columns. The new decoding algorithm is faster than the existing algorithms when more than three information columns fail. The proposed efficient decoding algorithm is also applicable to other Vandermonde array codes. Thus the proposed MDS array code is practically very meaningful for storage systems that need higher reliability.

Binary maximum distance separable (MDS) array codes are a special class of erasure codes for distributed storage that not only provide fault tolerance with minimum storage redundancy but also achieve low computational complexity. They are constructed by encoding $k$ information columns into $r$ parity columns, in which each element in a column is a bit, such that any $k$ out of the $k+r$ columns suffice to recover all information bits. In addition to providing fault tolerance, it is critical to improve repair performance in practical applications. Specifically, if a single column fails, our goal is to minimize the repair bandwidth by downloading the least amount of bits from $d$ healthy columns, where $k\leq d\leq k+r-1$. If one column of an MDS code is failed, it is known that we need to download at least $1/(d-k+1)$ fraction of the data stored in each of $d$ healthy columns. If this lower bound is achieved for the repair of the failure column from accessing arbitrary $d$ healthy columns, we say that the MDS code has optimal repair. However, if such lower bound is only achieved by $d$ specific healthy columns, then we say the MDS code has weak-optimal repair. In this paper, we propose two explicit constructions of binary MDS array codes with more parity columns (i.e., $r\geq 3$) that achieve asymptotically weak-optimal repair, where $k+1\leq d\leq k+\lfloor(r-1)/2\rfloor$, and "asymptotic" means that the repair bandwidth achieves the minimum value asymptotically in $d$. Codes in the first construction have odd number of parity columns and asymptotically weak-optimal repair for any one information failure, while codes in the second construction have even number of parity columns and asymptotically weak-optimal repair for any one column failure.

Erasure coding is widely used for massive storage in data centers to achieve high fault tolerance and low storage redundancy. Since the cross-rack communication cost is often high, it is critical to design erasure codes that minimize the cross-rack repair bandwidth during failure repair. In this paper, we analyze the optimal trade-off between storage redundancy and cross-rack repair bandwidth specifically for data centers, subject to the condition that the original data can be reconstructed from a sufficient number of any non-failed nodes. We characterize the optimal trade-off curve under functional repair, and propose a general family of erasure codes called rack-aware regenerating codes (RRC), which achieve the optimal trade-off. We further propose exact repair constructions of RRC that have minimum storage redundancy and minimum cross-rack repair bandwidth, respectively. We show that (i) the minimum storage redundancy constructions support a wide range of parameters and have cross-rack repair bandwidth that is strictly less than that of the classical minimum storage regenerating codes in most cases, and (ii) the minimum cross-rack repair bandwidth constructions support all the parameters and have less cross-rack repair bandwidth than that of the minimum bandwidth regenerating codes for almost all of the parameters.

Array codes have been widely used in communication and storage systems. To reduce computational complexity, one important property of the array codes is that only XOR operation is used in the encoding and decoding process. In this work, we present a novel family of maximal-distance separable (MDS) array codes based on Cauchy matrix, which can correct up to any number of failures. We also propose an efficient decoding method for the new codes to recover the failures. We show that the encoding/decoding complexities of the proposed approach are lower than those of existing Cauchy MDS array codes, such as Rabin-Like codes and CRS codes. Thus, the proposed MDS array codes are attractive for distributed storage systems.

To recover simultaneous multiple failures in erasure coded storage systems, Patrick Lee et al introduce concurrent repair based minimal storage regenerating codes to reduce repair traffic. The architecture of this approach is simpler and more practical than that of the cooperative mechanism in non-fully distributed environment, hence this paper unifies such class of regenerating codes as concurrent regenerating codes and further studies its characteristics by analyzing cut-based information flow graph in the multiple-node recovery model. We present a general storage-bandwidth tradeoff and give closed-form expressions for the points on the curve, including concurrent repair mechanism based on minimal bandwidth regenerating codes. We show that the general concurrent regenerating codes can be constructed by reforming the existing single-node regenerating codes or multiplenode cooperative regenerating codes. Moreover, a connection to strong-MDS is also analyzed. On the other respect, the application of RGC is hardly limited to "repairing". It is of great significance for "scaling", a scenario where we need to increase(decrease) nodes to upgrade(degrade) redundancy and reliability. Thus, by clarifying the similarities and differences, we integrate them into a unified model to adjust to the dynamic storage network., Comment: 12 pages, 7 figures