Search

Many distributed deep learning systems have been published over the past few years, often accompanied by impressive performance claims. In practice these figures are often achieved in high performance computing (HPC) environments with fast InfiniBand network connections. For average deep learning practitioners this is usually an unrealistic scenario, since they cannot afford access to these facilities. Simple re-implementations of algorithms such as EASGD  for standard Ethernet environments often fail to replicate the scalability and performance of the original works. In this paper, we explore this particular problem domain and present MPCA SGD, a method for distributed training of deep neural networks that is specifically designed to run in low-budget environments. MPCA SGD tries to make the best possible use of available resources, and can operate well if network bandwidth is constrained. Furthermore, MPCA SGD runs on top of the popular Apache Spark  framework. Thus, it can easily be deployed in existing data centers and office environments where Spark is already used. When training large deep learning models in a gigabit Ethernet cluster, MPCA SGD achieves significantly faster convergence rates than many popular alternatives. For example, MPCA SGD can train ResNet-152  up to 5.3x faster than state-of-the-art systems like MXNet  , up to 5.3x faster than bulk-synchronous systems like SparkNet  and up to 5.3x faster than decentral asynchronous systems like EASGD . [ABSTRACT FROM AUTHOR]

In recent years, there have been a number of large-scale deployments of P2P real-time streaming platforms in cloud storage systems due to their fast and convenient storage services for platform designers. Note that providing real-time streaming services for a platform is costly because of heavy bandwidth consumption from the platform to end-users. To cope with the above issue, our idea is to exploit the upload bandwidth of volunteers, namely, edge or fog nodes, to form an edge-node-assisted real-time streaming P2P platform. Specifically, according to designed strategies, the cloud distributes chunk replicas to edge nodes that can respond to end-user requests, and the public cloud supplements end-user demands only when the edge nodes cannot accommodate them. One of the important issues while introducing edge nodes into the system is how to utilize their resources efficiently. However, mainly due to the heterogeneous upload capacity of edge nodes and uneven request rates for different streaming chunks, the replica utilization at the edge nodes is diverse for different replica distribution strategies. Likewise bandwidth consumption in the public cloud varies correspondingly. Thus, this paper describes a mathematical model of the proposed replica distribution and designs an optimal algorithm for distributing chunk replicas. The results validate the effectiveness of the proposal. [ABSTRACT FROM AUTHOR]

Distributed storage systems (DSSs) in the presence of an external wiretapper are considered. A DSS is parameterized by $(n, k, d)$ , in which the data are stored across $n$ nodes (each with storage capacity $\alpha $ ), and must be recoverable by accessing the contents stored on any $k$ out of $n$ nodes. If a node fails, any $d \geq k$ out of $(n-1)$ nodes help in the repair (regeneration) of the failed node (by sending $d\beta $ units of repair data, where $\beta \leq \alpha $ ), so that the data can still be recovered from the DSS. For such a $(n, k, d)$ -DSS, security from the two types of wiretappers is investigated: 1) Type-I (node data) wiretapper, which can read the data stored on any $\ell

This paper presents and analyzes a novel concatenated coding scheme for enabling error resilience in two distributed storage settings: one being storage using existing regenerating codes and the second being storage using locally repairable codes. The concatenated coding scheme brings together a maximum rank distance code as an outer code and either a globally regenerating or a locally repairable code as an inner code. In addition, error resilience for combination of locally repairable codes with regenerating codes is considered. This concatenated coding system is designed to handle two different types of adversarial errors: the first type includes an adversary that can replace the content of an affected node only once; while the second type studies an adversary that is capable of polluting data an unbounded number of times. The paper establishes an upper bound on the resilience capacity for a locally repairable code. This paper also proves that the proposed concatenated coding approach attains the upper bound on the resilience capacity in the presence of the first type of adversary for both minimum storage regenerating codes and locally repairable codes. Further, this paper presents mechanisms that combine the presented concatenated coding scheme with subspace signatures to achieve error resilience for the second type of errors. [ABSTRACT FROM PUBLISHER]

We consider the $(n,k,d,\ell )$ secure exact-repair regenerating code problem, which generalizes the $(n,k,d)$ exact-repair regenerating code problem with the additional constraint that the stored file needs to be kept information-theoretically secure against an eavesdropper, who can access the data transmitted to regenerate a total of $\ell $ different failed nodes. For all known results on this problem, the achievable tradeoff regions between the normalized storage capacity and repair bandwidth have a single corner point, achieved by a scheme proposed by Shah, Rashmi, and Kumar (the SRK point). Since the achievable tradeoff regions of the exact-repair regenerating code problem without any secrecy constraints are known to have multiple corner points in general, these existing results suggest a phase-change-like behavior, i.e., enforcing a secrecy constraint ( $\ell \geq 1$ ) immediately reduces the tradeoff region to one with a single corner point. In this paper, we first show that when the secrecy parameter $\ell $ is sufficiently large, the SRK point is indeed the only corner point of the tradeoff region. However, when $\ell $ is small, we show that the tradeoff region can in fact have multiple corner points. In particular, we establish a precise characterization of the tradeoff region for the (7, 6, 6, 1) problem, which has exactly two corner points. Thus, a smooth transition, instead of a phase-change-type of transition, should be expected as the secrecy constraint is gradually strengthened. [ABSTRACT FROM PUBLISHER]

Centralized cloud infrastructures have become the popular platforms for data-intensive computing today. However, they suffer from inefficient data mobility due to the centralization of cloud resources, and hence, are highly unsuited for geo-distributed data-intensive applications where the data may be spread at multiple geographical locations. In this paper, we present Nebula: a dispersed edge cloud infrastructure that explores the use of voluntary resources for both computation and data storage. We describe the lightweight Nebula architecture that enables distributed data-intensive computing through a number of optimization techniques including location-aware data and computation placement, replication, and recovery. We evaluate Nebula performance on an emulated volunteer platform that spans over 50 PlanetLab nodes distributed across Europe, and show how a common data-intensive computing framework, MapReduce, can be easily deployed and run on Nebula. We show Nebula MapReduce is robust to a wide array of failures and substantially outperforms other wide-area versions based on emulated existing systems. [ABSTRACT FROM PUBLISHER]

In order to guarantee data reliability, erasure codes have been used in distributed storage systems. Nevertheless, this mechanism suffers from the repair problem that excess data are needed to repair a single failure, causing both high bandwidth consuming for the network and heavy computing load on the replacement node. To reduce repair traffic, researchers pointed out the tradeoff between storage and repair traffic and proposed regenerating codes by combining network coding. However, the combination only focuses on the storage terminal and the construction of the codes is quite complicated. Therefore, this paper further combines network coding with network structure and proposes a repair tree model based on general erasure codes to simplify the repair procedure. By decomposing repair computing and distributing it among the tree nodes, our model can mitigate the computing tension. The performance of repair tree is analyzed and evaluated by preliminary emulation. The result shows it can make about three times faster computing than conventional measure and the repair throughput is doubled if there are network bottlenecks. For proper topology, it can significantly reduce the repair traffic. We present algorithms to generate trees across the network topology. At last, we present the idea of extending repair tree to repair multiple failures. [ABSTRACT FROM AUTHOR]

In distributed storage systems, regenerating codes can achieve the optimal tradeoff between storage capacity and repair bandwidth. However, a critical drawback of existing regenerating codes, in general, is the high coding and repair complexity, since the coding and repair processes involve expensive multiplication operations in finite field. In this paper, we present a design framework of regenerating codes, which employ binary addition and bitwise cyclic shift as the elemental operations, named BASIC regenerating codes. The proposed BASIC regenerating codes can be regarded as a concatenated code with the outer code being a binary parity-check code, and the inner code being a regenerating code utilizing the binary parity-check code as the alphabet. We show that the proposed functional-repair BASIC regenerating codes can achieve the fundamental tradeoff curve between the storage and repair bandwidth asymptotically of functional-repair regenerating codes with less computational complexity. Furthermore, we demonstrate that the existing exact-repair product-matrix construction of regenerating codes can be modified to exact-repair BASIC product-matrix regenerating codes with much less encoding, repair, and decoding complexity from the theoretical analysis, and with less encoding time, repair time, and decoding time from the implementation results. [ABSTRACT FROM AUTHOR]

Big-data computing is a new critical challenge for the ICT industry. Engineers and researchers are dealing with data sets of petabyte scale in the cloud computing paradigm. Thus, the demand for building a service stack to distribute, manage, and process massive data sets has risen drastically. In this paper, we investigate the Big Data Broadcasting problem for a single source node to broadcast a big chunk of data to a set of nodes with the objective of minimizing the maximum completion time. These nodes may locate in the same datacenter or across geo-distributed datacenters. This problem is one of the fundamental problems in distributed computing and is known to be NP-hard in heterogeneous environments. We model the Big-data broadcasting problem into a LockStep Broadcast Tree (LSBT) problem. The main idea of the LSBT model is to define a basic unit of upload bandwidth, r, such that a node with capacity c broadcasts data to a set of [c/r] children at the rater. Note that r is a parameter to be optimized as part of the LSBT problem. We further divide the broadcast data into m chunks. These data chunks can then be broadcast down the LSBT in a pipeline manner. In a homogeneous network environment in which each node has the same upload capacity c, we show that the optimal uplink rate r* of LSBT is either c/2 or c/3, whichever gives the smaller maximum completion time. For heterogeneous environments, we present an O(nlog2n) algorithm to select an optimal uplink rater* and to construct an optimal LSBT. Numerical results show that our approach performs well with less maximum completion time and lower computational complexity than other efficient solutions in literature. [ABSTRACT FROM PUBLISHER]

Providing scalable video streaming services for heterogeneous users in dynamic networked environments requires efficient and adaptive quality management mechanisms which deliver quality-customized services according to the client’s preferences and adapt the services to cope with various network conditions. In this paper, we address the issue of quality adaptation for providing personalized scalable media streaming services in dynamic network environments. We propose a differentiated adaptive quality optimization algorithm, called Scalable Video Coding Quality Adaptation algorithm (SVC-QA), which adapts streaming quality based on both system-level and client-level optimization to optimize streaming quality according to network bandwidth conditions, content characteristics, a user’s quality preferences, and buffering capacities of different client devices (e.g., mobile phones, PCs, HDTVs, etc.). Comparative studies are conducted to compare our proposed algorithms with other adaptive methods. We show that two-level SVC quality adaptation method can achieve better SVC streaming quality with both high peak signal-to-noise ratio (PSNR) and low quality variance under dynamic resource constraints. Moreover, the proposed distributed method reduces the computational complexities at the server side substantially, making it practical and flexible for providing scalable streaming services. [ABSTRACT FROM AUTHOR]

Data availability is critical in distributed storage systems, especially when node failures are prevalent in real life. A key requirement is to minimize the amount of data transferred among nodes when recovering the lost or unavailable data of failed nodes. This paper explores recovery solutions based on regenerating codes, which have been designed to provide fault-tolerant storage and minimum bandwidth. Existing optimal regenerating codes are designed for single node failures. We build a system called CORE, which augments existing optimal regenerating codes for the recovery of a general number of failures including single and concurrent failures. We show theoretically that CORE achieves the minimum possible bandwidth for most cases. We implement a CORE prototype and evaluate it atop an HDFS cluster testbed with up to 20 storage nodes. We demonstrate that our CORE prototype conforms to ourtheoretical findings and achieves bandwidth savings when compared to the conventional recovery approach based on erasure codes. [ABSTRACT FROM AUTHOR]