Search

Data reliability and availability, and serviceability (RAS) of erasure-coded data centers are highly affected by data repair induced by node failures. In a traditional failure identification scheme, all chunks share the same identification time threshold, thus losing opportunities to further improve the RAS. To solve this problem, we propose RAFI, a novel risk-aware failure identification scheme. In RAFI, chunk failures in stripes experiencing different numbers of failed chunks are identified using different time thresholds. For those chunks in a high-risk stripe, a shorter identification time is adopted, thus improving the overall data reliability and availability. For those chunks in a low-risk stripe, a longer identification time is adopted, thus reducing the repair network traffic. Therefore, RAS can be improved simultaneously. We also propose three optimization techniques to reduce the additional overhead that RAFI imposes on management nodes and to ensure that RAFI can work properly under large-scale clusters. We use simulation, emulation, and prototyping implementation to evaluate RAFI from multiple aspects. Simulation and prototype results prove the effectiveness and correctness of RAFI, and the performance improvement of the optimization techniques on RAFI is demonstrated by running the emulator. [ABSTRACT FROM AUTHOR]

In large-scale replicated distributed storage systems consisting of hundreds to thousands of nodes, node failures are not rare and can cause data blocks to lose their replicas and become faulty. A simple but effective approach to prevent data loss from the node failures, i.e., ensuring reliability, is to shorten the identification time of the node failures and faulty blocks, which is determined by both timeouts and check intervals for node states. However, to maintain low repair network traffic, the identification time is actually relatively long and even dominates repair processes of critical blocks. In this paper, we propose a novel scheme, named RICK, to explore the potential in the identification time, and thus improve data reliability of replicated distributed storage systems while maintaining a low repair cost. First, by introducing an additional replica state, critical blocks (with two or more lost replicas) have individual short timeouts while sick blocks (with only one lost replica) preserve the long timeouts. Second, by replacing the static check intervals for node states with adaptive ones, the check intervals and the identification time of critical blocks are further shortened, which improves data reliability. Meanwhile, due to the low ratio of critical blocks in all faulty blocks, the repair network traffic remains low. The results from our simulation and prototype implementation show that RICK improves data reliability of replicated distributed storage systems by a factor of up to 14 in terms of mean time to data loss. Meanwhile, the extra repair network traffic caused by RICK is less than 1.5 percent of the total network traffic for data repairs. [ABSTRACT FROM AUTHOR]