Download - NCCloud : A Network-Coding-Based Storage System in a Cloud-of-Clouds

1

NCCloud: A Network-Coding-Based Storage System in a Cloud-of-Clouds

Henry C. H. ChenYuchong Hu

Patrick P. C. LeeYang Tang

IEEE Transactions on Computers, 15 August 2013

2

Outline

ه Introductionه Repair in Multiple Cloud Storageه FMSR Codesه NCCloudه Conclusion

3

Introduction

ه Cloud storage provides an on-demand remote backup solution.

ه A single cloud storage provider encounters the problem such as a single point of failure.

4

Introduction

ه The general solution is to distribute data across different cloud providers.ه stripe data

ه The fault-tolerance can be improved by the diversity of multiple clouds.

5

Introduction-Data Failure

ه This paper focuses on unexpected permanent cloud failure.ه a cloud fails permanently => activate repair.ه maintain data redundancy and fault-tolerance.

ه A repair operation ه retrieves data from existing surviving clouds.ه reconstructs the lost data in a new cloud.

6

Introduction-Data Failure

ه During repair, each surviving node ه encode its stored data chunks.ه send the encoded chunks to a new node

ه Regenerate the lost data.

7

Introduction-Cost Problem

ه Today’s cloud storage providers charge users for outbound data.

ه While repairing failures, moving the enormous amount of data (repair traffic) can introduce significant monetary costs.

8

Introduction-Repair Traffic Problem

ه In order to minimize repair traffic problem, regenerating codes [16] have been proposed. ه store data redundantly in a distributed storage

system.ه require less repair traffic, but with the same

fault-tolerance level.

[16] Network Coding for Distributed Storage Systems

9

Introduction-Regenerating Codes

ه But, most existing regenerating codes require storage nodesه equip with computation capabilities.ه perform encoding operations during repair.

10

Introduction-Regenerating Codes

ه In order to make regenerating codes portable to any cloud storage service.

ه This paper considers only a thin-cloud interface where storage nodes only support read/write.

11

Introduction-NCCloud

ه In this paper, we present the design and implementation of NCCloudه a proxy-based storage system.ه a fault-tolerant storage.ه over multiple cloud storage providers.

12

Introduction-FMSR

ه On top of NCCloud, we propose the functional minimum-storage regenerating (FMSR) codes.

ه The FMSR code implementation ه maintain double-fault tolerance.ه maintain the same storage cost as in RAID-6ه less repair traffic when recovering a single-cloud

failure.

13

Introduction-FMSR

ه FMSR codes are non-systematic ه the encoded chunks was formed by linear

combination of the original data chunks.ه not keep the original data chunks as in

systematic coding schemes.

14

Outline


15

Repair in Multiple Cloud Storage

ه Transient failureه is short-term, such that the failed cloud will

return to normal after some time and no outsourced data is lost.

16

Repair in Multiple Cloud Storage

ه Permanent failureه is long-term, in the sense that the outsourced

data on a failed cloud will become permanently unavailable.

ه example : .data center outages in disastersى.data loss and corruptionى.malicious attacksى

17

Outline

ه Introductionه Repair in Multiple Cloud Storageه FMSR Codes

ه Motivationه Implementation

ه NCCloudه Conclusion

18

Motivation

ه This paper considersه distributedه multiple-cloud storageه data is stripedه proxy-based design

19

Motivation

20

Fault-tolerant

ه Maximum Distance Separable propertyه (n, k)-MDS code

.divide file into equal-size native chunksى.linearly combined to form code chunksى

ه distribute over n (larger than k) nodes.ه reconstruct original file from any k of the n

nodes.ه tolerate the failures of any n − k nodes.

21

Fault-tolerant

ه The FMSR codes can reconstruct the data of failed node from the surviving nodes.ه download less data.ه not reconstruct the whole file.

22

Different Coding SchemesStorage size 2MRepair traffic M

Storage size 2MRepair traffic 0.75M

Storage size 2MRepair traffic 0.75M

23

Double-fault Tolerant FMSR Codes

ه divide a file M into 2(n − 2) native chunks.ه generate 2n code chunks.ه each node store two code chunks of size .ه repair a failed node, repair traffic is .ه RAID-6 codes, total storage size is , repair traffic

is M.50% saved

24

Outline

ه Introductionه Repair in Multiple Cloud Storageه FMSR Codes

ه Motivationه Implementation

ه NCCloudه Conclusion

25

FMSR Codes Implementation

ه FMSR codes do not require lost chunks to be exactly reconstructedه not identical to those in the failed node.

ه As long as the MDS property holds.

26


ه This paper propose a two-phase checking scheme to ensure the code chunks on all nodes always satisfy the MDS property.

27


ه The implementation assumes a thin-cloud interface.1. File upload2. File download3. Repair

28

File Upload

ه Native chunks :

ه Code chunks :

ه Encoding matrix of coefficients : ه size ه in the Galois field GF(pn)

29

File Upload

ه Galois field GF(pn)

Encoding coefficient vector

30

File Download

1. Download the k(n−k) code chunks from any k of the n storage nodes.

2. The ECVs of the k(n−k) code chunks can form a k(n−k)×k(n−k) square matrix.

3. Obtain the original k(n − k) native chunks. ه multiply the inverse of the square matrix with the code

chunks.

31

Iterative Repair

ه MDS property must hold even after iterative repairs.

ه This paper proposes a two-phase checking.ه MDS propertyه rMDS property

32

Satisfy MDS, but not rMDS

33

Iterative Repair

Step 1. Download the encoding matrix from a surviving node.

Step 2. Select one ECV from each of the n-1 surviving nodes.

Step 3. Generate a repair matrix .

Step 4. Compute the ECVs for the new code chunks and reproduce a new encoding matrix.

34

Iterative Repair

Step 5. Given EM’, verify if those properties are satisfied.ه verify MDS by enumerating all .ه verify rMDS by n(n−k)n-1 .ه The corresponding encoding matrices must form a full rank.

Step 6. Download the actual chunk data and regenerate new chunk data.

ه Step 4 : The new ECVsه Code chunks from surviving nodes

35

rMDS Sustaining

36

Time of Two-phase Checking

37

Double-fault Tolerant Codes

ه Markov Model

38

MTTDL, Compare to RAID-6

Mean Time To Data Loss

39

Outline


40

NCCloud

ه A proxy that bridges user applications and multiple clouds.

ه Its design is built on three layers.ه File system layerه Coding layerه Storage layer

41

NCCloud

ه It is mainly implemented in Python, while the coding schemes are implemented in C for better efficiency.

42

Goal of NCCloud

ه Compare the costs and response time of using RAID-6 and FMSR codes.

ه The cost advantage of FMSR over RAID-6, while maintaining acceptable response time.

43

Goal of NCCloud

ه Normal operationsه RAID-6 and FMSR incur similar storage costs.

ه Repair operationه FMSR save a significant amount of transfer

costs over RAID-6.

44

Cost Saving-Price

45

Cost Saving

ه Normal operationsه 1.25PB of data stored

FMSR : $86,851 monthly storage costىRAID-6 : $86,851 monthly storage costى

ه Repair operationه RAID-6 : 1PB of data, $56,832ه FMSR : 0.5625PB of data, $33,894

Saving of $ 22,938

46

Response Time-Local Cloud

47

Response Time-Local Cloud

48

Response Time-Commerical Cloud

49

Outline


50

Conclusion

ه This paper present NCCloud providing the reliability of today’s cloud backup storage.ه proxy-basedه multiple-cloud storage system

ه NCCloud not only provides fault tolerance in storage, but also allows cost-effective repair.

ه The FMSR code implementation eliminates the encoding requirement of storage nodes during repair.

Download - NCCloud : A Network-Coding-Based Storage System in a Cloud-of-Clouds

Top Related