Transcript
Page 1: Dependable Storage and Computing using Multiple Cloud Providers

1

Dependable Storage and Computing using Multiple Cloud Providers

Alysson Bessani

www.di.fc.ul.pt/~bessani

International Industry-Academia Workshop onCloud Reliability and Resilience

Berlin, 7-8 November 2016

Page 2: Dependable Storage and Computing using Multiple Cloud Providers

3

Cloud Sec. & Dep.

• Secure and dependable services are a necessary condition for the long-term existence of a cloud provider– Clients need to trust providers– Providers need to justify this trust

• To stay in the market, a provider needs to invest on Sec. & Dep., however…

Page 3: Dependable Storage and Computing using Multiple Cloud Providers

4

Unavailability

Number of nines99,9% = 8,76 hours 99,99% = 53 minutes

Page 4: Dependable Storage and Computing using Multiple Cloud Providers

5

Unavailability

Page 5: Dependable Storage and Computing using Multiple Cloud Providers

6

Data Loss

2011

2015

2016

Page 6: Dependable Storage and Computing using Multiple Cloud Providers

7

Privacy Issues…

Page 7: Dependable Storage and Computing using Multiple Cloud Providers

8

Page 8: Dependable Storage and Computing using Multiple Cloud Providers

9

Important!

To the best of my knowledge, there is no work on theexisting technical literature saying that using the cloud

is less secure than having everything on premises

Cloud services work pretty well… but they are not perfect!

Page 9: Dependable Storage and Computing using Multiple Cloud Providers

10

Our view

• The best way to use the cloud in a dependable and secure way is to not rely on a single provider– Don’t base your service on a single

provider, i.e., an application works if the provider where it is running is correct

– Instead, consider distributed trust: an application is correct if at most f out-of n services/providers are faulty

Page 10: Dependable Storage and Computing using Multiple Cloud Providers

12

Cloud-of-clouds Service

• Two fundamental characteristics1. No modification on existing cloud services2. No collaboration between providers

Page 11: Dependable Storage and Computing using Multiple Cloud Providers

13

Cloud-of-Clouds Storage

Page 12: Dependable Storage and Computing using Multiple Cloud Providers

14

Cloud Storage Diversity

• 24+ cloud object storage services– Locations: 68– Model: Standard - Archival– Cost per GB/month stored: $0,001 - $0,13– Cost per GB downloaded: $0,01 - $0,25– Availability SLA: undefined, 99,9%-100%– Access control: inexistent, URLs, …, ACLs,– Standards compliance: 0-33+

Page 13: Dependable Storage and Computing using Multiple Cloud Providers

15

Fundamentals

• Let’s consider the problem of implementing a cloud-of-clouds object storage tolerating one cloud failure

• Assume you cannot know for sure if a unresponsive cloud is faulty or not

Page 14: Dependable Storage and Computing using Multiple Cloud Providers

16

Writing DataOne needs to write at least two copies of the data!

What if one of the clouds take too much to acknowledge the write?

Use three clouds, and wait for at least two acks to complete the write

Page 15: Dependable Storage and Computing using Multiple Cloud Providers

17

Reading DataHow many clouds you need to access for reading?

One cloud: might access an outdated/empty cloudTwo clouds: always access the last complete write

writequorum

readquorum

Page 16: Dependable Storage and Computing using Multiple Cloud Providers

18

Reading Data IIWhat if data can be modified or corrupted?

writequorum

readquorum

Page 17: Dependable Storage and Computing using Multiple Cloud Providers

19

Reading Data IITo solve this we need more clouds and bigger quorums

writequorum

readquorum

Page 18: Dependable Storage and Computing using Multiple Cloud Providers

20

DepSky: Dependable Cloud-of-Clouds Object Storage

[EuroSys’11, ACM Trans. on Storage 2013]

Availability, Integrity and Confidentialitydespite the failure of up to f clouds

Page 19: Dependable Storage and Computing using Multiple Cloud Providers

21

Challenges for implementing Updatable Objects

• How to implement an efficient replication protocol using only passive storage nodes?

• How to make it affordable?

read

write

Page 20: Dependable Storage and Computing using Multiple Cloud Providers

22

Cloud A

Cloud B

Cloud C

Cloud D

DD

D

D

qwjdasjkhdahsd

qwjdasjkhdahsd

qwjdasjkhdahsd

qwjdasjkhdahsd

DepSky Write

WRITEDATA

D

ACK

D

D

D

D

WRITEMETADATA

qwjdasjkhdahsd

ACK

qwjdasjkhdahsd

qwjdasjkhdahsd

qwjdasjkhdahsd

qwjdasjkhdahsd

Page 21: Dependable Storage and Computing using Multiple Cloud Providers

23

Cloud A

Cloud B

Cloud C

Cloud D

DD

D

D

DepSky Read

READDATA

D

DATA

D

D

D

D

READMETADATA

qwjdasjkhdahsd

qwjdasjkhdahsd

qwjdasjkhdahsd

qwjdasjkhdahsd

METADATA

qwjdasjkhdahsd

Data will be fetched from other clouds if needed.

highest version number

Page 22: Dependable Storage and Computing using Multiple Cloud Providers

24

Cloud A Cloud B Cloud C Cloud D

DepSky Confidentiality and Storage Efficiency

Data Limitations:1. Data is accessible

by cloud providers2. Requires n×|Data|

storage space

Data Data DataData

Page 23: Dependable Storage and Computing using Multiple Cloud Providers

25

Cloud A Cloud B Cloud C Cloud D

DepSky Confidentiality and Storage Efficiency

S1 S2 S3 S4

share

K

generatedkeyData

disperse

F1 F2 F3 F4

F1 S1 F2 S2 F3 S3 F4 S4

encrypt

Inverse process for reading from f+1 shares/fragments.

Page 24: Dependable Storage and Computing using Multiple Cloud Providers

27

Practical Considerations

• Read/write latency ≈ accessing a single cloud• Storage costs roughly 50% higher• DepSky is a Java programming library

http://cloud-of-clouds.github.io/depsky

• It does not support concurrent writers to the same object without using locks– Recent extensions for multi-writer storage (2016)

http://github.com/cloud-of-clouds/mwmr-registers

• How to build a complete system around it?

Page 25: Dependable Storage and Computing using Multiple Cloud Providers

28

CoordinationService

SCFS: Shared Cloud-backed File System

Storage cloudsCloud Storage

Cache

Cache

Cache

LockService

AccessControl

Metadata

Computingclouds

SCFSAgent

SCFSAgent

SCFSAgent

[USENIX ATC 2014]

Page 26: Dependable Storage and Computing using Multiple Cloud Providers

29

SCFS Consistency on Close

fd = open(“x”,…);…read(fd,…);write(fd,…);…fsync(fd);…write(fd,…);…close(fd);

CoordinationService

Cloud(-of-Clouds)

PersistentStorage

Main Memory

READ

WRITE

Local Storage

If “x” is cached, it is only validated

1

2

1

2

Page 27: Dependable Storage and Computing using Multiple Cloud Providers

30

• SCFS can use different backends– i.e., different cloud storage and a coordination service plugin

• Operation: blocking, non-blocking and non-sharing

SCFS Backends

Page 28: Dependable Storage and Computing using Multiple Cloud Providers

32

Sharing Latency: SCFS vs DropBox

Amazon S3

Google Storage

Rackspace Files

Windows Azure Blob

DATA

DATA

?$

#!

%|

DATA

DATA

SCFS

Dropbox

Page 29: Dependable Storage and Computing using Multiple Cloud Providers

34

Practical Considerations

• SCFS improves the transparency of cloud-of-clouds(-backed) storage

• It is implemented as a Linux FUSE FS:http://cloud-of-clouds.github.io/SCFS

• Limitations:– Does not work well with big files– Require computing instances to run the

coordination service (e.g., Zookeeper replicas)

• Can we do better?

Page 30: Dependable Storage and Computing using Multiple Cloud Providers

35

Private Cloud

Private Cloud

Private Cloud

Charon Cloud-backed File System

Charon CharonCharon

+ big data storage and sharing +multiple storage locations+ serverless design

Page 31: Dependable Storage and Computing using Multiple Cloud Providers

37

Charon Architecture

CharonCharon TLS ChannelCache

Site 1 Site 2

Cache

CA

Public Cloud A

Public Cloud B

D

B

A

B

d1

D

d2

C

metadata

Page 32: Dependable Storage and Computing using Multiple Cloud Providers

38

Avoiding write-write Conflicts without External Coordination

f-tolerant Byzantine lease

leasereleaserenew

enqueue, dequeue, peekAll

put, get, del, cas

atomic transactions

leasereleaserenew Queue

leasereleaserenew

Queue

leasereleaserenew

DynamoDB

leasereleaserenew Data Store

f+1clouds

Page 33: Dependable Storage and Computing using Multiple Cloud Providers

40

Confidentiality & Storage-Efficiency

Cloud A Cloud B Cloud C Cloud D

S1 S2 S3

secret sharing

K

randomly generated

keyCompressed Chunk

erasure code

F1 F2 F3

F1 S1 F2 S2 F3 S3

encrypt

Big File

Chunk 1 Chunk N

compress…

Optional

Page 34: Dependable Storage and Computing using Multiple Cloud Providers

41

Processing Writes

Page 35: Dependable Storage and Computing using Multiple Cloud Providers

43

Upload/Download Latency

0.6 GB/min1 GB/min

relatedwork

relatedwork

Page 36: Dependable Storage and Computing using Multiple Cloud Providers

45

Cloud-of-clouds Computing(very briefly)

Page 37: Dependable Storage and Computing using Multiple Cloud Providers

46

Disaster Recovery/Tolerance

• Services in cloud A can have a backup in cloud B• For less than €1/month it is possible to keep up to

30GB of data in another cloud w/ a RPO of 3 min

DBMS

Failover

Recovers take lessthan 40s per GB

Page 38: Dependable Storage and Computing using Multiple Cloud Providers

47

Permissioned Blockchains and Smart Contracts

• Permanent decentralized ledger used for recording transactions; requires a practical BFT consensus

• Ex.: Hyperledger, Symbiont Assembly, Ethereum, …

operationresult

operationresult

operationresult

operationresult

operationresult

Page 39: Dependable Storage and Computing using Multiple Cloud Providers

48

Geo-replicated Services

• Largely used in (single domain) internet-scale apps• Wide-area Crash or Byzantine fault tolerant replication

protocols are needed

ServiceReplica

ServiceReplica

ServiceReplica

ServiceReplica

Page 40: Dependable Storage and Computing using Multiple Cloud Providers

49

Final Remarks

• In 2010– Paxos and Zookeeper were not very popular– RAFT didn’t exist– Bitcoin was a crazy new idea– Blockchain and smart contracts were mostly ignored

• The world is changing fast, and advanced distributed applications are here to stay...

• What will happen in 2020?

Page 41: Dependable Storage and Computing using Multiple Cloud Providers

50

Further reading…Storage

• Bessani, Correia, Quaresma, André, Sousa. DepSky: Dependable and Secure Storage in the Cloud of Clouds. ACM Transactions on Storage. 2013. (preliminary version on ACM EuroSys’11).

• Oliveira, Mendes, Bessani. Exploring Key-Value Stores in Multi-Writer Byzantine-Resilient Register Emulations. OPODIS’16.

• Mendes, Oliveira, Cogo, Neves, Bessani. Charon: A Dependable Cloud-Backed System for Storing and Sharing Big Data. Under Submission. 2016.

• Bessani, Mendes, Oliveira, Neves, Correia, Pasin, Verissimo. SCFS: A Shared Cloud-backed File System. USENIX ATC’14.

Computing• Sousa, Bessani. Separating the WHEAT from the Chaff: An Empirical Design for Geo-

Replicated State Machines. IEEE SRDS’15.

• Bessani, Sousa, Alchieri. State Machine Replication for the Masses with BFT-SMaRt. IEEE/IFIP DSN’14.


Top Related