Transcript
Page 1: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

November 12, 2014

Akhil Gupta, Head of Infrastructure, Dropbox

Page 2: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 3: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 4: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

web app

server

Dropbox data center

Databases

S3 Bucket

web app

server

Page 5: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 6: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 7: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 8: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 9: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 10: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 11: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 12: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 13: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 14: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 15: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 16: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 17: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 18: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 19: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 20: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 21: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 22: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server

web app

server

S3 Bucket

Page 23: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

web app

server web app

server

S3 Bucket

Page 24: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 25: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 26: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 27: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

S3 Bucket

web app

server

Livefill

Server

Livefill

Extractor

Livefill

Extractor

Virtual Cluster #1

Virtual Cluster #3

Virtual Cluster #2

In Queue

Out Queue

Page 28: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 29: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

S3 Bucket

web app

server

Livefill

Server

Livefill

Extractor

Livefill

Extractor

Virtual Cluster #1

Virtual Cluster #3

Virtual Cluster #2

Process File A

InQueue

Out Queue

Page 30: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

S3 Bucket

Livefill

Extractor

Virtual Cluster #1

InQueue

Out Queue

Page 31: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

S3 Bucket

Virtual Cluster #1

Read File A

Page 32: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

S3 Bucket

Fetch File A

Handler 1Handler 2

Page 33: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

Dropbox data center

Databases

S3 Bucket

web app

server

Livefill

Server

Livefill

Extractor

Livefill

Extractor

Virtual Cluster #1

Virtual Cluster #3

Virtual Cluster #2

Store extracted

metadata

InQueue

Out Queue

Page 34: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 35: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 36: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 37: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014
Page 38: (PFC308) How Dropbox Scales Massive Workloads Using Amazon SQS | AWS re:Invent 2014

http://bit.ly/awsevals


Top Related