Stateful Applications On The Cloud – A PayPal JourneyAnant Kumar and Jasdeep SinghPayPal Inc.August 23, 2016

2

About Paypal

3

x 500K Cores

x 10 PB Storage

OpenStack Cloud

> 10K Physical Servers

x 100K VMs

x 1000engineers

x 10M Lines

x 1000 Releases/year

> 1000 Services

x 2000 Builds/day

x 1000Deploy/day

• Developer Statistics

9 Availability Zones

3 Regions

Key Statistics

4 4

2012-2013 2014-2015 2016-2017+

• OpenStack POC with 16 servers

• Built 2 AZs with OpenStack Essex release

• 5 apps in production• 40% of holiday traffic for

Paypal Web and APIs

• PaaS on Cloud with full PDLC support (Node and Java applications)

• 100% of Web, API & Mid-tier workloads

• All Stateless Applications in Cloud

• Built largest OpenStack based AZ (for Dev & QA)

‒ Next frontiers in Developer Productivity

‒ Infrastructure Cost & Efficiency

‒ Stateful Applications in Cloud

PayPal Cloud Platform Journey

5

Problem Statement

• Applications Still On Bare Metal• Messaging• Streaming• Caching• Search• Databases

• Challenges With:• Agility• Efficiency• Elasticity• Onboarding

6

Guiding Principles

• Hybrid Cloud Ready

• Cost Efficiency

• Container Compatible

• No Loss Of Efficiency And Performance

7

Constraints

• Preserve State (Petabytes Of Data)

• Storage, Storage, Storage

• Recovery from VM failure

• Network Bandwidth

• Noisy neighbor

8

Two Columns

• Network Block Storage • VMs With Ephemeral Disk

VM 1

Hypervisor

VM 2 VM n

Disk 1 Disk 2 Disk n

Host

VM 1

Hypervisor

VM 2 VM n

Network Block Storage

Host

iSCSI

9

Options

• VMs With Attached Local Disk • Hyper Converged Storage

VM 1

Hypervisor

VM 2 VM n

Disk 1 Disk 2 Disk n

Host

VM 1

Hypervisor

VM 2 VM n

Hyperconverged Storage Manager

NAS Disk 1 Disk n

Host

10

What We Chose

• Winner (For Now) : VMs With Attached Local Disks

• Advantages• Do Not Lose Data When VM Is Lost• Network Bandwidth Not A Concern• Cheaper (Always Helps !!!)

• Disadvantages• Storage Is Lost If The Host Is Lost

11

Conclusion

• Moving Ahead Full Steam - “VMs With Attached Local Disk”

• Keeping A Close Eye On The “Hyper-converged Storage” Space.

• Will Likely Support Multiple Options In The Future

• Keep The Hybrid Cloud In Mind