how to build a high performance application using cloud foundry and redis (cloud foundry summit...

23
1 building lightning fast apps over CF and BOSH

Upload: pivotal

Post on 27-Jan-2015

114 views

Category:

Technology


3 download

DESCRIPTION

Technical Track presented by Yiftach Shoolman, CTO & Co-Founder of Redis Labs. Why Redis? Redis is one of the top 3 databases chosen by developers. Redis is the fastest database available today has many attractive data types and commands for powering modern applications. In this session, you will learn: Why companies like Twitter, Pinterest, and GitHub rely on Redis as a critical infrastructure component. How to leverage Redis for real time analytics, social app functionality, job management, geo-search, and many other use cases. How to utilize CloudFoundry’s PaaS offering to build and maintain an infinitely scalable, highly available, top performing, and fully managed Redis database to power your application.

TRANSCRIPT

Page 1: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

1

building lightning fast appsover CF and BOSH

Page 2: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

2

Leading the commercial Redis market

Founded in 2011; GA in 02/2013

2,200+ paying customers; 50,000+ DBs; 100+

new DBs/day

2nd largest contributor to open source Redis

Raised $13M - Bain/Carmel/Strategic/Angels

Offices in Santa Clara and Tel-Aviv

In a nutshell

Page 3: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

3

Redis Cloud Memcached Cloud

Our offering

Fully-managed cloud services.

On-prem server license - soon.

Page 4: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

4

Launching Redis Labs

Clusterwith BOSH

Page 5: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

5

Why Redis ?

Page 6: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

6

The newest NoSQL create by Salvatore Sanfilippo

(@antirez)

The fastest data store available today (served entirely

from RAM)

Among the top 3 databases chosen by developers

Much more than a simple key/value - Strings, Hashes,

Lists, Sets, Sorted Set, LUA, transactions, Bits

operations

Strong use cases, dynamic community, large eco-

system

Redis

Page 7: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

7

100msec =

Fast apps requirements

max E2E response time, under any load

50msec = average Internet latency

50msec = required app response time (includes processing & multi DB accesses)

1msec = required DB response time

The only database to meet requirement

=

Page 8: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

8

DB performance comparison@<1mse

c

@<1msec

@<1msec

@<20msec

@<10-50msec

@<10-50msec

@<100msec

@<100msec

@>100msec

Page 9: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

9

Timeline

Followers

Caching

Messaging

Geo search

Leaderboards

Job management

RT analytics

Verticals & main use cases

Online advertisin

g

Social Gaming

Financial Services

Page 10: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

10

• Multi-TB in memory

• ~ 300,000 reads/sec

• ~ 5,000*N writes/sec

N - # of followers

Twitter

Every Timeline

(800 tweets per user)

is on Redis

Page 11: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

11

• 20TB+ in memory

• ~ 6,000,000 reads/sec

• ~ 600,000 writes/sec

Weibo (Chinese Twitter)

• Counting

• Reverse cache

• Top 10 lists

• Last Index

• Relational list/Message Queue

• Fast transactions w/ LUA

Page 12: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

12

Pinterest

Object graph:

• Per user (Sorted Set w/ timestamp as

score)

store the users followed (explicit+

implicit)

store the user’s followers

(explicit+implicit)

• Per board

Redis Hash for storing explicit followers

Redis Set for storing explicit unfollowers

Page 13: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

13

Stack Overflow

Three levels of cache:

• Local cache (no persistence)

sessions, and pending view count

updates

• Site cache

hot question id lists, users acceptance

rates..

• Global cache

Inboxes, API usage quotas, …

Page 14: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

14

Github

• Redis is used for routing info

• Matching user repositories to server

names

Page 15: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

15

Hipchat

• Which users are in which room

• Who is online

• XMPP server balancing

Page 16: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

16

Youporn

Most data is found in Hashes with ordered Sets used to

know what data to show

(1) ZinterStore on:

{videos:filters:release}{videos:filters:orientation:straig

ht}

{videos:filters:categories(id)}{videos:ordering:rating}

(2) Perform a ZRANGE to get the pages we want and get

the list of video_ids back

(3) Start pipelining to get all the videos from Hashes

Page 17: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

17

Snapchat

• 500+ instances

• 15-50TB

• Running on GCE

400M messages/day

Page 18: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

18

Why Redis Labs ?

Page 19: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

19

Infinite seamless scalability

True high-availability

Stable top performance

Zero management

Users choose us because..

Page 20: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

20

We invested a lot…

• 20+ man-years spent on R&D

• 15 Redis experts on team

Optimal Redis deployment

environment

Page 21: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

• [BOSH] Bootstrapping Redis Labs cluster (over EC2)

• [Redis Labs Console] Create a plan & set CF Credentials

• [CF]: Create & bind service • Show a working app (try.redis)

• SSO

The Demo

Page 22: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

22

Questions ?

Page 23: How to Build a High Performance Application Using Cloud Foundry and Redis (Cloud Foundry Summit 2014)

23

Thank You