Vaibhav Puranik

Lambda Architecture with Cassandra

About me

Invented In Image advertising in 2008http://gumgum.com/gallery

Computer VisionProcessing 2.6B image impressions / month10B events / day2000 premium publishers165 employees in 3 continents

Lambda Architecture

Lambda Architecture

Architecture Before Cassandra

500 Ad Servers

Caps

Geo Filter

Keyword Filter

Cap Filter

Other Filters

Sort by revenue and pick one

Real Time Pipeline - The Problem500 Ad Servers

The Problem

6 minsroundtrip time

Enter Cassandra - Phase 1500 Ad Servers

Cassandra: US East - 21 Nodes, US West: 18 Nodes, EU West: 12 Nodes - r3.2xlarge

Caps

Geo Filter

Other Filters

Cassandra Cap Filter

Old Cap Filter

Sort by revenue and pick one

The Solution

6 mins 10 ms

Enter Cassandra - Phase 2500 Ad Servers

Cassandra: US East - 21 Nodes, US West: 18 Nodes, EU West: 12 Nodes - r3.2xlarge

Load data every 8 mins async way

Caps

Geo Filter

Other Filters

Cassandra RealtimeCap Filter

Cassandra Batch Data Filter

Sort by revenue and pick one

Scale

21 + 18 + 12 + 9 = 60 r3.2xlarge instances (Metadata)

9 + 6 + 3 = 18 r3.2xlarge instances (Tally)

400,000 records loaded every 8 mins by each Ad Server

20,000 read requests per second

10,000 write requests per second

Batch Layer

Realtime View

Batch Views

Lambda Architecture

Speed Layer

gumgum.com/careers