(gam301) real-time game analytics with amazon kinesis, amazon redshift, and amazon dynamodb | aws...

© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

November 12, 2014 | Las Vegas

GAM301

Real-Time Game Analytics with Amazon Kinesis,

Amazon Redshift, and Amazon DynamoDB

Suhas Kulkarni – VP Engineering, GREE International

Kandarp Shah – Engineering Manager, GREE International

Talk Outline

• Mobile Game Analytics – use case

• Decisions, Mistakes & Challenges

• Deep dive – Analytics Platform using AWS Tech

• Lessons we learned

GREE Headquarters

Tokyo, Japan

GREE International,

Inc.

San Francisco, CA

GREE Canada

Vancouver, BC

QUICK FACTS

6Continents playing GREE games

1,882Employees Worldwide

13Games made in North America

2004

2011

2013

MILESTONES GAME STATS - 4 titles in top 100 grossing*

Crime City (Studios)

Reached Top 10 Grossing in 140 countries

Top 100 Grossing in 19 countries, over 3 years

since launch

*As of Sep. 2014 – Source: App Annie

A Global Gaming Powerhouse

Knights & Dragons (Publishing)

Reached Top 10 Grossing in 41 countries

Top 100 Grossing in 22 countries

Success Factors in Mobile Gaming

• Great gameplay & mechanics

• Great content

• Effective engagement & retention

• Generate in-app purchases

….. keep adding new content, features

??

?

I know its a

great game.

Why is my

game not

successful

anymore?

It had good

KPIs initially.

As a game developer ...

Why?

• Game not performing well

• Players not spending?

• Retention so poor?

How?

• Optimize game design

• Improve ARPDAU, %Spenders

• Improve Retention

You need – Game Analytics & Insights

Analytics @ GREE

Ad Clicks

Downloads

Perf Data

Attribution

Campaign Performance

SC Balance

HC Balance

IAP

Player Targeting

Data Collection

• Mobile Devices

• Game Servers

• Ad Networks

• Size of event ~ 1 KB

• 500M+ events/day

• 500G+/day & growing

• JSON format

Source of Data Data Size & Growth

Database Schema

• Every game – database schema

• Each game event = table (e.g., battle_fight, iap)

• 40-50 tables per DB schema

• All game titles ~ 1000 tables in DW

Key Requirements

• Data collection & streaming to database

• Zero data loss

• Zero data corruption

• Guaranteed data delivery

History – Analytics Architecture

Gen 1 – Analytics Platform

Analytics DB

Game DB

LAMP

Built on a LAMP Stack

Sharded DBs

Not scalable

Game

Servers

Gen2 – Flume/MPP Data

Warehouse

Game DB

MPP Data WarehouseFlume

Collectors

Flume

Master

Consumer

Game

Servers

In-house: Relay Engine

Game DB

Replicator

Amazon S3

DW

Relay Pipeline

Senders

Copiers

Game

Servers

Listeners

Cost of maintenance - HIGH

Challenges

• Hard to maintain and scale

• Spike in Live Ops events can clog other events

• Difficult to add new sink

• Writes to DW impacted query performance for BI

users

• Poor data latency

Key Requirements – the list grew

• Data collection &

streaming to database

• Zero data loss

• Zero data corruption

• Guaranteed data

delivery

• Near real-time data latency

• Real-time ad-hoc analysis

• Ease of adding consumers

• Managed Service

New Analytics Architecture

Start Simple

Ingest

• Kafka

• HTTP PUT

• Amazon Kinesis

Store

• HDFS

• Amazon S3

• Amazon Redshift

Analyze

Producer & Data

Data

{"player_id":"323726381807586881","player_level":169,"device":"iPhone 5","version":"iOS 7.1.2”,"platfrom":"ios","client_build":"440”,"db":”mw_dw_ios","table":"player_login","uuid":"1414566719-rsl3hvhu7o","time_created":"2014-10-29 00:11:59”}

{"fight_time":"2014-10-29 00:11:59","attacker_id":"413142642356365377”, "attacker_clan_size":500,"attacker_level":270,"db":"mw_dw_ios","table":"battle_fight","uuid":"1414566719-p0oogk0bep","time_created":"2014-10-29 00:11:59"}

• PHP/Python Web Services

• Client side analytics SDK

Produce Send Consume Load

Amazon Kinesis Sender

Produce Send Consume Load

Sender

Sender

Worker

Kinesis

Stream

Shard 1

Shard 2

Shard 3

Shard n

Describe Stream

Sync Shards

Analytics

Files

Send

PutRecordCompress

Read Buffer

50KB

Design Choices for Sender

• Single stream VS stream per game

• Batch VS Single Event

• Compressed VS Uncompressed

• PartitionKey VS ExplicitHashKey

Sender Deployment

Elastic Load

Balancing

AMI

Amazon

EC2

Auto Scaling Group

Amazon

EBS Data

Volume

Pending: Wait

EC2 EBS Data

Volume

Amazon

S3

Sender

Scale

Out

Pending: Proceed

EC2 EBS Data

Volume

EC2

Auto Scaling Group

EBS Data

Volume

InService

Update

Terminating: Wait

EC2 EBS Data

Volume

Scale

InChecks

Terminating: Proceed

EC2 EBS Data

Volume

Terminated

Consumer – S3 Store

Produce Send Consume Load

Consumer – Amazon S3 Store

Kinesis

Stream

Shard 1

Shard 2

Shard n

S3File Metadata DB

Decompress De-Dupe

BufferTransformation

Validation Target Table

Compress

Size/

Timeout

Record

Consumer

Kinesis Client Library

Record Processor

Record Processor

Consumer

Kinesis Client Library

Record Processor

Auto Scaling Group

Loading data into Amazon

Redshift

Produce Send Consume Load

Loading data into Amazon Redshift

Amazon S3

File Metadata DB Amazon

Redshift

Update Status

Transaction

Create Manifest Execute COPY

Create Manifest Execute COPY

Create Manifest Execute COPY

Status

Architecture – Simple Store

Produce Send Consume Load

Architecture – Simple Store

S3

Amazon S3

Consumer

Game DB

Game

ServersKinesis

Amazon

Redshift

Amazon

Elastic

MapReduce

DSV

JSON

Real-time Aggregates

DynamoDB – Real-time Aggregates

Amazon

DynamoDB

DashboardConsumerKinesis

Spark

Spark – Real-time Analysis

Spark SQLSpark Streaming

SQLKinesis

Final Architecture

Amazon S3

Amazon

S3

Spark SQL

JDBC ServerSQL

Dashboard

Amazon

Redshift

Consumer

Amazon

EMR

Consumer

Game DB

Game

ServersKinesis

DynamoDB

DSV

JSON

Lessons Learned

Lessons Learned

Sender

• Decouple data generation from sending

• Batch and compress

• PutRecord HTTP:5XX can result in duplicates

• Monitor ProvisionedThroughputExceeded exception

Lessons Learned (Cont.)

Consumer

• Use KCL

• Auto-scale and monitor load

Overall

• Provision enough shards

• Handle shutdown gracefully

• Follow AWS best practices for error retries and

exponential back-off

Takeaway

Takeaway

Kinesis

• Data available for processing within seconds

• Robust API, KCL, and Connector libraries

AWS

• Managed

• Scalable

• Cost effective

• Quick to get up and running

Please give us your feedback on this session.

Complete session evaluations and earn re:Invent swag.

http://bit.ly/awsevals