tarantool: как сэкономить миллион долларов на базе данных...

Как сэкономить$1 000 000

набазах данных

Денис Аникин,ТехДир Почты/Облака Mail.Ru

Why use a DBMS?

Why use a DBMS?

+ QUERIES+ ACID+ INDEXING+ TABLES+ DURABILITY

Database File

Heavy read workload

APP

DB

APP

APP APP

Overloaded

Solution: replication

APP DB Master

Update

DB Slave

Replication

Select

DB SlaveDB Slave

DB SlaveDB Slave

Heavy write workload

APP

DB

APP

APP APP

Replication does not scale writes

Solution: sharding

DB DB DB

DB DB DB

DB DB DB

Replication

Sharding

Who is the next problem?…

The boss

The boss

Technology

We are loosing money

Close down

What should we do?

Cache!

DB DB

DB DB

DB DB

Cache

App

Goodlatency

But …

Inconsistency

DB DB

DB DB

DB DB

Cache

App

InconsistencyStill have sharding

DB DB

DB DB

DB DB

Cache

App

InconsistencyCache does not optimize updates out!

DB DB

DB DB

DB DB

Cache

App

InconsistencyApp

Cache DB

Cache rebooted

Data’s lost

InconsistencyApp

CacheDB

Cache rebooted

Data’s lost

Cache- Inconsistency- Still need sharding- What else?

Cache is not a DB

File

Inconsistency

WHATTO

DO?

Smart cache

DB DB

DB DB

DB DB

SmartCache App

Goodlatency

AppSmartCache DB

Cache rebooted

OK

Well it’s better

Well it’s better, but …

Still inconsistency

Cache rebooted

Data lost

AppSmartCache DB

Still sharding

DB DB

DB DB

DB DB

SmartCache App

Remember, your boss …

… doesn’t like sharding

… because it’s way too expensive

Cache- Inconsistency- Still need sharding- No database properties- What else?

Cold start

Cold start

DB DB

DB DB

DB DB

Cache

ColdStartKillsThe

Cache

Cold start

DB

DB

DB

Cache

DB

DB

DB

DB

DB

DB

Only to warmup

cache

Cache- Inconsistency- Still need sharding- No database

properties- Cold start

How to warm it up?

How to warm it up?

Persistent cache

DiskMemory

Dump

Do you like it?

Persistent cache

DiskMemory

Dump

- IOPS- STALE DATA+ FAST WARMUP

How to fix it?

Persistent cache

DiskMemory

Log

+ IOPS+ FRESH DATA+ FAST WARMUP- BIGGER SIZE

100Mb/sec

There’s still a problem

Persistent cache

DiskMemory

Dump & Log

+ IOPS+ FRESH DATA+ FAST WARMUP+ OPTIMAL SIZE

Solved!

Cache is warmed up

Cache

Cache

Why use a DBMS at all?

Why use a DBMS at all?• Durability• Replication

Why use a DBMS at all?• Durability. What does a

DB store?• Replication

Why use a DBMS at all?• Durability. What does a

DB store? Hot & Cold data

• Replication

Temperature of data

9.9Tb100Gb

Temperature of data

9.9Tb

100Gb

9.9Tb9.9Tb

9.9Tb9.9Tb

9.9Tb9.9Tb

9.9Tb9.9Tb

9.9Tb9.9Tb

Shard

Replicate

Why do we copy a DB?

Temperature of data

9.9Tb

100Gb

Shard

Replicate

100Gb100Gb

100Gb100Gb

100Gb100Gb

The boss’s still angry

Ideas?

Why use a DBMS at all?• Durability. What does a

DB store? Well, cache is also durable now

• Replication

An idea!CACHEIS THE

PRIMARYDATA SOURCE

An idea!CACHEIS THE

PRIMARYDATA SOURCE

With replication

An idea!CACHEIS THE

PRIMARYDATA SOURCE

With replicationAnd with all other DB features

Cache

Cache

Cache is not a DB

File

Cache can be a DB!

Cache

CanBeA

DATABASE

+ CACHE+ PERSISTENCE+ ACID TRANSACTIONS+ REPLICATION+ STORED PROCEDURES

Database

Tarantool – for hot data+ CACHE+ PERSISTENCE+ ACID TRANSACTIONS+ REPLICATION+ STORED PROCEDURES (LUA)

Database

Tarantool

We’ve done it!

9.9Tb

100Gb

9.9Tb9.9Tb

9.9Tb9.9Tb

9.9Tb9.9Tb

9.9Tb9.9Tb

9.9Tb9.9Tb

Shard

Replicate

We’ve done it!

9.9Tb

100Gb

SQL DBMS

Tarantool

Look back• Shard, replica, cache, smart

cache• Shard exists, replica back,• Persistent cache, replication,

acid. Now cache is a DB• 2 instances could be

enough

Cache. Why not?• Dependable, durable• You’re on the road to cache• No ACID, no consistency, no

durability• Leave cozy DB for cache• Tarantool helps come over• No magic, just a new tool

Tarantool in Mail.Ru

Mail.Ru Users’ Profiles

Profiles MySQLMySQL

MySQLMySQL

MySQLMySQL

16x1/8 of workload

Mail.Ru Users’ Profiles

$1.2M upfront$300K each year

128x MySQL 4x TarantoolVS

$20K upfront$5K each year

60 times!!!

Saved more than $1 000 000

Mail.Ru Email & Cloud

$100M+

Tarantool120+ instances of

The bottom line is …

Right tool for the job

Saved $1 000 000 on a feature

Summary

Questions?Денис Аникин,техдир Почты/Облака Mail.Ru

anikin@corp.mail.ruSkype: danikin2Whatsapp/Viber: +79165129650