mysql to cassandra: big data, high scale, data migration... oh my! scott bonneau, cto & evp of...

24
MySQL to Cassandra: Big Data, High Scale, Data Migration….Oh My! Scott Bonneau – CTO & EVP of Engineering RC Johnson – NYC Engineering Manager

Upload: -eric-david-benari-pmp

Post on 10-May-2015

355 views

Category:

Technology


0 download

DESCRIPTION

http://www.DatabaseMonth.com/database/bazaarvoice-cassandra-mysql

TRANSCRIPT

Page 1: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

MySQL to Cassandra: Big Data, High Scale, Data Migration….Oh My!Scott Bonneau – CTO & EVP of EngineeringRC Johnson – NYC Engineering Manager

Page 2: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Scott Bonneau

Page 3: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

RC Johnson

Page 4: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc. 4

Nobody reads ads.

Page 5: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice
Page 6: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

How we do it

6

Page 7: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

How it works

7

Page 8: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Over 2,500 Brands

Page 9: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Scale is our middle name

9

Page 10: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Technologies

10

Page 11: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice
Page 12: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Bazaarvoice History

Replication

WritesWrites

ReadsReads

ApplicationServer

ApplicationServer

Slave DBSlave DB Slave DBSlave DBSolr

IndexMaster

DBMaster

DB

Page 13: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Cracks in the foundation

Page 14: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Goals & Chaos

Page 15: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

I’ll take 4 Cassandra Rings & and Elastic Search to go.

Page 16: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Breaking Emo apart

Page 17: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Conditional writes & Eventual Consistency

{ "product": "Sceptre 32\" LCD", "rating": 5, "text": "Very nice TV great picture. Very Very light amazing!",}

{ "product": "Sceptre 32\" LCD", "rating": 5, "text": "Very nice TV great picture. Very Very light amazing!",}

t1

{ .., "status": "APPROVED"}

{ .., "status": "APPROVED"}

t2{ .., "facebookId": 387075234674416}

{ .., "facebookId": 387075234674416}

t3

Page 18: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

SQL NoSQL

Page 19: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Page 20: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

It was the best of times, it was the other time

Page 21: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Rules engine & Vocabulary

Page 22: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Gotchas we’ve found so far

Page 23: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Confidential and Proprietary. © 2013 Bazaarvoice, Inc.

Questions/Thoughts

Page 24: MySQL to Cassandra: Big Data, High Scale, Data Migration... Oh My! Scott Bonneau, CTO & EVP of Engineering, Bazaarvoice

Shameless plug

@bazaarvoice