lightning talks percona live mysql_2012

Lightning talksGiuseppe MaxiaContinuent, Inc

April 11, 2012

Wednesday, April 11, 12

LineupStewart Smith The 5 minutes tour of Drizzle

Jeff Mace What can possibly go wrong?

Nicolai Plum MAKE.DATABASES.FAST

Bill Karwin Schemadoc

Robert Hodges It's the application, stupid!

Robert Buck NuoDB overview

David Holoboff Diagnosis and possible fixes

Lawrence Schwartz OLTP and OLAP

Baron Schwartz Building a scalable system

Sheeri Cabral How to become a rock star DBA

Giuseppe Maxia Cruelty on database clusters


The 5 minutes tour of DrizzleStewart Smith


2008...


http://www.flickr.com/photos/shaylor/53819053/




2011...


Drizzle7


2012...


Drizzle7.1


What is Drizzle?


Modular architecture


Pluggable


Infrastructure Aware


Community Developed


Multicore


Concurrency


Web focus


UTF-8


EBCDIC!=


Enable others


Modern codebase


C++


STL


Boost


Nolinked-list innovation


Collaboration


Participation


Build businesses


Support & Consulting


Testing


Transactional


Logging


Replication


Multi-sourceReplication


Auth


LDAP


PAM


In database tables


No bad data


strict_mode


No GOTCHAs


No implicit COMMIT


BSD Client Libraries


New in 7.1


Multi-source Replication


XtraBackup


IPv6 data type


ZeroMQ


Syslog is default


query_log plugin


Auth from tables


Server-side Javascript


HTTP JSON API


Percona InnoDB Patches


What could possibly go wrong?

Jeff Mace


©Continuent 2012.

What Could Possibly Go Wrong?

Je! Mace


©Continuent 2012 59

http://mingle2.com/blog/view/web-developer-mind





http://www.flickr.com/photos/brhefele/4026340146/





http://www.flickr.com/photos/themoly/4336260658/




©Continuent 2012

What can I possibly do?

• Relax

• Have a plan

• Keep it simple

63



http://www.flickr.com/photos/devenlaney/5767271943




MAKE.DATABASES.FASTNicolai Plum


Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/offices

MAKE.DATABASES.FAST

Nicolai Plum



The business

• Booking.com–Leading online hotel reservation agency–Part of the Priceline Group of Companies–Sustained rapid business growth–All powered by MySQL



The challenge



The problem…

• Hundreds of database servers• Many replication slaves• Few DBAs, some more sysadmins• Need to make tens of servers each day



If you do this manuallyYou are doing it wrong



Among our weapons are…

• Network OS installation• Puppet configuration management• Database cloning• and a fanatical devotion to automation!



Server installation

• Server Database• PXE netboot• CentOS kickstart



System configuration

• Now we have a server that can run Puppet• Puppet does

–OS setup–Software installation–LVM setup and filesystem creation (tailored by role)

• Prefer ongoing configuration management tools to single perfect installation tools



Database cloning

• Now we have a server ready to run a database• Copying databases based on LVM snapshots• create_clone program

–Copy via SSH and rsync–LVM snapshot–Fix my.cnf (server id!) automatically



Things we learned

• Waiting for actions to complete takes a lot of person time – automate batches of work as much as possible

• You really need a Server Database, and drive everything from it

• Most hardware can’t support being a clone source and in service, you need spares

• Online snapshots break in subtle ways; best stop the database.



Things we learned

DBAs are toolsmiths, not database installers



Thank you

• … if you like the way we work, we are hiring• http://www.booking.com/jobs


SchemadocBill Karwin


SchemadocBill Karwin, Percona Inc.


www.percona.com

Schema Can Be Unclear

CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, Name VARCHAR(100), Status VARCHAR(20), EntityNum INT, Order INT);


www.percona.com



whose name?


www.percona.com



whose name?

what are the values?


www.percona.com



whose name?


what’s an entity?what’s a number?


www.percona.com



whose name?


what’s an entity?what’s a number?

commerce, or ordinal?


www.percona.com

Comments!

CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, Name VARCHAR(100) COMMENT ‘person who placed the order’, Status VARCHAR(20) COMMENT ‘new, open, or closed’, EntityNum INT COMMENT ‘how many items in the order’, Order INT COMMENT ‘reference to the Orders table’) COMMENT ‘any change to e-commerce orders’;


www.percona.com

Length

• Limits increased in MySQL 5.5.3:• Per column: 1024 characters• Per index: 1024 characters• Per table: 2048 characters• Per partition: 80 characters

(MySQL 5.6 increases this to 1024)


www.percona.com

“Is there a tool for MySQL like javadoc for Java code?”


www.percona.com

developmentdba

documentation


www.percona.com

schemadoc


www.percona.com

Usage

$ schemadoc -v h=dbserver,D=EcommerceDB,u=root,p=xxxxLoading metadata...Running online analysis...done.Outputting to html.Rendering pages for tables. done.Rendering pages for views done.Rendering pages for procedures done.Rendering pages for triggers done.Rendering page for schema summary... done.Rendering page for schema link... done.Rendering page for deprecated-list... done.Rendering page for all-index... done.Rendering page for help-doc... done.Rendering page for allschema-frame... done.Rendering page for index... done.


www.percona.com

hooray.


www.percona.com

But Wait, There’s More

•Parses SQL dump file•Reads from a live MySQL instance•Document many schemas•Update one schema at a time


www.percona.com

Schema Analysis

•Number of indexes and columns per data type•Columns with same name but different type•ENUM columns that aren’t NOT NULL•Use of FLOAT or DOUBLE• Tables with one index per column• Tables with one index over all columns• Tables with no indexes• INT(N) other than default N• Lack of INT UNSIGNED columns


www.percona.com

Schema Analysis

• Tables that look like Entity-Attribute-Value• Tables that look like Polymorphic Associations•Superfluous primary key•Report table with least / most indexes• IP addresses stored in VARCHAR•Excessive use of VARCHAR(255),

report actual max string length• ...And supports plugins for other checks!


www.percona.com

yes, we get it.


www.percona.com

Future of Schemadoc

• Integrate with Percona Toolkit library•Read extra metadata (foreign keys)•Show table sizes and growth rates•Show SQL privileges (who has access?)•CLI for adding / updating comments•Batch-mode output•Redesign HTML output with modern look & feel


www.percona.com

Future of Schemadoc


www.percona.com

launchpad.net/schemadoc


It's the application, stupid!Robert Hodges


©Continuent 2012.

It’s the Application, Stupid!

(Or: Multi-Master Enlightenment in 5 Minutes or Less)

Robert Hodges, Continuent CEO



Multi-Master Replication Is This Easy

tungsten-sandbox-2.0.4 -m 5.1.54 \ -i $RELDIR/tungsten-replicator-2.0.5 \ -n 3 --topology=star --hub=2

HubSpoke Spoke



Multi-Master Apps Not So Much

Airline Reservations



HubSpoke Spoke



Who Is in Which Seat on Flight 3523?

Book Bobin Seat 11E




HubSpoke Spoke

Book Suein Seat 11E

Bump Bob from Flight 3523



Rules for Multi-Master Apps

1. Isolate destructive changes to a single master to avoid harmful con!icts

➡ “Change gate for !ight 3523 to C28”

2. Use non-destructive INSERTs for everything else

➡ “Please assign Sue to seat 11A”



Rules for Multi-Master Apps

3. Coordinate across masters to aggregate results

➡ “Generate "nal passenger manifest for !ight 3523”

4. Be prepared to apologize

➡ “Add 5000 frequent !yer miles to Bob’s account”



Multi-Master Enlightenment

Scalable multi-master applications are eventually consistent

1. Think through the whole problem: Business processes are often eventually consistent already

2. Follow design rules for eventually consistent applications

3. Use appropriate replication technology for MySQL*

* As in Tungsten :)


NuoDB overviewRobert Buck


Copyright © 2012 NuoDB

The Elastically Scalable Database™


Copyright © 2012 NuoDB 107

‣ SQL/ACID: Emergent Architecture

‣ Elastically Scale: Demand-driven, auto-balancing

‣ Single Logical Database: No sharding, no partitioning

‣ Run Anywhere: On-Premises / Private, Public, or Hybrid Cloud

‣ Store Anywhere: KV-Store backend (flexible, e.g. Files, S3, HDFS)

‣ Multi-Tenant: DBs share machines dynamically

‣ Geo-Distributed: Active/active across several data centers

‣ Extreme Availability: Massively redundant, permitting live upgrade

‣ Minimal Admin: No knobs, auto everything

NuoDB Summary



Emergent Database Architecture

“An emergent behavior can

appear when a number of simple agents operate in an environment,

forming more complex behaviors

as a collective.”

- Wikipedia

108


http://en.wikipedia.org/wiki/Entity

http://en.wikipedia.org/wiki/Entity


Instantly Elastic

Second Machine Instant Performance Increase

‣ Second machine more than doubles TPS due to caching effects

‣ Second machine is added to live database while it is running at 1,000’s of TPS

‣ Performance increase is immediate

‣ BTW - you can take either machine away and the database keeps running without data loss

109



Number of Nodes

TPS

Linearly Scalable

110

Technical Details:‣ 2-9 Tx engines‣ 1 storage manager‣ Best sustained TPS and #

clients combination‣ 50% updates

Nodes TPS

MySQL 1 3,000

NuoDB 1 4,500

NuoDB 9 27,000

NuoDB running on 9 nodes was approx. 9x faster than MySQL running on 1 node.

0"

5000"

10000"

15000"

20000"

25000"

30000"

1" 2" 3" 4" 5" 6" 7" 8" 9"



... even on Amazon EC2

111

0"

2000"

4000"

6000"

8000"

10000"

12000"

1" 2" 3" 4" 5" 6" 7" 8" 9"

Number of EC2 Nodes

TPS

‣ NuoDB scales linearly on EC2

‣ EC2 performance bound by AMZ TCP/IP traffic shaping

‣ Linearly scales at 50% of rate for physical commodity hardware

‣ Whereas, Amazon RDS runs on 1 node, and gets overloaded at 10+ connections



We Work with Your SQL Tools

SQuirreLSQL

MS Excel (and other MS tools)

DbVisualizer

112



NuoDB System Architecture

TM1

Application(JDBC, ODBC, SQL) NuoDB Storage

ManagerNuoDB Transaction

Engines

NuoDB Brokers

Tx Scalability Add as many of B, TE, or SM as you like

I/O Scalability Distributed IO, e.g. Hadoop HDFS

100% Uptime No single point of failure, geo-distributed

Optimal Utilization Multiple Databases on finite resources

Low People Costs Single Console Management

B1

TM1TM1TM1TM1TM1TM1TM1TM1TM1TM1TM1TM1TM1TM1TE

B1B1B

SM

SM

SM

113


Copyright © 2012 NuoDB 114

‣ 100% SQL

‣ 100% ACID

‣ 100% Elastic Scaling

‣ If you have more questions...

‣ Prove it to yourself. Try it out!

‣ Download it at: www.nuodb.com

‣ Chat with us afterward

NuoDB Summary


http://www.nuodb.com

http://www.nuodb.com


NuoDBThe Elastically Scalable Database™


Diagnosis and possible fixes

David Holoboff


DIAGNOSIS ANDPOSSIBLE FIXES:

A MULTIPLE CHOICE PRIMERFOR BEGINNING DBAS


This talk is for entertainment value only. Any perceived similarity between what you will hear in the next five minutes and true life scenarios are merely coincidental, with

no intent of harm or liability.


Are you being asked the right question?


What do you do with a slow database server?


What do you do if swap memory is always being used up?


OLTP and OLAPLawrence Schwartz


OLTP and OLAPHave Your Cake and Eat it Too!

Lawrence Schwartz


Avoiding the B-tree Valley of Death

Read-intensive Write-intensive

Usa

ge

OLAP OLTP

B-treeValley of

Death

Columnar DBs HadoopMongoDB

However, Today’s database categories - OLAP, OLTP are compromises to get around dated, rigid indexing technology

“The real end-game for Big Data is to have transactional and analytic data on the same database.”


B-trees Perform Poorly on Disk

• Random Workloads Lead To Frequent Drive Head Movement

• MySQL Can’t Take Advantage of Full Drive I/O Potential

MySQL + InnoDB

• Continuous Rebalancing and Aggregation at Nodes

• Eliminates Fragmentation, Maximizes Disk I/O

MySQL + TokuDBBased on B-Tree Indexes Based on Fractal Tree Indexes


B-trees Can’t Handle Big Data

• Performance Drops to a Trickle above 50-100M Rows

• Delayed Checkpoints Diminish Performance

• 20x Insertion Rates at One Billion Rows

• Stand Up More Indexes for Better Query Performance

MySQL + TokuDBMySQL + InnoDB


B-trees Aren’t Agile

• Big Database Schema Changes Take Hours To Complete

• Changes Have to Wait for Scheduled Downtime

• Changes Complete in Seconds

• Allows Business' Data Models to Immediately Adapt



B-trees Don’t Compress Well

• Compression ~2x to ~2.5x Typical

• Compression Impacts Performance

• High Compression

• No Impact to Performance



TokuDB Eliminates Death Valley

Read-intensive Write-intensive

Usa

ge“While TokuDB® is effectively an operational database technology, it does blur the lines between operations and analytics”

OLAP OLTP

Fractal Tree® Indexesare the Solution

eCommercePersonalization

Cloud Assets

Logfile analysis

SocialNetworks

Real Time OLAP

Analytic OLTP


Building a scalable systemBaron Schwartz


How to become a rock star DBA

Sheeri Cabral


How to Become a Rock Star DBA

Sheeri Cabral, MozillaOracle ACE Director

MySQL Community Award Winner^2


Do what you love....


What you really love


Have a Twin Brother


Code


Write

forums.mysql.com

Tutorials / HOW-TOs

MariaDB/Percona Documentation

#irc


Make MisteaksMistakes


Teach others


Teach others

You know more than you think you do!


Keep Learning


Share


Organize

User Group

UnconferenceIgnite Event

Magazine


Connect People


Avoid burnout


Have a hobby


Have Support


Be tall...or not


Find Ways Others Can Help You


Some Help You Can Get

MySQL Community TeamVolunteers – UG video: Richard Laskey“What can I help you do?”Technocation for grants - meetup.com fees - Open SQL/DB Camp - Postgres Open - Open Source Bridge - Drizzle Day

(donate to www.technocation.org!)


It Takes a Community

Permission to use pictures granted by:

Ronald BradfordFrancesca Simon-Carboni PhotographyColin CharlesJeremy ColeJames Duncan DavidsonKeith LarsonGerry Narvaja


Cruelty on database clustersGiuseppe Maxia


©Continuent 2012.

Cruel things that you can inflict on your replicated

cluster ... and get away with it!Giuseppe MaxiaContinuent, Inc


©Continuent 2012

I have a criminal mindset

• Like a bank robber

• or an airplane hijacker

• I look around, and watch for weak spots to exploit

189


©Continuent 2012

Have you ever reported a bug ...

• about a failure that happens after 9000 switches?

• Or a master that slows down when handling 512 slaves?

195


©Continuent 2012

Have you ever reported a bug ...

• about a failure that happens after 9000 switches?

• Or a master that slows down when handling 512 slaves?

195

I DIDWednesday, April 11, 12

©Continuent 2012





• running the installer in a system with only 200 MB of free storage

• Counting the records after a performance test (and "nding that some are missing)

196


©Continuent 2012


• Because, most of the time:

• Developers are not users

• Users do what developers can't imagine

197


©Continuent 2012


• Because, most of the time:

• Developers are not users

• Users do what developers can't imagine

•To make a fool-proof system you need to act like a fool

197


lightning talks percona live mysql_2012

Technology

mace continuent

mind continuent

simple continuent

http jsonapiwednesday

database clusterswednesday

possiblygo wrong

tens of servers

fastnicolai plumwednesday