benchmark showdown: which relational database is the fastest on aws?

Enterprise Strategy Group | Getting to the bigger truth.™

© 2016 by The Enterprise Strategy Group, Inc.

Mike Leone, Senior ESG Lab Analyst

ESG Lab WebinarValidating ClustrixDB Performance in AWS

About Enterprise Strategy Group• ESG is an IT analyst, research, and strategy company.• Our firm was founded in 1999 with headquarters in Milford, MA / an analyst and client

relations presence in Silicon Valley, CA.• ESG conducts research with and for IT vendors, IT professionals, business professionals,

and channel partners.• We maintain ongoing analyst coverage in cloud computing, networking, storage, data

protection, cybersecurity, data management and analytics, application development and deployment, enterprise mobility, and channels.

• Capabilities include: Analyst services, market research, technical and economic validation, consulting, and custom content.• Mike Leone is a Senior ESG Lab Analyst with coverage across the entire IT industry.

• Provide in-depth testing and analysis of IT technology and products, using methods that simulate or recreate real-world environments.

• Extensive background in performance modeling, testing, and analysis.


Goals of this Webinar

Cloud Usage Trends and ChallengesWhat are the benefits of the cloud?How many people use it?What challenges exist when moving a traditional RDBMS to the cloud?Product Overview: ClustrixDBWhat is it? How is it different? How does it work?ESG Lab Performance ValidationGoals of the validation – comparing ClustrixDB performanceConfiguration, methodology, workload, etc.Performance analysis

– Single instance database comparison– ClustrixDB scale-out performance

ClustrixDB Performance TestingAdditional performance test comparisons and results


Benefits of the Cloud

Flexibility

Reliability

Cost-savings

Accessibility Manageability


Public Cloud Usage Trends

Platform-as-a-service (PaaS)

Infrastructure-as-a-service (IaaS)

Software-as-a-service (SaaS)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

33%

38%

65%

10%

13%

24%

22%

17%

16%

15%

9%

15%

11%

7%

3%

1%

1%

Please indicate your organization’s usage of or plans for each of the following cloud computing services. (Percent of respondents, N=633)

Currently use Do not currently use, but we have done so within the past two years

Do not currently use but we plan to No use or plans at this time but we are interested

No use, plans, or interest at this time Don’t know


None of the above

Hadoop

Business intelligence

Analytics

Data warehouses

Spark

Databases

4%

34%

38%

39%

41%

43%

43%

For which of the following are you considering public cloud services? (Percent of respondents, N=475, multiple responses accepted)

Big Data in the Cloud


Traditional SQL Database Challenges in the Cloud

ACID compliance is a requirement• Mission critical database applications serve as the lifeline of the business• Delivering high levels of performance is essentialDatabase Performance is a major challenge – how do you address it?1. Increase horsepower

– Database migration– Adds risk and potential for delays– Direct impact on the business

2. Workarounds– Just as expensive as #1– More people, resources, and complexity

3. Read slaves – Band-aid approach– Only work for so long– Workload dependentNone of these options are

futureproof

ClustrixDB Overview

Dave AnselmiDirector of Product Management, Clustrix

ClustrixDB: Scale-Out, Fault-tolerant, MySQL-Compatible

ClustrixDB Overview9

ClustrixDB

ACID Complaint

Transactions & Joins

Optimized for OLTP

Built-In Fault Tolerance

Flex-Up and Flex-Down

Minimal DB Admin

Also runs great inthe Data Center

Built to runin the Cloud

ClustrixDB Technical Overview

Fully Distributed & Consistent Cluster• Fully Consistent, and ACID-compliant database

– Supports transactions– Support joins– Optimized for OLTP– But also supports reporting SQL

• All nodes are equal (no “special” node)• All servers are read/write• All servers accept client connections• Tables & Indexes distributed across all nodes

– Fully automatic distribution, re-balancing& re-protection


Private N

etwork

ClustrixDB on commodity/cloud servers

HW or SW Load Balancer

SQL-based Applications

High Concurrency

Custom:PHP, Java, Ruby, etc

Packaged: Magento, etc

Bill

ions

of R

ows

DatabaseTables

S1 S2S2S3

S3S4S4

S5S5

Intelligent Data Distribution

• Tables auto-split into slices • Every slice has a replica on another node

– Slices are auto distributed, auto-protected


S1

ClustrixDB

S1

S2

S3

S3

S4

S4

S5

Database Capacity

• Easy and simple Flex Up (and Flex Down)– Single minimal interruption of service

• All servers handle writes + reads• Data is automatically rebalanced

across the cluster


S1

ClustrixDB

S2

S5

S1

S2

S3

S3

S4

S4

S5

Built-in Fault Tolerance

• Server node goes down…– Data is automatically rebalanced across

the remaining nodes

• Simply Add new Node – System automatically re-protects– Data is automatically redistributed


S1

ClustrixDB

S2

S5

ClustrixDB Rebalancer: Making the Complex, Simple

Q: How do you ensure data stays well distributed in a clustered environment?

A: You let the Rebalancer handle it!

The Rebalancer automatically:

• Initial Data: Distributes the data into even slices across nodes

• Data Growth: Splits large slices into smaller slices

• Flex-Up/Flex-Down: Moves slices to leverage new nodes and/or evacuate nodes

• Failed Nodes: Re-protects slices to ensure proper replicas exist

• Skewed Data: Re-distributes the data to even out across nodes

• Hotness Detection: Finds hot slices and balances then across nodes

…while the DB stays open for businessPatent 8,543,538Patent 8,554,726

ESG Lab Evaluation


• Same database size configured with the same scripts

• Similar test bed configuration – server configuration differed due to the distributed ClustrixDB architecture

Goals of the ESG Lab Validation

Validate ClustrixDB as a leading cloud database in AWSfor high-value, high-transaction workloads

• Identified a common, real-world OLTP workload

• Compared ClustrixDB to two competing cloud database offerings (referred to as CloudDB1 and CloudDB2)

• Same infrastructure pushing the workload, same time period, same scripts

• Measured the same performance metrics


Performance Test Bed

CloudDB1 and CloudDB2

One r3.8xlarge AWS Instance• 32 cores• 244GB of RAM• 2 x 320GB SSDs

WorkloadDrivers

sysbench

LOAD BALANCER

Four c3.2xlarge AWS Instances• 8 cores• 15GB of RAM• 2 x 80GB SSDs

One database instance with one tableconsisting of 40,000,000 records (20GB)


Transactions/sec• How busy is your database server?• Measures the database activity by tracking the number of serviced requests

Performance Metrics and Scaling the Workload

Average Transaction Latency• How is the end-user experience?• Industry-defined threshold of 20ms• Low, predictable latency tends be more valuable

Performance Curves• Scale-up workload by doubling the concurrent thread count of each test• Increase transactions/sec, but also increase latency


Comparing Single Instance OLTP Workload Performance

20 40 80 160 320 640 12800

2,000

4,000

6,000

8,000

10,000

CloudDB1 CloudDB2 ClustrixDB (4 nodes)Number of concurrent threads

Tran

sacti

ons/

sec


Comparing Scale-out Performance

20 40 80 160 320 640 1280 2560 51200

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

CloudDB1 CloudDB2 ClustrixDB (4 nodes) 8 nodes 12 nodes 16 nodes 20 nodesNumber of concurrent threads

Tran

sacti

ons/

sec


Near-linear Performance Scalability with ClustrixDB

4 8 12 16 200

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

50,000

0

20

40

60

80

100

Number of ClustrixDB Nodes

Tran

sacti

ons/

sec

Aver

age

Tran

sacti

on L

aten

cy (m

s)


Clustrix – Making a Representative Benchmark

Peter FriedenbachPerformance Architect, Clustrix

Making a Representative Benchmark

PROPRIETARY AND CONFIDENTIAL23

Why did we choose the workload that we did?• Other venders recently have published performance results with Sysbench.• Things that we think matter:

– Read/Write Mix Matters: Real workloads are a mixture of reads and writes.– Data Size Matters: OLTP databases, while not “big data”, still have record counts of 10 million

or more.– Latency Matters: OLTP workloads demand low latency.

• The ClustrixDB Sysbench Performance Benchmark– Mixed workload: 90% read and 10% writes– Size: 40m records / 20 GB file– Latency: 20ms is the upper range of “acceptable” latency (>3X the initial latency).

• Methodology: Performance curves show a more complete story.– Reveals the “capacity point of the machine”

• The “knee of the curve” where throughput gains succumb to growth in latency.

ClustrixDB Sysbench Benchmark


ClustrixDB Key Differentiators• MySQL Compatible, Scale-out ACID RDBMS

• Flex-Up and Flex-Down – Flex Licensing allows paying for only the instances you need– Add or remove multiple nodes at a time

• Massively Scaling Writes– ClustrixDB is the only cloud RDBMS that can massively scale

writes in addition to reads– E-commerce, AdTech, IoT are very write-intensive. Clustrix

has multiple satisfied customers in these sectors

• Built for Cloud (any cloud) and Datacenter– One RDBMS wherever the application is deployed


Thank You

Enterprise Strategy Group | Getting to the bigger truth.™

http://www.twitter.com/esg-global

http://www.facebook.com/ESGglobal

https://www.linkedin.com/groups?gid=1295607&trk=myg_ugrp_ovr

http://www.youtube.com/user/ESGglobal

FOLLOW ESG


Mike Leone, Senior ESG Lab AnalystE-mail: [email protected] Office: 508-244-4814

benchmark showdown: which relational database is the fastest on aws?

Technology