ficstar software: cassandra installation to optimization

Cassandra Installation to Optimization40-minutes of best practice and resources

@VictorFAnjos

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

Welcome to

Who Wants to perform 1M ops/s

50:50

@VictorFAnjos

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos

5© 2015. All Rights Reserved.

A: NAS / SAN

C: DAS SATA

B: SSD

D: DAS SCSI

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

This storage medium allows for best performance.

@VictorFAnjos


A: NAS / SAN

C: DAS SATA

B: SSD

D: DAS SCSI

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos

This storage medium allows for best performance.

Installation and considerations

how to store the datastore

Storage Area Network Solid State Drive




Local (DAS), iSCSI, Fiber Channel


● AVOID network storage like the plague

● Direct Attached Storage FTW

● Disk latency is a HUGE deal for performance



9© 2015. All Rights Reserved. 9@VictorFAnjos

SATA/SAS DAS

PCIe/NVMe DAS

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: ZFS

C: Ext4

B: Btrfs

D: F2FS

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

When using SSDs, this filesystem type

is best.

@VictorFAnjos

Congratulations!

You’ve Reachedthe 1,000 ops/s

Milestone!

Congratulations!Congratulations!@VictorFAnjos


i can’t believe it’s not btrfs


● easiest to use ext4 (it’s on most linux distros), but F2FS get 5-10% gains in write performance

● if NOT using F2FS, make sure to TRIM

● multiple disks → use RAID0

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: 0

C: Equal to HEAP

B: ½ of HEAP

D: EQUAL TO RAM

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

This is the sweetspot for SWAP

when using C*

@VictorFAnjos


to swap or not to swap


151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: 64G

C: 16G

B: 32G

D: 8G

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

Having 64G of RAM means you should optimize to have ___G of HEAP.

@VictorFAnjos


A: 64G

C: 16G

B: 32G

D: 8G

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos

Having 64G of RAM means you should optimize to have ___G of HEAP.


how much heap?


http://docs.datastax.com/en/cassandra/1.2/cassandra/operations/ops_tune_jvm_c.html

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: EC2Snitch

C: Simple Snitch

B: Dynamic Snitch

D: Property File Snitch

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

Definitely DO NOT use this snitch in

Multi-DC environments.

@VictorFAnjos


A: EC2Snitch

C: Simple Snitch

B: Dynamic Snitch

D: Property File Snitch

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos

Definitely DO NOT use this snitch in

Multi-DC environments.


son of a snitch


151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: Synchronous AND Full Queries

C: Synchronous AND Prepared Statements

B: Asynchronous AND Prepared Statements

D: Asynchronous AND Full Queries

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

To reduce latency and wire time to my

app, I should opt for.

@VictorFAnjos

Achieving performance through code/drivers

should I stay or should I go


● Client writes to any Cassandra node

● Coordinator node replicates to other nodes (in local and remote Data Center)

● Local write acks returned to coordinator

● Client gets ack when enough total nodes are committed

● Data written to internal commit log disks

● When data arrives, remote node replicates data

MULTI DC

● Ack direct to source region coordinator

● Remote copies written to commit log disks

lf a node or region goes offline, hinted handoff completes the write when the node comes back up (as long as there are enough nodes to satisfy consistency level).




Prepare ONCE...

Bind and Execute multiple times.

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: 1 / 1 = 1

C: 2 * 1 = 2

B: 2 / 1 = 2

D: 2 / 2 + 1 = 2

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

With RF=2 and CL=Quorum, operations failed when 1 node went down because of this.

@VictorFAnjos

Congratulations!

You’ve Reachedthe 32,000 ops/s

Milestone!

Congratulations!Congratulations!@VictorFAnjos


when friends aren’t enough


Replication Factor = 3

Insert into a cluster of size 6 with consistency Quorum

Two nodes in token range must be present for write to succeed


when friends aren’t enough


What happens now?

Cannot achieve consistency level QUORUM




Nodes in partition key DOWN

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: Truth table

C: CAP Theorem

B: Brewer’s Theorem

D: Entropy

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

This mathematical and CS concept helps when data modeling for query

optimization.

@VictorFAnjos


A: Truth table

C: CAP Theorem

B: Brewer’s Theorem

D: Entropy

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos

This mathematical and CS concept helps when data modeling for query

optimization.

Data modelling, CQLSH and more

the truth shall set you free


Motivated by CS, Math, Engineering

Allows for creating building blocks that yield a single output

More complex truth tables can arise


the truth shall set you free


How about searching for username?

And what about full_name?

user_stream

← ← ← Partition Key → → → user_id username full_name

1 0 00 1 00 0 1

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: Reads / Batches

C: Writes / Deletes

B: Writes / Batches

D: Reads / Deletes

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

A shift in paradigms, what should you

maximize and reduce for good performance.

@VictorFAnjos


A: Reads / Batches

C: Writes / Deletes

B: Writes / Batches

D: Reads / Deletes

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos

A shift in paradigms, what should you

maximize and reduce for good performance.


do the write thing



do the write thing


memtable --- < 100ns

commit log --- ~ 1 ms

DELETES / TTL cause compactions


do the write thing


151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: ACID

C: Rollback

B: Vector

D: Sharding

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

To not hit a 2B record limit (per row), this

RDBMS borrowed term can still makes sense.

@VictorFAnjos


sit on this and rotate


151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: Batches

C: Secondary Indexes

B: Synchronous

D: MySQL

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

Many say to use sparingly, I would say, avoid like the plague.

@VictorFAnjos


A: Batches

C: Secondary Indexes

B: Synchronous

D: MySQL

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos

Many say to use sparingly, I would say, avoid like the plague.

Performance must-haves

never be second best


writes are distributed among the cluster

each partition key refers to one exact position in which to get a row

but what do we do when we don’t have exactly the right type of index to specify a queryCREATE TABLE users ( user varchar, email varchar, state varchar, PRIMARY KEY (user));-- OPTION 1 : create an indexCREATE INDEX idxUBS on users (state);

-- OPTION 2 : create another table (store data twice)CREATE TABLE usersByState ( state varchar, user varchar, PRIMARY KEY (state, user));

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

@VictorFAnjos


A: UDT

C: JSON

B: Lightweight Transactions

D: Hinted handoff

50:50

151413121110987654321

1 Million500,000250,000125,00064,00032,00016,0008,0004,0002,0001,000500300200100

This recent addition to C* now helps with ACID like transactions, at a bit

of a performance hit.

@VictorFAnjos


slimfast agreement


Prepares a proposal that is sent to a number of Acceptors.Waits on a an acknowledgement (in form of promise) from Acceptors.Sends accept message to Quorum of Acceptors with new value to commit.Returns success? completion to client.

Determines if proposal is newer than what it has seen.Acknowledges/agree with its own highest proposal value seen AND the current value (of what is to be set).Receive message to commit new value.Accept and return on successful commit of value.



slimfast agreement

Thank you

ficstar software: cassandra installation to optimization

Technology