introduction to cassandra architecture
TRANSCRIPT
![Page 1: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/1.jpg)
Nick Bailey@nickmbailey
Intro to Cassandra Architecture
1
![Page 2: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/2.jpg)
4.1 Cassandra - Introduction
![Page 3: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/3.jpg)
Why does Cassandra Exist?
![Page 4: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/4.jpg)
Dynamo Paper(2007)•How do we build a data store that is:• Reliable• Performant• “Always On”•Nothing new and shiny• 24 papers cited
Also the basis for Riak and Voldemort
![Page 5: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/5.jpg)
BigTable(2006)
•Richer data model• 1 key. Lots of values• Fast sequential access• 38 Papers cited
![Page 6: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/6.jpg)
Cassandra(2008)•Distributed features of Dynamo•Data Model and storage from
BigTable• February 17, 2010 it graduated to
a top-level Apache project
![Page 7: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/7.jpg)
Cassandra - More than one server
• All nodes participate in a cluster• Shared nothing• Add or remove as needed•More capacity? Add a server
7
![Page 8: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/8.jpg)
8
Cassandra HBase Redis MySQL
THR
OU
GH
PUT
OPS
/SEC
)
VLDB benchmark
![Page 9: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/9.jpg)
Cassandra - Fully Replicated
• Client writes local• Data syncs across WAN• Replication per Data Center
9
![Page 10: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/10.jpg)
Cassandra for Applications
APACHE
CASSANDRA
![Page 11: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/11.jpg)
Summary
•The evolution of the internet and online data created new problems
•Apache Cassandra was based on a variety of technologies to solve these problems
•The goals of Apache Cassandra are all about staying online and performant
•Apache Cassandra is a database best used for applications, close to your users
![Page 12: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/12.jpg)
4.1.2 Cassandra - Basic Architecture
![Page 13: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/13.jpg)
Row
Column 1
Partition Key 1
Column 2
Column 3
Column 4
![Page 14: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/14.jpg)
Partition
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
![Page 15: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/15.jpg)
Partition with Clustering
Cluster 1
Partition Key 1
Column 1
Column 2
Column 3
Cluster 2
Partition Key 1
Column 1
Column 2
Column 3
Cluster 3
Partition Key 1
Column 1
Column 2
Column 3
Cluster 4
Partition Key 1
Column 1
Column 2
Column 3
![Page 16: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/16.jpg)
Table Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 2
Column 2
Column 3
Column 4
Column 1
Column 2
Column 3
Column 4
Column 1
Column 2
Column 3
Column 4
Column 1
Column 2
Column 3
Column 4
Partition Key 2
Partition Key 2
Partition Key 2
![Page 17: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/17.jpg)
Keyspace
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 2
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 2
Column 2
Column 3
Column 4
Column 1
Partition Key 2
Column 2
Column 3
Column 4
Column 1
Partition Key 2
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 2
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 1
Column 2
Column 3
Column 4
Column 1
Partition Key 2
Column 2
Column 3
Column 4
Column 1
Partition Key 2
Column 2
Column 3
Column 4
Column 1
Partition Key 2
Column 2
Column 3
Column 4
Table 1 Table 2Keyspace 1
![Page 18: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/18.jpg)
NodeServer
![Page 19: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/19.jpg)
TokenServer•Each partition is a 128 bit value
•Consistent hash between 2-63 and 264
•Each node owns a range of those values
•The token is the beginning of that range to the next node’s token value
•Virtual Nodes break these down further
Data
Token Range
0 …
![Page 20: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/20.jpg)
The cluster Server
Token Range0 0-100
0-100
![Page 21: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/21.jpg)
The cluster Server
Token Range0 0-50
51 51-100
Server
0-50
51-100
![Page 22: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/22.jpg)
The cluster Server
Token Range0 0-2526 26-5051 51-7576 76-100
Server
ServerServer
0-25
76-100
26-5051-75
![Page 23: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/23.jpg)
Summary
•Tables store rows of data by column•Partitions are similar data grouped by a partition key•Keyspaces contain tables and are grouped by data center•Tokens show node placement in the range of cluster data
![Page 24: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/24.jpg)
4.1.3 Cassandra - Replication, High Availability and Multi-datacenter
![Page 25: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/25.jpg)
Replication10.0.0.
1
DC1
DC1: RF=1
Node Primary
10.0.0.1 00-25
10.0.0.2 26-5010.0.0.3 51-75
10.0.0.4 76-100
10.0.0.100-25
10.0.0.476-100
10.0.0.226-50
10.0.0.351-75
![Page 26: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/26.jpg)
Replication10.0.0.100-25
10.0.0.476-100
10.0.0.226-50
10.0.0.351-75
DC1
DC1: RF=2
Node Primary Replica
10.0.0.1 00-25 76-100
10.0.0.2 26-50 00-2510.0.0.3 51-75 26-50
10.0.0.4 76-100 51-75
76-100
00-25
26-50
51-75
![Page 27: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/27.jpg)
ReplicationDC1
DC1: RF=3
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.100-25
10.0.0.476-100
10.0.0.226-50
10.0.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
![Page 28: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/28.jpg)
ReplicationDC1
DC1: RF=3
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.100-25
10.0.0.476-100
10.0.0.226-50
10.0.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
Client
Write to partition 15
???
![Page 29: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/29.jpg)
Consistency levelConsistency Level Number of Nodes Acknowledged
One One - Read repair triggered
Local One One - Read repair in local DC
Quorum 51%
Local Quorum 51% in local DC
![Page 30: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/30.jpg)
ConsistencyDC1
DC1: RF=3
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.100-25
10.0.0.476-100
10.0.0.226-50
10.0.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
Client
Write to partition 15CL= One
![Page 31: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/31.jpg)
ConsistencyDC1
DC1: RF=3
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.100-25
10.0.0.476-100
10.0.0.226-50
10.0.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
Client
Write to partition 15CL= One
![Page 32: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/32.jpg)
ConsistencyDC1
DC1: RF=3
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.100-25
10.0.0.476-100
10.0.0.226-50
10.0.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
Client
Write to partition 15CL= Quorum
![Page 33: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/33.jpg)
Multi-datacenterDC1
DC1: RF=3Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.100-25
10.0.0.476-100
10.0.0.226-50
10.0.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
Client
Write to partition 15
DC2
10.1.0.100-25
10.1.0.476-100
10.1.0.226-50
10.1.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
DC2: RF=3
![Page 34: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/34.jpg)
Multi-datacenterDC1
DC1: RF=3Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.100-25
10.0.0.476-100
10.0.0.226-50
10.0.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
Client
Write to partition 15
DC2
10.1.0.100-25
10.1.0.476-100
10.1.0.226-50
10.1.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
DC2: RF=3
![Page 35: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/35.jpg)
Multi-datacenterDC1
DC1: RF=3Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.100-25
10.0.0.476-100
10.0.0.226-50
10.0.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
Client
Write to partition 15
DC2
10.1.0.100-25
10.1.0.476-100
10.1.0.226-50
10.1.0.351-75
76-10051-75
00-2576-100
26-5000-25
51-7526-50
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-10010.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
DC2: RF=3
![Page 36: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/36.jpg)
Summary
•Replication Factor indicates how many times your data is copied
•Consistency Level specifies how many replicas are consistent at read or write
•Replication along with Consistency Factor are critical for uptime
![Page 37: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/37.jpg)
4.2.1.1.3 Cassandra - Read and Write Path (Node Architecture)
![Page 38: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/38.jpg)
WritesCREATE TABLE raw_weather_data ( wsid text, year int, month int, day int, hour int, temperature double, dewpoint double, pressure double, wind_direction int, wind_speed double, sky_condition int, sky_condition_text text, one_hour_precip double, six_hour_precip double, PRIMARY KEY ((wsid), year, month, day, hour) ) WITH CLUSTERING ORDER BY (year DESC, month DESC, day DESC, hour DESC);
![Page 39: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/39.jpg)
WritesCREATE TABLE raw_weather_data ( wsid text, year int, month int, day int, hour int, temperature double, PRIMARY KEY ((wsid), year, month, day, hour) ) WITH CLUSTERING ORDER BY (year DESC, month DESC, day DESC, hour DESC);
INSERT INTO raw_weather_data(wsid,year,month,day,hour,temperature) VALUES (‘10010:99999’,2005,12,1,10,-5.6);
INSERT INTO raw_weather_data(wsid,year,month,day,hour,temperature) VALUES (‘10010:99999’,2005,12,1,9,-5.1);
INSERT INTO raw_weather_data(wsid,year,month,day,hour,temperature) VALUES (‘10010:99999’,2005,12,1,8,-4.9);
INSERT INTO raw_weather_data(wsid,year,month,day,hour,temperature) VALUES (‘10010:99999’,2005,12,1,7,-5.3);
![Page 40: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/40.jpg)
Write PathClient INSERT INTO raw_weather_data(wsid,year,month,day,hour,temperature)
VALUES (‘10010:99999’,2005,12,1,7,-5.3);
year 1wsid 1 month 1 day 1 hour 1
year 2wsid 2 month 2 day 2 hour 2
Memtable
SSTable
SSTable
SSTable
SSTable
Node
Commit Log Data * Compaction *
Temp
Temp
Memory
Disk
![Page 41: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/41.jpg)
Read PathClient
SSTableSSTable
SSTable
Node
Data
SELECT wsid,hour,temperatureFROM raw_weather_dataWHERE wsid='10010:99999'AND year = 2005 AND month = 12 AND day = 1 AND hour >= 7 AND hour <= 10;
year 1wsid 1 month 1 day 1 hour 1
year 2wsid 2 month 2 day 2 hour 2
Memtable
Temp
Temp
Memory
Disk
![Page 42: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/42.jpg)
Summary
•By default, writes are durable•Client receives ack when consistency level is achieved•Reads must always go to disk•Compaction is data housekeeping
![Page 43: Introduction to Cassandra Architecture](https://reader035.vdocuments.net/reader035/viewer/2022062316/58edc8941a28ab9d1c8b45d9/html5/thumbnails/43.jpg)
43
Questions?