itdks.su.bcebos.com · mysql vs hbase mysql - w t « -galera/group replication - þ õ o/proxy -ndb...
TRANSCRIPT
![Page 1: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/1.jpg)
�����
![Page 2: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/2.jpg)
DBA BDA
�����
![Page 3: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/3.jpg)
■
■ FAE
Qunar DBA MySQL
HBase ,
RDBMS
■ xzwen96( )
![Page 4: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/4.jpg)
Contents
�����
![Page 5: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/5.jpg)
Part 01
�����
![Page 6: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/6.jpg)
-
PPTSUCAIPPT PPT
——PPTSUCAI
1
2
3
4
database
�����
![Page 7: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/7.jpg)
-
1 2 MySQL 3 Namespace
�����
![Page 8: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/8.jpg)
SQL ?
hash
MySQL/HBase
�����
![Page 9: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/9.jpg)
Part 02
�����
![Page 10: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/10.jpg)
01
03
05
02
04
06�����
![Page 11: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/11.jpg)
…�����
![Page 12: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/12.jpg)
-HBase
�����
![Page 13: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/13.jpg)
HBase
写 E+�h\)k�a`mE+6S)V�k�T=%�.I���Ce1
��5�row-keyO$�.Ik�,�:ZO� PUk�HYOqpsP��.I
列 J?L_*bkE+!J6)��k��MeQJ�.Ikcgi^�!�;[/�
HBasek�F?KV6Sd
�����
![Page 14: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/14.jpg)
HBase
�����
![Page 15: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/15.jpg)
MySQL VS HBase
MySQL
InnoDB
B+tree
HBase
Key-Value
LSM-tree Phoenix
web
String
SQL SQL kv scan�����
![Page 16: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/16.jpg)
MySQL VS HBase
MySQL
Galera/Group replication
/Proxy
NDB
HBase
Namenode QJM
Datanode
HBase Master
RegionServer
HDFS
RS�����
![Page 17: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/17.jpg)
HBase
GC G1
Zookeeper
HBase
OOM�����
![Page 18: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/18.jpg)
HBase
HBase
Scan.setCaching RPC
Get RPC
scan.setBlockCache(false)
rowkey <startrow,stoprow>
blockcache LRUBlockCache SlabCache BucketCache
HBase
rowkey
BlockCache LRUBLockCache+BucketCache
HFile hbase.hstore.compactionThresholdhbase.hstore.compaction.max.size
Compaction IO
vs BloomFilter row rowcol
HDFS
Region major compaction
Hedged Read dfs.client.hedged.read.threadpool.sizedfs.client.hedged.read.threshold.millis
Short Circuit Local Read dfs.client.read.shortcircuitdfs.domain.socket.path�����
![Page 19: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/19.jpg)
HBase
HBase
WAL ?
VS (PUT)
(keyvalue length) RS ( 100G)
rowkey
HBase
RegionServer flush memstore Flush
blockingStoreFiles hbase.hstore.blockingStoreFiles
Full GC Java GC�����
![Page 20: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/20.jpg)
HBase BlockCache
�����
![Page 21: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/21.jpg)
HBase BlockCache
�����
![Page 22: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/22.jpg)
Part 03
�����
![Page 23: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/23.jpg)
HBase
•&jlJavak�GHDFS0�8%�2•' �B4SQL>3kSchema-less�(!#K•�B4f��I•��f��IkRW�B4�7]X•"MySQL@%HBasek�AD�)9<
HBasek-�6SN������
�����
![Page 24: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/24.jpg)
SQL on Hadoop
�����
![Page 25: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/25.jpg)
SQL ? Hadoop Hive
�����
![Page 26: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/26.jpg)
SQL ? Hadoop Hive
1
2
3
, HQL
MR
Hive HadoopMySQL
Hive Client Hadoop
4Hive -> SparkSQL , SparkSQL Hive HiveContext (1.x)SparkSession(2.x)���
��
![Page 27: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/27.jpg)
MR VS Spark
�����
![Page 28: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/28.jpg)
Hadoop vs Spark
02
04
01
03
Hadoop: HiveSpark: SparkSQL
SQL Query
Hadoop: MahoutSpark: Spark ML Lib
Machine Learning
Hadoop: MR(Java,Pig,Hive)Spark: RDDs(Java,Scala,Python)
Batch processing
Hadoop: Strom+KafkaSpark: Spark Streaming
Strem Processing
�����
![Page 29: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/29.jpg)
SQL ? SparkSQL
�����
![Page 30: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/30.jpg)
SQL ? SparkSQL
1
2
3
Spark : Local Standalone YARN( ) Mesos
:
DataFrame vs SQL , vs
4 :spark.sql.shuffle.partitions: spark.sql.sources.partitionColumnTypeInference.enabled�����
![Page 31: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/31.jpg)
MySQL NoSQL/NewSQL
�����
![Page 32: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/32.jpg)
Part 04
�����
![Page 33: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/33.jpg)
1 2 SQL3 4 MySQL
�����
![Page 34: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/34.jpg)
TiDB
MySQL01
03
05
HTAP
02
04
06 �����
![Page 35: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/35.jpg)
TiDB
�����
![Page 36: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/36.jpg)
�����
![Page 37: itdks.su.bcebos.com · MySQL VS HBase MySQL - W T « -Galera/Group replication - þ õ o/Proxy -NDB ZHBase Namenode MQJM [ b Datanode M Z Å \ Ä Ü HBase M SMaster RegionServer Ô](https://reader033.vdocuments.net/reader033/viewer/2022060912/60a665147b718e74cf4628b7/html5/thumbnails/37.jpg)
�����