tidb && tikv 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfkv api...
TRANSCRIPT
![Page 1: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/1.jpg)
TiDB && TiKV 技术选型思考 刘奇
![Page 2: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/2.jpg)
仅代表个人观点
Lease >= 2017.01.14
![Page 3: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/3.jpg)
About me● Qi Liu (刘奇)● Co-founder & CEO of PingCAP● JD / Wandoulabs / PingCAP● Old programmer
○ asm/c/c++/go/rust
● Infrastructure software engineer / Open source hacker● Codis / TiDB / TiKV
![Page 4: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/4.jpg)
Why TiDB ?
一个老程序员夙愿解决分布式系统的一些问题
分布式缓存:Codis分布式数据库:TiDB
![Page 5: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/5.jpg)
Why TiDB ?
● No sharding● No inconsistent● Keep transaction● Scaling● Make everything much faster
![Page 6: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/6.jpg)
TiDB can be used as
● Unlimited MySQL for both OLTP/OLAP● 一周无限回档/全球同服● 异地多活● MySQL warehouse, query is much faster
![Page 7: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/7.jpg)
Why multiple layer ?Applications
MySQL Drivers(e.g. JDBC)
TiDB
TiKV
MySQL Protocol
RPC
F1
Spanner
![Page 8: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/8.jpg)
Architecture
KV API Coprocessor
Txn, Transaction
MVCC
RawKV, Raft KV
RocksDB
Placement Driver
TiKV TiDB
MySQL clients
Load Balancer (Optional)
MySQL Protocol
TiDB SQL Layer
KV APIDistSQL
API
TiDB Server (Stateless)
MySQL Protocol
TiDB SQL Layer
KV APIDistSQL
API
TiDB Server (Stateless)
Pluggable Storage Engine (e.g. TiKV)
![Page 9: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/9.jpg)
TiKV: The whole picture
Client
Store 1
Region 1
Region 3
Region 5
Region 4
Store 3
Region 3
Region 5
Region 2
Store 2
Region 1
Region 3
Region 2
Region 4
Store 4
Region 1
Region 5
Region 2
Region 4
RPC RPC RPC RPC
TiKV node 1 TiKV node 2 TiKV node 3 TiKV node 4
Placement Driver
PD 1
PD 2
PD 3
Raft Group
![Page 10: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/10.jpg)
TiDB query performanceTiDB Elapse MySQL Elapse
5.07699437s 19.93s
10.524703077s 43.23s
10.077812714s 43.33s
10.285957629s >20 mins
10.462306097s 36.81s
9.968078965s 1 min 0.27 sec
9.998030375s 44.05s
10.866549284s 43.18s
![Page 11: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/11.jpg)
Why TiDB is much faster? Predicate pushdown
TiKV Node1 TiKV Node2 TiKV Node3
TiDB Server
Region 2Region 1
Region 5
age > 20 and age < 30 age > 20 and age < 30
age > 20 and age < 30TiDB knows that Region 1 / 2 / 5stores the data of person table.
![Page 12: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/12.jpg)
More data
● Scale to 200T● Scale to 496 node
![Page 13: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/13.jpg)
But how ?
● 长久以来的正确姿势 ● make it run ● make it right ● make it fast
![Page 14: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/14.jpg)
But how ?
然后这不是一个简单的程序不是两三天就能搞定的
数据库是航母级别规模的东西How to make it right ?
How to make it run ?How to make it fast ?
![Page 15: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/15.jpg)
How to make it right ?
● Tests first● Mock layer by layer
![Page 16: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/16.jpg)
Tech choices:Paxos vs Raft
XXX-Paxos is complicatedDifficult to implement
Needs to test the implementation for years
![Page 17: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/17.jpg)
Programming language choices
go or rust or both?FAST, SAFE AND PRODUCTIVE — PICK THREE
Go: productiveRust: fast, safe
![Page 18: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/18.jpg)
But why not c++ ?
● Hard to control for large project.● Can’t detect data race when compiling the source● No package management tools● Personally programming cpp sucks
![Page 19: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/19.jpg)
Is rust really wonderful?
● Sounds too good to be true● Compiling speed makes you want to kill yourself● Lacks of libraries, we are still waiting for grpc
![Page 20: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/20.jpg)
Hold on, why MySQL protocol ?
● I heard that PostgreSQL is more beautiful and powerful.
● Users○ MySQL >> MongoDB >> PostgreSQL
![Page 21: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/21.jpg)
Embed storage: the war
● Lmdb● Leveldb● Innodb● Rocksdb● Wiredtiger
![Page 22: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/22.jpg)
grpc VS others
Some rpc frameworks are really fastBut we want the rpc that widely used, work with
protocol buffersCame from ex-googler
![Page 23: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/23.jpg)
prometheus VS others
Very activeWidely used
Works well with grafana
![Page 24: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/24.jpg)
grafana VS others
Widely usedCame from ex-googler
![Page 25: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/25.jpg)
Rebuilding all or use mysql with tikv ?
Everything should be distributedDistributed query plan
Multiple writable serversNo read/write bottleneck
![Page 26: TiDB && TiKV 技 型思考open.qiniudn.com/ecug-2016/thought-of-tidb-tech-choices.pdfKV API Coprocessor Txn, Transaction MVCC RawKV, Raft KV RocksDB Placement Driver TiKV TiDB MySQL](https://reader033.vdocuments.net/reader033/viewer/2022052808/607228b3ea224f7caa194392/html5/thumbnails/26.jpg)
We are still on our way
https://github.com/pingcap/tidbhttps://github.com/pingcap/tikv