Download - 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant [email protected] 2016-08-24
![Page 2: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/2.jpg)
2/117
程式開發 X 資訊安全 X 智慧財產權 X 創業
Profile
![Page 3: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/3.jpg)
3/117
剛好符合分寸。形容做事、說話十分恰當。
![Page 4: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/4.jpg)
4/117
Premature optimization is the root of all evil
過早最佳化是萬惡的根源
- Donald Knuth -
![Page 5: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/5.jpg)
5/117
架構是演進的,預想有益身心健康預先策想
![Page 6: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/6.jpg)
6/117
GB
TB
PB
MB
![Page 7: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/7.jpg)
7/117
新平台預計下一季上線。
營運預計導入多少流量?我可以先行準備。
有流量再說,快就好。
那我之後 ( 有空 ) 再做。
新平台上線
![Page 8: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/8.jpg)
8/117
為什麼網站很慢又常當機?
調整架構需要一週。
這麼久?不能快點嗎?
沒有辦法 ...
上線一週後
![Page 9: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/9.jpg)
9/117
MVP 若沒有控制好,技術債將迅速增長
Minimum Viable Product
先前程式常不能用
遺舊程式不忍棄之
結構變動反覆無常
![Page 10: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/10.jpg)
10/117
只做最低限度產品 (MVP)
最終得到的可能是壞產品
![Page 11: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/11.jpg)
11/117
Pivot
ObjectGo
✖✖
✖
✖✖
Research Development
![Page 12: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/12.jpg)
12/1172015
➊架構先決 無視人員、流程只講技術,是耍白痴
架構會影響公司文化、商業擴展;思維更要超越程式碼層次
![Page 13: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/13.jpg)
13/1172015
➋沒有完美的架構,只有最適的架構無視場景只講架構,是耍流氓
若無法舉出架構的缺陷,代表你還無法掌握
盲目套用別人的架構,並不會讓你變得跟他一樣好
![Page 14: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/14.jpg)
14/1172015
➌架構是演進的,預想但不過早調優無視未來只求現有,是耍自閉
兵馬未動,糧草先行,預想下一步,下下一步,甚至下下下一步 ...
![Page 15: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/15.jpg)
15/117
Monolith
Microservices
SOA
Nanoservices
Picoservices?
![Page 16: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/16.jpg)
16/117
Business
License
Elastic business
Workload
Technology
Scale-up
ApplicationConnectionDatabaseFile systemOS KernelHardware
Scale-out
ReplicationClusteringShardingDisaster RecoveryMulti Regional Resiliency
CONSILIENCE
Architecture
and more ...
![Page 17: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/17.jpg)
17/117
Business
License
Elastic business
Workload
Technology
Scale-up
ApplicationConnectionDatabaseFile systemOS KernelHardware
Scale-out
ReplicationClusteringShardingDisaster RecoveryMulti Regional Resiliency
CONSILIENCE
Architecture
and more ...
![Page 18: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/18.jpg)
18/117
License
![Page 19: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/19.jpg)
19/117
License
程式原始碼是否在意授權對方修改、散布?
GNU GPL 的互惠性 / 感染性
散布于對方時,強制啟動授權
( 接案 ) 軟體交付時
嵌入式設備
自己的程式 GPL 程式
GPL 程式
![Page 20: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/20.jpg)
20/117
License
C/Java + MySQL/Percona →
PHP/PDO + MySQL/Percona →
C/Java/PHP + MariaDB →
自己的程式 Database
GPL 程式
![Page 21: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/21.jpg)
21/117
Business
License
Elastic business
Workload
Technology
Scale-up
ApplicationConnectionDatabaseFile systemOS KernelHardware
Scale-out
ReplicationClusteringShardingDisaster RecoveryMulti Regional Resiliency
CONSILIENCE
Architecture
and more ...
![Page 22: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/22.jpg)
22/117
當業務需求變更程式設計時
預想但不過早調優
總工程師不該把每次新需求都認為獨立需求
( 多想二分鐘,團隊可以不必自虐 )
Elastic business
![Page 23: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/23.jpg)
23/117
狀態
原表格設計
Elastic business
id name ... is_deleted
1 Apple ... 0
2 Banana ... 1
![Page 24: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/24.jpg)
24/117
狀態
新業務需要儲存「鎖定」狀態
Elastic business
id name ... is_deleted
1 Apple ... 0
2 Banana ... 1
id name ... is_deleted is_locked
1 Apple ... 0 1
2 Banana ... 1 0
![Page 25: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/25.jpg)
25/117
狀態
其實若狀態是互斥的,則可以合併
Elastic business
id name ... status
1 Apple ... 0
2 Banana ... 1
id name ... is_deleted is_locked
1 Apple ... 0 1
2 Banana ... 1 0
{ 0: deleted, 1: enabled, 2: locked }
![Page 26: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/26.jpg)
26/117
標籤雲
原表格設計
Elastic business
id name tag1
1 Apple admin
2 Banana reporter
3 Cherry reporter
SELECT * FROM {Table}WHERE tag1 = ‘admin’
![Page 27: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/27.jpg)
27/117
標籤雲
新增標籤
Elastic business
id name tag1 tag2 tag3
1 Apple admin reporter programmer
2 Banana reporter programmer NULL
3 Cherry reporter admin NULL
SELECT * FROM {Table}WHERE (tag1 = ‘admin’ OR tag2 = ‘admin’ OR tag3 = ‘admin’) AND (tag1 = ‘reporter’ OR tag2 = ‘reporter’ OR tag3 = ‘reporter’)
SELECT * FROM {Table}WHERE ‘admin’ IN (tag1, tag2, tag3) AND ‘reporter’ IN (tag1, tag2, tag3)
ALTER TABLE !!
![Page 28: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/28.jpg)
28/117
Tag
Elastic business
id tag
1 admin
1 reporter
1 programmer
2 reporter
... ...
新方法
標籤雲
id name X X X
1 Apple X X X
2 Banana X X X
SELECT * FROM {Table} INNER JOIN ‘Tag’ AS t1 USING (id) INNER JOIN ‘Tag’ AS t2 USING (id)WHERE t1.tag = ‘admin’ AND t2.tag = ‘reporter’
![Page 29: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/29.jpg)
29/117
Elastic business
或是 M:N
標籤雲
id name
1 Apple
2 Banana
id tag_id
1 1
1 2
1 3
2 2
2 3
tag_id name
1 admin
2 reporter
3 programmer
![Page 30: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/30.jpg)
30/117
Elastic business廣告需求
營運有新的需求,受眾在一天內不要看到相同廣告。
瞭解,預計一個工作天。
第一天
![Page 31: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/31.jpg)
31/117
Elastic business廣告需求
營運有新的需求,受眾在小時內不要看到相同廣告。
瞭解,預計二個工作天。
第二天
時間粒度不同
![Page 32: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/32.jpg)
32/117
Elastic business廣告需求
營運有新的需求,受眾在一天內不要看到同類廣告。
瞭解,預計八個工作天。
第三天
目標粒度不同
不能一次講清楚嗎?
![Page 33: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/33.jpg)
33/117
Elastic business廣告需求
受眾在 M 時間內不要看到 N廣告
預想
該需求的延伸會是什麼?
M → 年 / 季 /月 / 週 / 時 / 分 /秒N → 相同 /同類看到的次數? 1/2/3...裝置有別?區域?廣告主屬性?
![Page 34: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/34.jpg)
34/117
Business
License
Elastic business
Workload
Technology
Scale-up
ApplicationConnectionDatabaseFile systemOS KernelHardware
Scale-out
ReplicationClusteringShardingDisaster RecoveryMulti Regional Resiliency
CONSILIENCE
Architecture
and more ...
![Page 35: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/35.jpg)
35/117
Workload
Processing Intensive Capacity
CPU intensive
Memory intensive
Storage/IO intensive
Bandwidth intensive
OLTP
OLAP
Data warehouse
Throughput
Latency
Memory footprint
Service-level agreement
Bond
Performance
Security
Cost restriction
![Page 36: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/36.jpg)
36/117
OLTP (On-Line Transaction Processing)
➊應用 : Customer-oriented➋回應時間 (response time) 要求較高。➌併發 (concurrency) 要求較多。➍資料處理量 (volume) 少。➎交易 (transaction)完整性高。➏安全性 (security) 要求較高。
WorkloadProcessing
![Page 37: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/37.jpg)
37/117
Workload
OLAP (On-Line Analytical Processing)
➊應用 : Market-oriented➋回應時間 (response time) 要求較低。➌併發 (concurrency) 要求較少。➍資料處理量 (volume) 多。➎交易 (transaction)完整性低。➏安全性 (security) 要求較低。
Processing
![Page 38: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/38.jpg)
38/117
Workload
Data warehouse
➊應用 : Subject-oriented➋熱資料 (Hot)本地、快取、粒度、一致性。➌暖資料 (Warm) 分布、快取、複製。➍冷資料 (Cold)索引、壓縮、合併、備份。
Processing
![Page 39: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/39.jpg)
39/117
WorkloadIntensive
BandwidthCPU
Memory Storage/IO
![Page 40: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/40.jpg)
40/117
WorkloadCapacity
Latency Memoryfootprint
Trade-off
Throughput
![Page 41: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/41.jpg)
41/1172015
High throughput Low latency
![Page 42: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/42.jpg)
42/1172015
千萬人同時在線電子商務、線上媒體
低延遲回應廣告平台 (30ms) 、高頻交易 (0.5~3ms) 、醫療等關鍵設備
![Page 43: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/43.jpg)
43/117
機房只能給你 4U共 2 台機器,請您務必提供每秒 1 萬次的處理效能。
每次請求包括產生 46張圖片及 46 次加密演算法
WorkloadCapacity
![Page 44: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/44.jpg)
44/117
WorkloadCapacity
![Page 45: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/45.jpg)
45/117
WorkloadCapacity
Optimal capacity
![Page 46: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/46.jpg)
46/117
WorkloadCapacity
Optimal capacity
![Page 47: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/47.jpg)
47/117
WorkloadCapacity
Language / Framework / Algorithm / Hardware
300 Server → Performance +10x → 30 Server(Price cost reduction)
(Maintenance cost reduction)
![Page 48: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/48.jpg)
48/117
WorkloadBond
Security CostRestriction
Trade-off
Performance
![Page 49: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/49.jpg)
49/117
Business
License
Elastic business
Workload
Technology
Scale-up
ApplicationConnectionDatabaseFile systemOS KernelHardware
Scale-out
ReplicationClusteringShardingDisaster RecoveryMulti Regional Resiliency
Architecture
and more ...
CONSILIENCE
![Page 50: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/50.jpg)
50/117
Scale-upHardware
CPU
➊快取對 InnoDB影響很大 (CPU Cache) 。➋超執行緒 (Hyper threading) 有助益。➌通常啟用 Node Interleaving ,可避免NUMA問題。➍多核心處理
MySQL 5.5最佳表現為 16核心,同時連線數 128 。
MySQL 5.6支援至少 64核心,同時連線數處理不受影響;但 RW 在同時處理 128連線數後顯著下降。
MySQL 5.7支援至少 64核心,同時連線數處理不受影響;解決RW同時處理連線數下降問題。
![Page 51: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/51.jpg)
51/117
Scale-upHardware
CPUs
North Bridge(MCH) Memory
PCIe
FSB (10.6 GB/s)
Intel Xeon (Older)
CPUs
North Bridge(IOH)
Memory
PCIe
QBI (25.6 GB/s)
Intel Xeon (Newer)
![Page 52: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/52.jpg)
52/117
Scale-upHardware
Memory
➊原則上愈多愈好➋確保能把所需資料表全儲存在記憶體中。
![Page 53: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/53.jpg)
53/117
Scale-upHardware
Storage
➊原則上, PCIe NVMe SSD > SSD > HDD 。
Ref: http://agigatech.com/blog/ssds-some-cold-hard-numbers-to-flavor-your-opinions/
![Page 54: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/54.jpg)
54/117
Scale-upHardware
Storage
➊原則上, PCIe NVMe SSD > SSD > HDD 。➋區塊大小 (Block size) 對 SSD 很重要。
![Page 55: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/55.jpg)
55/117Ref: http://www.thessdreview.com/our-reviews/intel-ssd-dc-p3608-review-1-6tb-over-5gbs-and-850k-iops/2/
![Page 57: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/57.jpg)
57/117Ref: http://jdevelopment.nl/2009/02/
Bandwidth
IOPS
ThroughputLatency
![Page 58: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/58.jpg)
58/117
Scale-upHardware
Storage
➊原則上, PCIe NVMe SSD > SSD > HDD 。➋區塊大小 (Block size) 對 SSD 很重要。➌循序寫 +RAID 的 HDD 不比 SSD差。
![Page 59: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/59.jpg)
59/117Ref: SSD Deployment Strategies for MySQL (2010-04-15).pdf (p16)Ref: http://yoshinorimatsunobu.blogspot.tw/2009/05/tables-on-ssd-redobinlogsystem.html
RAID-10 > RAID-5 (RAID 控制器很重要 )battery backed up write cache (BBWC)
![Page 60: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/60.jpg)
60/117
Scale-upOS Kernel
➊ vm.swappiness = 1
![Page 61: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/61.jpg)
61/117
Scale-upOS Kernel
➊ vm.swappiness = 1 ➋ vm.dirty_background_ratio / vm.dirty_ratio
![Page 62: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/62.jpg)
62/117
Scale-upOS Kernel
➊ vm.swappiness = 1 ➋ vm.dirty_background_ratio / vm.dirty_ratio ➌ IO scheduler (DEADLINE or NOOP)
![Page 63: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/63.jpg)
63/117
Scale-upFile system
OLTP
Ext4 / XFSJournal / O_DIRECTCOMPRESSION...
![Page 64: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/64.jpg)
64/117
Scale-upDatabase
Design
Configuration
MT-malloc: jemalloc / tcmalloc / etc.DB Engine: InnoDB / TokuDB / RocksDBSchema design / IDIndexPartitions
default_time_zone = ‘+00:00’max_connectionssort_buffer_sizejoin_buffer_sizeread_buffer_sizeinnodb_use_native_aio = 1innodb_file_per_table = 1innodb_buffer_pool_size = { 65~80% of Mem }innodb_thread_concurrency = { 2xCPUs }innodb_read_io_threads = { CPUs }innodb_write_io_threads = { CPUs }
![Page 65: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/65.jpg)
65/117
Scale-upDatabase
OLTP
innodb_flush_methodinnodb_max_dirty_pages_pctinnodb_page_sizeinnodb_io_capacityinnodb_flush_neighborsinnodb_random_read_aheadinnodb_read_ahead_threshold...
![Page 66: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/66.jpg)
66/117
Scale-upConnection
Connection pool (Client/Application)
➊不是所有應用程式都支援。➋無法得知伺服器的狀態及承載。➌遭遇錯誤時,必須執行完整的資源清理。➎會保持連線,佔用伺服器連線數及線程快取。
![Page 67: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/67.jpg)
67/117
Scale-upConnection
DatabaseApplication
架構問題
最大連線數 100 最大連線數 200
![Page 68: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/68.jpg)
68/117
Scale-upConnection
DatabaseApplication
架構問題
最大連線數 200{ 剩餘連線數 100}
最大連線數 100
keep
![Page 69: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/69.jpg)
69/117
Scale-upConnection
DatabaseApplication
架構問題
最大連線數 100
Application
最大連線數 100keep
keep
最大連線數 200{ 剩餘連線數 0}
![Page 70: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/70.jpg)
70/117
Scale-upConnection
DatabaseApplication
架構問題
最大連線數 100
Application
最大連線數 100
Application
最大連線數 100
keep
keep
最大連線數 200{ 剩餘連線數 0}
![Page 71: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/71.jpg)
71/117
Scale-upConnection
架構問題
MySQL
➊線程模式。➋不需 Connection pool 就可以支持高併發。➌支持短連接使用資料庫。➍新版5.7建立連線的開銷更少。5.6版的 62.5%;5.5版的 40%。
![Page 72: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/72.jpg)
72/117
Scale-upApplication
效能通常有 99%的問題在於Application
➊ N+1 queries / ORM ➋ Bad SQL ➌ Bad Schema Design ➍ Big SQL ➎ Big Transaction ➏ Big Batch
![Page 73: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/73.jpg)
73/117
Scale-upApplication
效能通常有 99%的問題在於Application
➊ N+1 queries / ORM ➋ Bad SQL ➌ Bad Schema Design ➍ Big SQL ➎ Big Transaction ➏ Big Batch
![Page 74: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/74.jpg)
74/117
Scale-upApplication
(MySQL) CHAR vs. VARCHAR
➀如果更新頻繁且長度不一, CHAR通常比較快。
➁在 MySQL 5.7.7之後, CHAR通常比VARCHAR 快。
![Page 75: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/75.jpg)
75/117
Scale-upApplication
(MySQL) VARCHAR vs. VARCHAR
➀某些編碼下, VARCHAR(760)與VARCHAR(770) 快得多。
➁某些編碼下, VARCHAR(190)比VARCHAR(200) 快得多。
➂不過在 MySQL 5.7.7之後,前兩者幾乎沒什麼差別。
![Page 76: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/76.jpg)
76/117
Scale-upApplication
INDEX
➀ Primary Index對 MySQL 很重要,循序式比亂序式快。
➁ Index愈多不一定愈好。
➂ Composite Index需善用。
![Page 77: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/77.jpg)
77/117
Business
License
Elastic business
Workload
Technology
Scale-up
ApplicationConnectionDatabaseFile systemOS KernelHardware
Scale-out
ReplicationClusteringShardingDisaster RecoveryMulti Regional Resiliency
Architecture
and more ...
CONSILIENCE
![Page 78: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/78.jpg)
78/117
Replication
Scale-out
Master
ReplicaReplica Replica
![Page 79: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/79.jpg)
79/117
Replication
Scale-out
Master
ReplicaReplica Replica
Applications
writeread
![Page 80: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/80.jpg)
80/117
Clustering
Scale-out
![Page 81: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/81.jpg)
81/1172015
Database#1
Database#2
Database#3 ...
Applications
RW RW RW RW
Master-Master
![Page 82: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/82.jpg)
82/1172015
Database#1
Database#2
Database#3 ...
Applications
RW RW RW RW
UPDATE tSET …
WHERE id = 1
T1
Master-Master
UPDATE tSET …
WHERE id = 1
T2
![Page 83: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/83.jpg)
83/1172015
Database#1
Database#2
Database#3 ...
Applications
RW RW RW RW
UPDATE tSET …
WHERE id = 1
T1
Master-Master
UPDATE tSET …
WHERE id = 1
T2
100 → 200 100 → 200
![Page 84: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/84.jpg)
84/1172015
Database#1
Database#2
Database#3 ...
Applications
RW RW RW RW
UPDATE tSET …
WHERE id = 1
T1
Master-Master
UPDATE tSET …
WHERE id = 1
T2
100 → 200 100 → 200
Deadlock / Rollback
![Page 85: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/85.jpg)
85/1172015
Database#1
Database#2
Database#3 ...
Applications
Master-Master
Load balancing
W RRR
![Page 86: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/86.jpg)
86/1172015
Database#1
Database#2
Database#3 ...
Applications
UPDATE tSET …
WHERE id = 1
T1
Master-Master
SELECT ...
T2
W RRR
T3
SELECT ...
T4
SELECT ...
T5
Load balancing
![Page 87: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/87.jpg)
87/1172015
Database#1
Database#2
Database#3 ...
Applications
UPDATE tSET …
WHERE id = 1
T1
Master-Master
SELECT ...
T2
W RRR
T3
SELECT ...
T4
SELECT ...
T5
Load balancingLoad balancingHA
Monitor
![Page 88: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/88.jpg)
88/1172015
Percona XtraDB Cluster: Multi-node writing and Unexpected deadlocks2012-08-17https://www.percona.com/blog/2012/08/17/percona-xtradb-cluster-multi-node-writing-and-unexpected-deadlocks/
Avoiding Deadlocks in Galera - Set up HAProxy for single-node writesand multi-node reads2013-09-17http://www.severalnines.com/blog/avoiding-deadlocks-galera-set-haproxy-single-node-writes-and-multi-node-reads
Optimizing Percona XtraDB Cluster for write hotspots2015-06-03https://www.percona.com/blog/2015/06/03/optimizing-percona-xtradb-cluster-write-hotspots/
![Page 89: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/89.jpg)
89/1172015
Database#1
Database#2
Database#3 ...
Applications
UPDATE tSET …
WHERE id = 1
T1
Master-Master
SELECT ...
T2
W RRR
T3
SELECT ...
T4
SELECT ...
T5
Load balancingLoad balancingHA
Monitor
Hot-Spot1
Complexity2
![Page 90: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/90.jpg)
90/1172015
Database#1
Database#2
Database#3 ...
Applications
RW RW RW RW
Master-Master
Smart Client
![Page 91: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/91.jpg)
91/117
Sharding
Scale-out
![Page 92: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/92.jpg)
92/117
Disaster Recovery
Scale-out
![Page 93: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/93.jpg)
93/117
Disaster Recovery
Scale-out
Latency ?Bandwidth ?
![Page 94: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/94.jpg)
94/117
Multi Regional Resiliency
Scale-out
Ref: https://cloud.google.com/solutions/scalable-and-resilient-apps
![Page 95: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/95.jpg)
95/117
Multi Regional Resiliency
Scale-out
Ref: https://securosis.com/blog/resilient-cloud-network-architectures-design-patterns
![Page 96: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/96.jpg)
96/117
Business
License
Elastic business
Workload
Technology
Scale-up
ApplicationConnectionDatabaseFile systemOS KernelHardware
Scale-out
ReplicationClusteringShardingDisaster RecoveryMulti Regional Resiliency
CONSILIENCE
Architecture
and more ...
![Page 97: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/97.jpg)
97/117
大數據BigData
![Page 98: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/98.jpg)
98/117Ref: https://www.talend.com/blog/2015/07/15/hadoop-summit-2015-takeaway-the-lambda-architecture
![Page 99: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/99.jpg)
99/117Ref: http://www.slideshare.net/akirillov/data-processing-platforms-architectures-with-spark-mesos-akka-cassandra-and-kafka#p4
![Page 100: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/100.jpg)
100/117Ref: https://medium.baqend.com/nosql-databases-a-survey-and-decision-guidance-ea7823a822d
![Page 101: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/101.jpg)
101/1172015
我們很少在大數據架構中見到 RDBMS 的蹤影
但 Google/Facebook/Twitter/Uber/Alibaba ...
![Page 102: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/102.jpg)
102/117
大數據BigData
微服務Micro-services
X
![Page 103: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/103.jpg)
103/117
大數據BigData
Hadoop (Java)
Spark (Java/Scala)
Cassandra (Java)
Kafka (Java/Scala)
Pig (Java)
Hive (Java)
HBase (Java)
Flink (Java)
ElasticSearch (Java)
JVM 的天下
![Page 104: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/104.jpg)
104/117Ref: http://eugenedvorkin.com/seven-micro-services-architecture-advantages/
![Page 105: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/105.jpg)
105/117
部署快(起滅快 )
效能高( 機數少 )
開發速( 碼量少 )
體積小
編譯式
GC編譯式
JIT
保護編程下限(IDE/除錯 )
保障破壞下限
![Page 106: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/106.jpg)
106/117
部署快(起滅快 )
效能高( 機數少 )
開發速( 碼量少 )
體積小
編譯式
GC編譯式
容器化
Java 混合雲
JIT
保護編程下限(IDE/除錯 )
保障破壞下限
![Page 107: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/107.jpg)
107/117
JavaContainer Size
Oracle JDK (~350 MB)
Oracle JRE (~70 MB)
Oracle Server JRE (~70 MB)
Alpine Java Docker Container ?
只是 JRE本身
![Page 108: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/108.jpg)
108/117
JavaContainer Size
docker-alpine-java
<snip>
curl -jksSLH "Cookie: oraclelicense=accept-securebackup-cookie" ...
rm -rf /opt/jdk/*src.zip \ /opt/jdk/lib/missioncontrol \ /opt/jdk/lib/visualvm \ /opt/jdk/lib/*javafx* \ /opt/jdk/jre/plugin \ /opt/jdk/jre/bin/javaws \ ...
</snip>
Ref: https://github.com/anapsix/docker-alpine-java/blob/master/8/92b14/jdk/standard/Dockerfile
![Page 109: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/109.jpg)
109/117
Business
License
Elastic business
Workload
Technology
Scale-up
ApplicationConnectionDatabaseFile systemOS KernelHardware
Scale-out
ReplicationClusteringShardingDisaster RecoveryMulti Regional Resiliency
License
CONSILIENCE
and more ...
![Page 110: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/110.jpg)
110/117
Java
▐問題一 :
Oracle/Java 安裝前需人工同意 Oracle/Java 授權。
▐問題二
Oracle/Java 為整體不可分割之授權。
<snip>
(i) you distribute the Redistributables complete and unmodified,
and only bundled as part of Programs,
</snip>
Ref: http://www.oracle.com/technetwork/java/javase/terms/license/index.html
License
Oracle Java部署時,不得刪減任何程式及文件
![Page 111: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/111.jpg)
111/117
Java
▐問題一 :
Oracle/Java 安裝前需人工同意 Oracle/Java 授權。
▐問題二
Oracle/Java 為整體不可分割之授權。
<snip>
(i) you distribute the Redistributables complete and unmodified,
and only bundled as part of Programs,
</snip>
Ref: http://www.oracle.com/technetwork/java/javase/terms/license/index.html
License
Oracle Java部署時,不得刪減任何程式及文件
OpenJDK( openjdk-7-jre-headless )
有些人不敢用
![Page 112: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/112.jpg)
112/117
部署快(起滅快 )
效能高( 機數少 )
開發速( 碼量少 )
體積小
編譯式
GC編譯式
容器化
Java 混合雲
JIT
保護編程下限(IDE/除錯 )
保障破壞下限
![Page 113: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/113.jpg)
113/117
部署快(起滅快 )
效能高( 機數少 )
開發速( 碼量少 )
體積小
編譯式
GC編譯式
容器化
Java 混合雲
JIT
保護編程下限(IDE/除錯 )
保障破壞下限
Go
![Page 114: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/114.jpg)
114/117
大數據BigData
小結
▐未捨棄既有累積的知識 (RDBMS)
▐降低開發與維運人員的焦慮感 (專注,技術深化 )
▐系統異質性低 (出錯少,除錯易 )
▐依然相容現行大數據架構
▐支持容器化、混合雲、私有雲 (顧問性質 )
▐大數據核心與對外服務層權責分離 (兼具安全性 )
▐其實大多時候我們不需要大數據解決方案資料多≠需要大數據解決方案,或許只是垃圾數據多
微服務Micro-services
處理慢≠需要大數據解決方案,大多都是程式差,架構弱
![Page 115: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/115.jpg)
115/117
GoGo
HDFS/RDBMS
HDFS/RDBMS
Go
MySQL
MySQL
![Page 116: 恰如其分的MySQL設計技巧 - s.itho.me · Modern Web 2016 恰如其分的MySQL設計技巧 Ant yftzeng@gmail.com 2016-08-24](https://reader031.vdocuments.net/reader031/viewer/2022040617/5f204ee24509fd444e063cdd/html5/thumbnails/116.jpg)
116/1172015
➊架構先決
➋沒有完美的架構,只有最適的架構
➌架構是演進的,預想但不過早調優