toi uu hoa he thong 30 trieu nguoi dung
TRANSCRIPT
![Page 1: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/1.jpg)
Tối ưu hóa hệ thống 30 triệu người dùng
IT Expert Club
![Page 2: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/2.jpg)
CONTENT
1. Introduction2. Solution3. Caching4. Asynchronous5. Scale 6. Distributed7. Profiling and Turning
![Page 3: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/3.jpg)
1. Introduction
CTO of Ecomobi
Contact info:• EMAIL: [email protected]• MOBILE: (+84)904.554.112• SKYPE: ITBACHKHOA
![Page 4: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/4.jpg)
1. Introduction
Ecomobi is a CPA Network focus on SEA Market.
![Page 5: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/5.jpg)
Main Flow
![Page 6: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/6.jpg)
Daily Information
Affiliate Networks• 1.5M+ DAU• 2-3M CLICK • 1TB TRANSFER• USER: VN, ID, IN, BR...
Ads Networks• 30M+ USERS• 1M+ DAU• 40M+ ADS REQUEST• 70K+ RPM• USERS: VN, ID, IN,
BR...
![Page 7: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/7.jpg)
Our Issues
• High Availability• Minimize Response time• Concurrency • High Bandwidth• International user
![Page 8: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/8.jpg)
2. Our Solutions
• Measurement tools• Select Services Provider• Programming language• Database• Others
![Page 9: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/9.jpg)
2. Measurement
• Analytics• Quick or Slow• Error or Accuracy• Stable or Unstable• Better or Worse
![Page 10: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/10.jpg)
2. Measurement Tools - New Relic
![Page 11: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/11.jpg)
2. Measurement Tools - New Relic
![Page 12: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/12.jpg)
Services Provider?
• Dedicated Server in Viet nam• Private cloud• Docker
• VPS Provider in Vietnamese• Amazon Web Service• Digital Ocean / Linode• VPS in Local Provider
![Page 13: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/13.jpg)
Programming language?
• PHP• JAVA• Erlang / Elixir• Node JS / Golang
![Page 14: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/14.jpg)
Database
• MySQL / MariaDB• PostgreSQL• MongoDB• Cassandra / couchbase / Riak• Redis / Memcache • Elasticsearch
![Page 15: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/15.jpg)
Message Queue
• RabbitMQ• Redis• Activemq / Zeromq• Kafka• Gearman (Job Server)
![Page 16: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/16.jpg)
Other
• Caching• Asynchronous• Scale out• Distributed
![Page 17: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/17.jpg)
Our Stack
![Page 18: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/18.jpg)
3.Caching
• In-memory / variable caching• Cache size => minimize (why) • Increasing Cache Hit / Cache Miss• Cache Read through issue
![Page 19: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/19.jpg)
3.Caching: memory
function getInfoWithCache($id) {
static $cache_contents = array();
if (isset($cache_contents[$id])) {
return $cache_contents[$id];
}
$data = getInfoWithoutCache($id)
$cache_contents[$id] = $data;
return $data;
}
![Page 20: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/20.jpg)
Caching - Minimize cache size
![Page 21: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/21.jpg)
3.Caching - increasing cache hit
$candidates = Campaign for
• Active campaigns
• Allow Android 4.0.1
• Allow Vietnam
• Not gambling
• CPI or CPC• IF CPC: PRICE > $0.03• IF CPI: PRICE > $0.5
• Did not install this game / app
…
Candidates = Campaign for
• Active Campaigns
• Allow Vietnam
Candidates = Filter(Candidates)
• Allow Android 4.0.1
• Not gambling
• CPC or CPC
…
![Page 22: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/22.jpg)
Cache read through issue!!!function getData($id){ $data = getDataFromCache($id); if (!empty($data) { return $data; } $data = getDataWithoutCache($id); setDataToCache($id, $data, $exprire_time); return $data;}
![Page 23: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/23.jpg)
4. Asynchronous
• Send a external HTTP request
• Send a email
• Working with file
• Working with DB
• …
![Page 24: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/24.jpg)
4. Asynchronous
• Decreasing Response time• Deal with huge of request
![Page 25: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/25.jpg)
4. Asynchronous - MQ
![Page 26: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/26.jpg)
4. Asynchronous - MQwhile (true) { $msg = getMessageQueue(); if (empty($msg)) { sleep(1); continue; } processMQ($msg); if ($need_stop) { break; }}
![Page 27: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/27.jpg)
5.1 Web Application Scale
![Page 28: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/28.jpg)
5.2 Database layer
![Page 29: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/29.jpg)
6. Distributed
![Page 30: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/30.jpg)
6. Distributed
![Page 31: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/31.jpg)
6. CDN
![Page 32: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/32.jpg)
7. Profiling
![Page 33: Toi uu hoa he thong 30 trieu nguoi dung](https://reader031.vdocuments.net/reader031/viewer/2022022202/588261ca1a28abd40e8b6e31/html5/thumbnails/33.jpg)