proxysql tutorial high performance & high availability percona … · 2016-10-14 · proxysql...
TRANSCRIPT
![Page 1: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/1.jpg)
ProxySQL TutorialHigh Performance & High Availability
Proxy for MySQL
With a GPL license!
Amsterdam, Netherlands | October 3 – 5, 2016
Visit us at booth #102
Percona Live Europe 2016
![Page 2: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/2.jpg)
Who we are
David Turner DBA, Uber
Derek Downey OSDB Practice Advocate, Pythian
René Cannaò MySQL SRE, Dropbox / ProxySQL
![Page 3: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/3.jpg)
Main motivations
empower the DBAs
improve operation
understand and improve performance
create a proxy layer to shield the database
High performance and High Availability
![Page 4: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/4.jpg)
ProxySQL FeaturesSome of the most interesting features:
1. on-the-fly rewrite of queries
2. caching reads outside the database server
3. connection pooling and multiplexing
4. complex query routing and read/write split
5. load balancing
6. real time statistics
7. monitoring
8. High Availability and Scalability
9. seamless failover
10. firewall
11. query throttling
12. query timeout
13. query mirroring
14. runtime reconfiguration
![Page 5: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/5.jpg)
Hostgroups and Query Routing
All backends are grouped into hostgroups
Hostgroups have logical functionalities
![Page 6: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/6.jpg)
Basic design
HG0
HG2
HG1
ProxySQL
APP1
APP2
APP3
![Page 7: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/7.jpg)
Hostgroups example #1
HostGroup0 (HG0): Write masters
HostGroup1( HG1): Read slaves
Read/Write split
![Page 8: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/8.jpg)
Hostgroups example #2
HG0: main write masters
HG1: main read slaves
HG2: reporting slaves
HG3: ad-hoc queries slaves
HG4: data warehouse write masters
HG5: data warehouse read slaves
HG6: remote site servers
HG7: test servers
HG8 : mirror for traffic on HG0
HG9 : mirror for traffic on HG1
![Page 9: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/9.jpg)
Basic design
HG0
HG2
HG1
APPAPP
ProxySQL
Very low latency usingUnix Domain Socket
![Page 10: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/10.jpg)
Basic design
HG0
HG2
HG1
APP1
ProxySQL
APP2
ProxySQL
APP3
ProxySQL
![Page 11: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/11.jpg)
Clustered ProxySQL Architecture
DB DBDB
APP
DB
PROXYSQL
DB DB
APP
PROXYSQL
APP
PROXYSQL
APP
PROXYSQL
PROXYSQL PROXYSQL
DBMANAGER
PROXYSQL
Clustered Proxy Layer
![Page 12: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/12.jpg)
Clustered ProxySQL at scale
Tested with:
● 8 app servers with 3k clients’ connections each (24k total)
● 4 middle layer proxysqls processing 4k connections each from local proxysqls (16k total)
● 256 backends/shard (meaning 256 routing rules) processing 600 connections each (150k total)
Single ProxySQL was tested with up to 150k connections
At today, ProxySQL is able to process up to 750k QPS
![Page 13: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/13.jpg)
ProxySQL Internals
![Page 14: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/14.jpg)
ProxySQL Modules
HG0
HG1
APP1 ProxySQL
Queries cache
Query Processor
APP2
APP3 Thre
ad #
1
Thre
ad #
2
Thre
ad #
3Users Auth
ConnectionPool
HostgroupManager
Admin
Monitoring
![Page 15: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/15.jpg)
Queries Processor
Based on Queries Rules
Defines what to cache
Defines the hostgroup target
Timeout/delay
Firewall
Mirroring
Rewrite queries
![Page 16: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/16.jpg)
Queries rules
Complex rules to match incoming traffic:
● regex on query
● regex on digest text
● username
● schemaname
● Source IP address
● Bind IP address/port
● digest
Rules can be chained
![Page 17: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/17.jpg)
Queries Cache and Rewrite
Caching on the wire
Internal key/value storage
In memory only
Pattern based
Expired by timeout
Rewrite on the wire
Regex match/replace on query on digest text
Optionally cached or mirrored
![Page 18: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/18.jpg)
Users Authentication
Credentials stored in the proxy
User login always possible (even without backends)
Max connections
Login credentials are encrypted
![Page 19: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/19.jpg)
Hostgroups Manager
Management of servers
Track servers status
Tightly integrated with the connections pool
![Page 20: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/20.jpg)
Connections Pool
Reduced the overhead of creating new connections, and are recycled when not in use
One to many connections
Multiplexing & maximum connections
Auto-reconnect and automatic re-execution of queries
Failover management
![Page 21: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/21.jpg)
Auto-reconnect and re-execution
Automatic detection of failures
Graceful handling
Auto-reconnect when possible
Pause until a backend becomes available
Re-execution of queries
![Page 22: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/22.jpg)
Multiplexing
Reduce the number of connections against mysqld (configurable)
Many clients connections (tens of thousands) can use few backend connections (few hundreds)
Tracks connection status (transactions, user variables, temporary tables, etc)
Order by waiting time
![Page 23: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/23.jpg)
Monitoring Module
It monitors backends and collects metrics
Monitors replication lag and shun hosts
Monitors read_only variables (replication hostgroups)
Ping and terminates unresponsive nodes
![Page 24: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/24.jpg)
Failover with ProxySQL
![Page 25: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/25.jpg)
Failover
2 phases process:
remove host
add host
Seamless switchover:
http://proxysql.blogspot.com/2015/09/proxysql-tutorial-seamless-replication.html
Managed by external process
Switchover in less than 1 second
![Page 26: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/26.jpg)
Distributed failover
Multiple ProxySQL are available in a network
Failover managed by an external process that:
remove host from each ProxySQL instance
add new host into each ProxySQL instance
Manager is not part of ProxySQL.
Ex: MHA or MySQL Utilities
![Page 27: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/27.jpg)
Replication hostgroups
Constantly monitor read_only variable
Defines replication topology as writer(s)/readers(s)
Automatically re-assign servers to the right hostgroup
Possible to define unlimited number of replication clusters
![Page 28: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/28.jpg)
Replication hostgroups
https://www.nylas.com/blog/growing-up-with-mysql/
Failover without ProxySQL (10 minute outage)
![Page 29: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/29.jpg)
Replication hostgroups
https://www.nylas.com/blog/growing-up-with-mysql/
Failover with ProxySQL (10 second outage)
![Page 30: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/30.jpg)
Failover highlight
improve failover time as perceived by the application
prevent errors sent to the application
perform transparent database failovers: gracefully redirecting traffic without the application knowing
existing applications do not have to be rewritten to autoreconnect since connections are not lost from failovers
![Page 31: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/31.jpg)
Admin Module
![Page 32: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/32.jpg)
Admin Interface
Allows runtime configuration
Exports internal statuses
It uses MySQL protocol
Configuration possible from any client/tool using MySQL API
Covered during the tutorial
![Page 33: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/33.jpg)
Try it!
Source code on GitHub:
https://github.com/sysown/proxysql/
Forum:
https://groups.google.com/forum/#!forum/proxysql
Tutorials on:
http://www.proxysql.com
Join us at booth #102
![Page 34: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/34.jpg)
Demo environment
![Page 35: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/35.jpg)
Demo
Code on GitHub:
https://github.com/dtest/plam16-proxysql
Run locally: Docker, Ansible
Some hosts provided:
Username: plam
Passwords: proxysql
![Page 36: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/36.jpg)
The 3 Rs of ProxySQL and related statistics
![Page 37: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/37.jpg)
ProxySQL
● The 3 Rs○ Rules○ Rewrite○ Routing
● Related Statistics○ Identify offensive queries and resources consumed○ View results of Remediation
![Page 38: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/38.jpg)
ProxySQL Troubleshooting Methodology
● Identify problem via stats tables● Arrive at a solution
○ Match columns○ Action columns
![Page 39: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/39.jpg)
Identify the problem
● stats_mysql_query_rules● stats_mysql_commands_counters● stats_mysql_processlist● stats_mysql_connection_pool● stats_mysql_query_digest● stats_mysql_query_digest_reset● stats_mysql_global
![Page 40: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/40.jpg)
![Page 41: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/41.jpg)
● mysql_users○ active○ fast_forward(bypass)
● global_variables● mysql_collations
○ Default(utf8)○ No stats
Match Related Tables
● mysql_rules○ username○ schemaname○ flagIN○ client_addr○ proxy_addr○ proxy_port○ digest○ match_digest○ match_pattern○ Negate_match_pattern
![Page 42: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/42.jpg)
Action related fields
● flagOUT● replace_pattern● destination_hostgroup● cache_ttl● reconnect● timeout● retries● delay● mirror_flagOUT● mirror_hostgroup● error_msg● log● apply
● active
![Page 43: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/43.jpg)
Let the fun begin
● ssh <your-ip>● sudo su -● cd /root/dba/admin/dev/plam16-proxysql/● ./run_proxy.sh● docker exec -it proxysql bash (at least to terminals)● export TERM=ansi● cd /root/plam-rrr● Please resist viewing the files( I will tell you why )
![Page 44: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/44.jpg)
Stats scripts
● show_stats_mysql_commands_counters.sh● show_stats_mysql_connection_pool.py● show_stats_mysql_global.sh● show_stats_mysql_processlist.sh● show_stats_mysql_query_digest.sh● show_stats_mysql_query_digest_reset.sh
![Page 45: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/45.jpg)
3 Levels for ProxySQL Tables
● Disk● Memory● Runtime
![Page 46: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/46.jpg)
Scenarios
● p1.py● p2.py● p3.py● p4.py
![Page 47: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/47.jpg)
Monitoring scripts
● show_mysql_query_rules.sh● show_mysql_servers.sh● show_mysql_users.sh● show_stats_mysql_commands_counters.sh● show_stats_mysql_connection_pool.py● show_stats_mysql_global.sh● show_stats_mysql_processlist.sh● show_stats_mysql_query_digest.sh● show_stats_mysql_query_digest_reset.sh● show_stats_mysql_query_rules.sh
![Page 48: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/48.jpg)
Precautions
● Confirm the query hits with stats_mysql_query_rules● double check rules and rewrite results● hold off on writing rules to disk until you're sure they are
working as expected● have a rollback plan
○ load from disk, then memory to runtime○ load from repo, then memory to runtime and later load to disk
● test select query rules and rewrites on a slaves first● test mutable rules on a mirror or other throw away db.
![Page 49: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/49.jpg)
● Derek Downey● Krzysztof Ksiazek● Marco Tusa
● Rene Cannao
Links● https://github.com/sysown/proxysql/tree/master/doc● https://www.percona.com/blog/2016/08/30/mysql-sharding-with-proxysql/● https://tusacentral.net/joomla/index.php/mysql-blogs/183-proxysql-percona-
cluster-galera-integration.html● http://severalnines.com/blog/how-proxysql-adds-failover-and-query-control-y
our-mysql-replication-setup
Thank you
![Page 50: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/50.jpg)
Uber is hiring
● Growing and talented team● Embracing Open Source Technologies● Awesome projects awaiting you● Cool campuses● Build the future● Uber Eats / Monthly credits (weekend coverage)● Automated vehicles for all employees (Derek)
![Page 51: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/51.jpg)
Failover with ProxySQL
![Page 52: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/52.jpg)
Failover scenario - without ProxySQL
![Page 53: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/53.jpg)
Failover scenario - without ProxySQL
![Page 54: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/54.jpg)
Failover highlights
improve failover time as perceived by the application
prevent errors sent to the application
perform transparent database failovers: gracefully redirecting traffic without the application knowing
existing applications do not have to be rewritten to autoreconnect since connections are not lost from failovers
![Page 55: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/55.jpg)
Failover scenario - with ProxySQL
![Page 56: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/56.jpg)
Failover scenario - with ProxySQL
![Page 57: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/57.jpg)
Failover scenario - with ProxySQL
![Page 58: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/58.jpg)
Failover scenario - with ProxySQL
![Page 59: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/59.jpg)
ProxySQL Failover ProTIP
ProxySQL does not handle promotion or re-slaving
External process needed, such as MHA
![Page 60: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/60.jpg)
Failover - MySQL Servers
mysql> SELECT hostgroup_id, hostname, status FROM mysql_servers
WHERE hostname IN ('master', 'slave')\G
*************************** 1. row ***************************
hostgroup_id: 1
hostname: master
status: ONLINE
*************************** 2. row ***************************
hostgroup_id: 2
hostname: slave
status: ONLINE
2 rows in set (0.00 sec)
![Page 61: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/61.jpg)
Failover - Replication Hostgroups
mysql> SELECT * FROM mysql_replication_hostgroups\G
*************************** 1. row ***************************
writer_hostgroup: 1
reader_hostgroup: 2
comment:
1 row in set (0.00 sec)
![Page 62: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/62.jpg)
Failover Demonstration
![Page 63: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/63.jpg)
Mirroring with ProxySQL
![Page 64: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/64.jpg)
What is mirroring?
![Page 65: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/65.jpg)
What is mirroring?
![Page 66: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/66.jpg)
Why mirror queries?
Validate performance on a different server using different hostgroups.
Validate performance of query rewrite or schema change
Pre-fetch slave replication (Replication Booster)
![Page 67: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/67.jpg)
How to mirror
mysql>SHOW CREATE TABLE mysql_query_rules\G
*********************** 1. row ************************
table: mysql_query_rules
Create Table: CREATE TABLE mysql_query_rules (
rule_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 0,
username VARCHAR,
schemaname VARCHAR,
flagIN INT NOT NULL DEFAULT 0,
*snip*
mirror_flagOUT INT UNSIGNED,
mirror_hostgroup INT UNSIGNED,
error_msg VARCHAR,
log INT CHECK (log IN (0,1)),
apply INT CHECK(apply IN (0,1)) NOT NULL DEFAULT 0,
comment VARCHAR)
![Page 68: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/68.jpg)
Mirroring flow
![Page 69: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/69.jpg)
Mirroring - example #1
mysql> SELECT username, destination_hostgroup,
mirror_hostgroup, mirror_flagOUT FROM mysql_query_rules
WHERE username='plam_mirror'\G
****************** 1. row **********************
username: plam_mirror
destination_hostgroup: 3
mirror_hostgroup: 4
mirror_flagOUT: NULL
![Page 70: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/70.jpg)
Mirroring - example #2
![Page 71: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/71.jpg)
Mirroring Demonstration
![Page 72: ProxySQL Tutorial High Performance & High Availability Percona … · 2016-10-14 · ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Amsterdam,](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e67f414537ae04a2f3f0256/html5/thumbnails/72.jpg)
Questions?