tcpdump & xtrabackup @ mysql casual talks #1

Click here to load reader

Post on 10-May-2015

14.572 views

Category:

Technology

0 download

Embed Size (px)

TRANSCRIPT

  • 1.tcpdumpxtrabackup @riywo

2. tcpdumptcpdumpmk-query-digestMyQueryShow(3)xtrabackupInnoDBHotbackupxtrabackup2 3. tcpdump3 4. tcpdumpSQLtcpdumplibpcapmysql(memcachedhttp) 4 5. tcpdumpSQLtcpdump -s 0 -l -w - dst port 3306|strings|perl -e while() { chomp; next if /^[^ ]+[ ]*$/; if(/^(SELECT|UPDATE|DELETE|INSERT|SET|COMMIT|ROLLBACK|CREATE|DROP|ALTER)/i) { if (defined $q) { print "$qn"; } $q=$_; } else { $_ =~ s/^[ t]+//; $q.=" $_"; }}http://www.mysqlperformanceblog.com/2008/11/07/poor-mans-query-logging/5 6. 6 7. tcpdumpSQLperlMySQL 5.1set global long_query_time = 0slow-log(set global) 7 8. mk-query-digestPerconaUtility scriptsMaatkit1http://www.maatkit.orgid:marqs++http://www.slideshare.net/marqs/maatkit-40989458 9. mk-query-digest# wget http://maatkit.googlecode.com/files/maatkit-7041.tar.gz# tar xvzf maatkit-7041.tar.gz# cd maatkit-7041# perl Makefile.PL# make & make install-> /usr/bin/mk-query-digest9 10. mk-query-digest(mysqldumpslow)# mk-query-digest slow.log 10 11. mk-query-digesttcpdump!# tcpdump -s 65535 -x -nn -q -tttt port 3306 >/tmp/tcp.log# mk-query-digest --type tcpdump /tmp/tcp.logmemcachedtcpdump!# tcpdump -s 65535 -x -nn -q -tttt port 11211 >/tmp/tcp.log# mk-query-digest --type memcached /tmp/tcp.log11 12. mk-query-digestreport95--review, --review-historySQL 12 13. mk-query-digestquery_reviewDB# export DSN=h=host,D=db,u=user,p=pw# mk-query-digest --review $DSN,t=query_review--review_history t=query_review_history --create-review-table --create-review-history-table ...13 14. mk-query-digesttcpdump.sh#!/bin/shSEC=$1; HOSTNAME=`hostname`/usr/sbin/tcpdump -s 65535 -x -nn -q -tttt (dst port 3306 orsrc port 3306) and host $HOSTNAME 2> /dev/null &PID=$!; sleep $SEC; kill $PIDmk-qd-review.sh#!/bin/shDSN=$1; TCPFILE=$2/usr/bin/mk-query-digest --no-report --type tcpdump --review-history t=query_review_history --review $DSN,t=query_review--create-review-table --create-review-history-table $TCPFILErm -f $TCPFILE14 15. mk-query-digestcrontab* * * * * /root/tcpdump.sh 30 > /tmp/tcp.log.`date +%Y%m%d_%H%M` 2> /dev/null* * * * * /root/mk-qd-review.sh h=host,D=test,u=test,p=test/tmp/tcp.log.`date -d 1 minutes ago +%Y%m%d_%H%M`2> /dev/null30tcpdumpDB() 15 16. mk-query-digestDBSQLSQLWEBWEB() 16 17. MyQueryShowhttps://github.com/riywo/MyQueryShow 17 18. MyQueryShow18 19. MyQueryShow19 20. MyQueryShowWAFAmon2(tokuhirom++)jQuery + jQuery Grid pluginAPIHTMLRRDToolDB20 21. MyQueryShow$ git clone git://...$ cd MyQueryShow$ cpanm -v --installdeps .RRDs(apt/yum/port,brew)$ plackup MyQueryShow.psgicpanm$ perl -I./lib script/sync_data_to_rrd.plquery_review(_history)RRDcron21 22. MyQueryShowUISQL explainDB 22 23. MyQueryShowmy $dbh = DBI->connect(dbi:mysql:host=host:dbname=test,test,test, {RaiseError => 1,Callbacks => {ChildCallbacks => {execute => sub {my ($obj, @binds) = @_;my (undef, $file, $line) = caller;$obj->{Database}->{Statement} .="/* $file:$line $obj->{Database}->{Name} */";return;},},},});select * from query_reviewwhere checksum = 2668448038919624623/* QueryComment.pm:30 host=localhost:dbname=test */ 24. 24 25. xtrabackup 25 26. InnoDBHotbackupHotbackup()LVMmysqldump --single-transactionibbackup & innobackupxtrabackup & innobackupex 26 27. InnoDBHotbackupLVMmysqldump --single-transactionbegin; mysqldumpibbackup & innobackup 27 28. InnoDBHotbackupxtrabackup & innobackupexibbackup & innobackup Percona++ WAL WAL (InnoDB) 28 29. [ InnoDB Internals: InnoDB File Formats and Source Code Structure ]@MySQL University http://forge.mysql.com/wiki/InnoDB_Internals:_InnoDB_File_Formats_and_Source_Code_Structure 30. 2. datafle1. InnoDBWALCopy of ibdataxtrabackup_logfile3. WAL[ InnoDB Internals: InnoDB File Formats and Source Code Structure ]@MySQL University http://forge.mysql.com/wiki/InnoDB_Internals:_InnoDB_File_Formats_and_Source_Code_Structure 31. xtrabackuprpm/debmakeMySQLbuildhttp://www.percona.com/downloads/XtraBackup/LATEST/xtrabackupWALCinnobackupex-XXXxtrabackupPerlhttp://www.percona.com/docs/wiki/percona-xtrabackup:start31 32. xtrabackupWALxtrabackup# innobackupex-1.5.1 --slave-info --user root--password pass /tmp/xtrabackup/innobackupexstart_ibbackup(); #xtrabackup WALwait_for_ibbackup_suspend(); #mysql_lockall(); #binlogbackup_files(); #.frmmysqlMyISAMresume_ibbackup(); #WALmysql_unlockall(); #32 33. xtrabackup# innobackupex-1.5.1 --apply-log/tmp/xtrabackup/YYYY-mm-dd_HH-MM-SS/xtrabackup: This target seems to be not prepared yet.xtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(538288766)xtrabackup: Starting InnoDB instance for recovery.InnoDB: Log scan progressed past the checkpoint lsn 538288766101211 11:25:25 InnoDB: Database was not shut down normally!InnoDB: Starting crash recovery.InnoDB: Doing recovery: scanned up to log sequence number 538934654 (34 %)101211 11:25:25 InnoDB: Starting an apply batch of log records to the database...InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 ...xtrabackup: starting shutdown with innodb_fast_shutdown = 1101211 11:25:26 InnoDB: Starting shutdown...101211 11:25:31 InnoDB: Shutdown completed; log sequence number 538934654xtrabackup: This target seems to be already prepared.xtrabackup: Starting InnoDB instance for recovery.101211 11:25:32 InnoDB: Log file ./ib_logfile0 did not exist: new to be createdInnoDB: Setting log file ./ib_logfile0 size to 1024 MBInnoDB: Database physically writes the file full: wait...InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000xtrabackup: starting shutdown with innodb_fast_shutdown = 1101211 11:26:07 InnoDB: Starting shutdown... 33101211 11:26:12 InnoDB: Shutdown completed; log sequence number 538934796 34. xtrabackupWALxtrabackuptar4ibdgzip(lzop)# innobackupex-1.5.1 --stream=tar --slave-info --userroot --password pass ./ | gzip - > backup.tar.gzwait_for_ibbackup_suspend(); #xtrabackuptar4ibdSTDOUTmysql_lockall();(STDOUT)xtrabackup --log-stream --target-dir=./WALsuspend34 35. xtrabackup3WALxtrabackupscp# innobackupex-1.5.1 --remote-host=host --slave-info--user root --password pass /tmp/xtrabackup/wait_for_ibbackup_suspend(); #xtrabackupscpmysql_loclall();()xtrabackup --log-stream --target-dir=./WALsuspend35 36. 36 37. MySQL CasualTalks #2 (^^)v 37 38. Thanks Oracle++myfnder++Speakers++Listeners++ 38 39. Thank you for listeningGod bless your MySQL! @riywo39