Profiling Your Server: Tips & Tricks of the TradeEvan Volgas

“The website is slow”Is it…???• The database• The disk(s)?• Not enough hardware?• DNS? • NGINX / Apache?• Goblins attacking your server?

Profiling your database: Part One

✤ SHOW FULL PROCESSLIST;

✤ You can also use:

SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST WHERE command = "query" ORDER BY info, user, time desc

✤ What you’re looking for is a lot of queries piling up in the database, a few queries taking a long time, a lot of queries taking a long time, lock waits, etc

Profiling your database: Part Two

✤ pt-query-digest

✤ Analyzes your MySQL Slow Query logs and analyzes them by their “fingerprint” to see which types of queries are eating up all your resources

✤ What you are looking for are queries that your database spends the most time doing

✤ Also pay attention to the queries with the highest variance in the time it takes to run them (as given by the V/M ratio).

✤ You may be able to rewrite some of these queries and improve your performance.

HTOP, Glances, Firehol, Top

✤ HTOP: http://hisham.hm/htop/

✤ Firehol: https://github.com/firehol/netdata/

✤ Glances: https://github.com/nicolargo/glances

✤ Top

✤ You are looking for consistently high CPU or memory and/or swapping.

✤ Basically, is your server overwhelmed? If so, what program(s) is (are) overwhelming it?

nginx (or Apache) logs

✤ GoAccess

✤ https://goaccess.io/

✤ What you are looking for is excessive 404s, unusually high traffic, aggressive hosts/bots that are pounding your server, outliers in general

Disk metrics

✤ dstat

✤ vmstat

✤ iostat

✤ iotop

✤ Tools like these are a little more specialized and might be harder to interpret unless you’ve got a fair amount of experience working with servers.

✤ But, if you work with a couple of different sites on a couple of different servers, you may find a server similar to yours that isn’t acting poorly. If you can find one, you might consider comparing the output of these tools across the servers to see if anything stands out to you as weird

✤ Pay close attention to swap and whether or not you’re doing it (if you’re not sure where to look, start with vmstat -S k 1 10 and pay attention to the si and so columns)

Network issues

✤ traceroute

✤ dig, nslookup

✤ vnstat, ifstat, iftop, etc

✤ You are looking for inaccurate name server records, excessive wait times on trace route

✤ You are also looking for spikes in network activity or bandwidth that consistently hovers near some high number (like maybe the network is maxed out)

✤ Basically the two questions you need to ask here are, is it your server that’s acting up or somebody else’s? If it’s your fault, is your network overwhelmed?