southeast linuxfest -- mysql queries-- - damned lies and best guesses
DESCRIPTION
This session covers how to understand what MySQL is trying to tell you about queries.TRANSCRIPT
![Page 1: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/1.jpg)
MySQL Queries -- DamnedLies and Best GuessesVery few programmers really understand SQL or how to speed queries to their databases. This session covers that basics of relational calculus (no actual math/calculus will be demanded of attendees), how a RDMS like MySQL tries to optimize the query, and introduces query tuning.
Dave [email protected]@stokerslideshare.net/davestokes
![Page 2: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/2.jpg)
SQL Structured Query Language
Structured Query Language (/ˈɛs kjuː ˈɛl/,[4] or /ˈsiːkwəl/; (SQL)[5][6][7][8]) is a
special-purpose programming language designed for managing data held in a
relational database management system (RDBMS).
Originally based upon relational algebra and tuple relational calculus, SQL consists of a data definition language
and a data manipulation language. The scope of SQL includes data insert, query, update and delete, schema
creation and modification, and data access control. Although SQL is often described as, and to a great extent is, a
declarative language (4GL), it also includesprocedural elements.
relational algebra is an offshoot of first-order logic and of algebra of sets concerned with operations over
finitary relations, usually made more convenient to work with by identifying the components of a tuple by a name
(called attribute) rather than by a numeric column index, which is called a relation in database terminology.
--Wikipedia
![Page 3: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/3.jpg)
![Page 4: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/4.jpg)
You send SQL to the server …
The mysqld process will take your input and parse it for VALID syntax.
Then it will build a query plan on how best to retrieve the data.
Finally it goes to fetch the data.MySQL’s NoSQL queries that skip these steps are MUCH faster
![Page 5: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/5.jpg)
GOALs
1. Get the data that you need and only what you need as fast as possible. No ‘SELECT * FROM’
2. Avoid unnecessary disk/memory reads and disk writes.
3. Make data as compact as is usefull, no BIGINTs for zipcodes.
![Page 6: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/6.jpg)
Cost Based Optimizer
C.5.6. Optimizer-Related Issues
MySQL uses a cost-based optimizer to determine the best way to resolve a query. In many cases, MySQL can calculate the
best possible query plan, but sometimes MySQL does not have enough information about the data at hand and has to make
“educated” guesses about the data.
So MySQL wants to get your data as cheaply as possible and plans accordingly.
![Page 7: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/7.jpg)
Query Plan not lockable with MySQL
Each time MySQL gets a query it will optimise it!
It builds a list of statistics over time to help keep track of data & speed retrieval of the data (5.6 lets you save/restore this information)
Clue: You want FAST!
![Page 8: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/8.jpg)
EXPLAIN
EXPLAIN is a tool to ask the server how it wants to optimize the query.
Prepend to a QUERY
![Page 9: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/9.jpg)
Example Table
![Page 10: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/10.jpg)
Example Query
![Page 11: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/11.jpg)
Example EXPLAIN
![Page 12: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/12.jpg)
Example EXPLAIN with \G
Query#
![Page 13: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/13.jpg)
A Quick Word on Indexes
Indexes allow you to go directly to the record(s) you want (think SSN) instead of reading all records to find the one(s) wanted.
But they require maintenance and overhead.
Not a panacea!
![Page 14: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/14.jpg)
What the heck is a B-Tree?!??!?!
![Page 15: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/15.jpg)
select_type
![Page 16: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/16.jpg)
Using WHERE
274 Records Read!
![Page 17: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/17.jpg)
Previous query w/o INDEX
No index used and all records in table readto find 274 re
![Page 18: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/18.jpg)
How to find index(es) already in use
![Page 19: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/19.jpg)
OR ...
![Page 20: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/20.jpg)
A more common example
Optimizer estimates 8 reads to get desired info
Could use the PRIMARY key but does not!!
Has to read all records
239 x 8 = 1,912 records to read
![Page 21: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/21.jpg)
Slightly more complex query
SELECT a.Name as 'City',
b.Name as 'Country',
a.population
FROM City a
JOIN Country b
ON (a.CountryCode = b.Code)
WHERE a.population > 3000000
AND b.LifeExpectancy > 66
ORDER BY b.name, a.Population
LIMIT 20;
![Page 22: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/22.jpg)
Gee, we add all those qualifiers and it still has to readAll those records AND we get a temp table plus a file sort!
And we ONLY wanted 20 records!!!
![Page 23: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/23.jpg)
Visual Explain
MySQL 5.6 and
Workbench 6.1 use
JSON format output
to generate diagram.
Costs published with
5.7 and 6.1!
![Page 24: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/24.jpg)
Yet a little deeper into complexitySELECT CONCAT(customer.last_name, ', ', customer.first_name) AS
customer,
address.phone, film.title
FROM rental INNER JOIN customer ON rental.customer_id = customer.customer_id
INNER JOIN address ON customer.address_id = address.address_id
INNER JOIN inventory ON rental.inventory_id = inventory.inventory_id
INNER JOIN film ON inventory.film_id = film.film_id
WHERE rental.return_date IS NULL
AND rental_date + INTERVAL film.rental_duration DAY < CURRENT_DATE()
LIMIT 5;
![Page 25: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/25.jpg)
**
****
****
****
****
****
****
* 1.
row
***
****
****
****
****
****
****
id: 1
sel
ect_
type
: SIM
PLE
ta
ble:
film
t
ype:
ALL
poss
ible
_key
s: P
RIM
AR
Y
key
: NU
LL
ke
y_le
n: N
ULL
r
ef:
NU
LL
row
s: 1
000
E
xtra
: N
ULL
****
****
****
****
****
****
***
2. r
ow *
****
****
****
****
****
****
**
id
: 1
sele
ct_t
ype:
SIM
PLE
ta
ble:
inve
nto
ry
typ
e: r
efpo
ssib
le_k
eys:
PR
IMA
RY,
idx_
fk_
film
_id
k
ey: i
dx_
fk_f
ilm_i
d
ke
y_le
n: 2
r
ef:
sak
ila.f
ilm.f
ilm_i
d
row
s: 2
E
xtra
: U
sing
inde
x**
****
****
****
****
****
****
* 3.
row
***
****
****
****
****
****
****
id: 1
se
lect
_typ
e: S
IMP
LE
tabl
e: r
enta
l
typ
e: r
efpo
ssib
le_k
eys:
idx_
fk_i
nve
nto
ry_i
d,id
x_fk
_cus
tom
er_
id
key
: id
x_fk
_in
vent
ory_
id
ke
y_le
n: 3
r
ef:
sak
ila.in
vent
ory.
inve
ntor
y_id
r
ows:
1
Ext
ra: U
sin
g w
here
****
****
****
****
****
****
***
4. r
ow *
****
****
****
****
****
****
**
id
: 1
sele
ct_t
ype:
SIM
PLE
ta
ble:
cus
tom
er
typ
e: e
q_re
fpo
ssib
le_k
eys:
PR
IMA
RY,
idx_
fk_
addr
ess_
id
key
: PR
IMA
RY
key_
len:
2
re
f: s
akila
.ren
tal.c
ust
omer
_id
r
ows:
1
Ext
ra:
NU
LL**
****
****
****
****
****
****
* 5.
row
***
****
****
****
****
****
****
id: 1
se
lect
_typ
e: S
IMP
LE
tabl
e: a
ddr
ess
t
ype:
eq_
ref
poss
ible
_key
s: P
RIM
AR
Y
key
: PR
IMA
RY
key_
len:
2
re
f: s
akila
.cu
sto
mer
.add
ress
_id
r
ows:
1
Ext
ra:
NU
LL5
row
s in
set
(0.
00 s
ec)
![Page 26: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/26.jpg)
A little easier to understand
![Page 27: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/27.jpg)
Compound Indexes
We can use this index searching on
1. City, State, and Zip
2. City, State
3. City
![Page 28: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/28.jpg)
Covering Indexes
ALTER TABLE city ADD INDEX country_idx (CountryCode, Population);
The INDEX contains all the data we are searching for which means less data to look-up
![Page 29: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/29.jpg)
Musts for better queries
1. Read chapter 8 of the MySQL Manual
2. Join on like data types, INTs with INTS
3. Keep columns as small as practical (PROCEDURE ANALYSE)
4. Maintain B-tree index with ANALYSE TABLE when things are quiet
5. Keep looking for improvments
![Page 30: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/30.jpg)
Hard to teach all in a few minutes
![Page 31: Southeast Linuxfest -- MySQL Queries-- - Damned Lies and Best Guesses](https://reader033.vdocuments.net/reader033/viewer/2022052820/54b6eacf4a7959ff2d8b4629/html5/thumbnails/31.jpg)
MySQL Central @ Oracle Open WorldMySQL Central @ Oracle Open World
Five days with the MySQL Engineers, innovative customers (Facebook, Twitter, Playful Play, Verizon, Paypal, & you), and the top professionals from the MySQL Community. Early Bird registration saves $500 before July 18th!
Starts September 28th in San Francisco!