monday, september 23, 13 - khan kennels client programs and the mysqld server %allusersprofile%...

$: Monday, September 23, 13 - Khan Kennels Client programs and the mysqld server %ALLUSERSPROFILE% \MySQL\MySQL Server 5.6\ Log files and databases (Windows XP, Windows Server 2003) The$
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1

Monday, September 23, 13

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

Insert Picture Here

2

Getting Started with MySQL:Learn the Essentials

Ligaya Turmelle Principle Technical Support Engineer - [email protected]@oracle.com



THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO

ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD

NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR

ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE.

3

Safe Harbor



Agenda - Part 1

Foundation Installation Talking to MySQL Architecture DataTypes



Agenda - Part 2

Continue DataTypes Log Files Security Backup Monitoring



Insert Picture Here

6

Foundation



Basics



MySQL in a Nutshell

▪Worlds most popular open source database

▪ “M” of LAMP

▪Main site: mysql.com

8



MySQL.com

▪mysql.com

▪ Commercial site

▪ http://dev.mysql.com

▪ Developer Zone

▪ http://downloads.mysql.com

▪ http://dev.mysql.com/doc/

▪ Documentation

▪ bugs.mysql.com9



Getting Help

▪Manual

▪MySQL Support

▪MySQL Forums (forums.mysql.com)

▪ Search the Web

▪ IRC on freenode - #mysql

10



Main Features

▪Works on multiple platforms

▪Multi-layered server design

▪ Pluggable storage engine architecture

▪Multi-threaded

▪ “Plays well” with most programming languages

11



Commercial



Enterprise Edition Add-ons

▪MySQL Enterprise Backup (MEB)

▪MySQL Enterprise Monitor (MEM)

▪Workbench

▪MySQL Enterprise Audit

▪MySQL Enterprise Security

▪ Thread pool plugin

13



Insert Picture Here

14

Installation



▪ Files

15



Option File

▪ AKA configuration files

▪ Lists specific startup options

16

# The following options will be passed to all MySQL clients[client]#password = your_passwordport = 3306socket = /tmp/mysql.5.1.66.sock

# Here follows entries for some specific programs

# The MySQL server[mysqld]port = 3306socket = /tmp/mysql.5.1.66.socklog-error = /Users/ligaya/mysql_installs/mysql-5.1.66-osx10.6-

x86_64/data/error.logserver-id = 1table_open_cache = 4000



Common Locations of Option File

▪Windows:

▪ INSTALLDIR\my.ini▪ INSTALLDIR\my.cnf

▪ Linux & Mac:

▪ /etc/my.cnf▪ /etc/mysql/my.cnf

17



Generic Unix/Linux Binary File Layout

18

Directory Contents of Directorybin Client programs and the mysqld server

data Log files and databases

docs Manual in Info format

man Unix manual pages

include include (header) files

lib Libraries

scripts mysql_install_db

share Miscellaneous support files, including error messages, sample option files, and SQL for database installation

sql-bench Benchmarks



Linux RPM File Layout

19

Directory Contents of Directory/usr/bin Client programs and scripts

/usr/sbin mysqld server

/var/lib/mysql log files and databases

/usr/share/info Manual in Info format

/usr/share/man Unix manual pages

/usr/include/mysql

Include (header) files

/usr/lib/mysql Libraries

/usr/share/mysql

Miscellaneous support files, including error messages, character set files, sample option files, and SQL for database installation

/usr/share/sql-bench

Benchmarks



Windows File Layout

20

Directory Contents of Directory Notesbin Client programs and the mysqld server

%ALLUSERSPROFILE%\MySQL\MySQL Server 5.6\

Log files and databases (Windows XP, Windows Server 2003)

The Windows system variable %ALLUSERSPROFILE% defaults to C:\Documents and Settings\All Users\Application Data

%PROGRAMDATA%\MySQL\MySQL Server 5.6\

Log files and databases (Windows 7 Windows Server 2008, and newer)

The Windows system variable %PROGRAMDATA% defaults to C:\ProgramData

examples Example Programs and scripts

include include (header) files

lib Libraries

scripts mysql_install_db

shareMiscellaneous support files, including error messages, sample option files, and SQL for database installation



▪ Installation

21



Installation

▪Well documented in the Manual

▪ Each OS is documented

▪ Unix/Linux Generic Binaries

▪Windows

▪ Mac

▪ RPM Packages / Debian Packages/ Native Package Managers

▪ Source

22



Generic Binary Installation

23

shell> groupadd mysqlshell> useradd -r -g mysql mysqlshell> cd /usr/localshell> tar zxvf /path/to/mysql-VERSION-OS.tar.gzshell> ln -s full-path-to-mysql-VERSION-OS mysqlshell> cd mysqlshell> chown -R mysql .shell> chgrp -R mysql .shell> scripts/mysql_install_db --user=mysqlshell> chown -R root .shell> chown -R mysql data



Starting it up

▪Methods

▪ mysqld▪ mysqld_safe▪ mysql.server▪Windows Service

▪Unix/Linux - run as a non-root account

▪Windows - admin rights

24



Shutting it down

▪Manually

▪ mysqladmin shutdown▪ /etc/init.d/mysql stop▪ NET STOP MySQL▪ Services Manager

25



Insert Picture Here

26

Talking to MySQL



CLI - mysql client

▪ Easy to use

▪ Simple SQL shell

▪ Server side help

▪Many options

▪Own commands

27

mysql> select count(*) from inventory;+----------+| count(*) |+----------+| 4581 |+----------+1 row in set (0.08 sec)

mysql> ALTER TABLE rental DROP FOREIGN KEY `fk_rental_inventory`;Query OK, 0 rows affected (0.25 sec)Records: 0 Duplicates: 0 Warnings: 0

mysql> SELECT rental_id from rental order by RAND() LIMIT 10;+-----------+| rental_id |+-----------+| 12539 || 8592 || 15699 |



Workbench

▪GUI

▪ Functionality

▪ SQL Development

▪ Data Modeling

▪ Server Administration

▪ Data Migration

▪ Integrated with MEB and Audit

28



SQL Editor in Windows

29



Administration in Linux

30



Insert Picture Here

31

Architecture



▪High Level

32



High Level MySQL Architecture

33



▪ Storage Engines

34



InnoDB

▪General-purpose storage engine

▪ as of 5.5 is the default engine

▪ Some Features:

▪ ACID compliant

▪ Row level locking with consistent reads

▪ Foreign keys

▪ Clustered primary key index

▪ FullText indexes (5.6)35



MyISAM

▪Default storage engine prior to 5.5

▪ stored on disk as 3 files

▪ .frm▪ .MYD▪ .MYI

▪ Features:

▪ FullText indexes

▪ Geospacial datatype support36



More

▪Memory

▪CSV

▪ Blackhole

▪ Archive

▪ etc.

37



Insert Picture Here

38

DataTypes



String

▪ CHAR/VARCHAR

39



String (con’t)

▪ CHAR/VARCHAR▪ BINARY/VARBINARY

40



String (con’t)

▪ CHAR/VARCHAR▪ BINARY/VARBINARY▪ BLOB/TEXT

41



String (con’t)

▪ CHAR/VARCHAR▪ BINARY/VARBINARY▪ BLOB/TEXT▪ ENUM

42

CREATE TABLE shirts ( name VARCHAR(40), size ENUM('x-small', 'small', 'medium', 'large', 'x-large');

INSERT INTO shirts (name, size) VALUES ('dress shirt','large'), ('t-shirt','medium'), ('polo shirt','small');



String (con’t)

▪ CHAR/VARCHAR▪ BINARY/VARBINARY▪ BLOB/TEXT▪ ENUM▪ SET

43

mysql> CREATE TABLE myset (col SET('a', 'b', 'c', 'd'));

mysql> INSERT INTO myset (col) VALUES ('a,d'), ('d,a'), ('a,d,a'), ('a,d,d'), ('d,a,d');Query OK, 5 rows affected (0.01 sec)Records: 5 Duplicates: 0 Warnings: 0



Numeric - Integer Types

▪ Exact value integer types

44

Storage Minimum Value Maximum Value

TINYINT 1-128 127

TINYINT 10 255

SMALLINT 2-32768 32767

SMALLINT 20 65535

MEDIUMINT 3-8388608 8388607

MEDIUMINT 30 16777215

INT 4-2147483648 2147483647

INT 40 4294967295

BIGINT 8-9223372036854780000 9223372036854775807

BIGINT 80 18446744073709600000



Numeric - Fixed Point Types

▪ Exact value fixed-point types

▪ DECIMAL▪ NUMERIC

▪ Important to preserve exact precision

▪ Ex: Monetary data

▪ Stores in binary format

45

salary DECIMAL(5, 2)



Numeric - Floating-Point Types

▪ Approximate value floating-point types

▪ FLOAT▪ 4 bytes

▪ DOUBLE▪ 8 bytes

▪Rounding performed when storing values

▪ Exact comparisons can lead to problems

46



Temporal

47



Temporal - DATE

▪ Format: YYYY-MM-DD

▪ Supported range: ‘1000-01-01’ - ‘9999-12-31’

▪ “Zero” Value: ‘0000-00-00’

▪ Assignment can be made with strings or numbers

▪ ‘Relaxed” formats accepted for string values

48

‘10.11.12’ -> 2010-11-12’04-04-31’ -> 0000-00-00‘10:45:15’ -> 0000-00-00



Temporal - TIME

▪Displays as ‘HH:MM:SS[.fraction]’ or ‘HHH:MM:SS[.fraction]’

▪Uses

▪ Time of day (24 hrs)

▪ Elapsed time or time interval (-838:59:59 to 838:59:59)

▪Recognizes time in multiple formats

▪ some include fractional seconds (6 digit precision)

▪ ‘Zero’ value: 00:00:0049



Temporal - YEAR

▪ 1 byte

▪ Format: YYYY

▪Range: 1901 - 2155

▪ ‘Zero’ value: 0000

▪ Variety of input formats are accepted

50



Temporal - DATETIME

▪Date and time combination

▪ Format: YYYY-MM-DD HH: MM:SS[.fraction]

▪Range: '1000-01-01 00:00:00.000000'

to '9999-12-31 23:59:59.999999'

▪ Assignment can be made with strings or numbers

▪ (5.6) Optional

▪ Automatic initialization and updating to current date & time

51



Temporal - TIMESTAMP

▪Range: '1970-01-01 00:00:01.000000' UTC to '2038-01-19 03:14:07.999999' UTC.

▪ Stored as seconds since epoch

▪ ‘Zero’ value: ‘0000-00-00 00:00:00’

▪Complicated

▪ Automatic initialization and updating to current date & time available

▪ (5.6) explicit_defaults_for_timestamp system variable

52



Insert Picture Here

53

Log Files



Error Log

▪ --log-error[=file_name]▪Default location: host_name.err in datadir

▪ logs

▪ start and stops

▪ critical errors

▪ InnoDB Monitor data (if turned on)

▪ MyISAM tables that need to be checked or repaired

▪ some OS’s - stack trace if mysqld crashes54



Slow Query Log

▪ First line of offense for tuning queries

▪Why?

▪ Performance usually

▪ Enable dynamically or with --slow-query-log

▪ Default location: host_name-slow.log in the datadir

▪Multiple options for controlling it

▪ aggregate its data with mysqldumpslow utility

55



EXPLAIN

▪Displays query execution plan

▪ Syntax:

▪ Prior to 5.6:

▪ EXPLAIN [explain_type] SELECT select_options▪ 5.6:

▪ EXPLAIN [explain_type] explainable_stmt

56



EXPLAIN(con’t)

▪ Talk all by itself

▪ 1 row == 1 table

▪Order matters

▪ EXTENDED + SHOW WARNINGS = rewritten query

▪Columns:

▪ id, select_type, table, partitions, type, possible_keys, key, key_length, ref, rows, filtered, Extra

57



Example CLI EXPLAIN

58

mysql> EXPLAIN EXTENDED SELECT ci.Name, -> co.Name, -> cl.IsOfficial -> FROM City ci -> INNER JOIN Country co -> ON ci.CountryCode = co.Code -> INNER JOIN CountryLanguage cl -> ON co.Code = cl.CountryCode -> WHERE cl.Language = 'ENGLISH' -> LIMIT 100;+----+-------------+-------+--------+---------------+---------+---------+----------------------------+------+-----------------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------+--------+---------------+---------+---------+----------------------------+------+-----------------------+| 1 | SIMPLE | ci | ALL | NULL | NULL | NULL | NULL | 4079 | NULL || 1 | SIMPLE | co | eq_ref | PRIMARY | PRIMARY | 3 | world.ci.CountryCode | 1 | NULL || 1 | SIMPLE | cl | eq_ref | PRIMARY | PRIMARY | 33 | world.ci.CountryCode,const | 1 | Using index condition |+----+-------------+-------+--------+---------------+---------+---------+----------------------------+------+-----------------------+3 rows in set (0.00 sec)



Workbench Visual EXPLAIN - Mac

59



mysqldumpslow

▪ Parses slow query log and summarizes the data

▪Default - groups queries that are similar

▪ “abstracts” out specific string and number data values

▪Options for sorting:

▪ t, at - sort by query time or average query time

▪ l, al - sort by lock time or average lock time

▪ r, ar - sort by rows or average rows sent

▪ c - sort by count60



Example mysqldumpslow output

61

Count: 6 Time=0.36s (2s) Lock=0.00s (0s) Rows=1.0 (6), agent[agent]@localhost SELECT @@version NOT LIKE "S" AND (SELECT COUNT(*) < N FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA != 'S') AND (SELECT COUNT(*) = N FROM INFORMATION_SCHEMA.TABLES WHERE CREATE_OPTIONS LIKE 'S')

Count: 2 Time=0.24s (0s) Lock=0.10s (0s) Rows=1.0 (2), agent[agent]@localhost SELECT ROUND(SUM(sum_timer_wait)/N) total_wait_time_ms, SUM(count_star) total_statements, ROUND(MAX(max_timer_wait)/N) max_wait_time_ms, SUM(sum_errors) total_errors, SUM(sum_warnings) total_warnings, SUM(sum_rows_sent) total_rows_returned, ROUND(SUM(sum_lock_time)/N) total_lock_time_ms FROM performance_schema.events_statements_summary_by_digest

Count: 100 Time=0.22s (22s) Lock=0.00s (0s) Rows=0.0 (0), root[root]@localhost CREATE TABLE `t1` (id serial,id0 varchar(N) unique key,intcol1 INT(N) ,intcol2 INT(N) ,intcol3 INT(N) ,intcol4 INT(N) ,intcol5 INT(N) ,charcol1 VARCHAR(N),charcol2 VARCHAR(N),charcol3 VARCHAR(N),charcol4 VARCHAR(N),charcol5 VARCHAR(N),charcol6 VARCHAR(N),charcol7 VARCHAR(N),charcol8 VARCHAR(N),charcol9 VARCHAR(N),charcol10 VARCHAR(N))

Count: 300 Time=0.20s (60s) Lock=0.00s (0s) Rows=0.0 (0), root[root]@localhost CREATE TABLE `t1` (id serial,id0 varchar(N) unique key,intcol1 INT(N) ,intcol2 INT(N) ,intcol3 INT(N) ,intcol4 INT(N) ,intcol5 INT(N) ,intcol6 INT(N) ,intcol7 INT(N) ,intcol8 INT(N) ,intcol9 INT(N) ,intcol10 INT(N) ,charcol1 VARCHAR(N),charcol2 VARCHAR(N),charcol3 VARCHAR(N),charcol4 VARCHAR(N),charcol5 VARCHAR(N),charcol6 VARCHAR(N),charcol7 VARCHAR(N),charcol8 VARCHAR(N),charcol9 VARCHAR(N),charcol10 VARCHAR(N))



General Query Log

▪General Record

▪Why:

▪ Order is important

▪ Exact query that came in

▪ minimal audit of what a connection did

▪ Enable Dynamically or with --general-log

▪ Default location: host-name.log in the datadir

▪Multiple options for controlling it62



Binary Log

▪ Logs database change events

▪Why?

▪ Replication

▪ Data recovery

▪ Enable with --log-bin

▪ A LOT of Options

▪ “Read” with mysqlbinlog

63



mysqlbinlog

▪Displays binary/relay logs in text format

▪ Sytax: mysqlbinlog [options] log_file

▪Can work with any replication format

▪ Reading RBR

▪ --verbose --base64-output=DECODE-ROWS

64



Example mysqlbinlog output

65

# at 316#130512 20:22:57 server id 1 end_log_pos 364 CRC32 0xa7a9fe8c ! GTID [commit=yes]SET @@SESSION.GTID_NEXT= '6e521de8-bb62-11e2-836e-821afda5425c:2'/*!*/;# at 364#130512 20:22:57 server id 1 end_log_pos 443 CRC32 0xf9bf1aaa ! Query!thread_id=3!exec_time=0!error_code=0SET TIMESTAMP=1368404577/*!*/;BEGIN/*!*/;# at 443#130512 20:22:57 server id 1 end_log_pos 542 CRC32 0x1b82a82c ! Query!thread_id=3!exec_time=0!error_code=0SET TIMESTAMP=1368404577/*!*/;insert into t1 values (1)/*!*/;# at 542#130512 20:22:57 server id 1 end_log_pos 573 CRC32 0x22965d3b ! Xid = 29COMMIT/*!*/;# at 573#130512 20:23:01 server id 1 end_log_pos 621 CRC32 0x89e5d253 ! GTID [commit=yes]SET @@SESSION.GTID_NEXT= '6e521de8-bb62-11e2-836e-821afda5425c:3'/*!*/;



Insert Picture Here

66

Security



Securing a New Installation

▪ Initial accounts

▪ Superuser (root)

▪ Anonymous (“”)

67



User Accounts

▪ 2 parts

▪ username

▪ hostname

▪ Ex:

▪ ‘root’@‘localhost’▪ ‘’@‘127.0.0.1’▪ ‘root’@‘192.168.1.%’▪ ‘lig’@’%’

68



Securing a New Installation (con’t)




▪ test database

69







▪ test database

▪ Securing the easy way

▪ mysql_secure_installation

70







▪ test database

▪ Securing the easy way

▪ mysql_secure_installation

▪Hard way

71



Access Control

▪ 2 stages

72



Access Control

▪ 2 stages

▪ Stage 1 - connecting

▪Who are you?▪ host ▪ username

▪ Prove it!

73



Access Control

▪ 2 stages

▪ Stage 1 - connecting

▪Who are you?▪ host ▪ username

▪ Prove it!▪ Stage 2 - request

▪ for each request▪ what are you doing and are you allowed to do that?

74



GRANT

▪Defines privileges and account characteristics

▪Multiple privileges

▪ Ex: SUPER, CREATE, ALTER, SELECT, INSERT

▪Multiple levels

▪ Ex: Global, Database, Table, Column

▪ Account characteristics

▪WITH/REQUIRE

75



Sample GRANTS

76

mysql> SHOW GRANTS;+---------------------------------------------------------------------+| Grants for root@localhost |+---------------------------------------------------------------------+| GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' WITH GRANT OPTION || GRANT ALL PRIVILEGES ON `mysql`.* TO 'root'@'localhost' || GRANT PROXY ON ''@'' TO 'root'@'localhost' WITH GRANT OPTION |+---------------------------------------------------------------------+3 rows in set (0.00 sec)

mysql> SHOW GRANTS FOR ''@'localhost';+----------------------------------------------------+| Grants for @localhost |+----------------------------------------------------+| GRANT USAGE ON *.* TO ''@'localhost' || GRANT ALL PRIVILEGES ON `test`.* TO ''@'localhost' |+----------------------------------------------------+2 rows in set (0.00 sec)



REVOKE

▪Removes the privileges GRANTed

▪ Does not extrapolate

▪Doesn’t remove the user

▪No host is given

▪ % is used

▪ Does not extrapolate

77



REVOKE clarification

78

mysql> SHOW GRANTS FOR ''@'localhost';+----------------------------------------------------+| Grants for @localhost |+----------------------------------------------------+| GRANT USAGE ON *.* TO ''@'localhost' || GRANT ALL PRIVILEGES ON `test`.* TO ''@'localhost' |+----------------------------------------------------+2 rows in set (0.00 sec)

mysql> REVOKE DELETE ON test.t1 FROM ''@'localhost';ERROR 1147 (42000): There is no such grant defined for user '' on host 'localhost' on table 't1'

mysql> REVOKE USAGE ON *.* FROM ''@'localhost';Query OK, 0 rows affected (0.02 sec)



Pluggable Authentication Module

▪Commercial Extension

▪Capabilities:

▪ External authentication

▪ Unix passwords▪ LDAP▪Windows Active Directory

▪ Proxy user support

79



Insert Picture Here

80

Backups



▪ Backup Types

81



Logical Backups

▪ Saved

▪ Logical structure

▪ Content

▪Machine independent

▪ Slower

▪ Server Up/Warm

▪ Full Granularity

82



mysqldump

▪ Logical

▪Commonly used

▪ Editable text files

▪ Very flexible

▪Doesn’t scale well

83



Physical Backup

▪Raw copies

▪ Faster

▪Compact

▪ File based granularity

▪Usually Server down or Locked

84



Filesystem Snapshot

▪ Physical

▪OS Dependent

▪ Basic Steps

▪ FLUSH TABLES WITH READ LOCK

▪ Take the snapshot

▪ UNLOCK TABLES

▪ copy files over

▪ startup instance and recover85



MySQL Enterprise Backup

▪MySQL’s solution for backups

▪ Physical backup

▪Hot Backup for InnoDB

▪Warm Backup for other storage engines

86



Insert Picture Here

87

Monitoring



SHOW

▪MySQL specific commands

▪Commands for

▪ Metadata

▪ Status information

▪Metric crap-ton

88



Examples

89

MetadataMetadataMetadata

SHOW DATABASES SHOW TRIGGERS SHOW PLUGINS

SHOW CREATE PROCEDURE SHOW CREATE TABLE SHOW CREATE TRIGGER

SHOW INDEXES SHOW GRANTS SHOW TABLES

StatusStatusStatus

SHOW SLAVE STATUS SHOW MASTER STATUS SHOW OPEN TABLES

SHOW ENGINE INNODB STATUS SHOW FULL PROCESSLIST SHOW GLOBAL STATUS



INFORMATION_SCHEMA

▪Mostly metadata

▪ But

▪ GLOBAL_STATUS

▪ PROCESSLIST

▪ INNODB_TRX

▪ INNODB_LOCKS

▪ INNODB_LOCK_WAITS

▪ INNODB_METRICS90



PERFORMANCE_SCHEMA

▪Monitors at a very low level

▪ Available:

▪ Current events

▪ Event histories

▪ Event Summaries

▪Configuration is dynamic

▪ Tables use PERFORMANCE_SCHEMA storage engine

▪Query using SQL91



PERFORMANCE_SCHEMA (con’t)

▪Using the PERFORMANCE_SCHEMA

▪ ps_helper

▪ By Mark Leith▪ Collection of views and procedures▪ Ex: statement_analysis, schema_unused_indexes▪ http://www.markleith.co.uk/ps_helper/

92



PERFORMANCE_SCHEMA(con’t)

▪Using the PERFORMANCE_SCHEMA

▪ Practical P_S

▪ By Todd Farmer▪ Blog post series of practical uses▪ Ex: How idle are you connections, Finding the KILLer▪ http://mysqlblog.fivefarmers.com/tag/performance_schema/

93



MySQL Enterprise Monitor

94



MEM - Dashboard

95



MEM - Query Analysis

96



MEM - Graphs

97



Insert Picture Here

98

Questions?



Insert Picture Here

99

Getting Started with MySQL:Learn the Essentials

Ligaya Turmelle Principle Technical Support Engineer - [email protected]@oracle.com
