mysql 4.0, 4.1 and 5.0 -- at the libre software meeting mysql_metz_juillet_2003.ppt kaj arnö vice...

MySQL 4.0, 4.1 and 5.0-- at the Libre Software Meeting

MySQL_Metz_Juillet_2003.ppt

Kaj ArnöVice President, MySQL AB

Campus du Saulcy, amphithéâtre C1, UFR SHA

Metz 9.7.2003

© MySQL AB 2003

2MySQL 4.0, 4.1 and 5.0 – at the Libre Software Meeting | Kaj Arnö | © MySQL AB 2003 | www.mysql.com

Copyright © 2003 MySQL AB

MySQL Overview: Goals

1. Direction– Past: MySQL 3.23– Present: MySQL 4.0– Next: MySQL 4.1– Future: MySQL 5.0– Schedule forecast

2. Subqueries– Syntax and limitations

3. Unicode, Charsets– COLLATE

4. Geographical Data– OpenGIS

5. Stages of Maturity– Commitments: Speed,

Reliability, Ease, Economy

– Release Philosophy– Stages of Maturity: pre-

Alpha, Alpha, Beta, Gamma, Production



How many of you ...

• Are decision makers? Or rather developers?• Use MySQL today? Have “just” tested MySQL?• Use MySQL under Linux? Windows? Solaris? AIX?• Use MySQL <=3.22? 3.23? 4.0? 4.1? 5.0?• Use MySQL with PHP? Perl? Java? C? ODBC?

Others?• Use Oracle? Microsoft SQL Server? DB2? Other

databases?



MySQL AB, the company

• founded by the developers of MySQL• owns the trademark MySQL™ • owns the Copyright zu dem MySQL-Programmcode• sells commercial MySQL licenses (440 € / Server),

distributes MySQL under GPL (0 €)• delivers commercial services,

MySQL Support (from 1500 € / year) and MySQL Training (2490 € / 5 days) and MySQL Consulting (160-250 € / hour)



Past: MySQL 3.23

• Majority of MySQL’s popularity grewfrom the rein of MySQL 3.23

• Feature set matched what most users needed from a database

• Debut of transactions, MyISAM, replication, fulltext search, and availability under the GNU GPL in the summer of 2000

• MySQL 3.23 is currently retired



Present: MySQL 4.0• Groundwork enhancements for future branches• Performance declared to match that of the

industries biggest players by eWeek (www.mysql.com/eweek)

"Of the five databases we tested, only Oracle9i and MySQL were able to run our Nile application as originally written for 8 hours without

problems."



Present: MySQL 4.0 (continued)

• Major new features:– InnoDB as ”Standard Issue” (previously only in MAX

versions)– Query Cache, huge performance increase– UNION command– Dynamic Variables, no server restarts for parameter

tweaks– Dual threaded replication– FullText Search enhancements– Debut of embedded server (’libmysqld’)

• Standard builds for non-traditional platforms such as Netware, QNX, and S/390

• MySQL 4.0 is available now as production



Next: MySQL 4.1

• Many new and big features added• Major New Features:

– Newer high-speed internal key cache– Sub-Queries and derived tables– Prepared Statements– Binary client/server protocol, faster transmission– OpenGIS (Geographical data)– Internationalization, UTF-8 and fine grained

control of character sets and collation

• MySQL 4.1 is available now as alpha



Future: MySQL 5.0• First push into the Enterprise level market• Feature additions built on MySQL 4.x foundation• Major new features:

– Stored Procedures (SQL-99 syntax)– Missing Core SQL-99 features– Foreign keys for MyISAM with cascading DELETE– Cursor Support– Multi-master replication and Online backup– Full Sub-Queries (MATCH, FOR ALL/ANY/SOME)– OLAP functions for Data Warehousing– More scalable thread/connection manager

• MySQL 5.0 is available now as pre-alpha



Current Schedule Forecasts

• We are currently estimating that MySQL 4.1 will be in alpha for 3-5 months (July-Sept)

• Best estimations put MySQL 5.0 at alpha in 3-5 months also. (available as pre-alpha right now)

• MySQL 5.0 will probably have a long’ish alpha period -- stretching more towards 6 months



Subqueries (MySQL 4.1-Sonderthema 1 von 3)



Current Status of Subqueries in MySQL

• Subquery development started in Feb 2002

• Subqueries are part of MySQL starting with version 4.1.0

• The Subquery code is currently in stabilisation/optimisation phase



What is a Subquery?

• A Subquery allows you to use the result of one query in another query

• An Example:– You have one table: City with data like population and

country for each city in the world– You want to find all cities that are larger than the largest city

in the Ukraine– This is impossible to do in one query without subqueries



Subquery Example

mysql> select * from City where City.Population >

-> (select MAX(Population) from City where country='UKR');

+------+---------------------+---------+-------------------+------------+

| ID | name | Country | District | Population |

+------+---------------------+---------+-------------------+------------+

| 69 | Buenos Aires | ARG | Distrito Federal | 2982146 |

| 130 | Sydney | AUS | New South Wales | 3276207 |

: :

| 3580 | Moscow | RUS | Moscow (City) | 8389200 |

| 3581 | St Petersburg | RUS | Pietari | 4694000 |

| 3769 | Ho Chi Minh City | VNM | Ho Chi Minh City | 3980000 |

| 3793 | New York | USA | New York | 8008278 |

| 3794 | Los Angeles | USA | California | 3694820 |

| 3795 | Chicago | USA | Illinois | 2896016 |

+------+---------------------+---------+-------------------+------------+

61 rows in set (0.01 sec)

The above subquery is non-correlated and hence very efficient



JOINs Instead of Subqueries?

• Many subqueries can be rewritten as JOINs

• People often feel that using subqueries is more natural• An example:

– Two tables: City and Country– The task: Find all countries without citiesmysql> select Country.Name, Country.Code from Country

-> where NOT EXISTS -> (select * from City where Country=Country.Code);

mysql> select Country.Name, Country.Code from Country -> left join City on (Country.Code=City.Country) -> where City.name is NULL;

– The above queries give exactly the same result• The subquery is not yet optimised



Subqueries vs. JOINs

• JOINs can only be used in SELECT statements– Limited usage with UPDATE and DELETE

• Subqueries are supported in the following statements:– SELECT – UPDATE – INSERT – DELETE – REPLACE – DO – SET



Supported Subquery Types 1/2

• Derived Tables SELECT * FROM t1,(SELECT * FROM t2 WHERE a>5) t3

WHERE t1.a=t3.a)

• Expression Sub-Queries: – Value-returning sub-selects

• Single value SELECT t1.a, (SELECT a FROM t2 WHERE t2.b=t1.b) FROM t1

• Row value SELECT t1.a FROM t1 WHERE (a,b)= (SELECT c,d FROM t2 WHERE t2.e=t1.e)



Supported Subquery Types 2/2

• Expression Sub-Query (continue): – Boolean

• EXISTS SELECT t1.a FROM t1 WHERE EXISTS (SELECT * FROM t2 WHERE t2.a>t1.b)

• IN

– Scalar SELECT t1.a FROM t1 WHERE a IN (SELECT a FROM t2)

– Row SELECT t1.a FROM t1 WHERE (a, b) IN (SELECT a, b FROM t2)

• ALL/ANY/SOMESELECT t1.a FROM t1 WHERE a < ALL (SELECT a FROM t2))



Properties of Subquery Types

fullfirst matchedfullfullExecution type

yesno*yes yesCan have LIMIT clause

yesignored*yesyesCan have ORDER BY clause

tablebooleanrowany scalar type

Result type

no*yesyesyesCan be correlated

FROM clause

scalar expressionrowscalar expression

Can appear as

Derived table

EXISTS INALL/ANY/SOME

RowScalar

* Limitations that are planned to be removed in the future (MySQL 5.0)



Derived Tables

• All queries for the derived tables will be executed before the parent query

• The results of the derived table queries will be internally stored as temporary tables and then used in the parent query

Why were derived tables implemented this way?• A derived table can not be dependent on the parent

query• It is very unusual to have several levels of queries with

derived tables that are correlated• This allowed us to make derived tables very efficient



The Execution of a Subquery

• Non-correlated subquery– Execution is done once– A cached result of the subquery is used for the parent query

• Correlated subquery– Preparation and Optimisation is done once– Execution / re-initialisation: the cached value gets updated

every time the parent query needs it (normally once for each row)



Scalar/single Row Subqueries

SELECT ... WHERE (a,b) = (SELECT c,d FROM ...)

• In the above statement c and d are stored in the Item_singlerow_subselect cache

• If the subquery is correlated then the new values will be recomputed for each row of the outer query



EXISTS Subqueries

SELECT ... WHERE EXISTS (SELECT * FROM...)

• The above query will be rewritten internally to the following:SELECT ... WHERE EXISTS (SELECT * FROM...LIMIT 1)

• An EXISTS query only needs one row!



IN/ALL/ANY/SOME Subqueries

• IN/ALL/ANY/SOME subqueries are internally rewritten in the following fashion:

select...where a IN (select b from table t1)

-> select... where EXISTS* (select 1 from t1 where a=b)

select...where a > ALL (select b from table t1)

-> select...where NOT EXISTS* (select 1 from t1 where a <= b)

SELECT...where a > ANY (select b from table t1)

-> select ... where EXISTS* (select 1 from t1 where a > b)

• * The above aren’t true EXISTS as they can return NULL



Currently Unimplemented Features

• UNIQUE• MATCH [UNIQUE] (SIMPLE | PARTIAL |

FULL)• FOR (ALL | ANY | SOME)

The above will be implemented as soon as the optimisation of the already implemented subqueries is done



Future Subquery Development

• The following optimisations are planned to be done in the future:

1. SELECT...WHERE a IN (SELECT key_value ...) should use an index

2. SELECT...WHERE a IN (<non-correlated SELECT>) => a IN (<list of values>)

3. EXISTS -> JOIN

4. The recalculation of correlated subqueries should only take place if the correlated values change



National Character Sets und Unicode (MySQL 4.1-Sonderthema 2 von 3)



Definitions

A character set is a set of symbols and encodings.

A collation is a set of rules for comparing characters in a character set.



What MySQL 4.1 Can Do

• Store strings using a variety of character sets

• Compare strings using a variety of collations • Mix strings with different character sets or collations in

the same server, database, table or even SQL query• Allow specification of character set and collation on

server, database, table, column, connection and literal level



Server Character Set and Collation

mysqld --default-character-set=latin1

mysqld --default-character-set=latin1_german1_ci



Database Character Set and Collation

CREATE DATABASE database_name CHARACTER SET latin1 COLLATE latin1_swedish_ci



Table Character Set and Collation

Example:

CREATE TABLE t1 ( ...fields... ) CHARACTER SET latin1 COLLATE latin1_danish_ci



Column Character Set and Collation

Example:

CREATE TABLE t1 ( s CHAR(10) CHARACTER SET latin1 COLLATE latin1_german1_ci );

Data Types:

CHAR, VARCHAR, TEXT



Character String Literal Character Set and Collation

Examples:

SELECT _latin1 'string';

SELECT _latin1 'string' COLLATE latin1_danish_ci;

SELECT 'string'; -- still working



Examples of the COLLATE clause

SELECT column1 FROM Table1 ORDER BY column1 COLLATE latin1_german2_de

SELECT column1 FROM Table1 GROUP BY column1 COLLATE latin1_german2_de

SELECT MAX(column1 COLLATE latin1_german2_de) FROM Table1

SELECT DISTINCT column1 COLLATE latin1_german2_de FROM Table1

SELECT * FROM Table1 WHERE _latin1 'Müller' COLLATE latin1_german2_de = column1



An Example of the Effect of Collation

SELECT x FROM t ORDER BY x COLLATE <collation name>

MySQLMySQLMySQL

MX SystemsMX SystemsMüller

MufflerMüllerMX Systems

MüllerMufflerMuffler

German DIN-2German DIN-1Swedish/Finnish



Unicode

variable-number-of-bytes variation

utf8UTF-8

16-bit variationucs2UCS-2

DescriptionMySQL nameOfficial Name



Compatibility with the SQL Standard

YES Coercibility

YES _introducer

YES COLLATE clause

YES multiple character sets in one table

YES separate character set / collation objects

Standard SQL? Feature



Spatial Data / OpenGIS (MySQL 4.1-Sonderthema 3 von 3)



OGC and OGC specifications

• OpenGIS Consortium (OGC) • OpenGIS Simple Feature Specifications For SQL • SQL With Geometry Types environment



Supported Spatial Data Formats

• OpenGIS Well-Known Text (WKT) representation. • OpenGIS Well-Known Binary (WKB) representation.



MySQL Spatial Data Types

• GEOMETRY • POINT • LINESTRING • POLYGON • MULTIPOINT • MULTILINESTRING • MULTIPOLYGON • GEOMETRYCOLLECTION



Creating Spatial Columns

CREATE TABLE

mysql> CREATE TABLE g1 (p1 GEOMETRY); Query OK, 0 rows affected (0.02 sec)



Populating Spatial ColumnsINSERT INTO geom VALUES (GeomFromText('POINT(1 1)')) INSERT INTO geom VALUES (GeomFromText('POLYGON((0 0,10 0,10 10,0 10,0

0),(5 5,7 5,7 7,5 7, 5 5))')) INSERT INTO geom VALUES

(GeomFromText('GEOMETRYCOLLECTION(POINT(1 1),LINESTRING(0 0,1 1,2 2,3 3,4 4))'))

INSERT INTO geom VALUES (PointFromText('POINT(1 1)')) INSERT INTO geom VALUES (LineStringFromText('LINESTRING(0 0,1 1,2

2)')) INSERT INTO geom VALUES

(GeomCollFromText('GEOMETRYCOLLECTION(POINT(1 1),LINESTRING(0 0,1 1,2 2,3 3,4 4))'))



Spatial Functions

Four major function groups: • Functions that convert geometries between various formats. • Functions that describe qualitative or quantitative properties of a

geometry. • Functions that describe relations between two geometries. • Functions that create new geometries from existing ones.



Functions To Convert Between Geometries Between Different Formats

GeomFromWKT(string wkt [,integer srid]): geometry – Converts WKT representation into internal geometry format.

GeomFromWKB(binary wkb [,integer srid]): geometry – Converts WKB representation into internal geometry format

AsWKT(geometry g): string – Converts internal geometry format into WKT representation.

AsWKB(geometry g): binary – Converts internal geometry format into WKB representation.

mysql> select AsText(GeomFromText('LineString(1 1,2 2,3 3)'));

+-------------------------------------------------+

| AsText(GeomFromText('LineString(1 1,2 2,3 3)')) |

+-------------------------------------------------+

| LINESTRING(1 1,2 2,3 3) |

+-------------------------------------------------+



Functions To Analyse Geometry Properties

Area(Polygon p):Double – The area of this Polygon, as measured in the spatial

reference system of this Polygon.

StartPoint(LineString l):Point – The start point of this LineString.

X(point p):Double – The x-coordinate value for this point.



Testing Relations On Geometry Minimal Bounding Rectangles

MBRContains(geom1,geom2)

MBRWithin(geom1,geom2)



Spatial Indexes

The most typical spatial searches: • A point query. Searching for all objects that contain a given

point. • A region query. Searching for all objects that overlap a given

region.



Creating Spatial Indexes

• Creating with ALTER TABLE: ALTER TABLE g ADD SPATIAL KEY(g);

• With CREATE TABLE: CREATE TABLE g (g GEOMETRY NOT NULL, SPATIAL INDEX(g));

• With CREATE INDEX: CREATE SPATIAL INDEX sp_index ON g (g);



Using a Spatial IndexSpatial search without index mysql> select fid,AsText(g) from g where mysql> MBRContains(GeomFromText('Polygon((30000 15000,31000 15000,31000 16000,30000 16000,30000

15000))'),g); ..... 20 rows in set (0.46 sec)

The same search with index mysql> SELECT fid,AsText(g) FROM g WHERE mysql> MBRContains(GeomFromText('Polygon((30000 15000,31000 15000,31000 16000,30000 16000,30000

15000))'),g); ..... 20 rows in set (0.00 sec)

Exemining query plan with EXPLAIN mysql> EXPLAIN SELECT fid,AsText(g) FROM g WHERE mysql> MBRContains(GeomFromText('Polygon((30000 15000,31000 15000,31000 16000,30000 16000,30000 15000))'),g); +----+-------------+-------+-------+---------------+------+---------+------+------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+-------+---------------+------+---------+------+------+-------------+ | 1 | SIMPLE | g | range | g | g | 32 | NULL | 50 | Using where | +----+-------------+-------+-------+---------------+------+---------+------+------+-------------+ 1 row in set (0.00 sec)



Commitments• Speed

• Reliability

• Ease

• Economy



Speed -- Commitments

• Latest and greatest features are added while preserving our top priority -- SPEED

• Great attention in terms of development time is spent on NOT ”adding bloat”

• All code is checked/approved by Monty and a small handful of lead developers to ensure optimization



Reliability -- Commitments

• New code must compile and pass the automated test suite before submission for approval

• Integrated memory checker runs (valgrind, purify) to spot memory leaks

• Heavy-duty benchmarking by our in-house Benchmark team

• Open Source community testing from pre-alpha thru gamma ensures solid production releases



Ease -- Commitments

• Ease of use is one of the key things that makes MySQL a success

• Internal task team to develop and address ”Product Installation and Ease of First Use” issues

• If MySQL is not easy to download, install, and be up and running ASAP, then we’re not doing our job



Economy -- Commitments

• Inexpensive commercial licenses

• No restrictions on:– number of connections

– number of CPUs

– amount of memory

• GPL provides try before you buy



Release Philosophy

Release Early

Release Often



Release Early

• pre-Alpha code repository made available immediately after prior goes ”Alpha”

• Binaries provided in Alpha, Beta, and Gamma stages

• Feedback from the community is essential– Developers monitor public discussion lists for

feedback about their features– Users interact first-hand with the developers

• Feedback fuels feature polishing



Release Often

• Our development model cycles every 30 days

• Our goal is to release every cycle (minimum)• This makes our users and customers

happy– Frequent releases keep the community engaged– Customers see action for their requests– Installation base grows– Bug fixes are good!



Schedule Patterns• Our aim is to have each version in

pre-Alpha development for about a year• Alpha lasts 3-6 months• Beta lasts 1-3 months• Gamma aims to be 1 month, but can go to

3 months• Production should go for 6-12 months• A newer branch will not enter Beta until the

preceeding branch has at leastreached Gamma



Stages of Maturity

Development Stages• pre-Alpha• Alpha• Beta• Gamma

Production Stages• Production• Retired• End of Life



pre-Alpha -- Stages of Maturity• Source code available via public Bitkeeper

repository• DIY, build your own• No binaries provided by MySQL AB• Useful for evaluation purposes

– Testing a new feature– Convincing your boss that yes, Sub-Queries are

coming soon

• Feedback with MySQL developers is highly prized



Alpha -- Stages of Maturity

• Official binaries are provided from www.mysql.com

• Alpha is still in development• Feature set may change• Connecting APIs may change• Major bugs and/or data corruption is more

likely (than other mature stages)



Beta -- Stages of Maturity

• Feature set frozen (mostly), minor feature changes are allowed for:– Design tweaks– Good argumentation that

convinces Monty– By high paying customers

• Major bug hunting stage• Might be used in production by the cutting

edge users that are very brave and knowledgeable



Gamma -- Stages of Maturity

• Feature set is FROZEN SOLID, very minor feature changes could be added for very high paying customers (and a case of vodka for Monty)

• Final polishing for bug fixes• We encourage production use for our

friends in the community that will give it a real work-out



Production -- Stages of Maturity• Only bug fixes• Most any bugs are fixed until there is a “gamma” of

the upcoming branch– For instance: 3.23 vs. 4.0-gamma

• Once the upcoming development branch has reached “gamma” only “hard bugs” are fixed– Core dumps, server process just crashes

– Memory leaks, server process gobbles more and more memory over time

– Data corruption, data in the database is lost or corrupted



Retirement -- Stages of Maturity• The recent 3.23 is an example of ”Retirement”• New releases can occur in retirement

because of:– Major security bugs

– Major memory leaks

– Major data corruption bugs

• Note that these late production stages (retirement, and end-of-life) are primarily for the GPL releases

• For commercial versions, support can continue for longer periods as agreed in licensing contracts



End of Life -- Stages of Maturity

• MySQL 3.22 and earlier would be examples of ”End of Life”

• We keep the last most stable version around (available for download) for historical purposes

• All bug fixing ceases• Again, for commercial versions, support

can continue for longer periods as agreed in licensing contracts



END

mysql 4.0, 4.1 and 5.0 -- at the libre software meeting mysql_metz_juillet_2003.ppt kaj arnö vice...

Documents