com336 web database development open source database systems
TRANSCRIPT
COM336 Web Database Development
Open Source Database Systems
Relational Databases
• Advantages over using flat files– RDB can provide faster access to data– RDB can be easily queried to extract sets of data
that fits certain criteria– RDB have built-in mechanism for dealing with
concurrent access (so you don’t have to worry about it)
– RDB have built-in privilege system.
Relational Databases
• The RDB engine that we will use in the module is MySQL
• Before getting into the MySQL specifics, we need to cover:– RDB Concepts and Terminology– Web Database Design
RDB Concepts and Terminology
Relational DB Concepts
• Relational Databases (RDB) are the most common type of databases, they depend on on a sound theoretical basis in relational algebra.
• However, we don’t need to understand relational theory to use them.
• However, some basic concepts need to be understood
Tables
• Relational databases are made up of relations, more commonly called tables
• A table is exactly that: A table of data just like the one produced in a
CustomerID Name Address
1 Bill Jobs 2011 Apple Way, SF, 10064 CA
2 Steve Gates 1 Microsoft Drive, SD, 98711 CA
CUSTOMERS
Columns
• Columns in the table have unique names and contain different data (each column has an associated data type) – Columns are called FIELDS– On the previous example, the first column
(customerID) is an integer and the other two Fields contain text strings
Rows & Values
• Each row in the table represents all the data relating to an individual (in this example, a customer) Rows are also called RECORDS or TUPLES
• Each row consists of a set of individual values that correspond to columns. Each value musst have the data type specified by its column.
Keys
• Usually, in a table, we need a way to uniquely identify each individual row.
• If the table is one like the example, using names might not be the best way of doing this. (think of a 3rd entry: Peter Smith with address 15 Main Street).
• What are the chances of finding another customer named Peter Smith?
• Peter this peter might be the only one living at the given address, but then we are using 2 fields
Keys
• The way around this is to create an additional field (customerID) in this example that contains this unique identifier to the subject
• This identifier field on the table is called the key or the PRIMARY KEY.
• A Key can also be made of more than one field.
Keys
• Databases usually consist of multiple tables and use a keys as a reference from one table to another.
• For example, the following table represents orders placed by the customers:
orderID customerID Amount Date
1 3 27000 10 Aug 2011
2 3 45000 22 Aug 2011
3 2 14270 1 Oct 2011
ORDERS
CustomerID Name Address
1 Bill Jobs 2011 Apple Way, SF, 10064 CA
2 Steve Gates 1 Microsoft Drive, SD, 98711 CA
3 Peter Smith 15 Main Street
orderID customerID Amount Date
1 3 27000 10 Aug 2011
2 3 45000 22 Aug 2011
3 2 14270 1 Oct 2011
CUSTOMERS
ORDERS
Schemas
• The complete set of table designs for a database is called the database schema.
• A schema shows tables along with their columns, data types of the column and indicate the primary key (PK) of each table (and any foreign key)
• Keys on the schema are recognised because they are underlined
Database Schema
• Schema Example
Customers (customerID, Name, Address)Orders (orderID,customerID*, Amount, Date)
Primary Key
Primary KeyForeign Key
Note, the * besides customerID represents a foreign key
How to Design your Web Database
Wed DB Design
• Knowing when you need a new table and what the key should be can be something of an art.
• Most of the time, a few design principles can be followed
Principles
1. Think about the real world objects you are modelling.
– Rule of Thumb: Each class of real world objects modelled will need its own table.
Principles
2. Avoid storing redundant data– On the working example, the customer
information could be stored in the orders table, but now imagine a more complex customer information table with 5 fields and then the order table with another 2 fields
– This “super table” will not only be huge but will also contain redundant data which is a waste of space (imagine Peter Smith doing weekly orders and you having to store all his details every time)
Principles– This kind of super tables can also lead to what is
known as update anomalies (situations where we change the database and end up with inconsistent data)
– There are 3 kinds of update anomalies that need to be avoided:
• Modification (i.e. a customer moves and we need to update the address fields…)
• Insertion (i.e. a new customer is inserted several times and the address needs to be entered correctly each time)
• Deletion (occurs when deleting rows from the database; imagine we delete the fulfilled orders from the database, when all the orders for a single customer have been fulfilled, we will no longer have his address on file)
Principles
3. Use Atomic Colum Values– In each attribute, in each row, store only ONE thing
4. Choose Sensible KEYS– Make sure that the keys you choose are unique
5. Thinks about the questions you want to ask the database
– Make sure that the database contains all the data required, and the appropriate links exists between tables to answer the questions you have
Principles
6. Avoid Designs with Many Empty Attributes– Having many NULL values in the database is a
bad idea, it wastes storage space and causes problems when working out totals and other functions on numerical columns.
– When a user sees a null in a database, they don’t know if it’s because this attribute is irrelevant, whether there is a mistake in the database, or whether the data just has not been entered yet.
MySQL Databases
SQL Language Tutorial
23
Web Database Architecture
web client (e.g browser)
HTTP response
Database
HTTP request
server system
web server PHP Preprocessor
Web Database Architecture
• Typical web database transaction stages:– HTTP request issued from client– Web Server receives request for .php script and passes it
for processing– PHP engine begins parsing the script. The script includes
a command to connect to the database and execute a query. PHP opens a connection to the MySQL server
– MySQL server receives the database query, processes it and sends the result back to the PHP Engine
– PHP Engine finishes running the script – this usually involves the formatting of the results in HTML. Returns resulting HTML to server
– Server passes the HTML back to the browser.
24
25
Web Database Architecture
• Process is basically same regardless of scripting engine or database server
• Often Web Server, script engine and database server all run on the same machine
• As your application grows in size, you may want to separate your PHP application into tiers – a database layer (deals exclusively wit MySQL), a business logic layer (core of application), and a presentation layer (manages the HTML output).
Managing access to the Database
• Access to the database can be provided by using the root administrator level or by creating a user for this database
• For testing purposes, admin access is enough
• For security reasons, deploying using admin level access is NOT recommended.
• MySQL provides different security levels for users, called privileges
26
27
MySQL Privilege System
• MySQL supports a sophisticated privilege system
• Principle of least privilege– A user (or process) should have the lowest level
of privilege required to perform his assigned task.– For example, to run queries, a user does not
need all privileges.• Three basic types of privileges
– Regular Users– Administrators– Special
28
MySQL Privilege System - USERS
Privilege Applies to Description
SELECT Tables, columns Allows users to select rows from tables
INSERT Tables, columns Allows users to insert new rows into tables
UPDATE Tables, columns Allows users to modify values in existing table rows
DELETE Tables Allows users to delete existing table rows
INDEX Tables Allows users to create and drop indexes on particular tables
ALTER Tables Allows users to alter the structure of existing tables
CREATE Databases, tables Allows users to create new databases or tables
DROP Databases, tables Allows users to drop (delete) databases or tables.
29
MySQL Privilege System - ADMINISTRATORS
Privilege Description
CREATE TEMPORARY TABLES Allows an administrator to use the keyword TEMPORARY in a CREATE TABLE statement
FILE Allows data to be read into tables from files and vice-versa
LOCK TABLES Allows the explicit use of a LOCK TABLES statement
PROCESS Allows an administrator to view server processes belonging to all user
RELOAD Allows an administrator to reload grant tables and flush privileges, hosts, logs and tables
REPLICATION CLIENT Allows use of Show Status on replication masters and slaves
REPLICATION SLAVE Allows replication slave servers to connect to the master server.
SHOW DATABASES Allows a list of all databases to be seen with a SHOW DATABASE statement. Without this, users see only databases to which they have other privileges
SHUTDOWN Allows an administrator to shut down the MySQL server
SUPER Allows an administrator to kill threads belonging to any user
30
MySQL Privilege System – SPECIAL
Privilege Description
ALL Grants all privileges listed in previous tables
USAGE Grants no privileges. This privilege creates a user and allows him to log on, but it doesn’t allow him to do anything. Usually you will add more privileges later.
MySQL Database Manipulation Strings
MySQL Databases
• Most commonly used database program for developing database-driven web sites with PHP
• MySQL is open source and runs on a majority of operating systems (UNIX, Linux, Mac Os and Windows)
• Although PHP can be used with any database through its set of ODBC functions, it comes loaded with MySQL specific functions which makes for a tight integration between PHP and MySQL
SQL Command/Queries
• SQL is a computer language, and like languages in general, SQL has its rules, grammar, and a set of special or reserved words.
• Different variants of the language have evolved over the years because different vendors offer additional features to manipulate data in the name of competition
SQL Command/Queries
• SQL has many commands, but they can be divided in two major categories:– The Commands to Manipulate the Tables in the
database– The Commands to Manipulate the Database itself
• There are many excellent tutorials on the web that cover all the SQL commands and how to use them:
• http://www.w3schools.com/sql/default.asp
English Like Grammar
• An SQL statement makes a “REQUEST” or “QUERIES” the database in the form of a statement
• The first word is an English verb, an action word called a command such as (show, use, select, drop, etc.)
• The command is followed by a list of noun-like words, such as show database, use database, or create database.
• The statement might contain prepositions such as in or from; for example: – show tables in database– Select phones from customer_table
English Like Grammar• The language also allows for conditional clauses to
refine the queries such as:– select companyname from suppliers where suppllierid > 20
• When listing multiple items in a query, the items are separated by commas– select companyname, phone, address from suppliers
• Semicolon (;) is the standard way to terminate each query statement. Some database systems do not require it, but MySQL does (except for the USE and QUIT command).
SQL Reserved WordsALTER CROSS JOIN FULL JOIN JOIN ON SELECT
AND DELETE GROUP BY LEFT JOIN OR SET
AS DROP INSERT LIKE ORDER BY UPDATE
CREATE FROM INTO LIMIT RIGHT JOIN WHERE
Case Use• Database and table names are case sensitive on UNIX but not in
WINDOWS.
• A convention is to always use lowercase names for databases and their tables
• SQL commands are not case sensitive, but by convention SQL commands are CAPITALIZED for clarity while only the first letter of the field, table, and database names is capitalized– SELECT * FROM Persons WHERE FirstName=‘John’;
• If Performing pattern matching with the LIKE and NOT LIKE commands, the the pattern being searched for is case sensitive when using MySQL
Useful SQL Commands
• SHOW DATABASES: Used to see what databases are available on your database server
• USE: The USE command makes the specified database your default database
SHOW databases;
USE database_name;
Useful SQL Commands
• SHOW TABLES IN: Displays all the tables within a database
• SHOW/DESCRIBE: Either of these commands is used to see what type of data can be assigned to a table. The DESCRIBE command is specific to MySQL databases
SHOW TABLES IN database;
DESCRIBE table_name;
SQL Data ManipulationLanguage (DML)
• SQL is a non procedural language providing syntax for extracting data, including a sytax to update, insert, and delete records.
• The Query and Update commands together form the Data Manipulation (DML) part of SQL– SELECT– UPDATE– DELETE– INSERT INTO
SQL-DMLSELECT
• SELECT: This command is mandatory when performing a query; it is used to retrieve data from a table based on some criteria
• It specifies a coma separated list of fields to be retrieved, and the FROM clause specifies the table(s) to be accessed.
• The results are stored in a result table known as the result-set.
• The * symbol can be used to represent all of the fields
SQL-DMLSELECT
Format:SELECT column_name(s) FROM table_name
Example:SELECT LastName, FirstName, Address FROM Students;
To select specified columns, the SELECT command is followed by a comma separated list of fields to be selected from the table
SQL-DMLSELECT
• SELECT DISTINC: this keyword is used to return only distinct (unique) values from the table.
• If there are multiple values of a specified field, the distinct result-set will display only one.
Format:SELECT DISTINCT column_name(s) FROM table_name
Example:SELECT DISTINCT ShipName FROM Orders;
SQL-DMLSELECT
• LIMIT(X): this keyword specifies the number of rows to be returned from the beginning of the result-set. X specifies the rows returned
Format:SELECT column_name(s) FROM table_name LIMIT X;
Example:SELECT ShipName FROM Orders LIMIT 10;
SQL-DMLSELECT
• WHERE Clause: It is used to select a field when a certain criteria set of conditions are desired
• The WHERE Clause is optional• To create the conditions (called selection
criteria) SQL provides a set of operators to further qualify what criteria should be specified
SQL-DMLWhere Operators
Operator Description Example
= Equal to WHERE country = ‘ireland’
<>, != Not equal to WHERE country != ‘USA’
> Greater than WHERE salary > 28000
< Less than WHERE age < 35
>=, <= Greater/Less Than or Equal WHERE cost >=1200
IS [NOT] NULL Is NULL (vo value) or Not NULL WHERE birth = NULL
BETWEEN Between an inclusive range WHERE last_name BETWEEN ‘Doherty’ AND ‘McDAID’
LIKE Search for a value like a pattern WHERE name LIKE ‘D%’
NOT LIKE Search for a value not like a pattern WHERE country NOT LIKE ‘Sw%’
! , NOT Logical not for negation WHERE age ! 10;
||, OR Logical OR WHERE order_number > 10 || part_number = 80
&&, AND Logical AND WHERE age>12 && age < 21
XOR Exclusive OR WHERE status XOR
SQL-DML
• Using Quotes: Quotes are always an issue in programming languages. (single quotes?, double quotes?, when?)
• SQL uses single quotes around text values (MySQL also accepts double quotes)
• Numeric Values should not be enclosed in quotes.
SQL-DML
• Comparing Strings: When comparing strings using =, the string must be exactly as typed for the condition to be true – this include length and type of characters.
• NULL: Null means that there is not a value in the field, or it is unknown, but does not mean a value of zero.
DML-SQL• LIKE – NOT LIKE: The pattern matching operator
can be used as a condition in the WHERE clause, allowing the selection of rows that are ‘like’ or match a pattern
• A percent sign (%) can be used as a wildcard to match any possible character that might appear before and/or after the character(s) specified.
• A _ is used to match a single character.• The LIKE/NOT LIKE condition can be used in any
valid SQL statement, including SELECT, INSERT, UPDATE or DELETE.
DML-SQL
• Examples of the wildcard % uses:• SELECT CompanyName, Country FROM Customers WHERE
country LIKE ‘SW%’;– Returns all the customers and countries in which the country starts
with “Sw” i.e. Sweden, Switzerland• SELECT City, Country FROM suppliers WHERE City LIKE ‘%o’;
– Returns all cities and countries where the % matches any city that ends with a letter o.
• SELECT CompanyName FROM customers WHERE CompanyName LIKE ‘%Super%’– Returns all company names where the % matches any company name
that contains the pattern “Super”
DML-SQL
• Examples of the wildcard _ uses:• SELECT Extension, Firstname FROM Employees WHERE
extension LIKE ‘4_ _’;– Returns all extensions and first names where the exetension has three
characters and the first character is a 4.
DML-SQL
• ORDER BY: Used to sort the output of a query in either ascending (ASC, the default) or descending (DESC) order where the values being sorted are either strings or numbers
Format:SELECT column_name(s) FROM table_name [WHERE condition] ORDER BY column [ASC, DESC]
Example:SELECT Company,Ordernumber FROM Orders ORDER BY Company;
SQL-DMLINSERT
• The INSERT statement is used ot insert new rows into a table.
• After the VALUES keyword, a comma-separated list of column names follows
Format:INSERT INTO table_name VALUES (value1, value2, … )
INSERT INTO table_name (column1, column2,…) VALUES (value1, value2, … )
Example:INSERT INTO Shippers (CompanyName, Phone) VALUES (‘FEDEX’,’416-555-1221’);
SQL-DMLINSERT
• Usually, the tables have a primary key column that is usually set to auto-increment; when this is the case, the id of the table is created by the database engine automatically
• Letting the database increment the PRIMARY KEY ensures that the value is always unique.
SQL-DMLUPDATE
• The UPDATE statement is used to modify data in a table.
• The UPDATE command is followed by the name of the table where the data will be changed, followed by the SET statement to inidcate what field will be changed, and then the new value that will be assigned to the field
• The WHERE clause further qualifies what data is to be modified, thereby limiting the scope of the UPDATE
SQL-DMLUPDATE
Format:UPDATE table_name SET column_name = new value WHERE column_name = some_value;
Example:UPDATE orders SET ShipCountry=‘Spain’ WHERE CustomerId = ‘whitc’;
SQL-DMLDELETE
• The DELETE statement is used to delete rows in a table and returns the number of rows that were deleted.
• DELETE uses the FROM clause to specify the name of the table that contains the data you want to delete
• The WHERE clause specifies the criteria to identify what data should be removed.
BE CAREFUL: Without the WHERE clause ALL ROWS are DELETED
SQL-DMLDELETE
• If the ORDER BY clause is specified, the rows are deleted in the order that is specified.
• The LIMIT clause places a limit on the number of rows that can be deleted.
Format:DELETE FROM table_name WHERE column_name = some_value;
Example:DELETE FROM orders WHERE ShipCountry = ‘Greenland’;
Database Creation using SQL
Data Definition Language
Data Definition Language• The DDL part of SQL permits database objects to
be created or destroyed• Indexes (keys) can be defined• Links between table can be specified• Constraints between database tables can be
imposed• Often decisions to create and remove databases
are handled by a database administrator and having permission to create and drop tables depends on what access rights are granted.
Data Definition Language
• The most important Data Definition Statements in SQL are:– CREATE TABLE– ALTER TABLE– DROP TABLE– CREATE INDEX– DROP INDEX
• DML and DDL can be executed on the MySQL command line or in the MySQL tab of PHPMyAdmin
MySQL Command Line• Windows
1. Start WAMP Server2. On the command prompt browse to find the mysql.exe file (usually
located in c:\wamp\bin\mysql\mysqlX.X.X\bin\) – (X.X.X=5.5.8)3. At the end of the command line add the following:..\mysql.exe --user=student --password=studentIf it is your own laptop, then user=root -- password=
• MAC1. Start MAMP and Start the Servers2. Go to Terminal3. Type in the following command:/Applications/MAMP/Library/bin/mysql --host=localhost -uroot -proot
DDLDATABASES on SERVER
• Once on the MySQL command line the SHOW DATABASES command will show you the existing databases on the server:
• mysql> show databases;
DDLCHOOSING a DATABASE
• In order to select a database from the list to use it, you will need the USE command:
• mysql> use test;
DDLShow & Describe
• To see what tables are in the database, use the SHOW TABLES command:
• mysql> show tables;
• To display the contents of a table, you can use the SELECT command.
DDLShow & Describe
• To see what type of data can be assigned to a table, use the DESCRIBE command (specific to MySQL) or the SHOW FIELDS IN command (standard SQL)
• mysql> describe table_name;
DDLCreating a Database
• To create a new database, use the CREATE DATABASE command:
• CREATE DATABASE database_name;
DDLCreating a Database
• mysql>CREATE DATABASE gallerydb;• mysql>USE DATABASE gallerydb;
Art Gallery Database
Creating Tables
• In order to create the tables, the database should be first properly designed:
• For each field on each table you should know what type of data is it going to store
• After that is done, then the required datatype should be selected
Note: it is a waste of space (memory) to, for example, declare a field as integer when you
know beforehand that it will not store a number bigger that 1000.
SQL Data TypesNumbers
72
Type Range Storage (Bytes)
Description
INTEGER -231..231-1 / 0.. 232-1 4 Whole Number
INT UNSIGNED 0..232-1 4 Non Negative Whole Number
TINYINT -127..128 / 0..255 1 Very small integers
SMALLINT -32768..32767 /0..65535
2 Small Integers
MEDIUMINT -8388608..8388607 / 0..16777215
3 Medium Sized Integers
BIGINT -263..263-1 / 0..264-1 8 Big Integers
SQL Data TypesNumbers
73
Type Range Storage (Bytes)
Description
FLOAT (precision) Depends on precision Varies Can be used to specify single or double precision floating point numbers
FLOAT [(M,D)] ±1.175494351E-38 .. ±3.402823466E+38
4 Single precision floating point number. These numbers are equivalent to FLOAT(4) but with a specified display width(M) and number of decimal places(D).
DOUBLE[(M,D)] ±1.7E+308 .. ±2.2E-308 8 Double precision floating point number. These numbers are equivalent to FLOAT(8) but with a specified display width (M) and number of decimal places(D).
DECIMAL[(M,D)] Varies M+2 Floating point stored as a CHAR
NUMERIC / DEC / FIXED As Above Synonym for DECIMAL
SQL Data Types STRINGS
74
Type Range Description
CHAR (M) 0 – 255 Characters
Fixed-length string of length M, where M is between 0 and 255. Using the BINARY keyword specifies that the data should be considered casesensitive.
VARCHAR (M) 1 - 255 Same as above, except they are variable length.
SQL Data TypesSTRINGS
75
Type Range Description
BLOB 65535 characters A normal sized Binary Large Object (BLOB) (Case sensitive)
TEXT 65535 characters Normal Sized TEXT object. (Case insensitive)
MySQL Data Types – DATE and TIME
76
Type Range Description
DATE 1000-01-01 .. 9999-12-31 A date. Will be displayed as YYYY-MM-DD
TIME -838:59:59 .. 838:59:59 A time. Will be displayed as HH:MM:SS. Note that the range is much wider than you probably will ever want to use.
DATETIME 1000-01-01 00:00:00 .. 9999-12-31 23:59:59
A date and time. Will be displayed as YYYY-MM-DD HH:MM:SS
TIMESTAMP (M) 1970-01-01 00:00:00 .. Sometime in 2037
A timestamp useful for transaction reporting. The display format depends on the value of M. The top range depends on the limit on Unix
YEAR[(2|4)] 70 – 69 (1970 – 2069)1901 - 2155
A year. You can specify two or four digits format. Each has a different range.
SQL Data TypesTimestamp Examples
77
Type Display
TIMESTAMP YYYYMMDDHHMMSS
TIMESTAMP(14) YYYYMMDDHHMMSS
TIMESTAMP(12) YYMMDDHHMMSS
TIMESTAMP(10) YYMMDDHHMM
TIMESTAMP(8) YYYYMMDD
TIMESTAMP(6) YYMMDD
DML-DDL-Example• We are going to create a database to store pets
information and on it we are going to create a table called dog
PETSdog(name,owner,breed,sex,birth,death)
• Note: when designing your database, be mindful of all the already learned database design techniques like normalization.
PET DBDog Table
• SQL Instructions:mysql>create database pets;mysql>use pets;mysql>create table dog ( -> name varchar(20), -> owner varchar(20), -> breed varchar(20), -> sex char(1), -> birth date, -> death date);mysql>describe dog;
Altering a TableAdding a Primary Key
• On the previous example, we forgot to add a primary key field; we are going to correct that example using the ALTER TABLE command:
• When altering a table, we redefine its structure by adding or dropping columns, keys, indexes and tables.
• The Alter command can also be used to change column names, types and the table name.
Altering a TableAdding a Primary Key
• First, we need to add a new column to serve as the primary key; we will call it pet_id:
mysql>alter table dog add pet_id int(11) first;
• Now we need to modify this new column to make it the primary key – primary key fields should not be null and should have an auto_increment property set:
mysql>alter table dog modify column pet_id int(11)-->not null auto_increment primary key;
• The following command accomplishes the same task in a single line:mysql>alter table dog add pet_id int(11) not null auto_increment first,
-->add primary key(pet_id);
Adding PK
Dropping TablesDropping Databases
• To delete a table use the DROP command:mysql>drop table dog;• To delete a database use the DROP command:mysql>drop database pets;
SQL Functions
• The following functions are used to alter or format the output of a SQL query. Functions are provided for strings, numbers, dates, server and information, and so on. They return a result-set.
• When using SELECT with a function, the function, as it was called, is displayed as the name of the column in the result-set
Numeric Functions
String Functions
Date & Time FunctionsFunction Example
NOW() select NOW() 2012-01-17 15:58:45
CURDATE() select CURDATE() 2012-01-17
CURTIME() select CURTIME() 15:58:45
DAYOFYEAR(date) select DAYOFYEAR(‘2006-12-15’)349
DAYOFMONTH(date) select DAYOFMONT(‘2012-01-19’) 19
DAYOFWEEK(date) select DAYOFWEEK(‘2012-01-19’) 5 (Thursday); Sunday is 1
WEEKDAY(date) select WEEKDAY(‘2012-01-19’) 3(week starts at 0 on Monday)
MONTHNAME(date) select MONTHNAME(‘2012-01-19’) January
DAYNAME(date) select DAYNAME(‘2012-01-19’)Thursday
YEAR(date) select YEAR(‘2012-01-19’)2012
QUARTER(date) select QUARTER(‘2012-01-19)1