Database FundamentalsDatabase FundamentalsDatabase FundamentalsDatabase Fundamentals

Who Needs to Know about Who Needs to Know about Database Design ConceptsDatabase Design Concepts

System Architects Applications Designers System Managers Operators End users and of course DBA’s

System Architects Applications Designers System Managers Operators End users and of course DBA’s

In short just about everyone can benefit

Why?Why?

DataDatabasebase

The database is usually the center of the system universe

The database is usually the center of the system universe

End Users

Reporting Development

OLTP

The supports the business Business drives the data

The supports the business Business drives the data

Everyone effects the Everyone effects the DatabaseDatabase

?

?

?

One users can bring the system to its kneesOne users can bring the system to its knees

Locking IssuesResource Hogging

The Database is a Common The Database is a Common ResourceResource

Poor database performance means poor business performance

Poor database performance means poor business performance

Inaccurate Reporting

Slower Development

Wasted Resources

Customer Complaints

DatabaseDatabaseDatabaseDatabase InflexibleArchitectures

How is a Database built?How is a Database built?

Business Needs Analysis Conceptual Design Logical Design Physical Design Testing Deployment Maintenance

Business Needs Analysis Conceptual Design Logical Design Physical Design Testing Deployment Maintenance

Design StepsDesign Steps

Same logic as Application Same logic as Application Development life cycleDevelopment life cycle

BNA Conceptual Design Logical Design Physical Design Tuning Deployment Maintenance

BNA Conceptual Design Logical Design Physical Design Tuning Deployment Maintenance

RFP Requirements Gathering Functional Design Detail Design & Build Acceptance Testing Roll-out Enhancements

RFP Requirements Gathering Functional Design Detail Design & Build Acceptance Testing Roll-out Enhancements

Blueprint for DisasterBlueprint for Disaster

A poor foundation means.... a shaky future...

A poor foundation means.... a shaky future...

Building BlocksBuilding Blocks

BNA

Conceptual Design

Logical Design

Physical Design

Test

Deploy

Maint

Each block builds on the previous ones

Each block builds on the previous ones

Like the PyramidsLike the Pyramids

Business Needs AnalysisBusiness Needs Analysis

The initial proposal...The initial proposal...

Scopes the project Mission Statement Mission Critical

Success Factors

Scopes the project Mission Statement Mission Critical

Success Factors

Project ScopeProject Scope

High level view of the project Establishment of project goals Establishment of project boundaries Rough view of data requirements

High level view of the project Establishment of project goals Establishment of project boundaries Rough view of data requirements

Mission StatementMission Statement

Defines project scope as clearly and concisely as possible

Defines project scope as clearly and concisely as possible

The Phoenix project will convert the existing paper order entry system to anelectronic one within one year and providethe basis for a customer tracking system at completion.

Helps retain project focus through delivery

Mission Critical Success Mission Critical Success FactorsFactors

Define the basic project metrics , to measure its success while controlling project scope

Define the basic project metrics , to measure its success while controlling project scope

System must be completed within one year System must process 1,500 orders per day Planned growth is 45,000 orders per year Response time must be 3 seconds for on-line Standard inventory reports will be produced System will use existing hardware

MCSF and Red FlagsMCSF and Red Flags

MCSF establish a framework with the customerand eliminate unrealistic “wish list” expectations

MCSF establish a framework with the customerand eliminate unrealistic “wish list” expectations

“45,000 orders per year are only 22.5 per day, are some days heavier for order entry?”

“If you want to use your existing PC-XT’s, you’re not getting three second response time...”

BNA DeliverablesBNA Deliverables

Basic system requirements Project Scope Mission Statement Mission Critical Success Factors

Conceptual DesignConceptual Design

Identify information sources– Existing forms, reports, documentation– Existing training materials– Key People

Identify information sources– Existing forms, reports, documentation– Existing training materials– Key People

Requirements gatheringRequirements gathering

Understanding the customer’s requirements meansunderstanding their business as well as they do...

Understanding the customer’s requirements meansunderstanding their business as well as they do...

Uncovering Business RulesUncovering Business Rules

A business rule is... Something you know about the data, upon which design decisions will be based.

A business rule is... Something you know about the data, upon which design decisions will be based.

The new system is great but how do I enter thecustomer’s billing address

?!??!?

Business Rules and MetadataBusiness Rules and Metadata

Metadata dictionary is data about dataMetadata dictionary is data about data

Used as a reference for all usersUsed as a reference for all users

Cash Batch Header

Customer addresses Billing

Shipping

Corporate

Conceptual Design Conceptual Design DeliverablesDeliverables

Methodology, Standards and Guidelines System Hardware and Software requirements Rough data requirements from Functional Design

Specification Beginning of Metadata Dictionary List of known Business Rules

Logical DesignLogical Design

Defining the data structures to fit the business needsDefining the data structures to fit the business needs

Entity Analysis and Normalization Producing the ERD CRUD Diagram Transaction Analysis Table Analysis

Entity Analysis and Normalization Producing the ERD CRUD Diagram Transaction Analysis Table Analysis

Where do you start?Where do you start?

School assignment, 5-10 tables, no problem...

Small subsystem, 10- 20 tables, not to tricky...

Real system design, 100+ tables, hmmmm...hmmmm...

Full blown mission critical, 800+ tables, Yikes!!!Yikes!!!

Which comes first, the Chicken or the Which comes first, the Chicken or the Attribute?Attribute?

Do you create the data sets from the applications? Or do you define the applications once you have the data?

Do you create the data sets from the applications? Or do you define the applications once you have the data?

It is an evolving process, functional design to detail design, data and applications needs will grow and develop together

It is an evolving process, functional design to detail design, data and applications needs will grow and develop together

???

Time to Cook, Time to Cook, Entity SoupEntity Soup

Order Form Order number Customer number Item 1...

Entity SoupEntity SoupCus

tom

er

Prof

ile

Throw all entity attributesinto the “entity soup”

Throw all entity attributesinto the “entity soup”

Use Normalization to sortthem out into coherenttables

Use Normalization to sortthem out into coherenttables

Use the followingUse the followingrules...rules...

Use the followingUse the followingrules...rules...

Pick your Starting EntitiesPick your Starting Entities

ORDER CUSTOMER

Order items

Order infoParts

info SalespersonEmployee

Customer info

Populate the EntitiesPopulate the Entities

ORDER CUSTOMER

Order items

Order infoParts

info SalespersonEmployee

Customer_nameAddress

Order item

Item desc

Populate the EntitiesPopulate the Entities

ORDER CUSTOMER

SalespersonEmployee

Customer_nameAddress

Order itemItem desc

Some are not... (an Employee fills an order, and also takesan order as the customer’s sales person... where does it go?)

Populate the EntitiesPopulate the Entities

ORDER CUSTOMER

Parts info Sales

personEmployee

Customer_nameAddress

Order itemItem desc

Don’t worry, pick one or the other, but not both!Don’t worry, pick one or the other, but not both!

You don’t wantto duplicate data

Populate the EntitiesPopulate the Entities

CUSTOMER

Parts info Sales

personEmployee

Customer_nameAddress

Eliminate similar entity attributesEliminate similar entity attributes

Sales person and Employee are the same

Employee#Name

A Word about attribute naming A Word about attribute naming conventionsconventions

Homonyms - Same name, different meaning Customer address and Employee Address

Synonyms - Same meaning, different nameBadge_ID, Manager_id, Emp_no

Homonyms - Same name, different meaning Customer address and Employee Address

Synonyms - Same meaning, different nameBadge_ID, Manager_id, Emp_no

Build in Intuitiveness where possible

NormalizationNormalization

Breaking entities down into the most molecular level needed to be viewed by the business

Tinkertoys

Lego’s Lincoln Logs

WOOD BLOCKS?

Breaking entities down into the most molecular level needed to be viewed by the business

Tinkertoys

Lego’s Lincoln Logs

WOOD BLOCKS?

The process of NormalizationThe process of Normalization

Orders Customer

Order info Cust info

Items infoEmp info

InventoryDependents

Cust Credit

Warehs loc Part Descr

Normalization will take the large “super structures” and break them down...

The process of NormalizationThe process of Normalization

Orders Customer

Order info Cust info

Items info

Emp info

Inventory

Dependents

Cust Credit

Warehs loc

Part Descr

into the smallest components needed by the business view

Normal FormsNormal Forms

1NF, Select “Candidate Key”, remove repeating groups.

2NF, Functional dependencies dependent on the whole key.

3NF, Removal of Transitive Dependencies. BCNF, 4NF,5NF Domain-key NF

1NF, Select “Candidate Key”, remove repeating groups.

2NF, Functional dependencies dependent on the whole key.

3NF, Removal of Transitive Dependencies. BCNF, 4NF,5NF Domain-key NF

Normal FormsNormal Forms

1NF - “The Key” 2NF - “The whole Key” 3NF - “Nothing but the Key”

1NF - “The Key” 2NF - “The whole Key” 3NF - “Nothing but the Key”

If you remember this, and it makes sense, you understand the basics of normalization

Determinants and Repeating Determinants and Repeating GroupsGroups

Determinant - A field with a known value that can uniquely identify the value in another field.

Based on the Business Rules governing those fields

Repeating Group - Field that has multiple values

Determinant - A field with a known value that can uniquely identify the value in another field.

Based on the Business Rules governing those fields

Repeating Group - Field that has multiple values

When in doubt, ask the entity...Hey Employee with Emp_ID = 100,Do you know your name?... (yes)Do you know your shoe size?... (yes)What car do you own?... (which one?)

1NF, “1NF, “the key”the key”

Select a “primary key” from “candidate keys”Select a “primary key” from “candidate keys”

Remove repeating groups into their own table

Remove repeating groups into their own table

EMP_ID SSN Name & Start_date Shoe size Dependents

Whether something is a key, or repeating group is determined by the Business Rules for that attribute

2NF, “2NF, “the whole key”the whole key”

In a “concatenated key”, the entire key is a determinant for every attribute, (again based on Business Rules)

In a “concatenated key”, the entire key is a determinant for every attribute, (again based on Business Rules)

WAREHOUSE & BIN_LOC Whs_address Bin_Size Qty

To get warehouse address using the key, you must first finda part that is stored there

3NF, “3NF, “nothing but the key”nothing but the key”

Removal of non-key determinants to their own tables

Removal of non-key determinants to their own tables

CUST_NO Name Address ZIP

ZIP City State

Normalization ResultsNormalization Results

Orders table

Order_no N6Cust_no N10Ord_date DateShip_date DateEmp_id N8Invt_flg BooleanC_rec C12

MetadataDictionary

The results of the normalization process are clearly defined data structures that fits the business needs and a Metadata Dictionary which clearly describes the data

The results of the normalization process are clearly defined data structures that fits the business needs and a Metadata Dictionary which clearly describes the data

Entity Relation DiagramEntity Relation Diagram

An ERD it the roadmap to navigate through the database. Establish relationships with foreign keys.

An ERD it the roadmap to navigate through the database. Establish relationships with foreign keys.

Orders

O_Items

CustomerParts

C_Credit

Inventory

WarehseRelationships are based on Business Rules Make the ERD as intuitive as possible

Converting Function to TransactionConverting Function to Transaction

Order Entry

EnterOrder

Transaction #102 Add_an_Order Start Transaction Validate Customer Validate Credit Add Order Commit

Business Function

Business Function

ApplicationRequirements

ApplicationRequirements

Database Transaction Analysis

Database Transaction Analysis

Transaction Analysis SheetsTransaction Analysis Sheets

Trans #102 Description

Tables Used

Retrieval Typ Time of Day Priority Frequency Security Business Rules

Add an Order

Order, Item Customer, Customer_credit Index and index only M-F 8:00am-5:30pm High, 3 seconds up to 1,500/hour Update by supervisor Customer credit validatedbefore order is entered

Lays groundwork for physical design.

Allows transactions to becataloged and quantifiedfor performance tuning.

Lays groundwork for physical design.

Allows transactions to becataloged and quantifiedfor performance tuning.

CRUD DiagramCRUD Diagram

Table

Tra

nsa

ctio

nAdd Order

Fill Order

Update Inv

Mailing Rep

Update Whs

Credit Rep

OR

DE

R

CU

STO

ME

RC

_CR

EDIT

INV

EN

TO

RY

C RC RC U

D

U

R R

CU

R R

CRUD stands for Create, Read, Update, Delete CRUD stands for Create, Read, Update, Delete

CRUD tables show the interaction betweentables and transactions

An excellent tool forperformance andlocking predictions

CRUD tables show the interaction betweentables and transactions

An excellent tool forperformance andlocking predictions

Table Analysis SheetsTable Analysis Sheets

Table: OrdersDescription

CardinalityGrowthPrimary KeyForeign KeysAddDeleteUpdateReadConstraintsSecurityViews

Stores normal orderprocessing records100,000 from old sys45,000 new per yearOrder_noCust_noTrans #102, 106, 114Trans #116, 96Trans # 69Rept # 4, 6 13, 61Must have valid Cust_noUpdate by Supervisor used by Cust_Ord view

Table analysis is looking at transactions from a tablepoint of view.

Often useful for uncovering forgotten maintenance routines

Table analysis is looking at transactions from a tablepoint of view.

Often useful for uncovering forgotten maintenance routines

Logical Design Sanity CheckLogical Design Sanity Check

Order_no N6Cust_no N10Ord_date DateShip_date DateEmp_id N8Invt_flg BooleanC_rec C12

Add_Order

Fill_OrderUpd_Order

Orders

O_Items

The last step in Logical Design is to make sure that the datastructures do indeed support the transactions the end usersmust perform.

The last step in Logical Design is to make sure that the datastructures do indeed support the transactions the end usersmust perform.

Logical Design DeliverablesLogical Design Deliverables

Logical Tables ERD Transaction Analysis CRUD Diagram Tables Analysis

Physical DesignPhysical Design

Physical Layout Locking Strategies Indexing Options

Physical database construction Physical database construction

Built on the foundation provided by the Logical Design deliverables

Physical LayoutPhysical Layout

How does the database layer on the Operating System?

What are its memory requirements? How is it backed up and restored? How is it horizontally and vertically partitioned? How are the transaction logs stored? How many users can it support?

The answers to these questions can usually be found on the software documentation for the product

Physical LayoutPhysical Layout

File Placement Physical Size Log file placement Rollback segment

location Memory usage CPU usage

Build for performance, flexibility, recoverability and cost savings

Build for performance, flexibility, recoverability and cost savings

Physical Layout Physical Layout

Table: OrdersDescription

CardinalityGrowthPrimary KeyForeign KeysAddDeleteUpdateReadConstraintsSecurityViews

Stores normal orderprocessing records100,000 from old sys45,000 new per yearOrder_noCust_noTrans #102, 106, 114Trans #116, 96Trans # 69Rept # 4, 6 13, 61Must have valid Cust_noUpdate by Supervisor used by Cust_Ord view

Use the Table Analysis work sheets to help planthe size and growth potential of the tables and storage areas.

Use the Table Analysis work sheets to help planthe size and growth potential of the tables and storage areas.

Locking issuesLocking issues

Databases have concurrent users Locking protects record integrity

Read operations are usuallycompatible with other reads

Updates are exclusive and lockthe record

Read operations are usuallycompatible with other reads

Updates are exclusive and lockthe record

Locking ScenarioLocking Scenario

UPDATE

READREADREAD

READ

?

?

?? Make sure to prototype transactions

Transaction InteractionTransaction Interaction

Tra

nsa

ctio

nAdd Order

Fill Order

Update Inv

Mailing Rep

Update Whs

Credit Rep

OR

DE

R

CU

STO

ME

RC

_CR

EDIT

INV

EN

TO

RY

C RC RC U

D

U

R R

CU

R R

Use the CRUD Diagram and Transaction Analysis to uncover potential problem areas

Use the CRUD Diagram and Transaction Analysis to uncover potential problem areas

Trans #102 Description

Tables Used

Retrieval Typ Time of Day Priority Frequency Security Business Rules

Add an Order

Order, Item Customer, Customer_credit Index and index onlyM-F8:00am5:30pmHigh200/day - 1500 max

Issues of IndexesIssues of Indexes

Indexes are like a “phone book” They speed record retrieval They speed join operations between tables The more indexes, the faster things can be

found But... they slow updates

YellowPages

Index ExampleIndex Example

Name SSN DOB

Indexes

TableSelect * from employees wherename=‘Jones’and dob=‘05/14/59’;

Jones 55-555-5555 05/14/59

Index ExampleIndex Example

Name SSN DOB

Indexes

TableDelete from employeeswherename=‘jones’;

Jones 55-555-5555 05/14/59UPDATE

UPDATEUPDATE

UPDATE

One deleteforces 4 updates

Trans #102 Description

Tables Used

Retrieval Type Time of Day Priority Frequency

Add an Order

Order, Item Customer, Customer_credit Range and index only exact match M-F 8:00am-5:30pm High, 3 seconds up to 1,500/hour

Selecting the correct fields to Selecting the correct fields to IndexIndex

Select indexes based on the Transaction Analysis accessstrategies. Optimize the indices to enhance performance.

Select indexes based on the Transaction Analysis accessstrategies. Optimize the indices to enhance performance.

Don’t index haphazardly Don’t over-index tables Select the correct

– structure

– fill factor

– placement

Don’t index haphazardly Don’t over-index tables Select the correct

– structure

– fill factor

– placement

Logical Design DeliverablesLogical Design Deliverables

Physical Database Schema optimized to support mission critical transactions– Table and Index structures– File and storage area sizes– File and storage area locations– Backup and recovery architecture

TestingTesting

Create a testing environment that mimics the production environment as closely as possible

Create a testing environment that mimics the production environment as closely as possible

Build in benchmark gathering from the start

DeploymentDeployment

Deploy as unobtrusively as possible Deploy as unobtrusively as possible

The New

System!

1/1/1997

Complete and update documentation Load production data Beta Test deployment Train end users

Always have “Plan B” readyin case of unforeseen problems

MaintenanceMaintenance

As part of the design process, build in performanceestablish a maintenance programtrain the support staff

As part of the design process, build in performanceestablish a maintenance programtrain the support staff

Reorganize storage areas Reoptimize statistics Monitor transaction performance Rebuild indexes