12/11/2008 seg 3550
Post on 06-May-2015
352 Views
Preview:
TRANSCRIPT
12/11/2008 SEG 3550
SEG3550 Fundamentals of Information System
Tutorial 8
Storage and File Structure
12/11/2008 SEG 3550
Overview
Storage Hierarchy Physical Storage Media RAID RAID Levels File Organization
12/11/2008 SEG 3550
Why do we need to know about storage/file structure
Many database technologies are developed to utilize the storage architecture/hierarchy
Data in the database needs to be organized and stored/retrieved efficiently
12/11/2008 SEG 3550
Physical Storage Media
Physical storage media classified according to:
Data access speed
Cost per unit of data
Reliability Can differentiate storage into:
Volatile storage: Loses contents when power is switched off
Non-volatile storage: Contents persist even when power is switched off
12/11/2008 SEG 3550
Storage Hierarchy
Magnetic Tape
Optical Disk
Magnetic Disk
Flash Memory
Cache
unit priceMemory
Volatile
primary storage
Non-volatile speed
Secondary storage
Tertiary storage
12/11/2008 SEG 3550
Primary Storage Cache Volatile - managed by hardware
Speed: 7 to 20 ns (1 nanosecond = 10–9 seconds) Capacity:
A typical PC level 2 cache 64KB-2 MB. Within processors, level 1 cache usually ranges in size from 8 KB
to 64 KB.
Main memory: Volatile
Speed: 10s to 100s of nanoseconds; Capacity: Up to a few Gigabytes widely used currently
per-byte costs have decreased roughly factor of 2 every 2 to 3 years)
12/11/2008 SEG 3550
Secondary Storage
Flash memory Non-volatile
Speed: read speed similar to main memory. But writes are slow (few microseconds), erase is slower.
Capacity: 32M to a few Gigabytes currently Forms: SmartMedia, memory stick, secure digital, BIOS Cost: roughly same as main memory
Magnetic-disk Non-volatile
Capacities: up to roughly 1 TB(1000 GB) currently Data must be moved from disk to main memory for access, and
written back for storage. Growing constantly and rapidly with technology improvements (factor of 2 to 3 every 2 years)
12/11/2008 SEG 3550
Tertiary Storage (Non-volatile)
Optical storage CD-ROM (640 MB) and DVD (4.7 to 17 GB) most popular
forms Reads and writes are slower than with magnetic disk
Tape storage used primarily for backup (to recover from disk failure), and
for archival data sequential-access – much slower than disk very high capacity (40 to 300 GB tapes available)
12/11/2008 SEG 3550
Between Memory and Disk
The permanent residency of database is mostly on disk In database, cost is usually measured by the number of disk I/O But disks are too slow and we need memory to be the buffers …
but memory is volatile this introduced a number of issues
12/11/2008 SEG 3550
RAID
RAID: Redundant Arrays of Independent Disks Disk organization techniques that manage a large
numbers of disks. high capacity and high speed by using multiple disks in
parallel, and high reliability by storing data redundantly
12/11/2008 SEG 3550
Mean time to failure (MTTF)
Average time the disk is expected to run continuously without any failure. Typically 3 to 5 years (1 year = 8,760 hours) MTTF = 30,000 to 1,200,000 hours for a new disk
an MTTF of 1,200,000 hours for a new disk means that given 1000 relatively new disks, on an average one will fail every 1200 hours
(assuming by Exponential Distribution)
When number of disks increase, the chance of some disk failure increase proportionally
12/11/2008 SEG 3550
Parallelism
Two main goals of parallelism in a disk system: 1. Load balance multiple small accesses to increase throughput
2. Parallelize large accesses to reduce response time.
Basic strategy: Stripping Compare and contrast bit stripping and byte stripping
12/11/2008 SEG 3550
Redundancy
store extra information that can be used to rebuild information lost in a disk failure
Basic strategy: mirroring, parity
Mean time to data loss depends on mean time to failure, and mean time to repair
10010010 1
Data Parity
12/11/2008 SEG 3550
RAID Levels
Twice the Read transaction rate of single disks, same Write transaction rate as single disks.
Due to its cost and complexity, level 2 never really "caught on". Therefore, much of the information below is based upon theoretical analysis, not empirical evidence.
12/11/2008 SEG 3550
RAID Levels (cont)
Very high read data transfer rate and very high write data transfer rate, but Controller design is fairly complex.
Very high read data transaction rate,but quite complex controller design. Difficult and inefficient data rebuild in the event of disk failure.
12/11/2008 SEG 3550
RAID Levels (cont)
Highest read data transaction rate and medium write data transaction rate. Most complex controller design. Difficult to rebuild in the event of a disk failure (compared to RAID level 1)
RAID 6 is essentially an extension of RAID level 5 which allows for additional fault tolerance by using a second independent distributed parity scheme (dual parity)
12/11/2008 SEG 3550
Choice of RAID Levels
Level 1 provides much better write performance than level 5 Level 5 requires at least 2 block reads and 2 block writes to write a
single block, whereas Level 1 only requires 2 block writes Level 1 preferred for high update environments such as log disks
Level 1 had higher storage cost than level 5 disk drive capacities increasing rapidly (50%/year) whereas disk
access times have decreased much less (x 3 in 10 years) I/O requirements have increased greatly, e.g. for Web servers When enough disks have been bought to satisfy required rate of I/O,
they often have spare storage capacity so there is often no extra monetary cost for Level 1!
Level 5 is preferred for applications with low update rate,and large amounts of data
Level 1 is preferred for all other applications
12/11/2008 SEG 3550
Buffer Management
Database can not fit entirely in memory, needs memory as a buffer for speed reasons
LRU is used in many OS Spatial and temporal locality due to loops
Database has a more predictable behavior Example: join
12/11/2008 SEG 3550
File Organization
The database is stored as a collection of files. Each file is a sequence of records. A record is sequence of fields
Approaches:
– assume record size is fixed
– each file has records of one particular type only
– different files are used for different relations
12/11/2008 SEG 3550
Fixed-Length Records
Simple approach:
– Store record i starting from byte n *(i − 1), where n is the size of each record
– Record access is simple but records may cross blocks
Deletion of record i – Several alternatives:
– move records i + 1, . . . , n to i, . . . , n − 1
– move record n to i
– link all free records on a free list
12/11/2008 SEG 3550
Free Lists
Store the address of the first record whose contents are deleted in the file header
Use this first record to store the address of the second available record, and so on
Can think of these stored addresses as pointers since they “point” to the location of a record
More space efficient representation: reuse space for normal attributes of free records to store pointers. (No pointers stored in in-use records)
12/11/2008 SEG 3550
Variable-Length Records
Variable-length records arise in database systems in several ways:
– Storage of multiple record types in a file
– Record types that allow variable lengths for one or more fields
– Record types that allow repeating fields (used in some older data models)
Byte string representation
– Attach an end-of-record ( ) control character to the end of each record
12/11/2008 SEG 3550
Organization of Records in Files
Sequential File Organization Suitable for applications that require sequential processing of the
entire file. The records in the file are ordered by a search-key. Need to reorganize the file from time to time to restore sequential order.
Deletion – use pointer chains Insertion – must locate the position in the file where the record is
to be inserted – If there is free space insert there – If no free space, insert the record in an overflow block – In either case, pointer chain must be updatedClustering File Organization Simple file structure stores each relation in a separate file Can instead store several relations in one file using a clustering
file organization
12/11/2008 SEG 3550
Exercises (1)
Show the structure of the file below after each of the following steps:
a. Insert(Brighton, A-323,1600) b. Delete record 2 c. Insert( Brighton, A-626,2000)
12/11/2008 SEG 3550
Exercises (2)
Show the structure of the file below after each of the following steps:
a. Insert(Mianus, A-101, 2800) b. Insert( Brighton, A-323, 1600) c. Delete( Perryridge, A-102, 400)
12/11/2008 SEG 3550
Thanks!
top related