files cs 470 - spring 2002. overview example: fat file system file organization file system...
TRANSCRIPT
Files
CS 470 - Spring 2002
Overview
• Example: FAT File System
• File Organization
• File System Organization– File Directories and File Sharing– Record Blocking– Disk space allocation
• Example: Unix File Systems
Fat Disk StructureBoot Sector
Other Reserved
File AllocationTable (FAT)
File AllocationTable Copy
Root Directory
File and DirectoryArea
Reserved Sectors
Allocation chainsand Free List
Redundant Copy
Fixed size Arrayof root directory entriesAll other directories andfiles are located here
FAT Boot Sector
• Location: First sector of the drive
• Contains:– Jump to first level boot code (byte offset 0)– Disk Parameter Block (byte offset 11) which
describes the drive– Other identifying information like the volume
label (byte offset 43) – First level boot code (typically at offset 0x40)
Disk Parameter Block
• Geometry– BytesPerSector[2] – SectorsPerTrack[2] – Heads[2] – Sectors[2] or
LargeSectors[4]– Media– HiddenSectors[4]
• Structure Sizes– ReservedSectors[2]– NumberOfFats – SectorsPerFat[2] – NbrOfRootEntries[2]– SectorsPerCluster
Field sizes in bytes are indicated in square brackets
FAT Directory Entries
Typedef struct _DIRENT {
UCHAR FileName[11];
UCHAR Attributes;
UCHAR NtByte;
UCHAR
CreationCSec;
FAT_TIME_STAMP
CreationTime;
FAT_DATE
LastAccessDate;
USHORT
ExtendedAttributes;
FAT_TIME_STAMP
LastWriteTime;
USHORT
FirstClusterOfFile;
ULONG FileSize;
} DIRENT;
File Allocation Table
18
0
0xff7
1a
0
0xfff
14
8dot3 Name
. . .
First Cluster (15)
File Size (5a0)
0xfffDirectory Entry 14
16
17
18
19
1aFree Cluster
Bad Cluster
FAT12
FOO BAR. . .1b32f
Dir Entry
• How to translate FAT tables entries to sector offsets in the disk partition
• RootDirSize = NbrRootEntries * sizeof(DIRENT) / SectorSize
• FileAreaOffset = NbrReservedSectors + NbrFat*SectorsPerFat + RootDirSize
• SectorOffset = FileAreaOffset + (FAT_Offset - 2) * SectorsPerCluster
Fat_Offset to SectorOffset
Long Filename Directory Entrytypedef struct _LFN_DIRENT {
UCHAR Ordinal;
UCHAR Name1[10]; // 5 Unicode chars
UCHAR Attributes; // Should be 0x0f
UCHAR Type;
UCHAR CheckSum; // of 8dot3 name
WCHAR Name2[6];
USHORT MustBeZero;
WCHAR Name3[2]; } LFN_DIRENT;
DIRENT with LFN
0x44
Characters 26 to 383
Characters 13 to 252
First 13 Unicode Chars1
Up to 13 Unicode Chars
8dot3 Name
Ordinal
All have sameChecksumField
Calculate rotated checksum from this
File Structure
CS 470 - Spring 2002
Segment II
File Organization• Vocabulary
– Field - basic element of data– Record - collection of related fields– File - collection of similar records
• File Organization Choice Criteria– Rapid access– Ease of updates– Economy of storage– Reliability
Some File Types
• Pile
• Sequential
• Indexed Sequential
• Indexed
• Direct or Hashed
Pile
• Variable length records with variable fields
• Fields must be self describing
• No order to records - exhaustive search
F1 F2 F5
F1 F3
Record 1
Record 2
F4 DataF4 Hdr
F3
Sequential File• Fixed length records of fixed format fixed
length fields. Distinguished key field.
• Records sorted in key field order
• Search is typically sequential
• Insertions are not easy - so usually accumulated in separate overflow file with periodic update of the master file
17 F2 F3 F4 F524 F2 F3 F4 F526 F2 F3 F4 F5
14 F2 F3 F4 F5 Record 1Record 2
Record 3Record 4
Key
Indexed Sequential Files
• Sequential file with index file and overflow file for updates
• Records still in order of key index
• Overflow file also accessible through index - Records have Next_Record field which can point into overflow file.
• Multiple level so indexing can speed lookups
Indexed Sequential File
5
42
35
18
9
11
18
35
42
13
38
15
5Index
OverflowFile
MasterFile
Key
Indexed Files
• Index for each searchable field, but each record must have at least one searchable field
• Records not sorted, no overflow needed
• Allows for variable length fields and records with missing fields
• Exhaustive index - one entry per record, versus Partial index - on entry per record containing the indexed field.
Indexed File
PartialIndex Exhaustive
Index
Indexed File
Hashed Files
• Hashed access with overflow tables to handle collisions
• Usually used with fixed length records
• Very fast access
• No way to sequentially access all records
Hashed File
8
37
91
231
19
54
14
83
24
87
23
43
Overflow Table37
87
23
91
24
14
Hashed File
Pointers calculated via hashing function
File Systems
CS 470 - Spring 2002
Segment III
Directories
• Contents– Name– File type and organization– Address: Size, location– Access control information– Usage Information - times, current usage
• Structure - e.g. directory tree
• Operations: Search, Create file, Delete file, and List Directory
File Sharing
• Access Rights - Allow various users or groups to share files
• Simultaneous Access control - Allow multiple threads to share the same file at the same time
Record Blocking
• Three approaches– Fixed Blocking– Variable length records with records
restricted to a single block - no spanning– Variable length records which can overlap
(span) two blocks
• First two may have internal fragmentation
• Third is hard to update
Record Blocking Methods
Rec 1 Rec 2 Rec 3
Rec 4 Rec 5 Rec 6
Rec 1 Rec 2 Rec 3 Rec 4
Rec 5 Rec 6 Rec 6 R7
Rec 3 Rec 4 Rec 5
Rec 1 Rec 2
Rec 4
Block 1
Block 2
Block 1
Block 2
Block 1
Block 2
SpanningRecords
Fragment
Disk Space Allocation (1 of 2)• Recorded in File Allocation Table
• Pre-allocation (at file creation) versus dynamic allocation of space
• Allocation size - variable or fixed, large or small, contiguous or not
• Allocation Methods– Contiguous - just need address and total
size– Chained - pointer to next block– Indexed - index refers to file portions
Allocation Methods
0 UU213
0 21
3 543
21
0 12
5
Indexed
Chained
Contiguous
Disk Space Allocation (2 of 2)
• Free Space Management (Disk Allocation Table)– Bit maps of Free Blocks– Chained Free Portions– Index of Free Portions
• Reliability Problems– Caused by improper shutdown when File
and Disk allocation tables kept in memory– Locking is often too expensive
Free List Management
0
14 42205
432 4344333
321
51 522
1
Indexed
Chained
Bit Map 1 1 0 0 1 0 1 1 1 1
Free Allocated
Free
Unix File System
CS 470 - Spring 2002
Segment IV
Classic Unix File System
Superblock
InodeTable
Files andDirectories
Superblock
Free Inode List
Free Block List
InodeTable
BufferCache
Main MemoryDisk Drive
FileTable
V6 Unix Inode Entries
• i_mode - type (dir, char or block special, or ordinary) and file permissions
• i_nlink - number of names (hard links)
• i_uid, i_gid - user and group owners
• i_size0, i_size1 - size of file in bytes
• i_ addr[8] - location of file on disk
• i_atime, i_mtime - access and modification times
Classic V7 Unix Inode
Rest of inode
256 More Block Nbrs
10 Direct Block Nbrs
256 More Indirect Block Nbrs
256 Block Nbrs 255 More of these
Block number ofTriple Indirect Block
Inode
File location part of inode
Double IndirectBlock
Single Indirect Block
Unix File Allocation Limits
Level Nbr of Blks Nbr of Bytes
Direct 10 10 KB
Single Indirect 256 256 KB
Double Indirect 256 x 256 64 MB
= 64 K
Triple Indirect 256 x 64 K 16 GB
= 16 M
Classic Unix Free Lists
Superblock Free Block List
255 Free Block Nbrs
255 Free Block Nbrs
Superblock Free Disk Inode List
One per diskPartition
One per diskPartition
Summary
• File Organization
• File System Organization– File Directories– File Sharing– Record Blocking– Disk space allocation
• Examples: Unix and FAT File Systems