chapter 11: file-system...

Chapter 11: File-System Implementation

Operating Systems CS 33211 2

Chapter 11: File System Implementation

� File-System Structure

� File-System Implementation

� Directory Implementation

� Allocation Methods

� Free-Space Management

� Efficiency and Performance

� Recovery

� Log-Structured File Systems

� NFS


Objectives

� To describe the details of implementing local

file systems and directory structures

� To discuss block allocation and free-block

algorithms and trade-offs


File-System Structure

� File structure� Logical storage unit

� Collection of related information

� File system resides on secondary storage (disks). � Allows read, write and modify block of data from disk� Sequence or Random access to any given block of information

� I/O transfers between disk and memory in blocks of typically 512bytes (efficiency)

� File system organized into layers� Efficiency

� Avoids duplication of code

� File System maintains file structure via File control block: storage structure consisting of information about a file� Ownership

� Permissions

� Locations of file contents …


The file system provides efficient and convenient

access to information on the disk

� File System Design Challenges:

1. File system UI

� File Attributes, Operations

� Directory structure

2. Mapping logical file system to physical storage (disk)

� Requires efficient algorithms and data structures

File system Design Challenges


File System

Layered Design

� In Layered Architecture� Higher levels make use of features in the lower

level

� I/O Control � Uses Device drivers to transfer info between

memory and disk

� Device driver

� Translates commands to hardware instructions for use by controller

� Basic file system:� Issues generic commands to read/write physical

blocks on disk

� File Organization� Translates logical block addresses to physical

block addresses

� Logical File System� Manages directory structure

� Metadata information

� Maintains file structure via File Control Block (FCB)


A Typical File Control Block


Creating a new file

1. Your application program calls the logical file system

2. Logical file system creates a new FCB (allocates one from free FCBs)

3. Systems reads appropriate directory into memory

� Creates new file name

� Creates new FCB

4. Writes back onto disk

Now the new file may be used for I/O operations


In-Memory File System Structures

� Necessary file system structures provided by the

OS.

� Figure 12-3(a) refers to opening a file.

� Figure 12-3(b) refers to reading a file.


In-Memory File System Structures

� The open( ) call first searches “system-wide-open file table” to see if file is in use

� If file is in use:� An entry is created in “per-process

open-file table”. Entry points to system-wide open-file-table

� Else:� Search directory structure for fie name

� Copy file’s FCB to system-wide open file table

� An “entry” is created in “per-process open-file table”. Entry points to system-wide open-file-table

� Open( ) call returns a pointer to the entry in the per-process open-file table

All I/O on the file are performed via pointer


Virtual File Systems

� Virtual File Systems (VFS) provide an object-oriented way to simplify, organize and modularize File-system implementation

� VFS provides a uniform system call interface (the API) to be used for different types of file systems.

� The API is to the VFS interface, rather than any specific type of file system:

� Using data structures and procedures

OS concurrently supports multiple type of file-systems via VFS interface


Schematic View of Virtual File System

� Layer 1: File System Interfaces handles:

� System calls: open(), read(),

write(), close()

� Layer 2: Virtual File System

� Separates generic operations from

implementation details

� API provides a uniform interface to different types of file system implementations

� Layer 3: Supported file system types


Directory Implementation

� Linear list simple to program but time-consuming to execute� Maintain a list of file names with pointers to the data blocks.� To create a new file:

� Search dir (or list) to be sure file does not exist� Add new at end of list

� To delete a file:� Search list for file then delete name from file

� Disadvantage: Linear search of directory is time consuming

� Hash Table – linear list with hash data structure.� Linear list stores directory entries

� Harsh table takes a value computed from file name and returns a pointer to file name in the list� decreases directory search time

� collisions – situations where two file names hash to the same location

� Disadvantage: Fixed size

Directory management algorithms can affect efficiency of file-systems


Allocation Methods

� An allocation method refers to how disk

blocks are allocated for files:

� Contiguous allocation

� Linked allocation

� Indexed allocation


Contiguous Allocation

� Each file occupies a set of contiguous blocks on the disk

� Simple – only starting location (block #) and length (number of

blocks) are required

� Random access

� Wasteful of space (dynamic storage-allocation problem)

� Files cannot grow


Contiguous Allocation

� Mapping from logical to physical

LA/512

Q

R

Block to be accessed = ! + starting address

Displacement into block = R


Contiguous Allocation of Disk Space


Extent-Based Systems

� Many newer file systems (I.e. Veritas File

System) use a modified contiguous allocation

scheme

� Extent-based file systems allocate disk

blocks in extents

� An extent is a contiguous block of disks

� Extents are allocated for file allocation

� A file consists of one or more extents.


Linked Allocation

� Each file is a linked list of disk blocks: blocks

may be scattered anywhere on the disk.

pointerblock =


Linked Allocation (Cont.)

� Simple – need only starting address

� Free-space management system – no

waste of space

� No random access

� Mapping

Block to be accessed is the Qth block in the linked chain of

blocks representing the file.

Displacement into block = R + 1

LA/511

Q

R


Linked Allocation


Indexed Allocation

� Brings all pointers together into the index

block.

� Logical view.

index table


Example of Indexed Allocation


Linked Allocation

File-Allocation Table (FAT)

� FAT resides at beginning of

each disk partition

� Each FAT index represents

block number of the file

� Last block has eof value

� Directory entry contains the

block number of first block

� Each FAT entry contains the

next block number of the file

MS-Dos and OS2 Operating Systems


Indexed AllocationSmall Files (max size 256K words; block size 512 words)

� uses directory entry table

� Uses Index table, size normally one disk block (512)

� Provides random access

� Provides dynamic access without external fragmentation,

but have overhead of index block.

LA/512

Q

RQ = displacement into index table

R = displacement into block

Recall, index allocation:

For small files: Mapping from logical to physical:

Requires only 1 index table


Indexed AllocationLarge Files (Unbounded Length; block size 512 words)

� Linked scheme – Link several blocks of index table (no limit on size).

LA / (512 x 511)

Q1

R1

Q1 = block of index table

R1 is used as follows:

R1 / 512

Q2

R2

Q2 = displacement into block of index table

R2 displacement into block of file:

Last word is pointer to another index block

For Large Files:


Indexed Allocation

Two-Level Index

� Two-level index (maximum file size is 5123)

LA / (512 x 512)

Q1

R1

Q1 = displacement into outer-index

R1 is used as follows:

R1 / 512

Q2

R2

Q2 = displacement into block of index table

R2 displacement into block of file:


Indexed Allocation

Multi-level Index

� Directory entry table points to

the start of the outer index

table

� Outer-index table points to a

set of second level index

blocks

� Second level block may

also point to another set of

third level index blocks

� Third level points to fourth

…

� Entry in index table contains

direct access block address

M

outer-index

index table file


Combined Scheme:

UNIX File Systems (4K bytes per block)

� Inode contains N pointers of the index

block (E.g., N =12)

� First nine pointers point to direct

blocks (small files ~ 4*9 KB)

� The 10th pointer points to an

indirect block (an index block)

� Index block contains address

of blocks that contain data

blocks

� The 11th pointer points to double

indirect block (an index block)

� The double indirect block

contains?

� The 12th pointer points to triple

indirect block


Free-Space Management

Bit Vector

� To create new files, we need to

re-use free space from deleted

files

� How do you keep track of free

space on the disk?

� Maintain a free-space list

� Space not allocated to

files or directory

� Implemented as Bit

Vector

…

0 1 2 n-1

bit[i] =

67

8 0 ⇒ block[i] free

1 ⇒ block[i] occupied

Block number calculation

(number of bits per word) *

(number of 0-value words) +

offset of first 1 bit

Block number calc applies to

First non-zero word

n-Bit vector:



n-Bit Vector Storage

� n-Bit vector requires extra space

� Example:

block size = 212 bytes

disk size = 230 bytes (1 gigabyte)

n = 230/212 = 218 bits (or 32K bytes)

� Requires 32K bytes to store bit maps

� Easy to get contiguous files



Linked Free Space List on Disk

� Basic idea is to link all free disk blocks

� Requires a pointer to first free block

� Pointer stored on disk

and cache

� Each block contains

a pointer to next free disk block

� Linked list (free list)

� Cannot get contiguous

space easily

� No waste of space


Efficiency and Performance

� Efficiency dependent on:

� Disk allocation and directory algorithms

� Linked allocation Vs Index allocation

� Types of data kept in file’s directory entry

� Pointer size limits the length of file

� 16-bit pointer � 216 (64 KB)

� 32-bit pointer � 232 (4 GB) � Requires more disk space to store index tables

� Performance

� Disk cache – separate section of main memory for frequently used Disk blocks

� Free-behind and read-ahead – techniques to optimize sequential access


Performance

Page Cache

� A page cache caches pages rather than disk

blocks using virtual memory techniques

� Memory-mapped I/O uses a page cache

� Routine I/O through the file system uses the

buffer (disk) cache

� This leads to the following figure


Performance

I/O Without a Unified Buffer Cache


Performance

Unified Buffer Cache

� A unified buffer cache uses the same page

cache to cache both memory-mapped pages

and ordinary file system I/O


Performance

I/O Using a Unified Buffer Cache

chapter 11: file-system...

Documents