operating systemslecturenotes session 10

7/30/2019 Operating SystemsLectureNotes Session 10

1/50

OPERATING SYSTEM

FILE SYSTEM

Session 10(Chapter 9)


2/50

Concept and Structure of a File

OperatingSystems-FileSystem

Contiguous logical

address space

Types: Data

numeric character

binary

Program

None - sequence of words,bytes

Simple record structure Lines Fixed length Variable length

Complex Structures Formatted document Re-locatable load file

Can simulate last two with first

method by inserting appropriatecontrol characters

Who decides: Operating system Program

2


3/50

Attributes of a File


Name only information kept in human-readable form

Identifier unique tag (number) identifies file within file

system

Type needed for systems that support different types

Location pointer to file location on device Size current file size

Protection controls who can do reading, writing,

executing

Time, date, and user identification data forprotection, security, and usage monitoring

Information about files are kept in the directory structure,

which is maintained on the disk

3


4/50

File Operation


File is an abstract data type

Create

Write

Read

Reposition within file Delete

Truncate

Open(Fi) search the directory structure on disk for

entry Fi, and move the content of entry to memory Close (Fi) move the content of entry Fi in memory to

directory structure on disk

4


5/50

What is needed to Open Files


Several pieces of data are needed to manage open files: File pointer: pointer to last read/write location, per process that

has the file open

File-open count: counter of number of times a file is open to

allow removal of data from open-file table when last processescloses it

Disk location of the file: cache of data access information

Access rights: per-process access mode information

5


6/50

Open File Locking


Provided by some operating systems and file systems

Mediates access to a file

Mandatory or advisory: Mandatory access is denied depending on locks held and

requested Advisory processes can find status of locks and decide what

to do

6


7/50

File typeName and Extension


7


8/50

What is File System?


The file system is a mechanism for storing data and programs.

Main characteristics of Secondary storage Large storage capacity

Non-volatile nature.

Consequence? Give raise to the convenience of storing data and programs on secondary

storage.

8


9/50

What do you Require of File

Systems?

Naming Should be flexible, e.g., allow multiple names for same files

Support hierarchy for easy of use

Persistence Want to be sure data has been written to disk in case crash occurs

Sharing/Protection Want to restrict who has access to files

Want to share files with other users

Speed & Efficiency for different access patterns Sequential access

Random access

Sequential is most common & Random next

Other pattern is Keyed access (not usually provided by OS


9


10/50

What do you Require of File

Systems?

Minimum Space Overhead Disk space needed to store metadata is lost for user data

Twist: all metadata that is required to do translation must be stored

on disk Translation scheme should minimize number of additional accesses for

a given access pattern

Harder than, say page tables where we assumed page tables

themselves are not subject to paging!


10


11/50

File System Organization

The file system on the disk of an operating system allows data to be stored,

searched and retrieved.

One part of the file system is the user interface.

This interface defines the definition of a file, file attributes, operations

allowed on files and a directory structure for organizing files.

The other part of the file system has algorithms and data structures to map

the logical file system onto the physical storage devices.

A file system has a layered design Each layer uses the features provided by the layer below it to provide features to the

layer above it.

The lowest layer is the I/O control. It consists of device drivers and interrupt handlers for information transfer between

the memory and the disk.

A device driver is a translator between the file system and the actual disk

hardware.

The device driver takes high level instructions as input.


11


12/50


The next layer, the basic file system issues generic commands to the

appropriate device driver to access (read / write) physical blocks on the disk. The layer above the basic file system is the file organization module.

This module has information about files, the logical records and the physical

blocks. The file organization module translates / maps logical addresses to physical block

addresses for the basic file system.

This module also keeps track of free blocks.

The logical file system makes use of the directory structure of the file system

and provides information to the file organization module given a file name.

Protection and security are also part of the logical file system.


12


13/50



13


14/50

Over-View


14


15/50

OperatingSystems-FileSystem15


16/50

Steps in Opening and Reading a

File

Look-up (via-directory) Find on disks file descriptors block number

Find entry in open file table. Create one if none, else increment ref count

Find where file data is located By reading on-disk file descriptor

Read data & return to user


16


17/50

Open File Table

inoderepresents file at most 1 in-memory instance per unique file

#number of openers & other properties

filerepresents one or more processes using an file

With separate offsets forbyte-stream dirrepresents an open directory file

Generally: None of data in OFT is persistent

Reflects how processes are currently using files

Lifetime of objects determined by open/close Reference counting is used


17


18/50

File Descriptors (inodes)

Term inode can refer to 3 things: 1.in-memory inode

Store information about an open file, such as how many openers, corresponds

to on-disk file descriptor

2.on-disk inode Region on disk, entry in file descriptor table, that stores persistent information

about a filewho owns it, where to find its data blocks, etc.

3.on-disk inode, when cached in buffer cache A bytewise copy of 2. in memory

Q.: Should in-memory inode store a pointer to cached on-disk

inode?


18


19/50

What information does

File System contain?

Contains super blockstores information such

assize of entire file system, etc. Location of file descriptor table & free map

Free Block Map Bitmap used to find free blocks

Typically cached in memory

Superblock & free map often replicated in different

positions on disk


19


20/50

Allocation of Disk Space

These mechanisms looks at how efficiently files are

stored and accessed. Contiguous

Linked

Indexed Commonly referred as file allocation strategies


20


21/50

Contiguous Allocation

Idea: allocate files in contiguous blocks

File Descriptor = (first block, length)

Good sequential & random access

Problems: hard to extend filesmay require expensive compaction

external fragmentation

analogous to segmentation-based VM


21


22/50

Linked Files

Idea: implement linked list either with variable sized blocks

or fixed sized blocks (clusters)

Solves fragmentation problem, but now need lots of seeks for sequential accesses and random

accesses

unreliable: lose first block, may lose file

Solution: keep linked list in memory DOS: FAT File Allocation Table


22


23/50

OperatingSystems-FileSystem23


24/50

Indexed Allocation

Single-index: specify maximum file-size, create index

array, then note blocks in indexRandom access okone translation step

Sequential access requires more seeksdepending oncontiguous allocation

Drawback: hard to grow beyond maximum


24


25/50

Directory Structure

Directory is collection of nodes containing

information about files.


Both the directory structureand the files reside on disk

Backups of these twostructures are kept on tapes

25


26/50

Disk Structure

Disk can be subdivided into partitions

Disks or partitions can be RAID protected against failure

Disk or partition can be used raw without a file system, or

formatted with a file system

Partitions also known as minidisks, slices Entity containing file system known as a volume

Each volume containing file system also tracks that file

systems info in device directory or volume table of

contents As well as general-purpose file systems there are many

special-purpose file systems, frequently all within the same

operating system or computer.


26


27/50

Disk Schematics


27


28/50

Tracks Sectors and Cylinders


28


29/50

From O.S Perspective

Disks are big & slow -compared to RAM

Access to disk requires Seek (move arm to track)to cross all tracks anywhere from 20-50ms, on

average takes 1/3.

Rotational delay (wait for sector to appear under track) 7,200rpm is 8.3ms

per rotation, on average takes : 4.15ms rot delay Transfer time (fast: 512 bytes at 998 Mbit/s is about 3.91us)

Seek+Rot Delay dominates

Random Access is expensive And unlikely to get better

Consequence: Avoid seeks

Seek to short distances

Amortize seeks by doing bulk transfers


29


30/50

Disk Scheduling

Can use priority scheme

Can reduce avg access time by sending requests to disk controller

in certain order Or, more commonly, have disk itself reorder requests

SSTF: shortest seek time first Like SJF in CPU scheduling, guarantees minimum avg seek time, but

can lead to starvation

SCAN: elevator algorithm Process requests with increasing track numbers until highest reached,

then decreasing etc.repeat

Variations: LOOKdont go all the way to the top without passengers

C-SCAN: -only take passengers when going up


30

T i l Fil S t


31/50

Typical File System

Organization


31


32/50

Operations Performed on a Directory


Search for a file

Create a file

Delete a file

List a directory

Rename a file Traverse the file system

32


33/50

Organize the Directory (Logically) to Obtain


Efficiency locating a file quickly

Naming convenient to users Two users can have same name for different files

The same file can have several different names

Grouping logical grouping of files by properties, (e.g.,all Java programs, all games, )

33


34/50

Single Directory

A single directory for all users

Naming and grouping problem


34


35/50

Single Directory

Separate directory for each user. Path name

Can have the same file name for different user Efficient searching

No grouping capability


35


36/50

Tree Structured Directories


Efficient searching

Grouping Capability

Current directory (working

directory)

cd /spell/mail/prog

type list

36


37/50

Tree Structured Directories


Absolute orrelative path name

Creating a new file is done in current directory

Delete a file

rm

Creating a new subdirectory is done in current directorymkdir

Example: if in current directory /mail

mkdir count

Deleting maildeleting the entire subtree rooted by mail

37


38/50

Acyclic Graph Directories


They have sharedSub-directories and files

38


39/50

Acyclic Graph Directories

Two different names (aliasing)

Ifdict deletes list dangling pointer

Solutions: Back-pointers, so we can delete all pointers

Variable size records a problem Back-pointers using a daisy chain organization

Entry-hold-count solution

New directory entry type Link another name (pointer) to an existing file

Resolve the link

follow pointer to locate the file


39


40/50

General Graph Directory

How do we guarantee no

cycles?Allow only links to file not

subdirectories

Garbage collection

Every time a new link is

added use a cycle detection

algorithm to determine

whether it is OK.


40


41/50

Directory Implementation

Linear List

list of file names with pointers to the data blocks is one way to implement a

directory.

linear search is necessary to find for a particular file

The method is simple but the search is time consuming.

To create a file a linear search is made to look for the existence of a file with

the same file name and if no such file is found the new file created is added to

the directory at the end.

To delete a file a linear search for the file name is made and if found allocated

space is released.

Every time making a linear search consumes time and increases access time

that is not desirable since a directory information is frequently used.

A sorted list allows for a binary search that is time efficient compared to the

linear search.

But maintaining a sorted list is an overhead especially because of file creations and

deletions. OperatingSystems-FileSystem

41


42/50

Directory Implementation

Hash Table

A linear list is used to store directory entries.

A hash table takes a value computed from the file name and returns a

pointer to the file name in the linear list.

Thus search time is greatly reduced.

Insertions are prone to collisions that are resolved.

The main problem is the hash function that is dependent on the hash

table size.

A solution to the problem is to allow for chained overflow with each hash

entry being a linked list.

Directory lookups in a hash table are faster than in a linear list.


42


43/50

How is File System Mounted?

A file system must be

mounted before it can be

accessed

A un-mounted file system ismounted at a mount point


(a) Existing. (b) Unmounted Partition

Mount point

43


44/50

File Sharing

Sharing of files on multi-user systems is desirable

Sharing may be done through a protection scheme

On distributed systems, files may be shared across a

network

Network File System (NFS) is a common distributed file-sharing method.

Multiple Users

User IDsidentify users, allowing permissions and protections to be

per-user

Group IDsallow users to be in groups, permitting group access

rights


44

File Sharing Remote File


45/50

Uses networking to allow file system access between systems Manually via programs like FTP Automatically, seamlessly using distributed file systems Semi automatically via theworld wide web

Client-servermodel allows clients to mount remote file systemsfrom servers Server can serve multiple clients Client and user-on-client identification is insecure or complicated NFS is standard UNIX client-server file sharing protocol CIFSis standard Windows protocol Standard operating system file calls are translated into remote calls

Distributed Information Systems (distributed naming services)

such as LDAP, DNS, NIS, Active Directory implement unified accessto information needed for remote computing

File Sharing Remote File

Systems


45


46/50

File Sharing Failure Modes

Remote file systems add new failure modes, due to

network failure, server failure

Recovery from failure can involve state information about

status of each remote request

Stateless protocols such as NFS include all information ineach request, allowing easy recovery but less security


46

File Sharing Consistency


47/50

File Sharing Consistency

Semantics

Consistency semanticsspecify how multiple users areto access a shared file simultaneously Similar to process synchronization algorithms

Tend to be less complex due to disk I/O and network latency (forremote file systems

Andrew File System (AFS) implemented complex remote filesharing semantics Unix file system (UFS) implements:

Writes to an open file visible immediately to other users of the sameopen file

Sharing file pointer to allow multiple users to read and writeconcurrently

AFS has session semantics Writes only visible to sessions starting after the file is closed


47


48/50

Protection

File owner/creator should be able to control:

what can be done

by whom

Types of access

Read

Write

Execute

Append

Delete

List


48


49/50

Access Lists and Groups

Mode of access: read, write, execute Three classes of users

Ask manager to create a group (unique name), say G, and addsome users to the group.

For a particular file (say game) or subdirectory, define anappropriate access.

Attach a group to a file


49

A Sample UNIX Directory


50/50

A Sample UNIX Directory

Listing50

operating systemslecturenotes session 10

Documents