1

Chapter 10 : Chapter 10 : CCase Study ase Study --

UNIXUNIX• History • Overview • Processes • Memory management • Input/output in • The unix file system • Security

Note: This case study covers only UNIX. Please read chapter 10 of the text book for LINUX.

2

History of UNIXHistory of UNIX

Originated from the MULTICS Originated from the MULTICS (Multiplexed Information and (Multiplexed Information and Computing Service) operating system Computing Service) operating system by M.I.T, Bell Labs and GEby M.I.T, Bell Labs and GE

There are two main versions:There are two main versions: AT&T System V Release 4 (SVR4) AT&T System V Release 4 (SVR4)

Originally developed by AT&T, now Originally developed by AT&T, now SCOSCO

BSD (Berkeley Software Distribution)BSD (Berkeley Software Distribution)

3

Overview of UNIXOverview of UNIX Supports various architecturesSupports various architectures Structure variesStructure varies Supports preemptive multitaskingSupports preemptive multitasking Multiuser environment - generally secureMultiuser environment - generally secure Supports multithreaded applicationsSupports multithreaded applications Protection/Security is high on modern Protection/Security is high on modern

versionsversions Supports symmetric multiprocessingSupports symmetric multiprocessing Highly scalabHighly scalablle/portable to various e/portable to various

systemssystems Many types/flavours of UNIX existMany types/flavours of UNIX exist

4

UNIXUNIX Layers Layers

The layers of a UNIX system.The layers of a UNIX system.UserInterface

5

UNIX Utility ProgramsUNIX Utility Programs

A few of the more common UNIX utility programs required by POSIXA few of the more common UNIX utility programs required by POSIX

6

UNIX KernelUNIX Kernel (1) (1)

Approximate structure of generic UNIX Approximate structure of generic UNIX kernelkernel

7

UNIX KernelUNIX Kernel (2) (2)

Bottom layerBottom layer Device driversDevice drivers for character and block for character and block

devicesdevices Process dispatcherProcess dispatcher which stops the which stops the

current process, saves its state and current process, saves its state and starts the appropriate driver when an starts the appropriate driver when an interrupt occursinterrupt occurs

8

ProcessProcess Creation Creation in UNIXin UNIX - fork- fork

Process creation in UNIX.Process creation in UNIX.

9

POSIX ShellPOSIX Shell

A highly simplified shellA highly simplified shell

10

The The lsls Command Command

Steps in executing the command Steps in executing the command lsls type to the shell type to the shell

11

Some UNIX Process Some UNIX Process ConceptsConcepts

DaemonsDaemons (background processes) (background processes) Cron daemonCron daemon which wakes up once a which wakes up once a

minute to check scheduled events (eg., minute to check scheduled events (eg., disk backup)disk backup)

PipesPipes - syncronized channels between - syncronized channels between processes to pass byte streamsprocesses to pass byte streams

SignalsSignals – software interrupts used for – software interrupts used for interprocess communication. Choices: interprocess communication. Choices: catch, ignore, or kill processcatch, ignore, or kill process

12

Signals Required By Signals Required By POSIXPOSIX

The signals required by POSIX.The signals required by POSIX.

13

System Calls for Process System Calls for Process ManagementManagement

ss is an error code is an error code

pidpid is a process ID is a process ID

residualresidual is the remaining time from the previous alarm is the remaining time from the previous alarm

14

ThreadThread Calls Calls in POSIX in POSIX

The principal POSIX thread calls.The principal POSIX thread calls.

15

ThreadThread Calls Calls in POSIX in POSIX

Threads were not in the first versions of UNIXThreads were not in the first versions of UNIX There are many thread packages in use which There are many thread packages in use which

are standardized in POSIXare standardized in POSIX Thread calls are the same for user-space or Thread calls are the same for user-space or

kernel-space kernel-space In kernel-space implementation calls are In kernel-space implementation calls are

system callssystem calls In user-space implementation calls are to a In user-space implementation calls are to a

run-time libraryrun-time library

16

Thread Communication - Thread Communication - mutexesmutexes

Threads use locks called Threads use locks called mutexes mutexes for short-for short-time locking a resource (say a shared buffer)time locking a resource (say a shared buffer)

A mutex must be first A mutex must be first createdcreated (and finally (and finally destroyed)destroyed)

Mutual exclusion is implemented by Mutual exclusion is implemented by locking locking a mutexa mutex before accessing a resource and before accessing a resource and unlock itunlock it when they are done (like binary when they are done (like binary semaphores which is “0” or “1”, a mutex is semaphores which is “0” or “1”, a mutex is either locked or unlocked)either locked or unlocked)

17

Thread Communication Thread Communication – condition variables– condition variables

For long-term synchronization (such as For long-term synchronization (such as waiting for a tape to become free) waiting for a tape to become free) condition variablescondition variables are used are used

Condition variables have to be Condition variables have to be createdcreated first and later first and later destroyeddestroyed like mutexes like mutexes

A condition variable is used by having one A condition variable is used by having one thread thread waitwait on it, and another thread on it, and another thread signalsignal it. If no thread is waiting when a it. If no thread is waiting when a signal is sent, the signal is lostsignal is sent, the signal is lost

18

UNIX SchedulerUNIX Scheduler (1) (1)

The UNIX scheduler is based on a multilevel queue The UNIX scheduler is based on a multilevel queue structurestructure (highest priority queue first, round-robin in each (highest priority queue first, round-robin in each queue)queue)

In this scheme, a process which was blocked and waiting for In this scheme, a process which was blocked and waiting for an event joins the appropriate queue when blocking is over an event joins the appropriate queue when blocking is over (a process whose disk I/O is finished joins,say, queue –4) (a process whose disk I/O is finished joins,say, queue –4)

19

UNIX SchedulerUNIX Scheduler (2) (2) Once a second the priority of all processes are Once a second the priority of all processes are

recalculated to avoid starvation using recalculated to avoid starvation using priority = CPU_usage + nice + basepriority = CPU_usage + nice + base

CPU_usage, represents the average number of CPU_usage, represents the average number of clock ticks per second that the process has had clock ticks per second that the process has had during the past few secondsduring the past few seconds

Nice is a value between –20 to 20 (default 0). Nice is a value between –20 to 20 (default 0). Nice system call can be used to set this value 0-Nice system call can be used to set this value 0-2020

Base is a system parameter in UNIX source code Base is a system parameter in UNIX source code The scheduler forces CPU bound (on positive The scheduler forces CPU bound (on positive

queues) get any service that is left over when all queues) get any service that is left over when all I/O bound and interactive processes are blockedI/O bound and interactive processes are blocked

20

Booting UNIXBooting UNIX (1) (1)

The first sector of the boot disk (The first sector of the boot disk (master master boot recordboot record) is read in and executed) is read in and executed

This sector loads the This sector loads the bootboot program program Boot reads root directory, loads Boot reads root directory, loads kernelkernel

and starts its executionand starts its execution Kernel reads the rest of the operating Kernel reads the rest of the operating

system (system (main C-code sectionmain C-code section)) C code does some initialization, allocates C code does some initialization, allocates

system data structures, loads device system data structures, loads device drivers and handcrafts the first process, drivers and handcrafts the first process, process 0process 0

21

Booting UNIXBooting UNIX (2) (2)

The sequences of processes used to boot some systemsThe sequences of processes used to boot some systems

cp

22

Handling MemoryHandling Memory

Each process has three segments (Each process has three segments (shown as one segment shown as one segment in the figure, but if hardware supports they can be in the figure, but if hardware supports they can be separateseparate): ): Text : executable code (Text : executable code (which is shared in the figurewhich is shared in the figure)) Data : variables, strings, arrays etc.Data : variables, strings, arrays etc.

initialized data – variables which must be initialized to initialized data – variables which must be initialized to some value when program startssome value when program starts

Uninitialized data (BSS) – not initialized but has value 0 as Uninitialized data (BSS) – not initialized but has value 0 as defaultdefault

StackStack Text is fixed in length, data and stack can grow and shrinkText is fixed in length, data and stack can grow and shrink

Process B

Process A

23

Paging in UNIXPaging in UNIX (1) (1)

Prior to 3BSD, UNIX systems used Prior to 3BSD, UNIX systems used swapping (if memory is full, swap swapping (if memory is full, swap processes to disk)processes to disk)

To run a process, all that is needed To run a process, all that is needed is the is the user structureuser structure and and page page table. table. The pages of the text, data The pages of the text, data and stack segments are brought in and stack segments are brought in on demandon demand

24

Paging in UNIXPaging in UNIX (2) – (2) – Core MapCore Map

Main memory: kernel, core map, pagesMain memory: kernel, core map, pages CCore mapore map has an entry for each page has an entry for each page and and

contains information about the contents of contains information about the contents of the page framesthe page frames

Page on Disk

Process table entry

25

PaPage Replacement ge Replacement Algorithm (1)Algorithm (1)

The page replacement algorithm is The page replacement algorithm is executed by the executed by the page daemonpage daemon (process 2) (process 2)

Page daemonPage daemon wakes up every 250 msec wakes up every 250 msec and transfers pages to disk if the amount and transfers pages to disk if the amount of free memory is less than the system of free memory is less than the system parameter parameter lotsfree lotsfree (typically set to ¼ of (typically set to ¼ of memory)memory)

Page daemon uses the Page daemon uses the two-handed clock two-handed clock algorithmalgorithm

26

Basic Clock Algorithm (1)Basic Clock Algorithm (1)

The pointer (hand) points to the oldest pageThe pointer (hand) points to the oldest page When a page fault occurs, When a page fault occurs,

if the if the RR bit of the pointed page is 0 (page not referred), this bit of the pointed page is 0 (page not referred), this page is evicted (the new page replaces this page - written to page is evicted (the new page replaces this page - written to disk first if it is dirty)disk first if it is dirty)

if the if the RR bit is 1 (page accessed), bit is 1 (page accessed), RR bit is cleared and the bit is cleared and the hand is advanced to the next pagehand is advanced to the next page

27

Two-handed Clock Two-handed Clock Algorithm (2)Algorithm (2)

Page daemon has to do two passes with Page daemon has to do two passes with oneone core map pointer. Pass 1 clears all core map pointer. Pass 1 clears all RR bits, bits, second pass removes pages (second pass removes pages (RR bits are set bits are set between pass 1 and 2)between pass 1 and 2)

Page daemon maintains two pointers into the Page daemon maintains two pointers into the core map to speed up the process (one pass core map to speed up the process (one pass instead of two) for large memoriesinstead of two) for large memories

When page daemon runs, it first clears the When page daemon runs, it first clears the RR bit bit at the at the front handfront hand, and then checks the , and then checks the RR bit at bit at the the back handback hand, after which it advances both , after which it advances both handshands

Each time the page daemon runs, the hands Each time the page daemon runs, the hands rotate less than a full revolution, the amount rotate less than a full revolution, the amount depending on the number of pages needed to depending on the number of pages needed to reach reach lotsfreelotsfree

28

I/O in UNIXI/O in UNIX

All I/O devices are integrated into All I/O devices are integrated into the file system as the file system as special filesspecial files

These special files are accessed like These special files are accessed like ordinary files (ie., file operations ordinary files (ie., file operations such as read, write, open are the such as read, write, open are the same for special filessame for special files

29

NetworkingNetworking in UNIX (1) in UNIX (1)

SSocketsockets are used to establish a are used to establish a connection between connection between network network nodesnodes

30

NetworkingNetworking in UNIX (2) in UNIX (2) Sockets are created and destroyed dynamicallySockets are created and destroyed dynamically Creating a socket returns a file descriptor, which Creating a socket returns a file descriptor, which

is needed for establishing a connection, reading is needed for establishing a connection, reading data, writing data, and releasing the connectiondata, writing data, and releasing the connection

One party makes a One party makes a listenlisten call on a local socket, call on a local socket, which creates a buffer and blocks until data which creates a buffer and blocks until data arrivearrive

The other party makes a The other party makes a connectconnect call giving as call giving as parameters the file descriptor of the local socket parameters the file descriptor of the local socket and the address of a remote socket (a sockets has and the address of a remote socket (a sockets has an address in the network like the internet)an address in the network like the internet)

Once a connection is established, a socket Once a connection is established, a socket functions like a functions like a pipepipe

31

UNIX I/O (1)UNIX I/O (1) When a user accesses a special file, the file system When a user accesses a special file, the file system

determines the major and minor device numbers determines the major and minor device numbers and whether it is a block or character special fileand whether it is a block or character special file

Major device number is used to index into either Major device number is used to index into either bdevswbdevsw array for block special or array for block special or cdevswcdevsw for for character special filescharacter special files

These structures contain pointers to the procedures These structures contain pointers to the procedures to open the device, read, write etc., to open the device, read, write etc., Some of the Some of the fields of a typical fields of a typical cdevswcdevsw table table are shown below are shown below

32

UNIX I/O (2)UNIX I/O (2)

The UNIX I/O system in BSDThe UNIX I/O system in BSD

C-list

33

UNIX I/O (UNIX I/O (33)) For block special files (eg., disks) the blocks For block special files (eg., disks) the blocks

are cached in a are cached in a buffer cachebuffer cache The buffer cache works for both reads and The buffer cache works for both reads and

writeswrites Usually dirty (modified) blocks are written to Usually dirty (modified) blocks are written to

the disk in every 30 secondsthe disk in every 30 seconds For character special devices, data is For character special devices, data is

buffered in a chain of buffered in a chain of C-listsC-lists. A C-list block . A C-list block is 64 characters long, plus a count and a is 64 characters long, plus a count and a pointer to the next block (BSD method of pointer to the next block (BSD method of character buffering)character buffering)

34

The UNIX File System (1)The UNIX File System (1)

Some important directories found in Some important directories found in most UNIX systemsmost UNIX systems

35

The UNIX File System (2)The UNIX File System (2)

Before linking.Before linking. After linking.After linking.

(a) Before linking. (b) After linking

36

The UNIX File System (3)The UNIX File System (3)

Separate file systemsSeparate file systems After mountingAfter mounting

(a) Before mounting (b) After Mounting

37

Locking FilesLocking Files in UNIX (1) in UNIX (1) Accessing a file by several processes need Accessing a file by several processes need

some critical section managementsome critical section management This is done by This is done by lockslocks A lock is defined by a file name, the starting A lock is defined by a file name, the starting

byte and the number of bytesbyte and the number of bytes When placing a lock, the process specifies toWhen placing a lock, the process specifies to

BlockBlock: when the existing block is removed, the : when the existing block is removed, the process is unblocked and the lock is placedprocess is unblocked and the lock is placed

Not to blockNot to block: the system call returns with a : the system call returns with a status code telling whether the lock succeeded or status code telling whether the lock succeeded or notnot

38

Locking FilesLocking Files (2) (2)

((a) File with one locka) File with one lock(b) Addition of a second lock(b) Addition of a second lock(c) A third lock(c) A third lock

39

System Calls for File System Calls for File ManagementManagement

ss is an error code is an error code (-1 if an error has occured) (-1 if an error has occured) fdfd is a file descriptor is a file descriptor (a positive number: 0 standard (a positive number: 0 standard

input)input) positionposition is a file offset is a file offset

40

The The statstat System Call System Call

Fields returned by the Fields returned by the statstat system call system call

41

System Calls for Directory System Calls for Directory ManagementManagement

ss is an error code is an error code dirdir identifies a directory stream identifies a directory stream direntdirent is a directory entry is a directory entry

42

UNIX File System (1)UNIX File System (1)

Disk layout in classical UNIX systemsDisk layout in classical UNIX systems

Block 0 is the boot blockBlock 0 is the boot block Block 1 is the Block 1 is the superblocksuperblock which contains which contains

information about the layout of the file information about the layout of the file system, including the number of i-nodes, system, including the number of i-nodes, number of disk blocks, and start of the list number of disk blocks, and start of the list of free disk blocksof free disk blocks

43

UNIX File System (2)UNIX File System (2)

Directory entry fields.Directory entry fields.

Structure of the i-node in System V

44

UNIX File System (3)UNIX File System (3)

The relation between the file descriptor table, the open file descriptionThe relation between the file descriptor table, the open file description

File descriptor table is indexed by the fd parameter and has one entry for each file

45

UNIX File System (4)UNIX File System (4)

A BSD directory with three filesA BSD directory with three files..The same directoryThe same directory after the file after the file voluminousvoluminous has been removed has been removed

File name can be 255 characters longFile name can be 255 characters long The first 4 fields are fixed lengthThe first 4 fields are fixed length

46

Security in UNIXSecurity in UNIX (1) (1)

Each UNIX user has a Each UNIX user has a UIDUID (User ID). (User ID). A UID is an integer between 0 and A UID is an integer between 0 and 65536. Files, processes and other 65536. Files, processes and other resources are marked with the UID of resources are marked with the UID of their ownertheir owner

The user with UID 0 is the The user with UID 0 is the superusersuperuser Users can be organized in groups, Users can be organized in groups,

which are also numbered with 16-bit which are also numbered with 16-bit GIDGID’s (Group ID) ’s (Group ID)

47

Security in UNIXSecurity in UNIX (2) (2)

Some examples of file protection modesSome examples of file protection modes

48

System Calls for File System Calls for File ProtectionProtection

ss is an error code is an error code uiduid and and gidgid are the UID and GID, respectively are the UID and GID, respectively