levels of abstraction in operating systems

7/28/2019 Levels of Abstraction in Operating Systems

1/45

Levels of AbstractioninOperating Systems

Robert L . BrownPeter J . DenningWalter F. Tiehy

J ul y , 191i4

Research In st itute for Advanced Computer ScienceN A S A Arnes Research Center

RlACS TR. 4.5{ N A S A - C R - 1 8 7 2 8 2 ) L E V E L S OF A B S T R A C T I O N I NO P E R A T I N G S Y S T E M S (Research I n s t - f o rAdvanced Compu te r S c ience) 4 5 p

0

h90-71363

uncl as00/61 0295376

RlACSesearch InstituteforAdvanced Computer Science


2/45

Levels of AbstractioninOperating Systems

Robert L. BrownPeter J . DenningWalter F. Tichy

Abetract:By defining the wr's view of a computer system and aiding in every command, the. operating systemplays a critical role in computing. Operatingsystems can be modeled by a hierarchy of fevers correspond-ing to important time scales or object sires within the computer system.

_ .

Key words and Phrases:Abstract data types, distributed file q s k m , distributed system, hierarchical design, levels of abstraction,network operating system, object-orimted system, operating system.

Work on his article w a s supported by the National Aeronautics and Space Administration underContract NASZ-11530

Authors' add-: Robert L. Brown,RUCS, NASA A m a Bcrurch Center, MS 230-6, Moffett Field, CA 94035 (net ad-dress: brornOricrcr); Peter 1. Denning, RUCS, NASA A m a R a e u c h Center, MS 2m5, Moffett ?%Id, CA 94035 net addreu:denningOriru); Walter F. Tichy, Computer Scienca Department, Purdue University, Weat Lafayette, IN 47907 (net add-:tichyOpurduc).


3/45

1.NTRODUCTION .................................................................................................1.1 - Definitions of Operating System ...........................................................................1.2 - Current Operating Systems ..................................................................................2 -MODEL O F AN OPE RAT ING SYSTEM ................................................................2 .1 -Th e Sing le-Machhe Levels: 1-8 ........... ..-............................................................2.2 - The Multi-Maehhe Levels: 9-14 ............................................................................2.5 -General Comments on Level Structure ..................................................................5 - DISTRIBUTED CAPABILITIES (Level 8) ..............................................................4 - COMMUNICATIONS (Level 9) ..............................................................................5 - FILES (Level 10) ..................................................................................................6 - DEVICES (Level 11) .............................................................................................7 - STREAM 1/0 Level 12) .......................................................................................8 - USER PROCESSES (Level 15) ...............................................................................9 - DIRECTORIES (Level 14) .....................................................................................10- SHELL (Level 15) ...............................................................................................11- NITIALIZATION ...............................................................................................12- CONCLUSION ...................................................................................................13 - ACKNOWLEDGEMENTS ............... .


4/45

1. INTRODUCTIONThe operating systems of 1955 were control programs a few thousand bytes long that

scheduled j obs , drove peripheral devices, and billed users. The operat ing systems of 1984 aremuch larger both in size and in responsibility. The largest ones, such as Honeywells Multics orIBMs MVS, re tens of millions of bytes long. Intermediate ones, such as Bell Labss UNIX orDigital Equipments VMS, re several hundreds of thousands of by tes long. Even the smallest,most pared-down systems for personal computers are tens of thousands of bytes long.

Th e intellectual content of the field.of opera ting systems was recognized in the early

1970s. Virtually every curriculum in computer science an d engineering includes a course onoperating systems. Texts are numerous. The continuing debates - over the set of conceptsth at should be taught and over th e proper mix between concepts and implementation projects

... *.&I- -- are signs of the vitality of the field. - . . -.aiSince 1975, personal computers for home and business have grown into a multi- billion

dollar industry. Advanced graphics workstations and microcomputers have been proliferating.Local networks - .g., Ethernet, ring nets, wideband nets - nd network protocols - .g., X.25,r U L ,TCP/!P -- a!!cw !arge eyskma to be constructed from many small ones. The availabler r nhardware has grown rapidly in power and sophistication.

In view of t he rapid advances in power and sophistication of available hardware, it isnatural to ask: Will hardware eventually obviate software control programs? Is the intellectualcore recorded in operating systems texts outmoded? Is operating systems a dying field? Inthis paper we will argue that the power and complexity of the new hardware intensifies theneed for operating systems, tha t the intellectual core contains the concepts needed for todayscomputer systems, and that operating systems are essential.


5/45

July 10,1984 2

1.1. Definitions ofOperating SystemBefore looking into these questions, we need to agree on a definition of operating

system. The oldest definition, which says an operating system is a control program jorallocating resources among competing tasks describes only a small portion of a modernoperating systems responsibilities. This definition is inadequate.

Among the great problems faced by operating systems designers is managing thecomplexity of operations at many levels of detail, ranging from hardware operations th at takeone billionth of a second to software operations th at take tens of seconds. An early strategemwa s information-hiding -- confining the detail s of managing a class of objects within amodule that has a good interface with its users. With information-hiding, the designers canprotect themselves from extensive reprogramming if the hardware or some part of the softwarechanges: the change affects only the small portion of the software interfacing directly with tha tsystem component. This principle has been extended from isolated subsystems to an entireoperating system. The basic idea is to create a hierarchy of levels of abstraction, so that a tany level one can ignore the details of what is going on at all lower levels. At the highest levelis the user of the system, who ideally is insulated from everything except what he aims toaccomplish. As a consequence of these developments, a better definition today is, an operatingsystem is a set of software eztensions ofprimitive hardware, culminating in a virtual machinethat 8erves a8 a high level programming environment.

** =

Operating systems of this type can support diverse environments: programming, t extprocessing, real-time processing, office automation, database, and hobbyist.


6/45

July 10,1984 3

1.2. C*wz-ezt.Operatirag System s -Most operating systems for large mainframes are direct descendants of third generation

systems - .g., Honeywell Multics, IBM MVS and VM/370, and CDC Scope. These systemsintroduced important concepts such as timesharing, multiprogramming, virtual memory,sequential processes cooperating via semaphores, hierarchical file systems, and deviceindependent I/O [Dennf1,Denn761.

During the 196Os, there were many projects to construct timesharing systems and test themany new operating systems concepts. These included MITs Compatible Time SharingSystem (CTSS), the University of Manchester Atlas, the University of Cambridge MultipleAccess System (CMAS),BM TSS/360, and RCA Spectra/70. The most ambitious project ofal l was Multics (Multiplexed Information and Computing Service) for the General Electrical645 (later renamed Honeywell 6180) processor [Orga72]. Multics simultaneoualy tested newconcepts of processes, interprocess communication, segmented virtual memory, pagereplacement, linking new segments to a computation on demand, autom atic multiprogrammedload control, access control and protection, hierarchical file system, device independence, 1 / 0redirection, and a high-level language shell.

Another important concept of third generation systems was he virtual machine, asimulated copy of the host. Virtual machines were first tested around 1966 on the M44/44Xproject at the IBM T. J. Watson Research Center. In the early 19709 virtual machines wereused in IBMs CP-67 system, a time sharing system tha t assigned each users process to its ownvirtual copy of the IBM 360/67 machine. Thi s system has been moved to the IBM 370machine and is now called VM/370 [Gold74, IBM731. Beeause each virtual machine can run adifferent copy of the operating system, VM/370 is effective for developing new operatingsystems within the current operating system. But because virtual machines are well isolated,


7/45

4uly 10, 1984

communication among them is expensive and awkward.Perhaps the most influential current operating system is UNIX, a complete reengineering

of Multics for the DE C PDP family of computers. It is an order of magnitude smaller thanMultics. I t retains the most useful concepts of Multics -- processes, hierarchical file system,device independence, 1 / 0 redirection, and a high-level language shell. I t dispensed with virtualmemory and t he detailed protection system; it introduced the pipe. I t offered a large library ofutility programs that were well integrated with the command language. Most of UNIX iswritten in a high-level language, C, which has allowed it to be transported to a wide variety ofprocessors, from mainframes to personal computers (Ritc74, Kern841.

-

In systems consisting of multiple UNIX machines connected by a high-speed localnetwork, i t is desirable to hide the locations of files, users, and devices from those who do notwish to dea l with those details. LOCUS is a distributed version of UNIX that accomplishes p .this by means of a directory hierarchy t ha t spans the entire network [PopeBl].

In recent years a large family of opera ting systems has been developed for personalcomputers . These include MS-DOS, C-DOS, APPLEDOS, CP/M, Coherent, and Xenix.These are all simple systems with limited function, designed for 8- and 16-bit microprocessorchips with small memories. In many respects, the development of personal computers isrepeating the history of mainframes in t he early 1960s - or example, multiprocess operatingsystems for microcomputers have appeared only recently in the forms of pared-down UNIX-likesystems such as Coherent and Xenix. Because only the large firms can sell enough machines tomake their own opera ting systems viable, there is strong pressure for standard operatingsystems. The emerging standards are PC-DOS, CP/M, an d UNIX.

Research on operating systems continues. There are numerous experimental systemsexploring new concepts of system structu re and distributed computation. The operating


8/45

July 10,1984 5

system for the Cambridge Z A P machine expioiia ihe hardwares microcode siippori fo rcapability addressing to implement a large number of processes in separately protecteddomains. Data abstraction is easy to implement on this machine (Wilk791.

StarOS is an operating system for the Cm* machine. Its central purpose is the supportofthe t a sk force, a group of concurrent processes cooperating in a computation. StarOS alsouses capabilities to control access to objects [Jone79]. Another operating system for the Cm*machine is Medusa. It is composed of several utilities, each implementing a particularabstraction such as a file system. Each utility can-include several parallel processes running onseparate processors. There is no central control [OustBO].

Grapevine is a distributed database and message delivery system used widely within theXerox Corporation. The network contains special nameservers that can find the locations ofusers, groups, and other services when given their symbolic names. There is no central controland it can survive the failures of the nameserver machines [Birr82]. Because it does no tprovide all the services of a high-level programming environment, Grapevine is not a true

operating system.

-

The V kernel is an experimental system aiming for efficient, uniform interfaces betweensystem components. A complete copy of the kernel runs on each machine of the network andhides the locations of files, devices, and users. V is a descendent of THOTH, an earlier systemworked on by the author of V [Cher84, Cher821.

The Provably Secure Operating System (PSOS) s a level-structured system whose high-level code has been proved correct in the context of a rigorous hierarchical design methodologydeveloped at SRI nternational [Neum80]. Although it was intended for secure computing,PSOS explored many principles that can help any operating system toward the goal ofprovable correctness.


9/45

July 10,1984 6

These examples demonstrate that the new technology has created new control problemsfor operating systems designers to solve. The need for operating systems is stronger than ever.

2. MODEL OF AN OPERATING SYSTEMThe hierarchical structure of a model operating system separates its functions according

to their characteristic time scales and their levels of abstraction. Table 1 shows anorganization spanning fifteen levels. It is not a model of any particular operating system butrather incorporates ideas from several systems. I t includes facilities for distributed processing.

Each level is the manager of a set of objects, which can be hardware or software andwhose na ture varies greatly from level to level. Each level also defines operations that can becarried out on those objects. The levels obey two general rules:

1. Hierarchy. Each level adds new operations to the machine and hides selected-operationsat lower levels. The set of operations visible at a given level form the instruction set of anabs tra ct machine. Hence a program written at a given level can invoke visible operationsof lower levels but no operations of higher levels.

2. Inform ation Hiding. The details of how an object of given type is represented or where itis stored are hidden within the level responsible for that type. Hence no pa rt of an objectcan be changed except by applying an authorized operation t o it.

The principle of data abstraction embodied in the levels model traces back to Dennis andVan Horns 1966 paper, which emphasized a simple interface between users and the kernel[Denn66]. The first instance of a working operating system whose kernel spanned Pveral levelsw a s reported by Dijkstra in 1968 [Dijk68]. Th e idea has been extended to generate families ofoperating systems for related machines (Habe761 and to increase the portability of an operating


10/45

July 10,1964 7

TABLE 1: An Operating System Design Hierarchy.~-.eve1 Name Objects Example Operations

Shell User programming environmentsca lu data, array data

statements in shell langaage15

14

13

12

11

10

9

Dircetoricr Directoh create, destroy, atta ch,detach, eenrch, lirt

U r r Proccua Uaer pmceu fork, quit, ki l l , suspend,resumeopen, close, read, writetream I/ O

Devica

F i i Sp tem

StrerunsExternal devicer andpcripheralr such Mprinter, dupky, keyboudFila

create, destroy, open, c k ,read, write

create, destroy, open, dac,r e d , write . -

Communications Pi p s create, datroy, open, c b ,re d , wri te

CapabiitiaVirtualMemoqLocal Secondary Store

Primitive Procaw

In tempt .Proeedura

Instruction Set

Electronic Circuits

CapAbiitia create, validate, attenuat eread, write, fetchr e d , write, allocate, free

SegmentsBlocb of data,device channelr

suspend, mum e,wait, signal

Primitive proceu,aemaphom, ready lirt

invoke, mark, unmark, retsault handler p r o ~ m rmark-stack, call, returnrocedure r p m t r ,

Call stack, dhpkyEvaluation stack,microprogram interpreter,

load, store, un-op, bin-opbraach, array-ref, etc.

Registen, g a b ,b u m k .

clear, trander, compkmemlactivate, etc.


11/45

July 10,1984 8

system kernel (Cher821.. The Provably Secure Operating System (PSOS) s the first completelevel-structured system reported and formally proved correct in the open literature [Neum80].

We now turn to a brief summary of each level in Table 1. Greater detail follows in latersections.

2.1. The Single-Machine Levels: 1-8Levels 1-8 are called s i n g le - m a c h i n e l e vel s because their operations ar e well understood

from primitive machines and require little modification for advanced operating systems.The lowest levels include the hardware of the system. Level 1 is the electronic circuitry,

where the objects are registers, gates, memory cells, and the like, and the operations areclearing registers, reading memory cells, and th e like. Level 2 adds the processors instructionset, which can deal with somewhat more abstract en tities such as an evaluation stack and a narray of memory locations. Level 3 adds the concept of a procedure and the operations of call

-.y*-

and return Level 4 introduces interrupts and a mechanism for invoking special procedures whenthe processor receives an interrupt signal.

The first four levels correspond roughly to the basic machine as it is received from t hemanufacturer, although there are some interactions with the operating system. For example,interrupts are. generated by hardware bu t the interrupt-handler routines are part of th eoperating system.

Level 5 adds primitive processes, which a re simply single programs in t he course ofexecution. The information required to specify a primitive process is its stateword, a datastructure that can hold the values of he registers in the processor. This level provides acontext switch operation, which transfers the at tention of of a processor from one process toanother by saving the stateword of he the first and loading the stateword of the second. This


12/45

July 10,1984 9

level contains a scheduier tha t seiects, from a reudy :i& of avai:ab!e pioe-, the zextprocess to run after the current process is switched off the processor. This level also providessemaphores, the special variables used to cause one process to stop and wait until anotherprocess has signalled the completion of a task. This level has a simple hardwareimplementation (Denn81J. Primitive processes are analogous to system processes in PSOS andlightweight procesecs in LOCUS.

Level 6 handles access to the secondary-storage devices of a particular machine. Theprograms at, this level are responsible for operations such as positioning the head of a diskdrive and transferring a block of data. Software at a higher level determines the address of thedat a on the disk and places a request for it in the devices queue of pending work; therequesting process then waits at a sempahore until the transfer has been completed.

Level 7 is a stan dard virtual memory, a.scheme,that gives the programmer the illusion of w e - -having a main memory space large enough to hold the program and all it s data even if th eavailable main memory is much smaller [Denn70]. Software at this levei handles the inter ruptsgenerated by the hardware when a block of data is addressed when it is n o t present in the mainmemory; this software locates the missing block in t he secondary store, frees space for it in themain store, and requests Level 6 to read in the missing block.

Level 8 implements capabilitiee, which are unique internal addresses for software objectsdefinable at higher levels. This level allows capabilities to be read, bu t not altered. This levelprovides a validate operation that enables the programmer of higher-level procedures to verifyth at actual parameters are of the expected types.

Up through level 8 , the operating system deals exclusively with the resources of a singlemachine. Beginning with the next level, the operating system encompasses a larger worldincluding peripheral devices such as erminals and printers and also other computers attached


13/45

July 10,1984 10

to the network. In this world, pipes, files, devices, user processes, and directories can be sharedamong all the machines.

2.2. The Multi-Machine Levels: 9.14Every object in the system has two names: its external name, a string of characters

having some meaning to users, and its internal name, a binary code used by the system tolocate the object. The mapping from external to internal names is controlled by th e userthrough directories. The mapping from internal names to physical locations is controlled bythe operating system, giving it the ability to move objects among several machines withoutaffecting any users ability t o use those objects. This principle, called delayed bind ing, wasimportant in third generation operating and is even more important today [Dennll].

To hide the locations of all sharable objects, both external and internal names must be4.Lglobal, Le., they can be interpreted on any machine. Unique external names can be constructed

as pathnames in the directory hierarchy (defined at Level 14). Unique internal names areprovided b y capabilities (Level 8). If the local network communication system (Level 9) isefficient, software at the higher levels can obtain access to a remote object with l ittle penalty.

Level 9 is explicitly concerned with communication between processes, which can bearranged through a single mechanism called a pipe. A pipe is a one-way channel: a stream ofda ta flows into one end an d out of the other. A request t o read items is delayed until they a reactually present in the pipe. A pipe can equally well connect two processes on the same

machine or on different machines. A set of pipes linking levels in al l the machines can serve asa broadcast facility, which is useful for finding resources that might be anywhere in thenetwork [Bogg83]. Pipes are implemented in UNIX [Ritc74] and have been copied in recentsystems such as iMAX [Orga83] or XINU [Come84].


14/45

July 10, 1984 11---_Levei 10 provides for iong-term storage of named fiies. Whereas Levei 6 deais with disk

storage in terms of tracks and sectors - he physical units of the hardware - Level 10 dealswith more abstract ent ities of variable length. Indeed a file may be scattered over manynoncontiguous tracks and sectors. To be examined or updated, a files contents must be copiedbetween virtual memory and the secondary storage system. If a file is kept on a differentmachine, Level-10 software can create a pipe to Level 10 on the files home machine.

Level 11provides access to external input and output devices such as printers, plotters,and the keyboards and display screens of terminals. There is a standard interface with allthese devices and again a pipe can be used to gain access to a device attached to anothermachine.

Level 12 provides a means by which user processea can be attached interchangeably t o .:.pipes, files, or devices for input and output. The idea is to make each of the fundamental - . -operations of Levels 9,10, and 11 (OPEN, LOSE, READ, and WRITE) look the same sothat the author of a program need not be concerned with the differences among these objects.

This is achieved in two steps. First, t he information contained in pipes, files, and devicesisregarded simply as streams of bytes; requests for reading or writing move segments of databetween streams and a user process. Second, a user process is programmed to request all inputand outp ut via ports, which are attached by the open operation a t run time to specific pipes,files, or devices.

Level 13 implements user processes, which are virtual machines executing programs. It isimportant to distinguish the user process from the primitive process of Level 5. All theinformation required to define a primitive process can be expressed in t he stateword thatrecords the contents of the registers in the processor. A user process includes not only aprimitive process, but also a virtual memory containing the program and i ts work space, a list


15/45

July 10,1984 12

of arguments supplied as parameters when the process was star ted, a list of objects with whichthe process can communicate, and certain other information about the context in which theprocess operates. A user process is much more powerful than a primitive process.

Level 14 manages a hierarchy of directories tha t catalogue the hardware and softwareobjects to which access must be controlled throughout the network: pipes, files, devices, userprocesses, and the directories themselves. The central concept of a directory is a table thatmatches external names ofobjects with capabilities containing their internal names. Ahierarchy arises because a directory can include among its entries the names of subordinatedirectories. Level 14 ensures that the subhierarchies encached a t each machine are consistentwith one another.

The directory level is responsible only for recording the associations between the externalnames and capabilities; other levels manage the objects themselves. Thus when a directory ofdevices is searched for the str ing laser, the result returned is merely a capability for the laserprinter. The capability must be passed to a program a t Level 11, which handles the actualtransmission to that printer.

Level 15 is the shell, so called because it is the level that separates the user from therest of the operating system. The shell is the in terpreter of a high level command languagethrough which th e user gives instructions to t he system. The shell incorporates a listenerprogram that responds to a terminals keyboard; i t parses each line of input to identifyprogram names and parameters; it creates and invokes a user process for each program andconnects it as needed to pipes, files, and devices.


16/45

July 10,1984 1s

2.3. General Comments on Levei StructureThe level struc ture is a hierarchy of functional specifications. Its purpose is to impose a

high degree of modularity and enable incremental verification, insta llation, and testing of thesoftware.

In a functional hierarchy, a program at a given level may directly call any visibleoperation of a lower level. No nformation flows through any intermediate level. The levelstructure can be completely enforced by a compiler, which can insert procedure calls or expandfunctions in-line [Habe76]. A recent example of its use is in XINU, an operating system for adistributed system based on LSI 11/02 machines (Come841.

It is important to distinguish the level structure discussed here from the layer structure ofthe I S 0 (International Standards Organization) model of long-haul network protocols(Tane8l j. In the I S 0 model, information is passed down through all the layers on the sendingmachine and back up through all the layers on the receiving machine. Each layer addsoverhead to a da ta transmission, whether or not tha t overhead is required. Models for long-haul network protocol structure may not be efficient in a local network fPope811.

.*

A significant advantage of a functional levels over information-transferring layers isefficiency: a program that does not use a given function will experience no overhead from thatfunctions presence in the system. For example, procedure calls will validate capabilities onlywhen they are expected. Common objects (such as pipes, files, devices, directories, and userprocesses) are implemented. by their own levels rather than as new types within a general

. type-extension scheme [Wulf81].Each level should be able to locate local objects by their internal names without having to

rely on a central mapping mechanism. This is not only a step toward reliability in adistributed network, but also efficiency because central mechanisms are prone to be


17/45

July 10,1984 14

bottlenecks.Operating system designers ought to take reasonable steps to verify th at each level of the

operating system meets its specifications. This too serves efficiency as well as reliability: run-time checks need be included in system procedures only for conditions that cannot easily beverified a priori. Thus, system procedure calls must check at run time that expectedcapabilities are present as parameters because the calling programs may be unverified; butother aspects of parameter type checking can be performed by a compiler.

In the.following sections we will give more detail about the mechanisms from thecapability level [Level 8) upwards.

3. DISTRIBUTED CAPABILITIES (Level 8 )The external names of sharable objects are character strings of arbitrary length having V

meaning to human users. Because these strings are difficult to manipulate efficiently, theoperating system provides internal names for quick access to objects. One purpose of Level 8 isto provide a standard way of representing and interpreting internal names for objects.

To prevent a process from applying invalid operations to an object whose internal name itknows, th e operating system can at tach a type code and an access code to a n internal name.The combination of codes (type, access, internal-name) is called a capability. All processes areprevented from altering capabilities. The system assumes th at the very fact tha t a processholds a capability for an object is proof of its authorization to use that object; processes areresponsible for controlling the capabilities they hold.

The simplest way to protect capabilities from being altered is to tag the memory wordscontaining them with a special bit and to permit only one instruction, the create-capabilityinstruction, to set that bit [Fabr74, Wilk79). The IBM System 38 is a recent example of an


18/45

July 10,1984 15

efficient system using tags to disiing&& c @ d % c s f:=m ether & j ~ t s r? memory IT.evv841.-- , -,Capabilities were first proposed as an efficient method of mplementing an object-orientedoperat ing system [Denn66]. Th is has continued to be the main reason for using themIWulf81, Levy84, Orga831.

All existing implementations of capabilities are based on a central mechanism formapping th e internal name t o an object. These mappings are direct extensions of v irtualmemory addressing schemes [Denn76, Fabr741. Unfortunately, a central mapping schemecann ot be used with a distributed system whose component machines may fail. So theresponsibility for mapping must be distributed by allowing each the procedures of Levels 9-14on each machine to read and interpret locally th e fields within validated parametercapabilities.

The storage structure and mapping scheme for capabilities is illustrated in Figure 1. The .name field of a capability of type T onsists of a code Mfor the machine on which thecapability was created and a n index number I. The machine number is needed because someCapabilities (those for open pipes, files, and devices) can only be used on the issuing machine.The access code specifies which of the 2'-type operat ions can be applied to th e object. Th eindex number I is used b y the level in charge of T-type objects on machine M o address adescriptor block for the given object. The descriptor block records control information aboutan object, times and da tes of creation and last update, a nd current size and attributes of theobject. The location of the descriptor block denotes the location of the object - moving thedescriptor block from one machine to another effectively moves the object.

The procedures implementing operations at Levels 9-15 must conform to certainstandar ds tha t ensure the proper use of capabilities. One is an agreement on the codes for theobject types; eight are listed in Table 2. We will use the notation T-cap to denote a


19/45

July 10,1984 16

name =(machine#,index)3rocess Type Access Name

Memory Segment' Level

-BlockStorageLevel

FIGURE 1. The storage structure for representing object consists of chain starting with acapability, through a local map under the control of the object's level, through a descriptarblock, to the object itself. A change in location of he object requires no change in anycapability. The index numbers re generated locally by the level when it c reates objects. Inthis example, the process holds two capabilitiesof ype T; ne of the objects is in a segment invirtual memory and the othe r is in a block of econdary storage.


20/45

July 10, 1984 17

capability of type T where T i s one of the abbreviations in the table (e.g. fiie-capj7The remaining s tandards concern the creation of capabilities pointing to new objects and

the application of specific operations to those objects. Suppose Level L (L>8) is the managerof T-type objects. This level contains a procedure to create new objects of type T and one ormore procedures to apply given functions to objects of t y p e T. The create operation must usea call of the form

T-cap := CREATE-T(initia1-value)

This procedure performs all the steps required to set up the storage for a new object of ype T:it obtains space in secondary storage for the object and stores in it the given initial value, i tsets u p a descriptor block, it finds an unused index and sets up the entry in the local map, andfinally it creates a capability of type T denoted T a p ) .

TABLE : Capability Type Marh and their Abbreviations.Level Type mark Abbreviation

14 d m t o y dir15 u.cr proccu UP11 device dev

open device 0P-d-

open f& op-file

o pen p ip e o p g i p e

10 file fde

1 9 PiPC PiPC


21/45

July 10,1984 18

Creating a capability is a critical operat ion. Level 8 implements a special operation forthis purpose:

T-cap := CREATE-CAP(I)

where l i s the local index number chosen by Level L. When used inside the CREATE-Toperation on machine M, CREATE-CAP constructs a capability (T, , M, ), sets to 1 thecapability bit of the memory word containing it, and returns the result. The code for Mcomesfrom a register in the processor. The code for A is the one denoting maximum access. Thecode for T omes from a field in the program sta tus word (PSW), a processor register that alsocontains the program counter of he current procedure. The compilers must be set up t ogenerate type(PSW)= T only for the CREATE-T procedure and type(PS W)=null for all otherprocedures. CREATE CAP fails if executed when type(PSW)=null . -_*-

The procedures for applying operations to a given object have the generic form

APPLY-OP(T-cap, parameters)

which means that OP(parameters) must be applied to the object denoted by T-cap. Thecompiler can validate that the first actual parameter on any call to A P P L Y O P is indeed oftype T by using another operation of Level 8, called VALIDATE, which checks that thisparameter is a capability whose type code is T nd whose access code enables operation OP.VALIDATE can also be used to verify the presence of other capabilities among the otherparameters.

A procedure may reduce the access rights of a capability it passes to another procedure byusing the Level-8 operation ATTENUATE.


22/45

July 10, 1984

Table 3 summarizes the operations impiemented at Levei 8.

19

4. COMMUNICATIONS (Level 9)The communications level provides a single mechanism, the pipe, for moving information

from a writer process to a reader process on the same or different machines. The mostimportant property is that the reader must stop and wait until t he writer has put enough da tainto the pipe to fill the request. Level 9 gives the higher levels the ability to move objectsamong th e nodes of the network.

The external interface presented by the communications level consists of the commands inTable 4. When two communicating processes are on the same machine, a pipe between themcan be stored in shared memory and th e READ-PIPE and WRITE-PIPE operations areimplemented the same as the send and t e cc i vc operations for message queues [Brin73]. _ a +

TABLE : Specification of Capabdity Operatiom (Level 8) .iI/ Form of call EffectI

T-cap := CREATE-CAP(I) If the typcmark in the current PSW u non-null, reate anew capability with t y p e fnld set to that mark, ace- codemaximum, machine field the local machin e identifHr, andindex I.V e m he capnhility at the cdkro virtud addreoo p. For atleast one i=l,..., th e following murt be true: the capp.bfitycont.ino Tiin ita type ield and permito u c e n .i. f Tidenoteo op g ip e . op-fde, or op-dev, the machine fieldmurt match the identifier of th e local machine. (Fail.ft h e w conditionr are not met.)

VALIDATE(p, I, T l , d ) ,...,(Ta,aa))

ca p := ATTENUATE(cap, mark) Returnr a copy of the given capability with the sec eu fieldreplaced by the bitwioeAND of mark and the a c c e u fieldfrom cap.


23/45

July 10,1984

Form of call Effect

20

pipe-cap := CREATE-PIPE( )

TABLE : Specification of Communication Level Interface (Level 9).

DESTROY -PIPE( pipe-cap),pgipe-cap := OPEN-PIPE(pipe-cap, m)

READ-PIPEWRITE-PIPECLOSE-PIPEopgipe-cap := BROADCAST(mrg)

Creates a new empty pipe and returns a capabili ty for it. (Ifthe caller is a urer process, it can s to re this capability in adirectory entry and make th e pipe available throughout thesptem.)Destroy the given pipe (undo a create pipe operation).Opens the pipe named by the pipe capability by allocatingitorage and retting up a dewr ipto r block. Initially, the pipeir empty. If rw=writc, th e open-pipe capability hM its writepermimion set and can be ured only by the procerr at theinput end of the pipe. If rw =r ed , th e open-pipe capabilityh a ts read permirrion s et and can be ured only by theprocesr at the output end of the pipe. Docr not return untilboth reader and writer have requested connections. (Fail# ifth e pipe ir already open for writing when rw=wrilc orreading when rw=red .) If both render and receiver er e onth e same machine, the open-pipe descriptor block willindicate tha t shared memory c a n be ured for the pipe;otherwise a network protocol must be ured.There have the same effects M t h e READ,WRITE, andCLOSE operationr described in the rection on stream I/O.Broadcast a message to all t y p e managen in th e networkthat manage objectr of th e same type M t h e c d h g ocaltype manager. Retu rnr an open pipe capability for readingresponses.

When the two processea are on different machines, the communications level mustimplement the network protocols required to move information reliably between machines.(SeeFigure 2.) These protocols are much simpler than long-haul protocols because congestionand routing control are not needed, packets cannot be received out of order, fewer error typesare possible, and errors are less common [PopeBl].

The read and write operations become ambiguous unless both a reader process and awriter process are connected to a pipe. Should a writer be blocked from entering informationuntil the reader opens its end? Wha t happens if either the reader or writer breaks its


24/45

July 10,1984

~

21

I

III

IIIII

I Machine BI Process 2

Machine A

Process 1

ICommunications \ I II&Level

StreamendingBuffer

T;>rocessI PacketsIIIIIIIIIIIIIIIIIIIIIIII

StreameceivingBuff-

ReceiveProcess

IPackets

FIGURE . A ne t w a k protocol muat be need when tw o procewee connectedby a pipe are ondifferent machinee. The WR.I!IE nqacrtsof he sender append segment. to a stream awaitingtrammission. The ender proceee t r d t s he stream ae a eequence of packets, which areconverted back intoa atream and placed in the receiving buffer. Each READ m e a t of hereceiver waits until the requested amount ofdata is in the buffer then return8it.

connection? Questions like these are dealt with by a c o n n e c t i o n p r o t o co l . A simple connectionprotocol is called rendezvous on open and close:

1. The open-for-reading and the open-for-writing requestsmay be called at different timesbut both returns are simultaneous.


25/45

July 10, 1984 22

2. The close operation, executed by the reader, shuts both ends of the pipe; executed by thewriter is deferred until the reader empties the pipe.

A pipe capability can be stored in a file or a message and passed to another machine overan existing open pipe or by broadcast. A pipe capability can also be listed in a directory (to bediscussed later), making the pipe a global object. (In this case it is like a FIFO file inSystem-5 U N IX [&1183]. )

The communications level also contains a broadcast operation to permit Levels 10,11,an d 12 to request mapping information from their counterparts on other machines. Forexample, if th e file level on one machine cannot locally open the file named by a given.capability, i t can broadcast that capability to the file levels of o ther machines; the machineactually holding the file responds with enough information to allow the broadcaster t ocomplete its pending open operation.

5. FILES (Level 10)Level 10 implements a long-term store for files. A file is a named string of bits of known,

but arbit rary length and is potentially accessible from all machines in the network. Theoperations for files are summarized in Table 5.

To establish a connection with a file, a process must present a file capability to th eOPEN-FILE operation, which will find the file in secondary storage and allocate buffers fortransmissions between the file and th e caller. The transmissions themselves are requested byREAD-FILE and WRITE-FILE operations. Each read operation copies a segment ofinformation from the file to the callers virtual memory and advances a read pointer by thelength of the segment. Each write operation appends a segment from the callers virtual


26/45

July 10,1984 23

TABLE : Speciflcation of Files Level Interface (Level 10).Form of call Effectfile-cap := CREATE-FIL.E( )

DES TROY-FILE(file-cap)op-file-cap := OPEN-FILE(fik-cap, m)

READ-FILEWRITE-FILECLOSE-FILERE WIND(op-file-cap)ERASE( op-file-cap)

Createa a ne w empty file an d returns a capabil ity for it. (Ifthe caller u a uwr p r ~ ~ c u ,t can store this capability in adirectory entry and make the fde availabk throughout thesyrtem.)Destroy the given file (undo a createfile operation).Opens the fie named by the file capability by allocatingstorage for buffen and setting up a dacr ipt or block. Thevalue of R (read, write, or both) u put in the acccu fnld oft h e open-filecapability. The ead pointer b wt to e r o andthe write pointer to t h e files length. (Failm if the fh balready open.)These have the same effects u he READ, WRITE, andCLOSE operations dacr ibed in th e eection on stream I/O.

R e u t red pointer t o zero.Se t fik kngth u d rite pointer to zero;nkuc secondarystorage blockoccupied by the fde.

memory to the end of the file.In a multi-machine system, the file level must deal with the problem of nonlocal files.

What happens when a process on one machine requests to open a file stored on anothermachine? There are t w o alternatives:

1. Open a pair of pipes to Level 10 on the files home machine; read and write requests arerelayed via the forward pipe for remote exeution; results are passed back over the reversepipe. (This is called remote open.)

2. Move the file from its current machine to the machine on which the file is being opened;thereafter all read and write operations are local. (This is called file migration.)


27/45

July 10,1984 24

Both methods are feasible. An instance of remote open is in the Berkeley COCANET system(Rowe821. An instance of migration is in the Purdue STORK file system (Pari831.

The open connection descriptor block for a file, which is addressed by an open-filecapability, indicates whether read and write operations can be performed locally or mustinteract with a surrogate process on another machine. In the latt er case, the required open-pipe capability will be implanted in the descriptor block b y the open-file command.

Figure 3 illustrates the types of capabilities generated and used during a typical file-editing session.

One important improvement t o the basic file system is to allow multiple readers andwriters by building in to the read and write operations a solution to the readers and writerssynchronization problem IHolt78). Another is to use a version control system to automaticallyretain different revisions of a file; the file system can then provide access to the older versionswhen needed [Tich82].

8. DEVICES (Level 11)The devices level implements a common interface to a wide range of external input and

output devices -- for example, terminal displays and keyboards, printers, plotters, time-of-dayclock, and optical readers. The interface at tempts to hide differences among devices by makinginput devices appear as sources of da ta streams and output devices appear as sinks. Obviously,the differences cannot be completely hidden -- for example, cursor-positioning commands mustbe embedded in the dat a stream sent to a graphics display -- but a surprising amount ofuniformity can be achieved.

Corresponding to each device is a device driver program that translates commands at th einterface to instructions for operating t ha t device. A considerable amount of effort may be


28/45

July 10,1984 25

UsersVirtualMemory Program

Buffer

DescriptorBlock

5

FIGURE . The steps of an editing session generate and use various capabilities. 1) CanVertthe external name &ring to a capability, e l ; thiscanbe done by a dbectary-aeeueh e-d.2) Open he file faa reading and wr i t ing by the command ed :=OPEN FILE e l , RW). 3)Copy the fik into a buffer by the command READ-FILE(ed, B, elf). 4)Edit the content. fthe buffer. 5) Replace the older redm of the file by the pair ofcmnman& EUSE(c2);WRITE FILE(&, B, L),where L is the length ofthe buffer. 6) Cloee the fUe by thecommurd CLOSEFILE( d).


29/45

July 10,1984 26

required to constructla reliable, robust device driver. When a new device is attached to+,hesystem, it s physical address is stored in a special file accessible to the device drivers.

Table 6 summarizes the interface for external devices.

7 . STREAM 1 / 0 (Level 12)An important principle adopted in the hypothetical operating system we are describing

here is input-output independence. A t Levels 9, 10, and 11 the same fundamental operations(namely OPEN, CLOSE, READ, and WRITE) are defined for pipes, files, and devices.Although writing a block of data to a disk calls for a sequence of events quite different fromth at needed to supply the same da ta to the laser printer or to the input of another program,

..

the au thor of a program does not need to be concerned with those differences. All read and

TABLE : Specification of Devices Level Interface (Level 11).Form of call Effectdev-cap := CREATE-DEV(type, addrera) Returnr a capability for a device of the given type at thegiven d d r e u . The accerr code of the returned capabilitywill not include W if t he device is read-only or R f t h edevice u write-only.DESTROY -DEV(dev-cap) Detach the given device from the system (undo a create-device operation) .op-dev-cap := OPENDEV(dev-cap, rw) Opens the device named by t he device capability byallocating storage for buffen and wttiag up a deicriptorblock. The value in the access field of the open-devicecapability is the logical AND of rw and the accem code ofthe device capability. (Fail6 if the device is already open.)READ-DEVWRITE-DEVCLOSE-DEV

There have the same effects M the READ, WRITE, andCLOSE operationr dercribed in the clection on stream I/O.

..


30/45

July 10,1984 27

write. statements in .a program can refer to input and -output ports, which are attached toparticular files, pipes, or devices only when the program is executed.

This strategy, which is an instance of delayed binding, can greatly increase the versatilityof a program. A library program (such as the pattern-finding grep program in UNIX) cantake its input from a file or directly from a erminal and can send i ts out put to another file, toa terminal, or to a printer. Without delayed binding, each program would have t o be writtento handle each possible combination of source and destination.

A common model of data must be used for pipes, files, and devices. The simplestpossibility is the stream model in which these objects are media for holding streams of bits.Corresponding to each of these objects is a pair of pointers, R or reading and W or writing; Rcounts the number of bytes read thus far and similarly for W. Each read request begins atposition R and advances R by-th e number of bytes read. Similarly, each write-request beginsat position W and advances W by the number of bytes written.

*T

The blocks of dat a moved by read or write requests a re called segments; seg(z,n) denotes acontiguous sequence of n bytes beginning at position z n a given dat a stream. The exactintcrpret.ationof read (o r write) request depends on whether the segment comes from a pipe,file, or device. For example, a read request can only be applied at the output end of a pipe an dthe reader is required to wait until the writer has supplied enough data to fill the request. Anoutput-only device, such as a laser printer, cannot be read and an input-only device, such as aterminal keyboard, cannot be written.

The OPEN operation of Level 12 returns an op-T-cap, corresponding to a given T-capfor T, a pipe, file, or device. The op-T-cap represents an active connection through whichdata may be passed efficiently to and from the object. The READ nd WRITE operationsrequest segment moves across such a connection. The CLOSE operation breaks the connection.

- -


31/45

July 10,1984 28

Because the-st ream model has already been incorporated into the pipes, files, and deviceslevels, the only new mechanism is a w ay of switching from a Level-12 operation to itscounterpart in the level for the type of object connected to a port. For example,OPEN( -cap, rw) means

CASE T OFpipe: RETURN OPEN-PIPE(T-cap, rw);file: RETURN OPEN-FILE(T-cap, rw);dev: RETURN OPEN -DEV(Tcap , rw);ELSE: error;END CASE

Table 7 summarizes the interpretat ions of the four operations of OPEN, CLOSE, READ,and WRI TE for the three kinds of input-output object.

The stream model is not used in every operating system. For example, in Multics,segments are explicit components of the virtual memory; there-is no need f o r a separateconcept of file because segments are retained indefinitely until deleted by their owners[Orga72]. In Multics the four operations of Table 7 are implicit. The first time a process refersto a segment, a missing-binding interrupt causes the operating system to load and bind t ha tsegment to the process. The process can thereaf ter read or write the segment using theordinary virtual-addressing mechanism. Certain segments of th e address space arepermanently bound to devices; reading or writing those segments is equivalent to reading orwriting the device. There is no concept of pipe, but the interprocess communicationmechanism allows a data stream to be transmitted from one process to another.

*.

8. USER PROCESSES (Level 13)A user process is a virtual machine containing a program in execution. It consists of a

primitive process, a virtual memory, a list of arguments passed as parameters, a list of ports,


32/45

July 10,1984

TABLE : Semantic. of 1/0 Operations on Objects (Level 12)Pipe

Verlry that T_cq refento M unopened pipe. UrOPENJWE(T_cap, R)to init-* an open-pipedner ipto r bloek in whichR = W =0; return theo p s 4 u w .

InvokeWRIT~PPE(OPS~P~_E~P,A, a) to copy MI a ,a ) tong (W ,a) and advanceW to R + a . (MAYawaken waiting d e r ) .

If t he pipe contains athat ruder the r e d i n g= p e n t in the pipe.InvokeCLOSl$-pIPE(opgipem)to dulkca te the open-pipe descriptor b b e t

waiting reader, return to

FileVerify that T_cap refersto an unopened tile. UwOPEN-lE(T_cap. cw)to init- an open-filedeuriptor b k k n whichR = O a n d W =L (fdelength); return theopJde-cap.

Set m =min[ L -R , 1InvokeR E A D _ F I L E ( o ~ i a p ,a, m) o COPY w (R m 1to reg(a,m)- If n = d ,return immediitely withwhatever u n the file,reg ( R .W R ).

InvokeWRITE_FILE(op_fe_cap,A, n ) as for pipe, plusadvance L to L + a .

InvokeCLOSEJILE(opxile_cap)to deallocate the open-filedescriptor block.

DeviceVerify that T-cq mfersto M unopened device.UK OPENqEV(TAap,iw ) to initiaIixean open-device descriptor bbc k inwhich R =0 or W =0,s ~ ~ ~ r d i n gthe device kinput or output; returnthe op-dev_cap.

~

InvokeREAD-DEV(op>ey_cap,a, m) M for f&. If n = d ,return immedii withwhatever input mavaihhk, ref (R W -R ).(No effect for outpatdevice.)

InvokeWRITEJ)EV(o&cv>p,a, n ) M for pipe. (Noeffect for input dcvicc.)

InvokeCL OSE-DEV( o p _ d e y ~ )to deallocate the open-device descriptor block.

and context. Each port is a capability for an open pipe, file, or device. The context is aset of variables characterizing the environment in which the process operates; it includea thecurrent working directory, the command directory, a link to the paren t process, a l i k e d l i t ofspawned processes, and a signal variable that counts the number of spawned processes whoseexecution is yet incomplete. Figure 4 illustrates the format of a user process descriptor block.


33/45

July 10,1984 30-tateword PORTS0 I default input Ipointer to

virtual memoryCONTEXT

ARGSpirgument list CDW Dsignalparentchildren command dir.~working dir.FIGURE . A user process is a virtual machine created for the purpose of executing a givenprogram. It contains a primitive procese, a v i r tua l memory holding the given program, a listofarguments supplied at the time of c a l l , a list ofports, and a set ofcontext variables. Byconvention, PORTS[O] is the default input and PORTS[l] is the default output; these twoports are bound to pipes, files, or devices when the process ie created. The process c a n openother ports as well after it commences execution.

A new user process is created by a FORK operation. The creator is called the parentand the new process a child. A parent can exercise control over its children by resuming,suspending, or killing them. A parent can stop and wait for it s children t o complete their t a s k sby a join operation, and a child can signal its completion by an exit operation. Table 8summarizes.


34/45

July 10,1984 Sl

TABLE : Specificationof User Proceas Operations (Level 13)Form of Call Effectup-cap := FORK(fi1e-cap, pa ra m, in, ou t)

EXIT

SUSPEND( up-cap)

RESUME( u p-cap)

op-T-cap :=OPEN(T-cap.ru)

Allocate a user pro ceu dex rip tor block. Create a rurpendedprimitive proctu and rtore ita index in a ne w user-proccucapability. Create a vir tud memory an d load the executablefile denoted by fde-cap. Copy the pu nmet en into theARGS Imt. Verify that ir and .ut are capabilities for pipes,fila, or deviceo; if so,open ir for readiig and p ut the open-capability in PORTS(O1, and open d or writing and putthe open-capability in PORTS(l1.Wait until callerr context variable, r i d a A, then return.Terminate the designated umer proccu , bu t only f it ia achild of the c a r . T h i eat& destroying the primitiveprocar an d virtual memory, closiig open pipes, fb, rd e v i c a connected to ports, rekaring th e storage held by thedacr ipto r block, and removing the deleted proccu from theh t of the cderr children.Terminate the c d e r procar and add 1 o the rigad variableof the parent proccu.Put the primitive procar contained within the ur n proccuLup-cap into th e ruapended date, ba t only if the ulkr bthe parent of proccu up-cap.Pu t the primitive procw contained within the user proccuup-cap int o the ready state, but only if the c&r in theparent of procen up-cap.Invoke he OPEN command in Level 12, store a copy of theresult in the next avdnbk pmition in the PORT S table an dreturn the result to the caller (T ia pipe. file,or devicd.

Notice tha t an OPEN operation appears in Table 8. ThisOPEN operation hides theLevel-12 OPEN rom higher levels. It allows Level 13 to store copies of all open-objectcapabilities in the PORTS table. When a process terminates, Level 13 can assure that al l openobjects are clo-ked by invoking the Level-12 CLOSE operation for each entry in the PORTStable.

. . - . .. -


35/45

July 10,1984 32

9. DIRECTORIES (Level 14)Level 14 is responsible for managing a hierarchy of directories containing capabilities for

sharable objects. In our hypothetical system, these a re pipes, files, devices, directories, anduser processes; capabilities for open pipes, files, and devices are not sharab le and cannot appearin directories. A hierarchy arises because a directory can contain capabilities for subordinatedirectories.

A directory is a table tha t matches an external name, stored as a string of characters,with an access code and a capability. In a tree of directories, the conca tenated sequence ofexternal names from the root to a given object serves as a unique, system-wide external namefor tha t object. A directory system of his kind has been implemented on the Cambridge CAP

roote - .

e denning. a bic ynet-hosts brown

laser clock 0 nroff eqn d eI passwdterminal

FIGURE . A directory hierarchy can be depicted asan nverted tree whose topmost node iscalled root. Some directories are permanently reserved for specific purposes. For example,the d eu directory lists all the external devicesof he system. The l ib directory lists the libraryof all the executable programsmaintained by the systems administration. A U I C ~ irectorycontains a subdirectory for each anthorhed user; hat subdirectory is the root o fa subtreebelonging to that user. In UNIX, the unique external name of an object is formed byconcatenating the external names along the pa th &om the root, separated by slash (/) andomitting root. Thus the laser printers external name is /dev/laser.


36/45

July 10,1984 33

The principal operation of Level 14 is a search command tha t locates and returns thecapability corresponding to a given external name. Thus he directory level is merely amechanism for mapping external to internal names. Only one type of capability can bemapped to an object at this level: a directory capability. All other capabilities must bepresented to their respective levels for interpretation. Information about object attributes,such as ownership or time of last use, is not kept in directories; it is kept in the objectdescriptor blocks within the object manager levels.

I IItdevttFIGURE . A directory is a table matching an external name String with an acceas code anda capability. Every directory c o n t a i n e a capability pointing to ita immediate parent and acapability pointing to itsee the eel-capability ie can be nsed to till in the parent entry in anew subordinate d i rec t or y . & c a w directorieeare at a higher evel thanfiles, the file systemcan be wed toetare directories. A directory conta in ing d y he eelf and parent entriesieconsideredempty.


37/45

July 10,1984 34

The requirement for system-wide unique names implies that the directory level also hasthe responsibility for ensuring th at portions of the directory hierarchy resident on each machineare consistent. This can be accomplished by methods for replication in a distributed databasesystem [Seli80]. To control the number of update messages in a large system, the full directorydatabase may be kept on only a small subset of machines (e.g., two or three) implementing astable store. Copies of the views of the directory database being accessed by a given user canbe stored in a workstation or other local system after that user logs in. Operations that modifyan entry in a directory must send updates to the stable-store machines, which relay them toaffected workstations.

Specifications of the principal operations of the directory level are given in Table 9.These operations allow higher-level programs to create objects and store capabilities for themin direcfories, This table is not a complete specification of a directory manager; for example, itcontains no command to change the name and access fields of a directory entry.

-

The attach operation is used to create new new entry in a directory. The access field ofthe capability returned by a search operation will be the conjunction of the entrys access codeand th e access field already in the capability.

In t he special case of at taching a directory to a directory, the attach operation must alsodefine the parent of the newly attached directory. The operation fails if a parent is alreadydefined (see Figure 6). The detach operation only removes entries from directories but has noeffect on the object to which a capability points.. To destroy an object, the destroy operationof the appropria te level must be used. To minimize inadvertent deletions, the destroy-directoryoperation fails if applied to a nonempty directory.

The attach and detach operations must notify the stable store so that changes becomeeffective throughout the system. To keep this simple, we have required a) tha t an empty


38/45

July 10,1984 ss

TABLE Specificationof a Directory Manager Interface (Level 14).Form of Call Effect

dir-cap := CREATEDIR(acceu) AbCAte an empty directory. Retu rn a capability with i kpermhiin bit. net to the given acceu code. (Thu dimto ryi ot attached to the dimtor y tm. )Dertroy (remove) the given ditectory. (Fib f the dimt oryi onempty.)Make an entry cd ed rrme in the given directory (dir-cap);Itore in it the given object-capability (dj-cap) and thegiven ac ceu code. If obj-ecp denoter a directory, set i kparent entry from the self entry of the di nctory hr-crp.Notify the d imto ry st abk store of the change. (Faib if t h en a m e already exists in the dimtory 4r-c.p. if the dimtorydir-crp u not attached, or if 06j-ecp denota an already-a ttached di t or y. )Remove the entry of the given lumc from the givendm to ry . Notify the dimtory rtable store of the change.(F& if the name d o n not exist in the given di mto ry or ifthe given dircctory & nonempty.)Find the entry of the given nrmc in the given dircctory andreturn a copy of th e associated capability. Set the acccufield in the returned capability to the minimum privilegeenabled by the ac cc ~.ields of the d im to ry entr y and of thecapability. ( F a f the n m e doer not exist in the givendirectory.)

In a segment of t h e calkr'r vir tua l memory, eturn a copy ofthe contents of the di mtor y. (A user-level program caninterrogate the other lev& for other information about theobjects lirted in t he dm to ry - .&., da te of lart change.)

DES TROY-DIR(du-cap )

ATTACH(obj-cap, du-cap, name, acceu)

DETACH(du-cap, name)

obj-cap := SEARCH(dir-cap, name)

oeg := LIST(dir-cap)

directory must first be attached to the global directory tree before entries are made in it, andb) that a directory must be empty before being detached. A more complicated notificationmechanism will be needed if a process is allowed to construct a directory subtree beforeattaching its root t o the global directory tree.


39/45

July 10,1984

10. SHELL (Level 15)36

Most users of the system spend most of their time employing existing programs, notwriting new ones. When a user logs in, the operating system creates a user process containinga copy of the shell program with i ts default input connected to the users keyboard and itsdefault ou tput connected to the users display. The shell is the program that listens to theusers terminal and interprets the input as commands to invoke existing programs in specifiedcombinations and with specified inputs.

The shell scans each complete line of input to pick ou t th e names of programs to beinvoked and the values of arguments to be passed to them. For each program called in thisw a y , the shell creates a user process. The user processes are connected according to the da taflow specified in the command line.

Operations of substantial complexity can be programmed in the commrind.language of theUNIX shell. For example, the operations that format then print a file named text can be setin motion by the command line:

-

tb l output

The first program is t b f , which scans the d at a on it s input st ream and replaces descriptions oftables of information with the necessary formatting commands. The symbol indicates that the output of fptroffisto be placed in a file named output. If > output were replaced with 1 laser, the datawould instead be sent directly to the laser printer.


40/45

July 10, 1984 57

Having identified itie -eoiiiponsntaof a ccmmand .!in,=, the she!! obtains capabilities forthem by a series of commands:

c l :=SEARCH(CD, tbl);c2 :=SEARCH(WD, text);c3 :=CREATEPIPEO;c4 :=SEARCH(CD, eqn);c5 :=CREATEPIPEO;c6 :=SEARCH(CD, lptroff);c7 :=CREATEFILE();ATTACH(c7, WD, output, all);

The variable CD holds a capability for a commands directory and WD olds a capabilityfor the current working directory. Both CD and W D are par t of the shells context (seeFigure 4).

The shell then creates and resumes user processes that execute the three-components ofthe pipeline and awaits their completion:

RESUME( FORK(c1, -, 2 , 3 );RESUME( FORK(c4, -, 3 , 5 );RESUME( FORK(c6, -, 5 ,?) );JOIN 3);

After the JOIN returns, the shell can kill these processes and acknowledge completion of theeniire command to the user (b y a p r~mpt haracter!.

If the specification < text were omitted, the shell would have connected t M to thedefault input, which is the same as it s own,namely the terminal keyboard. In this case, thesecond search command would be omitted and the first fork operation would be

FORK(c1, -,PORTS[O], ~ 3 )Similarly, if > output were omitted, t h e shell would have connected lpttoffto the defaultoutput, the shells PORTS[l].


41/45

July 10,1984 38

If an.elaborate command-line is to be performed .often , typing i t can become tedious.UNIX encourages users to store complicated commands in executable files called shell- sc r ip t sthat become simpler commands. A file named l p might be created with the contents:

tb l $2where the names of nput and output files have been replaced by variables $1 and $2. Whenthe command l p is invoked, the variables $1 and $2 are replaced by the arguments followingthe command. For example, typing

lp text outputwould substitute text for $1 and output for $2 and so would have exactly the same effectas the original command line. - ?.,- ..-_

11. INITIALIZATIONOne small but essential piece of an operating system has not been discussed -- the method

of starting up the system. The st ar tup procedure, called a bootstrap sequence, begins with avery short program copied into th e low end of main memory from a permanent read-onlymemory. This program loads a longer program from the disk, which then takes control andloads the operating system itself. Finally, the operating system creates a special login processconnected to each terminal of the system.

When a user correctly types an identifier and a password, the login process will create ashell process connected to the same terminal. When the user types a logout command, theshell process will exit and t he login process will resume its vigil over the terminal.


42/45

July 10, 1984 38

12 . CONCLUSIONWe have used the levels model to describe the functions of contemporary multi-machine

operat ing systems. This description shows how i t is possible to systematically hide t h e physicallocations of all sharable objects and yet be able to locate them quickly when given a name inthe directory hierarchy.

Th e directory function can be generalized from its tradit ional role by storing capabilities,rat her than file identifiers, in directory entries. No user machine need have a full, local copy ofthe directory structure; i t need only encache the view with which i t is currently working. Thefull structure is maintained by a small group of machines implementing a stable store.

The model can deal with heterogeneous systems consisting of general purpose usermachines, such as workstations, and special purpose machines, such as stable sto ra , fileservers, and supercomputers. Only the user machines need contain a full operating system; thespecial purpose machines require only a simple operating system capable of managing localtasks and communicating on the network.

.--

The levels model is based on the same principle found in nature to organize many scaleaof space and time. *4t each !eve1 of bstraction there are well defined rules of interaction forthe objects visible at that level; the rules can be understood without detailed knowledge of thesmaller elements making up those objects. The many part s of an operating system cannot befully understood without recourse to this principle.

13. ACKNOWLEDGEMENTSWe are grateful to many colleagues for advice and counsel while we formulated and

refined this model. These include J. Dennis, E. Dijkstra, N.Haberrnann, B.Randel1, and M.Wilkes for early inspirations about hierarchical structure; K. Levitt, M.Meliar-Smith, and P.


43/45

July 10,1984 40

Neumann for numerous discussions about PSOS; D. Cheriton, for demonstrating in theTHOTH system, D. Comer, for demonst rating in the XINU system, and G. Popek, fordemonstrating in the LOCUS system, many of the ideas of this paper; D. Schrader for adviceon distributed databases and help with the document; D.Denning for advice on access controls,capability systems, and verification; A. Birrell, B. Lampson, R. Needham, and M. Schroeder foradvice on server models of distr ibuted systems; and D. Farber, A. Hearn, T. Korb, and L.Landweber for advice on networks through the CSNET Project. We are grateful to theNational Aeronautics and Space Administration, which supported part of this work undercontrac t NAS2-11530 a t RIACS. We re also grateful to the National Science Foundation,which supported part of this work through grant MCS-8109513 at Purdue University.

14. REFERENCESBe1183.

Bell Laboratories,, UNIX, System Users Manual: System V, 301-905 Issue 1, WesternElectric (January 1983).

-

Birr82.Birrell, Andrew D., Roy Levin, Roger M. Needham, and Michael D. Schroder, Grapevine:An Exercise in Distributed Computing, Communications of the A C M 25(4) pp. 260-274(April 1982).

Bogg83.Boggs, David R., Internet Broadcasting, CSL-83-3, XEROX PARC, PaloAlto (October 1983).

Brin73.Brinch Hansen, P., Operating System Principles, Prentice-Hall, Englewood Cliffs,N J (1973).Cheriton, David R., The Thoth Syatern: Multi-procese Structunng and Portability, ElsevierScience, New York (1982).

Cher82.

Cher84.Cheriton, David R., The V Kernel: A Software Base for Distributed Systems, IEEESoftware l( 2) pp. 19-42 (April 1984).Come84.Comer, Douglas, Operating System Design: Th e XINU Approach, Prentice-Hall,Englewood Cliffs (1984).

Den n 70.Denning, Peter J., Virtual Memory, Computing Surveys 2(3) pp. 154-216 (September


44/45

July 10,1984 41

1970).Denning, Peter J., Third Generation Computer Systems, Computing SurveysS(4) pp.175-212 (December 1971).Denning, Peter J., Fault-Tolerant Operating Systems, Computing Surveys 8(4) pp. 359-389 (December 1976).Denning, Peter J., T. Don Dennis, and Jeffrey A. Brumfield, Low ContentionSemaphores and Ready Lists, Communications of he ACM 24( 10) pp. 687-699 (October1981).Dennis, J. B. and E. C. Van Horn, Programming Semantics for MultiprogrammedComputations, Communications of he ACM9(3) pp. 143-155 (March 1966).

Dijkstra, Edsger W ,The Structure of the THEMultiprogramming System,Communications of th e A CM l ( 5 ) pp. 341-346 (May 1968).Fabry, R. S. , Capability-Based Addressing, Communications of the A CM 7(7) pp.

Denn71.

Denn76.

Denn81.

Denn66.

Dijk68.

Fabr74.403-412 (July 1974).

Gold74. - +%-.Goldberg, Robert P., Survey of Virtual Machine Research, Computer, pp. 34-46 (June1974).Habermann, A. Nico, Lawrence Flon, and Lee W. ooprider, Moduiarization andHierarchy in a Family of Operating Systems, Communications of the ACM 19(5) pp.266-272 (May 1976).Holt, R. C., E. D.Lazowska, G. . Graham, and M. . Scott, Structured ConcurrentProgramming with Operating Systems Applicutions, Addison- Wesiey, Eieading, TvA (1978).IBM,, IBM Virtual Machine Facility/370: Introduction, GC20-1800-1, IBM (August1973).Jones, Anita K., Robert J. Chansler Jr., Ivor Durham, Karsten Schwans, and Steven R.Vegdahl, StarOS, A Multiprocessor Operating System for the Support of Task Forces,Proceedings of the Seventh Symposium on Opcruting Systems Principles, pp. 117-127(December 1979).Kernighan, Brian W. nd Rob Pike, Th e UNIX Programming Environment, Prentice-Hall,Englewood Cliffs (1984).Levy, Henry M., crpub;lityBuscd Computer Systems, Digital Press, Bedford, M A 1984).

Habe76.

Holt78.

IBM73.

Jone79.

Kern84.

Levy84.


45/45

July 10, 1984 42

Neum80.Neumann, Peter G. , Robert S. Boyer, Richard J . Feiertag, Karl N. Levitt, and LawrenceRobinson, A Provably Secure Operating System, its Applications, and Proofs, CSL-116(2nd edition), SRI International, Menlo Park, CA (May 7, 1980).

Orga83.Organick, Elliott, A Programmers View of the Intel 432 System, McGraw-Hill, NewYork (1983).

Orga72.Organick, Elliot I., The Multics System: A n Eza mina tion of i ts Structure, The MIT Press,Cambridge, MA (1972).Ousterhout, John K., Donald A. Scelza, and Pradeep S. Sindhu, Medusa: AnExperiment in Distributed Operating System Structure, Communicat iona of the A CM23(2) pp. 92-105 (February 1980).

Oust80.

Pari83.Paris, Jehan-Francois and Walter F. Tichy, Stork: An Experimental Migrating FileSystem for Computer Networks, CSD-TR-411, Purdue University, West Lafayette,IN (February 1983).

Pope81.Popek, G., B. Walker, J. Chow, D. Edwards, C. Kline, G. Rudisin, and G. Thiel,LOCUS: A Network Transparent, High Reliability Distributed System, Proceedings of- - - the Eighth Symposium o n Operating Systems Principles, pp. 169-177 (December 1981). .CRitc74.Ritchie, D. M. and K. L. Thompson, The UNIX Time-sharing System,Comm unicat ions of the A C M 17(7) pp. 365-375 (July 1974).

Rowe82.Rowe, Lawrence A. and Kenneth P. Birman, A Local Network Based on the UNIXOperating System, IEEE Transactions on Software Engineering SE-8(2) pp. 137-146(March 1982).

Seli80.Selinger, P. G., Replicated Data, pp. 223-231 in Distributed Data Bases, ed. F.Poole,Cambridge University Press, Cambridge, England( 1980).Tanenbaum, Andrew S., Com puter Networks, Prentice-Hall, Englewood Cliffs, N J (1981).Tichy, Walter F., Design, Implementation, and Evaluation of a Revision ControlSystem, pp. 58-67 in Proceedings of the 6th Internatio nal Conference on SoftwareEngineering, IPS, ACM, IEEE, NBS(September 1982).Wilkes, M. V. nd R. M. Needham, The cambridge C A P Com puter and i ts Operat ingSystem, ElsevierjNorth-Holland Publishing Co. (1979).Wulf, William A., Roy Levin, and Samuel P.Harbison, H Y D R A / C . m m p , A nEzperimental Computer System, McGraw-Hill (1981).

Tane81.Tich82.

Wilk79.

WUlf81.

levels of abstraction in operating systems

Documents