a flexibility-driven microkernel architecture · a flexibility-driven microkernel architecture...
TRANSCRIPT
Bachelor Thesis
A Flexibility-Driven Microkernel Architecture
Martin [email protected]
06th October, 2010
Supervisor: Dipl.-Inf. Stefan WinterExaminer: Prof. Neeraj Suri
DEEDS GroupDepartment of Computer ScienceDarmstadt University of Technology, Germany
www.moduli-os.de
Abstract
In a quickly-developing IT world the growing complexity and the ever-increasing re-quirements to software turn out to be an essential problem – portability, extendability,maintainability, performance, dependability and many more. Software developers areunable to create universal software, which puts them in substantial considerations abouttrade-offs and design choices. However, user software is tightly-coupled and dependenton the underlying operating system. Contemporary operating systems in turn are oftenoptimized for specific features and properties, imposing fixed abstractions and limitedsupport for the varying software requirements. This poses serious challenges for operat-ing system structures, which are often unable to catch up.
We argue that flexibility is a key feature of operating systems that can decrease soft-ware complexity and development time, enhance reconfigurability and adaptability, aswell as facilitate functional reuse. We investigate a novel operating system architecture- a hierarchical microkernel with a main emphasis on flexibility and an additional em-phasis on reliability. It is an upgrade of the original microkernel concept with modulesorganized in a hierarchical manner. The architecture implies useful features that userapplications and system services can take advantage of. This includes a fully recon-figurable operating system interface and implementation, spatial context, hierarchy oftrust and communication transparency.
We have implemented an operating system based on this concept for the Intel IA-32 architecture to show that our approach is promising. We also describe how varioussoftware engineering problems can effectively be recast by structural support from theoperating system. Our evaluation proves the potential of the hierarchical microkernel.
3
Contents
1 Introduction 71.1 Flexibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.4 Thesis organization and structure . . . . . . . . . . . . . . . . . . . . . . . 13
2 Related Work 142.1 Classes of kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.1 Monolithic kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.1.2 Layered kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.1.3 Microkernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.1.4 Exokernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Mach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3 L3 and L4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.4 Chorus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.5 SPIN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.6 Multikernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3 Proposed Architecture and Thesis Scope 203.1 System organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.1.1 Hierarchical microkernel . . . . . . . . . . . . . . . . . . . . . . . . 213.1.2 Buses and modules . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.1.3 Gateways . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.1.4 Rules and requirements . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 An example system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.3 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.3.1 Spatial context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.3.2 Two-way abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . 333.3.3 Hierarchy of trust . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.3.4 Custom communication mechanisms . . . . . . . . . . . . . . . . . 363.3.5 Communication transparency . . . . . . . . . . . . . . . . . . . . . 36
3.4 Comparison to traditional microkernels . . . . . . . . . . . . . . . . . . . . 373.4.1 Flexibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.4.2 Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.4.3 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.4.4 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4
4 Design & Implementation Issues 424.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.2 Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.2.1 Modules and buses . . . . . . . . . . . . . . . . . . . . . . . . . . . 444.2.2 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464.2.3 Memory layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.2.4 System calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.3 Memory management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484.4 Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.4.1 Sending . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.4.2 Receiving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.5 I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504.6 File systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514.7 Runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514.8 Sample modules and buses . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5 Discussion 535.1 Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.2 Current implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.3 Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545.4 Message passing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545.5 Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
6 Conclusion & Future Work 566.1 Co-processor acceleration . . . . . . . . . . . . . . . . . . . . . . . . . . . 566.2 Decentralized memory management and virtualization . . . . . . . . . . . 566.3 Formal verification of the isolation mechanisms . . . . . . . . . . . . . . . 576.4 Practical projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
7 Acknowledgements 59
5
”The main function of a microkernel is to provide a communication facilitybetween the client program and the various services that are also running inuser space.”
– Abraham Silberschatz (Operating System Concepts, 8th edition [SGG08])
6
1 Introduction
In today’s world operating systems (OS) are an inevitable part of every computersystem - from small sensor nodes [LMP+05] to big mainframe computers (e.g. z/OS).Operating systems are the intermediate level between user applications and hardware.They have the complex task of managing resources and providing a unified executionenvironment. This is accomplished by establishing an abstraction level over hardware,which minimizes the time and effort of developing software and allows developers to writehardware-independent programs. With the growing speed and capabilities of hardwareand in a world, where the demand of rapid software development rises dramatically,one can barely imagine running a user application without operating system. Even onsimple embedded chips, operating systems have not only become a standard, but alsothe most significant component in terms of revenue generation [BRW10]. Moreover, theOS abstraction has evolved to such levels that many software developers have no clearidea how operating systems are working “under the hood”.
Initially, no operating system was required, as software and hardware were relativelysimple. With the increasing requirements to software, by its growing complexity, and thefurther progress of hardware, the development processes became an issue. Developersfound a way out by creating libraries, containing frequently needed functionality. Suchlibraries were later unified to a single program that was run before the actual userapplications. This is how operating systems were born. Nowadays, this type of operatingsystem organization is known as monolithic 1 - the operating system is running as asingle program with higher privileges (known as kernel mode) than user applications. Inparticular, a monolithic kernel is a collection of system services and functions which arerather unstructured and unprotected from one another. Most of the available operatingsystems today are considered monolithic [Tan07].
Operating systems evolve relatively slowly. On the one hand, developers try to keepthe OS interface to user applications consistent to make them compatible with olderversions of the kernel. Besides, most of the development efforts go into the race againstthe evolution of hardware architectures [Pu92]. The history shows a vast number ofoperating systems which were discontinued due to the inability to cope with the rapidly-developing hardware (e.g. SkyOS, HeliOS, ARX). On the other hand, a great numberof ideas are not adopted. This is because they are either too complicated and time-consuming to develop, or too hardware-dependent. Indeed, some research is practically
1More information about monolithic kernels can be found in the book of Tanenbaum - “Modern oper-ating systems” [Tan07].
7
unusable because its contributions rely on narrow assumptions. Certainly, there is awell-known gap between theory and practice. Developing on a conceptional level canoften lead to outstanding results, but the practical implementation level strictly dependson the underlying hardware.
As user software requirements increase, so do operating systems requirements. De-pending on specific demands, a problem can be solved in a number of ways, underdifferent constraints and hardware architectures. This implies the existence of varioustypes of operating systems. Desktop operating systems, for example, are aimed at asingle-user scenario and therefore optimized for convenience and response-time, whereasmainframe operating systems are designed to achieve high throughput and performance.To meet all demands, developers often have to make choices, regarding the internal or-ganization of the operating system kernel. These design choices have a crucial impacton the properties of the resulting system. For example, one mobile operating systemrequirement is low power consumption. Mobile phone manifacturers are aimed at creat-ing phones with long battery liftimes and an operating system that does not meet thisconstraint is not a potential choice. Indeed, quality attributes are used to characterizeand compare operating systems. Some of them include: efficiency, reliability, flexibility,security, portability, scalability, robustness. We cannot generally say that a given prop-erty is more important than another one as this strictly depends on the application withthe particular operating system.
Efficiency is one of the commercially used terms when advertising in the IT world. Butopposed to most believes, performance is not always a required feature. A microwave,for example, will do the same job, no matter if it uses an 8-bit RISC microcontroller oran Intel R© CoreTM i7 processor. The latter will just affect the price, but not the heatingquality.
Reliability is the ability of the system to deliver services as specified. This has becomea major concern, as the rapid software development and time-to-market requirementspush developers against strict deadlines, thus often resulting in faulty software. Tech-nical research shows that on the average, each 100 lines of code contain a bug [Tan07].Therefore, operating system developers try to eliminate common software mistakes bydesign and consequently increase the overall reliability of the system.
Portability and scalability are also of growing importance with the emerging hardwarearchitectures. The move from one-core to many-core processors required software totake advantage of the additional processing units. Still, a great number of programsare designed to work with single core architectures only. The reasons for that includethe lack of support from the programming languages, the difficulties of parallelizing userapplications and the inflexibility of operating systems and software.
8
1.1 Flexibility
Flexibility is considered an important software property. In general, flexibility is amulti-dimensional and polymorphous term that can have different meanings in differentcontexts [GP00]. It is the ability to adapt to internal or external changes. But thisdefinition is too abstract in relation to software engineering. The Institute of Electricaland Electronics Engineers (IEEE) gives the following definition of flexibility [The90]:
The ease with which a system or component can be modified for use in applications orenvironment other than those for which it was specifically designed.
Flexibility consists of two sub-properties: extendability and maintainability. Extend-ability is the ease with which a software system can be modified to increase its functionalcapacity. Maintainability is the ease with which a software system can be modified tocorrect faults, improve performance or other attributes, or adapt to a changed environ-ment [The90].
Applications that do not meet these two characteristics are hard to use in a long-termperspective. This is because maintenance is a regular phase of the software developmentlift-cycle, following the deployment. It includes modifying software to fix bugs and meetthe changing requirements, as well as the constantly evolving software environment interms of software libraries or hardware.
Flexibility is often a required feature in the industry. Software has to be designed ina way that takes into consideration a possible future growth or change. This also helpsdecrease costs because eventual specification changes require less/slight modificationsto the existing software. A famous example of inflexibility is the Year 2000 problem(Y2K). Poorly designed software used to represent years with two digits only, implyingthey start with ”19”. The UK Government has estimated that the total cost to theUK would be about $50 billion to solve the problem [Neu97]. Indeed, the proportion ofsoftware maintenance costs has increased over the last 30 years from 67% to over 90%of the total cost [Kos10].
Reliability and robustness go hand in hand with flexibility. In particular, flexibleoperating systems can simplify the construction of reliable applications. Sometimes,simple operations can be curprisingly difficult to implement on top of a traditional OSinterface. Furthermore, bugs in the kernel (e.g. in system calls) can often be used tocompromise the reliability of user applications. In fact, some researchers even supportthe opinion that a flexible operating system is demanded to provide secure applicationsat all [MK97].
Flexibility is a natural way of simplifying software. Software engineers follow a two-step approach of managing the ever-increasing complexity - first, divide requirements
9
into subproblems and implement them in subsystems called modules and second, abstractfrom specific details of a component’s particular application context to enable its reusein different contexts [Win09]. A flexible system allows different divisions and abstractionlevels to be tested, so that an optimal choice can be made, thus adapting the remainingproperties of the system towards its requirements.
Striving for modularization has pushed software engineers to developing various pat-terns, tools and frameworks that support software reuse. However, such attempts areoften impeded by the inadequate design of hierarchically lower components - virtualmachines, operating systems or even hardware. We think that the design of an operat-ing system can explicitly support modularization and code reuse by providing a flexibleenough architecture as a base. There are hundreds of libraries, for example, which pro-vide functionality to manipulate images (resize, crop, save in different formats, etc.).If operating systems and library facilities were flexible enough, it would not have beennecessary to develop more than one such library. This would have saved time and effort.Unfortunately, compatibility problems and the lack of special interfaces between compo-nents often prevent connections between modules. If developers were never “reinventingthe wheel” and wasting their time in writing pieces of code that already exist, more new,unique and stable applications would have existed. If architectures were not imposingany compatibility problems and allowed code reuse at function granularity (no matterof programming language and hardware architecture), software development would havebeen much easier. In other words, flexibility is a good way of coping with the increasingrate at which software is required to progress.
The tension between flexibility and performance pervades system development. It isusually a trade-off between them - a system may not guarantee good performance andflexibility at the same time. Thus, a design choice is made depending on the require-ments. Sometimes, the flexibility of the system precedes its functionality, i.e. whenthanks to a flexible architecture we can make a component work, even with performanceloss. Emulation and virtualization are good examples.
Current operating system are not flexible enough. As stated earlier, most modernoperating systems are considered to be modular monolithic, which means that the kernelcan be extended by dynamically loading additional modules. However, such modularityis on the binary and not on the architectural level. It allows some flexibility, but itsmain idea is to reduce the size of the kernel that is kept in the main memory.
The monolithic architecture imposes a number of problems - the kernel size and com-plexity, as well as its bad extendability and maintainability. Microkernels try to solvethese issues by limiting the kernel to only the most critical components. Often, theseare the scheduler and the memory manager2, and inter-process communication (IPC).Other system components are isolated from one another and run on top of the kernel
2this includes the I/O control as well
10
as normal user processes (called servers), thus increasing the overall reliability of thesystem. User applications use the functionality provided by servers by making requeststo them using inter-process communication facilities. Unlike monolithic kernels, a failurein one component does not necessarily bring the whole system down, therefore allowingfaulty modules to be identified and restarted [Tan07]. Additionally, due to the explicitmodularization of the system, microkernels are considered to be more flexible and scal-able. Modules can be added and removed from the system dynamically during runtimewithout affecting other parts of the system. However, these advantages are believed tocome with the price of performance loss. This is due to the need of message exchangebetween system components, rather than direct system call execution as it is the casein monolithic kernels. As system components are running as user processes, contextswitches to the kernel are required. This is where the most performance loss comesfrom. Moreover, such “indirect” calls require packing and unpacking of messages, whichis also often accompanied with message exchange security checks. Some researchers ar-gue that microkernels are not inherently inefficient, but that this belief results from thevast number of poor implementations [Lie95].
Some developers argue that microkernels impose a restricted flexibility due to thelack of efficiency [Lie95]. In this way important mechanisms and principles that rely onperformance cannot be used in practice. We, however, think that microkernels provide ahigher flexibility level than monolithic architectures (see section 2.1.3), but yet not thehighest possible one. With this bachelor’s thesis we are proposing an improvement overthe original microkernel architecture with emphasis on flexibility and reliability.
1.2 Problem statement
The problem to be solved by this bachelor’s thesis is the lack of a highly-flexible op-erating system architecture. Microkernels are in this aspect superior to large integratedkernels, but yet not flexible enough. With this work we present a novel operating sysstemarchitecture based on the original microkernel concept that increases the flexibility andreliability of the system. Thus, our problem statement consists of:
1. P1: Proposing a theoretical operating system organization with a main emphasison flexibility and an additional emphasis on reliability.
2. P2: Implementing this concept on a modern hardware architecture to support thetheoretical arguments.
3. P3: Evaluating the proposed architecture and its implementation to make generalconclusion if such an operating system organization has a perspective.
As the term flexibility is disputable, we will define it here in the context of operatingsystems. We consider an operating system flexible if it possesses the following properties:
11
• Modularity: A system organization that consists of discrete, highly-isolated andindependent components.
• Reconfigurability: The ability to dynamically change the internal organization ofthe system. This includes adding, removing and substituting components at run-time, also highly-critical ones, as well as changing the way they interact.
• Encapsulation: The ability to combine components in abstracted higher-level struc-tures, thus hiding complexity and providing simplified interfaces to functionality.
• Adaptability: The ability of the system to adapt to internal or external changes.This includes identifying issues within the system – faults, performance or func-tionality problems – and trying to recover from them and/or tune the system toensure availability.
• Functionality reuse: The ability to reuse functionality from components withoutfacing interface issues, compatibility or structural problems.
• Rational generalization: The ability to provide a relatively unified system interfaceto user application, but yet allow variance.
• Simplicity : Provide/Support all of the above mentioned properties in a rathersimple manner, without the need of complex software.
1.3 Contributions
This thesis proposes a new theoretical concept of a hierarchical microkernel architec-ture that implies a high-flexibility and reliability of the system by design. The maincontributions presented are:
• C1: Hierarchical organization of the operating system, where the microkernel isthe root of the hierarchy.
• C2: Process concept that provides an advanced modularization mechanism byisolating components from one another and prohibiting any direct interaction.
• C3: A new type of hierarchical scheduling that allows for distributed scheduling,thus enabling the combination of various scheduling mechanisms and policies.
• C4: Support for a hierarchy of trust to gradually guarantee the reliability andavailability of the system.
• C5: Inter-component communication, which supports more than two componentsin the same virtual channel as well as communication transparency.
• C6: Ability to provide custom communication facilities and use various messageexchange protocols.
As an proof of concept a partial implementation under the Intel R© x86 architecturehas been developed. It is presented in Section 4.
12
1.4 Thesis organization and structure
The reminder of this bachelor thesis is organized as follows. Section 2 discusses ex-isting operating system architectures and their flexibility properties as well as relatedwork on microkernels. Section 3 then presents the proposed architecture on a relativelytheoretical level. An implementation of the proposed architecture is presented in Section4. Some additional aspects and conclusions are then discussed in Section 5. Section 6concludes the thesis and elaborates on how the work on the proposed architecture willcontinue in the future.
13
2 Related Work
In this section we discuss different operating system architectures, where the emphasisis on how architectures by design influence flexibility properties of the system. We startwith the basic kernel classes - monolithic kernels, layered kernels, microkernels, andexokernels. We do not get into details about the organization of specific kernels asdetailed information on this is presented in a vast number of books [Tan07, SGG08],papers [Lie95, EKJ95], and on the Internet. Afterwards, we will discuss several state-of-the-art microkernel implementations that had significant impact for the microkerneldevelopment over the years or are prominent for their flexibility features. This includesMach [ABB+86], Jochen Liedtke’s L3 and L4 [Lie95, Lie96, Lie93], Chorus [RAA+91],SPIN [BSP+95], and Multikernel [BBD+09].
2.1 Classes of kernels
2.1.1 Monolithic kernels
As stated earlier, a monolithic kernel is a single program running with the highestpossible privileges. It is a set of functions that implement all system functionality –memory management, I/O, interrupt handling, drivers, etc. These functions are not byany means isolated from one another and each one is allowed to call any other func-tion. This kernel organization does not have a well-defined structure. Although kernelfunctions might be logically separated in groups, this does not lower the complexity ofthe system, nor does it provide any in-kernel abstractions. Monolithic systems usuallystarted as very small and efficient kernels that gradually grew up to big, complex andhard to maintain systems.
Monolithic kernels are considered highly-inflexible. They are not organized in mod-ules, rather the whole system is one enormous binary executable. However, modularmonolithic architectures exist as well. They allow the dynamic addition and removalof functionality to the kernel at a binary level, which makes the system a little moreflexible, but still does not provide any isolation between kernel and modules. Thus, thereconfigurability of monolithic kernels is considered very low – changes to the internalorganization require a kernel recompilation1. The system provides only one abstractionlevel – the kernel in the form of system calls. It is the basic encapsulated unit of func-tionality that user applications can use. Further abstractions can be created by usersoftware on the top of the kernel, but they are not natevely supported by the kernel.
1Runtime-loadable kernel extensions may exist, which require no recompilation, but this does not affectreconfigurability as extensions are loaded at binary level and are not isolated.
14
Because they are running with equal rights as user applications, it is more like an em-ulation of abstraction [EKJ95]. Furthermore, monolithic kernels are not fault-tolerant– considering the kernel running as one functional unit, faults, for example, can easilypropagate within the kernel and are very likely to result in unrecoverable system er-rors that require a restart of the whole system. In addition, monolithic kernels do notby-design support functionality reuse. Because abstraction levels can only be emulated,there exists the possibility to create libraries on the user-side. However, this does notguarantee any common interfaces or compatibility. Nevertheless, the interface betweenkernel and user applications is standardized in terms of a fixed set of system calls pro-vided by the kernel. Still, it is not extendable and no interfaces between user programsare structurally provided. Clearly, the major drawback of monolithic kernels is theircomplexity. Owing to the lack of any structure, system services in monolithic kernelscannot be independently developed, which results in huge kernel sizes, bad maintain-ability and extendability, slow development process, and difficult deployment. Indeed,this is the reason why monolithic architectures slowly evolve to microkernels.
2.1.2 Layered kernels
The functions and data structures of a layered kernel are physically grouped in layers.Each layer uses the functionality of the lower-leveled layers and provides an abstractionwhich higher-leveled layers can use. In this way, a layer hides the complexity of lowerlayers and offers a clean view over its capabilities. Usually, the very first layer in asystem is the hardware and the very last one - the interface to applications. In thisway, all the layers are running in kernel mode. However, implementations may vary, sothe operating system designers may draw the line between user and kernel space on theinterface between any two adjacent layers [SGG08]. Layered kernels are more structuredthan monolithic ones, but the layers are still organized in one dimension only. Moreover,the layering is usually only followed up to the interface to user applications, which meansthat applications are not layered themselves and cannot profit from this organization.
Layered kernels are also considered inflexible. Despite the reduced complexity andthe somehow modularized organization, the kernel is still not reconfigurable. The layersand the interfaces between them are rather fixed and no layers can be added or removeddynamically. The encapsulation in the kernel is relatively good, as each layer provides awell-defined abstraction over the lower-leveled ones. Still, from a user software point ofview, the kernel is one big black-box that provides a set of fixed system calls. Similarlyto monolithic architectures, the kernel does not include any mechanisms for user-levelabstraction and emulation is the only possibility. Because usually all the layers run inkernel mode, a fault in any layer is likely to bring the whole system down, thus speakingfor weak-adaptability. Regarding the functional reuse, layered kernels are superior tomonolithic ones because of the several abstraction levels. However, there is a subtledrawback - user applications have no profit of the layered organization as the kerneldoes not provide any inherent mechanisms of layering on top of it. Furthermore, thekernel still does not provide a unified interface between applications. Layered kernels are
15
considered an improvement over monolithic kernels as layers can be developed, testedand verified independently, which results in a more organized and maintainable kernel.However, this organization still does not impose a direct influence on user applications,which makes it rather inflexible.
2.1.3 Microkernels
The main idea of microkernels is to minimize the size of the kernel by distributingfunctionality from kernel space to user space, leaving only very critical system serviceslike I/O, interrupt handling, and inter-process communication facilities in the kernel.All other system services are organized as isolated modules (servers) that run in usermode like normal applications. User applications (clients) and the microkernel accessuse the functionality that servers provide by communicating with them. Obviously,communication mechanisms are what the microkernel architecture relies on - it is theonly way of interaction between modules [SGG08]. Typically, this is accomplished byexchanging messages between components using point-to-point virtual communicationchannels.
Microkernels are inherently flexible due to their well-organized structure [Lie95]. Themodularity of components does not apply only to the kernel, but the whole system orga-nization is based on it. Moreover, modules are isolated from one another which allows aprecise separation of functionality. In addition, microkernels are well-reconfigurable sys-tems, as modules can be added, removed, or substituted dynamically. This property alsospeaks for good adaptability. As most of the system is running in user space, faults inmodules have little influence on the kernel, which allows faulty modules to be identifiedand eventually restarted. In this way, the kernel provides by design a higher reliabilityof the whole system. Because of the good modularity, microkernels also impose goodencapsulation properties – modules are allowed to provide abstraction levels on the topof the kernel that other modules can use. However, combinations of modules in higher-level abstractions are not structurally supported. A major drawback of microkernels isthat all user space modules are running with the same privileges and on the same level.This allows no physical separation of modules based on their importance. Functionalityreuse in microkernels is also architecturally supported – as functionality is encapsulatedin modules, other modules can easily use it by issuing IPC requests. However, com-munication between components is restricted to virtual channels that include exactlytwo communication partners. Regarding rational generalization, microkernels provide aunified way of interaction in the whole system – namely the communication facilities inthe form of inter-process communication. Still, the kernel does not provide unified pro-tocols or rules for exchanging information – this strictly depends on the communicatingpartners. Also, the communication facility is provided by the kernel, which does notallow any addition or changes from the user-space. Therefore, custom communicationchannels and different ways of exchanging information are not supported.
16
All these advantages and flexibility of microkernels come in a relative simple way. Asthe kernel is kept as small as possible and system services are implemented indepen-dently, the development of the operating system is simpler in comparison to monolithickernels, for example. However, microkernels have a known drawback - a performancepenalty for exchanging messages and thus making the interactions in the system indirect.Fortunately, communication facilities can be optimized for specific processor [Lie95]. Thedisadvantage is that in this way, the microkernel looses its portability property.
2.1.4 Exokernels
Similarly to microkernels, exokernels try to minimize the size of the kernel by providinga very thin abstraction layer over hardware. This layer does not provide any criticalsystem services, but rather securely multiplexes the available hardware resources. Inthis way, multiple operating systems can be running on top of it, which allows each oneof them to use only part of the available resources. Basically, all the exokernel does ismanagement - spatial and temporal. As a result, operating systems and in particularuser applications can provide custom mechanisms for managing the allocated resources.In a vast number of situations, such flexibility leads to performance gains, as well aslowered software complexity [EKJ95].
Exokernels are considered flexible architectures. Their modularity is moderate, be-cause isolation is provided only between operating systems running concurrently on topof the exokernel. It is not predefined how operating systems should be structured – thusthe modularity property is actually not considered global. For instance, a monolithicoperating system can run on top of the exokernel, without using a modular structure.The same applies to reconfigurability as well – operating systems can be added andremoved at runtime, but this does not guarantee that these operating systems in turnensure the same property for their components or applications running on top of them.However, exokernels are more reconfigurable in comparison to monolithic architectures,for example. The encapsulation property of exokernels is excellent – the kernel allowsthe creation of arbitrary custom abstractions by operating systems and applications. Nolimitations apply to how they are organized, deployed or used. Exokernels even allow theextension, specialization and replacement of abstractions [EKJ95]. Their adaptability ismedium – failures in one operating system have no influence on others. Failures in oneuser application, however, might still affect the whole operating system. Speaking aboutfunctionality reuse, exokernels do not provide any mechanisms to explicitly support it.The thin layer over the hardware provides only a unified interface to operating systems,which allows variance and custom interfaces in one operating system. However, no in-terfaces are provided between operating systems. The exokernel architecture is verysimple, as the hardware abstraction layer is thin and compact. The major drawbackis that exokernels provide just an isolation and resource multiplexing mechanisms andnot a full system organization. This results in a certain degree of flexibility, but not tosuch an extent that necessarily affects the whole system, but rather only the operatingsystems at the lowest level.
17
2.2 Mach
Mach [ABB+86] is one of the earliest microkernel implementations, designed with theintention to support distributed and parallel computation. Components are decoupledfrom one another and interactions are allowed only using the introduced capability-basedinter-process communication facility. It allows components to communicate throughvirtual communication channels (ports) by exchanging messages.
Mach provides a higher flexibility than a traditional UNIX distributions. Among itsstrongest properties are modularity, adaptability and simplicity. The message exchangebase can also transparently be extended over network boundaries, which makes is suitablefor creating clusters and distributed applications. However, Mach does not structurallyprovide strong encapsulation and functionality reuse features. Its reconfiguration ismedium as ports are physically attached to components. Another disadvantage of Machis that the message exchange protocol is fixed, allowing no variance in the system.
2.3 L3 and L4
The L3 microkernel is one of the first second-generation microkernels. It was designedwith the intention to correct the IPC layer of the Mach microkernel with careful attentionto performance. L3 and its successor L4 prove that efficient and competitive microkernelscan be developed. L4 is considered the ancestor of today’s microkernel implementations.It provides a flat modularized system organization, where IPC is a basic concept forcommunication between subsystems and a foundation of independence.
2.4 Chorus
Chorus [RAA+91] is another example of an operating system aiming at distributedcomputing. Similarly to Mach, it contains a microkernel (called nucleus) which abstractsthe hardware and implements basic functionality (I/O and interrupt handling, low-levelmemory management, scheduling). The provided interface on the top of the nucleus isan inter-process communication facility. Over this communication basis different sub-systems or even operating systems can be implemented. The approach can be comparedto a mix of Mach and Exokernels.
Chorus imposes good reconfigurability and encapsulation properties. Unlike Mach,it implements named ports and port groups, which can dynamically be changed, fullytransparent for servers and applications. The grouping of services allow the creation ofdifferent abstraction layers within the system, which makes it flexible and adaptable.However, like most microkernels, Chorus implements the inter-process communicationfacility completely in the kernel.
18
2.5 SPIN
SPIN is an operating system designed with three goals in mind: flexibility, safety andperformance. It is an extendable monolithic architecture, where modules can be dynam-ically loaded into the kernel. Therefore, the operating system is highly-reconfigurable,where even critical system components can be changed (e.g. scheduler). The idea is toallow applications to profit from custom mechanisms in the kernel. SPIN and its ex-tensions are programmed in Modula-3. The modularity and isolation properties in thekernel are enforced by the type-safety of the programming language. The communica-tion between extensions and the base kernel is controlled and allows various interactionstyles.
The base kernel generates a number of events when internal/external changes happen(e.g. an interrupt). The extensions can install handlers that respond to given events,which the kernel transparently disguises as procedure calls. However, the operatingsystem is not adaptable - a fault in one extension can propagate to the kernel and resultin a hanged system. Certainly, SPIN is an example of an well-extensible monolithicoperating system.
2.6 Multikernel
The multikernel [BBD+09] is a new operating system architecture using ideas fromdistributed systems in order to provide a more adequate scalability for operating systems.It strongly relies on the message passing paradigm, even between cores on the samechip. Barrelfish[BBD+09], an operating system instance of this architecture mimicsthe exokernel approach, but additionally provides functionality for message exchangebetween local or remote cores in its lowest layer.
19
3 Proposed Architecture and Thesis Scope
In this section we present a novel flexibility-driven operating system architecture ona theoretical level. In section 3.1 we first explain the main idea of the architecture,followed by an example system organization in section 3.2. Afterwards, in 3.3 we detailthe basic features, provided by the architecture. As a conclusion, in 3.4 we compareour architecture to traditional microkernels based on four criteria – flexibility, reliabil-ity, scalability, and performance. For details on how the proposed architecture can beimplemented and the dependence on hardware, refer to sections 4 and 5.
The kernels discussed in the previous section have 3 major disadvantages – the equalprivileges for large classes of components, the limited support for functional reuse, andthe low customizability degree of the system.
First, none of the architectures considers that some applications (no matter if useror system) are more important than others and moreover – that components rely onone another. The software relations in a computer system can usually be expressedusing dependency graphs. Consider the following example (figure 3.1). A picture editorapplication allows users to manipulate images. To be able to accomplish that, theapplication relies on several libraries - an image library for reading, decoding and storingthe images, a drawing library that implements some basic image manipulation functionsand an event library that informs the application about mouse and keyboard inputs.The image library uses the file system component to read and store files, which inturn uses the disk driver that actually writes and stores the bytes on specific sectorsof the hard drive. The driver itself directly uses the I/O subsystem and the memorymanager. A malfunction of the image library will prevent the image editor from loadingimages. However, a malfunction of the file system driver will prevent both the imagelibrary (thus also the picture editor) and the email client from reading and storing files.Obviously, the system is supposed to ensure the reliability of the file system driver beforethe reliability of the image library and the email client (i.e. it has a higher priority).In this way, the system will be able to assure the availability of other services despiteof faults. Such logical dependencies are usually omitted by operating systems and anadaptable architecture is supposed to be built upon them.
Second, most kernels do not inherently support the reuse of functionality. By providingrather fixed high-level abstractions, they reduce the performance, increase the complexityand limit the functionality and flexibility of application programs [EKJ95]. Moreover,most kernels do not allow any abstractions to be created at user-level, but allow their
20
Picture editor Image library File system Disk driver I/O subsystem
memory manager
Event library Mouse/Keyboard driver
Drawing library Windowing system VGA driver
E-mail client
dependency
Figure 3.1: A sample (and incomplete) dependency between software components in acomputer system
emulation in the form of user libraries. Exokernels try to target this problem [EKJ95],but this solution just leaves an “open space” above the exokernel layer and does notoffer a structural support that can be used on higher software levels.
Third, contemporary architectures provide only a low customizability degree due totheir weak modularity and reconfigurability properties. Highly adaptable and flexiblesystems should allow the exchange of critical components at runtime for optimizationpurposes. Applications can often profit from low-level highly optimized custom mech-anisms like memory management, scheduling, or inter-process communication, for ex-ample. An exokernel allows that, but if the requirements to the system change duringruntime (for example due to an external event), the applications might require a changeof the custom mechanisms as well (e.g. for better optimization). An exokernel is flexibleenough to allow support of custom mechanisms, but not their dynamical reconfiguration.User applications will not only have to be restarted, but also relinked, using the new OSlibrary.
3.1 System organization
3.1.1 Hierarchical microkernel
Microkernels are traditionally organized in a flat or layered fashion as shown in figure3.2. All modules run as isolated components over the same abstraction layer (the micro-kernel) and communicate using a point-to-point communication paradigm. We proposea slightly different organization of the system, where modules form a hierarchy builtupon the microkernel as shown in figure 3.3. The root (level 0) of the hierarchy is themicrokernel. It it directly connected to the modules above it (level 1 modules) via asingle communication channel (called bus), which is shared by all level 1 modules andthe microkernel. Modules can use more than one communication channel, which allows
21
the expansion of the hierarchical communication network as shown on the figure – somelevel 1 modules have virtual channels to level 2 modules, which in turn are connected tolevel 3 modules. The working system1 forms an interconnected network of modules thatcan be represented as a tree graph. Similarly to traditional microkernels, the interactionbetween components in this hierarchical kernel is accomplished via these communicationchannels, in particular by exchanging messages. A message sent over a bus eventuallypropagates through the whole system, thereby reaching all intended recipients and/orall modules interested in this message.
Microkernel
App 1 App 2 VGA driver Disk driver File server Keyboard driver
IPC channel
Figure 3.2: The organization of a traditional microkernel
Because of the hierarchical organization, modules have a spatial context, which meansthat their position in the hierarchy is of importance for the whole system. We will seethe advantages of this in section 3.3 when we discuss the features of the architecture.Any two modules may be in one of the following transitive relations – above, below, orat the same level. In figure 3.3 the App 1 module is above Apps, below Library 2 andon the same level as Keyboard. However, Library 3 and VGA are not in any suchrelation.
Two modules are directly connected, if they share a common communication channeland are indirectly connected, if there exists a sequence of directly connected modulesbetween them. Connections can also be divided in horizontal and vertical. A verticalconnection between two modules is a connection, where each module on the path betweenthem is on a different level. A horizontal connection between two modules is a connection,where at least two modules on the path between them are on the same level.
The microkernel itself provides only the most critical system services - I/O, interrupthandling, basic scheduling, and management of the hierarchy. All other system and userfunctionality is implemented as modules in the hierarchy.
1The working system consists of all the components that are directly or indirectly connected to themicrokernel. Refer to section 3.1.2 for more information.
22
Microkernel
DriversApp
VGAApp 1
File system
Keyboard
Communication channel(Bus)
App 2
Library 1 Library 2 Library 3
Level 0
Level 1
Level 2
Level 3
Figure 3.3: A sample organization of a hierarchical microkernel system
3.1.2 Buses and modules
A system built upon the hierarchical microkernel concept has only two building blocks– modules and buses. A module is a unit of functionality – it encapsulates data structuresand functions, similarly to a process. The microkernel is also considered a module. A busis a virtual communication channel, which is responsible for providing a communicationfacility between two or more modules. A module is said to be attached to a bus when itis allowed to use the bus as a communication channel. A bus is said to be attached to amodule when the module is a communication end-point on the communication channel.The relation is symmetric – if a module is attached to a bus, then the bus is attachedto the module as well. A module is only allowed to be attached to buses. Attaching amodule to another module or a bus to another bus is impossible. Actually, attachinga module to another module is of no use, as buses constitute the only inter-modulecommunication mechanisms.
Modules are isolated units – all they are allowed to do in the system is executinginstructions from their own binary and communicating by sending and receiving messagesvia their attached buses. Other direct interaction with modules, buses or the microkernelis not possible. The goal here is to force modules to use an explicit way of interaction (e.g.communication) so that control is applicable. If possible on the hardware architecture,this also means not providing any system calls2. If not, then just a send system call
2By system call here is meant a software exception/trap, similar to an interrupt that causes a switchto kernel mode.
23
to send a message on a specific bus like in MINIX [TW06]. The same isolation rulesapply to buses – they are allowed to execute instructions from their own binary and toaccept/deliver messages from/to modules attached to them.
Modules use buses to interact with other modules by exchanging messages. These mes-sages might include requests, answers, data structures, or arbitrary types of information.Modules send information over a bus attached to them and the bus is responsible fordelivering it to other modules attached to the same bus. Unlike traditional microker-nels, where the communication goes through the kernel, here a bus, which constitutesthe fundamental communication mechanisms, is implemented and running in user space.Buses in the system can be different, implementing various communication mechanisms,protocols and policies. In order to be able to attach a module to a bus, the module issupposed to known (and obey) the communication rules of this bus or the bus is respon-sible for checking compatibility. For example, if a bus implements a connection-orientedcommunication, then all the modules attached to it should be aware of how to establisha connection and how messages are being encoded. Otherwise, they will not be able tocommunicate with one another.
Having only two building blocks and a set of rules for connecting them, is a simple, yetpowerful, mechanism of organizing a system. A system is therefore highly-configurablelike a puzzle, where different system organizations can be tested. Moreover, a dynamicalreconfiguration of the system at runtime is also supported. Changing the connectionsbetween modules or even substituting buses is fully transparent to other parts of thesystem.
The main idea of the hierarchical microkernel is to create a system that mimics theorganization of hardware. In hardware design, functionality is encapsulated in chipsthat communicate through buses. As the design and production of chips is expensive,designers tend to organize architectures in ways that allow reuse and concurrent usageof these chips. We target a similar approach for interconnecting functional softwarecomponents.
3.1.3 Gateways
A gateway is a module that has more than one bus attached to it. Gateways areimportant for the system because they allow the hierarchical structure to grow upwards,function as controllers for interaction of their subtrees with the rest of the system, andare therefore significant for providing flexibility.
A gateway interconnects the modules in the subtree above it with the rest of thesystem. As it is connected to more than one bus, it can allow a message to propagatefrom one bus to another. In addition, if buses are different, the gateway is required tofollow all the communication rules and eventually to do message translation from oneprotocol to another.
24
Gateways can peek at the messages that go through to identify harmful content. Suchmessages can be blocked to ensure the reliability of the components above and below.For instance, if App 2 in figure 3.3 sends an output request to the microkernel, thenApps will block this message as only drivers are allowed to issue output requests.
The main advantage of gateways is the two-way abstraction they provide. A gatewaymight serve as an interface to the components above or below it, thus hiding complexity.Because a gateway is allowed to control the passing messages, it can, for example, providevirtual services that are then translated to a sequence of messages to actual servers. Sucha combination of abstractions can also solve compatibility problems - as is discussed ingreater detail in section 3.3.
3.1.4 Rules and requirements
The hierarchical microkernel has several rules and implied a number of requirementsto assure the correct and reliable operation of the system.
1. The root of the system is always the microkernelThe lowest level of the hierarchy should always be the microkernel and it shouldrun with the highest privileges in the system. It has to implement the most criticalsystem functionality – I/O, interrupt handling, management of the hierarchy andbasic scheduling.
2. The working system is always interconnected, starting from the microkernel, butno cycles are allowed in it.The working system is the interconnected network of modules starting from themicrokernel. All modules and buses that are not part of it, are considered non-working as they are not being scheduled. Therefore, a message sent over a busin the system should be able to reach each working module, if the gateways allowthat. However, no cycles are allowed in the system. Cycles would destroy thehierarchical organization. In addition, this results in one definite path betweentwo modules which eliminates the need of message routing.
Module
Parent module
Child Child Childchild bus
parent bus
bus
Figure 3.4: Parents and children in the system from the viewpoint of a module and abus
25
3. A module can be attached to at most 2 busesA module is limited to be attached to at most 2 buses. Therefore, the bus above themodule is called a child bus and the bus below it - parent bus (figure 3.4). Similarly,from the viewpoint of a bus the modules can be child modules and parent modules.This is more a simplification than a limitation, as modules attached to more than2 buses can be emulated by using one more bus.
4. System provides a distributed schedulingAs stated earlier, the microkernel does only basic scheduling. Scheduling in thesystem is done is a distributed manner, where each parent is responsible for schedul-ing its children. This means that the kernel is only responsible for scheduling itschild bus (see section 4.2.2). Similarly, the child bus (level 0 bus) is responsiblefor scheduling the child modules attached to it and so on. Distributed schedulingallows different scheduling mechanisms and policies to be applied at the same timewhich results in greater flexibility.
3.2 An example system
Figure 3.5 depicts an example system built on the hierarchical microkernel concept.The root of the hierarchy is the Microkernel and its child bus (here also called mainbus or level 0 bus) has 4 child modules attached to it:
• File system - a file system server that accepts requests for opening and closingfiles as well as performing read and write operations on them.
• Apps - a gateway that connects the main bus with the application bus and controlswhat information may pass between them.
• Memory manager - a memory management server which accepts requests forallocating and deallocating memory.
• Drivers - a gateway that connects the main bus with the driver bus and controlswhat information may pass between them.
26
Microkernel
App 2
DriversApps
KeyboardWrapper
Memory manager
VGAWrapper
App 1
Level 0
Level 1
Level 2
Level 3
File system
Library 1 Disk
Text mode Video mode
main bus
application busdriver bus
Figure 3.5: An example hierarchical system
The application bus has 3 modules attached to it. Library 1 is a dynamically-loadedlibrary, which provides some kind of functionality. There are also two user applicationsrunning in the system, App 1 and App 2, that are indirectly connected to the appli-cation bus via Wrappers. This is simply a preventive mechanism to assure that userapplications do not have direct access to this bus. Therefore, wrappers can explicitlycontrol what these applications send and receive. The application bus can be used byApp 1, App 2 (via the wrappers) and Library 1 to make requests, send replies, orexchange data. Apps can forward signals to the application bus that come from othercomponents in the system, such as drivers. The wrappers can then, for example, preventa signal from reaching applications that are not supposed to receive it.
The driver bus is organized in a similar manner. There are 3 drivers running inthe system – Keyboard, VGA and Disk driver. The drivers are not attached viawrappers as the system assumes that they are trustworthy. An untrusted driver cansafely be connected to the driver bus via a wrapper. The VGA module has a childbus, on which 2 child modules are attached – Text mode and Video mode. Thesetwo components act like servers that implement the two different modes that the VGAdriver supports. They are separated in 2 isolated units, which allows their separatedevelopment. In addition, the drivers gateway can prevent unintended interactions withother system components. The VGA module serves as an interface to them. Evenif servers change the supported requests in future, the VGA gateway can keep theinterface to them fixed, which will eliminate possible compatibility issues. This canbe accomplished by forcing the VGA module to translate requests and thus emulateprevious server.
27
In the following an example scenario is presented, where a key on the keyboard ispressed, which results in a terminal program printing the corresponding character onthe screen. The scenario illustrates what types of messages are exchanged between thecomponents to accomplish this task.
1. HardwareA key is pressed and the corresponding interrupt is triggered.
2. Microkernel → Main busThe Microkernel receives that interupt and translates it to a messages that issent on the main bus. The message contains the number of the key that is pressed.
3. Main bus → Drivers → Driver busAll the components attached to the main bus receive the message, but only Driversconsiders it relevant. It forwards the message to the driver bus. All other compo-nents on the main bus ignore the message.
4. Driver bus → Keyboard → Driver busFrom the driver bus, only the Keyboard component is interested in the messageand it processes it by translating the key number to a symbol. Then it sends amessage back to the driver bus marked as a “key pressed signal”, containing thesymbol.
5. Driver bus → Drivers → Main busNone of the other components on the driver bus besides Drivers is interested ina signal message. Drivers forwards the message to the main bus.
6. Main bus → Apps → Application busApps receives the message and forwards it to the application bus.
7. Application bus → Wrapper → Local bus → App 1Both wrappers and the library receive the message. The library ignores it as itdoes not await any keystrokes. Both wrappers check if the applications they arecontrolling await a keystroke. If so, they forward the message on the bus betweenthe user application and wrapper. In our case, Wrapper forwards the message toApp 1 which is a terminal program.
8. App 1 → Local busThe terminal program (App 1) receives the message and processes it by requestingthe symbol to be printed on the screen. This request is directed to the videodriver as recipient. The message is, however, sent via the local bus. Intermediatecomponents are required to forward the message. Gateways know which modulesare attached in the hierarchy above them, so they can decide whether to forwardor not.
9. Local bus → Wrapper → Application bus → Apps → Main bus → Drivers →Driver bus → VGA
28
The message goes through Wrapper, Apps, Drivers until is reaches the VGAmodule. It in turn accepts the requests and depending on the current graphicmode sends a specialized request in the form of a message either to Text modeor to Video mode.
10. VGA → Local bus → Text/Video mode → Local busDepending on the mode, in which the operating system is, one of the two com-ponents processes the message and creates an I/O request to the video adapter.Modules run in user mode and are not allowed to directly invoke I/O operations.Thus, the request is sent as a message on the driver bus directed to the microkernel.
11. Local bus → VGA → Driver bus → Drivers → Main bus → MicrokernelThe message is forwarded via VGA, Drivers and finally reaches Microkernel.The microkernel processes it by checking if the corresponding driver is allowed toperform this specific output and if so, writes the symbol in the VGA RAM.
12. HardwareThe character appears on the screen.
Keep in mind that this is just an example scenario on an example system to illustrateinter-module interactions. Other systems built on the same concept can work in differentways even if they are using the same components. As we will see in the next section,from the viewpoint of a module, sending a message is just passing it to the child orparent bus. A module itself does not need to know the complete route of a messagethough the system, nor does it know the actual organization of the system.
3.3 Features
In the following section we discuss the main features resulting from the hierarchicalmicrokernel concept. The explanations are accompanied with anumer of examples toillustrate the practical relevance of the features.
3.3.1 Spatial context
The hierarchical microkernel concept exhibits an isolation property, similar to tradi-tional microkernels [TW06]. Modules and buses run in isolated address spaces, thereforeno direct interaction between them is possible. They are allowed to execute instructionsfrom their own binary and communicate directly with their attached modules/buses bysending and receiving messages. Clearly, all harmful things a component can do in thesystem are limited to: executing a privileged instruction or sending a harmful message.The former one is a problem to be identified by the processor on a hardware-level. Mostmodern CPU architectures trigger an internal exception, followed by a context-switchto the kernel, which is then responsible for handling it. In the latter case, a harmfulmessage can be identified by the bus itself, other components attached to the same bus
29
or gateways on the way to the recipient(s). Hence, reliability mechanisms are distributedwithin the whole hierarchy (see section 3.3.3).
The hierarchical organization provides an advanced modularity in comparison to tra-ditional microkernels. Modules have the so-called spatial context, which means that theirposition in the system and the surrounding modules are of importance. The closer amodule is to the microkernel, the more important it is considered. For example, gate-ways can control the hierarchy above them in terms of scheduling and message relay.It is then normal to assure that gateways are correctly functioning before the modulesabove them. In fact, a failure in a gateway can cause the above modules not to be sched-uled. Therefore, system reliability is guaranteed using an on-level basis, starting fromthe microkernel (level 0). In case of faults, the operating system can make choices andtrade-offs based on the priority information. For instance, if the physical memory is fulland the operating system has to store some critical information until a given deadline(e.g. in a real-time application), then it will rather kill an arbitrary user applicationthan the disk driver. As a consequence, the memory manager will be able to use thedisk driver to swap out several applications and afterwards restart the killed one.
Considering the example scenario with the picture editor and its dependency graph infigure 3.1, a system organization might look like in figure 3.6. The disk driver is runningon level 1, the file system server on level 2, and the image library on level 3. Obviously,the disk driver is more important than the file system, which is in turn more importantthan the image library. The spatial context can be used to take dependency graphs intoconsideration. An interesting question that might arise here is why the image editor isrunning on level 2 when it relies on the image library. This is simply a design choice –placing the image editor over the image library can equally valid choice, depending onapplication characteristics. Here, user applications in this system are attached above the“Servers” module and not above the driver they use. Moreover, the image editor mightnot be necessarily dependent only on the image library, but rather loosely coupled to allkinds of image libraries. If the disk driver fails, the whole sub-hierarchy above it will notfunction. However, this does not mean that the image editor will not be able to sendrequests to other image libraries on remote systems using the network. Therefore, theidea of splitting the actual user application and the components it relies on in differenthierarchies can be advantageous.
30
Microkernel
Disk driverServers
Image editor
Network
E-mail client
Level 0
Level 1
Level 2
Level 3
File system
Image library Database
Figure 3.6: The importance of modules based on their position in the hierarchy. Thecloser a module is, the more important it is considered by the operatingsystem.
An essential side-effect of the spatial context is that the functionality of a moduleis no longer independent. As gateways can forward, modify and block messages, theycan influence the message flow from/to a given module. Let us assume that we have adatabase module which is used to manipulate databases stored in normal files. It acceptsdatabase queries and in turn sends a sequence of file system requests to manipulate thecorresponding files. This module can then be used in at least 3 different situations asdepicted in figure 3.7. In the first system, the database is communicating with the localfile system server, thus manipulating a local database stored as a normal file. In thesecond one, the database module is still communicating with the local file system server,but the requests of the file server are then sent by the Network module to a remote diskdriver over the network. The result is that the same Database module is transparentlymanaging a database file on a remote computer. In the third configuration, the Interfacemodule is serving like a virtual file server that is actually accepting the same requests asa file server, but is translating them to memory accesses, allocations and deallocations.Therefore, the Database module is managing a database that is actually stored in themain memory rather than in a normal file on the disk. These three configurationsshow how the same component can actually be used for different tasks in a relativelysimple manner. Similar adaptations often require complex middleware in other operatingsystems.
In addition, the previous example demonstrates the good reconfigurability of the hi-erarchical microkernel. Because modules are loosely-coupled, it is possible to substitutemodules or reorganize how buses and modules are attached at runtime. What modulesand buses “see” is only the flow of messages in their local environment. Where these
31
Database
File system
Database
Network
File system
Network Disk driver
Database
Interface
Memory manager
...
...
... ...
...
...
......
(1) (2) (3)
Figure 3.7: Functional dependence of modules
come from and where they go is not always important. In the above example, where theDatabase module is managing a local database file (1), it is possible to attach a Networkmodule below the File system at runtime, resulting in a fully-transparent change of themanipulated databases.
The hierarchical microkernel also allows the reorganization of very critical systemcomponents. For instance, it is possible to change the memory manager of the operatingsystem during runtime (figure 3.8). The problem with touching critical components isthat they have an internal state, which eventually has to be “transfered” to the newcomponent that is taking over. In a hierarchical manner, this can be accomplished in arelatively simple way. For the memory manager it might happen as follows:
1. The memory manager is attached to some bus, no matter which one.
2. A Buffer module is attached under the memory manager. This buffer collects allmessages in both directions and does not forward them.
3. The new memory manager is attached on the bus between the old one and thebuffer. The new memory manager sends a message to the old one, requesting itsinternal state. The old memory manager is supposed to support such a request,otherwise the change of the critical components is impossible3.
4. The two memory managers exchange data and the new manager accepts the inter-nal state of the old one. Its internal data representations can though be different.In this case, the state data might include tables of the reserved physical memoryfor each module/bus and their page tables.
5. After the state information has been exchanged, the old memory manager can bedetached from the bus.
3Alternatively, the new memory manager can request permission from the old one to use its data sectionin the main memory. However, this is a very complex mechanism and will not be discussed here.
32
6. The buffer then sends all the collected messages to the two buses and is alsodetached. The system is supposed to continue functioning like before.
(1) (2)
Old MM Buffer
Old MM
(3)
Buffer
Old MM New MM
Buffer
New MM
New MM
(4) (5)
Figure 3.8: Changing the memory manager at runtime
3.3.2 Two-way abstraction
Gateways in a hierarchical system provide abstractions both for the modules aboveand below them, called a two-way abstraction. They can act like interfaces, which hidecomplexity, adjust the communication, or do translations in both directions. An exam-ple of this is a traditional database application (figure 3.9). Queries to the database arearriving on the parent bus of the SQL interface. It hides the complexity of the abovemodules by allowing applications to do requests over the clean and standardized SQL in-terface. In this way, applications do not have to know the specifications of each databasemodule. The abstraction in the other direction hides the database-specific representa-tion of the query results and returns it based on the standard SQL specifications. Herethe SQL interface acts like a translator between user applications and the database.It is the only component that knows the specifications of each database module.
33
...
SQL interface
Storage engine
Relational engine
Transaction engine
Query Engine
...
Figure 3.9: A database application
Actually, a gateway can abstract not only from the specifics of modules directly aboveit, but from the details of the whole sub-hierarchy. This combination of abstractionlevels is an advanced encapsulation mechanism that makes the software organization ofthe system highly flexible. Building on the top of existing functionality can dramaticallyreduce software development time and effort. Moreover, abstractions can hide reconfig-urability effects. In the example above, if we change one of the database modules ortheir organization, this action will be transparent for user applications, as long as thischange is supported by the other components beyond the interface.
One of the essential advantages of the two-way abstraction is the simplified functionalreuse without compatibility problems. Consider the case where two applications usedifferent versions of the same library. Depending on the operating system, it might notbe able to install both versions at the same time. Therefore, programs cannot be usedin parallel. The hierarchical microkernel can solve the problem as shown in figure 3.10.The gateway will not only forward the requests to the corresponding library dependingon the application. Also, the gateway can emulate deprecated functions or even allowapplications to use both libraries at the same time. In some cases, this can also preventsecurity problems. Consider the case with an older library version with known-bugs inseveral functions. The gateway can redirect calls to these functions to the newer libraryto avoid errors.
The hierarchical organization of the system allows the simplified creation of monitors,patches, wrappers and compatibility interfaces. For instance, to implement a keyloggerin the example in figure 3.5, a keylogger module is added between the keyboard driverand the driver bus. Its task is to forward all messages between the driver and the localbus and to monitor the messages containing a keyboard interrupt. The implementa-tion of such a module would be around 20 lines of code4. Similarly, small patches can
4Not including the lines of code of the standard runtime library to send/receive messages.
34
be implemented as gateways, which log the internal state of the patched module andeventually block/translate messages to prevent problems.
...
Common library interface
Library ver 3.4 Library ver 3.5 Library ver 0.9
...
Figure 3.10: Using different versions of the same library in parallel
3.3.3 Hierarchy of trust
To guarantee the reliability of the system the hierarchical microkernel follows anarchitecture-inherent bottom-up approach, called hierarchy of trust. Higher-level com-ponents assume that lower-level components below them are reliable and therefore trustthem. The microkernel trusts the underlying hardware and it is practically the baseof the trust hierarchy. If a component in the system is untrusted, so is the whole sub-hierarchy above it. Indeed, a failure in a module is likely to affect only the hierarchyabove it and impose no influence on the hierarchy below it. Knowledge about the relia-bility of components can help system designers to choose an optimal position to attach amodule, so that a possible failure has a minimum effect on the whole system. Untrustedapplications or drivers can be attached at higher-levels via wrappers (like in figure 3.5),so that their behavior can be monitored and evaluated. In addition, the organization ofa system structurally allows lower-level components (gateways) to control higher-levelones above them.
The hierarchy of trust can help a system adapt to changes and ensure its availability.Let us consider an example where the hardware malfunctions, which usually results infailure in the corresponding driver and eventual hanging of the whole system (knownin Microsoft Windows as “Blue Screen of Death”). We will use the example system infigure 3.5. If the graphic card malfunctions, the Driver module can identify this in oneof the following ways:
• An interrupt is received by the microkernel, which indicates a hardware bus prob-lem.
• There are only requests to the VGA driver, but no answers going through.
35
• The VGA driver has created more threads than the maximum allowed number(this indicates that a received message in the VGA is endlessly processed).
To adapt the system to this change, the Drivers module attaches a Router modulebetween the driver bus and the VGA driver, which redirects the input/output requestsof the driver over the network to a remote VGA driver. In this way, the screen of themalfunctioned system can be seen as a virtual console on a remote machine.
3.3.4 Custom communication mechanisms
Buses in the hierarchical microkernel are running completely in user-space, whichallows custom communication mechanisms, based on message exchange, to be used. Thisincludes the potential to use various message protocols, data representations, connectiontypes and policies. Thus, the communication between modules can be optimized forspecific purposes. Such customizations can help to increase performance and reduce thefunctional limitations of modules [EKJ95].
This is what we described as rational generalization in section 1. The interactionsbetween components in the system are based on a unified mechanism, namely messageexchange (generalization). However, custom user-implemented buses can also be used,thus allowing variance in the hierarchical system (rational). What is aimed at hereis to provide a mechanism balanced between generalization and customization. Unlikethe exokernel approach, we believe that flexibility does not imply the need for extremecustomization, as this limits other aspects of a flexible architecture (e.g. functionalreuse).
3.3.5 Communication transparency
From the viewpoint of a module, it can only directly communicate with the buses at-tached to it. There can be other modules on the same bus as well, but the module will notbe able to make a direct interaction with them. As a result, a module explicitly “knows”only its surrounding environment. Of course, message semantics may include requestsand answers that give information about other modules as well. However, a module isstructurally prevented from seeing the full organization of the system. Therefore, com-munication in the hierarchical microkernel is transparent. This means that buses andmodules do not know the route of a message. All they see is a flow of messages, comingfrom their surrounding components. Similarly, a bus knows only the modules attachedto it, but has no clue which other buses take part in the communication.
This transparency has several advantages. First, there is no need to exchange infor-mation between modules when the system organization changes. This allows a fasterand easier reconfiguration. Indeed, modules are not interested in who is serving theirrequests, as long as there is someone to do the job. If we change a server during runtime(like in the example with the memory manager), modules will certainly not notice this.
36
Second, the transparency reduces the development time and complexity of new modules.To create a new module and attach it to the existing system, the modules does not needto know how other modules work internally. It is only required to know their interfaces,so that it can use their services.
Basically, the hierarchical microkernel is based on a distributed functionality principle.Each module offers some kind of functionality in terms of services. Whenever the modulecannot solve a given task by itself, it can send it either on the child bus5 or on the parentbus, so that a server can eventually process it.
3.4 Comparison to traditional microkernels
Our proposed architecture builds upon the microkernel concept and aims at increas-ing its flexibility and reliability properties. In the following sections, we discuss theadvantages and disadvantages of the hierarchical microkernel organization in compari-son to traditional microkernels. The comparison will be based on 4 criteria – flexibility,reliability, scalability, and performance.
3.4.1 Flexibility
We will follow the introduced flexibility properties (section 1.2) and give several ex-amples of how they are influenced by the hierarchical organization.
The hierarchical microkernel is a modularized architecture like the traditional micro-kernel architecture. Nevertheless, its organization is built using two types of components- modules and buses (instead of only modules). The isolation mechanisms in the hierar-chical microkernel are slightly different, resulting from the organization of components.The attachment of modules to buses builds a hierarchical organization that limits thepossibilities for interaction among components. The idea of this limitation is to separateimportant from not so important components and provide more control in the system.Indeed, isolation in the hierarchical microkernel is stronger than in traditional microker-nels. Two modules cannot directly communicate without a bus and even may not be ableto create a shared communication channel, if they are parts of different sub-hierarchies.
In addition, modules possess a spatial context, which allows to observe functionalitynot only of separate modules, but also of groups of modules (e.g. sub-hierarchies).Therefore, components are more flexible – they can be used for different purposes withoutthe need of modification of the functional units themselves, but rather only the systemorganization. Unlike traditional microkernels, where the IPC facilities are hardcoded inthe kernel, hierarchical microkernels allow custom IPC mechanisms based on messageexchange. Thus, applications are free to provide their own communication protocols and
5Usually, a module will send a request on the child bus, when it explicitly knows that there is acooperative module. In all other cases, it will send the request on the parent bus.
37
policies. Similarly to traditional microkernels, modules and buses are relatively easilyportable, whereas the microkernel is not portable because it constitutes an abstractionfrom a specific hardware architecture. However, due to the small size of the kernel, thisis often not considered an essential disadvantage (in our implementation the compiledkernel size is around 30 kilobytes binary code, see section 4).
The hierarchical microkernel is also reconfigurable like the traditional microkernelarchitecture. System organization can be changed dynamically at runtime. However,the hierarchical organizations and the features deriving from it imply better reconfig-urability properties. Abstraction and communication transparency hide reconfigurationeffects, which increases the flexibility. Therefore, even critical system components canbe changed at runtime. As the communication channels are physically separate frommodules and running in user space, there is no problem (de)attaching a module/bustransparently. In comparison, traditional microkernels often impose problems when de-taching a module from the system that has an open virtual channel. In addition, tradi-tional microkernels do not poses communication transparency, which sometimes makesreconfiguration impossible. For example, when two modules are connected and com-municating, detaching the one module destroys the communication channel, so a newone has to be created. This makes the reconfiguration non-transparent for the secondmodule.
Traditional microkernels impose good encapsulation properties. However, physicalcombination of functionality units is not supported. In contrast, hierarchical microker-nels allow arbitrary higher-level abstractions from groups of modules in 2 directions tounlimited levels. This simplifies software development, as well as creation of interfaces,monitors, wrappers, patches, etc.
Because of the good modularity, isolation, and reconfigurability properties, the hierar-chical microkernel is considered adaptable. Similarly to traditional microkernels, faultsdo not affect the whole system, but rather separate modules. In contrast to traditionalmicrokernels, where reconfigurability is not always transparent, the hierarchical organi-zation allows better adaption to internal/external changes, in order to ensure maximumavailability of the system.
Traditional microkernels count on modularization and indirect communication viamessage exchange to provide a service-oriented architecture, which is inherently proneto functional reuse. The hierarchical microkernel builds upon the same concept, but alsoadds two-way abstraction, which makes functional reuse easier. As all microkernels, thesystem is service-oriented with physically separated sub-systems. As a result, applica-tions can be developed using the separation of concerns paradigm. If a module cannotaccomplish a task by itself, it just sends a message to the corresponding server. In atraditional microkernel, this often requires the requester to know the exact server. In
38
contrast, in the hierarchical microkernel this is not necessary. The requester just sendsthe request on the bus and waits for an answer.
Traditional microkernels generalize the interactions between system components bybuilding upon message exchange as communication paradigm, implemented in the kernel.Unfortunately, this allows no variance in the inter-process communication mechanisms.In contrast, the hierarchical microkernel provides custom inter-module communicationin the form of buses. This allows a higher customization degree of the system, as well asthe possibility to optimize applications’ communication. The hierarchical organizationdoes not destroy the generalization of the system as all working components are stillinterconnected.
All the flexibility gains a hierarchical microkernel is providing in comparison to atraditional one come in a rather simple manner. Because of the relatively small set ofrules to build a working system and the separation of concerns, modules can be createdeasily and quickly. The hierarchical organization actually reduces the complexity of thesystem by providing two-level abstractions.
3.4.2 Reliability
Microkernels are inherently more reliable than other kernel types [Tan07]. Each com-ponent is structurally independent and running in user space. Thus, a failure in onecomponent can only affect the component itself, but not the whole kernel. Specialservers may track such failures and try to restart failed services, therefore increasing theavailability of the system. However, traditional microkernels do not consider the logi-cal dependencies between software components. In the hierarchical microkernel, systemdesigners can construct the system in a way that observes them. The concept for ensur-ing the reliability is slightly different, building on inherent properties of the hierarchy.The so-called hierarchy of trust (section 3.3.3) guarantees the reliability of componentsgradually, in levels. Each module trusts and relies on the modules below, resulting in asystem with different degrees of trust attributed to components. Therefore, hierarchicalmicrokernels allow for more fine-grained reliability guarantees.
However, bad system design may reduce flexibility in hierarchical microkernels. Wronglyidentified software dependencies and the corresponding module organization may leadto the effect that one failed gateway prevents the correct and continuous operation ofan actually independent module. Therefore, systems must be created with caution andstrictly defined relations.
3.4.3 Scalability
Scalability is often defined as the ability of a system to handle growing amounts ofwork and being ready to be enlarged [Bon00]. Due to the modular structure of microker-nels, they are considered well-scalable. On a many-core hardware architecture, different
39
modules can run on different cores, thus increasing the performance and throughput ofthe system. In addition, as microkernels are organized based on a message exchangecommunication paradigm, they are also easily scalable over networks (e.g. clusters).Messages can be sent/received over the network in the same way as local IPC communi-cation. The disadvantage of traditional microkernels is that communication facilities arepart of the kernel. This limitation does not allow the facilities to be scaled over cores,resulting in a communication bottleneck.
In the hierarchical microkernel, communication facilities are not centralized. They aredistributed among the different sub-hierarchies and are all implemented in user-space.Therefore, buses can be scaled over cores without any problems. In addition, due tothe nature of the system, subsystems can expand over networks, without destroyingthe hierarchical organization. For example, clusters of computers can be representedas large interconnected hierarchies, where communication transparency allows them towork together without requiring middleware (figure 3.11).
Network Drivers
...
Microkernel
Apps
...
...
Apps Drivers
...
Microkernel
Network
...
...
Network (Ethernet,
Internet, Serial port, etc.)
NetworkNetwork
......
Figure 3.11: Hierarchical microkernels in a cluster
3.4.4 Performance
Performance is the big disadvantage of the microkernel class. Although many re-searchers believe that reliability is more important than performance [Tan10], it it stillone of the essential properties for a system to be used in practice. Due to the message ex-change paradigm, microkernels impose a high context-switch overhead, resulting in a sig-nificant performance penalty. There have been numerous tries to decrease the penalty, sothat the speed of microkernels is comparable to monolithic kernels [Lie93, Lie96, Lie95].Often researchers blame developers for improper implementations of the microkernel con-cept. Commercially, many microkernel operating systems are performing well enough
40
and are preferred in comparison to monolithic architectures (e.g. Symbian, QNX).
Due to the microkernel nature, the hierarchical microkernel is also imposing a per-formance penalty. However, depending on the underlying hardware, it is sometimespossible to implement the inter-process communication facility in a different way thansystem calls. In contrast to traditional microkernels, the hierarchical ones use communi-cation channels with more than one end-point. This means that a message is likely to bedelivered to several modules (meaning several copies of the same message). Dependingon the message exchange implementation, this might impose an additional performancepenalty. Moreover, a message is often required to go through several levels and passthrough a number of modules until if finally reaches a recipient. This results in a higherdelivery latency, which we believe is a fair price from the provided flexibility.
In fact, it is not easy to discuss the performance property in theory. The assumptionis that the increased number of messages to be exchanged in the system will make itless-performant, yet more-flexible. However, such measures are implementation-specificand often optimizable.
41
4 Design & Implementation Issues
As a proof of the hierarchical microkernel concept, we implemented an operatingsystem based on it. In this section we present you the main aspects regarding theimplementation - the microkernel organization and services, the process representationand memory management, the communication facility, as well as the various issues thatarise. We also occasionally elaborate on the design and implementation variants weconsider and actual design choices. In order to completely undestand this section, somebasic knowledge on the Intel IA-32 architecture is required.
The operating system we developed is named moduli. The initial idea was to developonly an instance of the hierarchical microkernel concept, but the OS turned out to bea suitable foundation for further research on the topic. We believe that a hierarchicalsystem organization has a lot of research potential. There are numerous open questionsand optimization problems related to it, some of which will be discussed in section 6.Moduli turned into a long-term project and we aim at expanding it in many directions.This includes porting it to various architectures and integrating support for standardexecutable formats. Nevertheless, we are also planning to use moduli in several practicalprojects. You can find more information, news and actual papers about the architectureand its implementation on its official website – www.moduli-os.de.
Moduli is a 32-bit operating system implemented for the Intel R© IA-32 architecture[Int10](often generically called x86 or i386 ). It is written mainly in pure C, whereas some crit-ical parts are implemented in assembly for optimization and security1 purposes. The Cstandard library is still not ported for moduli, but a standard runtime is provided andwill be presented later is this chapter. The operating system has been mainly tested on2 popular x86 emulators (Bochs[Boc10] and Qemu[Qem10]) and 4 real-hardware com-puters (Table 4.1).
We will not get into details about the current implementation as it is a project indevelopment prone to changes.
Important! The presented version of the moduli operating system implements the origi-nal hierarchical microkernel concept with several exceptions. The Intel architecture turnsout to be non-supportive for hierarchically organized systems in implementing several ofthe required features (e.g. hierarchical isolation), thus an emulation of these is providedin the microkernel. This and further problems, resulting from the hardware architecture
1To prevent the compiler from eventually generating wrong code sequences (this happens!).
42
Computer IBM ThinkCentre 8187-D1G (original IBM-PC architecture)Type Desktop PCCPU Intel R© PentiumTM 4 2.66 GHzRAM 512 MB SDRAMYear of production 2001Computer Gigabyte W556UType Notebook PCCPU Intel R© CoreTM 2 Duo T7300 2.00 GHzRAM 2048 MB DDR2Year of production 2007Computer Lenovo Thinkpad T61Type Notebook PCCPU Intel R© CoreTM 2 Duo T7300 2.00 GHzRAM 2048 MB DDR2Year of production 2007Computer n/a (self-assembled)Type Desktop PCCPU AMD R© AthlonTM XP 2200+ 1.8 GHzRAM 128 MB SDRAMYear of production n/a
Table 4.1: Hardware configurations of the computers on which moduli was tested
are discussed in section 5. The emulation still allows the system to work as describedin the previous section, but some performance penalty is imposed. Therefore, we areunable to provide adequate benchmarks.
4.1 Overview
The moduli operating system consists of 3 parts - bootloader, microkernel and com-ponents (modules and buses). The bootloader initializes the system and loads the mi-crokernel. The microkernel is kept minimal and implements the management of thehierarchy, I/O subsystem, interrupt handling and basic scheduling. Additionally, thereis an integrated memory manager and a file system stub. As a design choice, the mainbus (bus 0) of the hierarchy is implemented in the kernel as well. The microkernel buildsthe initial hierarchy of components upon system start. Afterwards, components are al-lowed to interact by exchanging messages. The system can be dynamically reconfiguredby requesting attaches/detaches to the kernel.
The microkernel is the only component running in kernel mode with full privileges(ring 0 ), whereas modules and buses execute in user mode (ring 3 ). Isolation betweencomponents is provided by the memory management unit (MMU), using a paging mem-ory protection scheme. Each module has its own address space, in which the kernel is
43
mapped. The interface of system calls is kept minimal. Unfortunately, the Intel archi-tecture does not allow an implementation without system calls, because it provides noother mechanisms to accomplish the message exchange and scheduling paradigms.
Scheduling is hierarchical, starting from the microkernel. Each components, that haschild components attached to it is allowed to schedule them via a special system call.Sending of messages to a direct neighbor is accomplished using a system call as well.The kernel copies the message to the address space of the recipients and automaticallycreates a new processing thread for them. Direct input/output is only possible for themicrokernel. Modules have to send requests to it in order to accomplish reads and writeson I/O ports or memory-mapped devices.
4.2 Processes
4.2.1 Modules and buses
Modules and buses form the processes in the moduli operating system. Internally, abus does not differ from a module – they have the same memory layout, system calls,executable files and process management. The difference is actually logical and is madein the microkernel, where to each running process a type is stored. In this way, the kernelcan control interactions and will not, for example, allow the attachment of module tomodule. In the following, a process/component will refer to both modules and buses.
A new process is always created by the microkernel. This happens either at theinitialization of the hierarchy, or during runtime, after another component requests theattachment of a module/bus. Therefore, upon the creation of a process three things areclear: the component has just been attached to the hierarchy, it has a parent and it hasno children. Processes are identified by the so-called module IDs (MID)2. Each moduleor bus receives an unique and fixed MID number, which it can afterwards use to identifyitself when interacting with other modules or buses. This is often needed in messageexchange protocols, where a sender’s identification is explicitly included in the messageheader.
Unlike other operating systems, in moduli a process is not executed in the so-calledprocess environment. Usually, it includes a list of command-line arguments and en-vironment variables. In moduli, everything is communicated via messages and similarinformation should be delivered via the communication channels. Therefore, a program’sentrypoint in moduli is different from the popular C main method:
int main(int argc, char** argv);
2Buses are also processes, they are also identified by MIDs.
44
In moduli, the program entrypoint is also called message entrypoint. Every time anew message is received by a component, a new thread is created, which is responsiblefor processing the message. This thread starts its execution at this message entrypoint.This might be confusing at a first glance, but is actually a convenient way of writingprograms that continuously receive messages. It is similar to starting a new programexecution each time a message arrives. Here is how the declaration of the main methodlooks in a moduli program:
void main(unsigned short bus, char* data);
First, a main method does not return any values. To inform the caller/creator of theprocess for success or failure, the program is supposed to send a message. Second, thepassed parameters point to a received message. The bus parameter informs from whichdirect neighbor the message comes from. From a bus point of view, this will be the MIDof a child or the parent. From a module point of view, this is either the parent or the childbus. The data parameters is a pointer to the actual message. There are no restrictionsabout the structure of the message, the corresponding component is supposed to knowhow to handle it. A common standard, however, is that the first 4 bytes of the messagecontain its size. In this way, repeater gateways, that simply blindly forward messages,can work with any message representation.
Upon process creation, the passed parameters point to a special creation message,which contains information about the created process (e.g. MID). The special messagecomes from an invalid direct sender, which allow the program to differentiate betweennormal message and the very first one. An interesting point is that when the mainmethod has finished executing (the last instruction is executed and a jump to the calleris to be done), the process is neither deleted, nor detached from the hierarchy. It staysalive, but has no working threads. If a message is received, then a new tread is created.This is extremely useful for the development of servers, which serve remote requests buthave no own control flow. Having no own threads, results in no overhead schedulingthem, which speeds up the system.
As already mentioned, moduli supports threads. Threads can be created either ondemand by sending a special request to the microkernel, or automatically upon thereception of a new message. The operating system differentiates basically between twotypes of threads that can run in two protection modes, which results in four combinations(table 4.2). Normal threads are either threads that processes created on their own ormessage processing threads. Scheduling threads have special privileges that will bedescribed in the next section. The distinction of the execution mode is important is asecurity aspect as it is the base for implementing the context-switch routine.
45
Type/Mode kernel-mode user-modenormal normal kernel thread normal user thread
scheduling scheduling kernel thread scheduling user thread
Table 4.2: Types of threads in moduli
4.2.2 Scheduling
Following the hierarchical nature of the microkernel, scheduling is hierarchical as well(figure 4.1). There are 2 schedulers in the kernel. The first one is integrated in the timerinterrupt handler and schedules all the threads of the kernel itself. This also includesthe threads of bus 0 as it is part of it. This scheduler is preemptive and based on theround-robin principle. It is kept minimal – just a few lines of code, so that it does notimpose a high overhead. The second scheduler is the bus 0 scheduler, which is runninglike a kernel thread. Its task is to schedule the threads of all child modules attached tobus 0. This scheduler is also working on a round-robin basis.
Normalthread
Schedulingthreadschedules
A B
Figure 4.1: Hierarchical scheduling in moduli
Level 1 modules are supposed to schedule their child buses, which in turn have toschedule their child modules (figure 4.1). To accomplish hierarchical scheduling, eachcomponent (bus or module) is allowed to have one special thread, marked as schedulingthread. It is allowed to use a special system call to schedule a thread of a child compo-nent. The system call expects as a parameter the thread id (TID3) of the thread to be
3A TID is similar to MID, but identifying a thread in the system.
46
scheduled. Additionally, as the kernel knows the complete hierarchical organization ofthe system, it can check, if a scheduling request is acceptable.
However, to know the TIDs of its children, a parent has to perform some managementoperations. It has to keep track of the list of threads for each of its children. This consistsof two tasks. First, the parent has to peek in the messages of its children and identifyrequests to the kernel to create new normal threads. Afterwards, it has to identify thekernel answers and read the TID. Second, the parent has to keep track of the TIDs of thenewly-created child processing threads, after delivering a message. This is accomplishedby the send system call, which returns the TID of the created processing thread. Inaddition, moduli also supports auto-scheduling. A scheduler thread is allowed to makea scheduling system call with a special parameter that requests the kernel to performautomatic round-robin of its children. This saves some CPU and management overhead,but limits the flexibility.
Unlike other operating systems, hierarchical microkernels perform scheduling in aslightly different way. In order to schedule a higher-level component, a chain of schedulershas to be passed. This imposes a relatively high overhead on Intel architectures, butcomes with a greater flexibility as sub-hierarchies can use custom scheduling policies andschedulers. On the Intel IA-32 architecture, a scheduling from the user-side has to bemade using an emulation system call. However, on other architectures there might bean easier way - for example, if they support more than 2 privilege levels. Schedulingthreads can be executed in a ring, where they can accomplish scheduling by themselves.
A thread can also inform the scheduler, that it does not need the CPU anymore. Thisis also accomplished via a specialized system call. Additionally, the thread can alsorequest to be killed. This is usually used by processing threads after the message hasbeen processed.
4.2.3 Memory layout
The memory layout of a moduli process is depicted on figure 4.2. The lower 64 MBin the virtual address space are reserved and the kernel is mapped there. The kernelis however protected and a process is not allowed to directly access its data. The onlyway to switch to the kernel is via a system call, which changes the privilege level andjumps to the lower 64 MB. After the reserved space comes the actual code, data andother sections of the executing process. They are allowed to grow almost to the end ofthe address space (apart from the 4 MB at the end, which are reserved). This allows arelatively huge address space that can be used by an application (in comparison to otheroperating systems). The last 4 MB of the address space are reserved by the kernel andused for memory management purposes.
47
microkernel
magic page
module / bus
64 MB
4 MB
3.94 GB
Figure 4.2: Memory layout of a moduli process
4.2.4 System calls
Moduli supports only a limited number of system calls (table 4.3). Despite of therequirement not to have any system calls, on the Intel architecture there are no otherpossibilities to implement some of the functionality (e.g. different levels of protection).This requires its emulation via the microkernel itself.
System call Descriptionschedule Schedules a user space thread. Supports two modes. Manual mode –
schedules a thread by a given TID; Auto mode - automatically schedulesthreads using round-robin policy
send Sends a message to an attached neighbor. Expects neighbor MID and apointer to the message as parameters.
yield Informs the scheduler that the given thread does not require the CPUanymore. Additionally, the thread can request to be killed and option-ally to pass a pointer to a message to be deleted. Useful for processingthreads after having processed a message.
Table 4.3: System calls of moduli
4.3 Memory management
The microkernel of moduli has an integrated memory manager for 2 reasons. First,to build the hierarchy when initializing the operating system, a memory manager isrequired. Second and more important – the kernel cannot rely on an external memorymanager as this will destroy the hierarchy of trust. Therefore, as a design choice weintegrated a memory manager in the kernel that is responsible for managing its own
48
memory and address space. It is however possible to attach an external memory managerto be used by the rest of the system.
The memory protection in moduli is accomplished using 32-bit paging with a pagesize of 4 KB. The virtual address space is achieved via page translation in the MMUusing page tables, however no swapping to the disk is possible at the current stage ofdevelopment (due to the lack of a disk driver). The memory allocator uses a first-fitstrategy to allocate memory zones. The zones of the physical/virtual memory are storedin special linked-lists, aiming of a minimum memory overhead.
Memory allocation and deallocation for user-space components is accomplished viamessage requests (not system calls).
4.4 Communication
An essential aspect of the implementation is how communication between modulesand buses is achieved. It consists basically of two parts – sending and receiving of themessages.
4.4.1 Sending
Given that a component wants to send a message to an attached neighbor, this meansthat this component has reserved some memory (buffer) and written the message there.In order to send this buffer, 2 variants are possible - either the buffer is moved from theaddress space of the sender to the address space of the recipient, or it is copied to a newbuffer in the address space of the recipient.
The former one will be faster, but has a subtle disadvantage. Consider the case wherea bus is sending a message to all of its children. Mapping the buffer to the addressspace of all children will impose a security problem. The modules would share the samememory and will practically be able to modify the message, thus possibly influencingthe functionality of other modules accessing the same message. The mapped memoryregions can be marked as read-only. This would eliminate the security problem, but hasanother disadvantage. If a gateway receives a message as a read-only buffer and wants tomodify it before forwarding, it will first have to copy it to a non-read-only local buffer.Therefore, moving the buffer between address spaces is inconvenient. However, the mainproblem is that, if a buffer is removed from the address space of a given module, themodule will have to request a new buffer in order to create a new message. The requestin turn has to be sent as a message. Moreover, this prevents using the stack as a buffer.To get out of this paradoxical cycle, our design choice was simply to take the lattervariant and to copy all messages between the address spaces.
49
The sending itself had to be implemented as a system call, because the Intel x86architecture does not support multiple protection levels within a paging-based memoryprotection mechanism. In this way, the microkernel can accomplish the copying of themessage between the address spaces and additionally perform security checks.
4.4.2 Receiving
After a message has been copied to the address space of the recipient, the questionarises - how to inform the recipient that a new message has arrived. The two mainvariants are event-based and query-based approaches.
In a query-based message reception, the module itself periodically checks if a new mes-sage has arrived. This can, for example, be done via a system call or a check at a specialmemory address that the kernel will mark (for example, by setting a special value/bit).Query-based approaches have 2 advantages - they are easy to implement from the oper-ating system viewpoint and require no additional control flows (threads). However, thepersistent querying adds performance overhead and is generally not considered a goodidea.
In contrast, the event-based reception does not impose overhead as an event is gener-ated only when a message is received. This allows a process to have no threads and notto be scheduled at all until a message arrives, which is ideal for implementing servers(as discussed in 4.2.1). The problem with event-based approaches is that they requiremore than one control flow, which can also lead to synchronization problems. Our de-sign choice was to use the event-based approach, where for each received message a newnormal thread is created (called processing thread). The newly-created thread gets apointer to the message buffer as described in section 4.2.1.
After a processing thread is done processing the message/request, it usually calls theyield system call to give the control to the scheduler. Normally, the thread will alsorequest to be killed and optionally can provide a pointer to the message buffer, so thatthe kernel can deallocate it.
4.5 I/O
The microkernel of moduli prevents components from directly accessing input/outputports and memory-mapped I/O devices, but allows indirect reads and writes via requests.No control mechanism is applied here, it is awaited that higher-level gateways make surethat components have the rights to perform such requests. However, the microkerneldoes not allow the modification of the following devices/data structures, because thisbreaks the hierarchy of trust and can lead to serious security problems:
• Interrupt controller
50
• Timer
• CPU data structures - interrupt vector table, general descriptors table, TSS
Interrupts received by the microkernel are handled by user-space drivers, thereforethey are sent on the main bus as messages. An exception here is the timer interrupt,which the scheduler of the microkernel accepts and does not forward. As the interrupthandlers in the kernel are uninterruptible (cannot be preempted), they have to be asfast as possible. The sending of messages can however be slow and other interrupts canbe lost in the meantime. Therefore, a so-called lazy sending concept is applied. Aninterrupt handler just prepares the message, but does not send it. A new kernel threadis created that will later send the message.
4.6 File systems
The current release of moduli does not include a disk driver, therefore no file systemsare supported. The bootloader uses the BIOS disk driver to load the microkernel andcache all the files in the main memory. The file system that the bootloader supportsis specifically developed for moduli. It is called sfat and is quite similar to FAT16, butwithout directory support and a statical one. No files can be added or deleted. Thismakes it ideal for prototype-typical testing purposes.
Because moduli is flexible, it is possible to implement a disk driver and file systemservers as modules. These can later be attached to the hierarchy. No changes of themicrokernel will be required.
4.7 Runtime
In addition to the operating system, we also developed a runtime library, which speedsup the development of new modules and buses. It abstracts the system calls and provideseasy to use functions for building new messages, extracting data from messages, as wellas synchronizing threads. Furthermore, the runtime includes a default message handlerthat is capable of serving standard requests like ping, attachment of children, moduleinformation, etc. It also emulates synchronous message sending, where the thread blocksuntil an answer is received.
The runtime is compiled as an object file that is ready to be linked with the objectfiles of the module/bus.
4.8 Sample modules and buses
In the following, some sample modules will be presented that the moduli operatingsystem comes with. These can be used to build sample systems.
51
Simple bus (bus) The simple bus is just an example bus that has a very primitivemessage protocol. It blindly forwards messages from the parent to all children or froma child to all other children and the parent.
Repeater (module) The repeater is the simplest possible gateway to be constructed.It consists of just a few lines of code4 and its task is just to blindly forward messages be-tween the child and the parent bus. The repeater does not support message translation,therefore it works only if the child and the parent bus are using the same protocols. Anadvantage is that works with arbitrary protocols. Because of its simplicity, this gatewaycan be used to expand the hierarchy with one level at places, which require no additionalcontrol.
Dummy (module) The dummy module is just a test module that can be attached onany bus. It contains one running thread, that is simulating mathematical calculations.It accepts only ping requests, to which it answers. All other messages it receives areignored. The module does not support the management of a child bus and thereforecannot be used as a gateway.
Keyboard driver (module) The keyboard driver receives the interrupts of the keyboardcontroller and translates them to signal messages. It also controls the LEDs of thekeyboard.
VGA driver (module) The VGA driver accepts requests to print text on the screen. Itonly supports text mode of the video adapter.
Serial port driver (module) The serial port driver allows control over the serial porton the computer. This can for example be used by application to exchange data betweentwo or more computers.
Shell (module) This is a sample shell that accepts several commands for changingthe hierarchy of the system. It can load new modules, attach/detach components andreorganize the system. It can also be used to inject messages in the hierarchy for simu-lation/testing purposes.
Flooder (module) This is an example of a malicious module that continuously sendsmessages. Its idea is to emulate a malicious application that performs a denial of serviceattack. A stable hierarchy will not get affected by it and will eventually identify itswrong behavior.
4Not including the standard runtime.
52
5 Discussion
In the following we discuss several aspects regarding the hierarchical microkernel ar-chitecture, as well as its current implementation. We briefly comment on the perspectiveof the architecture, its current implementation issues, the similarity to virtualization, themessage exchange basis, and the possible alternative implementations.
5.1 Perspective
We believe that the hierarchical microkernel architecture has the potential to becomea widely-used concept among operating system developers, because of its strong flexibil-ity and reliability features. It is an essential upgrade of the microkernel approach, whichbrings various advantages for the fair price of some performance overhead. Indeed, per-formance can be a serious problem. However, we believe that optimizations can reducethe performance penalty to minimum and eventual hardware support can help even toeliminate it at all. We have been witnessing numerous approaches in the history thatwere initially known as quite inperformant, but later turned out to be essential for thepractice (e.g. the microkernel concept itself, virtualization). Virtualization technology,for example, was considered slow for many years. Now, processors natively supportvirtualization, thus minimizing the overhead.
The test results of the current implementation prove the ease to construct an operatingsystem based on the hierarchical microkernel, as well as developing software for it. Theextended flexibility turns out to solve many of the problems of traditional operatingsystem architectures.
We are going to continue researching on the topic and further improving our concept.There is a long list of open problems and aspects to be considered, investigated andquantified. Some of them will be discussed in section 6. Moduli is a long-term projectthat will try to prove that the hierarchical microkernel is a natural continuation of themicrokernel concept. We strongly believe that flexibility is the key to future softwaredevelopment.
5.2 Current implementation
Due to the lack of time and the strict deadlines, we were unable to release a stableversion of our operating system. There are a numerous optimization that are still pendingand will be implemented in the near future. Also, the reduced number of sample modules
53
we developed does not completely illustrate the practical importance of the hierarchicalmicrokernel. However, our work on the project continues and we strongly encourage thereader to occasionally check the project’s website for updates (www.moduli-os.de).
It turns out that the Intel IA-32 architecture is not suitable for implementing a hier-archical operating system. The main problem is the limited number of protection ringsthat the hardware supports (2 in paging memory protection mode). Therefore, only twodifferent components can be kept in the same virtual memory (kernel and one compo-nent). Ideally, we would like to keep 2 buses and a module in the virtual memory atthe same time. This is the local environment of a standard gateway. Moreover, the twoprotection rings force us to emulate the hierarchy using the kernel (for sending messages,doing scheduling). A completely decentralized management and no protection emulationfrom the kernel side is desired. Exactly this emulation leads to an increased number ofcontext-switches and TLB flushes, which results in a higher overhead. The second mainproblem is the lack of a manageable integrated high-speed memory in the processor. Itcould be used to make message exchange quite faster than the case where main mem-ory is used. The L1/L2 caches are an example for such memory, but unfortunately notdirectly controllable from the software side.
As we discuss in section 6, a hardware support for the hierarchical microkernel conceptcan be developed and we are plannig to perform such an implementation on an FPGAco-processor.
5.3 Virtualization
The hierarchical microkernel concept has a lot in common to virtualization. Virtual-ization basically provides an abstraction over the hardware that emulates a real hardwareenvironment in software. Gateways in the hierarchical microkernel provide two-way ab-straction that can also be used for such emulation purposes. However, several openquestions arise – first, how to base an existing operating system/software on messagepassing, when it is not inherently based on it and second, how to decentralize some ofthe resource management in the system (e.g. memory management). Theoretically, itshould be possible to run another operating system as a sub-hierarchy in a hierarchicalmicrokernel. It is an interesing research aspect that we will work on in the future (seesection 6).
5.4 Message passing
The hierarchical microkernel is based on a communication paradigm, namely on mes-sage exchange. We believe that this is of great advantage as it is a powerful conceptthat is used in many areas of the IT world (Internet, networks, programming languages,WWW, etc.). Moreover, this makes the operating system inherently scalable over net-
54
works, with no (or minimal) need of middleware. We will present several of our ideas forfuture projects that include moduli, some of which are based on hierarchical clusters.
In addition, message passing does not imply a particular protocol to be used. There-fore, different communication approaches can be built on top of it. It can virtually beused for exchanging all kinds of information.
5.5 Alternatives
Our implementation of the hierarchical microkernel architecture is just a sample one.There are various ways of implementing the different aspects of the operating system- organizing memory, implementing send/receive functionality, achieving protection be-tween components, etc. We do not expect that our implementation is optimal for everypossible application.
The hierarchical microkernel can be ported to various architectures. It should bepossible to emulate the system organization on each processor that supports some kindof memory protection and executing in at least two protection modes (kernel and usermode). One of our future goals will be to seek for a more appropriate architecture to portmoduli to and afterwards to compare its performance to the Intel x86 implementationin order to determine the dependence on hardware.
Ideally, the hardware architecture will be suitable for implementing hierarchical oper-ating systems by supporting several and manageable memory protection levels as well ascontrollable integrated high-speed memory. For instance, a processor can be built whichinternally supports a send instruction that automatically copies messages between ad-dress spaces.
55
6 Conclusion & Future Work
The presented hierarchical microkernel architecture proves that flexibility and relia-bility properties of an operating system can be improved by design. By upgrading theoriginal microkernel concept, we created a new type of operating system organizationand implemented it on a modern desktop computer architecture. We also gave examplesof how the concept increases the ease of reconfiguring the system on a low- or high-level.
Our research on the topic continues and we will try to further improve the concept andits implementation. One of our main goals will be to decrease the performance penaltyand eventually eliminate the performance bottleneck that the current implementationof message exchange imposes. Furthermore, we are planning to use the implementedoperating system in several practical projects. Actual information, news and papers canbe found on our official website – www.moduli-os.de.
In the following we will present the main directions for future work.
6.1 Co-processor acceleration
An interesting aspect to investigate is how dependent the hierarchical microkernel ison the underlying hardware in terms of performance. We are planning to develop a spe-cial FPGA-based co-processor that implements some of the concepts in hardware, thusproviding an acceleration for message exchange and/or multi-level process isolation. Theco-processor is to be used in cooperation with a standard x86 CPU. We believe that themessage exchange bottleneck can be eliminated by providing the corresponding hard-ware support, which will hopefully not only reduce the imposed performance overheadbut also impose a performance gain.
6.2 Decentralized memory management and virtualization
One of our main goals for the further development of the concept is to decentralizethe management in the system. This means, for example, decoupling the memory man-agement from one single component (the memory server) and distributing it into thegateways. In this way, a partial virtualization environment can be created. It can, forexample, allow the execution of various operating systems as a part of the hierarchicalsystem structure. In fact, such virtualized environment may provide only support forthe executable formats of other operating systems (e.g. COFF, Windows PE, ELF).
56
The idea of virtualization inside the system organization has the advantage that exist-ing software can be connected to the hierarchical microkernel without being changed. Inthis way, existing applications can partially profit from the advantages of the hierarchicalconcept. However, this idea is connected with many problems like emulating system callsof other operating systems or providing interfaces that do not rely on communicationbasis.
6.3 Formal verification of the isolation mechanisms
A higher degree of modularity in a system composition intuitively provides a higherdegree of fault isolation, i.e. absence of error propagation among modules, if modules arein some way isolated from each other. In order to prove the intuition that hierarchicalmicrokernels provide a higher degree of fault isolation than existing OS architectures, weare currently developing a formal framework for reasoning about module compositions,isolation, and error propagation paths among modules.
6.4 Practical projects
Within the next few years we are planning to use the moduli operating system inseveral practical projects. We not only want to assess the practical importance of thehierarchical microkernel, but also measure performance, throughput, utilization, real-time guarantees and other properties. For our projects we intend to mainly use legacyhardware (Pentium 2 to Pentium 4 computers).
MD5 Rainbow Tables The first project is to build a cluster of 3 computers runningmoduli OS. They will have the task of distributed brute-force calculation of MD5 hashesand storing them as rainbow tables on local hard drives. We want to measure thecomputing speed in terms of hashes per second of 3 old computers, working in a clusterand compare the results to one modern computer running alone. This will be a greatpossibility to calculate a price/computing speed ratio. Also, we would like to measurethe overhead of a standard monolithic architecture by running the same MD5 calculationprogram on a standard Linux distribution. We expect to observe performance advantagesdue to high customizability similar to Exokernel [EKJ95].
The collected rainbow tables can later be used for the reverse search of passwordscorresponding to a given hash.
Virtual book Another project is the creation of a device to emulate a book as shownin figure 6.1. The open book in the middle has two blank pages and it is the projectoron the top that projects its contents. A human is able to turn over the pages by movinghis hand over the corresponding page. Two cameras (one at the bottom and one next tothe projector) are taking photos at a regular interval (e.g. 0.5 sec). The cameras and theprojector are connected to a desktop computer, running the moduli operating system.
57
Several applications are running over the microkernel – one for projecting the pages andtwo to analyze the photos from the cameras. The applications are real-time, becausethe analysis of the photos has to be accomplished within the given interval, before thenext photo comes. This will be a possibility to test moduli in a real-time environmentand to try optimizing it for giving timeliness guarantees. A similar project is SixthSense[Six10].
Figure 6.1: A virtual book device
58
7 Acknowledgements
I would like to thank Dipl-Inf. Stefan Winter for his continuous support, revisions andthe time he spent with me discussing on the topic. Further thanks to Prof. Neeraj Surifor giving me the possibility to write this thesis as well as all the staff of the DEEDSGroup at the Darmstadt University of Technology for the reviewing remarks.
59
List of Tables
4.1 Hardware configurations of the computers on which moduli was tested . . 434.2 Types of threads in moduli . . . . . . . . . . . . . . . . . . . . . . . . . . 464.3 System calls of moduli . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
60
List of Figures
3.1 A sample (and incomplete) dependency between software components ina computer system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 The organization of a traditional microkernel . . . . . . . . . . . . . . . . 223.3 A sample organization of a hierarchical microkernel system . . . . . . . . 233.4 Parents and children in the system from the viewpoint of a module and a
bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.5 An example hierarchical system . . . . . . . . . . . . . . . . . . . . . . . . 273.6 The importance of modules based on their position in the hierarchy. The
closer a module is, the more important it is considered by the operatingsystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.7 Functional dependence of modules . . . . . . . . . . . . . . . . . . . . . . 323.8 Changing the memory manager at runtime . . . . . . . . . . . . . . . . . 333.9 A database application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.10 Using different versions of the same library in parallel . . . . . . . . . . . 353.11 Hierarchical microkernels in a cluster . . . . . . . . . . . . . . . . . . . . . 40
4.1 Hierarchical scheduling in moduli . . . . . . . . . . . . . . . . . . . . . . . 464.2 Memory layout of a moduli process . . . . . . . . . . . . . . . . . . . . . . 48
6.1 A virtual book device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
61
Bibliography
[ABB+86] Mike Accetta, Robert Baron, William Bolosky, David Golub, Richard Rashid,Avadis Tevanian, and Michael Young. Mach: A new kernel foundation forunix development. pages 93–112, 1986.
[BBD+09] Andrew Baumann, Paul Barham, Pierre E. Dagand, Tim Harris, RebeccaIsaacs, Simon Peter, Timothy Roscoe, Adrian Schupbach, and Akhilesh Sing-hania. The multikernel: a new os architecture for scalable multicore systems.In SOSP ’09: Proceedings of the ACM SIGOPS 22nd symposium on Operat-ing systems principles, pages 29–44, New York, NY, USA, 2009. ACM.
[Boc10] Bochs IA-32 Emulator Project. http://bochs.sourceforge.net/, 2010.[Online; accessed 22-Sep-2010].
[Bon00] Andre B. Bondi. Characteristics of scalability and their impact on perfor-mance. In WOSP ’00: Proceedings of the 2nd international workshop onSoftware and performance, pages 195–203, New York, NY, USA, 2000. ACM.
[BRW10] Steve Balacco, Chris Rommel, and Jared Weiner. 2010service year; track 2: Embedded system engineering sur-vey data. Technical report, VDC Research, August 2010.http://newsletters.eetimes.com/portal/wts/ccmciv2cg82qecov3LqFFsqfFsjUg.
[BSP+95] Brian N. Bershad, Stefan Savage, Przemys Pardyak, Emin Gun Sirer, Marc E.Fiuczynski, David Becker, Craig Chambers, and Susan Eggers. Extensibility,safety and performance in the spin operating system. pages 267–284, 1995.
[EKJ95] Dawson R. Engler, M. Frans Kaashoek, and James O’Toole Jr. Exokernel:An operating system architecture for application-level resource management.pages 251–266, 1995.
[GP00] William Golden and Philip Powell. Towards a definition of flexibility: insearch of the holy grail? Omega, 28(4):373–384, 2000.
[Int10] Intel Corporation. Intel’s 64 and IA-32 Architectures Software Devel-oper’s Manual. Specification, 2010. http://www.intel.com/products/processor/manuals/index.htm.
[Kos10] Jussi Koskinen. Software Maintenance Costs. http://users.jyu.fi/
~koskinen/smcosts.htm, 2010. [Online; accessed 13-Sep-2010].
62
[Lie93] Jochen Liedtke. Improving ipc by kernel design. In In 14th ACM Symposiumon Operating System Principles (SOSP, pages 175–188, 1993.
[Lie95] Jochen Liedtke. On micro-kernel construction. In SOSP, pages 237–250,1995.
[Lie96] Jochen Liedtke. Towards real micro-kernels, 1996.
[LMP+05] P. Levis, S. Madden, J. Polastre, R. Szewczyk, K. Whitehouse, A. Woo,D. Gay, J. Hill, M. Welsh, E. Brewer, and D. Culler. Tinyos: An operatingsystem for sensor networks. In Werner Weber, Jan M. Rabaey, and EmileAarts, editors, Ambient Intelligence, chapter 7, pages 115–148. Springer-Verlag, Berlin/Heidelberg, 2005.
[MK97] David Mazieres and M. Frans Kaashoek. Secure applications need flexibleoperating systems. In In Proceedings of the 6th Workshop on Hot Topics inOperating Systems (HotOS-VI, pages 56–61, 1997.
[Neu97] Peter G. Neumann. UK and Y2K: 50 billion. http://catless.ncl.ac.uk/Risks/19.07.html#subj6.1, 1997. [Online; accessed 13-Sep-2010].
[Pu92] Calton Pu. Evolving operating systems and architectures: How do kernelimplementors catch up? In IEEE Computer Society, pages 143–145, 1992.
[Qem10] QEMU - open source processor emulator. http://wiki.qemu.org/, 2010.[Online; accessed 22-Sep-2010].
[RAA+91] M. Rozier, V. Abrossimov, F. Armand, I. Boule, M. Gien, M. Guille-mont, F. Herrmann, C. Kaiser, S. Langlois, P. Leonard, and W. Neuhauser.Overview of the chorus distributed operating systems. Computing Systems,1:39–69, 1991.
[SGG08] Abraham Silberschatz, Peter Baer Galvin, and Greg Gagne. Operating Sys-tem Concepts. Wiley Publishing, eight edition, 2008.
[Six10] SixthSense - integrating information with the real world. http://www.pranavmistry.com/projects/sixthsense/, 2010. [Online; accessed 5-Oct-2010].
[Tan07] Andrew S. Tanenbaum. Modern Operating Systems. Prentice Hall Press,Upper Saddle River, NJ, USA, 2007.
[Tan10] FOSDEM 2010: Andrew Tanenbaum Sets Reliability Before Performance.http://blogs.techrepublic.com.com/security/?p=4369, 2010. [Online;accessed 25-Sep-2010].
[The90] The Institute of Electrical and Eletronics Engineers. Ieee standard glossaryof software engineering terminology. IEEE Standard, September 1990.
63
[TW06] Andrew S. Tanenbaum and Albert S. Woodhull. Operating Systems Designand Implementation (3rd Edition) (Prentice Hall Software Series). PrenticeHall, January 2006.
[Win09] Stefan Winter. Choosing Error Models for OS Robustness Evaluations.Diploma thesis, Department of Computer Science, TU Darmstadt, April2009.
64