operating systems (2inc0) 2019/20 introduction (01)

Operating Systems (2INC0)2019/20

Introduction (01)

Dr. Tanir OzcelebiCourtesy of Prof. Dr. Johan Lukkien

System Architecture and Networking Group

Tanir Ozcelebi

Course Overview

• Introduction to operating systems

• Processes, threads and scheduling

• Concurrency• atomicity and interference• action synchronization• condition synchronization • deadlock

• Memory management

• Input/output• general issues • file systems

213-Nov-19

Tanir Ozcelebi

Agenda

• Course overview• OS: place in the computer system • Some common notions• Motivation & OS tasks• Extra-functional requirements

313-Nov-19

Tanir Ozcelebi

Agenda


413-Nov-19

Tanir Ozcelebi

Elements of a computer system

513-Nov-19

Images from Techstereo

Users• humans• machines

Software• application programs• OS, system programs

Hardware• CPU, memory (storage), battery• input/output devices

USER

DATA & APPs

OS

HW

Tanir Ozcelebi

Computer system hardware

• HW operates at a very low abstraction level compared to user programs.• A mediator in between is needed.

• HW resources are shared among running programs (in time / in space). • A burden that must be taken away from the programmer.

• HW can be expensive.• Use of HW resources needs to be optimized.

613-Nov-19

Bus connection

Tanir Ozcelebi

Operating system (not a formal definition):

• …is a piece of software that acts as an intermediary between users/applications and computer hardware.

• …provides a level of abstraction that hides the gory details of the HW architecture from applications.

• …organizes sharing of HW resources among applications and users (virtualization),

• …maximizes computer performance (resource management).

713-Nov-19

OS kernel: The one program that runs at all times on a computer.

Everything else is either a system program or an application program.

Tanir Ozcelebi

OS: place in computer system

813-Nov-19

Chrome Dropbox Outlook

Tanir Ozcelebi

How a computer works…

• CPU and I/O devices execute concurrently• I/O: data transfer between device and the

local buffer of device controller− CPU does other things in the meantime

• Device controller (hardware) informs CPU that the device is ready for I/O by means of an interrupt

• CPU moves data between MM (main memory) and these local buffers

913-Nov-19

Tanir Ozcelebi

How a computer works…

• CPU and I/O devices execute concurrently• I/O: data transfer between device and the

local buffer of device controller− CPU does other things in the meantime

• Device controller (hardware) informs CPU that the device is ready for I/O by means of an interrupt

• CPU moves data between MM (main memory) and these local buffers

• Alternative: Direct Memory Access (DMA)

1013-Nov-19

Tanir Ozcelebi

OS’s are interrupt driven

• An interrupt is a signal to the CPU that transfers control to an interrupt service routine• caused by hardware (e.g. a result of clock

tick, ready for i/o, i/o completion)• interrupt vector contains address of the

service routine• OS preserves CPU state by storing registers

and program counter (address of the interrupted instruction is saved)

• A trap or exception is a software-generated interrupt • caused either by a software error (e.g.

division by zero, floating pt error) • or by a system call (explicit SuperVisor Call

or SVC), in line with control flow of program

1113-Nov-19

Tanir Ozcelebi

OS: Dual-mode operation

• Different privileges required for different types of code• Most instructions (user code) can be executed in user mode.

− In user mode CPU cannot access memory locations reserved for OS.• Some “privileged” instructions are only executable in kernel mode.

• Dual-mode allows OS to protect itself and other system components• Mode bit provided by hardware

− Provides ability to distinguish when system is running user code or kernel code− System call changes mode to kernel, return from call resets it to user

1213-Nov-19

Tanir Ozcelebi

OS: place in computer system

1313-Nov-19

separationbetween kerneland user mode

Chrome Dropbox Outlook“Privileged” instructions are only executable inside the kernel (i.e. in kernel mode)!

Tanir Ozcelebi

Example interaction with OS: read data from file

1413-Nov-19

• Interaction is named a ‘system call’• Execute C language statement:

status = read(fd, buffer, nbytes);• is part of a program running on top of the OS• read nbytes bytes from fd, storing it in buffer

• memory: kernel space/user space• kernel space only accessible with

processor in kernel mode• parameters: either via registers or

via memory

• 1-3: pushing parameters• 4: call library function read• 5: put code for read in reg.• 6: trap (Linux@x86: int 0x80): switch mode & call trap handler• 7: handler calls read function handler• 8: handler performs read actions (store data

at address). Here suspension of the calling process may occur if data needs to come froman i/o device.

• 9-11: give control back to caller

picture from slide by A.S. Tanenbaum

suspensionpossible

Tanir Ozcelebi

status = read(fd, buffer, nbytes);

• The filepointer, fd, refers to a data structure (probably within the kernel space) that stores• information where the file is to be found• current access state (particularly, the read position)• .....

• The requested data can be found in system buffers that store disk blocks• already available or needing disk access

• Suspension of calling process: Copying data from disk to these buffers is by hardware read requests that• specify disk block and destination buffer• issue the disk operation and wait for completion interrupt

• The file system may do look-aheads concurrently with user activities

• When this data is not available in system buffers at the time of reading, this results in the suspension of the calling process. Similar suspension occurs when the process has consumed its alotted time.

1513-Nov-19

0

Tanir Ozcelebi

Agenda


1613-Nov-19

Tanir Ozcelebi

Policy vs mechanism

• Policy: defines what you want a system (e.g. an OS) to do• Mechanism: defines how to do it

Policies need appropriate mechanisms for their realization.

• Examples:• policy: increase home-ownership

− mechanism: tax deduction• policy: execute first the task with the earliest deadline

− mechanism: priority based scheduling & pre-emption

1713-Nov-19

Tanir Ozcelebi

Transparency

• Hide details with respect to a given issue.

• Examples:• processor architecture (Instruction Set Architecture - ISA)

− mechanism: use compiler

• physical memory size− mechanism: virtual memory

• physical memory location− mechanism: indirection using logical memory address

1813-Nov-19

Tanir Ozcelebi

Virtualization

1913-Nov-19

• Provides a simple, abstract, logical model of the system• virtual memory, virtual CPU, virtual disk

• Current systems virtualize the entire hardware• virtual machine

Main Memory

Tanir Ozcelebi

Layering and virtualization

• An OS layer with a well-defined API yields a virtual machine• for a programmer, the virtual machine is what he sees• porting programs amounts to porting such virtual machines

• Virtualization can support several OS’s on top of one OS (right)• Implemented on a run-time virtualization layer (e.g. Vmware, VirtualBox)

• This virtualization can go down to immediately above the hardware (left)

2013-Nov-19

Tanir Ozcelebi

Other examples of virtual machines

2113-Nov-19

compilationcompilation

compCommon Language Runtime

Java virtual machine

.NET Framework

Tanir Ozcelebi

Agenda


2213-Nov-19

Tanir Ozcelebi

OS Motivation: Deal with diversity

2313-Nov-19

• CPU and main memory are required for basic operation

• There are many alternatives• diverse CPUs

− Instruction Set Architectures

• diverse architectures− (a) multi processor with shared memory (e.g.

current multi-core systems)− (b) multi computer with private memory and

shared file system (e.g. cluster)− (c) independent computers on a network (e.g.

cluster, but also, internet-connected machines)

Tanir Ozcelebi

OS Motivation: Transparency and Virtualization

• OS task:• e.g. hide inner workings and details of

processing platform− Transparency: Processor architecture,

# processors, particular devices− note: also compiler plays a role in this

• e.g. abstract from the complicated memory hierarchy and physical limitations − Virtualization: present linear memory model

that is larger than physical− called: memory virtualization

2413-Nov-19

registers

cache

main memory

solid-state disk

magnetic disk

optical disk

magnetic tapes

Tanir Ozcelebi

OS Motivation: Shared functionality

• User interface (UI)• e.g. Command-Line (CLI), Graphical User

Interface (GUI), Batch

• Program execution • load a program into MM and run• end execution, either normally or abnormally

(indicating error)

• I/O operations• File-system management• Communication between processes

• e.g. via shared memory or through message passing (packets)

• Error detection• Debugging facilities

2513-Nov-19

• Resource sharing & allocation – CPU cycles, main memory, file

storage, I/O devices (using timers)• Accounting

– Who’s using which resources for how long?

• Protection – concurrent processes should not

interfere with each other (memory)• Security

– of the system against running processes and outsiders

Large collection of services needed by virtually all programs...

Tanir Ozcelebi

OS Motivation: Shared functionality

• OS task: provide functionality common to most programs• introduce well-defined abstractions of concepts

− files and filesystems instead of disk blocks− exceptions and traps rather than ‘something goes wrong’− linear memory rather than memory blocks, pages and disk space

• provide system calls for these functions provided by the OS

2613-Nov-19

Tanir Ozcelebi

OS shared functionality: System calls

• Programming interface to the services provided by the OS,…• …typically written in a high-level language (C or C++)

2713-Nov-19

Tanir Ozcelebi

OS shared functionality: System calls

• System calls are mostly accessed by programs via a high-level

Application Program Interface (API) rather than direct system

call use

• Win32 API for Windows, POSIX API for POSIX-based systems (including

virtually all versions of UNIX, Linux, and Mac OS X),

• Java API for the Java Virtual Machine (JVM),

• Cocoa Touch for iOS, Java based Android API for Android (runs on top of

Android Runtime – previously on top of Dalvik VM).

Why are API’s needed?

1) direct system calls are more difficult to work with than an API

2) system calls can be very frequent

3) a program written using an API can compile/run on any system supporting

the API.

2813-Nov-19

Tanir Ozcelebi

Types of system calls

• Process management• create, destroy, communication, synchronization, ...

• File management• open, close, read, write, ...

• Memory management• allocation, free, virtual memory, ...

• Device management• access control, open, attach, send/receive, ....

• Communications• setup communications, exchange messages, ....

• Miscellaneous• timers, inspect system resources, ...

2913-Nov-19

Tanir Ozcelebi

Example of system calls

• System call sequence to copy the contents of one file to another file

3013-Nov-19

Tanir Ozcelebi

OS Motivation: Concurrency

• The machine must be shared: multiple activities (‘tasks’, ‘processes’)

& multiple users

• devices and processor(s) operate concurrently (unavoidable)

• many common OS tasks are always running; termination is rare

− efficiency: utilize waiting times

• OS task:

• realize concurrency transparency (virtualization)

− each task virtually has a (virtual) machine of its own

• manage and protect (enforce) resource usage limits between tasks and between users

− resources: processor, memory, i/o equipment, keyboard, mouse,

screen, disks, network and other communication facilities, ....

3113-Nov-19

Tanir Ozcelebi

OS Motivation: Portability

• Protect investments in application software• support source code portability

• OS task (or design criterion)• give a unified machine view to applications

− i.e., a portable view, a good abstraction of commonly used architectures

• standardize on the API – effectively, define a virtual machine− POSIX (Portable Operating System Interface for Unix) initiative

3213-Nov-19

Tanir Ozcelebi

Provides standard interfaces to API (device capabilities).

System apps are those that come with the device. Default apps can

be changed after purchase.

Java API for creating apps.

An app runs its own process and with its own ART VM.

Does not use JIT compile like Dalvik VM used to do. Instead translatesDalvik bytecode to machine instructions at installation time: faster and more energy efficient at runtime!

Libraries needed by some core system components and services.

Example:

[image from

developer.android.com]

3313-Nov-19

Tanir Ozcelebi

Agenda

• Course overview• OS: place in the system • Some commons notions• Motivation & OS tasks• Extra-functional requirements

3413-Nov-19

Tanir Ozcelebi

OS extra-functional requirements

• Efficiency• the sacrificed efficiency (of having an OS rather than direct access)

should be reasonable− applications must be able to obtain close-to-optimal machine use− “tweak-able” control by programmer

− example: provide raw disk access as well as file systems

• rule of thumb: − if a (new) function can be implemented with the available ones, don’t provide it(unless this indirect implementation needs to sacrifice an unreasonable amount of performance)

3513-Nov-19

Tanir Ozcelebi

OS extra-functional requirements

• Extensibility • support for adding application-specific (domain-specific) functionality

• Scalability• wide range of environments, functionalities, machines

• Dependability • robust, correct, safe & secure• How dependable? à depends… on the application domain

3613-Nov-19

Tanir Ozcelebi

Domain-specific requirements

• Real-time OS• predictability

− known performance instead of high performance (all resources)− maximum latencies (response times) of API calls

• support for dealing with real-time control− (pre-emptive) scheduling policies− explicit control over resources− real-time facilities: clocks and timers

• stringent dependability

• Embedded OS• small footprint (e.g., leave all superfluous parts out)• low system requirements (e.g. processor speed, energy)• stringent dependability

3713-Nov-19

Tanir Ozcelebi

Summarizing - OS views

• Abstraction• provide useful generic concepts• .... to handle complexity

• Virtualization• provide the same abstract model for a wide range of systems• help in sharing the system

− each process/user sees single machine, linear memory

• Resource management• resource sharing, protection• optimized performance• accounting and access control

3813-Nov-19

operating systems (2inc0) 2019/20 introduction (01)

Documents