cs4023 – operating systems (week 4) processes and concurrency dr. atif azad [email protected] 1
TRANSCRIPT
2
Acknowledgement
• Significant material in this set of lectures has been borrowed from:– http://www.cs.berkeley.edu/~kubitron/courses/cs162/.
Copyright © 2010 UCB
– http://www.cs.berkeley.edu/~kubitron/cs194-24/index_lectures.html. Copyright © 2013 UCB
– http://www-inst.eecs.berkeley.edu/~cs162/fa14 . Copyright © 2014 David Culler. Copyright ©2014 UCB.
– http://www.os-book.com . © 2014 Silberschatz et al.
– Dr Patrick Healy at CSIS Department, University of Limerick.
3
Review – Week 3• Operating System Services
– User View– Efficient Operation View (System View)
• System Calls– Provide links to Kernel Services (Resource Allocator)– Implementation– Viewing System Calls (strace)
• Policy vs Mechanisms – Policy: what needs to be done.– Mechanism: how it should be done.
• Structure of Operating Systems
FetchExec
R0…
R31F0…
F30PC
…Data1Data0
Inst237Inst236
…Inst5Inst4Inst3Inst2Inst1Inst0
Addr 0
Addr 232-1
What happens during execution?
• Execution sequence:– Fetch Instruction at PC – Decode– Execute (possibly using registers)– Write results to registers/mem– PC = Next Instruction(PC)– Repeat
PCPCPCPC
• Register: small, very fast memory for quick access to frequently used information• Program Counter (PC): A special register; holds address of current/next instruction
OS Bottom Line: Run Programs
• Program (passive) becomes alive to be a Process/job.– Load instruction and data segments of executable file into
memory– Create data structures in memory (stack and heap)– “Transfer control to it”– Provide services to it– While protecting OS and it
• One program can be several processes?– Consider multiple users executing the same program
int main() { … ; }
edito
r
com
pile
r
Program SourceExecutable
foo.c a.out
Load
&
Exec
ute
0x000…
0xFFF…
instructions
data
instructions
data
heap
stack
Memory
Processor
registers
PC:
OS
(passive) Program Process (active)
Process
Processorregisters
PC:
0x000…
0xFFF…
Code Segment
Static Data
heap
int i; flot w; chr *x stack
instruction
SP:
Pro
gra
m A
dd
ress
Sp
ace int array[6]; //Global data
float globalF=3.0f;
int main(){ int i=20; //local data float w=2; //dynamic allocation char* x=malloc( i ); //like new Object(); func(); free(x);}
void func(int a){ char c=‘a’; printf(“func”);}
• What’s in the code segment? Data?• What’s in the heap segment?
– How is it allocated? How big?• What’s in the stack segment?
– How is it allocated? How big is it?• What if an instruction accesses data outside heap/stack?
int a; char c;
Stack and Heap
Processorregisters
PC:
0x000…
0xFFF…
Code Segment
Static Data
heap
int i; flot w; chr *x stack:
instruction
SP:
Pro
gra
m A
dd
ress
Sp
ace int array[6]; //Global data
float globalF=3.0f;
int main(){ int i=20; //local data float w=2; //dynamic allocation char* x=malloc( i ); //like new Object(); func(); free(x);}
void func(int a){ char c=‘a’; printf(“func”);}
• When func exits, stack decreases.• When main exits, stack finishes.
– Heap still has memory allocated– But nothing points to it. – Needs to be free()d– Otherwise, heap may run out.
int a; char c;
free(x)
Recall Key Concept: Address Space• Program operates in an address space that is distinct
from the physical memory space of the machine
Processor Memory
0x000…
0xFFF…
translator
“virt
ual a
ddre
ss”
“phy
sical
add
ress
”
Example of Address Translation
Prog 1Virtual
AddressSpace 1
Prog 2Virtual
AddressSpace 2
CodeDataHeapStack
CodeDataHeapStack
Data 2
Stack 1
Heap 1
OS heap & Stacks
Code 1
Stack 2
Data 1
Heap 2
Code 2
OS code
OS dataTranslation Map 1 Translation Map 2
Physical Address Space
Q: Still vulnerable?
A simple address translation: Base & Bound
• Can the pgm touch OS?• Can it touch other pgms?9/3/14
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
FFFF…
1000…
0000…
Programaddress
Base Address
Bound <
1000…
1100…1100…
Concurrency• “Thread” of execution
– Independent Fetch/Decode/Execute loop– Operating in some Address space– Single-Threaded processes for this lecture. (also called “heavy
weight” process) • Uniprogramming: one thread at a time
– MS/DOS, early Macintosh, Batch processing– Easier for operating system builder– Get rid of concurrency by defining it away– Does this make sense for personal computers?
• Multiprogramming: more than one thread at a time– Multics, UNIX/Linux, OS/2, Windows NT/2000/XP, Mac OS X
• ManyCore Multiprogramming, right?
The Basic Problem of Concurrency• The basic problem of concurrency involves resources:
– Hardware: single CPU, single DRAM, single I/O devices– Multiprogramming API: users think they have exclusive access to
shared resources• OS Has to coordinate all activity
– Multiple users, I/O interrupts, …– How can it keep all these things straight?
• Basic Idea: – abstract the notion of an executing program (process representation)– Then, worry about multiplexing these abstract beings
• Dijkstra did this for the “THE system”– Few thousand lines vs 1 million lines in OS 360 (1K bugs)
Multiprogramming – Many Processes
OS
Proc 1
Proc 2
Proc n…
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
1000…
1100…
3000…
3080…
Base 1000 …
1100…Bound
regs
sysmode
…
1
PC
0000…
FFFF…
00FF…
0000 1234
xxxx…uPC
Multiprogramming – Interrupt (kernel code)
OS
Proc 1
Proc 2
Proc n…
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
1000…
1100…
3000…
3080…
Base 1000 …
1100 …Bound
0000 1234uPC
regs
sysmode
…
0
PC
0000…
FFFF…
00FF…• How to save registers
and set up system stack?
IntrpVector[i]
Multiprogramming – Process 2
15
OS
Proc 1
Proc 2
Proc n…
code
Static Data
heap
stack
code
Static Data
heap
stack
code
Static Data
heap
stack
0000…
1000…
1100…
3000…
3080…
Base 3000 …
0080 …Bound
xxxx xxxxuPC
regs
sysmode
…
1
PC
0000…
FFFF…
00D0…• How to save registers
and set up system stack?
000 0248
1000 …
1100 …
0000 1234
regs
00FF…
Concurrency: illusion of many processorsCPU3CPU2CPU1
Shared Memory
• Concurrency provides the illusion of multiple processors?– Multiplex in time!
• Each process is like a virtual “CPU”; needs a structure to hold:– Program Counter (PC), Stack Pointer (SP)– Registers (Integer, Floating point, others…?)
• Q: What structure holds this information: PCB. (coming next)• How switch from one CPU to the next?
– Save PC, SP, and registers in current state block– Load PC, SP, and registers from new state block
• What triggers switch?– Timer, voluntary yield, I/O, other things
• Q: what states can a process go to after a switch?
CPU1 CPU2 CPU3 CPU1 CPU2
Time
Diagram of Process State
• As a process executes, it changes state– new: The process is being created– ready: The process is waiting to run– running: Instructions are being executed– waiting: Process waiting for some event to occur– terminated: The process has finished execution
ProcessControlBlock
Multiplexing with Process Control Blocks (PCB)• The current state of process held in a
process control block (PCB):– This is a “snapshot” of the execution and
protection environment– Only one PCB active at a time
• PCB also notes multiplexing process info :– CPU time to different processes
(Scheduling):• Only one process “running” at a time• Give more time to important processes
– Resources held by Process):• Memory Mapping: based and bound registers• I/O info: opened files, network connections etc.
Process Representation in LinuxRepresented by the C structure task_struct
pid t_pid; /* process identifier */ long state; /* state of the process */ unsigned int time_slice /* scheduling information */ struct task_struct *parent; /* this process’s parent */ struct list_head children; /* this process’s children */ struct files_struct *files; /* list of open files */ struct mm_struct *mm; /* address space of this process */
CPU Switch From Process to Process
• This is also called a “context switch”• Code executed in kernel above is overhead
– Overhead sets minimum practical switching time
21
Context Switch in Detail
• A context switch is the switching of the CPU from one process or thread to another.
• In detail, kernel does the following:– Suspend one process and store its state as a PCB– retrieve the PCB for the next process from memory and restore it in
the CPU’s registers– Start executing the new process as pointed by its program counter
(PC)– The more complex the OS and the PCB the longer the context
switch• Time dependent on hardware support
– Some hardware provides multiple sets of registers per CPU multiple contexts loaded at once
Operations on Processes
• Operating systems must provide mechanisms for:– process creation,– process termination,
Process Creation• Parent process create children processes,
which, in turn create other processes, forming a tree of processes
• Generally, process identified and managed via a process identifier (pid)
• Resource sharing options– Parent and children share all resources– Children share subset of parent’s resources– Parent and child share no resources
• Execution options– Parent and children execute concurrently– Parent waits until children terminate
A Tree of Processes in Linux
• Run pstree command. Can’t see init? Instead see systemd? • http
://www.pcworld.com/article/2841873/meet-systemd-the-controversial-project-taking-over-a-linux-distro-near-you.html
Process Creation (Cont.)• Address space
– Child duplicate of parent– Child has a program loaded into it
• UNIX examples– fork() system call creates new process– exec() system call used after a fork() to
replace the process’ memory space with a new program
C Program Forking Separate Process
Parent Process Child Process
??
Process Termination
• Process executes last statement and then asks the operating system to delete it using the exit() system call.– Returns status data from child to parent (via wait())– Process’ resources are deallocated by operating system
• Parent may terminate the execution of children processes using the abort() system call. Some reasons for doing so:– Child has exceeded allocated resources– Task assigned to child is no longer required– The parent is exiting and the operating systems does not
allow a child to continue if its parent terminates
Process Termination
• Some operating systems do not allow child to exist if its parent has terminated. If a process terminates, then all its children must also be terminated.– cascading termination. All children, grandchildren, etc. are
terminated.– The termination is initiated by the operating system.
• The parent process may wait for termination of a child process by using the wait()system call. The call returns status information and the pid of the terminated process
• pid = wait(&status); • If no parent waiting (did not invoke wait()) process is a zombie• If parent terminated without invoking wait , process is an
orphan
Process Scheduling• Maximize CPU use, quickly switch processes onto CPU for time sharing• Process scheduler selects among available processes for next
execution on CPU• Maintains scheduling queues of processes
– Job queue – set of all processes in the system– Ready queue – set of all processes residing in main memory, ready and
waiting to execute– Device queues – set of processes waiting for an I/O device– Processes migrate among the various queues
• Short-term scheduler (or CPU scheduler) – selects which process should be executed next and allocates CPU
– The processes are ready to execute. – Scheduler should be fast.
• Long-term scheduler (or job scheduler) – selects which processes should be brought into the ready queue
– Long-term scheduler is invoked infrequently (seconds, minutes) (may be slow)
– The long-term scheduler controls the degree of multiprogramming
Ready Queue And Various I/O Device Queues
Process Scheduling
• PCBs move from queue to queue as they change state– Decisions about which order to remove from queues are
Scheduling decisions– Many algorithms possible (few weeks from now)
Multiprocess Architecture – Chrome Browser
• Many web browsers ran as single process (some still do)– If one web site causes trouble, entire browser can hang or
crash• Google Chrome Browser is multiprocess with 3 different
types of processes: – Browser process manages user interface, disk and network
I/O– Renderer process renders web pages, deals with HTML,
Javascript. A new renderer created for each website opened• Runs in sandbox restricting disk and network I/O, minimizing effect
of security exploits
– Plug-in process for each type of plug-in
Interprocess Communication• Processes within a system may be independent or cooperating• Cooperating process can affect or be affected by other processes,
including sharing data• Reasons for cooperating processes:
– Information sharing– Computation speedup– Modularity– Convenience
• Cooperating processes need interprocess communication (IPC)• Two models of IPC
– Shared memory– Message passing
Communications Models
(a) Message passing. (b) shared memory.
Shared Memory: Producer-Consumer Metaphor
• Paradigm for cooperating processes, producer process produces information that is consumed by a consumer process– unbounded-buffer places no practical limit on
the size of the buffer– bounded-buffer assumes that there is a fixed
buffer size
Bounded-Buffer – Shared-Memory Solution• Shared data
#define BUFFER_SIZE 10typedef struct {. . .
} item;
item buffer[BUFFER_SIZE];int in = 0;int out = 0;
item next_produced; while (true) {
/* produce an item in next produced */ while (((in + 1) % BUFFER_SIZE) == out) ; /* do nothing */ buffer[in] = next_produced; in = (in + 1) % BUFFER_SIZE;
}
• Solution is correct, but can only use BUFFER_SIZE-1 elements
Bounded Buffer – Consumer
item next_consumed; while (true) {
while (in == out) ; /* do nothing */
next_consumed = buffer[out]; out = (out + 1) % BUFFER_SIZE;
/* consume the item in next consumed */ }
Interprocess Communication – Shared Memory
• An area of memory shared among the processes that wish to communicate
• The communication is under the control of the users processes, not the operating system.
• Major issues is to provide mechanism that will allow the user processes to synchronize their actions when they access shared memory.
• Synchronization is discussed in great details in Chapter 5 of SGG.
IPC POSIX Producer
IPC POSIX Consumer