linux internals - kernel/core
TRANSCRIPT
Linux Kernel Architecture
What is Linux ?
What is the Linux Kernel ?
How does the Kernel effect system performance ?
Kernel
P
PP P P
P
HH H H
Kernel
Linux Kernel Architecture
Monolithic Kernelhttp://en.wikipedia.org/wiki/Monolithic_kernel
Supports for System Calls
Support for Loadable Modules
Implements Preemptive Multitasking
Virtual Memory / Filesystem
Demand Paging / Loading
etc..
Linux Kernel Architecture
Kernel Function Libraries
– System (/proc, generic drivers)
– Networking (TCP, sockets, drivers)
– Storage (VFS, block devices)
– Memory (VM, PM)
– Processing (scheduler, fork, exec)
– Human Interface (HI devices)
Linux Kernel Architecture
Kernel Layers
– User space interface
– Virtual subsystem
– L2V bridge / transformer
– Logical subsystem
– Hardware interface
Linux Kernel and Processes
Linux Kernel and Processes
The Linux Kernel supports pre-emptive multitasking which allows multiple processes to run individually at the same time while prioritising these with higher importance
What is a “Context Switch” ?
Linux Kernel and Processes
Context Switch
In order to support multi tasking a context switch is being handled
Context switch stores or restores the CPU state to where is 'stopped' before the last CS
Linux Kernel and Processes
Context switch can be forced by an interrupt timer in case of a low latency operation e.g: Disk I/O
Context Switch is also being used in order to handle Interrupt requests e.g: Completed read request
Linux Kernel and Processes
Context switch is being handled by the OS
Each process stack has a Process Control Block which holds the relevant meta data for the CSe.g: Program Counter, Registers Address List
Some CPUs (all x86 variants included) has a Task State Segment (TSS) which is used to store the PCB
Linux Kernel and Processes
In case a process needs to transit to Kernel mode (during a system call) a Mode Switch is being initiated
Price
Context switch is the most resource consuming task of the OS.
Linux Kernel and Processes
To review Context Switch statistics use
– 'sar -w'
– 'vmstat'
Exercise:
- Review the reciprocal relations between User / System CPU utilization and CS
Linux Kernel and Processes
To review an individual process structure use
– 'pmap'
Exercise
- Review the structure of several processes. Can you identify the process stack ?
Linux Kernel and Processes
CPU Scheduler
The Linux Kernel has a power full and high performance CPU scheduler. The CUP scheduler is designed to provide the most appropriate amount of CPU [time]slices
– Scheduling is based on a dynamic process priority rank
– Priority is changed to make sure no process is 'hogging' the CUP nor starving
Linux Kernel and Processes
CPU Scheduler
– For each CPU the scheduler maintain two priority ordered arrays for active and expired tasks. After the active one is done arrays are swapped
4
3
2
1
4
3
2
1
Linux Kernel and Processes
CPU Scheduler
– The scheduling policy involves assessing the priority for each task
– There are two classifications for tasks in terms of scheduling
• CPU bound
• I/O bound
These can be either Interactive, Batch or RT processes
Linux Kernel and Processes
CPU Scheduler
– Process pre-emption occurs, when a process receives a hardware interrupt
– Pre-emptive kernel, can ‘prefer’ a user mode process over a kernel mode process (even during an exception handling), in order to support interactivity
Linux Kernel and Processes
CPU Scheduler
–When setting ‘nice’ value to a process, one set a static portion of the scheduler priority for that task
–Exercise‘grep a /etc/services >/dev/null’
Can you ‘tune’ this command group to run faster ?
Linux Kernel and Processes
Multi Threading
Multi Threading is being used in order to have several processing objects within the same process
– Each Thread is being scheduled separately by the Kernel
– Multiple threads share the same Memory segment
Linux Kernel and Processes
Symmetric Multi Process systems can run threads of the same process simultaneously
Linux implements 1-1 (1 scheduling object per thread) threading via the Native Posix Thread Library (NTPL)* Just like Solaris and FreeBSD
Linux Kernel and Processes
To review Threads information use
– 'ps -eLf'
Exercise
– Can you identify multi threaded processes on your system ?
– Can you find any difference between PID and Thread ID ?
Inter Process Communication
IPC
IPC is an implementation of techniques to exchange data between different threads
IPC can be used internally or across a computer network
Inter Process Communication
IPC has 4 techniques
– Message passing
– Synchronization
– Shared memory
– Remote procedure call
Inter Process Communication
Message passing
Message can be a signal or data packet
Passing can be synchronous or asynchronous
Synchronisation
Used in order to keep several threads synchronized and able to provide the same / parallel service(s)
Inter Process Communication
Shared memory
Simply a shared form of memory
No control from of OS side
RPC
Method of running a procedure (subroutine) on a remote process
Inter Process Communication
IPC implementations
PIPE – pass data between hierarchy related threads
FIFO – pass data between non-related threads
Socket – pass large amount of data. Can implement TCP internally
Shared Memory (SHM) – share defined amount of storage between threads
Inter Process Communication
IPC implementations
Semaphore (SEM) – shared abstract data (like a class) used to control access on a shared resource
Inter Process Communication
IPC
Use the following commands to review IPC information
– 'ipcs' – Review SHM, SEM, MSG
– 'ipcrm' – clean up IPC objects
– 'rpcinfo' – Review RPC
– 'popen' – Create a PIPE
– 'mkfifo' – Create a FIFO pipe
*Try 'man ipc'
Inter Process Communication
IPC
Exercise
– Review IPC information on your systemAre there any SHMs ?
– Review RPC Services available on your system
– Create a named pipe. Try to use it to pass a message between two processes (e.g: bash)Can you name any advantages of using fifo ?
Linux Kernel
Kernel Tune able parameters
Changing the kernel's behaviour when working with processes, IPC, etc
Tune able parameters can be used via the '/proc' filesystem or the 'syscall' command
Linux Kernel
Kernel Tune able parameters
General parameters
– 'ctrl-alt-del': sets the kernel reaction to 'ctrl-alt-del' key sequence
– 'domainname': sets machine's domain name
– 'hostname': sets machine's host name
– 'panic': sets timer before restart after panic
– 'pid_max': sets maximum PID number
Linux Kernel
Kernel Tune able parameters
IPC parameters
– 'shmax': maximum size of shared memory segment (bytes)
– 'shmall': total amount of shared memory pages system wide
– 'shmmni': maximun number of shared memory segments system wide
Linux Kernel
Kernel Tune able parameters
IPC parameters
– 'sem': holds 4 parameters
• maximum semaphores per array
• total maximum semaphores
• maximum operations per semaphore operation call
• total maximum arrays
Linux Kernel
Kernel Tune able parameters
IPC parameters
– 'msgmni': maximun message queues system wide
– 'msgmax': maximum size of message (bytes)
– 'msgmnb': maximum size of queue (bytes)