linux system programmingcfs3.tistory.com/upload_control/download.blog?fhandle... · unix system...

Unix System Programming

UNIX (Linux) System Programming

2Unix System Programming

References

Advanced Programming in the UNIX Environment Linux manual pages Understanding the Linux Kernel


Contents 1. Overview 2. UNIX Standards and Implementations 3. File I/O 4. Files and Directories 5. Standard I/O Library 6. System Data Files and Information 7. Environment of a Process 8. Process Control 9. Process Relationships 10. Signal


1. Overview


Role of the Linux Operating System

virtual machine operates on the physical machine which contains the low

level programming interface provides the high-level abstractions as well as a more

advanced programming and user interface interfaces between applications and the machine

applications

kernel

hardware



sharing the processor Linux system operates in multi-tasking mode

run several programs(processes) at the same time system implements a scheduling system

scheduling system connects each of the processes in turn to the processor.

by cycling the processes rapidly, machine users have the impression that the processes are being carried out in parallel.



memory management in a multi-user and multi-tasking computer, very rigorous

control of memory is necessary. each process has its own address space. the kernel must be capable of efficiently controlling memory in

order to meet the demands of the different processes as quickly as possible.

zones of memory allocated to different processes are protected so as to prevent unauthorized modifications



memory management (cont’d) physical memory is typically insufficient

the system uses part of the disk as auxiliary memory(swap area)



resource manager the processor and the memory are particular instances of

resource provides processes with an interface allowing resources

(such as disks, printers, etc.) to be shared prevent conflicts of access

implements a protection system which allow users and system administrators to protect access to their data



communication hub of the machine the OS should manage different events

events arising from either the physical layer (interrupts), or from applications(system calls)

the system must deal with the events if necessary send them to the processes concerned.

the system must be able to put several processes in communication with each other

signal, message queue, semaphore, shared memory, pipe, socket


System Structureprocesses

System calls interface

File systems Central kernelext2fs, proc,

minix, ...task manage, schedulerIPC, memory manage

machine

machine interface

Network manageripv4, ethernet, ...

peripheral managerblock character

buffer cache


Kernel mode & User mode

kernel mode privileged mode no restriction is imposed on the kernel of the system may use all the instructions of the processor manipulate the whole of the memory talk directly to the peripheral controllers


Kernel mode & User mode

user mode normal execution mode for a process has no privileges

certain instructions are forbidden only allowed to zones allocated to it cannot interact with the physical machine

process carries out operations in its own environment, without interfering with other processes

process may be interrupted at any moment


System Calls

system call system call is a request transmitted from the process to the

kernel process in user mode cannot directly access the machine

resources the kernel deals with the request in kernel mode, without

any restrictions, and sends the result to the process trap instruction

causes the CPU to switch into kernel mode Intel CPU: int 0x80


System Calls

wrapper routine

#define _syscall0(type,name) \type name(void) \{ \long __res; \__asm__ volatile ("int $0x80" \ : "=a" (__res) \ : "0" (__NR_##name)); \__syscall_return(type,__res); \}


System Calls

syscall( ) int syscall(int syscall_number, arg0, arg1, ...);

#include <asm/unistd.h>main(){ int p1, p2;

p1 = syscall(__NR_getpid); p2 = getpid();

printf("syscall: %d\n", p1); printf("getpid: %d\n", p2);}


Structure of the Ext2 file system

Block group 0 Block group 1 ... Block group nBootblock

Superblock

Groupdescriptors

Blockbitmap

Inodebitmap

Inodetable Data blocks

partition 2 partition 3

hard disk

Ext2 file system

block group

partition 1


Structure of the Ext2 file system

boot block contains bootstrap code

the machine code needed for loading the kernel when the system is started up

block group a file system is divided into a number of block groups reduce the disk seek time (exploit locality)

keep data blocks close to their inodes keep file inodes close to their directory inode


Block Group

superblock contains the control information of the file system

number of inodes and blocks number of free inodes and blocks block size blocks per group inodes per group

duplicated in each group of blocks enable the corruption of the file system to be easily overcome


Block Group

group descriptor contains control information of a block group

block number of the inode bitmap block number of the block bitmap block number of the first block of inode table the number of free inodes and free blocks the number of directories in this block group

used by the inode allocation algorithm for the directories attempts to spread directories as evenly as possible over

the block groups


Block Group

block bitmap block allocation status a bit indicates whether the block is allocated or available

inode bitmap inode allocation status a bit indicates whether the inode is allocated or available


Block Group

inode table contains a part of a table of inodes of the file system

data blocks used to store data contained in files and directories also used to store indirect blocks


Inode

inode internal representation of a file every file has one inode inode contains

type/permission user(UID), group(GID) file size, number of blocks, link counter access time, modification time, change time list of addresses of data blocks


Inode and Data Blocks

direct 0

direct 1

direct 2

direct 9

direct 10

direct 11

double indirect

triple indirect

single indirect

Address List in Inode Data Blocks


File and Directory Blocks

inodetable

datablock

datablock

directoryblock

datablock

inodenumber filename

inode


Ext2 Directory Entry

2 12 1 . 2017 20 10 lost+found

0 4 6 8 12 16 18 20 32

struct ext2_dir_entry {unsigned long inode; /* inode number */unsigned short rec_len; /* length of directory entry */unsigned short name_len; /* length of filename */char name[EXT2_NAME_LEN]; /* filename */

};


Kernel Data Structure for Open Files

fd flag ptrfd 0fd 1fd 2

user file descriptor table file status flagcurrent file offset

inode ptr

file status flagcurrent file offset

inode ptr

file tableinode table



user file descriptor table allocated per process identifies all open files for a process when a process “open” or “creat” a file, the kernel allocates

an entry return value of “open” and “creat” is the index into the user

file descriptor table contains pointer to file table entry



file table global kernel structure contains the description of all open files in the system

file status flag (open mode) current file offset

contains pointer to in-core inode table entry



in-core inode table global kernel structure when a process opens a file, the kernel converts the

filename into an identity pair(device number, inode number) the kernel then loads the corresponding inode into in-core

inode table


Idea of Process

process an instance of a program in execution

at any given time a single instruction is carried out within the process

processes are often called “tasks” in Linux source code. process descriptor

task_struct contains all the information related to a single process


Process Descriptor

state flagsprioritynext_taskprev_tasknext_runprev_runp_pptrp_optr...ttyfsfilesmmsig...

tty_struct

fs_struct

files_struct

mm_struct

signal_struct

tty associated with the process

current directory

pointers to file descriptors

pointers to memory area descriptors

signals received

task_struct


Process States

process states state of a process is defined by its current activity executing

the process is being executed by the processor ready

the process could be executed, but another process is currently being executed


Process States

process states (cont’d) suspended

the process is suspended (sleeping) until some condition becomes true.

hardware interrupt releasing a system resource the process is waiting for delivering a signals, etc.


Process States

process states (cont’d) stopped

process execution has been stopped caused by signals

SIGSTOP, SIGTSTP, SIGTTIN, SIGTTOU zombie

the process has finished execution, but it is still referenced in the system


Process States

stopped

suspended

ready executing zombie

creation

scheduling

signal signal

termination

input/outputend ofinput/output


Attributes of a Process

attributes state identification (unique number) values of the registers, including the program counter user identity under whose name the process is executing information used by the kernel to establish the schedule of

the processes priority, execution time


Attributes of a Process

attributes (cont’d) information concerning the address space of the process

segments for the code, data, stack information concerning the inputs/outputs carried out by the

process descriptions of open files, current directory, ...

information summarizing resources used by the process


Login

/etc/passwd login name:password:UID:GID:comment:home directory:shell

Shell : command interpreter Bourne shell: /bin/sh C shell: /bin/csh Korn shell: /bin/ksh Tenex shell: /bin/tcsh

File and directory file system directory file name


User Identity

user identifiers real user id

identifier of the user who started up the process effective user id

identifier which is used by the system for access control can be different from the real user, especially in the case of

programs with the setuid bit set


User Identity

user identifiers (cont’d) saved user id

effective user id is saved when a process executes a new program

saved user id is used to authorize the modification of the effective user id

setuid( ), seteuid( ), setreuid( )


User Identity

user identifiers (cont’d) file system user id

used to check for all accesses to the file system normally equal to the effective user ID whenever the effective user id is changed, file system user id

will also be changed to new value of effective user id effective user id is not changed when file system user id is

changed


User Identity

group identifiers real group id

identifier of the primary group of the user who started the process

effective group id identifier which is used by the system for access control can be different from the real group, especially in the case of

programs with the setgid bit set


User Identity

group identifiers (cont’d) a list of group identifiers

any user can belong to several groups simultaneously the kernel keeps a list of groups associated with each process

in order to carry out access control process can possess up to 32 groups

saved group id effective group id is saved when a process executes a new

program


Shell

Shell Command line interpreter Reads and executes user command, and then shows the

result Bourne shell: /bin/sh C shell: /bin/csh Korn shell: /bin/ksh Tenex shell: /bin/tcsh


File and Directory File System, File name, and Pathname Working Directory, Home Directory Ex : myls.c

#include <sys/types.h>#include <dirent.h>#include “ourhdr.h”

int main(int argc, char *argv[]){ DIR *dp; struct dirent *dirp;

if (argc != 2) err_quit(“\”the directory name\” is required”);

if ((dp == opendir(argv[1])) == NULL) err_sys(“ca’t open %d”, argv[1]); while ((dirp = readdir(dp)) != NULL) printf(“%s\n”, dirp->d_name); closedir(dp); exit(0);}


How to build and run myls.c

# cc myls.c - or - #gcc myls.c# a.out /dev

# cc –o myls myls.c - or - # gcc –o myls myls.c# myls /dev


Input and Output stdin, stdout, stderr Unbuffered I/O and

standard I/O(buffered I/O)

# ls -vs-# ls > file.list

#include “ourhdr.h”

#define BUFFSIZE 8192

int main(void){ int n; char buf[BUFFSIZE];

while ((n = read(STDIN_FILENO, buf, BUFFSIZE)) > 0) if (write(STDOUT_FILENO, buf, n) != n) err_sys(“write error”);

if (n < 0) err_sys(“read error”); exit(0);}


Input and Output

Another example


int main(void){ int c; while ((c = getc(stdin)) != EOF) if (putc(c, stdout) == EOF) err_sys(“output error”); if (ferror(stdin)) err_sys(“input error”); exit(0);}


Program and Process

Process ID


int main(void){ printf(“hello world from process ID %d\n”, getpid()); exit(0);}

# a.outhello world from process ID 851# a.outhello world from process ID 854


Process control fork(), exec(), waitpid()

#include <sys/types.h>#include <sys/wait.h>#include “ourhdr.h”

int main(void){ char buf[MAXLINE]; pid_t pid; int status;

printf(“%% “);

while (fgets(buf, MAXLINE, stdin) != NULL) { buf[strlen(buf) -1] = 0; if ((pid = fork()) < 0) err_sys(“form error”); else if (pid == 0) { execlp(buf, buf, (char *) 0); err_ret(“couldn’t execute: %s”, buf); exit(127); } if ((pid = waitpid(pid, &status, 0)) < 0) err_sys(“waitpid error”); printf(“%% “); }}


ANSI C

Functionsssize_t read(int, void *, size_t);ssize_t write(int, const void *, size_t);pid_t getpid(void);

Generic pointer void * : char * is used in early UNIX system.

Primitive System Data Types


Error handling• Related Variable and Functions

extern int errno;

#include <string.h> char *strerror(int errnum);

#include <stdio.h> void perror(const char *msg);

#include <error.h>#include “ourhdr.h”

Int main(int argc, char *argv[]){ fprintf(stderr, “EACCES: %s\n”, strerror(EACCES)); errno = ENOENT; perror(argv[0]); exit(0);}


User Identification

UserID, GroupID


Int main(void){ printf(“uid = %d, gid = %d\n”, getuid(), getgid()); exit(0);}


Signal Signal Actions

Ignore Default User specified function

#include <sys/types.h>#include <sys/wait.h>#include “ourhdr.h”

int main(void){ char buf[MAXLINE]; pid_t pid; int status;

if ((signal(SIGINT, sig_int) == SIG_ERR) err_sys(“signal error”); printf(“%% “);

while (fgets(buf, MAXLINE, stdin) != NULL) { buf[strlen(buf) -1] = 0; if ((pid = fork()) < 0) err_sys(“form error”); else if (pid == 0) { execlp(buf, buf, (char *) 0); err_ret(“couldn’t execute: %s”, buf); exit(127); } if ((pid = waitpid(pid, &status, 0)) < 0) err_sys(“waitpid error”); printf(“%% “); }}void sig_int(int signo){ printf(“interrupt\n%% ”); }


UNIX time

Calendar time Time elapsed since 1970/1/1 00h00m System data type : time_t Ex) file modification time, …

Process time (clock time, user CPU time, system CPU time) System data type : clock_t Typically 50Hz, 60Hz, 100Hz

$ cd /usr/include$ time grep _POSIX_SOURCE */*.h > /dev/null

real 0m19.81suser 0m0.43ssys 0m4.53s


System call

Application

code

Memory allocation function malloc

sbrk

system call

KERNEL

USER PROCESS


System call

Application

code

C library

functions

System calls

USER PROCESS

KERNEL


2. UNIX standards and implementations


ANSI C

ANSI Standard X3. 159-1989 ISO/IEC 9899:1990 Goal : Portable C program for various OS


OS standards

IEEE POSIX POSIX :Portable Operating System Interface for Computer

Environments IEEE Standard 1003.1-1988 : OS Interface 1003.2 : Shell and utilities 1003.7 : System management

X/Open XPG3 X/Open : International Consortium of Computer Manufacturers X/Open Portability Guide, Issue 3 (XPG3)

FIPS (Federal Information Processing Standards)


UNIX Implementations Developed for PDP-7 by Ben Tompson in Bell Lab AT&T UNIX System V (SVR 3, SVR 4)

Complies POSIX 1003.1 and XPG3 Provides ANSI C Compiler

Berkeley Software Distributions (BSD) 4.3 BSD Tahoe – BSD Networking Software, Release 1.0

Microsoft Xenix, System V SunOS

4.2 BSD + SVR 4 CMU Mach

Support multi-processor, multi-thread


Limitations

Compile time options Compile time limits Runtime limits


Definitions for standards & implementations


ANSI C Limits


POSIX Limits


XPG3 Limits


sysconf, pathconf, fpathconf

Synopsis #include <unistd.h> long sysconf(int name); long pathconf(const char *pathname, int name); long fpathconf(int filedes, int name);

Ex)#include <errno.h>#include “ourhdr.h”static void pr_sysconf(char *, int);static void pr_pathconf(char *, char *, int);

int main(int argc, char * argv[]){ if (argc != 2) err_quit(usage:a.out <dirname>”);

pr_sysconf(“CHILD_MAX = “, _SC_CHILD_MAX); pr_pathconf(“PATH_MAX = “, argv[1], _PC_PATH_MAX);}

static void pr_sysconf(char *mesg, int name){ fputs(msg, stdout); printf(“%ld\n”, sysconf(name));}

static void pr_pathconf(char *mesg, char *path, int name){ fputs(mesg, stdout); printf(“%ld\n”, pathconf(path, name));}


Example of Configuration


Indeterminate runtime limits#include <errno.h>#include <limits.h>#include “ourhdr.h”

#ifdef PATH_MAXstatic int pathmax = PATH_MAX;#elsestatic int pathmax = 0;#endif

#define PATH_MAX_GUESS 1024

char *path_alloc(int *size){ char *ptr;

if (pathmax == 0) { if ((pathmax = pathconf(“/”, _PC_PATH_MAX)) < 0) { pathmax = PATH_MAX_GUESS; else pathmax++; } ptr = malloc(pathmax+1); *size = pathmax+1; return ptr; }

maxpath


Indeterminate runtime limits(con’t)

max open fileEx1)

#include <sys/param.h>

for (i = 0; i < NOFILE, i++) close(i);

Ex2)#include <unistd.h>

#define OPEN_MAX_GUESS 256

openmax = sysconf(_SC_OPEN_MAX);

if (openmax < 0) openmax = OPEN_MAX_GUESS;

for (I = 0; I < openmax; i++) close(i);


Feature test macros

_POSIX_SOURCE cc –D _POSIX_SOURCE file.c

__STDC__

#ifdef __STDC__void *myfunc(const char *, int);#elsevoid *myfunc();#endif

linux system programmingcfs3.tistory.com/upload_control/download.blog?fhandle... · unix system...

Documents