UNIT : I

SYSTEM

PROGRAMMING II SEMESTER (MCSE 201)

PREPARED BY ARUN PRATAP SINGH

PREPARED BY ARUN PRATAP SINGH 1

1

OVERVIEW OF LANGUAGE PROCESSORS :

System programming (or systems programming) is the activity of computer programming system

software. The primary distinguishing characteristic of systems programming when compared to

application programming is that application programming aims to produce software which

provides services to the user (e.g. word processor), whereas systems programming aims to

produce software which provides services to the computer hardware (e.g. disk defragmenter). It

requires a greater degree of hardware awareness.

The following attributes characterize systems programming:

The programmer will make assumptions about the hardware and other properties of the

system that the program runs on, and will often exploit those properties, for example by using

an algorithm that is known to be efficient when used with specific hardware.

Usually a low-level programming language or programming language dialect is used that:

can operate in resource-constrained environments

is very efficient and has little runtime overhead

has a small runtime library, or none at all

allows for direct and "raw" control over memory access and control flow

let’s the programmer write parts of the program directly in assembly language

Often systems programs cannot be run in a debugger. Running the program in a simulated

environment can sometimes be used to reduce this problem.

Systems programming is sufficiently different from application programming that programmers

tend to specialize in one or the other.

In system programming, often limited programming facilities are available. The use of automatic

garbage collection is not common and debugging is sometimes hard to do. The runtime library, if

available at all, is usually far less powerful, and does less error checking. Because of those

limitations, monitoring and logging are often used; operating systems may have extremely

elaborate logging subsystems.

Implementing certain parts in operating system and networking requires systems programming,

for example implementing Paging (Virtual Memory) or a device driver for an operating system.

UNIT : I

PREPARED BY ARUN PRATAP SINGH 2

2

PREPARED BY ARUN PRATAP SINGH 3

3

Division of of language processors

PREPARED BY ARUN PRATAP SINGH 4

4

PREPARED BY ARUN PRATAP SINGH 5

5

PREPARED BY ARUN PRATAP SINGH 6

6

PREPARED BY ARUN PRATAP SINGH 7

7

PREPARED BY ARUN PRATAP SINGH 8

8

PREPARED BY ARUN PRATAP SINGH 9

9

ELEMENTS OF ASSEMBLY LANGUAGE PROGRAMMING :

An assembly language is a low-level programming language for a computer, or other programmable device, in which there is a very strong (generally one-to-one) correspondence between the language and the architecture's machine code instructions. Each assembly language is specific to a particular computer architecture, in contrast to most high-level programming languages, which are generally portable across multiple architectures, but require interpreting or compiling.

A utility program called an assembler is used to translate assembly language statements into the target computer's machine code. The assembler performs a more or less isomorphic translation (a one-to-one mapping) from mnemonic statements into machine instructions and data. This is in

PREPARED BY ARUN PRATAP SINGH 10

10

contrast with high-level languages, in which a single statement generally results in many machine instructions.

Many sophisticated assemblers offer additional mechanisms to facilitate program development, control the assembly process, and aid debugging. In particular, most modern assemblers include a macro facility (described below), and are called macro assemblers.

Elements of Assembly Language :-

Assembly language is basically like any other language, which means that it has its words, rules

and syntax. The basic elements of assembly language are:

Labels; Orders; Directives; and Comments.

PREPARED BY ARUN PRATAP SINGH 11

11

Key concepts :

Assembler

An assembler is a program which creates object code by translating combinations of mnemonics

and syntax for operations and addressing modes into their numerical equivalents. This

representation typically includes an operation code ("opcode") as well as other control bits. The

assembler also calculates constant expressions and resolvessymbolic names for memory

locations and other entities. The use of symbolic references is a key feature of assemblers, saving

tedious calculations and manual address updates after program modifications. Most assemblers

also include macro facilities for performing textual substitution—e.g., to generate common short

sequences of instructions as inline, instead of called subroutines.

Some assemblers may also be able to perform some simple types of instruction set-

specific optimizations. One concrete example of this may be the ubiquitous x86 assemblers from

various vendors. Most of them are able to perform jump-instruction replacements (long jumps

replaced by short or relative jumps) in any number of passes, on request. Others may even do

simple rearrangement or insertion of instructions, such as some assemblers

for RISC architectures that can help optimize a sensible instruction scheduling to exploit the CPU

pipeline as efficiently as possible.

Like early programming languages such as Fortran, Algol, Cobol and Lisp, assemblers have been

available since the 1950s and the first generations of text based computer interfaces. However,

assemblers came first as they are far simpler to write than compilers for high-level languages.

This is because each mnemonic along with the addressing modes and operands of an instruction

translates rather directly into the numeric representations of that particular instruction, without

much context or analysis. There have also been several classes of translators and semi automatic

code generators with properties similar to both assembly and high level languages,

with speedcode as perhaps one of the better known examples.

Number of passes

There are two types of assemblers based on how many passes through the source are needed

to produce the executable program.

One-pass assemblers go through the source code once. Any symbol used before it is defined

will require "errata" at the end of the object code (or, at least, no earlier than the point where

the symbol is defined) telling the linker or the loader to "go back" and overwrite a placeholder

which had been left where the as yet undefined symbol was used.

Multi-pass assemblers create a table with all symbols and their values in the first passes, then

use the table in later passes to generate code.

In both cases, the assembler must be able to determine the size of each instruction on the initial

passes in order to calculate the addresses of subsequent symbols. This means that if the size of

PREPARED BY ARUN PRATAP SINGH 12

12

an operation referring to an operand defined later depends on the type or distance of the operand,

the assembler will make a pessimistic estimate when first encountering the operation, and if

necessary pad it with one or more "no-operation" instructions in a later pass or the errata. In an

assembler with peephole optimization, addresses may be recalculated between passes to allow

replacing pessimistic code with code tailored to the exact distance from the target.

The original reason for the use of one-pass assemblers was speed of assembly— often a second

pass would require rewinding and rereading a tape or rereading a deck of cards. With modern

computers this has ceased to be an issue. The advantage of the multi-pass assembler is that the

absence of errata makes the linking process (or the program load if the assembler directly

produces executable code) faster.

High-level assemblers

More sophisticated high-level assemblers provide language abstractions such as:

Advanced control structures

High-level procedure/function declarations and invocations

High-level abstract data types, including structures/records, unions, classes, and sets

Sophisticated macro processing (although available on ordinary assemblers since the late

1950s for IBM 700 series and since the 1960s for IBM/360, amongst other machines)

Object-oriented programming features such as classes, objects, abstraction, polymorphism,

and inheritance

See Language design below for more details.

Assembly language

A program written in assembly language consists of a series of (mnemonic) processor instructions

and meta-statements (known variously as directives, pseudo-instructions and pseudo-ops),

comments and data. Assembly language instructions usually consist of an opcode mnemonic

followed by a list of data, arguments or parameters. These are translated by

an assembler into machine language instructions that can be loaded into memory and executed.

For example, the instruction below tells an x86/IA-32 processor to move an immediate 8-bit

value into a register. The binary code for this instruction is 10110 followed by a 3-bit identifier for

which register to use. The identifier for the AL register is 000, so the following machine code loads

the AL register with the data 01100001.

10110000 01100001

This binary computer code can be made more human-readable by expressing it in hexadecimal as

follows.

PREPARED BY ARUN PRATAP SINGH 13

13

B0 61

Here, B0 means 'Move a copy of the following value into AL', and 61 is a hexadecimal

representation of the value 01100001, which is 97 in decimal. Intel assembly language provides

the mnemonic MOV (an abbreviation of move) for instructions such as this, so the machine code

above can be written as follows in assembly language, complete with an explanatory comment if

required, after the semicolon. This is much easier to read and to remember.

MOV AL, 61h ; Load AL with 97 decimal (61 hex)

In some assembly languages the same mnemonic such as MOV may be used for a family of

related instructions for loading, copying and moving data, whether these are immediate values,

values in registers, or memory locations pointed to by values in registers. Other assemblers may

use separate opcodes such as L for "move memory to register", ST for "move register to memory",

LR for "move register to register", MVI for "move immediate operand to memory", etc.

The Intel opcode 10110000 (B0) copies an 8-bit value into the AL register, while 10110001 (B1)

moves it into CL and 10110010 (B2) does so into DL. Assembly language examples for these

follow.[6]

MOV AL, 1h ; Load AL with immediate value 1

MOV CL, 2h ; Load CL with immediate value 2

MOV DL, 3h ; Load DL with immediate value 3

The syntax of MOV can also be more complex as the following examples show.[7]

MOV EAX, [EBX] ; Move the 4 bytes in memory at the address contained in EBX into EAX

MOV [ESI+EAX], CL ; Move the contents of CL into the byte at address ESI+EAX

In each case, the MOV mnemonic is translated directly into an opcode in the ranges 88-8E, A0-

A3, B0-B8, C6 or C7 by an assembler, and the programmer does not have to know or remember

which.

Transforming assembly language into machine code is the job of an assembler, and the reverse

can at least partially be achieved by a disassembler. Unlike high-level languages, there is usually

a one-to-one correspondence between simple assembly statements and machine language

instructions. However, in some cases, an assembler may provide pseudo instructions (essentially

macros) which expand into several machine language instructions to provide commonly needed

functionality. For example, for a machine that lacks a "branch if greater or equal" instruction, an

assembler may provide a pseudo instruction that expands to the machine's "set if less than" and

"branch if zero (on the result of the set instruction)". Most full-featured assemblers also provide a

rich macro language (discussed below) which is used by vendors and programmers to generate

more complex code and data sequences.

PREPARED BY ARUN PRATAP SINGH 14

14

Each computer architecture has its own machine language. Computers differ in the number and

type of operations they support, in the different sizes and numbers of registers, and in the

representations of data in storage. While most general-purpose computers are able to carry out

essentially the same functionality, the ways they do so differ; the corresponding assembly

languages reflect these differences.

Multiple sets of mnemonics or assembly-language syntax may exist for a single instruction set,

typically instantiated in different assembler programs. In these cases, the most popular one is

usually that supplied by the manufacturer and used in its documentation.

PREPARED BY ARUN PRATAP SINGH 15

15

DESIGN OF ASSEMBLER :

PREPARED BY ARUN PRATAP SINGH 16

16

PREPARED BY ARUN PRATAP SINGH 17

17

PREPARED BY ARUN PRATAP SINGH 18

18

PREPARED BY ARUN PRATAP SINGH 19

19

PREPARED BY ARUN PRATAP SINGH 20

20

PREPARED BY ARUN PRATAP SINGH 21

21

ONE PASS ASSEMBLER :

One-pass assemblers go through the source code once. Any symbol used before it is defined will

require "errata" at the end of the object code (or, at least, no earlier than the point where the

symbol is defined) telling the linker or the loader to "go back" and overwrite a placeholder which

had been left where the as yet undefined symbol was used.

PREPARED BY ARUN PRATAP SINGH 22

22

PREPARED BY ARUN PRATAP SINGH 23

23

PREPARED BY ARUN PRATAP SINGH 24

24

PREPARED BY ARUN PRATAP SINGH 25

25

PREPARED BY ARUN PRATAP SINGH 26

26

PREPARED BY ARUN PRATAP SINGH 27

27

PREPARED BY ARUN PRATAP SINGH 28

28

MULTI-PASS ASSEMBLER:

Multi-pass assemblers create a table with all symbols and their values in the first passes, then

use the table in later passes to generate code.

In both cases, the assembler must be able to determine the size of each instruction on the initial

passes in order to calculate the addresses of subsequent symbols. This means that if the size of

an operation referring to an operand defined later depends on the type or distance of the operand,

the assembler will make a pessimistic estimate when first encountering the operation, and if

necessary pad it with one or more "no-operation" instructions in a later pass or the errata. In an

assembler with peephole optimization, addresses may be recalculated between passes to allow

replacing pessimistic code with code tailored to the exact distance from the target.

The original reason for the use of one-pass assemblers was speed of assembly— often a second

pass would require rewinding and rereading a tape or rereading a deck of cards. With modern

computers this has ceased to be an issue. The advantage of the multi-pass assembler is that the

absence of errata makes the linking process (or the program load if the assembler directly

produces executable code) faster.

PREPARED BY ARUN PRATAP SINGH 29

29

PREPARED BY ARUN PRATAP SINGH 30

30

PREPARED BY ARUN PRATAP SINGH 31

31

PREPARED BY ARUN PRATAP SINGH 32

32

The difference between one pass and two pass assemblers are:-

A one pass assembler passes over the source file exactly once, in the same pass collecting the

labels, resolving future references and doing the actual assembly. The difficult part is to resolve

future label references and assemble code in one pass.

A two pass assembler does two passes over the source file ( the second pass can be over a file

generated in the first pass ). In the first pass all it does is looks for label definitions and

introduces them in the symbol table. In the second pass, after the symbol table is complete, it

does the actual assembly by translating the operations and so on.

MACRO DEFINITION :

A macro (short for "macroinstruction") in computer science is a rule or pattern that specifies how

a certain input sequence (often a sequence of characters) should be mapped to a replacement

output sequence (also often a sequence of characters) according to a defined procedure. The

mapping process that instantiates (transforms) a macro use into a specific sequence is known as

macro expansion. A facility for writing macros may be provided as part of a software application or

as a part of a programming language. In the former case, macros are used to make tasks using

the application less repetitive. In the latter case, they are a tool that allows a programmer to

enable code reuse or even to design domain-specific languages.

PREPARED BY ARUN PRATAP SINGH 33

33

Macros are used to make a sequence of computing instructions available to the programmer as

a single program statement, making the programming task less tedious and less error-

prone. (Thus, they are called "macros" because a big block of code can be expanded from

a small sequence of characters). Macros often allow positional or keyword parameters that dictate

what the conditional assembler program generates and have been used to create

entire programs or program suites according to such variables as operating system, platform or

other factors. The term derives from "macro instruction", and such expansions were originally

used in generating assembly language code.

A macro instruction (macro) is a notational convenience for the programmer

It allows the programmer to write shorthand version of a program (module

programming)

The macro processor replaces each macro instruction with the corresponding

group of source language statements (expanding)

Normally, it performs no analysis of the text it handles.

It does not concern the meaning of the involved statements during macro

expansion.

The design of a macro processor generally is machine independent!

Macro:-

Macro instructions are single line abbreviations for group of instructions.

Using a macro, programmer can define a single “instruction” to represent block of

code.

PREPARED BY ARUN PRATAP SINGH 34

34

Macro Expansion –

Replacement of macro call by corresponding sequence of instructions is called as

macro expansion

PREPARED BY ARUN PRATAP SINGH 35

35

Example of macro definition-

A macro invocation statement (a macro call) gives the name of the macro instruction

being invoked and the arguments to be used in expanding the macro.

macro_name p1, p2, …

Difference between macro call and procedure call

Macro call: statements of the macro body are expanded each time the macro is

invoked.

Procedure call: statements of the subroutine appear only one, regardless of how

many times the subroutine is called.

How does a programmer decide to use macro calls or procedure calls?

From the viewpoint of a programmer

From the viewpoint of the CPU

PREPARED BY ARUN PRATAP SINGH 36

36

Example: more than one arguments-

A 1,data1

A 2,data2

A 3,data3

:

A 1,data3

A 2,data2

A 3,data1

:

data1 DC F’5’

data2 DC F’6’

PREPARED BY ARUN PRATAP SINGH 37

37

PREPARED BY ARUN PRATAP SINGH 38

38

Two ways of specifying arguments to a macro call-

A) Positional argument

Argument are matched with dummy arguments according to order in which they appear.

INCR A,B,C

‘A’ replaces first dummy argument

‘B’ replaces second dummy argument

‘C’ replaces third dummy argument

B) keyword arguments

This allows reference to dummy arguments by name as well as by position.

e.g.

INCR &arg1 = A,&arg3 = C, &arg2 =’B’

e.g.

INCR &arg1 = &arg2 = A, &arg2 =’C’

Two pass Macro Processor –

General Design Steps

Step 1: Specification of Problem:-

Step 2 Specification of databases:-

Step 3 Specification of database formats

Step 4 : Algorithm

Specify the problem –

In Pass-I the macro definitions are searched and stored in the macro definition table and

the entry is made in macro name table

In Pass-II the macro calls are identified and the arguments are placed in the appropriate

place and the macro calls are replaced by macro definitions.

Specification of databases:-

Pass 1:-

The input macro source program.

The output macro source program to be used by Pass2.

PREPARED BY ARUN PRATAP SINGH 39

39

Macro-Definition Table (MDT), to store the body of macro defns.

Macro-Definition Table Counter (MDTC), to mark next available entry MDT.

Macro- Name Table (MNT), used to store names of macros.

Macro Name Table counter (MNTC), used to indicate the next available entry in MNT.

Argument List Array (ALA), used to substitute index markers for dummy arguments

before storing a macro-defns.

Pass 2:-

The copy of the input from Pass1.

The output expanded source to be given to assembler.

MDT, created by Pass1.

MNT, created by Pass1.

Macro-Definition Table Pointer (MDTP), used to indicate the next line of text to be used

during macro-expansion.

Argument List Array (ALA), used to substitute macro-call arguments for the index

markers in the stored macro-defns

Specification of database format:-

PREPARED BY ARUN PRATAP SINGH 40

40

Macro Names Table (MNT):

PREPARED BY ARUN PRATAP SINGH 41

41

RELOCATING AND LINKING CONCEPTS : Relocation is the process of assigning load addresses to various parts of a program and adjusting the code and data in the program to reflect the assigned addresses. A linker usually performs relocation in conjunction with symbol resolution, the process of searching files and libraries to replace symbolic references or names of libraries with actual usable addresses in memory before running a program. Although relocation is typically done by the linker at link time, it can also be done at execution time by a relocating loader, or by the running program itself.

Relocation is typically done in two steps:

1. Each object file has various sections like code, data, .bss etc. To combine all the objects

to a single executable, the linker merges all sections of similar type into a single section

of that type. The linker then assigns run time addresses to each section and each symbol.

At this point, the code (functions) and data (global variables) will have unique run time

addresses.

2. Each section refers to one or more symbols which should be modified so that they point

to the correct run time addresses based on information stored in a relocation table in the

object file.

PREPARED BY ARUN PRATAP SINGH 42

42

PREPARED BY ARUN PRATAP SINGH 43

43

PREPARED BY ARUN PRATAP SINGH 44

44

PREPARED BY ARUN PRATAP SINGH 45

45

PREPARED BY ARUN PRATAP SINGH 46

46

LINKING CONCEPT :

PREPARED BY ARUN PRATAP SINGH 47

47

PREPARED BY ARUN PRATAP SINGH 48

48

PREPARED BY ARUN PRATAP SINGH 49

49

DESIGN OF LINKER : Execution of Programs- A has to be transformed before it can be executed Many of these transformations perform memory bindings

• Accordingly, an address is called compiled address, linked address, etc

Linking

The process of collecting and combining various pieces of code and data into a single file that can be loaded (copied) into memory and executed.

Linking time Can be done at compile time, i.e. when the source code is

translated Or, at load time, i.e. when the program is loaded into memory Or, even at run time.

Static Linker

Performs the linking at compile time.

Takes a collection of relocatable object files and command line arguments and generate a fully linked executable object file that can be loaded and run.

Performs two main tasks Symbol resolution: associate each symbol reference with exactly

one symbol definition Relocation: relocate code and data sections and modify symbol

references to the relocated memory locations Dynamic Linker

Performs the linking at load time or at run time.

PREPARED BY ARUN PRATAP SINGH 50

50

Performing Relocation relocation_factorP = l_originP – t_originP lsymb = tsymb + relocation_factorP Relocation Requirement

Linking Requirement

Name Table (NTAB) – with the field of Synbols name and Linked_address

eg. : Pg: 231 – Example : 7.9

Self-Relocating Programs-

Which program can be modified, or can modfied it self , to execute from a given Load Origin.

Classifications :

1) Non Relocatable Program

2) Relocatable Programs

3) Self Relocating Programs

Non Relocatable Program –

1) Can't Execute in Any Memory Area, Only on Traslated Origin

2) Lack of Address Sensitive Instructions

3) e.g. Hand Coded Machine Language Program

Relocatable Programs-

1) Can Execute in Any Desired Memory Area

2) Availability of Address Sensitve Instruction

3) e.g. Object Module

Self-Relocating Programs-

1) Can Execute in Any Memory Area

2) Availability of Own Address Sensitve Instruction

3) Relocating Logic Specified on the start of the Program

4) Useful for Time sharing Operating System

PREPARED BY ARUN PRATAP SINGH 51

51

Linking for Overlays-

Overlay:

It is a part of a program that use same load origin as some part of the program.

Advantages:

► Keep in memory only those instructions and data that are needed at any given time.

► Needed when process is larger than amount of memory allocated to it.

► Implemented by user, no special support needed from operating system, programming

design of overlay structure is complex.

►Used to reduce the Main Memory Requirements.

An Overlay Tree

Loader –

An operating system utility that copies programs from a storage device to main memory, where

they can be executed. In addition to copying a program into main memory, the loader can also

replace virtual addresses with physical addresses.

Most loaders are transparent, i.e., you cannot directly execute them, but the operating system

uses them when necessary.

Absolute loader can only load origin = linked Origin.

PREPARED BY ARUN PRATAP SINGH 52

52

PREPARED BY ARUN PRATAP SINGH 53

53

PREPARED BY ARUN PRATAP SINGH 54

54

PREPARED BY ARUN PRATAP SINGH 55

55