system programming unit-4 by arun pratap singh

8/12/2019 System Programming Unit-4 by Arun Pratap Singh

1/83

UNIT : IV

SYSTEMPROGRAMMING

II SEMESTER MCSE 201)

PREPARED BY ARUN PRATAP SINGH


2/83

PREPARED BY ARUN PRATAP SINGH 1

1

DISTRIBUTED OPERATING SYSTEM AND ITS DESIGN ISSUES:

UNIT : IV


3/83


2


4/83


3


5/83


4


6/83


5


7/83


6


8/83


7


9/83


8


10/83


9


11/83


10


12/83


11


13/83


12


14/83


13


15/83


14


16/83


15


17/83


16


18/83


17

NETWORKING ISSUES :

NETWORK TOPOLOGY :

Network topologyis the arrangement of the various elements (links,nodes,etc.) of acomputer

network.Essentially, it is thetopological structure of a network, and may be depicted physically or

logically. Physical topology refers to the placement of the network's various components,

including device location and cable installation, whilelogicaltopology shows how data flows within

a network, regardless of its physical design. Distances between nodes, physical interconnections,

transmission rates, and/or signal types may differ between two networks, yet their topologies may

be identical.

The study of network topology recognizes seven basic topologies:

Point-to-point

Bus

Star

Ring or circular

Mesh

Tree

Hybrid
http://en.wikipedia.org/wiki/Data_linkhttp://en.wikipedia.org/wiki/Node_(networking)http://en.wikipedia.org/wiki/Computer_networkhttp://en.wikipedia.org/wiki/Computer_networkhttp://en.wikipedia.org/wiki/Topologicalhttp://en.wikipedia.org/wiki/Logical_topologyhttp://en.wikipedia.org/wiki/Logical_topologyhttp://en.wikipedia.org/wiki/Logical_topologyhttp://en.wikipedia.org/wiki/Logical_topologyhttp://en.wikipedia.org/wiki/Topologicalhttp://en.wikipedia.org/wiki/Computer_networkhttp://en.wikipedia.org/wiki/Computer_networkhttp://en.wikipedia.org/wiki/Node_(networking)http://en.wikipedia.org/wiki/Data_link


19/83


18

COMMUNICATION OVER THE NETWORK :


20/83


19


21/83


20

SWITCHING TECHNIQUES IN NETWORK :

In large networks there might be multiple paths linking sender and receiver. Information may beswitched as it travels through various communication channels. There are three typical switching

techniques available for digital traffic.

Circuit Switching

Message Switching

Packet Switching


22/83


21

Circuit Switching :

Circuit switching is a technique that directly connects the sender and the receiver in an

unbroken path.

Telephone switching equipment, for example, establishes a path that connects the caller's

telephone to the receiver's telephone by making a physical connection.

With this type of switching technique, once a connection is established, a dedicated path

exists between both ends until the connection is terminated.

Routing decisions must be made when the circuit is first established, but there are no

decisions made after that time.

Circuit switching in a network operates almost the same way as the telephone system

works.

A complete end-to-end path must exist before communication can take place.

The computer initiating the data transfer must ask for a connection to the destination.

Once the connection has been initiated and completed to the destination device, the

destination device must acknowledge that it is ready and willing to carry on a transfer.

Message Switching :

With message switching there is no need to establish a dedicated path between two

stations.

When a station sends a message, the destination address is appended to the message.

The message is then transmitted through the network, in its entirety, from node to node.

Each node receives the entire message, stores it in its entirety on disk, and then transmits

the message to the next node.

This type of network is called a store-and-forward network.


23/83


22

A message-switching node is typically a general-purpose computer. The device needs sufficient

secondary-storage capacity to store the incoming messages, which could be long. A time delay

is introduced using this type of scheme due to store- and-forward time, plus the time required to

find the next node in the transmission path.

Packet Switching :

Packet switching can be seen as a solution that tries to combine the advantages of

message and circuit switching and to minimize the disadvantages of both.

There are two methods of packet switching: Datagram and virtual circuit.

In both packet switching methods, a message is broken into small parts, called

packets.

Each packet is tagged with appropriate source and destination addresses.

Since packets have a strictly defined maximum length, they can be stored in main

memory instead of disk, therefore access delay and cost are minimized.

Also the transmission speeds, between nodes, are optimized.

With current technology, packets are generally accepted onto the network on a first-

come, first-served basis. If the network becomes overloaded, packets are delayed or

discarded (``dropped'').

Packet Switching: Datagram

Datagram packet switching is similar to message switching in that each packet is a self-

contained unit with complete addressing information attached.

This fact allows packets to take a variety of possible paths through the network.

So the packets, each with the same destination address, do not follow the same route,

and they may arrive out of sequence at the exit point node (or the destination).


24/83


23

Reordering is done at the destination point based on the sequence number of the

packets.

It is possible for a packet to be destroyed if one of the nodes on its way is crashed

momentarily. Thus all its queued packets may be lost.

Packet Switching:Virtual Circuit-

In the virtual circuit approach, a preplanned route is established

before any data packets are sent.

A logical connection is established when

a sender send a "call request packet" to the receiver and

the receiver send back an acknowledge packet "call accepted packet" to the sender if

the receiver agrees on conversational parameters.

The conversational parameters can be maximum packet sizes, path to be taken, and

other variables necessary to establish and maintain the conversation.

Virtual circuits imply acknowledgements, flow control, and error control, so virtual circuits

are reliable.

That is, they have the capability to inform upper-protocol layers if a transmission problem

occurs.

ROUTING OF NETWORK TRAFFIC :

Routingis the process of selecting best paths in a network. In the past, the term routing was also

used to mean forwarding network traffic among networks. However this latter function is much

better described as simply forwarding. Routing is performed for many kinds of networks, including

the telephone network (circuit switching), electronic data networks (such as the Internet),

and transportation networks. This article is concerned primarily with routing in electronic data

networks usingpacket switching technology.

In packet switching networks, routing directs packet forwarding (the transit of logically

addressed network packets from their source toward their ultimate destination) through

intermediate nodes. Intermediate nodes are typically network hardware devices suchas routers, bridges, gateways, firewalls, or switches. General-purpose computers can also

forward packets and perform routing, though they are not specialized hardware and may suffer

from limited performance. The routing process usually directs forwarding on the basis ofrouting

tables which maintain a record of the routes to various network destinations. Thus, constructing

routing tables, which are held in the router'smemory,is very important for efficient routing. Most
http://en.wikipedia.org/wiki/PSTNhttp://en.wikipedia.org/wiki/Circuit_switchinghttp://en.wikipedia.org/wiki/Computer_networkhttp://en.wikipedia.org/wiki/Internethttp://en.wikipedia.org/wiki/Transport_networkhttp://en.wikipedia.org/wiki/Packet_switchinghttp://en.wikipedia.org/wiki/Packet_forwardinghttp://en.wikipedia.org/wiki/Network_packethttp://en.wikipedia.org/wiki/Node_(networking)http://en.wikipedia.org/wiki/Router_(computing)http://en.wikipedia.org/wiki/Bridging_(networking)http://en.wikipedia.org/wiki/Gateway_(telecommunications)http://en.wikipedia.org/wiki/Firewall_(computing)http://en.wikipedia.org/wiki/Network_switchhttp://en.wikipedia.org/wiki/Computerhttp://en.wikipedia.org/wiki/Routing_tablehttp://en.wikipedia.org/wiki/Routing_tablehttp://en.wikipedia.org/wiki/Computer_storagehttp://en.wikipedia.org/wiki/Computer_storagehttp://en.wikipedia.org/wiki/Routing_tablehttp://en.wikipedia.org/wiki/Routing_tablehttp://en.wikipedia.org/wiki/Computerhttp://en.wikipedia.org/wiki/Network_switchhttp://en.wikipedia.org/wiki/Firewall_(computing)http://en.wikipedia.org/wiki/Gateway_(telecommunications)http://en.wikipedia.org/wiki/Bridging_(networking)http://en.wikipedia.org/wiki/Router_(computing)http://en.wikipedia.org/wiki/Node_(networking)http://en.wikipedia.org/wiki/Network_packethttp://en.wikipedia.org/wiki/Packet_forwardinghttp://en.wikipedia.org/wiki/Packet_switchinghttp://en.wikipedia.org/wiki/Transport_networkhttp://en.wikipedia.org/wiki/Internethttp://en.wikipedia.org/wiki/Computer_networkhttp://en.wikipedia.org/wiki/Circuit_switchinghttp://en.wikipedia.org/wiki/PSTN


25/83


24

routing algorithms use only one network path at a time.Multipath routing techniques enable the

use of multiple alternative paths.

In case of overlapping/equal routes, the following elements are considered in order to decide

which routes get installed into the routing table (sorted by priority):

1. Prefix-Length: where longer subnet masks are preferred (independent of whether it is

within a routing protocol or over different routing protocol)

2. Metric: where a lower metric/cost is preferred (only valid within one and the same routing

protocol)

3. Administrative distance:where a lower distance is preferred (only valid between different

routing protocols)

Routing, in a more narrow sense of the term, is often contrasted with bridging in its assumption

that network addresses are structured and that similar addresses imply proximity within the

network. Structured addresses allow a single routing table entry to represent the route to a group

of devices. In large networks, structured addressing (routing, in the narrow sense) outperforms

unstructured addressing (bridging). Routing has become the dominant form of addressing on the

Internet. Bridging is still widely used within localized environments.
http://en.wikipedia.org/wiki/Multipath_routinghttp://en.wikipedia.org/wiki/Administrative_distancehttp://en.wikipedia.org/wiki/Administrative_distancehttp://en.wikipedia.org/wiki/Bridging_(networking)http://en.wikipedia.org/wiki/Network_addresshttp://en.wikipedia.org/wiki/Network_addresshttp://en.wikipedia.org/wiki/Bridging_(networking)http://en.wikipedia.org/wiki/Administrative_distancehttp://en.wikipedia.org/wiki/Multipath_routing


26/83


25


27/83


26


28/83


27

COMMUNICATION PROTOCOLS :

In telecommunications, a communications protocol is a system of digital rules for data exchange

within or between computers.

Communicating systems use well-defined formats for exchanging messages. Each message has

an exact meaning intended to elicit a response from a range of possible responses pre-

determined for that particular situation. Thus, a protocol must define the syntax, semantics, and

synchronization of communication; the specified behavior is typically independent of how it is tobe implemented. A protocol can therefore be implemented as hardware, software, or both.

Communications protocols have to be agreed upon by the parties involved. To reach agreement

a protocol may be developed into a technical standard. A programming language describes the

same for computations, so there is a close analogy between protocols and programming

languages: protocols are to communications as programming languages are to computations.

Basic requirements of protocols-

Messages are sent and received on communicating systems to establish communications.

Protocols should therefore specify rules governing the transmission. In general, much of the

following should be addressed:

Data formats for data exchange. Digital message bitstrings are exchanged. The bitstrings are

divided in fields and each field carries information relevant to the protocol. Conceptually the

bitstring is divided into two parts called the header area and the data area. The actual

message is stored in the data area, so the header area contains the fields with more relevance


29/83


28

to the protocol. Bitstrings longer than themaximum transmission unit (MTU) are divided in

pieces of appropriate size.

Address formats for data exchange. Addresses are used to identify both the sender and the

intended receiver(s). The addresses are stored in the header area of the bitstrings, allowing

the receivers to determine whether the bitstrings are intended for themselves and should beprocessed or should be ignored. A connection between a sender and a receiver can be

identified using an address pair (sender address, receiver address). Usually some address

values have special meanings. An all-1s address could be taken to mean an addressing of

all stations on the network, so sending to this address would result in a broadcast on the local

network. The rules describing the meanings of the address value are collectively called

an addressing scheme.

Address mapping. Sometimes protocols need to map addresses of one scheme on addresses

of another scheme. For instance to translate a logical IP address specified by the application

to an Ethernet hardware address. This is referred to as address mapping.

Routing. When systems are not directly connected, intermediary systems along the routeto

the intended receiver(s) need to forward messages on behalf of the sender. On the Internet,

the networks are connected using routers. This way of connecting networks is

calledinternetworking.

Detection of transmission errorsis necessary on networks which cannot guarantee error-free

operation. In a common approach, CRCs of the data area are added to the end of packets,

making it possible for the receiver to detect differences caused by errors. The receiver rejects

the packets on CRC differences and arranges somehow for retransmission.

Acknowledgements of correct reception of packets is required for connection-oriented

communication.Acknowledgements are sent from receivers back to their respective senders.

Loss of information - timeouts and retries. Packets may be lost on the network or suffer from

long delays. To cope with this, under some protocols, a sender may expect an

acknowledgement of correct reception from the receiver within a certain amount of time. On

timeouts, the sender must assume the packet was not received and retransmit it. In case of

a permanently broken link, the retransmission has no effect so the number of retransmissions

is limited. Exceeding the retry limit is considered an error.

Direction of information flowneeds to be addressed if transmissions can only occur in one

direction at a time as on half-duplex links. This is known as Media Access Control.

Arrangements have to be made to accommodate the case when two parties want to gain

control at the same time.

Sequence control. We have seen that long bitstrings are divided in pieces, and then sent on

the network individually. The pieces may get lost or delayed or take different routes to their

destination on some types of networks. As a result pieces may arrive out of sequence.

Retransmissions can result in duplicate pieces. By marking the pieces with sequence
http://en.wikipedia.org/wiki/Maximum_transmission_unithttp://en.wikipedia.org/wiki/Internetworkinghttp://en.wikipedia.org/wiki/Internetworkinghttp://en.wikipedia.org/wiki/Internetworkinghttp://en.wikipedia.org/wiki/Connection-oriented_communicationhttp://en.wikipedia.org/wiki/Connection-oriented_communicationhttp://en.wikipedia.org/wiki/Half-duplexhttp://en.wikipedia.org/wiki/Media_Access_Controlhttp://en.wikipedia.org/wiki/Media_Access_Controlhttp://en.wikipedia.org/wiki/Half-duplexhttp://en.wikipedia.org/wiki/Connection-oriented_communicationhttp://en.wikipedia.org/wiki/Connection-oriented_communicationhttp://en.wikipedia.org/wiki/Internetworkinghttp://en.wikipedia.org/wiki/Maximum_transmission_unit


30/83


29

information at the sender, the receiver can determine what was lost or duplicated, ask for

necessary retransmissions and reassemble the original message.

Flow control is needed when the sender transmits faster than the receiver or intermediate

network equipment can process the transmissions. Flow control can be implemented by

messaging from receiver to sender.

Getting the data across a network is only part of the problem for a protocol. The data received

has to be evaluated in the context of the progress of the conversation, so a protocol has to specify

rules describing the context. These kind of rules are said to express the syntax of the

communications. Other rules determine whether the data is meaningful for the context in which

the exchange takes place. These kind of rules are said to express the semantics of the

communications.


31/83


30


32/83


31


33/83


32


34/83


33


35/83


34


36/83


35


37/83


36


38/83


37


39/83


38

Common types of protocols :

The Internet Protocol is used in concert with other protocols within the Internet Protocol Suite.

Prominent members of which include:

Transmission Control Protocol (TCP) User Datagram Protocol (UDP)

Internet Control Message Protocol (ICMP)

Hypertext Transfer Protocol (HTTP)

Post Office Protocol (POP3)

File Transfer Protocol (FTP)

Internet Message Access Protocol (IMAP)

Other instances of high levelinteraction protocols are:

IIOP

RMI

DCOM

DDE

SOAP

MESSAGE PASSING :

Message passingis a concept fromcomputer science that is used extensively in the design and

implementation of modern software applications; it is key to some models of

concurrency andobject-oriented programming.Message passing is a way of invoking behavior

through some intermediary service or infrastructure. Rather than directly invoke a process,

subroutine, or function by name as in conventional programming, message passing sends a

message to a process (which may be an actor or object) and relies on the process and the

supporting infrastructure to select and invoke the actual code to run.

Message passing is used ubiquitously in modern computer software. It is used as a way for the

objects that make up a program to work with each other and as a way for objects and systemsrunning on different computers (e.g., the Internet) to interact. Message passing may be

implemented by various mechanisms, includingchannels.

Messages are collection of data objects and their structures

Messages have a header containing system dependent control information and a

message body that can be fixed or variable size.
http://en.wikipedia.org/wiki/Internet_Protocolhttp://en.wikipedia.org/wiki/Transmission_Control_Protocolhttp://en.wikipedia.org/wiki/User_Datagram_Protocolhttp://en.wikipedia.org/wiki/Internet_Control_Message_Protocolhttp://en.wikipedia.org/wiki/Hypertext_Transfer_Protocolhttp://en.wikipedia.org/wiki/Post_Office_Protocolhttp://en.wikipedia.org/wiki/File_Transfer_Protocolhttp://en.wikipedia.org/wiki/Internet_Message_Access_Protocolhttp://en.wikipedia.org/wiki/Interaction_protocolhttp://en.wikipedia.org/wiki/IIOPhttp://en.wikipedia.org/wiki/Java_remote_method_invocationhttp://en.wikipedia.org/wiki/Distributed_Component_Object_Modelhttp://en.wikipedia.org/wiki/Dynamic_Data_Exchangehttp://en.wikipedia.org/wiki/SOAPhttp://en.wikipedia.org/wiki/Computer_sciencehttp://en.wikipedia.org/wiki/Concurrency_(computer_science)#Modelshttp://en.wikipedia.org/wiki/Concurrency_(computer_science)#Modelshttp://en.wikipedia.org/wiki/Object-oriented_programminghttp://en.wikipedia.org/wiki/Actor_modelhttp://en.wikipedia.org/wiki/Object_(computer_science)http://en.wikipedia.org/wiki/Channel_(programming)http://en.wikipedia.org/wiki/Channel_(programming)http://en.wikipedia.org/wiki/Object_(computer_science)http://en.wikipedia.org/wiki/Actor_modelhttp://en.wikipedia.org/wiki/Object-oriented_programminghttp://en.wikipedia.org/wiki/Concurrency_(computer_science)#Modelshttp://en.wikipedia.org/wiki/Concurrency_(computer_science)#Modelshttp://en.wikipedia.org/wiki/Computer_sciencehttp://en.wikipedia.org/wiki/SOAPhttp://en.wikipedia.org/wiki/Dynamic_Data_Exchangehttp://en.wikipedia.org/wiki/Distributed_Component_Object_Modelhttp://en.wikipedia.org/wiki/Java_remote_method_invocationhttp://en.wikipedia.org/wiki/IIOPhttp://en.wikipedia.org/wiki/Interaction_protocolhttp://en.wikipedia.org/wiki/Internet_Message_Access_Protocolhttp://en.wikipedia.org/wiki/File_Transfer_Protocolhttp://en.wikipedia.org/wiki/Post_Office_Protocolhttp://en.wikipedia.org/wiki/Hypertext_Transfer_Protocolhttp://en.wikipedia.org/wiki/Internet_Control_Message_Protocolhttp://en.wikipedia.org/wiki/User_Datagram_Protocolhttp://en.wikipedia.org/wiki/Transmission_Control_Protocolhttp://en.wikipedia.org/wiki/Internet_Protocol


40/83


39

When a process interacts with another, two requirements have to be satisfied.

Synchronization and Communication.

Fixed Length

Easy to implement

Minimizes processing and storage overhead.

Variable Length

Requires dynamic memory allocation, sofragmentation could occur.

Message passing is a technique for invoking behavior (i.e., running a program) on a computer. In

contrast to the traditional technique of calling a program by name, message passing uses

an object model to distinguish the general function from the specific implementations. The

invoking program sends a message and relies on the object to select and execute the appropriatecode. The justifications for using an intermediate layer essentially falls into two categories:

encapsulation and distribution.

Encapsulation is the idea that software objects should be able to invoke services on other objects

without knowing or caring about how those services are implemented. Encapsulation can reduce

the amount of coding logic and make systems more maintainable. E.g., rather than having IF-
http://en.wikipedia.org/wiki/Object_modelhttp://en.wikipedia.org/wiki/Encapsulation_(computer_science)http://en.wikipedia.org/wiki/Encapsulation_(computer_science)http://en.wikipedia.org/wiki/Object_model


41/83


40

THEN statements that determine which subroutine or function to call a developer can just send a

message to the object and the object will select the appropriate code based on its type.

Distributed message passing provides developers with a layer of the architecture that provides

common services to build systems made up of sub-systems that run on disparate computers in

different locations and at different times. When sending a distributed object a message the

messaging layer can take care of issues such as:

Finding the appropriate object, including objects running on different computers, using

different operating systems and programming languages, at different locations from where

the message originated.

Saving the message on a queue if the appropriate object to handle the message is not

currently running and then invoking the message when the object is available. Also, storing

the result if needed until the sending object is ready to receive it.

Controlling various transactional requirements for distributed transactions, e.g.ensuringACID properties on data.

Synchronous versus asynchronous message passing

One of the most important distinctions among message passing systems is whether they use

synchronous or asynchronous message passing. Synchronous message passing occurs between

objects that are running at the same time. With asynchronous message passing it is possible for

the receiving object to be busy or not running when the requesting object sends the message.

Synchronous message passing is what typical object-oriented programming languages such asJava and Smalltalk use. Asynchronous message passing requires additional capabilities for

storing and retransmitting data for systems that may not run concurrently.

The advantage to synchronous message passing is that it is conceptually less complex.

Synchronous message passing is analogous to a function call in which the message sender is

the function caller and the message receiver is the called function. Function calling is easy and

familiar. Just as the function caller stops until the called function completes, the sending process

stops until the receiving process completes. This alone makes synchronous message unworkable

for some applications. For example, if synchronous message passing would be used exclusively,

large, distributed systems generally would not perform well enough to be usable. Such large,

distributed systems may need to continue to operate while some of their subsystems are down;

subsystems may need to go offline for some kind of maintenance, or have times when subsystems

are not open to receiving input from other systems.

Imagine a busy business office having 100 desktop computers that send emails to each other

using synchronous message passing exclusively. Because the office system does not use
http://en.wikipedia.org/wiki/ACIDhttp://en.wikipedia.org/wiki/ACID


42/83


41

asynchronous message passing, one worker turning off their computer can cause the other 99

computers to freeze until the worker turns their computer back on to process a single email.

Asynchronous message passing is generally implemented so that all the complexities that

naturally occur when trying to synchronize systems and data are handled by an intermediary level

of software. Commercial vendors who develop software products to support these intermediate

levels usually call their software "middleware". One of the most common types of middleware to

support asynchronous messaging is calledMessage Oriented Middleware (MOM)

With asynchronous message passing, the sending system does not wait for a response.

Continuing the function call analogy, asynchronous message passing would be a function call

that returns immediately, without waiting for the called function to execute. Such an asynchronous

function call would merely deliver the arguments, if any, to the called function, and tell the called

function to execute, and then return to continue its own execution. Asynchronous message

passing simply sends the message to the message bus. The bus stores the message until the

receiving process requests messages sent to it. When the receiving process arrives at the result,

it sends the result to the message bus. And the message bus holds the message until the original

process (or some designated next process) picks up its messages from the message bus.

Synchronous communication can be built on top of asynchronous communication by using

aSynchronizer.For example, the -Synchronizer works by ensuring that the sender always waits

for an acknowledgement message from the receiver. The sender only sends the next message

after the acknowledgement has been received.

In addition to the distinction between synchronous and asynchronous message passing the otherprimary distinguishing feature of message passing systems is whether they use distributed or

local objects. With distributed objects the sender and receiver may exist on different computers,

running different operating systems, using different programming languages, etc. In this case the

bus layer takes care of details about converting data from one system to another, sending and

receiving data across the network, etc. The Remote Procedure Call (RPC) protocol in Unix was

an early example of this. Note that with this type of message passing it is not a requirement that

either the sender or receiver are implemented using object-oriented programming. It is possible

to wrap systems developed using procedural languages and treat them as large grained objects

capable of sending and receiving messages.

Distributed or asynchronous message passing has some overhead associated with it compared

to the simpler way of simply calling a procedure. In a traditionalprocedure Call,arguments are

passed to the receiver typically by one or more general purpose registers or in a parameter

list containing the addresses of each of the arguments. This form of communication differs from

message passing in at least three crucial areas:

total memory usage
http://en.wikipedia.org/wiki/Middlewarehttp://en.wikipedia.org/wiki/Message-oriented_middlewarehttp://en.wikipedia.org/wiki/Synchronizer_(algorithm)http://en.wikipedia.org/wiki/Synchronizer_(algorithm)http://en.wikipedia.org/wiki/Calling_conventionhttp://en.wikipedia.org/wiki/General_purpose_registerhttp://en.wikipedia.org/wiki/Parameter_(computer_science)http://en.wikipedia.org/wiki/Parameter_(computer_science)http://en.wikipedia.org/wiki/Parameter_(computer_science)http://en.wikipedia.org/wiki/Parameter_(computer_science)http://en.wikipedia.org/wiki/General_purpose_registerhttp://en.wikipedia.org/wiki/Calling_conventionhttp://en.wikipedia.org/wiki/Synchronizer_(algorithm)http://en.wikipedia.org/wiki/Message-oriented_middlewarehttp://en.wikipedia.org/wiki/Middleware


43/83


42

transfer time

locality

In message passing, each of the arguments has to copy the existing argument into a portion of

the new message. This applies regardless of the size of the argument and in some cases the

arguments can be as large as a document which can be megabytes worth of data. The argument

has to be copied in its entirety and transmitted to the receiving object.

By contrast, for a standard procedure call, only an address (a few bits) needs to be passed for

each argument and may even be passed in ageneral purpose register requiring zero additional

storage and zero transfer time.

RPC IN HETEROGENEOUS ENVIRONMENT :

IPC part of distributed system can often be conveniently handled by message-passing

model.

It doesn't offer a uniform panacea for all the needs.

RPC emerged as a result of this.

It can be said as the special case of message-passing model.

It has become widely accepted because of the following features:

Simple call syntax and similarity to local procedure calls.

It specifies a well defined interface and this property supports compile-time type

checking and automated interface generation.

Its ease of use, efficiency and generality.

It can be used as an IPC mechanism between

processes on different machines and

also between different processes on the same machine.

RPC MODEL

It is similar to commonly used procedure call model. It works in the following manner:

1. For making a procedure call, the caller places arguments to the procedure in some

well specified location.

2. Control is then transferred to the sequence of instructions that constitutes the body

of the procedure.

3. The procedure body is executed in a newly created execution environment that

includes copies of the arguments given in the calling instruction.
http://en.wikipedia.org/wiki/Bit_ratehttp://en.wikipedia.org/wiki/General_purpose_registerhttp://en.wikipedia.org/wiki/General_purpose_registerhttp://en.wikipedia.org/wiki/Bit_rate


44/83


43

4. After the procedure execution is over, control returns to the calling point, returning

a result.

The RPC enables a call to be made to a procedure that does not reside in the address

space of the calling process.

Since the caller and the callee processes have disjoint address space, the remote

procedure has no access to data and variables of the callers environment.

RPC facility uses a message-passing scheme for information exchange between the caller

and the callee processes.

On arrival of request message, the server process

1. extracts the procedures parameters,

2. computes the result,

3. sends a reply message, and

4. then awaits the next call message.

Only one of the two processes is active at any given time.

It is not always necessary that the caller gets blocked.


45/83


44

There can be RPC implementations depending on the parallelism of the caller and the

callees environment.

The RPC could be asynchronous, so that the client may do useful work while waiting for

the reply from the server.

Server could create a thread to process an incoming request so that server is free to

receive other requests.

Transparency of RPC-

A transparent RPC is one in which the local and remote procedure calls are

indistinguishable.

Types of transparencies:

Syntactic transparency

A remote procedure call should have exactly the same syntax as a localprocedure call.

Semantic transparency

The semantics of a remote procedure call are identical to those of a local

procedure call.

Syntactic transparency is not an issue but semantic transparency is difficult.

Difference between remote procedure calls and local procedure calls:

1. Unlike local procedure calls, with remote procedure calls,

Disjoint Address Space

Absence of shared memory.

Meaningless making call by reference, using addresses in arguments and

pointers.

2. RPCs are more vulnerable to failure because of:

Possibility of processor crashes or

communication problems of a network.

3. RPCs are much more time consuming than LPCs due to the involvement of

communication network.

Due to these reasons, total semantic transparency is impossible.


46/83


45

Implementing RPC Mechanism-

To achieve semantic transparency, implementation of RPC mechanism is based on the

concepts of stubs.

Stubs

Provide a normal / local procedure call abstraction by concealing the underlying

RPC mechanism.

A separate stub procedure is associated with both the client and server processes.

To hide the underlying communication network, RPC communication package

known as RPC Runtime is used on both the sides.

Thus implementation of RPC involves the five elements of program:

1. Client

2. Client Stub

3. RPC Runtime

4. Server stub

5. Server

The client, the client stub, and one instance of RPCRuntime execute on the client machine.

The server, the server stub, and one instance of RPCRuntime execute on the server

machine.

As far as the client is concerned, remote services are accessed by the user by making

ordinary LPC


47/83


46

Client Stub-

It is responsible for the following two tasks:

On receipt of a call request from the client,

it packs a specifications of the target procedure and the arguments into a

message and

asks the local RPC Runtime to send it to the server stub.

On receipt of the result of procedure execution, it unpacks the result and passes it

to the client.

RPCRuntime

It handles transmission of messages across the network between Client and the server

machine.

It is responsible for

Retransmission,

Acknowledgement,


48/83


47

Routing and

Encryption.

Server Stub-

It is responsible for the following two tasks:

On receipt of a call request message from the local RPCRuntime, it unpacks it and

makes a perfectly normal call to invoke the appropriate procedure in the server.

On receipt of the result of procedure execution from the server, it unpacks the

result into a message and then asks the local RPCRuntime to send it to the client

stub.

Stub Generation-

Stubs can be generated in one of the following two ways:

Manually

Automatically

Automatic Stub Generation-

Interface Definition Language (IDL) is used here, to define the interface between a client

and the server.

Interface definition:

It is a list of procedure names supported by the interface together with the types oftheir arguments and results.

It also plays role in reducing data storage and controlling amount of data

transferred over the network.

It has information about type definitions, enumerated types, and defined constants.

Export the interface

A server program that implements procedures in the interface.

Import the interface

A client program that calls procedures from an interface.

The interface definition is compiled by the IDL compiler.

IDL compiler generates

components that can be combined with client and server programs, without making

any changes to the existing compilers;


49/83


48

Client stub and server stub procedures;

The appropriate marshaling and unmarshaling operations;

A header file that supports the data types.

RPC Messages-

RPC system is independent of transport protocols and is not concerned as to how a

message is passed from one process to another.

Types of messages involved in the implementation of RPC system:

Call messages

Reply messages

Call Messages

Components necessary in a call message are

1. The id. Information of the remote procedure to be executed.

2. The arguments necessary for the execution.

3. Message Identification field consists of a sequence number for identifying lost and

duplicate messages.

4. Message Type field: whether call or reply messages.

5. Client Identification Field allows the server to:

Identify the client to whom the reply message has to be returned and

To allow server to authenticate the client process.

Call Message Format

Reply Messages

Sent by the server to the client for returning the result of remote procedure execution.

Conditions for unsuccessful message sent by the server:


50/83


49

The server finds that the call message is not intelligible to it.

Client is not authorized to use the service.

Remote procedure identifier is missing.

The remote procedure is not able to decode the supplied arguments.

Occurrence of exception condition.

Reply message formats


51/83


50

RESOURCE ALLOCATION :


52/83


51

Most Realtime systems are distributed across multiple processors. Different techniques are

used to manage resource allocation in such distributed systems. Some of these techniques

are discussed below. They are:

Centralized Resource Allocation Hierarchical Resource Allocation Bi-Directional Resource Allocation Random Access

Centralized Resource Allocation-

In this technique, a centralized allocator keeps track of all the available resources. All entities

send messages requesting resources and the allocator responds with the allocated

resources. The main advantage of this scheme is simplicity. However, the disadvantage is

that as the system size increases, the centralized allocator gets heavily loaded and becomes

a bottleneck.For example, inXenon,the space slot allocation strategy uses this scheme. Space slot free-

busy status is maintained at the CAS. XEN cards needing space slots request the CAS for a

space slot via a message. CAS allocates the space slot and informs the requesting XEN by

a message.

Hierarchical Resource Allocation-

In this technique, the resource allocation is done in multiple steps. First, the centralized

allocator takes the high level resource allocation decision. Then, it passes on the allocation

request to the secondary allocator which takes the detailed resource allocation decision. This

technique is explained by an example given below.

InXenon trunk allocation is carried out in following steps.:

1. The centralized allocator at CAS determines which trunk group should be used to routeoutgoing calls.

2. The call request is then forwarded to the XEN handling the trunk group.

3. The XEN level allocator selects the actual trunk from the trunk group.

The advantage of this scheme is that the centralized allocator is not burdened with the detailed

resource allocation decisions. In this design, the detailed allocation decisions are taken by the

XEN. Thus this design scales very well when the number of XENs in the switch is increased.Note that this example shows only a two step resource allocation. This technique could be

implemented in multiple levels of hierarchical resource allocations.

Bi-Directional Resource Allocation-
http://www.eventhelix.com/realtimemantra/patterns/resourceallocationpatterns.htm#Centralized%20Resource%20Allocationhttp://www.eventhelix.com/realtimemantra/patterns/resourceallocationpatterns.htm#Hierarchical%20Resource%20Allocationhttp://www.eventhelix.com/realtimemantra/patterns/resourceallocationpatterns.htm#Bi-Directional%20Resource%20Allocationhttp://www.eventhelix.com/realtimemantra/patterns/resourceallocationpatterns.htm#Random%20Accesshttp://www.eventhelix.com/ThoughtProjects/Xenon/http://www.eventhelix.com/ThoughtProjects/Xenon/http://www.eventhelix.com/ThoughtProjects/Xenon/http://www.eventhelix.com/ThoughtProjects/Xenon/http://www.eventhelix.com/realtimemantra/patterns/resourceallocationpatterns.htm#Random%20Accesshttp://www.eventhelix.com/realtimemantra/patterns/resourceallocationpatterns.htm#Bi-Directional%20Resource%20Allocationhttp://www.eventhelix.com/realtimemantra/patterns/resourceallocationpatterns.htm#Hierarchical%20Resource%20Allocationhttp://www.eventhelix.com/realtimemantra/patterns/resourceallocationpatterns.htm#Centralized%20Resource%20Allocation


53/83


52

This scheme allows two independent allocators to allocate the same set of resources. It is

used in situations like bi-directional trunk groups. The switch at each end of the trunk group

allocates the trunks from one specific end. This logic avoids both ends allocating the same

resource under light and medium load. However, under heavy load, there is a possibility of

both ends allocating the same resource. This situation is resolved by a fixed arbitration

algorithm leading to one of the switches withdrawing its claim on the resource.

Consider the example of two switches A and B sharing a bi-directional trunk group with 32

trunks. Switch A searches for a free resource starting the search from from trunk number 1.

Switch B searches for a free resource starting from trunk number 32. Most of the time there

will be no clash in resource allocation. However, under heavy utilization of the trunk group, on

occurrence of a clash for a trunk, the incoming call takes precedence over the outgoing call.

Random Access-

Whenever a resource needs to be shared between multiple entities which cannot synchronize

to each other and they do not have access to a centralized allocator, designers have to resort

to random access to the resource. Here all the entities needing the resource just assume that

they have the resource and use it. If two or more entities use the resource at the same time,

none of them gets service and all the entities retry again after a random backoff timer.

Consider the case of a mobile system, where subscribers that need to originate a call need

to send a message across to the base station. This is achieved by using a random access

channel. If two or more mobile terminals use the random access channel at the same time,

the message is corrupted and all requesting terminals will timeout for a response from the

base station and try again after a random backoff.The main disadvantage of this technique is that the random access channel works well only

when the overall contention for the random access channel is very small. As contention

increases, hardly any resource requests are serviced.

DISTRIBUTED DEADLOCK DETECTION :

Distributed deadlocks can occur in distributed systems when distributed

transactions orconcurrency control is being used. Distributed deadlocks can be detected either

by constructing a globalwait-for graph from local wait-for graphs at a deadlock detector or by

adistributed algorithm likeedge chasing.

Phantom deadlocks are deadlocks that are falsely detected in a distributed system due to

system internal delays but don't actually exist. For example, if a process releases a resource R1

and issues a request for R2, and the first message is lost or delayed, a coordinator (detector of

deadlocks) could falsely conclude a deadlock (if the request for R2 while having R1 would cause

a deadlock).
http://en.wikipedia.org/wiki/Distributed_systemshttp://en.wikipedia.org/wiki/Distributed_transactionhttp://en.wikipedia.org/wiki/Distributed_transactionhttp://en.wikipedia.org/wiki/Concurrency_controlhttp://en.wikipedia.org/wiki/Wait-for_graphhttp://en.wikipedia.org/wiki/Distributed_algorithmhttp://en.wikipedia.org/wiki/Edge_chasinghttp://en.wikipedia.org/wiki/Edge_chasinghttp://en.wikipedia.org/wiki/Distributed_algorithmhttp://en.wikipedia.org/wiki/Wait-for_graphhttp://en.wikipedia.org/wiki/Concurrency_controlhttp://en.wikipedia.org/wiki/Distributed_transactionhttp://en.wikipedia.org/wiki/Distributed_transactionhttp://en.wikipedia.org/wiki/Distributed_systems


54/83


53


55/83


54

WAIT-FOR-GRAPH (WFG):


56/83


55


57/83


56


58/83


57

ISSUES IN DEADLOCK DETECTION:


59/83


58

RESOLUTION OF A DETECTED DEADLOCK :

MODELS OF DEADLOCKS :


60/83


59

THE AND MODEL :


61/83


60


62/83


61

Deadlocks in distributed systems are similar to deadlocks in single processor systems,

only worse.

They are harder to avoid, prevent or even detect.

They are hard to cure when tracked down because all relevant information is

scattered over many machines.

People sometimes might classify deadlock into the following types:

Communication deadlocks -- competing with buffers for send/receive

Resources deadlocks -- exclusive access on I/O devices, files, locks, and other

resources.

We treat everything as resources, there we only have resources deadlocks.

Four best-known strategies to handle deadlocks:

The ostrich algorithm (ignore the problem)

Detection (let deadlocks occur, detect them, and try to recover)

Prevention (statically make deadlocks structurally impossible)

Avoidance (avoid deadlocks by allocating resources carefully)

The FOUR Strategies for handling deadlocks-

The ostrich algorithm

No dealing with the problem at all is as good and as popular in distributed systems

as it is in single-processor systems.

In distributed systems used for programming, office automation, process control,

no system-wide deadlock mechanism is present -- distributed databases will

implement their own if they need one.

Deadlock detection and recovery is popular because prevention and avoidance are so

difficult to implement.

Deadlock prevention is possible because of the presence of atomic transactions. We will

have two algorithms for this.

Deadlock avoidance is never used in distributed system, in fact, it is not even used in

single processor systems.

The problem is that the bankers algorithm need to know (in advance) how much

of each resource every process will eventually need. This information is rarely, if

ever, available.

Hence, we will just talk about deadlock detection and deadlock prevention.


63/83


62

Distributed Deadlock Detection

Since preventing and avoiding deadlocks to happen is difficult, researchers works on

detecting the occurrence of deadlocks in distributed system.

The presence of atomic transaction in some distributed systems makes a major

conceptual difference.

When a deadlock is detected in a conventional system, we kill one or more

processes to break the deadlock --- one or more unhappy users.

When deadlock is detected in a system based on atomic transaction, it is resolved

by aborting one or more transactions.

But transactions have been designed to withstand being aborted.

When a transaction is aborted, the system is first restored to the state it had before

the transaction began, at which point the transaction can start again.

With a bit of luck, it will succeed the second time.

Thus the difference is that the consequences of killing off a process are much less

severe when transactions are used.

The Chandy-Misra-Haas algorithm:

Processes are allowed to request multiple resources at once -- the growing phase

of a transaction can be speeded up.

The consequence of this change is a process may now wait on two or more

resources at the same time.

When a process has to wait for some resources, a probe message is generated

and sent to the process holding the resources. The message consists of threenumbers: the process being blocked, the process sending the message, and the

process receiving the message.

When message arrived, the recipient checks to see it it itself is waiting for any

processes. If so, the message is updated, keeping the first number unchanged,

and replaced the second and third field by the corresponding process number.

The message is then send to the process holding the needed resources.

If a message goes all the way around and comes back to the original sender -- the

process that initiate the probe, a cycle exists and the system is deadlocked.


64/83


63


65/83


66/83


65

If a young process wants a resource held by an old process, the young process

will wait.

Centralized Deadlock Detection-

We use a centralized deadlock detection algorithm and try to imitate the non distributed

algorithm.

Each machine maintains the resource graph for its own processes and resources.

A centralized coordinator maintain the resource graph for the entire system.

When the coordinator detect a cycle, it kills off one process to break the deadlock.

In updating the coordinators graph, messages have to be passed.

Method 1) Whenever an arc is added or deleted from the resource graph,

a message have to be sent to the coordinator.

Method 2) Periodically, every process can send a list of arcs added and

deleted since previous update.

Method 3) Coordinator ask for information when it needs it.


67/83


66

MECHANISM FOR BUILDING DISTRIBUTED FILE SYSTEM :

A distributed file system is a client/server-based application that allows clients to access and

process data stored on theserver as if it were on their own computer. When a user accesses a

file on the server, the server sends the user a copy of the file, which is cachedon the user's

computer while the data is being processed and is then returned to the server.

Ideally, a distributed file system organizes file anddirectory services of individual servers into a

global directory in such a way that remote data access is not location-specific but is identical from

any client. All files are accessible to all users of the global file system and organization is

hierarchical and directory-based.

Since more than one client may access the same data simultaneously, the server must have a

mechanism in place (such as maintaining information about the times of access) to organize

updates so that the client always receives the most current version of data and that data conflicts

do not arise. Distributed file systems typically use file or database replication (distributing copies

of data on multiple servers) to protect against data access failures.

A distributed file system is a resource management component of a distributed operating

system. It implements a common file system that can be shared by all the autonomous

computers in the system.

Two important goals :

1. Network transparency to access files distributed over a network. Ideally, users

do not have to be aware of the location of files to access them.

2. High Availability - to provide high availability. Users should have the same easy access

to files, irrespective of their physical location.

ARCHITECTURE

In a distributed file system, files can be stored at any machine and the computation

can be performed at any machine.

The two most important services present in a distributed file system are the name server and

cache manager. A name server is a process that maps names specified by clients to stored

objects such as files and directories. The mapping (also referred to as name resolution) occur

when a process references a file or directory for the first time. A cache manager is a process that

implements file caching. In file caching, a copy of data stored at a remote file server is broughtto the clients machine when referenced by the client. Subsequent accesses to the data are

performed locally at the client, thereby reducing the access delays due to disk latency. Cache

managers can be present at both clients and file servers. Cache managers at the servers cache

files in the main memory to reduce relays due to disk latency. If multiple clients are allowed to

cache a file and modify it, the copies can become inconsistent. To avoid this inconsistency

problem, cache managers at both servers and clients coordinate to perform data storage and

retrieval operations.
http://searchnetworking.techtarget.com/definition/client-serverhttp://searchsoftwarequality.techtarget.com/definition/applicationhttp://whatis.techtarget.com/definition/serverhttp://searchstorage.techtarget.com/definition/cachehttp://searchwinit.techtarget.com/definition/directoryhttp://searchwinit.techtarget.com/definition/directoryhttp://searchstorage.techtarget.com/definition/cachehttp://whatis.techtarget.com/definition/serverhttp://searchsoftwarequality.techtarget.com/definition/applicationhttp://searchnetworking.techtarget.com/definition/client-server


68/83


67

Architecture of a Distributed File System

A request by a process to access a data block is presented to the local cache (client cache) of

the machine (client) on which the process is running (see Figure 5.1). If the block is not in the

cache, then the local disk, if present, is checked for the presence of the data block. If the blockis present, then the request is satisfied and the block is loaded into the client cache. If the block

is not stored locally, then the request is passed on to the appropriate file server (as determined

by the name server). The server checks its own cache for the presence of the data block before

issuing a disk I/O request. The data block is transferred to the client cache in any case and loaded

to the server cache if it was missing in the server cache.

MECHANISMS FOR BUILDING DISTRIBUTED FILE SYSTEMS

1. Mounting

This mechanism provides the binding together of different filename spaces to form a single

hierarchically structured name space. It is UNIX specific and most of existing DFS ( DistributedFile System ) are based on UNIX. A filename space can be bounded to or mounted at an internal

node or a leaf node of a namespace tree. A node onto which a name space is mounted is called

mount point. The kernel maintains amount table, which maps mount points to appropriate

storage devices.

Uses of Mounting in DFS


69/83


68

File systems maintained by remote servers are mounted at clients so that each client have

information regarding file servers. Two approaches are used to maintain mount information.

Approach 1: Mount information is maintained at clients that is each client has to individually

mount every required file system. When files are moved to a different server then mount

information must be updated in mount table of every client.

Approach 2:Mount information is maintained at servers. If files are moved to a different servers,

then mount information need only be updated at servers.

2. Caching

This mechanism is used in DFS to reduce delays in accessing of data. In file caching, a copy of

data stored at remote file server is brought to client when referenced by client so subsequent

access of data is performed locally at client, thus reducing access delays due to netwrok latency.

Data can be cached in main memory or on the local disk of the clients. Data is cached in main

memory at servers to reduce disk access latency.

Need of Caching in DFS:

File system performance gets improved accessing remote disks is much slower than accessing

local memory or local disks. It also reduces the frequency of access to file servers and the

communication network so scalability gets increased.

3. Hints

Caching results in the cache consistency problem when multiple clients cache and modify shared

data. This problem can be avoided by great level of co-operation between file servers and clientswhich is very expansive. Alternative method is to treat cached data as hints that is cached data

are not expected to be completely accurate. Only those class of applications which can recover

after discovering that cached data are invalid can use this approach.

Example:After the name of file or directory is mapped to physical object, the address of object

can be stored as hint in the cache. If the address is incorrect that is fails to map the object, the

cached address is deleted form the cache and file server consult the same server to obtain the

actual location of file or directly and updated the cache.

4. Bulk Data Transfer

In this mechanism, multiple consecutive data blocks are trasferred from server to client. This

reduces file access overhead by obtaining multiple number of blocks with a single seek, by

formatting and transmitting multiple number of large packets in single context switch and by

reducing the number of acknowledgement that need to be sent. This mechanism is used as many

files are accessed in their entirety.


70/83


69

5. Encryption

This mechanism is used for security in Distributed systems. The method was developed by

Needham Schrodkar is used in DFS security. In this scheme, two entities which want to

communicate establish a key for conversation with help of authentication server.

DISTRIBUTED SHARED MEMORY (DSM) :

A distributed shared memory is a mechanism allowing end-users' processes to access shareddata without using inter-process communications. In other words, the goal of a DSM system is tomake inter-process communications transparent to end-users. Both hardware and softwareimplementations have been proposed in the literature. From a programming point of view, twoapproaches have been studied:

Shared virtual memory: This notion is very similar to the well-known concept of paged

virtual memory implemented in mono-processor systems. The basic idea is to group alldistributed memories together into a single wide address space. Drawbacks: suchsystems do not allow to take into account the semantics of shared data: the datagranularity is arbitrarily fixed to some page size whatever the type and the actual size ofthe shared data might be. The programmer has no means to provide informations aboutthese data.

Object DSM: in that class of approaches, shared data are objects i.e. variables withaccess functions. In his applications, the user has only to define which data (objects) areshared. The whole management of the shared objects (creation, access, modification) ishandled by the DSM system. In opposite of SVM systems which work at operating systemlayer, objects DSM systems actually propose a programming model alternative to theclassical message-passing.

In any case, implementing a DSM system implies to address problems of data location, dataaccess, sharing and locking of data, data coherence. Such problems are not specific to parallelismbut have connections with distributed or replicated databases management systems(transactionnal model), networks (data migrations), uniprocessor operating systems (concurrentprogramming), distributed systems.


71/83


70

What- The distributed shared memory (DSM) implements the shared memory

model in distributed systems, which have no physical shared memory- The shared memory model provides a virtual address space shared

between all nodes- The overcome the high cost of communication in distributed systems,

DSM systems move data to the location of access How:

- Data moves between main memory and secondary memory (within anode) and between main memories of different nodes

- Each data object is owned by a node- Initial owner is the node that created object- Ownership can change as object moves from node to node

- When a process accesses data in the shared address space, themapping manager maps shared memory address to physical memory(local or remote)


72/83


71

Advantages of distributed shared memory (DSM) :

Data sharing is implicit, hiding data movement (as opposed to Send/Receive inmessage passing model)

Passing data structures containing pointers is easier (in message passing model datamoves between different address spaces)

Moving entire object to user takes advantage of locality difference Less expensive to build than tightly coupled multiprocessor system: off-the-shelf

hardware, no expensive interface to shared physical memory Very large total physical memory for all nodes: Large programs can run more efficiently No serial access to common bus for shared physical memory like in multiprocessor

systems Programs written for shared memory multiprocessors can be run on DSM systems with

minimum changes

Algorithms for implementing DSM

Issues

- How to keep track of the location of remote data

- How to minimize communication overhead when accessing remote data- How to access concurrently remote data at several nodes

1. The Central Server Algorithm

- Central server maintains all shared data Read request: returns data item Write request: updates data and returns acknowledgement message

- Implementation A timeout is used to resend a request if acknowledgment fails Associated sequence numbers can be used to detect duplicate write

requests If an applications request to access shared data fails repeatedly, a failure

condition is sent to the application- Issues: performance and reliability- Possible solutions

Partition shared data between several servers Use a mapping function to distribute/locate data

2. The Migration Algorithm

- Operation

Ship (migrate) entire data object (page, block) containing data item torequesting location Allow only one node to access a shared data at a time

- Advantages Takes advantage of the locality of reference DSM can be integrated with VM at each node

- Make DSM page multiple of VM page size- A locally held shared memory can be mapped into the VM page

address space


73/83


72

- If page not local, fault-handler migrates page and removes it fromaddress space at remote node

- To locate a remote data object: Use a location server Maintain hints at each node Broadcast query

- Issues

Only one node can access a data object at a time Thrashing can occur: to minimize it, set minimum time data object resides

at a node

3. The Read-Replication Algorithm

Replicates data objects to multiple nodes DSM keeps track of location of data objects Multiple nodes can have read access or one node write access (multiple readers-

one writer protocol) After a write, all copies are invalidated or updated DSM has to keep track of locations of all copies of data objects. Examples of

implementations: IVY: owner node of data object knows all nodes that have copies PLUS: distributed linked-list tracks all nodes that have copies

Advantage The read-replication can lead to substantial performance improvements if

the ratio of reads to writes is large

4. The FullReplication Algorithm

- Extension of read-replication algorithm: multiple nodes can read andmultiple nodes can write (multiple-readers, multiple-writers protocol)

- Issue: consistency of data for multiple writers

- Solution: use of gap-free sequencer All writes sent to sequencer Sequencer assigns sequence number and sends write request to

all sites that have copies Each node performs writes according to sequence numbers A gap in sequence numbers indicates a missing write request:

node asks for retransmission of missing write requests


74/83


73

DISTRIBUTED SCHEDULING :

Scheduling refers to assigning a resource and a start time end to a task

Much of scheduling research has been done in Operations Research Community

e.g Job Shop, Flow shop scheduling etc.

Scheduling is often an overloaded term in Grids.

A related term is mapping that assigns a resource to a task but not the start time.

Systems taxonomy :

Parallel Systems

Distributed Systems

Dedicated Systems

Shared Systems

Time Shared e.g. aludra

Space Shared e.g. HPCC cluster

Homogeneous Systems

Heterogeneous Systems

Scheduling Regimens :

Online/Dynamic Scheduling

Offline/Static Scheduling

Resource level Scheduling

Application level Scheduling

Applications taxonomy :

Bag of tasksIndependent tasks

Workflowsdependent tasks

Generally Directed Acyclic Graphs (DAGs)

Performance criteria

Completion time (makespan), reliability etc.


75/83


74

Scheduling Bag of Tasks on Dedicated Systems:

Min-Min

Max-Min

Sufferage

Min-Min Heuristic :

For each task determine its minimum completion time over all machines

Over all tasks find the minimum completion time

Assign the task to the machine that gives this completion time

Iterate till all the tasks are scheduled

Max-Min Heuristic :

For each task determine its minimum completion time over all machines

Over all tasks find the maximum completion time

Assign the task to the machine that gives this completion time



76/83


75

Sufferage Heuristic :

For each task determine the difference between its minimum and second minimum

completion time over all machines (sufferage)

Over all tasks find the maximum sufferage

Assign the task to the machine that gives this sufferage



77/83


76

Grid Environments :

Time-shared resources

Heterogeneous resources

Tasks require input files that might be shared

Data transfer times are important


78/83


77


79/83


78

Scheduling Heuristic :

Schedule()

1. Compute the next Scheduling event

2. Create a Gantt Chart G

3. For each computation and file transfer underway

1. Compute an estimate of its completion time

2. Update the Gantt Chart G

4. Select a subset of tasks that have not started execution: T

5. Until each host has been assigned enough work

1. Heuristically assign tasks to hosts

6. Convert G into a plan

Sample Gantt Chart-


80/83


79

Scheduling Task Graphs

Task Graphs have dependencies between the tasks in the Application

Scheduling methods for bag of task applications cannot be directly applied


81/83


82/83


81

ALGORITHM FOR DISTRIBUTED CONTROL :

In a distributed operating system using a symmetric design for its kernel, each node in the system

has identical capabilities with respect to each OS control function. This provides availability of the

function in the presence of failure, and reduces delays involved in performing a controlled

function. However a symmetric design requires synchronization of activities in a various nodes to

ensure absence of race conditions over OS data structures. For example, even if each node iscapable of querying the resource status and performing allocation, only one node is permitted to

do so at any time. Multiprocessor operating system use this approach. However in a distributed

OS, data structure containing resource status information may be distributed in different nodes.

Presence of local memories and communication links creates difficulties in obtaining global state

information. This is the fundamental difference between distributed and non-distributed

algorithms.

Here two groups of distributed control algorithms discussed. First group of algorithms implements

mutual exclusion in the OS. Algorithms of the second group are used for distributed deadlock

handling.

Understanding distributed control algorithms :-

A distributed control algorithm (DCA) implements control activities in different nodes of the

system. Each DCA provides a service of some kind, e. g. providing mutual exclusion, and has

clients which can be user or OS processes. A DCA performs some action when a client needs

its service, or when an event related to its service occurs in the system.


83/83

82

entity called privilege token exists in the system. Only a process processing this token (called

the token holder) can enter a critical section. When a new process wishes to enter, it sends a

request to the token holder and waits till it receives the token.

system programming unit-4 by arun pratap singh

Documents