distributed systems santa clara university 2016tschwarz/coen 317/communication.pdf · remote object...

CommunicationDistributed Systems

Santa Clara University 2016

Protocol Stack• Each layer has its own protocol • Can make changes at one layer without changing

layers above or below • Use well defined interfaces

Protocols

• OSI model was never popular • Physical layer:

• Transmission of bits • Data link layer:

• Organizes bits into frames • Add checksum to frames in order to detect /

correct errors

Layered Protocols• Open Systems Interconnection Reference Model

Layered Protocols

Example Data link layer

Transport Protocols

• Transport layer • Offer a reliable connection

• TCP protocol vs. UDP protocol • There are many new proposals for the transport

layer • E.g. Real-time transport protocol (RTP) for real-

time data transfer

Transport Protocol• Client Server TCP

• TCP not tailored for synchronous request-reply behavior of client-server interaction

• Can use UDP with added reliability features • Transactional TCP:

• Proposal that combines connection setup with sending request.

Higher Level Protocols• Session Protocol

• Enhancement of transport layer: • Dialog control • Synchronization • Not present in IP suite

• Presentation layer: meaning of bits • Application layer:

• OSI intended a collection of network applications • email, file transfer, terminal emulation

Middleware Protocols

• Middleware lives at the application layer • Can contain general purpose protocols that

warrant their own layer • Examples for middleware services

• Authentication protocols • Authorization protocols • Distributed commit protocols

Middleware Protocols

• Remote procedure calls • Remote object invocation • Message queuing systems • Communication of continuous media through

streams

Network Model for networked communication

Remote Procedure Calls• Birrell & Nelson (1984)

• Programs call procedures on other machines • Remote Procedure Call (RPC) • When process on A calls procedure on B, calling

process on A is suspended • Information is passed through the parameters

from A to B and through the result from B to A

Remote Procedure Calls• Conventional procedure call uses stack

Remote Procedure Calls• Conventional procedure call:

• Call by value • Call by reference • Call by copy / restore

• Caller copies variable to stack • Callee copies overwrites variable on the stack

• Almost like call by reference • But not if same parameter occurs several

times in the parameter list

Remote Procedure Calls• RPC

• Make remote procedure call look like a local one • Use stubs at client and server

Remote Procedure Calls• RPC execution

1. Client procedure calls the client stub in the normal way 2. Client stub builds a message and calls the local OS 3. Client’s OS sends the message to the remote OS 4. Remote OS gives message to the server stub 5. Server stub unpacks parameters and calls server 6. Server does the work and returns the result to the stub 7. Server stub packs result in a message and calls local OS 8. Server’s OS sends message to client’s OS 9. Client’s OS gives message to the client stub 10.Stub unpacks the result and returns to the client

Remote Procedure Calls

Remote Procedure Calls• Passing reference parameters

• How to pass pointers: • Not at all (pointer to an object) • Pack what is pointed to into the message

• as in copy / restore

Remote Procedure Calls• Caller and callee need to use the same standard

for formatting data

A. Message on Pentium B. Message as received on SPARC C. Message on SPARC after reformatting

Remote Procedure Calls• Example


• To allow implementation of stubs for RPC • Define interface • Interface Definition Language

Remote Procedure Calls• RPC model assumes that caller and callee only

communicate through message passing • Sometimes caller and callee live in the same

machine • IP calls to the same machine are unnecessarily

complex

Doors

• Some OS offer RPCs for processes collocated on the same machine

• Doors • OS needs to support doors • Sever must register a door before it can be called

• OS returns a door identifier • E.g. Solaris: each door has a file name

Remote Procedure Calls• Asynchronous RPC

• Request reply behavior can prevent client from doing useful work • as long as it does not depend on the result

• Asynchronous RPC is not blocking


(a) Traditional RPC (b)Asynchronous RPC

Remote Procedure Calls• If server gives a result, can combine two

asynchronous RPC to yield deferred synchronous RPC

Remote Procedure Calls• One way RPC:

• Client does not wait for an acknowledgment

DCE RPC• Distributed Computing Environment

• Middleware system • Layer of abstraction between os and distributed

application • Uses client server model

• DCE consists of (among others) • distributed file service • directory service • security service • distributed time service

DCE RPC• RPC system allows accessing remote services by

calling a local procedure • RPC locates correct server and binds client and

server software • Handles message transport, fragmentation and

reassembly • Data conversions

• Interface Definiton Language • IDL files define types, constants, … for

parameter marshaling • Use IDL compiler to generate header file, client

stub, and server stub

DCE RPC

DCE RPC

DCE - RPC

Client to Server Binding in DCE

DCE RPC

• Semantic options • At most once operation • Idempotent

Remote Object Invocation• RPC principles can be applied also to Remote

Objects • Sophisticated systems CORBA and DCOM

• Objects encapsulate data (state) and methods • Methods are made available through an interface

• Distributed Systems: • Separate between interfaces and objects

Remote Object Invocation

Remote Object Invocation• When a client binds to a distributed object • Implementation of the interface — the proxy — is loaded

into the client’s address space • Works like a client stub for RPC

• Proxy: • marshals method invocation into messages • unmarshals reply messages

• Actual object sits at a server machine • Skeleton — server stub:

• Unmarshals method invocations • Marshals replies

Remote Object Invocation• Objects can be compiled

• Compile-time objects• Define an object by a class implemented in Java/C

++, … • Compilation of class definition gives code to

instantiate Java / C++ … objects • Interfaces are compiled into client-side and server-

side stubs

Remote Object Invocation• Compile time objects depend on a particular programming

language • Alternative:

• Explicitly instantiate objects at run-time • Allows development in several programming languages • But need to result in objects that can be invoked from a remote

machine • Common method: object adapter

• Wrapper around the implementation • Objects are defined in terms of their interface

• Implementation is registered at an adapter • Adapter makes interface available for remote invocations

Remote Object Invocation• Persistent vs. Transient Objects

Remote Object Invocation• Binding a client to an object

• Systems with distributed objects can provide systemwide object references • Can be passed freely around

• Client with object reference needs to bind to the referenced object • Binding places proxy into the client’s address

space

Remote Object Invocation• Implicit binding:

• Client can invoke methods using a reference to the object • Implementation needs to allow client to bind to

the object • Explicit binding

• Client needs to explicitly bind and obtain a pointer to a distributed object


Code with (a) implicit binding using only global references and (b) with explicit binding with local and global references

Remote Object Invocation• Possible Implementation of implicit binding

• Object reference gives • network address of server • endpoint identifying the server that manages

the object • endpoint = local port dynamically assigned

by server’s OS • indication of which object

Remote Object Invocation• Drawback

• Server crashes and endpoint changes • Object references are now invalid • Could use a local daemon to keep track of the

endpoints • But now server cannot be relocated

• So use a location server • which turns out to not be scalable

Remote Object Invocation• Full object reference

• can also identify network protocol etc. • could have an implementation handle that

contains a complete implementation of the proxy • Client follows link and installs the proxy

Remote Object Invocation• After binding

• Client issues remote method invocation - RMI • Difference between RMI and RPC

• RMI supports systemwide object references • Support of RMI

• specify interface with an interface definition language • or: use a single language to generate stubs

automatically • This is static invocation

• interface of an object are known when client application is developed

Remote Object Invocation• Dynamic Invocation:

• compose a method at runtime • Application selects at runtime which method it

will invoke • invoke(object, method, input_par, output_par)

Remote Object Invocation• Example:

• Static invocation • fobject.append(integer)

• Dynamic invocation • invoke(fobject, id(append), int)

• where id(append) returns an identifier for a method “append”

Remote Object Invocation• Example:

• Object browser that examines objects • Browser support remote object invocation • Binds to a distributed object and presents the

objects interface to the user • At this point, user decides on a method and

parameters

Remote Object Invocation• Passing parameters

• Example: All objects are distributed objects • Use object references to interact with methods • References are passed by value and copied between

machines • For efficiency: treat remote and local objects

differently • If reference is remote:

• Copy reference: pass by reference • If reference is local:

• Referred object is copied pass by value


Client program on A, server program on C Client uses local and remote reference as parameters in a RMI Copy O1 to server, pass remote reference

Remote Object Invocation• Invoking a method with an object reference can

result in copying an object • Cannot hide this

• there are suddenly two replicas of the object • Need to make a distinction between local and

distributed objects • Violates transparency • Makes it harder to write distributed applications

Example 1: DCE Remote Objects

• DCE Distributed Object Model • Added as extensions to their IDL • With C++ language bindings

• Objects are created by a server • creates local C++ objects • makes methods available to clients

• Two types of objects: • Distributed dynamic object• Distributed named object

Example 1: DCE Distributed Object Model

• Distributed dynamic object • Server creates local object on behalf of a client • Accessible only to that client • Clients call a create RPC • DCE runtime administers the new object and associates it with the

client • Distributed named object

• Created by server to be shared amongst clients • Registered with a directory service • Client can look up directory service and bind to the object • Registration gives

• Unique identifier for the object • Info on how to contact the objects server


Dynamic (a) vs named objects in DCE


• Remote object invocation • done with RPC • To invoke a method, client passes to the server

• method identifier • identifier of interface that contains the method • parameters

• Server maintains an object table • Objects can be stored on disk instead of staying in

memory • Server then might have to retrieve the object first


• DCE have no mechanism to handle transparent object references

• Clients can use binding handles associated with a named object • Contains identification of an interface of the

object • transport protocol used to communicate with

server • server’s host address and endpoint

Example 2: Java RMI• Java:

• Distributed objects are part of the language • Tries to maintain high transparency

• Differences between local and remote objects are hidden (unless it become to inefficient)

Example 2: Java RMI• Java only uses remote objects for distributed

objects • State always resides on a single machine • Interface can be made available to remote

processes • Interfaces are implemented by using a proxy • Proxy appears as a local object in the client’s

address space

Example 2: Java RMI• Differences between local and remote objects

1.Cloning local and remote objects is different • Cloning a local object results in a new object

with the same state • Cloning a remote object can only be done at

the server • Hence, proxies are not cloned • Clients need to bind to the clone if they want

to use it

Example 2: Java RMI• Differences between local and remote objects

2.Blocking an object • Objects can be constructed as “monitors" • by declaring a method as “synchronized" • If two processes call simultaneously a synchronized method, one has to be

blocked • Could block client process in the client stub

• Problem: Need to synchronize between clients on different machines (which is complicated)

• Could only block at the server • Problem: What happens if a client fails (which results in

complications) • Therefore: Java RMI only block object on proxies • Remote objects are not protected natively against simultaneous access

from processed operating on different proxies • Will have to use an explicit distributed locking technique

Example 2: Java RMI• Java Remote Object Invocation

• Java hides most of the differences between local and remote objects

• Primitive or object type can be passed as a parameter to an RMI • If it can be marshaled

• Called serializable• Example: file descriptors and sockets depend

on platform —> cannot be serialized

Example 2: Java RMI• When passing objects during an RMI

• Local objects are passed by value (even if they are arrays)

• Remote objects are passed by reference • Java RMI reference to a remote object

• network address and endpoint of server • local identifier of the object in the server’s

address space • protocol stack used for communication between

server and client

Example 2: Java RMI• Remote object is build from two different classes

• Server class: • implementation of server side code

• Client class: • implementation of a proxy

• Proxies can be marshaled • Can be sent to another client • Serve as references to remote objects

Example 2: Java RMI• Marshaling proxies:

• First possibility: Convert state and code into a byte stream • Would lead to very large references

• Better alternative: Generate implementation handle • Specifies which classes are needed to

construct the proxy • Server might need to download classes

Comparison• Java RMI allows passing proxies as parameters

because every client and server runs the same JVM

• DCE does not share platform, so cannot pass stubs • DCE needs to link to a dynamically to a locally

available stub • But can then use this stub as a parameter in an

RPC

Message oriented communication

• RPC and RMI can hide communication in distributed systems

• Need for alternative communication services • Use explicit messaging

Persistence and Synchronization

Communication System


• Persistence • Message is stored until it can be delivered

• Sender or receiver might be down while message travels between communication servers


• Transient communication • Message is stored by the communication system

only as long as sender and receiver execute


• Asynchronous communication • Sender continues immediately after submitting

message for transmission


• Synchronous communication • Sender blocks until its message is

• stored in the local buffer at the receiver • or actually delivered to the receiver


• Persistent asynchronous communication • Electronic mail

• Persistent synchronous communication • Messages are stored only at the receiving host • Sender is blocked until message reaches

receiver’s buffer • (But receiver does not have to run)


• Transient asynchronous communication • UDP • One-way RPC

• Transient synchronous communication • Sender is blocked until the message is stored in

a local buffer at the receiving host • Sender receives ack • Sender continues

• Asynchronous RPC

(a) persistent asynchronous (b)persistent synchronous (c) transient asynchronous (d)receipt based transient

synchronous (e) delivery based transient

synchronous (f) response based transient

synchronous


• Persistent communication is necessary in large-scale and widely dispersed interconnected networks

• Persistency is useful when dealing with failure

• Berkeley UNIX sockets interface

Message Oriented Transient Communication


General pattern for socket based communication


• Message passing interface (MPI) • Standard for message passing • Supports several communication models

Message Oriented Persistent Communication

• Important class of middleware services: • Message Queuing Systems• Message Oriented Middleware (MOM)

• support persistent asynchronous communication • offer intermediate-term storage capacity for

messages


• Message Queueing Model • Applications communicate by inserting messages in specific

queues • For sender:

• Guarantee that message will be eventually inserted into the recipients queue


• Message Queuing System Interface


• Message queuing system • Many allow installing a handler as a callback

function • Invoked when message is put into the queue

• Allow daemon etc to monitor queue for incoming messages


• Message-queueing system architecture • Local: on the same machine or the same LAN • Queues distributed over network


• Message Queueing System Architecture • Queues managed by queue manager • Some act as routers / relays

• Generate an overlay network • Existence simplifies network administration

• Only routers need to be updated when a queue is moved, inserted, or deleted

• Can also be used to log messages • for security, fault tolerance


• Message Brokers • Integrate existing and new applications

• Applications need to understand messages • Sender can format outgoing messages

according to the receiver • or: Use a common message format • or: Message brokers convert incoming

messages to a format understood by receiver


• Message queuing systems are almost like email servers • But:

• email servers directly support endusers • have additional requirements such as

automatic message filtering, …


• IBM MQSeries


• IBM MQSeries • Queues are managed by queue managers • Message channels connect two queue managers

• unidirectional • reliable • managed by Message Channel Agent (MCA)

• Queue managers can be linked into the same process as an application • Queues are hidden to application • Standard interface for messaging


• IBM MQSeries • Message channels

• one send queue • one receive queue • both MCA need to be up for sending a message

• Initialization: • Manually • or: application starts MCA • or: Send queue triggers when message is put into queue, which

starts a handler, to start the sending MCA • or: start MCA over the network

• Termination: • Automatic after a specific time without sending messages


Attributes for message channel agents


Routing tables and aliases for MQSeries

Stream Oriented Communication

• Streams: • Timing is crucial

• Continuous media • Meaning of message depends on temporal

relationship to previous messages • Motion , Audio

• Discrete media • text, still images


• Data stream: sequence of data units • Asynchronous transmission mode:

• Data items are transmitted in order • without timing constraints

• Synchronous transmission mode: • Data items are transmitted in order • Maximum end-to-end delay for each unit in the stream

• Isochronous transmission mode: • Data units are transferred on time • Maximum and minimum end-to-end delay

• “Bounded delay jitter"


• Simple streams • Complex streams

• consist of several related simple streams, the sub streams


(a) stream between two processes

(b) stream between two devices


• Multicasting • Receivers can have different requirements • Use filters to adjust quality


• Quality of Service (QoS) • Flow specification: bandwidth requirements,

transmission rates, delays, …


Token bucket algorithm

Generate tokens at constant rate If bucket is full, tokens fall away Each time application sends data, needs to remove tokens from bucket


• Currently, no model for • specifying QoS parameters • describing resources in a communication system • translating QoS parameters to resource usage


• QoS protocol: Resource reSerVation Protocol (RSVP)


• RSVP • Senders provide flow specification • Hand it over to RSVP process • RSVP process stores specification • Sender sets up path to receiver(s)

• providing flow specification to all intermediate nodes • RSVP server when receiving a reservation request:

• Checks whether enough resources are available • Checks whether receiver has permission to make

the reservation


• Stream synchronization • Sub streams in a complex stream need to be

synchronized • Simple form: discrete data stream (slides) and

continuous data stream (audio) • Complex form: Synchronizing video and audio

stream, two audio streams for stereo (with max. jitter of less than 20 μsec

• Need to synchronize between data units


• Explicit synchronization at the data level


• Synchronization at high level


• Synchronization at high level: • Multimedia middleware offers interfaces for

controlling video and audio streams • Multiplex different streams into a single stream:

• MPEG streams

distributed systems santa clara university 2016tschwarz/coen 317/communication.pdf · remote object...

Documents