communication in distributed systems cheng-zhong xu

78
Communication in Distributed Systems Cheng-Zhong Xu

Upload: alicia-carroll

Post on 02-Jan-2016

222 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Communication in Distributed Systems Cheng-Zhong Xu

Communication in Distributed Systems

Cheng-Zhong Xu

Page 2: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

TaxonomyCommunication

Point-to-point Multicast group comm

Two-way comm One-way comm

Socket prog RPC/RMI Message-oriented

Page 3: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Outline

TCP/UDP socket programming RPC/RMI programming

– RPC: Remote Procedure Call– RMI: Remote Method Invocation

Distribute Objects– Local object vs remote object

Message-oriented comm

Page 4: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Limitations of Socket Programming Limited support for data structure in comm

– In Java TCP/UDP programming, data in transmission is defined as a stream of bytes

– client and server are coupled in development: they need to know how the data are interpreted at other side

Limited support for heterogeneity– Network protocols: Internet protocol masks the differences between

networks, but– computer hardware: e.g. data types such as integers can be represented

differently– operating systems, e.g. the API to IP differs from one OS to another

– programming languages, data structures (arrays, records) can be represented differently

– implementations by different developers, » they need agreed standards so as to be able to inter-work

Page 5: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Architectural heterogeneity: an example

Representations of integer 5 and String “JILL” on different machines:

• Original message on the Pentium (little endian)• The message after receipt on the SPARC (big endian)• The message after being inverted. The little numbers in boxes indicate the

address of each byte• Integers are reversed by byte ordering, but strings are not!!

Page 6: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Remote Procedure Call (RPC)Remote Method Invocation (RMI)

Middleware provides a simple programming abstraction and masks heterogeneity of client and server systems

Similar interface as in procedure call or method invocation between client and service– client sends command and arguments

– server computes

– Server sends back reply data

Implementation of the procedures or methods are on remote servers

Page 7: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Conventional Procedure Call

a) Parameter passing in a local procedure call: the stack before the call to readb) The stack while the called procedure is active

For example, count = read(fd, buf, nbytes);

Page 8: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Remote Procedure Call (RPC) Build a mechanism, which allows clients

– call procedures over a network as if the code was stored and executed locally

The mechanism automates the writing of data-handling and formatting code

Advantages: fewer errors, faster to code and evolve interfaces

Page 9: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

RPC Structure: Client/Server Proxies (stub, skeleton)

clientprogram

serverprogram

clientstub

serverskeleton

network

call callreturnreturn

Page 10: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Passing Parameters Steps involved in doing remote computation through RPC

2-8

Page 11: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Outline

TCP/UDP socket programming RPC/RMI programming

– RPC: Remote Procedure Call– RMI: Remote Method Invocation

Distribute Objects– Local object vs remote object

Message-oriented comm

Page 12: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Distributed Objects An object is an individual unit of run-time data storage used

as the basic building block of programs.– Each object is capable of receiving msgs, processing data, and sending

msgs to other objects; methods together form the object’s interface with the outside world or other objects.

– A class is the blueprint from which individual objects are created. Distributed objects have a separation between interfaces and

objects that implement the interfaces; interfaces and the objects are in different machines.

Allow programs to treat remote objects just like local objects– remote references– remote invocation

First good impl: DEC SRC network objects for Modula-3 Recent implementation: Java Remote Method Invocation

Page 13: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Organization of Distributed Objects Separate proxy for each remote object that is referenced proxy looks just like real object. but loaded at run-time

when remote object is to be bound. – implements the same interface

proxy’s methods do RPC to real object– handle arguments, return values, exceptions correctly

proxy code generated by RMI compiler

Page 14: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Remote Object References Naming a remote object (“global address”)

– machine (IP address) where actual object resides– port number, identifying the server that manages the object– (boot time)– unique object ID number (generated by server)– Object references can be passed between processes, as

parameters to method invocations Before invocation, object ref. must first bind to its

object. Binding leads to a placement of corresponding proxy

Programs referenced a remote object by using stand-in proxy object

Page 15: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Remote Invocation when a proxy method is invoked:

– invocation message sent to remote process» contains global address of object, method ID, arguments

– remote process invokes real object

– return message sent back to proxy object» contains return value or exception

– proxy method call returns correct value, or throws correct exception

Under the Hood each process keeps a table of all proxy objects

– maps global address to proxy object use table to maintain one-to-one mapping from

proxy object to remote object interactions with garbage collection

Page 16: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

compiler/linker

compiler/linker

Building an RMI App

clientcode

serverinterface

stubgenerator

serverskeleton

clientstub

clientapp

serverapp

Server code

serverimpl

Page 17: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

RMI Binding (rmi registry) How does a client find the service code Registry to keep track of distributed objects

– simply a name repository– provides a limited central management point

for RMI Server binds methods to names in the reg Client looks up names in the reg for

availability of an object

Page 18: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Interface Definition

import java.rmi.*;

public interface myServer extends Remote {public void doSomething( … ) throws RemoteException;

}

All remote objects are referenced through interfaces– It extends the Remote interface as all RMI

interfaces must– Methods must throw RemoteException

Page 19: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

// myClient// myServer object has registered itself as “xuObject”

import java.rmi.*;import java.rmi.server.*;

public class myClient {public static void main( String[] args ) {

System.setSecurityManager( new RMISecurityManager() );try {

myServer ro= (myServer)Naming.lookup(“xuObject”);ro.doSomething();

} catch (Exception e) {}}

}

Page 20: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Notes

All RMI-based app need to import– java.rmi and java.rmi.server packages

Inside the main method, the first thing is set the Java security manager, which grant or deny permissions on the operations

create a try/catch block that performs the remote method invocation

Page 21: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

// myServer implementationimport java.rmi.*;import java.rmi.server.*;import java.rmi.registry.*;public class myServerImpl extends UnicastRemoteObject

implements myServer {public myServerImpl() throws RemoteException{ }public void doSomething(… ) throws RemoteException{… …}public static void main( String[] args) {

System.setSecurityManager( new RMISecurityManager());try {

myServerImpl obj = new myServerImpl();Naming.rebind(“xuObject”, obj);System.out.println(“doSomething bound in registry”);

}catch (Exception e) {}}

}

Page 22: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Notes myServerImpl extends

– java.rmi.server.UnicastRemoteObject class, which defines a non-replicated remote object whose references are valid only while the myServer process is alive (Transient Objects).

– It supports point-to-point active object reference via TCP streams.

Bind vs rebind A remote implementation must have a zero-

argument constructor

Page 23: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Deployment Generate stub and skeleton

– rmic myServerImpl, which generates» myServerImpl_Stub.class, myServerImpl_Skel.class

– Since JDK 5.0, stub classes can be generated on-demand at runtime

rmiregistry– starts the registry service in the background

java ServerImpl– starts the server, which registers the remote object

with the registry java Client

Page 24: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Passing Arguments Passing primitive types (int, boolean, etc.) Remote object vs. non-remote objects

– remote object is an instance of RemoteObject– remote object refers to an object whose

method can be invoked from another JVM.– Remote objects passed by reference, while non-

remote objects passed by copy A difference between RPC and RMI

– passing objects, – passed object may interact with receipt itself

Page 25: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Parameter Passing Remote object vs Local Object

– Pass-by-value is ok for all distributed object references, but not efficient The situation when passing an object by reference or by value

– E.g. Client on machine A calls a method of remote object in machine C, which contains two parameters: O1 in machine A and O2 in machine B

Page 26: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Passing objects Object Serialization

– To be passed between client/server, objects have to be serialized

For example, – CloudServer, which do jobs for clients in clouds– Different clients may ask CloudServer do

different jobs– Each job is specified in arguments passed from

a client to the server

Page 27: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

//Task.javaimport java.io.*;public class Task implements Serializable{

public Object execute() { return null; }}

//foo.javapublic class foo extends Task{

int n;public foo (int m) {n=m;}public Object execute() {

return new Integer( n*n );}

}

Page 28: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

// CloudServer.java

import java.rmi.*;import java.util.*;public interface CloudServer extends Remote {

Object execute( Task o) throws RemoteException

}

Page 29: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

// CloudServerImpl.javaimport java.rmi.*;import java.io.*;import java.rmi.server.*;public class CloudServerImpl implements CloudServer {

public CloudServerImpl() throws RemoteException{}public Object execute( Task t)

throws RemoteException {return t.execute();

}public static void main (String args[]) {

System.setSecurityManager( new RMISecurityManager());try {

CloudServer csr = new CloudServerImpl();Naming.rebind(“CloudServer”, csr);

} catch (IOException ioe) {}}

Page 30: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

// example.java// java example registry-hostimport java.rmi.*;import java.io.*;

public class example{public static void main( String[] args ) throws RemoteException {System.setSecurityManager( … );

new example(args[0]);}

public example( String host ) {try {

CloudServer csr = (CloudServer)Naming.lookup(“rmi://”+host+“/CloudServer”);

System.out.println( csr.execute( new foo(10) );

catch (IOException ioe) {}

… … }

Page 31: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Running on different machines Server site

– CloudServer.class

– CloudServerImpl.class

– CloudServerImpl_Stub.class

– CloudServerImpl_Skel.class

– Task.class

java CloudServerImpl

Client site– CloudServer.class

– example.class

– CloudServerImpl_Stub.class

– Task.class

– foo.class

At client sitejava -Djava.rmi.server.codebase=‘…’ example server

Page 32: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Remote Object References

Asynchronous Client/Server Comm– Client calls CloudServer’s asynExecute(…)– On the completion of the work, the

CloudServer notifies the client of the result Client itself is declared as a Remote Object,

which reference is passed to CloudServer CloudServer calls workCompleted(…) of

the remote object.

Page 33: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

// CloudServer.java

import java.rmi.*;import java.util.*;public interface CloudServer extends Remote {

Object execute( Task work ) throws RemoteException

void asynExecute( Task work, WorkListener listener)throws RemoteException

}

Page 34: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Public class CloudServerImplextends UnicastRemoteObjectimplements CloudServer {

… …public void asynExecute(Task work,

WorkListener listener)throws RemoteException {

Object result = work.execute();listener.workCompleted( work, result );

}}

Page 35: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

// WorkListener.javapublic interface WorkListener extends Remote {

public void workCompleted( Task request, Object result )

throws RemoteException;}

WorkListener Remote Interface

Example.java will be modified to implement the remote interface. This turns example into a remote object

Page 36: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

import java.rmi.*;import java.io.*;public class example extends UnicastRemoteObject

implements WorkListener{public static void main( ) throws RemoteException {…}public void workCompleted( Task t, Object result)

throws RemoteException {System.out.println(“Async work result = “ + result);

}public example ( String host ) throws RemoteException {

try {CloudServer csr = (CloudServer)

Naming.lookup(“rmi://”+host+“/CloudServer”); csr.asynExecute( new foo(10), this );catch (IOException ioe) {}

}

Page 37: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Remarks Asynchronous RPC: A client and server

interacting through two asynchronous RPCs

Asynchronous RMI ??

Page 38: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

RPC vs RMI

RMI supports system-wide object references RMI can have object-specific stubs, not

necessarily have general-purpose client-side and server-side stubs

Pass objects as arguments (object migration) Migrated objects can interact with both original

and new hosts

Page 39: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Mobile Objects Objects be passed as arguments

– Single-hop vs multi-hop migration

– Dispatched by its host, or migrated according to its own itinerary

Mobile objects can be performed on hosts (cloud servers) according to their own agenda, under a security sandbox– access to local computing and data resources

Mobile objects can communicate back to their home hosts.

Weak mobilityStop before migration, and restart on arrival at a new server; running status not transferred

Strong mobilitySuspend before migration, and resume on arrival at a new server; running status carried over

Page 40: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Naplet: A mobile object middleware system

Naplet 0.2 source code availablehttp://www.ece.eng.wayne.edu/~czxu/software/naplet.html

Structured Itinerary Mechanism– Sequential/parallel/alternative visit of a group of servers

Naplet directory based location + Forward Pointer Message-oriented communication Persistent comm between mobile objects Un/secure execution

– Multithreaded objects– Secure access to local resource

Java based

Page 41: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

TaxonomyCommunication

Point-to-point Multicast group comm

Two-way comm One-way comm

Socket prog RPC/RMI Message-oriented

Page 42: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Message-Oriented Persistent Communication

Page 43: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Organization of Comm System

Examples: Email --- Client/Outbound Mail Server/Inbound ServerMPI --- Message Passing LayerDifference??

Page 44: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Another Example of Persistent Comm

Persistent communication of letters back in the days of the Pony Express.

Although the means of transportation and the means by which letters are sorted have changed, the mail principle remains the same

Page 45: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Persistence and Synchronicity Persistent vs Transient Communication

– Persistent: submitted messages are stored by the comm. system as long as it takes to deliver it the receiver

– Transient: messages are stored by the comm system as long as the sending and receiving app. are executing.

Asynchronous vs synchronous – Sync: a client is blocked until its message is stored in a

local buffer at the receiving host, or actually delivered to the receiver

– Async: A sender continues immediately after it has submitted its message for transmission. (The message is stored in a local buffer at the sending host, or otherwise at the first communication server.

Page 46: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

a) Persistent asynchronous communication E.g. Email

b) Persistent synchronous communication (??) (a weaker form: when the sender is blocked until its message is stored at receiver-side comm server )

Page 47: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

c) Transient asynchronous communication (e.g. UDP, asynchronous RPC). Transmission fails if the receiver is not executing at the time the msg arrives.

d) Receipt-based transient synchronous communication . Sender is blocked until the msg is stored in a local buffer at the receiving host. (Example?)

2-22.2

Page 48: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

e) Delivery-based transient synchronous communication at message delivery. (Sender is blocked until the msg is delivered to the receiver)

f) Response-based transient synchronous communication (e.g. RPC and RMI) (Sender is blocked until it receives a reply msg from the receiver)

Page 49: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Different Forms of Communication Persistent Asynchronous Persistent Synchronous Transient Asynchronous Transient Synchronous

– Receipt-based (weakest, e.g. async. RPC) – Delivery-based – Response-based (strongest, e.g. RPC)

Need for persistent comm. in middleware, large-scale system– Components of a large scale distributed system may not

always be immediately accessible– Failure should be masked with a recovery procedure– But, persistent comm. incurs long delays

Page 50: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Message-Oriented Transient Comm. Socket is a communication endpoint to which an appl can write data to

and from which incoming data can be read Socket primitives for TCP/IP.

Primitive Meaning

Socket Create a new communication endpoint

Bind Attach a local address to a socket

ListenAnnounce willingness to accept connections

AcceptBlock caller until a connection request arrives

Connect Actively attempt to establish a connection

Send Send some data over the connection

Receive Receive some data over the connection

Close Release the connection

Page 51: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Berkeley Sockets Connection-oriented communication pattern using sockets.

Page 52: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

MPI: Another Transient Comm Basic message-passing primitives of MPI.

Primitive Meaning

MPI_bsendAppend outgoing message to a local send buffer of MPI runtime

MPI_sendSend a message and wait until copied to local or remote buffer

MPI_ssend Send a message and wait until receipt starts

MPI_sendrecv Send a message and wait for reply

MPI_isend Pass reference to outgoing message, and continue

MPI_issendPass reference to outgoing message, and wait until receipt starts

MPI_recv Receive a message; block if there are none

MPI_irecv Check if there is an incoming message, but do not block

Page 53: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Synchrony in MPI

MPI_bsend Transient synchronous MPI_send (blocking send)

– Block the caller until the msg be copied to MPI runtime system at the receiver’s side; Receipt-based transient comm

– or until the receiver has initiated a receive operation delivery-based transient comm

MPI_ssend delivery-based MPI_sendrecv response-based

Page 54: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Message Queuing Model MQ: a software-engineering component used for

interprocess comm. It utilizes a queue for messaging—the passing of control or of content. [Wikipedia]

MQ provides a Persistent Asynchronous comm protocol– Sender and receiver do not need to connect to the msg

queue at the same time; msgs placed on the queue are stored until the recipient retrieves them.

– Most msg queues have set limits on the size of data that can be transmitted in a single msg. Those with no such limits are often referred to as mailboxes.

Page 55: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Four combinations for loosely-coupled comm. using queues (sender and receiver can execute completely independently of each other)

2-26

MQ Model

Page 56: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

MQ: Interface to a Queue

Primitive Meaning

Put Append a message to a specified queue (non-blocking)

GetBlock until the specified queue is nonempty, and remove the first message

PollCheck a specified queue for messages, and remove the first. Never block.

NotifyInstall a handler to be called when a message is put into the specified queue.

One-way Communication Primitives:

• Addressing: System wide unique destination queue• Active message: The handler to be executed on arrival at the dest.• Mobile object: messages in principle contains any data. How about objects?

Page 57: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Msg Oriented Middleware (MOM)

Many impl function internally in OS or appl– In UNIX: msgctrl(), msgget(), msgsnd()

Distributed impl across different systems for msg passing – Enhanced fault resilient: msgs don’t get lost even in the

event of a system failure. Examples:

– IBM’s WebSphere MQ (MQ series), – Oracle Advaced Queueing (AQ)– Microsoft’s MSMQ– Sun’s Java API: Java Message Service (JMS) since 2005– Amazon’s Simple Queue Service for building automated

workflow (aws.amazon.com/sqs)

Page 58: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

MOM (Cont’) Msgs are forwarded over comm servers and eventually

delivered to dest, even if it was down when the msg was sent.

Each app has its own queue and a queue can be read only by its assoc. app.

Multiple apps can share a single queue Guaranteed that a msg will eventually be inserted in the

recipient’s queue, but no guarantee about when, or if the msg will actually be read. – Immediate-term storage capacity for messages

Messaging domain: – Point-to-point vs publish/subscribe

Page 59: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

General Architecture: Source Queue, Destination Queue, or 3rd-party messaging service (Amazon) Mapping of queue-level addressing to network-level addressing. (analogous to DNS in

email system) Queue manager Special manager working as routers: forward msg between queue managers

Page 60: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Queue Manager, Router, and Overlay Network

2-29

• Static vs dynamic routing with routers• Multicasting with routers

Page 61: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Message Brokers (e.g. Amazon’s SQS Integration of legend applications with new ones to form a single coherent

distributed information system Message broker acts as an appl-level gateway to convert incoming msg to

a format that can be understood by the dest app. The general organization of a message broker in a msg-queuing system.

Page 62: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Amazon Simple Queue Service Amazon’s SQS offers a reliable, highly scalable, hosted

queue for storing messages as they travel between computers. (aws.amazon.com/sqs)

By using Amazon SQS, developers can simply move data between distributed components of their applications that perform different tasks, without losing messages or requiring each component to be always available.

Make it easy to build an automated workflow Pricing: $0.01 for 10K SQS requests.

Page 63: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Basic Queue Requests CreateQueue: Create queues for use with your AWS account. ListQueues: List your existing queues. DeleteQueue: Delete one of your queues. SendMessage: Add any data entries to a specified queue. ReceiveMessage: Return one or more messages from a specified queue. DeleteMessage: Remove a previously received message from a

specified queue. SetQueueAttributes: Control queue settings like the amount of time

that messages are locked after being read so they cannot be read again. GetQueueAttributes: See information about a queue like the number

of messages in it.

Page 64: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Example: IBM WebSphere MQ Launched in 1992, previously known as MQ series De-facto standard for messaging across platforms Queue Manager: handles storage, timing issues, triggering, and all other

functions not directly actual movement of data QM comm with outside via local Binding, or remote Client connection

Page 65: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Message Channels MC is a unidirectional, reliable connection between a

sending and a receiving queue manager Each end is managed by a Message Channel Agent (MCA)

– check send queue, wrap messages into transport-level packets, send to associated receiving MCA

– listening for incoming packets, unwrap them, store into queues

Attribute Description

Transport type Determines the transport protocol to be used

FIFO deliveryIndicates that messages are to be delivered in the order they are sent

Message length

Maximum length of a single message

Setup retry count

Specifies maximum number of retries to start up the remote MCA

Delivery retries

Maximum times MCA will try to put received message into queue

Page 66: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Message Transfer Address: queue manager + destination queue; Each queue manager has a system-wide unique id; Local aliases for queue

manager names Routing tables: (Dest, SendQueue) Transfer along a channel can take place only if both MCAs are up and

runing

Page 67: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Message Queue Interface Primitives available in an MQSeries MQI

Primitive Description

MQopen Open a (possibly remote) queue

MQclose Close a queue

MQput Put a message into an opened queue

MQget Get a message from a (local) queue

MQSerie also provides facilities to signal applications when msgs arrive. For more info, see http://www.redbooks.ibm.com/abstracts/sg247128.html

Page 68: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Messaging Domain

Point-to-Point

Publish/Subscribe– Clients address messages to a topic

Page 69: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Publish/Subscribe Messaging Request/Reply (RPC, MRI) model has limitations in two

types of applications– Update of state info that changes over time

» Send requests regularly for update of the new info. NOT efficient– Action in response to event occurrence:

» Send event info to ALL nodes who are interested in Publish/Subscribe messaging

– Any number of customers of info can receive messages that are provided by one or more producers of the info

– Publisher: producer of the info– Subscriber: consumer of the info– Topic (or named logical channel) to be subscribed by consumers to

register their interests– Publish/subscribe broker: track subscriptions to individual topics and

provide facilitates for a publisher to publish msgs on a given topic Topic-based vs content-based vs hybrid systems

Page 70: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

JMS Programming Available in Java System Message Queue V3.6 Interfaces:

– ConnectionFactory» Administrated object used to create a connection to JMS provider» Defined by administrator, rather than programmer

– Connection» A conn represents a comm link between app and msg server» Administrated object

– Destination: where msg is delivered and consumed» Queue or topic

– MessageConsumer– MessageProducer– Message– Session: Single threaded context for sending and receiving messages

http://www.sun.com/software/products/message_queue/index.xml

Page 71: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

JMS Programming Model

Page 72: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Producer (1/3)injects resources for a connection factory, queue, and topic:

@Resource(mappedName="jms/ConnectionFactory")private static ConnectionFactory connectionFactory;

@Resource(mappedName="jms/Queue")private static Queue queue;

@Resource(mappedName="jms/Topic")private static Topic topic;

Assigns either the queue or topic to a destination object, based on dest type:Destination dest = null;try {  if (destType.equals("queue")) {    dest = (Destination) queue;  } else {    dest = (Destination) topic;  }} catch (Exception e) {}

Page 73: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Producer (2/3)Creates a Connection and a Session:

Connection connection =    connectionFactory.createConnection();Session session = connection.createSession(false,    Session.AUTO_ACKNOWLEDGE);

Creates a MessageProducer and a TextMessage:MessageProducer producer = session.createProducer(dest);TextMessage message = session.createTextMessage();

Sends one or more messages to the destination:for (int i = 0; i < NUM_MSGS; i++) {  message.setText("This is message " + (i + 1));  System.out.println("Sending message: " + message.getText());  producer.send(message);}

Page 74: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Producer (3/3)Sends an empty control message to indicate the end of the message stream:

producer.send(session.createMessage());

Sending an empty message of no specified type is a convenient way to indicate to the consumer that the final message has arrived. Closes the connection in a finally block, automatically closing the session and MessageProducer:

} finally {  if (connection != null) {    try {      connection.close();    } catch (JMSException e) {    }  }}

Page 75: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Async Consumer1. injects resources for a connection factory, queue, and topic.2. Assigns either the queue or topic to a destination object, based on the specified dest type.3. Creates a Connection and a Session.4. Creates a MessageConsumer.5. Creates an instance of the TextListener class and registers it as the message listener for the MessageConsumer:

listener = new TextListener();consumer.setMessageListener(listener);

6. Starts the connection, causing message delivery to begin.7. Listens for the messages published to the destination, stopping when the user types the character q or Q:

System.out.println("To end program, type Q or q, " +  "then <return>");inputStreamReader = new InputStreamReader(System.in);while (!((answer == 'q') || (answer == 'Q'))) {  try {    answer = (char) inputStreamReader.read();  } catch (IOException e) { }}

8. Closes the connection, which automatically closes the session and MessageConsumer.

Page 76: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Async Consumer (2/2)When a message arrives, the onMessage method is called automatically.The onMessage method converts the incoming message to a TextMessage and displays its content. If the message is not a text message, it reports this fact:

public void onMessage(Message message) {  TextMessage msg = null;

  try {    if (message instanceof TextMessage) {      msg = (TextMessage) message;      System.out.println("Reading message: " + msg.getText());    } else {      System.out.println("Message is not a " +  "TextMessage");    }  } catch (JMSException e)   } catch (Throwable t) {  }}

Page 77: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Email vs Message Queue Email is a special case of Message Queue

– Email aims at providing direct support for end users

– Email makes direct use of the underlying transport services (SMTP over TCP); routing is left out

General MQ can satisfy a different set of requirements:

» guaranteed message delivery,

» message priorities,

» logging facilities,

» efficient multicasting,

» load balancing,

» fault tolerance, etc.

Page 78: Communication in Distributed Systems Cheng-Zhong Xu

© C. Xu, 1998-2011

Comm in DS: In Summary

Communication

Point-to-point Multicast group comm

Two-way comm One-way comm

Socket prog RPC/RMI Message-oriented