1 interprocess communications natalia khuri & seetharam samptur
TRANSCRIPT
1
Interprocess Communications
Natalia Khuri &
Seetharam Samptur
2
Outline
• Introduction to IPC • Computer networks• Protocols for remote operations • Remote Procedure Calls• Conclusion
3
What is IPC?
• IPC mechanisms allow processes to communicate and to synchronize their actions without sharing the same address space.
• IPC mechanism in a distributed system should make communication transparent.
4
Traditional IPC
• How do we pass information from one process to another?– Pipes, shared memory.
• How do we avoid race conditions and achieve mutual exclusion?– Mutexes, lock variables, semaphores.
• How do we schedule processes when dependencies are present?– Batch, interactive, real-time.
5
Distributed IPC
• How do we support communication over a network?
• How do we hide the distinction between local and remote communication?
• How do we deliver information with minimum latency, maximum throughput and minimum jitter?
• How do we shield one process from failures of another?
• How do we ensure security?
6
Distributed Communication
• All communication in distributed system is based on low-level message passing as offered by the underlying network.
• Process A builds a message in its own address space and executes a system call that causes the operating system to send the message over the network to process B.
7
What Do We Need?
• Good network infrastructure.• Efficient protocol stack.• Transport protocol to handle various types
of data.• Communication models.• Security.
8
Network Requirements
• Ability to transfer any data type:– Bulk, voice, video.
• Minimum latency, maximum throughput, minimum jitter.
• Error detection and correction:– Physical medium is never error-free.
• Different types of networks:– Local Area Networks (LAN)– Metropolitan Area Networks (MAN)– Wide Area Networks (WAN)
9
Asynchronous Transfer Mode
• Communication technology that uses high-bandwidth, low-delay transport technology, and multiplexing techniques.
• ITU standard for cell relay in which multiple service type (such as voice, video, or data) are conveyed in fixed-length.
• ATM networks are connection-oriented.• ATM Services:
– Constant/variable bit rate– Available/unspecified bit rate
10
Switching
• Packet Switching:– No predefined paths, e.g. traditional LAN.
• Circuit Switching:– Specific path set up for each connection, e.g.
telephone networks.
• ATM Switching:– Hybrid of packet and circuit switching.– Multiple connections over a predefined path.
11
ATM Network Operation
ATM devices:– ATM Switches.– ATM endpoints, e.g. workstations, video
coders/decoders.– User-Network Interface (UNI) & Network-Node
Interface (NNI).
12
ATM Virtual Connection• Virtual Channel:
– End-to-end connection
• Virtual Path:– Bundle of virtual channels that have local significance.
13
ATM Path SwitchingATM Switching:• Uses VCI & VPI from the cell.• Uses translation/routing table.
14
ATM Protocol Data Unit
Generic Flow Control Virtual Path Identifier
Virtual Path Identifier Virtual Channel Identifier
Virtual Channel Identifier
Virtual Channel Identifier Payload Type Cell Loss Priority
Header Error Control
48-byte Payload
Bit-8 Bit-4 Bit-1
5-byte Header
48-byte Payload
ATM Protocol Data Unit [CELL]
15
Why 48-byte Cells?
• Telephone quality audio is sampled at 8000 Hz using 8-bit samples.
• PTTs send a packet every 6 msec each containing 48 bytes of audio.– (1/8000) * 48 = (125 usec) * 48 = 6 msec
• Smaller header is required to assist in faster switching of packets in the network.
16
Protocol Organization
• Different agreements (protocols) are needed in message exchanges:– Number of volts per bits– Data type representation– Error detection and recovery
• The Open Systems Interconnection Reference (OSI) model identifies the standard rules that govern the format, contents, and meaning of the messages.
17
OSI Layers
• Physical Layer: – Interface to physical medium.
• Data Link Layer:– Reliable transit of data over the physical link.
• Network Layer:– Routing packets in the network.
• Transport Layer:– Consistent end-to-end delivery of the packet to
applications.
18
Mapping ATM to OSI
• Physical Layer:– medium dependent
transmissions.
• ATM layer: – Connection setup and
passing of the cells.
• ATM Adaptation Layer:– isolates higher layer
protocols from ATM processes.
Physical Layer
ATM
ATM Adaptation Layer
Upper Layer Applications
ATM Layers
19
Demultiplexing
• ISO OSI model allows multiplexing and demultiplexing in six of the seven layers.
• Faster in ATM:– Fewer layers in ATM stack.– Support for virtual circuits at MAC level.– Virtual channels expedite demultiplexing.
20
Classes of Communication Protocols
• Four classes of communication protocols:– Remote Operations (Client/Server,
Request/Response)– Bulk data management (File transfer protocol)– One-to-many communication
(Multi-/Broadcasting)– Continuous Media (Video/Audio)
• The diversity of communication requirements in a distributed system prohibits the use of a single transport protocol.
21
Remote Operations
• The most basic form of communication in distributed systems.– Process A (client process) sends a message to
process B asking it to do some work.– Process B (server process) carries out work and
returns a message with the result.
• Client and server can be the same process.
• Return message can merely be ACK.
22
Bulk Data Transfer
• Very large request or response messages.• Bulk data transfer can be viewed as a
special form of remote operation.– When data goes to the client, the operation
can be viewed as a read operation.– When data comes from the client, the
operation can be viewed as a write operation.
• Request/response protocols are very efficient.
23
One-to-Many Communication
• Services that use replication to tolerate failures.
• Large amounts of internal communication is needed to keep the replicas consistent.
• It is important that this communication reaches all replicas in a well-defined order.
• In practice, replication protocols are always multicast protocols.
24
Continuous Media
• Multimedia networks require low-latency and constant-rate data transport for continuous media and low-latency for “normal” data.
• ATM networks appear to be good at allowing the mixture of these data types.
25
Fundamental Properties of Protocols
• Message delivery:– At-least-once.– At-most-once.
• Feedback:– End-to-end application-level acknowledgement.– Error report on failures.
• Low end-to-end latency.• Support large request/response messages.
26
Communication Failures
• Networks exhibit failures that manifest themselves as lost packets.
• If not all packets are lost, these failures can be corrected using feedback:– Acknowledgements, timeouts.
• Communication protocols cannot be made completely reliable:– Client cannot distinguish a server that went
down from one that has become disconnected.
27
Unresponsive Server
Client Process Server Process
Request
Client Process Server Process
Request
Response
28
Amnesia Failure
• A processor suddenly stops (crashes) and forgets all its state.
• It resumes operations from the initial state (reboots).
• At-most-once message delivery cannot be achieved:– Aim for at-least-once message delivery.
29
Recovering From Failure
• Session management: – Unique session identifier.– Guaranteed at-most-once message delivery.
• End-to-end acknowledgement:– No dependence on protocol stack.– Positive feedback about delivery and response
can only be provided by an end-to-end application level acknowledgement.
30
At-most-once Delivery
• Session management is essential to provide at-most-once delivery.
• Delta-T protocol– Sessions are alive until packets are
exchanged and until a no-traffic timer expires.– New sessions are created by timeouts that
last for a period of ∆T.– ∆T = transmission delay + max. # of retrans.
+ time interval between retransmissions.
31
Delta-T ProtocolClient Server
∆T
Request
End of session
Delta -T protocol
32
Two-way Handshake with Piggybacking
• Positive feedback when no failure occurs. • Protocol:
– Client sends a request and starts a retransmission timer
– Server responds to the client before the retransmission timer expires. The response is also an acknowledgement for the request.
– Client sends the next request, which also acts as an acknowledgement for the prior response.
– If the client does not have any request to send, the client will acknowledge the last response before terminating.
33
Two-way Handshake with Piggybacking
Client Server
Request
Two-way handshake with piggybacking
Response/ACK
Request/ACK
Response/ACK
34
Optimized Two-way Handshaking
• Error report on failures.• Protocol:
Client sends a request and starts a timerWhile retransmission count is not zeroDo
If ACK receivedSend request header only
ElseSend request
Restart timerReduce retransmission count
End While
35
Optimized Two-Way Handshaking
Client Server
Request-0
Optimized Two-way handshake with piggybacking
ACK-0
Response
Seq #0Timer = sACK = False
Timer Expired Request-0 (R)
Seq #0Timer = sACK = TRUE
Timer ExpiredRequest-0
(R)
ACK-0
Seq #1Timer = sACK = False Request-0
36
Packet Blast Protocol
• Support large request and response messages.
• Fixed number of packets sent as a blast and acknowledged as one unit.
• Retransmit all packets in the blast if no acknowledgement received.
37
Packet Blast ProtocolClient Server
Request-0
Packet Blast Protocol
ACK-0
Blast 0Timer = s
Timer Expired Request-0 (R)
Request-1
ACK-1
38
Messages• Initially: hand-coded messages to send
requests and responses:– Hand-coding messages gets tiresome.– Need to worry about message formats.– Have to pack and unpack data from messages.– Have to decode and dispatch messages to
handlers.– Messages are often asynchronous.
• Messages are not a natural programming model (Birrell and Nelson, 1984):– Could encapsulate messaging into a library.– Invoke library routines to send a message.
39
Procedure Calls• Procedure calls - more natural way to
communicate:– Every language supports them.– Natural for programmers to use.
• Idea: Servers can export a set of procedures that can be called by client programs– Similar to module interfaces, class definitions, etc.– Clients just do a procedure call as it they were
directly linked with the server.– Under the covers, the procedure call is converted
into a message exchange with the server.
40
Implementing RPC• No architectural support for making
remote procedure calls.• Simulate it with available tools:
– local procedure calls and – sockets for network communication.
• This simulation makes remote procedure call a language level construct as opposed to sockets, which are an operating system level construct:– Compiler knows that remote procedure calls
need special code.
41
Implementing RPC
• Trick:– Create stub functions that make it
appear to the user that the call is really local.
– A stub function looks like the function that the user intends to call but really contains code for sending and receiving messages over a network.
42
Functional Steps in RPC
W. Richard Steven “UNIX Network Programming”
43
Benefits
• Procedure call interface.• Writing applications simplified:
– RPC hides all network code into stub functions.
– Application programmers don’t have to worry about details:• Sockets, port numbers, byte ordering.
• Presentation layer in OSI model.
44
Implementation Issues
• How do we make this invisible to the programmer?
• What are the semantics of parameter passing?
• How do we bind (locate, connect to) servers?• How do we support heterogeneity (OS,
architecture, language)?• How do we make it perform well?• How do we deal with failures?
– Client and server can fail independently.
45
RPC Messages
• RPC uses a message-based communication scheme to provide remote service.
• RPC messages are well structured (no longer just packets of data):– addressed to an RPC daemon listening to a
port on a remote system– contain an identifier of the function to
execute and the parameters to pass to that function.
46
RPC Model• A server defines the server’s interface
using an interface definition language (IDL)– The IDL specifies the names, parameters, and
types for all client-callable server procedures.• A stub compiler reads the IDL and
produces two stub procedures for client and server:– The server programmer implements the server
procedures and links them with the server-side stubs.
– The client programmer implements the client program and links it with the client-side stubs.
– The stubs are responsible for managing all details of the remote communication between client and server.
47
RPC Stubs• A client-side stub looks to the client as if it
were a callable server procedure.• A server-side stub looks to the server as if a
client called it.• The client program thinks it is calling the
server (In fact, it is calling the client stub).• The server program thinks it is called by
the client (In fact, it is called by the server stub).
• The stubs send messages to each other to make the RPC happen “transparently”.
48
Marshalling
• Marshalling is the packing of procedure parameters into a message packet.
• The RPC stubs call type-specific procedures to marshal (or unmarshall) the parameters to a call– The client stub marshals the parameters into a
message.– The server stub unmarshalls parameters from
the message and uses them to call the server procedure.
• On return:– The server stub marshals the return parameters.– The client stub unmarshalls return parameters
and returns them to the client program.
49
Marshalling Goals
• Linearize the data structures for transport in messages and reconstructing the original data structures at the other end.
• Convert data structures from the data representation on the calling process to that of the called process.
50
Linearizing• Pointers:
– Do not marshal the data pointed to, but generate a call-back handle.
– Avoid unnecessary copying of data across the interface.
– Multiple pointers to the same data will usually go unrecognized.
• Implicit typing:– only values are transmitted, not data types or
parameter info, e.g., Sun XDR.• Explicit typing:
– Type is transmitted with each value, e.g., ASN.1, XML.
51
Data Representation
• Use representation identical to that of one of the participating machines.
• Support multiple representations to avoid translations.
• Define a machine-independent representation of data– Sun’s XDR (eXternal Data Representation).
52
Semantics of RPC
• Local procedure calls fail only under extreme circumstances.
• RPCs can fail, or be duplicated and executed more than once due to common network errors.– Ensure at-most-once delivery by attaching to
each message a timestamp and require the server to keep history of the processed messages.
53
Where To Bind?
• Solution 1 (Birrell & Nelson’s 1984 proposal):– Centralized database that can locate a host that
provides a particular service.– Server sends message to central authority stating
its willingness to accept certain remote procedure calls (and sends port number).
– Clients then contact this authority when they need to locate a service.
• Solution 1 is problematic for Sun NFS:– identical file servers serve different file systems.
54
Where To Bind?
• Solution 2: Require client to know which host it needs to contact:– A server on that host maintains a
database of locally provided services.
55
Rendezvous
• Typically, an operating system provides a rendezvous daemon also known as Port Mapper on a fixed RPC port.
• A client sends a message, containing the name of the RPC to the rendezvous daemon requesting the port address of the required RPC.
• The port number is returned, and the RPC call may be sent to that port until the process terminates or the server crashes.
56
Transport Protocol
• Which one?– Some implementations may offer only
one (e.g. TCP).– Most support several:
• Allow programmer (or end user) to choose.
57
When Things Go Wrong• Local procedure calls do not fail:
– If they core dump, entire process dies.
• More opportunities for error with RPC:– Server could generate error.– Network problems.– Server crash.– Client might crush while server is still
executing code for it.
• Transparency breaks here:– Applications should be prepared to deal with
RPC failure.
58
When Things Go Wrong
• Local procedure call: exactly once.• Exactly once may be difficult to achieve
with RPC.• A remote procedure call may be called:
– 0 times: server crashed or server process died before executing server code.
– 1 time: everything worked well.– 1 or more: excess latency or lost reply from
server and client retransmission.
59
RPC Semantics
• Most RPC systems will offer either:– At-least-once semantics.– Or at-most-once semantics.
60
Programming With RPC
• Language support– Most programming languages have no
concept of remote procedure calls.– Language compilers will not generate client
and server stubs.
• Common solution:– Use a separate compiler to generate stubs
(pre-compiler).
61
Interface Definition Language
• Allow programmer to specify remote procedure interfaces (names, parameters, return values)
• Pre-compiler can use this to generate client and server stubs:– Marshaling code– Unmarshaling code– Network transport routines– Conform to defined interface– Similar to function prototypes
62
Writing an RPC Program
• Client code has to be modified:– Initialize RPC-related options.– Transport type.– Locate server/service.– Handle failure of remote procedure call.
• Server functions– Generally need little or no modification.
63
ONC RPC
• RPC for Unix System V, Linux, BSD:– ONC RPC (Open Network Computing) and Sun RPC.
• Sun provides a compiler that takes the definition of a remote procedure interface and generates the client and server stub functions.– This compiler is called rpcgen.
• Before running this compiler, the programmer has to provide the interface definition. – The interface definition contains the function
declarations, grouped by version numbers (to support older clients connecting to a newer server) and a unique program number.
64
Sun RPC
• Other components provided by Sun are XDR and a run-time library that implements the necessary protocols and socket routines to support RPC.
• All the programmer has to write is a client procedure, the server functions, and the RPC specification.
65
SUN RPC IDL
name.xprogram GETNAME {
version GET_VERS {long GET_ID(string<50>) = 1;string GET_ADDR(long) = 2;
} = 1; /* version */} = 0x31223456;
66
rpcgen
$ rpcgen name.x• produces:
– name.h– name_svc.c – name_clnt.c– [ name_xdr.c ]
• Function names are derived from IDL function names and version numbers
• Client gets pointer to result– Allows it to identify failed RPC (null return)
67
Server-side
• Start server– Server stub creates a socket and binds any
available local port to it.– Calls a function in the RPC library:
• svc_register to register program#, port #– contacts portmapper (rpcbind on SVR4):
• Name server• Keeps track of {program#,version#,protocol}
port# bindings– Server then listens and waits to accept
connections.
68
Client-side
• Client calls clnt_create with:– Name of server– Program #– Version #– Protocol#
• clnt_create contacts port mapper on that server to get the port for that interface– early binding – done once, not per every
procedure call.
69
Advantages
• No need to get a unique transport address (port):– With SUN RPC you need a unique program
number per server.• Greater portability.• Transport independent:
– Protocol can be selected at run-time.• Application does not have to deal with maintaining
message boundaries, fragmentation, reassembly.• Applications need to know only one transport
address:– Port mapper
• Function call model can be used instead of send/receive.
70
RPC Transparency
• One goal of RPC is to be as transparent as possible:– Make remote procedure calls look like local
procedure calls.– We have seen that binding breaks transparency.
• Failures – remote nodes/networks can fail in more ways than with local procedure calls:– Need extra support to handle failures well.
• Performance – remote communication is inherently slower than local communication.– If program is performance-sensitive, could be a
problem.
71
Conclusion
• Distributed IPC• ATM and ATM protocol stack • Transport layer
– Session management– Two-way handshaking– Blast protocol
• Communication models– Remote procedure calls
72
References
• Black U.D., Emerging Communications Technologies.
• Mullender, S.J. Distributed Systems, 2nd edition, 1993.
• Cisco ATM Internetworking.• Krzyzanowski P., Notes on Distributed Computing.• Voelker, G.M. “Principles of Operating Systems”,
lecture notes, 2004.• http://www.webopedia.com.