1

XDRXDRExternal Data RepresentationExternal Data Representation

Process AProcess A

XDR Encode/DecodeXDR Encode/Decode

TransportTransport

Process BProcess B

XDR Encode/DecodeXDR Encode/Decode

TransportTransport

2

XDR as a case studyXDR as a case study

Sun RPC uses XDR.Sun RPC uses XDR. A good example of a “layer”.A good example of a “layer”. Interesting API.Interesting API. Powerful paradigm for creation of complex Powerful paradigm for creation of complex

data structures.data structures.

3

XDRXDR

XDR provides a service associated with the XDR provides a service associated with the OSI Presentation Layer.OSI Presentation Layer.– Common data representationCommon data representation– Library (not part of the O.S.). Library (not part of the O.S.). – Easy to port to new architectures.Easy to port to new architectures.– Independence from transport layer.Independence from transport layer.

4

Data ConversionData Conversion

Asymmetric Data ConversionAsymmetric Data Conversion– client always converts to the server’s data client always converts to the server’s data

representation.representation. Symmetric Data ConversionSymmetric Data Conversion

– both client and server convert to some standard both client and server convert to some standard representation.representation.

5

XDR Data TypesXDR Data Types

booleanboolean charchar shortshort int int longlong floatfloat doubledouble

enumerationenumeration structurestructure stringstring fixed length array (1-D)fixed length array (1-D) variable length array (1-D)variable length array (1-D) unionunion opaque dataopaque data

6

Implicit vs. Explicit TypingImplicit vs. Explicit Typing

Explicit Typing means that each piece of Explicit Typing means that each piece of data includes information about the type.data includes information about the type.

Implicit typing means that the sender and Implicit typing means that the sender and receiver must agree on the order and type of receiver must agree on the order and type of all data.all data.

XDR uses Implicit TypingXDR uses Implicit Typing

7

XDR ProgrammingXDR Programming

Most XDR implementations are based on a Most XDR implementations are based on a bufferbuffer paradigm. paradigm.

A buffer is allocated that is large enough to A buffer is allocated that is large enough to hold an entire message.hold an entire message.

Individual data items are added to the buffer Individual data items are added to the buffer one at a time.one at a time.

XDR buffers can be attached to a file, pipe, XDR buffers can be attached to a file, pipe, socket or memory.socket or memory.

8

Conversion TerminologyConversion Terminology

Converting from local representation to Converting from local representation to XDR representation is called XDR representation is called Encoding.Encoding.

Converting from XDR representation to Converting from XDR representation to local representation is called local representation is called Decoding.Decoding.

SenderSender ReceiverReceiverDECODEXDRXDRENCODE

9

XDR Buffer CreationXDR Buffer Creation

There are a number of buffer creation There are a number of buffer creation functions in the XDR library.functions in the XDR library.– xdrmem_create xdrmem_create

» destination for encoding -or- source for decoding is destination for encoding -or- source for decoding is a chunk of memory.a chunk of memory.

– xdrstdio_create xdrstdio_create » destination for encoding -or- source for decoding is destination for encoding -or- source for decoding is

a file descriptor.a file descriptor.

10

XDR filtersXDR filters

The XDR library includes an extensive set The XDR library includes an extensive set of of filters filters that perform encoding/decoding that perform encoding/decoding operations.operations.

Each XDR stream includes an attribute that Each XDR stream includes an attribute that determines the specific operation that will determines the specific operation that will be performed by a filter (encoding or be performed by a filter (encoding or decoding).decoding).

11

XDR FiltersXDR Filters

The filter to encode/decode an integer is The filter to encode/decode an integer is called xdr_int:called xdr_int:

bool_t xdr_int( XDR *xdrs, int *ip);bool_t xdr_int( XDR *xdrs, int *ip);

the return value indicates success or failure.the return value indicates success or failure. xdrs is a pointer to an XDR streamxdrs is a pointer to an XDR stream ip is a pointer to an integer.ip is a pointer to an integer.

12

xdr_int()xdr_int()

If the XDR stream operation is XDR_ENCODEIf the XDR stream operation is XDR_ENCODE

int count;int count;

XDR *xstream;XDR *xstream;

xdr_int(xstream, &count);xdr_int(xstream, &count);

will convert (encode) the value of count to the will convert (encode) the value of count to the integer representation used by XDR (big-integer representation used by XDR (big-endian) and put the result on the XDR stream.endian) and put the result on the XDR stream.

13

xdr_int()xdr_int()

If the XDR stream operation is XDR_DECODEIf the XDR stream operation is XDR_DECODE

int count;int count;

XDR *xstream;XDR *xstream;

xdr_int(xstream, &count);xdr_int(xstream, &count);

will get an XDR integer from the stream, convert will get an XDR integer from the stream, convert (decode) the value to local integer representation and (decode) the value to local integer representation and put the result in count.put the result in count.

14

xdr_int()xdr_int()

int count;xdr_int(xstream,&count);

int count;xdr_int(xstream,&count);

Source Encodes

Destintation decodes

15

Complex Data FiltersComplex Data Filters

The XDR library includes a set of filters The XDR library includes a set of filters designed to translate complex C data designed to translate complex C data structures to and from XDR representation.structures to and from XDR representation.

Many of these filters make use of the Many of these filters make use of the simpler filters to convert individual simpler filters to convert individual components.components.

16

xdr_array()xdr_array() The xdr_array() filter provides support for The xdr_array() filter provides support for

encoding/decoding a variable length array.encoding/decoding a variable length array.bool_t xdr_array( bool_t xdr_array( XDR *xdrs, XDR *xdrs, char *arrpchar *arrp, ,

u_int *sizepu_int *sizep, , u_int maxsizeu_int maxsize, , u_int elsizeu_int elsize, , xdrproc_t elproc)xdrproc_t elproc); ;

sizep sizep is a pointer to the size of the array.is a pointer to the size of the array.

elsize elsize is the size of each element (in bytes).is the size of each element (in bytes).

elproc elproc is a pointer to a function that can encode/decode is a pointer to a function that can encode/decode individual array elements.individual array elements.

17

xdr_array()xdr_array()

0 1 2 3

SourceArray ->SourceArray ->

0 1 2 3DestinationArray ->DestinationArray ->

elproc()elproc()

18

Inside xdr_array()Inside xdr_array()

xdr_int(xdrs,&sizep);xdr_int(xdrs,&sizep);

for (i=0;i<sizep;i++)for (i=0;i<sizep;i++)

elproc(xdrs,arrp+elsize*i);elproc(xdrs,arrp+elsize*i);

encode/decode thenumber of elements in the array

encode/decode each array element.

19

xdr_string()xdr_string()

the string conversion filter is a little the string conversion filter is a little different since XDR strings have a different since XDR strings have a maximum size.maximum size.

bool_t xdr_string( XDR *xdrs,bool_t xdr_string( XDR *xdrs,

char *string, u_int maxsize);char *string, u_int maxsize);

20

Problem!!Problem!!

We want to send an array of strings We want to send an array of strings between processes.between processes.

What is the problem (using xdr_array)?What is the problem (using xdr_array)?

What is a possible solution?What is a possible solution?

21

Distributed Program DesignDistributed Program Design

Communication-Oriented DesignCommunication-Oriented Design– Design protocol first. Design protocol first. – Build programs that adhere to the protocol.Build programs that adhere to the protocol.

Application-Oriented DesignApplication-Oriented Design– Build application(s).Build application(s).– Divide programs up and add communication Divide programs up and add communication

protocols.protocols.

Typical Typical

SocketsSockets

Approach

Approach

RPCRPC

22

RPCRPCRemote Procedure CallRemote Procedure Call

Call a procedure (subroutine) that is Call a procedure (subroutine) that is running on another machine.running on another machine.

Issues:Issues:– identifying and accessing the remote procedureidentifying and accessing the remote procedure– parametersparameters– return valuereturn value

23

blah, blah, blah

bar = foo(a,b);

blah, blah, blah

blah, blah, blah

bar = foo(a,b);

blah, blah, blah

int foo(int x, int y ) { if (x>100)

return(y-2); else if (x>10)

return(y-x); else

return(x+y);}

int foo(int x, int y ) { if (x>100)

return(y-2); else if (x>10)

return(y-x); else

return(x+y);}

ClientClient ServerServer

protocol

24

Sun RPCSun RPC

There are a number of popular RPC There are a number of popular RPC specifications.specifications.

Sun RPC (ONC RPC) is widely used.Sun RPC (ONC RPC) is widely used. NFS (Network File System) is RPC based.NFS (Network File System) is RPC based. Rich set of support tools.Rich set of support tools.

25

Sun RPC OrganizationSun RPC Organization

Procedure 1Procedure 1 Procedure 2Procedure 2 Procedure 3Procedure 3

Shared Global DataShared Global Data

Remote ProgramRemote Program

26

Procedure ArgumentsProcedure Arguments

To reduce the complexity of interface To reduce the complexity of interface specification Sun RPC includes support for a specification Sun RPC includes support for a single argument to a remote procedure.*single argument to a remote procedure.*

Typically the single argument is a structure Typically the single argument is a structure that contains a number of values.that contains a number of values.

* Newer versions can handle multiple args.* Newer versions can handle multiple args.

27

Procedure IdentificationProcedure Identification

Each procedure is identified by:Each procedure is identified by:– Hostname (IP Address)Hostname (IP Address)– Program identifier (32 bit integer)Program identifier (32 bit integer)– Procedure identifier (32 bit integer)Procedure identifier (32 bit integer)

28

Procedure IdentificationProcedure Identification

Each procedure is identified by:Each procedure is identified by:– Hostname (IP Address)Hostname (IP Address)– Program identifier (32 bit integer)Program identifier (32 bit integer)– Procedure identifier (32 bit integer)Procedure identifier (32 bit integer)

– Program Version identifierProgram Version identifier» for testing and migration.for testing and migration.

29

Program IdentifiersProgram Identifiers

Each remote program has a unique ID.Each remote program has a unique ID. Sun divided up the IDs:Sun divided up the IDs:

0x00000000 - 0x1fffffff0x00000000 - 0x1fffffff

0x20000000 - 0x3fffffff0x20000000 - 0x3fffffff

0x40000000 - 0x5fffffff0x40000000 - 0x5fffffff

0x60000000 - 0xffffffff0x60000000 - 0xffffffff

SunSun

Sys Admin Sys Admin

TransientTransient

ReservedReserved

30

Procedure Identifiers &Procedure Identifiers &Program Version NumbersProgram Version Numbers

Procedure Identifiers usually start at 1 and Procedure Identifiers usually start at 1 and are numbered sequentiallyare numbered sequentially

Version Numbers typically start at 1 and are Version Numbers typically start at 1 and are numbered sequentially.numbered sequentially.

31

Iterative ServerIterative Server

Sun RPC specifies that at most one remote Sun RPC specifies that at most one remote procedure within a program can be invoked at procedure within a program can be invoked at any given time.any given time.

If a 2nd procedure is called the caller blocks until If a 2nd procedure is called the caller blocks until the 1st procedure has completed.the 1st procedure has completed.

This is useful for applications that may share data This is useful for applications that may share data among procedures.among procedures.

Example: database - to avoid Example: database - to avoid insert/delete/modify collisions.insert/delete/modify collisions.

32

Communication SemanticsCommunication Semantics

To act like a local procedure (exactly one To act like a local procedure (exactly one invocation per call) - a reliable transport invocation per call) - a reliable transport (TCP) is necessary.(TCP) is necessary.

Sun RPC does not support reliable call Sun RPC does not support reliable call semantics.semantics.

At Least Once SemanticsAt Least Once Semantics Zero or More SemanticsZero or More Semantics

If the procedure returnsIf the procedure returns

No replyNo reply

33

Dynamic Port MappingDynamic Port Mapping

Servers typically do not use well known Servers typically do not use well known protocol ports.protocol ports.

Clients known the Program ID (and host).Clients known the Program ID (and host). A port lookup service runs on each host that A port lookup service runs on each host that

contains RPC servers.contains RPC servers. RPC servers register themselves with the RPC servers register themselves with the

port mapperport mapper

34

The portmapperThe portmapper

Each system which will support RPC Each system which will support RPC servers runs a servers runs a port mapper port mapper server that server that provides a central registry for RPC services.provides a central registry for RPC services.

Servers tell the port mapper what services Servers tell the port mapper what services they offer.they offer.

Clients ask a remote port mapper for the Clients ask a remote port mapper for the port number corresponsing to Remote port number corresponsing to Remote Program ID.Program ID.

35

More on the portmapperMore on the portmapper

The portmapper is itself an RPC server!The portmapper is itself an RPC server! The portmapper is available on a well-The portmapper is available on a well-

known port (111).known port (111).

36

Sun RPC ProgrammingSun RPC Programming

The RPC library is a collection of tools for The RPC library is a collection of tools for automating the creation of RPC clients and automating the creation of RPC clients and servers.servers.

RPC clients are processes that call remote RPC clients are processes that call remote procedures.procedures.

RPC servers are processes that include RPC servers are processes that include procedure(s) that can be called by clients.procedure(s) that can be called by clients.

37

RPC ProgrammingRPC Programming

RPC libraryRPC library– XDR routinesXDR routines– RPC run time libraryRPC run time library

» call rpc servicecall rpc service

» register with portmapperregister with portmapper

» dispatch incoming request to correct proceduredispatch incoming request to correct procedure

– Program GeneratorProgram Generator

38

RPC Run-time LibraryRPC Run-time Library

High- and Low-level functions that can be High- and Low-level functions that can be used by clients and servers.used by clients and servers.

High-level functions provide simple access High-level functions provide simple access to RPC services.to RPC services.

39

High-level Client LibraryHigh-level Client Library

int callrpc( char *host,int callrpc( char *host,

u_long prognum,u_long prognum,

u_long versnum,u_long versnum,

u_long procnum,u_long procnum,

xdrproc_t inproc,xdrproc_t inproc,

char *in,char *in,

xdrproc_t outproc,xdrproc_t outproc,

char *out);char *out);

40

High-Level Server LibraryHigh-Level Server Library

int registerrpc(int registerrpc(

u_long prognum,u_long prognum,

u_long versnum,u_long versnum,

u_long procnum,u_long procnum,

char *(*procname)()char *(*procname)()

xdrproc_t inproc,xdrproc_t inproc,

xdrproc_t outproc);xdrproc_t outproc);

41

High-Level Server Library High-Level Server Library (cont.)(cont.)

void svc_run();void svc_run();

svc_run() svc_run() is a is a dispatcher. dispatcher. A dispatcher waits for incoming A dispatcher waits for incoming

connections and invokes the appropriate connections and invokes the appropriate function to handle each incoming request.function to handle each incoming request.

42

High-Level Library LimitationHigh-Level Library Limitation

The High-Level RPC library calls support The High-Level RPC library calls support UDP only (no TCP).UDP only (no TCP).

You must use lower-level RPC library You must use lower-level RPC library functions to use TCP.functions to use TCP.

The High-Level library calls do not support The High-Level library calls do not support any kind of authentication.any kind of authentication.

43

Low-level RPC LibraryLow-level RPC Library

Full control over all IPC optionsFull control over all IPC options– TCP & UDPTCP & UDP– Timeout valuesTimeout values– Asynchronous procedure callsAsynchronous procedure calls

Multi-tasking ServersMulti-tasking Servers BroadcastingBroadcasting

44

RPCGENRPCGEN

There is a tool for automating the creation There is a tool for automating the creation of RPC clients and servers.of RPC clients and servers.

The program The program rpcgen rpcgen does most of the work does most of the work for you.for you.

The input to rpcgen is a The input to rpcgen is a protocol definition protocol definition in the form of a list of remote procedures in the form of a list of remote procedures and parameter types.and parameter types.

45

RPCGENRPCGEN

Input File

rpcgen

Client Stubs XDR filters header file Server skeleton

C Source CodeC Source Code

Protocol DescriptionProtocol Description

46

rpcgen Output Filesrpcgen Output Files

> rpcgen foo.x> rpcgen foo.x

foo_clnt.c (client stubs)foo_clnt.c (client stubs)

foo_svc.c (server main)foo_svc.c (server main)

foo_xdr.c (xdr filters)foo_xdr.c (xdr filters)

foo.h (shared header file)foo.h (shared header file)

47

Client CreationClient Creation

> gcc -o fooclient foomain.c foo_clnt.c foo_xdr.c> gcc -o fooclient foomain.c foo_clnt.c foo_xdr.c

foomain.c is the client main() (and possibly foomain.c is the client main() (and possibly other function) that call rpc services via the other function) that call rpc services via the client stub functions in foo_clnt.cclient stub functions in foo_clnt.c

The client stubs use the xdr functions.The client stubs use the xdr functions.

48

Server CreationServer Creation

gcc -o fooserver fooservices.c foo_svc.c foo_xdr.cgcc -o fooserver fooservices.c foo_svc.c foo_xdr.c

fooservices.c contains the definitions of the fooservices.c contains the definitions of the actual remote procedures.actual remote procedures.

49

Example Protocol DefinitionExample Protocol Definitionstruct twonums {struct twonums {

int a;int a;

int b;int b;

};};

program UIDPROG {program UIDPROG {

version UIDVERS {version UIDVERS {

int RGETUID(string<20>) = 1;int RGETUID(string<20>) = 1;

string RGETLOGIN( int ) = 2;string RGETLOGIN( int ) = 2;

int RADD(twonums) = 3;int RADD(twonums) = 3;

} = 1;} = 1;

} = 0x20000001;} = 0x20000001;

50