douglas - cs.wustl.eduschmidt/pdf/atm.pdfdouglas c. schmidt high-p erfo rmance, real-time orbs...
TRANSCRIPT
High-performance,Real-time
CORBAORBsforATM
Networks
DouglasC.Schmidt
WashingtonUniversity,St.Louis
www.cs.wustl.edu/�
schmidt/TAO.html
Sponsors
NSF,DARPA,Bellcore,Boeing,CDI,
Kodak,Lucent,Motorola,OTI,SAIC,
SiemensSCR,SiemensMED,SiemensZT,Sprint
Douglas C. Schmidt High-performance, Real-time ORBs
Problem: Lack of Real-time Middleware
WIDE AREA
NETWORK
SATELLITESSATELLITESTRACKINGTRACKINGSTATIONSTATION
PEERSPEERS
STATUS INFO
COMMANDS BULK DATA
TRANSFER
LOCAL AREA NETWORK
GROUNDSTATION
PEERS
GATEWAY
� Many applications require
QoS guarantees
{ e.g., telecom, avionics,
WWW
� Building these applications
manually is hard
� Existing middleware doesn't
support QoS e�ectively
{ e.g., CORBA, DCOM, DCE
� Solutions must be integrated
Washington University, St. Louis 1
Douglas C. Schmidt High-performance, Real-time ORBs
Candidate Solution: CORBA
DIIDII ORBORBINTERFACEINTERFACE
ORBORBCORECORE
operation()operation()
OBJECTOBJECT
ADAPTERADAPTER
IDLIDLSKELETONSKELETON
DSIDSI
in argsin args
out args + return valueout args + return value
CLIENTCLIENT
GIOPGIOP//IIOPIIOP
SERVANTSERVANT
STANDARD INTERFACESTANDARD INTERFACE STANDARD LANGUAGESTANDARD LANGUAGE
MAPPINGMAPPING
ORB-ORB-SPECIFIC INTERFACESPECIFIC INTERFACE STANDARD PROTOCOLSTANDARD PROTOCOL
IDLIDLSTUBSSTUBS
www.cs.wustl.edu/�schmidt/corba.html
� Goals of CORBA
{ Simplify distribution
by automating
� Object location
and activation
� Parameter
marshaling
� Demultiplexing
� Error handling
{ Provide foundation
for higher-level
services
Washington University, St. Louis 2
DouglasC.Schmidt
High-performance,Real-timeORBs
MotivationforCORBA
�
Simpli�esapplicationinterworking
{CORBAprovideshigherlevelintegrationthantraditionaluntyped
TCPbytestreams
�
Providesafoundationforhigher-leveldistributedobjectcollaboration
{e.g.,WindowsOLEandtheOMGCommonObjectService
Speci�cation(COSS)
�
Bene�tsfordistributedprogrammingsimilartoOOlanguagesfor
non-distributedprogramming
{e.g.,encapsulation,interfaceinheritance,andobject-basedexception
handling
WashingtonUniversity,St.Louis
3
Douglas C. Schmidt High-performance, Real-time ORBs
The ACE ORB (TAO)
NETWORKNETWORK
ORBORB QQOOSSINTERFACEINTERFACE
REALREAL--TIMETIMEORBORB CORECORE
operation()operation()
RIDLRIDLSTUBSSTUBS
REALREAL--TIMETIME
OBJECTOBJECT
ADAPTERADAPTER
RIDLRIDLSKELETONSKELETON
in argsin args
out args + return valueout args + return value
CLIENTCLIENT
OS KERNELOS KERNEL
HIGHHIGH--SPEEDSPEED
NETWORK ADAPTERSNETWORK ADAPTERS
REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM
RIOPRIOP
SERVANTSERVANT
OS KERNELOS KERNEL
HIGHHIGH--SPEEDSPEED
NETWORK ADAPTERSNETWORK ADAPTERS
REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM
www.cs.wustl.edu/�schmidt/TAO.html
� TAO Overview
{ A high-performance,
real-time ORB
� Telecom and
avionics focus
{ Leverages the ACE
framework
� Runs on RTOSs,
POSIX, and Win32
� Related work
{ QuO at BBN
{ ARMADA at U.
Mich.
Washington University, St. Louis 4
Douglas C. Schmidt High-performance, Real-time ORBs
The ADAPTIVE Communication Environment (ACE)
PROCESSESPROCESSES//THREADSTHREADS
DYNAMICDYNAMIC
LINKINGLINKING
MEMORYMEMORY
MAPPINGMAPPING
SELECTSELECT//IO COMPIO COMP
SYSTEMSYSTEM
VV IPCIPCSTREAMSTREAM
PIPESPIPES
NAMEDNAMED
PIPESPIPES
CAPIS
SOCKETSSOCKETS//TLITLI
COMMUNICATIONCOMMUNICATION
SUBSYSTEMSUBSYSTEM
VIRTUAL MEMORYVIRTUAL MEMORY
SUBSYSTEMSUBSYSTEM
GENERAL POSIX AND WIN32 SERVICES
PROCESSPROCESS//THREADTHREAD
SUBSYSTEMSUBSYSTEM
FRAMEWORKS ACCEPTORACCEPTOR CONNECTORCONNECTOR
SELF-CONTAINED
DISTRIBUTED
SERVICE
COMPONENTS
NAMENAME
SERVERSERVER
TOKENTOKEN
SERVERSERVER
LOGGINGLOGGING
SERVERSERVER
GATEWAYGATEWAY
SERVERSERVER
SOCKSOCK__SAPSAP//TLITLI__SAPSAP
FIFOFIFO
SAPSAP
LOGLOG
MSGMSG
SERVICESERVICE
HANDLERHANDLER
TIMETIME
SERVERSERVER
C++WRAPPERS
SPIPESPIPE
SAPSAP
CORBACORBA
HANDLERHANDLER
SYSVSYSVWRAPPERSWRAPPERS
SHAREDSHARED
MALLOCMALLOC
THE ACE ORBTHE ACE ORB
((TAOTAO))
JAWS ADAPTIVEJAWS ADAPTIVE
WEB SERVERWEB SERVER
MIDDLEWARE
APPLICATIONS
REACTORREACTOR//PROACTORPROACTOR
PROCESSPROCESS//THREADTHREAD
MANAGERSMANAGERS
ADAPTIVE SERVICE EXECUTIVE ADAPTIVE SERVICE EXECUTIVE (ASX)(ASX)
SERVICESERVICE
CONFIGCONFIG--URATORURATOR
SYNCHSYNCH
WRAPPERSWRAPPERS
MEMMEM
MAPMAP
OS ADAPTATION LAYER
www.cs.wustl.edu/�schmidt/ACE.html
� ACE Overview
{ Concurrent OO
networking
framework
{ Ported to C++
and Java
{ Runs on RTOSs,
POSIX, and
Win32
� Related work
{ x-Kernel
{ SysV STREAMS
Washington University, St. Louis 5
DouglasC.Schmidt
High-performance,Real-timeORBs
ACEStatistics
�
ACEcontain>
125,000linesofC++
{Over10person-yearsofe�ort
�
PortedtoUNIX,Win32,MVS,and
embeddedplatforms
{e.g.,VxWorks,LynxOS,pSoS
�
Largeusercommunity
{www.cs.wustl.edu/�
schmidt/ACE-
users.html
�
Currentlyusedbydozens
ofcompanies
{Bellcore,Boeing,
Ericsson,Kodak,
Lucent,Motorola,
SAIC,Siemens,
StorTek,etc.
�
Supportedcommercially
{www.riverace.com
WashingtonUniversity,St.Louis
6
Douglas C. Schmidt High-performance, Real-time ORBs
TAO's Real-time ORB Endsystem ArchitectureRR
UU
NN
TT
II
MM
EE
S S
CC
HH
EE
DD
UU
LL
EE
RR
SOCKETSOCKET QUEUEQUEUE DEMUXERDEMUXER
ATM PORT INTERFACECONTROLLER (APIC)
ZZ
EE
RR
OO
CC
OO
PP
YY
BB
UU
FF
FF
EE
RR
SS
II//OOSUBSYSTEMSUBSYSTEM
OBJECTOBJECTADAPTERADAPTER
SERVANTSERVANT DEMUXERDEMUXER
CLIENTSCLIENTSSTUBSSTUBS
SERVANTSSERVANTSSKELETONSSKELETONS
U
ORBORB CORECORE
REACTORREACTOR
(20 (20 HZHZ))
REACTORREACTOR
(10 (10 HZHZ))
REACTORREACTOR
(5 (5 HZHZ))
REACTORREACTOR
(1 (1 HZHZ))
QQooSSAPIAPI
� Solution Approach
{ Integrate RT dispatcher into
ORB endsystem
{ Support multiple request
scheduling strategies
� e.g., RMS, EDF, and MUF
{ Requests ordered across
thread priorities by OS
dispatcher
{ Requests ordered within
priorities based on data
dependencies and importance
Washington University, St. Louis 7
Douglas C. Schmidt High-performance, Real-time ORBs
Real-time Experiments over ATM
����
C 1C 0
Requests
C n
�� ��������������������������
Client
...
...
2ATM Switch
Ultra 2 Ultra 2
I/O SUBSYSTEM
Server
Object Adapter
ORB Core
Servants RU
NT
IME
SCH
ED
UL
ER
www.cs.wustl.edu/�schmidt/RT-perf.ps.gz
� One high-priority client
� 1..n low-priority clients
� Server factory implements
thread-per-priority
{ Highest real-time
priority for high-priority
client
{ Lowest real-time
priority for low-priority
clients
Washington University, St. Louis 8
DouglasC.Schmidt
High-performance,Real-timeORBs
ORBLatencyResultsoverATM
04812
16
20
24
28
32
36
40
44
48
52
15
10
15
20
25
30
35
40
45
50
Nu
mb
er o
f L
ow
Pri
ori
ty C
lien
ts
Latency per Two-way Request in Milliseconds
CO
RB
Aplu
s H
igh P
riority
MT
-OR
BIX
Hig
h P
riority
min
iCO
OL H
igh P
riority
CO
RB
Aplu
s Low
Priority
MT
-OR
BIX
Low
Priority
min
iCO
OL L
ow
Priority
TA
O H
igh P
riority
TA
O L
ow
Priority
�
Synopsisofresults
{COOL'slatencyislower
forsmall#ofclients
{TAO'slatencyislowest
forlarge#ofclients
{TAOavoidspriority
inversion
�
i.e.,highpriority
clientalwayshas
lowestlatency
WashingtonUniversity,St.Louis
9
Douglas C. Schmidt High-performance, Real-time ORBs
ORB Jitter Results over ATM
1 5 10 15 20 25 30 35 40 45 50
TAO High Priority
TAO Low Priority
miniCOOL High Priority
miniCOOL Low Priority
MT-ORBIX High Priority
MT-ORBIX Low Priority
CORBAplus High Priority
CORBAplus Low Priority
0
50
100
150
200
250
300
Jitt
er in
mill
isec
on
ds
Number of Low Priority Clients
� De�nition
{ Variance from
average latency
� Synopsis of
results
{ TAO's jitter is
lowest and most
consistent
{ MT-Orbix's
jitter is highest
and more
variable
Washington University, St. Louis 10
DouglasC.Schmidt
High-performance,Real-timeORBs
User-levelandKernel-levelLockingOverhead
41
94
199
175
100
231216
599
0
100
200
300
400
500
600
700
TA
Om
iniCO
OL
CO
RB
Aplus
MT
OR
BIX
OR
Bs T
ested
User Level Lock Operations per Request
client
server
00
44
0
13
8
14
0 2 4 6 8 10 12 14 16
TA
Om
iniCO
OL
CO
RB
Aplus
MT
OR
BIX
OR
Bs T
ested
Kernel Level Lock Operations per Request
client
server
WashingtonUniversity,St.Louis
11
Douglas C. Schmidt High-performance, Real-time ORBs
Integrating TAO with a Real-time ATM I/O Subsystem
REACTORREACTOR
(20 (20 HZHZ))
REACTORREACTOR
(10 (10 HZHZ))
REACTORREACTOR
(5 (5 HZHZ))
REACTORREACTOR
(1 (1 HZHZ))
ORBORB CORECORE
APICAPIC ATMATM DRIVERDRIVER
II//OOSUBSYSTEMSUBSYSTEM
PERPER--VCVC
SOCKETSOCKET
QUEUESQUEUES
APICAPIC PACINGPACING
QUEUESQUEUES
ZZ
EE
RR
OO
CC
OO
PP
YY
BB
UU
FF
FF
EE
RR
SS
� Key Features
{ Vertical integration of QoS
through ORB, OS, and ATM
network
{ Real-time I/O enhancements to
Solaris kernel
{ Provides rate-based QoS
end-to-end
{ Leverages APIC features for cell
pacing and zero-copy bu�ering
Washington University, St. Louis 12
DouglasC.Schmidt
High-performance,Real-timeORBs
ConcludingRemarks
�
Developersofdistributedapplicationsconfrontrecurringchallengesthat
arelargelyapplication-independent
{e.g.,serviceinitializationanddistribution,errorhandling, ow
control,eventdemultiplexing,concurrencycontrol,persistence,fault
tolerance
�
Successfuldevelopersresolvethesechallengesbyapplyingappropriate
designpatternstocreatecommunicationframeworksandcomponents
�
CORBAORBsareane�ectivewaytoachievereuseofdistributed
softwarecomponents
�
Thenext-generationofORBswillprovidemuchbettersupportfor
real-timeQoSoverATM
WashingtonUniversity,St.Louis
13