open architecture publish- subscribe benchmarking - · pdf fileopen architecture...
TRANSCRIPT
Slide 1 6/30/2005
Open Architecture Publish-Subscribe Benchmarking
Bruce McCormick, EG&GLeslie Madden, NSWCDD
Slide 2 6/30/2005
ObjectivesObjectives! To gain familiarity with existing publish-subscribe
middleware! Understand ability of products to support bounded
latencies and sustained throughput required by combat system applications
! Understand architectural differences in products that may affect ability of products to scale to large numbers of processors & processes
! Provide a baseline of comparison as products migrate toward DDS compliance
Slide 3 6/30/2005
Data Store
SPLI
CE
queries
Application
Publish/Subscribe API
queries
Application
Publish/Subscribe API
SPLICE Daemon
Network
Node
Pub/SubM/W
Pub/SubM/W
pub
pub
sub
sub
pub
pub
sub
sub
waitsetwaitset waitsetwaitset
wait fordata
THALES SPLICE 2THALES SPLICE 2
Slide 4 6/30/2005
ND
DS
Application
Publish/Subscribe API
Publish/Subscribe M/W
Application
Publish/Subscribe API
Publish/Subscribe M/W
Network
Node
subs
crip
tion
subs
crip
tion
publ
icat
ion
publ
icat
ion
subs
crip
tion
subs
crip
tion
publ
icat
ion
publ
icat
ion
poll for datapoll for datacallbackcallback
RTI NDDS 3.0MRTI NDDS 3.0M
Slide 5 6/30/2005
Architectural Features Affecting Results
Architectural Features Affecting Results
Implements Reliable/FIFO – all issues delivered. Improperly sized queues can lead to flow control behavior
Implements reliable but not FIFO – most recent message issue (based on timestamp) overwrites previous issue
Receives data into message queues
Stores data into in-memory data store for future access
NDDSSPLICE
Slide 6 6/30/2005
Mett ToolMett Tool
Slide 7 6/30/2005
Mett DesignMett Design
Display/ControlHost 1
METT EngineHost 2
METT EngineHost 3
Test Stub Test Stub
groupthread
groupthread
rsh rsh
multicastrun test
command
Data Messages
tcp configure/launch
stub
Slide 8 6/30/2005
Test Stub Window
Test Configuration Window
Mett StubsMett Stubs
Slide 9 6/30/2005
Test EnvironmentTest Environment
! All nodes in the same subnet! Use multicast! Privileged User
– Mett sets high (Max – 5) RT thread and process priorities
– Not bound to any CPU (SMP)
Slide 10 6/30/2005
Host OS Description NetworkInterface
Alphe1 Solaris8 SUN Ultra2 w/ Dual 200 MHz UltraSparc - sun4u128 MB RAM Fast Ethernet
Alphe2 Solaris8 SUN Ultra2 w/ Dual 200 MHz UltraSparc - sun4u128 MB RAM Fast Ethernet
Alphe3 Solaris8 SUN Ultra2 w/ Dual 200 MHz UltraSparc - sun4u128 MB RAM Fast Ethernet
Alphe4 Solaris 8 SUN Ultra2 w/ Dual 200 MHz UltraSparc - sun4u128 MB RAM Fast Ethernet
Serpens Solaris 8SUN Ultra-Enterprise 8 processor UltraSparc – sun4u Fast Ethernet
Alphe4 Solaris 8 SUN Ultra2 w/ Dual 200 MHz UltraSparc - sun4u128 MB RAM Fast Ethernet
Alphe5 Solaris 8 SUN Ultra2 w/ Dual 200 MHz UltraSparc - sun4u128 MB RAM Fast Ethernet
Alphe6 Solaris 8 SUN Ultra2 w/ Dual 200 MHz UltraSparc - sun4u128 MB RAM Fast Ethernet
Host ConfigurationHost Configuration
Slide 11 6/30/2005
Single topic with one publisher and 5 subscribers
Single topic with 5 publishers and one subscriber
Host 1
Host 2 Host 3
Host 4 Host 6
Host 6
Host 1
Host 2 Host 3
Host 4 Host 6
Host 6
Single topic with one publisher and onesubscriber
Host 1 Host 2
Configuration Setup
Test SetupsTest Setups
Slide 12 6/30/2005
MethodologyMethodology
Subscriber Publisher
One-way latencies
Time recv - Time sent
• NTP for clock sync• NDDS 3.0m configuration
– Reliable queue size 1
• Splice 2• Vendor tool code generation
– C libraries
Time Synchronization
0
0.02
0.04
0.06
0.08
0.1
Mill
isec
onds
Max Offset
Slide 13 6/30/2005
Latencies of 200 Byte Message Delivery at 10 Hz
Small Message Low RateSmall Message Low Rate
Slide 14 6/30/2005
Average Latencies of Different Message Sizes at Different Rates
Reliable Delivery - Average Latencies
Reliable Delivery - Average Latencies
Slide 15 6/30/2005
Maximum Latencies of Different Message Sizes at Different Rates
Reliable Delivery - Maximum Latencies
Reliable Delivery - Maximum Latencies
Slide 16 6/30/2005
Maximum Latencies for SPLICE Reliable, NDDS Reliable/FIFO, and UDP at 2000 Hz
UDP, NDDS, & SPLICE – Maximum Latencies
UDP, NDDS, & SPLICE – Maximum Latencies
Slide 17 6/30/2005
Average Latencies of Different Message Sizes at Different Rates with Collocated Hosts
Simultaneous NDDS & SPLICE –Average Latencies
Simultaneous NDDS & SPLICE –Average Latencies
Slide 18 6/30/2005
Maximum Latencies of Different Message Sizes at Different Rates with Collocated Hosts
Simultaneous NDDS & SPLICE -Maximum Latencies
Simultaneous NDDS & SPLICE -Maximum Latencies
Slide 19 6/30/2005
Average Latencies of Five Publishers Sending Data To One Subscriber
5 Publishers to 1 Subscriber -Average Latencies
5 Publishers to 1 Subscriber -Average Latencies
Slide 20 6/30/2005
Maximum Latencies of Five Publishers Sending Data to One Subscriber.
5 Publishers to 1 Subscriber –Maximum Latencies
5 Publishers to 1 Subscriber –Maximum Latencies
Slide 21 6/30/2005
Maximum Latencies of One Publisher Sending Data to Five Subscribers – NDDS Best Effort
NDDS 1 Publisher to 5 Subscribers Best Effort – Maximum Latencies
NDDS 1 Publisher to 5 Subscribers Best Effort – Maximum Latencies
Slide 22 6/30/2005
Maximum Latencies of One Publisher Sending Data to Five Subscribers – NDDS Reliable/FIFO
NDDS 1 Publisher to 5 Subscribers Reliable - Maximum Latencies
NDDS 1 Publisher to 5 Subscribers Reliable - Maximum Latencies
Slide 23 6/30/2005
Maximum Latencies of One Publisher Sending Data to Five Subscribers – SPLICE Best Effort
Splice 1 Publisher to 5 Subscribers Best Effort - Maximum Latencies
Splice 1 Publisher to 5 Subscribers Best Effort - Maximum Latencies
Slide 24 6/30/2005
Maximum Latencies of Publisher Sending Data to Subscribers – SPLICE Reliable
Splice 1 Publisher to 5 Subscribers Reliable - Maximum Latencies
Splice 1 Publisher to 5 Subscribers Reliable - Maximum Latencies
Slide 25 6/30/2005
Middleware Solaris CPU Usage For SubscribersTake and read
Subscriber CPU UsageSubscriber CPU Usage
Slide 26 6/30/2005
Middleware Solaris CPU Usage For PublishersTool overhead included
Publisher CPU UsagePublisher CPU Usage
Slide 27 6/30/2005
Middleware Message Overhead
0
5000
10000
15000
20000
25000
30000
sent recv sent recv
Publish Subscribe
19,000 Content Messages
Tota
l Mes
sage
s
Splice
NDDS
Middleware Message Overhead
IPC Message OverheadIPC Message Overhead
Slide 28 6/30/2005
SummarySummary! Overall, both products perform well within the
range of performance requirements of typical US Surface Navy Domain combat systems.
! Current indications are that both will provide good performance and scalability as a publish-subscribe middleware for combat system applications.
! Based on the results to date, it appears likely that performance of the products may not be a major differentiator, but other features not examined in this evaluation, such as life-cycle cost and support may influence the selection of one product over the other.
Slide 29 6/30/2005
What’s Next?What’s Next?! Testing of DDS-compliant products as they
become available! Evaluation of marshalling overhead associated
with different, domain applicable IDL representations
! Evaluation of cross-platform performance issues, including marshalling
! Performance with large message sizes! Performance on real-time operating systems such
as LynxOS, RT Linux