rti data-distribution service (dds) master class 2011

The Real-TimeMiddleware ExpertsThe Real-TimeMiddleware Experts

DDS: A Next-Generation Approach to Building Distributed Real-Time Systems

2011 Masterclass

Gerardo Pardo-Castellote, Ph.D.Co-chair OMG DDS SIGCTO, Real-Time [email protected]

http://www.rti.com

Outline

Day 1– Part I

• Concepts• Introduction to DDS and RTI• Overview of Technology• Run-time services (Persistence, Recording, Routing, …)

Day 2– Part II

• Basic Hand’s on Examples• Development tools• Future directions and Standards:

– Part II• Application development cycle• Advanced Exercise

Beyond real-time:Systems that interact directly with the Real World

Must adapt to changing environment

Cannot stop processing the information

Live within world-imposed timing

Beyond traditional interpretation of real-time

© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 3

© 2009 Real-Time Innovations, Inc. 4

Challenge: More Data, More Speed, More Sources

TRENDS: Growing Information Volume Lowering Decision Latency Increasing System Availability Accelerating technology insertion and

deployment

Next-generation systems needs: Performance Scalability Robustness & Availability Platform Integration & Evolution Safety-Critical Certification Security

“Real World” Systems are integrated using a Data-Centric Model

Grounded on the “physics” of the problem domain– Tied to the nature of the sensors and real objects in the system

(vehicles, device types, …)

Provides governance across disparate teams & organizations– Central authority can define data model necessary for interoperability

Increased decoupling from use-cases and components– Avoids over constraining applications

Open, Evolvable, Platform-Independent– The use-cases, algorithms might change between missions or

versions of the system

Copyright © 2010 RTI - All rights Reserved. COMPANY CONFIDENTIAL. 5

Realizing this data-model requires a middleware infrastructure

App AppApp

Data Centricity 101

Everyday Example: Calendaring

Alternative Process #1:

1. Email: “Meeting Monday at 10:00.”

2. Email: “Meeting moved to Tuesday.”

3. Email: “Here’s dial-in info for meeting…”

4. Rick: “Where do I have to be? When?”

5. Rick: (sifting through email…)

7

Example: Calendaring


1. Calendar: (add meeting Monday at 10:00)

2. Calendar: (move meeting to Tuesday)

3. Calendar: (add dial-in info)

4. Rick: “Where do I have to be? When?”

5. Rick: (check calendar)

8

Example: FaceBook


Use the FaceBook web site.


The web site doesn’t exist.

Deduce your friends’ information based on the notifications FaceBook sends you.

9

What’s the Difference? State.

Things have attributes and characteristics– The meeting will run 1:00–2:00 in

the conference room.– My friend’s phone number is 555-

1234 and he’s currently grooming his cat.

– The car is blue and is traveling north from Sunnyvale at 65 mph.

…whether they exist in the real world, in the computer, or both

…whether or not we observe or acknowledge them

“State” (“data”) is a snapshot of those attributes and characteristics.

Best Practice: operate on state directly, not dialogs about state.

Data-Centricity =

Describing the world

as it is

Implication: State of the world can be maintained by infrastructure, not each app

the part of you care about

at a certain point in time

Not Data-Centricity =

Saying anything else… “Hey you: go do this.”

“The thing changed in this way.”

Implication: State must be inferred, reconstructed, managed by each app

(Sometimes called “message-centricity”: focus on what’s said vs. what is)

Why is it better to just describe the world?

Reconstructing the state of the world is hard– Must infer based on all previous messages– Maintaining all these messages is expensive– Each app makes these inferences

=> duplicate effort

People make mistakes– Many copies of state => may be different => bugs

vs.– Uniform operations on state => fewer bugs

Why is it better to just describe the world?

Computers are stupid– Integrate based on what to

do=> must know everything you might do=> future extension hard

– Integrate based on what exists=> do whatever you like; no one else cares=> future extension easy

Integration effort doesn’t scale– Integrate at level of function

=> everything integrates with everything else=> effort scales with square of number of apps

– Integrate at level of state=> everything integrates with common model=> effort scales with number of apps

14

So it’s “better.” Who cares?

Faster to implement=> Save time and money

Easier to integrate and update=> Protect your investment

More reliable systems=> Protect your business

15

Before We Forget: the Definition

A data-centric architecture:

1. …is based on a data model that is:– Appropriately documented—

i.e. understandable by humans– Formally defined—

i.e. understandable by machines– Discoverable—i.e. can be found during execution

2. The data model is independent of any domain-specific functionality / application.– i.e. made of nouns, not verbs

l The (instantiated) data model is the only authoritative source of state in the system.

For example, data structures in IDL file.

Calendar Event =•Start Time•Duration•Location•Organizer

For example, data structures in IDL file.

Calendar Event =•Start Time•Duration•Location•Organizer

DDS Lets You Observe a Changing World

Other data-centric technologies:– Databases: SQL– Web: HTTP (mostly)

…assume the world changes slowly

…use network resources inefficiently

…are highly centralized

StateState

ServerServerAppApp

AppApp

AppAppAppApp

AppApp

AppApp

SlowA few updates/sec

Not scalable100 apps => 100x load

UnreliableFailure here kills many apps


DDS:

…allows you to observe frequent changes

…uses network resources efficiently

…is decentralized

AppApp AppApp AppApp AppApp AppApp AppApp

Fast100,000’s updates/sec

ScalableLoad indep. # apps

ReliableNo single pt. failure

Managedwith QoS

State: Global Data SpaceState: Global Data Space


JBC-P replaced home-brew messaging w/ DDS:

Tracks 20x more objects with fewer failures

…with 97% less code (1.5M lines 50K)

…with 99% less CPU resources (88 cores 0.8)

AppApp AppApp AppApp AppApp AppApp AppApp

Fast100,000’s updates/sec

ScalableLoad indep. # apps

ReliableNo single pt. failure

Managedwith QoS

State: Global Data SpaceState: Global Data Space


Domain: world you’re talking about

Topic: group of similar things– Similar structure (“type”) what– Similar way they change when

over time (“QoS”) how

Instance: individual thing

DataWriter: source of observations about part of the world (topic)

DataReader: observer of part of the world (topic)

Domain(e.g. Yellowstone Park)


Topic(e.g. bears in the park)


Instance(e.g. Yogi the bear)


The OMG DDS Standard


DDS: Standard Data-Centric middleware for Application Integration

Data Distribution Service

StreamingData

Sensors Events

Real-TimeApplications

EnterpriseApplications

Actuators


Open Architecture

Vendor independent– API for portability– Wire protocol for interoperability

Multiple implementations– 7 of API– 4 support RTPS (+1 non-DDS)

Heterogeneous– C, C++, Java, .NET (C#, C++/CLI)– Linux, Windows, VxWorks, other

embedded & real time

Loosely coupled

Real-Time Publish-Subscribe

Wire Protocol (RTPS)

DDS MiddlewareLibrary

DDS API

Cross-vendor portability

Cross-vendor interoperability

Web-EnabledDDS

2011

Family of Specifications


DDSImplementation

Network / TCP / UDP / IP

App

DDSImplementation

App

DDSImplementation

DDS Spec

2004

DDSInteroperablity

2006

UML Profilefor DDS

2008

DDS forLw CCM

2009

DDS X-Types

2010

DDSSecurity

20112010

DDS-STD-C++DDS-JAVA5

App

DDS adopted by key programs

US Navy Open Architecture– Mandates DDS for Pub-Sub

SPAWAR NESI– Mandates DDS for Pub-Sub SOA

DISR– Mandates DDS for Pub-Sub API– Mandates DDS-RTPS for Pub-Sub Interop

European Air Traffic Control– DDS used to interoperate ATC centers

UK Generic Vehicle Architecture– Mandates DDS for vehicle comm.– Mandates DDS-RTPS for interop.


RTI DDS Application Examples

Aegis Weapon System

Lockheed Martin

Radar, weapons, displays, C2

B-1B Bomber

Boeing

C2, communications, weapons

Common Link Integration Processing (CLIP)

Northrop Grumman

Standards-compliant interface to legacy and new tactical data links

Air Force, Navy, B-1B and B-52

ScanEagle UAV

Boeing

Sensors, ground station

Advanced Cockpit Ground Control Station

Predator and SkyWarrior UAS

General Atomics

Telemetry data, multiple workstations

RoboScout

Base10

Internal data bus and link to communications center



Multi-ship simulator

FORCE Technology

Controls, simulation display

Mobile asset tracking

Wi-Tronix

GPS, operational status over wireless links

Highway traffic monitoring

City of Tokyo

Roadway sensors, roadside kiosks, control center

Driver safety

Volkswagen

vision systems, analysis, driver information systems

Medical imaging

NMR and MRI

Sensors, RF generators, user interface, control

computers

Automated trading

Automated Trading Desk (ATD, now Citigroup)

Market data feed handlers, pricing engines, algorithmic

trading applications



Full-immersion simulation

National Highway Transportation Safety Authority

Migrated from CORBA, DCOM for performance

Air-Traffic Management

INDRA.

Deployed in

UK, Germany, Spain

Standards, Performance, Scalability

Industrial Control

Schneider Electric

VxWorks-based PLCs

communicate via RTI-DDS

Signal Processing

PLATH GMBH

RTI supports modular programming across

product line

Large Telescopes

European Southern Observatory

Performance & Scalability

1000 mirrors, 1sec loop

Radar Systems

AWACS upgrade

Evolvability, Mainteinability, and supportability

Architecture for the next-generation systems

Existing technologies are reaching robustness/performance/scalability limits

RTI DDS brings a fundamental new architecture and approach– Fully decentralized, peer-to-peer, “no bottlenecks” architecture– Superior Wire Protocol– Powerful data-centric model– Built-in Robustness and High-Availability – Standards-based, multi-platform

Single-lane trafficNo prioritization

Brokers aschoke-points RTI Approach

Benefits of the DDS approach

Simple & Powerful Data-Centric Pub-Sub Model Reduces Risk and Development/Integration Time Enhances effective performance by delivering the

right data to the right place with the right QoS Standards-based: API and Protocol

1. Unsurpassed Performance and Scalability Priority-aware no choke-points architecture

2. Builds higher quality systems and lowers TCO Built-in high-value capabilities Handles Availability & other “hard problems” Easy to maintain and Evolve Leverage multicore


Messaging & Caching

Event Processing

Database Bridge

Persistence& Durability

RecordingRedundancy & Failover

SQL

DDS Global Data SpaceAnd

Programming Model

© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

Data-Centric Pub-Sub Model

Source (key) Latitude Longitude Altitude

UAV1 37.4 -122.0 500.0

UAV2 40.7 -74.0 250.0

UAV3 50.2 -0.7 2000.0

PersistenceService

RecordingService

Essentially a virtual, decentralized global data space


Data-Centric Model

Data WriterData Writer


Data ReaderData Reader

Data Reader

Data ReaderData Writer

Data Object

“Global Data Space” generalizes Subject-Based Addressing– Data objects addressed by DomainId, Topic and Key– Domains provide a level of isolation – Topic groups homogeneous subjects (same data-type & meaning) – Key is a generalization of subject

• Key can be any set of fields, not limited to a “x.y.z …” formatted string

example

Data-Centric Model




Data Reader


Topic



example

Data-Centric Model




Data Reader


Key (subject)



example


Demo: Publish-SubscribeShapesDemo

DataReader“Alarm”

DomainParticipant

DataWriter

“Alarm”

DomainParticipant

DDS communications model

Participants scope the global data space (domain)

Topics define the data-objects (collections of subjects)

Writers publish data on Topics

Readers subscribe to data on Topics

QoS Policies are used configure the system

Listeners are used to notify the application of events

Listener

Offered

QoS Listener

Got newdata

Requested

QoS

New subscriber

!

example


Demo: Real-Time Quality of Service

Content filter

Time-based filter

History

Deadline

ShapesDemo

Analyzer


Real-Time Quality of Service (QoS)

QoS Policy

DURABILITY

HISTORY

READER DATA LIFECYCLE

WRITER DATA LIFECYCLE

LIFESPAN

ENTITY FACTORY

RESOURCE LIMITS

RELIABILITY

TIME BASED FILTER

DEADLINE

CONTENT FILTERS

Vo

lati

lity

User Q

oS

Del

ive

ry

Presen

tation

Red

un

dan

cy

Infr

astr

uct

ure

Tran

spo

rt

QoS Policy

USER DATA

TOPIC DATA

GROUP DATA

PARTITION

PRESENTATION

DESTINATION ORDER

OWNERSHIP

OWNERSHIP STRENGTH

LIVELINESS

LATENCY BUDGET

TRANSPORT PRIORITY

Copyright © 2010 RTI - All rights Reserved. . 40

Integrating components to generic middleware technology

Comp Comp Comp Comp

Middleware Artifacts

DataModel

Custom Mapping

Custom Integration

Akin to implementing an OO design on a Procedural Language:Requires mapping inheritance, encapsulation, exceptions, …


Integrating components to data-centric middleware technology

Comp Comp Comp Comp

DDS Global Data Space

DataModel

Standard Mapping(*)

Standard API

No custom mappings / code necessaryDirect support for data-centric actions: create, dispose, read/take

Example: Message-Centric LegacyDefine message-sets / handshakes


Pu

blish

Su

bscribe

x=float(45.6)x=float(45.6)

y=float(78.9)y=float(78.9)

id=“AA123”id=“AA123”0x0000000641410102030042366666429DC

0x0000000641410102030042366666429DC

“My app knows this means dispose.”

“My app knows this means dispose.”

Component or Use-case based

Schema,Limited QoS)

Component or Use-case based

Schema,Limited QoS)

Nothing to base filters, xforms on

Error checking dev integration

Self-describing data is verbose

Example: Modern Data-Centric DesignStart with Data Model / Schemas / Meaning


Pu

blish

Su

bscribe

Data SchemaData Schema

x : floatx : float

y : floaty : float

id : string (key)id : string (key)

NewNew

45.645.6

78.978.9

“AA123”“AA123”

UpdateUpdate

56.756.7

89.089.0

“AA123”“AA123”

NewNew

65.465.4

32.132.1

“DL987”“DL987”

DisposeDispose

“AA123”“AA123”

X

Map this into XML; rows + cols

Express content-based filters

Propagate data efficiently

Example: Modern Data-Centric DesignAttach QoS to Data Model


Once infrastructure understands data items, can attach QoS contracts to them

“Keep only the latest value” or “I need updates at this rate” make no sense unless per-item– Flight AA123 updates shouldn’t overwrite DL987, even if

AA123 is updated more frequently– Update rate for one track shouldn’t change just because

another track appeared

Pu

blish

Su

bscribe

Data SchemaData Schema

x : floatx : float

y : floaty : float

id : string (key)id : string (key)

Quality of ServiceQuality of Service

DeadlineDeadline

Time-Based FilterTime-Based Filter

HistoryHistory

Domain and Domain Participants

Node 1 - App 1Pub/Sub

Node 2 - App 1Subscribe


Node 4 - App 2Publish


Node 5 - App 1Subscribe

Domain 1



Domain 2

Domain 3Added Func.

Multiple Domain System

Using Multiple domains for Scalability, Modularity & Isolation

demo_domain_0 demo_domain_1

DDS Theoretical Foundation:

- Observer Pattern- Directed State Pattern

Observer Pattern: Use Cases

How do I model state that changes over time? How do I decouple data consumption from data production in time? What if consumers need to come and go while the system is

operational? What if new functionality needs to be added to the system while it’s

operational?

Examples– Vehicles move from place to place– Bid/offer prices of an asset change– Sensors take new measurements periodically

47

Observer Pattern: Introduction

Roles– Subject: Changes its state over time

• Has identity, abstract “address”

– Observer: Sees those changes when it cares to• Known to infrastructure, anonymous to subjects• Notified when subject changes; may examine new state or not

Operations (CRUD)– Create: create new subject exists with certain state– Read: access state(s) of subject(s)– Update:modify existing subject with new state– Delete: delete existing subject

48

Observer Pattern: Introduction

49

Update Delete

Create

ObserverObserver

Read

(Data Bus: Notify)(Data Bus: Notify)

SubjectSubject

Observer Pattern: How To

Premier pattern in DDS, central to data-centric design!

Roles– Subject address = domain topic instance– Observer = DataReader

Operations– Create = write new instance– Read = read or take samples– Update = write existing instance– Delete = dispose instance

50


Domain: world you’re talking about– Best practice: use to isolate subsystems– Best practice: connect subsystems with Routing Service

Topic: group of similar objects– Similar structure (type)

• Best practice: model complete state, not diffs• Best practice: use DDS type system, not opaque

bytes or custom encapsulations

– Similar way they change over time (QoS)• Best practice: differentiate

– …how subject changes (writer QoS)– …from what observer sees (reader QoS)

Instance: individual object– Best practice: declare key fields always– Best practice: key fields establish identity

• e.g. Social Security Number, Track ID, etc.

– Best practice: key fields allow resource management per object over time

• e.g. “keep last 5”

51







Subject Change Contract:– History keep last 10– Deadline 1 sec

Observer Requirements:– History keep last 5– Deadline 5 sec– Minimum separation 2

sec

Example Type:

typedef long BearID;

struct Bear {

@Key BearID which_bear;

long pos_x;

long pos_y;

float speed;

bool is_growling;

bool is_charging;

};


Two variants:

1. Publish when it changes– Uses network efficiently when changes are slow / sporadic– Examples:

• e.g. DDS discovery• e.g. Vehicles / devices that start and stop

– Best practice: use durability, reliability, keep-last history depth >= 1

2. Publish periodically– Good for continuous feedback control algorithms

• e.g. Vehicles / devices while they’re moving• e.g. Stuff that tends to be connected to Matlab / Simulink, LabView

– Smoothes resource use when changes are regular– Examples:

• Sensors, actuators in a control loop– Best practice: declare finite deadline– Best practice: consider time-based filter for slow readers and best

efforts

53

Observer Pattern: Benefits

Supports dynamic system composition, evolution– Subject states don’t depend on previous states: observers can

come, go whenever without being out of sync– Uses of data unspecified: functions can be added, removed

Maximizes parallelism, minimizes blocking– Subject changes on its own time scale– Observer reads on its time scale– Data bus can overwrite, filter, down-sample as necessary

Allows flexible deployment– Subjects don’t care about identities, locations, number of

observers– Observers don’t care about physical locations of subjects; just

need abstract “address”

54

decoupled

Observer Pattern: Challenges

Challenge: Large state may change in small ways– Communicating new state may be unnecessarily expensive– If Observer cares what changed, may be expensive to determine

Response: Don’t sweat it– Are the data rates high enough that it matters?– How much network/memory/CPU/complexity will you pay to “fix” it?

Response: Send diffs and reassemble– Tools at your disposal:

• DDS-XTypes: optional fields (also in RTI [dynamic] sparse types)• Portable today: sequence<MyType, 1>• Multiple topics (with same key)

– Topic 1: complete object state– Topic 2: cumulative changes to last state from Topic 1– Use durability and explicit correlation IDs to merge 1 and 2

– Cost: increased logic, state in your applications

55

Observer Pattern: See Also

High-Volume patterns

Low-Latency patterns

[GoF] “Observer”

[EIP] “Publish-Subscribe Channel”

[EIP] “Datatype Channel”

[EIP] “Quality-of-Service Channel”

56

DDS Theoretical Foundation:

- Observer Pattern- Directed State Pattern

Directed State Pattern: Use Cases

How can one party request that another party do something?

How can a producer know that observers(s) have acted on its data?

How can a producer observe the “result” of a remote action?

Examples:– Defense: Fire control– Financial Services: Order execution

58

Directed State Pattern: Introduction

Best practice: keep interactions data-centric– Insight: Objectives are state and can be distributed– Insight: Desires and intentions are state and can be distributed

Anti-pattern: point-to-point commands, replies– Why not: Couple applications based on function and role

• Why not: Constrain future functional changes to system• Why not: Integration effort doesn’t scale as number of apps increases

59

Directed State Pattern: Introduction

3 States:1. Current2. Objective (“goal”)—Optional

3. Requested State

2+ Roles (special case of Observer pattern):1. Effector

• aka Subject wrt Current State and Objective State• aka Observer wrt Requested Objective State

– Requester1. aka Subject wrt Requested Objective2. aka Observer wrt Current State and Objective State

– (Observer—with respect to any state)

Current State

Current State

Objective State

Objective Statechangechange

Effector executes this

Requested State

Requested State

Requester changes

this

RequesterRequesterEffectorEffector

Directed State Pattern: How To

Reliability QoS policy:– Current: Reliable or Best Effort– Requested: Reliable

History QoS policy:– Current, Objective: Keep Last n– Requested: Keep Last 1

61

Data BusData Bus

if long-runningif long-running

if short-runningif short-running

Current State

Current State

write

Objective State

Objective State

write

Requested State

Requested State

read, filter

request processed?

feedbackfeedback

Current State

Current State

read, filter

Objective State

Objective State

read, filter

same typesame or different types same key

Requested State

Requested State

write


Variant: Simple state,Fast transition

Current + Requested States; no Objective State

States use same type

Flow:1. Current = x2. Requested y3. Current y4. Requested unregister

Variant: Simple state,Slow transition

Current, Objective, Requested States

States use same type

Flow:1. Current, Objective= x2. Requested y 3. Objective y4. Current x + 1 x + 2 … y5. Requested unregister

62


Variant: Complex state, slow transition Current, Objective, Requested States Requested States use different type than Current

– Requests have identity and lifecycle: e.g. REQUESTED, PROCESSING, DONE– Objective indicates which request is in process

Flow:– Effector: Current, Objective= x– Requester: Requested id = 1, value = y, state = REQ– Effector: Objective id = 1, value = y, state = PROC– Requester: Requested unregister id = 1– Requester: Requested id = 2, value = z, state = REQ– Effector: Current x + 1 x + 2 … y– Effector: Objective id = 1, value = y, state = DONE– Effector: Objective id = 2, value = z, state = PROC– Requester: Requested unregister id = 2– Effector: Current y + 1 y + 2 … z– Effector: Objective id = 2, value = z, state = DONE

63

Directed State Pattern: How ToExample: Fire Missile

64

Fire ControlFire Control Fire Effector

Fire Effector

Launcher = 5State = IdleLauncher = 5State = Idle

Launcher #5

Request ID = 5Launcher = 5Action = FireState = Requested


1

Request ID = 5Launcher = 5Action = FireState = Processing

Request ID = 5Launcher = 5Action = FireState = Processing

2



4

Request ID = 5Launcher = 5Action = FireState = Done

Request ID = 5Launcher = 5Action = FireState = Done

7

Launcher = 5State = Prep’ingLauncher = 5State = Prep’ing 3

Launcher = 5State = FiringLauncher = 5State = Firing 5

Launcher = 5State = IdleLauncher = 5State = Idle 6

Firing Authorized

Firing Authorized

Target Identified

Target Identified

(Objectives ) (Current )

Directed State Pattern: Benefits

Allows components to express intent and communicate about actions

Allows additional observing components to leverage request, objective data

Avoids tight application-to-application coupling of RPC-like (anti-)patterns– More flexible asynchronous behavior– Stronger type-based, QoS-based governance– Easier insertion, removal of application components– Easier insertion, removal of infrastructure components

65

Directed State Pattern: Challenges

Beware degradation into RPC-style request-reply– Code smell: blocking behavior, e.g.

wait_for_acknowledgments– Code smell: point-to-point comm. back channels,

e.g. payloads in app ACKs– Code smell: stateless interactions, e.g. many

occurrences of keep-all reliability

66

Directed State Pattern: Challenges

Manage state ownership– One party established state, another needs to change it—why not

“pass” ownership on single topic?• Why not: Embedding workflow of current objective state transition into

multiple components=> maintenance pain

• Why not: “Same” objects may be written by different parties but with different ownership,e.g. I own #1, you own #2, she owns #3, …

• Why not: Moving ownership => hard to manage fault tolerance,e.g. persistence

– Best practice: always distinguish Current, (Requested) and Objective states

• Each should have a single owner• Fail-over should occur among redundant copies, not different functional

units

67

Directed State Pattern: See Also

Patterns– Observer pattern

Anti-Patterns– [GoF] “Command”– [EIP] “Command Message”– [EIP] “Correlation Identifier”– [EIP] “Point-to-Point Channel”– [EIP] “Request-Reply”– [EIP] “Return Address”– [EIP] “Scatter-Gather”

68

Part I Summary

Real-World Systems are facing information volume, velocity, and mgmt. challenges

Common solution is integration around a Data Model

DDS is a family of OMG specifications that directly supports data-centric publish-subscribe communications

Use of RTI DDS results in reduced programming, decreased cost, and lowered risk


Cost and Interoperability are the key drivers

Resources:http://dds.omg.orghttp://community.rti.comhttp://www.rti.com/dds

The Real-TimeMiddleware ExpertsThe Real-TimeMiddleware Experts

Day 2: Exercises

Gerardo Pardo-Castellote, Ph.D.Co-chair OMG DDS SIGCTO, Real-Time [email protected]

http://www.rti.com

Part II

Solving integration use-cases with RTI– Recording, Persistence, Database Integration, Web Access

Hello World with RTI DDS

Configuring QoS via XML

Development and Run-time Tools: – Ping, Spy, Analyzer, Wireshark, Excel– Discovery and Builtin-Topics

Protocol, Performance & Scalability.

Future directions and Standards

RTI DDS builds Higher quality, Lower TCO Systems

DDS Global

Data Space

Messaging & Caching

Event Processing

Database Bridge

Persistence& Durability

RecordingRedundancy & Failover

SQL

72

CompComp

Comp

© 2010 Real-Time Innovations, Inc.

Presence Discovery Historical Cache Availability Redundancy Failover Mgmt Durable Data DDS QoS Security Guard

Hooks

Pre-built components address many challenging use-cases Extended QoS (*) Durable Writers (*) Parallel Persistence (*) Excel Visualization (*) Monitoring (*) Recording (*) Database Connectivity (*) Web Accessibility (*) Transformation (*) Event Processing (*) Secure TLS Transport (*) WAN Routing (*) Configurability via XML (*) Dynamic Data & Types (*) Pluggable Transports,

Pluggable Discovery(*)

(*) Extended capabilities in RTI DDS

(*) Extended capabilities in RTI DDS

Real-Time Recording Service

Applications:– Future analysis and

debugging– Post-mortem– Compliance checking– Replay for testing and

simulation purposes

Record high-rate data arriving in real-time

Non-intrusive – multicast reception


Demo:1. Start RecordingService2. Start ShapesDemo3. See output files4. Convert to: HTML XML5. View Data: HTML XML

sqlite stop_all start

Data Persistence

A standalone service that persists data outside of the context of a DataWriter

Data Writer

Global Data Space

Data Reader

PersistenceService

PersistenceService

Data Reader

Data Writer

PermanentStorage

PermanentStorage

Can be configured for:

• Redundancy

• Load balancing


Demo:1. PersistenceService2. ShapesDemo3. Application failure4. Application re-start5. Persistence Svc failure6. Application re-start start

stop_all

Ownership and High Availability

Owner determined per Topic and Key Only writer with highest strength can publish a Key Automatic failover when highest strength writer:

– Loses liveliness– Misses a deadline– Stops writing the subject

Shared Ownership allows any writer to update any object

Producer / Writer

strength=10

Topic T1

K1 K2Producer / Writer

strength=5

Producer / Writer

strength=1

K1 Primary

K1 Backup

K2 PrimaryK2 Backup


ShapesDemo

Relational Actions

Relational Database Integration

Topic T1

I1 I2 I3I1I2I3

Table T1

Publish-Subscribe ActionWrite()Read() & Take()Dispose()Wait() & Listener

UPDATE [2,3] & INSERTSELECTDELETE

Event driven – The fastest way to observe database changes!© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 76

RTIReal-TimeConnect

1. start mysql2. start RTC

ShapesDemo

sql_gui

stop_all

sql_shell

RTI Routing Service

Selective, real-time data forwarding and transformation

Can Change Topic Name and Topic Schema– Allows for custom transformations via “plugin”– Can filter/guard data

QoS managed, can cache last-knownvalue for data

Dynamically configured

Location independent deployment

DDS Router

GUARDXFORM

Topic A Topic B

ShapesDemo_0

start

ShapesDemo_1

stop_all

Global Scalability: LAN to WAN……without sacrificing Performance and Security

DDSDDSRoutingRouting


Site ASite A

Site BSite B

Site CSite C

Site DSite D

WAN / Internet

TCP/TLS/SSL

WAN / Internet

TCP/TLS/SSL

Topics:Site StatusAlarmsHealth LogsSensor DataProc Sensor Data

Topics:Site StatusResult Data

Topics:Site StatusProc Sensor DataResult DataAlarms

Topics:Site StatusSensor Data








79

Web Accessibility

Direct access to real-time data from Web-Based Applications

Tactical Real-Time

Data

Web Enabled

DDS

Web Enabled

DDS

Web Enabled

DDS

GUARD

RecordedData

DDS Recording

Service

RecordedData FilesRecordedData Files

RecordedTrack FilesRecorded

Track Files

1. start replay & router

2. start

view maps

simulatedtracks

stop_all

COTS tools: Excel – Interacting with your data

Display live RTI DDS Data in Excel

Perform real-time computations and charts

Publish RTI DDS data from Excel

blank saved demoShapesDemo

Part II







Hands-on Example (C++)

TypeDefinition

MyType.idl

rtiddsgen

MyType.h

MyTypeSupport.c MyTypePublisher.cpp

MyTypeSubscriber.cpp

MyType.sln

Publisher.exe Subscriber.exe

Three minutes to a running app!!1. Define your data2. Create your project3. Build4. Run: publisher subscriber

Aux:File BrowserConsoleDelete Filesrtiddsspy

compiler

Exercise #1 - Hello World

Define you data type:

Create a directory “HelloWorld”

Create a file called HelloWorld.idl and open it in VisualStudio

Add the following contents:

const long MSG_LEN=256;struct Hello {

string<MSG_LEN> user; //@keystring<MSG_LEN> msg;

};

Run rtiddsgen (for C++)

rtiddsgen HelloWorld.idl -language C++ -example i86Win32VS2005 \

-replace -ppDisable rtiddsgen HelloWorld.idl -language Java -example i86Win32jdk \

-replace -ppDisable

Look at the directory you should see:– HelloWorld-vs2005.sln– And Several other files…

Open the Solution File (type hello-vs2005.sln on the console)– Look at HelloMsgPublisher.cxx– Look at HelloMsgSubscriber.cxx

Build the Solution

Run rtiddsgen (for Java)

rtiddsgen hello.idl -language Java -example i86Win32jdk \

-replace -ppDisable

Look at the directory you should see:– makefile_hello_i86Win32jdk– And Several other files…

• Look at HelloMsgPublisher.java• Look at HelloMsgSubscriber.java

You can use the makefile to build and the Java programs:– gmake –f makefile_hello_i86Win32jdk

Execute the program

C++:– On one window run:

• objs\i86Win32VS2005\HelloMsgPublisher.exe

– On another window run:• objs\i86Win32VS2005\HelloMsgSubscriber.exe

Java– On one window run:

• gmake –f makefile_hello_i86Win32jdk HelloMsgPublisher

– On another window run:• gmake –f makefile_hello_i86Win32jdk HelloMsgSubscriber

You should see the subscribers getting an empty string…

Modify the program to produce something

C++: Open HelloMsgPublisher.cxx in VisualStudio

Java: Open HelloMsgPublisher.java in your preferred tool

Look for the comment:/* Modify the data to be sent here */

Add the line:

strcpy_s(instance->user, MSG_LEN, “Your name here");

strcpy_s(instance->msg, MSG_LEN, "Hello world");

Kill the Publisher, Rebuild the publisher and run it again

Playing with rtiddsspy

Run rtiddsspy while the other applications are running

Start and stop applications. What do you see in rtiddsspy

Example: Publication

// Entities creationDomainParticipant participant = TheParticipantFactory->create_participant(

domain_id, participant_qos, participant_listener);

Publisher publisher = domain->create_publisher( publisher_qos, publisher_listener);

Topic topic = domain->create_topic( “MyTopic”, “Text”, topic_qos, topic_listener);

DataWriter writer = publisher->create_datawriter( topic, writer_qos, writer_listener);

TextDataWriter twriter = TextDataWriter::narrow(writer);

TextStruct my_text;twriter->write(&my_track);

Example: Subscription

// Entities creationSubscriber subscriber = domain->create_subscriber(

subscriber_qos, subscriber_listener);

Topic topic = domain->create_topic( “Track”, “TrackStruct”, topic_qos, topic_listener);

DataReader reader = subscriber->create_datareader( topic, reader_qos, reader_listener);

// Use listener-based or wait-based access

How to Get Data? (Listener-Based)

// Listener creation and attachmentListener listener = new MyListener();reader->set_listener(listener);

// Listener codeMyListener::on_data_available( DataReader reader ){ TextSeq received_data; SampleInfoSeq sample_info; TextDataReader reader = TextDataReader::narrow(reader); treader->take( &received_data, &sample_info, …) // Use received_data printf(“Got: %s\n”, received_data[0]->contents);}

How to Get Data? (WaitSet-Based)

// Creation of condition and attachementCondition foo_condition = treader->create_readcondition(…);waitset->add_condition(foo_condition);

// WaitConditionSeq active_conditions;waitset->wait(&active_conditions, timeout);

// Wait returns when there is data (or timeout)FooSeq received_data;SampleInfoSeq sample_info;

treader->take_w_condition(&received_data,

&sample_info, foo_condition);

// Use received_data printf(“Got: %s\n”, received_data[0]->contents);

Listeners, Conditions & WaitSets

Middleware must notify user application of relevant events:– Arrival of data– But also:

• QoS violations• Discovery of relevant entities

– These events may be detected asynchronously by the middleware… Same issue arises with POSIX signals

DDS allows the application to choice:– Either to get notified asynchronously using a Listener– Or to wait synchronously using a WaitSet

Both approaches are unified using STATUS changes

Status ChangesDDS defines A set of enumerated STATUS The statuses relevant to each kind of DDS Entity

DDS entities maintain a value for each STATUSSTATUS Entity

INCONSISTENT_TOPIC Topic

DATA_ON_READERS Subscriber

LIVELINESS_CHANGED DataReader

REQUESTED_DEADLINE_MISSED DataReader

RUQESTED_INCOMPATIBLE_QOS DataReader

DATA_AVAILABLE DataReader

SAMPLE_LOST DataReader

SUBSCRIPTION_MATCH DataReader

LIVELINESS_LOST DataWriter

OFFERED_INCOMPATIBLE_QOS DataWriter

PUBLICATION_MATCH DataWriter

struct LivelinessChangedStatus { long active_count; long inactive_count; long active_count_change; long inactive_count_change;}

Listeners, Conditions and Statuses

A DDS Entity is associated with:– A listener of the proper kind (if attached)– A StatusCondition (if activated)

The Listener for an Entity has a separate operation for each of the relevant statuses

STATUS Entity Listener operation

INCONSISTENT_TOPIC Topic on_inconsistent_topic

DATA_ON_READERS Subscriber on_data_on_readers

LIVELINESS_CHANGED DataReader on_liveliness_changed

REQUESTED_DEADLINE_MISSED DataReader on_requested_deadline_missed

RUQESTED_INCOMPATIBLE_QOS DataReader on_requested_incompatible_qos

DATA_AVAILABLE DataReader on_data_available

SAMPLE_LOST DataReader on_sample_lost

SUBSCRIPTION_MATCH DataReader on_subscription_match

LIVELINESS_LOST DataWriter on_liveliness_lost

OFFERED_INCOMPATIBLE_QOS DataWriter on_offered_incompatible_qos

PUBLICATION_MATCH DataWriter on_publication_match

Listeners & Condition duality

A StatusCondition can be selectively activated to respond to any subset of the statuses

An application can wait changes in sets of StatusConditions using a WaitSet

Each time the value of a STATUS changes DDS– Calls the corresponding Listener operation– Wakes up any threads waiting on a related status change

Asynchronous notificationvia Listener operation

Synchronous notificationvia activation/wakeup of

conditions/waitsets

DDS EntityStatus Change

Exercise #2 – Shapes Publisher

Create a new directory Shapes

In the Directory create a file called ShapeType.idl

Edit the file to have the following content:

const long COLOR_LEN=64;struct ShapeType { string<COLOR_LEN>color; //@key long x; long y; long shapesize;};

Run:

rtiddsgen ShapeType.idl -language C++ -example i86Win32VS2005 –replace -ppNotRun

rtiddsgen Details

rtiddsgen [-d <outdir>] [-language <C|C++|Java|C++/CLI|C#>]

[-namespace] [-package <packagePrefix>]

[-example <arch>] [-replace] [-debug]

[-corba [client header file]] [-optimization <level of optimization>]

[-stringSize <Unbounded strings size>]

[-sequenceSize <Unbounded sequences size>]

[-notypecode] [-ppDisable] [-ppPath <path to the preprocessor>]

[-ppOption <option>] [-D <name>[=<value>]]

[-U <name>] [-I <directory>] [-noCopyable] [-use42eAlignment]

[-help] [-version] [-convertToIdl | -convertToXml | -convertToXsd |

-convertToWsdl]

[[-inputIdl] <IDLInputFile.idl> | [-inputXml] <XMLInputFile.xml> | [-inputXsd] <XSDInputFile.xsd> | [-inputWsdl] <WSDLInputFile.wsdl>]

DefinitionFile can be IDL, XSD and XML file

-example generates example pub/sub apps and makefiles for compilation.

-replace replaces everything that’s generated. Use if the data type definition has changed. Always use with caution if you’ve made modifications.

99

IDL vs. XML: IDL Example

struct MemberStruct{ short sData;}

typedef MemberStructType; //@top-level false

100

IDL vs. XML: XML Example

<?xml version="1.0“ encoding="UTF-8"?><types xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:noNamespaceSchemaLocation="../rti_dds_topic_types.xsd">

<struct name="MemberStruct" topLevel="false"> <member name="sData“ type="short"/> </struct>

<typedef name="MemberStructType" type="nonBasic“ nonBasicTypeName="MemberStruct“ topLevel="false"/>

</types>

101

IDL vs. XSD: XSD Example

<?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:dds="http://www.omg.org/dds" xmlns:tns="http://www.omg.org/IDL-Mapped/" targetNamespace="http://www.omg.org/IDL-Mapped/"> <xsd:import namespace="http://www.omg.org/dds" schemaLocation="rti_dds_topic_types_common.xsd"/> <xsd:complexType name="MemberStruct"> <xsd:sequence> <xsd:element name="sData" minOccurs="1" maxOccurs="1" type="xsd:short"/> </xsd:sequence> </xsd:complexType> 

<xsd:complexType name="MemberStructType"> <xsd:complexContent> <xsd:restriction base="tns:MemberStruct"> <xsd:sequence> <xsd:element name="sData" type="xsd:short" minOccurs="1" maxOccurs="1"/> </xsd:sequence> </xsd:restriction> </xsd:complexContent> </xsd:complexType> </xsd:schema>

Part II



Development and Run-time Tools: – Ping, Spy, Analyzer, Wireshark, Excel– Discovery and Builtin-Topics– Monitoring




Tools provide insight into a distributed system

RTI Analyzer– Understand connections and data flow– Tune QoS properties without changing

code

RTI Excel Addin– View Data– Inject Data– Compute / Analyze RT Data

RTI Protocol Analyzer– Sniff the wire and analyze traffic

RTI Monitoring– Visualize and monitor system

operation– Detect system problems, bottlenecks,

etc.

Let’s try the Tools!

105

XML-Defined Qos Profiles:Overview

QoS can be specified in XML file(s) and is automatically loaded when the application starts

XML – specified QoS will be automatically used when DDS entities are created – Globally for all new entities– Selectively, depending on the Topic associated with the

entity

XML –specified QoS can be accessed via new API calls:create_<ENTITY>_with_profile()get_< ENTITY >_qos_from_profile()set_default_< ENTITY >_qos_with_profile()

Benefits of using QoS profiles

Simplifies application programming. – Editing XML is simpler than editing/compiling source code– Same settigs can be shared between applications/languages

Accelerates development– Changing QoS does not require re-compilation & re-deployment

Increases flexibility and maintainability– Can change QoS applications to meet changing system needs without

requiring re-deployment

Improves code reuse – Separates configuration/deployment from application logic– Same code can be re-deployed on different

hardware/networks/systems just changing the QoS file

Enhances portability– Application code is not dependent on vendor QoS extensions

More on using QoS profiles

Could increase brittleness– QoS files can change between runs. What was working no longer

works the same way…

Portability– XML profile format now standardized as part of “DDS for light-

weight CCM” specification• Will be folded in the next revision of the standard

– File-based use of QoS profiles is always portable– Direct calls to the C/C++/Java are not part of the standard yet

Makes deployment dependent on accessing a file-system…– XML can be compiled into the code, but that negates some of the

benefits

What is a QoS Profile?

Set of related Entity-QoS, usually one per entity kind, identified by a name.– Provides all the information needed to fully configure the QoS for

any DDS Entity ( DomainParticipant, Publisher, DataWriter, …)

QoS policies and attributes can be set explicitly, in the profile or otherwise are inherited from it’s “base profile”– Profiles that do not specify a “base profile” automatically inherit from

the DDS-Specified Default QoS.

May contain multiple definitions of a EntityQos (e.g. DataWriterQos)– Can be disambiguated by Topic related to the Entity– If ambiguous the last definition prevails

Defined inside Qos libraries

What is inside an XML QoS file?

A collection of QoS profiles organized inside QoS Libraries<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation = "C:/RTI/ndds.4.4d/resource/qos_profiles_4.4d/schema/rti_dds_qos_profiles.xsd" version="4.4d"> <qos_library name="MyQosLib"> <qos_profile name="ReliableKeepLast" base_name="Basic"> <datawriter_qos> <reliability> <kind>RELIABLE_RELIABILITY_QOS</kind> <max_blocking_time> <sec>1</sec> <nanosec>0</nanosec> </max_blocking_time> </reliability> <history> <kind>KEEP_LAST_HISTORY_QOS</kind> <depth>10</depth> </history> </datawriter_qos> <datareader_qos> <reliability> <kind>RELIABLE_RELIABILITY_QOS</kind> </reliability> <history> <kind>KEEP_LAST_HISTORY_QOS</kind> <depth>1</depth> </history> </datareader_qos> </qos_profile> </qos_library> </dds>

Example

Field names are the same as in C++/Java source code. Eg.:

datawriter_qos.reliability.max_blocking_time.sec

How are XML Qos files specified?

1. NDDSHOME directory (first to be loaded)– $NDDSHOME/resource/qos_profiles_XXX/xml/NDDS_QOS_PROFILES.xml– Automatically loaded if present unless: DDS_ProfileQosPolicy::ignore_resource_profile == DDS_BOOLEAN_TRUE

2. NDDS_QOS_PROFILES Environment variable– NDDS_QOS_PROFILES= [<url1> | <url2>]; …– <file_url> ::= file://usr/local/default_dds.xml– <string_url> ::= str://<dds><qos_library>…</qos_library></dds>– Automatically loaded if defined unless: DDS_ProfileQosPolicy::ignore_environment_profile ==

DDS_BOOLEAN_TRUE

3. Application directory– USER_QOS_PROFILES.xml– Automatically loaded if present unless: DDS_ProfileQosPolicy::ignore_user_profile == DDS_BOOLEAN_TRUE

4. DDS_ProfileQosPolicy::url_profile– Automatically loaded if defined

5. DDS_ProfileQosPolicy::string_profile– Automatically loaded if defined Duplicate Qos Libraries and/or

Qos Profiles within a library are not allowed

QoS Profile Inheritance

A QoS profile can inherit the values of another QoS profile

Base profile set using the base_name tag Base profile can be in another file and/or library A Profile that does not specify a base_name

implicitly inherits from the Default Profile Example:

113

<qos_profile name="ReliableKeepLast" base_name="BasicQos"> <datawriter_qos> <reliability> <kind>RELIABLE_RELIABILITY_QOS</kind> <max_blocking_time> <sec>1</sec> <nanosec>0</nanosec> </max_blocking_time> </reliability> <history> <kind>KEEP_LAST_HISTORY_QOS</kind> <depth>10</depth> </history> </datawriter_qos> </qos_profile>

Modifying the default QoS using XML files

The “default” QoS used by DDS can be modified via an explicit call to:– set_default_<ENTITY>_qos_with_profile()

Alternatively a QoS profile can be marked such that it is used as the default QoS– In combination with the “topic filter” (next slide) this can be

used to surgically modify the QoS of a deployed application

<qos_library name="MyQosLib"> <qos_profile name="ReliableKeepLast" is_default_qos="true"> <datawriter_qos> … </datawriter_qos> <datareader_qos> … </datareader_qos> </qos_profile> </qos_library>

Fine-tuning which QoS applies to an Entity via Topic filters

Allows a QoS profile to affect only selected DataReaders or DataWriters

Affected DataReaders or DataWriters are specified via an expression in their Topic name

Single QoS file and profile can be used to configure the whole system

QoS of selected entities can be changed without application code changes

QoS can be inherited from another library. E.g.:

115

<datawriter_qos topic_filter="Alarm*"> …</datawriter_qos>

Configuration of QoS via XML-defined profiles

Try it out using the shapes demo!

Part II







20X Faster than JMS / Broker-based solutions

Throughput with a single publisher

0

5

10

15

20

25

30

35

40

45

2 4 6 9 11 18

CPU load [%]

[10

00

's s

am

ple

/s]

RTI DDS

JMS

Platform: Linux 2.6 on AMD Athlon, Dual core, 2.2 GHz

0

10,000

20,000

30,000

40,000

50,000

60,000

16 32 64 128 256 512 1024

Message/Data Size (bytes - without batching)

Po

int-

to-P

oin

t U

pd

ates

per

Sec

on

d

1-11-101-24

RTI DDS is about 20X faster than JMSRTI DDS is about 20X faster than JMS

RTI DDS reliable multicast exhibits near perfect scalabilityRTI DDS reliable multicast exhibits near perfect scalability

( 2KB messages )



Extremely low latency and jitter

Reliable, ordered delivery overGigabit Ethernet between 2.4 GHz Core 2 Quad

processors running 32-bit Red Hat Enterprise Linux 5.0

Orders of magnitude more scalable than broker-based solutions

Going from 1 to 888 subscribers of the same data has only a 10% impact on throughput

New topics can be added to a system without impacting the latency and throughput on other topics

Throughput with 8 topics is 8x the throughput with 1 topic

http://www.rti.com/products/dds/benchmarks-cpp-linux.html


Realizing Performance & Scalability

RTI DDS operates peer-to-peer, without brokers

RTI DDS uses RTPS, an Advanced Multi-Session protocol supporting Reliable Multicast

RTI DDS Approach

RTPS

JMSAMQPESBs

…

Others: Broker-based middleware

122© 2010 Real-Time Innovations, Inc.

Realizing Performance & Scalability

RTI DDS completely isolates each process and communicates peer-to-peer

Other DDS implementations communicate vis shared memory with local daemon

RTI DDS Approach

RTPS

SharedMem

RTPS

Others: Broker-based middleware


Advanced Scalability & Performance Techniques

Latency and Priority Aware message batching

Content-Aware multi-channel reliable multicast

Enhanced Reliable Protocol– Selective ACKs (SACKs) for Confirmed Reliability– NACK-only Reliable Protocol for Massive Scalability

Smart caching integrated with the message protocol

Content-Filtering at the source


Message Batching

write()

sender receiver

write()

sender

Send queue Receive queue

Send queue Receive queue

Without batching each message is separately sent. For small messages protocol headers might be bigger than payload

With batching messages are held a little and combined into larger batches maximizing throughout and minimizing CPU

receiver

Transparent:

Receiver still sees individual messages


Reliability with Batching

Reliability must work even when messages are batched

ACK or NACK of individual samples would negate some of the benefits of batching…

=> Protocol must be batch aware so that it can ACK/NACK complete batches!

B3

B2

B1

B3

B2

B1

ACK(B3), NACK(B2)

Repair B2

B3

B2

B1

write()sender

receiver


Batching has a huge payoff!

RTI DDS 4.3b perftest results

0

100

200

300

400

500

600

700

800

900

1000

0 1000 2000 3000 4000 5000

Sample size (bytes)

Th

rou

gh

pu

t (M

bp

s)

Linux Baseline

Linux 10Kb Batch

Intel Core2Duo Single-CPU Dual-Core 2.4GHz, 4MB cache32-bit CentOS 5 (RHEL 5), 2GB memory, Intel E1000 NIC


Company Confidential

Classic (TCP Style) Reliable ProtocolNo packet loss situation

01

02

03

04

01

02

03

04, HB

01

02

03

04ACK 1-405

06

07

08

05

06

07

08, HB

05

06

07

08ACK 1-8


ShapesDemo

Classic (TCP Style) Reliable Protocolwith some packet loss

01

02

03

04

01

02

03

04, HB

01

02

X

ACK 1-2, NACK 305

06

07

08

05

06

07

08, HB

06

07

08ACK 1-8

030405

X

X

Packets 04 and 05 are received but the protocol drops them because a prior packet 03 is missing.This wastes valuable bandwidth


RTI DDS Reliability (Reader Cache + SACK)improves performance when packet loss occurs

01

02

03

04

01

02

03

04, HB

01

02

X

04ACK 1-2, SACK 305

06

07

08

05

06

07

08, HB

05

06

07

08ACK 1-8

03

Packets 04 and 05 are received and cached waiting for the repair of 03.

No bandwidth is wasted.


RTI DDS NACK-only reliabilityeliminates ACK traffic if there no packet loss

01

02

03

04

01

02

03

04, HB

01

02

03

0405

06

07

08

05

06

07

08, HB

05

06

07

08

No ACK traffic under normal operating conditions


RTI DDS NACK-only reliabilitygreatly reduces traffic even with packet loss

01

02

03

04

01

02

03

04, HB

01

02

X

04NACK 305

06

07

08

05

06

07

08, HB

05

06

07

08

03

Negative Acknowledgments sentonly when some message is lost

This approach is far more scalable when there are many subscribers


Part II







OMG Standards update

DDS API - 2003

DDS Wire Interoperability - 2006

UML Profile for DDS - 2007

DDS for lwCCM – 2008

Extensible Topics for DDS – 2010

ISO C++ API – 2010

Java 5 API - 2010

Upcoming standards:+ Web Enabled DDS (Mar 2011)+ Secure DDS (Dec 2011)

135

Web-Protocol Access to DDS Global Data Space

Stateless access of data via standard Web Technologies & Protocols

Not a bridge, broker, ormessage router

WebApp

DDS

RTPS

Global Data Space

App

App

App

App

Web Integration Service

HTTP

DDS Security RFP

A PIM and language PSMs for set of “plugins” that provide Authentication, Authorization, Data Tagging, Signing, and Encryption

A set of pre-defined plugins that can be used to define and enforce security policies.

Extensions to the DDS-RTPS Wire Protocol to support secure interoperability between DDS Systems.

136

Pub/Submiddleware(DDS)

Pub/Submiddleware(DDS)

ApplicationApplication

Secure OSSecure OS

Secu

re

Tran

spor

tSe

cure

Tr

ansp

ort

DDSDDS

ApplicationApplication

SecureDDS Global DataSecureDDS Global Data

Credential MgmtCredential MgmtAuthenticationAuthentication

Access ControlAccess Control

Data TaggingData Tagging

EncryptionEncryption

?

?

DDSDDS

ApplicationApplication?

DDSDDS

ApplicationApplication?

App.App.

Other DDS

System

Other DDS

System

Secure DDS middleware

AuthenticationPluginAuthenticationPlugin

Access ControlPluginAccess ControlPlugin Data Encryption

PluginData EncryptionPlugin

Data TaggingPluginData TaggingPlugin

Secure Kernel (e.g. SE Linux, MILS)

Crypto Module(e.g. TPM ) Crypto Module(e.g. TPM )

Secure Transport (e.g. TLS)Secure Transport (e.g. TLS)

CS application componentCS application componentcertificatescertificates

?

Datacache

ProtocolEngine

KernelPoliciesKernelPolicies

DDS Entities

NetworkDriverNetworkDriver

?

Network

1

2

3

4

5

6

7

8

9

Encrypted DataTAGS

Other DDS

System

Other DDS

System

Other DDS

System

Other DDS

System

App.App.App.App.

Part III

Application development cycle

Advanced Exercise

Exercise: How could “chat rooms” be implemented?

Different Topics for each Chat room?

Map to Partitions?

Add field to the message and use content-filtered Topics?

Same as before and also make room part of the Key?

Others?

Discuss pros and cons of each approach

Exercise: How could we implement Ground control stations that monitor UAVs

Different Topics for each UAV?– Or use Keys?

Different Domains for each Ground Station?– Or Partitions?

How to control multiple UAVs from the same ground station?

How to switch the ground station that controls the UAV?

How to do failover between ground stations?

How to direct a message to one or all UAVs?

How to detect loss of connection to an UAV?

Discuss pros and cons of each approach

Edit the chat_publisher.cxx

Go to the line with comment: /* Main loop */– Add the line:strcpy_s(instance->name,

NAME_LEN, "Gerardo Pardo");

(Use your own name)

Go to the line with comment:– /* Modify the data to be sent here */– Add the lines:instance->age = count; strcpy_s(instance->msg,

NAME_LEN, “Como va todo?");(Use your age and personalized message)

Rebuild and execute

Exercise #3 – Chat Room

Create a new directory Chat

In the Directory create a file called chat.idl

Edit the file to have the following content:

const long NAME_LEN=64;const long MSG_LEN=256;struct ChatMsg { string<NAME_LEN>name; //@key long age; string<MSG_LEN> chatRoom; string<MSG_LEN> msg;};

Run:

rtiddsgen chat.idl -language C++ -example i86Win32VS2005 –replace -ppNotRun

Edit the chat_publisher.cxx

Go to the line with comment: /* Main loop */– Add the line:strcpy_s(instance->name,

NAME_LEN, "Gerardo Pardo");

(Use your own name)

Go to the line with comment:– /* Modify the data to be sent here */– Add the lines:instance->age = count; strcpy_s(instance->msg,

NAME_LEN, “Como va todo?");(Use your age and personalized message)

Rebuild and execute

Exercise #3 (continued) Use Qos

Set RELIABILITY

Set HISTORY to KEEP_LAST or KEEP_ALL– Test different depths

Use Partitions– Create several Partitions:

• E.g. by ChatRoomName

– Publish in your ChatRoom– Subscribe to one or more ChatRooms

Exercise #4 (continued) Use content filters

Edit the chat_subscriber.cxx

Add the lines:DDSContentFilteredTopic *cftopic;

DDS_StringSeq filter_params;

filter_params.maximum(0);

cfTopic = participant->

create_contentfilteredtopic(

"Selected Chats", topic,

"age > 4", filter_params);

Look of the call to create_datareader– Replace “topic” with “cfTopic” in the paramater list.

Exercise #3 (continued) Use Exclusive Ownership

Set up in pairs edit the chat_publisher.cxx and use the same “name” for both of you Re-run the publisher application you will see mixed messages.

Edit the chat_publisher.cxx Before creating the data writer add the lines

publisher->get_default_datawriter_qos(dwq); dwq.ownership.kind = DDS_EXCLUSIVE_OWNERSHIP_QOS; dwq.ownership_strength.value = 10;

Replace DDS_DATAWRITER_QOS_DEFAULT with dwq In the create_datawriter() call

Edit the chat_subscriber.cxx Before creating the data reader add the lines

DDS_DataReaderQos drq; subscriber->get_default_datareader_qos(drq); drq.ownership = DDS_EXCLUSIVE_OWNERSHIP_QOS;

Replace DDS_DATAWRITER_QOS_DEFAULT with drq in the create_datareader() call

Exercise

Record the data

Exercise

Use durable data with a persistence service

Exercise

Isolate domains and transform data using the routing service

Summary

Reduces software lifecycle costs– Loose coupling– Replaces need for custom

middleware in high performance, real time applications

Reduces risk– Standards-compliant API

and wire protocol– Multiple implementations– Widely adopted

Most widely proven and mature implementation

Highest performance

Industry-leading expertise and services capability

Free trial, research and IR&D licenses

Comprehensive platform support

rti data-distribution service (dds) master class 2011

Technology

state data

data centricity

meeting rick

real world systems

realtime innovations

innovations email

data model necessary

facebook alternative