the omg real time data distribution service hans-arno jacobsen e-mail: [email protected]...

The OMGReal Time Data Distribution Service

Hans-Arno JacobsenE-mail: [email protected]

Phone: 416-946 7586

Department of Electrical and Computer Engineering &

Department of Computer ScienceUniversity of Toronto

ICDCS 1st DDRTS Workshop, 2003Hans-Arno Jacobsen, University of

Toronto

Outline

• Motivating Example• Requirements & Characteristics• The OMG DDS Service• Our Research in this Space


Toronto

Motivating Application Examples

• Air traffic control• Network equipment management• Industrial automation• Traffic control• Railroad control• Command and control (e.g., C4I)• Distributed control and simulations• …


Toronto

Characteristics and Requirements

• Data-centric exchange of information and coordination of activities

• Loosely coupled• Operate on global data space• Publish/Subscribe-style interactions• Continuously and dynamically evolving• All are (soft, hard, or quasi) real-time needy

– Primary concern: efficient data distribution with minimal overhead– Requires ability to control QoS properties: predictability, overhead,

and resources used– Also requires ability to scale to hundreds and even thousands of

entities• Hierarchical data model• Need for reliability and fault tolerance


Toronto

Global Data Space

node node

nodenode


Toronto

Hierarchical Data Representation

sensors

sub-systems

plants

controller(s)

1000+

subscribe


Toronto

Publish/Subscribe

Broker

Publisher Publisher

Subscriber Subscriber

Subscriptions

Publications

NotificationNotification

IBM=84

MSFT=27 INTC=19 JNJ=58ORCL=12

HON=24

AMGN=58

Stock marketsNYSE

NASDAQTSX

Subscriptions:IBM > 85ORCL < 10JNJ > 60


Toronto

The OMG DDS

• Overview• Architecture• Data-Centric Publish/Subscribe

Layer• Data Local Reconstruction Layer

Aims to address many of the afore mentioned requirements and more.


Toronto

The OMG DDS

• Request for Proposal issued in 9/2001• Speared-headed by Real-Time Innovations,

THALES, Objective Interface Systems, MITRE, and University of Toronto

• Submission “proof-of-concept” in existing products and research prototypes– NDDS from RTI– SPLICE from THALES– ToPSS (Toronto Publish/Subscribe System) MSRG @ UofT


Toronto

DDS Overview

• DDS comprises two layers: – DCPS (Data Centric Publish/Subscribe)

• To distribute the data– DLRL (Data Local Reconstruction Layer)

• To build a local object cache allowing applications to access the data “as if it was local”

• Specified according to OMG’s MDA approach– PIM (abstract view of the service)– CORBA PSM (concrete view when using CORBA)

• Focuses only on the interface between the application and the service– Does not address the exchanges between the different actors

implementing the service– Does not address scheduling of resources internal to DDS


Toronto

DDS Logical Architecture I

Data-centric Publish/Subscribe

Data Local Reconstruction Layer

Application


Toronto

DDS Logical Architecture II

DCPS

Application

DLRL

Application

DLRL

Application

DLRL

Application

DLRL

Communication

The Application{


Toronto

DCPS Overview• DCPS Models the DDS service with

– DomainParticipant– Topic & Data Type– DataWriter – Publisher– DataReader– Subscriber

• Configurable by QoS profiles• Data can be accessed in two ways

– Listener based (asynchronous via callbacks)– Wait based (synchronous)

• Similar for queriying communication status


Toronto

Domains and Participants

1

2

312

3

1

1

DomainParticipant NodeDomain

Domain Domain

•like a virtual private network


Toronto

DomainParticipantPublisher

DataWriter

Subscriber

DataReader

• DomainParticipant ~ Represents participation of the application in the communication collective

• DataWriter ~ Accessor to write typed data on a particular Topic

• Publisher ~ Aggregation of DataWriter objects. Responsible for disseminating information (i.e., for publishing).

• DataReader ~ Accessor to read typed data regarding a specific Topic

• Subscriber ~ Aggregation of DataReader objects. Responsible for receiving information (i.e., for subscribing).


Toronto

Topic-Based Publish/Subscribe

• Topic is used to associate DataWriters and DataReaders– Described by a string. E.g. “Pressure”– Bound on a single data-type e.g.

struct AnalogSensor { … };– Represents a collection of data-objects distinguished by

means of an application-defined key e.g.struct AnalogSensor { string sensor_id; // key float value; // other sensor data};

• Provision for content-based subscriptions


Toronto

Types

• Represent the data-type (format of the information)– E.g. “IDL struct” in the CORBA PSM– Provides description of application-defined key

• Manipulated only by means of automatically-generated typed interfaces

• Given type “Foo” associated with generated code:– FooDataWriter // write values– FooDataReader // read values)– FooType // register type

// description in DDS


Toronto

Topic-Based Publish/Subscribe

• DataWriter is bound (at creation time) to a single Topic

• DataReader is bound (at creation time) with one or more topics (Topic, ContentFilteredTopic, or MultiTopic)

• ContentFilteredTopic and MultiTopic provide means for content-based subscriptions

PressureTemperature


Toronto

Subscription = Topic + DataReader

Topic 1

application

Topic 2 Topic n

DataReader

DataReader

DataReader

subscriber subscriber

QoS

QoS

QoS


Toronto

QoS

• All DCPS entities configurable by means of QoS.• QoS tailored to data-distribution in real-time systems.

– RELIABILITY (BEST_EFFORT, RELIABLE)• Enables use of efficient transports for repetitive data

– DEADLINE• Establishes contract regarding rate at which repetitive data is refreshed

– LATENCY_BUDGET• Establishes guidelines for acceptable delays

– RESOURCE_LIMITS• Controls resources utilized by the service

– TIME_BASED_FILTER• Decouples fast producers from slow consumers

– HISTORY (KEEP_LAST, KEEP_ALL)• Supports semantics of state-data

– DURABILITY (VOLATILE, TRANSIENT, PERSISTENT)– and others…

• Request/offered compatibility checked by DDS


Toronto

Content-based Subscriptions

• ContentFilteredTopic (like a DB-selection)– Enables subscriber to only receive data-

updates whose value verifies a condition. – E.g. subscribe to “Pressure” of type AnalogData

– where “value > 200”

• MultiTopic (like a DB-join operation)– Enables subscription to multiple topics and re-

arrangement of the data-format– E.g. combine subscription to “Pressure” and

“Temperature” and re-arrange the data into a new type: struct { float pres; float temp; };


Toronto

Support for State-data Coherence

• Allow to access a set of modifications as-if made “atomically”– Restricted to modifications by the

same Publisher


Toronto

Example: Subscription

Subscriber subs = domain->create_subscriber(

subscriber_qos, subscriber_listener);

Topic topic = domain->create_topic( “Track”, “TrackStruct”, topic_qos, topic_listener);

DataReader reader = subscriber->create_datareader(

topic, reader_qos, reader_listener);


Toronto

Example: Publication

Publisher publisher = domain->create_publisher(

publisher_qos, publisher_listener);

Topic topic = domain->create_topic(

“Track”, “TrackStruct”, topic_qos, topic_listener);

DataWriter writer = publisher->create_datawriter(

topic, writer_qos, writer_listener);

TrackStruct my_track;Writer->write(&my_track);


Toronto

Summary of DCPS features• Efficient Publish/Subscribe interfaces• QoS suitable for real-time systems

– deadlines, levels of reliability, latency, resource usage, time-based filter

• Listener and wait-based data access– Suits different application styles

• Support for content-based subscriptions• Support for data-ownership• Support for history and persistence of

data-modifications


Toronto

Overview of DRLR

• Goals of DLRL• DLRL objects• Structural Mapping• DLRL entities• Layer is optional and it still is under

discussion


Toronto

Data Local Reconstruction Layer

TrackTrack 3D_Track

Track_Topic 3D -Track

DLRL

DCPS

Cache


Toronto

Goals of DLRL

• Data Local Reconstruction Layer• DLRL model is local to an application• Object oriented access to data• Application developer will be able to

– describe classes with their methods, data fields and relations

– attach some of those data fields to DCPS entities– manipulate these objects (i.e. create, read, write,

delete) using native language constructs – activate attached DCPS entities to update objects– have these objects managed in a cache

• Like a mapping or binding (intuition only)


Toronto

DLRL Objects• DLRL objects are (native) language objects

with:– data members and methods

• Only the data members are relevant to data distribution; they can be:– attributes, i.e., values– relations, i.e., reference other objects– plain local data membres (i.e., not known or

involved in data distribution) are also supported • DLRL classes can be organised by inheritance• DLRL objects maps to one or more DCPS

Topics


Toronto

DLRL Object Examples

Track

x : real

y : real

comments [*] : string

w : integer

Track3D

z : real

Radar

x : real

y : real


z : real

tracks a_radar

* 0..1

w : integer


Toronto

Structural Mapping

• The structural mapping describes the links between DLRL objects and DCPS data– e.g., in which 'topic' a given DLRL

attribute will be put...– close to Object-to-Relational mapping


Toronto

Example

Oid2

z300

T3D_TOPIC

Oid11

RADAR_TOPIC

Oid1

2

x100

102

y200

201

TRACK_TOPIC

ClassTrack

Track3D

radar11

11

COMMENTS_TOPIC

Oid1

1

commentsa comment

another comment

ClassTrack

Track

index0

1

R_Oid11

11

RADAR-TRACKS_TOPIC

T_Oid1

2

T_ClassTrack

Track3D

Track

x : real

y : real


w : integer

Track3D

z : real

Radar

x : real

y : real


z : real

tracks a_radar

* 0..1

w : integer

index0

1


Toronto

How This May Work?

Modeldescription

DLRL Generator ModelTags

Nativemodel

description

DCPSdescription

Dedicated DLRLentities


Toronto

Mapping DLRL to DCPS continued

• Structural mapping

relationship between DLRL objects and DCPS data (i.e., topics)

• Operational mappingmapping of DLRL objects to DCPS Entities (i.e., Publisher, DataWriter, …)

• Functional mappingrelation between DLRL functions and DCPS functions (write/publish)

• DLRL entities: Cache …


Toronto

The MSRG’s Research Projects in this Context

research projects


Toronto

Overview of Our Research Interests

• Efficient single-broker design (ToPSS project)– Publish/subscribe matching problem– Application programming model and data model– DDS design, implementation, and evaluation

• Efficient data-routing in multi-broker network (distributed ToPSS project)

• Design and implementation of non-functional requirements in middleware via aspect oriented programming (AOM project)


Toronto

Qualitatively Speaking - Research Challenges &

Requirements• large number of subscriptions (several per

user)• volatile subscriptions (changing subscriber

interests)• high rates of information input (e.g., news,

location information per user … )• different content formats (XML, HTML, WML,

ASCII)We develop efficient filtering, matching, correlationand data routing algorithms


Toronto

The Toronto Publish/Subscribe System Project (ToPSS)

• overall middleware platform framework to address these challenges

• support information dissemination applications in networked environments (mobile wireless, Internet, Intranets)

• support location-based services• support network management and monitoring &

control applications• framework components:

– ToPSS: Toronto Publish/Subscribe System• high-performance matching kernel

– pervasive notification engine (unified messaging)– context-aware extension to pervasive notification engine

ToPSS Family of Publish/Subscribe Systems

ToPSS, the Toronto Publish/Subscribe System is a flexible, high-performance matching kernel with numerous extensions:

Location-based matching (L-ToPSS)

Semantic matching-based (S-ToPSS)

Approximate matching-based (A-ToPSS)

Distribution and mobility support

Applications: Selective information dissemination and location-based services etc.

sub

scri

bers p

ub

lisher

s

Prof. Arno Jacobsen

ToPSS

Information consumer subscribe to information of interest.Information producer publish information. ToPSS-broker(s) match and route relevant information to interestedsubscriber.


Toronto

Research Projects Overview • ToPSS: Toronto Publish/Subscribe System

– research prototype that can manage up to 10 million subscribers with an event matching performance in the millisecond range

– High-performance matching kernel• L-ToPSS: location-based

– support location information as additional dimension• A-ToPSS: approximate matching-based

– processing of uncertain information• subscribe to events “close to downtown” Toronto• publish approximate subscriber location: “nearby CN tower”

• X-ToPSS: XML-based– subscribe to documents marked up in XML (e.g., web-pages)– publish XML content

• S-ToPSS: semantic-based– subscribe to one concept (e.g., in English “born in 1962”)– publish an ontologically related concepts (e.g., in French, Chinese, …

“Elle a 42 ans”)


Toronto

Subscription Matching Performance

(byte / bit-matrix vs. list-based)

bit

list

10 M


Toronto

Thanks for coming !


Toronto

Selected References• www.eecg.toronto.edu/~jacobsen soon www.msrg.org• S-ToPSS. VLDB 2003.• A-ToPSS VLDB 2002• Quantifying Aspect in Middleware. AOSD’02.• Middleware Services for Location-based Information

Dissemination. H.-A. Jacobsen. Middleware 2001.• The Push-Mi, Pull-Yu Event Notification Kernel for Enhanced

Publish/Subscribe. H.-A. Jacobsen. Middleware 2001.• Publish/Subscribe Systems. H-Arno Jacobsen, F. Llirbat Tutorial at

ICDE 2001.• Filtering Algorithms and Implementation for Very Fast

Publish/subscribe Systems . F. Fabret, H-Arno Jacobsen, F. Llirbat, J. Pereira, K. A, Ross, D. Shasha, SIGMOD2001 .

• WebFilter: High-Throughput XML-based Publish/Subscribe. F. Fabret, H-Arno Jacobsen, F. Llirbat, J. Pereira, D. Shasha, VLDB 2001.

• …


Toronto

Conformance Statement• Several profiles:

– Minimum: Just mandatory features of DCPS• must always be supported

– Content-subscription: Minimum plus…• classes: ContentFilteredTopic, QueryCondition, MultiTopic

– Persistence: Minimum plus…• ‘PERSISTENT’ setting of the DURABILITY QoS • Enables use of permanent storage to persist data

– Ownership: Minimum plus…• ‘EXCLUSIVE’ setting of the OWNERSHIP QoS . • OWNERSHIP_STRENGTH policy. • Setting of depth > 1 for the HISTORY QoS.

– Object model: Minimum plus…• ‘GROUP’ setting of PRESENTATION access_scope• DLRL layer

• Implementation might support multiple profiles– E.g. Minimum + Content-subscription + Ownership


Toronto

Predicate Matching Performance(tables-based vs. list-based

scheme)for the mixed domain

4 M


Toronto

Predicate Matching Data Structure Size(table-based vs. list-based scheme)

20 MB

1.4 GB

4 M


Toronto

Subscription Matching Data Structure Size

bit

list

10 M


Toronto

DBMS-based Matching

# subs kdb bit-wise list-based1,000 89 0.3 0.15,000 206 1.9 0.27

10,000 397 3.8 0.6350,000 1800 20.0 3.85

100,000 3451 40.4 8.1500,000 17299 264.0 79.0

1,000,000 33473 562.4 190.75,000,000 338166 1555.7 1056.0


Toronto

DLRL backup slides


Toronto

Operational Mapping

• Describes associations between a DLRL class and DCPS entities– DLRL class -> several 'topics' -> several 'data-

readers/writers'– All attached to the same 'subscriber/publisher'

• DLRL operations are provided to create all the entities that correspond to a set of DLRL classes

• No QoS settings at DLRL level– Means to retrieve the DCPS entities to set QoS if

needed


Toronto

Functional Mapping

• Describes translation between DLRL functions and DCPS functions

• DLRL functions– Publishing only nodes

• create objects, modify them, possibly destroy them • request for publication of the performed changes (creations,

modifications, destructions)

– Subscribing only nodes• load objects (i.e. make subscribed DCPS data, DLRL objects)• read their attributes and/or relations,• possibly use the relations to navigate among the objects,• be made aware on changes on the objects that are there, or on

the arrival of new objects.


Toronto

DLRL Entities

• Cache and CacheAccess– A Cache represents a set of objects managed

consistently. It comprises a Publisher and/or a Subscriber

– It comprises one or more CacheAccess that isolate a set of objects in a given access mode (read/write)

• ObjectHome– An ObjectHome is an object that manages all the

instance of a given application class, inside a Cache– It is typed according to the application type it

manages • ObjectRoot is the root for all the application classes


Toronto

DLRL Entities at Run-Time 2/2

• A set of typed entities represent relations and collections

• Selections are typed objects to gather a set of objects matching a given criterion

• Listeners are interfaces that the application may implement to be notified of the arrival of incoming modifications:– CacheListener

• begin/end of a set of modifications

– ObjectListener• creation/update/deletion of an object

– SelectionListener• entering/exiting/update of an object into the selection


Toronto

Typical Publishing Node

1. Create a Cache and attach the ObjectHomes; enable the infrastructure

2. Create a CacheAccess (if not using the default one)

3. Populate that CacheAccess (i.e., create and/or attach objects) and/or modify the attached objects (native calls)

4. Write the CacheAccess (i.e., write all the modifications on objects attached to it)

5. Eventually, purge the CacheAccess


Toronto

Typical Subscribing Node

1. Create the Cache and attach the ObjectHomes; enable the infrastructure

2. If needed, create Selections3. Attach Listeners4. Just wait...

– In the listener, access to the objects is made using native constructs

– If needed to take a consistent picture of a part of the graph, use a CacheAccess


Toronto

How to get data (listener-based)

Listener listener = new MyListener();Reader->set_listener(listener);

MyListener::on_data_available( DataReader reader ){ FooSeq received_data; SampleInfoSeq sample_info;

reader->take( &received_data, &sample_info, READ | NOT_READ, NEW | MODIFIED | DISPOSED);

// Use received_data}


Toronto

How to get data (wait-based)

Condition foo_condition = reader->create_readcondition(

READ | NOT_READ, NEW | MODIFIED | DISPOSED);

waitset->add_condition(foo_condition);

ConditionSeq active_conditions;waitset->wait(&active_conditions, timeout);…

FooSeq received_data;SampleInfoSeq sample_info;

Reader->take_w_condition(&received_data, &sample_info, foo_condition);

// Use received_data

the omg real time data distribution service hans-arno jacobsen e-mail: [email protected]...

Documents