1

Encore Data Distribution Services

WorkshopFebruary 19, 2004

Encore Data Distribution Services

WorkshopFebruary 19, 2004

the

optionsclearingcorporation

2

What is DDS?What is DDS?

Data Distribution Services (DDS) - ENCORE outbound data processing implementation

Replaces all Data Files (Data Service, Series, Prices, etc.)

Takes advantage of new and more flexible technologies: - XML based messages - Event Driven Processing - Real Time Delivery

Data Distribution Services (DDS) - ENCORE outbound data processing implementation

Replaces all Data Files (Data Service, Series, Prices, etc.)

Takes advantage of new and more flexible technologies: - XML based messages - Event Driven Processing - Real Time Delivery

3

XML (Extensible Markup Language) allows for a more cost-effective, faster and easier means of modifying transmission content

No restrictions on message size, allowing data that belongs together to stay together

Self-describing in nature

Easy to read

Well suited for dynamically repeating data elements

XML (Extensible Markup Language) allows for a more cost-effective, faster and easier means of modifying transmission content

No restrictions on message size, allowing data that belongs together to stay together

Self-describing in nature

Easy to read

Well suited for dynamically repeating data elements

Advantages of XMLAdvantages of XML

4

FIX (Financial Information eXchange) defines a standard industry protocol for content in both front and back office processing

OCC is one of several organizations working with FIX to create these standards

FIX releases are published in an 18 month cycle

The current FIXML version, available since April 2003, is 4.4

Transmissions will no longer be in a format proprietary to OCC

Goal for Subscribers: Utilize the same process for transmissions from multiple organizations

FIX (Financial Information eXchange) defines a standard industry protocol for content in both front and back office processing

OCC is one of several organizations working with FIX to create these standards

FIX releases are published in an 18 month cycle

The current FIXML version, available since April 2003, is 4.4

Transmissions will no longer be in a format proprietary to OCC

Goal for Subscribers: Utilize the same process for transmissions from multiple organizations

Industry Standard Definition and SupportIndustry Standard Definition and Support

5

A FIX message, also referred to as a report, represents a group of fields used to describe a common functionality

The number of distinct message formats will decrease from 34 current data service layouts to 9 FIX messages

When OCC’s data service content does not map to any existing FIX messages customized fields or messages will be used

Customizations will be formalized through a proposal process with the goal of receiving an approval from the FPL (FIX Protocol Limited) committees

A FIX message, also referred to as a report, represents a group of fields used to describe a common functionality

The number of distinct message formats will decrease from 34 current data service layouts to 9 FIX messages

When OCC’s data service content does not map to any existing FIX messages customized fields or messages will be used

Customizations will be formalized through a proposal process with the goal of receiving an approval from the FPL (FIX Protocol Limited) committees

Current Data Service to DDS MappingCurrent Data Service to DDS Mapping

6

DDS Process OverviewDDS Process Overview

DDS SubscriberCMTA Transfer

entered into ENCORE

CMTA Transfer is validated and posted to positions

PUSHDDS Subscriber

receives Post Trade Messages

immediately

PULLDDS Subscriber opens MQ channel or downloads file when they are ready to

process

Post Trade

Message

Post Trade

MessageCMTA Transfer is

translated into FIXML format

Messages are loaded onto

subscriber MQ or FTP server

7

Real Time processing - transactional data will be available as soon as it is processed in the Encore system

Real Time outbound transmissions examples: trades and post trades

DDS will also offer real time subscription to master file data (product and contract data service records)

Batch data will be packaged and delivered as soon as it becomes available

Real Time processing - transactional data will be available as soon as it is processed in the Encore system

Real Time outbound transmissions examples: trades and post trades

DDS will also offer real time subscription to master file data (product and contract data service records)

Batch data will be packaged and delivered as soon as it becomes available

DDS - Event Driven ProcessingDDS - Event Driven Processing

8

Subscriber - entity (clearing member, exchange, regulatory agency) that represents the final beneficiary of DDS data

Recipient - entity (clearing member, exchange, regulatory agency or service bureau) that owns the systems where DDS data will be delivered

A recipient can receive both real-time and batch transmissions

Possible setup scenarios: 1). a given entity can act as a subscriber and recipient at the same time

2). a given subscriber can have its data distributed to one or more recipients

3). a given recipient can receive data for multiple subscribers

Subscriber - entity (clearing member, exchange, regulatory agency) that represents the final beneficiary of DDS data

Recipient - entity (clearing member, exchange, regulatory agency or service bureau) that owns the systems where DDS data will be delivered

A recipient can receive both real-time and batch transmissions

Possible setup scenarios: 1). a given entity can act as a subscriber and recipient at the same time

2). a given subscriber can have its data distributed to one or more recipients

3). a given recipient can receive data for multiple subscribers

Subscriber - Recipient ProfilesSubscriber - Recipient Profiles

9

A subscriber can choose to have one or more transmissions bundled as one package

Every package will be associated with one or more recipient destinations of the following type:

1). real time - an MQ channel between OCC’s server and recipient’s systems 2). batch file - to be pushed to or pulled by recipient’s systems

Data level security can be applied to the transmission packages (e.g. a package can contain data related to a specific group of tier accounts)

Packages to be delivered in batch mode will become available when processing of the last transmission in the package has completed

A subscriber can choose to have one or more transmissions bundled as one package

Every package will be associated with one or more recipient destinations of the following type:

1). real time - an MQ channel between OCC’s server and recipient’s systems 2). batch file - to be pushed to or pulled by recipient’s systems

Data level security can be applied to the transmission packages (e.g. a package can contain data related to a specific group of tier accounts)

Packages to be delivered in batch mode will become available when processing of the last transmission in the package has completed

DDS PackagesDDS Packages

10

For real-time transmissions the only supported mechanism will be MQ Series

For subscribers of real-time transmissions that already have MQ in house, the setup of a proper MQ channel will be required

For batch delivery, multiple communication lines are supported (leased lines, dial-up, internet, etc)

As we are in the process of testing various batch delivery software, details related to this subject will be forthcoming

For real-time transmissions the only supported mechanism will be MQ Series

For subscribers of real-time transmissions that already have MQ in house, the setup of a proper MQ channel will be required

For batch delivery, multiple communication lines are supported (leased lines, dial-up, internet, etc)

As we are in the process of testing various batch delivery software, details related to this subject will be forthcoming

Delivery MechanismsDelivery Mechanisms

11

DDS FIXML message sizes will increase compared to ODS data service records by a certain expansion ratio

The latest efforts of reduction in message size are reflected in the FIXML 4.4 Schema version

The main enhancements are a heavier usage of attribute based fields and the abbreviations of field names

Moving from COBOL fixed length records to XML is a more fundamental system change than simply changing a record layout

To enhance performance, XML parsers that can pipeline content rather than require the hierarchy of the XML document to be formed should be considered

DDS FIXML message sizes will increase compared to ODS data service records by a certain expansion ratio

The latest efforts of reduction in message size are reflected in the FIXML 4.4 Schema version

The main enhancements are a heavier usage of attribute based fields and the abbreviations of field names

Moving from COBOL fixed length records to XML is a more fundamental system change than simply changing a record layout

To enhance performance, XML parsers that can pipeline content rather than require the hierarchy of the XML document to be formed should be considered

Implications - Performance and Storage RequirementsImplications - Performance and Storage Requirements

12

Subscribers must have the ability to check for duplicate messages based on a supplied unique key(s) for each transmission

Subscriber’s systems need to be able to support multiple iterations of a transaction and take the appropriate actions

Example:

- A trade is received and processed within Encore and subsequently sent as a FIXML message to subscribers

- If the same trade gets busted (deleted) by an exchange, a new DDS message will be sent indicating that a backout needs to be applied for the given trade

Subscribers must have the ability to check for duplicate messages based on a supplied unique key(s) for each transmission

Subscriber’s systems need to be able to support multiple iterations of a transaction and take the appropriate actions

Example:

- A trade is received and processed within Encore and subsequently sent as a FIXML message to subscribers

- If the same trade gets busted (deleted) by an exchange, a new DDS message will be sent indicating that a backout needs to be applied for the given trade

Implications - Real Time Processing ChallengesImplications - Real Time Processing Challenges

13

Redundant data will be reduced by offering Product Issue transmissions and Option Series/Futures Contract transmissions separately

The message at the product level will be more detailed (delivery components and listing exchanges will be included)

Subscribers are advised to be able to process the Product updates independent of the Series/Contract updates

An incremental version for these messages will be available in real time and batch mode

The incremental message formats will contain a FROM section and a TO section allowing the retrieval of existing product/contract definition using the business keys from the FROM section and the update with the full definition supplied in the TO section

The 2 incremental message types will be delivered over the same MQ queue or in the same batch file in the chronological order of the occurrence of adds, modifies or deletes

Redundant data will be reduced by offering Product Issue transmissions and Option Series/Futures Contract transmissions separately

The message at the product level will be more detailed (delivery components and listing exchanges will be included)

Subscribers are advised to be able to process the Product updates independent of the Series/Contract updates

An incremental version for these messages will be available in real time and batch mode

The incremental message formats will contain a FROM section and a TO section allowing the retrieval of existing product/contract definition using the business keys from the FROM section and the update with the full definition supplied in the TO section

The 2 incremental message types will be delivered over the same MQ queue or in the same batch file in the chronological order of the occurrence of adds, modifies or deletes

Implications - Separate Product and Contract TransmissionsImplications - Separate Product and Contract Transmissions

14

FIX standard encourages the use of dates to drive processing and OCC views the usage of dates as a critical factor in the processing of DDS data.

OCC will provide clearing business dates on all transactional and positions based data

For all security master transmissions, OCC will provide activate and inactivate/expiration dates as well as listing dates for each exchange

Subscriber back office and/or order routing processes may need to be modified to interrogate these dates when receiving transmissions

OCC expects that subscribers will use inactivation/expiration dates to drive purge processing when applicable

FIX standard encourages the use of dates to drive processing and OCC views the usage of dates as a critical factor in the processing of DDS data.

OCC will provide clearing business dates on all transactional and positions based data

For all security master transmissions, OCC will provide activate and inactivate/expiration dates as well as listing dates for each exchange

Subscriber back office and/or order routing processes may need to be modified to interrogate these dates when receiving transmissions

OCC expects that subscribers will use inactivation/expiration dates to drive purge processing when applicable

Implications - Date Driven ProcessingImplications - Date Driven Processing

15

The final schema used for all DDS messages will be an extension to the current FIXML schema version at the time of DDS implementation

OCC expects to participate in a proposal process which involves submitting the necessary documentation to FPL to include the customized fields in FIX

The FPL committees and other FIX members will validate the viability of the content and representation of customized fields

OCC will periodically transition its FIXML messages to future FIX releases

OCC plans to provide at any time support for 2 concurrent FIXML releases

The final schema used for all DDS messages will be an extension to the current FIXML schema version at the time of DDS implementation

OCC expects to participate in a proposal process which involves submitting the necessary documentation to FPL to include the customized fields in FIX

The FPL committees and other FIX members will validate the viability of the content and representation of customized fields

OCC will periodically transition its FIXML messages to future FIX releases

OCC plans to provide at any time support for 2 concurrent FIXML releases

Implications - Adherence To Standards TradeoffsImplications - Adherence To Standards Tradeoffs

16

DDS TimelineDDS Timeline

Q3 2005

Data Distribution Systems

Publish Formats and

Technical Specifications

Q1 2004 Q3 2004

DDS Go LiveDecommission

current data service

12 month overlap between DDS and current

data service

17

Plan a project to accommodate processing of new DDS data service

Understand benefits and implications to your organization presented in the DDS external documentation

Get accustomed to the new layouts in FIXML format

Participate as a pilot user in the implementation of DDS

Plan for a testing phase of DDS transmissions between OCC and your organization

Contact your OCC CM Representative or email to ddshelp@theocc.com questions or comments related to DDS

Plan a project to accommodate processing of new DDS data service

Understand benefits and implications to your organization presented in the DDS external documentation

Get accustomed to the new layouts in FIXML format

Participate as a pilot user in the implementation of DDS

Plan for a testing phase of DDS transmissions between OCC and your organization

Contact your OCC CM Representative or email to ddshelp@theocc.com questions or comments related to DDS

Next StepsNext Steps