high precision timestamps unveiling market microstructure · 2019. 10. 21. · act as a measure to...

High Precision Timestamps

Unveiling market microstructureSeptember 2019

Data Services

The HPT products:

Deutsche Börse is introducing the High Precision Timestamp (HPT)

information product suite

Provide valuable information on the flow of an order in the T7 ® architecture

Act as a measure to evaluate and optimise participants’ investment strategies

Can be used in algorithm backtesting

Are available for all order book updates sent via Co-location 2.0 Service

Data Services / Deutsche Börse Group 3

Are available for Eurex® and Xetra®

The ETICaptTime timestamp is captured at the outer boundary of

the exchange’s network and offers valuable insights into trading

system dynamics

RequestTime

▪ RequestTime timestamp is

captured at the T7 Gateway and

is publicly available in

EMDI/EOBI for aggressive orders

▪ Latency is introduced by the

Exchange network before the

incoming orders enter the

Gateway which is not reflected in

the RequestTime

▪ Using RequestTime to evaluate

the reaction time may not be

sufficient

ETICaptTime

▪ ETICaptTime timestamp is

captured as soon as an order

arrives at the outer boundary of

the exchange’s network

▪ It allows participants to isolate

the latency introduced by the

network

▪ ETICaptTime is made available to

subscribers via a daily file service

Use Case

▪ Two orders are sent to T7 and

compete for the same price

▪ The first order is matched, the

second order loses

▪ Using ETICaptTime participants

can calculate how long the 2nd

order lagged behind the 1st one

and use this information to

improve their strategies


The HPT offering includes the HPT Executions and HPT All

information products

HPT Executions HPT All

▪ Available for all market data updates

where the order was sent via Co-

Location 2.0 service

▪ Covers all Eurex (futures and options)

and all Xetra instruments

▪ Increases transparency for clients who

compete to add limit orders after a

trigger event

2

▪ Provides the ETICaptTime (t3_a)

timestamp for aggressive orders only

sent via Co-Location 2.0 service

▪ Covers all products traded on Eurex and

Xetra

▪ Allows participants to isolate the latency

introduced by the network and

accurately calculate the time delta to

winning orders

High Precision Timestamp File Service

1


Currently RequestTime is captured at the Gateway and is used to

estimate the time delta to the winning order

Matcher

t_1 t_3

Participant Gateway T7 PartitionNetwork

Latency

Exchange Network

Market Data

Publisher

(EMDI)

Step Description

1

Participant sends order from their

system at t_1 (t_1 is captured by

client)

2

Order arrives at the Gateway at

RequestTime (t_3). t_3 is captured

and published in EMDI/EOBI in field

RequestTime for aggressive orders

only

order

1 2


The ETICaptTime timestamp is captured at the outer bounds of the

Exchange Network and allows to determine network latency

Step Description

1Participant sends order from their

system at t_1 (captured by client)

2

The network timestamp ETICaptTime

(t_3a) is captured at the outer

boundaries of the Exchange’s

network. ETICaptTime is published

daily for all market updates in a CSV

file

3

Order arrives at the Gateway at t_3.

t_3 is captured and published in

EMDI/EOBI in field RequestTime for

aggressive orders only

Matcher

t_1 t_3

Participant Gateway T7 PartitionNetwork

Latency

Market Data

Publisher

(EMDI)

order

1 3

t_3a

2

Exchange Network


Participant

Example: Orders A to F are submitted at different times. The time lag

between A and F is currently calculated based on Gateway timestamps

A0

B0

C0

Session A

Session B

Session C

Session D

Session E

Session FF0

B2F2C2D2 A2E2

t

Switch

A2-F2 is the time lag

using RequestTime

Exchange Network

Gateway T7 Partition

Matcher

Incoming orders are

serialised at the network

switches

Currently, only RequestTime (t_3)

in EMDI/EOBI is captured as the

orders enter the Gateway

Incoming orders

D0

Market

Data

Publisher

(EMDI)

t_3A1

F1

B1C1D1

E1

RequestTime


E0

t_3a is captured at the outer boundaries of the network

allowing a highly accurate calculation of the time delta

between A and F

Example (cont.): The new timestamp ETICaptTime allows the calculation

of the time lag between orders A and F with higher precision

t RequestTime

Switch

Exchange Network

A2-F2 is the time lag

using RequestTime

t ETICaptTime

A0-F0 is a more

accurate calculation

of the real time lag

Participant

A0

B0

C0

Session A

Session B

Session C

Session D

Session E

Session F

E0

B2F2C2D2 A2E2

Gateway T7 Partition

Matcher

D0

Market

Data

Publisher

(EMDI)

t_3A1B1C1D1

E1

t_3a

Incoming orders


F0

F1

High Precision Timestamp

Subscription Details

1. HPT Executions File Service: Available for all Eurex and Xetra instruments

2. HPT All File Service: Available for all Eurex and Xetra instruments

Product

Scope

▪ The file is available at T+1 in CSV format via the Deutsche Börse Data Shop subscription

service

▪ For a full description of the information in the files please check the next slides

▪ Data availability

▪ HPT Executions Futures: from 01 Jun 2018

▪ HPT Executions Options (EMDI): from 13 Aug 2018 to 14 Mar 2019

▪ HPT All Futures: from 17 Jan 2019 onwards

▪ HPT All Options: from 16 Jul 2019 onwards

Data

Availability

Contact

DS Analytics Sales

[email protected]

EMEA US

Maria Boutsikou

[email protected]

Chris DeMaso

[email protected]

+44 (0) 207 862 7524 +1 212 309 9305


mailto:[email protected]



HPT Executions File: Field Description

Field Description

MarketSegmentID Product identifier, extracted from EOBI1 Packet Header

SecurityID Unique instrument identifier, extracted from EOBI Execution Summary (ES)

ExecID Matching timestamp, extracted from EOBI ES

ApplSeqNum Message sequence number, extracted from EOBI Packet Header

PartitionID Grouping of T7 products, extracted from EOBI

CompletionIndicator Indicates whether a unit of work fits into a single datagram, extracted from EOBI Packet Header

TradeCondition 1 = Implied Trade, extracted from EOBI ES

AggressorSide 1= Triggered by the buy side, 2= triggered by the sell side, extracted from EOBI ES

LastQty Total quantity of this match, extracted from EOBI ES

LastPx Worst price of this match, extracted from EOBI ES

RestingHiddenQty Quantity of matched passive orders that is not displayed to the market, extracted from EOBI ES

RestingCxlQty Extracted from EOBI ES

RequestTime Gateway request in timestamp of aggressing order, extracted from EOBI ES

AggressorTime Matching Engine In timestamp of aggressing order, extracted from EOBI ES

TransactTime Time when market data feed handler writes packet on the wire, extracted from EOBI

EOBICaptTime Time when market data feed packet is captured by distribution layer tap (t_9d)

ETICaptTime Time when aggressing order packet is captured by access layer tap (t_3a)

1Data Services / Deutsche Börse Group 11

1 T7 Enhanced Order Book Interface (EOBI) manual: www.eurexchange.com > Technology > T7 Trading architecture > System Doumentation

Note: EMDI timestamps was provided only for options between 13 Aug 2018 and 14 Mar 2019. After the EOBI Options feed launch 18 Feb 2019,

this file was discontinued. The Options data can be subsequently found within the standard HPT Executions file.

https://www.eurexchange.com/resource/blob/1614652/bd0a55fd889a2385c2686dc1ce3f1e37/data/T7_EOBI_Manual_v.8.0.1.pdf

HPT All File: Field Description

Field Description

ApplSeqNum Application sequence number, extracted from EOBI Packet Header1

MsgSeqNum Message sequence number, extracted from EOBI Message Header

Message TemplateID, extracted from EOBI Message Header. See next page for details

CompletionIndicator Indicates whether a unit of work fits into a single datagram, extracted from EOBI Packet Header

PartitionID Grouping of T7 products, extracted from EOBI / EMDI Packet Header

MarketSegmentID Product identifier, extracted from EOBI/EMDI1 Packet Header

SecurityID Unique instrument identifier, extracted from EOBI Execution Summary (ES)

Side 1= buy side, 2= sell side, in case of executions: triggering side

Price Price, in case of executions: worst price of this match

Qty Quantity, in case of executions: Total quantity of this match

TradeCondition 1 = Implied Trade, extracted from EOBI ES

RestingHiddenQty Quantity of matched passive orders that is not displayed to the market, extracted from EOBI ES (empty otherwise)

RestingCxlQty Extracted from EOBI ES (empty otherwise)

ExecID Matchier central timestamp, extracted from EOBI

TrdRegTSTimePriority Priority time, extracted from EOBI

RequestTime Gateway request in timestamp of aggressing order, extracted from EOBI ES or ETI packet

AggressorTime Matching Engine In timestamp of aggressing order, extracted from EOBI ES

TransactTime Time when market data feed handler writes packet on the wire, extracted from EOBI Packet Header

EOBICaptTime Time when market data feed packet is captured by distribution layer tap (t_9d)

ETICaptTime Time when triggering ETI packet is captured by access layer tap (t_3a), network entry time of an order in case of an Instrument State Change

message


1 T7 Enhanced Order Book Interface (EOBI) manual: www.eurexchange.com > Technology > T7 Trading architecture > System Doumentation


HPT All File: Message Types Covered

Message Description

13100 Order add

13101 Order modify

13102 Order delete

13106 Order modify same priority

13202 Execution summary

13104 Full order execution

13105 Partial order execution


HPT All File: Additional Information


▪ File naming convention:▪ Options data: 2402_00_D_03_A_YYYYMMDD_PP_O.zip

▪ Futures data: 2402_00_D_03_A_YYYYMMDD_PP.zip

Where YYYYMMDD is a date string and PP is the Partition Number the instruments will trade in

▪ For partitions where both options and futures are traded (i.e. Partitions 8, 9 and 10), two files will appear one for

futures and one for options. The options filename is appended with “ _O”. The rest of the partitions are either for

futures or options only.

▪ Files ending in “.zip_0” are MD5 checksums which can be used to verify your download

▪ For options, only the first message of the EOBI datagram will be represented in the HPT file. Users will need totake the MarketSegmentID + ApplSeqNum + TransactTime fields within the HPT file and join that to their own

EOBI data, using the same MarketSegmentID + ApplSeqNum + TransactTime fields contained within the

PacketHeader, as illustrated in the diagram below

▪ All messages within that packet can be assigned with the same t3a, t3n and t9d values made available in the HPT file

▪ By using this logic, we are able to reduce the number of rows by approximately 88%, attaining a combined zipped file

size of approximately 5GB for Eurex options

T7 Architecture and

Timestamps

Appendix 1

T7 Architecture and Timestamps1

T7 PartitionParticipant Gateway

EMDI

t_8

Market data publisher

(EOBI)EOBI

Core matching

ETI

t_1

t_2

t_3n

t_5

t_7t_9

t_8

t_11

t_10

t_6t_4’t_4

t_3

Market data publisher

(EMDI)

t_3a

t_9d

t_8d

Access

LayerDistribution

Layer

The above T7 architecture reflects the new timestamp t_3a. Network timestamps shown in light blue.


1 More details can be found in T7 Enhanced Order Book Interface (EOBI) manual and T7 Enhanced Market Data Interface (EMDI) manual

https://www.eurexchange.com/resource/blob/1470880/c1df7d4321b678c8ae0f1570262cad51/T7_EOBI_Manual_070.007.070-data.pdf

https://www.eurexchange.com/resource/blob/1470872/a9e0d336d87a774e7cc1158bbfb8823f/T7_EMDI_MDI_RDI_Manual_070.007.024-data.pdf

T7 Timestamp Reference1

The time stamps t_3 to t_9 are available via the following EMDI/EOBI fields:

t_3, t_3n: Tag 5979 (“RequestTime”) in the T7 ETI Response

in the T7 EMDI Depth Incremental message, in case a trade is reported

in the T7 EOBI Execution Summary message

t_4’: Tag 7765 (“ResponseIn”) in the T7 ETI Response (from the matching engine)

Tag 25043 (“NotificationIn”) in the T7 ETI Notification (from the matching engine)

t_4: Tag 52 (“SendingTime”) in the T7 ETI Response and Notification

t_5: Tag 21002 (“TrdRegTSTimeIn“) in the T7 ETI Response (from the matching engine)

Tag 21002 (“TrdRegTSTimeIn”) in the T7 EOBI Order Add, Order Modify, Order Modify Same Priority and Order Delete messages

Tag 28820 (“AggressorTimestamp“) in the T7 EMDI Depth Incremental message, in case a trade is reported

Tag 28820 (“AggressorTimestamp“) in the T7 EOBI Execution Summary message

t_6: Tag 21003 (“TrdRegTSTimeOut“) in the T7 ETI Response and Notification (from the matching engine)

t_7: Tag 17 (“ExecID“) in the T7 ETI Response (from the matching engine)

in the T7 EOBI Execution Summary message

Tag 273 (“MDEntryTime”) in the T7 EMDI Depth Incremental message

Tag 21008 (“TrdRegTSTimePriority”) in the T7 EOBI Order Add and Order Modify messages

Tag 60 (“TransactTime”) in the T7 EOBI Order Modify Same Priority and Order Delete messages

t_8: no Tag (“SendingTime”) in the T7 EMDI UDP packet header

t_9: Tag 60 (“TransactTime”) in the T7 EOBI packet header

(t_8-t_5): no Tag (“PerformanceIndicator“) in the T7 EMDI UDP packet header of the T7 EMDI Depth Incremental stream.

Notes on time stamps:

All time stamps provided are 8 byte integers (in nanoseconds after Unix epoch).

The PerformanceIndicator is a 4 byte integer (in nanoseconds as well).





Description of Timestamps

Definition1

Timestamp Description

t_1,t_2 can be taken by a Participant (e.g. via a network capture) when a request/ response is read from/written to the socket.

t_4 taken by the ETI gateway when a response/ notification is written to the socket on the Participant´s side of the gateway

t_3n taken by the ETI gateway when the first bit of a request arrives on the HF gateway NIC

t_3 taken by the ETI gateway application when a request is read from the socket on the Participant´s side of the gateway

t_3’ taken by the ETI gateway right before a request is sent towards the matching engine

t_4’ taken by the ETI gateway when a response/ notification is received by the ETI gateway from the matching engine

t_5, t_6 taken by the matching engine when a request/response is read/written

t_7 time at which the matching engine maintains the order book

t_8 time taken by EMDI publisher just before the first respective UDP datagram is written to the UDP socket.

t_9 time taken by EOBI publisher just before the first respective UDP datagram is written to the UDP socket.

t_10, t_11 can be taken by a Participant (e.g. via a network capture) when a UDP datagram is read from the UDP socket.





September 2019

Deutsche Börse AG is a public company registered under German law. This publication is published for information only and shall not

constitute investment advice. CEF®, DAX® and Eurex® are registered trademarks of Deutsche Börse AG. STOXX® is a registered trademark

of STOXX Ltd.

high precision timestamps unveiling market microstructure · 2019. 10. 21. · act as a measure to...

Documents