technology risk modeling and analysis seminar - exchange trading

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com

1

Технологические риски в биржевой торговле

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 2460, +1 415 830 38 49www.exactpro.com

14 Октября 2016Иосиф Иткин, Exactpro CEO, London Stock Exchange Group

Научно-практический семинарМоделирование и анализ технологических рисков

http://www.exactpro.com/

http://exactpro.com/


2

Technology Risk Typology

Financial Services

Money and Reputation

Minimal Health & Safety



3

It argues that if machine brains surpass human brains in general intelligence, then this new superintelligence could replace humans as the dominant lifeform on Earth.

Sufficiently intelligent machines could improve their own capabilities faster than human computer scientists. As the fate of gorillas now depends more on humans than on the actions of gorillas themselves, so will the fate of future humanity depend on the actions of the machine superintelligence. The outcome could be an existential catastrophe for humans.

Solving the control problem is surprisingly difficult because most goals, when translated into machine-implementable code, lead to unforeseen and undesirable consequences.

Advanced Technology



4

Existential Risk

Google DeepMind published a paper claiming that it is

possible to implement "a button" to interrupt undesired behaviour

Will we have enough time to press the button once?



5

Existential Risk



6

Robot Apocalypse Lessons

The only reason the characters were able to make it until the end of the second season is that they were relying on the very same technology that was trying to kill them.

They were protected by systems having the same level of sophistication as those they were struggling with. This is the mentality that we should adapt.

When we use a complex platform, we need to build software to test our software. And you do not want your risk control and test instruments to be inferior to what will hit you.

Having a good robot on your side is the only way to survive the robot apocalypse.



7

Build Software to Test Software

Automated testingCompares intended and received results

Automated end 2 end clearing system testing

Automated monitoring, analysis and reporting

75K messages / second from a single CPU coreMeasures latencies in microsecond range

A variety of algoes simulating end-clients will run and see how system performs



8


Cat on a Screen

Bug in the Code



9


Cat on a Screen

Bug in the Code

The Next Generation of Software Testing Tools



10

Learning from failure has the status of a cliché, but this book reveals the astonishing story behind the most powerful method of learning known to mankind, and reveals the arsenal of techniques wielded by some of the world's most innovative organizations. It also reveals the dangers of failing to learn from mistakes. In healthcare, hundreds of thousands of patients die from preventable medical errors every year due to a chronic lack of Black Box Thinking.

Using gripping case studies, exclusive interviews and really practical takeaways, Matthew Syed – the award-winning journalist and best-selling author of Bounce - explains how to turn failure into success, and shows us how we can all become better Black Box Thinkers.

Learning from Failure



11

What Can Possible Go Wrong



12

Fat Finger Errors12 October 2005Attempts to sell a single J-Com stock for 610,000 Yen ($5,041)Mistakenly price and quantity were swappedRisk systems failure:• Mizuho Securities• Tokyo Stock ExchangeEstimated loss $225 millions

This type of errors is called: Fat Finger



13

Fat Finger Errors18 September 2012, Stockholm. A trader had the intention of posting a sell order for 5,000 SKF B shares. Due to an input error with the Client, the order volume field was populated with a negative value (-5,000)

Instead of returning an error, the system converted the value into a random 9-digit figure - 294,962,296

The Sell Order corresponded to approximately 71 % of the total outstanding volume in the SKF B share. The Sell Order resulted in execution of 813,442 shares



14

Knight Capital Events• 1 August 2012, USA• Knight Capital – one of the most successful HFT firms• Implemented changes related to Retail Liquidity Program

at NYSE• SMARS – an ultra-fast order router• Source code responsible for legacy functionality PowerPeg• 212 parent orders, millions child orders• Accumulated loss – $460m or $170k/sec• Incorrectly configured risk systems• Deployment on 7 servers instead of 8…



15

Knight Capital Events



16

Knight Capital Events



17

Regulated Exchange



18

Facebook IPO on NASDAQ• 18 May 2012, NASDAQ, One of the largest IPOs in history• Secondary trading is preceded by a designate Display Only Period

(DOP)• Multi-component architecture that included Matching Engine, IPO

Cross Application and Execution Application• At the end of the DOP, NASDAQ’s “IPO Cross Application” analyzes all

of the buy and sell orders to determine the price at which the largest number of shares will trade; then NASDAQ’s matching engine matches the buy and sell orders at that price. Usually takes 1-2 ms

• NASDAQ allowed orders to be cancelled at any time up until the end of the DOP, including the very brief interval during which the IPO cross price is calculated. After calculation was completed, the system performed orders validation check between ME and “IPO Cross Application”. If any of the orders were cancelled after the start of the cross, the system would have to repeat the calculation



19

Failover Proposal



20

Facebook IPO on NASDAQ• Over 496k orders participated in the cross, and its duration exceeded

20ms• Order cancellation arrived during this period, and the application had to

repeat the calculation. Two more cancellations arrived during the second iteration, and four more - during the third one

• IPO Cross Application went into infinite loop at 11:05• The NASDAQ team switched off validation check on the secondary

system and performed failover 25 minutes after the start of the loop• Unknown at that moment, 38k orders submitted between 11:11 and

11:30 were stuck and did not participate in the uncross. It created another discrepancy, this time with Execution App and Members who were not able to receive confirmation for orders executed in the cross until 13:50



21

Failover Proposal



22

Facebook IPO Stock Chart

SEC Release No. 34-69655 / May 29, 2013


http://www.sec.gov/litigation/admin/2013/34-69655.pdf


23

Flash CrashThe May 6, 2010, Flash Crash, was a United States trillion-dollar stock market crash, which started at 2:32 p.m. and lasted for approximately 36 minutes.

Event is attributed to HFT, large directional sell, index arbitrage and market manipulation



24

Defects Management

Areas of research in defect management:

• automatic defect fixing

• automatic defect detection

• metrics and predictions of defect reports

• quality of defect reports

• triaging defect reports



25

Defects ManagementDefect clustering• Understand weaknesses of software

• Improve testing strategy

Use obtained data to:• Predict the probability of a defect to be re-opened• Highlight the area of the code where the bug is most likely located• Identify requirements and areas most prone to defects in the next release of software



26

The seventh EXTENT conference will be held in London, UK .

Find out more: www.extentconf.com

– LSEG Technology and Quality Assurance– Risk controls and FPGA– Market Surveillance systems– Blockchain and trading technology trends– Artificial intelligence

Thank You

Software Testing Conferences


http://www.extentconf.com/

technology risk modeling and analysis seminar - exchange trading

Education