Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
1
Технологические риски в биржевой торговле
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 2460, +1 415 830 38 49www.exactpro.com
14 Октября 2016Иосиф Иткин, Exactpro CEO, London Stock Exchange Group
Научно-практический семинарМоделирование и анализ технологических рисков
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
2
Technology Risk Typology
Financial Services
Money and Reputation
Minimal Health & Safety
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
3
It argues that if machine brains surpass human brains in general intelligence, then this new superintelligence could replace humans as the dominant lifeform on Earth.
Sufficiently intelligent machines could improve their own capabilities faster than human computer scientists. As the fate of gorillas now depends more on humans than on the actions of gorillas themselves, so will the fate of future humanity depend on the actions of the machine superintelligence. The outcome could be an existential catastrophe for humans.
Solving the control problem is surprisingly difficult because most goals, when translated into machine-implementable code, lead to unforeseen and undesirable consequences.
Advanced Technology
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
4
Existential Risk
Google DeepMind published a paper claiming that it is
possible to implement "a button" to interrupt undesired behaviour
Will we have enough time to press the button once?
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
5
Existential Risk
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
6
Robot Apocalypse Lessons
The only reason the characters were able to make it until the end of the second season is that they were relying on the very same technology that was trying to kill them.
They were protected by systems having the same level of sophistication as those they were struggling with. This is the mentality that we should adapt.
When we use a complex platform, we need to build software to test our software. And you do not want your risk control and test instruments to be inferior to what will hit you.
Having a good robot on your side is the only way to survive the robot apocalypse.
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
7
Build Software to Test Software
Automated testingCompares intended and received results
Automated end 2 end clearing system testing
Automated monitoring, analysis and reporting
75K messages / second from a single CPU coreMeasures latencies in microsecond range
A variety of algoes simulating end-clients will run and see how system performs
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
8
Build Software to Test Software
Cat on a Screen
Bug in the Code
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
9
Build Software to Test Software
Cat on a Screen
Bug in the Code
The Next Generation of Software Testing Tools
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
10
Learning from failure has the status of a cliché, but this book reveals the astonishing story behind the most powerful method of learning known to mankind, and reveals the arsenal of techniques wielded by some of the world's most innovative organizations. It also reveals the dangers of failing to learn from mistakes. In healthcare, hundreds of thousands of patients die from preventable medical errors every year due to a chronic lack of Black Box Thinking.
Using gripping case studies, exclusive interviews and really practical takeaways, Matthew Syed – the award-winning journalist and best-selling author of Bounce - explains how to turn failure into success, and shows us how we can all become better Black Box Thinkers.
Learning from Failure
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
11
What Can Possible Go Wrong
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
12
Fat Finger Errors12 October 2005Attempts to sell a single J-Com stock for 610,000 Yen ($5,041)Mistakenly price and quantity were swappedRisk systems failure:• Mizuho Securities• Tokyo Stock ExchangeEstimated loss $225 millions
This type of errors is called: Fat Finger
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
13
Fat Finger Errors18 September 2012, Stockholm. A trader had the intention of posting a sell order for 5,000 SKF B shares. Due to an input error with the Client, the order volume field was populated with a negative value (-5,000)
Instead of returning an error, the system converted the value into a random 9-digit figure - 294,962,296
The Sell Order corresponded to approximately 71 % of the total outstanding volume in the SKF B share. The Sell Order resulted in execution of 813,442 shares
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
14
Knight Capital Events• 1 August 2012, USA• Knight Capital – one of the most successful HFT firms• Implemented changes related to Retail Liquidity Program
at NYSE• SMARS – an ultra-fast order router• Source code responsible for legacy functionality PowerPeg• 212 parent orders, millions child orders• Accumulated loss – $460m or $170k/sec• Incorrectly configured risk systems• Deployment on 7 servers instead of 8…
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
15
Knight Capital Events
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
16
Knight Capital Events
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
17
Regulated Exchange
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
18
Facebook IPO on NASDAQ• 18 May 2012, NASDAQ, One of the largest IPOs in history• Secondary trading is preceded by a designate Display Only Period
(DOP)• Multi-component architecture that included Matching Engine, IPO
Cross Application and Execution Application• At the end of the DOP, NASDAQ’s “IPO Cross Application” analyzes all
of the buy and sell orders to determine the price at which the largest number of shares will trade; then NASDAQ’s matching engine matches the buy and sell orders at that price. Usually takes 1-2 ms
• NASDAQ allowed orders to be cancelled at any time up until the end of the DOP, including the very brief interval during which the IPO cross price is calculated. After calculation was completed, the system performed orders validation check between ME and “IPO Cross Application”. If any of the orders were cancelled after the start of the cross, the system would have to repeat the calculation
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
19
Failover Proposal
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
20
Facebook IPO on NASDAQ• Over 496k orders participated in the cross, and its duration exceeded
20ms• Order cancellation arrived during this period, and the application had to
repeat the calculation. Two more cancellations arrived during the second iteration, and four more - during the third one
• IPO Cross Application went into infinite loop at 11:05• The NASDAQ team switched off validation check on the secondary
system and performed failover 25 minutes after the start of the loop• Unknown at that moment, 38k orders submitted between 11:11 and
11:30 were stuck and did not participate in the uncross. It created another discrepancy, this time with Execution App and Members who were not able to receive confirmation for orders executed in the cross until 13:50
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
21
Failover Proposal
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
22
Facebook IPO Stock Chart
SEC Release No. 34-69655 / May 29, 2013
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
23
Flash CrashThe May 6, 2010, Flash Crash, was a United States trillion-dollar stock market crash, which started at 2:32 p.m. and lasted for approximately 36 minutes.
Event is attributed to HFT, large directional sell, index arbitrage and market manipulation
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
24
Defects Management
Areas of research in defect management:
• automatic defect fixing
• automatic defect detection
• metrics and predictions of defect reports
• quality of defect reports
• triaging defect reports
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
25
Defects ManagementDefect clustering• Understand weaknesses of software
• Improve testing strategy
Use obtained data to:• Predict the probability of a defect to be re-opened• Highlight the area of the code where the bug is most likely located• Identify requirements and areas most prone to defects in the next release of software
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
26
The seventh EXTENT conference will be held in London, UK .
Find out more: www.extentconf.com
– LSEG Technology and Quality Assurance– Risk controls and FPGA– Market Surveillance systems– Blockchain and trading technology trends– Artificial intelligence
Thank You
Software Testing Conferences