algorithmic trading: statistical significance

IntroductionES 100-day momentum example

Statistical SignificanceClosing Thoughts

Algorithmic Trading: Statistical Significance

Antony Jackson, CFA

University of East Anglia

London Quant Club, July 2016

Antony Jackson, CFA London Quant Club 1/ 19



Overview

Economic Significance versus Statistical Significance.




Overview


What is a trading strategy? Binary Signals.




Overview



100-day ES momentum example.




Overview



100-day ES momentum example.

Statistical Significance.




Economic Significance versus Statistical Significance

Economic Significance can be thought of as the degree to which a tradingstrategy satisfies the goals of an investor.






What impact does the strategy have on the investor’s portfolio? Do its returnshave a low correlation (diversification) and high stand-alone Sharpe Ratio(risk-adjusted returns)?







Economic Significance rule-of-thumb:








Accept strategy into existing portfolio if








Accept strategy into existing portfolio if

Stand-alone SR > Existing portfolio SR× correlation





Statistical significance is concerned with how likely the results of our tradingstrategy could have been generated by pure chance.






If you have enough monkeys and typewriters, how likely is it that one of themwrites the complete works of William Shakespeare?







It’s possible to have statistical significance without economic significance. Forexample, you design a strategy with a Sharpe Ratio of 0.05, and you are 99%sure this is significantly larger than zero.







It’s possible to have statistical significance without economic significance. Forexample, you design a strategy with a Sharpe Ratio of 0.05, and you are 99%sure this is significantly larger than zero.

Equally, we can have economic significance without statistical significance. Forexample, a strategy with a Sharpe Ratio of 0.40, but where the buy and holdSharpe Ratio is 0.50.




What is a Trading Strategy? Binary Signals.

I recommend starting algorithmic trading by coding up strategies with simplesignals; e.g., 1 for buy and 0 for out of the market.






Traditional technical trading systems can be coded up like this. For example,simple moving averages, channel breakouts, filter rules.






Traditional technical trading systems can be coded up like this. For example,simple moving averages, channel breakouts, filter rules.

Simplifying strategies to a series of ones and zeros helps a lot in establishingthe likelihood of a trading strategy being statistically significant, because purechance strategies can be easily constructed using coin tossing to generatesignals.




100-day ES momentum example

There’s a lot of evidence in the academic literature of momentum in assetprices.






Cross-sectional momentum involves buying a portfolio of recent winners andfunding the purchase by selling a portfolio of recent losers.






Cross-sectional momentum involves buying a portfolio of recent winners andfunding the purchase by selling a portfolio of recent losers.

Time-series momentum looks at an asset in isolation. We buy if the asset hasgone up, and sell if it has gone down. This is the simple (but effective!)method I’ll consider today.





Preliminaries:

The Matlab scripts for this presentation make use of the Quantiacs Toolbox.





Preliminaries:


You will need to open Matlab and place the files from the presentation in thesame directory as the Toolbox files such as runts and optimize.





Preliminaries:


You will need to open Matlab and place the files from the presentation in thesame directory as the Toolbox files such as runts and optimize.

The files ESbootstrap.m and ESmomentum.m are Quantiacs trading systemfiles, and the file ESbootstrapCaller.m is a script file used to process theresults of random trading strategies.





The ESbootstrap.m file contains the following code:





The ESbootstrap.m file contains the following code:

function [p, settings] = ESbootstrap(CLOSE, settings)

settings.markets = {’F ES’};

settings.lookback = 260;

setting.slippage = 0;

trial = 1; %#[1:1:1000]#

p = 1; % uncomment this to see the performance of buy and hold

p = round(rand()); % uncomment this to generate random strategies

end





An important line is






settings.slippage = 0;







At this stage we are trying to identify a property of the time series of pricesthat offers an opportunity, either in terms of identifying a market anomaly or interms of harvesting a risk premium.







At this stage we are trying to identify a property of the time series of pricesthat offers an opportunity, either in terms of identifying a market anomaly or interms of harvesting a risk premium.

We don’t want to distort the results at this stage with transaction costs.





If we uncomment p = 1;, we simulate a buy-and-hold strategy for ES.





If we uncomment p = 1;, we simulate a buy-and-hold strategy for ES.

Enter the command runts(’ESbootstrap’) within the Quantiacs Toolbox, andwe obtain the following chart for the buy and hold strategy:





1000000

1221663

1492461

1823285

2227440

2721182

3324368

4061258

4961490

996368

4943467

Per

form

ance

(lo

g)

Factsheet of fundEquity in ESbootstrap

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

0

0.2

0.4

0.6

0.8

1

Long

/Sho

rt

Max DrawdownMax Time Off PeakLongShort

Your Income Estimation($ 1Mio Investment, 10%

performance fee)total $

per anno $324408.4512747.24

Performance Numbers

0.34740.5555

0.06400.1843

0.6292

0.1018

3352

Sharpe RatioSortino Ratio

Performance (prc/y)Volatility (prc/y)

Maximum Drawdown

MAR Ratio

Max Time off peak





The buy and hold strategy has a Sharpe Ratio of 0.3474.






Let’s now have a look at the code for a 100-day momentum strategy, which iscontained in the file ESmomentum.m






Let’s now have a look at the code for a 100-day momentum strategy, which iscontained in the file ESmomentum.m

function [p, settings] = ESmomentum(CLOSE, settings)

settings.markets = {’F ES’};

setting.lookback = 260;

setting.slippage = 0;

p = CLOSE(end) > CLOSE(end - 100);

end





1000000

1158838

1342905

1556209

1803394

2089841

2421787

2806458

3252230

999574

3250845

Per

form

ance

(lo

g)

Factsheet of fundEquity in ESmomentum

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

0

0.2

0.4

0.6

0.8

1

Long

/Sho

rt

Max DrawdownMax Time Off PeakLongShort

Your Income Estimation($ 1Mio Investment, 10%

performance fee)total $

per anno $196232.847710.73

Performance Numbers

0.39200.5974

0.04390.1119

0.2584

0.1697

2428

Sharpe RatioSortino Ratio

Performance (prc/y)Volatility (prc/y)

Maximum Drawdown

MAR Ratio

Max Time off peak





Not bad. The Sharpe Ratio has been boosted from 0.3474 to 0.3920.






So at this stage, we could argue that we have discovered an economicallysignificant trading strategy, because we have boosted the Sharpe Ratio andwe’ve come up with a way of staying out of the worst of the financial crisis,without cheating.






So at this stage, we could argue that we have discovered an economicallysignificant trading strategy, because we have boosted the Sharpe Ratio andwe’ve come up with a way of staying out of the worst of the financial crisis,without cheating.

The next step is to quantify the probability that a “pure chance” strategy couldgenerate a similar (or better) Sharpe ratio.




Statistical Significance

In the ESbootstrap.m file, comment out the p = 1; line, and uncomment thep = round(rand()); line.






The mysterious %#[1:1:1000]# line is there so that the optimize function inthe Quantiacs Toolbox runs a random trading system 1000 times, collecting theSharpe Ratio each time.






The mysterious %#[1:1:1000]# line is there so that the optimize function inthe Quantiacs Toolbox runs a random trading system 1000 times, collecting theSharpe Ratio each time.

Essentially, we are going to generate 1000 strategies by tossing a coin every dayto decide whether we are long or out of the market.





The ESbootstrapCaller.m script contains the following code:






% Script to call the ESbootstrap function

out = optimize(’ESbootstrap’);

hist(out.sharpe)

title(’Histogram of Trading Strategy Sharpe Ratios’)






% Script to call the ESbootstrap function

out = optimize(’ESbootstrap’);

hist(out.sharpe)

title(’Histogram of Trading Strategy Sharpe Ratios’)

Enter run ESbootstrapCaller at the Matlab command line, and we obtain ahistogram something like this:





−0.2 0 0.2 0.4 0.6 0.80

50

100

150

200

250

300Histogram of Trading Strategy Sharpe Ratios





Finally we add the following line to the ESbootstrapCaller script to work outthe proportion of random strategies that obtain a Sharpe Ratio greater thanour “real-life” strategy Sharpe Ratio of 0.3920:






pValue = sum(out.sharpe(:) > 0.3920)/1000;







For this particular sequence of coin tosses, we obtain a p-value of 0.1530.







For this particular sequence of coin tosses, we obtain a p-value of 0.1530.

We prefer low p-values, as it’s then less likely that our results could begenerated by pure chance.




Closing Thoughts

The methods I’ve introduced today can be extended to testing multiple ruleson the same time series.




Closing Thoughts


For instance, if we decide to test both a 100-day and a 250-day momentumrule, we need to take into account that we are now testing multiple strategies,and so the bootstrap methodology needs to be adjusted accordingly.




Closing Thoughts



In this case, for each trial, we would run two random strategies and pick thehighest Sharpe Ratio each time.




Closing Thoughts



In this case, for each trial, we would run two random strategies and pick thehighest Sharpe Ratio each time.

We balance the increased likelihood of obtaining a good Sharpe Ratio bytesting multiple models against the rightward shift in the distribution ofrandom Sharpe Ratios that results from picking the best of multiple randomstrategies each time.


algorithmic trading: statistical significance

Economy & Finance