learning from the insurance industry - statistrade: know ... · learning from the insurance...

www.statistrade.comStatisTrade LLC proprietary and confidential. Distribution without prior written consent is strictly prohibited.

Learning from the Insurance Industry

Using Stochastic Modeling to Improve Trading System Development

April 2016

Dave Walton Partner, StatisTrade


11

About Me and StatisTrade

Evaluate system components (entry, exits, filters, etc.) to identify performance contribution

Safeguard from Data Mining Bias (DMB) / over-fitting by applying statistical robustness checks

Independently code strategy to identify potential issues

Benchmark analysis and development

Evaluate performance using bias-free (survivorship, dividends, etc.) data

Subject strategy to a variety of stressors and scenarios

Provide easy-to-understand Report Card of significant findings

Identify relevant portfolio-level correlation and diversification impact

WHAT WE DO

Dave Walton

Partner, StatisTrade

2014 NAAIM Wagner Award Winner

Provide marketing and sales support for your high-value prospects

Compare strategies to active and passive benchmarks & similar, public-domain systems

Degrees in computer engineering, computer science

MBA in finance

Completed Dr. Van Tharp’s Super Trader program

Portfolio manager for actuarial-based hedge fund


Today’s Focus

2

In about 45 minutes…

Understand the problem and why it is hard to address

Discuss “traditional” solutions

Introduce stochastic modeling

Insurance company applications

Trading applications

Trading examples

Key assumptions

Working knowledge of trading system development processes

Basic understanding of probability and statistics


3

The ProblemHow Can I Realistically Estimate Future Returns?

Goal:

Avoid creating unrealistic performance expectations only to underperform and get fired!

Development Live Trading


4

... focus only on a single possible historical path (in this case the best)...

... when there are many equally likely outcomes.

ImportantSingular data points are insufficient to estimate financial market outcomes. We need to think in terms of probabilities and statistical distributions.

Why does it Happen?How Can I Realistically Estimate Future Returns?

There are many reasons, but a big one is the tendency to…


5

“Use the average of all test

results...”

Reasonable Performance ExpectationsThink in Terms of a Class of Related Systems

“Specifying reasonable parameter ranges is important when evaluating the test results. It is better to assume that the price patterns change; you cannot tell which combination of parameters will be the best. Regardless of the past returns for the parameters you choose, your expectations should be the average performance of all tests.”

-- Both quotes from Perry Kaufman, Trading Systems and Methods


Traditional Solutions

6

Method Pros Cons

In-Sample, Out-of-Sample

Quick and easyUnbiased estimate

Single outcomeLarge sampling errorArbitrary choice of data split determines outcomeAlmost impossible not to “cheat”

Scenario Testing

Multiple outcomes Limited by historical context and human biasesTime consuming and tedious

Monte Carlo Techniques

Relatively quick and easyNearly infinite outcomes; provides a distribution

Uses fixed set of predetermined trades; over-fit input -> over-fit outputAssumes IID trades/returnsIgnores portfolio effects


Traditional Solutions + Stochastic Modeling

7

Stochastic Modelling

Nearly infinite outcomes; provides a distributionTrades are independently determined for each simulationProperly models portfolio effects

Can be challenging to implementRequires experience to interpretQuality of results depend on how well stochastic inputs model reality

Method Pros Cons

In-Sample, Out-of-Sample



Scenario Testing

Multiple outcomes Limited by historical context and human biasesTime consuming and tedious

Monte Carlo Techniques


Uses fixed set of predetermined trades; over-fit input -> over-fit outputAssumes IID trades/returnsIgnores portfolio effects


8

Stochastic ModelingWhat Is It?

Definition

"Stochastic" means being or having a random variable.

A stochastic model is a tool for estimating probability distributions of potential outcomes by allowing for random variation in one or more inputs over time.

Distributions of potential outcomes are derived from a large number of simulations(stochastic projections) which reflect the random variation in the input(s).

Types of questions we can answer through stochastic modeling

What is a reasonable range of future performance?

At what point does real-life performance indicate likely system/model failure?

Stochastic Inputs1

Input 1

Input 2

Input n

Processing

Simulation Engine

2Outputs

3


9

Stochastic Modeling: Insurance Company

You are CEO of the fictional AIT Insurance Company, Inc.

Assets need to exceed liabilities in the future (i.e., company remains solvent).

Future assets and liabilities are unknown; both depend on a variety of factors. You must estimate both.

Estimation Approaches

DeterministicProject assets and liabilities by extrapolation (e.g., corporate budgeting process). Outcome is a single result (point estimate).

StochasticProject assets and liabilities by varying key parameters like interest rates, inflation. Outcome is distribution of potential results.

Solvency

Input

Output


10

Stochastic Modeling: Insurance Company

Typically, stochastic inputs are statistical distributions approximating empirically observed features.

Example:

Stochastic Inputs1

Interest Rates

Equity Returns

Mortality Rates

Default Rates

Processing

Simulation Engine

2Outputs

Distribution of PotentialSolvency Outcomes

3


11

Stochastic Modeling: Trading System Development

You are CEO of the fictional Showcase Capital, LLC hedge fund.

Profits need to exceed a performance minimum while staying within a certain drawdown limit (otherwise clients will withdraw their capital).

Future profits and losses are unknown; both depend variety of factors. You must estimate both.

Estimation Approaches

DeterministicProject profits and losses by extrapolation of backtest results or track record. Outcome is a single result (point estimate) for both key figures.

StochasticProject profits and losses by varying key parameters like trading rule parameters and price action. Outcome is a distribution of potential results.

Solvency


12


Unlike the insurance company example, general statistical approximations are notsufficient for some inputs. Specifically, price action can be critically important as trading logic targets specific patterns in historical data.

Stochastic Inputs1

Price Action

Corporate Actions

Broker / Exchange Actions

Investor Behavior

Processing

Simulation Engine

2Outputs

Distribution of PotentialEquity Outcomes

3

Trading Logic

Other


Stochastic Inputs

Trading Logic

Rule 1

Rule 2

Rule n

13


1

Price Action

Corporate Actions

Broker / Exchange Actions

Investor Behavior

Processing

Simulation Engine

2Outputs

Distribution of PotentialEquity Outcomes

3

Other

Today‘s Focus


Price Action Variation

Method Applications Caveats

Add Noise to Price ActionOverlay original historical price data with randomly generated noise (e.g. Gaussian)

Price action based Trading system evaluation

Must select “reasonable” noise levelLimited by historical data available

Simulate Price ActionGenerate new price series that aims to mimic the information content (e.g., GARCH) of the original data.

Asset allocation evaluationLonger-term trading system evaluation

Difficult to mimic information content with statistical processes (e.g. serial co-variances)

Return Series ResamplingUses historical return series to create “new” versions (e.g. block bootstrap)

Asset allocation evaluationLonger-term trading system evaluation

Limited by historical data availableMust choose appropriate block length

14

Stochastic Modeling: Trading System DevelopmentInput Variation: Price Action

1

Price Action

Stochastic Inputs


15

Stochastic Modeling: Trading System DevelopmentInput Variation: Rule Parameters

1

Rule Parameter Variation

Method Applications Caveats

Parameter RandomizationFor each simulation, all rule parameters are randomly selected from specified ranges.

Parameter-based trading system evaluation

Must select appropriate parameter ranges

Trading Logic

Rule 1

Rule 2

Rule n

Vary allparameters

Stochastic Inputs

“[Focusing on historically optimal parameters] assumes that the market will continue to perform in a way that allows those parameters to generate profits during the next year.”

--Perry Kaufman, Trading Systems and Methods


16

Stochastic Modeling: Trading System DevelopmentWhy Does Using Randomness Help Solve the Problem?

Pure signal signal & noise

<- cycle -> <- cycle ->

Question:

What is the value of adding randomness to our input data (e.g., prices, rule parameters)?

Answer:

It recognizes that they are already influenced by randomness in their original, “pure” state.


17

Stochastic Modeling: Trading System DevelopmentWhy Does Using Randomness Help Solve the Problem?

Pure Signal Signal & Noise

Market data contains noise

System rules must be able to deal with noise=> deliberate injection of randomness into data

Specific settings of system rules were influenced by noise in training data=> deliberate variation of rule parameters

<- cycle -> <- cycle ->


Performance Estimation

Method Pros Cons

Stochastic ModelingGenerate performance distribution through stochastic simulation of trading inputs (e.g. price action).

Nearly infinite outcomes; provides a distributionTrades are independently determined for each simulationProperly models portfolio effects

Quality of results depend on how well stochastic inputs model realityCan be challenging to implementRequires experience to interpret

Out of Sample TestingOptimize (train) trading model on in-sample data and validate on out-of-sample data



18

Stochastic Modeling: Trading System DevelopmentApplication: System Performance Estimation

Versus


19


As CEO of the fictional Showcase Capital, LLC hedge fund, you are evaluating an RSI2 mean reversion trading system

You want an estimate (unbiased) of long-run performance

You use tried and true IS/OOS validation

The rule of thumb is OOS performance should be > 50% of IS performance

This result a fail! But should you drop the system?


20


Embedded in OOS validation is an assumption – IS performance is the best predictor of OOS

Virtually non-existent relationship between IS and OOS performance

Reversion to the mean sucks!

Optimized OOS performance within range of results from stochastic model

Do you specify a single (random?) estimate or a range?

OOS 3.4% = 8th percentile


Position Size Optimization

Method Pros Cons

Stochastic OptimizationFor selected position sizing settings, generate performance distribution through stochastic simulation of trading logic/price action.

Dynamic trade generationAllows comparison of distributions (e.g., median, 90% range) for position sizing settings

Quality of results depend on how well stochastic inputs model reality

Monte Carlo Trade ReshufflingFrom a set of trade results, generate performance distribution through reshuffling trades


Uses fixed set of predetermined trades; over-fit input -> over-fit outputAssumes IID tradesIgnores portfolio effects

21

Stochastic Modeling: Trading System DevelopmentApplication: Position Size Optimization

Versus


22

Stochastic Modeling: Trading System DevelopmentApplication: Position Size Optimization

Stochastic Optimization (Parameters)Monte Carlo Trade Reshuffling

Using your Showcase Capital RSI2 mean reversion system your objectives are: >10% Annual Returns with <-30% Maximum Drawdown

You need to determine your optimal position size to achieve these objectives

Which method would you rely on?


23

Stochastic Modeling: Trading System DevelopmentApplication: Determine Value of Individual Rule

Determine Value of Individual Rule

Method Process Caveats

Single Rule ReplacementReplaces signal generation for a rule with a random sample drawn from a distribution created by the original rule (e.g., trade durations).

Multi-pass process.1. Original stochastic model run2. Single rule replaced with “recorded”

characteristics and 2nd stochastic model run

3. Original and 2nd run outputs compared

For some rules finding a method of rule replacement by a random equivalent can be challenging.

Trading Logic

Rule 1

Rule 2

Rule n

1

Trading Logic

Rule 1

Random Equivalent

Rule n

swap with random equivalent

Stochastic Inputs2

Stochastic Inputs COMPARE


24

Stochastic Modeling: Trading System DevelopmentApplication: Determine Value of Individual Rule

Individual Rule Analysis Performance Contribution

Your Showcase Capital RSI2 mean reversion system is OK, but you wonder if all rules add value.

How do you determine the value of each rule to overall performance?

Looks like you‘ve got some work to do!


Stochastic Modeling: Trading System DevelopmentOther Applications

25

Stochastic Inputs Modeling Applications

Market Impacts Slippage effectsStrategy capacity constraints

Corporate Actions Dividend payout changes

Investor Behavior Redemption acceleration during drawdowns

Broker/Exchange Actions Margin requirement changesPartial fillsShorting limitations

Detecting System Failure Compare live trading results to “low probability” occurrencesMaximum drawdown violationMinimum return over some short run period

Bayesian Approach Simulations are re-run periodically to incorporate new dataInitial stochastic run = prior distributionSubsequent runs = posterior distributions

Oh and by the way...


Summary / Key Points

26

Singular data points are insufficient to estimate financial market outcomes

We need to think in terms of probabilities and statistical distributions

Traditional solutions to trading system performance estimation have many limitations

Stochastic modelling is another potential solution with many trading system applications

Thank you!


27

Questions

You can also approach us later, but please don’t be shy! We love talking about this stuff!

learning from the insurance industry - statistrade: know ... · learning from the insurance...

Documents