wood for trees 1wood for trees 1 testing jon kelly, director of analysis @ wood for trees march 2015...
TRANSCRIPT
wood for trees 1wood for treeswww.woodfortrees.net
1
Testing
Jon Kelly, Director of Analysis @ Wood for Trees
March 2015
Outstanding Marketing Services Provider 2011 (Database Marketing awards); Supplier of the Year 2012 (IoF SIG Awards); Innovation in Analytics for Marketing 2013 (Database Marketing Awards)
wood for trees 2wood for treeswww.woodfortrees.net
2
Some thoughts on testing
Determine the reason for the test – what are you expecting to find out
Beware of averages
Beware of balancing effects
Test as wide as possible
Explore behaviour changes within sub populations of the test audience
Action the test
Re-test
Understand the longer term effects
wood for trees 3wood for treeswww.woodfortrees.net
3
Prompt Testing – Top Level Results
Top level views and analysis previously done shows
that the testing had little effect.
However looking at certain segments in more detail
tells a different story..
Audience Prompts Response Rate Average Gift Income per Thousand
Control £5, £18 ,£25, £50, £150 10.30% £11.48 £1,183Test £10 10.43% £11.46 £1,195
wood for trees 4wood for treeswww.woodfortrees.net
4
Response Pattern
Overall response pattern in line with other campaigns
However, different response picture where testing took place
wood for trees 5wood for treeswww.woodfortrees.net
5
May 2012 By Last Amount
This behaviour is the same across all giving levels, so
£5 donors give £10 – GOOD
£25 donors give £10 – BAD
As long as responder volumes remain consistent……
Previous Gift =
£5
Previous Gift =
£25
wood for trees 6wood for treeswww.woodfortrees.net
6
May 2012 by Last Amount
Low value responders were prompted to give more but less
responded
High value responders were prompted to respond more but to give
less
Previous Gift = £5Prompts Response Rate Average Gift Income per
Thousand£5, £18 ,£25, £50, £150 13.04% £6.59 £1,210
£10 12.26% £7.34 £1,093
Previous Gift = £25Prompts Response Rate Average Gift Income per
Thousand£5, £18 ,£25, £50, £150 10.60% £23.26 £2,475
£10 11.34% £19.59 £2,222
wood for trees 7wood for treeswww.woodfortrees.net
7
TEST - £5 and under
Test 1: £5, £8, £10, £15, Other
Test 2: £8, £10, £15, £20, Other
Test 3: £10, £15, £20, £25, Other
wood for trees 8wood for treeswww.woodfortrees.net
8
Cold Mailing - £5 and under
HR £0 - £5.01 Mailed Responded Response Rate Index Sig Average Gift Index Sig Income per 000 Mailed IndexControl 2,500 205 8.2% £5.76 £472.20Test Total 7,500 548 7.3% 89 84.7% £6.89 120 100.0% £503.67 107Test 1: £5, £8, £10, £15, Other 2,500 203 8.1% 99 8.2% £6.47 112 95.6% £525.60 111Test 2: £8, £10, £15, £20, Other 2,500 161 6.4% 79 98.3% £6.92 120 100.0% £445.80 94Test 3: £10, £15, £20, £25, Other 2,500 184 7.4% 90 73.3% £7.33 127 100.0% £539.60 114TOTAL 10,000 753 7.5% £6.58 £495.80
All tests show a fall in response rate which is perhaps unexpected, although only the £8 test is significant.Average gift is up around £1 on all tests as supporters are encouraged to give at higher amounts.Around £30 extra per 000 across all tests.So tests are likely to increase income but there might be a small hit in responses.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
2014 - April Warm Cash Mailing - HR £0 - £5.01 Prompt Control
2014 - April Warm Cash Mailing - HR £0 - £5.01 Prompt Test 1
2014 - April Warm Cash Mailing - HR £0 - £5.01 Prompt Test 2
2014 - April Warm Cash Mailing - HR £0 - £5.01 Prompt Test 3
b:<£5
c:£5
d:£5-£10
e:£10
f:£10-£15
g:£15
h:£15+
wood for trees 9wood for treeswww.woodfortrees.net
9
Max Warm Donation in last 5 years
<=£10
Responded to a previous campaign
Max Response
Amount Ever > £12
Max Response
Amount Ever £6 - £12
Max Response
Amount Ever < £6
Not responded to
a previous campaign
Mailed once or less
(Warm) in last year
First payment in
last 15 months
First Payment
before last 15 months
Mailed more than once in
last 12 months
First relationship within last 2
years
Given via self sponsorship
Not given via self
sponsorship
First relationship more than 2 years ago
GROUP A: £12 ASK – Max Warm Donation in last 5 years <=£10
Max Warm Donation in last 5 years
<=£10
Responded to a previous campaign
Max Response
Amount Ever > £12
Max Response
Amount Ever £6 - £12
Max Response
Amount Ever < £6
Not responded to
a previous campaign
Mailed once or less
(Warm) in last year
First payment in
last 15 months
First Payment
before last 15 months
Mailed more than once in
last 12 months
First relationship within last 2
years
Given via self sponsorship
Not given via self
sponsorship
First relationship more than 2 years ago
wood for trees 10wood for treeswww.woodfortrees.net
10
Max Warm Donation in last 5 years
<=£10
Responded to a
previous campaign
Max Response Amount
Ever > £12
Max Response Amount
Ever £6 - £12
Max Response Amount
Ever < £6
Not responded
to a previous campaign
Group A1
wood for trees 11wood for treeswww.woodfortrees.net
11
Ask: £5 £12 £15
No. mailed: 1,431 4,310 1,418
Response rate:
8.4% 7.3% 7.8%
Average gift:
£12.20 £15.16 £20.05
ROI: 2.56 2.76 3.89
Group A1
Group A1: Max donation in last 5 years less than £10 –
but previously responded at more than £12
Winner = £15
Ask: £5 £12 £15
No. mailed: 1,431 4,310 1,418
Response rate:
8.4% 7.3% 7.8%
Average gift:
£12.20 £15.16 £20.05
ROI: 2.56 2.76 3.89
wood for trees 12wood for treeswww.woodfortrees.net
12
Max Warm Donation in last 5 years <=£10
Responded to a previous campaign
Max Response
Amount Ever > £12
Max Response
Amount Ever £6 - £12
Max Response Amount
Ever < £6
Not responded to a previous campaign
Group A3
wood for trees 13wood for treeswww.woodfortrees.net
13
Ask: £5 £12 £15
No. mailed: 6,016 17,814 5,966
Response rate:
16.3% 13.1% 12.2%
Average gift:
£4.43 £4.57 £4.48
ROI: 1.80 1.49 1.36
Group A3
Group A3: Max donation ever is less than £6
Winner = £5
Ask: £5 £12 £15
No. mailed: 6,016 17,814 5,966
Response rate:
16.3% 13.1% 12.2%
Average gift:
£4.43 £4.57 £4.48
ROI: 1.80 1.49 1.36
wood for trees 14wood for treeswww.woodfortrees.net
14
Most effective ask levels - summary
Less than before Same as before More than before
How the asks changed
75%
15%
10%