High Tempo Testing-a system for growth
Michał, welcome to the team!
Michał Fiech
ooooooołłłłł, stop you!
Last time:◉ Pirate Metrics are Acquisition, Activation,
Retention, Revenue, Referral (AARRR!!!!)◉ Each customer segment/channel is different and
should be treated accordingly - aka should have its own, different AARRR factors
◉ You should find 3 channels:Best performing (%)Largest volume (#)Lowest cost ($)
◉ There is qualitative and quantitative data analysis
Next meetings will focus on:◉ Pirate Metrics Workshop◉ Tests and Experiments◉ B2B◉ B2C
How long will it take today?
What will we cover today?
How to be smart in managing growth?
Testing - this is the key to success!
You have to test new solutions to improve your funnel (AARRR) quickly and cheaply and scale what works.
Test to discover vs. test to optimize
In order to grow - you must know in which direction, have the proper framework and implement tests in an agile way.
Build
TestAnalyse
Ok, what are the three things crucial for growth?
Process TeamSyste
m
Tell me more about the process!
First - set up goals to achieve, metrics, strategy (for M,Q or Y)
◉Which funnel level are we working on? (like activation)
◉What are we strong at? (i.e. SEO, SEM)
◉What budget do we have?◉What overall company goals to
achieve? (i.e. revenue increase 50% YoY)
◉What do we measure? What’s growth for us?
Second - build a repeatable and scalable machine, that looks like this
Ideation
Prioritisation
Experiments
Analysis
Build a backlog of ideas
Tomasz Duda
zostaje czy out?
Experiment Doc:1. What is the test?
2. What was the hypothesis?3. What was the experiment
configuration?4. What were the texts, graphics
5. Did it succeed or not?6. Why did it (not) work?7. What can we improve?8. What have we learned
Where to find ideas?
◉Watch the competition◉Read, read, read and again - read a lot
(growthhackers.com , medium.com, Quora etc.)
◉Brainstorm◉Organize workshops◉Use external advisors◉Ask the entire company to participate
Who here wants to grow their business?
Each idea has to have a hypothesis and KPIs
So you know if it failed or succeeded - based on data
Scoring - we suggest the ICE score
ImpactConfidenceEase
Impact (potential impact) Confidence (how certain are you
that it will work)Ease (how easy is to introduce it)
On a scale of 1-10+ working hours, if needed
+ anything that suits an organisation
They all have to focus on how will they move the One-Metric-That-Matters (OMTM)
+ Each idea should be linked to particular level
of a funnel (AARRR)
How to prioritize ideas?
How to test them?
Each test must have a hypothesis, which we are
verifying
For example: We have X traffic from Facebook a week (no ads) and we want to
improve it. I think we should increase the number of posts x2 daily and the traffic will increase
by 50%.
Weekly tempo goal (ie. 3 tests a week) – depending
on the team
Crucial– total focus on the sprint tasks
How to analyze results and draw conclusions?
Did it work, did it not work, we don’t know?
Ok, we know about the process, now let’s
systemize the tests!
Tell me more about the System!
Each test has to be analyzed and described,
keep learning!
Weekly growth meeting – Settling on the priorities and growth coordination
Growth Meeting agenda:● 15 min: KPI review and update focus
arena● 10 min: Review last week’s testing
sprint● 15 min: Key lessons learned from
analyzed tests● 15 min: Select tests for this week’s
sprint● 5 min: Check growth of ideas backlog
Ideation
Prioritisation
Experiments
Analysis
Ideation
Prioritisation
Experiments
Analysis
Trello (Growth Template)
Ideation
Prioritisation
Experiments
Analysis
Ok, we know the process and the system, but what about the growth team?
Let’s go back to definitionof Growth Hacking
SEO PPC Email CRO Product
SEO guyDevContentAnalytics
Ads guyDevGraphicAnalytics
Email guyDevContentAnalytics
DevAnalyticsUX
DevUXAnalytics
‘Growth Master’Someone who’s making sure, you’re
focusing only on growth
DeveloperFront-end / Full Stack /
Full Stack Overflow
‘Growth Master’Someone who’s making sure, you’re
focusing only on growth
DeveloperFront-end / Full Stack /
Full Stack Overflow (joke)
‘Growth Master’ Developer
Analyst
‘Growth Master’ Developer
Analyst Graphic designer
‘Growth Master’ Developer
Analyst Graphic designer SEO Specialist
‘Growth Master’ Developer
Analyst Graphic designer SEO Specialist Content Writer
‘Growth Master’ Developer
Analyst
UX/UI designer
Graphic designer SEO Specialist Content Writer
‘Growth Master’ Developer
Analyst
UX/UI designer Email Marketer
Graphic designer SEO Specialist Content Writer
‘Growth Master’ Developer
Analyst
UX/UI designer Email Marketer
Graphic designer SEO Specialist
Community Expert
Content Writer
‘Growth Master’ Developer
Analyst
UX/UI designer Email Marketer
Graphic designer SEO Specialist
Community Expert PPC Specialist
Content Writer
Growth Team might therefore consist of 1 - 2 - 3 - 4 - ∞*
Growth Team might therefore consist of 1 - 2 - 3 - 4 - ∞*
* but remember you’ll need to manage them, so infinity is not a good idea. Up to a few people full time (4 - 6) should be enough in most cases
No matter the size, some things must be jointly shared
Is your funnel leaking?
3012
110
60
48
39
3012
110
60
48
39 1.3%
3.6%
2%
1.6%
5412
641
358
12
120
5412
641
358
12
120
11.8%
6.6%
0.22%
2.2%
7612
4301
3127
1275
134
7612
4301
3127
1275
134
56.5%
41%
16.7%
1.7%
7612
4301 56,5%
3127 72,7%
1275 40,77%
134 10,5%
5871
763
143
83
53
5871
763
143
83
53
13%
2.4%
1.4%
0.9%
5871
763 13%
143 19%
83 58%
53 64%
LogMeIn Case Study
3 step goal:1.Register your details2.See download page3.Download the
software
Problem90% drop-off at the download step
The team ran dozens of tests focusing on improving the button’s appeal. Nothing seemed to reverse the trend.
So they decided to do some qualitative research to come up with new hypotheses. It turns out they didn’t have to ask many people.
There was a clear theme that was killing their Activation numbers.
People didn’t believe the product was free. They thought it was too good to be true.
Based on their new insight, they added a second button. This button was to download a free trial of the paid version of the software.
They hadn’t offered the paid version during the signup process before (probably because you’d assume that would hurt conversion).
300% improvement!
How to do it in your business?
1. Goal setting is critical to sustainable growth
2. Focus on high leverage goals
3. Communicate goals widely
4. Follow growth hacking process to achieve goals
What can you do?
Figure out your:Best performing (%)Largest volume (#)Lowest cost ($)
Maximize your return from these 3 channels.
Find out where are the leaks in these funnels
Or talking to find out where is the obstacle?
Spare less attention to the other channels
Most important sentence of the day!
You won’t know until you test it!
To sum it up:◉ Growth is built on proper
Process, People and Systemization
◉ The process should consist of Ideation, Prioritization, Experimentation and Analysis
◉ Focus on the areas where you can have the biggest impact
Next timePirate Metrics Workshops
Thanks for coming!Let’s have a beer together now!
3 steps to success
Any questions ?You can find us at◉ [email protected]◉ [email protected]◉ [email protected]
Thanks!