a/b testing with react · o introduction o our project • about the project • the stack •...
TRANSCRIPT
A/B Testing with React
Eivind Arvesen and Lars Dølvik
o Introductiono Our project
• About the project• The stack• Hypothesis- and goal-driven development/Workflow
o Story-timeo What is A/B testing?
• react-ab-test (library)▪ Isomorphism
o Evaluation• Measurements (GA: scrolling and events…)• Interpreting the numbers
o Practical demonstration
Outline
o A recipe focused application
o Runs on Google Cloud Platformo Isomorphico Built with React and Relay in fronto GraphQL as data query languageo Uses ElasticSearch to hold our contento Enonic CMS to publish content
o Hypothesis- and goal-driven development/Workflow
Our project
o Tactical goalo Action What?o Hypothesis Why?o Measurement How?o Estimated effect
Workflow
Story-time
Story-time
A/B-TESTING!
A form of statistical hypothesis testing.
What is A/B testing?
Illustration by Maxime Lorant
Usual steps:
● Have an idea or supposition about our solution● Verify or falsify the hypothesis via experiment(s)
○ Collect various measurements
What is A/B testing?
Why?
It’s normal to be wrong.
Measuring how our development impacts our users makes us able to make decisions based on facts!
What is A/B-testing?
React-AB-Test (on npm)
https://github.com/pushtell/react-ab-test
What is A/B-testing?
Legg gjerne inn et portrett av personen du siterer på dette lysbildet!
”The only true wisdom is in knowing you know nothing.Socrates - allegedly...
● Be skeptical● Don’t measure too much● Have a good basis for comparison
o Google Analytics APIo Scroll and time trackingo Event tracking
o Tracking the correct evento Follow a standardo Write documentationo Less is more(ish)o Evaluating the numbers
Evaluation
o A good grounds for comparisono Avoid noise in your datao Supplement qualitatively...
"Lies, damned lies, and statistics"
Evaluation
Evaluation
Let’s say one of the media elements is an autoplaying video. Clicks lead to another recipe.
What is the user’s real intent?
Evaluation
Evaluation
Let’s see how we can perform A/B-testing in practice: look at code and run it in the browser.
https://github.com/eivind88/ab-testing-react-meetup/tree/master
Demo
o It is not as complicated as it might seem• The thought process is more the half the job
o Keep it simple and structured• Follow a standard and write documentation
o Keep your experiments limitedo You don’t know it if you haven’t proven it
In summary