UXPA UK - Toolkits and Tips for Blending UX, Analytics and CRO

Download UXPA UK - Toolkits and Tips for Blending UX, Analytics and CRO

Post on 27-Jan-2015




1 download

Embed Size (px)


This is a talk I gave to the UXPA-UK, focusing on how you can blend UX,agile, UCD, tools and CRO into one nice quant and qual mashup.


  • 1. UX and CRO : Toolkits and Tips UXPA UK - 20th March 2014 @OptimiseOrDie

2. @OptimiseOrDie UX and Analytics (1999) User Centred Design (2001) Agile, Startups, No budget (2003) Funnel optimisation (2004) Multivariate & A/B (2005) Conversion Optimisation (2005) Persuasive Copywriting (2006) Joined Twitter (2007) Lean UX (2008) Holistic Optimisation (2009) Was : Group eBusiness Manager, Belron Now : Consulting 3. @OptimiseOrDie Became obsessed with UX Save the Trees Discovered UX Breaking the bonds Getting the mix right UX Hype Cycle 4. @OptimiseOrDie Timeline Tested stupid ideas, lots Most AB or MVT tests are bullshit Discovered AB testing Triage, Triangulation, Prioritisation, Maths Zen Plumbing AB Test Hype Cycle 5. @OptimiseOrDie Hands on! 6. Nice day at the office, dear? @OptimiseOrDie 7. Craigs Cynical Quadrant Improves revenue Improves UX YesNo No Yes Client delighted (and fires you for another UX agency) Client fucking delighted Client absolutely fucking furious Client fires you (then wins an award for your work) 8. Top Tools & Tips #1 Get out of the office #2 Immerse yourself #3 Session Replay #4 Voice of Customer #5 Get the right inputs #6 Act like a P.I. #7 Experience testing #8 Split testing tools #9 Get performance #10 Analytics health check #11 Going agile Examples @OptimiseOrDie 9. #1 : GET OUT OF THE OFFICE @OptimiseOrDie 10. @OptimiseOrDie 1a : Lab Based Testing 11. @OptimiseOrDie 1b : Remote UX Testing 1 2 3 3 3 1 Moderator Participant2 Viewers3 3 12. Som, feedbackRemote UX tools (P=Panel, S=Site recruited, B=Both) Usertesting (B) www.usertesting.com Userlytics (B) www.userlytics.com Userzoom (S) www.userzoom.com Intuition HQ (S) www.intuitionhq.com Mechanical turk (S) www.mechanicalturk.com Loop11 (S) www.loop11.com Open Hallway (S) www.openhallway.com What Users Do (P) www.whatusersdo.com Feedback army (P) www.feedbackarmy.com User feel (P) www.userfeel.com Ethnio (For Recruiting) www.ethnio.com Feedback on Prototypes / Mockups Pidoco www.pidoco.com Verify from Zurb www.verifyapp.com Five second test www.fivesecondtest.com Conceptshare www.conceptshare.com Usabilla www.usabilla.com 13 @OptimiseOrDie 1c : Crowdsourced Testing 13. @OptimiseOrDie 1d : Beer, Caffeine and Work Breaks 14. DESKTOP & LAPTOP CamStudio (free) www.camstudio.org Mediacam AV (cheap) www.netu2.co.uk Silverback (Mac) www.silverbackapp.com Screenflow (Mac) www.telestream.net MOBILE UX Recorder (iOS) www.uxrecorder.com Skype Hugging bit.ly/tesTfm Reflection bit.ly/GZMgxR Reflector bit.ly/JnwtMo @OptimiseOrDie 1e : Guerrilla Testing 15. @OptimiseOrDie 1f : The Secret Millionaire Tesco placed IT users in front line roles with product You have to to create this kind of feedback loop If it isnt there, you need to push/encourage Connect the team with pain points AND outcomes of their work, split tests and changes Hugely motivational strategy One last tip learn how to interview like a pro Read these: Dont Make Me Think amzn.to/1gIZEJn Rocket Surgery Made Easy amzn.to/1e0hnUL Talking to Customers bit.ly/1e0hT58 Talking with Participants bit.ly/1kKL3LE Dont listen to Users bit.ly/1cQpiIE Interviewing Tips bit.ly/1fKqu03 More interviewing Tips bit.ly/1bmvGT 16. #2 : IMMERSE YOURSELF @OptimiseOrDie Test ALL key campaigns Use Real Devices Get your own emails Order your products Call the phone numbers Send an email Send 11 shoes back Be difficult Break things Experience the end-end Do the same for competitors Team are ALL mystery shoppers Wear the magical slippers Be careful about dogfood though! 17. Vital for optimisers & fills in a missing link for insight Rich source of data on visitor experiences Segment by browser, visitor type, behaviour, errors Forms Analytics (when instrumented) are awesome Can be used to optimise in real time! Session replay tools Clicktale (Client) www.clicktale.com SessionCam (Client) www.sessioncam.com Mouseflow (Client) www.mouseflow.com Ghostrec (Client) www.ghostrec.com Usabilla (Client) www.usabilla.com Tealeaf (Hybrid) www.tealeaf.com UserReplay (Server) www.userreplay.com @OptimiseOrDie #3 : GET SESSION REPLAY 18. Sitewide Omnipresent Feedback Triggered (Behavioural) Feedback Use of Features, Cancellation, Abandonment 4Q Task Gap Analysis very good Kampyle www.kampyle.com Qualaroo www.qualaroo.com Feedback Daddy www.feedbackdaddy.com 4Q 4q.iperceptions.com Usabilla www.usabilla.com #4 : GET THEIR VOICE 19. Make contact and feedback easy & encouraged Add contact & feedback to everything (e.g. all mails) Read Caroline Jarrett, run surveys (remember them?) Run regular NPS and behaviourally triggered surveys Get ratings on Service Metrics Find what drives the level of delight Ask your frequent, high spend, zealous users questions Make the team spend a day a month at the Call Centre Meet with your Sales and Support teams ALL the time Tip : Take them for Beers and encourage bitching #4 : GET THEIR VOICE 20. Insight - Inputs #FAIL Competitor copying Guessing Dice rolling An article the CEO read Competitor change Panic Ego Opinion Cherished notions Marketing whims Cosmic rays Not on brand enough IT inflexibility Internal company needs Some dumbass consultant Shiny feature blindness Knee jerk reactons #5 : Your inputs are all wrong @OptimiseOrDie 21. Insight - Inputs Insight Segmentation Surveys Sales and Call Centre Session Replay Social analytics Customer contact Eye tracking Usability testing Forms analytics Search analytics Voice of Customer Market research A/B and MVT testing Big & unstructured data Web analytics Competitor evalsCustomer services #5 : These are the inputs you need @OptimiseOrDie 22. For your brand(s) and competitors Check review sites, Discussion boards, News Use Google Alerts on various brands & keywords See what tools theyre using (www.ghostery.com) Sign up for all competitor emails Run Cross Competitor surveys This was VITAL for LOVEFiLM Use Social & Competitor Monitoring tools : slidesha.re/1k7bflG #6 : ACT LIKE A PI 23. #4 Test or Die! Email testing www.litmus.com www.returnpath.com www.lyris.com Browser testing www.crossbrowsertesting.com www.browserstack.com www.spoon.net www.saucelabs.com www.multibrowserviewer.com Mobile devices www.appthwack.com www.deviceanywhere.com www.mobilexweb.com/emulators www.opendevicelab.com @OptimiseOrDie #7 : MAKE MONEY FROM TESTING! 24. Google Content Experiments bit.ly/Ljg7Ds Optimizely www.optimizely.com Visual Website Optimizer www.visualwebsiteoptimizer.com Multi Armed Bandit Explanation bit.ly/Xa80O8 New Machine Learning Tools www.conductrics.com www.rekko.com @OptimiseOrDie #8 : MAKE MORE MONEY FROM TESTING! 25. #9 Fight! Google PageSpeed Tools Webpagetest.org Mobitest.akamai.com 26 @OptimiseOrDie 26. Site Size Requests The Daily Mail 4574k 437 Starbucks 1300k 145 Direct line 887k 45 Ikea (.se) 684k 14 Currys 667k 68 Marks & Spencers 308k 45 Tesco 234k 15 The Guardian 195k 35 BBC News 182k 62 Auto Trader 151k 47 Amazon 128k 16 Aviva 111k 18 Autoglass 25k 10 Real testing : mobitest.akamai.com @OptimiseOrDie 27. Slides : slidesha.re/PDpTPD If you really care, download this deck: @OptimiseOrDie 28. Scare the Ecom or Trading director: 29. #10 : Your analytics tool is broken! @OptimiseOrDie 30. Get a Health Check for your Analytics Mail me for a free pack Invest continually in instrumentation Aim for at least 5% of dev time to fix + improve Stop shrugging : plug your insight gaps Change I dont know to Ill find out Look at event tracking (Google Analytics) If set up correctly, you get wonderful insights Would you use paper instead of a till? You wouldnt do it in retail so stop doing it online! How do you win F1 races? With the wrong performance data, you wont @OptimiseOrDie #10 : Your analytics tool is broken! 31. #11 : Go Agile @OptimiseOrDie 32. Methodologies - Lean UX Positive Lightweight and very fast methods Realtime or rapid improvements Documentation light, value high Low on wastage and frippery Fast time to market, then optimise Allows you to pivot into new areas Negative Often needs user test feedback to steer the development, as data not enough Bosses distrust stuff where the outcome isnt known The application of UX design methods into product development, tailored to fit Build-Measure-Learn cycles. 33 33. Agile UX / UCD / Collaborative Design Positive User centric Goals met substantially Rapid time to market (especially when using Agile iterations) Negative Without quant data, user goals can drive the show missing the business sweet spot Some people find it hard to integrate with siloed teams Doesnt work with waterfall IMHO Wireframe Prototype TestAnalyse Concept Research An integration of User Experience Design and Agile* Software Development Methodologies *Sometimes 34 34. CRO 35 35. Lean Optimisation Positive A blend of several techniques Multiple sources of Qual and Quant data aids triangulation Focus on priority opportunities drives unearned value and customer delight for all products Negative Needs a one team approach with a strong PM who is a Polymath (Commercial, Analytics, UX, Technical) Only works if your teams can take the pace you might be surprised though! A blend of User Experience Design, Agile PM, Rapid Lean UX Build-Measure-Learn cycles, triangulated data sources, triage and prioritisation. 36 36. Lean CRO Inspection Immersion Identify Triage & Triangulate Outcome Streams Measure Learn Instrument 37 37. We believe that doing [A] for People [B] will make outcome [C] happen. Well know this when we observe data [D] and obtain feedback [E]. (reverse) @OptimiseOrDie 38. agile - Summary Design your own methodology Experiment and optimise with your team Dont be a slave The methodology is the slave, not your master http://tcrn.ch/1gPpUNo Collaborative working Harvard study into teams its an all the time thing Ask me later Questions see me on Twitter, G+ or ask by mail @OptimiseOrDie 39. Scarcity principle... #1 : EXAMPLES 40. Scarcity principle... #1 : EXAMPLES 41. 20M+ visitor tests with People Images Some interesting stuff at Autoglass (Belron) Negative body language is a turnoff Uniforms and branding a positive (ball cap) Eye gaze and smile are crucial Hands are awkward without a prop Best prop tested was a clipboard Single image better than groups In most countries (out of 33) with strong female and male images in test, female won So a question about this test @OptimiseOrDie #2 : SPLIT TESTING PEOPLE 42. @OptimiseOrDie 43. Terrible Stock Photos : headsethotties.com & awkwardstockphotos.com Laughing at Salads : womenlaughingwithsalad.tumblr.com Other Stock Memes : linkli.st/optimiseordie/7Fdxz BBC Fake Smile Test : bbc.in/5rtnv @OptimiseOrDie 44. UK Wave 2 - Isi 45. TV - Off TV - On Isi went on to star in the TV slot and helped Autoglass grow recruitment of female technicians, as well as proving a point! #3 : TV ADVERTISING 46. SPAIN +22% over control 99% confidence @OptimiseOrDie #3 : PHOTOGRAPHY UX 47. @OptimiseOrDie #4 : VOC, NPS, EXPERIMENTS Belron NPS programme is huge Millions of people every year, across the world 35% survey takeup, 6% dropout rate! (Try @lukew and @cjforms and @stickycontent) Higher scores than some consumer products Why? Measuring the drivers of delight Even on A/B tests, we could split NPS data We could see a new funnel drove a 5.5% rise Lovefilm beat their competitors using NPS How? Measuring key service metrics Regression to find high value investment areas Contact deflection using self service Analytics, split testing, UX 48. How is it working out for Craig? Methodologies are not Real Life Its mainly about the mindset of the team and managers, not the tools or methodologies used Not all my clients have all the working parts Use some, any techniques instead of guessing Bringing together UX techniques with the excellent tools available along with analytics investment - will bring you successful and well- loved products Blending Lean and Agile UX with conversion optimisation techniques (analytics, split testing, Kaizen, Kano) is my critical insight from the last 5 years. UX got hitched to numbers, they ran away and lived happily ever after 49 49. If it isnt working, youre not doing it right @OptimiseOrDie 50. Email Twitter : sullivac@gmail.com : @OptimiseOrDie : linkd.in/pvrg14 Slides on Slideshare.net/sullivac tonight! 51. RESOURCE PACK Maturity model Best CRO people on twitter Best Web resources Good recent books to read Triage and Triangulation The Bucket outcome methodology Belron methodology example CRO and testing resources Companies and people to watch Building a ring model Manual Models for Analytics @OptimiseOrDie 52. Ad Hoc Local Heroes Chaotic Good Level 1 Starter Level Guessing A/B testing Basic tools Analytics Surveys Contact Centre Low budget usability Outline process Small team Low hanging fruit + Multi variate Session replay No segments +Regular usability testing/research Prototyping Session replay Onsite feedback ________________________________________________________________________ _____________________ _ Dedicated team Volume opportunities Cross silo team Systematic tests Ninja Team Testing in the DNA Well developed Streamlined Company wide +Funnel optimisation Call tracking Some segments Micro testing Bounce rates Big volume landing pages + Funnel analysis Low converting & High loss pages + offline integration Single channel picture + Funnel fixes Forms analytics Channel switches +Cross channel testing Integrated CRO and analytics Segmentation +Spread tool use Dynamic adaptive targeting Machine learning Realtime Multichannel funnels Cross channel synergy ________________________________________________________________________ _______________________ ________________________________________________________________________ ________________________ Testing focus Culture Process Analytics focus Insight methods +User Centered Design Layered feedback Mini product tests Get buyin _________________________________________________________________________ _______________________Mission Prove ROI Scale the testing Mine value Continual improvement + Customer sat scores tied to UX Rapid iterative testing and design + All channel view of customer Driving offline using online All promotion driven by testing Level 2 Early maturity Level 3 Serious testing Level 4 Core business value Level 5 You rock, awesomely ________________________________________________________________________ ________________________ 53 53. Best of Twitter @OptimiseOrDie @danbarker Analytics @fastbloke Analytics @timlb Analytics @jamesgurd Analytics @therustybear Analytics @carmenmardiros Analytics @davechaffey Analytics @priteshpatel9 Analytics @cutroni Analytics @Aschottmuller Analytics, CRO @cartmetrix An...