|
Crowdsourcing KeysThe Never-Ending Journey To Define Scope & Briefs
Matt Johnston | @matjohnston | CMO @ uTest | April 2012
The Challenge
||2
What We’reTalking About
|3In-The-Wild Testing for Functional + Security + Load + Usability + Localization
• Two critical components to c-sourcing success:1. What’s the nature & intent of the project?
2. Who’s the best fit to do a project?
Boiled Down
The Challenge
||4
Who’s The Best Fit
|5In-The-Wild Testing for Functional + Security + Load + Usability + Localization
• This 2nd question gets the attention, so we’ll focus elsewhere• For uTest, matching algorithms based upon
– Based upon profile– Based upon merit-based ratings– Based upon other criteria (eg: load balancing)
• Structured data is vital to good matches
Who’s The Best Fit?
The Challenge
||6
Defining Nature &Intent Of Projects
|7In-The-Wild Testing for Functional + Security + Load + Usability + Localization
• Fundamental nature of your projects– Creative brief ≠ technical spec ≠ advertising guidance– Design vs. animation vs. dev vs. testing vs. advertising vs. content
• What customer is really trying to do and cares about– Effectiveness vs. efficiency
- Signal-to-noise ratio?- Precision?- Brute force?
• Those who can’t articulate the
problem won’t find the solution.
First Things First
|8In-The-Wild Testing for Functional + Security + Load + Usability + Localization
• Our journey:– 2008 – We launched… and we sucked– 2009 – More sucking – 2010 – Started wrapping head around nature of the problem– 2011 – More sophisticated– 2012 – Solved most of the problem… and created several new ones
- Launched 4 new testing types, each w/ their own project definition challenges
• Key takeaways for us:– Garbage in, garbage out– Real customer of such info is our community– Nature of project matters:
- If goal is predictability (technical work), unstructured data is the enemy- If goal is creativity, unstructured data is still the enemy (just a necessary evil)
Defining Work To Be Done
The Challenge
||9
ExamplesPast & Future
|10In-The-Wild Testing for Functional + Security + Load + Usability + Localization
Examples: Project Setup
|11In-The-Wild Testing for Functional + Security + Load + Usability + Localization
Examples: Project Scope
|12In-The-Wild Testing for Functional + Security + Load + Usability + Localization
• Even with right questions, we had room for improvement:– Ux for customers to create input
- Information architecture and GUI design- Integrations with legacy systems
– Ux for community to consume output- Information architecture and GUI design- Change management
Input / Output Matters
|13In-The-Wild Testing for Functional + Security + Load + Usability + Localization
Examples: Project Setup
|In-The-Wild Testing for Functional + Security + Load + Usability + Localization
Examples: Project Scope
14
|15In-The-Wild Testing for Functional + Security + Load + Usability + Localization
• Two critical components to c-sourcing success:1. What’s the nature & intent of the project?
2. Who’s the best fit to do a project?
• Should spend equal calories, hours and brain cells on each
Bottomline
