testing beyond software, a case study
DESCRIPTION
So far, testing has always been related to hardware and, specially, to software development. We tend to agree that building new pieces of software, programs, applications... requires testing efforts and we, as a community, struggle to establish, strengthen and refine these efforts; because we believe that testing is never in vain, we believe that testing adds value to wherever it is attached to. If we believe that testing adds value to wherever it is attached to, why don’t apply testing efforts beyond software development? In this presentation, I will explain a work in progress case study of testing applied not to software development, but to software usage, for real-time production and launching of on-line market research surveys in a complex and highly demanding environment.TRANSCRIPT
@Mauri_Edo
Testing beyond softwareA case study
@Mauri_Edo
Where is the action?
Online market research projects+1.850.000 interviews in +1.000 projects (2012)
Proprietary tools
@Mauri_Edo
Online market research?Yes, online surveys mainly
But online surveys are simple!Hmm... Are they?
@Mauri_Edo
Mental image ≈ 3 minutes
Text inputsSingle response
Multiple responseSimple routing
@Mauri_Edo
Real image ≈ 15 minutes
Plus:Flash / jQuery
LoopsClient-side
scripting…
@Mauri_Edo
The problem:Faulty surveys being launched
Let’s add some testing and quality to this process!
Cool! But… where to start?
@Mauri_Edo
Gather information!
Primary sourceAsk questions!
Secondary sourcesObservation
Current issuesCurrent quality level
@Mauri_Edo
Information gathering results
Mission:Compliance with requirementsCreate a safety net before survey reaches the customer for approval
Stakeholders:CustomersRespondents
@Mauri_Edo
ApproachIterate
Little bets*
MonitorDiversify
Triage
DIYPersevere!
*A Peter Sims idea
@Mauri_Edo
Disclaimer:Work in progress,
potential fatal errors included
@Mauri_Edo
Current status
Formreview
Form LaunchSpec Coding C.A.T.
C.A.T. = Customer Acceptance Testing
Survey testing
Spec reviewTest design
Test executionE.T.
Data validation
@Mauri_Edo
Challenge #1: culture
Missing background on quality and testing:Knowledge transference (and recycling)Message consistencyCommunicate, communicate, communicate
Aversion to change:Start small, gain momentumProve your victoriesGain followers to get more followers
@Mauri_Edo
Challenge #2: miscommunication
Miscommunication = confusion, mistakes, stressLead by example:
- Daily stand-up meetings- Weekly sync meetings
Empower communication in others:- General, non-project related, meetings- Cross-team trainings- Stand up and talk!
@Mauri_Edo
Challenge #3: development pace
“Survey programming cycle shouldn’t be dramatically extended in time” (anonymous co-worker)
Challenge everything!Any deadlocks available?Testability + test utilitiesSmoke tests“Smart” checklistsTesting depth ↔ Time available
@Mauri_Edo
Error #1: inattention to specs
Specs are valuable!
Promote research and improvement in order to achieve specs ownership
Make related issues visible
Test against specs compliance
@Mauri_Edo
Error #2: client modifications
Before...
After...
Full regression testing? Maybe...
Form LaunchSpec Coding C.A.T.
Express Testing
Form LaunchSpec Coding C.A.T.
Fixes
@Mauri_Edo
Error #3: issue unawareness
Active search for issues:Soft launch + data consistency analysisRepeatable process, anytime
Don’t wait for bugs to knock at your door
Who is aware of the issues?Respondent support team (functional, content)Application support team (internal)
Connect the teams
@Mauri_Edo
Lessons learned
@Mauri_Edo
About quality systems...
The (successful) implementation of a quality system needs the bosses’ approval AND the embrace of the regular workers
Quality might start with testing but it has to permeate through the rest of teams / areas
@Mauri_Edo
About testing...
Be adaptable, a full regression test suite execution is not the solution to all problems
Communication is key
Make yourself visible!
Test strategies require lots of testing as well
@Mauri_Edo
About challenges...
Quality can add value everywhere, beyond software development
Future job, anyone?
Thanks!Questions?
testingfuncional.wordpress.com
@Mauri_Edo
Mauri Edo