tips for conducting long-term diary studies - upa 2009
TRANSCRIPT
![Page 1: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/1.jpg)
Conducting Longitudinal Diary Studies
Amanda Nance
Sage
June 10, 2009
![Page 2: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/2.jpg)
Early diary study
![Page 3: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/3.jpg)
Goal of this presentation
• How to run a better diary study
– Create effective template
– Understand the data
– Keep users engaged
• See benefits of the method
![Page 4: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/4.jpg)
Topics for this presentation
• Reason for the project
• About the diary study
• Diary study tips
• Unique contributions of the diary method
![Page 5: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/5.jpg)
Reasons for the project
• Assess “getting started” experience of our desktop accounting software
• Focus on tasks after initial set up
• Create requirements for next release
![Page 6: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/6.jpg)
Used multiple methods to get truer picture
Site visits Support calls Survey analysis
Researcher calls Diary studyDiary from: http://www.flickr.com/photos/bdorfman/15846725/
![Page 7: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/7.jpg)
About the diary study
• 5 participants: small business owners, bookkeepers
• Diaries kept for up to 8 weeks beginning at install
• Diaries submitted weekly by email
• Compensation – at least $40/week
![Page 8: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/8.jpg)
About the diary study
• Provided template in Word to record
– Daily Work: tools they used, other tasks
– Issues and Comments: problems with our product
![Page 9: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/9.jpg)
Tips
![Page 10: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/10.jpg)
Recruit extra participants
• Recruited 12
• 5 completed
• Reasons for drop outs were interesting findings, too
![Page 11: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/11.jpg)
Use a recruiter to save time
• Pros and cons
• They can recruit and more– Email documents– Remind of deadlines
– Collect diaries
![Page 12: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/12.jpg)
Communicate deadlines
• Communicate due date for first submission
• Disqualify participants who don’t respond
• Close the loop
![Page 13: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/13.jpg)
Template tips
![Page 14: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/14.jpg)
![Page 15: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/15.jpg)
Make it easy for participants
• Keep it short
– This study: 3 questions daily, 6-7 per incident
• Separate questions from answers
![Page 16: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/16.jpg)
Separate questions from answers
![Page 17: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/17.jpg)
Show sample entries
What did you do for your business today?
• I answered several customer phone calls.
• Called automated bank system to make sure we have enough money for payroll.
• Called in payroll to ADP (we outsource payroll).
• Ordered parts for one of our jobs by phone.
![Page 18: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/18.jpg)
Ask for screenshots
“Looking at Aged Receivables…. They do not match. Why are they different?”
![Page 19: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/19.jpg)
Tips for screenshots
• If they are important, require them
• Include instructions
![Page 20: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/20.jpg)
Structure template for easy analysis
• Separate daily entries from critical incidents
• Include participant name and week number in footer and file name
![Page 21: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/21.jpg)
Iterate on the template
• Prior to launching
• As you receive user feedback
![Page 22: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/22.jpg)
Process tips
![Page 23: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/23.jpg)
Require daily entries
• Keeps participants engaged
• Gives you a reason to pay them
• Opportunity to learn about the broader work context
![Page 24: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/24.jpg)
Talk to the participants yourself
• Intro call to understand business and product expectations
• Follow up on unclear entries
• Provide support
![Page 25: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/25.jpg)
Research tips
• Read entries as soon as possible
• Leave time to research
![Page 26: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/26.jpg)
Data analysis
• No different from other methods
• Evaluate data– Excel to categorize theme, second theme, week #
• Share and act on data– Summarize key findings, provide recommendation
for each
– Use customer quotes
– Share complete diaries
![Page 27: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/27.jpg)
Unique contributions of diary study
• Infrequent tasks
• Help use
• Users resolving issues
• Tasks over time
![Page 28: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/28.jpg)
Reasons to run a diary study
• No travel required
• Non-intrusive, in context
• Cheaper than site visits
![Page 29: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/29.jpg)
Engagement and meaningful data
• Pay for more complete responses
• Encourage useful responses through feedback on recent entries
• Continue participants who are useful
![Page 30: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/30.jpg)
Key Points
• Daily entries
• Explain what you expect
• Sample entries
• Reasonable number of questions
• Personal contact with researcher
• Questions?
![Page 31: Tips for conducting long-term diary studies - UPA 2009](https://reader033.vdocuments.net/reader033/viewer/2022052509/55a1da681a28abd0198b4660/html5/thumbnails/31.jpg)
Contact
More tips in conference proceedings
Twitter: @amandaux