introduction to web surveys 10-10.ppt - uic | cuppa to web surveys 10-10.pdf · introduction to web...
TRANSCRIPT
![Page 1: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/1.jpg)
Survey Research Laboratory
Introduction to Web Surveys
Survey Research Laboratory
University of Illinois at Chicago
October 2010
![Page 2: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/2.jpg)
Web Surveys
� First reported use in early 1990’s
� Dramatic increase in use over the past decade
� Numerous web survey software packages now availableavailable
![Page 3: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/3.jpg)
Basic Advantages of Web
Surveys� Speed
� Cost
� Convenient (self-administered)
� Multi-media delivery (sound, video)
� Power of computer-assisted programming� Power of computer-assisted programming
� Unique, hi-tech
� Similar arguments were made regarding CATI (in the 1970s) and CAPI (in the 1980s) technologies
![Page 4: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/4.jpg)
![Page 5: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/5.jpg)
![Page 6: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/6.jpg)
![Page 7: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/7.jpg)
Typology of Web Surveys
With thanks to Mick Couper, University of Michigan Survey Research Center(see: Couper, Web Surveys: A Review of Issues and Approaches. Public (see: Couper, Web Surveys: A Review of Issues and Approaches. Public Opinion Quarterly 2000;
64: 464-494)
•Non-probability methods
•Probability methods
![Page 8: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/8.jpg)
Non-probability Methods
1. Polls as entertainment
2. Unrestricted self-selected surveys2. Unrestricted self-selected surveys
3. Volunteer opt-in panels
![Page 9: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/9.jpg)
Probability Methods
4. Intercept surveys
5. List-based samples
6. Web option in mixed-mode surveys6. Web option in mixed-mode surveys
7. Pre-recruited panels of internet users
8. Pre-recruited panels of full population
![Page 10: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/10.jpg)
Type 1: Entertainment Polls
� “question of the day”
� Intended primarily for entertainment
� No pretense at “science” or representativeness� No pretense at “science” or representativeness
� Basically harmless so long as audience can distinguish them from real surveys
![Page 11: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/11.jpg)
![Page 12: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/12.jpg)
Type 2: Unrestricted self-selected
surveys
� Open invitation on portals, frequently-visited web sites, or dedicated “survey” sites
� No access restrictions� No access restrictions� Ballot-stuffing possible
� Equivalent of 1-900 poll or magazine insert survey
� Main distinction with type 1 are claims to legitimacy are sometimes made here
![Page 13: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/13.jpg)
![Page 14: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/14.jpg)
Type 3: Volunteer panels of internet
users
� Create volunteer (opt-in) panel � using open invitation, recruit large number of
people willing to do web surveys
� Use quota controls or random sampling of select persons from this group for a select persons from this group for a particular survey
� Control access through invitation & PIN
� Although 2nd step (selection within panel) is controlled, 1st step is self-selected and uncontrolled
![Page 15: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/15.jpg)
Type 3: Continued
� Claims to generalize to total population
� Maybe most common approach to web surveys now
� As with all panels, some of these pay respondents to participaterespondents to participate
� Some are claiming that these panels are equal or better than other forms of data collection based on probability methods
![Page 16: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/16.jpg)
![Page 17: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/17.jpg)
![Page 18: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/18.jpg)
Probability-Based Methods
#4: Intercept Surveys
� Target is visitors to a web site� Customer satisfaction
� Web site evaluation
Systematic sample commonly used – every � Systematic sample commonly used – every nth visitor
� No coverage problem because the population of interest is active web users
� Biggest problem is nonresponse
� Also problem of timing – when to intercept?
![Page 19: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/19.jpg)
![Page 20: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/20.jpg)
![Page 21: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/21.jpg)
#5: List-Based Samples of High-
Coverage populations
� Recruit and invite those with web access to participate
� Restricted to current internet usersRestricted to current internet users
� Controlled access
� Nonresponse occurs at many stages in the process but can be measured (via analyses of baseline data)
� High coverage population examples:� College students
� Members of professional organizations
� IT professionals
![Page 22: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/22.jpg)
![Page 23: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/23.jpg)
#6: Mixed-Mode Designs with Choice
of Completion Method
� Web survey as part of a mixed mode design
� Example: mail survey with invitation to do via web
� Requires controlled access
� Concerns over equivalence of measurement� “mode effects”
� More expensive than single mode survey
� Offering respondents a choice helps confront nonresponse problems
![Page 24: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/24.jpg)
![Page 25: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/25.jpg)
#7: Pre-recruited Panels
of Internet Users
� Random probability methods used to contact and invite persons with internet access to participate
� This approach is limited to active internet users� This approach is limited to active internet users
� More expensive than some other approaches
� May be difficult to assess nonresponse
![Page 26: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/26.jpg)
#8: Probability Samples
of Full Population
� Start with probability sample of target population
� Give everyone access to internet in exchange for
participation
� Only approach with potential to be representative of
general population
� Biases and errors can be measured
� Nonresponse, panel effects and costs are big
concerns
� Example: Knowledge Networks
![Page 27: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/27.jpg)
![Page 28: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/28.jpg)
Designing Web Questionnaires
![Page 29: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/29.jpg)
Basic Design Approaches
• Static web questionnaire• Survey in single HTML document
• Respondents can scroll through document
• Data sent to server once when survey is completed
• Interactive web questionnaire• Interactive web questionnaire• Questions are delivered one at a time or in modules
• Data is sent to server after each screen is completed
• Conducive to use of skip patterns, consistency checks, range checks, etc.
![Page 30: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/30.jpg)
Static Web Questionnaires
• Very similar to mail and other self-administered questionnaires
• Can minimize download time
• Respondents can skip questions, but the • Respondents can skip questions, but the process is not usually automated• Hypertext links can be used to facilitate skips
• All information is lost if respondent quits before finishing
• More advantageous for short questionnaires
![Page 31: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/31.jpg)
Interactive Web Questionnaires
• This approach permits the use of all
computer-assisted programming devices
• May increase length of survey due to
additional download timeadditional download time
• Partial data is captured for respondents who
quit before finishing questionnaire
• More advantageous for longer and more
complex questionnaires
![Page 32: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/32.jpg)
Progress Indicators
• The purpose is to motivate respondents to complete the questionnaire in the absence of an interviewer• Couper et al. (2001): 89.9% completed survey with progress
indicator vs. 86.4% completing survey without one
• Very useful in interactive questionnaires, where respondent does not know how long the questionnaire is
• Not necessary in static questionnaires where • Not necessary in static questionnaires where respondents can determine the length by scrolling through it
• May add to survey length if increase download time• There is some concern of increased break-offs
• Transition sentences are an alternative
• Empirical evidence regarding effectiveness not clear
![Page 33: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/33.jpg)
![Page 34: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/34.jpg)
![Page 35: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/35.jpg)
![Page 36: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/36.jpg)
![Page 37: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/37.jpg)
![Page 38: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/38.jpg)
![Page 39: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/39.jpg)
General Screen Design• Do not use background color or images
• Background colors can create contrast & reading
problems
• “visual noise”
![Page 40: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/40.jpg)
![Page 41: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/41.jpg)
General Screen Design #2
• Be aware that images may bias responses• Witte et al (2004) – National Geographic Survey
• Images increased support for species protection
• Couper et al (2007) – healthy vs. sick person image• When exposed to fit person, respondents consistently rated their • When exposed to fit person, respondents consistently rated their
own health as lower than when exposed to sick person
• Use upper right corner for contact information• Privacy/IRB information can be clickable from there
• If top of screen format is consistent:• Respondents will tend to ignore that section across
pages• “Banner-Blindness”
![Page 42: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/42.jpg)
General Screen Design #3
• Access to other relevant information can also be provided:
• Answers to commonly asked questions about the
surveysurvey
• pdf versions of the full questionnaire
![Page 43: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/43.jpg)
![Page 44: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/44.jpg)
![Page 45: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/45.jpg)
![Page 46: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/46.jpg)
![Page 47: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/47.jpg)
Effect of Color on Web Survey
Completion
• Do not overuse color but use it consistently• Use red only for emergency messages
• Red-green distinctions a problem with persons who are color-blind • 10% of males are color blind
• 99% of color blind persons cannot distinguish green & red
• White or off-white backgrounds seem to work best• Some evidence that R’s view black-on-white web
pages as being more ‘professional’ than white-on-black web pages
• Couper (2008) prefers light blue backgrounds
![Page 48: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/48.jpg)
Color, continued
• For maximum readability, should be high contrast between text color and background color
• Bright colors are easier to see than pastels
• Colored backgrounds often used by spammers • Colored backgrounds often used by spammers and may reduce response rates
![Page 49: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/49.jpg)
Text
• Always avoid small font sizes (use 10-12 point)
• Appears to be some preference for Arial over Times Roman font
• Do not overuse bold, underline, italics and other forms of emphasisforms of emphasis
![Page 50: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/50.jpg)
Question Presentation
• Avoid requiring R to horizontally scroll
• Avoid any scrolling may be best
![Page 51: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/51.jpg)
![Page 52: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/52.jpg)
Question Presentation
• Avoid requiring R to horizontally scroll• Avoid any scrolling may be best
• No agreement about inclusion of question numbers • Excluding them may avoid skip logic confusion• Excluding them may avoid skip logic confusion
• Likert questions (fully-labeled) should be displayed vertically
![Page 53: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/53.jpg)
![Page 54: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/54.jpg)
Question Presentation #2
• Respondents less likely to skip words when lines are kept short
• Provide computer-operating instructions at the precise point when a R may need to use that informationinformation
• When # of responses cannot be fitted on single screen:• Double- or triple-banking may be best approach
• Place a box around the categories in order to ‘group’ them as being relevant to the question
![Page 55: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/55.jpg)
![Page 56: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/56.jpg)
![Page 57: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/57.jpg)
Question Presentation #3
• Visibility principle
• Options that are visible are more likely to be
selected than those that are not visible until the R
takes some action to display them
• Response models• Response models
• Serial processing model
• Search options for pre-existing judgment
• Deadline processing model
• Spend certain amount of time and select best answer found before cognitive deadline (a form of satisficing)
![Page 58: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/58.jpg)
Common Types of Response Options
for Web Surveys
1. Radio buttons or boxes
2. Drop-down boxes
3. Check boxes
4. Slider bars
5. Text boxes
6. Open-ended questions
![Page 59: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/59.jpg)
Radio Buttons
•Options are typically mutually exclusive
![Page 60: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/60.jpg)
![Page 61: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/61.jpg)
![Page 62: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/62.jpg)
![Page 63: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/63.jpg)
![Page 64: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/64.jpg)
![Page 65: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/65.jpg)
Be careful not to use long grids
that lose column headings:
![Page 66: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/66.jpg)
![Page 67: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/67.jpg)
![Page 68: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/68.jpg)
Boxes instead of buttons
![Page 69: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/69.jpg)
![Page 70: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/70.jpg)
![Page 71: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/71.jpg)
![Page 72: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/72.jpg)
Drop-Down Boxes
•Useful only for closed lists of response options
•Can be designed to allow for single or multiple •Can be designed to allow for single or multiple
choices
•Options provided must be exhaustive
•Drop boxes more difficult to use than radio buttons
![Page 73: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/73.jpg)
![Page 74: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/74.jpg)
![Page 75: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/75.jpg)
![Page 76: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/76.jpg)
![Page 77: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/77.jpg)
Beware of Scroll Mice� Healy (2007)
� Drop-downs (compared to radio buttons) led to
higher item nonresponse and longer response times
� Respondents using scroll mice to complete the
survey were prone to accidentally changing an survey were prone to accidentally changing an
answer if presented with drop-down questions
![Page 78: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/78.jpg)
Check Boxes
•Unlike radio buttons, multiple choices can be clicked
via check boxesvia check boxes
![Page 79: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/79.jpg)
![Page 80: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/80.jpg)
Radio Button/Check Box Hybrid
![Page 81: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/81.jpg)
![Page 82: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/82.jpg)
Slider Bars (a.k.a. visual analog
scale, graphic rating scales,
“sliders”)
![Page 83: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/83.jpg)
![Page 84: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/84.jpg)
![Page 85: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/85.jpg)
Slider Bars - Research• Random experiment by Bayer & Thomas
(2004) of Harris Interactive• Slider bars took about twice as long for completion
as any other scale type (including semantic differentials, likert, etc)
• Answering 2 slider bar questions averaged 42.3 • Answering 2 slider bar questions averaged 42.3 seconds, compared to 21.3 seconds for semantic differential questions
• Couper (2008) says results using slider bars are “quite similar” to what is obtained from a scale that uses radio buttons
![Page 86: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/86.jpg)
![Page 87: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/87.jpg)
![Page 88: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/88.jpg)
![Page 89: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/89.jpg)
Open-ended Questions
•providing more space encourages respondents to •providing more space encourages respondents to
provide longer answers
![Page 90: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/90.jpg)
![Page 91: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/91.jpg)
Present Single or Multiple Items per
Screen?
• For interactive questionnaires, multiple items per screen:• Are completed more quickly by respondents
• May provide more context• May provide more context
• Intercorrelations among items are consistently higher when grouped together on one screen (Couper et al. 2001).
• Also, multiple item screen versions:• take less time to complete
• produce less missing data
![Page 92: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/92.jpg)
Survey Navigation
• A consistent format should be followed
• Use action buttons that are different from any
response input elements such as radio buttons
• “next screen” or “next question” buttons should be on • “next screen” or “next question” buttons should be on
all pages
• Crawford et al (2005) recommends putting them in the
lower left corner
• “previous screen” or “previous question” buttons should
be in the bottom right corner
![Page 93: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/93.jpg)
![Page 94: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/94.jpg)
Key Point
• Never force respondents to answer a question
• Adds to frustration
• IRB implications
• No other questionnaire formats ‘force’ answers• No other questionnaire formats ‘force’ answers
![Page 95: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/95.jpg)
Key Questionnaire Design Principles
Summary
• Minimize respondent burden and frustration
• The fewer ‘clicks,’ the better
• The less scrolling, the better
• The fewer distractions, the better• The fewer distractions, the better
• The fewer problems knowing how to navigate the
questionnaire, the better
• The less download time required, the better
• Never forcing respondents to answer questions
![Page 96: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/96.jpg)
Some other design
recommendations to consider (from
Couper 2008):
• Remove unneeded content and clutter
• Minimize the number of different colors and fonts being used
Use consistent design formats through the • Use consistent design formats through the entire instrument
• Avoid putting too much material on any page
![Page 97: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/97.jpg)
Summary� Web surveys vary greatly in their goals, design,
execution, analysis, etc.
� Evaluation must be done in the context of the type of survey being conducted
� Cannot say that all web surveys are good or bad
� Methodological research is being done on a moving target
![Page 99: Introduction to Web Surveys 10-10.ppt - UIC | CUPPA to Web Surveys 10-10.pdf · Introduction to Web Surveys University of Illinois at Chicago October 2010. Web Surveys ... No pretense](https://reader035.vdocuments.net/reader035/viewer/2022062603/5f03e2987e708231d40b3f37/html5/thumbnails/99.jpg)