using surveys for assessment · 2020-06-30 · what is your question/objective? know your...
TRANSCRIPT
![Page 1: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/1.jpg)
Using Surveys for Assessment
Best Practices in AssessmentApril 3, 2013
![Page 2: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/2.jpg)
Goals for the WorkshopAll things considered before conducting a survey
when developing survey questions
when developing a questionnaire
from administration to reporting
![Page 3: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/3.jpg)
What to consider before conducting a survey?
![Page 4: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/4.jpg)
What is your question/objective?Know your question/objective- Be specific about what you want to know.
Be clear about why you want to know it.
Be clear about what you plan to do with the results.
Make sure the information is not available elsewhere.
Make sure this is the right source to get the information needed.
![Page 5: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/5.jpg)
Who are you asking?Knowing your population can affect:
The survey design
The questions asked
The wording of the questions
The way(s) you contact prospective participants
The administration of the survey
The response rate
The results
![Page 6: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/6.jpg)
How will the survey be administrated?
Paper Forms:By mailFace-to-face
Telephone On-line
![Page 7: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/7.jpg)
How will you analyze and report the results?
Answer will depend on: The purpose of the study (i.e. to
describe, explain, predict or explore)
Your audience (i.e. who will be using the results and/or report)
The available software applications (i.e. Excel/SPSS/SAS)
![Page 8: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/8.jpg)
What to consider when developing survey questions?
![Page 9: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/9.jpg)
Survey QuestionsThere are two basic types:
Open-ended
Closed-ended/Partially Closed-ended
![Page 10: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/10.jpg)
Open-ended Questions
Advantages: To supplement a close-ended question (e.g. to solicit and explanation of a particular answer) To explore a new topic To offer comments and suggestions To vent frustration To highlight issue(s) that is overlookedDisadvantages: Often ignored by respondents because:
- there are no answer choices- respondents must formulate their own answers
Answers are extremely difficult to code, process and analyze Answers are hard to read
Example: What should be done to improve this program?
![Page 11: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/11.jpg)
Closed-ended Questions Yes/No
Multiple Choice
Ranking
Rating Scale
![Page 12: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/12.jpg)
Yes/No Questions
Advantages: Useful in simple and complex analyses Useful as filter questions to create skip patterns.
Disadvantages: Provides little information Difficult to reduce complex information to yes/no Often boring for respondents
Example: Have you used the BCC online tutoring resources?
![Page 13: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/13.jpg)
Multiple Choice Questions
Advantages: Yields factual information
Quick and easy to answer
Easy to tabulate
Disadvantages: Difficulty including all possible answers
More than one answer is often given
Answers are usually categorical or ordered
Example: In which of the following ways have you used the BCC online tutoring resources?
Which one of the following components of the online resources was most helpful in ....
![Page 14: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/14.jpg)
Ranking Questions
Advantages: Easy to understand
Disadvantages: Can be difficult to complete
Assumes differences in respondent’s rankings
Example: Please rank each factor below to indicate how important it is to you as a reason for your enrollment at BCC? Rank the most important reason ‘1’…
![Page 15: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/15.jpg)
Rating Scale QuestionsLikert ScaleExample: My instructor grades fairly. Strongly agree, Agree, Disagree, Strongly
disagreeAdvantages: Popular and easy to complete
Efficient
Good measure of attitudes and opinions
Facilitate easy comparison of answers using the same scale
Disadvantages: Item are often ambiguous
Not suitable for factual information
![Page 16: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/16.jpg)
Rating Scale QuestionsSemantic ScaleExample: Accessing your BCC email is:
Easy__ __ __ __ __DifficultAdvantages: Efficient
Facilitate easy comparison of answers using the same scale
Disadvantages: Directions can be difficult to write
![Page 17: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/17.jpg)
Rating Scale QuestionsGoal Attainment ScaleExample: Please review the skills listed below and try to assess what your expectations were when you started at BCC and if you met those expectations through your experiences while attending the college. For example: How much did you expect to improve your communication skills in college and how much did you actually improve these skills?Advantages: Measure the attainment of a goal Efficient Provides interpretation of answers Facilitate easy comparison of answers using the same
scaleDisadvantages: Directions are difficult to write Completion can be frustrating and time consuming
![Page 18: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/18.jpg)
Question Construction: Tips Use simple language Ask answerable questions Be specific in information required Define terms that may be unclear Ask interesting questions Be sensitive to cultural differences Ask questions that are quick to read,
understand, and answer
![Page 19: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/19.jpg)
Tips (continued) Avoid double negatives
Avoid double barrel questions
Avoid yes/no questions when possible
Minimize biased questions
Minimize fence sitting/ floating
Avoid loaded and sensitive questions
![Page 20: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/20.jpg)
What to considering when developing a questionnaire?
![Page 21: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/21.jpg)
Maximizing your response
Factors affecting response rate: The nature of your topic
Your population’s reaction to your topic
Your consideration of your population
The appearance of your study
Survey/Questionnaire length
![Page 22: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/22.jpg)
Formatting Choose first question carefully (i.e. easy to answer,
interesting, general and impersonal).
Order of remaining questions should be grouped logically and flow naturally.
Keep questions addressing similar topic together.
Group questions with the same format together.
Keep use of contingency questions to a minimum.
Save the sensitive, complex, boring, and open-ended questions for last.
![Page 23: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/23.jpg)
Other Considerations Always include directions
Do you need to know who is responding?
Close with a ‘Thank you’ (if applicable, directions on how to return completed questionnaires)
Check for spelling and grammar
Stack responses (esp. for multiple choice questions)
Provide space for your check marks (e.g. O, __, (), [ ])
Leave plenty of ‘white space,’ if possible
Use color paper
![Page 24: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/24.jpg)
From Administration to Reporting
![Page 25: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/25.jpg)
From Administration: A Few Words State the purpose of the survey. Consider an email, cover letter,
introductory script
State how the results will be used
Include promise of confidentiality
Always include a title and identify the sponsor
Consider the length of your collection period
Consider reminders, if applicable
Incentives
Respect your population/respondents
![Page 26: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/26.jpg)
To Results: More Than WordsThe type of instrument (online or paper) will determine the treatment of the completed surveys.
Be prepared to edit(clean) responses Code to lighten your load text to numeric for closed-ended questions thematic coding of open-ended questions
What types of analyses will be done? Frequencies/counts and percentages Crosstabs More complex analyses (statistical)
![Page 27: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/27.jpg)
To Results: More Than WordsReporting Results Identify limitations, if applicable Small samples/responses and the ability to generalize Know/anticipate your audience
Verbal, written or graphic
What is nice to know or what really matters
![Page 28: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/28.jpg)
The Center for Institutional Effectiveness
IR is here to help!
Contact:Tonia McKoy @ [email protected] ext. 5564 Jack Latona @ [email protected] ext. 7209
Room A-335Please come with a draft of your questions!!
![Page 29: Using Surveys for Assessment · 2020-06-30 · What is your question/objective? Know your question/objective- Be specific about what you want to know. Be clear about why you want](https://reader031.vdocuments.net/reader031/viewer/2022041118/5f2e55cad1ada5690d473852/html5/thumbnails/29.jpg)
Questions