ASKING THE RIGHT QUESTIONS IN EVALUATIONS TO PROMOTE CHANGEStephen Ashton, Ph.D.; Thanksgiving Point Institute

Kari Nelson, MA MS; Museum of Peoples and Cultures

Purpose

• Understand the purpose of evaluation questions

• Learn basic principles for drafting surveys and survey questions

• Discuss ways to distribute surveys

• Practice drafting evaluation and survey questions

“The evaluation questions form the basis for . . . knowing what to do in the evaluation.”

Marvin C. Alkin

Evaluation Essentials: From A to Z

Evaluation Questions

Evaluation Questions

• The “BIG” questions

– Inform your practice

– What decisions do you need to make?

– Meeting goals and outcomes?

– What’s working and what’s not?

• Involve stakeholders

• Ask: How will you use the information?

SURVEYS ARE NOT THE END-ALL, BE-ALL.

…BUT THEY ARE HELPFUL.

Evaluation Methods

1. Surveys

2. Observations

3. Interviews

4. Timing and tracking studies

5. Focus groups

6. Document analysis

7. Experiments

8. Etc.

Survey Benefits and Disadvantages

Benefits

• Good for general feedback

• Relatively quick

• Convenient

• Inexpensive

• Can be quantitative and qualitative

Disadvantages

• May not get to the heart of an issue

• Does not tell you everything

• Requires sufficient sample

IT’S SURVEY TIME!

Question 1

People like museums that have interactive elements, good interpretations, and friendly staff. Tell me a little bit about this museum.

• The question is leading and is looking for specific feedback. It needs to be more open-ended.

Question 2

What is your annual household income?

• The question is actually good, but the placement is bad.

• It is best to ask demographic questions at the end, once you have “gained the trust” of the participant.

Question 3

I wish this museum had more docents or more interpretive signage.

• Do not ask about more than one thing. Generally avoid using “and” and “or.”

• Instead split the question into two separate questions.

Question 4

I would not recommend this museum to a friend.

• Avoid using “not” or other negatives. It makes the question difficult to understand.

• Sometimes people skip the “not” when they read the question.

Question 5

How would you rate the helpfulness of museum staff on a scale of 1 to 10 . . .

• It is often best to label each of the points on a scale with a description (e.g. very satisfied, satisfied, etc.), because we may each interpret the meanings of the numbers differently.

• Follow up with “Why?”

Question 5 continued

There are exceptions to this rule:

Net Promoter Score (NPS)

Question 6

Did you purchase any of the following…

(Check all that apply.)

• Too long and too many can lead to survey fatigue.

• Don’t extend a question to two pages.

• Consider using a table with Yes/No.

Question 7

The Water Play exhibit was interactive.

• Keep the scales going the same direction for the entire survey.

• Most often surveys go from negative to positive. It’s ok to go the other way as long as the whole survey is consistent.

Question 8

How would you rate your interactions with the museum staff?

• Balance the scale: equal number of negative and positive categories.

Question 9

How satisfied were you with your overall museum experience?

• This one is outright terrible.

• Like car dealership satisfaction surveys.

Question 10

How old are you?

• Categories should be mutually exclusive, meaning there is no overlap.

Question 11

Please rate the food in the Awesomeness Café.

• Make sure the question is applicable for everyone.

• Where needed, include “Not applicable” or “I did not eat at the café.”

Question 12

What is your favorite color?

Only ask it if you really need it.

SURVEY DISTRIBUTION

Paper vs. Computer

Paper

1. Inexpensive

2. May be more comfortable to create and take

3. Must manually input data into database

4. Must take it in the moment

Computer

1. Can be expensive for nice packages

2. Takes some technical skills

3. Participant does all the data entering

4. Can take it when it’s convenient

More Reasons to Consider Going Online

1. Website does all the data managing

2. Can easily send survey link in emails, post to Facebook, or create QR code

3. More fruitful open-ended comments (typing is faster)

4. Higher response rates

5. Can be longer

6. Consider making it a “requirement” for program participation

Online Survey Platforms

…or just search the web for “online surveys”

PILOT THE SURVEY

Survey Writing Activity

1. Write down an evaluation question you have about your museum

– Such as: What do our visitors learn as a result of visiting Exhibit X?

2. Draft 2-3 survey questions that would help you answer your evaluation question

3. Share your questions with those around you for feedback

QUESTIONS?

Stephen Ashton, Ph.D.sashton@thanksgivingpoint.org

Kari Nelson, MA MSkari.ross.nelson@gmail.com

Please put your name on the email list (or leave your business card) if you want additional survey resources, including copies of the presentation and Poor Sample Survey.