focus group workshop steven m. culver, ph.d. associate director office of academic assessment...

27
Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech [email protected] Fall 2009

Upload: casey-doughty

Post on 31-Mar-2015

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Focus Group Workshop

Steven M. Culver, Ph.D.Associate Director

Office of Academic AssessmentVirginia Tech

[email protected] 2009

Page 2: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

A focus group is . . .

Controlled, carefully planned discussion Gathers information about a particular topic Conducted in a non-threatening environment Conducted by a moderator/facilitator Group members influence each other by

responding to ideas, comments of others 7-10 participants

Page 3: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Why a focus group

To collect qualitative data To gather feelings and perceptions regarding

services, programs, products To promote self-disclosure among

participants (note: familiarity inhibits disclosure), though not appropriate for emotionally charged environments

To stimulate interaction among participants to gather more information

Page 4: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Uses of Focus Groups

Improve planning & design of new programs Improve existing programs Recruit new participants Understand decision making processes Generate information for larger studies

Page 5: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Advantages of a focus group

Socially oriented research procedure – more interesting to participants than individual interviews

Format allows moderator to probe High face validity – easily understood More accurate than information from just one

person Low cost Speedy results

Page 6: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Disadvantages of a focus group

Researcher has less control Data more difficult to analyze Need carefully trained interviewer

Can introduce biases May fail to follow up on crucial information

Groups can vary considerably Groups may be difficult to assemble

Page 7: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Produces qualitative information

Can precede quantitative approach Can be used same time as quantitative

approach Can follow quantitative approach Can be used alone

Page 8: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Preparing for the session

Identify major objective(s) of the meeting Develop 5 or 6 questions Plan session – think of maximum time for

session as 1 ½ to 2 hours Invite participants

Be careful of mixing levels of education, authority, etc.

Incentives

Page 9: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Moderator traits

Informed about the topic to be discussed Able to encourage all members to participate Exhibit empathy, but maintain control Able to encourage group members to discuss

in greater detail Able to allow the session to flow smoothly –

be adaptable.

Page 10: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Beginning the Focus Group

Welcome Overview and topic Ground Rules First question

Page 11: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

The Welcome

You are the host. Make participants feel welcome and comfortable.

Much of the success of the group interview is attributable to the development of an open environment.

First few moments of a focus group are critical.

Page 12: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Overview and Topic

Provide your name and who you represent Explain the purpose of the group and how the

data will be collected and used Note that there are no right/wrong answers,

but rather differing opinions, so please share your point of view even if different from what others have said

Amount of time / breaks

Page 13: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Ground Rules

Please speak up – one person at a time We will be on a first-name basis for the

discussion. Talk about note-takers and tape recording if

applicable. Confidentiality – assured from your

perspective and ask participants to respect confidentiality of others when outside the group

Page 14: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

The Questions

Get participants involved as soon as possible Use open-ended questions: What did you think of

the program? Be careful of phrases like “how satisfied” or “to what extent”

Avoid dichotomous, yes/no questions. Avoid “why?” questions – implies cause/effect that

might not exist Use “think back” questions. Take people back to an

experience, not forward to the future

Page 15: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Identifying focus questions

Opening: get people talking and feeling comfortable Introductory: Introduce topic, get people thinking

and connecting with the topic Transition: move conversation into key questions

that guide the study Key questions: those that drive the study Ending: Bring to a close. Use “what is the most

important thing we talked about,” “have we missed anything,” “summarize; is this an adequate summary?”

Page 16: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Types of questions (Patton)

Behaviors – what person has done or is doing

Opinions/values Feelings Knowledge Sensory – what people have seen, touched,

heard Background/demographics – age, gender,

etc.

Page 17: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Keeping it moving

May be helpful to think in terms of time blocks.

Introduction – 10 minutes Purpose – 10 minutes Topics & discussion – 50 minutes Conclusion – 10 minutes

Page 18: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Balancing

Use balancing to help the group round out its discussion rather than just follow the lead of a few

“are there other ways of looking at this?” “what do others think?” “So, we’ve heard x and y points of view, is

there another?”

Page 19: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Encouraging

Encouraging is about creating an opening for people to participate.

“Who else has an idea?” “Is there a student’s perspective on this

idea?” “A lot of men have been talking; let’s hear

from the women.” “Let’s hear from someone who has spoken in

a while.”

Page 20: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Paraphrasing

Paraphrasing helps support people to think out loud, helps clarify, provides calming effect.

“It sounds like what you’re saying is . . .” “Let me see if I’m understanding you . . .”

Page 21: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Tracking/Reviewing

Tracking lets the group see that several elements are being discussed.

First, “I hear three conversations going on right now; I want to make sure I’m tracking them.”

Second, “It sounds like one conversation is about …”

Third, “Am I getting it right?”

Page 22: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Considerations for Analysis

Be aware of the actual words used by the participants and the meaning of those words

Participant responses are triggered by a stimulus—examine the context of that response in that light

Look at frequency/extensiveness of comments – some topics may be more important than others

Consider intensity of the comments Give more weight to specific comments based on

experiences rather than vague, impersonal responses

Page 23: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

The Analysis Process

Begin while still in the group: listen for inconsistent or cryptic comments & probe further

Immediately after: diagram seating arrangement, debrief moderator, note takers, prepare report in question-by-question format

Later: compare/contrast results, look for emerging themes, construct typologies, use quotes to illustrate

Prepare report: narrative style, use quotes to illustrate, share report with the team for verification

Page 24: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Analysis options

Transcript-based analysis Tape-based analysis Note-based analysis Memory-based analysis

Page 25: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Reporting

Purpose is to report views of the group, not to generalize to larger groups

Statement of problem, key questions used Results/findings Summary of themes Limitations Recommendations

Page 26: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Helpful references

These slides based on the following sources: Douglah, M. (2002, October 11). Focus

group workshop. Madison, WI: University of

Wisconsin-Madison. Bledsoe, M. Focus group manual. Daytona

Beach, FL: One Voice for Volusia.

Page 27: Focus Group Workshop Steven M. Culver, Ph.D. Associate Director Office of Academic Assessment Virginia Tech sculver@vt.edu Fall 2009

Other references

Gibbs, A. (1997). Focus groups. Social research update. University of Surrey.

http:/www.soc.surrey.ac.uk/sru/

Krueger, R. A. (1997). Developing questions for

focus groups. London: Sage.

Rubin, H., & Rubin, I. (2004). Qualitative interviewing: The art of hearing data. 2nd ed. Sage.