national rusa institute: “serving 21st century users: opportunities & challenges”
DESCRIPTION
Reference Assessment Programs: Evaluating Current and Future Reference Services INTERVIEWS & FOCUS GROUPS. National RUSA Institute: “Serving 21st Century Users: Opportunities & Challenges” Marie L. Radford, Ph.D. Associate Professor, School of Information & Library Science, Pratt Institute - PowerPoint PPT PresentationTRANSCRIPT
1
Reference Assessment Programs: Evaluating Current and Future
Reference Services
INTERVIEWS & FOCUS GROUPS
National RUSA Institute:
“Serving 21st Century Users: Opportunities & Challenges”
Marie L. Radford, Ph.D.Associate Professor,
School of Information & Library Science, Pratt Institute
October 12-14, 2000
Omni Inner Harbor Hotel
Baltimore, MD
Introductions
• Presenter- Marie L. Radford, Ph.D.
Associate Professor, School of Information & Library Science
Pratt Institute,
200 Willoughby Ave.
Brooklyn, NY 11205-3897
(718) 636-3512 (o)
(718) 636-3733 (fax)
• Workshop Participants
AGENDA
• Interviews – Types of interviews– Questioning types– Listening effectively– Advantages & Uses– Disadvantages– Planning & Conducting Interviews
• Focus Groups – Advantages & Uses– Disadvantages– Planning & Conducting FGs
• Analysis• Reporting Results• Limitations
4
INTERVIEWS
• What is an Interview?
“An interview is a conversation involving two or more people, which is guided by a predetermined purpose.”
(Lederman, 1996, p. 5)
5
TYPES of INTERVIEWS
• Person-to-Person with Individuals
• Telephone• Focus Groups
KEY COMPONENTS
Good Questions
Good Listening Skills
Good Interpersonal Skills
7
TYPES OF QUESTIONS
• 4 valuable types for information gathering interviews– OPEN
• “What is it like when you visit the library?”
– DIRECTIVE• “What happened when you asked for
help at the reference desk?”
– REFLECTIVE• “It sounds like you had trouble with
the online catalog?”
– CLOSED• “Have I covered everything you wanted
to say?”
8
NEUTRAL, LEADING, AND LOADED QUESTIONS
• NEUTRAL– “What are your impressions of
the M. L. Radford Memorial Library?”
• LEADING– “You don’t like the librarians at
the M. L. Radford Memorial Library, do you?”
• LOADED– “How many other unreasonable
requests have you made to the reference librarians?”
Questioning Do’s & Don’ts
DO’S Be clear Be focused 1 idea per
question Use specific
questions Give sufficient
time PRETEST!
DON’TS Hypotheticals Wandering
interviewee Too many
choices Ambiguous or
abstract questions
Leading or loaded questions
10
Effective ListeningDo’s & Don’ts
DO’S• Pay attention• Listen actively• Invite talk• Open mind• Supportive
atmosphere• Show
acceptance & understanding
• Paraphrase• Note NV cues
DON’TS
• Rush to fill silences
• Give opinions
• Be defensive
• Forget to listen for feelings
• Hesitate to clarify
• Glaze over
11
In-Person Interviews: Advantages & Uses• Face-2-face interaction
• In-depth info.
• Understand experiences & meanings
• Highlight individual’s voice
• Preliminary information to “triangulate”
• Control of sampling
• Greater range of topics
12
Telephone Interviews
• 3 Types of Sampling1. All who phone are asked to
participate in a short interview immediately
2. Phoning users are asked to participate in a call-back interview
3. Random dialing of community members (samples non-users)
• Disadvantages– Not all users have phones– Nonverbal cues missing– Follow-up may be difficult
13
Interviews: Disadvantages
• Time Factors– Varies by # & depth– Transcription: 1.5 hrs. per hr.– Lots of prep. & administration
• Cost Factors– Higher the #, higher the cost– Training interviewers– Tape purchase & transcription
• Additional Factors– Self-report data– Errors in transcription or note
taking possible
14
Library Applications
• Academic– Reference
Encounter
• Public– Family
Place Project
15
ACADEMICThe Reference Encounter
(Radford, 1999)
• Interviews & observation
• 3 sites– Community College– Undergraduate College– Research University
• Qualitative methodology to capture complexity & depth
• 27 pairs of librarians/users
• Analysis– Critical Incident– Paired Perceptions– Categories
16
PUBLIC
Family Place Project (Middle Country Public Library, Centereach, NY & Libraries For the
Future, NYC) • Interviews & questionnaire
• 5 sites– 3 Urban– 1 Suburban– 1 Rural
• Qualitative methodology to evaluate change process
• Analysis– Critical Incident– Categories
17
Phase I: Planning
• Determine purpose of interviews in overall assessment
• Formulate Q’s & pretest
• Identify: Interviewees & Interviewers (& train ‘em)
• Choose notes or tape recorder
• Decide when & where
• Decide f-2-f or telephone
• Plan schedule of interviews
• Choose method of data analysis
• Identify time & private place
18
Phase II: Conducting Interviews
• Be on time & don’t waste their time• Obtain permission to use info. (report
and/or publication) & if taping• If taping check equipment & have
back-up• Create safe climate, assure
confidentiality• Be prepared, flexible, & stay on task• Listen & know when to probe• Accept that some interviews won’t go
well• Thank them!
19
Focus Group Interviews
• What is a Focus Group?
“A focus group is an in-depth, face-to-face interview of a group of 8 to 12 people representing some target group and centered on a single topic.”
(Zweizig, Johnson, Robbins, & Besant, 1996)
20
Focus Groups: Advantages & Uses
• Advantages of individual interviews plus…
• Take less time
• Creates synergy:– Comments stimulate others
– Unexpected insights
– More complete info.
– Elicit strong & common opinions
– Less inhibiting, less formal
• Access needs of under-served or underrepresented groups
• Control of sampling
21
Focus Groups: Disadvantages
• Skilled moderator required
• Participants may be too quiet or too outspoken
• Analysis, summarization & interpretation of responses may be difficult
• Loads of planning & administration time
• Transcription of tapes is time consuming & costly
22
Library Applications
• ACADEMIC– Assessment of
AV Delivery Service
– Undergraduate College
– Focus Groups of students, faculty, users and non-users of service
– Used to develop survey to evaluate and improve service
• PUBLIC– Preparation for
service excellence training
– Urban library system
– Focus Groups of staff (mixed librarians & non librarians) and users
– Used to develop survey to inform training
23
Phase I: Planning
• Determine role of focus group study in overall assessment
• Identify: – target group(s)– moderator & assistant
• Decide # of groups (3-4 groups per targeted group)
• Develop interview guide (& pretest)• Plan schedule (1.5-2 hours each), when,
& where• Choose notes or tape recorder• Choose method of data analysis• Con’t...
24
More Planning
• RECRUIT VOLUNTEERS– Identify pool of potentials
– Plan for 8-12 per group (over-recruit and remind)
– Offer reward if possible (is not $, free ILLs, photocopying, food)
– Try to get representative members
– Form homogeneous groups:
• Academic (e.g., faculty, undergrads, grad. students)
• Public (e.g., adults, teenagers, non-users)
25
Moderator Characteristics
• Excellent communication skills• Experience in group dynamics• Knows when to probe• Restrained• Ability to:
– Involve all participants– Ask neutral questions– Quickly establish rapport– Summarize areas of
agreement/disagreement– Direct but not control discussion
26
Phase II: Conducting Focus Groups• Be on time, don’t waste time
• Bring/check supplies (flip chart, markers, masking tape, etc.)
• Obtain permission to use info. (report and/or publication) & if taping
• If taping check equipment & have back-up
• Begin by creating safe climate
• More...
27
Conducting Focus Groups Continued
• Use effective listening
• Help quiet people talk, limit talkative people
• Design well constructed guide, including:– Introduction (purpose, ground
rules)– Ice breaker or warm-up set of
questions– Relevant major/probe questions– Summary or closing
• Thank them!
28
Interviews & Focus GroupsPhase III: Analysis
• Listen/transcribe tapes, review notes• Ways to analyze data:
– Code data into pre-determined categories
– Use data to identify categories
– Use data as basis for summary statements “capture the essence”
– Interpret the data- intensive analytic method
• Compare all interviews/focus groups• Compile and summarize• Look for trends or problems to study• Do not overgeneralize from results!
29
Interviews & Focus GroupsPhase IV: Reporting Results
• Parts of Report– Statement of purpose– List of issues purpose– Explanation of how data was collected
and analyzed– Summary of findings: What was said
on each issue– Sample quotes (anonymous)– Interviewer’s or moderator’s
impressions– Recommendations
• Short term (low hanging fruit)
• Long term
30
Limitations:Interviews & Focus Groups
• Difficult to generalize• May not be representative • Analyzes perceptions, not facts• Subjective analysis (reliability requires
more than 1 person)• Raw data could be misleading• Results can be limited by:
– Poor/inexperienced moderator/ interviewer
– Poorly constructed discussion guide/interview questions
• Compensate by combining methods (survey/questionnaires/ observation)
31
Summary & More Information
• Where to get more information
• List of books, articles, in packet.