On The Qualitative Frontline:Fieldwork Recruiters’ Views On

Respondent Cooperation

Prepared by:QRCA Field Committee

March 2010

1

Table of ContentsStudy Overview 3Background 4Study Objectives 5Methodology 6Note of Thanks 8Caveat 9Summary and Implications 10Detailed Discussion 26 Trends in Respondent Cooperation 27 Factors that Affect Respondent Cooperation 30 Recruiter Suggestions for Decreasing Cheating/Repeating 56 Recruiter Suggestions for Increasing Respondent Cooperation 61Appendix A: Moderator’s Guide 73

Page hyperlinks work directly in the Slide Presentation View, otherwise right click on page links above, then click ‘open hyperlink’ to jump directly to a page

2

STUDY OVERVIEW

3

Background• QRCA’s Field Committee formed a Task Force to gain insights into respondent

cooperation in qualitative market research studies by interviewing fieldwork service recruiters. The Committee believes that recruiters are “on the frontline” and, therefore, can be a valuable source of information and insights.

• The goal is to understand what qualitative research consultants (QRCs), fieldwork services (facilities), and clients can do to increase respondent cooperation, by:– Developing best practices for increasing respondent cooperation rates.– Improving communication between QRCs/clients and fieldwork services on recruiting

practices.– Reducing the incidence of cheating and repeating (respondents who lie in order to get

into studies and/or to participate too often).

4

Study Objectives• Specific study objectives are to:

– Identify trends in respondent cooperation – Identify factors that increase or decrease respondent cooperation– Obtain recruiters’ suggestions for increasing respondent cooperation– Obtain recruiters’ suggestions for reducing the incidence of cheating/

repeating

5

Methodology• Five QRCA Field Committee members who are practicing qualitative research

consultants, most of them also facility owners, volunteered for the Task Force. Each conducted 2 groups with recruiters. – Jan Lohs, Lohs Research Group, in association with Smith Research, Chicago, IL– Anndel Martin, Opinions Unlimited, Houston/Dallas, TX– Andrea Schrager, Meadowlands and New York Consumer Centers, New Jersey and New

York– Merrill Shugoll, Shugoll Research, Bethesda, MD– Diane Trotta, Trotta Associates, Los Angeles and Irvine, CA

6

Methodology• A series of 10 focus groups was conducted, divided between:

– Recruiters specializing in consumer recruiting – Recruiters specializing in executive/professional and medical recruiting

• Focus groups were about 90 minutes, with 6-8 respondents each. • Recruiters interviewed were a mix in terms of recruiting experience (minimum 6

months), gender and age.• Recruitment criteria and the moderator’s topic guide, developed by the Task

Force, are appended.

7

Note of Thanks• Special thanks to the following companies for providing transcription services:

Transcription Services, IncTSItranscripts.com909-484-8185

Mark Hampton Transcriptsmarkhampton88@yahoo.com212-595-4083

MCC Field and TranscriptionNadya@consumercenters.com201-537-5256

8

Caveat• The qualitative research methodology seeks to develop direction rather than

quantitatively precise measures. Because of the limited number of respondents involved in this type of research, the study should be regarded as exploratory in nature and the results should be used to generate hypotheses for further testing.

• This study presents views of qualitative recruiters. Comments by the Field Committee are discussed separately.

9

SUMMARY AND IMPLICATIONS

10

A. RECRUITERS’ REPORTS AND SUGGESTIONS

11

Trends in Participation• Several trends are contributing to respondent participation

– Awareness – more people know about market research generally and qualitative in particular, especially focus groups.

– Positive past participation – people who have participated usually have a good experience, are more likely to come again, give referrals, spread word-of-mouth.

– Recession – increases interest because of incentives.

12

Trends in Participation• But certain trends present obstacles to respondent cooperation:

– Continuing lack of research awareness among some consumers and office gatekeepers.– Company policies against employee participation, affecting business-to-business studies.– The recession – more people working longer hours, don’t want to leave office for a

study.– Busy lifestyles – less time for screening, participation. – Cellphone use – harder to reach respondents, safety issues while driving.

13

Factors in Cooperation: Overview

• Recruiters/facilities look at cooperation both in terms of the individual study and keeping respondents in their database long-term.

• The screener plays a major role in cooperation (discussed below).

14

Factors Helping Cooperation• Study topic – those respondents consider interesting, personally relevant.• Incentives – appropriate to the market, respondent segment, amount of pre-work.• Facility contact prior to screening – emails/letters/fax introducing the research

company are a good first step in recruiting.• The actual research experience – respondents who find the interview worthwhile/

enjoyable are open to participating again; seeing results of the research makes them feel they have had an impact (on product development, etc.).

15

Factors Hurting Cooperation: Sampling/Requirements

• Narrow study requirements, quotas – database respondents become frustrated to be called repeatedly but not qualify; some opt out.

• Client list studies – inaccurate information is common; respondents are wary if not told the client name.

• Random digit dialing – people unaware of research are less likely to participate.

16

Factors Hurting Cooperation: Sampling/Requirements

• Homework assignments have become more demanding – resulting in lower acceptance and show rates.

• Incentives – too low for homework required, no incentive for completed homework if respondents are cancelled; respondents refuse if less than prior studies.

• Scheduling – times of day that are difficult for the respondent segment.• Last-minute changes – in interview scheduling, new/revised screener questions.• Confirmation of holds – if not done quickly, respondents make other plans.

17

Factors Hurting Cooperation: Screeners

• Length – many screeners are longer today; with questions “for information” or as a mini-survey, rather than for the purpose of screening.

• Questions that are complicated or poorly written are more common:– Unclear wording. – Run-on sentences.– Long response lists. – Respondents can’t remember what they did a while ago, especially everyday usage. – Redundant questions, repeated verbal scales annoy/offend respondents. – Terminology that is hard to pronounce/understand.– Poorly translated from other languages.

• Organization/flow– Difficult skip patterns– Terminate questions at end – waste respondents’ and recruiters’ time.

18

Cheating/Repeating• Cheaters/repeaters have become more knowledgeable about “gaming the

system” in order to get into studies.– The recession raises facility concerns about respondents doing this for the money, lying

about employment.

• Facility procedures can help reduce cheaters/repeaters.– Electronic monitoring recruiting, confirmation calls checking screener information using a

different recruiter.– Keeping the database updated on participation.

19

Recruiters’ Suggestions: Screeners

• Introduction – be engaging and honest about the study topic/purpose; ask the project manager how to present the subject in an interesting way.

• Respondents desired – talk with the project manager about what you want. • List studies – tell respondents the client name, if possible.• Length – keep screeners short (under 10 minutes at most), ask only questions

needed to recruit.• Flow – make skip patterns and instructions simple/clear.• Terminates – do this on the question after the one where respondents disqualify,

especially on sensitive issues (e.g., age, ethnicity) • Algorithms – use them only if absolutely necessary; recognize they complicate

recruiting and increase costs.

20

Recruiter Suggestions: Screener Questions

• Ask key questions, including security, in an open-ended way, without reading pre-coded answers – more difficult for cheaters/repeaters to game the system.

• Ask children questions in simple, age-appropriate language. • Use numeric scales instead of verbal ones on a list of statements.• Ask articulation questions related to study subject – avoid ones seen as “silly.”• Put terminate questions as close to beginning as possible. • Phonetically spell out difficult terminology.• Translate screeners originally in another language into colloquial English.

21

Recruiter Suggestions: Other Issues

• Requirements/quotas – recognize that studies with very narrow specifications are more difficult, expensive, take longer to recruit; ethnicity is more complex today.

• Incentives – consult with the project manager on what’s appropriate to the market, segment, amount of pre-work required; partial payment for cancelled respondents who complete homework; give a small gift in addition to cash at the end of the interview to create goodwill.

• Confirm holds – do this as soon as possible to retain desired respondents.• Homework – keep it to a minimum, especially for businesspeople; make help

available on weekends for Web assignments; consider cameras as incentives.• Scheduling – ask the project manager for the best interview start times in that

market; consider weekends and holidays for longer interviews (e.g., mock juries) or to recruit certain segments (e.g., students).

• Refreshments – serve meals at standard mealtimes.

22

A. FIELD COMMITTEE COMMENTS

23

Overview• This study with qualitative research recruiters provides a number of very useful

observations and suggestions for QRCs and end clients in planning their fieldwork in today’s world.– QRCs should try to avoid practices that cause recruiting problems and/or price

increases. – QRCs should make their clients aware of the consequences of practices that complicate

recruiting and/or are more costly. • It is important that QRCs/clients work collaboratively with facility project

managers, asking their advice on recruiting and their particular market.• We also strongly recommend that QRCs and facilities follow the guidelines for best

practices in communication and recruiting practices, including email recruitment, on the QRCA website.

24

Better Communication• There are reasons why QRCs/clients do not or cannot follow practices suggested

by recruiters. Explaining the reasons to the facility helps the project manager and recruiters understand the situation. Examples:– Study specifications may be narrow in order to find people relevant to the client.– Homework assignments sometimes need to have multiple parts in order to enrich the

research. – Modifications in respondent qualifications may be necessary after interviews in one

market, such as learning that certain types of people are not true prospects for the client.

– Small changes in an answer (e.g., one rating point) may disqualify a respondent – while this is frustrating, it may be necessary.

– Respondent callbacks to ask additional or revised questions may be needed based on new information/ideas or on the mix of recruited respondents.

– A few terminate questions may need to be toward the end of the screener – e.g., respondents can be put off if household income is asked too early.

25

B. DETAILED DISCUSSION

26

1: TRENDS IN RESPONDENT COOPERATION

27

Respondent Cooperation Trends

• The economic downturn is having a slight positive impact on respondent cooperation. More people are returning recruiters’ calls and agreeing to participate in qualitative research.

“Maybe it has to do with the economy; everyone needs more money.”

“People are more willing to listen to you and take your call. They will put dinner on hold to answer your questions – the kids could be screaming in the background but they keep on going.”

• Negatively, business-to-business studies can be more difficult to recruit today. – Some professionals have heavier-than-usual workloads and are concerned about being

out of the office during the recession.– More companies have policies against employees’ participation in market research.

28

Respondent Cooperation Trends

• Many people remain unfamiliar with market research. They are more likely to disengage from a phone conversation with a recruiter, are skeptical when told that the research company wants their opinions and will compensate them for their time.

“Executive and medical recruiting is more difficult if the gatekeeper is not familiar with qualitative research.”

“They have to be familiar in order to trust you with whatever you are talking about. When you call them, they do not know who you are. You mention money – ‘we’re going to pay you’ – that is a scam in their mind.”

“If you are not allowed to say the company that you are calling on behalf of and they don’t know anything about you, they don’t care; they don’t want to talk to you because they don’t know anything about a focus group. They don’t know about marketing research.”

“’Are you serious! You pay me?’”

“You can’t get past the administrator. They think you are trying to sell something; they are paid to protect the doctor, not let people get through to them.”

29

2: FACTORS IN RESPONDENT COOPERATION

30

Respondents’ Past Experience

• Many respondents have had positive experiences in qualitative studies, which makes them willing to participate again.– They enjoyed expressing their opinions and being heard. Most are anxious the first time

and then find the experience pleasurable.– Seeing an actual outcome of the research (e.g., new product/advertising discussed

come to market) makes respondents excited about having had an impact.

• Respondents expect the same level of incentives in future studies, are less likely to participate when it is lower.

“There are people who have told me [no] if it’s not $100.”

31

Study Topic• Interesting or personally relevant topics attract people to participate. The gratuity

becomes secondary, although it is still important. “I think it depends on the topic. It could be something that’s very interesting to them as a person and that makes them much more interested in participating.”

“They will make an effort to come, even if it’s inconvenient, because they enjoy it [the topic].”

“Some doctors want to learn about new techniques. They’re not just doing it for the money. And they want to get feedback about medications from other doctors.”

• Topics that may not sound inherently interesting can be positioned in a positive (and honest) way.

– Example: a study on “what consumers do for personal grooming” vs. “a study of deodorants.”

32

Study Topic• If the actual interview topic differs from what respondents expected/were told or

is about issues of little interest to them, they may feel misled or be reluctant to participate again.

“Very often, we find out after the fact that the vague topic ended up being an ad campaign. ‘What color or what font should we be using?’ Some doctors can get angry to think they have attended something that was quite different than what they thought it would be.”

33

Screener Introduction• Introductions need to capture respondent interest, provide reassuring information.

– Generic introductions that give no real information about the study topic (e.g., “some

issues”) do not intrigue respondents. – Clients willing to reveal their sponsorship of the study, especially on a list job, sometimes

leave it out of the introduction, which can result in terminates by people who would be interested in participating.

– The recruiter’s tone at the outset also plays a role in encouraging participation.

“I think a good intro is one where we can say within a couple of sentences who we are, what we’re trying to do, and if you qualify we’d like to invite you to participate.”

“The more the qualitative research people can provide you with ‘this is what’s going to be happening in the group,’ the more we can really sell them on the idea of coming to the session.”

“Sometimes it’s not what you say but how you say it. If the topic sounds interesting in your introduction I think you can sell the idea that it will be fun to participate.”

34

Screener Length• Screeners have been growing longer (up to 20 minutes), creating major problems

with participation rates.– More screeners today include questions for general information purposes or as a mini-

survey, which goes beyond screening. – Screeners longer than about 7 minutes can cause respondents to become frustrated, to

lose interest and terminate midway.“Some questions are a whole paragraph, and that’s not necessary. By the time you get to the question, they’ve already forgotten.”

“Respondents sometimes feel like the screener goes on forever and they get lost in the detail.”

• Terminate questions near the end of a screener annoy respondents, waste both their and the recruiters’ time.

“I try to get permission from the project manager to go directly to the termination question [when I begin a screener].”

35

Screener Questions• Simple and direct questions are most easy to answer correctly.

– Long questions with run-on sentences confuse respondents – they don’t know what part of the sentence they should answer.

– A difficult-to-administer screener makes respondents think the study will also be difficult.“Respondents tune out to long lists. They just answer and are not listening to what you are really saying.” “Directions can be so long that [respondents] don’t remember what they are doing.”

36

Screener Questions• Numeric scales with anchor points are simpler, less confusing and more time-

efficient than verbal scales that recruiters have to reread for each statement. – Numeric scale example: “for each statement, choose a number from 1 to 5, where 1

means disagree completely and 5 means agree completely.” Verbal scale example: “for each statement, tell me if you agree completely, agree somewhat, neither agree nor disagree, disagree somewhat, disagree completely.”“You have to make sure that they understand the scale. Sometimes when they are filling out a rescreener before the group, they answer differently and I think it is because they didn’t understand the scale on the phone because it wasn’t logical or explained well.”“When I ask how many times they were at a particular restaurant or drank a beverage in the last year and then drill down to the last month, the numbers change and the respondent gets confused. Start with the last month. That changes when you ask about purchasing a big item like a car or taking a cruise. Those things they can remember.”

37

Screener Questions• Redundant questions are seen as being used to verify answers – they can annoy or

even offend some respondents, who feel they are being tricked. “They say, ‘Do not waste my time.’”

• Skip patterns are often not clear – more screeners now are questionnaires with complicated, hard-to-follow instructions.

“The screeners used to be simpler and they have become more complicated over time. It’s not about ‘keep it simple.’”

• Some screeners originally in foreign languages are not translated well.“With doctors, they use the same screener in Europe, India, China, wherever, and sometimes it doesn’t fit into our language or how we talk.”

• Terminology that is not commonplace lay language (e.g., financial, medical) can be difficult for recruiters to pronounce if they are not spelled out phonetically.

“I had a couple of screeners where it seemed like the screeners were like research papers. Some of them are cumbersome; they’re poorly written. The terminology they use do not encourage a successful completion.”

38

Study Specifications• Increasingly, narrow specifications and quotas mean that fewer people qualify for

study participation. – Respondents in facility databases become discouraged about repeatedly failing to

qualify, and are negative about being screened for other studies. “I’ve had some get very angry on the phone. They say, ‘Why are you wasting my time? I never qualify.’”

39

Algorithms• Algorithms are a major concern – these multi-question formulas have to be

calculated, recruiters cannot determine immediately if a respondent qualifies. – Respondents feel their time has been wasted if they make it through most of the

screener, then don’t qualify because of the algorithm. They may be less willing to be screened for the future studies. “The problem with the algorithm is that the person doing the interview can’t determine if they qualify.”“Respondents do not want to waste a lot more time on the phone while I figure out if they qualify or not. They get annoyed and hang up if the calculation takes too long.”

“It builds frustration. If it happens too often, they just say ‘I don’t want to do this any more.’”

– Some recruiters explain the rationale to respondents (clients look for people based on their attitudes); calculate the algorithm offline, calling back those who qualify. “Once I understood what an algorithm was and why it is important, I could help the respondent feel more comfortable going through the process.”

– Some facilities pre-screen respondents with a form online, screen those who pass.

40

Follow-Up Calls• Last-minute changes in interview scheduling, screener questions and respondent

approval are more common today – these irritate and put off respondents. – Calling respondents back with new questions is seen as an imposition. A small change in

an answer (e.g., one rating point) can result in disqualification.

“Once we are halfway through with the recruit, they’ll [the client] call back with a new question.”

“You feel badly when you have to go through lots of questions that they already answered.”

“It is ridiculous sometimes. Once they give a 7 and then an 8 when re-asked and now they do not qualify.”

41

Confirming Holds• Respondents put “on hold” for participation because they do not totally fit

screening qualifications can become annoyed or make other plans if the client “waits too long” in accepting them for the interview.

“Sometimes, the client won’t get back to the project manager until a day or two before [the session]. We’ve lost holds because they’re not available anymore.”

42

Incentives • Respondent cooperation is largely affected by the amount and type of gratuity.

– The right amount varies by market – the project manager has experience to know what works best in their market with different segments.

– Beyond the basic incentive, it can be useful to pay additional gratuities, such as for completed homework, for parking, and drawings for early arrival at the facility.

– While high-interest topics increase participation rates, this does not mean that lower incentives are a good idea – show rates can be hurt.

“If it’s two hours, and it’s $75, they’re usually not interested. I would say if you have somewhere between $100 to $125, you’ve got them.”

“Some people, that’s the first thing they’ll ask me, ‘Is the gratuity [amount]?’”

“I don’t know why but respondents seem to like it better when we separate the incentive for babysitting or parking rather than telling them one lump sum amount.”

43

Sampling Method• Cold calling through random digit dialing is the most challenging method – many

people contacted may not be familiar with research generally or the company. – Similarly, client list studies on which recruiters are not authorized to name the study

sponsor are difficult – respondents are wary about where the research company got their name/information. “[They] are more suspicious, they don’t know who we are, what we do. You have to give them a level of comfort, or they’ll hang up on you.”“’I’m so-and-so from such-and-such’ – it suddenly sounds like a telemarketing call.” “We need to spend a lot of time explaining the nature of the call, the difference between telemarketing and research to establish their trust.”“A lot of the time they think I’m a bill collector until I tell them about [our company].”

44

Sampling Method• Client lists need to be up-to-date, with full contact information. Many lists have

inaccuracies in addresses, telephone numbers and respondent information.– Recruiters sometimes are not warned that people listed may have negative reactions to

a call and/or to the client.“There are a lot of wrong numbers. It seems like some lists are old.”"It is great when lists have more than one phone number. You can try them during the day or at night.”

“We need to know the type of person on the list so we can calm them down. If they are people who have called to complain, someone with a lousy credit history, they can get angry.”

45

Contacts Before Calls • Emails, letters by mail or a fax introducing the research company and the study

can be a helpful first step in recruiting.– With respondents who know the facility, these steps avoid calls bothering them about a

study they are not qualified for or interested in. “It is best that way. They want to be included if they respond to an email.”

– People on a client list have heard the research company’s name when they are called, making them more receptive.“If we can, emailing ahead of time really helps. They know the name [of the research company] and they will answer my call.”

46

B2B and Consumers• Recruiting B2B respondents is generally more challenging than consumers.

– Businesspeople and professionals are often more difficult to reach, have less time to be screened or to participate in a study.“For business recruits, it’s more challenging because you’re interfering in their day, because it’s hard to get them on the phone to screen them, and then it’s hard to get them to come in. It’s easier with consumers because word-of-mouth and past participation makes them want to do it again.”

“You need to get through, you have to work your way through the people, the receptionist.”

• Consumers are easier to recruit than businesspeople, once they are contacted, because of experiences in previous studies or word-of-mouth recommendations.– Characteristics like age and gender do not seem to make a major difference in

respondent cooperation but do play a role in show rates.“It’s not that younger/older is easier or harder to recruit, it depends on whether the topic relates to them or not.”

47

Type of Interview• Most respondents are willing to participate in a range of interview types.

– Businesspeople /professionals have divided preferences for depth interviews and focus groups. Depth interviews tend to be shorter and have greater flexibility in scheduling, giving respondents choice of the time. Focus groups give them the opportunity to hear others’ opinions and learn from the interaction.“Sometimes they say they only want to do groups – especially if the topic is interesting or they get to learn about new products from others.”

• Respondent concerns about ethnography home visits were noted by some recruiters. – Even respondents who know/trust the field service can be hesitant about strangers

visiting; researchers coming to see how they live seems foreign/“weird” to some. – Respondents want to know details, e.g., who will be visiting, the visit purpose. Securing

cooperation is especially difficult when recruiters cannot tell them. Some women will not allow males into their home unless their husband or a friend is present. “They’re like, ‘You want to come in and watch my family eat dinner? I don’t feel comfortable with that.’”

“It is the fear factor. They know my company but do not know the person coming. They sometimes ask if I can be there too.”

“They are just uncomfortable and it is even worse if more than one person is going. They feel overpowered in their space.”

48

Type of Interview• Store shopalongs are easier to recruit than home visits but some respondents

express concerns that they might be asked to make purchases. “I get asked these days if I have to purchase things. They say they just shopped and do not want to spend more money on things they do not need at the moment.“

49

Time of Day• Interviews scheduled at times when respondents are busy/unavailable make

recruiting far more difficult. “In the old days, it was what was convenient for the respondent and now it’s what’s convenient for the researcher or the clients.”

• Best start times for early evening interviews vary by market.– Traffic patterns and distance to the facility from the highway or downtown area need to be considered.

“If a focus group is scheduled any time around rush hour, it is a problem for some to come.”

“For respondents who live or work far away from the facility, getting to a 6pm group can be very challenging during rush hour traffic.”

50

Time of Day• Demographic segments have different time preferences that should be

considered; these also vary by market.– Elderly respondents generally prefer daytime or early evening to 8pm interviews. – Businesspeople and professionals generally prefer early morning and evening for focus

groups. Midday can work for shorter IDIs but not for two-hour focus groups.“Maybe a year ago doctors were willing to reschedule a patient. Now they are not as willing to tweak their schedule in fear of losing a patient.” “We find that many doctors do not see patients on Wednesdays or Fridays so those days work well.”

• Serving a meal near/around typical mealtimes encourages cooperation. “For a 4:00 pm group we are asked, ‘You are serving dinner, aren’t you?’”

51

Day of the Week• Certain days of the week can be a problem in some communities (e.g., church

night) – the project manager can advise on this.• Weekends and school holidays may be preferred by some consumers, especially

for longer interviews and mock trials – their schedules may more flexible then.– However, some respondents, researchers and facilities do not like these times.

52

Pre-Interview Homework• “Homework” assignments are increasingly used today. While a number of

respondents find these interesting, they can make recruiting more difficult and require more recruiting time. – Certain professionals dislike or won’t do homework assignments, according to some

recruiters, while other recruiters see no problem.“The doctors don’t like doing homework at all. They did enough of that in school.”

“I actually heard a doctor ask his nurse to pull ads from magazines for him for his homework when called to remind him to bring it in. I tried to stop him but do not know who actually did it.”

“I don’t think they mind doing the homework. They’ll laugh about it, but for the most part, they’re all right with it.”

– “One-step” homework is easier to “sell” to respondents than multiple tasks.“If we have jobs with homework, we should allow at least one weekend because they don’t have a lot of time for themselves. If we have a turnaround of a week, then they don’t have time to do it.”

“I think they (respondents) like to do homework sometimes if it is fun and not too much work. A 50-page diary is way too much work; [sometimes] they don’t show up because they didn’t do it.”

53

Pre-Interview Homework• Homework that is not clearly spelled out, is more difficult or takes more

time/effort than promised may lead to respondent frustration, cancellations and no-shows. – Similarly, incentives that are seen as insufficient for the assignment are a problem.

“[On a recent project] respondents were angry. They said ‘even though you told me 3 to 4 hours, it’s taking me 5 or 6.’”

“I have had people drop out of the study because the homework is much more involved than either I or the respondent expected.”

• Incentives that increase cooperation include a still or video camera for doing the homework, a bonus for the most complete homework.

• Cancelled respondents who complete and submit homework become upset, are less likely to cooperate on future studies. – Offering partial incentives in these cases promotes cooperation.

54

Cancellations• Last-minute cancellations create negative feelings and affect future cooperation.

– Client changes in study specifications near the interview date can result in the cancellation of some respondents. This is often done without explanation or compensation. Respondents who adjusted their schedules to be available resent the short notice. “The client changes the specifications and tells us to replace respondents. The client may have numerous reasons why and they don’t want to tell the respondents why we can’t use them.” “That leaves a bad taste in their mouths. The next time we call to recruit them, they’re going to be skeptical and not take us seriously. They need advance notice if they’re going to be cancelled.”“We feel that these people should get paid if cancelled at the last minute. Often they have rescheduled other meetings around our schedule.”

55

3: RECRUITER SUGGESTIONS FOR DECREASING CHEATING AND REPEATING

56

Screener Questions • Cheaters who lie to get into qualitative research studies and repeaters who

participate too often are a major concern for the industry as a whole.• Closed-ended questions make it easier for respondents to guess how they can

qualify for a study. Some say “no” to all occupations listed in the security question and “no” to past 6-month participation in order to be recruited. – Questions in which answers are pre-coded but not read aloud are more difficult for

these respondents to figure out.

“Some of our respondents are experts on qualifying themselves because if you give them a range, they’ll pick the biggest range.”

“They tend to cheat on income because income is usually a scale.”

“Especially when they’re from the database, then they know [to say “no” to each occupation].”

“A lot of them will fudge on their answers. A lot of them know it’s 6 months [participation] and they say, ‘I don’t really know’ and they’ll get it wrong.”

57

Screener Questions• Terminating respondents right after a terminate answer enables some to guess

why they do not qualify for the study. This can be sensitive in the case of demographic quotas.

“I don’t tell them that they’re not qualified. I say ‘that quota’s filled and if it changes then we’ll call you back’ so they’re not thinking about why they didn’t qualify.”

58

Preventing Cheaters/Repeaters

• Facility monitoring of recruiting and confirmation calls help to identify respondents falsifying information and ensures that recruiters follow procedures.– A facility staff member other than the original recruiter should make the confirmation

calls to double check recruiting.“It is funny when they ask, ‘What am I supposed to say’ or ‘what did I say the last time?’ And we say, ‘Well, just answer the truth.’ That is a clue to block them in our system.”

59

Preventing Cheaters/Repeaters

• Keeping the facility database updated is crucial.– A list of cheaters/repeaters (“professional respondents”) and those who are suspect

(e.g., arrive late on repeated occasions) is used by recruiters. “Keeping the database updated and being on the lookout for names you recognize. You know these

names are going to pop in your head after a couple of calls.”

“A person will call in with a different name and the same number. We’ll put notes on that person’s profile and the database manager will delete them from the database.”

“We make a list of people who are professional respondents and keep the list on our desk.”

“ We share information with other facilities when we find a ‘nest of phony respondents.’”

“We keep them in our database and flag them so that they can’t sign up again. We don’t call them. We don’t recruit people who call us on the phone asking about studies, Everyone has to be checked in our database.”

“There are also problems with people who are scamming the system, coming only at the time when they will be paid and sent.”

60

4: RECRUITER SUGGESTIONS FOR INCREASING RESPONDENT COOPERATION

61

Recruiting Techniques• Good recruiters play a major role in encouraging cooperation:

– Friendly/pleasant yet professional manner.– Convey a sense of credibility. – Develop rapport quickly, engage respondents in conversation, but avoid talking too

much, which can sound “sales-y.”– Upbeat tone of voice, communicating that the study will be interesting/enjoyable.– Listen to respondents, make them feel special/valued.– Demeanor appropriate to the study and respondent (more formal with businesspeople).– Display confidence without coming across too strong.

62

Recruiting Techniques• Recruiter manner is professional and personable.

“You have to know what you’re talking about because you have to give them confidence. They have to feel that this is something that would interest them, that you’re calling them because they’re important and they’re one of the few people that might be able to contribute to your study, you have to make them feel special.”

“Be confident. You’ve got to sound like you know what you’re talking about, speak clearly, and try to be professional.”

“You have to dig in and sell yourself and you have to put that smile in your voice in order to get through.”

• Good recruiters follow interviewing protocols:– Read screener questions verbatim.– Never lead the respondent.

63

Study Requirements• Recruiters stressed that the QRC/client needs to fully explain the screener and

study goals. – A written explanation of the type of respondent the client is seeking is very helpful – the

screener questions and specifications may not be sufficient. “I think the client has a part to play in us being informed about the screener. Sometimes we get screeners and they’re a little obscure, they’re not clear.”

“All project managers should understand their screeners, but they don’t.”“Sometimes even our client contact does not understand the questions.”“I have worked with one project manager who really reads the screener and writes notes. If [something is unclear], he’ll go over it with you.”

“I think that we are sometimes the best people to find mistakes in screeners. Maybe the skip pattern is wrong or terminates are not marked. We need to understand in broad strokes who they are looking for so we can see the logic in the screener.”

64

Email Blasts• Email blasts to the facility database and/or a client-provided list can be very

effective in pre-screening respondents.– Short online surveys, such as Survey Monkey, can be used. – Respondents who pass the initial stage are then formally screened. – This approach is especially useful on low incidence studies and with younger people. – The emails should be worded carefully to avoid biasing respondents.

“With the e-blast you know that this person may potentially qualify versus just pulling from demographics in the database.”“It’s easier because you get more people calling in than calling out.”“With young people, it’s better to email them than call them.”

65

Screener Introductions• Introductions are very important in encouraging respondent participate, especially

on contacts with people not in the facility database. – Explain the purpose of the call and the study clearly/concisely, being as specific as

possible about who, what, when, where and why, etc. – The study descriptions should sound interesting – the project manager can give advice

on wording.

“I think the opening is the most important part. You must tell them when the group is, what the subject is, and that you’re not trying to sell them anything.”

“I have to understand what they’re trying to get and then shorten it to let the respondent know that it’s something interesting to them.”

66

Screener Wording/Length• Keep the screener short (under 10 minutes), place terminate questions early, make

skip patterns clear. – The screener should be only/mainly questions needed for recruiting, not extraneous

information. If needed, a brief questionnaire can be filled out in the waiting room.

• Recognize that memories of recent experiences are better than ones a while ago.• For children’s research, use simple questions with age-appropriate wording.

– “Do you like SpongeBob a lot or a little?” vs. “on a scale of 1 to 5 where 1 means [etc.]”– Ask the child first. Parents do not always know what their child watches on TV or what

characters they like. Do not allow parents to prompt the child.

67

Screener Questions• Articulation questions should be engaging and appropriate to the study, not

threatening or annoying. – Respondents can be put off or intimidated by questions requiring knowledge on a

subject irrelevant to the research. – Certain “creative” questions are considered “dumb” by some respondents (e.g., “name

10 things you can do with a paperclip”). • Ethnicity questions and quotas need to keep up with the times.

– It is harder to find “Caucasians” now because so many people are a mix. – People with Hispanic names may be different from people living the Hispanic culture.

“The best example in the world is Tiger Woods. What ethnicity do you call him? Better yet, what ethnicity do you call his children?” “The client asked for no more than 2 ‘Others’ for ethnicity. Everyone is an ‘Other’ these days.“I wish they would tell us what they are looking for. I am Hispanic because my great grandfather on my mother’s side came from South America. I am not Hispanic in the same sense as some screeners ask for.”

68

Screener Algorithms• Algorithms should be avoided if possible. If they must be used, explain the system

to respondents at the start or ask them online in a pre-screener, and keep the rest of the screener short.

“I tell them upfront that their answers will go through a computer-generated program after they answer the questions so they are not surprised when I say I need to call them back. It is a pain though.”

69

Homework• Keep homework to a minimum, especially for businesspeople/professionals.• Provide appropriate incentives for the pre-work.

– If it is necessary to cancel a respondent who completed homework, pay partial incentive.

• If daily Web-based homework is required, provide a telephone number and/or email address for the facility and respondents if problems arise on a weekend.

“I had people just drop out after a few days because the website did not work. They were good people but a bad site.”

70

Other Suggestions• Location:

– Choose a facility or other place with reasonable time for desired respondents.

• Interview time– Schedule interviews for respondents’ convenience.

– Consider weekends and holidays for longer interviews and certain segments. • Specifications/quotas

– If possible, use broader specifications so recruiting is not so restrictive.

71

Other Suggestions• Incentives

– Compensate respondents fairly for their time and effort. – A small gift in addition to cash incentives is appreciated; these should be given out at

the end of the interview. • Confirmations

– Let respondents on hold know as soon as possible if they are confirmed.

72

APPENDIX:MODERATOR’S GUIDE

73

Moderator’s Topic Guide• Group 1 - Professional B2B • Group 2 - Consumer Recruiters• Introduction

– Purpose of group– Who I am and what I do– What you must do – speak to me

• Ground Rules– Audio taping, speak up and loud, one way mirror, observers

• Respondent Introductions– Your first name– Length of time working as a recruiter/working at (company) as a recruiter – Primarily recruit consumer/business– Purpose: determine how, if at all, respondent cooperation rates for qualitative research

have changed and how they could be improved

74

Moderator’s Topic Guide• Trends in Respondent Cooperation

– Do respondents seem more or less willing to participate in qualitative research now or a year ago, or is there no change?

– If a change, what do you feel are the reasons for change?– Has the economy had an impact on current respondent cooperation rates?

• Identify Factors that Affect Respondent Cooperation– What factors make it easier or more difficult to gain respondent cooperation?– Identify factors that most impact cooperation rates for consumers/business professionals and

how.

• Type of recruit impact on cooperation rates – Database, list, or random digit dial, blind recruit versus client identified, etc. – Any impact of telemarketing, fundraising, “do not call” on respondent cooperation.

• Respondent type needed impact How cooperation rates vary based on age, race, economic status, occupation, experience level, etc.

• Interview type impact Focus group, IDI, shop-a-longs, ethnographies, etc.

75

Moderator’s Topic Guide– Screener impact

Length of screener, type of introduction, type of invitation, type and order of questions asked, use of algorithms, etc.

Specific screener features that you believe increase or decrease cooperation.

– Time impact Length of the interview (2-hour group, 1-hour IDI, several-hour shop-a-long or ethnography,

etc.) Time of day/evening the interview is scheduled, etc.

– Topic impact – Homework projects impact

By type of homework

– Incentives/honoraria Gift, check, cash or value/amount of money, etc.

76

Moderator’s Topic Guide– Recruiter style impact

Do certain telephone personality styles increase or decrease cooperation rates? Role of recruiter experience.

– Initial email contact impact – Respondents’ experience in previous research impact

Do respondents differentiate between survey research and qualitative; if so, how? What, if anything, respondents tell recruiters about their experiences. How not qualifying for a previous qualitative research study impacts respondent cooperation rates.

• Obtain Recruiters’ Suggestions for Decreasing the Incidence of Cheating/Repeating– Do you ever suspect that a respondent is not answering the questions on a screener honestly;

if so, how?

• Obtain Recruiters’ Suggestions for Increasing Respondent Cooperation Rates– For list/random dial recruits, for consumer recruits, for business executive/professional

recruits.– Suggestions for screener design, recruiter training programs, etc.– Prioritize the one or two most important things qualitative researchers could do to improve

rates.

77