THE TAILORED DESIGN THE TAILORED DESIGN METHODMETHOD

Damon Burton Damon Burton

University of IdahoUniversity of Idaho

SURVEY EXAMPLESJob Survey – A federal agency surveys businesses monthly on their number of employees, but the short survey must be completed in two weeks.

TV Survey – Nielson Ratings collect people’s viewing habits each week in each of several hundred television markets.

Conference Survey – AASP is a professional sport psychology organization wants to find out how conference attendees like the new programming format.

TAILORED DESIGN METHOD

Tailored Design involves using multiple motivational features in compatible and supportive ways to encourage high quantity and quality of response. Social Exchange Theory hypothesizes that respondent behavior is motivated by the return or reward that behavior is expected to bring from others.Survey response rate and quality is enhanced when respondents trust that expected rewards will outweigh anticipated costs.

TAILORED DESIGN METHOD (TDM-2) FUNDAMENTALS

TDM-2 is a scientific approach to surveying that attempts to reduce 4 sources of error (i.e., coverage, sampling, nonresponse & measurement). TDM-2 uses a set of survey procedures that work together to encourage the sample population to respond to the survey.Tailoring survey procedures build positive social exchange and encourages high quality responses from the sample surveyed.

REDUCING COVERAGE ERROR

Well-designed surveys minimize coverage, sampling, nonresponse and measurement error. “Coverage error” occurs when not all members of the population have a known, nonzero chance of being included in the sample.Coverage error can occur because the choice of survey mode may not provide adequate coverage of the population (e.g., internet). The list where the sample is drawn may also not include everyone in the population (e.g., tax rolls or utility bills).

REDUCING SAMPLING ERROR

“Sampling error” is the limit to precision of survey estimates due to not every person in the population being sampled. Random sampling is a technique that allows great precision in estimating the behavior of the entire population from a small sample.Randomly surveying 100 people give estimates with a +/- 10% margin of error (MOE), and sampling 2000 increases MOE to +/- 2%. Smaller random samples decrease costs but at the cost of inevitable error in the estimate.

REDUCING NONRESPONSE ERROR

“Nonresponse error” is due to not everyone who is sampled responds to the survey request. “Nonresponse error” occurs when people selected for the survey who do not respond are different from those who do in ways important to the study.For example, if Democrats are much more likely not to respond to a presidential voting survey than Republicans, then nonresponse error may seriously increase survey inaccuracy. Surveyers must motivate most people sampled to respond to the survey so all sociodemographic groups are represented.

REDUCING MEASUREMENT ERROR

“Measurement error” occurs when the respondent’s answer is inaccurate or imprecise. Measurement error is often the result of poor question wording or design and other aspects of questionnaire construction.Common problems include (a) questions that are too long, (b) too high question readability level, (c) compound sentences, (d) confusing wording and (e) cliches.Survey researchers must monitor item length, readability, focus and simplicity if they want to reduce measurement error.

ECONOMIC EXCHANGE VIEW OF SURVEY RESPONSE

“Economic exchange” is the view that people want something in return for completing a survey, and it attempts to determine how to provide a reward and/or benefit that will prompt survey completion.

People’s “price point” varies widely.

Payment is typically not feasible unless you have a grant or other funding.

Small token cash payments up front are better than large payments following survey completion.

Many other factors influence survey response rates.

MODELS OF SURVEY RESPONSE

People are motivated for many different intrinsic and extrinsic reasons. Cialdini (1984) argues that (a) scarcity of opportunity, (b) consistency with previous behavior, (c) desire to reciprocate, (d) task enjoyment, and (e) social proof (i.e., what other people do) influence survey response. Comley (2006) used transactional analysis to emphasize creating adult-to-adult rather than adult-to-child interactions.

LEVERAGE-SALIENCY THEORY

Groves, Singer & Corning (2000) propose that respondents are motivated to respond by different aspects of the survey (i.e., leverage) and the emphasis put of each aspect by the surveyer (i.e., salience). For example, cash may be important to some, while for others, topic, sponsorship, personal interest or community involvement are more critical factors to response. How much is the topic emphasized and how much do respondents relate to the topic?The value of letters from sponsoring agencies depends on the agency’s creditability with the respondent.

LEVERAGE-SALIENCY THEORY

Leverage-saliency theory suggests that overemphasis on a single appeal may produce a serious nonresponse error. Groves, Presser & Dipko (2004) found topic-based appeals increased response rates for people interested in the topic, overall results were mixed. Implications of this research is that it is important to use broad, multifaceted appeals to encourage all segments of the population to respond.

SOCIAL EXCHANGE THEORY

“Social exchange” hypothesizes that people’s voluntary actions are motivated by the return that these actions are expected to bring from others. Social exchange differs from economic exchange because there is only a general expectation of return to the respondent and various types of rewards and costs are involved. Social exchanges don’t specify the exact nature of the benefits or when they will be provided up front.Small token incentives up front (i.e., SE) increase response rates compared to large cash payments following survey completion (i.e., EE).

SOCIAL EXCHANGE SURVEY DESIGN QUESTIONS

How can the perceived rewards for responding be increased?How can the perceived costs of responding be reduced?How can trust be established so that people believe the rewards will outweigh the costs of responding?

STRATEGIES TO INCREASE PARTICIPATION BENEFITS

Respondents are motivated to act by the benefits they expect to receive.The more survey requests received, the more benefits are reduced?Survey researchers must distinguish their survey from the rest by emphasizing that it offers more benefits.9 strategies can be used to increase perceived participation benefits.

BENEFITS STRATEGIES

1. Provide Information about Survey – how can results benefit respondents and others.

Prenotice letters and brochures that explain why survey is being conducted and the importance of participation.

2. Ask for Help or Advice – Helping others is a reward for many.

Appealing to social responsibility encourages participation.

Mowen & Caldini (1980) found adding phrase “it would really help us out” increased response 19%.

BENEFITS STRATEGIES

3. Show Positive Regard – being regarded positively by others enhances motivation.

Personally addressing contacts, providing contact information for questions and providing multiple response modes shows regard for respondents.

4. Say Thank You – verbal appreciation can be an important reward.

“We appreciate your help” and “Many thanks in advance” can increase response rates.

Dillman et al. (1974) found a followup postcard designed to thank respondents for “prompt return of the survey sent to them recently” doubled response rate.

BENEFITS STRATEGIES

5. Support Group Values – people identify with certain groups (e.g., citizen of Moscow, Democrat, Catholic) and supporting people’s values can be rewarding.

Tailor communication to appeal to these group values. Demonstrating how survey results can benefit group

values is also helpful in increasing response rates.

6. Give Tangible Rewards – token financial incentives (e.g., $1-10) or other tangible rewards (e.g., IPOD Shuffle drawing) increases response rate.

Providing token incentives in advance creates a sense of reciprocal obligation.

Tangible rewards can be geared to available funding, but is often a key benefit strategy.

BENEFITS STRATEGIES

7. Make Questionnaire Interesting – designing surveys with questions that a wide variety of people find interesting enhances response rates.

Improve visual layout and design, place engaging questions early, and craft questions that are simple and easy to understand and respond to.

8. Provide Social Validation – knowing that others similar to themselves have completed the survey can enhance participation.

Compliance increases when they believe others have responded.

Followup contacts should emphasize that many others have already responded.

BENEFITS STRATEGIES

9. Inform People that Opportunities to Respond Are Limited – people perceive rewards are more valuable when opportunities become scarce (Groves et al., 1992).

Petrie et al. (1998) found participation increased when people were told that their opportunity to respond was limited and they must act quickly.

Explaining that only a small number of people were selected for survey response also is motivating.

STRATEGIES TO DECREASE PARTICIPATION COSTS

Respondents are also motivated by reducing participation costs that they expect to incur.Time, energy and loss of focus may prompt many survey requests to go unanswered.Survey researchers must distinguish how their survey will have minimal costs to respondents.5 strategies can be used to decrease perceived participation costs.

COSTS STRATEGIES

1. Make It Convenient to Respond – offer a desired response that fits the population.

For online surveys, include a link to your survey in the email invitation that respondents just click.

Include prepaid envelopes with mail surveys.

2. Avoid Subordinating Language – people don’t like to feel dependent on others.

Make people feel part of the solution by “requesting” not “demanding” they help you better understand the nature of the problem.

Asking people for help makes them feel self-determining and in control.

COSTS STRATEGIES

3. Make the Questionnaire Short and Easy to Complete – the shorter and easier to fill out surveys appear, the lower the costs to the respondent.

Shorter questionnaires enhance response rate. Respondent-friendly designs for survey enhance

response rates.

4. Minimize Requests to Obtain Personal or Sensitive Information – revealing sensitive information is a “deal breaker” for many respondents.

Ask for sensitive information on when necessary. Put demographic info at end of survey. Explain why info is needed and the confidentiality of

all responses.

COSTS STRATEGIES

5. Emphasize Similarity to Other Requests Which the Person Has Already Responded – people like to appear consistent in the attitudes, beliefs and actions, so they are more likely to comply requests that are consistent with their position on an issue or previous behavior.

In panel surveys, once people respond to the initial request, it is much easier to get them to respond to subsequent requests.

Dillman et al. (1995) used the “foot in the door” technique to get national park visitors to complete a 3-question survey entering the park and a much longer survey when leaving the park.

ESTABLISHING TRUST

Because social exchange involves future obligations, trusting the surveyor to follow through and provide the reward as expected is critical.

Response rate increases when respondents trust that the rewards will be provided as promised.

Trust is critical to believing benefits outweigh costs for a survey.

4 strategies can be used to increase trust.

ESTABLISHING TRUST

1. Obtaining Sponsorship by Legitimate Authority – legitimate authority sources enhance compliance (e.g., U.S. Census Bureau).

Government surveys have higher response rates.

Mandatory surveys increase response rates.

2. Provide a Token of Appreciation in Advance – a dollar include with a survey increases rewards and enhances trust.

Emphasizing the the incentive is a “small token of appreciation” conveys trust and respect.

ESTABLISHING TRUST

3. Make the Task Appear Important – make both the use of survey results and the individual response seem valued.

Personalized cover letters on letterhead Increase trust.

Color pictures on front of survey add a personal touch. Brochures about survey project enhances credibility.

4. Ensure Confidentiality and Security of Information – explain how confidentiality will be maintained and security procedures used to protect data.

Ask for sensitive information on when necessary. Put demographic info at end of survey. Explain why info is needed and the confidentiality of

all responses.

SURVEY SCAMS

Frugging – fund raising disguised as a survey.Slugging – selling disguised as survey research.Phishing – tricking people to provide personal information.Push Polling – negative campaigning disguised as a political poll.

DESIGN CONSIDERATIONS

TAILORED DESIGN METHOD

The End