do monetary incentives increase business survey response rates?

Click here to load reader

Post on 12-Jan-2016




1 download

Embed Size (px)


Do Monetary Incentives Increase Business Survey Response Rates? Results from a Large Scale Experiment Paul Biemer, RTI International and University of North Carolina Christopher Ellis, Angela Pitts and Kimberly Robbins RTI International. Outline of the Presentation. - PowerPoint PPT Presentation


  • Do Monetary Incentives IncreaseBusiness Survey Response Rates?Results from a Large Scale ExperimentPaul Biemer, RTI International and University of North CarolinaChristopher Ellis, Angela Pitts and Kimberly RobbinsRTI International

  • Outline of the PresentationEssential information about the establishment surveyDesign of the incentive experimentResultsConclusions

  • The O*NET SurveyOccupation Information Network Survey (O*NET)Provides descriptive ratings on 800+ U.S. occupationsTarget population is all U.S. employees in these 800+ occupationsContinuing survey since 1997Mode of interviewTelephone contact of establishmentsPoint of contact (POC) is usually Human Resources ManagerPAPI or Internet questionnaires completed by selected employees

  • The O*NET Survey (contd)Two-stage sample designEstablishments and employees within establishments in selected occupations ~24,000 establishments/year (79% RR)~36,000 employee respondents/year (69% RR)

  • O*NET Incentive SystemAll incentives are prepaid (prior to receipt of questionnaires)For the establishment:O*NET toolkit (personnel job description aid)For POC:Desk clockFramed certificate of appreciation$20 money order (to be tested)For employees$10 cash

  • POC Incentive Experiment: MotivationConcern that POC incentives not commensurate with POC tasksSome evidence of diminished POC effort during employee nonresponse follow-up phaseEstablishment and employee nonresponse rates were low for some occupationsPOC does not receive monetary incentive although employees do ($10)

  • Overview of Data Collection Protocol

  • Overview of Data Collection Protocol

  • Cover of the POC Incentive BrochureFRONTBACK

  • Inside of the POC Incentive Brochure

  • Experimental DesignSplit sample design (75/25 split)Incentive group: POCs offered $20 money order in addition to clock and framed certificateControl group: POCs offered only clock and framed certificateMoney orders could be made out to POC, the business or a favorite charityInterviewers (called business liaisons or BLs) handled both incentive and non-incentive cases

  • Sample Allocations to the Control and Incentive Groups

  • Results

  • Unweighted Response Rates (%)

  • Other ResultsNo effect on response rates bysize of establishmentindustry groupingurban or rural regionsoccupation of employeeSome evidence of a negative effect for some sectors

  • Cumulative Employee Response Rates for by Week of Data Collection

  • Summary of ResultsNo evidence of an effect on establishment cooperation ratesWeak evidence of an effect on employee cooperation ratesBoth negative and positive effects detected for small subgroups Overall, no significant differenceNo cost advantageFollow-up costs were the sameSpeed of response was the sameIncentive costs were not offset by efficiency gains

  • Interpreting the FindingsNo establishment-level effect$20 incentive viewed as trivial by POCsFramed certificate and clock could have much larger perceived valueDecision to participate shared by POC and companyPOC cannot accept money in many establishmentsNo effect at the employee-levelPOC has limited ability to affect employee response$20 ineffective for increasing follow-up intensity