use of a 2nd reminder mailing, quick response code currently, arbitron uses a mailed screener survey...

Use of a 2nd Reminder Mailing, Quick Response Code and Optimized Mobile

Survey to Increase Response to a Screener Questionnaire

American Association for Public Opinion ResearchOrlando, Florida

May 18, 2012

Michelle Cantave, Robin Gentry, Arbitron Inc and Survey Technology & Research

© 2012 Arbitron Inc.

Outline

»Background

»Research Questions

»Methodology

»Results

»Conclusions


Background

»Currently, Arbitron uses a mailed screener survey sent to an Address Based sample (ABS) to recruit the non-landline portion of our hybrid sample frame.

»The screener is only the first step of a multi-mode data collection process: • Mailed screener • Phone diary placement• Mailed diary package

»This means that the overall response rate for the ABS sample can never be higher than the return rate to the initial screener survey


Background cont…

»ABS screener methodology:• Wave 1:

–Mail a screener questionnaire with $2 cash gift in the envelope and a promised incentive of $5 for completing and returning the screener.

– Respondents are offered the option of completing the survey by» Mailing the paper survey» Completing the survey over the internet or» Calling in to complete the survey by phone

• Wave 2:– If the screener is not received after three weeks a second

screener is sent to the household. –There is no cash included in the envelope but the promised

incentive is increased to $10


Research Questions

»Can we improve the ABS screener returns rate by adding an inexpensive third wave postcard reminder?

»Can we drive late respondents to complete on the web?

»Will respondents use a Quick Response code to access the survey?

»Can we use the third wave reminder to increase the number of responses from our hard to reach demos?


Methodology

»Designed a secure mailer that contained the login information for the respondents •The postcard encouraged respondents to either:

– Go online to complete the screener survey – Call into the 1-800 line to complete the survey– Return the paper survey that was previously mailed to the household – Scan the QR code to go directly to online survey

»The website that the QR code went to was optimized for mobile phone use.

»Finally, we included the message “LAST CHANCE” on the outside of the postcard. • We have found in other testing that this message lead to a spike in returns


Methodology cont…


Methodology cont…

»For the Wave 1 mailing 156,000 screeners were mailed out

»After the close of the Wave 2 mailings, there were 111,290 households that still had not completed the screener

»We randomly split the non-responding addresses into a test and control group.

»We then sent the reminder postcard to half of the non-responding households


Results

»Overall, there was a significant 1.4 point gain in the number of late returns that were received from the test group

Table 1: Screener Return Rate after Wave 3 Reminder

Control Test Pt. Diff % Diff

Wave 3 Reminder 1.7 3.1 1.4* 82.4%*

N 55645 55645*significant p<0.0001 in a one tailed test


Results cont…

»The sample of third wave reminder respondents was relatively small, making it difficult to find significance.

»There was a general trend toward more Cell Phone Only/Cell Phone Mostly returns from the test group as well as those from households containing an 18-34 year old or a Hispanic.

»However, there was also a trend towards a reduction in returns from Black households.

Table 2: Proportion of Screener Returns by Household Type

Control Test Pt. Diff % DiffCell Phone Households 66.9 69.5 2.6 3.9%Black 18.0 16.0 -2.0 -11.1%Hispanic 14.8 16.0 1.2 8.1%Young Adult 48.4 51.5 3.1 6.4%Total Returned 928 1721


Results cont…

»While mailing the screener back was the preferred mode of responding, when compared to Wave 1 and Wave 2 respondents, the proportion of respondents that responded via the web was triple that of Wave 1 and double that of Wave 2

Table 3: Method of Return by WaveInternet Mail Telephone

Wave 1 10.4% 87.7% 1.8%Wave 2 16.8% 80.1% 3.2%Wave 3 (with Post Card) 35.4% 58.1% 6.5%

Results cont…

»QR Code• In order to track QR Code usage we had the QR code

direct to a separate web site. • While the a few respondents accessed the survey via

the QR code, none of them completed the survey

»Mobile Devices• While some respondents did complete the survey via a

mobile device, we do not have browser information for all respondents and cannot determine if optimizing the website for mobile phones had an effect on website response rates

13© 2012 Arbitron Inc.


Conclusions

»The addition of the Wave 3 mailer resulted in a significant increase in the number of returned screeners after the close of the Wave 2 mailing window

»The mailer appears to be successful in bringing in more cell phone households, Young Adults, and Hispanics but appears to not be successful with Black households

»Offering the website as primary method of completing the survey appears to have an effect in the method of completion

Possible Future Uses

»Use the mailer to delay the raising of the incentive until Wave 3

»Target the sending of the mailer to areas that have higher concentrations of Cell Phone households or Young Adults


Thank You

Questions?

[email protected]

[email protected]

use of a 2nd reminder mailing, quick response code currently, arbitron uses a mailed screener survey...

Documents