Surveying Migrant Surveying Migrant Households: Households: A Comparison of Census-A Comparison of Census-Based, Snowball, and Based, Snowball, and Intercept Point SurveysIntercept Point Surveys

David McKenzieDavid McKenzieWorld BankWorld Bank

(with Johan Mistiaen)(with Johan Mistiaen)

MotivationMotivation

Increasing research and policy attention Increasing research and policy attention being given to migrationbeing given to migration

But few detailed and representative But few detailed and representative surveys of households of migrants existsurveys of households of migrants exist

Many specialized surveys of migrants Many specialized surveys of migrants are non-probability samples, making it are non-probability samples, making it hard to generalize from themhard to generalize from them

=> Paper reports on experiment in Brazil => Paper reports on experiment in Brazil to compare three different survey to compare three different survey methodsmethods

ContextContext

World Bank survey of Brazilians of Japanese World Bank survey of Brazilians of Japanese descent (descent (Nikkei)Nikkei)

Approximately 1.2-1.9 million of them among Approximately 1.2-1.9 million of them among Brazil’s 170 million populationBrazil’s 170 million population

Japanese law allows third-generation and Japanese law allows third-generation and below unrestricted access to labor marketbelow unrestricted access to labor market

Estimated 265,000 Brazilian migrants in Estimated 265,000 Brazilian migrants in Japan, sending approximately $US2billion in Japan, sending approximately $US2billion in remittances annually.remittances annually.

Survey methods used here equally applicable Survey methods used here equally applicable to attempts to survey migrants in destination to attempts to survey migrants in destination countries.countries.

Surveying methodsSurveying methods

In most countries, surveying migrants is In most countries, surveying migrants is survey of survey of rare elementrare element

Three approaches used in practice:Three approaches used in practice:– Stratified sampling using population censusStratified sampling using population census– Chain-referral methods such as snowball Chain-referral methods such as snowball

samplingsampling– Sampling at locations where immigrants Sampling at locations where immigrants

tend to cluster. Time and space/intercept tend to cluster. Time and space/intercept sampling.sampling.

=> Question as to how well other methods => Question as to how well other methods perform compared to random sample.perform compared to random sample.

Our experimentOur experiment

Compare three different methods in BrazilCompare three different methods in Brazil Use the same questions and same survey firmUse the same questions and same survey firm Wish to compare characteristics of households Wish to compare characteristics of households

with and without migrants, estimate with and without migrants, estimate proportion of households receiving proportion of households receiving remittances and with migrants in Japanremittances and with migrants in Japan

Context: population predominately urban; Context: population predominately urban; crime a concerncrime a concern

=> Communications with community => Communications with community organizations, use where possible of Nikkei organizations, use where possible of Nikkei interviewers.interviewers.

Method 1: Stratified Method 1: Stratified random sample random sample Restrict to Sao Paulo and Parana provinces Restrict to Sao Paulo and Parana provinces

where 80% estimated to livewhere 80% estimated to live (still 47 million people)(still 47 million people) Use 2000 Brazilian Census to classify Use 2000 Brazilian Census to classify

households as Nikkei or not (approximately)households as Nikkei or not (approximately) Census doesn’t ask ethnicity, instead asks: Census doesn’t ask ethnicity, instead asks:

race (yellow), place of birth, whether race (yellow), place of birth, whether individual has lived elsewhere in last 10 individual has lived elsewhere in last 10 yearsyears

Second complication: only 10% long form of Second complication: only 10% long form of census asks this.census asks this.

Method 1: Stratified Method 1: Stratified random sample random sample Steps:Steps:

– Select 75 census tracts, with stratified Select 75 census tracts, with stratified random samplingrandom sampling

– Carry out door-to-door listing of households Carry out door-to-door listing of households in these census tractsin these census tracts

– Return and administer questionnaire to Return and administer questionnaire to households identified as Nikkeihouseholds identified as Nikkei

– Follow-up round to re-try and interview Follow-up round to re-try and interview those who refused on first set of interviews, those who refused on first set of interviews, use shorter interview if refuse longer one.use shorter interview if refuse longer one.

ListingListing

Letters sent out to 150 Nikkei Letters sent out to 150 Nikkei associations with bases in the areas associations with bases in the areas chosen, to explain surveychosen, to explain survey

Census tract averages 301 housesCensus tract averages 301 houses 42 interviewers in Sao Paulo and 24 in 42 interviewers in Sao Paulo and 24 in

Parana used for 50 and 25 tracts.Parana used for 50 and 25 tracts. Three attempts made to interview Three attempts made to interview

household; proxy-reporting used if no household; proxy-reporting used if no one at home.one at home.

Summary of listingSummary of listing

Listed 22, 539 dwellingsListed 22, 539 dwellings Among these, detected 839 Nikkei households Among these, detected 839 Nikkei households

– 528 interviewed in person, 311 by proxy-– 528 interviewed in person, 311 by proxy-reportingreporting

Initial phase of interviewing interviewed 247 Initial phase of interviewing interviewed 247 Nikkei householdsNikkei households

Returned and interviewed another 156 (45 long Returned and interviewed another 156 (45 long interview, 111 short interview)interview, 111 short interview)

Found some of those identified as Nikkei by Found some of those identified as Nikkei by proxy were notproxy were not

Estimate in the end:Estimate in the end:– Parana: 76% interviewed, 12% refuse, 10% absentParana: 76% interviewed, 12% refuse, 10% absent– Sao Paulo: 45% interviewed, 31% refuse, 17% absentSao Paulo: 45% interviewed, 31% refuse, 17% absent

Comparing households Comparing households in 2 phasesin 2 phases Main difference is that households Main difference is that households

that required the additional effort that required the additional effort to survey are much more likely to to survey are much more likely to refuse to say how much they refuse to say how much they receive in remittances.receive in remittances.

Look similar in many other Look similar in many other respects to those who answered respects to those who answered first time.first time.

Snowball surveySnowball survey

Contacted 25 Nikkei organizations and Contacted 25 Nikkei organizations and asked each to provide 3 names as seeds.asked each to provide 3 names as seeds.

Targeted sample of 300 householdsTargeted sample of 300 households Received 67 seed namesReceived 67 seed names Two main problems:Two main problems:

– Some of seed households refusedSome of seed households refused– Many households refused to provide referralsMany households refused to provide referrals

In the end have sample of 100 households, In the end have sample of 100 households, 60 seed households, 40 referrals.60 seed households, 40 referrals.

Intercept surveyIntercept survey

Consulted with local researchers, Nikkei Consulted with local researchers, Nikkei organizations, and organizations, and SudamerisSudameris officers to select officers to select broad range of locations which Nikkei broad range of locations which Nikkei community frequentscommunity frequents

Chose 9 fixed points, and 6 eventsChose 9 fixed points, and 6 events– Sports club, metro station in Liberdade neighborhood, Sports club, metro station in Liberdade neighborhood,

two two FeirasFeiras (Sunday marketplaces), hospital, grocery (Sunday marketplaces), hospital, grocery stores, language school, outside branch of bank.stores, language school, outside branch of bank.

– Japanese film event, large cultural festivals, Japanese Japanese film event, large cultural festivals, Japanese food festival, Japanese art exposition, Christmas food festival, Japanese art exposition, Christmas concert and music festival.concert and music festival.

Short questionnaire – 62 questions, 7 minutes.Short questionnaire – 62 questions, 7 minutes.

Intercept surveyIntercept survey

At each location, 2 interviewers used, one to At each location, 2 interviewers used, one to count number passing through location, one count number passing through location, one to interviewto interview

Interviewers there for 129 total hours over 2 Interviewers there for 129 total hours over 2 weeks (~8.5 hours/location).weeks (~8.5 hours/location).

Each person asked how often in past 2 weeks Each person asked how often in past 2 weeks had frequented any of the other locations – had frequented any of the other locations – used to reweight answers.used to reweight answers.

Only 19% had visited only one location – on Only 19% had visited only one location – on average individuals had visited 3.18 out of 15 average individuals had visited 3.18 out of 15 locations during the two week period.locations during the two week period.

Weight:Weight:

15

1

Pr,1minj

i jatsampledsindividualofFractionjvisitsip

Table 4: Sao Paulo Intercept Survey: December 9-20, 2006Number of Number of Refusal Time spent Approximate

Intercept Point interviews refusals rate (%) in location number in locationFixed point locations

Coopercotia Atlético Clube 34 23 40.4 8.5 hours 368Estação Metrô São Joaquim 49 37 43.0 14 hours 1436Feira da Liberdade 34 3 8.1 5 hours 1282Feira Livre da Rua Carneiro 34 3 8.1 7 hours 1635Hospital Santa Cruz 42 12 22.2 8 hours 374Mercearia Marukai 54 76 58.5 13 hours 2583Mercearia Satsuyama 36 69 65.7 11 hours 1922Sociedad Brasileira de Cultura Japonesa-Bunkyo 34 25 42.4 9 hours 311Agencia Sudameris 34 24 41.4 8 hours 186EventsCinema Bunkyo 34 19 35.8 4 hours 97ACAL Toyo Matsuri - Festival Oriental 30 22 42.3 9 hours 824ACESA Motitsuki Matsuri (Festival Gastronômico) 29 1 3.3 8 hours 424Fundação Mokiti Okada -Exposição de Obras de Arte 4 2 33.3 9 hours 67Coral Bunkyo - Concerto de Natal 34 3 8.1 4 hours 704Grupo the Friends-Koohaku Utagassen 2006 (Festival musical) 34 6 15.0 11 hours 1731

516 325 38.6 128.5 hours 13944

Table 7: Which individuals go to more intercept locations?Dependent variable: Number of locations visited in past 2 weeks

(1) (2)

OLS OLSMale -0.288* -0.294*

Age 0.0151*** 0.00719

Married 0.101 0.0965

Has University Education 0.00686 0.0842

Has worked/studied in Japan 0.614*** 0.313

Works for Pay 0.0787 0.125

Receives Remittances 0.436 0.433

Issei 0.485

Nissei 0.433**

Reads Japanese newspapers 0.688***

Refuses to give income range -1.005***

Observations 492 491R-squared 0.07 0.13

Fourth method?Fourth method?

Use phone book to select individuals Use phone book to select individuals with minority names? (Osili with with minority names? (Osili with Nigerians in Chicago)Nigerians in Chicago)

90% have landline phone (most of rest 90% have landline phone (most of rest have cellphones)have cellphones)

With intermarriage, 74.8% have With intermarriage, 74.8% have recognizably Japanese surnamerecognizably Japanese surname

Only 24% appear in the phone bookOnly 24% appear in the phone book Only 20% in phone book with Japanese Only 20% in phone book with Japanese

name!name!

Comparison of Comparison of different methodsdifferent methods Hypotheses:Hypotheses:H1: intercept and snowball households will be H1: intercept and snowball households will be

more closely connected to Nikkei community more closely connected to Nikkei community than random sample.than random sample.

H2: weighting intercept survey should bring it H2: weighting intercept survey should bring it closer to random samplecloser to random sample

H3: snowball and intercept surveys will H3: snowball and intercept surveys will overestimate proportion of households with overestimate proportion of households with migrant experience.migrant experience.

H4: refusal rates for questions about remittances H4: refusal rates for questions about remittances will be higher for intercept survey, since they will be higher for intercept survey, since they take place in public location.take place in public location.

Results: Connection to Results: Connection to JapanJapan

Table 11: Comparison of Characteristics of Nikkei Across Different Sampling Methods

Stratified Unweighted WeightedSurvey Intercept Intercept Snowball

Household Characteristics

Household Size 3.25 3.29 3.45 3.66*Percentage of households with member who:

Reads Japanese/Nikkei newspapers 13 39*** 28*** 45***, +++

Watches Japanese/Nikkei TV programs 23 43*** 36** 54***, +++

Reads Japanese/Nikkei books/magazines 15 42*** 30*** 52***, +++

Reads newspapers from Nikkei associations 12 39*** 28*** 63***, +++

Checks Japanese/Nikkei websites on the internet 5 24*** 18*** 25***

Sample Size (households) 204 516 516 100

Sao Paulo

Results: Migration and Results: Migration and RemittancesRemittances

Table 12: Comparison of Migration and Remittances of Nikkei Across Different Sampling Methods

Stratified Unweighted WeightedSurvey Intercept Intercept Snowball

% of households which:Migration

Have a Member currently in Japan 19 35*** 33** 30 Have a Member who has returned

from work/study in Japan 41 65*** 59*** 51Remittances

Receive remittances from Japan 14 10 10 10 Refuse to say if they receive remittances 3 2 3 7

Refuse to say how much they receive if receiving 31 73*** 82*** 50+

Sample Size 204 516 516 100

Sao Paulo

RemittancesRemittances

Proportion refusing to report how Proportion refusing to report how much they receive is much higher much they receive is much higher (82%) in intercept survey than in (82%) in intercept survey than in household survey (31%). household survey (31%).

Fear of crime likely to lead to Fear of crime likely to lead to more reticence in public places.more reticence in public places.

Do they give different Do they give different results in regressionsresults in regressions Probit regressions to consider Probit regressions to consider

determinants of being a return determinants of being a return migrantmigrant

Stratified and intercept survey Stratified and intercept survey give quite similar resultsgive quite similar results

Snowball gives quite different Snowball gives quite different picture – gives different answers picture – gives different answers to education and gender to education and gender selectivity of migrants.selectivity of migrants.

Marginal effects from Probit estimation of being a return migrantSTRATIFIED INTERCEPT SNOWBALL

(1) (3) (5)Age -0.0000349 -0.00132 -0.00656

Female -0.0835 -0.0381 0.130**

Married 0.0779 0.0260 0.0395

Undergraduate Education -0.136* -0.0156 0.257***

Postgraduate Education 0.333 -0.0579 -0.311***

Sample Size 195 362 140

Comparison of costsComparison of costs

Total cost per household interviewed:Total cost per household interviewed:– Stratified survey: US$212 ($2 per dwelling Stratified survey: US$212 ($2 per dwelling

listed; $80 per interview)listed; $80 per interview)– Snowball survey: US$100Snowball survey: US$100– Intercept survey: US$30Intercept survey: US$30

Recall survey lengths differ:Recall survey lengths differ:– Stratified and Snowball: 36-page Stratified and Snowball: 36-page

questionnaire, 1 hour to completequestionnaire, 1 hour to complete– Intercept: 3-page questionnaire, 7 minutes Intercept: 3-page questionnaire, 7 minutes

to completeto complete

ConclusionsConclusions

Snowball and Intercept methods Snowball and Intercept methods oversample individuals more closely oversample individuals more closely connected to communityconnected to community

Intercept method does seem to give Intercept method does seem to give results closer to random sample when results closer to random sample when reweight data – suggests multiple reweight data – suggests multiple location sampling importantlocation sampling important

Snowball, often seen as cheapest and Snowball, often seen as cheapest and easiest, may be just as difficult to easiest, may be just as difficult to carry out in some applications.carry out in some applications.

ConclusionsConclusions

Choice of survey method does make Choice of survey method does make large and significant differences to large and significant differences to estimates of means of interest, and to estimates of means of interest, and to regression analysis.regression analysis.

Cost considerations may suggest:Cost considerations may suggest:– Intercept survey for exploratory analysis, or Intercept survey for exploratory analysis, or

where target population attends community where target population attends community locationslocations

– Stratified survey for representative Stratified survey for representative analysis.analysis.

Discussion: when are Discussion: when are these methods these methods applicableapplicable Methods here easily applicable in migrant Methods here easily applicable in migrant

destinations (e.g. Salvadoreans in D.C.)destinations (e.g. Salvadoreans in D.C.) Are methods applicable in sending Are methods applicable in sending

countries when group is not ethnically countries when group is not ethnically different?different?– Listing based methods clearly areListing based methods clearly are– Snowball used by sociologistsSnowball used by sociologists– Intercept – festivals, transportation hubs, Intercept – festivals, transportation hubs,

money-transmitting branches, churches, money-transmitting branches, churches, social support networks, …social support networks, …