survey research (3): survey experiments & survey examples · pdf filesurvey research (3):...
TRANSCRIPT
Survey Research (3): Survey Experiments & Survey Examples
Benjamin GrahamOffice Hours: M 11:30-12:30, W 10:30-12:30
SSB 447
SOCI 60: Lecture 18 Benjamin Graham
Wednesday, April 3, 13
Housekeeping
• No class on Thursday• Homework 4 is due a week from Thursday
• I will e-mail it out this afternoon
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
Experiments (Review Question)
• What are the three components of a true experiment?• A. At least 2 groups; treatment precedes the post-test, random
assignment• B. At least 2 groups, treatment precedes the post-test, non-random
assignment• C. At least 2 groups, pre-test precedes the treatment, random assignment• D. None of the above
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
Survey Experiments
• The “treatment” is the version of the survey you are given.• Generally we vary the information you are provided
• The dependent variable is respondents’ answer to some question on the survey• Comes after the treatment• Same question for everyone
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
Survey Experiments: so so good, so so bad
• What is totally awesome about survey experiments?• Ethically and logistically much easier than “real” experiments• Random assignment is super easy
• What are some of the drawbacks?• Does our “treatment” actually look like the concept we’re interested in?
• i.e. Do people respond to stories the way they respond to real events?
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
Example
• If you were born in an odd numbered month, you are group A• If you were born in an even numbered month, your are group B
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
Group A ONLY
• A terrorist group has taken 100 US citizens hostage and threatens to kill them if the US does not release several of their members currently being held at Guantanamo bay. • What should the President do?
A. Agree to the dealB. Refuse to negotiateC. Launch a military assault against the Iranian government
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
Group B ONLY
• A terrorist group has taken 2 US citizens hostage and threatens to kill them if the US does not release a Yemeni cleric currently being held at Guantanamo bay. • What should the President do?
A. Agree to the dealB. Refuse to negotiateC. Launch a military assault against the Iranian government
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
Survey Experiments
• First we take our sample (randomized, if possible)• Then divide it into a treatment group and one or more comparison groups
(definitely randomized)
• The treatment is some form of vignette (story). We describe a situation that we can’t actually create.
• Varying the details of a story is much easier than varying actual events.
• Hypothesis: Americans are more willing to support negotiating with terrorists when the risk of US casualties (from non-negotiation) is large.
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
Another Example
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Firm # ______
6""
________________________________________________________________________________________________________________________________________________________________________________"
![SURVEY!EXPERIMENT]![Respondents!are!randomly!assigned!to!either!scenario!1!or!scenario!2]!C8.2."For"this"next"question,"we’ll"ask"you"to"think"of"some"of"the"entry[level"professional"jobs"at"your"firm."I"am" going" to" read" a" description" of" a" candidate," and" based" on" this" description," I" will" ask" you" to" rate" the"candidate"on"a"scale"of"1"to"10,"where"1"indicates"a"very"low"quality"candidate,"and"10"is"the"highest"quality"candidate."After"that,"I"will"also"ask"you"to"rate"on"a"scale"of"1"to"7"on"how"likely"you"would"be"to"hire"this"candidate."""
On"a"scale"of"1"to"10,"where"1"is"poor"quality"and"10"is"highest"quality,"how"would"you"rate"this"candidate?"[C821]"
How"likely"would"your"firm"be"to"hire"this"candidate?"[C822]"
Scenario!1:"Candidate"A"is"a"recent"graduate"from"a"reputable"academic"institution"in"the"Philippines"and"studied"business."He"has"very"good"academic"credentials,"and"graduated"in"the"top"10%"of"his"class."He"has"heard"a"lot"about"your"firm,"and"he"seems"enthusiastic"about"the"work."
" "
C822:""""1["Extremely"unlikely"" 2["Unlikely"" 3["Somewhat"unlikely"" 4["Neither"likely"nor"unlikely"" 5[Somewhat"Likely"" 6["Likely" "7[Extremely"Likely"
On"a"scale"of"1"to"10,"where"1"is"poor"quality"and"10"is"highest"quality,"how"would"you"rate"this"candidate?"[C823]"
How"likely"would"your"firm"be"to"hire"this"candidate?"[C824]"
Scenario!2:"Candidate"A"is"a"recent"graduate"from"a"reputable"academic"institution"in"the"Philippines"and"studied"business."He"has"very"good"academic"credentials,"and"graduated"in"the"top"10%"of"his"class."He"has"heard"a"lot"about"your"firm"because!his!uncle!is!one!of!the!managers,"and"he"seems"enthusiastic"about"the"work.""
" "
C824:""""1["Extremely"unlikely"" 2["Unlikely"" 3["Somewhat"unlikely"" 4["Neither"likely"nor"unlikely"" 5[Somewhat"Likely"" 6["Likely" "7[Extremely"Likely"
!! !Section!D:!Political!Risk!and!Dispute!Resolution(Renumbering"Necessary)"Now"I"have"a"few"questions"about"the"business"and"regulatory"environment"in"the"Philippines.""We"are"interested"in"understanding"the"business"climate"your"firm"faces.""D7.5."Does"your"firm"ever"take"steps"intended"to"affect"government"policy"in"the"Philippines"at"the"national"level?"_____"[D75]"" " 1"–"Yes""" 2"–"No""
If"answer"to"question"D7.5"is"yes:"D7.5A."Compared"to"other"firms"in"your"sector,"is"your"firm"more"or"less"effective"in"influencing"government"policy"at"the"national"level?"______[D75A]""" 1"–"Less"Effective"" 2"–"Equally"Effective"" 3"–"More"Effective""
"D7.5B."What"makes"your"firm"more"or"less"effective"than"other"firms"in"your"sector"at"influencing"government"policy"at"the"national"level?"[D75B]"
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________"
Ben Graham� 4/2/13 10:36 AMFormatted Table
Wednesday, April 3, 13
Combining Survey Experiments and Shielded Response
• I want to know if people under-report a particular behavior in surveys• Group A: Ask the question directly• Group B: Use shielded response
• If people are under-reporting, what should be true?
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
Example
Listed below are the various things that businesses can do to improve rela6ons and influence government policy. Please indicate which of these strategies your firm has used in the past. (Put 1 to all that apply)
_______ D8.1 Contribute to governmental funding for public works and services
_______ D8.2 Invite officials for dinner and entertainment in order to get more “face 6me” _______ D8.3 Privately increase individual officials’ “gray income.” (lagay)
_______ D8.4 Contribute to governmental capacity: e.g. giN a new wing to the government building, giN office equipment etc.
_______ D8.5 Invite officials to visit/inspect your business to highlight your contribu6on to the local economy and community.
_______ D8.6 AUend business forums and func6ons organized by the government.
Group A: Ask each item individually.Group B: Asked how many items on the list they par6cipated in.
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
Checking Understanding
•We add up the total number of items that each firm in Group A admits to doing, and compare that to the number of items that firms in Group B admit to doing. What should be true about Group A (aswered each item individually) and Group B (told us the total number of things they did) if no one is underreporting?•A. Group A reports a higher average number of items•B. Group B reports a higher average number of items•C. Both groups report EXACTLY the same number of items•D. Both groups report the same number of items, plus or
minus random sampling error.
IR 211: Lecture 19 -- Survey Research (3) Benjamin Graham
Wednesday, April 3, 13
The Survey
1.First survey to compare diaspora-‐owned firms to other foreign firms.
2.Random sample of foreign firms in Georgia.• Snowball sample of 7 addi6onal diaspora-‐owned firms
3.Final sample: 161 firms from 38 countries, primarily registered 2004-‐2007.• 30 diaspora-‐owned firms• 121 non-‐diaspora-‐owned foreign firms (10 unknown)
4.Respondents were:• 75% firm managers, 25% owners• 91% ethnic Georgians
Political Risk, Social Networks, and Social Responsibility Benjamin A.T. Graham
Wednesday, April 3, 13
Tes6ng the Diaspora Difference
1. Similarity in terms of firm demographics.
2. Social networks are important to firm loca6on decisions.
3. Diaspora-‐owned firms use social networks to gain compe66ve advantages.
4. Diaspora-‐owned firms do not engage in more pro-‐development or socially responsible behavior.
Political Risk, Social Networks, and Social Responsibility Benjamin A.T. Graham
Wednesday, April 3, 13
Wednesday, April 3, 13
Using Social Networks to Gain Compe66ve Advantage
1. How important are your owners’ and managers’ family rela*onships to increasing the profitability of your firm?
• Also asked about friendships.2. Has your firm ever rented or purchased real estate
with the help of a family member of one of your firm’s owners or managers?
• Also asked about friends.
Political Risk, Social Networks, and Social Responsibility Benjamin A.T. Graham
Wednesday, April 3, 13
Use of Social Networks
-1.5 -0.5 0.5 1.5
Friendships for Real Estate
Family for Real Estate
Friendships and Profitability
Family and Profitability
Social Networks and Firm Entry
Expectations of My TheoryDiaspora Mean - Non-Diaspora Mean
Mean = 0, Standard Deviation = 1
Wednesday, April 3, 13
Use of Social Networks
-1.5 -0.5 0.5 1.5
Friendships for Real Estate
Family for Real Estate
Friendships and Profitability
Family and Profitability
Social Networks and Firm Entry
Actual Findings:Difference in Means
Diaspora Mean - Non-Diaspora Mean
Standard Deviations Normalized to 1
Wednesday, April 3, 13
Pro-‐Development and Socially Responsible Behavior
Ques6ons include:
1. “Compared to employees at other firms in this sector, employees at this firm are able to provide a higher quality of life for their families.”
2. “This firm contributes to economic development in Georgia more than other firms in this sector.”
Political Risk, Social Networks, and Social Responsibility Benjamin A.T. Graham
Wednesday, April 3, 13
Political Risk, Social Networks, and Social Responsibility Benjamin A.T. Graham
Wednesday, April 3, 13
Pro-‐Development and Socially Responsible Behavior
-1.5 -0.5 0.5 1.5
Contribute to Charity
Contribute to Development
Prioritize Local Labor
Pay Higher Salaries
Higher Quality of Life
Professional Development
*Minimize Total Labor Costs
Expectations of My TheoryDiaspora Mean - Non-Diaspora Mean
Mean = 0, Standard Deviation = 1
Wednesday, April 3, 13
Pro-‐Development and Socially Responsible Behavior
-1.5 -0.5 0.5 1.5
Contribute to Charity
Contribute to Development
Prioritize Local Labor
Pay Higher Salaries
Higher Quality of Life
Professional Development
*Minimize Total Labor Costs
Actual Findings:Difference in Means
Diaspora Mean - Non-Diaspora Mean
Standard Deviations Normalized to 1
Wednesday, April 3, 13