critical lessons for training bilingual interviewers on a
TRANSCRIPT
Critical Lessons for Training Bilingual
Interviewers on a Longitudinal Study
May 20, 2012
Presentation to the 67th Annual Conference of the American Association for
Public Opinion Research
Prepared by
Rebecca Weiner ● Joe Baker ● Cleo Jacobs Johnson ● Shawn Marsh
2
Introduction
Five trainings on Building Strong Families
(BSF) in-home assessment held between 2009
and 2011
Adjustments were made with each successive
training to improve certification rates
Challenges certifying and retaining bilingual
staff
A bilingual-only training was held to address
these challenges
Multiyear, multisite project sponsored by the
Administration for Children & Families
Goal: To learn whether well-designed voluntary
interventions can help unwed parents build
stronger relationships with each other and
their children
Data collection: 2005 to 2011
3
Study Overview: Building Strong Families
4
Building Strong Families: Sites
Seven sites offered marriage and relationship
counseling:
– Georgia
– Maryland
– Louisiana
– Florida
– Indiana
– Oklahoma
– Texas
Hispanic sample concentrated in Georgia,
Florida, Texas, and Oklahoma
5
Study Participants
Couples were eligible to participate if they were
– Expecting a child OR had a child younger than three
months old
– Not married, or married after conception of the focal child
Couples were randomly assigned to a control or
program group
– Control group: 5,098 participants
– Program group: 5,108 participants
– Total: 10,206 participants
1,260 participants indicated that their primary
language was Spanish
6
Couples were followed for three years after entering
the study:
Data Collection Mode Time Frame
Enrollment and Baseline Data
Collection
Paper and
pencil
2005–2008
15-Month Follow-Up CATI 2006–2009
36-Month follow-Up CATI 2008–2011
In-Home Assessments In person 2009–2011
Data Collection: Overview
7
In-Home Assessment: Overview
Purpose: To measure the quality of the parent-
child relationship
Trained field interviewers conduct
assessments in respondents’ homes with focal
child and mother and/or father
Interviewers follow a written protocol and
conduct multiple assessments
Administration time: approximately 1 hour
8
In-Home Assessment: Protocol
Assessment Dimension Measured
Peabody Picture Vocabulary Test 4
(PPVT-4)
Receptive Language
Development (English)
Test de Vocabulario en Imagenes
Peabody (TVIP)
Receptive Language
Development (Spanish)
Walk a Line Slowly Self-Regulation
Two Bags Task Quality of Parent-Child
Interaction
Toddler Attachment Sort-45
(TAS-45)
Parent-Child Attachment
9
Initial Training Model: All Staff
Three days of training
Direct instruction, demonstration, paired
group practice
Live and video certification
– Certification conducted with volunteer families
on second day of training
– Interviewers submitted a video of an assessment
with a volunteer family after training
– Mathematica staff scored administration
10
Initial Training Model: Bilingual Staff
English monolinguals and bilingual observers
attended same training
– Entire training conducted in English
– Majority of trainees were English monolinguals
– Bilingual trainees received one extra hour of training
on bilingual protocol
– Bilingual trainees required to submit a post-training
certification video in English AND Spanish
Assumption: bilingual interviewers can
translate what they learned when administering
Spanish protocol
11
Initial Training Results: Bilingual Staff
Two bilingual staff attended training
One trainee failed certification
One trainee passed certification, but later left the
project
– Felt that assessment was more complicated than she
expected
– Did not produce enough work
12
Adjustments to Training Model
Project staff assisted field manager with
interview process and hiring decisions
– Hired staff with skill set best suited to the task
Trainees were sent a video of Mathematica staff
administering assessment with a volunteer
family before training
– Trainees had a better understanding of what they would
be doing and the flow of the protocol
13
Adjustments to Training Model
Lengthened training to four days
– More time devoted to challenging assessments
Increased paired practice time
– More time to address areas of concern before
certification
Moved certification to last day of training
– More time to become familiar with protocol
Asked trainees to schedule a family for video
certification before training
– Fewer delays with at-home certification
14
Less than half of the bilingual staff were
certified during the first four trainings
Training Date # Bilingual Trainees
# Passed
Certification
February 2009 2 1
June 2009 4 2
December 2009 2 1
July 2010 1 0
Total 9 4
Challenges Certifying and Retaining
Bilingual Staff
15
Challenges Certifying and Retaining
Bilingual Staff
Bilingual staff did not have time to practice TVIP
or bilingual protocol during training
– Difficulty with protocol evident in post-training
certification video
– TVIP was often scored incorrectly
– Difficulty with protocol resulted in low certification and
retention rates
We decided to hold a bilingual staff-only training
to address these challenges
16
Adjustments to Training Model: Bilingual
Staff
Hired staff that could travel to all sites
– Few qualified applicants lived local to BSF sites
– Hired staff with best skills, regardless of location
– Traveled with them to sites with most need
Pre-training video demonstration in Spanish
Adjustments to Training Model: Bilingual
Staff
17
Recruited bilingual staff to assist with practice
and certification
All practice sessions conducted in Spanish
Recruited bilingual volunteer families for
certification
Devoted more time to TVIP practice and created
a “TVIP Window” to assist with scoring
TVIP Window: Example
18
19
Bilingual Training: Results
All staff were certified!
20
Bilingual Training: Results
Staff from the bilingual-only training
– Had never attended a bilingual-only training previously
and found it beneficial
– Completed 66% of bilingual in-home assessments
– Worked more efficiently than previously trained
bilingual staff
21
Lessons Learned
Bilingual staff benefit from a training geared
specifically to them
Language of administration has an effect on
performance
Interviewers hired to travel to sites worked more
efficiently than those hired local to sites
22
Recommendations for Training Bilingual
Field Interviewers
Training should allow for bilingual trainees to
practice in Spanish
Spanish-speaking project staff should assist
with training
Spanish-speaking families should be used for
certification
23
Thank You
Thank you to the Administration for Children &
Families of the U.S. Department of Health and
Human Services
Thank you to the BSF field interviewers and the
families participating in the BSF study
For additional information, email Rebecca Weiner