appendix a. the incredible years: parent, child and teacher training series
TRANSCRIPT
Appendix A.The Incredible Years: Parent, Child and Teacher Training Series
The Incredible Years Award winning “exemplary best practices” program
with demonstrated effectiveness. [Office of Juvenile Justice and Delinquency Prevention, Substance Abuse and Mental Health Services Administration]
Comprehensive—targets home, school and child.
Designed to prevent, reduce and treat conduct problems and to increase children’s social competence.
Parent skills training uses group discussion, videotape modeling, and homework exercises .
Child series (Dina Dinosaur) uses puppets and videotapes to teach social skills, conflict management, and successful classroom behavior.
Research indicates that children’s emotional, social and behavioral adjustment is as important for school success as cognitive and academic preparation.
Child program promotes children’s cognitive problem-solving strategies, increases use of pro-social skills and reduces conduct problems at home and school.
Parent program helps strengthen parenting skills, reduces parenting stress and increases parent’s involvement in school activities.
Teacher program strengthens classroom management skills, reduces classroom aggression and improves teachers’ ability to focus on student’s social, emotional and academic competence.
The Incredible Years Outcomes
CMHRC commitment to excellence, demonstrated through:
Training and certification of facilitators, dedicated FT coordinator for Incredible Years programming.
Purchase of all required curriculum and program materials.
Use of treatment fidelity groups, peer review, videotape feedback.
Continued use of ongoing training and technical support from the Incredible Years developers
Ongoing evaluation of parent and child groups.
Incredible YearsAdherence to Model Fidelity
1. Implement a comprehensive model that employs evidence-based multiple strategies for providing supportive environments and that accommodates ethnic, cultural and economic diversity.
2. Expand the target population to universal and earlier prevention—reaching children with certain family characteristics that put them at particular risk for developing conduct problems, delinquency and substance abuse.
3. Strengthen partnerships with local agencies that serve at-risk populations and, with the community at large, promoting a holistic view of health that includes physical and mental well-being.
CMHRC Goals for Incredible Years
The Incredible Years and Ben Franklin School
Partnership sponsored through a two-year grant from the Rochester Area Foundation First Steps Initiative
Implements the comprehensive model-teacher, child and parent.
First year, Franklin Kindergarten teachers and para-professionals receive extensive training and mentoring from certified Incredible Years trainers.
Second year, Franklin Kindergarten teachers implement independently with guidance and support from CMHRC IY staff.
Second year, Franklin 1st Grade teachers will receive comprehensive training and mentoring from CMHRC IY staff (like Kindergarten teachers this year).
Why offer Dina Dinosaur in kindergarten classes at Ben Franklin school?
Social and emotional competencies are important predictors of school readiness and academic success.
Prevalence of aggressive behavior in early childhood 10% and may be as high as 25% for socio-economically deprived children.
Dina program cost effectively reaches high number of at-risk youth early in life when behavior is most malleable and before severe behavior problems arise.
Incredible YearsDina Dinosaur Prevention
Goals for program at Ben Franklin [Kindergarten]: Prevent early behavior problems from escalating. Integrate children into peer group –classroom as
“family”. Improve social competence for entire class,
provide common vocabulary and problem solving steps to use in everyday conflict.
Magnify dosage of IY intervention through teacher reinforcement of key concepts throughout the school day.
Incredible YearsDina Dinosaur Prevention
Implementation at Ben Franklin during School Year 2007-2008:
Offered 2 times a 6-day cycle in 5 kindergarten classrooms
Lessons over entire school year, seven units in sequential order [rules/behavior, feelings, problem solving, anger management, peer relations etc.]
Led by Children’s Mental Health Incredible Years staff with Ben Franklin teachers participating in planning and delivery of lessons
Incredible YearsDina Dinosaur Prevention - Children
Incredible YearsTeacher Training at Ben Franklin
Teacher training at Ben Franklin during School Year 2007-2008 included:
Five full day interactive classroom management workshops offered throughout the school year, and 3 days of training from Seattle IY trainer on Dina Dinosaur curriculum.
Sessions on promoting pro-social behavior and school readiness, reducing non-cooperation and aggressive behaviors, and ways to collaborate with parents to support school involvement and school-home consistency.
Teachers implementing a classroom action plan with ongoing support from CMH IY staff.
Incredible YearsParent Training at Ben FranklinParent training at Ben Franklin during School Year 2007-2008 included:
A 12-week parent group with two CMH IY facilitators meeting once a week. Child care, supper and transportation offered to program participants.
Used a collaborative process to promote positive parent-child relationships through play, praise and attention.
Worked on effective limit-setting, use of incentives, building social competence and handling misbehavior.
All Kindergarten parents received detailed handouts on the Dina Dinosaur curriculum throughout the year and received tips on how to use the Dina techniques at home.
Evaluation of Incredible Years at Ben FranklinEvaluation on all three components of the model, data collection includes:
Classroom Environment Scale [pre/post]
Social Competence Scale on Student Behavior [pre/post]
Teacher Classroom Strategies [pre/post]
Teacher Workshop Satisfaction Surveys
Parent Dina Dinosaur Satisfaction Surveys
Parent Stress Index [parent group, pre/post]
Eyberg Child Behavior Inventory [parent group, pre/post]
Parent Group Satisfaction Surveys
Classroom Environment ScaleTeachers filled out the scale in October 2007 and again in May 2008. Instructions were to:
Think about the general atmosphere of your classroom during the last month. Use the scale below where ‘10’ stands for always, ‘0’ stands for never and ‘5’ stands for half the time.
0.…1 .... 2….3.…4….5.…6….7.…8.…9….10 Rarely Sometimes Almost always
Classroom Environment ScaleOctober 2007 versus May 2008
8.9 8.8 8.6 8.8
5.6 5.5
2.3
4.2
Q1. Follows RulesQ2. Work Cooperatively
Q3. Problem Solve ConflictsQ4. Manage Anger
0
2
4
6
8
10
Rarely
Some-times
AlmostAlways
Teachers rate their classrooms on scale of 1 through 10 for behaviors exhibited in past month (N=8 Classrooms)
Q1. Students follow classroom rules
Q2. Students work cooperatively (helping sharing, teamwork).
Q3. Students attempt to problem solve when conflicts arise (talk about problem, discuss feelings, suggest solutions).
Q4. Students manage anger (uses words not fists, takes deep breaths, remains calm).
p≤.002 p≤.001 p≤.000 p≤001Statistically significant =
Source: Classroom Environment Scale database
Rarely
Some-times
AlmostAlways
Teachers rate their classrooms on scale of 1 through 10 for behaviors exhibited in past month (N=8 Classrooms)
Q5. Students express feelings appropriately (positive or negative)
Q6. Students handle transitions well.
Q7. Students stay on task during class activities.
Q8. Students show awareness and concern for individual differences
8.9 9.1 8.9 8.9
4.6
5.4 5.6 5.8
Q5. Express feelings Q6. Handle transitions Q7. On Task Q8. Show concern
0
2
4
6
8
10
p≤.000 p≤.002 p≤.003 p≤000Statistically significant =
Source: Classroom Environment Scale database
Classroom Environment ScaleOctober 2007 versus May 2008
How confident are you in managing current behavior problems in your classroom?
How confident are you in managing future behavior problems in your classroom?
Confident7
88%
Somewhat Confident1
13%
October
Confident2
25%
Very Confident6
75%
May
Teachers rate their confidence on six point scale: very unconfident, unconfident, somewhat unconfident, somewhat confident, confident, very confident (N=8 Classrooms)
Confident6
75%
Somewhat Confident2
25%
October
Confident2
25%
Very Confident6
75%
May
Source: Classroom Environment Scale database
Classroom Environment ScaleOctober 2007 versus May 2008
Average TOTAL score on Classroom Environment Scale by Program (maximum =80 points)
7073.5
69.7 70.1
44.749.5
26.7
39.1
AM classesPM classes
Extended Day ALL classes0
10
20
30
40
50
60
70
80
Source: Classroom Environment Scale database
Classroom Environment ScaleBy Franklin Program (AM, PM, Extended Day)October 2007 versus May 2008
Social Competence Scale – Teacher* Teachers rate individual student behavior on 25 questions using scale: ‘not at all’, ‘a little’, ‘moderately well’, ‘well’ ‘very well’ .
Ratings occurred in October 2007 and again in May 2008.
Questions look at: Emotional regulation
Problem solving skills
Attentiveness and initiation
N= 89 students
(17 students moved before post data was collected and are not included in the analysis.)
* Tool used by Incredible Years Programs – Denver Colorado School District
Social Competence Scale TeacherEmotional Regulation
73 7580
7073
7983
7673
1511
2115
1117
25
15 17
2. Accepts things3. Copes w/ failure
6. Accepts limits7. Expresses needs
8. Thinks before acting11. Calms down
12. Wait patiently13. Understands other's feelings
14. Effect behavior others
0
20
40
60
80
100
Source: Social Competence Scale Teacher database
N=89 students
Percentage who do “well” or “very well”
October 2007 versus May 2008
Social Competence Scale TeacherSocial Skills/Peer Interaction
66
73
7982 81 80
7672
8
1620
2619
2518
14
9. Resolves peer problems15. Works well group
16. Plays by rules19. Shares materials
20. Cooperates peers22. Helpful others
23. Listens others24. Not bossy
0
20
40
60
80
100
Source: Social Competence Scale Teacher database
N=89 students
Percentage who do “well” or “very well”
October 2007 versus May 2008
Social Competence Scale TeacherAttentiveness/ Initiative
6366
78
66
7580
15 15
26
10
1825
1. Ignores distractions4. Self starter
5. Works/plays w/out adult10. Stays on task
17. Pays attention21. Follows teacher directions
0
20
40
60
80
100
Source: Social Competence Scale Teacher database
N=89 students
Percentage who do “well” or “very well”
October 2007 versus May 2008
Social Competence Total Score by Franklin Program (AM, PM, Extended Day)
October 2007 versus May 2008
Average TOTAL score by Program (maximum =125 points)
98
112.9
99.3103.9
59.8
78.6
52.2
64.1
AM classesPM classes
Extended Day ALL classes0
20
40
60
80
100
120
All statistically significant at p≤000
N=89 students
Source: Social Competence Scale Teacher database
Social Competence Total Score by Ethnicity
October 2007 versus May 2008
Average TOTAL score by Program (maximum =125 points)
104.1
93.397.7
108.8 108.5
65.861.5
51.5
6370.2
CaucasianAfrican American
African NativeHispanic
Asian0
20
40
60
80
100
120
All statistically significant at p≤01Source: Social Competence Scale Teacher database
N=42N=10
N=6N=20
N=8
1 Native American pre 75, post 125
Social Competence Total Score by Gender / Ethnicity/ Disability: October 2007 versus May 2008
Average TOTAL score by Program (maximum =125 points)
99.7
110.3104.1 104.4
55.5
107.8
59.6
70.265.8 62.7
34.3
66.2
MaleFemale Caucasian
Minority Dev. DisabilityNo Disability
0
20
40
60
80
100
120
All statistically significant at p≤01Source: Social Competence Scale Teacher database
N=6N=83
N=51N=38 N=42
N=47
Social Competence Total Score by Teacher
October 2007 versus May 2008
Average TOTAL score by Program (maximum =125 points)
82.3
112.9
105.6
114.1
58.463.5
50.4
95.7
Frana Ilstrup Moskalik Schlake0
20
40
60
80
100
120
Source: Social Competence Scale Teacher database
N=18 N=20 N=33 N=18
Social Competence Total Score by Gender / Ethnicity/ Disability: October 2007 versus May 2008
Average TOTAL score by Program (maximum =125 points)
99.7
110.3104.1 104.4
55.5
107.8
59.6
70.265.8 62.7
34.3
66.2
MaleFemale Caucasian
Minority Dev. DisabilityNo Disability
0
20
40
60
80
100
120
All statistically significant at p≤01Source: Social Competence Scale Teacher database
N=6N=83
N=51N=38 N=42
N=47
Social Competence Total Score by Ethnicity
October 2007 versus May 2008
Average TOTAL score by Program (maximum =125 points)
104.1
93.397.7
108.8 108.5
65.861.5
51.5
6370.2
CaucasianAfrican American
African NativeHispanic
Asian0
20
40
60
80
100
120
All statistically significant at p≤01Source: Social Competence Scale Teacher database
N=42N=10
N=6N=20
N=8
1 Native American pre 75, post 125
Social Competence Total Score by Franklin Program (AM, PM, Extended Day)
October 2007 versus May 2008
Average TOTAL score by Program (maximum =125 points)
98
112.9
99.3103.9
59.8
78.6
52.2
64.1
AM classesPM classes
Extended Day ALL classes0
20
40
60
80
100
120
All statistically significant at p≤000
N=89 students
Source: Social Competence Scale Teacher database
Social Competence Total Score by Teacher
October 2007 versus May 2008
Average TOTAL score by Program (maximum =125 points)
82.3
112.9
105.6
114.1
58.463.5
50.4
95.7
Frana Ilstrup Moskalik Schlake0
20
40
60
80
100
120
Source: Social Competence Scale Teacher database
N=18 N=20 N=33 N=18
Teacher Strategies Scale* Teachers rate how often they use various teaching strategies on 34 questions using scale: ‘rarely/never’, ‘sometimes’, ‘half the time’, ‘often’ ‘very often’ .
Ratings occurred in October 2007 and again in May 2008.
Questions divided into five scales**: Positive Approaches with Parents
Praise and Incentives
Proactive Strategies
Limit-Setting Strategies
Inappropriate Strategies (lower score better)
N= 5 teachers, parent scale =4 teachers
*Tool used by Incredible Years Programs – Denver Colorado School District** Incredible Years in process of completing psychometric analyses, currently using these scales
Average total score on each subscale, n=5 (parent n=4)
33.2
26.227.7
16.4
11.8
29.7
18.4
22.4
18.4 18
Positive ParentPraise/Incentives
Proactive StrategiesLimit-Setting Innapropriate Strategies
0
5
10
15
20
25
30
35
Source: Teacher Strategies database
Teacher Strategies ScaleFall versus Spring
Increase in proactive strategies and decrease in inappropriate strategies significant at p≤05, increase in praise marginally significant p≤06
Teacher Training: Satisfaction with Workshops 1-5
Very helpful85
86%
Helpful13
13%
Neutral1
1%
ContentTotal: 99
Very helpful50
58%
Helpful33
38%
Neutral3
3%
VideosTotal: 86
Very helpful67
68%
Helpful29
29%
Neutral3
3%
Role PlaysTotal: 99
There were no “not helpful” answers
Source: Teacher workshop satisfaction database
Ben Franklin Staff Only
Teacher Training: Satisfaction with Workshops 1-5
Very helpful96
97%
Helpful3
3%
Leader's TeachingTotal: 99
Very helpful90
91%
Helpful9
9%
Group DiscussionTotal: 99
There were no “neutral” or “not helpful” answersSource: Teacher workshop satisfaction database
Ben Franklin Staff Only
Teacher Training: Satisfaction by Workshop
Average scoreScale: 1=not helpful, 2=neutral, 3=helpful, 4=very helpful
WORK SHOP
CONTENT VIDEOS LEADER’S TEACHING
GROUP DISCUSSION
ROLE PLAYS
1N=18
3.9 3.4 3.9 3.9 3.8
2N=18
3.7 3.3 3.9 3.9 3.5
3N=19
3.8 3.2 4.0 3.9 3.6
4N=22
4.0 3.8 4.0 3.9 3.7
5N=22
3.9 3.9 4.0 4.0 3.7
Source: Teacher workshop satisfaction database
Ben Franklin Staff Only
Parent Feedback Survey Dinosaur SchoolParents were asked to read a description of the Dinosaur School (puppets named Dina and Wally and the lessons being taught)
They were then asked to think about the last three months and rate the following questions on a 4-point scale “very often”, “somewhat often”, “once in awhile”, “not at all”.
1. How often does your child talk about the things he or she is
learning in Dinosaur School?
2. How often have you noticed your child doing the following:
a. Talks about his/her feelings
b. Identifies / recognizes when there is a problem
c. Tries to solve problems when they come up (uses solutions/ideas
from Dinosaur School)
d. Shares or helps friends/siblings
Tools Used to Measure Outcome:
1. Pre and Post Parenting Stress Index [PSI] Paper/pencil assessment parent fills out to assess parent-child relationship (three subscales and total stress score)
2. Pre and Post Eyberg Child Behavior Inventory [ECBI] Paper/pencil behavioral rating scale parent fills out to measure their perception of child conduct problems and the extent to which it is a problem for the parent.
Parent BASIC GroupWeekly parent group using Incredible Years curriculum to strengthen parenting skills and reduce stress.
Nine parents completed [3 fathers, 4 mothers, 1 stepparent, 1 grandparent]
Significant Drop in Percentage of Parents at a “Clinical Level” of Parental Stress (TSS*)
Average stress level for the Ben Franklin parent group [beginning and end] is less than for the parent “treatment groups” held at the CMHRC.
Average Entry Score – 81.2 (SD=8)Average Exit Score - 47.7 (SD=13)t{5}=6.5, significant at p<.001
0
3
0
1
2
3
4
5
6
Nu
mb
er
at
Clin
ica
l Le
vel
Pre Post
*Total Stress Score (TSS) is an indication of the stress level experienced within the role as parent.
Clinical level is at or above the 90th percentile on PSI scale.
N=6 with pre and post PSI*
Parenting Stress Index Pre to Post Program
Source: PSI / ECBI Database
Significant Improvement in Behavior Intensity and Child Behavior Problems (as perceived by parents)
INTENSITYAverage Entry Score - 132 (SD=19)Average Exit Score - 89 (SD=8)t{8}=8, significant at p<.001
PROBLEMAverage Entry Score – 12 (SD=10)Average Exit Score - 1 (SD=3)T{8}=3 , significant at p<.01
0 0
3
4
Intensity Problem 0
1
2
3
4
5
Num
ber
at C
linic
al L
evel
Pre Post
Eyberg Child Behavior Inventory [ECBI] Pre to Post Program
N=9 with pre and post ECBI
Average ECBI levels for the Ben Franklin parent group [beginning and end] is lower than for the parent “treatment groups” held at the CMHRC.
Source: PSI / ECBI Database
Parent Satisfaction with BASIC IY Parent Program, n=9 parents
44% 33%100%
56%
100%
67%
Original problemUse of methods
Child's progressOther family issues
0
10
20
30
40
50
60
70
80
90
100
Per
cent
age
helped slightly helped greatly helped1. Problem(s) that
originally prompted me to take this program for my child….
2. Use of methods to try and change child’s problems…
3. My feelings about my child’s progress…
4. Has helped with other personal/family problems…
Source: Tally sheet for parent BASIC satisfaction surveys
Parent Satisfaction with BASIC Parent Program
Source: Tally sheet for parent BASIC satisfaction surveys
Of 9 respondents: All would “strongly recommend” the program to a friend or relative.
All felt “positive” [3] or “very positive” [6] about achieving their goal in the program for child and family.
All were “confident” [5] or “very confident” [4] they could manage current behavior problems.
All were “optimistic” [1] or “very optimistic” [8] about good results from the program.
All felt the parenting approach used to change child behavior was “appropriate” [1] or “greatly appropriate” [8].
Parent Satisfaction with BASIC Parent Program
Source: Tally sheet for parent BASIC satisfaction surveys
33% 33% 22% 22%
67% 67%
78% 78%
Teaching 1Teaching 2 Preparation 1
Preparation 20
10
20
30
40
50
60
70
80
90
100
Per
cent
age
Slightly Above Average High Superior
Two facilitators:1.I feel the leader’s teaching was…2.The leader’s preparation was…
N=9 responses
All participants felt “very supported” [6] or “supported” [3] by the group.
Eight of nine program participants would like to keep meeting as a group. (The one participant who did not want to keep meeting was a grandparent).
Seven felt it was “likely” or “somewhat likely” they would continue meeting. The other two were “neutral”
Response to Facilitators Response to Group
Parent Satisfaction with BASIC Parent ProgramWhat did you see as the main benefit of the Incredible Years program?
I have become so much better with my children in all the ways that a father should be. I feel this will help me stay that #1 dad that I’ve always wanted. Thank you very much.
Positive talk was the most beneficial for me. Also the skills taught – how and when to ignore, using calming down techniques.
How easy it is to make small changes and have such wonderful results. When the program is broken down the way it was, everything makes so much sense.
Learning skills to improve communication, bringing parents into the children’s setting really helped my kids. They have been so proud and excited to have me here every Monday.
Consistency and reinforcing positive/expected behaviors.