anglicare victoria teachar program · 2.4. anglicare victoria teachar program 13 2.5. evaluation...
TRANSCRIPT
1
Anglicare Victoria TEACHaR Program
Year 2 Evaluation ReportBRIDGING THE EDUCATION GAP FOR CHILDREN AND
YOUNG PEOPLE IN OUT OF HOME CARE
AUGUST 2015
2
About the author
Laura David (BA, BSW, Dip Creative Arts, Cert IV Training & Assessment) is a research officer with Anglicare Victoria’s Policy, Research and Innovation Unit. Laura is a social worker, with particular experience working with families and children. Before joining Anglicare Victoria, she worked in a research capacity in the community mental health sector, during which time she received a Department of Health Emerging Researchers Scholarship for a project exploring the experiences of carers in the context of dual diagnosis. In her current role, Laura is leading the evaluation of the TEACHaR education program for children and young people in Anglicare Victoria’s out-of-home care services. Laura has also been involved in a range of other projects including: the evaluation of the ‘Hey Babe’ early parenting support program, and the ‘Baby and Me’ research project which explored the systems of supports available to young pregnant women and mothers in, or exiting out of home care.
The author would like to thank the following people for their particular contributions and support to this evaluation:
• Anglicare Victoria’s Principal Researcher Dr Tatiana Corrales for her assistance with data collection and analysis, guidance with evaluation methodology, and her contribution to the final report.
• Dr Sarah Wise, past General Manager of Anglicare Victoria’s Policy, Research and Innovation Unit. Dr Wise had a key role in the development of the TEACHaR program model, devised the original evaluation design and outcomes framework, and was co-author of the evaluation’s initial research outputs.
The author also thanks the following Anglicare Victoria staff members; Dennis Sherwell (Grants Coordinator) and Jenny Potten (Director Quality) for their editing assistance; Paul McDonald (CEO) for his guidance and input; Tori Smith (Case Manager) for her research assistance; Karen Fairhurst (Research Officer) for assistance with ethics; Andrea Dwyer (Regional Director, Southern Region) for her input regarding program implementation; and Eddy Sara (Senior Graphic Designer) for the design of the research report.
Importantly, the author extends her sincere thanks to the TEACHaR Educators for their valuable input and collaboration over the past two years:
• Greg Rollings, Victoria Bramall-White, Brett Anstey and Natasha Chidgey (TEACHaR 1 - pilot program)
• Jo Myers, Alicia Sefton and Hayley Pethica (TEACHaR 2 – Disabilities focused program)
• Janice Marks, Karyn Bryant and Claire Drake (TEACHaR 3 – Residential care focused program)
ISBN: 978-0-9808054-6-8
Suggested citation: David, L. (2015). Anglicare Victoria TEACHaR Program: Year 2 Evaluation Report. Anglicare Victoria: Melbourne.
3
Funders
The three-year pilot TEACHaR program was made possible via major philanthropic support from the Helen Macpherson Smith Trust.
Other significant financial support for the pilot was also received from:
• The Alec Prentice Sewell Gift, managed by The Ian Potter Foundation
• The Vera Moore Foundation
• The R.M. Ansett Trust, the Mars-Stride Trust, and the William Henry Pawsey Charitable Trust, managed by Equity Trustees
• The Kimberley Foundation.
Since 2014, TEACHaR program has been delivered in partnership with The Victorian Department of Education and Training (DET), The Federal Government’s ‘More Support for Students with Disabilities’ Education Initiative and The Victorian Department of Health and Human Services (DHHS).
Anglicare Victoria gratefully acknowledges the generosity, support and continued collaboration of all philanthropic and Government partners.
Governance Group
Anglicare Victoria would like to thank past and current members of the program’s Governance Group. Members have included Kris Arcaro (Director of Student Wellbeing, DET), Sam Cavarra (past Manager, OHC Services, DHHS), Stephen Newton (Anglicare Victoria Board Member), Gerard Jones (Executive Director, MacKillop Family Services), Bonnie Toale (Project Officer, DET), and the following Anglicare Victoria past and current staff members: Andrea Dwyer (Director, Southern Region), Jenny Potten (Director Quality), Angela Deliyannis (Area Director Northern Region), Dr Sarah Wise (past General Manager, Policy, Research and Innovation) and Dani Ascenzo (past General Manager, Placement and Support).
4
5
CONTENTS
1. Executive Summary 6
1.1. Recommendations 8
2. Introduction 10
2.1. Background 10
2.2. Policy 10
2.3. Literature 11
2.4. Anglicare Victoria TEACHaR Program 13
2.5. Evaluation Aims 15
3. Method 18
3.1. Sample 18
3.2. Measures 18
3.3. Data collection procedures 20
3.4. Analysis Framework 21
4. Results 22
4.1. Demographic Data 22
4.2. Quantitative Data 24
4.3. Qualitative Data 35
5. Discussion 40
6. Case Studies 46
8. References 49
9. Appendix A: Data Cleaning 51
6
EXECUTIVE SUMMARYAnglicare Victoria has a long history of supporting children and young people in out-of-home-care (OHC). Today it is Victoria’s largest provider of foster and kinship care, and one of the state’s major providers of residential care. The agency is highly committed to supporting the successful transition of children and young people in OHC; be that via re-unification with their families, movement into permanent care, or ultimately the transition into independent living in the community.
The agency also recognises the impact that positive educational experiences and attainment can have in later life, particularly for vulnerable children. Not only does a quality education have the potential to strengthen future employment opportunities and economic stability, but it can contribute to increased psychosocial wellbeing, self-esteem, and community belonging and connectedness into adulthood.
The TEACHaR program (‘Transforming Educational Achievement for Children in Home-based and Residential care’) was designed and implemented by Anglicare Victoria in response to evidence that children and young people living in OHC often experience poor education outcomes in comparison to the general student population. As such, the program aims to:
• Strengthen student school engagement and attendance in compulsory schooling
• Lift student literacy, numeracy and academic skills
• Support students to obtain results that are more consistent with their potential and ability
• Support students to complete Year 12 or its equivalent at the same rate as others in the community
• Support students to develop more positive feelings and attitudes towards learning
The program was first implemented in late 2012 with the support of the Helen Macpherson Smith Trust and other Trusts, and has since been expanded in partnership with The Department of Education and Training (DET) and The Department of Health and Human Services (DHHS).
There are currently three concurrent trials of the program in operation:
TEACHaR 1 (Pilot program)
Operates across the Eastern and Southern Metropolitan regions of Melbourne. The pilot was first implemented in approximately December 2012, and has been funded by a number of philanthropic trusts over three years. TEACHaR 1 supports children and young people living in Anglicare Victoria’s foster care and residential care programs.
TEACHaR 2 (Disabilities-focused)
This trial is targeted to children and young people in OHC with disabilities and special needs. It was funded in 2014 by the Department of Education and Training, and the Federal Government’s ‘More Support for Students with Disabilities’ Education Initiative. This program operates in the Eastern Metropolitan Region, and is available across Anglicare Victoria’s foster, residential and kinship care programs.
TEACHaR 3 (Residential care-focused)
This trial operates in the Northern Metropolitan Region and was funded by The Department of Health and Human Services in 2014. This program has a particular focus on supporting the educational needs of children and young people living in residential care, and is further differentiated on the basis that it engages both Anglicare, and non-Anglicare OHC clients.
7
At the time of publishing this report (approximately 2 ½ years into service delivery) the three TEACHaR program teams have supported a growing number of students, schools and school staff. In total the programs have together supported:
• Approximately 172 children and young people in OHC
• At least 156 schools or alternative learning environments
• Over 425 teachers and other staff within schools
The program model
One of TEACHaR’s unique strengths is that it employs and remunerates experienced, registered teachers who bring a range of classroom, welfare and senior level teaching experience to their work. Each TEACHaR program comprises one Lead Educator (who also has a coordination role) and two Educators; that is, a combined team of nine staff.
Where possible and appropriate, the TEACHaR model prioritises direct support to children and young people in OHC, both in their homes and their classroom environments, rather than referrals to other services. Interventions are flexible and holistic, encompassing: 1:1 tuition, group work, informal learning, systemic advocacy, brokerage and assistance for students to access Allied Heath and other assessments. The model is also highly collaborative, working closely with schools, care teams, OHC settings and other professionals.
Evaluation aims and design
This evaluation is targeted to the original pilot TEACHaR program only (TEACHaR 1). It does not currently extend to the work of the two additional TEACHaR teams.
The aim of the evaluation is to test whether the TEACHaR Operating Model, via its key service activities, can affect positive change for children and young people receiving the intervention across a range of educational domains.
Results
The 12 month outcomes data show positive change across the majority of educational domains for children and young people receiving the TEACHaR 1 (pilot) intervention, consistent with the program’s operating model (refer to Figure 3).
Key Outcomes – Change from Baseline to 12 months:
Support in schools:
• A further 30.6% of clients had an individual learning plan (57.6% to 88.2%)
• 100% of clients had access to a school-based student support group at the 12 month follow-up, up from 82.4% at baseline
• An additional 58.1% of clients had a school-based learning mentor (33.3% to 91.4%).
Engagement and attitudes to learning
• Average school attendance increased from 26.6 hours per week to 31.2 hours
• A further 35.8% of students were frequently ‘happy at school’ (58.8% at Baseline to 94.6% at 12 month follow-up)
• An additional 27% of students were assessed as working as hard, or harder than other students (32.4% to 59.4%)
• An additional 19.6% of students were always or usually actively involved in learning tasks (58.8% to 78.4%)
8
Academic skills and achievement
The proportion of children and young people with improvements on a range of academic skills and achievement measures increased from baseline to 12 month follow-up.
• A further 39.7% of students were rated as average or above on literacy skills (17.1% to 56.8%)
• A further 32.9% of students were rated as average or above on numeracy skills (13.9% to 48.6%)
• A further 37.4% of students were rated as average or above on concentration skills (19.4% to 56.8%)
• A further 23.8% were performing to their ability (16.7% to 40.5%)
• A further 39.7% achieved average or above on overall academic ability (17.1% to 56.8%)
Barriers to learning
• Individual and school based barriers reduced over the 12 month period. However there was little change on placement barriers.
The evaluation also shows that the program had a positive role with respect to school attendance and school stability for highly disengaged students, particularly at the critical period of new placement transitions. Together with the quantitative increase in school attendance observed in the data, there were fewer school changes observed for students once they engaged with the program, an indication of improved school stability.
Recommendations
This evaluation supports the value and viability of the TEACHaR program as a targeted, holistic education support model that can improve the educational outcomes of children and young people in OHC.
Nine key recommendations are presented:
1. That Government consider continued investment in the TEACHaR program trials beyond 2015.
2. That the differential education needs of children and young people in OHC are prioritised by the Department of Education and Training (DET) and the Department of Health and Human Services (DHHS), particularly with respect to targeted funding to address this cohort’s disproportionate experiences of educational disadvantage.
3. That early and thorough education transition planning be a mandated component of all OHC placement coordination. To achieve this it is essential that DHHS and DET assume a leadership role, particularly in the context of cross regional placement transitions. Lack of early educational planning can significantly increase the risk of children and young people becoming disengaged from school.
4. That the basic student unit funding allocated to all schools at the beginning of each year upon student enrolment, be transferable should a student move schools later in the same calendar year. Given the higher frequency with which children and young people in OHC change schools (often as a direct consequence of moving into OHC) this will help ensure that the schools they attend have access to the basic unit funding required to support their enrolment.
5. That the educational ‘Student Resource Packages’ attached to all children and young people in OHC have the flexibility to be transferred to OHC service providers and/or other Community Service Organisations (CSOs) if students become disengaged from school over a sustained period. Organisations could then utilise these resources to fund alternative educational supports for the child or young person whilst they are not attending school; in turn minimising the potential for entrenched school disengagement.
6. That the ongoing delivery of the TEACHaR program continue to adhere to the TEACHaR Operating Model (Table 3), the value of which is supported through this evaluation.
9
7. That the key program learnings identified in this report be shared broadly across the State’s OHC programs. This should include:
a. Strong efforts to ensure all children and young people in OHC attend school or an alternative educational setting, and that educational engagement be a high priority during new placement transitions. These should be minimum standards for the agency’s OHC programs, guided by a clear set of practice principles - informed by the experience and knowledge of the TEACHaR Educators
b. That all residential care staff and foster carers across the State’s OHC programs are supported to institute home learning and homework routines for children and young people in care
c. That access to a wide range of books and informal educational materials be made available within care settings, together with opportunities for informal educational enrichment via brokerage for extra-curricular activities
d. That information about key aspects of the OHC Education Partnering Agreement be included in training provided to case managers, residential care staff and carers across the state.
8. That student teachers have the opportunity to undertake a proportion of their placement practicum hours within OHC settings. This would provide the opportunity for student teachers to enrich their practice skills with an at-risk student cohort, and to develop an applied understanding of the needs and experiences of students who have entered the care system. Importantly, this is also likely to be of great benefit to students in care, providing them with additional educational support, particularly for those who may be disengaged or not attending school, and/or not reaching expected academic benchmarks.
9. That ongoing research and evaluation of the TEACHaR program be undertaken including:
a. A process evaluation to assess the key aspects of the program model, and ways of ensuring fidelity as the program expands and is replicated
b. A study comparing educational outcomes for OHC clients receiving the TEACHaR intervention to those who have not received the intervention.
10
INTRODUCTIONBackground
Anglicare Victoria is one of the state’s largest providers of out-of-home-care (OHC) services. OHC services are provided to children and young people who, due to substantiated concerns regarding their health, wellbeing or safety, have been placed into the care of state in loco parentis, via statutory orders granted by the Family Court.
Anglicare Victoria’s suite of state-funded and regulated OHC programs operate across the Eastern, Southern, Northern and Western Metropolitan areas of Melbourne, and in the regional areas of Gippsland and the Loddon Mallee. On any one night, the agency has responsibility for the care of approximately 325 children and young people in its foster care programs, 80 in its residential care programs, 55 in its kinship care programs, and 30 in its permanent care programs.
The agency’s OHC programs bring a commitment to supporting positive transitions for children and young people into adulthood. Strengthening student engagement and achievement in education is viewed as a key factor in assisting such transitions, by raising the potential for long term employment, economic independence and social, emotional and physical wellbeing.
Policy context
The importance of supporting all children and young people to access quality, state-funded education is enshrined in Government policy and legislation both nationally and in Victoria. The role of positive school achievement in improving the future life outcomes of children and young people is often cited in policy and Government frameworks, in particular the role that education can play in mitigating intergenerational cycles of poverty and socio-economic disadvantage (COAG, 2006; Commonwealth of Australia, 2011; Department of Education & Early Childhood Development, 2011).
The primary objective of the Council of Australian Governments National Education Agreement (COAG 2006, pg. 4) is that “all students acquire the knowledge and skills to participate effectively in society and employment in a globalised economy”. Further, one of the Agreement’s five primary outcomes focuses specifically on reducing educational disadvantage and increasing social inclusion, particularly for Indigenous children. Similarly, three of the 13 current ‘National Standards for Out-of-Home-Care’ (Department of Families, Housing, Community Services and Indigenous Affairs, 2011, pg. 7) explicitly reference the importance of education, and one references the importance of strengthening access to extra-curricular activities as a valuable source of informal learning.
The commitment to improving educational outcomes for children and young people in OHC is also found on a state policy level. ‘Victoria’s Vulnerable Children - Our Shared Responsibility: Baseline Performance Data Report’ (Department of Human Services, 2013, pg. 55) articulates the importance of positive educational outcomes, stating that the “Victorian Government recognises that children and young people who are in out-of-home-care require intensive targeted support to improve their educational progress and achievement”.
A key policy document in the context of this evaluation is the ‘Out-of-Home Care Education Commitment: A Partnering Agreement between the Department of Health and Human Services, Department of Education and Early Childhood Development, Catholic Education Commission and Independent Schools Victoria’ (DEECD, 2011).
The OHC Education Partnering Agreement acknowledges the disparity in school engagement and achievement that exists between students in OHC and the broader student population. To support more equitable education outcomes for these students, the OHC Education Partnering Agreement aims to improve collaboration between schools, and the Child Protection/OHC systems. It outlines a set of best practice standards to help ensure all children and young people in care attend school, that students in care have access to a range of school supports and state-funded assessments and services, and that more timely and routine information is shared between schools and case managers.
11
According to the Partnering Agreement, schools are particularly responsible for ensuring that all students in care are referred for an Educational Needs Assessment, and have a holistic Individual Learning Plan, Student Support Group, and a school-based Learning Mentor.
Literature
There is a large body of research literature that shows children and young people living in OHC are at greater risk of poorer education outcomes than those in the community more generally. Students in care often do not develop foundational education skills, are at risk of disengaging from school at an early age, complete fewer years of compulsory schooling and many do not achieve any academic qualification (Jackson & Cameron, 2012; Victorian Department of Human Services, 2013; Townsend, 2012; Australian Institute of Health and Welfare, 2011; Trout, Hagaman, Casey, Reid, & Epstein, 2008).
Lower levels of numeracy and literacy are commonly found amongst traumatised and neglected children and young people, including those who eventually enter the care system. These children are more likely to perform below grade level, to be described as ‘academically at risk’ by teachers and to be held back at school (Department of Human Services, 2013; Connelly & Furnivall, 2013; Townsend 2012; Jackson & Cameron, 2012; Mallett 2012; AIHW, 2011; Hyames & De Hamas, 2000; Wise et al; 2010; Coulombe, 2013; Wise et al, 2010; Trout et al., 2008).
The 2012 data released by the Victorian Department of Human Services (2013) show that the number of students in OHC in Victoria whose literacy and numeracy skills met minimum benchmark levels was lower than that of the broader student population at all years measured (Year 3, 5, 7 and 9 via the National Assessment Program – Literacy and Numeracy). For both literacy and numeracy, discrepancies between groups generally increased as students became older. For example by Year 9, 77% of students in OHC met the minimum benchmark for literacy and 84% for numeracy, compared to 92% and 96% of students in the wider community respectively. It should be noted that these figures are based on NAPLAN data, and only represent those enrolled in school who sat the national test.
In addition to disparities in academic performance, research has found that children and young people in OHC are likely to experience more school changes, to attend school less frequently, to be suspended more often, and are more likely to leave school prior to, or at the end of compulsory schooling (i.e at or before year 10) than other students in the community (Flynn, Tessier & Coulombe, 2013; Wise et al, 2010; Trout et al., 2008).
The recent Victorian Auditor General Report on Residential Care Services for Children (2014), painted a particularly concerning picture of the state of education performance in residential care settings, indicating that despite improvements in recent years, in 2012 only:
• 85.7% of children and young people were enrolled in school (below state-wide levels)
• 40% attended school less than five days per week, and
• 11.5% had been suspended.
Children and young people who have experienced maltreatment and entered the OHC system are also more likely to be diagnosed with a special education need or disability in their early school years (approximated at upwards of 35%), and/or to be enrolled in a special or alternative education program (Mallett, 2012; Trout et al 2008). They are also more likely to experience behavioural and emotional difficulties, than can present in the form of poor concentration, irritability, hyperactivity, violence, restlessness and a tendency to ‘shut down’ in the classroom. Less-positive interactions with peers are also common for this group, leading to difficulties maintaining friendships, and more frequent experiences of bullying, or being a ‘bully’ themselves (Flynn, Tessier & Coulombe, 2013; Wheldall & Beaman-Wheldall, 2013). Together these difficulties can have deleterious impacts on the ability of students to engage in learning tasks, to develop positive beliefs about school and their capacity as learners, and ultimately their ability to obtain foundational literacy and numeracy concepts from a young age, and to remain in school (Flynn et al., 2013; Jackson & Cameron, 2012).
12
The importance of carergiver factors on a child’s educational trajectory is also supported the research. Lack of positive educational role models and enriching home learning environments (common amongst children and young people who have experienced childhood trauma and entered the care system) have been positively correlated with poorer educational performance and aspirations across vulnerable student groups (Flynn, Tessier & Coulombe, 2013; Cheung, Lwin & Jenkins, 2012; Jackson & Cameron, 2012; Wise et al, 2010). Specific factors that have been shown to predict poor educational outcomes include intergenerational experiences of poor school engagement, caregivers spending fewer hours reading with, or completing homework tasks with children at home, lack of educational routines, fewer books in the home, less positive expectations or aspirations regarding educational achievement communicated by parents, and parents having poor educational attainment themselves (Flynn, Tessier & Coulombe, 2013; Buckingham, Wheldall & Beaman-Wheldall, 2013; Cheung, Lwin & Jenkins, 2012; Jackson & Cameron, 2012; Jackson, 2007).
Caregiver factors have the potential to be particularly salient in statutory care settings. For example, some carers and OHC providers/care staff have limited knowledge, resources, time and personal capacity to provide optimal learning support in the OHC home environment, or to demonstrate genuine or consistent interest in a student’s learning interests and needs (Connelly and Furnivall, 2013; Wise et al, 2010; Jackson, 2007). The care system also has the potential to exacerbate educational disengagement due to placement instability, or when there is a lack of robust education case planning and information sharing between schools and care teams. Such factors can increase the risk of school disengagement for students, and the potential for further emotional, psychological and social distress to arise during new school transitions (Connelly and Furnivall, 2013; Ferguson & Wolkow, 2012, Townsend 2012).
Further, Wise et al (2010) identified the need for increased training and resources to help teachers and schools more effectively respond to students demonstrating complex trauma-related behaviours. This reflects a clear recognition that many schools, classroom teachers and senior leadership teams are often inadequately equipped in terms of knowledge and/or resources, to provide differentiated curricula and the supportive school environments required by children in care, particularly in the context of complex trauma (Connelly & Furnivall, 2013; Flynn et al, 2013).
Whilst there is agreement across the literature that children in OHC are more likely to underperform compared to other students, the research also shows that many students in care in fact do well at school. The AIHW (2011) report on educational outcomes for children on guardianship orders found a number of diverse educational pathways for students in care, including that for students on continuous orders of four years, approximately 60% achieved positive patterns of numeracy and literacy benchmarks achievement over time. Further, Wise et al’s CIAO research (2011) identified a clear group of children in OHC characterised as ‘doing well’. These children were described as liking and working hard at school, with good grades, lower rates of school suspensions and a higher tendency to complete homework. These children were also more likely to have carers that placed a high importance on achieving well at school, with higher expectations of what the child in their care had the ability to achieve.
Lastly Hojer & Helena (2013) identify the potential for school and education to be viewed as a positive ‘protective environment’ by disadvantaged children. Their interviews with students in care identified the importance of the following particular factors as contributing to this ‘protective potential’: access to skilled, empathic teachers; holistic teaching principles; non-stigmatising inclusive environments; the ability to repeat a school year if needed, and school stability. It was concluded that together these factors helped students to develop more positive beliefs and aspirations about what they could achieve in school and into the future.
13
Anglicare Victoria’s TEACHaR program
TEACHaR (‘Transforming Educational Achievement for Children and in Home-based and Residential care’) seeks to curb the poor educational trajectories often experienced by children and young people in OHC.
The program was first implemented late 2012 in Anglicare Victoria’s Eastern foster care and Southern residential care programs via significant philanthropic funding from the Helen Macpherson Smith Trust and others. The pilot program was then expanded in 2014 into the Northern Metro Region, and more broadly in the Eastern Metro region via additional funding from State and Federal Governments.
Note:
Despite the recent expansion of the TEACHaR 1 (pilot program), this evaluation is targeted to the outcomes associated with the first two years of the pilot TEACHaR program.
Results associated with the TEACHaR 2 and 3 program streams are not presented in this report. These may be made available in the future.
Table 1. TEACHaR program streams
Programs and Targets Funding Period
Funders
TEACHaR 1 (Pilot)
Eastern metro home-based care and Southern metro residential care
December 2012 to January 2016
Helen Macpherson Smith Trust; The Alec Prentice Sewell Gift managed by The Ian Potter Foundation; The Vera Moore Foundation; The R.M. Ansett Trust, the Mars-Stride Trust, and the William Henry Pawsey Charitable Trust all managed by Equity Trustees; and The Kimberley Foundation.
TEACHaR 2 (Disabilities focused)
Eastern metro home-based, residential and kinship care.
Targeted to children and young people in OHC with a disability or additional need as defined by the Disability Discrimination Act
April 2014 to December 2015
The Victorian Department of Education and Training
The Federal Government’s ‘More Support for Students with Disabilities’ Education Initiative
TEACHaR 3 (Residential care focused)
Northern metro residential and home-based care. Clients of Anglicare Victoria and other residential care service providers
June 2014 to December 2015
The Victorian Department of Health and Human Services
14
i) Program aims
The primary objectives of the TEACHaR program are to:
• Improve the education opportunities and outcomes of children and young people living in OHC
• Develop an education support model for children in OHC that is programmatically and financially sustainable
Further, the program aims to support children and young people in OHC to:
• Attain literacy and numeracy skills to the same standard as other Victorian students, consistent with their ability and potential
• Participate in compulsory schooling
• Complete Year 12 or equivalent at the same rate as other Victorian students.
ii) Program model
The TEACHaR model prioritises direct support to children and young people in OHC, both in their homes and their classroom environments, rather than through referrals to other services. The program also aims to work with students for at least six months, although actual duration is dependent upon need, and a client’s continued placement in an Anglicare Victoria OHC program.
A key aspect of the model’s approach is the employment and remuneration of highly experienced, registered teachers. These educators bring a range of classroom, welfare and senior level teaching experience, together with a range of complementary higher qualifications, such as special education and social work.
The program’s holistic operating model is summarised in Table 3. It shows the three contexts within which the program works (student, school and placement), the ‘areas of change’ within which these contexts operate, and the key service intervention activities employed.
The TEACHaR program logic is presented in Figure 1.
As identified in the operating model, the program supports children and young people within classrooms and care settings, via one-on-one tuition, group work and flexible, informal learning. The program is also widely collaborative, working in partnership with schools, carers, case managers, residential workers, student wellbeing officers, Child Protection and Placement Coordination staff.
Other key interventions include:
• Assisting and providing funding for students to access educational and allied health assessments, complementary services, extra-curricular activities and learning resources
• Providing advocacy and liaison in the context of school disengagement, complex classroom behaviours, placement changes, care planning and school transitions
• Providing information, in-class support and professional development to teachers and schools about the care system, the impacts of developmental trauma, and the special education needs and learning difficulties often experienced by children and young people in care
• Supporting schools to implement the OHC Education Partnering Agreement. TEACHaR educators often attend care team meetings and Student Support Groups, assist teachers to develop individual educational plans, and help schools to identify student mentors.
15
The TEACHaR pilot program comprises a team of one Lead Educator/Coordinator (0.8 EFT), and two Educators (one at 0.8 EFT, the other at 1.0 EFT). The Lead Educator has primary operational responsibility for the program, and provides supervision to the other TEACHaR staff in addition to carrying a caseload of clients. The pilot program aims to support approximately 40 children and young people in OHC at any one time.
iii) Program implementation
At the time of publishing this report (approximately 2 ½ years into service delivery) the three TEACHaR program teams have supported a growing number of students, schools and school staff, as presented in Table 2 below.
Further, the number of children and young people that the program has also reached via informal support to school teachers and case managers, and through inclusive teaching practices (such as group work in classrooms and OHC settings) is also considerable.
Whilst for operational and evaluation purposes the programs are delivered as three separate streams, the teams work closely together, and their practice is guided by the same operating model, evaluation framework, service documents and governance structure.
Evaluation aims
This evaluation seeks to measure the impact of the TEACHaR 1 (Pilot) on children and young people who have been engaged with the program over a 12 and/or 24 month period. Outcomes were measured across the three contexts (student, school and placement) that the program operates within, at baseline and then at six month intervals. The evaluation framework utilises a repeated measures design, drawing on quantitative and qualitative data.
The overarching aims of this 24 months outcomes report are to:
1. Determine the level of change from baseline to 12 months across the domains of student support, school engagement and attitudes to learning, academic skills and achievement, the home/care learning environment, cultural activities and barriers to learning
2. Determine the level of change from baseline to 24 months across the domains identified above
This report outlines the evaluation methodology, provides 12 month and 24 month quantitative outcomes data, together with qualitative data derived from the service experiences of TEACHaR Educators. Recommendations are also presented.
Table 2. Summary of TEACHaR service delivery, December 2012 – July 2015.
TEACHaR 1 (Pilot Program)
TEACHaR 2 (Disabilities-focused)
TEACHaR 3 (Residential care-focused)
TOTAL
Approx. number of children/young people
70 58 44 172
Approx. number of schools/educational placements
75 48 33 156
Approx. number of school staff (including teachers, leadership staff, and wellbeing officers)
200 135 90 425
16
• Educational specialists
• Brokerage
• Management and governance
• Professional development
• Office infrastructure
INPUTS
• Education assessment and planning
• Direct educational support (tutoring)
• Arrangement of supplementary assistance and support (including flexible learning options)
• Facilitation of participation in cultural and leisure activities
• Student counselling and support
• Engagement and re-engagement activities
• Student Support Group meetings
• Teacher liaison and enrichment of classroom environment
• Carer/parent liaison and enrichment of home learning environment
• Case manager liaison
• Advocacy
ACTIVITIES
Table 3. TEACHaR Operating Model: Intervention Targets and Key Service Activities
Holistic ApproachStudent Context School Context School Context Placement Context
Targets for intervention
Engagement with school and learning
Intensive learning support
Personal support Teaching approach Education planning and support
Intensive learning support
Home-based learning Care and case planning
Key service activities
Funding of cultural, sporting and other extra-curricular activities
Supporting transitions into alternative educational placements
1:1 tutoring
Sourcing and funding subject-specific tutoring for clients in secondary school
Advocating for specialist educational, optometry, occupational, speech and auditory processing assessments
Funding of Allied Health and alternative therapies
Transport to, and from educationally-based appointments
Rapport-building activities
Teacher liaison on the impact of trauma on a child’s ability to engage in learning
Support and discussion in relation to behavioural issues, and appropriate management strategies
Heightening schools’ awareness of their responsibilities regarding the OHC Education Partnering Agreement
1:1 and peer group teaching
Educational information and support to carers
Development of educational plans for residential care units
Support and advocacy when transitioning clients into new educational placements
Provision of support and information to agency placement workers in relation to educational matters
Attendance at case plan and care team meetings
Figure 1. TEACHaR Program Logic Model
17
• Increased understanding of education needs
• Improved education planning
• Increased school (or alternative education) participation and engagement
• Improved academic skills and performance
• Enrichment of home learning environment
• Increased participation in cultural and leisure activities
• Reduction in personal issues/difficulties that create a barrier to education
SHORT-TERM OUTCOMES:
IMPROVEMENTS IN LEARNING OUTCOMES
• Increased proportion of OHC clients completing year 12 or other education qualification
• Increased proportion of OHC clients with skills for employment
LONG-TERM OUTCOMES:
HIGHER PROPORTION OF OHC CLIENTS WITH EDUCATION QUALIFICATIONS AND SKILLS FOR EMPLOYMENT
Table 3. TEACHaR Operating Model: Intervention Targets and Key Service Activities
Holistic ApproachStudent Context School Context School Context Placement Context
Targets for intervention
Engagement with school and learning
Intensive learning support
Personal support Teaching approach Education planning and support
Intensive learning support
Home-based learning Care and case planning
Key service activities
Funding of cultural, sporting and other extra-curricular activities
Supporting transitions into alternative educational placements
1:1 tutoring
Sourcing and funding subject-specific tutoring for clients in secondary school
Advocating for specialist educational, optometry, occupational, speech and auditory processing assessments
Funding of Allied Health and alternative therapies
Transport to, and from educationally-based appointments
Rapport-building activities
Teacher liaison on the impact of trauma on a child’s ability to engage in learning
Support and discussion in relation to behavioural issues, and appropriate management strategies
Heightening schools’ awareness of their responsibilities regarding the OHC Education Partnering Agreement
1:1 and peer group teaching
Educational information and support to carers
Development of educational plans for residential care units
Support and advocacy when transitioning clients into new educational placements
Provision of support and information to agency placement workers in relation to educational matters
Attendance at case plan and care team meetings
18
METHOD Sample
Over the first two years of the TEACHaR 1 (Pilot), a total of 63 children and young people received some length of intervention.
Of these 63 children and young people, 26 were involved with the program for less than 12 months. This report presents outcomes data for an overlapping sample of children and young people who have received at least 12 months of intervention. The 12 month group is comprised of 37 children and young people. The 24 month group (n=15) is comprised of a sub-sample of the 12 month group who have continued with the program for 24 months.
There is substantial variation in intervention length between children and young people in foster and residential care. For example, of the 26 children and young people who received less than 12 months of intervention, 53.8% (n=14) were in residential care. As service length increases, the proportion of children and young people in residential care decreases. Of the 37 children and young people who have been involved in the program for more than 12 months, 75.6% are in foster care. This difference is mostly at the 18 and 24 month assessment periods. For example, at the 18 month assessment, 80% of the sample was in foster care, increasing to 87% at the 24 month assessment.
Demographic information pertaining to the 37 children and young people who have received at least 12 months of intervention is presented in the Results section of this report.
Measures
The TEACHaR ‘Assessment and Intervention Tool’ (devised for this program by Sarah Wise, 2012) draws upon the AUSVELS curriculum standards and the Looking After Children (LAC) Assessment. Building on the results of the CIAO study (Wise et al., 2010) the tool also contains measures designed for the purpose of the program, specifically in relation to the importance of school engagement and learning attitudes, the role of the care environment and potential barriers to learning.
The assessment tool contains 108 items across the student, school and placement contexts, covering the following:
• Client demographics and placement information
• Education planning and support
• School participation and motivation/engagement
• Academic skills and performance
• The home and care environment
• Engagement in extracurricular and cultural activities
• Personal factors that impact engagement with learning
Outcome indicators were clustered into six domains, relating to student support in schools, school engagement and attitudes to learning, academic skills and achievement, the OHC learning environment, cultural activities and barriers to learning. Given the variety of questions used to measure each indicator, the following section presents a brief overview of the main variables used in this report, along with information about coding and scoring.
19
DOMAIN 1. STUDENT SUPPORT IN SCHOOLS
Student support in schools was measured through three items reflecting the core elements of the Victorian Government’s OHC Education Partnering Agreement, including the presence of Student Support Groups, Learning Mentors and Individual Learning Plans. Each item was rated dichotomously as either present or absent.
A summed score, reflecting overall student support in school was also created. This score ranges from 0 to 3 with higher scores indicating a greater number of in-school supports.
DOMAIN 2. SCHOOL ENGAGEMENT AND ATTITUDES TO LEARNING
School engagement and attitudes to learning was measured through a combination of items, reflecting the broad scope of this construct. While school attendance is important, the extent to which students are actively engaged as learners will also impact on their academic outcomes. For the purpose of this report, five items used to assess school engagement and attitudes to learning: school attendance, school stability, happiness to be at school, level of active involvement in learning, and how hard a child or young person were working.
School attendance was measured as the average number of hours attended per week during the four week period prior to an assessment. School stability was measured as the number of school changes a child or young person had experienced over the 12 months preceding a given assessment (i.e., 12 months prior to the baseline assessment)
Happiness to be at school and level of active involvement were measured on a 4-point Likert-type scale, from 1 (always) to 4 (never). Working hard at school was measured on a 7-point Likert-type scale, from 1 (much less than other children) to 7 (much more than other children).
DOMAIN 3. ACADEMIC SKILLS AND ACHIEVEMENT
This domain was operationalised as: age-appropriate literacy, numeracy and concentration skills, academic performance matching a student’s ability, and overall academic achievement. The literacy, numeracy and concentration skills variables were measured on a 4-point Likert-type scale from 1 (Above average) to 4 (Well below average). Overall achievement was measured on a 5-point Likert-type scale, from 1 (Excellent) to 5 (Well below average). The performance matches ability variable was measured on a 3-point Likert-type scale (1=Performance matches ability and 3=Performance seriously below ability).
DOMAIN 4. THE OHC HOME LEARNING ENVIRONMENT
Three items were used to measure supportive learning environments in an OHC setting. The first item assessed the frequency that carers talked to a child or young person about school activities and was rated on a 5-point Likert-type scale (1= Daily and 5 = Rarely or never). The second item assessed the frequency that a child or young person participated in literacy-based activities in the home (e.g., reading or being read to). This item was rated on a 3-point Likert-type scale, from 1 (Frequently) to 3 (Rarely or never). The final item assessed how often the child or young person read for pleasure, rather than as a schooling requirement. This item was rated on a 6-point Likert-type scale, ranging from 1 (Every day) to 6 (Almost never).
20
DOMAIN 5. CULTURAL ACTIVITIES
Client participation in a range of cultural or extra-curricular activities was measured via a cultural activities scale comprising the following seven items: play a musical instrument, had lessons or gave a dance performance, had lessons or gave a singing performance, participated in drama, visited a public library, visited a museum or art gallery and attended a performing arts event. Each item was dichotomously score as either Yes (1) or No (0). Individual scores for the seven items were summed to create an overall ‘cultural activities’ score , with a range of 0 to 7. Higher scores indicate a greater number of cultural activities engaged in over a 12 month period.
DOMAIN 6. BARRIERS TO LEARNING
The barriers to learning domain is comprised of three indicators covering individual, school-based and placement barriers. The individual barriers indicator is comprised of six items that assess a student’s confidence and engagement with the learning environment. The school-based barriers indicator is also comprised of six items assessing the school culture and approach to teaching. Finally, the placement-based barriers indicator is comprised of two items, that assess stability and relationships. All items are rated on a 3-point Likert-type scale ranging from 0 (Not a barrier to learning) to 2 (Major barrier to learning).
Three summed scores were created, corresponding to each of the indicators. For individual and school-based barriers, the summed score ranged from 0 to 12, with 0 representing no barriers and 12 representing major barriers. For placement-based barriers the summed score ranged from 0 to 4. As with the other two summed scores, 0 represents no barriers and 4 represents major barriers.
Strengths and Difficulties Questionnaire (SDQ: Goodman, 1997)
The SDQ was utilised to measure the emotional and behavioural difficulties experienced by a sub-sample of 18 children and young people in foster care placements. This data was extracted from available LAC data contained in client case file records. The SDQ is comprised of 25 items, which produce five ‘scale’ scores corresponding to the emotional symptoms, conduct problems, hyperactivity, peer problems and prosocial skills. A total score is also obtained. Each item is rated on a 3-point Likert-type scale, ranging from 0 (Not true) to 2 (Very true). Scale scores range from 0 to 10, with higher scores indicative of greater ‘dysfunction’ that signals the potential for clinically significant problems. The only exception is the Prosocial scale, where higher scores are indicative of greater ‘functioning’ in this domain.
The total score is calculated by summing the scale totals for ‘Emotional Symptoms’, ‘Conduct Problems’, ‘Hyperactivity’ and ‘Peer problems’, leading to a range of 0 to 40, where higher scores are indicative of greater difficulties in emotional, behavioural and psychosocial functioning. The ‘Prosocial’ scale is not included in the Total Difficulties score.
Each scale, as well as the Total Difficulties score is converted to a ‘clinical’ score, reflecting broad-based age-norms. These clinical cut-off points are used to identify the proportion of the population showing elevated dysfunction that can signal the need for clinical intervention.
Qualitative data
Supplementary qualitative data was also collected to gain some understanding of the program’s implementation and perceived benefit, from the perspective of educators. This includes case studies devised by educators, guided by a common template (created by the researcher), and secondary data derived from quarterly operation reports and regular attendance at Governance Group and Educator team meetings by the research team.
Data Collection Procedures
The TEACHaR ‘Assessment and Intervention Tool’ is utilised by Educators as part of their routine practice with clients. Data is collected on a 6 monthly basis whilst children/young people are clients of the program, commencing at service entry. The initial assessment is completed over the first month of the intervention, however for some clients this process can take longer if service engagement and/or data collection requires more time.
21
Completed assessments are informed by educators’ direct practice with, and observation of students, together with information derived from agency case file and LAC data, school report cards and test results, and information provided by clients, classroom teachers, carers and caseworkers. The Assessment Tool is also intended to strengthen practice, in particular the assessment of educational needs and progress, and the ongoing formulation and review of service goals and activities.
All data used for the evaluation has been stored electronically in a shared, protected network drive accessible only by the educators and Research team. Client names were recorded on the research database, with students identifiable via their Client Information (CRIS/SP) number.
Analysis framework
Rationale for statistical analyses
Descriptive and inferential data are presented in this report according to six key domains:
1. Support in schools
2. Engagement and attitudes to learning
3. Academic skills and achievement
4. Engagement in cultural activities
5. Learning in the context of the care environment
6. Barriers to learning
Due to the high ratio between outcome variables and the maximum number of overall cases (n=37), it was only possible to undertake a limited number of inferential analyses via paired-sample t-tests . Five tests were conducted between baseline and 12 month data for the following key variables.
Domain One: Summed Score of three in school student support variables
Domain Two: Active engagement in learning
Domain Three: Overall academic achievement
Domain Six: Scale of individual barriers to learning
Scale of school based barriers to learning
Given the small sample size and the number of tests applied to the data, a Bonferroni correction was calculated, resulting in an adjusted significance level of p>0.01.
As the 24 month data presented in this report reflect a more limited sample of 15 cases, inferential analyses for these data were not conducted.
For all data presented in this report, the valid percentage has been presented to represent the impact of missing data on each analysis.
Further information about the evaluation’s approach to ‘missing and ‘don’t know’ response data is located in Appendix A.
22
RESULTSDEMOGRAPHIC DATA
Demographic data for the 63 children and young people who have been involved with the TEACHaR 1 (Pilot) over the two year evaluation period are presented in Tables 4 and 5.
Table 4. Profile of TEACHaR clients at time of entry into the program (n=63).
Age at service commencement
5-8 years
9-12 years
13-15 years
16-18 years
23%
33%
22%
22%
GenderMale
Female
57%
43%
Country of birthAustralia
Other
95%
5%
Aboriginal and/or Torres Strait Islander
Yes 16%
Placement typeFoster care
Residential care
63%
37%
School level
Primary School
Secondary
Alternative School/TAFE
Not attending school
56%
27%
6%
11%
Challenges to Learning
Cognitive impairment, learning disability or learning related difficulties
Yes
Referral in place
22%
11%
Diagnosed mental health issuesYes
Referral in place
29%
10%
Emotional or behavioural difficulties Yes 79%
23
Table 4 presents the SDQ data for a sub-sample of 18 children and young people in foster care who have been engaged with TEACHaR, against a larger comparison sample of 250 Anglicare Victoria OHC clients not engaged in the pilot TEACHaR intervention. This comparison data was collected as part of Anglicare Victoria’s Children In Care Report Card, 2015.
The SDQ data shows that a greater proportion of TEACHaR clients scored within the ‘very high’ range of scores for emotional symptoms, conduct problems and hyperactive, compared to the broader OHC population of Anglicare Victoria clients. This pattern was reversed for the peer problems and prosocial behaviour scales, where a lower proportion of TEACHaR clients scored within the ‘very high range’. However, when comparing the Total Difficulties Scale, an equivalent proportion of TEACHaR clients and children and young people in the broader OHC population scored within the ‘very high’ range.
While it was not possible to test for statistically significant differences between the two samples, the patterns in Table 5 indicate that a unique cluster of emotional and behavioural difficulties may be apparent among children and young people who have received the TEACHaR service. This is an area of the project that clearly warrants further research.
Table 5. SDQ data for TEACHaR clients at service entry and comparison group of Anglicare OHC clients
SDQ Scale and Score TEACHaR clients (n=18)
Comparison OHC clients (n=222)
Emotional symptoms:
High
Very High
17.6%
23.5%
21.2%
11.2%
Conduct problems: High
Very High 29.4%
23.5%
25.4%
18.5%
Hyperactivity:
High
Very High
23.5%
29.4%
6.9%
16.1%
Peer problems:
High
Very High
29.4%
5.9%
16.1%
26.9%
Prosocial behaviours:
High
Very High
11.8%
23.5%
10.9%
32.3%
Total Difficulties Score:
High
Very High
23.5%
29.4%
12.0%
28.4%
24
QUANTITATIVE DATA
12 MONTH OUTCOMESOutcomes data for the 37 children and young people who have received at least 12 months of the TEACHaR 1 (Pilot) over the two year evaluation period are presented below3.
DOMAIN 1. STUDENT SUPPORT IN SCHOOLS
Figure 2 shows that over the course of 12 months there was an increase in the proportion of children and young people who had been allocated a student support group or learning mentor, and who had an individual learning plan.
The largest proportional increase was seen in the category of learning mentor, where close to all students were allocated a mentor by 12 months, compared to only 33% at Baseline.
In order to test for change across this outcome domain, a summed Student Support in School variable was utilised. A significant difference was found between the baseline (M=1.6, SD=1.0) and 12 month assessment (M=2.8, SD=0.5): t (32) = - 7.016, p<.01. This indicates that children and young people experienced a significant increase in global student supports over a 12 month period.
3. Please note that in some instances there is substantial variation in sample sizes across measurement periods. These variations have been noted where possible.
Figure 2. Student support in schools
% O
F SA
MP
LE
STUDENT SUPPORT GROUP
Child/young person has a Student
Support Group within schools
LEARNING MENTOR
Child/young person has an identified school-
based learning mentor
INDIVIDUAL LEARNING PLAN
Child/young person has an Individual
Learning Plan
100
90
80
70
60
50
40
30
20
10
0
BASELINE (n=34) 6 MONTHS (n=36) 12 MONTHS (n=35)
82.4%
33.3%
57.6%
77.8%
88.2%
80.6%
91.4%97.2% 100%
25
DOMAIN 2. SCHOOL ENGAGEMENT AND ATTITUDES TO LEARNING
There was a general pattern towards improvement across the three school engagement and attitudes to learning variables, as demonstrated in Figure 3.
The largest increase was observed in the happy at school variable, where just over half of students were rated as always happy at school at baseline, increasing to close to 95% at the 12 month follow-up.
For the active engagement in learning variable, a pairwise analysis was conducted between baseline (M= 2.3, SD=0.73) and 12 month data (M=2.1, SD=0.57). Despite the decrease in scores, this difference was not statistically significant. These results indicate that while there was a general improvement in the proportion of children and young people who were happy at school, actively involved in learning and working hard at school, the was no evidence of a statistically significant improvement for the measure of active engagement specifically.
Figure 3. School Engagement and Attitudes to Learning – Baseline to 12 months
% O
F SA
MP
LE
Always or usually happy at school
Always or usually actively involved in
learning tasks
Working hard at school at an average
level or above
100
90
80
70
60
50
40
30
20
10
0
BASELINE (n=34) 6 MONTHS (n=37) 12 MONTHS (n=37)
58.8% 58.8%
32.4%
51.2%
59.4%64.9%
78.4%78.4%
94.6%
26
School attendance:
There was a trend of consistent improvement in average hours of school attendance as children and young people progressed through the program, as indicated in Figure 4.
Attendance increased by two hours between baseline and 6 months, and by 1.5 hours between 6 months and 12 months, reaching a maximum mean of 29.9 hours per week – which is close to full time attendance. Further, at baseline a greater proportion of the sample had not attended school over the previous four weeks (11.1%, n=4), compared to 2.8% (n=1) at 6 months, and 0% at 12 months.
A pairwise comparison of baseline and 12 month school attendance data was not conducted, due to the fact that at least 70% of the sample were attending school full time at baseline, in particular the majority of students in foster care. Any change between means is likely to be impacted by a small group within the sample with particularly low school attendance compared to the majority of the sample.
School stability
The measure of school stability reflects the number of school changes a child or young person has experienced in the 12 month period prior to an assessment. As such, the baseline measure of school stability refers to the number of school changes experienced prior to the baseline assessment. Similarly, the measure of school stability at follow up refers to the number of school changes that occurred between the original assessment at baseline and the 12 month assessment.
Figure 4. Change in average hours of school attendance per week - Baseline to 12 months
ME
AN
HO
UR
S P
ER
WE
EK
32.5
30
27.5
25
22.5
20
17.5
15
12.5
10
7.5
5
2.5
0
Baseline (n=35)
26.4
6 months (n=35)
28.4
12 months (n=35)
29.9
27
Figure 5 shows a pattern of improved school stability following exposure to the TEACHaR intervention. Over the course of the 12 months, 86.5% of students remained in the one school setting, compared to 69.4% of students in the 12 month period preceding engagement with TEACHaR. Further to this:
• In the period prior to engagement with TEACHaR, 25% of students had attended two or three schools, which reduced to 13.5% of the sample following 12 months exposure to the intervention (capped at two schools only).
• 5.6% of students were not enrolled in school at any time prior to TEACHaR engagement, compared to 0% of students over the 12 month TEACHaR intervention period.
Figure 5. Change in school stability, 12 months prior to, and 12 months following exposure to the TEACHaR intervention.
% O
F SA
MP
LE
100
90
80
70
60
50
40
30
20
10
0
12 months prior to baseline (n=36)
No schools
5.6%0.0%
One school
69.4%
86.4%
Two schools
11.1% 13.5%
Three schools
13.9%
0.0%
Baseline to 12 months period (n=36)
NUMBER OF SCHOOLS ATTENDED OVER PREVIOUS 12 MONTH PERIOD
284. For the five academic achievement variables, there was a high number of ‘Don’t Know’ responses within the Baseline data. Potential explanations for this are outlined in the methodology section of this report.
DOMAIN 3. ACADEMIC SKILLS AND ACHIEVEMENT
Figures 6 and 7 show positive shifts across the five academic skill and achievement variables measured over 12 months4.
There was a general trend towards improvement across the three ‘skills’ variables, with a higher proportion of children and young people scoring ‘average or above’ on literacy, numeracy and concentration skills from baseline to 12 months. The largest proportion increase was seen for literacy skills, with just under 20% of children and young people scoring ‘average or above’ at baseline compared to over 50% at the 12 month follow-up.
Figure 6. Percentage change in literacy, numeracy and concentration skills – Baseline to 12 months.
% O
F SA
MP
LE
Literacy skills, average or above
Numeracy skills, average or above
Concentration skills, average or above
100
90
80
70
60
50
40
30
20
10
0
BASELINE 6 MONTHS 12 MONTHS
17.1%13.9%
19.4%
48.6%
56.8%
36.4%
48.6%45.7%
56.8%
29
There was also a general pattern of improvement in the proportion of children and young people whose academic performance was assessed as matching their ability across the three assessment periods (baseline, 6 months and 12 months). Improvement on this variable appears to have plateaued from the 6 month to 12 month assessment however.
There was a more marked increase in the proportion of children and young people rated at average or above for overall academic achievement over time. A significant difference between baseline (M=4, SD=0.87) and 12 months (M=3.5, SD=1.0) was also found for this variable, t(28) = 2.93, p<.01 This result lends increased support for the intervention’s capacity to benefit student academic performance.
Figure 7. Percentage change for overall academic performance – Baseline to 12 months.
% O
F SA
MP
LE
Academic performance matches ability
Overall academic achievement, average or above
100
90
80
70
60
50
40
30
20
10
0
BASELINE 6 MONTHS
17.1%
45.7%
56.8%
16.7%
38.9% 40.5%
12 MONTHS
30
DOMAIN 4. THE OHC HOME LEARNING ENVIRONMENT
Three variables comprise the OHC learning environment domain, covering carer involvement in learning, literacy activities in the home and the frequency with which students read for pleasure.
As seen in Table 6, for both carer involvement in student learning outside of school and student engagement in literacy in the home, there were trends towards improvement from baseline to the 6 month assessment, which were then sustained at 12 months. The exception was the reading for pleasure variable, which remained stable from baseline to 6 months, but then increased at the 12 month assessment. While it is not possible to state that this increase was statistically significant, there was nevertheless a slight trend towards improvement on this variable over the 12 month period.
These patterns suggest that while positive gains were observable in the early phase of program involvement, they may be more difficult to sustain over time, in part due to the nature of OHC environments, particularly in residential care. However, given the quantity of ‘Don’t know’ responses (refer to Appendix A) it is also possible that Educators may not be in the best position to directly or accurately assess the quality of carer involvement.
The only variable that showed an improvement from the 6 month to 12 month assessment was reading for pleasure which is likely to be correlated to more academically oriented variables, and therefore may be more directly impacted by changes on the academic skills and achievement domain. As discussed in the previous section, there was an overall trend towards improvement on this domain, which provides some, albeit tentative support for the contention that ‘reading for pleasure’ may be more closely affected by changes in academic ability, than changes in the home/carer learning environment.
DOMAIN 5. CULTURAL ACTIVITIES
There was a general trend towards increased cultural activities across the three assessment periods. While it was not possible to test for statistical significance, the data in Table 7 nevertheless show that children and young people experienced an increased range of cultural activities as they progressed through the program.
Table 6. Mean barriers to learning score, Baseline to 12 months
Proportion (n)
Domain Baseline 6-months 12-months
Learning in the home/careenvironment
Number of carers who talk to the child/young person about school activities on a daily basis
88.2% (n=34)
94.6% (n=36)
91.7% (n=36)
Number of clients who participate in literacy-based activities at least several times per week in the OHC home environment
50% (n=36)
73% (n=37)
67.6% (n=33)
Number of clients who read for fun or pleasure a few times per week or more5
48.6% (n=33)
48.6% (n=35)
59.5% (n=37)
5. There were a consistently high number of ‘Don’t Know’ responses to this item over each data point: baseline n=12, 6 months n=11, 12 months n=7. Any change observable from baseline should be considered in the context of these ‘Don’t Know’ responses. 6. Given the variability in sample sizes across assessment periods, means were calculated based on only the 22 children and young people who had information on cultural activities at baseline, 6 months and 12 months. While the means were lower at 6 months and 12 months (M=3, SD=2.4 and M=2.9, SD=2), the pattern of improvement is nevertheless observable regardless of the variability in sample size.
Table 7. Cultural Activities, Baseline to 12 months
Mean (Standard Deviation)(n)
Domain Baseline 6-months 12-months
Cultural Activities Summed Score (Range 0 to 7)
2.5 (2.5) (n=22)6
3.1 (2.2) (n=32)
3.3 (2.1) (n=37)
Indicators
31
DOMAIN 6. BARRIERS TO LEARNING
Table 8 provides information about students’ experiences of barriers to learning across individual, school-based and placement contexts.
For the individual barriers summed score, there was a clear trend towards a decreasing number of barriers from baseline to 6 months, and from 6 months to 12 months. This decrease was statistically significant when comparing baseline (M = 6.0, SD=3.5) to 12 month assessments (M = 4.1, SD=3.1), t(28) = - 5.255, p<.01.
There was also a noticeable trend towards decreased school-based barriers over the three assessment periods, such that by the 12 month assessment the mean score for this barrier had reduced to 1.3. However, this change was not statistically significant.
Finally, it is noteworthy that placement barriers were moderate to low across the three assessment periods. Given the over-representation of children and young people in foster care, and the fact that these placements tend to be relatively more stable than residential care placements, the relatively low average placement barriers scores may simply be a function of the sample, rather than an accurate reflection of the extent and impact of placement barriers on educational outcomes. This is an area that also warrants further research.
24 MONTH OUTCOMES
Data for the 15 children and young people who received 24 months of the TEACHaR 1 (Pilot) intervention over the evaluation period are presented below. As this is a small sample, pairwise comparisons are not advised. As such, the data presented throughout this section of the report provide a preliminary assessment of the profile of children and young people who have experienced 24 months of intervention, and an initial, descriptive analysis of outcomes.
The majority of this sample (87%) reside in foster care, representing a skew in these data towards outcomes for students in this care group in particular. This may be explained by the fact that children and young people in residential care have typically experienced shorter interventions (see sample description in the Method), potentially as a result of the greater instability of placements associated with this care type.
A summary of baseline, 12 month and 24 month data across six outcome domains are presented in Tables 9 and 10, and Figure 9. Where data are based on a reduced sample, the alternative sample size is presented.
7. There was a trend towards significance from the baseline assessment (M = 2.4, SD = 3.3) to the 12 month assessment (M = 1.2, SD = 2.0), t(28) = 1.871, p = 0.07.
Table 8. Mean barriers to learning score, Baseline to 12 months
Mean (Standard Deviation)(n)
Domain Indicators Baseline 6-months 12-months
Barriers to Learning
Personal barriers score (Range 0-12)
6.0 (3.5) (n=30)
5.1 (3.2) (n=34)
4.5 (3.1) (n=36)
School-based barriers score(Range 0-12)
2.6 (3.3) (n=30)
2.3 (3.1) (n=33)
1.3 (2.2) (n=36)
Placement barriers score(Range 0-4)
1.7 (1.6) (n=34)
1.6 (1.6) (n=36)
1.5 (1.6)(n=37)
32
Support in schools
For the Support in schools domain, the overall pattern was one of improvements from baseline to 24 month assessment, although in most instances this was associated with one extra child receiving additional supports. The greatest improvement can be seen for the proportion of students with a school-based learning mentor, which increased from 14.3% (n=2) at baseline to 93.3% at the 24 month assessment (n=14).
This may be attributed to a greater emphasis on the OHC Education Partnering Agreement in conjunction with the increased awareness at the school-level of the TEACHaR program, and its intervention model. In contrast, there was a decrease in the proportion of children and young people with an Individual Education Plan at the 24 month assessment, although this is due to one student moving into an alternative educational setting where other forms of student support are provided.
Table 9. Change in school support, engagement, academic skills and achievement, and learning in the care environment – Baseline to 24 months (n=15)
Number and valid percent of clients
Domain Indicators Baseline 12-months 24-months
Support in schools
Child/young person has a Student Support Group within schools
85.7% (n=14)
100% 100%
Child/young person has an identified school-based learning mentor
14.3% (n=14)
86.7% 93.3%
Child/young person has an Individual Education Plan
57.1%(n=14)
93.3% 86.7%
School engagement and to attitudes to learning
Always or usually happy at school 78.6%(n=14)
100% 80%
Always or usually actively involved in learning tasks
71.4%(n=14)
86.7% 80%
Working hard at school at an average level or above
50%(n=14)
73.4% 66.7%
Academic Skills and Achievement8
Literacy skills average or above 26.7% 46.7% 46.7%
Numeracy skills average or above 25%(n=12)
33.3% 42.9%(n=14)
Concentration skills average or above 33.3% 60% 66.7%
Academic performancematches their ability
33.3% 40% 46.7%
Overall academic achievement, average or above
13.3% 53.3% 60%
Learning in the home/care environment
Carers talks to client about school activities on a daily basis
92.9%(n=14)
93.3% 86.7%
Participates in literacy-based activities several times per week in the OHC home environment
78.6%(n=14)
66.7% 46.7%
Reads for fun or pleasure a few times per week or more
57.1.%(n=14)9
73.3% 80%
8. The number of ‘Don’t Know’ responses at Baseline for this domain were as follows; Literacy n=2, Numeracy n=3, Concentration n=2, Overall achievement n=3 and performance matches ability n= 3. The number of ‘Don’t Know’ responses diminished to 1 at 12 months and 0 at 24 month data across all five variables. 9. Don’t Know responses for this variable were as follows: Baseline=4, 12 months=2, 24 months=0.
33
School engagement and attitudes to learning
For this domain there was an overall trend towards improvement from baseline to 24 months, although here too the gains were modest. The greatest improvement can be seen in the ‘working hard at school’ indicator, where only 50% (n=7) of students were rated as average or above on this variable at baseline, compared to 66.7% (n=10) students at the 24 month assessment. There was, however, a small decrease from the 12 month to the 24 month assessment, a pattern which can be seen throughout this domain. These patterns may reflect a ‘critical dosage’ effect, whereby the greatest gains associated with the TEACHaR model are observed during the first 12 months, but taper thereafter. This is an aspect of the model that requires greater research attention.
School attendance
There was little change observed across the 24 month intevention period for school attendance; with 93.3% of the students attending school on an approximate full-time basis at each assessment. Average weekly attendance in hours was as follows: baseline (M=30, SD 8.3), 12 months (M=30.4, SD=8.0) and 24 months (M=29.7, SD=7.7). There was n=1 student in the sample not attending school at baseline however, and while there was some improvement observed in this students’ attendance over time (12 months = 1.5 hours, 24 months = 2 hours), this change was nonetheless small.
School stability
A pattern of high school stability was also observed across the sample, with very few school changes observed either before or after students’ exposure to the intervention. Whilst there is a minor trend of improvement in Figure 8, this is largely attributable to change in n=1 student between baseline and 12 months, and 12 months and 24 months respectively.
Figure 8. Change in school stability, 12 months prior to, and following 24 months exposure to the TEACHaR intervention (n=15)
% O
F SA
MP
LE
12 months prior to baseline
Baseline to 12 months 12 months to
24 months
100
90
80
70
60
50
40
30
20
10
0
NO SCHOOLS ONE SCHOOL TWO SCHOOLS
6.7%0.0% 0.0%
93.3%
6.7%
86.7%
13.3%
80.0%
13.3%
NUMBER OF SCHOOLS ATTENDED OVER PREVIOUS 12 MONTH PERIOD
34
Table 10. Participation in cultural activities and barriers to learning, Baseline to 24 months (n=15)
Domain
Mean (Standard Deviation)(n)
Baseline 12-months 24-months
Cultural Activities
Number of cultural activities experienced by client (Range 0-7)
Missing data for over 50% of the sample
3.1 (2.3)(n=15)
2.9 (2.6)(n=15)
Barriers to learning
Individual barriers score (Range 0-12)
5.2 (3.8)(n=12)
3.5 (2.6)(n=14)
3.1 (2.1)(n=15)
School-based barriers score(Range 0-12)
2.0 (1.9)(n=10)
0.5 (1.1)(n=15)
1.0 (1.6)(n=15)
Placement barriers score(Range 0-4)
1.1 (1.6)(n=14)
0.8 (1.5)(n=15)
1.1 (1.7)(n=15)
Academic skills and achievement
There was a more consistent pattern in the Academic Skills and Achievement domain, where the majority of indicators showed progressive, albeit modest improvement from baseline to 24 months.
School attendance remained at stable, full-time levels across the 24 month period for the majority of the sample (n=14)10. However, the greatest gains were observed for overall academic achievement, which increased from 13.3% (n=2) at baseline to 60% (n=9) at the 24 month assessment. While these improvements are promising, they need to be contextualised against the amount of ‘Don’t know’ responses at baseline (see Appendix A). Given the small sample size at 24 months and the degree of missing data, only tentative conclusions can be drawn from the patterns observed for the ‘academic skills’ domain at this stage.
Learning in the home/care environment
Patterns for this domain are less clear, and again may be a reflection of this sample, where patterns are easily influenced by change among one or two children and young people. In general, however, at the 24 month assessment carer engagement in learning activities had decreased from the levels recorded at the baseline and 12 month assessments. The only exception was for ‘reading for pleasure’ which increased from 57.1% (n=8) to 80% (n=12) at the 24 month assessment. While this improvement is promising, it also needs to be understood in the context of the missing data recorded for this variable at baseline (see Appendix A).
Cultural activities and barriers to learning
Data for the cultural activities domain is difficult to interpret given the large amount of missing data at the baseline measurement. While there appears to have been a marginal decrease in the cultural activities summed score from the 12 month to the 24 month assessment, it is unclear whether throughout the 24 month intervention period there was an overall increase, decrease or stabilisation on this variable.
The pattern for the Barriers to Learning domain at 24 months reflects what was observed at the 12 month assessment, namely an overall reduction in individual, school-based and placement barriers. While significance testing was not possible with this sample, the data in Table 9 shows a clear decrease from baseline to 24 months especially on individual barriers to learning. This provides preliminary support for the program’s efficacy over a longer-period of intervention, at least among a sample of children and young people in foster care.
As with the 12 month data, placement barriers were low across the three assessment periods. This again suggests that while the TEACHaR program works across multiple contexts its ability to influence educational outcomes through placement-based factors may be limited, given the multiple complex factors that influence placement stability and relationship breakdowns among this sample of children and young people.
Indicators
10. The exception was one student living in residential care with complex care needs who refused to attend school. This young person was not enrolled at school at baseline, but showed some improvements in attendance throughout the 24 month period of program involvement.
35
QUALITATIVE DATA
Secondary qualitative data sourced from TEACHaR Educators were made available to the evaluator through quarterly operations reports, discussions at quarterly Governance Group meetings, and attendance at Educator team meetings. The following insights reflect some of the program’s core practices and service highlights. Three case developed by TEACHaR Educators are presented in Appendix B.
Engagement of registered teachers
The employment and remuneration of experienced registered teachers is a foundation of the model. The TEACHaR educators bring a range of classroom and senior-level teaching experience, and considerable commitment to working with vulnerable students. These skills have supported Educators to gain the trust and respect of schools, to inform and support the teaching practices of teachers, and to become valuable systemic advocates.
Co-location within OHC teams
The holistic nature of the program is reflected in the co-location of Educators within OHC care teams, and the delivery of interventions in a range of school, OHC and alternative settings. Educators report that working in this flexible manner has strengthened their understanding of students’ care circumstances and past experiences, and their capacity to deliver interventions in a more informed and sensitive manner. Educators indicate that they have become a ‘bridge’ between care teams, carers and schools, and can be a consistent point of support in the context of school and placement changes.
Flexible student-focused interventions
The program’s holistic practice model supports the use of both creative and traditional teaching interventions, matched to the needs of the student. These include:
• Direct teaching or tutoring for students in both classroom and care settings, consistent with their level of need, areas of interest, learning strengths and Individual Learning Plan. For example: 1:1 support and group work, literacy and numeracy support, individual projects, introduction of complementary learning and problem solving and implementation of learning strategies
• Small group work in classrooms that aims to strengthen students’ academic needs, together with their social relationships with peers and their feelings of belonging within the classroom. This is particularly important for children and young people in care who often feel ‘different’ or isolated from other students, and may have difficulty making friends due to multiple school transitions and the complexity of their behavioural presentations. Educators indicate that the group work can also help normalise the TEACHaR intervention for the student, and reduce potential stigma.
• Implementing creative, individual projects with students that are less formal, responsive to student interests, and which often lack an obvious link to learning. Such intervention can be very important when working to re-engage children back into school, and to improve their learning confidence, for example: games, electronics, Lego, music, outdoor experiences, drumming and art.
36
The school context
Drawing on their teaching expertise and understanding of the OHC system, TEACHaR Educators report that they have strengthened schools’ understanding of trauma, and the ongoing impacts that such experiences can have on student wellbeing, classroom behaviours and learning. In addition to the direct student supports described above, other typical interventions within schools include:
• Provision of ‘reverse teaching’, whereby TEACHaR Educators ‘take over’ the broader class group in order to enable the classroom teacher to have valuable 1:1 time with the student targeted for the program. This can support rapport building between student and teacher, help the teacher to develop a better understanding of the student’s learning needs on an ongoing basis, and provide opportunities for teachers to institute tailored learning practices in an individualised manner. This ‘reverse teaching’ has also provided opportunities for TEACHaR Educators to model teaching strategies congruent with the needs of the student in care, including positive ways of responding to challenging classroom behaviours.
• The provision of informal support, education and professional development to teachers both within and outside of classrooms. For example, assistance with modified lesson planning, helping teachers to plan for, and manage challenging behaviours, contributing to care team meetings, and providing education about the impacts of trauma and the care system.
The care team context
Working alongside OHC teams has provided opportunities for TEACHaR Educators to raise the profile of education across Anglicare Victoria’s OHC services. Educators report that case managers and management staff are becoming more cognisant of the importance of addressing education in their everyday work. In particular, Educators report that case managers have developed strengthened understanding of the OHC Education Partnering Agreement and its implementation, are more sensitive to the challenges that teachers and schools typically face, and have become more confident and informed ‘advocates for education’ in the context of care teams and care planning, particularly with respect to school stability, early school transition planning and seeking funding for learning support. Further examples of the role of the TEACHaR program at the level of the OHC context include:
• Taking responsibility for new school enrolments and transitions for children and young people who are not attending school. This is often the case in residential care, where Educators provide support to the child or young person in the care setting (e.g. rapport building, informal play-based learning and direct tutoring) whilst also working closely with schools and the care team to support a future transition back into school.
• The importance of assessing the educational needs of all children and young people as soon as they move into a new residential care placement is a key program feature. It is not uncommon for children and young people to enter a new placement without having being previously enrolled into a new school as part of the transition and case planning process. Given the risk of school disengagement for these students, TEACHaR has assumed an active role managing new school transitions. This includes matching the best possible school to the student, supporting communication between the school, residential care house, and case manager, and providing practical ongoing support to both students and key teaching staff over the transition period.
• Providing student advocacy, and being a voice for education during care team meetings, Student Support Groups and liaisons with the Department of Health and Human Services, and the Department of Education and Training.
37
Education support within residential care
Working within residential care units is a key program component. Interventions include:
• Helping students in residential care houses to be school ready at the beginning of every term and school year, supported by the development of a checklist of ‘educational expectations’ for all units.
• Instituting communication books between residential care staff, school teachers and the TEACHaR staff
• Supporting staff to establish daily routines that can support school attendance and learning: such as helping clients wake up in the morning, taking them to school if necessary, establishing set homework times if possible, and encouraging improved sleep routines
• Training residential care staff to facilitate 1:1 and peer-based literacy interventions with students, and to lead homework sessions
• Providing learning tubs within units containing developmentally appropriate resources for children and young people and staff to utilise. These contain books tailored to student interests and reading levels, and a range of problem solving resources and games that have an informal literacy and numeracy focus.
• Modelling, though their own practice, the importance of school attendance and engagement. This includes raising organisational expectations of what students can achieve as learners, providing hope that better educational outcomes for even the most disengaged clients is possible, and raising aspirations.
Support to foster carers
Recognising the important role foster carers can play in a child’s education pathway is another key focus of the program. Interventions include modelling positive reading and homework routines and strategies, assisting carers to implement specific learning interventions in the home, acting as a bridge between carers, schools and Allied Health services, providing advocacy for carers within schools, and supporting carers to be hopeful about, and positively engaged in the child or young person’s learning.
The value of brokerage
The program’s brokerage allowance has provided opportunities for students to obtain assessments and Allied Health services who would not otherwise meet the benchmarks for state-funded support services (despite demonstrable need). The provision of specialist interventions (such as Behavioural Optometry, Occupational Therapy, Speech Therapy and Psychology), have contributed to shifts in student learning for a number students, particularly with respect to reading, writing, spelling and numeracy. Brokerage has also enabled the program to:
• Work with more clients by providing those with less complex needs with support from a pool of carefully selected, supervised tutors
• Fund therapeutic and/or creative interventions, such as equine, music, dance, drumming and art therapies, together with a range of extra-curricular activities to complement formal learning
• Enrich the quality of the student home learning environment, in particular residential care houses through the introduction of formal and informal learning resources.
38
Other innovations
A number of creative initiatives informed by Educator expertise and interests have become important parts of the program and its approach. Examples include:
• REAL PLAY
An initiative to support children and young people in residential care to to take part in fun, engaging and pro-social activities connected with nature over the schools holidays, when children and young people are often disengaged from activities outside of the unit in the absence of school. The initiative was driven by the TEACHaR Educator in the Southern Metro Region, and activities included fishing, scuba diving, surfing lessons, skate-park visits, tree surfing, farm experiences, horse riding, and visits to the Zoo. Funding for these activities was sourced from TEACHaR brokerage:
• Sexual health education
Students disengaged from school often miss out on the sexual health education schools provide. TEACHaR arranged for residential care workers to receive professional development to help them to respond to questions raised by children and young people about sexual health, and provide age–appropriate sexual health information when needed. The training included information about contraception, issues of consent, internet safety and pornography, grooming behaviours, same sex attracted youth, gender diversity and how to respond to disclosures of sexual abuse.
• The literacy library
The program has drawn on brokerage to stock residential houses with books that are consistent with client interests and reading levels, and to improve the culture and accessibility of learning within these settings. Training has also been provided to staff with regard to utilising these resources, with a particular focus on building reading confidence and establishing routines.
• Carer information sessions and carer newsletters
Providing foster carers with information about the OHC Education Partnering Agreement, the role of Allied Health assessments, age-appropriate ways of supporting learning in the home, and the importance of the school/home relationship.
Service challenges
• Engagement with schools
Over the pilot period, TEACHaR Educators have implemented a theoretical model whilst simultaneously developing relationships and rapport with care teams, carers and schools. Given that CSOs do not typically provide direct education interventions, navigating this new service context has at times raised challenges, particularly in terms of establishing the program’s credibility within traditional school settings.
As a classroom and home-based program, engagement with schools has been critical. Such engagement has required careful communication and sensitive negotiation; e.g. helping Principals and teachers understand the aims of the program, the qualifications of its staff, its model of service delivery and the ways in which it might benefit both individual students and wider classroom settings.
Educators report that whilst a number of schools were initially reluctant to involve them in classroom activities or in student care and planning meetings, this has largely improved over the course of the two year evaluation period. Gaining access to student report cards and other assessments/tests was also difficult for Educators early in the program, with schools initially reluctant to share student information. This be may be attributable to confidentiality concerns, or due to the program sitting outside of the formal school system.
39
• Application of the model in residential care
Due to the typically more complex needs of children and young people in residential care, application of the TEACHaR program in that context has raised particular challenges. Whilst the program model seeks to provide direct education support to clients via 1:1 or group tuition, in residential care the immediate focus of the intervention is often to strengthen school engagement, provide informal education support and learning, and help transition children and young people back into school if they are not engaged or enrolled.
Educators report that strengthening the learning culture and education opportunities available within residential care environments remains an ongoing, evolving process. Obstacles can include low staff ratios in houses at busy times (such as after school), inconsistent staff expectations of clients with respect to school attendance and achievement, and difficulties establishing core routines (such as sleep, getting ready in the morning, completing homework) in the context of busy house environments populated by children and young people with a variety of often complex needs.
40
DISCUSSIONThe data presented in this report indicate positive change across a number of educational domains for students receiving the TEACHaR intervention, particularly over 12 months, consistent with the TEACHaR Operating and Program Logic Models.
The supplementary qualitative data also provides some insight into the translation of the model into a working program, particularly its broad benefits to clients, schools and OHC teams.
DEMOGRAPHIC DATA
The program has worked with a range of clients, in terms of age, school level and gender, but with few clients with an Indigenous or Torres Strait Islander background. The data shows that clients in foster care represent a more stable group in terms of placement length, with prior placement length at service entry for the residential care group averaged at only 3.3 months (compared 21.9 months for foster care). This disparity is consistent with the lengths of TEACHaR intervention received by each care group; with residential care clients receiving shorter term interventions generally due to increased placement changes. For example, 60% of residential care clients received less than 12 months of the TEACHaR intervention over the evaluation period, compared to only 30% of the foster care sample.
The demographic data also demonstrate that a high proportion of children and young people entered the program with established emotional and behavioural problems, typically associated with past experiences of trauma that are often acted out in the classroom, and can impede learning. According to the SDQ data available for the sample (n=22 clients), between 35% and 63% of clients were found to have clinically elevated (high or very high) conduct problems, peer problems, hyperactivity, reduced prosocial behaviours and emotional symptoms close to program entry. It is also notable that a greater proportion of the TEACHaR client cohort scored in the clinically high or very high range for all SDQ scales compared to the wider sample of Anglicare Victoria OHC clients. Such difficulties can make participation in learning challenging, especially with respect to school engagement, attendance, concentration, friendships and disruptive behaviours in the classroom, and may in part suggest why this cohort were referred into the program over others.
12 MONTH OUTCOMES
The 12 month data presented in this report show observable positive change for the majority of variables under investigation. These data provide support for the program’s educational benefit for children and young people in care, especially for those who experience at least 12 months of program intervention.
Student support in schools
The student support variables that form part of the TEACHaR Assessment Tool align with the best practice guidelines for schools contained within the OHC Education Partnering Agreement, which seeks to improve the supports available to students in OHC, together with their level of school attendance and their academic outcomes.
Both the frequency data, and the significant change between baseline and 12 months on the global school support measure, support the program’s positive influence on the implementation of the OHC Education Partnering Agreement by schools, in the form of improved use of Individual Learning Plans, Learning Mentors and Student Support Groups. At 12 months, those who did not have access to these supports were largely attending community school environments where alternate forms of student supports are utilised.
41
School engagement and attitudes to learning
A number of patterns were observed for the school engagement and student attitudes domain, although no statistically significant differences were identified from baseline to 12 months. However, the number of students working hard at school at an average level or above nearly doubled over 12 months (32.4% to 59%), and the majority of students (94.6%) were assessed as usually happy to be at school following 12 months of service provision.
Further approximately 20% more students were actively engaged in learning tasks following 12 months of support (58.8% to 78.4%). The importance of school engagement and positive attitudes to school has been widely acknowledged, with research indicating that these constructs are predictive of better educational and general wellbeing outcomes (Pecora, 2012; Font & Maguire-Jack, 2013). The 12 month outcome data presented in this report are consistent with research in this area, and further highlight the complexity of working with children and young people in the OHC system.
Some of this complexity can be observed in the data on school attendance. It is noteworthy that for the ten children and young people in residential care, only two were attending school full-time at baseline, and seven were either not attending at all, or attending less than ten hours per week (in contrast to those in foster care who were largely attending full time at baseline). By 12 months, the number of these students attending full time had increased to seven, and only one young person was not attending school regularly. This is consistent with the qualitative data which describes the program’s focus on assisting children in residential care to re-engage or re-enrol with school.
School stability
Increased school stability was also observed over the course of the 12 intervention period, with a greater number of students remaining in the same educational placement whilst clients of the program (86.5%), compared to the 12 month period prior to program entry (69.5%). Given the deleterious impacts that multiple school changes can have on school engagement and performance (Jackson & Cameron, 2012; Flynn, Tessier & Coulombe, 2011), this finding of increased school stability, also extending to residential care, is encouraging. The qualitative data also provide further insight into the program’s influence on school stability. For example, core aspects of the program include assisting schools to respond to challenging behaviours and to implement modified curricula, the provision of teacher mentoring on trauma-informed practice, and participation in student support group meetings. Further, where students did experience a school change over the course of the intervention period (13.5%), these changes were largely described as positive and appropriate by the Educators.
Academic skills and achievement
Despite the apparent complexity of the service system and the client group, there was a significant improvement in overall academic achievement between baseline and 12 months. Although not significant, improvements were also observed for all five variables under this domain, in particular for literacy and concentration skills. These patterns provide some support for the program’s benefit to clients, but once again highlight the difficulty associated with achieving consistent gains in educational outcomes among a group of children and young people with multiple, complex needs.
The least change was observed with respect to the number of students performing according to their ability, which reached 40.5% at 12 months, after plateauing from 6 months (38.9%). There are a few possible interpretations for these findings.
• On the one hand, it may be that for at least some of these children and young people, longer periods of intervention and support may be needed to help overcome early experiences of educational disadvantage, despite the academic improvements they may make in the interim.
• Alternatively, these data may indicate that performing at an average level may not be an appropriate goal for all students, particularly in the short term and in the context of considerable academic delay, and/or in the context of significant disruption as evident in OHC settings.
42
Further, the ‘plateau effect’ observed after 6 months may reflect a more accurate understanding of a child’s ability as Educators became more familiar with their clients, or may be influenced by a range of other factors, including for example: school engagement, a child and young person’s feelings of self-efficacy, attitudes towards school, and placement stability to name a few. These factors may interact to influence a child or young person’s performance and perceived ability.
It is also noteworthy that for overall academic achievement, the number of students performing at an above average or excellent level improved from two students at baseline, to three students at 6 months, to four students at 12 months. These improvements highlight the variability that exists in the educational ability and performance of children and young people in care, and suggest a range of potential within the client group, further reinforcing the value and importance of aspiring beyond the achievement of minimum benchmarks.
This is consistent with the literature that highlights the diversity that exists amongst this cohort of students, demonstrating that whilst a proportion of students in care may experience difficulties at school, many others may reach minimum academic benchmarks, and/or academically thrive (AIWH, 2011). The literature also indicates that education can become an important protective factor in the context of adversity (Hojer & Helena, 2013), as can carers or care staff who demonstrate congruent behaviours and attitudes such as valuing education, providing consistent support for learning in the home, and communicating positive messages about what the child or young person has the potential to achieve at school (Flynn, Tessier & Coulombe, 2013; Cheung, Lwin & Jenkins, 2012; Jackson, 2007; Wise et al).
Learning in the OHC environment
There were non-significant but nevertheless observable changes on variables related to the quality of the student’s OHC home learning environment, with some growth in the frequency with which carers talked about school activities, and the percentage of children and young people reading for fun, rather than as part of their school-work. There was however a small reduction in the percentage of students frequently participating in literacy-based activities in the home. This finding is worthy of consideration given one of the aims of the intervention is to help enrich the student home learning environment (Flynn, Tessier & Coulombe, 2013; Jackson, 2007).
The work that the TEACHaR program is undertaking within placement settings against this domain is summarised in the qualitative data, which shows that the facilitation and encouragement of informal and formal leaning in care settings is infact a core program practice. Direct interventions in residential care generally take place on a daily basis, including: 1:1 and peer-based sessions, homework groups facilitated by both the TEACHaR Educator and trained residential care staff, and the use of informal learning resources, such as games and comic books. Similarly, in foster care, the program works with students directly in the home, and often includes carers and siblings. Such accounts demonstrate the program’s potential role in enriching the quality of the OHC home learning environment, despite the challenges found in residential care where educational routines can be hard to establish, and staff capacity is limited.
Cultural activities
Engagement in cultural activities is an important element of the TEACHaR model. These activities provide a range of opportunities for social engagement and learning, consistent with the holistic and multi-systemic approach that underpins the model. At 12 months there was some evidence of change in the number of cultural activities experienced by clients, equated to approximately one additional cultural activity experienced in the preceding 6 month period. Despite the limited sample size at baseline, this change is consistent with the qualitative data, which describes the program’s holistic commitment to increasing students’ access to informal learning activities, such as: music lessons, dance, drumming, art, sport, alternative therapies and a range of outdoor activities over the school holidays. This core program focus is noteworthy given that children and young people in OHC are potentially missing out on valuable experiences that can enhance their psychosocial development (Jackson & Cameron, 2012; Townsend 2012; Flynn, Tessier & Coulombe, 2011).
43
Barriers to learning
There was a clear trend towards decreased barriers across individual, school-based and placement contexts from baseline to the 12 month assessment. Individual barriers were moderate at baseline (M=6), and progressively decreased to an average of 4.5 at the 12 month assessment. This decrease was statistically significant, and provides some evidence to support the program’s ability to influence change at the individual, cognitive levels. Specifically, these differences may reflect cognitive shifts towards more positive attitudes toward learning, greater feelings of self-efficacy and self-worth, and improved relationships within a school setting. The mean for school-based barriers also halved over the 12 month service period, providing further support for the program’s multi-systemic approach.
The mean score for placement barriers was consistently low to moderate across the three assessment periods. This is a surprising result, given the complexity and instability associated with OHC populations, and the OHC context itself (e.g., Fernandez, 2009; Ward, 2009; Riggs, 2010; Murray & Goddard, 2014). There are a couple of plausible explanations for this finding.
• First, there is an over-representation of children and young people in foster care at the 12 month and 24 month assessment periods. The composition of the sample therefore lends itself towards a lower rating of placement barriers, given that foster care placements tend to be more stable, and therefore better able to support the development of longer-term, nurturing interpersonal relationships. Further, low scores on placement barriers may simply reflect the fact that by virtue of greater stability, children and young people in foster care are more likely to remain engaged with the TEACHaR program over longer periods, compared to their counterparts in residential care.
• Secondly, these results may also indicate that placement barriers are less amenable to change through the TEACHaR program model. Despite its multi-systemic and holistic approach, promoting placement stability and avoiding relationship breakdown is not a core function of the TEACHaR program. Moreover, these issues are often influenced by a multitude of factors external to the program, and are therefore much more difficult to target. This may be particularly true for children and young people in residential care who are more likely to experience multiple placement breakdowns within relatively short time frames. There is therefore a clear need to further investigate how placement barriers differentially influence educational outcomes for children and young people in foster and residential care.
24 MONTH OUTCOMES
A total of 15 children and young people were engaged with the TEACHaR 1 (Pilot) program for a continuous period of 24 months. Based on the available data, a number of preliminary and descriptive conclusions can be drawn. For example, across almost every outcome indicator, there was a general trend towards a ‘maintenance effect’ such that the gains observed from baseline to 12 months were maintained at the 24 month assessment. It is currently unclear whether this ‘maintenance’ is due to the sample size, the composition of the sample, a combination of these two factors, or whether it reflects a critical ‘dosage’ beyond which the TEACHaR program begins to have a diminishing impact. This is an area of the program model that requires further research.
There is some support for this contention in the pattern of findings for specific domains and outcome indicators. For example, in the Academic skills and achievement domain there was a slight increase in the proportion of children and young people achieving at or above average for literacy skills and overall academic achievement. While these improvements are important, the overall trend was stable from the 12 to 24 month assessment. Similarly, there was little variation in the proportion of carers who were actively involved in children’s learning in the home at baseline, the 12 month assessment or the 24 month assessment.
School attendance (measured as the average number of hours attended per week over a four week period) also remained stable and at the equivalent of full-time attendance, from baseline to 12 months, and from 12 months to 24 months, as did school stability, (measured as the average number of school changes, which was consistently low prior to, and over the 24 months of the intervention).
44
There were also a number of findings that run counter to the program logic, and the intended targets of the intervention. For example, one of the intervention targets within the placement context is to enrich the student home learning environment. The patterns observed for the 12 and 24 month outcomes indicate that participation in literacy-based activities in the OHC environment decreased from baseline to 12 months, and then again from 12 months to 24 months. Although these patterns need to be contextualised against the nature and size of the sample, they nevertheless raise the possibility that the program’s impact in this area needs further research with a larger sample size.
Consistent with the 12 month data, average placement barriers were low at every assessment period (baseline, 12 months and 24 months). In fact, among the 15 children and young people who had received 24 months of TEACHaR intervention, the average number of placement barriers at 12 months and 24 months hovered around one. The ability of the TEACHaR program to affect change at the ‘placement level’ is therefore an area that requires greater conceptual and research attention.
Qualitative data
The supplementary qualitative data presented in this report provide a counterpoint to some of the quantitative findings, and highlight the benefit of a number of key program aspects that are consistent with the TEACHaR Operating Model.
• Flexibly embedding the program within school and OHC settings is an important aspect of supporting this client group. This flexibility enables Educators to act as informed conduits between schools and the care system, and to build a cohesive and informed team of support around the student.
• Collaborative practice has enabled Educators to actively drive school enrolments for disengaged students, to support and collaborate with schools to deliver more informed and modified teaching practices, and to work with OHC teams and foster carers to improve the quality of students’ home learning environments, particularly in residential care. Such practices have enabled the program to strengthen school stability for students, and diminished the potential for school breakdown.
• The ability to provide multi-systemic interventions, including direct education support to students, schools, carers and OHC teams has been critical. Given the level of school disengagement and academic difficulty experienced across this client group, working in ways that can enhance engagement and break through barriers by drawing on student interests and strengths cannot be over stated. The capacity to deliver direct student interventions in a variety of contexts, modalities and settings also stands out as a key aspect of the service approach.
• The employment and remuneration of experienced, registered and committed teachers is further foundation of the model. This has helped Educators gain the trust and respect of schools, to work with classroom teachers as peers to help support and develop their teaching practices, to provide strong advocacy, and to deliver targeted, innovative interventions to students.
• The program’s brokerage allowance has provided opportunities for Educators to fund Allied Health and learning assessments, and other services for at-risk students. In many cases, the provision of specialist and therapeutic interventions has contributed to demonstrable shifts in student learning.
45
Limitations
The main limitation of this evaluation extends to the sample size, which has impacted the number and type of analyses that could be conducted. Such limitations are common to research conducted in the early phases of new programs. The results presented nonetheless provide an early indication that a range of positive outcomes have been achieved, and sustained through participation in the TEACHaR intervention. As the program continues to grow, additional analyses will be possible to further test the impact of the program.
The evaluation would have also been strengthened via inclusion of more rigorous qualitative research methods, such as: analysis of case file records, and collection of stakeholder service feedback via interviews or questionnaires, including TEACHaR Educators, schools, carers, and OHC staff, and ideally the views of students receiving the intervention. Unfortunately, this was beyond the scope of the current evaluation, and thus secondary qualitative sources via operational reports, case studies and Educator presentations at Governance meetings were drawn upon.
Future directions
Program sustainability is an immediate priority for the TEACHaR program, as funding across all three trials is currently limited to the end of 2015. To ensure that the program can continue to provide support to this vulnerable student cohort, and that the agency is in a position to retain the operational and practice knowledge held by the program’s experienced group of staff, securing ongoing funding for the program beyond 2015 is important in the short, and ultimately long term.
The Recommendations contained in the Executive Summary of this report (page 8) provide further directions for consideration.
46
CASE STUDIESThree case studies were developed from the perspective of TEACHaR Educators. They provide qualitative accounts of the program’s approach, and its engagement with, and potential impacts for clients, schools and carers and other stakeholders.
Julian’s story
14 year old Julian was my first client in the TEACHaR program. I was an experienced teacher, but had never met any kids living in residential care before. At our first meeting we spoke animatedly for an hour about music, and shopped for jeans. I cried most of the way home, and thought, “What on earth is this polite and respectful young person doing in residential care? How can we live in a society where there aren’t families that want to include these children into their homes?” These days I sit more comfortably with questions like this, but I still ask them an awful lot.
I saw Julian every week for at least an hour for the full two years that he was in our care. Early on I found that his school was struggling to work with him; some teachers labelled him as ‘lazy’ and struggled to recognise his strengths. Through my role, I helped his classroom teacher better understand trauma and its impacts, and helped her to modify Julian’s curriculum and work tasks. We had many, many meetings to help make sure that the relationship between the residential house and the school was solid.
However, six months after first meeting him, Julian was in an increasingly difficult place. After three residential house changes, he had almost completely dropped out of school. The Assistant Principal, despite trying really hard, felt Julian had become too much of a danger to himself and others. Julian wanted to change schools too, and to leave residential care altogether – but there were no other options for him. He was also frequently suicidal. I recall one time when he flicked a lighter in my face and then apologised profusely upon realising what he had done. Our whole team were worried, and we didn’t even know if we had reached rock bottom yet.
As his mental health further deteriorated and he couldn’t be at school, Julian and I met more regularly and did Maths and English together. I also referred him to another service for extra tuition. When driving, we listened to a lot of music and talked. We talked about bands, what the songs meant, about being in residential care and his identity, about drugs and respectful relationships, and about his childhood.
Julian loved food, so we ate out - a lot. Over these meals, I introduced him to different books, and he soon developed an interest in fantasy novels. His residential house and case manager were just fantastic. They nurtured him and held the line. They encouraged him to take up sport, and regularly took him swimming. He also became interested in cooking, and loved trying out creative new recipes. During this time, we also found a new school for him that was going to be less stressful, and he was soon attending regularly, with reduced hours consistent with what he could handle. His mental health gradually began to stabilise, and with this his concentration improved.
When Julian eventually moved into a foster care placement, I continued to work with him to support this important transition. In those first few months, the process of settling in with his new foster carers was challenging. When his carers called me I would listen with understanding, and draw upon my experiences to gently encourage them to “try this…” and “wait it out a little longer”. During our last sessions, we read The Old Man and The Sea together; my parting gift to him. During those sessions I realised that Julian had become one of my greatest teachers.
Fast forward eight months, and Julian is going well. Recently a colleague remarked that he was going to school full-time, had joined a local swimming squad and is happy with his new foster family. Hearing this, I felt inspired by Julian’s resilience and very proud of my team and of the good work we had achieved together.
Victoria Bramall-White
TEACHaR Educator
47
Lola’s story
Lola has been a student of the TEACHaR program since she was in Grade four, and nine years of age. She was one of the program’s first referrals, and when TEACHaR first started working with her she was many years below the expected level in English and Maths, and often refused to go to school.
The trauma that Lola experienced through her childhood greatly impact her capacity to trust people, attend school consistently and engage with learning. Lola had missed a lot of school in the past, often due to the active caring role she had assumed for her younger siblings. Consequently, she had not only fallen behind her peers academically, but had developed an entrenched pattern of non-school attendance, and ultimately school refusal. Lola later carried these beliefs and feelings about school and her own abilities into her foster care placement.
The TEACHaR Educator soon learnt that Lola responded well to humour, and to the opportunity to take on responsibility. When given a choice or when asked for her opinion, Lola would respond with confidence and a willingness to prove herself as capable. Lola’s strengths however, could also be a hindrance. She would often guess and rush through tasks in order to finish on or before time, and would rarely ask for help or assistance, preferring to give the appearance of understanding what was expected of her.
Over the last two years, TEACHaR has provided Lola with weekly in-school support. Lola’s carer often joins in at the end of these sessions too, which has helped to enrich the home-learning practices that she and Lola can utilise together at home.
The TEACHaR Educator has also been an active member of Lola’s school-based Student Support Group (SSG) – consisting of Lola, her classroom teacher, the school leadership team and student support staff. The united support and warm approach provided by the SSG has supported Lola’s growing happiness and engagement at school, together with her academic progress. The group has also supported the development of an Individual Learning Plan consistent with Lola’s needs and strengths, which is used consistently in the classroom and by the TEACHaR Educator.
The program has supported Lola and her school/carers in the following ways:
• Weekly intensive tutoring in reading, writing, spelling and mathematics, and additional tutoring through the Anglicare Volunteer Friends program
• Advocacy for Lola to receive an Educational Needs Assessment, and funding for an assessment for dyslexia after identifying this as a potential issue through the 1:1 sessions completed together
• Provision of tailored strategies and resources to Lola’s teachers, including visual organisational aides, in-class mentoring, and a multisensory learning program to support spelling development
• Referral to a Behavioural Optometrist who recommended glasses, in turn purchased via TEACHaR brokerage. Lola continues to receive six monthly check-ups.
Through the one-on-one support provided by the TEACHaR Educator, and the united approach embraced by Lola’s school, carer and care team, we have seen Lola grow and progress over the last two years. She is happily attending school each day, has made secure friendships, is excelling in school sporting events and has taken on leadership roles within the school with enthusiasm, putting her ‘responsibility strengths’ to great use. She is also progressing well in all learning areas, and is now only 6 to 12 months behind her peers, rather than multiple years behind as she was when she first commenced with the program. This represents a very positive change in Lola’s education trajectory.
Lola is now aware that she has the genuine support of her carer, case manager, classroom teacher, school Principal and her TEACHaR Educator. I believe that this consistent and holistic approach has contributed significantly to Lola’s academic gains and other successes.
Natasha Chidgey
TEACHaR Educator
48
Cara’s story
Cara and her sibling are in a long term foster care placement. Prior to moving into care, Cara had experienced significant trauma that had substantially undermined her learning and access to positive educational opportunities.
Cara was just finishing Grade three when referred to the program due to significant concerns about her writing, reading and comprehension skills. While Cara enjoyed school, she found school work very difficult, and she was well behind her peers. Despite their efforts, her supportive carers were also finding it difficult to support her learning at home.
Cara’s in-home and class-based support has focused on developing her understanding of a range of reading and comprehension strategies, and helping her to consolidate basic number facts and numeracy concepts. The ability to work flexibly between home and school has been important, particularly after voiced that she did not want to be seen as different from her peers. A strong focus on supporting the school’s implementation of the OHC Education Partnering Agreement has also been a focus of Cara’s intervention.
Due to ongoing concerns about her reading and comprehension, the TEACHaR Educator referred Cara to a Behavioural Optometrist, who identified that she had low visual information processing skills. Cara subsequently underwent a TEACHaR-funded ten week visual training program that focused on developing her:
• Visual analysis skills (appreciate visual clues important in letter and word recognition)
• Eye movement control (enable her to read without losing her space and facilitate quicker and more accurate copying from a book or white board)
• Visual sequencing and visualisation skills
Both the carer and TEACHaR Educator supported Cara’s visual training, assisting her to practise set tasks on a daily basis. The post-training report from the optometrist showed that Cara had made significant gains in the areas of visual analysis and sequential visualisation and was now performing above the expected level in these areas; an important hurdle with respect to her reading.
Late last year, when Cara’s foster family relocated, her TEACHaR Educator supported her and her new school through the transition process. He contributed to the newly-formed student support group, and provided a bridge between old and new school settings. Tutoring in the home continued through the transition, and soon the TEACHaR Educator began to observe a significant change in Cara’s learning. Not only was she reading at a much higher level, but she was doing so for fun, and her writing skills had blossomed, with Cara now able to use a range of sentence structures and descriptive language.
Following two years of the program, Cara has become a confident student, with a great desire to learn. She has set herself a goal of being in the top of her reading group in class, and her willingness to take risks and aim high is a remarkable achievement.
TEACHaR is just one of the many elements that continues to make a difference in Cara’s life, and the program is proud to have contributed to her more positive experience of education now and into the future.
Greg Rollings
Lead TEACHaR Educator
49
REFERENCESAustralian Institute of Health and Welfare (2011). Educational outcomes of children on guardianship or custody orders: A pilot study, stage 2. Child Welfare Series No. 49. Cat. No. CWS 37. Canberra: AIHW. Available at: http://www.education.vic.gov.au/healthwellbeing/wellbeing/alternativeprograms.htm
Australian Institute of Health and Welfare (2007). Educational outcomes of children on guardianship or custody orders: A pilot study. Child Welfare Series No. 42. Cat. No. CWS 30. Canberra: AIHW. Available at: http://www.education.vic.gov.au/healthwellbeing/wellbeing/alternativeprograms.htm
Buckingham, J., Wheldall, K. & Beaman-Wheldall, R. (2013) Why poor children are more likely to become poor readers: The school years. Australian Journal of Education, 57 (3), 190-213.
Cheung, C., Lwin, K., & Jenkins, J. (2012). Helping youth in care succeed: Influence of caregiver involvement on academic achievement. Children and Youth Services Review, 34, 1092-1100.
Connelly, G. & Furnivall, J. (2013). Addressing low attainment of children in public care: The Scottish experience. European Journal of Social Work, 16 (1), 88-104.
Council of Australian Governments (2006). National Education Agreement.
Department of Education and Early Childhood Development (2011). Out-of-Home Care Education Commitment: A Partnering Agreement between the Department of Health and Human Services, Department of Education and Early Childhood Development, Catholic Education Commission and Independent Schools Victoria. Melbourne: Department of Education and Early Childhood Development, Student Wellbeing Division.
Department of Families, Housing, Community Services and Indigenous Affairs (2011). An Outline of National Standards for out-of-Home care: A priority Project under the National Framework for Protecting Australia’s Children 2009-2010. Commonwealth of Australia.
Department of Human Services (2013). Baseline Performance Data Report. Victoria’s Vulnerable Children, Our Shared Responsibility. The Victorian Government, Melbourne.
Ferguson, B.H & Wolkow, K. (2012). Educating children and youth in care: A review of barriers to school progress and strategies for change. Children and Youth Services Review, 34, 1143-1148.
Fernandez, E. (2009). Children’s wellbeing in care: Evidence from a longitudinal study of outcomes. Children and Youth Services Review, 31, 1092-1100.
Font, S., & Maguire-Jack, K. (2013). Academic engagement and performance: Estimating the impact of out-of-home care for maltreated children. Children and Youth Services Review, 35, 856-864.
Flynn, R.J., Tessier, G. & Coulombe, D. (2013). Placement, protective and risk factors in the educational success of young people in care: cross sectional and longitudinal analyses. European Journal of Social Work, 16(1), 70-87.
Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research note. The Journal of Child Psychology and Psychiatry, 38(5), 581-586.
Hojer, I. & Johansson, H. (2013). School as an opportunity and resilience factor for young people placed in care. European Journal of Social Work, 16 (1), 22-36.
50
Hyames, S. & De Hamas, M.(2000). Educational Experiences and achievement of children and youth in the care of the department receiving services from Chicago public schools. Urbana-Champaign: Children and Family Resource Center, University of Illinois.
Jackson, S. & Cameron, C. (2012). Leaving Care: Looking ahead and aiming higher. Children and Youth Services Review, 34, 1107-1114.
Jackson, S. (2007). Progress at last? Adoption and fostering, 3 (1), 3-5.
Mallett, C.A. (2012). The school success program: Improving maltreated children’s academic and school-related outcomes. Children & Schools, 34(1), 13-26.
Murray, S., & Goddard, J. (2014). Life after growing up in care: Informing policy and practice through research. Australian Social Work, 6(1), 102-117. doi: 10.1080/0312407X.2013.868010.
Pecora, P.J. (2012). Maximising educational achievement of youth in foster care and alumni: Factors associated with success. Children and Youth Services Review, 34, 1121-1129.
Riggs, S.A. (2010). Childhood emotional abuse and the attachment system across the life cycle: What theory and research tell us. Journal of Aggression, Maltreatment & Trauma, 19(1), 5-51. doi: 10.1080/10926770903475968
Townsend, M. (2012). Are We Making the Grade? The Education of Children and Young People in Out-of-Home-Care. NSW Government, Family and Community Services.
Trout, A.L., Hagaman, J., Casey, K., Reid., & Epstein, M.H. (2008) the academic status of children and youth in out-of-home care: A review of the literature. Children and Youth Services Review, 30, 979-994.
Victorian Auditor General (2014). Residential Care Services for Children, State Government of Victoria.
Ward, H. (2009). Patterns of instability: Moves within the care system, their reasons, contexts and consequences. Children and Youth Services Review, 31, 1113-1118.
Wise, S., Pollock, S., Mitchell, G., Argus, C., & Farquhar, P. (2010). Care-system Impacts on Academic Outcomes: Research Report. Anglicare Victoria and Wesley Mission Victoria: Melbourne.
51
APPENDIX AData cleaning
‘Don’t know’ response data
All variables measured contain a ‘Don’t know’ response option. For the majority of analyses, cases containing a ‘Don’t know’ response have not been removed, as this represents a valid response option given the constructs that were measured. . For the 12 month outcomes, the number of ‘Don’t know’ responses to any item is noted when this exceeds three (approximately 7% of the sample). For the 24 month outcomes, the number of’ ‘Don’t know’ responses to any item is noted when this exceeds two (approximately 10% of the sample).
A greater number of ‘Don’t Know’ responses were found within the baseline data, which is likely attributable to:
• Difficulty obtaining information, or assessing academic performance when clients are disengaged from learning and/or not attending school
• Student engagement challenges impacting the ability of Educators to assess student ability
• Improved Educator understanding and application of the assessment tool over time
• Educators gaining better access to student data due to the program’s growing relationships with schools and OHC care teams over time.
Missing data
‘Not Applicable’ (N/A) response items were re-categorised as ‘missing’, and were removed from the analyses. In the majority of cases, N/A responses were recorded for students who were not attending or enrolled at school at the time of the assessment, or if a scale or measure was inapplicable to the student (for example, standardised performance measures (AUSVELS) are often not used for students with learning disabilities or for those attending community schools).
For analyses utilising comparisons of means, cases containing either a ‘Don’t Know’ or ‘N/A’ were removed on a variable-by-variable basis so as not to distort the mean
Lastly, for analyses utilising comparisons of mean scores, missing data were conservatively pro-rated in order to maximise the potential sample available for analysis. This approach was utilised for sets of variables that contributed to overall constructs or ‘summed scores’ (i.e. Barriers to Learning and Cultural Activities). For the remaining variables presented in this report, missing data were not pro-rated and analyses were conducted on reduced samples only.
52
anglicarevic.org.au
anglicarevic.org.au 1800 809 722
youtube.com/anglicarevic twitter.com/anglicarevic
facebook.com/anglicarevic © Anglicare Victoria 2015