Measuring intervention effects in children with ASD: The use of specific and global outcome measures

A. Nordahl-Hansen1, S. Fletcher-Watson2, H. McConachie4, & A. Kaale1University of Oslo, 2University of Edinburgh , 3Newcastle University 4Oslo University Hospital

Contact: Anders Nordahl-Hansen: a.j.n.hansen@iped.uio.no

Background

Results

Objectives

References

Conclusion

Method

1. Green, J., Charman, T., McConachie, H., Aldred, C., Slonims, V., Howlin, P. … the PACT consortium. (2010). Parent-mediated communication focused treatment in children with autism (PACT): A controlled randomised controlled trial. The Lancet, 375, 2152-2160.

2. Yoder, P. J., Bottema-Beutel, K., Woynaroski, T., Chandrasekhar, R., & Sandbank, M. (2014). Social communication intervention effects vary by dependent variable type in pre-schoolers with autism spectrum disorders. Evidence-Based Communication Assessment and Intervention, 7, 150-174.

3. Grzadzinski, R., Carr, T., Colombi, C., McGuire, K., Dufek, S., & Lord, C. (in preparation). Development of a measure to identify change in ASD behaviors: Preliminary reliability and validity of the Brief Observation of Social Communication Change (BOSCC)

4. Bakeman, R., & Adamson, L. B. (1984). Coordinating attention to people and objects in mother-infant and peer-infant interaction. Child Development, 55, 1278-1289.

5. Kaale, A., Fagerland, M.W., Martinsen, E.W., & Smith, L. (2014). Preschool-based social communication treatment for children with autism: 12-month follow-up of a randomized trial. Journal of the American Academy for Child and Adolescent Psychiatry, 53, 188-198

6. Jacobson, N. S., & Truax, P. (1991). Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-9.

7. Fletcher-Watson, S., O’Hare, A., Pain, H., Petrou, A., & McConachie, H. (2013). Click-East: a randomised controlled trial of a new iPad-based social attention intervention for toddlers and pre-schoolers with autism. Developmental Medicine and Child Neurology, 55 (Suppl s2), PS9.3, p17.

. Acknowledgements: The authors thanks the children and parents for their

participation and COST ESSEA for financial support

• Outcome measures in social-communication interventions are often closely related to the intervention targets.

• Adding global outcome measures would strengthen validity of conclusions as to whether interventions also impact behaviours beyond treatment targets1,2.

• There is a need for more knowledge of the relationship between specific and global outcome measures for use in evaluation of interventions targeting young children with ASD.

• The children in the intervention group had a mean change of time in joint engagement of 13.8% compared to -1.3% for the control group (p= 0.013, d= 0.67).

• The differences between control and intervention group did not reach significance for BOSCC social- and total –change scores (Table 1).

• McNemar’s test showed there was a statistical significant distribution of the two measures (p= 0.004, 95% CI = 1.50, 10.65).  

• Our findings are in concurrence with earlier studies indicating that change in skills closely related to intervention targets may be apparent where more global change is not1,2.

• More investigations using longitudinal designs are needed to assess whether, with time, treatment effects seen in proximal measures can also deliver downstream effects on measures of global change.

• Thus, the use of multiple outcome measures, specific and global, can elucidate important mechanisms and potential pathways from intervention to global change.

To investigate whether a global ASD symptom measure; the Brief Observation of Social Communication Change3 (BOSCC) and a more specific measurement procedure; Joint engagement states coding4 (JE) identified similar group-level treatment effects.

• 59 children (2-4 year) diagnosed with autistic disorder who participated in a randomized controlled trial testing the effect of a brief social-communication intervention5 were assessed with the JE and a preliminary version of BOSCC.

• Group differences in JE and BOSCC change scores (baseline- to post–intervention) were analysed using independent samples t-tests and effect size estimates (Cohen’s d).

• A reliable change index6 (RCI) were calculated to analyse difference in distribution of change between measures using McNemar test of correlated proportions.

• Inter-rater reliability (IRR) for JE coding was high (ICC= .94). IRR for BOSCC was done on a separate sample7 with ICCs of .989 for BOSCC total and .986 for BOSCC social communication sub-total scores.