Sally Boa and Joan MurphyFunded by NHS Education Scotland Conducted by Talking Mats Limited

Communication Matters outcome measurement project

Aims:“To provide information to service

providers and users in order to equip them with knowledge about tools that are appropriate to identify change associated with services and an individual’s communication skills”

Points arising

• Confusion between assessment and outcome measurement

• Lots of Outcome Measures and assessments being used

• Why do we collect data?– To improve our individual practice– To provide feedback to others

Literature review

• Methods

• Review questions and data extraction

• Findings

Methods

• Rapid Review (Khangura et al 2012)

– “streamlined approach to synthesising evidence”

– Systematic search but a quick overview of existing research on a topic

Review questions

• Which Outcome Measures (OM) are used in Health, Education and Social Services?• What are the purposes of the

measures?• How do these relate to AAC?• Are there any gaps in the literature?

Papers screened• Google scholar, ASSIA,

Cochrane reviews– Total: 5121

Full papers read• Google scholar: 5• ASSIA: 109• Cochrane reviews: 31

Data extracted• Author; Year; Country; Population; OM • Outcome measures mentioned in papers– Total number: 72

• Categorisation:– Global/disease specific– Measuring what?– Valid and reliable?– Application to AAC?

Global outcome measures

• AUSTOMS• Barthel• COPM• EQ5D• Expanded Disability status scale• FIM/FAM• Frenchay Activity Index• GAS• Nottingham Health Profile• TOMS

Other measures• Disease specific

– Functional Assessment in Multiple Sclerosis (FAMS)– Aphasia Communication Outcome Measure

• Age specific:– Gross Motor Function Measure (Paediatric)– Mental Status Questionnaire (older people)

• Aimed at patients– COSSS Children's Overall Satisfaction with Schooling Scale– measure yourself medical outcome profile (MYMOP)

• Aimed at carers – Carers of Older People in Europe (COPE) Index – Family Impact of Childhood Disability measure

Things to consider

• Validity and reliability of measures

• What are they measuring?

• Are they relevant to AAC?

Survey

• Survey Monkey• Email • Twitter • Scotland, UK, International• Friday 1st – 8th March• Total responses 153

Sector

Geographical location

Profession

Extent of knowledge

Purpose

Outcome measure used Numbers

None used 58

Care Aims 9

TOMs 9

GAS 3

Talking Mats 3

CODES Framework 2

EFFC 2

Functional Communication 2

AFROM

Advocacy projects Outcomes 1

Outcome measure used Numbers

Carer feedback 1

East Kent Outcome Scale 1

Emotion Talks 1

Goal setting 1

In house 1

Sarah Blackstone’s Circles 1

Sign List 1

Talking Points 1

VASES 1

WAB 1

Frequency of use

Value

Why do we need Outcome Measures?

• I am not sure what an outcome measure is...• Demonstrates the value of interventions• ....

Strengths

• It allows therapists to be outcome led and aim for functional, achievable goals that are personal to the patients

• It provides consistent approach across whole caseload

• Allows us to check as a multi agency team that we are all assessing clients similarly within reasonable limits

Limitations

• No way of demonstrating outcomes with different types of AAC

• Difficult to share with service users• Not formally evidence based or standardised. Difficult

to prove effectiveness if measured against others• Too formal• Not consistently used in social settings• More difficult as intervention for higher tech users

usually cuts across a multi disciplinary/agency setting where it is harder to agree and measure outcomes

Expert panel days

•Presentation of literature review and survey

•What is an Outcome Measure?

•Directions for future work

Main points arising from expert panel days

• Agreed on what we mean by ‘Outcome Measure’

• Agreed on what is the best measure to use in AAC

• Identified the need for a measure to be workable across agencies

Definition of an outcome measure

Outcome Measures should:Measure changeAre repeated over timeCan be used across clients, settings and disciplinesCan be used to give feedback to services, professionals, carers and clients (about what works as well as what doesn’t)Are robust/standardised/well respectedInform dischargeAre short and straightforward to useAllow for comparison with different types of AAC/No AAC

Main contender for outcome measure in AAC

• TOMs AAC• (still in development – needs to be tested for

reliability and validity)

Final report available on our websitewww.talkingmats.com