EVALUATING POSTGRADUATE CBT TRAINING IN NEW ZEALAND

DR BEVERLY HAARHOFF MASSEY UNIVERSITY, AUCKLAND.

Transfer of training: “Fade away or show and tell!”

Agenda

Why is the evaluation of maintenance and transfer of training in psychotherapy important?

Obstacles and difficulties in assessing transferBrief look at current state of play in terms of

research in the CBT fieldOur study (descriptive data)Problems

A way forward

Defining transfer of training

“Transfer of training is defined as the generalisation and maintenance of new information, knowledge, attitudes and skills into the everyday practice of trainees”

Baldwin & Ford, 1988.

Transfer of training why measure?

Management and allocation of resources

Increasing access to evidenced based therapies (IAPT program)

What works? For whom?To improve

patient/client outcome

Transfer of training in Psychotherapy?

*Difficult area to researchComparatively few studies (increasing in CBT)Lack of funding (the field falls between

psychotherapy & education)Competing psychotherapy training programs

and paradigms(profession specific, short-term,

comprehensive, scholar-practitioner, scientist-practitioner, ‘apprenticeship model’, emphasis on personal therapy etc.)

Evaluating CBT transfer of training

More complications!What represents competence? (which

competecies? General or specific?)Trainee Self-report ?Supervisor ‘indirect’ report?Objective scoring of competence? Evaluation of patient outcome?The willingness of graduates to participate? Small participant samples (limited access

programs)

Transfer of training in CBT

Empirical studies have increased since 1999Still heavily reliant on the self report of

trainees The few studies using ‘objective’ observation

methods such as the Cognitive Therapy Scale (Young & Beck, 1980) measure overall competence have have generally been positive.

Very few linked to outcome or praticing CBT therapists

Recent studies

McManus, Westbrook, Vazquez-Montes, Fennell & Kennerley, 2010.Oxford Diploma CBT Course278 trainees between 1998-2009Increased competence Clinical Psychologists achieved consistently better resultsAge negatively correlated with improved competency

Brosan, Reynolds & Moore, 2006Naturalistic studyPractitioners recruited from a range of professionsmid t/m recorded CBT session rated using the CTS5380 practitioners approached47 responded & only 24 submitted tapes

Psychologists found to be the most competentHowever a number of accredited therapist scored below the expected level of competence

The Massey University Postgraduate CBT diploma

Minimum of two years Part-timePhase one: theoretical: four papers taught in

block mode namely Theory and Practice of CBT, CBT for Depression, CBT for the Anxiety Disorders and CBT for chronic and complex problems.

Phase two: Supervised clinical practicum over two semesters. x Two clients, two case studies and two verbal case presentations. (all supervision in-house)

Final oral exam with the focus on case conceptualisation

The study

Questions:★Is CBT competence maintained and transferred

beyond training? Is therapist self-reported competence in using CBT related

to observed use of CBT?

Is professional development as therapist related to self-reported competence and observed competence?

Is work involvement positively related to self-reported and observed competence as a CBT therapist?

Measures

*Cognitive Therapy Scale (Young & Beck, 1980, 1988) (the most widely used & validated instrument for assessing CBT competence McManus et al., 2010)

Adapted Survey of PGDipCBT (Kennedy-Merrick et al., 2006)

Therapist Professional Development questionnaire (Orlinsky & Rønnestad, 2005)

Psychotherapist work involvement questionnaire (Orlinsky & Rønnestad, 2005)

The Cognitive Therapy Scale (Young & Beck, 1980, 1988)

General therapeutic procedures & interpersonal

effectiveness

1. Agenda2. Feedback3. Understanding4. Interpersonal

effectiveness5. Collaboration6. Pacing & efficient use of

time

‘Red-line’ = 39-44 (RCT)

Specific CBT skills

1. Guided discovery2. Focusing on key

cognitions and behaviours

3. Strategy for change4. Application of CBT

Techniques5. Homework

Each item rated on a (0-6) Likert scale yielding a total score of 66

Participants

The participants were drawn from two groups of CBT trainees

Group one: trainees enrolled in the PGDipCBT during 2009

*Group two: Graduates from the PGDipCBT (2000-2009)

Mental health practitioners across multiple professions: psychologists, social workers, nurses, psychiatric registrars and consultants, GP’s, Occupational therapists, psychotherapists & counsellors

Participants : Group Two

Of the 88 graduates between 2000-2009, 42% (n=37) were not contactable

Of the remaining 58% (n=51), 20 were currently not practicing CBT and 9 declined to participate.

A total of 29 agreed to participate in the study but of this group only 12 provided recorded examples of their clinical work.

Of this sample one tape was unusable and only 11 clinicians eventually took part (12.5% of the total sample)

Results: Cognitive Therapy Scale (Young & Beck,1980,1988)

Participant professional background

General therapeutic competency (6)

Specific CBT competency (5)

Total Cognitive Therapy scale Score (66)

1. Nurse 25.5 18.5 44

2. CBT therapist 33 26.5 59.5

3. Psychologist/CS

32 26 58

4. Nurse 34 26.5 60.5

5. OT 32.5 28 60.5

6. Psychologist/CS

32 24.5 56.5

7. Psychologist/CS

31.5 20.5 52

8. Psychotherapist

18.5 8 26.5

9. Psychologist/GS

16 5 21

10. OT 18.5 8 26.5

11.Nurse 20.5 11 31.5

Competent?

Out of 11 participants n=7 (64%) achieved competence scoring above the 39 ‘red-line’ competence for RCT criteria

3 of 7 were clinical psychologists, 43%

agen

da F/B u/st

Inte

r/per

colla

bora

tion

stra

tegy

/chan

ge GD

key c

og. &

Beh

avio

ur

stra

tegy

for c

hang

e

appl

icat

ion

of C

BTHW

0

1

2

3

4

5

6

PsychotherapistPchol GSOTNurse

Poor performance in specific CBT competencies and skills

Participants below the ‘red-line’

Difficulties: Participant recruitment

Difficulties

Fear of negative evaluation

Obtaining recorded clinical material (patient consent, graduate resistance, technological difficulties)

Expensive (scoring of competence measures)

A way forward

Providing a strong rationale for participation in competency evaluation

Building in positive reinforcement for participation

Building a culture of accountability

Improving access to supervision

Technological support & know how

Deseminating feedback from research

Acknowledgements

Robyn Gedye (PhD candidate)

Dr Mei Williams Senior Lecturer Massey University

Lynley Stenhouse (Clinical Psychologist)Sarah Kennedy-Merrick PhD

Study participants Graduates of the Massey University Post Graduate Diploma in CBT 2000-2009.

questions