SDQ added value score

Why do we need it, where it came from and what it can and cannot

do

Why do we need it?

• The chronic and fluctuating nature of childhood psychiatric symptoms

• Attenuation

• Regression to the mean

Change in psychopathology

0

5

10

15

20

25

Baseline six months

SD

Q t

ota

l dif

fic

ult

ies

sc

ore

No disorder

Emotional

Conduct

HKD

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Baseline Six months

No disorder

Emotional

Conduct

HKD

3 year follow-up of 1999 survey

a) Symptoms are persistent at a group level

ASD HK Cond Emot None

19992002 Follow-up

Totalsymptoms(SDQ)

3 year follow-up of 1999 survey

b) Impact is persistent at a group level

ASD HK Cond Emot None

19992002 Follow-up

Totalimpact(SDQ)

% reporting in the first interview

% reporting

in the second

interview

Agreed contacts between

two interviews

as % of total

contacts from both interviews

Kappa(standard

error)

Assessment only 40 28 70 0.74 (0.14)

CBT 56 52 80 0.76 (0.13)

Medication 40 36 73 0.75 (0.14)

-2

-1

0

1

2L

aten

t S

DQ

to

tal

(z-s

core

)

Time 1 Time 2

Where did it come from?

Children in the DoH British child and adolescent mental health survey 2004 and its six-month follow up who:-

• Were rated as having a psychiatric disorder• Or children with parents who had approached

primary health care or teachers in relation to this child’s mental health within the previous year.

• (n=604)

How did we calculate it?

• Calculated empirically using linear regression after exploring the factors that were known or suspected to increase the persistence of psychopathology.

• Added value (in SDQ points) = 2.3 + 0.8*baseline total difficulties score + 0.2*baseline impact score – 0.3* baseline emotional difficulties subscale score – follow up total difficulties score.

y= a+bx

y

x a = y intercept

b = slope

y= a+b1x1+b2x2+b3x3….

• Added value (in SDQ points) = 2.3 + 0.8*baseline total difficulties score + 0.2*baseline impact score – 0.3* baseline emotional difficulties subscale score – follow up total difficulties score.

• y = added value score• a = 2.3• b1 = 0.8 and x1= baseline total difficulties score• b2 = 0.2 and x2 = baseline impact score• b3 = 0.3 and x3 = baseline emotional difficulties subscale

score

Added value score = expected score– observed score

Added value (in SDQ points) = 2.3 + 0.8*baseline total difficulties score + 0.2*baseline impact score – 0.3* baseline emotional difficulties subscale score – follow up total difficulties score.

Where

Expected score given baseline parameters

Observed score

• Expected > observed: added value = positive; follow up score lower than expected; better than expected change

• Expected < observed; added value = negative; follow up score higher than expected; worse than expected change.

• Expected – observed; added value =0 => same change as in the community

-15.00 -10.00 -5.00 0.00 5.00 10.00 15.00

vatot

0

10

20

30

40

50

60

70

Fre

qu

en

cy

Mean = 0.0527Std. Dev. = 4.85165N = 455

Complexity factors

We looked at:-• Type or severity of

diagnosis• Age and gender• Poor physical health• Maternal educational level• Maternal anxiety and

depression• Family (type, function and

size)• Housing tenure• Neighbourhood

characteristics

Using stepwise linear regression, these factors explained:-

• 0.6% of variance of the added value score

• 35.9% of baseline SDQ scores• 24.2% of follow up SDQ

scores

• Ie. Very small influence of these factors on the SDQ added value score

Lot worse Bit worse same Bit better Lot better

Parent says how they were at T2 compared with T1

-3.00

-2.00

-1.00

0.00

1.00

2.00

3.00

4.00

Me

an

va

tot

-10 -5 0 5 10 15

vatot

0

5

10

15

20

25

Fre

qu

ency

Mean = -0.13Std. Dev. = 4.577N = 148

Added value score applied to children with psychiatric disorder from 1999 survey

-20.00 -10.00 0.00 10.00

vatot

0

2

4

6

8

10

Fre

qu

ency

Mean = 2.6949Std. Dev. = 6.72243N = 39

Added value score applied to 39 clinic cases; mean =2.7

Effect size =0.56

95% confidence interval 0.12-0.99

Does it work?

• Tested with data from a community based trial of the IY parenting program that had used to the SDQ at two time points 4-8 months apart, including the impact scale and detected a difference.

• If the SDQ value added scale worked it should accurately predict the change measured by the trial for the intervention group and while the control group should show no change

• We tested the added value score against simple change scores

(T1 total difficulties score – T2 total difficulties)

Effect size in standard deviation units

Expected value

Added value score

Change score

Control group

0 -0.03 0.35 *

Intervention group

0.37 0.36 0.65 **

The SDQ added value score can:-

• Accurately estimate change among groups of children with significant levels of impairing psychopathology, such as high risk groups or those attending services.

• We estimate that teams should be able to produce effect sizes of 0.1 or greater, but this needs testing with clinical data

• The test trial was on a Sure Start intervention in a community setting using sure start staff

Caveats The added value score is only calibrated for use with therapeutic or

targeted interventions and will overestimate change in groups with low levels of psychopathology, so it should not be applied to universal interventions.

• The added value score is a tool for evaluating the impact of interventions on groups of children, but the confidence intervals around the scores of individual children will be too wide to interpret in most instances.

The added value score requires follow up to occur between 4 and 8 months after the initial measure. Follow up after a fixed interval is preferable to administration at discharge because of the risk that discharge may follow soon after a spontaneous improvement, and thereby capitalize on chance remission.

Lee et al (2005)

Caveats The added value score is based on the SDQ, which is a “wide

angle” measure. Clinicians may want supplement the SDQ with more specific outcome measures relating to each child’s individual problems.

The use of multiple measures (clinician, parent, child, process, satisfaction) will provide services with richer data for improving services.

Services need to aim for high response rates from parents in order to obtain representative data. This requires resources.

• The added value score cannot provide all the answers to outcome monitoring; it is one of an array of tools

• There is a need for further replication with clinical and research data