item analysis of the seated postural control measure (spcm ... · analysis. hypothesis 2: item...

1
Research Version of the Seated Postural Control Measure (SPCM) - for children with neuromotor dis- abilities - an evaluative measure - 34 criterion-referenced items - 22 Alignment, 12 Function - NCMRR - impairment and func- tional limitations - ICF 3 - impairment, activity - content validity, concurrent validity, rater reliability and test- retest reliability established 1,2 To analyze the item properties of the SPCM to determine whether there are redundant items and whether items have the capacity to detect change. Hypotheses Item homogeneity - inter-item correlations will be less than or equal to 0.95 The item difficulty index in the pre-seating condition will be less than or equal to 3.50 for multi-point items and less than or equal to 3.83 for limb items with only two score levels. The item discrimination index, individual gain (DIS IG ), will be greater than or equal to 0.2 (proportional increase from the pre-seating to post-seating condition). This study involved secondary analysis of data from 2 previous repeated measures reliability studies 1,2 . In the first reliability study (n=41) the SPCM was administered 8 times - 2 raters, 2 occasions and 2 seating conditions (with and without seat). In the second reliability study (n=51) two raters administered the SPCM on two occasions three weeks apart. Characteristics of Participants For Hypothesis 1: Item Homogeneity Analysis consisted of calculation of correlation coefficients for each of the 561 item pairs using data from the 1993 reliability study. Scatter plots were graphed to determine whether the assumptions of the correlation coefficients were met. If all assumptions were met, the Spearman rank correlation coefficient was calculated for each item pair in each data set. The mean correlations across all four data sets were calculated for items with at least one cor- relation greater than 0.90 (so that means would only need to be calculated for those item pairs which might possibly have mean correlations nearing 0.95). For Hypothesis 2: Item Difficulty Data from the Fife et al. (1991) reliability study collected in the pre-seating condition were used to calculate the item difficulty indices. Because item reliability was not sufficiently high for all items across raters and time, item scores could not be combined across data sets. The item difficulty (mean score) for each item was calculated for each of the four data sets and the mean item difficulty then calculated. The mean item difficulty index was exam- ined in relation to the preset levels for multi-point and two-point items to determine whether hypothesis 2 was upheld for each item. For Hypothesis 3: Item Discrimination The individual gain (Berk 4 ) discrimination index (DIS IG ) was calculated for each of the 34 SPCM items using data sets from the 1991 study. The four discrimination indices for each item were aver- aged and the mean DIS IG examined to determine whether each was greater than or equal to 0.2 and thus whether hypothesis 3 was upheld for each item. Missing Data Thirty-eight of a possible 532 tests were missing and 147 of the remaining 16,796 items scores. Items with missing data were omitted from the calculations. Hypothesis 1: Item Homogeneity Scatter plots of the 561 item pairs in each of the four data sets revealed 37 item pairs which did not meet one or more of the assumptions for use of the correlation coeffi- cient; these item pairs were thus excluded from the data analysis. Hypothesis 2: Item Difficulty Hypothesis 3: Item Discrimination The Function items that had a low discrimination index but acceptable difficulty index and item homogeneity will be retained.Only minor changes to 3 items and elimina- tion of one redundant item is required to achieve opti- mum item properties for this evaluative measure prior to undertaking a full responsiveness study. References: 1. Fife SE, Roxborough LA, Armstrong RW, Harris SR, Gregson JL, Field D (1991) Development of a clinical measure of pos- tural control for assessment of adaptive seating in children with neuromotor disabilities. Physical Therapy, 71, 981-993. 2. Fife SE, Roxborough LA, Story M, Field D, Harris SR, Armstrong R (1993) Reliability of a measure to assess out- comes of adaptive seating in children with neuromotor disabili- ties. Canadian Journal of Rehabilitation, 7, 11-12. 3. World Health Organization (2001) International Classification of Functioning, Disability and Health. Geneva: World Health Organization. 4. Berk R (1984) Conducting the item analysis. In Berk RA (Ed), A Guide to Criterion-Referenced Test Construction. Baltimore MD: Johns Hopkins University Press. Item analysis of the Seated Postural Control Measure (SPCM) Lori Roxborough*, MSc OT/PT and Beth Ott, MSc PT Children's and Women's Health Centre of British Columbia, Sunny Hill Site 1991 Study 1993 Study (N=41) (N=51) Sex Male 23 27 Female 18 24 Age <3 6 4 3 - 5 8 8 6 - 9 6 5 9 - 16 17 20 >16 4 14 Medical Diagnosis Cerebral palsy 20 25 Brain injury 7 6 Meningomyelocele 2 4 Muscle disease 3 2 Developmental delay 2 0 Chromosomal abnormality 0 3 Other 7 11 Item difficulty > 3.5 multi-point items 0 1 2 3 4 A19 A20 F1 Acknowledgements: Dr. Susan Harris, Thesis Supervisor, Graduate Studies School of Rehab Sciences, University of British Columbia Funding: CASMA and COTF Poster design: David Jordan Item - Item correlations > 0.95 0 1 Introduction Purpose Data Analysis Item Difficulty A-Items Acceptable F-Items Acceptable A-Items Unacceptable F-Items Unacceptable Pre to post seating increase < .2 0.00 0.05 0.10 0.15 0.20 A14 F1 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12 Results Methods Conclusion Item Homogeneity A-Items Acceptable F-Items Acceptable A-Items Unacceptable F-Items Unacceptable Item Discrimination A-Items Acceptable F-Items Acceptable A-Items Unacceptable F-Items Unacceptable DIS IG = # increased scores pre to post seating total # of scores

Upload: others

Post on 19-Jun-2020

10 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Item analysis of the Seated Postural Control Measure (SPCM ... · analysis. Hypothesis 2: Item Difficulty Hypothesis 3: Item Discrimination The Function items that had a low discrimination

Research Version of the SeatedPostural Control Measure (SPCM)- for children with neuromotor dis-abilities- an evaluative measure- 34 criterion-referenced items- 22 Alignment, 12 Function- NCMRR - impairment and func-tional limitations- ICF

3- impairment, activity

- content validity, concurrentvalidity, rater reliability and test-retest reliability established

1,2

To analyze the item properties of the SPCM to determinewhether there are redundant items and whether itemshave the capacity to detect change.

Hypotheses

Item homogeneity - inter-item correlations will be lessthan or equal to 0.95

The item difficulty index in the pre-seating condition willbe less than or equal to 3.50 for multi-point items and lessthan or equal to 3.83 for limb items with only two scorelevels.

The item discrimination index, individual gain (DISIG),will be greater than or equal to 0.2 (proportional increasefrom the pre-seating to post-seating condition).

This study involved secondaryanalysis of data from 2 previousrepeated measures reliabilitystudies

1,2. In the first reliability

study (n=41) the SPCM wasadministered 8 times - 2 raters, 2occasions and 2 seating conditions(with and without seat). In thesecond reliability study (n=51)two raters administered the SPCMon two occasions three weeks apart.

Characteristics of Participants

For Hypothesis 1: Item HomogeneityAnalysis consisted of calculation of correlation coefficientsfor each of the 561 item pairs using data from the 1993reliability study. Scatter plots were graphed to determinewhether the assumptions of the correlation coefficientswere met. If all assumptions were met, the Spearmanrank correlation coefficient was calculated for each itempair in each data set. The mean correlations across all fourdata sets were calculated for items with at least one cor-relation greater than 0.90 (so that means would only needto be calculated for those item pairs which might possiblyhave mean correlations nearing 0.95).

For Hypothesis 2: Item Difficulty Data from the Fife et al. (1991) reliability study collectedin the pre-seating condition were used to calculate theitem difficulty indices. Because item reliability was notsufficiently high for all items across raters and time, itemscores could not be combined across data sets. The itemdifficulty (mean score) for each item was calculated foreach of the four data sets and the mean item difficultythen calculated. The mean item difficulty index was exam-ined in relation to the preset levels for multi-point andtwo-point items to determine whether hypothesis 2 wasupheld for each item.

For Hypothesis 3: Item DiscriminationThe individual gain (Berk

4) discrimination index (DISIG)

was calculated for each of the 34 SPCM items using datasets from the 1991 study.

The four discrimination indices for each item were aver-aged and the mean DISIG examined to determine whethereach was greater than or equal to 0.2 and thus whetherhypothesis 3 was upheld for each item.

Missing DataThirty-eight of a possible 532 tests were missing and 147of the remaining 16,796 items scores. Items with missingdata were omitted from the calculations.

Hypothesis 1: Item HomogeneityScatter plots of the 561 item pairs in each of the four datasets revealed 37 item pairs which did not meet one ormore of the assumptions for use of the correlation coeffi-cient; these item pairs were thus excluded from the dataanalysis.

Hypothesis 2: Item Difficulty

Hypothesis 3: Item Discrimination

The Function items that had a low discrimination indexbut acceptable difficulty index and item homogeneity willbe retained.Only minor changes to 3 items and elimina-tion of one redundant item is required to achieve opti-mum item properties for this evaluative measure prior toundertaking a full responsiveness study.

References:

1. Fife SE, Roxborough LA, Armstrong RW, Harris SR, GregsonJL, Field D (1991) Development of a clinical measure of pos-tural control for assessment of adaptive seating in children withneuromotor disabilities. Physical Therapy, 71, 981-993.2. Fife SE, Roxborough LA, Story M, Field D, Harris SR,Armstrong R (1993) Reliability of a measure to assess out-comes of adaptive seating in children with neuromotor disabili-ties. Canadian Journal of Rehabilitation, 7, 11-12.3. World Health Organization (2001) InternationalClassification of Functioning, Disability and Health. Geneva:World Health Organization.4. Berk R (1984) Conducting the item analysis. In Berk RA(Ed), A Guide to Criterion-Referenced Test Construction.Baltimore MD: Johns Hopkins University Press.

Item analysis of the Seated Postural Control Measure (SPCM)Lori Roxborough*, MSc OT/PT and Beth Ott, MSc PT

Children's and Women's Health Centre of British Columbia, Sunny Hill Site

1991 Study 1993 Study

(N=41) (N=51)

Sex

Male 23 27

Female 18 24

Age

<3 6 4

3 - 5 8 8

6 - 9 6 5

9 - 16 17 20

>16 4 14

Medical Diagnosis

Cerebral palsy 20 25

Brain injury 7 6

Meningomyelocele 2 4

Muscle disease 3 2

Developmental delay 2 0

Chromosomal abnormality 0 3

Other 7 11

Item difficulty > 3.5

multi-point items

0

1

2

3

4

A19 A20 F1

Acknowledgements:

Dr. Susan Harris, Thesis Supervisor, Graduate Studies

School of Rehab Sciences, University of British Columbia

Funding: CASMA and COTF

Poster design: David Jordan

Item Difficulty > 3.5

3

4

Item - Item correlations >0.95

0

1

Introduction

PurposeData Analysis

Item Difficulty

A-Items Acceptable F-Items Acceptable

A-Items Unacceptable F-Items Unacceptable

Pre to post seating increase <.2

0.00

0.05

0.10

0.15

0.20

A14 F1 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12

Results

Methods

Conclusion

Item Homogeneity

A-Items Acceptable F-Items Acceptable

A-Items Unacceptable F-Items Unacceptable

Item Discrimination

A-Items Acceptable F-Items Acceptable

A-Items Unacceptable F-Items Unacceptable

DISIG = # increased scores pre to post seatingtotal # of scores