![Page 1: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/1.jpg)
Chapter 5
Instrument Selection, Administration, Scoring, and Communicating Results
![Page 2: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/2.jpg)
Selection of an Assessment Instrument
Identify possible alternatives Educational Testing Services may be
useful Mental Measurement Yearbook (MMY)
reviews many tests and is available at Morris or reviews can be ordered on-line for a fee.
Client’s lives can be adversely affected by selecting and using a faulty instrument
![Page 3: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/3.jpg)
Evaluating an Instrument Test Purpose Instrument
Development Appropriate Norm
Group or Criterion Reliability
Validity Bias Interpretation and
Scoring User Qualifications
![Page 4: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/4.jpg)
Test purpose
Does instrument meet counseling needs?
Manual may state purpose Purpose may not relate to name Purpose may not be met because of
construction problems
![Page 5: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/5.jpg)
Instrument development
Construction of items crucial Look for detailed item analysis data Face validity
![Page 6: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/6.jpg)
Selection of norm group
Is norming group appropriate for client? Age Gender Ethnicity Socioeconomic representation Educational level Geographic location
![Page 7: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/7.jpg)
Selection of criterion
How did developers determine criterion? Standards from professional
organizations Reviews by leading researchers Studies of curriculum
Manual must supply enough data to be convincing
![Page 8: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/8.jpg)
Reliability
No clear guidelines for what constitutes “good” reliability
Type of instrument affects how reliability should be viewed
Can be estimated in different ways and manuals will often provide range of coefficients calculated using different methods
![Page 9: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/9.jpg)
Validity
Look for preponderence of evidence Manual must provide enough
information to judge GO TO THE LITERATURE!
![Page 10: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/10.jpg)
Bias
Test bias in context of multicultural issues Differential item functioning (item
difficulty differs for ethnic groups) Reliability coefficients may differ
ethnically Construct validity may be compromised
as the construct may be culturally based
![Page 11: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/11.jpg)
Interpretation and scoring materials
Manual must provide adequate description of what scores mean
Hand vs. machine scoring
![Page 12: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/12.jpg)
User qualifications Level A: manual is sufficient (Holland’s Self-
Directed Search) Level B: master’s degree or equivalent
training (Myers-Briggs, Strong Interest Inventory)
Level C: doctoral level or equivalent training (Wechslers, Binet, MMPI-2)
Code of Fair Testing Practices in Education ACA Code of Ethics, Section E
![Page 13: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/13.jpg)
Practical issues
Cost Time
![Page 14: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/14.jpg)
Evaluating instruments
Form in text, page 95 Instrument evaluation paper
![Page 15: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/15.jpg)
Administering Assessment Instruments
Pre-read administration materials Follow standardized procedures
![Page 16: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/16.jpg)
Administrator Effects
Expectancy (pygmalion effect) Some data support minor effects
![Page 17: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/17.jpg)
Examiner/Examinee Relationship
Some studies support impact of relationship for kids and even more so for lower SES kids
Standardized test manuals usually address relationship boundaries
![Page 18: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/18.jpg)
Examiner race
Research equivocal
![Page 19: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/19.jpg)
Scoring
Hand scoring Clients can score some assessments
(SDS) but may be clinical reason not to Some tests require multiple templates
Computer scoring and interpretation Few to no errors Counselors ethically bound to know
integrity of service and steps taken to develop scoring and interpretation programs
![Page 20: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/20.jpg)
Performance and Authentic Assessment Scoring
Associated with achievement testing (rather than grammar test, write business letter)
Goal is to see if knowledge can be applied
Big issue is objectivity of scoring
![Page 21: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/21.jpg)
Performance and Authentic Assessment Scoring Guidelines
Assessment has specific focus
Scoring plan is based on observable qualities
Scoring is designed to reflect the intended target
Setting for assessment is appropriate
Checklist or rating scales are used
Scoring procedures have been field tested before they are used
![Page 22: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/22.jpg)
Communicating Results
Must know manual information Optimize power of test (It’s useful in
these ways for these reasons) rather than allowing client to maximize (test speaks truth)
Use effective counseling skills Develop multiple methods of
explaining in “plain” English
![Page 23: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/23.jpg)
Communicating Results (cont’d)
Use descriptive terms rather than numerical scores and tie them to the reason for the assessment
Put results in context of other client information
Involve client in interpretation and ask for feedback often
![Page 24: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/24.jpg)
Communicating Results (cont’d)
Limitations are discussed in non-technical terms
Encourage client to ask questions Summarize results to iterate and
stress important points
![Page 25: Chapter 5 Instrument Selection, Administration, Scoring, and Communicating Results](https://reader030.vdocuments.net/reader030/viewer/2022020715/56649d4c5503460f94a2a251/html5/thumbnails/25.jpg)
Communicating Results to Parents
Be empathic Monitor parents’ reactions so child
does not internalize as blame Provide therapeutic environment