principles of instrument & measurement development bonnie l. halpern-felsher, ph.d. professor...
TRANSCRIPT
![Page 1: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/1.jpg)
Principles of Instrument & Measurement Development
Bonnie L. Halpern-Felsher, Ph.D.
Professor
University of California,
San Francisco
![Page 2: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/2.jpg)
What should be considered when we develop an entire
survey or instrument?
![Page 3: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/3.jpg)
Principles of Instrument Development
![Page 4: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/4.jpg)
Identify Research Aims/Questions
• What are the goals of your study?
• Are you exploring:– Patterns over time?– Within subject changes?– Associations or Predictions?
• Are your measures addressing these aims?
![Page 5: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/5.jpg)
Identify Conclusions, Discussion Points, & Implications
• What do you hope to conclude?
• Will your instrument and measures provide you with the information to make such conclusions?
![Page 6: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/6.jpg)
Know Your Research Design
• Quantitative, qualitative, mixed-methods?
• Cross-sectional?
• Repeated Measures?
• Longitudinal?– Interval between measurement waves?
![Page 7: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/7.jpg)
Longitudinal Studies
• In general, use identical measures over time.
• Can add or subtract items or measures.
• Can substitute measures for more developmentally appropriate ones, but must assure they are highly correlated.
![Page 8: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/8.jpg)
Know Your Research Subjects
• Children? Adolescents? Young adults?
• Age comparisons?
• Clinic sample?
• At-risk sample?
• School-based sample?
• Reading level?
![Page 9: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/9.jpg)
Identify & Define Constructs
• What are you trying to measure?• Do constructs address research
aims/questions?• Can respondents answer questions?• Will venue allow you to ask these
questions?• Are there existing measures for constructs?
OR Need to develop new measures?
![Page 10: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/10.jpg)
Consider the Length of the Instrument
• Depends on the sample• Depends on the administration method• Depends on time and level of permission
given
![Page 11: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/11.jpg)
Consider the Instrument Format
• Paper/pencil
• Web-based
• ACASI
• IPad
![Page 12: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/12.jpg)
Want an Interesting Instrument
• Pictures
• Mix of questions
• Mix of format
• Reduce redundancy
![Page 13: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/13.jpg)
Want Consistency across Measures within the
Instrument
• Order of response (yes/no; scales)
![Page 14: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/14.jpg)
Principles of Measurement Development
![Page 15: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/15.jpg)
Principles of Measurement Development
• Measures need to provide answers containing important information that directly addresses research aims.
• Measures should provide comparable information about people, events or behavior.
• Measures should provide reliable and valid information.
![Page 16: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/16.jpg)
Provide Important Information
• Go back to research aims and questions.
![Page 17: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/17.jpg)
Provide Comparable Information
• All respondents must be answering the same question.
• All respondents must understand each item and attribute the same meaning to it.
• Assure that researchers and respondents are using the same definition.
![Page 18: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/18.jpg)
Provide Comparable Information
- Will items produce meaningful information from adolescents and adults? • Cannot conclude age or developmental
differences if it is really just a reflection of interpretation.
![Page 19: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/19.jpg)
Respondents are Able to Respond
• Items are specific, not ambiguous.
• Respondents know and remember information.
• Minimize burden of recall.
• If recall is needed, help respondent:– Place events or behavior in time– Use memory aids
![Page 20: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/20.jpg)
Respondents are Able to Respond
• Examples:– Factual Data: • SES, Parent Education, Income, Medical Data.• Can ask the source (e.g., parents, teachers, chart
reviews
– Event-Specific Data: • Since high school graduation…• Since we last surveyed you…• Provide calendars…
![Page 21: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/21.jpg)
Respondents are Willing to Respond
• Items should be written such that respondent does not feel the need to respond inaccurately.
• Assure confidentiality.
![Page 22: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/22.jpg)
Respondents are Willing to Respond
• Handling “Don’t Know” and “Not Sure” Responses:– Can provide a screening question– Can provide an option for N/A, None, Don’t
Know– Don’t assume or judge
![Page 23: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/23.jpg)
For Next Week
• Find existing measures of interest• Create/continue creating/find an
instrument with:– At least 2:• factual questions
• frequency and quantity questions
• “feelings” subjective questions
• Evaluative questions
• Scales
![Page 24: Principles of Instrument & Measurement Development Bonnie L. Halpern-Felsher, Ph.D. Professor University of California, San Francisco](https://reader036.vdocuments.net/reader036/viewer/2022062422/56649f125503460f94c257a1/html5/thumbnails/24.jpg)
For Next Week
• Keep in mind the principles of instrument and survey development when constructing your survey