chapter overview deciding on a method tests and their development types of tests observational...
TRANSCRIPT
![Page 1: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/1.jpg)
CHAPTER OVERVIEW
• Deciding on a Method• Tests and Their Development• Types of Tests• Observational Techniques• Questionnaires
![Page 2: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/2.jpg)
SOME IMPORTANT FIRST CONSIDERATIONS
• Is the tool you propose to use reliable and valid?
• Base your choice of research tools on how you have asked the research question
![Page 3: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/3.jpg)
TESTS AND THEIR DEVELOPMENT
• A test should measure the nature and extent of individual differences
• A good test differentiates people based on true scores
![Page 4: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/4.jpg)
WHY USE TESTS?
• Help determine outcomes of experiments
• Can be used to diagnose strengths and weaknesses
• Assist in placing individuals in appropriate programs
• Assist in selecting applicants• Used to evaluate a program’s
effectiveness
![Page 5: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/5.jpg)
TYPES OF TESTSOverview
What It Does
Achievement Test Assesses an individual’s knowledge of a specific area
Attitude Test Assesses an individual’s feelings about an issue
Personality Test Assesses stable individual behavior patterns
![Page 6: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/6.jpg)
ACHIEVEMENT TEST SOURCES
• Standardized– Commercially prepared for wide use– Scoring instructions included
• Researcher-made– Designed by user for specific purpose– Scoring instructions specific to test
![Page 7: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/7.jpg)
ACHIEVEMENT TESTS REFERENCING
What Comparison Do Tests Make?
• Norm-referenced—Individual’s scores are compared to results from a larger group
• Criterion-referenced—Individual’s scores are compared to defined performance standards
![Page 8: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/8.jpg)
MULTIPLE-CHOICE ACHIEVEMENT ITEMS
Anatomy of a Multiple-Choice Item
12. Intelligence tests that are given to preschool children a. favor middle-class children b. have questionable construct validity c. are based on motor skills d. are no fun at all
STEMClearly written
CORRECT ANSWER
DISTRACTERSShould be plausible (b & c), not easily ruled out
(d)
![Page 9: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/9.jpg)
TO USE OR NOT TO USE?• Advantages
– Versatile– Easy to score– Simple to take– Poor writers not
penalized– Good items used
again– Good distracters are
diagnostic– Hard to fake correct
answer
• Disadvantages– Limit student’s
options– No opportunity to
practice writing– Some people don’t do
well on them– Limits content to be
assessed– Items must be well
written
![Page 10: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/10.jpg)
ITEM ANALYSIS: HOW TO TELL IF YOUR ITEMS WORK• Questions should discriminate
those who know the material from those who don’t
• Item analysis provides two measures of a question’s ability to discriminate– Difficulty index– Discrimination index
![Page 11: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/11.jpg)
COMPUTING INDICESFirst Steps
1. Rank scores from highest to lowest2. Choose top 27% of scores for “high” group3. Choose bottom 27% of scores for “low”
group
![Page 12: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/12.jpg)
COMPUTING INDICES12. Intelligence tests that are given to preschool children
a. favor middle-class children b. have questionable construct validity c. are based on motor skills
d. are no fun at all
Item Alternative A B C D Total
High Group (n = 41)
23 12 4 2 41
Low Group (n = 41)
11 9 15 6 41
Total 34 21 19 8 82
![Page 13: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/13.jpg)
COMPUTING INDICESDifficulty index• Proportion who
answered item correctly
• D = NCh + NCl
T
Discrimination index• Proportion in high
group who answered correctly – proportion in low group who answered correctly
• D = NCh - NCl
(.5)T
NCh
= number of people in high group answering correctly
NCl= number of people in low group answering correctly
T = total number of people in high and low groups
![Page 14: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/14.jpg)
RELATIONSHIP BETWEEN ITEM DISCRIMINATION AND DIFFICULTY
0%
+1.00
50% 100%
-1.00
0
Difficulty Level
Dis
crim
inat
ion
Lev
el
Perfect Discrimination When: ½ gets item right, ½ gets item wrong
& ½ right is in upper half, ½ wrong is in lower half
![Page 15: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/15.jpg)
ATTITUDE TESTSAssess an Individual’s Feelings About
a Topic
Item Agree No Strong Feeling
Disagree
The day before Thanksgiving should be a holiday.
____
____
____
Final exams should be elective. ____ ____ ____
The dining room should serve gourmet food.
____
____
____
My parents don’t appreciate how smart I am.
____
____
____
My professors don’t appreciate how smart I am, either.
____
____
____
![Page 16: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/16.jpg)
THURSTONE SCALESMethod of Equal Appearing
Intervals
• Statements indicating an attitude toward a topic are written
• Judges rank the statements from least favorable to most favorable
• Statements receiving consistent ratings are given the average score
• A set of statements covering the entire range of attitudes is selected
![Page 17: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/17.jpg)
THURSTONE SCALESAdministration
• Respondents check items with which they agree– Well-formed attitudes indicated by
consistently checking either high or low items
– Poorly-formed or inconsistent attitudes indicated by inconsistent pattern or by checking off many neutral items
![Page 18: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/18.jpg)
LIKERT SCALES
• Statements indicating an attitude toward a topic are written
• Items with clearly positive or negative attitudes are selected
• Statements are listed with a space for respondent to indicate degree of agreement
![Page 19: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/19.jpg)
Item Rating
Government has no business funding child care programs.
SD D U A SA
Child care should be supported by federal, state, and local tax dollars.
SD D U A SA
A LIKERT SCALE
Directions: Indicate to what extent you agree or disagree with the statements listed below by circling one of the following:
SA means that you strongly agree with the statementA means that you agree with the statementU means that you are undecided about the statementD means that you disagree with the statementSD means that you strongly disagree with the statement
![Page 20: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/20.jpg)
SCORING LIKERT RESPONSESMethod of Summated Ratings
• Items are weighted• Weights of unfavorable items are reversed• Average score is computed
Item Rating
Government has no business funding child care programs.
SD D U A SA
Child care should be supported by federal, state, and local tax dollars.
SD D U A SA
![Page 21: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/21.jpg)
PERSONALITY TESTS
• Projective tests– Present respondent with ambiguous
stimulus
• Structured tests– Questions are objective
![Page 22: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/22.jpg)
OBSERVATIONAL TECHNIQUES
• Researcher observes and records• Does not interfere with behavior
![Page 23: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/23.jpg)
TECHNIQUES FOR RECORDING BEHAVIOR
• Duration—how long the behavior lasts• Frequency—how frequently the
behavior occurs • Interval (time sampling)—target
behaviors of each subject during set period of time
• Continuous—all behaviors of the subject
![Page 24: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/24.jpg)
TECHNIQUES FOR RECORDING BEHAVIOR
Technique How it Works Example
Duration recording The researcher records the length of time that a behavior occurs.
How much time is spent in verbal interaction between two children?
Frequency recording The researcher records the number of times a behavior occurs.
How often are questions asked?
Interval recording The researcher observes a subject for a fixed amount of time.
Within a 60-second period, how many times do members of the group talk to another person?
Continuous recording The researcher records everything that happens.
During a 1-hour period, all the behavior of a 6-year-old boy is recorded.
![Page 25: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/25.jpg)
OBSERVATIONAL TECHNIQUES? BE CAREFUL!
Pitfalls to Avoid
• Observer effects• Observer bias• Fatigue• Changing definitions
![Page 26: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/26.jpg)
QUESTIONNAIRES
• What they are– Paper and pencil tests with structured
questions– Self-administered
![Page 27: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/27.jpg)
QUESTIONNAIRES
• Advantages– Can be mailed out– Survey broad geographic area– Cheaper than one-on-one interview– Respondents may be more honest– Data easy to share with other
researchers• Disadvantage
– Low return rate
![Page 28: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/28.jpg)
BASIC ASSUMPTIONS OF A QUESTIONNAIRE
• Does not make unreasonable demands upon the respondent
• Does not have a hidden purpose• Requests information that
respondents presumably have• Contains interesting questions• Does not request information that
could be obtained by other means
![Page 29: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/29.jpg)
THE QUESTIONS
• The questionnaire contains questions that can be answered
• The questionnaire contains questions that are straightforward
![Page 30: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/30.jpg)
THE FORMAT• The presentation is attractive,
professional, and easy-to-understand• Questions and pages are clearly
numbered• Directions are clear and explicit• Questions are objective• Questions are ordered sensibly• Transitions are used from one topic to
the next
![Page 31: CHAPTER OVERVIEW Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649f395503460f94c55d15/html5/thumbnails/31.jpg)
THE IMPORTANCE OF USING A COVER LETTER
• Informs the recipient about the research
• Establishes the importance of the research
• Makes the recipient a part of the research