chapter 8 developing written tests and surveys physical fitness knowledge

Chapter 8Developing Written Tests and

Surveys

PhysicalFitnessKnowledge

Test Planning

Types

Mastery

Achievement

Table of Specifications

Content Objectiveshistory, values, equipment, etiquette,safety, rules, strategy, techniquesof play

Educational Objectivesknowledge, comprehension,application, analysis, synthesis,evaluation

Table 8.1 Table of Specifications for a 60 Item Written Test on

Badminton

Educational objectives

Con

ten

t ob

ject

ives

Test Characteristics

How to measure

When to test

How many questions?

What format should be used?

What types of questions should be used?

Essay Questions

When to use

Weaknesses

Scoring

Objectivity

Construction recommendations

Scoring recommendations

Semi-Objective Questions

When to use

Weaknesses

Construction recommendations

Scoring recommendations

Objective Questions

True/False

MatchingMultiple Choice

Construction recommendationsScoring recommendations

CompetenciesCompetency A

correctB

correct

Ccorrect

D correct

Total

Motor development and motor leaning – Competency 1 23 16 13 9 61

Fitness and fitness development/maintenance – Competency 2 21 14 13 14 62

Lifetime individual, dual, and group activities – Competency 3 6 7 10 12 35

Fitness-related health, nutrition, and safety – Competency 4 16 13 15 10 54

Affective development – Competency 5 8 7 6 5 26

Social development – Competency 6 4 2 6 5 17

Cognitive development – Competency 7 8 8 8 6 30

Physical education program – Competency 8 5 6 9 10 30

Learner assessment – Competency 9 4 13 8 18 43

Managing physical education classes – Competency 10 11 12 9 13 45

Legal, ethical, medical, and safety concerns – Competency 11 10 9 14 18 51

Total correct response 116 107 111 120 454

% Correct response by foil 26% 24% 24% 26% 100%

Figure 8.1 The difference Between Extrinsic and Intrinsic Ambiguity

Administering the Written Test

Before the test

During the test

After the test

Characteristics of Good Test Items

Leave little to "chance"ReliableRelevantValidAverage difficultyDiscriminate

Gotten correct by more knowledgeable studentsMissed by less knowledgeable students

Time consuming to write

Some Test-Taking SkillsReview these important skills on pages 171-172

Preparing for the testGetting started and taking the testAfter taking the test

Quality of the Test

Reliability and validity

Overall test qualityIndividual item quality

Alpha Coefficient

Recall the alpha coefficient from chapter 6

total

trials

S

S*

K

K2

2

11

KR20

Notice the Similarities

total

trials

S

S*

K

K2

2

11

KR20 =

alpha =

These two ARE the same.Use alpha with continuous data.Use KR20 with dichotomous data.

KR21

Notice the Similarities

total

trials

S

S*

K

K2

2

11

KR21 =

KR20 =

alpha =

Notice the Similarities

total

trials

S

S*

K

K2

2

11

KR20 =

alpha =

These two ARE the same.Use alpha with continuous data.Use KR20 with dichotomous data.

Notice the Similarities

KR21 =

KR20 =

KR21 is a conservative estimate ofthe reliability. KR20 will ALWAYS begreater than or equal to KR21 but KR21 is much easier to calculate.

Item Analysis

Used to determine quality of individual test items

Item difficultyPercent answering correctly

Item discriminationHow well the item "functions“Also how “valid” the item is based on the total test score criterion

One Way to Organize Data for Item Analysis

1) Score the tests2) Arrange answer sheets from high to low3) Separate answer sheet into three subgroups

a) upper 27%b) middle 46%c) lower 27%

4) Count and record responses per foil in upper group5) Count and record responses per foil in lower group6) See figure 10.3

Item Difficulty

100*nn

cc

LU

LUDifficulty

Item Discrimination

100*U

LUationminDiscri

n

cc

Item Difficulty

100*nn

cc

LU

LUDifficulty

Item Discrimination

100*U

LUationminDiscri

n

cc

Figure 8.3—Item #5

Responses A B C D* E Omit Diff. Net D

Upper 27% = 50 28 2 1 19 0

36% 4%Lower 27% = 50 24 8 1 17 0

Figure 8.3—Item #25

Responses A B C D* E Omit Diff. Net D

Upper 27% = 300 69 10 5 216 0

53% 37%Lower 27% = 300 89 52 54 104 1

Figure 8.4 The Relationship Between Item Discrimination and Difficulty

Maximum, Minimum, and Desired Item Difficulty and Item Discrimination

Difficulty Discrimination

Maximum 100 100

Minimum 0 -100

Desired 50 100

Sources of Written Tests

Professionally constructed tests

Textbooks

Periodicals, theses, and dissertations

McGee & Farrow

Item Analysis Practice27% = 50 people

Item # Upper Lower Difficulty Discrimination

1 32 21

2 18 12

3 40 21

4 50 40

5 36 42

6 25 25

7 15 35

8 35 15

Item Analysis Practice27% = 50 people

Item # Upper Lower Difficulty Discrimination

1 32 21 53% 22%

2 18 12 30% 12%

3 40 21 61% 38%

4 50 40 90% 20%

5 36 42 78% -12%

6 25 25 50% 0%

7 15 35 50% -40%

8 35 15 50% 40%

Each Item Was Completed By the Same Number of People

Complete the chart below by fillingin the empty boxes

Item Uppercorrect

Lowercorrect

Difficulty Discrimination

1 10 100%

2 5 50%

3 5 65%

Each Item Was Completed By the Same Number of People

Complete the chart below by fillingin the empty boxes

Item Uppercorrect

Lowercorrect

Difficulty Discrimination

1 10 10 100% 0%

2 10 5 75% 50%

3 5 8 65% -30%

Questionnaires

Determine the objectivesDelimit the sampleConstruct the questionnaireConduct a pilot studyWrite a cover letterSend the questionnaireFollow-up with non-respondentsAnalyze the results and prepare the report

Constructing Open-Ended Questions

AdvantagesAllow for creative answersAllow for respondent to detail answersCan be used when possible categories are largeProbably better when complex questions are involved

DisadvantagesAnalysis is difficult because of non-standard responsesRequire more respondent time to completeCan be ambiguousCan result in irrelevant data

Constructing Closed-Ended Questions

AdvantagesEasy to codeResult in standard responsesUsually less ambiguousEase of response

DisadvantagesFrustration if correct category is not presentRespondent may chose inappropriate categoryMay require many categories to get ALL responsesSubject to possible recording errors

Factors Affecting the Questionnaire Response

Cover letterBe brief and informative

Ease of returnYou DO want it back!

Neatness and lengthBe professional and brief

InducementsMoney and flattery

Timing and deadlinesTime of year and sufficient time to complete

Follow-upAt least once

The BIG Issues in Questionnaire Development

ReliabilityConsistency of measurement

ValidityTruthfulness of response

Representativeness of the sampleTo whom can you generalize?