inter item reliability with surveys
TRANSCRIPT
2
Inter-item Reliability
With inter-item reliability or consistency we are trying to determine the degree to which responses to the items follow consistent patterns.
3
For exampleLet’s imagine that two items are designed to assess how “teachable” someone is. Item 1 is a difficult item for someone to agree with. So on a scale of 1 – 5 with 1 being “Strongly Disagree” and 5 being “Strongly Agree” here is what the response pattern might look like:
4
For exampleLet’s imagine that two items are designed to assess how “teachable” someone is. Item 1 is a difficult item for someone to agree with. So on a scale of 1 – 5 with 1 being “Strongly Disagree” and 5 being “Strongly Agree” here is what the response pattern might look like:
Respondent
Item 1
Bob 1
Carla 2
Don 1
Elsa 3
Fiona 1
Gordon 2
Etc.
5
For exampleNow take a look at item 2. It is much easier to agree with:
Respondent
Item 1
Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Fiona 1 4
Gordon 2 5
Etc.
6
For example
Respondent
Item 1
Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Fiona 1 4
Gordon 2 5
Etc.
Notice the
consistency in the
item response patterns
7
For example
Respondent
Item 1
Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Fiona 1 4
Gordon 2 5
Etc.
Bob & Fiona are less likely to agree
with these items
targeting teachablen
ess.
8
For example
Respondent
Item 1
Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Fiona 1 4
Gordon 2 5
Etc.
But even though
they are both less likely to
agree they consistentl
y agree more with
item 2 than 1.
9
For example
Respondent
Item 1
Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Fiona 1 4
Gordon 2 5
Etc.
This shows a
consistent pattern across items.
10
For example
Respondent
Item 1
Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Bob 1 4
Gordon 2 5
Etc.
Notice that this pattern
holds across all
of the other
respondents. Item 2 is easier to
agree with than Item 1
11
For example
Respondent
Item 1
Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Bob 1 4
Gordon 2 5
Etc.
Notice also that Elsa is the most agreeable
across both items.
12
For example
Respondent
Item 1
Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Bob 1 4
Gordon 2 5
Etc.
And Bob & Fiona are consistently the least agreeable
across both items.
13
For example
Respondent
Item 1
Item 2
Bob 1 4
Carla 2 5
Don 1 4
Elsa 3 6
Bob 1 4
Gordon 2 5
Etc.
The item responses to this instrument, therefore,
demonstrate high inter-item reliability or consistency
across persons with different levels of the trait.