Reliability and Validity

Thinking Questions

• What is an IV?• What is a DV?• What is repeated measures design?• What is matched pairs design?• What is independent groups design?• What is a hypothesis? Can you name the 3

types of hypothesis?

Learning Objectives:1) To develop an understanding of

reliability and validity in psychological studies.

Success Criteria:1) Complete page 21 of your booklet making notes on types of reliability.

2) Answer practice questions on page 22 of your booklet on validity.

BUT FIRST…

Past Exam Question• A researcher carried out an experiment to investigate how many

numbers could be held in short-term memory. The participants were 15 children and 15 adults. Participants were asked to repeat lists of random numbers, in the correct order, as soon as they were read out by the researcher. For example, when the researcher said, “3, 4, 2, 8” the participant immediately repeated “3, 4, 2, 8”. When the researcher then said,“7, 5, 9, 6, 4” the participant immediately repeated “7, 5, 9, 6, 4”. One number was added to the list each time until participants were unable to recall the list correctly. Each participant’s maximum digit span was recorded.

• (a) Write an appropriate non-directional hypothesis for this experiment.

AO3 = 2 marks

0 marks for a directional hypothesis.

1 mark if not operationalised, e.g. “Age affects memory.” “There will be a difference between the two conditions.”

2 marks for e.g. “There will be a difference in how many numbers are correctly recalled by children and adults.”

“Children and adults have different short-term memory spans.”

Or

“The capacity of short-term memory is different for adults and children.

Past Exam Question• A researcher investigated whether memory for words presented

with pictures was better than memory for words presented without pictures. The researcher used an independent groups design.

• In Condition 1, participants were given a limited time to learn a list of 20 words. They were then asked to recall the 20 words in any order.

• In Condition 2, participants were given the same time to learn the same 20 words, but this time each word was presented with a picture. For example, the word ‘apple’ was presented alongside a picture of an apple. They were then asked to recall the 20 words in any order.

1. State a non-directional hypothesis for this experiment. (2)2. Explain two reasons why it was more appropriate to use an

independent groups design than a repeated measures design. (4)

1. AO3 = 2 marks0 marks for a directional / correlational / null hypothesis.1 mark for an appropriate non directional hypothesis where either or both variables are not operationalised e.g. memory will be different in the two conditions and / or when the hypothesis is not written as a statement e.g. “ To see if ...” or “ Is there.....?” 2 marks for an appropriate non directional hypothesis where both variables are operationalised e.g. there will be a difference in the number of words correctly recalled when words are presented with pictures and without pictures.

2. AO3 = 4 marksReasons for using an independent groups design rather than repeated measures include:-• There are no order effects because participants only do the task once.• The same words can be used in both conditions so one set of words is no easier to

recall than the other set of words.• Demand characteristics are less likely because participants will be unaware of the

other condition.

Credit other appropriate reasons.Simply stating IGD is quicker / saves time = 0.In each case 1 mark for a very brief / slightly muddled potentially relevant reason that could explain the use of IGD.2nd mark for some elaboration of a reason that is relevant / appropriate to this study.

ReliabilityWe could just measure

something using a piece of string, but it wouldn’t be

very accurate

We need to be able to measure or observe

something time after time and produce the same or

similar results

Reliability

The best way to test the reliability of an experiment is to repeat it. If the same results are obtained, then the experiment is reliable.

Scientists need their results to be reliable in order for them to be considered valid. Results are reliable if they are consistent; this means a test gives the same results each time.

A questionnaire’s reliability can be tested using the test-retest method – if the same answers are given the second time, it is a reliable test. Open questions and unstructured interviews lack reliability because they are difficult to replicate.

Why do the darts in the picture represent reliability?

Reliability

• The equation for checking reliability is:

Total agreements >80%

Total observations

PAGE 21Booklet – answer the questions at the bottom of...

Using the textbook (page 106)

Practice Question

• Mary has failed her maths GCSE last year. She got the lowest mark in the school, despite being given a Maths award before the exam results were announced. The school were really confused about Mary’s score and allowed her to retake the exam. On her second attempt Mary scored 100%.– Do you think that the Maths GCSE paper was a

reliable measure of Mary’s Maths ability? Discuss your answer with your neighbour.

Validity

PAGE 22Booklet

Internal Validity

• The extent to which study is free of design faults, which may affect results.

• Internal validity also looks at whether the IV really caused the changes to the DV or whether the results were due to extraneous variables.

• To have high internal validity you must make sure that you are testing what you intended to test.

External Validity

• This means the extent to which generalisation can be made from the test environment to other situations. If the study lacks internal validity that external validity will also be low and you cannot generalise the results.

• Types of external validity include:– Ecological validity – generalising results to other places– Population validity – generalising results to other people– Historical validity – generalising results to different times

Generalisability

• All psychologists want to be able to generalise their results beyond the research setting and into real life.

• A study could even be in the natural environment, but you may not be able to generalise, for example if the study was conducted using American students...why?

• So, remember, even when the study is in the natural setting you can not always generalise!

Does a driving test measure your competence to drive on the road or is it a measure of your ability

to pass the driving test? Would you be able to pass it again in six months time? Would you do

better? Is it a reliable and valid test?

L

Task

• Complete the table at the bottom of page 22

Look at this image...

Homework

• Create flashcards reliability and validity.