328 2.8, 2.9, 2.10 the statistical enquiry cycle 20

328 2.8, 2.9, 2.10 The statistical enquiry cycle

20What is an experiment?

An experiment involves manipulating variables in a controlled environment. Interventions (deliberate changes and choices) are made by the person running the experiment. Observations are recorded so that the effects of the change or choice can be assessed.

TIP

An experiment involves more than just collecting observations. The question ‘Do boys with larger feet tend to be taller?’ could be investigated by measuring foot lengths and heights. This does not involve making any deliberate changes to a situation, so it is not an experiment – rather, it is an observational study.

Example

Which of the following investigations are experiments?1 Taking soil samples to investigate when past seismic activity occurred.2 Bottle-feeding infant formula to kittens to see whether they have comparable growth to kittens

that are nursed by their mothers.3 Timing male and female students to find out which group can send text messages more quickly.

Statistical experiments20Mathematics and Statistics in the New Zealand Curriculum

Statistics and Probability: Statistical investigation and statistical literacy

Level 7• S7-1Carryoutinvestigationsofphenomena,usingthestatisticalenquirycycle: –conductingexperiments –evaluatingthechoiceofmeasuresforvariablesandthesamplinganddatacollectionmethodsused –usingrelevantcontextualknowledge,exploratorydataanalysis,andstatisticalinference• S7-2Makeinferencesfromsurveysandexperiments: –makinginformalpredictions,interpolationsandextrapolations –usingsamplestatisticstomakepointestimatesofpopulationparameters

Achievement Standard

Mathematics and Statistics 2.10 – Conduct an experiment to investigate a situation using statistical methods

9781442549586_C20_328-348.indd 328 9/22/12 1:10 PM

32920 Statistical experiments

20

Answers

1 Not an experiment. These are observations, and there are no deliberate changes involved.2 An experiment. The food source for the kittens is being deliberately selected. Comparing the

weight gains of the kittens may help us to compare the benefits of the two foods.3 Not an experiment. Gender is not something that an experimenter can change for any

individual. The comparison that could be made between males and females is not the result of an intervention.

Variables

Variable – a quantity or category that can be changed or take different values in an experimentVariables can be either:Categorical – has a particular description or belongs to a group of some kind, and cannot be ordered, e.g. gender, colourQuantitative or numerical – has a value that can be measured, e.g. weight or temperatureInput variables (also known as factors) – the variable manipulated by the experimenterTreatment – the specific combination of input variables applied to a particular subject or group. There can be several input variables, and each input variable could be set at different levels. Treatment describes the combination of factors, and sometimes a description of the level that factors are set atResponse variable – the variable that is measured to gain data from the experimentNuisance variable – other variables (besides our planned input variables) that may fluctuate, and therefore affect our results. Sometimes steps can be taken to control nuisance variables

Example

A student wishes to investigate whether taking your own pulse gives similar results to having someone else take it. Identify some of the variables that could be involved in such an experiment.

Answer

The variable that is deliberately altered is the choice of person taking the pulse; therefore this is the input variable and it is categorical. It has two levels: taken by self and taken by another person. The response variable is the pulse measurement (beats per minute). Possible nuisance variables could be the location where the pulse is taken (e.g. neck or wrist), and the nature of the relationship between the subject and the person taking the pulse. (The heart rate may be faster if the person is attracted to the pulse reader!) Other nuisance variables could include whether the subject had taken caffeine or exercised before a pulse reading. These nuisance variables can be controlled by ensuring that the pulse is read in the same place, the person taking the pulse is neutral, and the pulses are read under the same conditions, e.g. no caffeine consumed beforehand.

9781442549586_C20_328-348.indd 329 9/22/12 1:11 PM


20

1 For each of the following examples, state whether the investigation is an experiment. If it is, identify the intervention (deliberate choice or alteration the experimenter makes). If the investigation is not an experiment, explain why.

a A sports scientist is investigating the link, if any, between caffeine consumption and performance in a memory test. The data collected includes the number of words remembered during a one-minute recall period, and whether or not the subject is given strong coffee to drink before the test.

b A student who gains their restricted drivers licence decides to buy a car. He hypothesises that it will be cheaper to buy a car through a private sale than from a dealer. He gathers price data for similar cars from the newspaper, that allows him to compare these two sources.

c An environmental scientist wants to investigate the effectiveness of different filtering media to minimise run-off of pollutants into the local stormwater system. It is important that wastewater flows through any filter reasonably quickly, as otherwise the filter can become blocked. Two different media are chosen – a filter containing fine sand, and a filter containing topsoil.

d A student wonders whether sprinters tend to have longer toes than distance runners. He hypothesises that longer

toes would enable the runners to have greater rebound off each step. He measures the toe length of the school’s athletes, and records whether their preferred event is a sprint distance or longer distance.

e An optometrist tests the sight of a customer by asking him to read letters from a chart. She asks him to do this using three different sets of lenses.

2 A dehumidifier is used to remove excess moisture from a damp room in the ground floor of a house. The dehumidifier works by sucking in air across a refrigeration coil, and the water vapour condenses onto the coil and drips into a container. The dried air is then returned to the room. A heating engineer wants to investigate whether the dehumidifier is more effective at a low temperature than a higher temperature. Define two suitable variables to measure when investigating this relationship. State whether each variable is categorical or quantitative.

3 A groundsman wants to encourage the best possible grass growth in a sports field. He designs an experiment in which he will apply different amounts of fertiliser to the grass and observe the effect on grass growth. List some possible nuisance variables in this situation and explain how to minimise the impact of these.

EXERCISE 20.01

9781442549586_C20_328-348.indd 330 9/22/12 1:11 PM


20

4 A student wears fingerless gloves in the winter because he thinks that full-fingered gloves slow his text message typing speed. Suggest suitable input and response variables that could be used to investigate this situation. State whether each variable is categorical or quantitative.

5 A sports performance student wants to investigate how great a difference in long jump distance the choice of leading foot can have.

a What is the input variable in this situation? What levels does it have? Is it categorical or quantitative?

b What is the response variable in this situation? How could it be measured?

c Identify some possible nuisance variables in this situation. How could you manage the variation they might cause?

6 A student decides to settle the ‘Coke vs Pepsi’ debate once and for all by running a series of blind taste tests of the two drinks.

a There are a variety of response variables that could be used. Identify one that is:

i categorical ii numerical. b If each person is given multiple drinks

to taste, a nuisance variable could be introduced. Identify this nuisance variable, and suggest how it could be managed.

c What is needed to make certain that the investigation is an experiment?

9781442549586_C20_328-348.indd 331 9/22/12 1:11 PM


20

Types of experimentThere are many types of experiment, but in Year 12 we restrict ourselves to experiments that involve comparisons. Here are some questions we could investigate:• Does this treatment have an effect? Investigate using a before and after design.• Which treatment has the greater effect? Investigate using a two-group comparison. Each group is given one of the treatments. Subjects or

units are assigned to the groups using random selection.• Which option is preferred/most popular? Investigate using a one-group design. Every subject receives all options, e.g. taste-testing three

different recipes for similar foods. The order in which the options are presented is randomised. If the subjects are only given two options at a time to compare, the experiment is known as a paired comparison.

In any of these designs, the following three aspects must be considered:

9 Three core components of experiment design

Randomisation

Random selection is used to assign treatments to subjects/units, with the exception of the ‘before and after design’ (unless a control group is used). In the ‘two-group design’, groups are formed by random selection (e.g. by drawing names from a hat). In the ‘paired comparison’, the possible orders for the treatments are written down and assigned randomly to subjects.

Why? To avoid bias. Differences in the composition of the groups may cause differences in the response variable that could be confused with the treatment effect we are investigating. Random selection gives all units an equal chance of receiving each treatment.

Replication

Each possible treatment must be given more than once. This means either multiple trials of an experiment or multiple subjects/units treated.

Why? So we are able to observe the variability of results. It also helps control random variation in the results and provides results that are typical for each treatment.

Controls

Nuisance variables can affect the results of an experiment unless they are allowed for. Sometimes this is done using a ‘control’ group (a group that is not given a treatment but is as similar as possible to the treated group and is subject to the same conditions). Using the same technique each time to measure the response variable helps control unwanted variation and measurement errors. Keeping conditions as constant as possible may help to control other nuisance variables.

Why? To try and ensure that the changes we observe are due to our treatment and not due to other influences.

9781442549586_C20_328-348.indd 332 9/22/12 1:11 PM


20

Before and after designTo test whether a treatment has an effect we measure a variable (e.g. pulse), apply a treatment (e.g. caffeine or exercise) and measure the variable again. Because the measurements come from the same subjects, we can take the size of the change as our response variable (subtract the ‘before’ measurement from the ‘after’). Our analysis can then focus on the distribution of this change data. We are particularly interested in whether a large majority of the changes are on one side of zero. If the changes are split fairly evenly between positive and negative, the reason could be that the treatment has no effect and the variation in the results has random causes.

Example

A new cholesterol-lowering medication is tested on 30 participants with high cholesterol. Their cholesterol reading (in mg/dL) is recorded before they start the treatment and again after six weeks. These measurements are shown in the table below. The investigative question is: ‘Is this treatment effective in lowering cholesterol?’

Subject number Before After



1 239 219 11 220 220 21 200 192

2 252 204 12 214 182 22 203 181

3 261 218 13 245 232 23 217 199

4 228 201 14 261 215 24 275 265

5 241 199 15 203 205 25 269 243

6 238 214 16 218 212 26 232 230

7 271 240 17 222 194 27 210 196

8 257 213 18 246 203 28 245 204

9 209 193 19 235 214 29 281 251

10 256 231 20 219 220 30 277 262

Based on the data, can you infer that the medicine tends to lower cholesterol in a similar group of patients?

Answer

• The ‘before’ measurements are subtracted from the ‘after’ measurements to create a new data set: the size of the change.

• The calculations are shown in the spreadsheet ‘Cholesterol-lowering medicine.xls’. This is provided on the Theta Dimensions student CD and can also be downloaded from www.mathematics.co.nz.

• This data is plotted below both as a dot plot (to show the distribution) and as a box plot (to show the summary statistics)

DATA

9781442549586_C20_328-348.indd 333 9/22/12 1:11 PM


20

The summary statistics for the change in cholesterol level for each participant are:

Min LQ Median UQ Max Interquartile range (IQR)−48 −32 −23 −13.25 2 18.75

The change data is widely spread, and lacks dense clusters. All but three participants had negative change values, suggesting that the drug was effective in lowering cholesterol for most subjects of this study. The two participants that had an increase in their cholesterol reading had only a small increase (1 or 2) and this may be due to variability of measurement rather than a medication effect.We calculate an informal confidence interval for the median (see page 313).

median ± 1.5× IQRn

= −23 ± 1.5 × 18 7530.

= −23 ± 5.13That is, −28.13 < median < −17.87The informal confidence interval does not enclose 0, and we can infer that this treatment does tend to lower cholesterol.

Another analysis approach is to plot the before and after measurements as bivariate data. Onto this scatter plot, we superimpose the line y = x. If the treatment has caused a change in the measured variable, we expect most of the points to fall on one side of the line.An example of this approach is shown below:

Most points fall above the line: the treatment has caused most measurements to increase.

Most points fall below the line: the treatment has caused most measurements to decrease.

The points fall either side of the line. The treatment does not appear to have had a predictable effect.

TIP

Drawing the line y = x on a ‘before’ and ‘after’ measurement scatter plot is not the same as producing a trend line or a ‘line of best fit’. Instead, we are comparing the location of the points to the line that represents no change.

−48

−32

2

−13Change

−28.2 −17.8−23

9781442549586_C20_328-348.indd 334 9/22/12 1:12 PM


20

1 A researcher investigated whether taking a health supplement based on shellfish could help alleviate asthma symptoms. He measured the peak flow breath speed in L/min; before the trial and then eight weeks later. The measurements were taken in the morning in case the time of day affected the results. The results are displayed below and can also be found on the spreadsheet ‘Peak flow rates.xls’ on the Theta Dimensions student CD and available at www.mathematics.co.nz.




1 341 372 11 376 402 21 367 387

2 382 401 12 362 415 22 378 380

3 365 405 13 344 382 23 381 380

4 344 376 14 363 395 24 362 410

5 352 360 15 369 399 25 357 382

6 366 392 16 341 368 26 349 375

7 381 358 17 352 379 27 344 389

8 365 398 18 356 385 28 351 406

9 367 401 19 373 402 29 375 412

10 345 385 20 355 395 30 366 370

a Create the set of change measurements by subtracting the before data from the after data. Display the distribution of this change data using a suitable graph. Explain whether the treatment is effective in increasing the peak flow rate.

b Create a scatterplot of after vs before measurements. Superimpose the line y = x onto the graph. Explain how this display confirms your observations in part a.

2 A maths class, knowing that exercise affects pulse, decided to investigate the size of this effect and how much it varies from person to person. Their data (pulse rates from 30 class members taken before and after a 5-minute run) is provided on the spreadsheet ‘Exercise and pulse.xls’ on the Theta Dimensions student CD and available at www.mathematics.co.nz.

a Identify some other variables that may affect the results of this experiment. How could you control these?

b Create the set of change data and produce summary statistics. Comment on how great an effect the exercise seemed to have on the pulse rate.

EXERCISE 20.02

DATA

DATA

9781442549586_C20_328-348.indd 335 9/22/12 1:12 PM


20

c Create a graph displaying the distribution of the change data. Comment on at least two features of the distribution that can be seen from this graph.

d Calculate an informal confidence interval for the median of the change data. Explain what you can conclude from this confidence interval.

3 Eight dogs were brought to the SPCA. They had been badly treated and had suffered a great deal of hair loss. Vets measured the percentage hair coverage of the dogs’ bodies when they first arrived and again after two weeks of treatment with hair growth medication. This data is displayed below.

Dog number

% hair coverage

when admitted

% hair coverage

after 2 weeks’

treatmentDog

number

% hair coverage

when admitted

% hair coverage

after 2 weeks’

treatment

1 32 65 5 81 100

2 56 81 6 45 76

3 48 73 7 58 92

4 65 88 8 70 100

a Calculate and graph the change data. Comment on the size of the change. b Explain why this data is not sufficient to prove that the treatment for hair loss is effective. c Explain why a control group may not be desirable in this situation. d Explain whether this study is an experiment.

4 A nutritionist has developed a health supplement called ‘Even Keel’, which is designed to balance the body’s insulin level. She thinks that people who take this product will have fewer cravings for sugary snacks and drinks. She asks participants in her trial to keep a food journal for two weeks before taking the supplement, and for two weeks while taking it. She records the number of instances in a week where participants chose to eat (or drink) a sugary food option. This data is found in the spreadsheet ‘Sugary snack intake.xls’ on the Theta Dimensions student CD and available at www.mathematics.co.nz.

a Plot the before and after counts on a scatterplot and superimpose the line y = x. What impression does this give you of the product’s effectiveness?

b Create the set of change data and record its summary statistics. What conclusions can you draw about the effectiveness of the product?

c Construct a 95% informal confidence interval for the median of the change data. Explain what you can infer from this confidence interval.

d Comment on what nuisance variables may exist in this situation.

DATA

9781442549586_C20_328-348.indd 336 9/22/12 1:12 PM


20

5 Two weight loss products are to be compared for effectiveness. One is a complete liquid shake diet and the other is a series of packaged meals. Participants in the study had their weight (in kg) recorded at the outset of the trial and again after two months on their particular diet. Their before and after data are recorded in the spreadsheet ‘Weight-loss products.xls’ on the Theta Dimensions Student CD and available at www.mathematics.co.nz.

a The two treatments have been assigned at random. Sort the spreadsheet data by column B (weight before treatment) to check whether the heavier subjects are more likely to get a certain treatment option. Does the treatment assignment seem fair?

b Now sort the data by column C (treatment given). Cut and paste the liquid shake subjects’ data into new columns. Create scatter plots of after vs before data for each treatment and draw the line y = x onto them. What do these displays suggest about the effectiveness of the two options?

c Identify some possible nuisance variables that could affect the reliability of the study’s results.

d Generate change data for each plan and graph these two data sets so that their distributions can be compared. Make three comparison comments.

e Calculate informal 95% confidence intervals for the median of both sets of change data. Comment on whether the confidence intervals overlap and hence comment on whether you can claim one plan is more effective than the other.

f Identify which plan you believe to be the more effective weight loss method. Justify your choice with reference to your displays.

DATA

TIP

The experiment in this last question contains elements of a two-group design as well as a before and after design. If you use a control group in a before and after design, your experiment will need to be a composite of the two approaches.

SCENARIO

Scenarios for before and after experimentsAchievement standard 2.10 requires you to carry out an experiment yourself. Here are some scenarios that could be investigated with before and after style experiments.1 Athletes often warm up and stretch before vigorous

physical pursuits. Can a warm hand stretch further (and produce a wider handspan) than a cold one?

2 Is it possible that the choice of foot (left or right) that a long-jumper launches from could influence how far he or she jumps?

3 At dance parties, people have sometimes reported feeling that their heart was beating as fast as the music. Can fast music really have an effect like this?

9781442549586_C20_328-348.indd 337 9/22/12 1:12 PM


20

Two-group experiment designA two-group design is used when there are at least two treatments you would like to compare, but it may not be sensible to give both treatments to the same subjects. One of the treatments may be a control, i.e. no intervention given. For comparisons to be valid, it is extremely important that the groups be randomly chosen. For example, placing all boys in one group and all girls in the other group would not be a valid way to assign people to groups!

In a two-group design, we compare data between the groups using summary statistics, side-by-side box plots, dot plots, etc. We may choose to take measurements of only one response variable. We may also choose to take before and after measurements and compare the change data sets.

Example

Two dishwasher powders are advertised as giving a complete clean without any need for pre-rinsing. This claim was tested by giving two busy restaurants one brand of powder each to use for a month. The percentage of dishes that had to be rewashed after going through the dishwasher was recorded each day. The 30 daily observations from each restaurant (for Brand A and Brand B) are shown in the graphs below.

4 Exercising may make you breathe faster – does it also mean that you are unable to hold your breath for as long?

5 Does the colour of the text that words are written in (or the colour of the background they are printed on) affect the ease with which we read them?

Choose one of these scenarios. Write an investigative question and design an experiment to gather data that could help you answer it. (Tips on experimental write-ups are given at the end of this chapter.)

With your teacher’s permission, run this experiment, analyse your data and state your conclusions.

9781442549586_C20_328-348.indd 338 9/22/12 1:12 PM


20

1 Why is the response variable a percentage rather than a count?2 Calculate informal confidence intervals for the median rewash percentage for each brand. What

conclusion can we draw?3 Identify some potential nuisance variables.

Answer

1 Percentages are used because a count could be misleading if different numbers of dishes are being washed. For instance, a count of 3 is 3% of the plates if 100 are washed but 10% of the plates if only 30 are washed.

2 The formula for an informal confidence interval for the median is

(median of observations) ± 1 5. × IQRn

where IQR stands for interquartile range and n is the

number of observations.Brand A: n = 30, median = 2, interquartile range (IQR) = 1

The confidence interval for the median percentage of dishes that will need rewashing when using Brand A is given by:

2 1 5 130

2 1 5 130

1 73 2 27

− < < + ×

< <

. .

. .

median

median Brand B: n = 30, median = 5.5, interquartile range (IQR) = 2 The confidence interval for the median percentage of dishes that will need rewashing when

using Brand B is given by:

5 5 1 5 230

5 5 1 5 230

4 95 6 04

. . . .

. .

− < < + ×

< <

× median

median The screenshot from the Pearson app

‘Informal 95% confidence interval for medians’ shows the two confidence intervals (Brand A in green, and Brand B in brown) superimposed on the box plots.

Because the two confidence intervals do not overlap, we conclude that a higher percentage of dishes tend to need rewashing when Brand B is used. The informal confidence intervals imply that the difference could be as much as 6.04 – 1.73 = 4.31 or could be as small as 4.95 – 2.27 = 2.68.

3 Several nuisance variables could be different at each restaurant and therefore affect the results. These include: the types of dishwasher used at the restaurants, how full the dishwashers are when they are run, whether the same amount of powder is used each time, how thoroughly the dishes are checked after cleaning – it could be that dishes considered acceptably clean at one restaurant would be rejected as dirty at the other restaurant.

Brand A

Brand B

0 21.7 2.3

1.5 2.5

5

2 85.5

4.5 6.5

5 6

TIP

The graphing and analysis skills for comparing two groups have been covered earlier in this textbook. Revise Chapters 18 and 19 if you are unfamiliar with these skills.

9781442549586_C20_328-348.indd 339 9/22/12 1:12 PM


20

1 A publisher is interested in carrying out a study to provide evidence that e-learning provides better academic results than more traditional methods. A random group of 40 students at Mathville Girls High School are given laptops and an e-book, and then tested after the unit of work is finished. At the same time, a control group at Mathville Boys High School are taught the same unit of work using pen and paper, and the same test is administered.

a Suggest a suitable response variable that could be used to analyse this experiment.

b State at least two faults with the selection of the control group for this experiment, and suggest a more appropriate way for the control group to be assigned.

2 A petrol company is considering two options for speeding up its service.

The first option is to have the customer prepay for their petrol using an EFTPOS terminal at the pump. The second option is to have attendants monitor the pump so that the customer is able to immediately go inside. The company will allow up to 50 of its petrol stations to be used in an investigation into these two methods.

a Write an investigative question that clearly sets out the company’s aims for this study

b Identify the treatment factor and a suitable response variable for this study.

c Explain why before data needs to be collected from petrol stations in the study.

d Describe how 50 petrol stations could be selected and assigned to two groups.

3 A student decided to compare two different bread dough recipes to see which bread would rise the most. He did this by punching the dough with metal tubes to take 2 cm-high core samples. He recorded the heights of the dough in the tubes from each recipe. (Note that the responses are final heights and not change data, because all before heights are 2 cm.)

Recipe 1

Recipe 2

46

7.2

5.75

5.15

6.25

6.3

3.8

4.7

7.1

a Based on the appearance of the graph, what call would you make about the difference (or lack of difference) in the recipes’ dough-raising effects?

b Given that there were 20 samples from each recipe, calculate the confidence intervals for the median height for a dough core sample from each recipe. Use these to generate a confidence interval for the difference between the medians.

c What conclusions can you make based on the given information? What other observations from the actual experiment would help you interpret this data?

EXERCISE 20.03

Medical experiments often use a control group because of the placebo effect. At times, an improvement can be observed in a patient’s condition when they are taking an inert compound (placebo) that resembles the treatment. The effect might stem from the patient believing they are being treated.

9781442549586_C20_328-348.indd 340 9/22/12 1:13 PM


20

4 Thirty patients with brain tumours agree to trial an experimental medicine. The response recorded is the reduction (in mm3) in the size of the tumour after four weeks of treatment. Fifteen patients are given the experimental medicine and the other 15 are given a treatment that is already known to be beneficial. This data is given in the spreadsheet ‘Tumour reduction.xls’ provided on the Theta Dimensions student CD and available at www.mathematics.co.nz.

a This experiment does not involve an untreated control group. Why would this be impractical in this situation?

b Comment on how the placebo effect might cause problems for our analysis.

c Is there enough evidence to choose which medicine is the more effective option? Justify your answer with statistics and/or graphs.

5 A group of Year 12 students were delighted when their teacher announced they were carrying out a chocolate experiment! The investigative question was: ‘Does cold chocolate take longer to melt in the mouth than room-temperature chocolate?’ Half of the class were given a square of chocolate from a block kept at room temperature, and the other half were given a square from a block that had been refrigerated. The students timed how long it took to eat the chocolate, given that they were only allowed to suck it and not chew it. The data is given in the spreadsheet ‘Dissolving chocolate.xls’ provided on the Theta Dimensions student CD and available at www.mathematics.co.nz.

a Identify a potential nuisance variable in this situation.

b Sort the data by treatment and create separate columns for the responses from the different temperature groups. Calculate summary statistics for each group.

c Create a box plot to show the summary data for the two groups. Indicate the confidence interval for each median on each plot. Is there evidence to suggest there is a significant difference?

d Produce a histogram and then comment on the distribution of the two data sets.

e Write a conclusion to address the investigative question.

DATA

DATA

SCENARIO

Scenarios for two-group comparison experimentsAchievement standard 2.10 requires you to carry out an experiment yourself. Here are some scenarios that could be investigated with two-group comparison experiments:1 How much effect does sight have

on people’s perceptions of flavour? Suggestions: blind taste testing or use of food colouring.

2 Does the font used on a Snellen eye chart alter its readability?

3 Do tennis balls bounce higher when they have been frozen?

4 What is the best way to estimate a period of 30 seconds? (e.g. ‘one Mississippi, two Mississippi …’ or ‘one one thousand, two one thousand’ or tap your finger, etc.)

5 Which is the faster way to cook frozen peas: putting them in cold water and bringing them to the boil, or boiling the water, putting the peas in, then bringing the water back to the boil?

6 Do fresh eggs or older eggs produce more foam when they are beaten?

Design and run an experiment based on one of these suggestions. Ask for teacher approval first.

9781442549586_C20_328-348.indd 341 9/22/12 1:13 PM


20

One-group experiment designWe may wish to make the kinds of comparison that require every subject to have tried every treatment option. A common reason is to gauge preference, e.g. ‘Which of these four chocolate chip biscuit brands tastes the best?’ In this type of experiment, we ask the subjects to make comparisons (giving us qualitative data), or we use measures that will enable us to rank responses. This kind of experiment is usually designed to investigate all possible arrangements of subjects and treatment options in a group, but does not involve comparison of groups; rather, it involves comparing treatments while the group itself is held constant. Hence it is sometimes referred to as a within group experiment.

Giving subjects a series of treatments can present three problems:1 Fatigue. There may be a limit to how many treatments the subject can try before their senses

become less acute, they lose concentration, etc.2 Practice effect. If the treatment involves skill, the subject may improve at the skill, and this

improvement may be falsely credited to the treatments given later in the test.3 Carryover effects. A treatment may affect the response to the next treatment. For example, if

several batches of biscuit are compared, the remaining taste in the mouth from the most recent biscuit may change the way the taste of the next biscuit is perceived.

There are several ways to mitigate these effects:1 Present only two options at a time to prevent fatigue and make the comparison decision easier.

This is known as a paired comparison.2 Randomise the order in which the treatments are given. If enough subjects are selected, any

unwanted order effects should be cancelled out by the fact that the treatments are occurring in different orders.

3 Do not perform treatments immediately following each other. Instead, let time pass, or give a drink of water so that the carryover effects are minimised.

Analysis options

The data collected is likely to be ordinal (e.g. 1, 2, 3 for first, second, third preference) or it could be a measure, e.g. proportion of goals scored from each shooting position trialled.

For each of the treatments trialled, statistics could be calculated (e.g. mean of the rankings; percentage of the time that an option has been ranked as first choice). Bar graphs and pie charts may both be useful displays. If a quantitative measure (count or measurement rather than a ranking) has been recorded, other types of graphs may be considered also.

TIP

Although we receive several pieces of data from individuals, it is the group results that are of interest – these are compiled from individual preferences.

9781442549586_C20_328-348.indd 342 9/22/12 1:13 PM


20

Example

A panel of taste testers are given three brands of potato chip to try. All three are the same flavour. The brands are ‘Budget’ (B), ‘Classic’ (C) and ‘Dinner Mate’ (D). The taste testers have no information about the brands, and are simply asked to choose the best-tasting chip from two presented options. Results from two of the panellists are shown below:

Panellist number

Options presented

Preferred option of pair

1 B, C B

1 C, D D

1 D, B B

2 C, D D

2 B, D D

2 C, B C

This translates to the following rankings:

Panellist 1st (best) 2nd 3rd (worst)

1 B D C

2 D C B

Results from the whole panel were as follows:

Panellist 1st (best) 2nd 3rd (worst)

1 B D C

2 D C B

3 B D C

4 B C D

5 D B C

6 C B D

7 B D C

8 B C D

9 D C B

10 B D C

Mean of rankings (where lower numbers = best): B = 1.6, D = 2, C = 2.4

The image above shows how the data counts should be entered into a spreadsheet to generate a bar graph chart using the ‘Insert chart’ function in Excel 2010. Conclusion: Most panelists preferred the Budget brand, and the Classic brand was least favoured.

TIP

While paired comparison is considered a more reliable ranking method, at this level you may choose to take the simpler option (present all three at once and ask for them to be ranked).

9781442549586_C20_328-348.indd 343 9/22/12 2:28 PM


20

1 In their PE lessons, a group of primary school children were taught to catch a baseball using a special mitt. Twenty children were tested to see whether using the mitt could generally make children more successful at catching a baseball. The children were given ten throws and asked to catch five with the mitt and five without. The treatment order (e.g. with, with, without, with, without, without) was chosen using random selection.

Here are the results:

Child number

Catches with mitt

Catches without mitt

1 4 3

2 3 2

3 2 2

4 0 1

5 4 2

6 5 2

7 2 1

8 0 2

9 2 3

10 5 3

11 5 4

12 3 1

13 4 1

14 5 5

15 2 3

16 4 1

17 3 2

18 5 2

19 4 3

20 3 0

EXERCISE 20.04

a Calculate appropriate statistics that will allow you to compare the two sets of data. Were the mitt attempts generally more successful than the barehanded attempts?

b Calculate difference data for the two columns. What proportion of the children was more successful with the special mitt than without it?

c Explain why it is important to have multiple attempts with each catching method.

d Explain why it is important to randomise the order of attempt methods (with or without special mitts).

2 A couple went on a wine-tasting tour. At one winery, they were given three different red wines to compare by the glass: a cabernet, a merlot and a shiraz. Because the wines were cheaper by the case than by the bottle, they decided to ask the rest of the tour party what their preferences were, in case any of them wanted to share in the purchase of a case. The resulting data is found in the spreadsheet ‘Wine choices.xls’ provided on the Theta Dimensions student CD and available at www.mathematics.co.nz

a The data shows ranked preferences, where 1 = best and 3 = worst. Create a bar graph to show the frequency for the rankings received by each wine variety.

b Calculate the mean ranking for each wine. Use this to rank the overall group preferences.

c If people would generally consider buying their first or second preference wines, how many people might be interested in sharing in a Shiraz case purchase?

DATA

9781442549586_C20_328-348.indd 344 9/22/12 1:13 PM


20

3 A group of PE students wanted to investigate which position on the hockey shooting circle gives the best chance of success for scoring a goal. They theorised that a direct frontal attack (position C) would be the easiest shot to aim, but might also be easier for the goalie to defend. They had each player in the hockey team (except for the goalie, who tried to deflect the shots) take 12 shots from each position, and recorded how many shots from each position were successful. This data (number of successful shots out of 12) can be found in the spreadsheet ‘Hockey shots.xls’ provided on the Theta Dimensions student CD and available at www.mathematics.co.nz.

a Identify some potential nuisance variables in this situation. Explain how each one might be controlled.

b Calculate the mean, median, quartiles and standard deviations for the five shooting positions.

c Create side-by-side box plots for the five positions. Which seems to be the most successful?

d Explain what nuisance variable is neutralised by taking this approach, rather than using five different groups of students for the five positions.

CD

EA

BDATA

SCENARIO

Scenarios for one-group comparison experimentsAchievement standard 2.10 requires you to carry out an experiment yourself.Here are some scenarios that could be investigated with one (within) group comparison style experiments:1 Choose several varieties of the same product to rate (e.g. by taste or ease of use).2 Ask members of the school basketball team to shoot from several different positions to estimate

which are the easiest shots to make. (Don’t forget to randomise the order in which these positions are presented and use each position more than once per subject.)

3 Investigate the fastest way to get across a classroom: hop, crab-crawl or walking on one’s knees.

Design and run an experiment based on one of these suggestions. Ask for teacher approval first.

Writing up an experimentBefore starting to design a statistical experiment, you will be given a scenario where gathering data could help address a problem. You will first need to state what you are trying to find out in terms of an investigative question. You may also like to state a hypothesis, identifying what outcomes you expect (and why).

9781442549586_C20_328-348.indd 345 9/22/12 1:13 PM


20

Example

It is widely believed that music can help people concentrate and perform well in academic pursuits. You have been asked to investigate this phenomenon.Some possible investigative questions for this scenario:Do classical music and rock music have different effects on students’ ability to concentrate?Do students work better when listening to classical music than they do in silence?

9 Defining the variablesBefore collecting and analysing data, a researcher must have some idea of exactly what measurements or data values are required.

Variables should be carefully defined and should relate to the hypothesis or situation being investigated. Explain what you are manipulating (your input variable) and what you are measuring (your response variable).

If you are investigating something that is relatively abstract, your chosen response variable may be an indirect measure. For example, if you wished to investigate whether a treatment puts a person in ‘better health’, you would need to choose a tangible variable to represent this, e.g. lower cholesterol, change in blood pressure, weight loss, etc.

Example

In the music example above, the input variable is likely to be categorical (music vs no music, classical vs rock). ‘Concentration’ and ‘work better’ need some tangible measure to represent them, e.g. number of times table questions completed in a fixed time, number of words copied in a given time, score on a memory test, etc.

9 Selecting the group of individualsIndividuals should be chosen at random, if possible, because this reduces variability. A well-designed experiment should keep as many factors constant as possible to focus on how changes to the explanatory variable affect the response variable.

Working with larger group sizes in a sample also helps reduce variability, but there is a trade-off between ideal design and the cost/inconvenience involved if too many units or individuals are sampled.

Many experiments feature a control group – a group that is kept constant or untreated – so that the treatment effect can be assumed to be represented by the differences between the two groups. The control group should have similar characteristics to the treated group

When using sampling to obtain your data, describe the sampling method in sufficient detail so that someone else could repeat your method. Justify your choice of sampling method, with references to the population. Consider factors such as avoiding bias, ensuring convenience, minimising cost, etc.

TIP

Because of practical constraints, you may end up with a non-representative sample of subjects, e.g. the students in your class. Make comments in your critique of the experiment about whether the results would be likely to differ with a larger or more diverse sample, and explain why.

9781442549586_C20_328-348.indd 346 9/22/12 1:13 PM


20

9 ProcessThe experiment you are carrying out should be described in enough detail that someone else could carry it out. Describe what takes place in a trial and how many trials/subjects you are using. If you are using equipment, a diagram may be useful. Identify what timeframes are involved.

Example

Each student will be given a page of 200 times-table problems and asked to complete as many as they can in two minutes. They will be asked to do this task twice: once with music playing in the background and once without. The order of treatments (background music or silence) will be randomised for each student participating. The two trials per student will be carried out on different days to avoid tiredness or improvement due to practice effects. I intend to carry out this process with ten different students.

9 Recording dataWhen recording the data, remember that it is vital that you be able to show the data to others if it is questioned afterwards, and to share the data with other researchers. The experiment should be able to be replicated, and if your summary calculations or conclusions are disputed, the original data, with working, can be provided to support your analysis.

Obvious but important steps are to record the data correctly and consistently; and to decide how to handle measurements that are inconsistent with others – are these measurements genuine outliers, or have they been reported using the wrong units, or do they result from some kind of misunderstanding? Note visual observations as well as data records, e.g. ‘during trial 19 the wind was gusty, and this may have affected the temperature reading’. Measurement and rounding errors should be avoided.

9 Sources of variationSources of variation can include random variation – whenever a statistical experiment is carried out, there is some element of chance. Remember that, generally, you are working with a sample – not with all the population under consideration. Use a different sample, and the results of the experiment will also be different.

Other sources of variation can be controlled to some extent by carrying out the experiment under strictly similar conditions. Comment on what steps you are taking to control nuisance variables.

9 Analysing the data and presenting a summary

Displays

Make sure any graph or displays are appropriate. Obviously, they should relate to the variables you have defined, and should be suitably sized with a choice of scale that best represents the results of the experiment. Using displays that allow you to discuss distribution as well as central tendency is desirable.

9781442549586_C20_328-348.indd 347 9/22/12 1:13 PM


20

Data summaries

In some cases, it may be appropriate to summarise the data. Calculate medians, means, interquartile range and standard deviation, etc. so that you can describe central tendency and spread accurately. Standard deviation also gives a measure of the variability of your sample data. A confidence interval will indicate the likely range of values back in the population you are sampling from.

The conclusion

In your conclusion, address the underlying problem. If possible, set your conclusions in context to show you have a general understanding of the comparison you are making or the relationship between two groups.• Is it possible to check the conclusion independently?• Was the sample size reported, and is it appropriate and large enough to allow for some random

variation?• Are there other factors that may have influenced the result?• A critical review of the experiment• You should mention factors that may not have been taken into account in your experiment.

Consider these key questions:• What wider population (if any) could your results be applied to?• If some of the population was excluded, did this bias your results?• Is it possible that some items were sampled more than once?• Was your hypothesis correct? If not, can you explain why the results differed from your

expectations?• Comment on how constant and controlled the environment in which you carried out your

experiment was. Were there any factors that could interfere with your results? How would you deal with them if you ran the experiment again?

• How variable were your results for each treatment? If there was high variability, what do you think caused this?

For a fully worked example, see the pdf ‘Conducting a statistical experiment ‘available in the Theta Mathematics Teaching Resource.

P20.20

9781442549586_C20_328-348.indd 348 9/22/12 1:13 PM

534 Answers

20

Note that the analysis focuses on the box parts of the boxplots. This discounts the effect of outliers in the data. The sampling distributions for walking and other modes of transport show some shift or displacement. It appears that,

from the group of students who have short sleeps, the ones who walk get about 30 minutes more sleep than those who travel to school by other modes of transport.

Informal confidence intervals for the median sleeping times are as follows:8.1 hours < median sleeping time for walking < 8.6 hours7.8 hours < median sleeping time for other modes of transport < 8.2 hours.There is a small overlap between the two informal confidence intervals. This means that we cannot make the claim that

Year 12 students who walk to school tend to have a longer sleeping time than Year 12 students who travel to school by other modes of transport.

There could be some non-sampling variation caused by the day on which the students answered the survey – for example, a Sunday night/Monday morning may not be typical of the other nights. However, this effect should be independent of whether students walk to school or not. Another factor could be that if a student sleeps in, a family member has to deliver them to school by car. One reason for posing the question could have been that students who walk to school tend to live nearby, meaning that they may be allowed more time to sleep in. However, the results are inconclusive, so we cannot accept the claim based on this sample data.

Sampling variability was kept to a minimum by using all the relevant data units in the data set.


1 a This is an experiment. The intervention is the choice of whether or not the subject is given coffee.

b Not an experiment. The student is collecting observations but not altering factor levels.

c This is an experiment. The intervention is the choice of filtration media.

d Not an experiment. Observations are collected without any deliberate changes to conditions.

(page 330)EXERCISE 20.01

(Source of data: www.CensusAtSchool.org.nz)

9781442549586_ANS_512-544.indd 534 9/22/12 1:30 PM

535Answers

20 1 a

Although there are two measurements below zero, the lower quartile is 25. The overwhelming majority of measurements have increased peak flow, suggesting that the treatment is effective.

b

All but two points are above the line y = x, confirming the observation in part a that the treatment has had the desired effect in most subjects.

e This is an experiment. The intervention is the choice of lens.

2 One variable (explanatory) is the temperature in the room. This is a quantitative variable. The experimenter would need to select which temperatures to test. The response variable is the volume of water collected over a set time period. This also is a quantitative variable.

3 Sources of variation include: the time of year (more growth in spring, less growth in summer and winter); the amount of rainfall; the soil quality, including whether it is compact or loose; the soil type (clay, loam or sand); the composition of the fertiliser; the amount of sunshine; etc.

The sports field should be broken up into a grid of different square plots. There should be sufficient distance between the plots so that there is no cross-contamination where the effect is muddled from nearby plots. All plots should have a similar soil/grass type; they should all receive equal amounts of sunshine (in other words, not near trees, grandstands, banks, etc.), and all should be given the same amount of water.

4 Input variable could be the type of glove used (fingerless, full-fingered, etc.; possibly also ‘none’). This is a categorical variable. Response variable could be the time (in seconds) taken to type a predetermined text message. This is a quantitative variable.

5 a Choice of foot. This could be left vs right, or ‘usual choice’ vs ‘alternate choice’, and is a categorical variable.

b Length of jump. Could be measured in the usual way – from the launch board to the backmost mark in the sand.

c Left-foot or right-foot preference may be more common among the jumpers. Choosing ‘preferred foot/alternative choice foot’ instead of ‘left foot/right foot’ as the input variable may be a better approach. Different conditions (time of day, temperature, moisture, wind, light) may affect jumps. Having the jumpers trial in the same conditions will control this. There may be a practice effect or a fatigue effect. Randomising the order for the choice of launch foot will help negate any practice effect.

6 a i Preferred drink identification: ‘Coke’ or ‘Pepsi’. ii Proportion of people in each trial group that chose

a particular drink as first preference. b Taste perception could be altered by trying two drinks

in a row. This could be controlled by taking a drink of water in between.

c The experimenter should make deliberate choices as to who is given which drink and when, and how the drinks are presented (in plain containers) and have a response measure (such as a rating) in place. It is not an experiment if people are just asked, ‘Which of these two colas do you prefer?’


9781442549586_ANS_512-544.indd 535 9/22/12 1:30 PM

536 Answers

2 a The experiment could be affected by how fast the students ran, whether any had consumed caffeine, sugar, etc. beforehand, and how soon after the walk the second pulse was taken. These particular variables could be controlled by having the class run together at the same speed, not allowing caffeinated drinks, and timing the second pulse at a set time after the exercise.

b Minimum LQ Median UQ Maximum Mean Standard deviation

23 41 50 56 71 47.87 (2 dp) 12.64 (2 dp)

All students had an increase in pulse rate, and this generally seemed to be between 30 and 60 beats per minute. c

6

Change data – pulse rates

Change in pulse rate

5

4

3

2

20 25 30 35 40 45

Mean Median

50 55 60 65 70 75

1

Some features are: left-skew, gaps at 30–35 and 65–70, only two values over 70, and a mode group (cluster) at 50–55.

d The confidence interval for the median is 50 15± ×1 530

. i.e. 45.89 < median < 54.11.

A five-minute run tends to increase the median pulse by between about 46–54 beats per minute for students of a similar age and fitness level.

3 a

Median and mean increase in percentage hair coverage is 27.5. All but one dog has had an increase of more than 20 in its percentage hair coverage.

b The dogs’ hair may have grown back without this treatment and becasue the abuse had stopped. We cannot be sure that the treatment is effective unless it is compared with a control or another treatment, for example.

c A control group is not desirable because it is unethical not to treat a sick animal. d While this resembles an experiment (a deliberate intervention with observations taken to measure a change), it does not

follow the general purpose of an experiment, which is to investigate a question. 4 a

20

There are numerous points on both sides of the line. If the product has an effect it is a subtle or inconsistent one.

9781442549586_ANS_512-544.indd 536 9/22/12 1:30 PM

537Answers

b Minimum LQ Median UQ Maximum Mean Standard deviation−5 −2.75 −1 1 6 −0.78 2.67 (2 dp)

The change data is close to being symmetric about a zero effect, and is widely spread, indicating that there is considerable variation in the effect, if any.

c The 95% informal confidence interval for the median change in this case is −1.795 to −0.205, which would suggest that the supplement does, on average, cause a small reduction in the number of sugary snacks consumed. However, since the count of snacks is in whole numbers, if we round the limits to integers we get −2 to 0. The evidence for the supplement being effective is not strong.

d Nuisance variables may include the particular circumstances of the participants; if the weeks chosen were atypical in some way (e.g. contained a birthday, or morning tea shouts at work); and whether the participants recorded their food intake accurately.

5 a It does seem fair. The heaviest half of the participants have equal numbers of the two treatment options, and so does the lightest half.

b

These displays suggest that both treatments are effective, but perhaps the packaged meals are more consistently so, with no participants gaining weight and some larger losses.

c Whether the participants stuck to the diets or not; differing amounts of exercise.

d

The shakes change data has a much wider range of values, including some positive ones (i.e. weight gain). The middle 50% is less spread out than that of the packaged meals changes and the box is shifted to the right. The median of the packaged meals change data is not enclosed by the box of the shakes change data, suggesting that perhaps the packaged meals users tend to lose more weight. Both distributions are left-skewed.

e Liquid shakes: −12.46 < median < −7.54 Packaged meals: −19.38 < median < −10.62 The two confidence intervals overlap so we cannot claim than one plan tends to be more effective at weight loss than

the other. f The packaged meals seems to be the more effective option because no participants gained weight on this plan. There

is less overall variability in results. In the scatterplot, there seem to be a larger number of points well below the line y = x. However, the box plots for the two plans contain overlap in the boxes and the median confidence intervals overlap. Therefore it is a little too close to call based on the evidence available.

20

9781442549586_ANS_512-544.indd 537 9/22/12 1:30 PM

538 Answers


1 a The number of correct questions on the end of topic test. b The control group does not come from the same school and has a different gender. The students will have had different

teachers as well as different learning media. The control group should have a mixture of boys and girls chosen at random from the two schools. Alternatively, both test and control group should come from the same school. Ideally, the students should have the same teachers, too.

2 a ‘Will customers who prepay for their petrol and pump it themselves spend a shorter time at the station than those who have their petrol pumped by an attendant?’ Other questions are possible.

b The treatment factor is the service option (prepay and self-pump, or attendant pump). The response factor could be the time in minutes spent at the station (e.g. measured from the time the person exits their car to the time they drive off).

c The company wants to speed up its customer service. If the two options being studied are not quicker than what is already in place, then there is no need to adopt either of them.

d Select stations from the list of stations in the company’s chain at random and then randomly assign them to two groups. They may also like to consider stratified sampling by island, e.g. three-quarters North Island to one-quarter South Island, or a mixture of city and country stations.

3 a While there is overlap in the boxes, the lower quartile for recipe 1 is above the median for recipe 2. It seems likely that bread made using recipe 1 rises more.

b Recipe 1 median confidence interval is 5.82–6.18. Recipe 2 median confidence interval is 4.63–5.67. Recipe 1 median could be higher than recipe 2 median by 0.15–1.55 cm.

c Because the confidence interval for difference of medians does not contain zero, we can claim that bread made using Recipe 1 tends to rise more. We might also wish to know about the appearance/texture of the dough, how long they were kneaded for, where the samples were placed to rise (whether some were closer to a heat source than others), among other considerations.

4 a Because it would involve not treating a person with a tumour, which would be unethical and potentially life threatening.

b If there is evidence of some effect from the new medication, we do not know the extent to which this is a treatment effect or whether some of the effect can be explained by the placebo effect. We will have to compare the results with the medicine known to be effective to see whether the medicine produces an effect of an acceptable magnitude.

c

The median confidence intervals for the new and known treatments overlap. This means that the treatments could produce the same median change in tumour size. We do not have enough evidence to claim that one tends to be more effective than the other – the observed differences could be attributed to random variation.

5 a Nuisance variables could include how hard the students sucked, whether the chocolate broke evenly, and whether the pieces were the exact same size.

b Minimum LQ Median UQ Maximum Mean Standard deviation

Room temperature 89 95 102.5 112.5 128 104.7 (1 dp) 12.3 (1 dp)

Refrigerated 96 147 161.5 192 218 166.4 (1 dp) 33.2 (1 dp)

20

9781442549586_ANS_512-544.indd 538 9/22/12 1:30 PM

539Answers

c

Room temperature confidence interval: 95.3 seconds < median time < 109.7 seconds Refrigerated confidence interval: 143.5 seconds < median time < 179.5 seconds Because the confidence intervals do not overlap we can claim that the colder chocolate takes longer to melt in the mouth.

d

Room temperature chocolate times are more densely clustered than the refrigerated chocolate times, and there is a clear shift with the room temperature times on the left.

e Chocolate that has been refrigerated takes longer to melt in the mouth than chocolate that has been kept at room temperature.

1 a LQ Median UQ Mean

With mitt 2 3.5 4.25 3.25

Without mitt 1 2 3 2.15

The mitt attempts have higher values for all statistics, and the lower quartile with the mitt is equal to the median catches without the mitt. If a box plot were drawn there would be some overlap between the boxes. The differences in the statistics are generally small (less than 2) and might be the result of random variation in the catching.

b 14 out of the 20 children were more successful with the mitt than without it (70%). c It is important to have multiple attempts with each method to get a good estimate of how well the children are

catching. With one attempt, only 100% and 0% catch rates are possible. d Randomising the order helps eliminate the ‘practice effect’ between the two treatments. If the catches with the mitt

were the last five catches, the child may be better at catching because they are beginning to gauge the ball speed better by then, and not because the mitt is more helpful.


20

9781442549586_ANS_512-544.indd 539 9/22/12 1:30 PM

540 Answers

2 a

b Mean rankings are Merlot: 1.65, Cabernet: 2.1, Shiraz: 2.25. Merlot is the favourite choice and Shiraz the least favourite. c 20 people 3 a The goalie might be improving because he or she is deflecting the same shot many times; or getting fatigued for the

same reason; or the light may be different (e.g. sun in eyes for certain shots). These nuisance variables can be controlled by randomising order, and choosing an overcast day or a time when the sun will not be a problem.

b Position LQ Median UQ Mean Standard deviation

A 2.25 3 4 3.4 1.65

B 3.25 4.5 7 5.1 1.97

C 1.25 2 2.75 2.2 1.23

D 2.25 3 4 3.4 1.35

E 1.25 2 2.75 2.1 0.99

c

Position B seems the most successful, although it has some overlap with A and D. d Differences in ability amongst the players.

20

9781442549586_ANS_512-544.indd 540 9/22/12 1:30 PM

328 2.8, 2.9, 2.10 the statistical enquiry cycle 20

Documents