Slide 1

Slide 2 Statistical Decision Making Analysts must often make decisions about some condition in the real world. Assume that you have finished your BA in Political Science or MA in policy studies and have been hired as the environmental affairs officer for the city of Morgantown. You must make the following assessment: Does the water supply in Morgantown comply with safe drinking water standards? Lets see! Slide 3 Hypothesis Testing We can decide the facility is In Compliance. or We can decide the facility is in Violation. (not in compliance) Slide 4 If we decide: The facility is in compliance and it is. or We decide the facility is in violation and it is. Then we are fine. We have made a correct decision But sometimes No, make that often were not correct. We make mistakes! Statistics gives us some rules to reduce the likelihood of making these mistakes. Slide 5 In order to test the facilities water we will collect several bottles of water at different times. We take this sample (Do not be confused by the chemistry. Each bottle of water is a sample to the chemist, while all of the separate bottles of water are the sample to the statistician.) And we calculate the average amount of the pollutant e.g. trihalomethanes. Example of a statistical test Slide 6 The average of the samples is thus Our hypothesis is thus ppb (the standard for trihalomethanes)trihalomethanes or 80 micrograms per Liter (15 drops in your average swimming pool) Because we will say that the standard actually refers to a population distribution with a mean equal to the standard, we can restate this is conventional statistical terms For info on EPA standardsinfo on EPA standards Slide 7 An aside on Standards Drinking water standard are set by EPA to establish the Maximum Legal Contaminant Level (MCL) The level is set to the point where the concentration is expected to produce an acceptable level of morbidity & mortality Acceptable is a social construct, not an absolute fact or a hard fact. It a belief based on research and societal norms. Acceptable dose is inferred from LC 50LC 50 What is the value of a human life What is it worth to you? Slide 8 Hypotheses The hypothesis is: The facility is in violation. Sometimes referred to as The alternate hypothesis. The null hypothesis is: The facility is in compliance. In this instance, you want to be able to reject the alternate hypothesis, or more properly, fail to reject the null hypothesis. Slide 9 Decision Slide 10 In Compliance Slide 11 Decision In ComplianceIn Violation Slide 12 Real World Decision In ComplianceIn Violation Slide 13 Real World Decision In ComplianceIn Violation In Compliance Slide 14 Real World Decision In ComplianceIn Violation In Compliance Slide 15 Real World Decision In ComplianceIn Violation In Compliance Slide 16 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Slide 17 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Slide 18 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Slide 19 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Incorrect Slide 20 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Incorrect Slide 21 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Incorrect Prob. = Incorrect Slide 22 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Prob. = 1- Correct Incorrect Prob. = Incorrect Slide 23 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Prob. = 1- Correct Incorrect Prob. = Incorrect Prob. = Slide 24 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Prob. = 1- Correct Incorrect Prob. = (Type I error) Incorrect Prob. = Slide 25 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Prob. = 1- Correct Incorrect Prob. = (Type I error) Incorrect Prob. = (Type II error) Slide 26 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Prob. = 1- Correct Prob. = 1- Incorrect Prob. = (Type I error) Incorrect Prob. = (Type II error) Slide 27 Real World Decision In Compliance Do not reject H 0 In Violation Reject H 0 In Violation In Compliance Correct Prob. = 1- Correct Prob. = 1- (Power of the test) Incorrect Prob. = (Type I error) Incorrect Prob. = (Type II error) Slide 28 Power of the Test (1- ) We wish to use statistical tests that are Powerful, since they will be more likely to detect statistically significant differences. We can improve the power of our tests by Increasing the sample size Choosing a statistical tests with greater power than another appropriate test Testing in instances where the relationship, or difference, is stronger which often means waiting until the effect has grown enough The concept of effect size is an important related topic. Slide 29 Effect Size Statistical tests with large data sets will often produce results that are statistically significant. There are tools for informing the researcher how meaningful the difference that is found is Most noticeably, effect size: Calculated several ways (Cohens D, Pearsons R), effect size is standardized with the following rule of thumb: < 0.1 = trivial effect 0.1 - 0.3 = small effect 0.3 - 0.5 = moderate effect > 0.5 = large difference effect See Zint & Montgomery MEERA: Power Analysis, Statistical Significance, & Effect SizePower Analysis, Statistical Significance, & Effect Size Slide 30 Margin of error for Sampling From Wikipedia: Margin of ErrorMargin of Error Slide 31 Alpha () Note that alpha () is: The probability of rejecting the null hypothesis when the null is in fact true. It is the probability of making a Type I error By convention, we usually set =.05 (1 time out of 20 by chance alone) A good working rule is to always use =.05 until you know when not to. Slide 32 Beta ( ) Beta ( ) is: The probability of failing to reject the null hypothesis when the alternate is in fact true. It is thus the probability of making a Type II error. We can never really know , because we never know the true situation. and are inversely related. Slide 33 Balancing and If we have complete information then we would wish to balance the costs of potential errors with their likelihood of occurring. The expected result of an event is the sum of the costs and benefits of that outcome times the probability of each outcome. To balance testing. (e.g. given only 2 possible outcomes) Costs of Type I error * = Costs of Type II error * In reality, we never have enough information to properly assess this. Slide 34 Environmental vs. Industrial Protection provides us a measure of environmental protection. provides us a measure of industrial protection. If you increase the possibility of making one type of error, you will decrease the likelihood of the other type of error.