Basic probability Sep. 16, 2013 Introduction Our formal study of probability will base on Set theory Axiomatic approach (base for all our further studies of probability) We will solve a class of probability problems via counting methods Determining the probability of obtaining a royal flush in poker Obtaining a defective item from a batch of mostly good items. Set Theory Set theory is the branch of mathematical logic that studies, sets, which are collections of objects. The language of set theory can be used in the definitions of nearly all mathematical objects. It is foundational system for mathematics. Set theory, was founded by a single paper in 1874 by Georg Cantor: On a Characteristic Property of All Real Algebraic Numbers Georg Cantor was born in Russian Empire in 1845 and died in Germany in 1918. Review of Set Theory The set A can be defied either by the enumeration method description method Each object is called an element and it is distinct. Set are equivalent. Sets are said to be equal if they contain the same elements. If and then Review of Set Theory An element of x of a set is denoted and is read x is contained in A. If then Empty set or null set If the instructor in the class does not give out any grades A then the set of students receiving an A is. Universal set If the instructor is an easy grader and give out all A, then, where S is the set of all students enrolled in the class. Review of Set Theory Example 1. Set concepts Consider the set of all outcomes of a tossed die. This is The numbers 1,2,3,4,5,6 are its elements, which are distinct. 1.The set of integers numbers from 1 to 6 or is equal to A. 2.The set A is also the universal set since it contains all the outcomes. 3.The set is called a subset of A. 4.A simple set is a set containing a single element, Review of Set Theory Element versus simple set Sometimes elements in a set can be added, as, for example = 5, but it makes no sense to add sets as in {2} + {3} = {5}; More formally, a set B is defined as a subset of a set A if every element in B is also an element of A. It is denoted as. We can say that if and. New sets may be derived from other sets. If, then is a subset of S. The complement of A, denoted by or by is the set of elements in S but not in A. This is. Two sets can be combined together or intersected. Review of Set Theory Two sets can be combined together to from a new set. If Then the union of A and B, denoted by is the set of elements that belongs to A or B or both A and B. Hence,. The union of multiple sets is denoted by The intersection of sets A and B, denoted by or by is defined as the set of elements that belong to both A and B. Hence, for the sets above. The intersection of multiple sets is denoted by The difference between sets, denoted by is the set of elements in A but not in B. Hence,. Venn diagrams A Venn diagram is useful for visualizing set operations. The darkly shaded regions are the sets described. The dashed portions are not included in the sets Venn diagrams Example: One may inquire whether the following is true Applying Venn diagram it is easy to see To formally prove, let and prove that a. b. a. Assume that then by def. but. Hence,. Since,. Hence, and since this is true for every we have that b. Try to prove yourself. Algebra of sets If two sets A and B have no elements in common, they are said to be disjoint i.e.. If the sets contain between them all the elements of S, then the sets are said to partition the universe. Mutually disjoint sets for all are said to partition the universe if Algebra of sets Example: the set of students enrolled in the probability class is defined as the universe is partitioned by Some useful algebraic rules are 1. Commutative properties 2. Associative properties 3. Distributive properties Algebra of sets In either case we can perform the conversion by the following set of rules: 1.Change the unions to intersections and intersection to unions 2.Complement each set 3.Complement the overall expression Unions to intersections Intersections to unions 4. De Morgans law. Algebra of Sets: Examples Example 1 Verify by using the example sets when Example 2 Demonstrate for three sets Size of sets Discrete sets The set {2,4,6} is a finite set. The set {2,4,6,} is an infinite set, but countably infinite. (we can pair up each element in the set with an element in the set of natural numbers N). Continuous sets The set is infinite. Examples finite set discrete countably infinite set discrete infinite set continuous Set definitions: Examples Example 1. Identify the set A = {1,3,5,7} B = {1,2,3} C = {0.5 < c 8.5} D = {0.0} E = {2,4,6,8,10,12,14} F = {-0.5 < f 12.0} Example 2. Suppose we consider the problem of rolling die, The universal set of a rolling die is S = {1,2,3,4,5,6}. The person wins if the number comes up odd A = {1,3,5}, another person wins when the number is 4 or less A = {1,2,3,4}. Both A and B are subsets of S. For any universal set with N elements, there are 2 N possible subsets of S. So, there are 2 N = 64 ways one can define winning with one die. Tabularly specified, countable and finite Tabularly specified, countable and infinite Rule specified, uncountable and infinite Tabularly specified, countable and finite Rule specified, uncountable and infinite Assigning and Determining Probabilities The concept of sets and operations on sets provide an ideal description for probabilistic model and the means for determining the probabilities associated with the model. S = {1,2,3,4,5,6} is a universal set for a fair die. S is termed sample space and its elements are the outcomes. We are interested in particular outcomes. Even number outcome E even = {2,4,6}. Simplest events with one elements E 1 = {1}, E 2 = {2} Other events are S certain event and 0={} impossible event. Disjoint sets {1, 2} and {3, 4} are said to be mutually exclusive, hence events are mutually exclusive. An event occurs if the outcome is an element of the defining set of that event A summary of equivalent terms Set theoryProbability theoryProbability symbol UniverseSample space (certain event)S ElementOutcome (sample point)s SubsetEventE Disjoint setsMutually exclusive events Null setImpossible event0 Simple setSimple eventE = {s} Assigning and Determining Probabilities What is the probability P[E even ] that tossed die will produce an even outcome? Intuitively, there are 3 chances out of 6 so P[E even ] = ; Note, that P is a probability function that assigns a number between 0 and 1 sets. Examples For coin toss there are two events head H or T, all events are E 1 = {H}, E 2 = {T}, E 3 = S and E 4 = 0; For a die toss all the events are E 0 = 0, E 1 = {1},,{6}, E 12 = {1,2},, E 56 = {5,6},, E = {1,2,3,4,5,6}. There are total 64 events. In general, if the sample space has N simple events, the total number of events is 2 N. Assigning and Determining Probabilities: Axiomatic approach Axiom 1 P[E] 0 for every event E Axiom 2 P[S] = 1, Axiom 3 P[E U F] = P[E] + P[F] for E and F mutually exclusive. Axiom 3 for all E i s mutually exclusive. Axiom 4 Not a trivial transition. Mathematical model of Experiment A real experiment is defined mathematically by three things: 1.Assignment of a sample space; 2.Definition of events of interest; 3.Making probability assignments to the events such that axioms are satisfied. Example: Observe the sum of the numbers showing up when two dice are thrown. Sample space consists of 6 2 = 36 points We are interested in A = {sum ==7}, B = {8 < sum 11}, and C = {10 < sum } E ij = {sum of outcomes (i,j) = i + j} Using axiom 3 Die toss example Determine the probability that the outcome of a fair die toss is even E even = {2,4,6}. Defining E i as the simple event {i} we note that And from Axiom 2 we must have But since each E i is a simple event and by definition the simple events are mutually exclusive, we have from axiom 3 that Assume that outcomes are equally likely P[E 1 ] = P[E 2 ] = = P[E 6 ] = p. Hence, P[E i ] = 1/6 for all i. We can determine P[E even ] since E even = E 2 U E 4 U E 6 by applying Axiom 3 again P[E even ] = P[E 2 U E 4 U E 6 ] = P[E 2 ] + P[E 4 ] + P[E 6 ] = 1/6 + 1/6 + 1/6 = 1/2 Assigning and Determining Probabilities In, general P need not to be the same (weighted die). Letting P[{s i }] be the probability of the ith simple event we have that To simplify the notation instead P[{1}] we will use P[1]. Countably infinite sample space example A person arrives at the theater late by s i minutes, where s i = i i = 1, 2, 3 If P[s i ] = (1/2) i, what is the probability that hi will be more than 1 minute late? The even is E = {2,3,4,}. Using we have Using the formula for the sum of a geometric profession we have that 24 Countably infinite sample A habitually tardy person arrives at the theater later by s i minutes, where s i = i, i = 1,2,3, If P[s i ] = (1/2) i what is the probability that he will be more than 1 minute late? The event is E = {2,3,4,}. Then we have Using formula for the sum of a geometric progression we have that Properties of the Probability function Based on four axioms we may derive useful properties Property 1. Probability of complement event P[E c ] = 1 - P[E] Proof: By definition E U E c = S, and sets E and E c are mutually exclusive. Hence From which the property follows. Property 2. Probability of impossible event is 0. Proof: Since we have but if P[E] = 0, does not mean E is impossible. (Axiom 2) (definition of complement set) (Axiom 3) Properties of the Probability function Property 3. All probabilities are between 0 and 1. Proof: But from Axiom 1 P [ E c ] 0 and therefore P [ E ] = 1- P [ E c ] 1. Combining this result with Axiom 1 proves Property (Axiom 3) (definition of complement set) (Axiom 2) Properties of the Probability function Property 4. Formula for P [ EF ] where E and F are not mutually exclusive. P [ EF ] is a shortened from. Proof: By definition of E F we have that Also, the events E F and F are by definition mutually exclusive. It follows But by definition and E F and EF are mutually exclusive. Thus Combining (*) and (**) produces property * (Axiom 3) ** (Axiom 3) Properties of the Probability function Example when Axiom 3 could have been mistakenly applied to sets that are not mutually exclusive. Suppose given a die and we want the probably of these events E = {1,2,3} F = {3} So that EF = {3} Using property 4, we have that 29 Properties of the Probability function Example Switches in parallel Each switch is potentially faulty, say P = that switch is closed, P = that both switches close simultaneously. What is the probably that circuit will operate correctly? Define events E 1 = {switch 1 closes} and E 2 = {switch 2 closes} and event that at least one switch closes is. 30 Property 4 switch 2 switch 1 Properties of the Probability function Property 5. Monotonicity of probability function The larger the set, the larger the probability of that set. Proof: If the by definition, where E and F E are mutually exclusive by definition. Hence, Note that since and we have that and also that. 31 (Axiom 3) (Axiom 1) Properties of the Probability function Example: Potentially faulty switches in series P = that switch close. What is the probability that circuit operate correctly? So, we need to find P[E 1, E 2 ]. From previous example P[E 1, E 2 ] = . Property 5 E1E1 E2E2 Properties of the Probability function Property 6. Probability of unions of more that two events Booles inequality Probability of continuous sample space Recall is applicable to discrete events. Simple events of the continuous space are not countable Fair wheel example A number x is obtained by spinning the pointer on a fair wheel of chance that is labeled from 0 to 100 points. The sample set is S = {s: 0 < s 100}. The probability of the pointer falling between x 2 x 1 should be (x 2 x 1 ) / 100 Axiom 1 is satisfied for all x 1 and x 2 : 0 (x 2 - x 1 )/100 Axiom 2 is satisfied when x 2 = 100 and x 1 = 0 If break wheels periphery into N segments Then and for any N. If (x n x n-1 ) 0 then P(A n ) P(x n ) is the probability of the pointer falling exactly on the point x n. P(x n ) = 0, because N Axiom 3 Practice problems 1. 2. Practice problems