STA347 - week 1 1

What is Probability?

• Quantification of uncertainty.

• Mathematical model for things that occur randomly.

• Random – not haphazard, don’t know what will happen on any one experiment, but has a long run order.

• The concept of probability is necessary in work with physical biological or social mechanism that generate observation that can not be predicted with certainty. Example…

STA347 - week 1 2

Basic Set Theory

• A set is a collection of elements. Use capital letters, A, B, C to denotes sets and small letters a1, a2, … to denote the elements of the set.

• Notation: means the element a1 is an element of the set AA = a1, a2, a3 .

• The null, or empty set, denoted by Ф, is the set consisting of no points. Thus, Ф is a sub set of every set.

• The set S consisting of all elements under consideration is called the universal set.

Aa ∈1

STA347 - week 1 3

Relationship Between Sets

• Any two sets A and B are equal if A and B has exactly the same elements. Notation: A=B.

• Example: A = 2, 4, 6, B = n; n is even and 2 ≤ n ≤ 6

• A is a subset of B or A is contained in B, if every point in A is also in B. Notation:

• Example: A = 2, 4, 6, B = n; 2 ≤ n ≤ 6 = 2, 3, 4, 5, 6

BA ⊂

STA347 - week 1 4

Venn Diagram

• Sets and relationship between sets can be described by using Venn diagram.

• Example: We toss a fair dice. The universal set S is …

STA347 - week 1 5

Union and Intersection of sets

• The union of two sets A and B, denoted by , is the set of all points that are in at least one of the sets, i.e., in A or B or both.

• Example 1: We toss two fair dice…

• The intersection of two sets A and B, denoted by or AB, is the set of all points that are members of both A and B.

• Example 2: The intersection of A and B as defined in example 1 is …

BA∪

BA∩

STA347 - week 1 6

Properties of unions and intersections

Unions and intersections are:

• Commutative, i.e., AB = BA and

• Associative, i.e.,

• Distributive, i.e.,

• These laws also apply to arbitrary collections of sets (not justpairs).

.ABBA ∪=∪

( ) ( ) CBACBA ∪∪=∪∪

( ) ( ) ( )CABACBA ∪∩∪=∩∪

( ) ( ) ( )CABACBA ∩∪∩=∪∩

STA347 - week 1 7

Disjoint Events

• Two sets A and B are disjoint or mutually exclusive if they have no points in common. Then .

• Example 3: Toss a die. Let A = 1, 2, 3 and B = 4, 5.

Φ=∩ BA

STA347 - week 1 8

Complement of a Set

• The complement of a set, denoted by Ac or A’ makes sense only with respect to some universal set. Ac is the set of all points of the universal set S that are not in A.

• Example: the complement of set A as defined in example 3 is…

• Note: the sets A and Ac are disjoint.

STA347 - week 1 9

De Morgan’s Laws

• For any two sets A and B:

• For any collection of sets A1, A2, A3, … in any universal set S

( ) CCC BABA ∩=∪

( ) CCC BABA ∪=∩

( )IU∞

=

∞

=

=⎟⎟⎠

⎞⎜⎜⎝

⎛

11 n

Cn

C

nn AA

STA347 - week 1 10

Finite, Countable Infinite and Uncountable

• A set A is finite if it contains a finite number of elements.

• A set A is countable infinite if it can be put into a one-to-one correspondence with the set of positive integers N.

• Example: the set of all integers is countable infinite because …

• The whole interval (0,1) is not countable infinite, it is uncountable.

STA347 - week 1 11

The Probability Model

• An experiment is a process by which an observation is made. For example: roll a die 6 times, toss 3 coins etc.

• The set of all possible outcomes of an experiment is called the sample space and is denoted by Ω.

• The individual elements of the sample space are denoted by ωand are often called the sample points.

• Examples ...

• An event is a subset of the sample space. Each sample point is a simple event.

• To define a probability model we also need an assessment of the likelihood of each of these events.

STA347 - week 1 12

σ – Algebra• A σ-algebra, F, is a collection of subsets of Ω satisfying the

following properties:F contains Ф and Ω.F is closed under taking complement, i.e., F is closed under taking countable union, i.e.,

• Claim: these properties imply that F is closed under countable intersection.

• Proof: …

.FAFA C ∈⇒∈

FAAFAA ∈∪∪⇒∈ ...,..., 2121

STA347 - week 1 13

Probability Measure

A probability measure P mapping F [0,1] must satisfy

• For , P(A) ≥ 0 .

• P(Ω) = 1.

• For , where Ai are disjoint,

This property is called countable additivity.

• These properties are also called axioms of probability.

FA∈

FAAA ∈,...,, 321

( ) ( ) ( ) ( ) L+++=∪∪∪ 321321 ... APAPAPAAAP

STA347 - week 1 14

Formal Definition of Probability Model

• A probability space consists of three elements (Ω, F, P)

(1) a set Ω – the sample space.(2) a σ-algebra F - collection of subsets of Ω. (3) a probability measure P mapping F [0,1].

STA347 - week 1 15

Discrete Sample Space

• A discrete sample space is one that contains either a finite or a countable number of distinct sample points.

• For a discrete sample space it suffices to assign probabilities to each sample point.

• There are experiments for which the sample space is not countable and hence is not discrete. For example, the experiment consists of measuring the blood pressure of patients with heart disease.

STA347 - week 1 16

Calculating Probabilities when Ω is Finite

• Suppose Ω has n distinct outcomes, Ω = ω1, ω2,…, ωn. The probability of an event A is

• In many situations, the outcomes of Ω are equally likely, then,

• Example, when rolling a die for i = 1, 2, …, 6.

• In these situations the probability that an event A occurs is

• Example:

( ) ( )∑∈

=A

ii

PAPω

ω

( ) .1n

P i =ω

( )61

=iP

( )nn

nAP a==

occursA for which outcomes of #

STA347 - week 1 17

Rules of Probability

• for all

• Corollary:

• The probability of the union of any two events A and B is

Proof: …

• If then, Proof:

( ) ( )APAP −= 1

( ) .0=ΦP

( ) ( ) ( ) ( )BAPBPAPBAP ∩−+=∪

BA ⊆ ( ) ( ).BPAP ≤

.FA∈

STA347 - week 1 18

• Inclusion / Exclusion formula:For any finite collection of events

• For any finite collection of events

Proof: By induction

nAAA ,...,, 21

( ) ( ) ( ) ( ) ( )∑ ∑∑<

+

<<==

⋅⋅⋅∩∩−+⋅⋅⋅−∩∩+∩−=⎟⎟⎠

⎞⎜⎜⎝

⎛

jin

n

kjikjiji

n

ii

n

ii AAAPAAAPAAPAPAP 21

1

\11

1U

nAAA ,...,, 21

( )∑==

≤⎟⎟⎠

⎞⎜⎜⎝

⎛ n

ii

n

ii APAP

11U

STA347 - week 1 19

Example

• In a lottery there are 10 tickets numbered 1, 2, 3, …, 10. Two numbers are drown for prizes. You hold tickets 1 and 2. What is the probability that you win at least one prize?

STA347 - week 1 20

Conditional Probability• Idea – have performed a chance experiment but don’t know the outcome

(ω), but have some partial information (event A) about ω.Question: given this partial information what’s the probability that the outcome is in some event B?

• Example: Toss a coin 3 times. We are interested in event B that there are 2 or more heads. The sample space has 8 equally likely outcomes.

The probability of the event B is …

Suppose we know that the first coin came up H. Let A be the event the first outcome is H. Then and The conditional probability of B given A is

TTTTTHTHTHTTTHHHTHHHTHHH ,,,,,,,=Ω

( )( )AP

BAP ∩==

8483

43

HTTHTHHHTHHHA ,,,= HTHHHTHHHBA ,,=∩

STA347 - week 1 21

• Given a probability space (Ω, F, P) and events A, B F with P(A) > 0The conditional probability of B given the information that A has occurred is

• Example:We toss a die. What is the probability of observing the number 6 given that the outcome is even?

• Does this give rise to a valid probability measure?

• TheoremIf A F and P(A) > 0 then (Ω, F, Q) is a probability space where Q : is defined by Q(B) = P(B | A).Proof:

( ) ( )( )AP

BAPABP ∩=|

RF →

∈

STA347 - week 1 22

• The fact that conditional probability is a valid probability measure allows the following:

, A, B F, P(A) >0

for any A, B1, B2 F, P(A) >0.

( ) ( )ABPABP |1| −=

∈

∈

( ) ( ) ( ) ( )ABBPABPABPABBP |||| 212121 ∩−+=∪

STA347 - week 1 23

Multiplication rule

• For any two events A and B,

• For any 3 events A, B and C,

• In general,

• Example:An urn initially contains 10 balls, 3 blue and 7 white. We draw a ball and note its colure; then we replace it and add one more of the same colure. We repeat this process 3 times. What is the probability that the first 2 balls drawn are blue and the third one is white?Solution:

( ) ( )APABPBAP |)( =∩

( ) ( ) ( ) ⎟⎟⎠

⎞⎜⎜⎝

⎛⋅⋅⋅∩=⎟⎟

⎠

⎞⎜⎜⎝

⎛ −

==II

1

1213121

1

|||n

iin

n

ii AAPAAAPAAPAPAP

( ) ( ) ( ) ( )BACPABPAPCBAP ∩=∩∩ ||

STA347 - week 1 24

Law of total probability

• Definition:For a probability space (Ω, F, P), a partition of Ω is a countable collection of events such that

and

• Theorem:If is a partition of Ω such that then

for any .• Proof:

iB

Φ=∩ ji BB .Ui

iB Ω=

,..., 21 BB ( ) iBP i ∀> 0( ) ( ) ( )∑=

iii BPBAPAP | FA∈

,FBi ∈

STA347 - week 1 25

Examples1. Calculation of for the Urn example.

2. In a certain population 5% of the females and 8% of the males areleft-handed; 48% of the population are males. What proportion of the population is left-handed?Suppose 1 person from the population is chosen at random; what is the probability that this person is left-handed?

( )2BP

STA347 - week 1 26

Bayes’ Rule

• Let B1,B2,… be a partition of Ω such that P(Bi) > 0 for all i then

for any .

• Example:A test for a disease correctly diagnoses a diseased person as having the disease with probability 0.85. The test incorrectly diagnoses someone without the disease as having the disease withprobability 0.1 If 1% of the people in a population have the disease, what is the probability that a person from this population who tests positive for the disease actually has it?(a) 0.0085 (b) 0.0791 (c) 0.1075 (d) 0.1500 (e) 0.9000

( ) ( ) ( )( ) ( )∑

=

iii

jjj BPBAP

BPBAPABP

||

|

FA∈

STA347 - week 1 27

Independence

• Example:Roll a 6-sided die twice. Define the following eventsA : 3 or less on first rollB : Sum is odd.

• If occurrence of one event does not affect the probability that the other occurs than A, B are independent.

STA347 - week 1 28

• DefinitionEvents A and B are independent if

• Note: Independence ≠ disjoint. Two disjoint events are independent if and only if the probability of one of them is zero.

• Generalized to more than 2 events:A collection of events is (mutually) independent if for anysubcollection

• Note: pairwize independence does not guarantee mutual independence.

( ) ( ) ( )BPAPBAP =∩

nAAA ,..., 21

miii AAA ,...,

21

( ) ( ) ( ) ( )imiiimii APAPAPAAAP ⋅⋅⋅=∩⋅⋅⋅∩∩2121

STA347 - week 1 29

• DefinitionEvents A and B are independent if

• Note: Independence ≠ disjoint. Two disjoint events are independent if and only if the probability of one of them is zero.

• Generalized to more than 2 events:A collection of events is (mutually) independent if for anysubcollection

• Note: pairwize independence does not guarantee mutual independence.

( ) ( ) ( )BPAPBAP =∩

nAAA ,..., 21

miii AAA ,...,

21

( ) ( ) ( ) ( )imiiimii APAPAPAAAP ⋅⋅⋅=∩⋅⋅⋅∩∩2121

STA347 - week 1 30

Example

• Roll a die twice. Define the following events;A: 1st die oddB: 2nd die oddC: sum is odd.