03: Intro to ProbabilityLisa Yan
April 10, 2020
1
Lisa Yan, CS109, 2020
Quick slide reference
2
3 Defining Probability 03a_definitions
13 Axioms of Probability 03b_axioms
20 Equally likely outcomes 03c_elo
30 Corollaries 03d_corollaries
37 Exercises LIVE
Todayβs discussion thread: https://us.edstem.org/courses/109/discussion/24492
Defining Probability
3
Gradescope quiz, blank slide deck, etc.
http://cs109.stanford.edu/
03a_definitions
Lisa Yan, CS109, 2020
Key definitions
An experiment in probability:
Sample Space, π: The set of all possible outcomes of an experiment
Event, πΈ: Some subset of π (πΈ β π).
Outcome
4
Experiment
π
πΈ
Lisa Yan, CS109, 2020 5
Key definitions
Event, πΈ
β’ Flip lands headsπΈ = Heads
β’ β₯ 1 head on 2 coin flipsπΈ = (H,H), (H,T), (T,H)
β’ Roll is 3 or less:πΈ = 1, 2, 3
β’ Low email day (β€ 20 emails)πΈ = π₯ | π₯ β β€, 0 β€ π₯ β€ 20
β’ Wasted day (β₯ 5 TT hours):πΈ = π₯ | π₯ β β, 5 β€ π₯ β€ 24
Sample Space, π
β’ Coin flipπ = Heads, Tails
β’ Flipping two coinsπ = (H,H), (H,T), (T,H), (T,T)
β’ Roll of 6-sided dieπ = {1, 2, 3, 4, 5, 6}
β’ # emails in a dayπ = π₯ | π₯ β β€, π₯ β₯ 0
β’ TikTok hours in a dayπ = π₯ | π₯ β β, 0 β€ π₯ β€ 24
Lisa Yan, CS109, 2020
What is a probability?
A number between 0 and 1
to which we ascribe meaning.*
6
*our belief that an event πΈ occurs.
Lisa Yan, CS109, 2020
What is a probability?
π = # of total trials
π(πΈ) = # trials where πΈ occurs
7
Let πΈ = the set of outcomes
where you hit the target.
Lisa Yan, CS109, 2020
What is a probability?
π = # of total trials
π(πΈ) = # trials where πΈ occurs
8
Let πΈ = the set of outcomes
where you hit the target.
Lisa Yan, CS109, 2020
What is a probability?
π = # of total trials
π(πΈ) = # trials where πΈ occurs
9
Let πΈ = the set of outcomes
where you hit the target.
π πΈ β 0.500
Lisa Yan, CS109, 2020
What is a probability?
π = # of total trials
π(πΈ) = # trials where πΈ occurs
10
Let πΈ = the set of outcomes
where you hit the target.
π πΈ β 0.667
Lisa Yan, CS109, 2020
What is a probability?
π = # of total trials
π(πΈ) = # trials where πΈ occurs
11
Let πΈ = the set of outcomes
where you hit the target.
π πΈ β 0.458
Lisa Yan, CS109, 2020 12Not just yetβ¦
Axioms of Probability
13
03b_axioms
Lisa Yan, CS109, 2020
Quick review of sets
14
Review of Sets
E F
S πΈ and πΉ are events in π.
Experiment:
Die roll
π = 1, 2, 3, 4, 5, 6Let πΈ = 1, 2 , and πΉ = 2,3
Lisa Yan, CS109, 2020
Quick review of sets
15
Review of Sets
πΈ and πΉ are events in π.
Experiment:
Die roll
π = 1, 2, 3, 4, 5, 6Let πΈ = 1, 2 , and πΉ = 2,3
E
S
F
def Union of events, πΈ βͺ πΉ
The event containing all outcomes in πΈ or πΉ.
πΈ βͺ πΉ = {1,2,3}
Lisa Yan, CS109, 2020
Quick review of sets
16
Review of Sets
πΈ and πΉ are events in π.
Experiment:
Die roll
π = 1, 2, 3, 4, 5, 6Let πΈ = 1, 2 , and πΉ = 2,3
E
S
F
def Intersection of events, πΈ β© πΉ
The event containing all outcomes in πΈ and πΉ.
πΈ β© πΉ = πΈπΉ = {2}
def Mutually exclusive events πΉand πΊ means that πΉ β© πΊ = β
G
Lisa Yan, CS109, 2020
Quick review of sets
17
Review of Sets
πΈ and πΉ are events in π.
Experiment:
Die roll
π = 1, 2, 3, 4, 5, 6Let πΈ = 1, 2 , and πΉ = 2,3
E
S
F
def Complement of event πΈ, πΈπΆ
The event containing all outcomes in that are not in πΈ.
πΈπΆ = {3, 4, 5, 6}
Lisa Yan, CS109, 2020
3 Axioms of Probability
Definition of probability: π πΈ = limπββ
π(πΈ)
π
Axiom 1: 0 β€ π πΈ β€ 1
Axiom 2: π π = 1
Axiom 3: If πΈ and πΉ are mutually exclusive (πΈ β© πΉ = β ),then π πΈ βͺ πΉ = π πΈ + π πΉ
18
Lisa Yan, CS109, 2020
Axiom 3 is the (analytically) useful Axiom
Axiom 3: If πΈ and πΉ are mutually exclusive (πΈ β© πΉ = β ),then π πΈ βͺ πΉ = π πΈ + π πΉ
More generally, for any sequence ofmutually exclusive events πΈ1 , πΈ2 , β¦ :
19
(like the Sum Rule
of Counting, but for
probabilities)
π
πΈ1 πΈ2
πΈ3
Equally Likely Outcomes
20
03c_elo
Lisa Yan, CS109, 2020
Equally Likely Outcomes
Some sample spaces have equally likely outcomes.
β’ Coin flip: S = {Head, Tails}
β’ Flipping two coins: S = {(H, H), (H, T), (T, H), (T, T)}
β’ Roll of 6-sided die: S = {1, 2, 3, 4, 5, 6}
If we have equally likely outcomes, then P(Each outcome)
Therefore
21
(by Axiom 3)
Lisa Yan, CS109, 2020
Roll two dice
Roll two 6-sided fair dice. What is P(sum = 7)?
π = { (1, 1) , (1, 2), (1, 3), (1, 4), (1, 5), (1, 6),
(2, 1) , (2, 2), (2, 3), (2, 4), (2, 5), (2, 6),
(3, 1) , (3, 2), (3, 3), (3, 4), (3, 5), (3, 6),
(4, 1) , (4, 2), (4, 3), (4, 4), (4, 5), (4, 6),
(5, 1) , (5, 2), (5, 3), (5, 4), (5, 5), (5, 6),
(6, 1) , (6, 2), (6, 3), (6, 4), (6, 5), (6, 6)}
πΈ =
22
π πΈ =|πΈ|
|π|
Equally likely
outcomes
Lisa Yan, CS109, 2020
Target revisited
23
Lisa Yan, CS109, 2020
Target revisited
Screen size = 800 Γ 800
Radius of target: 200
The dart is equally likely to land anywhere on the screen. What is π πΈ , the probability of hitting the target?
24
Let πΈ = the set of outcomeswhere you hit the target.
π = 8002 πΈ β π β 2002
π πΈ =|πΈ|
|π|
Equally likely
outcomes
Lisa Yan, CS109, 2020
Target revisited
Screen size = 800 Γ 800
Radius of target: 200
The dart is equally likely to land anywhere on the screen. What is π πΈ , the probability of hitting the target?
25
Let πΈ = the set of outcomeswhere you hit the target.
π = 8002 πΈ β π β 2002
π πΈ =|πΈ|
|π|
Equally likely
outcomes
Lisa Yan, CS109, 2020
Play the lottery.
What is π win ?
π = {Lose,Win}
πΈ = {Win}
26
Not equally likely outcomes π πΈ =|πΈ|
|π|
Equally likely
outcomes
The hard part: defining outcomes consistently
across sample space and events
Lisa Yan, CS109, 2020
Cats and sharks
4 cats and 3 sharks in a bag. 3 drawn.
What is P(1 cat and 2 sharks drawn)?
27
Note: Do indistinct objects give you an equally likely sample space?
A.
B.
C.
D.
E. Zero/other
π πΈ =|πΈ|
|π|
Equally likely
outcomes
π€
(No)
Make indistinct items distinct
to get equally likely outcomes.
Lisa Yan, CS109, 2020
Cats and sharks (ordered solution)
4 cats and 3 sharks in a bag. 3 drawn.
What is P(1 cat and 2 sharks drawn)?
28
Define
β’ π = Pick 3 distinct items
β’πΈ = 1 distinct cat,2 distinct sharks
π πΈ =|πΈ|
|π|
Equally likely
outcomes
Make indistinct items distinct
to get equally likely outcomes.
Lisa Yan, CS109, 2020
Cats and sharks (unordered solution)
4 cats and 3 sharks in a bag. 3 drawn.
What is P(1 cat and 2 sharks drawn)?
29
π πΈ =|πΈ|
|π|
Equally likely
outcomes
Define
β’ π = Pick 3 distinct items
β’πΈ = 1 distinct cat,2 distinct sharks
Make indistinct items distinct
to get equally likely outcomes.
Corollaries of Probability
30
03d_corollaries
Lisa Yan, CS109, 2020
Axioms of Probability
Definition of probability: π πΈ = limπββ
π(πΈ)
π
Axiom 1: 0 β€ π πΈ β€ 1
Axiom 2: π π = 1
Axiom 3: If πΈ and πΉ are mutually exclusive (πΈ β© πΉ = β ),then π πΈ βͺ πΉ = π πΈ + π πΉ
31
Review
Lisa Yan, CS109, 2020
3 Corollaries of Axioms of Probability
Corollary 1: π πΈπΆ = 1 β π(πΈ)
32
Lisa Yan, CS109, 2020
Proof of Corollary 1
Corollary 1: π πΈπΆ = 1 β π(πΈ)
Proof:
πΈ, πΈπΆ are mutually exclusive Definition of πΈπΆ
π πΈ βͺ πΈπΆ = π πΈ + π πΈπΆ Axiom 3
π = πΈ βͺ πΈπΆ Everything must either bein πΈ or πΈπΆ, by definition
1 = π π = π πΈ +π πΈπΆ Axiom 2
π πΈπΆ = 1 β π(πΈ) Rearrange
33
Lisa Yan, CS109, 2020
3 Corollaries of Axioms of Probability
Corollary 1: π πΈπΆ = 1 β π(πΈ)
Corollary 2: If πΈ β πΉ, then π πΈ β€ π(πΉ)
Corollary 3: π πΈ βͺ πΉ = π πΈ + π πΉ β π πΈπΉ(Inclusion-Exclusion Principle for Probability)
34
Lisa Yan, CS109, 2020
Selecting Programmers
β’ P(student programs in Java) = 0.28
β’ P(student programs in Python) = 0.07
β’ P(student programs in Java and Python) = 0.05.
What is P(student does not program in (Java or Python))?
35
1. Define events& state goal
2. Identify knownprobabilities
3. Solve
Lisa Yan, CS109, 2020
Corollary 3: π πΈ βͺ πΉ = π πΈ + π πΉ β π πΈπΉ(Inclusion-Exclusion Principle for Probability)
General form:
Inclusion-Exclusion Principle (Corollary 3)
π πΈ βͺ πΉ βͺ πΊ =
π πΈ + π πΉ + π(πΊ)
β π πΈ β© πΉ β π πΈ β© πΊ β π πΉ β© πΊ
+ π πΈ β© πΉ β© πΊ
36
E
FG
r = 1:
r = 2:
r = 3:
(live)03: Intro to ProbabilityOishi Banerjee and Cooper RaterinkAdapted from Lisa YanJune 26, 2020
37
Lisa Yan, CS109, 2020
Reminders: Lecture with
β’ Turn on your camera if you are able, mute your mic in the big room
β’ Virtual backgrounds are encouraged (classroom-appropriate)
Breakout Rooms for meeting your classmates⦠Just like sitting next to someone new Our best approximation to sitting next to someone new
We will use Ed instead of Zoom chatβ’ Lots of activity and questions, thank you all!
38
Todayβs discussion thread: https://us.edstem.org/courses/667/discussion/82037
Lisa Yan, CS109, 2020
Summary so far
39
Sort objects
(permutations)
Distinct
(distinguishable)
Some
distinct Distinct Indistinct
Distinct
1 group π groups
Counting tasks on π objects
Choose π objects
(combinations)
Put objects in πbuckets
π
ππππ!
Ordered Unordered
π πΈ =|πΈ|
|π|
Equally likely
outcomes
Review
Lisa Yan, CS109, 2020
Indistinguishable? Distinguishable? Probability?
We choose 3 books from a set of 4 distinct (distinguishable) and 2 indistinct (indistinguishable) books.
Let event πΈ = our choice does not include both indistinct books.
1. What is |πΈ|?
2. What is π πΈ ?
40
Review
distinguishable,
equally likely
outcomes
report
countmake indistinct
compute
probability
Think, then Breakout Rooms
Then check out the question on the next slide (Slide 44). Post any clarifications here!
https://us.edstem.org/courses/667/discussion/82037
Think by yourself: 2 min
Breakout rooms: 5 min. Introduce yourself!
41
π€
Lisa Yan, CS109, 2020
Poker Straights and Computer Chips
1. Consider 5-card poker hands.β’ βstraightβ is 5 consecutive rank
cards of any suit
What is P(Poker straight)?
2. Consider the βofficialβ definition of a Poker Straight:β’ βstraightβ is 5 consecutive rank cards of any suit
β’ straight flushβ is 5 consecutive rank cards of same suit
What is P(Poker straight, but not straight flush)?
3. Computer chips: π chips are manufactured, 1 of which is defective.π chips are randomly selected from π for testing.
What is P(defective chip is in π selected chips?)
42
β’ What is an example of an outcome?
β’ Is each outcome equally likely?
β’ Should objects be
ordered or unordered?
π€
Lisa Yan, CS109, 2020
Any Poker Straight
1. Consider 5-card poker hands.
β’ βstraightβ is 5 consecutive rankcards of any suit
What is P(Poker straight)?
43
Define
β’ π (unordered)
β’ πΈ (unordered,consistent with S)
Compute π Poker straight =
Lisa Yan, CS109, 2020
Consider 5-card poker hands.
β’ βstraightβ is 5 consecutive rank cards of any suit
β’ βstraight flushβ is 5 consecutive rank cards of same suit
What is P(Poker straight, but not straight flush)?
44
βOfficialβ Poker Straight
Define
β’ π (unordered)
β’ πΈ (unordered,consistent with S)
Compute π Official Poker straight =
Lisa Yan, CS109, 2020
Define
β’ π (unordered)
β’ πΈ (unordered,consistent with S)
Compute
π chips are manufactured, 1 of which is defective.π chips are randomly selected from π for testing.
What is π(defective chip is in π selected chips?)
45
Chip defect detection
π πΈ =
Lisa Yan, CS109, 2020
π chips are manufactured, 1 of which is defective.π chips are randomly selected from π for testing.
What is P(defective chip is in π selected chips?)
46
Chip defect detection, solution #2
Redefine experiment
1. Choose π indistinct chips (1 way)
2. Throw a dart and make one defective
Define
β’ π (unordered)
β’ πΈ (unordered,consistent with S)
Interlude for announcements
47
Lisa Yan, CS109, 2020
Section
48
Week 1βs section: pre-recorded Python review session
Week 2+: 12:30-1:30PT Thursdays, live on Zoom (will be recorded)
Lisa Yan, CS109, 2020
Interesting probability news
49
βThe study finds that very few chords govern most of
the music, a phenomenon that is also known in
linguistics, where very few words dominate language
corporaβ¦. It characterizes Beethoven's specific
composition style for the String Quartets, through a
distribution of all the chords he used, how often they
occur, and how they commonly transition from one to
the other.β
https://actu.epfl.ch/news/de
coding-beethoven-s-music-
style-using-data-scienc/
Lisa Yan, CS109, 2020
3 Corollaries of Axioms of Probability
Corollary 1: π πΈπΆ = 1 β π(πΈ)
Corollary 2: If πΈ β πΉ, then π πΈ β€ π(πΉ)
Corollary 3: π πΈ βͺ πΉ = π πΈ + π πΉ β π πΈπΉ(Inclusion-Exclusion Principle for Probability)
50
Review
Lisa Yan, CS109, 2020
Takeaway: Mutually exclusive events
51
π α«π=1
β
πΈπ =
π
πΈ1 πΈ2
πΈ3
Axiom 3,
Mutually exclusive
events
E
F G
Inclusion-
Exclusion
Principle
The challenge of probability is in defining events.
Some event probabilities are easier to compute than others.
π α«π=1
β
πΈπ =
Review
Lisa Yan, CS109, 2020
Serendipity
52
Lisa Yan, CS109, 2020
Serendipity
β’ The population of Stanford is π = 17,000 people.
β’ You are friends with π = people.
β’ Walk into a room, see π = 360 random people.
β’ Assume you are equally likely to see each person at Stanford.
What is the probability that you see someone you know in the room?
53
Breakout Rooms
Check out the question on the next slide (Slide 57). Post any clarifications here!
https://us.edstem.org/courses/667/discussion/82037
Breakout rooms: 5 min. Introduce yourself if you havenβt yet!
54
π€
Lisa Yan, CS109, 2020
Serendipity
β’ The population of Stanford is π = 17,000 people.
β’ You are friends with π = 100 people.
β’ Walk into a room, see π = 360 random people.
β’ Assume you are equally likely to see each person at Stanford.
What is the probability that you see someone you know in the room?
Define
β’ π (unordered)
β’ πΈ: β₯ 1 friend in the room
55
What strategy should you use?
A. π exactly 1 + π exactly 2π exactly 3 + β―
B. 1 β π see no friends
π€
Lisa Yan, CS109, 2020
Serendipity
β’ The population of Stanford is π = 17,000 people.
β’ You are friends with π = 100 people.
β’ Walk into a room, see π = 360 random people.
β’ Assume you are equally likely to see each person at Stanford.
What is the probability that you see someone you know in the room?
Define
β’ π (unordered)
β’ πΈ: β₯ 1 friend in the room
56
It is often much easier to compute π πΈπ .
Lisa Yan, CS109, 2020
The Birthday Paradox Problem
What is the probability that in a set of n people, at least one pair of them will share the same birthday?
For you to think about (and discuss in section!)
57
Lisa Yan, CS109, 2020
In a 52 card deck, cards are flipped one at a time.After the first ace (of any suit) appears, consider the next card.
Is P(next card = Ace Spades) < P(next card = 2 Clubs)?
Card Flipping
π€(by yourself)
Lisa Yan, CS109, 2020
In a 52 card deck, cards are flipped one at a time.After the first ace (of any suit) appears, consider the next card.
Is P(next card = Ace Spades) < P(next card = 2 Clubs)?
Card Flipping
Sample space | S | = 52!
Event πΈπ΄π , next card
is Ace Spades
πΈ2πΆ, next card
is 2 Clubs
1. Take out Ace of Spades.
2. Shuffle leftover 51 cards.
3. Add Ace Spades after first ace.
|πΈπ΄π| = 51! β 1
1. Take out 2 Clubs.
2. Shuffle leftover 51 cards.
3. Add 2 Clubs after first ace.
|πΈ2πΆ| = 51! β 1
π πΈπ΄π = π πΈ2πΆ