CS2210DiscreteStructuresDiscreteProbability

Fall2017SukumarGhosh

SampleSpace

DEFINITION.ThesamplespaceSofanexperimentistheset

ofpossibleoutcomes.Anevent E isasubset ofthesample

space.

Whatisprobability?

Probabilitydistribution

Consideranexperimentwheretherearen possibleoutcomesx1,x2,x3,x4,…xn .Then

1.0≤p(xi)≤1(1≤i<n)2.p(x1)+p(x2)+p(x3)+p(x4)+…+p(xn)=1

Youcantreatp asafunction thatmapsthesetofalloutcomestothesetofrealnumbers.Thisiscalledtheprobabilitydistributionfunction.

Probabilityofindependentevents

• WhentwoeventsEandFareindependent,theoccurrenceofonegivesnoinformationabouttheoccurrenceoftheother.

• Theprobabilityoftwoindependenteventsp(E∩F)=p(E).p(F)

Exampleofdice

Whatistheprobabilityoftwo1’s ontwosix-sideddice?

ExamplefromCardgames

Thereare(13x 4)=52cardsinapack

Pokergame:Royalflush

Moreonprobability

Probabilityoftheunionofevents

Example

Whenisgamblingworth?

Disclaimer.Thisisastatisticalanalysis,notamoralorethicaldiscussion

Powerballlottery

Disclaimer.Thisisastatisticalanalysis,notamoralorethicaldiscussion

ConditionalProbability

Youareflippingacoin3times.Thefirstflipisatail.Giventhis,whatistheprobabilitythatthe3flipsproduceanoddnumberoftails?

DealswiththeprobabilityofaneventEwhenanothereventFhasalreadyoccurred.TheoccurrenceofFactuallyshrinksthesamplespace.

GivenF,theprobabilityofEisp(E|F)=p(E ⋂ F)/p(F)

ConditionalProbability

SamplespaceS={TTT,THH,THT,TTH,HTT,HHH,HHT,HTH}F={TTT,THH,THT,TTH}(thereducedsamplespace)E={TTT,THH} {thetargeteventset)

p(E ⋂ F)=2/8,p(F)=4/8.

Sop(E|F)=p(E ⋂ F)/p(F)=1/2

ExampleofConditionalProbability

Whatistheprobabilitythatafamilywithtwochildrenhastwoboys,giventhattheyhaveatleastoneboy?

F={BB,BG,GB}E={BB}

Ifthefourevents{BB,BG,GB,GG}areequallylikely,then

p(F)=¾,andp(E ⋂ F)=¼

Sotheansweris¼dividedby¾=1/3

MontyHall3-doorPuzzle

Whatisbehindthedoors?

WarmupProblem1.Asequenceof10bitsisrandomlygenerated.Whatistheprobabilitythatatleastoneofthesebitsis0?

Answer.Probabilitythatnoneofthesebitsis0is1/210

So,theprobabilitythatatleastoneofthesebitsis0is(1-1/210)=1023/1024

WarmupProblem2.Findtheprobabilityofselectingnoneofthecorrectsixintegers inalottery,(wheretheorderinwhichtheseintegersareselecteddoesnotmatter)fromthepositiveintegers1-40?

Answer.Thenumberofwaysofselectingallwrongnumbersisthenumberofwaysofselectingsixnumbersfromthe34incorrectnumbers.ThereareC(34,6)waystodothis.SincethereareC(40,6)waystochoosenumbersintotal,theprobabilityofselectingnoneofthecorrectsixintegersis

C(34,6)/C(40,6)

Bernoullitrials

Anexperimentwithonlytwooutcomes(like0,1or

T,F)iscalledaBernoullitrial.Manyproblemsneed

tocomputetheprobabilityofexactlyk successes

whenanexperimentconsistsofn independent

Bernoullitrials.

Bernoullitrials

Example.Acoinisbiased sothattheprobability

ofheadsis2/3.Whatistheprobabilitythat

exactlyfourheadscomeupwhenthecoinis

flippedexactlyseventimes?

BernoullitrialsThenumberofways4-out-of-7flipscanbeheadsisC(7,4).

HHHHTTTTHHTHHTTTTHHHH

Eachflipisanindependentflips.Foreachsuchpattern,the

probabilityof4heads(and3tails)=(2/3)4.(1/3)3. So,inall,the

probabilityofexactly4headsisC(7,4).(2/3)4.(1/3)3=560/2187

TheBirthdayProblemTheproblem.Whatisthesmallestnumberofpeoplewhoshouldbeina

roomsothattheprobabilitythatatleasttwoofthemhavethesamebirthdayisgreaterthan½?

13 2

Considerpeopleenteringtheroomoneafteranother.Assumingbirthdaysarerandomlyassigneddates,theprobabilitythatthesecondpersonhasthesamebirthdayasthefirstoneis1- 365/366

Probabilitythatthirdpersonhasthesamebirthdayasanyoneofthepreviouspersonsis1– 364/366x 365/366

TheBirthdayProblemContinuinglikethis,probabilitythatthenth personhasthesamebirthdayas

oneofthepreviouspersonsis1– 365/366x 364/366x …x (367–n)/366

13 2

Solvetheequationsothatforthenth person,thisprobabilityexceeds½.Youwillgetn =23

Alsosometimesknownasthebirthdayparadox.

RandomvariablesDEFINITION.Arandomvariableisafunction fromthesample

spaceofanexperimenttothesetofrealnumbers

Note.Arandomvariableisafunction,notavariableJExample.Acoinisflippedthreetimes.LetX(t)betherandom

variablethatequalsthenumberofheadsthatappearwhentheoutcomeist.Then

X(HHH)=3X(HHT)=X(HTH)=X(THH)=2X(TTH)=X(THT)=X(HTT)=1X(TTT)=0

ExpectedValueInformally,theexpectedvalueofarandomvariableisitsaverage

value.Like,“whatistheaveragevalueofaDie?”

DEFINITION.TheexpectedvalueofarandomvariableX(s)isequalto∑s∈S p(s)X(s)

EXAMPLE1.ExpectedvalueofaDieEachnumber1,2,3,4,5,6occurswithaprobability1/6.Sotheexpectedvalueis1/6(1+2+3+4+5+6)=21/6=7/2

ExpectedValue(continued)

EXAMPLE2.Afaircoinisflippedthreetimes.LetXbetherandomvariablethatassignstoanoutcomethenumberofheadsthatistheoutcome.WhatistheexpectedvalueofX?

Thereareeightpossibleoutcomeswhenafaircoinisflippedthreetimes.TheseareHHH,HHT,HTH,HTT,THH,THT,TTH,TTT.Eachoccurswithaprobabilityof1/8.So,

E(X)=1/8(3+2+2+2+1+1+1+0)=12/8=3/2

Geometricdistribution

Considerthis:Youflipacoinandtheprobabilityofatailisp.Thiscoinisrepeatedlyflippeduntilitcomesuptails.

Whatistheexpectednumberofflipsuntilyouseeatail?

GeometricdistributionThesamplespace{T,HT,HHT,HHHT…}isinfinite.

Theprobabilityofatail(T)isp.Probabilityofahead(H)is(1-p)Theprobabilityof(HT)is(1-p)pTheprobabilityof(HHT)is(1-p)2petc.Why?

LetXbetherandomvariablethatcountsthenumberofflipstoseeatail.Thenp (X=j)=(1-p)j-1.p

Thisisknownasgeometricdistribution.

ExpectationinaGeometricdistribution

X=therandomvariablethatcountsthenumberofflipstoseeatail.

So, andsoon

Thisinfiniteseriescanbesimplifiedto1/p.

Thus,ifp =0.2thentheexpectednumberofflipsafterwhichyouseeatailis1/0.2=5

X(T ) = 1,X(HT ) = 2,X(HHT ) = 3

E(X) =   j.p(X = j)1

∞

∑         =   1.p + 2.(1− p).p + 3.(1− p)2.p + 4.(1− p)3.p + ...

Explanation

Probability Value

0.20.30.5

304020

Whatistheaveragevalue?

0.2x 30+0.3x 40+0.5x 20=28

LinearityofExpectationTheorem

IfXi,i =1,2,...,nwithnapositiveinteger,arerandomvariablesonS,andifaandb

arerealnumbers,then

(i) E(X1 +X2 +···+Xn)=E(X1)+E(X2)+···+E(Xn)

(ii) E(aX+b)=aE(X)+b.

Example1.Whatistheexpectedvalueofthesumofthenumbersthatappearwhenapairoffairdiceisrolled?

LetX1 andX2 betherandomvariablessothatX1 appearsinthefirstdieandX2 appearsontheseconddie.E(X1 +X2)=E(X1)+E(X2)=7/2+7/2=7.

UsefulFormulas

p(E) = 1− p(E)

p(E ∩ F) = p(E).p(F) (EandFaremutuallyindependent)

p(E ∪ F) = p(E)+ p(F) (EandFaremutuallyindependent)

p(E ∪ F) = p(E)+ p(F)− p(E ∩ F) (EandFarenotindependent:Inclusion- Exclusion)

p(E | F) = p(E ∩ F)

p(F) (Conditionalprobability:givenF,theprobabilityofE)

MonteCarloAlgorithms

Aclassofprobabilisticalgorithmsthatmakearandomchoiceatoneormoresteps.

Example.Hasthisbatchofn chipsnot beentestedbythechipmaker?

Randomlypickachipandtestit.Ifitisbad,thentheansweristrue(i.e.ithasnotbeentested).Ifthechipisgoodthentheansweris“don’tknow.”Thenrandomlypickanother.

Aftertheansweris“don’tknow”forKdifferentrandompicks,withyoucertifythebatchtobegood.

Whatistheprobabilityofawrongconclusion?

MonteCarloAlgorithms

Assumethatinpreviouslyuntestedbatches,theprobabilitythataparticularchipisbadhasbeenobservedtobe0.1.Sotheprobabilityofachipbeinggoodfromanuntestedbatchis(1-0.1)=0.9.

Eachtestisindependent.SotheprobabilitythatallKstepsproducetheresult“don’tknow”is0.9k.BymakingKlargeenough,onecanmaketheprobabilityassmallaspossible.Thus,ifK=66,then0.966 <0.001

ThefactthatsomanychipsareOKtellsthattheprobabilitythatthebatchhasnotbeentestedisverysmall.Sowecertifythebatch.UsuallyKisaconstant.Eachtesttakesaconstanttime– sowecancertify(ordiscard)abatchinconstanttime.

-- Certificationviarandomwitnesses-- MonteCarloalgorithmfortestingprimenumbers

Bayes’theoremThisisrelatedtoconditionalprobability.Wecanmakearealisticestimatewhensomeextrainformationisavailable.

Problem1.Therearetwoboxes.Bobfirstchoosesoneofthetwoboxesatrandom.Hethenselectsoneoftheballsinthisboxatrandom.

IfBobhasselectedaredball,whatistheprobabilitythatheselectedaballfromthefirstbox?

(Seepage469ofyourtextbook)

Bayes’theoremLetE= Bobchosearedball.SoE’=BobchoseagreenballF=BobchosefromBox1.SoF’ =BobchosefromBox2Wehavetocomputep(F|E)

p(E|F)=7/9,p(E|F’)=3/7

Wehavetofind

p(F)=p(F’)=1/2

p(F | E) = p(F∩E)p(E)

p(E∩F) = p(E | F).p(F) = (7 / 9).(1 / 2) = 7 /18

p(E∩F ' ) = p(E | F ' ).p(F ' ) = (3 / 7).(1 / 2) = 3 /14

Bayes’theorem

p(E) = p(E∩F)+ p(E∩F ' ) == 7 /18 + 3 /14 = 38 / 63

p(F | E) = p(F∩E)p(E)

= 7 /1838 / 63

= 4976

ThisistheprobabilitythatBobchosetheballfromBox1

Bayes’theoremLetEandFbeeventsfromasamplespaceSsuchthatp(E)≠0andp(F)≠0.Then

p(F | E) = p(E | F).p(F)p(E | F)p(F)+ p(E | F).p(F)

cComputethis

cgiven

Bayes’theorem

1. Supposethatonepersonin100,000hasaparticularrarediseaseforwhichthereisafairlyaccuratediagnostictest.

2.Thistestiscorrect99.0%ofthetimewhengiventoapersonselectedatrandomwhohasthedisease;

3.Thetestiscorrect99.5%ofthetimewhengiventoapersonselectedatrandomwhodoesnothavethedisease.

Findtheprobabilitythatapersonwhotestspositiveforthediseasereallyhasthedisease.(Seepage471ofyourtextbook)

Problem2

Bayes’theorem

ü 1in100,000hastheraredisease (1)ü Thistestis99.0%correctifactuallyinfected; (2)ü Thetestis99.5%correctifnotinfected (3)

LetF=eventthatarandomlychosenpersonhasthediseaseandE=eventthatarandomlychosenpersontestspositive

So,p(F)=0.00001,p(F’)=0.99999 {from(1)}Also,p(E|F)=0.99,andp(E’|F)=1- 0.99=0.01 {from(2)}Alsop(E’|F’)=0.995,andp(E|F’)=1- 0.995=0.005{from(3)}

Now,plugintoBayes’theorem.

Bayes’theorem

p(F | E) = p(E | F).p(F)p(E | F)p(F)+ p(E | F).p(F)

=

0.99 × 0.000010.99 × 0.00001+ 0.005 × 0.99999

! 0.002

So,theprobabilitythataperson“whotestspositiveforthedisease”reallyhasthediseaseisonly0.2%