Probabili(esandProbabilis(cModels

EduardoEyrasComputa(onalRNABiology

PompeuFabraUniversity-ICREABarcelona,Spain

Master in Bioinformatics UPF 2017-2018

Probabili(es

Probabili(esandProbabilis(cModels

VariablesConsideravariableXtorepresentanevent:“theoccurrenceofsomething”VariableXcantakeanumberofpossiblevalues,e.g.x1,x2,….(Xisusuallycalledarandomvariable,eitherdiscreteorcon(nue)ProbabilityTheprobabilityofapar(culareventxiisthepropor(onof(mesthatittakesplaceifwemeasureXasufficientnumberof(mes.WecanwritethisasP(X=xi)orP(xi)

Probabili(esandProbabilis(cModels

Probabilitydistribu0onThefunc(ongivenbyP(X=s)foreachsistheprobabilitydistribu0onofX,alsodenotedasP(X)Probabili(estakevaluesbetween0(somethingneveroccurs)and1(somethingalwaysoccurs),andthesumofprobabili(esofallpossibleeventsisalwaysnormalizedto1.

€

0 ≤ P(s) ≤1, and P(s) =1s∈S∑

€

s∈ S

SamplespaceItisthesetofpossibleoutcomesforsomeeventorrandomvariable.S

Probabili(esandProbabilis(cModels

€

0 ≤ P(s) ≤1, and P(s) =1s∈{sun,...}∑

Theprobabilitydistribu(onforX:

Example Xrepresentstheweatherfortomorrow.TheprobabilityofrainingtomorrowiswriVenas:

Whichcanbealsosimplifiedas€

P(X = rain)

€

P(rain)

event probability

sun 0.1

clouds 0.3

rain 0.3

snow 0.2

sleet 0.1

P(not − sunny) = P(s) = 0.9s∈{clouds,rain,snow,sleet}

∑

Probabilityofasubset:

Joint,condi(onalandmarginalprobabili(es

ThejointprobabilityfortwoeventsA=aandB=biswriVen

€

P(a,b) ≡ P(A = a∩ B= b)

Thisdistribu(onisdefinedoverallpossiblevaluesthatAandBcantakeP(A = s, B = r)(foralls,rvaluesfromthesamplespace)

Jointprobability

Joint,condi(onalandmarginalprobabili(es

3importantproper(esofthejointprobabilityJointprobabili(esaddupto1ReversibilityNotalways P(A = a∩B = b) = P(A = a)P(B = b)

P(r,s∑ A = r∩B = s) =1

Thisdistribu(onisdefinedoverallpossiblevaluesthatAandBcantakeP(A = s, B = r)(foralls,rvaluesfromthesamplespace)

P(A = a∩B = b) = P(B = b∩A = a)

P(b) = P(b,ai )ai

∑

Whenthejointprobabili(esareknownwecancalculatethemarginaldistribu(on:

Itdescribesthedistribu(onofB“ignoring”othervariables.Itisobtainedfromthejointdistribu(onbysummingoveroneofthevariables:

Joint,condi(onalandmarginalprobabili(es

Generallyif

€

ai∩ a j =∅

aii = A

Marginaldistribu0on

P(a | b) = P(a,b)P(b)

=P(a,b)P(b,a ')

a '∑

€

P(a | b)a∑ =1

Joint,condi(onalandmarginalprobabili(es

P(a,b) = P(a | b)P(b)

Wecanalsodefinethejointprobabilityintermsofthecondi0onalprobability:

Condi0onalprobabilityThecondi(onalprobabilityofaneventA=awithrespectaneventB=bisdefinedas:

Generally,wedonotusejointprobabili(esdirectly,butcondi(onalprobabili(es

Notethatsince P(a,b) = P(b,a) P(a | b)P(b) = P(b | a)P(a)Itfollowsthat

€

P(A |B)Thecondi0onalprobabilityrepresentsthedistribu(onofAgiven

thatweknowthevalueofB(thisisdifferentfromajointprobability)

Joint,condi(onalandmarginalprobabili(es

P(a | b) = P(a,b)P(b)

=P(a,b)P(b,a ')

a '∑

P(b) = P(b,a)a∑ = P(b | a)P(a)

a∑

Wecanalsocalculatethemarginaldistribu0onalsofromthecondi(onalprobabili(es:

Joint,condi(onalandmarginalprobabili(es

P(b) = P(b,a ')a '∑

P(a | b)a∑ = P(a,b)P(b)a

∑ =P(a,b)

a∑P(b,a ')

a '∑

=1

Notethatthecondi0onalprobabilityisnormalizedto1

P(a | b) = P(a,b)P(b)

Themeaningofprobability

Probabili(escandescribe“frequenciesofoutcomesinrandomexperiments”

Probabili(escandescribe“frequenciesofoutcomesinrandomexperiments”

Themeaningofprobability

Probabili(escanbeusedmoregenerallytodescribedegreesofbeliefinproposi(ons(notnecessarilyinvolvingrandomvariables).Forexample:“Theprobabilitythatthebutleristhemurderer,giventheevidence”Thuswecanuseprobabili(estodescribeassump(onsorhypotheses,andtodescribeinferencesgiventhoseassump(ons.Therulesofprobabili(esensurethatiftwopeoplemakethesameassump(onsandreceivethesamedata,thentheywilldrawiden(calconclusions.

ConsiderHtobethesetofassump(onsorhypotheses,whicharepartofourprobabilis(cmodelWecanconsiderthepreviousdefini(ons:

Themeaningofprobability

€

P(a,h) ≡ P(A = a∩H = h) = P(a,h is true)

€

P(a) = P(a,h)h∈H∑Marginalprobability:

€

P(h) = P(h,a)a∑

Condi(onalprobability: P(a | h) = P(a,h)P(h)

=P(a,h)P(h,a ')

a '∑

Jointprobability:

H = h{ }

Exercise:verifytheseproper(es

€

P(a,b | h) = P(a | b,h)P(b | h) = P(b | a,h)P(a | h)

Productrule(followsfromthedefini(onofcondi(onalprobabili(es)

Sumrule(rewri(ngthemarginalprobabilitydefini(on)

€

P(a | h) = P(a,b | h)b∑ = P(a | b,h)

b∑ P(b | h)

Bayes’Theorem

Fromthepreviousofdefini(onsofcondi(onalprobabilitywecanwrite:

PosteriorprobabilityandBayes’theorem

P(a | h) = P(h | a)P(a)P(h)

=P(h | a)P(a)P(h | ak )P(a k )

ak

∑

P(a,h) = P(a | h)P(h) = P(h | a)P(a)

Thisallowstowritetheprobabilityofthedataacondi0onedtothehypothesish:

€

P(h | a) = P(a | h)P(h)P(a)

=P(a | h)P(h)P(a | h' )P(h' )

h'∑

Ortheotherwayaround,theprobabilityofthehypothesishcondi0onedtothedataa

Likelihood:

Posterior:

Bayes’theorem:

€

P(h | a) = P(a | h)P(h)P(a)

=P(a | h)P(h)P(a | h' )P(h' )

h'∑

ThisformoftheBayes’theoremdescribestheprobabilityofanhypothesis(H=h),giventhemeasurement(A=a).This probability is wriVen in terms of condi(onal probabili(es of themeasurement given the hypothesis (called likelihoods). From them, usingBayes theorem,we can es(mate howprobable is the hypothesis, given theobserveddata.

PosteriorprobabilityandBayes’theorem

Exercise:verifythisproperty

€

P(b | a,h) = P(a | b,h)P(b | h)P(a | h)

=P(a | b,h)P(b,h)P(a | b' ,h)P(b' ,h)

b'∑

Example:TransmembraneProteins

Wewanttogenerateamodelthatisabletoseparatesequencesthatformtransmembranehelicesfromsequencesthatformtheloops.Importanthypotheses(priorknowledge):heliceshavehighcontentofhydrophobicaminoacids(e.g.Isoleucine(I),Leucine(L),Valine(V),…).Thereisadifferencefromrandomness!!àWecanapplyprobabili(es

Ques0on:Givenoneormoreaminoacid,e.g.L,canwedis(nguishbetweenLinhelicesorLinloops?

Considerthefollowingprobabilis(cmodel,whereforeachaminoacid,theoccurrenceprobabilityisgivenbyqa,suchthatWiththismodel,theprobabilityofasequenceofresiduesx1…xncanbewriVenlikethis(assumingindependence):

€

P(x1x2...xn ) = qx1qx2qxn = qxii=1

n

∏

0 ≤ qa ≤1, and qa =1a∑

Example:TransmembraneProteins

Example:TransmembraneProteins

Consideranumberofknownstructures:sequencewithannotatedhelicesWecancalculatethefrequenciesofeachaminoacidainhelices=qa,andinloops=pa

Weestablishtwohypotheses(ormodels):(1)theloopmodelMloopgivenbytheprobabili(esobservedinloops:p(2)thehelixmodelMhelix givenbytheprobabili(esobservedinhelices:q

Example:TransmembraneProteins

Weestablishtwohypotheses(ormodels):(1)theloopmodelMloopgivenbytheprobabili(esobservedinloops:p(2)thehelixmodelMhelix givenbytheprobabili(esobservedinhelices:q

Consideranumberofknownstructures:sequencewithannotatedhelicesWecancalculatethefrequenciesofeachaminoacidainhelices=qa,andinloops=pa

GivenasequenceofNaminoacidss=x1…xNfromanunknownprotein,wecancalculate:

€

P(s |Mloop ) = px1 px2 pxN = pxii=1

N

∏ = L(s |Mloop ) Likelihoodofbeinginaloop

€

P(s |Mhelix ) = qx1qx2qxN = qxii=1

N

∏ = L(s |Mhelix ) Likelihoodofbeinginahelix

Theseprobabili(esarethelikelihoodofthedatagivenaspecificmodel

PosteriorProbability

WeactuallywanttoknowtheprobabilitythatthedataisdescribedbymodelMk,andnottheprobabilitythatthedatawouldariseifthemodelweretrue.Thatis,wewantthe:Posteriorprobability=TocalculateitweneedBayes´theorem:

€

P(Mk |D)

€

P(A,B) = P(A |B)P(B) = P(B | A)P(A)⇒ P(B | A) = P(A |B)P(B)P(A)

ThebasicprincipleofBayesianmethodsisthatwemakeourinferencesusingtheposteriorprobabili(es.

€

P(Mk |D) =L(D |Mk )P(Mk )L(D |Mi)P(Mi)

i∑

Iftheposteriorprobabilityofourmodelismuchhigherthantheothermodels,wecanbeconfidentthatthisisthebestmodeltodescribethedata.

Posteriorprobability Likelihood

PosteriorProbability

P(Mloop |D) =P(D |Mloop )P(Mloop )

P(D |Mloop )P(Mloop )+P(D |Mhelix )P(Mhelix )

P(Mhelix |D) =P(D |Mhelix )P(Mhelix )

P(D |Mhelix )P(Mhelix )+P(D |Mloop )P(Mloop )

P(Mk |D) =P(D |Mk )P(Mk )

P(D)

=P(D |Mk )P(Mk )P(D |Mi )P(Mi )

i∑

PriorProbability

Priorprobability:probabilityassociatedtoeachofthehypotheses(eachofthemodels),e.g.:

€

P(Mloop )

€

P(Mhelix )

Weassignpriorprobabili(estoeachmodel.Theymustadduptoone

€

P prior(Mk ) =1k∑

Thereisnopar(cularwaytoassigntheseprobabili(es.Theymustbees(matedusingsomepriorknowledgeaboutthedata(e.g.isitveryunlikelytofindhelicesinproteins?)Safechoice:uniform(uninforma(ve)probabili(es

NotethattheposterioriswriVenintermsoftheprobabilityofthehypothesisP(M).Thisiscalledtheprioranditrelatestotheques(on:Areallmodels(helix,non-helix)equallylikely?

€

P(Mk |D) =L(D |Mk )P(Mk )L(D |Mk )P(Mk )

k∑

Bayes’Theorem

ThebasicprincipleofBayesianmethodsisthatwemakeourinferencesusingtheposteriorprobabili(es.ThisprobabilityiswriVenintermsofcondi(onalprobabili(esofthemeasurementgiventheassump(on(calledlikelihoods).Fromthem,usingBayestheorem,wecanes(matehowprobableistheassump(on,giventheobserveddata.

€

P(Mk |D) =L(D |Mk )P(Mk )L(D |Mk )P(Mk )

k∑

Bayes’Theorem

€

posterior = likelihood × priorevidence

Yourini(albeliefYourimprovedbelief

degreetowhichyourbeliefexplainstheevidence

Alltheevidence

€

P(Mk |D) =L(D |Mk )P(Mk )L(D |Mk )P(Mk )

k∑

Bayes’Theorem

MedicaldoctorsstudyP(Symptom|Disorder)inpa(entsHowever,whentheyneedtomakeadiagnosis,theyes(mateP(Disorder|Symptom)E.g.,ifweknowP(Fever|Flu)andP(Flu),wecanthenuseBayes’Theoremtododiagnosis

Thedenominatoristhemarginal:Probabilityofhavingfeverwhetherornotyouhaveflu

Problem:Doesapa(enthavethediseaseornotApa(enttakesalabtestandtheresultcomesbackposi(ve.Thetestreturnsacorrectposi(veresultinonly98%ofthecasesinwhichthediseaseisactuallypresentandacorrectnega(veresultinonly97%ofthecasesinwhichthediseaseisnotpresent.Furthermore,0.008oftheen(repopula(onhavethisdisease.

€

P(c)

€

P(c )

€

P(+ | c)P(− | c)

€

P(+ | c )P(− | c )

€

P(c | +)?

Exercise

References

BiologicalSequenceAnalysis:Probabilis0cModelsofProteinsandNucleicAcidsRichardDurbin,SeanR.Eddy,AndersKrogh,andGraemeMitchison.CambridgeUniversityPress,1999ProblemsandSolu0onsinBiologicalSequenceAnalysisMarkBorodovsky,SvetlanaEkishevaCambridgeUniversityPress,2006Bioinforma0csandMolecularEvolu0onPaulG.HiggsandTeresaAVwood.BlackwellPublishing2005.