lecture 14: approximate inference sampling methods · lecture 14: approximate inference sampling...
TRANSCRIPT
![Page 1: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/1.jpg)
CS839:ProbabilisticGraphicalModels
Lecture14:ApproximateInferenceSamplingMethods
TheoRekatsinas
1
![Page 2: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/2.jpg)
Approachestoinference
2
• Exactinferencealgorithms• Theeliminationalgorithm• Message-passingalgorithm(sum-product,beliefpropagation)• Junctiontreealgorithm
• Approximateinferencetechniques• Variational algorithms
• Loopybeliefpropagation• Meanfieldapproximation
• Stochasticsimulation/samplingmethods• MarkovchainMonteCarlomethods
![Page 3: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/3.jpg)
Howtorepresentajointdistribution?
3
• Closedformrepresentation
• Sample-basedrepresentationCollectsamplesX(m) ~P(x)ifwedrawalotofsampleswecanusethelawoflargenumberstogetthatEp(f(x))=Σmf(X(m))/|m|
![Page 4: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/4.jpg)
MonteCarloMethods
4
• Drawrandomsamplesfromthedesireddistribution• Yieldastochasticrepresentationofacomplexdistribution
• marginals andotherexpectationscanbeapproximatedusingsample-basedaverages• Ep(f(x))=Σmf(X(m))/|m|
• Asymptoticallyexactandeasytoapplytoarbitrarymodels• Challenges:
• howtodrawsamplesfromagivendist.(notalldistributionscanbetriviallysampled)?
• howtomakebetteruseofthesamples(notallsampleareuseful,orequallyuseful,seeanexamplelater)?
• howtoknowwe'vesampledenough?
![Page 5: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/5.jpg)
MonteCarloMethods
5
• DirectSampling• Wehaveseenit.• Verydifficulttopopulateahigh-dimensionalstatespace
• RejectionSampling• Createsampleslikedirectsampling,onlycountsampleswhichisconsistentwithgivenevidences.
• Likelihoodweighting,...• Samplevariablesandcalculateevidenceweight.Onlycreatethesampleswhichsupporttheevidences.
• MarkovchainMonteCarlo(MCMC)• Metropolis-Hasting• Gibbs
![Page 6: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/6.jpg)
Rejectionsampling
6
• Supposewewishtosamplefromdist.Π(X)=Π'(X)/Z.• Π(X)isdifficulttosample,butΠ'(X)iseasytoevaluate• SamplefromasimplerdistributionQ(X)• Rejectionsampling
• Correctness:
• Pitfall:Wegainedasamplebutwhatdidwepay?
![Page 7: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/7.jpg)
Unnormalized ImportanceSampling
7
• SupposesamplingfromP(·)ishard.• Supposewecansamplefroma"simpler"proposaldistributionQ(·)instead.• IfQdominatesP(i.e.,Q(x)>0wheneverP(x)>0),wecansamplefromQandreweight:
![Page 8: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/8.jpg)
Normalizedimportancesampling
8
• SupposewecanonlyevaluateP’(x)=aP(x)
![Page 9: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/9.jpg)
Weightedresampling
9
• Problemofimportancesampling:performancedependsonhowwellQmatchesP• IfP(x)f(x)isstronglyvaryingandhasasignificantproportionofitsmassconcentratedinasmallregion,rm willbedominatedbyafewsamples
• Solution:useaheavytailQandweightedresampling
![Page 10: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/10.jpg)
LimitationsofMonteCarlo
10
• Directsampling• Hardtogetrareeventsinhigh-dimensionalspaces• InfeasibleforMRFsunlessweknowthenormalizerZ
• Rejectionsampling,Importancesampling• WeneedagoodproposalQ(x)thatisnotverydifferentthanP(x)
• Howaboutweuseanadaptiveproposal?
![Page 11: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/11.jpg)
MarkovChainMonteCarlo
11
• MCMCalgorithmsfeatureadaptiveproposals• InsteadofQ(x’)useQ(x’|x)wherex’isthenewstatebeingsampledandxistheprevioussample• AsxchangesQ(x’|x)canalsochange
![Page 12: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/12.jpg)
Metropolis-Hastings
12
• Drawasamplex’fromQ(x’|x)wherexistheprevioussample• Thenewsamplex’isacceptedorrejectedwithsomeprobabilityA(x’|x)
• Acceptanceprob:
• A(x’|x)islikearationofimportancesamplingweights• P(x’)/Q(x’|x)istheimportanceweightforx’,P(x)/Q(x|x’)istheimportanceweightforx• Wedividetheimportanceweightforx’bythatofx• NoticethatweonlyneedtocomputeP(x’)/P(x)ratherthanP(x’)orP(x)
• A(x’|x)ensuresthataftersufficientlymanydraws,oursamplescomefromthetruedistribution.
![Page 13: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/13.jpg)
Metropolis-Hastings
13
![Page 14: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/14.jpg)
ExampleofMH
14
• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)
![Page 15: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/15.jpg)
ExampleofMH
15
• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)
![Page 16: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/16.jpg)
ExampleofMH
16
• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)
![Page 17: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/17.jpg)
ExampleofMH
17
• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)
![Page 18: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/18.jpg)
ExampleofMH
18
• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)
![Page 19: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/19.jpg)
ExampleofMH
19
• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)
![Page 20: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/20.jpg)
ExampleofMH
20
• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)
![Page 21: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/21.jpg)
ExampleofMH
21
• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)
![Page 22: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/22.jpg)
SometheoreticalaspectsofMCMC
22
• TheMHalgorithmhasaburn-inperiod• InitialsamplesarenottrulyfromP
• WhyaretheMHsamplesguaranteedtobefromP(x)?• TheproposalQ(x’|x)keepschangingwiththevalueofx;howdoweknowthesampleswilleventuallycomefromP(x)?
• WhyMarkovChain?
![Page 23: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/23.jpg)
MarkovChains
23
• AMarkovChainisasequenceofrandomvariablesx1,x2,…,xN withtheMarkovProperty
• Therighthandsideisthetransitionkernel.Nextstatedependsonlyonprecedingstate
• Let’sassumethekernelisfixedwithtime.
![Page 24: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/24.jpg)
MCConcepts
24
![Page 25: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/25.jpg)
MCConcepts
25
• StationarydistributionsareofgreatimportanceinMCMC.Somenotions• Irreducible:anMCisirreducibleifyoucangetfromanystatextoanyotherstatex’withprobabilityx>0inafinitenumberofsteps• Aperiodic:anMCisaperiodicifyoucanreturntoanystatexatanytime• Ergodic(orregular):anMCisergodicifitisirreducibleandaperiodic
• Ergodicityisimportant:itimpliesyoucanreachthestationarydistributionnomattertheinitialdistribution.
![Page 26: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/26.jpg)
MCConcepts
26
• Reversible(detailedbalance):anMCisreversibleifthereexistsadistributionπ(x)suchthatthedetailedbalanceconditionholds
• ReversibleMCsalwayshaveastationarydistribution
![Page 27: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/27.jpg)
WhydoesMHwork?
27
• Wedrawasamplex’accordingtoQ(x’|x)andthenaccept/rejectaccordingtoA(x’|x).Hencethetransitionkernelis:
• WecanprovethatMHsatisfiesdetailedbalance.
![Page 28: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/28.jpg)
WhydoesMHwork?
28
• NowsupposeA(x’|x)<1andA(x|x’).=1.Wehave
• Thisisthedetailedbalancecondition:• TheMHalgorithmleadstoastationarydistributionP(x)• WedefinedP(x)tobethetruedistributionofx• Thus,MHeventuallyconvergestothetruedistribution
![Page 29: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/29.jpg)
GibbsSampling
29
• GibbsSamplingisanMCMCalgorithmthatsampleseachrandomvariableofagraphicalmodel,oneatatime
• GSisfairlyeasytoderiveformanygraphicalmodels
• GShasreasonablecomputationandmemoryrequirements(becausewesampleoner.v.atatime)
![Page 30: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/30.jpg)
GibbsSamplingAlgorithm
30
![Page 31: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/31.jpg)
GibbsSamplingExample
31
![Page 32: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/32.jpg)
ParallelGibbsSampling
32
x
VariableTally
CompleteModelCopies
• RunGibbsindependentlyonfullcopiesofthesamemodel
• Feweriterationspercopy• Moresamplesmeansmore
accuratemarginals
Datatomaterializefactorgraph
RunsequentialGibbs
![Page 33: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/33.jpg)
ParallelGibbsSamplingVariableAssignments
ColoredModel
Datatomaterializefactorgraph
VariableTally• Computeak-coloringofthe
factorgraph• Sampleallvariableswithsame
colorinparallel• Loadbalancingisakey
challenge
t1
t1
t1
CoordinatedWorkers
![Page 34: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms](https://reader033.vdocuments.net/reader033/viewer/2022043008/5f9658a3de01165f581de924/html5/thumbnails/34.jpg)
Summary
34
• Samplingcanbeeasytoimplementbutwecangetpoorqualitysamples• Weneedagoodproposaldistribution
• MarkovChainMonteCarlomethodsuseadaptiveproposalsQ(x’|x)tosamplefromthetruedistributionP(x)
• Metropolis-HastingsallowsyoutospecifyanyproposalQ(x’|x)
• GibbssamplingsetstheproposalQ(x’|x)totheconditionalP(x’|x)• Acceptancerateisalways1butthismeansslowexploration
• Burn-inisanart!