16.410 lecture 04: soundness and completeness of …...• conjecture: for any n, the algorithm...
TRANSCRIPT
7/2/11
1
1
Soundness and Completeness of State Space Search
Sertac Karaman 16.410‐13 Sept 20th, 2010
2
Assignments • Remember:
Problem Set #2: Uninformed search Out last Wednesday, Due this Wednesday, September 22nd
• Reading: – Today: Proofs and InducNon: Lecture 2 and 3 Notes of 6.042. – Wednesday: [AIMA] Ch. 6.1; 24.3‐5 Constraint SaNsfacNon.
• To learn more: Constraint Processing, by Rina Dechter – Chapter 2: Constraint Networks – Chapter 3: Consistency Enforcing and PropagaNon
9/20/1
AutonomousSystems:
• PlancomplexsequencesofacVons
• ScheduleVghtresources• Monitoranddiagnosebehavior
• Repairorreconfigurehardware. formulateasstatespacesearch.
BrianWilliams,Fall10 3
FormalizingGraphSearch
BrianWilliams,Fall10 4
C
S
B
G A
D
Input:AsearchproblemSP=<g,S,G>where• graphg=<V,E>,• startvertexSinV,and• goalvertexGinV.
Output:AsimplepathP=<S,v2,…G>ingfromStoG.(i.e.,<vi,v >∈E,andv ifi≠j).i+1 i≠vj
0
2
9/20/10
3
BrianWilliams,Fall10 5
S
B
G
A
<S>
<B,S><A,S>
<G,B,S><A,B,S>
GraphSearchisaKindofStateSpaceSearch
S
BA
GB
GraphSearchisaKindOfTreeSearch
BrianWilliams,Fall10 6
SoluVon:DepthFirstSearch(DFS)S
D
B A
C G
C G
D
C G
S
D
B A
C G
C G
D
C G
SoluVon:BreadthFirstSearch(BFS)
9/20/10
4
BrianWilliams,Fall10 7
SoluVon:DepthFirstSearch(DFS)S
D
B A
C G
C G
D
C G
S
D
B A
C G
C G
D
C G
SoluVon:BreadthFirstSearch(BFS)
Depth-first:
Add path extensions to front of Q
Pick first element of Q
Breadth-first:
Add path extensions to back of Q
Pick first element of Q
BrianWilliams,Fall10 8
TheWorstofTheWorst
Search Method
Worst Time
Worst Space
Shortest Path?
Guaranteed to find path?
Depth-first ~bm b*m No Yes for finite graph
Breadth-first ~bm bm Yes unit lngth Yes
Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q
S
D
B A
C G
C G
D
C G
Whichisbeler,depth‐firstorbreadth‐first?
C
S B
G A
D
• Assumed=mintheworstcase,andcallbothm.
• Best‐firstcan’texpandtolevelm+1,justm.
9/20/10
5
ElementsofAlgorithmDesignDescripVon:(lastMonday)
– Problemstatement.– Stylizedpseudocode,sufficienttoanalyzeandimplementthe
algorithm.– ImplementaVon(lastWednesday).
Analysis:(lastWednesday)• Performance:
– Timecomplexity:• howlongdoesittaketofindasoluVon?
– Spacecomplexity:• howmuchmemorydoesitneedtoperformsearch?
• Correctness:(today)– Soundness:
• whenasoluVonisreturned,isitguaranteedtobecorrect?– Completeness:
• isthealgorithmguaranteedtofindasoluVonwhenthereisone?
BrianWilliams,Fall10 9
Outline
• Review
• ProoftechniquesandtheaxiomaVcmethod
• Proofsofsoundnessandcompletenessofsearchalgorithms
• LimitsofaxiomaVcmethod
BrianWilliams,Fall10 10
9/20/10
6
EnvelopegameProbabili(es do not work!
• Iputanamount$Nand$2Nintotwodifferentenvelopes(you do not know N).
• Iopenoneofthem,ithas$X.
• Wouldyoupicktheopenoneortheother?
• Reasoning 1:(I pick one at random) – seeinginsideanenvelopedoesnotmaler…
• Reasoning 2: (I pick the second one) – IfIgetthisenvelope,Iget$X.– IfIgettheotherenvelope,Iget,onaverage:
(1/2)X/2+(1/2)2X=(5/4)X
UnexpectedhangingparadoxInduc(on does not work!
• Ajudgetellsacriminalthat“the criminal will be hanged on a weekday at noon next week, he will not know when he will be hanged, it will be a total surprise”.
• Criminal’sreasoning:– HecannotbehangedonaFriday(byThursdayawernoon,he
willknow–itwon’tbeasurprise).– Then,hecannotbehangedonThursdayeither.– Then,hecannotbehangedatall…Sohefeelssafe.
(HewashangedonWed.atnoon–itwasatotalsurprise…)
• Whatwentwrongwithcriminal’sdeducVon?
9/20/10
7
TheaxiomaVcmethod
• InventedbyEuclidaround300BC(inAlexandria,Egypt).
• 5axioms ofgeometrymenVonedinhisworkElements.
• StarVngfromtheseaxioms,Euclidestablishedmany“proposi2ons”byproviding“proofs”.
EuclidofAlexandria
OldestsurvivingcopyoftheElements onapapyrusfoundinOxyrhynchus, Egypt. EuclidstatuteinOxford
TheaxiomaVcmethod
• AdefiniVonofa“proof”:• Anysequenceoflogical deduc(onsfromaxioms andpreviouslyprovenproposi(ons/statements thatconcludeswiththeproposiVoninquesVon.
• Therearemanytypesof“proposi2ons”:• Theorem:Importantresults,mainresults• Lemma:apreliminaryproposiVonforproving
laterresults• Corollary:Aneasy(butimportant)conclusion,
anawerthoughtofatheorem.
EuclidofAlexandria
EuclidstatuteinOxford
Images are in the public domain.
9/20/10
8
TheaxiomaVcmethod
• Euclid’saxiom‐proofapproachisnowfundamentaltomathemaVcs!
• Amazingly,essenVallyallmathemaVcscanbederivedfromjustahandfulofaxioms…
• Howtoevenstartaproof?• Therearemany“templates”(outlines,ortechniques)
• Thedetailsdiffer…
EuclidofAlexandria
EuclidstatuteinOxford
Provinganimplica2on
• SeveralmathemaVcalclaimsareoftheform:• “If P, then Q”,orequivalently“P implies Q”.
• QuadraJcs:• Ifand,2 b
"ax + bx + c = 0 a = 0 thenx =
! ±
• InequaliJes:• If,then
• Goldbach’sConjecture:• Ifn>2,thennisasumoftwoprimes.
b2 ! 4ac
2a
0 ! x ! 2 !x3 + 4x + 1 > 0
!
9/20/10
9
ProvingimplicaVons:Simplestprooftechnique
• Toprove“PimpliesQ”,– “AssumeP”andshowthatQlogicallyfollows.
• Theorem:– If,then0 ! x ! 2 !x3 + 4x + 1 > 0
• Proof:– Assume(P)0 ! x ! 2– Then,,,andarx 2! x x + 2 eallnon‐negaVve– Then,isnon‐negx(2! x)(2 + x) aVve– Then,isnon‐negx(2! x)(2 + x) + 1 aVve– Then,mulVplyingoutthelewsidegives
• (Q!x3 + 4x + 1 > 0 )
ProofbyContradicVon
• ToprovethatastatementPisTrue.– Assumethatitisnot.– Showthatsomeabsurd(clearlyfalse)statementfollows.
• Formalized:InordertoproveastatementPisTrue – AssumethatPisFalse,– DeducealogicalcontradicVon(negaVonofapreviousstatementoranaxiom).
9/20/10
10
ProofbyContradicVon
• Theorem:isanirr!
aVonalnumber.• Proof:
• AssumethatisnotirraVonal.• Then,isaraVonalnumberandcanbewrilenas=a/bwherea andb areintegersandfrac(on is in lowest terms
• Then,squaringbothsidesandrearranginggives2=a2/b2• Then,a mustbeeven• Then,a2mustbeamulVpleof4• Then,2b2mustalsobeamulVpleof4• Then,bisalsoeven• Then,thefracVonisnotinthelowestterms(sincea andbarebotheven)
2!
2!2
!2
ProofbyInducVon
PickparameterNtodefineproblemsize.• NumberofedgesinthesoluVon.• Numberofnodesingraph.
BaseStep:N=0(orsmallN)• ProvepropertyforN=0
InducJonStep:• AssumepropertyholdsforN• ProvepropertyforN+1
• Conclusion:propertyholdsforallproblemsizesN.20
9/20/10
11
ProofbyInducVonformalized
• LetP(i)beastatementwithparameteri.
• ProofbyinducVonstatesthefollowingimplicaVon:• “P(0)isTrue”(1)and“P(i)impliesP(i+1)”(2)• (1)and(2)implies“P(i)isTrueforalli”.
• InducVonisoneofthecore principles ofmathemaVcs.
• Itisgenerallytakenasanaxiom,ortheaxiomsaredesignedsothatinducVonprinciplecanbeproven.
21
AninducVonexample
• Theorem:fn(n + 1)1 + 2 + · · · + n = oralln.
• Proof:– Base case:P(0)isTrue.
• Because,0=0.– InducVonstep:P(n)impliesP(n+1)
• Assumethatthehypothesisholdsforn. • Forn+1:
2
n(n + 1)1 + 2 + · · · + n + (n + 1) = + n + 1
2(n + 1)(n + 2)
=2
9/20/10
12
AfaultyinducVon
• Theorem[!]:Allhorsesarethesamecolor.• Proof[!]:
– Base case:P(1)isTrue.• because,thereisonlyonehorse.
– Induc(on step:P(i)impliesP(i+1).• AssumethatP(i)isTrue.• Bytheinduc(on hypothesisfirstihorsesarethesamecolor,andthelastihorsesarealsothesamecolor.
h1, h2, . . . , hi,hi+1 h1, h2, . . . , hi, hi+1
• Soallthei+1horsesmustbethesamecolor.• Hence,P(i+1)isalsoTrue.
• Whatwentwronghere?
ProofbyInvarianceA common technique in algorithm analysis • Showthatacertainpropertyholdsthroughoutinanalgorithm.
• AssumethatthepropertyholdsiniVally.• Showthatinanystepthatthealgorithmtakes,thepropertysVllholds.
• Then,propertyholdsforever.
• ItisasimpleapplicaVonofinducVon.Why?
9/20/10
13
ProvingstatementsaboutalgorithmsHandle with care!
• Correctnessofsimplestalgorithmsmaybeveryhardtoprove…
• Collatzconjecture:• Algorithm(HalfOrTriplePlusOne‐HOTPO):
• Givenanintegern. 1. Ifniseven,thenn=n/22. Ifnisodd,thenn=3n+13. Ifn=1,thenterminate,elsegotostep1.
• Conjecture:Foranyn,thealgorithmalwaysterminates(withn=1).
ProvingstatementsaboutalgorithmsHandle with care!
Collatzconjecture:• Firstproposedin1937.
• Itisnot knownwhethertheconjectureistrueorfalse.
First1000numbers
PaulErdős(1913‐1996) ‐famousnumbertheorist–
“Mathema(cs is not yet ready for such problems”, 1985.
First100numbers
Images are in the public domain. Images by Keenan Pepper and Jon McLoone.
9/20/10
14
SoundnessandCompletenessofSearchAlgorithms
• Today:
• provestatementsaboutthesearchalgorithmswehavestudiedintheclass.
• studywhetherthealgorithmreturnsacorrectsoluVon.
• studywhetherthealgorithmreturnsasoluVonatallwhenoneexists.
SoundnessandCompleteness
Givenaproblem PR,analgorithm thataYemptstosolvethisproblemmayhavethefollowingproperJes:
Soundness: • ThesoluVonreturnedbythealgorithmiscorrect.
Completeness:• ThealgorithmalwaysreturnsasoluVon,ifoneexists.• IfthereisnosoluVon,thealgorithmreportsfailure.
Also,OpJmality:• ThealgorithmreturnstheopVmalsoluVon,ifitreturnsone.
28
9/20/10
15
29
SomeOtherNoVonsofSoundnessandCompleteness
ProbabilisJcCompleteness:• ThealgorithmreturnsasoluVon,ifoneexists,withprobability
approachingtooneasthenumberofiteraVonsincreases.• IfthereisnosoluVon,itmayrunforforever.
ProbabilisJcSoundness:• Theprobabilitythatthe“soluVon”reportedsolvesthe
problemapproachesone,asthenumberofiteraVonsincreases.
AsymptoJcOpJmality:• Thealgorithmdoesnotnecessarilyreturnanop(mal soluVon,
butthecostofthesoluVonreportedapproachestheopVmalasthenumberofiteraVonsincreases.
BrianWilliams,Fall10 30
Problem:StateSpaceSearch
C
S
B
G A
D
Input:AsearchproblemS=<g,S,G>where• graphg=<V,E>,• startvertexSinV,and• goalvertexGinV.
Output:AsimplepathP=<S,v2,…G>ingfromStoG.
9/20/10
16
PseudoCodeForSimpleSearchLet g be a Graph G be the Goal vertex of g. S be the start vertex of g Q be a list of simple partial paths in GR,
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2. 31
SoundnessandCompletenessTheorems
Wewouldliketoprovethefollowingtwotheorems:
Theorem1(Soundness):Simplesearchalgorithmissound.
Theorem2(Completeness):Simplesearchalgorithmiscomplete.
Wewilluseablendofprooftechniquesforprovingthem.
32
9/20/10
SoundnessandCompletenessTheorems
Theorem1(Soundness):Simplesearchalgorithmissound.
Letusprove3lemmasbeforeprovingthistheorem.
33
Alemmatowardstheproof
• Lemma1:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenvk=S.
• Proof:(byinvariance)– Base case:IniVally,thereisonly<S>inthequeue.Hence,theinvariant holds.
– Induc(on step:Let’scheckthattheinvariant conVnues toholdineverystepofthealgorithm.
17
9/20/10
18
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenJme,thenvk=S.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
35
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenJme,thenvk=S.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 36
9/20/10
19
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenJme,thenvk=S.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 37
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenJme,thenvk=S.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue(apathisremoved) 38
9/20/10
20
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenJme,thenvk=S.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Severalpathsadded,eachsaJsfytheinvariantsinceNsaJsfiesit. 39
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenJme,thenvk=S.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 40
9/20/10
21
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenJme,thenvk=S.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 41
Anotherlemmatowardstheproof
• DefiniJon:Apath<v0,v1,…,vk>isvalidiff(vi!1, vi) ! E oralli ! {1, 2, . . . , k}
• Lemma2:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisvalid.
• Proof:(byinvariance)– Base case:IniVallythereisonlyonepath<S>,whichisvalid.Hence,theinvariant holds.
– Induc(on step:Let’scheckthattheinvariant conVnues toholdineverystepofthealgorithm.
9/20/10
22
PseudoCodeForSimpleSearchInvariant: If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisvalid
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
43
PseudoCodeForSimpleSearchInvariant: If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisvalid
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 44
9/20/10
23
PseudoCodeForSimpleSearchInvariant: If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisvalid
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 45
PseudoCodeForSimpleSearchInvariant: If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisvalid
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 46
9/20/10
24
PseudoCodeForSimpleSearchInvariant: If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisvalid
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 47
PseudoCodeForSimpleSearchInvariant: If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisvalid
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Notethatvalidityholdsforallnewlyaddedpath(fromLine4.b) 48
9/20/10
25
PseudoCodeForSimpleSearchInvariant: If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisvalid
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 49
PseudoCodeForSimpleSearchInvariant: If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisvalid
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 50
9/20/10
26
Yetanotherlemmatowardstheproof
• Lemma3:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath(containsnocycles).
• Proof:(byinvariance)– Base case:IniVally,thereisonly<S>inthequeue.Hence,theinvariant holds.
– Induc(on step:Let’scheckthattheinvariant conVnues toholdineverystepofthealgorithm.
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
52
9/20/10
27
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 53
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 54
9/20/10
28
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 55
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 56
9/20/10
29
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
WewouldliketoshowthateachnewlyaddedpathissimpleassumingNissimple.Proof: (by contradic2on) Assume one path is not simple. Then, a children of head(N) appears in N. But, this is contradicts Line 4.b 57
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 58
9/20/10
30
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
Before this line: assumethatinvariantholds.AIer this line: showthatinvariantissJlltrue.
Inthiscasenonewpathisaddedtothequeue. 59
SoundnessandCompletenessTheorems
Theorem1(Soundness):Simplesearchalgorithmissound.
Proof:bycontradicVon…
60
9/20/10
31
ProofofSoundness
Assumethatthesearchalgorithmisnotsound:Letthereturnedpathbe < v0, v1, . . . , vk >
Then,oneofthefollowingmustbeTrue:• 1.ReturnedpathdoesnotstartwithS:
vk = S
• 2.ReturnedpathdoesnotcontainGathead:v0 = G
• 3.SometransiVoninthereturnedpathisnotvalid:(vi!1, vi) !/ E forsome i ! {1, 2, . . . , vk}
• 4.Returnedpathisnotsimple:fvi = vj orsomewithi, j ! {0, 1, . . . , k} i = j
61
ProofofSoundness
• 1.ReturnedpathdoesnotstartwithS: vk = S
• But,thiscontradictsLemma1!
• Lemma1:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenvk=S.
62
!
!
!
!
9/20/10
32
ProofofSoundness
• 2.ReturnedpathdoesnotcontainGathead:v0 = G
• Butclearly,thereturnedpathhasthepropertythatHead(N)=G
• Recallthepseudocode:
63
PseudoCodeForSimpleSearchInvariant:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath.
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2.
64
!
9/20/10
33
ProofofSoundness
• 3.SometransiVoninthereturnedpathisnotvalid:f(vi!1, vi) !/ E orsome i ! {1, 2, . . . , vk}
• ContradictsLemma2!
• Lemma2:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisvalid.
65
ProofofSoundness
• 4.Returnedpathisnotsimple:fvi = vj orsomewithi, j ! {0, 1, . . . , k} i = j
• ContradictsLemma3!
• Lemma3:If<v1,v2,…,vk>isapathinthequeueatanygivenVme,thenitisasimplepath(containsnocycles).
66
!
9/20/10
34
ProofofSoundness
Assumethatthesearchalgorithmisnotsound:Letthereturnedpathbe < v0, v1, . . . , vk >
Then,oneofthefollowingmustbeTrue:• 1.ReturnedpathdoesnotstartwithS:
vk = S
• 2.ReturnedpathdoesnotcontainGathead:v0 = G
• 3.SometransiVoninthereturnedpathisnotvalid:(vi!1, vi) !/ E forsome i ! {1, 2, . . . , vk}
• 4.Returnedpathisnotsimple:fvi = vj orsomewithi, j ! {0, 1, . . . , k} i = j
67
ProofofSoundness
Assumethatthesearchalgorithmisnotsound:
WereachacontradicVoninallcases.
Hence,thesimplesearchalgorithmissound.
!
!
!
9/20/10
35
ProofofCompleteness
Theorem2(Completeness):Simplesearchalgorithmiscomplete.
Needtoprove:
• IfthereisapathtoreachfromStoG,thenthealgorithmreturnsonepaththatdoesso.
69
PseudoCodeForSimpleSearchLet g be a Graph G be the Goal vertex of g. S be the start vertex of g Q be a list of simple partial paths in GR,
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2. 70
9/20/10
36
Acommontechniqueinanalysisofalgorithms
• Let’sslightlymodifythealgorithm
• Wewillanalyzethemodifiedalgorithm.
• Then,“project”ourresultstotheoriginalalgorithm.
PseudoCodeForSimpleSearchLet g be a Graph G be the Goal vertex of g. S be the start vertex of g Q be a list of simple partial paths in GR,
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. // If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2. 72
9/20/10
37
ProofofCompleteness
• Themodifiedalgorithmterminateswhenthequeueisempty.
• Letusproveafewlemmasregardingthebehaviorofthemodifiedalgorithm
ProofofCompleteness
• Lemma1:Apaththatistakenoutofthequeueisnotplacedintothequeueagainatalaterstep.
• Proof:(using logical deduc(on) • Another way to state this:“Ifp=<v0,v1,…,vk>isapath
thatistakenoutofthequeue,thenp=<v0,v1,…,vk>isnotplacedintothequeueatalaterstep.”
• Assumethatp=<v0,v1,…,vk>istakenoutofthequeue.
• Then,pmustbeplacedintothequeueatanearlierstep.
• Then,v0mustbeinthevisitedlistatthisstep.• Then,p=<v0,v1,…,vk>cannotplacedintothequeue
atalaterstep,sincev0isinthevisitedlist.
9/20/10
38
ProofofCompleteness
• DefiniJon:Avertexvisreachable fromS,ifthereexistsapath<v0,v1,…,vk>thatstartsfromSandendsatv,i.e.,vk=Sandv0=v.
• Lemma2:IfavertexvisreachablefromS,thenvisplacedintothevisitedlistawerafinitenumberofsteps.
ProofofCompleteness• Lemma2:IfavertexvisreachablefromS,thenvisplacedintothevisitedlist
awerafinitenumberofsteps.• Proof:(by contradic(on)
• AssumevisreachablefromS,but itisneverplacedonthevisitedlist.• SincevisreachablefromS,thereexistsapaththatisoftheform<v0,v1,…,
vk>,wherev0=vandvk=S.• Letvibethefirstnode(starVngfromvk)inthechainthatisneveraddedto
thevisitedlist.• (1)Notethatviwasnotinthevisitedlistbeforethisstep.• (2)Notealsothat(vi+1,vi)isinE.• Sincevi+1wasinthevisitedlist,thequeueincludedapath<vi+1,…,vk>(not
necessarilythesameasabove),wherevk=S.• Thispathmusthavebeenpoppedfromthequeue,sincethereareonly
finitelymanydifferentparValpathsandnopathisaddedtwice(byLemma1)andviwasnotinthevisitedlist(seestatement1above).
• Sinceitispoppedfromthequeue,then<vi+1,vi,…,vk>mustbeplacedintoqueue(seestatement2above)andviplacedintothevisitedlist
• Redstatementscontradict!
9/20/10
39
ProofofCompleteness
• Lemma2:IfavertexvisreachablefromS,thenvisplacedintothevisitedlistawerafinitenumberofsteps.
• Corollary:Inthemodifiedalgorithm,Gisplacedintothevisitedqueue.
• “Project” back to the original algorithm: • Thisisexactlywhentheoriginalalgorithmterminates
ProofofCompleteness
Theorem2(Completeness):Simplesearchalgorithmiscomplete.
• Proof:FollowsfromLemma2evaluatedintheoriginalalgorithm.
78
9/20/10
40
PseudoCodeForSimpleSearchLet g be a Graph G be the Goal vertex of g. S be the start vertex of g Q be a list of simple partial paths in GR,
1. Initialize Q with partial path (S) as only entry; set Visited = ( ); 2. If Q is empty, fail. Else, pick some partial path N from Q; 3. If head(N) = G, return N; (goal reached!) 4. Else
a) Remove N from Q; b) Find all children of head(N) (its neighbors in g) not in Visited
and create a one-step extension of N to each child; c) Add to Q all the extended paths; d) Add children of head(N) to Visited; e) Go to step 2. 79
SummarizeCompletenessandSoundness
• Hence,wehaveproventwotheorems:
Theorem1(Soundness):Simplesearchalgorithmissound.
Theorem2(Completeness):Simplesearchalgorithmiscomplete.
• Soundnessandcompletenessisarequirementformostalgorithms,althoughwewilltheirrelaxaVonsquiteowen
9/20/10
41
BacktotheAxiomaVcMethodDoes it really work?
• EssenVallyallofwhatweknowinmathemaVcstodaycanbederivedfromahandfulofaxiomscalledtheZarmelo‐FrankelsettheorywiththeaxiomofChoice(ZFC).
• TheseaxiomsweremadeupbyZarmelo(theydidnotexista priori,unlikephysicalphenomena).
• Wedonotknowwhethertheseaxiomsarelogically consistent!– Soundscrazy!But,happenedbefore…
Around 1900, B. Russell discovered that the axioms of that (me were logically inconsistent, i.e., one could prove a contradic(on.
BacktotheAxiomaVcMethodDoes it really work?
• ZFCaxiomsgivesonewhatshe/hewants:– Theorem:5+5=10.
• However,absurdstatementscanalsobedriven:– Theorem(Banach‐Tarski):Aballcanbecutintoafinitenumberofpiecesandthenthepiecescanberearrangedtobuildtwoballsofthesamesizeoftheoriginal.
Clearly, this contradicts our geometric intui(on! Image by Benjamin D. Esham, in the public domain.
9/20/10
42
BacktotheAxiomaVcMethodDoes it really work?
BacktotheAxiomaVcMethodDoes it really work?
Onthefundamentallimitsofmathema2cs • Godelshowedin1930thattherearesomeproposiVonsthat
aretrue,butdonotlogicallyfollowfromtheaxioms.
• Theaxiomsarenotenough!
• But,Godelalsoshowedthatsimplyaddingmoreaxiomsdoesnoteliminatethisproblem.Anysetofaxiomsthatis
notcontradictorywillhavethesameproblem!
• Godel’sresultsaredirectlyrelatedtocomputaVon.TheseresultswerelaterusedbyAlanTuringin1950stoinventarevoluVonaryidea:computer…
Images of Godel, Turing, and Einstein removed due to copyright restrictions.
9/20/10
Whatyoushouldknow
• ThedefiniVonsofaproposiVon,proof,theorem,lemma,andcorollary.
• ProoftechniquessuchasproofbycontradicVon,inducVon,invarianceproofs.
• NoVonsofsoundnessandcompleteness.• Proving soundness and completeness of search algorithms.
43
MIT OpenCourseWare http://ocw.mit.edu
16.410 / 16.413 Principles of Autonomy and Decision Making Fall 2010
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.