designing programs that check their work
DESCRIPTION
Designing Programs that Check Their Work. Manuel Blum Sampath Kannan. by Jeffrey Corbell. Overview. Introduction to a Program Checker Other Methods of Determining Correctness Definition of a Program Checker Example of a Checker: Graph Isomorphism Beigel’s Theorem. - PowerPoint PPT PresentationTRANSCRIPT
Designing Programsthat Check Their Work
Manuel BlumSampath Kannan
by Jeffrey Corbell
Overview
• Introduction to a Program Checker• Other Methods of Determining Correctness• Definition of a Program Checker• Example of a Checker: Graph Isomorphism• Beigel’s Theorem
What is a program checker
• Program that checks the output of a program to determine if the program is correct or buggy
Formally:– P and C are programs, I is the input– For any I run on P, C is run and determines
whether P is correct for I or buggy
Other Methods of Determining Correctness
• Program verification– Use a proof to prove a program is correct– Very difficult to do– Argued that it doesn't improve confidence in
correctness• very complex• may contain errors which would be difficult to detect
Other Methods of Determining Correctness
• Program testing– Run program on input that you know the correct
output for– Compare program output to expected output– Problems
• No general way to create test data• No theorems to describe behavior if they do pass tests
Differences Between a Checker and Testing
• A checker is a program that uses its own algorithm that allows it to check the output
• Program testing usually only uses a small amount of predetermined cases for specific input
Definition of a Bug
• Let π represent a decision or search problem• x represents an input to π with π(x)
representing the output• P is a deterministic program that supposedly
solves π
P has a bug if for some instance x of πP(x) ≠ π(x)
Definition of a Checker• Let Cπ be the checker, k be the number of
different cases the checker tries, and I be the group of test inputs
• CπP(I,k) is the output of the checker and
follows these conditions:1. If P(x) = π(x), then with probability ≥ 1- 1/2k
CπP(I,k) = CORRECT
2. If P(x) ≠ π(x), then with probability ≥ 1- 1/2k
CπP(I,k) = BUGGY
Definition of a Checker
• However, if P has bugs but P(I)=π(I) then Cπ
P(I,k) may output either CORRECT or BUGGY
Definition of a Checker
• Assumed P halts on all inputs• Not always the case• If P(x) exceeds a predetermined bound then
the checker should raise a flag, CπP(I,k) = TIME
Definition of a Checker
• Runtime includes the time it takes to submit input and receive output from P
• Does not include the time it takes P to run
Definition of a Checker
• If a checker is a program, how can you be sure the checker is correct?
• You can’t really• Checker must have the little oh property with
respect to the runtime of P– Ensures the checker is programmed differently
than the original program
Graph Isomorphism
b
e
a
c
d
1
2
4
3
5f (a) = 1f (b) = 2f (c) = 3f (d) = 4f (e) = 5
Graph Isomorphism Checker
• Let P be a program that solves graph isomorphism– Input: two graphs G and H– Output: YES if G is isomorphic to H; NO otherwise
• CGIP(G, H, k) checks P on input G and H
Graph Isomorphism Checker
• Compute P(G,H)• If P(G,H)=YES then
– Use P to search for an isomorphism from G to H– Check if the resulting correspondence is an
isomorphism– If not, return BUGGY; if yes, return CORRECT
Graph Isomorphism Checker
• If P(G,H)=NO then– Do k times:
• Toss a fair coin• If coin = heads then
– Generate a randompermutation G’ of G
– Compute P(G,G’)– If P(G,G’)=NO then
return BUGGY
• If coin = tails then– Generate a random
permutation H’ of H– Compute P(G,H’)– If P(G,H’)=YES then
return BUGGY
• Return CORRECT
Graph Isomorphism Checker
• CGIP runs in polynomial time
• If P has no bugs and G is isomorphic to H, then CGI
P(G,H,k) creates an isomorphism from G to H and outputs CORRECT
• If P has no bugs and G is not isomorphic to H, then CGI
P(G,H,k) tosses coins. It discovers P(G,G’)=YES for all G’ and P(G,H’) for all H’ so outputs CORRECT
Graph Isomorphism Checker• If P(G,H) is incorrect then there are two cases:
– If P(G,H)=YES but G is not isomorphic to H, then CGI
P fails to construct an isomorphism and outputs BUGGY
– If P(G,H)=NO but G is isomorphic to H, the only way that C will return CORRECT is if P(G,G’)= YES if the coin is heads and P(G,H’)= NO when it is tails. But G and H are permuted randomly to produce G’ and H’. Therefore P correctly distinguishes G’ from H’ only by chance for just 1 of 2k possible sequences
Beigel’s Theorem
• Let π1 and π2 be two polynomial-time equivalent decision problems. Then from any polynomial time checker for π1 it is possible to construct a polynomial-time checker for π2.
Beigel’s Theorem
• Have a checker Cπ1 for π1 and a program P2 for π2
• Also have two way polynomial time transformations f1,2 and f2,1
• This gives us a program for π1
– P1(x) =P2(f1,2(x))
Beigel’s Theorem
• To check P2 on an input y, compute P2(y) then transform into an input z for π1 using f2,1
• Then use Cπ1 to check z.
• Any call Cπ1 makes to P1 is transformed to a call to P2
P2 Cπ1
f1,2
f2,1y z
P1
Beigel’s Theorem
• If P2 is correct then P1 will be correct because P1 is defined in terms of P2
• Thus if P1 is correct on z then P2 is correct on y• If P2 is wrong on y and P1 is correct on z then
there’s a contradiction because P2(y)=P1(z)
• If P1 is wrong on z then the checker Cπ1 will catch it
Beigel’s Theorem
• This checker for π2 runs in polynomial time– Running the checker for π1
– One transformation of f2,1
– Polynomial number of applications of f1,2
Bibliography• Designing programs that check their work - M. Blum and S.
Kannan• Social Processes and Proofs of Theorems and Programs - R.A.
De Millo, R.J. Lipton, and A.J. Perlis.• Introduction to the Theory of Computation – M. Sipser• www.wikipedia.org