proof translation and smt lib certification
DESCRIPTION
Proof translation and SMT LIB certification. Yeting Ge Clark Barrett SMT 2008 July 7 Princeton. SMT solvers are more complicated. CVC3 contains over 100,000 lines of code Are SMT solvers correct?. Quest for correct SMT solvers?. To verify a SMT solver is correct? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/1.jpg)
PROOF TRANSLATION AND SMT LIB CERTIFICATION
Yeting Ge Clark Barrett
SMT 2008
July 7 Princeton
![Page 2: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/2.jpg)
SMT solvers are more complicated
CVC3 contains over 100,000 lines of code Are SMT solvers correct?
![Page 3: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/3.jpg)
Quest for correct SMT solvers?
To verify a SMT solver is correct? To develop a correct SMT solver?
![Page 4: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/4.jpg)
Good news: we have proofs
Some SMT solvers could produce proofs Proof checking should be easier than
proving the correctness of a SMT solver A proof could be represented as a proof
tree
1| ba 1| b
0| a
![Page 5: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/5.jpg)
Bad news: Proof checking for SMT solvers is not so easy
Theory proof rules require the proof checker to have theory reasoning ability a/2 = b
Choice of proof rules A small set of simple proof rules?
Good for proof checking Large set of complex proof rules?
Good for performance (CVC3 has 298 rules) The correctness of the proof checker becomes
questionable SMT solvers are in constant change
![Page 6: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/6.jpg)
The idea
Use a second prover to check the proof Translate the proof into the second prover The benefits
Could easily handle both simple and complex proof rules Flexible
The challenges A suitable second prover
The correctness is reduced to the second prover Efficiency Translation
This is feasible!
![Page 7: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/7.jpg)
SMT LIB certification
SMT LIB A collection of over 40,000 SMT benchmarks,
most of which from industry applications Each file contains a status field
Some files are incorrectly labeled The proof in the second prover is a certificate A certified SMT LIB will be beneficial to SMT
community Prove as many unsatisfiable cases as possible
(benchmark tmp:source {piVC} :status unsat :category { industrial } :difficulty { 0 } :logic AUFLIA :extrafuns ((V_6 Int))
![Page 8: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/8.jpg)
CVC3
A proof is a tree A proof rule maps a set of proofs to a
proof
Some proof rules are rather complex
![Page 9: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/9.jpg)
The second prover: HOL Light
Simple The core:
430 lines of Ocaml, 10 inference rules, 3 axioms Definitional extension guarantees
correctness Except equality, all logic symbols are defined
All proofs in HOL Light can be broken down into the 10 rules and 3 axioms, if needed
“it sets a very exacting standard of correctness” Efforts to verify the correctness of the core
![Page 10: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/10.jpg)
HOL Light
Powerful Capable of formalizing most mathematics (up to
axiom of choice) Flexible
Programmable Ocaml as meta-language
A number of built-in theories Reals, integers
A lot of useful tools Decision procedures for first-order logic, propositional
logic Decision procedures for reals, integers, …
![Page 11: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/11.jpg)
Translation of terms
HOL Light and CVC3 are connected through C API functions of CVC3
distinct(x1,x2,…,xn) Define a predicate on the fly
Mixed integers and reals Lift to reals
Skolem constant
Choice operator (@x.P)
)()(. skoPxPx
![Page 12: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/12.jpg)
Translation of proof rules
An Ocaml function for each proof rule Naïve method
call HOL Light’s decision procedure Exploit HOL Light’s capability of higher
order reasoning Prove a meta-theorem off-line During the translation, instantiate the meta-
theorem Engineering the translation of a proof rule
![Page 13: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/13.jpg)
Propositional reasoning
SAT solvers can dump a resolution proof
Sequent representation
Definitional CNF and ITE
hole5 Time(s)
Try 1 255
Try 2 155
Seq 37
Sorted
2.8
![Page 14: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/14.jpg)
Results
catetory cases CVC3 Translation
proved Ave time proved Ave time
simplify1 833 833 0.98 833 19.51
Simplify2 2329 2306 1.11 2164 8.85
burns 14 14 0.02 14 1.38
ricart 14 13 0.07 13 17.60
piVc 41 41 0.12 41 1.45
Hard cases
CVC3 Translation
No Prep 5 47.25 5 41.49
With Prep 4 48.91 4 64.27
Hard cases in simplify1: CVC3 spent more than 20 seconds
![Page 15: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/15.jpg)
Results
Found one proof rule that does not preserve validity in CVC3
Found one faulty proof rule in CVC3 Found two mis-labled SMT LIB cases in
AUFLIA
![Page 16: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/16.jpg)
Discussion
Instantiating a meta-theorem in HOL Light is almost like rewriting
Most proof rules can be converted into some meta-theorem
Other methods to improve efficiency Compiling
HOL Light
![Page 17: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/17.jpg)
Conclusion
It is feasible to translate proofs from CVC3 into HOL Light
It is possible to certify many SMT LIB cases in HOL Light
![Page 18: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/18.jpg)
Future works
Prove more SMT LIB cases Improve the translation of arithmetic
proof rules Support more proof rules Support more theories Improve the proof rules of CVC3
![Page 19: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/19.jpg)
Thanks
John Harrison for help with HOL Ligh Sean McLaughlin for writing the first
version of the translator
![Page 20: Proof translation and SMT LIB certification](https://reader034.vdocuments.net/reader034/viewer/2022051114/56813a7e550346895da277bb/html5/thumbnails/20.jpg)
Reference
C. Barrett and C. Tinelli. CVC3. In W. Damm and H. Hermanns, editors, Proceedings of the 19th International Conference on Computer Aided Verification (CAV ’07), LNCS 4590, pages 298–302. Springer-Verlag, July 2007. Berlin, Germany.
J. Harrison. Hol light: A tutorial introduction. In M. K. Srivas and A. J.Camilleri, editors, FMCAD, LNCS 1166, pages 265–269. Springer, 1996.
S. McLaughlin, C. Barrett, and Y. Ge. Cooperating theorem provers: A case study combining HOL-Light and CVC Lite. In A. Armando and A. Cimatti, editors, Proceedings of the 3rd Workshop on Pragmatics of Decision Procedures in Automated Reasoning (PDPAR ’05), volume 144(2) of Electronic Notes in Theoretical Computer Science, pages 43–51. Elsevier, Jan. 2006. Edinburgh, Scotland.
M. Moskal. Rocket-fast proof checking for smt solvers. In K. Jesen and A. Podelski, editors, TACAS, LNCS 4963, pages 486–500. Springer, 2008.
T. Weber. Efficiently checking propositional resolution proofs in isabelle/hol. volume 212 of CEUR Workshop Proceedings, 2006.