introduction to formal property verification (fpv) erik seligman cs 510, lecture 8, january 2009
TRANSCRIPT
Introduction to Formal Property Verification (FPV)
Erik Seligman
CS 510, Lecture 8, January 2009
Agenda
Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints
Agenda
Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints
Definitions
• Assertion • Statement that must be true in all cases.
• Assumption • Assertion treated as always-true constraint for FPV.
• Cover Point• Condition that must be reachable for valid proof env
• Formal Property Verification (FPV)• Mathematical proofs, not simulation
• Proves assertions: all possible test vectors
Simulation: spot coverage of design space
Motivation for Formal Property Verification
Formal Property Verification (ideal case): full coverage of design space
Simulation: spot coverage of design space
Motivation for Formal Property Verification
Formal Property Verification (ideal case): full coverage of design space
Simulation: spot coverage of design space
Motivation for Formal Property Verification
Formal Property Verification (real life): full coverage in some areas
Major Benefits of FPV for ASIC Projects
Improving Design Process
Major Benefits of FPV for ASIC Projects
Improving Design Process• Force Designer to Think Through Logic
Major Benefits of FPV for ASIC Projects
Improving Design Process• Force Designer to Think Through Logic
• Help Identify Hidden Assumptions
Major Benefits of FPV for ASIC Projects
Improving Design Process• Force Designer to Think Through Logic
• Help Identify Hidden Assumptions
Bug Hunting
Major Benefits of FPV for ASIC Projects
Improving Design Process• Force Designer to Think Through Logic
• Help Identify Hidden Assumptions
Bug Hunting• Unit-Level Validation (before testbench)
Major Benefits of FPV for ASIC Projects
Improving Design Process• Force Designer to Think Through Logic
• Help Identify Hidden Assumptions
Bug Hunting• Unit-Level Validation (before testbench)
• Find Corner Cases Missed in Simulation
Major Benefits of FPV for ASIC Projects
Improving Design Process• Force Designer to Think Through Logic
• Help Identify Hidden Assumptions
Bug Hunting• Unit-Level Validation (before testbench)
• Find Corner Cases Missed in Simulation
• Quickly Verify Design Changes
Major Benefits of FPV for ASIC Projects
Improving Design Process• Force Designer to Think Through Logic
• Help Identify Hidden Assumptions
Bug Hunting• Unit-Level Validation (before testbench)
• Find Corner Cases Missed in Simulation
• Quickly Verify Design Changes
“Peace of Mind”
Useful Assertions for FPV Focus on high-level intent
• Assertions = “executable comments”• Add insight to design
– Micro-assert on a couple of RTL lines less usefulassign foo = bar & baz;A1: assert property (foo == bar & baz);
Don’t be afraid of some modeling code• Auxiliary calculations / wires are fine
– Provide `ifdef to exclude from synthesis• Full reference models in areas of concern• Smaller “shadow models” often very useful
Agenda
Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints
Prerequisites for FPV
RTL design with assertions Clocks must be identified
• Critical since FPV runs over time
• Clocks are ‘special’: driven 1/0/1/0/…
• Need explicit ratios if multiple clocks
Reset pattern must be identified• FPV resets model to known state at start
• Common method: single rst signal (easy)
• More complex design may have reset sequence– Hold RST 10 cycles, then PowerGood for 5 cycles, …
FPV Run Verilog RTL with
Assertions
FPV
FPV Run Verilog RTL with
Assertions
Passing Assertions
FPV
FPV Run Verilog RTL with
Assertions
Passing Assertions
Bounded-Passing
Assertions
FPV
Bounded vs Full Proofs Which of these do you need?
• “Assertion can NEVER be violated”
• “Assertion can never be violated by any possible simulation of length up to <n>”
Bounded proof usually easier for tools
• Use cover point proofs to judge good bound
• Bound == lengths of interesting scenarios
• Some coverage lost vs full proofs
– But often at point of diminishing ROI
Consider modifying starting state
• Fill queue at start of proof…?
FPV Run Verilog RTL with
Assertions
Passing Assertions
Bounded-Passing
Assertions
FPV
Failing Assertions
FPV Run Verilog RTL with
Assertions
Passing Assertions
Bounded-Passing
Assertions
FPV
Failing Assertions
Unknown Assertions
FPV Debug Loop
Verilog RTL with Assertions
Passing Assertions
Bounded-Passing
Assertions
FPV
Failing Assertions
Unknown Assertions
Analyze Failures: RTL error, assertion error, or assumption
needed?
Edit RTL: Fix bugs, add
assumptions
FPV Debug Loop
Verilog RTL with Assertions
Passing Assertions
Bounded-Passing
Assertions
FPV
Failing Assertions
Unknown Assertions
Analyze Failures: RTL error, assertion error, or assumption
needed?
Edit RTL: Fix bugs, add
assumptions
• This is where FPVers spend their time!
Cover Points and FPV Cover Point is opposite of assertion
• “Good”: FPV creates example trace
• “Bad”: FPV proves point unreachable
• Also may aid simulation coverage checks
Examplecover property (opcode == `ADD);
• If it passes, FPV reports trace with ADD op
• If it fails, ADD op cannot exist in FPV env– Maybe bad assumption prevents ADD op
– All proofs are suspect unless this was expected
Add cover points when doing FPV!• Good tools auto-generate some
Agenda
Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints
Running Jasper
Just run ‘JG’ from command line• Can use –batch to run without GUI
Runs are automatically logged• See jgproject/jg.log
GUI hints• Right-clicking usually gets useful options
• Pass over button with cursor for name
Basic Jasper Command File# Load into JG using "source <file>.tcl".# load modelanalyze -clearanalyze -sva traffic_start.velaborate -top traffic # set clocks and resetsclock -clearclock clkreset -clearreset rst# Set engine mode & run proofsset_engine_mode {H D B3 H2}prove -all
Proof Results
View Violation Trace
• How would this work for liveness properties, like a |-> ##[0:$] b?
Violation Trace: “Why”?
Violation Trace: Why? Again
Violation Trace for Liveness
• Trace shows possible infinite violation loop
Alternate JG Debug Tool: The Visualizer
Using The Visualizer
Visualizing Constraints
Visualize Options
Replot With Constraints
Adding More Constraints
Replot Again
Agenda
Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints
Failing Assertions
Your first FPV run on a block *will* fail• Nobody writes right assumptions in advance!
• Always something you didn’t think of
Thus most of FPVers time is debug• This is OK– debug process gives insight
• Often debugging one assert can help identify other issues
• More assumptions improve counterexample
Assumption Creation Loop: Majority of FV Time
Mindset change from sim; prepare team!
• Early runs have many false negatives More assumptions == more interesting CEX
• Interesting bugs not found on first run
• Several rounds of assumes deep traces
• Be sure to check assumptions too, in simulation or FV
Analyze Failures
Run FV
Add Assumes
Example: Adding an Assumption
In one unit, many assertions failed
• Mid-transaction address changes Needed input assumption to prove
• Bug found when assumption fired in simulation!
RTL Under Test(several cycles per
transaction)
Address Bus
ASSUME (held for ASSUME (held for nn cycles)cycles)
Assumption Count Exploding?
Possible bad choice of boundary• Effectively reimplementing neighbor block?
Consider increasing hierarchy level• Add upper level & many blackboxes?
Also consider simplifying problem•Only cover certain modes
–PCIE: prove for x16, not x4, x8?•Restrict data
–Will one bit test most major logic?–Are fully general payloads needed?
Example: FV too hard?
MPE0
MRA1
MRA0
MPE1
MPE = Memory Protocol Engine
MRA = Memory Read Arbiter
Correct Hierarchy Makes FV Easy
MPE0
MRA1
MRA0
MPE1
MSB
Pick Your Battles
FPV can be effort-intensive• Need good understanding of requirements
• Should concentrate on high-risk areas
FPV owner needs deep understanding• Tool pokes at unusual behaviors, not typical
– Very different from simulation
• If not author of block, need to study intensely– Block owner should be available for questions
• Don’t assign random intern to FPV!
Effective FPV At Intel (Info from public DAC 2008 talk) Chipsets: Found numerous bugs missed in
simulation (PCIe, memory controllers)
• Also uncovered flaw in one validation env CPU designs expanding this usage mode
• Competitive: Recent project devoted 8% of validation resources in front-end design, found 8% of bugs
• 30-35% of bugs found by assertion FV were unlikely to be found in simulation
References / Further Reading•http://www.aracnet.com/~eseligma/docs/dvcon_2006.pdf
• Jasper documentation in /pkgs/jasper/current/doc on ECE systems
•http://www.systemverilog.org/pdf/3b_AssertionsUserTechnology.pdf
•http://www.aycinena.com/index2/index3/iccd%202006%20verification%20panel.html
• http://oskitech.com/papers/datta-mc-vlsi08.pdf