qualification of verification environments using formal ...l5 not observed 1. modify each location...

Copyright OneSpin Solutions 2014

Qualification of Verification Environments Using

Formal Techniques

Raik Brinkmann

DVClub on Verification Qualification

April 28 2014

www.onespin-solutions.com

Copyright OneSpin Solutions 2014 2

OneSpin SolutionsInnovative Formal Technology

Unique Formal Technology

•Push button observation coverage analysis for verification progress

•Functional coverage using unique gap free verification

•Mature sequential equivalence checking for FPGA and ASIC synthesis

Advanced Usability Features

•Structural assertion debugger and active value/driver tracing through RTL

• Incremental compilation of assertions for quick turn around

•Faster assertion development with operational assertions

World Wide Company Success

•Doubled Revenue in 2013

•Tripled Bookings in 2013

•Increased World Wide Adoption

• Simulation-based verification methodologies

predominant

• Significant Market Growth for Formal

Verification

• Significant EC FPGA market growth

• OneSpin customers benefit from Mature Formal

Design Verification products

• OneSpin keeps innovating!Verification Market Sizes 2008 - 2013Source EDAC MSS 2013

OneSpin Solutions is

growing faster than the

market.

Accumulated Growth Since 2008Source EDAC MSS 2013

Pioneering, Leading Technology

Broad Range of Solutions

User Oriented Approach


From Automated Solutions

To Advanced Verification

OneSpin Advanced Formal Proof Engine

Quick & Easy,

Automated,

Comprehensive

Design Analysis

Rapid, Exhaustive

Coverage-Driven Property

Verification

ASIC & FPGA

Tool Sign-off Accuracy

Sequential

Equivalency Checking

High Performance, Easy to use, Accessible Technology Platform

Assertion

Constraint

Set Formal

Engine

Integrated

DebugAssertion

Set

RTL

Code

Quantify™

Observation

Coverage

Solutions: Protocol Analysis, Register &

Connectivity Checking, Score-boarding

Formal

Engine

Structural

Assertion Synthesis

Integrated

Debug

HDL

LintCoverage

Assertion Synthesis

RTL

Code

RTL-RTL,

RTL-Gate

Gate-Gate Sequential

EC

Place & Route

Gate

Synthesis

RTL

X-Propagation Analysis

Assertion Synthesis

Handwritten SV / PSL Assertions

Operational Transaction Assertion Library

Silicon


• Analyzing Code Reachability

• Extracting Finite State Machine from RTL Code

• Covering FSM State Transitions

• Postulating Operations by Inspecting Transition Table

• Capturing Operations using Formal Assertions

• Mapping Operations to Code Regions Using Observation Coverage

• Analyzing Operational Assertion Set for Functional Completeness

Outline of the Talk


Qualification of Verification Environments

Coverage is Key

DUV

Tests / Scenarios Checkers

Bus Functional

Model (BFM)

Test #1

Test #2

Stimulus / Scenario Generation

ConstraintsFormal

Engine

Stimulus / Scenario Exclusion

Check #1

Check #2

Check #3

Check #4

Simulation

Formal

Verification Environment (simplified):

Reachability Coverage:

• Focused on quality of stimuli

• But what about the checkers?

Observation Coverage:

• Focused on quality of checkers

• Exposes unverified DUV parts

How good are mytest vectors & constraints?

How good are mycheckers andassertions?


Reachable Code Coverage

• Generated cover statement for each code

branch in DUT

• OneSpin Inspect dead_code check

– Either proves that code branch cannot be covered or

– Shows simulation trace from reset where code branch

gets activated

• Dead code often points to DUT issue

case (state)

2'b00: nstate = 2'b01;

2'b01: nstate = 2'b11;

2'b10: nstate = 2'b00;

2'b11: if (ack)

state = 2'b10;

else

state = 2'b11;

endcase

Can this branch

be covered?


Reachable Code Coverage Example


Extending Code Coverage Analysis

FSM State/Transition Coverage

Problem:

• Is the FSM correctly reset?

• Is there any unreachable state?

• Are there deadlocks between FSMs?

Solution:

• Automatically detect FSMs in RTL

• Synthesize and check cover properties and assertions to

• Cover initialization

• Cover each state

• Cover each transition

• Assert absence of deadlocks


Example: SDRAM Controller

SDRAM

clk

reset

write_request_i

address_i[23:0]

write_data_i[31:0]

ready_o

read_data_o[31:0]

SDRAMController

we_n_o

cs_n_o

ras_n_o

sdram_addr_o[11:0]

cas_n_o

sdram_write_data_o[31:0]

sdram_read_data_i[31:0]

read_request_i

burst_single_i

CPU

• Two Interfaces:

- CPU

- SDRAM

• Operations:

- Nop (Idle)

- Single Read

- Single Write

- ...


Extracting FSM and Analyzing Reachability

Formally Analyze

Reachability

Formal

Engine

Structural

Assertion Synthesis

Integrated

Debug

HDL

LintCoverage

Assertion Synthesis

RTL

Code

Extract StructureRTL

Code

OneSpin 360 DV Inspect


Explore Behavior using Cover Properties

Active Code

Evaluated Code

State Sequence

SVA

Cover

Property

sr1

sr2

sr4

row_act

sr2

The FSM can transition from idle through sr1 back to idle

It passes through row_act twice

Does it always have to do this`?

What are the conditions for this transition sequence?

Formal Cover Property is

used to demonstrate some

postulated behavior


Capture Operations of Design and Validate

Operations Start and End in Important States

Basic Operations

• Reset

• NOP (Idle)

• Pre Charge

• Row Activation

• Single Read

• Single Write

sw1

bw1

bw2

bw4

br1

br2

br7

sr1

sr2

sr4

pr ra

idle

row_act

idle

row_act

Complex Operations

• Burst Read

• Burst Write

Capture Central Control for

Each Operation using an

Operational Assertion


Conceptual State Machine

• Abstraction of design FSM

• Need to capture all operations in CSM with

operational assertions

idle row_act

burst read

burst writerow activationnop

prechargesingle write

single read


Operational Assertion

Operation

suppose

prove

endstart

t##0 t##1 t_ack t_ack##1timepoints

conceptual

state

outputs

inputs

conceptual

state

Cause

Effect


SDRAM Controller: Single Read Operation

write_request_i

address_i

read_data_o

write_data_i

ready_o

cas_n_o

we_n_o

sdram_addr_o

sdram_read_data_i

burst_single_i

cs_n_o

ras_n_o

read_request_i

single read

C

D

D

state row_act row_act

{R,?}{R,C}

D

D

C

t +1 t +2 t +3 t +4 t +5 t +0

Effect

clk

Cause

Start & End

State


Proving Central Control Conditions for

Single Read Operation

sr1

sr2

sr4

row_act

sr2

Assertion proves that cause defining operation conditions always has the same effect

Cause

Effect


How much Code is Observed by Assertions?

RTL Code

Assertions /

Constraints

Formal

Check

Assertion

exhaustively

proven

CounterexampleCoverage Metric

Coverage

Analysis

Debugging

How do we know we have understood all of the design?

Has all RTL code been understood and captured?

• Observation Coverage provides an Answer!

Has all functionality been captured?

OneSpin 360 DV Verify with Quantify Observation Coverage


always @(posedge clk or posedge reset)

if (reset)

z <= 1’b0;

else

begin

case (i)

3'b001: z <= a;

3'b010: z <= b;

3'b100: z <= c;

default: z <= <input>;

endcase

end

M5


if (reset)

z <= 1’b0;

else

begin

case (i)

3'b001: z <= a;

3'b010: z <= b;

3'b100: z <= <input>;

default: z <= 1'b1;

endcase

end

M4


if (reset)

z <= 1’b0;

else

begin

case (i)

3'b001: z <= a;

3'b010: z <= <input>;

3'b100: z <= c;

default: z <= 1'b1;

endcase

end

M3

Observation Coverage

What Causes the Design to Satisfy the Assertions?

module select1(onehot, a, b, c, z, clk, reset);

input clk;

input reset;

input [2:0] i;

input a;

input b;

input c;

output reg z;


if (reset)

z <= 1'b0; // L1: not covered (reset case)

else

begin

case (i)

3'b001: z <= a; // L2: covered by assertion

3'b010: z <= b; // L3: not covered

3'b100: z <= c; // L4: not covered

default: z <= 1'b1; // L5: not covered

endcase

end

// if there is no reset, then 'a' is stored in 'z' if ‘i' is 3'b001

A: assert property

( @(posedge clk)

disable iff (reset)

i == 3'b001 |=> z == $past(a)

);

endmodule

Which assignment locations Lx in design

M are observed by proven assertion A?

2. Re-Check property A for each M1..M5


if (reset)

z <= <input>;

else

begin

case (i)

3'b001: z <= a;

3'b010: z <= b;

3'b100: z <= c;

default: z <= 1'b1;

endcase

end

M1


if (reset)

z <= 1’b0;

else

begin

case (i)

3'b001: z <= <input>;

3'b010: z <= b;

3'b100: z <= c;

default: z <= 1'b1;

endcase

end

M2

Assertion A holds on M1:

L1 not observed

Assertion A fails on M2:

L2 is observed

M

A

L3 not observed

L4 not observed

L5 not observed

1. Modify each location L1..L5

of M: Producing M1..M5

The locations Lx for which A fails after replacing the

assignment with a free input are observed.Module M with verified assertion A


Identifying Unobserved and Uncovered Code

Analyzing Basic Operations of SDRAM

Burst Write

Completely

Uncovered

Unobserved

For

Single Read


Quantifying Observation Code Coverage

What is the status after capturing the state transitions

for the basic operations?


Full Single Read Operation

property single_read_p;

t ##0 row_act_state() and // start state

t ##0 read_request_i && // trigger: non-burst read request! write_request_i &&! burst_single_i and

t ##0 address_i[23:12] == $past(address_i[23:12]))

implies

t ##1 sdram_read() and // SDRAM interfacet ##1 sdram_addr_o == $past(address_i[11:0]) andt ##2 sdram_stop() and

t ##4 ready_o and // CPU interfacet ##4 read_data_o == $past(sdram_read_data_i) andt ##5 ! ready_o and

t ##3 sdram_nop() [*3] and

t ##5 row_act_state() // end state

endproperty

Cause

Effect

Operational SVA can express complex properties intuitively

function:

!cs_n_o && ras_n_o &&

!cas_n_o && we_n_o


Complete Behavior of Single Read Operation

Can we be sure?

How can we be sure about functional completeness?


Formal Design Specification

sw1

bw1

bw2

bw4

br1

br2

br7

sr1

sr2

sr4

pr ra

idle

row_act

RTL Code

idle

row_act

Equivalence

burst write

idle row_act

nop

precharge

single read

single write

row_actburst read

Formal

Design Spec

reset

Operational SVA:

Timing Diagrams

Operations

Intuitive

How can we Prove Equivalence?


Functional Coverage Analysis

• Finds errors and omissions in assertion set

– Guarantees that assertion set verifies every possible infinite-long

sequence of operations

– This makes the assertion set a deterministic reference model for the

design

• Proving all assertions on the design closes the design understanding

Requirements for Complete Functional Coverage:

– Input Coverage

– Output Coverage (Determination Test)

– Sequence Coverage (Successor Test)

Do I have enough assertions?

Are my assertions checking

everything?


Example: Missing Operation

property single_read_p;


t ##0 read_request_i && // trigger: non-burst read request

! write_request_i &&

! burst_single_i and


implies

…

Cause

burst excluded

property single_write_p;


t ##0 ! read_request_i && // trigger: non-burst write request

write_request_i &&

! burst_single_i and


implies

…

Cause

burst excluded


Functional Input Coverage Analysis –

Case Split

• Exhaustively checks whether all possible input scenarios are

checked by operational assertions

– Uncovered input sequences are shown as CEX

nop

row_act

single_write

single_read

inputs

state

outputs

matches

neither state

nor inputs

matches

state, but

not inputs

matches

inputs but not

state

IDLE

IDLE

ROW_ACT

write_burst_req

write_burst_req

write_burst_req

write_burst_req = write_request_i &&

burst_single_i

ROW_ACT?


Input Coverage Analysis –

Case Split Test: Counterexample

single read?

write burst request

in row_act state

add burst write (BW)

operation property


Phases to Capture and Validate

Complete Specification as Assertion Set

• Phase 1: Capture/verify central control of core operations– Termination: the internal control and sequencing of the core operations

has been fully captured

Input: RTL code of the DUV & Some Design Understanding

Output: Complete Specification of the DUV

• Phase 4: Capture/verify extended functionality– Termination: 100% input scenarios coverage, 100% output behavior

coverage (full completeness check)

• Phase 2: Capture/verify full control of core operations

– Termination: the full control and sequencing of the core operations has

been captured without gaps

• Phase 3: Capture/verify full behavior of core operations

– Termination: all output signals of core operations are verified to have

the expected value - always


Qualification of Verification Environment

Coverage Analysis has Many Aspects

DUV

Tests / Scenarios Checkers

Spec

Verification Environment

Verification

Plan

Coverage

Analysis

Control

Coverage

Observation

Coverage

Functional

Coverage

Have I written

enough stimuli?

Which parts of

my DUV have

been exercised?

Which parts of my

DUV have been

checked?

Did I write

enough checks?

Are all specified

functions

implemented?

Are all specified

functions verified?

Functional

Structural


OneSpin Targeting Specification Driven

Verification Management

• Verification Requirement:

Leverage functional specification detail to derive test plan

and manage verification process

• Raising the abstraction of both test benches and coverage

to functional level is critical for rigorous, realistic testing

• OneSpin innovation (Quantify Coverage, Operational

Assertions, etc) targeting this requirement


Thank You!

Visit OneSpin at

www.onespin-solutions.com

ChipeEx - TelAviv

DAC - San Francisco

DVCon Europe - Munich

http://www.onespin-solutions.com/

qualification of verification environments using formal ...l5 not observed 1. modify each location...

Documents