FMCAD 2008 1

A Theory of Mutations with Applications to Vacuity,

Coverage, and Fault Tolerance

Orna Kupferman1

Wenchao Li2

Sanjit A. Seshia2

1 Hebrew University 2 UC Berkeley

FMCAD 2008 2

FMCAD 2008 3

Bob

This system is correct even under faults (e.g. flips in

latches)

Why? Convince me.

It satisfies its specification under these faults.

Doesn’t this mean the specification coverage is low?

Adam

So is my specification not good enough or is my system

fault-tolerant?

Need fault-tolerance! But also need to certify it!

FMCAD 2008 4

Problem Current mutation-based metrics are

inadequate to reason about specification coverage for fault-tolerant circuits in model checking.

FMCAD 2008 5

Preliminaries Coverage

Introduce ∆ to an implementation I and check I’ ² S.

Fault Tolerance I with fault f still

satisfies S.

Vacuity Introduce ∆ to a

specification S and check I ² S’.

All three involve introducing mutations in the verification process!

j=

j= (2)

¹

FMCAD 2008 6

ContributionsA theory of mutations: formally ties together coverage

and vacuity in model checking; enables reasoning coverage for

fault-tolerant circuits.

FMCAD 2008 7

Agenda Related Work

Coverage Vacuity

A Theory of Mutations Coverage and Vacuity are dual Aggressiveness amongst mutations

Applications Conclusion

FMCAD 2008 8

Coverage Is my specification complete? Coverage metrics for model checking

[HKHZ 99; KGG 99; CKV 01,03]

FSM Coverage

statepath

FMCAD 2008 9

Coverage Functional Coverage in BMC [GKD 07] Detect “forgotten cases” [Claessen 07] Coverage for fault-tolerant systems

[FPFRT 03, DBBDCMF 05] Single stuck-at fault model

FMCAD 2008 10

Vacuity Is my specification satisfied trivially? Vacuity detection [KV 99, 03; BBER 01;

AFFGP 03; CG 04; BFGKM 05; BK 08]

G (req → F grant) G (req → false)Replace a sub-formulae in the most challenging way.

Trivially true in a system where req

is never sent.

FMCAD 2008 11

Agenda Related Work

Coverage Vacuity

A Theory of Mutations Coverage and Vacuity are dual Aggressiveness amongst mutations

Applications Conclusion

FMCAD 2008 12

Examples of Mutations Can mutate inputs, outputs, or latches Stuck-at

Restricting a signal to a value

Freeing (abstracting) a signal

1000

1001

1000

10011000

100X100X

old

new

10011000 Removes behaviors

Adds behaviors

Modifies behaviors

FMCAD 2008 13

A Theory of Mutations Properties:

Invertability: (Cμ)ν = C

Monotonicity: I ² S → Iμ ² Sμ Duality

Interesting Mutations: Conditional stuck-at Conditional add/remove transitions Permuting events

FMCAD 2008 14

Duality

Iμ ² S ↔ I ² Sν

,where ν and μ are dual mutations.

low coverage vacuity

FMCAD 2008 15

Circuit with input = {z}, control signals = {x, y}, output = {x}, described by the state representation on the right. xy

x

z

S simulates I’ and S’ simulates I

010

0,1

I 0,1

010

000

10S’

remove behavior

I’

add behavior

010

111 0,10,1

0,1 0,1

010

000

111

101

S

0

10

1

FMCAD 2008 16

Aggressiveness Mutation is more aggressive

than if applying makes it harder for the design to satisfy its specification.

I ² S → I ² S

orI ² S → I ² S

≥imp

≥spec

FMCAD 2008 17

Some Aggressive Orders Free(x) ≥ k-SEU(x) Free(x) ≥ Stuck_at_0(x) Free(x) ≥ Flip(x) Delay_k+1 ≥ Delay_k k-SEU(x) ≥ m-SEU(x) ≥ for k ≥ m More interesting ones can be found in the

paper.

FMCAD 2008 18

Coverage for Fault-tolerance For a fault-tolerant system I and a set

of mutations {j} such that Ij

² S for all 1≤j≤k.

The fault-tolerant system loosely satisfies S if there is a mutation such that j ≤imp for all 1≤j≤k;

I ² S.

FMCAD 2008 19

Agenda Related Work

Coverage Vacuity

A Theory of Mutations Coverage and Vacuity are dual Aggressiveness amongst mutations

Applications Conclusion

FMCAD 2008 20

Applications Useful vacuity information can be

obtained for free from coverage checks.

Analyze coverage for fault-tolerant systems.

Improving specifications Catch bugs Strengthen environmental assumptions

FMCAD 2008 21

Vacuity from Coverage S: G (sp[2..0] = 3’b110 → X (sp[2..0] =

3’b111) In our experiment, applying the “Flip(x)”

mutation to sp[0] still satisfies S. S’: G (sp[2..0] = 3’b110 → X (sp[2..0] =

3’b110) S & S’ → G ¬(sp[2..0] = 3’b110)

FMCAD 2008 22

Certifying Fault-Tolerance

System behaviors

Original low-coverage spec.

System behaviors

High-coverage spec. certifies system’s target resilience

1-SEU

System behaviors

2-SEU

FMCAD 2008 23

Experiments

VIS benchmarks, results obtained with Cadence SMV model checker

FMCAD 2008 24

Improving Specifications Chip Multiprocessor Router [Peh 01]

However, the process still requires some user assistance.

Simplied model

S: G (ξ → X ¬(grant = 2’b11)

S’: G (ξ → X (grant = 2’b10)

FMCAD 2008 25

Conclusion A theory of mutations that

Unifies coverage and vacuity Can be used to certify the correctness

of fault-tolerant circuits A new technique to tighten

specifications The ideas here can be applied to

other verification techniques.

FMCAD 2008 26

Q & A

Thank you!

FMCAD 2008 27

References