Paper Report

Presenter: Jyun-Yan Li

Design Fault Directed Test Generation for Microprocessor

Validation

Deepak A. Mathaikutty, Sandeep K. Shukla FERMAT Lab, Virginia Tech, BlacksburgSreekumar V. Kodakara, David Lilja The University of Minnesota, MinneapolisAjit Dingankar Validation Tools, Intel Corporation, Folsom

Design, Automation & Test in Europe Conference & Exhibition, 2007. DATE '07

2

Functional validation of modern microprocessors is an important and complex problem. One of the problems in functional validation is the generation of test cases that has higher potential to find faults in the design.

We propose a model based test generation framework that generates tests for design fault classes inspired from software validation. There are two main contributions in this paper. Firstly, we propose a microprocessor modeling and test generation framework that generates test suites to satisfy modified condition decision coverage (MCDC), a structural coverage metric that detects most of the classified design faults as well as the remaining faults not covered by MCDC.

Abstract

3

Secondly, we show that there exists good correlation between types of design faults proposed by software validation and the errors/bugs reported in case studies on microprocessor validation. We demonstrate the framework by modeling and generating tests for the microarchitecture of VESPA, a 32-bit microprocessor. In the results section, we show that the tests generated using our framework's coverage directed approach detects the fault classes with 100% coverage, when compared to model-random test generation

Abstract (cont.)

4

Simulation is widely used to validate large system Depend on the quality of tests Coverage metric measures the quality

。Software validation Statement, branch and path coverage

。State machine representation State, transition and path coverage

。Functionality

Effectiveness of coverage directed test generation Type of faults

。Can occur Strength of the coverage metric

。Detect faults

What is the Problem

5

Related workcoverage

Micro architecture

coverage[12]

randomExtended fault

class[7]

Design Fault Directed Test Generation for Microprocessor ValidationThis

paper:

coverage of formal models

[5]

Graph-based[9]

MCDC[6]Generate

assembler test program directly

for PowerPC

For superscaler processor

Build graph model of

processor and generate test program to detect fault

9 fault classes related to boolean

expressions

Strategy of coverage metric

Functional coverage

6

Fault class Example

Expression Negation Fault (ENF)

Term Omission Fault (TOF)

Term Negation Fault (TNF)

Literal Omission Fault (LOF)

Literal Insertion Fault (LIF)

Literal Negation Fault (LNF)

Literal Reference Fault (LRF)

Disjunctive Operator Reference Fault (ORF[+])

Conjunctive Operator Reference Fault (ORF[-])

Fault classes [7]

Correct

7

Statement coverage Invoke executable statement

Branch coverage Execute both the true and false

Modified condition Decision Coverage (MCDC) Test each of condition within a decision Affect outcome of the decision independently m conditions, m+1 test cases

Coverage Metrics

The same

with T2

The same

with T1

8

Correlate the bug categories in [2, 13] and map into the fault class

Detect all the design fault except LIF None of the errors observed was classified as a LIF

Motivation

9

Modeling language as a metamodel Describe architectural and microarchitectural

Proposal method

Validate the Microcode

Validate the RTL implementation

Statement, branch, MCDC Generate golden

result

Simulation result

10

Processor is specified Register/memory-level description

。Registers, their relationship and memory schematic Instruction set capture

。Instruction behavior

Construct Function blocks, if-else block, statement sequence and

loop, register map and memory layout Ex:

Architecture modeling

Register map

Register/memory reference or

immediate value

jump-if-not-zero (JNZ) instruction

11

Construct Configure pipe stage, provide ports with memory and

registers, insert instruction registers and describe the control logic

Combined to complex stages by basic components as MUX, ALU, INC, etc

Ex:

Microarchitecture modeling

Instruction fetch stage

Instruction memory

Z

PC

Increment by 4

PC2

1

1

0 2 3

0

12

Constraint Satisfaction Problem (CPS) Build Program Flow Graph (PFG) Static Single-Assignment(SSA) analysis for multiple

incoming path Coverage Constraint Generator (CCG) generates

constrains for decisions

Test Case Generator (TCG) ILOG solver generates test case Binary generator initializes registers, memory locations

and variables

Test generation Framework

13

Convert the architectural and micro-architectural model into Program Flow Graph (PFG) Architectural model

。Function block -> hierarchical states。Statement -> state。If-else -> decision。Identifier, registers, memory location -> variables

Micro-architectural model。Basic components -> bunch of if-else sequences

If a state with multiple incoming transition Resolve in the Static Single-Assignment (SSA) analysis

Constraint Satisfaction Problem(CSP)

14

JNZ instruction

IF stage

PFG example

variable

decision

state

15

Help CSP to generate constrains for complete run Focus on states with multiple incoming transitions

If no change the value。the state will be ignored。Insert additional decisions and variables

Ex:

Static Single-Assignment (SSA) analysis

SSA result

SSA analysis

Ignore S3

Rename variables

16

ILOG is a commercial solver designed for performance and scalability Generate test case that satisfying every constraint

variable Ex:

ILOG solver

17

Two set of inputs Coverage annotation

。expedite the reachability of the coverage goal。Setting the attribute MARK associated with a state and

decision Coverage type

。Statement coverage。Branch coverage。MCDC。Design Fault coverage

Coverage constraint generator (CCG)

18

MCDC

Arrange D by graph depth

Graph G

Problematic decision point D

Node d

Find a path p form root to d

Generate a case that the

constraints for path p and

decision is true (d=1)

Boolean operator

?

Generate a case that the

constraints for path p and

decision is false (d=0)

Total test case = combine and

Generate a case that the

constraints for path p and

conditions are true

Generate a case that the

constraints for path p and

conditions are false

Total test case = combine and

Remove the duplicate test

cases

N

Y

19

The fault detection capability of MCDC is 7/9 classes coverage Literal omission fault (LOF) can be satisfied by MCDC

。Every literal will affect the output of decision。Ex:

Literal Reference Fault (LRF) coverage by LRF_list。A list of variables to tell CCG which variable can be

replaced for resulting in a fault。If LRF_list is not provided, the CCG replaces every

variabled。Lead to large set of test case。Recommend LRF_list with a variable and mark it

Design Fault coverage

20

How many test cases are generated

。Not describes in the paper Conditions in a decision

。330 unique decisions, 400 unique conditions。1~10 conditions in a decision

Fault in each fault class

Before Experiment

21

Environment VESPA 32-bit processor, 5-stage pipeline 50 decisions are chosen and replaced randomly for LRF The number of test cases are the same for each

coverage metrics

Experimental result

22

A metamodel-based modeling framework to capture structure and behavior Architecture – registers, ISA Micro-architecture – pipeline structure, control logic

TGF results in test case for covering design faults CSP formulation TCG

My comment Overview the test case generating flow for code

coverage Some algorithm is not specification

Conclusion