a novel collaborative scheme of simulation and model checking for system properties verification
TRANSCRIPT
-
mpr
B
ogy
7 A
ine
of i
The
che
stru
atio
rks
# 2006 Elsevier B.V. All rights reserved.
main bottleneck in design flow. The overall idea of verification
ques [3], such as theorem proving, equivalence checking and
cooperative scheme is implemented on classified properties
Computers in Industry 57 (20and refined internal model.
The remainder of this paper is organized as follows: Section
2 looks through the basic and improved verification methods.
Section 3 depicts the collaborative flow of our verification* Corresponding author. Tel.: +86 10 62795428; fax: +86 10 62785564.
E-mail address: [email protected] (M. Zhu).
0166-3615/$ see front matter # 2006 Elsevier B.V. All rights reserved.doi:10.1016/j.compind.2006.04.006incorrect or incomplete specifications [2]. The gap between
chip scale and design ability increases incessantly, the system
complexity increases 58% annually and the design ability only
improves 21%, as the arrow shown in Fig. 1 [3]. Usually a
project needs to be staffed with three verification engineers for a
design engineer. Time to market is being delayed, and many
bad bugs are showing up after tape-out on account of
incomplete verification.
Functional verification of a complex system becomes the
to their inherent deficiency. The feasibility may be found in a
collaborative approach that employs multiple verifying
engines, including coverage analysis, module simulation, and
formal verification, etc.
This paper presents a novel collaborative verification
scheme which employs static analysis, module simulation
and BDD-based model checking. Prior to the verification
process, properties preprocessing and model refinements are
performed on a hierarchical CDFG structure. Then theor functional flaws; 82% of designs with re-spins resulting from
logic or functional flows have design errors and 47% havemodel checking [4]. Neither simulation nor formal methods can
cope with a very large and complex circuits design alone owingabout 50% 2 years ago; more than 60% of designs contain logic1. Introduction
According to most estimates, nowadays the verification in
the designs of complex ASICs (Application Specific Integrated
Circuits) and SoCs (Systems-on-a-Chip) takes 70% or more of
the overall design cycle [1]. From Collett International
Research, first silicon success rate has fallen to 39% from
is to find as many bugs as possible in the earliest stage and get to
an experience-based confidence level. Special corner cases
coverage and improved specification techniques are needed
during the verification process.
Current verification strategies span a wide spectrum that
ranges from brute-force manner, such as dynamic simulation,
random test vectors generation to formal verification techni-Keywords: Simulation; Model checking; Collaborative verification; Property; CDFGA novel collaborative sche
checking for system
Ming Zhu *, Jinian
Department of Computer Science and Technol
Accepted 2
Available onl
Abstract
The conventional methods cannot easily handle the verification task
collaborative scheme for verifying functional properties of a design.
module simulation, BDD-based (Binary Decision Diagram) model
cooperative scheme is performed on a refined model from CDFG
verification interactively and collectively. For speeding up the verific
reordering, properties pre-grouping, and model refining. The benchma
validity and practicality of the collaborative scheme.e of simulation and model
operties verification
ian, Weimin Wu
, Tsinghua University, Beijing 100084, China
pril 2006
7 July 2006
ncreasingly complex hardware designs. This research proposes a novel
classified properties are designed for three verification engines, i.e.
cking and CDFG (Control Data Flow Graph) static analysis. The
cture and three methods are employed to complete the properties
n process, optimization techniques are introduced, such as variables
ITC99 (International Test Conference) are used to demonstrate the
www.elsevier.com/locate/compind
06) 752757
-
scheme. Experimental results and conclusions are presented in
Sections 4 and 5, respectively.
2. Preliminaries of verification methods
Traditional simulation is effective for medium-scale
designs, and well suited for verifying design units. As the
number of circuit gates increases to several millions, the
simulation cycles explodes by orders of magnitude. The annual
increasing simulation cycles is shown as histogram in Fig. 1,
and now 40 billion random cycles without finding a bug is one
of the criteria for accepted designs tape outing [5]. Simulation
methods have not the ability to keep up with design
complexities. In spite of the increasing simulation and
emulation speed, the number of problem cases that can be
and BDDs to generate input vectors, and significant improve-
ment in state space coverage is reached. Hazelhurst et al. [8]
proposes a hybrid technology combining symbolic trajectory
evaluation with either symbolic model checking or SAT-based
model checking to reduce significantly the cost of verifying
circuits with complex initialization. Presented by Ho et al. [9],
Ketchum integrates symbolic simulation with SAT-based BMC
(satisfiability-based BoundedModel Checking) which provides
the ability for verifying 4500 latches and 170 K gates. RFN
proposed by Wang et al. [10] employs multiple formal
verification engines, including a conventional BDD-based
fix-point engine and a BDD-ATPG hybrid engine.
M. Zhu et al. / Computers in Industry 57 (2006) 752757 753covered by manually generated test vectors or pseudo-random
testing is declining.
As an attractive alternative to simulation, formal approach
verifies a design statically and automatically without test
benches. Model checking is one of important formal method for
verifying finite state systems. Compared with simulation, the
superiority of model checking is full automation with useful
counterexamples as by-product. However state explosion that
constantly occurs in large-scale systems [6] is the main problem
in model checking. On the other hand, model checking needs a
detailed specification of the interface behavior of the block
being verified, so that only legal inputs are checked. Now the
research on model checking has been mainly focused on
verification of abstract designs rather than actual ones.
In order to suppress the shortcomings and employ the
advantages of different approaches, many collaborative
strategies have been put forward in simulation and formal
verification. Simulation provides the statically analyzed
information to formal method, and the latter explores
exhaustively the corner case for the prior. It has been observed
that combining the two techniques may be more effective and
practical. Many hybrid methods are proposed for special
applications.
Developed by Ganai et al. [7], SIVA employs the
combination of ATPG (Automatic Test Pattern Generation)Fig. 1. Chip Scale vs. design ability.Most of the existing hybrid strategies focus on the
cooperation between test vector generation and state space
exploration. In our project, we emphasize how to provide a
unified frame and activate their inherent relations to improve
the verification quality.
3. Collaborative verification scheme
Based on previous works, a collaborative verification
scheme is applied in our project. System property checking
is the basic target and CDFG [11] acts as the common structure.
Static analysis, module simulation and BDD-based model
checking are employed, respectively, and cooperatively.
Differing from existing works, our research concentrates on
properties classification, special model refinement, and inter-
action among different methods.
In this section, assertion for describing system properties
will be introduced, and then as an intermedium bridging the gap
between simulation and formal method, CDFG will be
presented briefly. Next properties classification strategies are
proposed according to different verification methods. Lastly,
collaborative verification flow is presented and explained
detailedly.
3.1. Assertion-based verification
Assertions are some clauses to describe system behaviors
that can be monitored and checked in verification. A
fundamental system function can be defined as a property. In
essence, an assertion is a statement that a property of a design
must be true. The roles of assertion are shown in Fig. 2.Fig. 2. The role of assertions.
-
attention to data processing; static analysis properties focus on
the condition translation, and model checking properties
n Industry 57 (2006) 752757
Table 1
Typical properties declaration
Type Procedure declaration
Simulation assert_always (expr: in std_logic)
assert_always_on_edge (expr: in std_logic)
assert_unchange (expr, count: in std_logic)
assert_period (expr, period: in std_logic)
Static analysis assert_consistent (expr1, expr2: in std_logic)
assert_exclusive (expr1, expr2: in std_logic)
assert_ifcond (expr1, expr2: in std_logic)
assert_switch (val, cond: in std_logic)
Model checking assert_AG (expr: in std_logic)
assert_AX (expr: in std_logic)
assert_EF (expr: in std_logic)
assert_EfUg (expr1, expr2, clk: in std_logic)More than 25% of designers may begin with using assertion-
based verification to start their next design [2]. As an effective
and standardized effort, assertion-based technique is shaping up
as the best way for clearing up the bottleneck in IP (Intelligent
Property) integration and verification. Many companies are
developing innovative methodologies based on assertions. In
the foreseeable future, assertion-based verification will con-
tinue to gain ground as the best hope for steady improvements
in verification efficiency.
In our project, two assertion languages, CTL (Computation
Tree Logic) [4] from CMU University and OVL (Open
Verification Library) [12] based on Accellera package are use
for reference. As an assertion expression, CTL language is a very
expressive logic that combines both branching-time and linear-
time operators. At the same time, CTL is very complicated and
deficient in theexpressionofwide-dataoperation.OVLassertions
are developed originally byVerplex for simulation, not formodel
checking. The implementation of OVL package increases the
difficulties formodel checking.An effective property description
should have the ability to express not only data operations in
simulation but also temporal logic in formal verification.
3.2. New property definition and classification
An execution of a system is formalized as an infinite
sequence of states, and any set of such sequences is a property.
Property checking refers to the automatic checking of a design
to ensure that it satisfies certain properties. By stepping through
a significant part of a designs state space, property checkers
overcome the well-known code and event coverage limitations
of simulation, and significantly reduce the verification time.
Property checking can be performed with logic simulation,
static analysis or model checking.
Although simulation cannot provide a full guarantee of
correctness, data simulating is effective and fast in a block or a
module, especiallywithout regard to the datawidthwhich is fatal
for model checking; Model checking provides a workable and
practical solution, and it examines a design and mathematically
proves its functional properties. The exhaustive exploration in all
possible states space improves its competence. Compared with
simulation, full automation and temporal characters are its
exciting advantages. At the same time it is suitable only for
medium-scale designs with clearly specified interfaces.
Leveraging simulation and model checking, we define and
classify the properties with improved descriptions to employ
their good qualities and discard their shortcomings. Specially
supported by CDFG structure, signals coverage and data path
are quite distinct, hence CFGs (Control Flow Graph) to FSMs
(Finite State Machine) creation and DFGs (Data FlowGraph) to
simulation are convenient. Inheriting the good qualities of CTL
and OVL assertions, three types of properties are defined as
some procedures. In terms of their flexibility in parameters,
properties declaration can be changed as the demand of
verification. These properties can be embedded into processes
or parallelized with processes.
Three categories of property prototypes are designed
M. Zhu et al. / Computers i754according to these principles: simulation properties pay moreemphasize the temporal logic.
Some typical propertiesdeclarations are listed inTable 1. The
parameterexpr1 and expr2 are a logical expression. For example,
the property,assert_always can detect thevariation of signals; for
static analysis, assert_ifcond is used to judge the relation
betweenvariables and conditions; the property assert_AG asserts
that the expression is satisfied on all paths forever.
Although the properties are classified, it does not mean that
one property can only be verified with one certain method. The
supplement among different strategies is necessary, so as find
the bugs as many as possible.
3.3. The bedrock structureCDFG
CDFG is an essential intermediate data structure for design
implementation, used in high-level synthesis and hardware/
software co-design. A CDFG structure represents the behavior
of a target design naturally and intuitively. The CDFG
implementation [11] used in our scheme is basically an acyclicFig. 3. CDFG structure and data flow.
-
graph with nodes and edges. It is comprised of two graphs: DFG
including operations and data dependencies, and CFG
including conditional branching, iterations, and modules. All
the subgraphs are described with the same structure. Vertices
and edges represent the dependency of data and control,
respectively. All referenced ports and signals are attached with
the subgraph, as shown in Fig. 3; hence it is easy to recognize
whether a subgraph is influenced by the given signals.
Based on CDFG, it is easy to get control signals and create
an FSM with relatively few states space. Additionally a design
hierarchy can also be kept. In a CDFG structure, branches
provide the matching abilities to check conditional conflicts
and boundary values, while hierarchical model facilitates the
elimination of redundant information. CDFG acts as the bridge
spanning simulation and model checking.
3.4. Collaborative verification flow
M. Zhu et al. / Computers in InIn the verification scheme, CDFG acts as the fundamental
structure. A refined model is obtained from it. The collaborative
verification flow is shown in Fig. 4. At top of the flow, the
VHDL designs with their properties defined by the designer are
compiled into CDFG internal model. The division of CFG and
DFG in CDFG provides a remarkable advantage for verifica-
tion. Not only is it useful for simulation and model checking,
but CDFG itself can be used to verify some properties by static
analysis. With the help of hierarchical structure of CDFG, it is
easy to trace a multi-branch signal and partition the whole
design into modules, to find the value conflicts, and to check the
branch conditions.
For a large design, it is almost impossible to create a whole
model for model checking; therefore a refined sub-model is
competitive and practical. Based on the CDFG structure, first
the data dependencies are eliminated, and then signals
executive trees and conditional signals selection are created
for BDD operation in model checking.
During the construction of FSM for model checking, data
nodes only involve calculating within a module should be gotFig. 4. Collaborative verification flow.rid of, while the signals-cutting method, presented in Ref. [13],
can be applied to the control signals, which are identified easily
in CFG.
Grouping together the properties that refer to the same set of
signals is a good way to diminish the size of model. The signals
that neither exist in a specific group of properties nor are
referred to in corresponding modules should be removed
temporarily from the model.
Usually a propertys checking need not involve the whole
design. That is to say, the signals referenced in a property are
limited. For example, in an 8-bit counter, high 4-bit value can
be ignored when a property only aims to verify low 4-bit. Prior
to model checking, directed by properties, model abstracting
will reduce the count of states effectively. The core of
abstraction is to hold the necessary signals as few as possible.
The initial order of signals is critical for the scale of BDD
nodes, although many algorithms [14] provide the ability to
reorder variables. A good initial order will shrink the cost of
reordering. Before model checking, the order of signals is
adjusted according to their appearance, not to their definition.
Thus a relatively satisfactory initial order is provided for
symbolic model checking.
A hierarchical simplification method for model refinement is
adopted, according to which segments not referred by current
properties will be hidden until they are called by subsequent
properties. To reduce the amount of search effort, pruning
techniques to state space are also adopted. Word-level data
operations, which will lead to states exploding, are treated
specially.
These model refinements are introduced detailedly in our
previous paper [15].
Simulation is good at data modules, limited in large circuits
and test-benches generation. Model checking is highly
automatic and unique in temporal logic, but consumptive
hugely in states space, especially for word-level data
operations. To employ the advantages of different methods,
simulation, static analysis and model checking are performed
separately. Whereas they are also cooperative closely, as the
black arrow shows in Fig. 4: model checking proceeds in large
amount of states and pruning is necessary, and static analysis
needs some data values to continue its work, then simulation
can provide initial values for some critical signals which are
decisive in model checking and static analysis; when signal
mismatching occurs in static analysis process or counter-
example is found in model checking, it means that any design
bug has been found, then the condition will be given to
simulation for test benches generation. Through checking of
control signals, the efficiency of test benches generation can be
improved remarkably.
A similar research was performed by Nakata et al. [16], and
they employed the simulation and SMC (Symbolic Model
Checking) for RTL (Register Transfer Level) description. We
provide the distinct fundamental structureCDFG and
optimization techniques on the internal model, especially in
property classification and grouping for diminishing the model.
By eliminating unnecessary calculations, cycle-based
dustry 57 (2006) 752757 755simulators [17] achieve huge performance gains in verifying
-
functionality, concretely, 510 times improvement in speed and
one-fifth to one-third save in space, over conventional event-
driven simulators. This advantage is inherent in our scheme, for
the elimination of unnecessary calculations.
Proposed by Aziz et al., saturated simulation [18]
decomposes a design into a set of interacting controllers to
explore substantially more of the controller interactions and
cover more design space. This will be referred to our project
later. S2-FSM verification proposed by Bei et al. [19] is specific
for synchronous circuits and reduces the state variables greatly.
The experimental results of collaborative verification are
shown in Table 4. Not only can three methods work separately,
but they overlap each other as well. Such as the property, No. 2
for B01, can be verified with simulation and static analysis,
respectively. This enhances the ability to find as many bugs as
possible. Through the collaborative scheme, we found several
bugs added manually those are difficult to be verified with a
single method.
5. Conclusion
In this paper, we propose a novel collaborative scheme that
utilizes advantages of simulation in data operation and model
checking in temporal logic. CDFG structure acts as the
intermedium role for bridging the two verification methods.
Based on the OVL and CTL, classified properties are defined
and verified with corresponding verification techniques. In our
verification flow, several optimization techniques, i.e. property
grouping, model refining, signals reordering, have been
adopted and examined. The experimental results demonstrate
the validity and practicality of collaborative strategy.
Traditional simulation devotes itself to verification, but its
capability is not equivalent to its ambition. Promising formal
verification is energetic, whereas the shortcoming of space
explosion restricts its application to a wider extent. The
collaborative approach is a new way to improve the ability for
M. Zhu et al. / Computers in In756
Module simulation.b Static analysis.c Model checking.After our refinement in their model, the number of nodes
decreases obviously.
In our collaborative scheme, the problem is how to set the
environmental values for modules that have close relation with
others. Before we come to a satisfactory solution, the random
initial values are adopted.
4. Experimental results
This collaborative verification scheme is performed on some
classical testbench and ITC99 benchmarks [20]. The inter-
mediate results after property pre-grouping and model refining
are compared with the original ones.
The different results after the preprocessing of grouping
properties are shown in Table 2. For the sake of simplification,
properties are only divided automatically into two sub-groups.
The second column is the signal number in a testbench. Columns
35 and columns 68 list the internal node and byte counts in
different groups, respectively. It is obvious that the node and byte
counts are both reduced for one group although the sum of two
groups exceeds the original. This provides a practical way to deal
with large designs. The grouping effect depends on the signals
related by properties. For count8 and arbiter, two groups are
separated well and the nodes count is decreased remarkably,
whereas properties in B01 are close and the grouping effect is
inconspicuous. The number of used bytes during the verification
is also reduced reasonably after grouping operation.
Table 3 shows the comparison after internal model
refinement in last two columns. As previous introduction,
signals reordering, model hierarchizing, and model abstracting,
etc. are adopted. After these optimizations, the verification
model becomes a subset of the original one, and the nodes count
is fairly reduced, especially in B01. It is a valuable assistant to
solve large-scale designs although some bugs maybe get away
from the verification.
Table 2
Result of property grouping
Design
name
Vars
num
Node counts Byte counts
BGa G1b G2c BG G1 G2
Count8 16 114 71 114 72544 68032 72528
TLC 14 284 170 172 77216 75408 75440
Arbiter 13 126 32 117 75888 67312 74608
B01 10 340 330 325 74848 73664 73360
a Before grouping.b Group 1.
c Group 2.dustry 57 (2006) 752757
Table 3
Result of model refining
Design
name
Vars
num
Potential
states
Reachable
states
Node counts
BRa ARb
Count8 16 65536 65536 114 93
TLC 14 16384 96 284 243
Arbiter 13 8192 1792 126 124
B01 10 1024 432 340 257
a Before refining.b After refining.
Table 4
ITC99 benchmark property verification
Properties description Type Result
ITC99 B01 FSM that compares serial flows
assert_always (stato = e and overflw = 1) MSa Valid
assert_switch (stato = wf1 and line2 = 1, stato = a) MS Invalid
assert_switch (stato = wf1 and line2 = 1, stato = a) SAb Invalid
assert_AG (reset = 1 and stato = e) MCc Invalid
assert_EF (reset = 1 and stato = e) MC Valid
ITC99 B03 resource arbiter
assert_period (code0 = U1, 2) MS Invalid
assert_exclusive (ru1 = 1 and fu1 = 1,code0 = U1) SA Valid
assert_AX (code0 ! = code0) MC Valid
assert_EF ((ru1 = 1 and ru2 = 1) and code0 = U2) MC Valid
averifying large-scale designs.
-
Of course, more cooperation strategies among different
methods need to be examined, still new problems for hardware/
software co-verification require considering. Our future effort
will involve these problems.
Acknowledgements
This research was supported by the National Natural Science
Foundation of China (NSFC) 60273011, Hi-Tech Research &
Development (863) Program of China 2003AA115110 and the
National Foundation Research (973) Program of China
G1998030403.
[15] M. Zhu, J. Bian,W.Wu,Optimization techniques in a functional verification
platform for embedded system, in: Proceedings of the First International
Conference on Embedded Software and System (ICESS), Hangzhou, 2004.
[16] T. Nakata, S. Kowatari, et al., Techniques for effectively applying model
checking to design projects, Fujitsu Scientific and Technical Journal 36
(2000) 1.
[17] K. Westgate, D. McInnis, Cycle-based simulation reducing logic verifica-
tion time with cycle simulation, http://www.quick-turn.com/tech/cbs.htm.
[18] A. Aziz, T.R. Shiple, J.H. Kukula, Hybrid verification using saturated
simulation, in: Proceedings of 35th DAC, 1998, pp. 615618.
[19] B. Jinsong, L. Hongxing, B. Jinian, et al., S2-FSM: a verification-oriented
model of synchronous sequential circuits and its modeling algorithm,
Journal of CAD/CG 11 (3) (1999) 196199 (Chinese).
[20] ITC99, http://www.cerc.utexas.edu/itc99-benchmarks.
M. Zhu et al. / Computers in Industry 57 (2006) 752757 757Reference
[1] D. Maliniak, Assertion-based verification smoothes the road to IP reuse,
in: Electronic Design Technology Report, 2002.
[2] R. Schutten, T. Fitzpatrick, Design for verificationblueprint for pro-
ductivity and product quality, Synopsys Technical Papers, 2003.
[3] Veritable, Formal validation manual, http://www.veri-table.com/.
[4] E.M. Clarke Jr., O. Grumberg, D.A. Peled, Model Checking, MIT Press,
Massachusetts, 1999.
[5] K. Albin, Nuts and bolts of core and SoC verification, in: Proceedings of
38th DAC, 2001, pp. 249252.
[6] J.R. Burch, E.M. Clarke, K.L. McMillan, D.L. Dill, L.J. Hwang, Symbolic
model checking 10E20 states and beyond, Information and Computation
98 (2) (1992).
[7] M. Ganai, A. Aziz, A. Kuehlman, Enhancing simulation with BDDs and
ATPG, in: Proceedings of 36th DAC, New Orleans, LA, (1999), pp. 385
390.
[8] S. Hazelhurst, O. Weissberg, G. Kamhi, et al., A hybrid verification
approach: getting deep into the design, in: Proceedings of 39th DAC,
2002, pp. 111166.
[9] P.H. Ho, T.R. Shiple, K. Harer, et al., Smart simulation using collaborative
formal and simulation engines, in: Proceedings of ICCAD, 2000, pp. 120
126.
[10] D. Wang, P.-H. Ho, et al., Formal property verification by abstraction
refinement with formal, simulation and hybrid engines, in: Proceedings of
38th DAC, 2001, pp. 3540.
[11] J. Jeon, Y. Ahn, K. Choi, CDFG toolkit users guide, http://inspire.s-
nu.ac.kr.
[12] Accellera, Open verification library assertion monitor reference manual,
2002, http://www.accellera org.
[13] R.C.-Y. Huang, K.-T. Cheng, A new extended finite state machine (EFSM)
model for RTL design verification, in: Proceedings of International High
Level Design Validation and Test Workshop, 1998.
[14] A. Aziz, S. Tasiran, R.K. Brayton, BDD variable ordering for interacting
finite state machines, in: Proceedings of 31st DAC, 1994, pp. 283288.Weimin Wu received his BS degree (1989) inComputer Science from Jilin University, Masters
degree (1992) from Harbin Institute of Engineering,
and PhD degree (1995) from Harbin Institute of
Technology, all in PR China. From 1996 to 1998
and from 1998 to 2000, he worked as a Post-doctor
researcher in Zhejiang University and Tsinghua Uni-
versity, respectively. Now he is an Associate Profes-
sor with the Department of Computer Science and
Technology, Tsinghua University.Department of Computer and Science of Tsinghua
University, Beijing, China. His research interests
include logic-level and high-level design specifica-
tion, simulation, verification, synthesis and DA
systems. He was a visiting scholar in Kyoto Uni-
versity, Japan from 1985 to 1986, and in Kyushu
University, Japan in 1999. He has contributed to a
number of national projects on VLSI CAD systems
development including the PANDA VLSI CAD
system. He is the Associate Editor-in-Chief of the Journal of CAD/CG,
China.Ming Zhu received his BS degree (1999) in Com-puter and Science from Tsinghua University, Beijing,
China. Currently he is a PhD candidate of the
Department of Computer and Science, Tsinghua
University, Beijing, China. His research interests
include high-level design specification, system func-
tional checking, formal and hybrid verification.
Jinian Bian is a Processor in EDA laboratory at the
A novel collaborative scheme of simulation and model checking for system properties verificationIntroductionPreliminaries of verification methodsCollaborative verification schemeAssertion-based verificationNew property definition and classificationThe bedrock structure-CDFGCollaborative verification flow
Experimental resultsConclusionAcknowledgementsReference