using constraint satisfaction ... - research.ibm.com€¦ · ibm research lab in haifa using...
TRANSCRIPT
![Page 1: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/1.jpg)
IBM Research Lab in Haifa
Using Constraint Satisfaction Using Constraint Satisfaction Formulation and Solution Techniques Formulation and Solution Techniques for Random Test Program Generationfor Random Test Program Generation
Roy Emek
12-Sep-2002
![Page 2: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/2.jpg)
HRL
Random test program generationConstraint Satisfaction Problems (CSP)Modeling test programs as CSPCSP for random test generation: characteristicsSolution building blocks
Outline
[ Based on a paper by E. Bin, R. Emek, G. Shurek and A. Ziv ]
![Page 3: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/3.jpg)
HRL
Verification Through Simulation
Architecture Simulator
Design Simulator
Test program:InstructionsResource initializations
Actual behavior
Expected behavior
?
![Page 4: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/4.jpg)
HRL
Random Test Program Generator
Random Test Generator
System model:What's validWhat's interesting
User requirements
Generate N tests
N distinct testsValid, InterestingSatisfy user requirements
![Page 5: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/5.jpg)
HRL
Test Program Constraints
add R1 bbbb R2 + R3
load Rx bbbb 1000(Ry)???? ??, Rz
mult Rz bbbb R6 %%%% R7
User request: same register
Test quality: sum overflow
Validity: x !!!! y
![Page 6: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/6.jpg)
HRL
CSP Definition
Variables of the problemaddress, register_value
Domain (set) for each variableaddress: 0x0000 - 0xFFFFnumber of bytes in a 'load': { 1, 2, 4, 8, 16 }
Constraints (relations) over variables(load n bytes) u (align address to n bytes boundary)value(base_reg) + displacement = address
[ Mackworth, Freuder, Montanari, Dechter, Rossi, ...]
![Page 7: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/7.jpg)
HRL
CSP Definition (cont)
Solution for a CSPEvery variable is assigned a value from its domainThe assignments satisfy all the constraints
ExampleVariables: a, b, cDomains:
A = {1,2,3} ; B = {2,3,4,5} ; C = {1,3,5}Constraints:
a2 < b ; c g b ; a < c - 1Solution:
a = 1 ; b = 4 ; c = 3
![Page 8: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/8.jpg)
HRL
Random Test Program Generator (2)
Random Test Generator(CSP solver)
System model:What's validWhat's interesting
Generate N tests
N distinct tests (solutions)Valid, InterestingSatisfy user requirements
User Requirements
A Constraint Satisfaction
Problem
![Page 9: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/9.jpg)
HRL
CSP@RTG Characteristics
Random, uniform distribution solution [Yuan et al. '99, Dechter et al. '02]
As opposed to one, all, or 'best' solutionHuge domains: 264 and more
Example: address spaceRepresenting and operating on large sets becomes an issue
Hierarchy of constraints [Borning et al., '87]Mandatory: test case validityNon-mandatory: makes the test 'interesting'
![Page 10: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/10.jpg)
HRL
And we want it fast ...
![Page 11: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/11.jpg)
HRL
Solution Algorithm: Consistency - A Single Constraint
{1, 2, 3}X
R: (x,y,z) c X%Y%Z, x=y+z
{1, 2, 3}Y
{1, 2, 3}Z
{1, 2, 3} {1, 2, 3} {1, 2, 3}
![Page 12: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/12.jpg)
HRL
Solution Algorithm: Maintaining Arc Consistency (MAC)
The process: reducing domains to single-values1. Make all constraints locally consistent
Some constraints are handled repeatedlyAchieve fixed-point
2. Choose a variable: address3. Choose a value: address b 0x1234
0x1234 c domain ( address )4. Go to step 15. On failure - backtrack
Failure results in an empty set / domain
Arc = Constraint[ Mackworth, 1977 ]
![Page 13: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/13.jpg)
HRL
Consistency as Projection
C
A
B
1 C
A
B
2 C
A
B
3B'
A'
![Page 14: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/14.jpg)
HRL
Formula Based Constraint Projector
ProjectorA
B
C
A'B'
C'
C Constraint (formula)a = b + c 1 a
Reduced output sets:
Input sets:
MAC scheme – projectors for constraintsDeveloping arithmetic / logical / bit-wise projectors time after time again ?
a=b+c, a+b=c+d, a + b = c bit-xor d, ...Error prone, labour intensive
![Page 15: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/15.jpg)
HRL
Example: Projecting Disjoint Constraints
Constraint: (a=b) - (b=a+c) - (c=3$a)Domains: A={1,2,3}; B={3,4,5};C={4,5}
A B CInput domain
{1,2,3} {3,4,5} {4,5}
a=b {3} {3} {4,5}b=a+c {1} {5} {4}c=3a ! ! !Results {1,3} {3,5} {4,5}
(2) Join* sub-constraints projections
A B CInput domain
{1,2,3} {3,4,5} {4,5}
a=b {3} {3} -b=a+c {1} {5} {4}c=3a ! - !
(1) Project sub-constraints separately
![Page 16: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/16.jpg)
HRL
Domain (Set) Representation Example: bit-vectors
Origin of the challenge: large H/W resources128-bit registers64-bit wide memory address space
All the addresses such thataddr = base + displacement // architecturaladdr[3:6] = 01?1 // cache lineaddr c [0x2000 : 0x10FFF] // memory space
'Masks' (DNF) representation:01?1 d 0101, 0111
![Page 17: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/17.jpg)
HRL
Problem: Exponential Explosion
01010101 + 0?0?0?0? dddd {10101010, 01101010, 10011010, 01011010, 10100110, 01100110, ..., 10010101}
The general case: a + b dddd 2(n/2) clausesCoping with the problem
Binary Decision Diagrams (BDDs)Sometimes: space explosion
Approximations
We only have partial solutions
![Page 18: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/18.jpg)
HRL
Summary
Viewing test generation as CSPCharacteristics: random, huge domainsSolution scheme
Consistency based
HRL's test generators are CSP based The basis for test generators for all the processors designed in IBM
![Page 19: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/19.jpg)
IBM Research Lab in Haifa
Using Constraint Satisfaction Using Constraint Satisfaction Formulation and Solution Techniques Formulation and Solution Techniques for Random Test Program Generationfor Random Test Program Generation
End of Presentation
12-Sep-02
![Page 20: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/20.jpg)
HRL
Floating Point Operations – Stochastic Approach
a$$$$2✍✍✍✍ op b$$$$2✎✎✎✎ = c$$$$2✏✏✏✏op: +, -, $, ...Limited number of bits: non-continuous domain, rounding
mantissa:53 exp:11op
mantissa:53 exp:11
mantissa:53 exp:11Constraints:'op' itself bit #n = '0' Number of '1's = m a c [a1 ... a2]
Stochastic solution scheme: assign random 64 %%%% 3 bitsHill-climbing
Simple heuristicsLocal maximum: flip random bitsAfter some time - give up and start all over again
![Page 21: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/21.jpg)
HRL
== ++
Load DS(RA) →→→→ RT
RegisterMemory
Instruction
Operands Format
Register Immediate
Address Operands
Address DataData
Instruction ConstraintsModelling a PowerPC instruction
![Page 22: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/22.jpg)
HRL
Problem Partition, Dynamic Modeling
Dynamic problem structure
?
Partitioning the problemEasier to model, easier to solveHard to handle interdependencies
![Page 23: Using Constraint Satisfaction ... - research.ibm.com€¦ · IBM Research Lab in Haifa Using Constraint Satisfaction Formulation and Solution Techniques for Random Test Program Generation](https://reader033.vdocuments.net/reader033/viewer/2022060520/604f1d488c0a52062474844e/html5/thumbnails/23.jpg)
HRL
CSP Definition
Variables of the problemaddress, register_value
Domain (set) for each variableaddress: 0x0000 - 0xFFFFnumber of bytes in a 'load': { 1, 2, 4, 8, 16 }
Constraints (relations) over variables(load n bytes) u (align address to n bytes boundary)value(base_reg) + displacement = addresslast_instruction = "branch" ?
yes: PC = branch-targetno : PC = increase (last_instruction_address)
[ Mackworth, Freuder, Montanari, Dechter, Rossi, ...]