distributed verification of multi-threaded c++ programs
DESCRIPTION
Distributed Verification of Multi-threaded C++ Programs. Stefan Edelkamp joint work with Damian Sulewski and Shahid Jabbar. Motivation: IO-HSF-SPIN. Same states in both parts. Arrives at the final state. Large jumps due to 2nd heuristic. Current state. Already seen final state. - PowerPoint PPT PresentationTRANSCRIPT
Distributed Verification Distributed Verification of Multi-threaded C++ of Multi-threaded C++
ProgramsPrograms
Distributed Verification Distributed Verification of Multi-threaded C++ of Multi-threaded C++
ProgramsPrograms
Stefan Edelkamp
joint work with Damian Sulewskiand Shahid Jabbar
Motivation: IO-HSF-SPIN
Arrives at the final
state
Arrives again at
same final state
Same states in
both parts
Current state
Already seen final
state
Large jumps due
to 2nd heuristic
2.9 TB20 days1 node
----8 days
3 nodes
Overview
• Software Checking in StEAM
Externalization
Virtual Addresses
Parallelization
Overview
•Software Checking in StEAM
Externalization
Virtual Addresses
Parallelization
Software Checking
• Advantages+ Building a model unnecessary
+ Learning specification language unnecessary + Checking can be done more often
Disadvantages
- Code has to be executed
- Huge number of states
- Huge states
StEAM
• Can check concurrent C++ programs Uses a virtual machine for execution supports BFS, DFS, Best-First, A*, IDA* finds
Deadlocks Assertion Violations Segmentation Faults
Objectcode
StEAM - Checking a C++ Program
igccCompiler
Model checker
Virtual Machine
char globalChar;
int globalBlocksize = 7;
int main(){allocateBlock(blocksize);
}
void allocateBlock(int size){
void *memBlock;
memBlock = (void *) malloc(size);}
StEAM - Interpreting the Object Code
char globalChar;
int globalBlocksize = 7;
int main(){allocateBlock(blocksize);
}
void allocateBlock(int size){
void *memBlock;
memBlock = (void *) malloc(size);}
Register
BSS Section
Data Section
Text Section
Stack
Memory Pool
ICVM Virtual Machine
Objectcode
StEAM – Generating States
Register
BSS Section
Data Section
Text Section
Stack
Memory Pool
ICVM Virtual Machine StEAM
Register
BSS Section
Data Section
Text Section
Stack
Memory Pool
Initial StateRegister
BSS Section
Stack
Memory Pool
State 1Register
BSS Section
Data Section
Stack
State 2
Overview
•Software Checking in StEAM
Externalization
Virtual addresses
Parallelization
Externalization - Motivation
Internal
External
time
problem size
Externalization – Mini States
• pointer to a state in RAM or on Disk
pointer to the predecessor mini state
constant size
DiskRAM
[EJMRS 06]
Externalization – Expanding a State
Mini States Secondary MemoryCache
Internal Memory
Externalization – Flushing the Cache
Mini States Secondary MemoryCache
Internal Memory
Externalization – Collapse Compression
Register
BSS Section
Data Section
Text Section
Stack
Memory Pool
State Caches Files on Disk
Overview
•Software Checking in StEAM
Externalization
Virtual Addresses
Parallelization
Virtual Addresses • programs request memory memory assignment done by system
moving program between nodes impossible two possible strategies
converting the addresses before executing
using virtual addresses
Virtual Addresses – Memory Management
Stack
Stack pointer
Text BSS Data
Program counter
Memory pool
0
RAM
real address: x
virtual address: y
yx, size
AVL-Tree
Stack pointer
Virtual Addresses - Overhead
real
virtual
nodes
time
Overview
•Software Checking in StEAM
Externalization
Virtual Addresses
Parallelization
Parallelization – Motivation
Distributed (Shared) Memory MPI channels/shared RAM communication
Sending full states too expensive (if not used for expansion) Exploit externalization DualChannel (Speedup vs. Load Balance)Appropriate State Space Partitioning
Parallelization – Dual Channel
Communication
Parallelization – Hash Partitioning
Partitioning by hashing full stateProblem: Successors often not in same
partition high communication overhead
Partitioning by hashing partial state,e.g. memory pool
Problem: Too many states map to one hash value Load balancing
Parallelization – Incremental Tree
Hashing
h(3,1) = 3*3+1*9 mod 17= 1
h(1,2) = 1*3+2*9 mod 17 = 4
h(2,2,1,2) = 9 = 6+h(2,1,2)*3^1 =6+1*3 mod 17
h(2) = 2*3^1 mod 17= 6
h(s) = (Σi si 3^i) mod 17
h(1,2,3,1,2,2,1,2) = 4+1*3^2 + 9*3^(2+2) mod 17 = 11
[EM05]
Parallelization – Search Partitioning
DFS[Holzman & Bosnacki 2006]
Best-First, A*
horizontal slices vertical slices
Parallelization - Hardware
• Cluster Vision System (PBS)• Linux Suse 10.0• MPI via infiniband• Files via GBit Ethernet• 224 nodes (464 procs), < 15 used • AMD Opteron DP 50 (2.4 GHz)
Experiments: 15-Puzzle Partial Hash
time
nodes
speedup
Experiments – Depth-First Slicing 200
Philosopherstime
processors
Top Result: 600 Phils / 6 nodes
97 KB /stateEx Collapse
Compression & Distribution
16GB 1.5 GB per node
Experiments - Bath-Tub Effect (50 phils-
avg.)Time
Size of Depth Layer
validates Holzmann &
Bosnacki
Experiment - Shared Memory Bakery
(pthread)• 4 Opteron MP 852 (2.6 GHZ)
nodes
speedup
time
Conclusion
Preceeding Work: Full Externalization of States, inIO-HSF-SPIN Constant-Size RAM, e.g. 1.8 GB RAM, 20 days 1 proc, 8 days 4 procs, 2.9TB disk [EJ06], Distribution via (g+h)-Value
Problem: Huge & Highly Dynamic States Solution: Mini States as Constant Size Finger
Prints of States in RAM for Dual-Channel Communication to combine External and Parallel Search with Memory-Pool, Best-First Slicing Partitioning