joão lourenço diogo sousa vasco pessanha - ibm research · , 2010 / 1 joão lourenço diogo sousa...
TRANSCRIPT
![Page 1: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/1.jpg)
Haifa Verification Conference, October 4, 2010 / 1
João Lourenço Ricardo Dias Diogo Sousa Bruno Teixeira Ricardo Pinto Vasco Pessanha Center for Informatics and Information Technologies Computer Science Department New University of Lisboa, Portugal
![Page 2: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/2.jpg)
Haifa Verification Conference, October 4, 2010 / 2
Agenda
1. Software Transactional Memory: what, why and how?
2. Detection of dataraces in Transactional Memory
3. Visualization of Transactional Memory computations
4. Final notes
![Page 3: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/3.jpg)
Haifa Verification Conference, October 4, 2010 / 3
The problem
Multicores are here to stay
Moore’s law
Power consumption
Home computers are multicore now
Soon will be manycore
![Page 4: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/4.jpg)
Haifa Verification Conference, October 4, 2010 / 4
The problem
Computers are data processing tools
Data may be scattered among different computing units
A set of data manipulations may be inter-related and inter-dependent
The isolated change of a data item may leave the overall system in an inconsistent state
![Page 5: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/5.jpg)
Haifa Verification Conference, October 4, 2010 / 5
Parallel programming is hard
Multiple (asynchronous) control flows
Concurrency control
Synchronization
Contention management
The problem
![Page 6: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/6.jpg)
Haifa Verification Conference, October 4, 2010 / 6
For developing parallel code we need
Parallel programming skills
Programming languages constructs
System / run-time support
Techniques and tools for (parallel) program development
The problem
![Page 7: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/7.jpg)
Haifa Verification Conference, October 4, 2010 / 7
Programming Constructs / / System Support
Blocking methodologies
Locks
Condition variables
Semaphores
Non-blocking algorithms
e.g., java.util.concurrency
![Page 8: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/8.jpg)
Haifa Verification Conference, October 4, 2010 / 8
Locks: Insert in a Double Linked List
Pred
Succ
Pred
Succ
Pred
Succ
Pred
Succ
Pred
Succ
![Page 9: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/9.jpg)
Haifa Verification Conference, October 4, 2010 / 9
Coarse grain locks public synchronized void insertNode (node, precedingNode) { node.prec = precedingNode; node.succ = precedingNode.succ; precedingNode.succ.prec = node; precedingNode.succ = node; }
Locks: Insert in a Double Linked List
Pred
Succ
Pred
Succ
Pred
Succ
Pred
Succ
![Page 10: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/10.jpg)
Haifa Verification Conference, October 4, 2010 / 10
Fine grain locks public void insertNode (node, precedingNode) { synchronized (precedingNode) { synchronized (precedingNode.succ) { node.prec = precedingNode; node.succ = precedingNode.succ; precedingNode.succ.prec = node; precedingNode.succ = node; } } }
Locks: Insert in a Double Linked List
Pred
Succ
Pred
Succ
Pred
Succ
Pred
Succ
![Page 11: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/11.jpg)
Haifa Verification Conference, October 4, 2010 / 11
Fine grain locks lead to deadlocks
Fine grain locks synchronized (precedingNode) { synchronized (precedingNode.succ) { … } }
synchronized (precedingNode.succ) { synchronized (precedingNode) { … } }
Deadlock
![Page 12: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/12.jpg)
Haifa Verification Conference, October 4, 2010 / 12
Problem: locking granularity
Coarse grain locks
Threads lock each other out
Lower concurrency
Fine grain locks
Higher overhead
Risk of deadlocks
Lower maintainability
![Page 13: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/13.jpg)
Haifa Verification Conference, October 4, 2010 / 13
Example: bank account transfer
Transfer 50! from account A to account B
1. read (A);
2. A = A – 50;
3. write (A);
4. Read (B);
5. B = B + 50;
6. write (B);
Fault
![Page 14: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/14.jpg)
Haifa Verification Conference, October 4, 2010 / 14
Example — Atomicity
Transfer 50! from account A to account B
1. read (A);
2. A = A – 50;
3. write (A);
4. Read (B);
5. B = B + 50;
6. write (B);
Atomicity requirement — if the transaction fails after step 3 and before step 6, the system must ensure that that updates are not reflected in the database
![Page 15: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/15.jpg)
Haifa Verification Conference, October 4, 2010 / 15
Example — Consistency
Transfer 50! from account A to account B
1. read (A);
2. A = A – 50;
3. write (A);
4. Read (B);
5. B = B + 50;
6. write (B);
Consistency requirement — the sum of A and B is unchanged by the execution of the transaction
![Page 16: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/16.jpg)
Haifa Verification Conference, October 4, 2010 / 16
Example — Isolation
Transfer 50! from account A to account B
1. read (A);
2. A = A – 50;
3. write (A);
4. Read (B);
5. B = B + 50;
6. write (B);
Isolation requirement — no other successful transaction can see an intermediate state of this transaction (where the temporary sum of the accounts is less than it should)
![Page 17: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/17.jpg)
Haifa Verification Conference, October 4, 2010 / 17
Example — Durability
Transfer 50! from account A to account B
1. read (A);
2. A = A – 50;
3. write (A);
4. Read (B);
5. B = B + 50;
6. write (B);
Durability requirement — once the user has been notified that the transaction has completed (i.e., the transfer of the 50! has taken place), the updates to the database by the transaction must persist despite failures
![Page 18: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/18.jpg)
Haifa Verification Conference, October 4, 2010 / 18
Properties
Consistency — the transaction must obey legal protocols
Atomicity — it either happens or it does not; either all are bound by the contract or none are
Isolation — transactions must behave as if executed alone in the system
Durability — once a transaction is committed, it cannot be abrogated
![Page 19: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/19.jpg)
Haifa Verification Conference, October 4, 2010 / 20
Fine grain locks public void insertNode (node, precedingNode) { synchronized (precedingNode) { synchronized (precedingNode.succ) { node.prec = precedingNode; node.succ = precedingNode.succ; precedingNode.succ.prec = node; precedingNode.succ = node; } } }
Locks: Insert in a Double Linked List
Pred
Succ
Pred
Succ
Pred
Succ
Pred
Succ
![Page 20: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/20.jpg)
Haifa Verification Conference, October 4, 2010 / 21
Transactional memory
A possible solution !"#$%&&'&(")$()"*+,-'&.+$/"#+*01(*#+-'1+2'"%$#3%".')&*+,'1(#4*$'1$(*#+&'56#4%('7899-'Shavit & Touitou 1995, Herlihy et al. 2003:
Transactional memory public void insertNode (node, precedingNode) { atomic { //means “transaction” node.prec = precedingNode; node.succ = precedingNode.succ; precedingNode.succ.prec = node; precedingNode.succ = node; } }
![Page 21: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/21.jpg)
Haifa Verification Conference, October 4, 2010 / 22
Transactional memory
Transactional operations Established in the domain of databases Easy to understand
Transaction Series of read and write operations Executed at least atomically and isolated (AI from ACID properties) Performance is an issue
On conflict Transaction is repeated
![Page 22: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/22.jpg)
Haifa Verification Conference, October 4, 2010 / 24
Transactional memory
Advantages Developer worries less about parallelism
Code is marked as atomic/isolated
Code is generated and run-time ensure the properties
Disadvantages Not always ideal
Successive collisions repetitive rollbacks
Memory transactions can not contain certain operations (e.g., I/O operations)
![Page 23: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/23.jpg)
Haifa Verification Conference, October 4, 2010 / 25
Expectations
Preserving real-time ordering The conceptual point in time a transaction appear to execute lies within its lifespan
i.e., transactions do no execute “in the past” neither “in the future”
Precluding inconsistent views Assumption 1: transactions that access an inconsistent memory state will generate an exception Assumption 2: transactions that generate exceptions are aborted
![Page 24: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/24.jpg)
Haifa Verification Conference, October 4, 2010 / 31
Types of Actions in STM
Non-transactional operations:
Action whose effects are observed by an entity external to the system
Killing a process
user seeing a printf
Irreversible and cannot be undone
![Page 25: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/25.jpg)
Haifa Verification Conference, October 4, 2010 / 32
Types of Actions in STM
Transactional operations:
Pure transactional
Storing a value in memory
Execute a gettimeofday() system call
Revertible / Compensable
Memory management operations
Execute a lseek() system call
Appending data to a file
Execute the creat() system call
![Page 26: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/26.jpg)
Haifa Verification Conference, October 4, 2010 / 33
Non-Transactional Operations in STM
Possible approaches: Don’t care
Leave to the programmer the responsibility to avoid or manage its use in TM blocks
Common approach for library-based TM frameworks
Do not allow The programming language type system do not allow its use inside TM blocks
Approach in Concurrent Haskell
IO operations are typified and statically verified by the compiler
e.g., putChar :: Char -> IO()
![Page 27: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/27.jpg)
Haifa Verification Conference, October 4, 2010 / 34
Non-Transactional Operations in STM
Possible approaches:
Enter a non-speculative mode
Enforce that transactions execution NTO will never abort
May use an optimistic approach, assuming transactions will not execute NTO, otherwise…
Wait for all other concurrent transactions to finish
Restart and run the transaction that will execute NTOs as the only transaction in the system
Once finished, resume the normal operation
![Page 28: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/28.jpg)
Haifa Verification Conference, October 4, 2010 / 35
Revertible / Compensable Operations in STM
Possible approaches:
Defer
Some operations may/must be postponed to the end of the atomic block
Compensate
Some operations are not revertible, but may be compensated later without compromising the application semantics
![Page 29: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/29.jpg)
Haifa Verification Conference, October 4, 2010 / 36
Handling Revertible Operations in STM
Deferred operations can be supported by transaction handlers
Handlers extend the transaction life cycle with new states
Example for memory transactions in next slide
![Page 30: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/30.jpg)
Haifa Verification Conference, October 4, 2010 / 38
Memory Transaction States
BEGIN
committed
active
partially committed
END aborted Transaction States
![Page 31: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/31.jpg)
Haifa Verification Conference, October 4, 2010 / 39
void *my_malloc(...) { void *m = malloc (...); register_pre_abort(free, m); }
#define my_free (m) \ register_pos_commit (free, m);
Postponed
free()
postponed free()
free local malloc()
Memory Transaction States
BEGIN
committed
active
partially committed
END aborted Transaction States
pre-abort pos-abort
prepare-commit
pre-commit
pos-commit
Handler System States
![Page 32: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/32.jpg)
Haifa Verification Conference, October 4, 2010 / 40
An Handler System
pre-abort
pos-abort
prepare-commit
pos-commit
pre-commit
![Page 33: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/33.jpg)
Haifa Verification Conference, October 4, 2010 / 41
void TxDBStart(...) { TxStart(); register_pre_abort(...); register_prepare_commit(...); register_pre_commit(...); }
Mem Tx ABORT
Mem Tx COMMIT
Database Transaction COMMIT
DB Tx ABORT
Unifying Memory and DB Transactions
BEGIN
committed
active
partially committed
END aborted Transaction States
pre-abort pos-abort
prepare-commit
pre-commit
pos-commit
Handler System States
![Page 34: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/34.jpg)
Haifa Verification Conference, October 4, 2010 / 42
A Static Approach for Detecting Concurrency Anomalies in Transactional Memory
![Page 35: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/35.jpg)
Haifa Verification Conference, October 4, 2010 / 43
Problem Statement
Concurrency errors have always been a problem
Dataraces Deadlocks Priority inversion Convoying ...
There are still errors observed in TM Low-level dataraces High-level dataraces No tools for TM
Our goal is to statically detect these errors
![Page 36: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/36.jpg)
Haifa Verification Conference, October 4, 2010 / 44
Low-level Dataraces (LLDR)
private boolean hasSpaceLeft() { return (list.size() < MAX_SIZE); }
private void store(Object obj) { list.add(obj); }
public void attemptToStore(Object obj) { if (hasSpaceLeft()) {
store(obj); } }
![Page 37: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/37.jpg)
Haifa Verification Conference, October 4, 2010 / 45
Low-level Dataraces (LLDR)
private boolean hasSpaceLeft() { return (list.size() < MAX_SIZE); }
private void store(Object obj) { list.add(obj); }
public void attemptToStore(Object obj) { if (hasSpaceLeft()) {
store(obj); } }
Low-level datarace:
list is read and concurrently
updated
![Page 38: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/38.jpg)
Haifa Verification Conference, October 4, 2010 / 46
Low-level Dataraces (LLDR)
private boolean hasSpaceLeft() { synchronized { return (list.size() < MAX_SIZE); } }
private void store(Object obj) { synchronized { list.add(obj); } }
public void attemptToStore(Object obj) { if (hasSpaceLeft()) {
store(obj); } }
![Page 39: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/39.jpg)
Haifa Verification Conference, October 4, 2010 / 47
High-level Dataraces (HLDR)
private boolean hasSpaceLeft() { synchronized { return (list.size() < MAX_SIZE); } }
private void store(Object obj) { synchronized { list.add(obj); } }
public void attemptToStore(Object obj) { if (hasSpaceLeft()) { // list may become full store(obj); } }
![Page 40: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/40.jpg)
Haifa Verification Conference, October 4, 2010 / 48
High-level Dataraces (HLDR)
private boolean hasSpaceLeft() { synchronized { return (list.size() < MAX_SIZE); } }
private void store(Object obj) { synchronized { list.add(obj); } }
public void attemptToStore(Object obj) { if (hasSpaceLeft()) { // list may become full store(obj); } }
High-level datarace:
accesses are synchronized but list can overflow
![Page 41: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/41.jpg)
Haifa Verification Conference, October 4, 2010 / 49
LLDR Detection
Automatically convert TM to lock based programs
New program is processed by a datarace detector
Detected dataraces are also present in TM program
Control with TM
Ø T Ø T
Ø Ø T T
DR DR DR
Control with Locks
Ø L1 Ø L1 L1
Ø Ø L1 L2 L1
DR DR DR DR
Ø : Unprotected access L : Lock-guarded access T : Transactional access
![Page 42: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/42.jpg)
Haifa Verification Conference, October 4, 2010 / 50
LLDR Detection
Automatic Transformer
(Polyglot based tool)
Datarace Detector (JChord)
TM-based Program
Lock-based (single lock) Program
Datarace Report
LLDR Detector
![Page 43: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/43.jpg)
Haifa Verification Conference, October 4, 2010 / 51
Example
private boolean hasSpaceLeft() { atomic { return (list.size() < MAX_SIZE); } } private void store(Object obj) { atomic { list.add(obj); } }
public static Object GLOCK;
private boolean hasSpaceLeft() { synchronized (GLOCK) { return (list.size() < MAX_SIZE); } } private void store(Object obj) { synchronized (GLOCK) { list.add(obj); } }
![Page 44: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/44.jpg)
Haifa Verification Conference, October 4, 2010 / 52
Validation: Lee-TM
Renowed benchmark
2 versions (lock- and TM-based)
Ran JChord in lock version 52 dataraces – nolocks and wrong-locks
Ran LLDR detector in TM version 48 dataraces – accesses outside transactional blocks
![Page 45: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/45.jpg)
Haifa Verification Conference, October 4, 2010 / 53
High-level Dataraces (HLDR)
private boolean hasSpaceLeft() { synchronized { return (list.size() < MAX_SIZE); } }
private void store(Object obj) { synchronized { list.add(obj); } }
public void attemptToStore(Object obj) { if (hasSpaceLeft()) { // list may become full store(obj); } }
High-level Anomaly:
list could overflow, although accesses are synchronized
![Page 46: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/46.jpg)
Haifa Verification Conference, October 4, 2010 / 54
HLDR Detection
TM-based Program
AJEX Parser
(Polyglot extension)
Symbolic Executor/ /Tracer
Pattern Matcher
Anomaly Reports
AST Traces
HLDR Detector
![Page 47: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/47.jpg)
Haifa Verification Conference, October 4, 2010 / 55
HLDR Detection (Example)
private boolean hasSpaceLeft() { atomic { return (list.size() < MAX_SIZE); } }
private void store(Object obj) { atomic { list.add(obj); } }
public void attemptToStore(Object obj) { if (hasSpaceLeft()) store(obj); }
public void run() { attemptToStore("I’m a misbehaving program."); }
![Page 48: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/48.jpg)
Haifa Verification Conference, October 4, 2010 / 56
Tracer (Example)
hasSpaceLeft()
IF
attemptToStore()
CALL: store()
CALL: attemptToStore()
ATOMIC R: list R: MAX_SIZE
ATOMIC R: list W: list
store()
END
-
CALL: hasSpaceLeft()
run()
![Page 49: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/49.jpg)
Haifa Verification Conference, October 4, 2010 / 57
Tracer (Example)
hasSpaceLeft()
ATOMIC R: list R: MAX_SIZE
ATOMIC R: list W: list
store()
run()
IF
CALL: store()
END
-
CALL: hasSpaceLeft()
CALL: attemptToStore()
![Page 50: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/50.jpg)
Haifa Verification Conference, October 4, 2010 / 58
Tracer (Example)
ATOMIC R: list R: MAX_SIZE
ATOMIC R: list W: list
store()
run()
IF
CALL: store()
END
-
CALL: hasSpaceLeft()
![Page 51: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/51.jpg)
Haifa Verification Conference, October 4, 2010 / 59
Tracer (Example)
ATOMIC R: list R: MAX_SIZE
run()
IF
END
- CALL: store()
ATOMIC R: list W: list
![Page 52: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/52.jpg)
Haifa Verification Conference, October 4, 2010 / 60
Tracer (Example)
ATOMIC R: list R: MAX_SIZE
ATOMIC R: list W: list
run()
IF
END
-
![Page 53: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/53.jpg)
Haifa Verification Conference, October 4, 2010 / 61
Tracer (Example)
ATOMIC R: list R: MAX_SIZE
ATOMIC R: list W: list
run()
IF
END
-
ATOMIC R: list R: MAX_SIZE
END
START
END
START
ATOMIC R: list R: MAX_SIZE
ATOMIC R: list W: list
Trace 1 Trace 2
![Page 54: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/54.jpg)
Haifa Verification Conference, October 4, 2010 / 62
Tracing: Calls and Flow Control
Method calls Target method is expanded (inlined)
If statement Both branches are considered, but independently
Loops 0, 1, and 2 iterations are considered
Recursive calls Method is expanded twice, avoid infinite expansions but yield all possible anomalies
![Page 55: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/55.jpg)
Haifa Verification Conference, October 4, 2010 / 63
HLDR Detection
TM-based Program
AJEX Parser
(Polyglot extension)
Symbolic Executor/ /Tracer
Pattern Matcher
Anomaly Reports
AST Traces
HLDR Detector
![Page 56: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/56.jpg)
Haifa Verification Conference, October 4, 2010 / 64
Anomaly Patterns
Relation between transactions may be inferred by data access patterns
Certain access patterns may indicate anomalies
![Page 57: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/57.jpg)
Haifa Verification Conference, October 4, 2010 / 65
Anomaly Patterns
Read(x) Read(y)
Write(x,y)
Thread 1
Thread 2
RwR Anomaly
Transaction1
Transaction2
Transaction3
Write(x) Write(y)
Read(x,y)
Thread 1
Thread 2
WrW Anomaly
Transaction1
Transaction2
Transaction3
Read(x) Write(x)
Write(x)
Thread 1
Thread 2
RwW Anomaly
Transaction1
Transaction2
Transaction3
![Page 58: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/58.jpg)
Haifa Verification Conference, October 4, 2010 / 66
Pattern Matcher (Example)
ATOMIC R: list R: MAX_SIZE
END
START START
ATOMIC R: list R: MAX_SIZE
ATOMIC R: list W: list
END
Trace 1 Trace 2
Anomaly Pattern: RwW
list is retrieved, and updated in another
transaction
START
ATOMIC R: list R: MAX_SIZE
ATOMIC R: list W: list
END
Trace 1’
R: list
W: list
W: list
![Page 59: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/59.jpg)
Haifa Verification Conference, October 4, 2010 / 67
Validation
Hand Rewritting
TM HLDR Detection (Polyglot)
Well-known Lock-based Buggy Programs
TM-based Buggy Programs
Anomaly Reports
![Page 60: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/60.jpg)
Haifa Verification Conference, October 4, 2010 / 68
Coordinates’03
//Thread 3 public void run() { double x = c.getX(); //atomic double y = c.getY(); //atomic // use x and y... }
//Thread 1 public void run() { c.setXY(...); //atomic }
//Thread 4 public void run() { Coord d4 = c.getXY(); double x4 = d4.getX(); double y4 = d4.getY(); }
Anomaly: RwR
Coordinates could be from different states
False Positives
pointer analysis required
![Page 61: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/61.jpg)
Haifa Verification Conference, October 4, 2010 / 69
Counter
public class Counter { int i; int inc(int a) { atomic { i = i + a; return i; } } }
// Thread code public void run() { int i = c.inc(0); //get i // value could have changed c.inc(i); //increment by i }
Anomaly: RwW
Updates could be lost
No False Positives
![Page 62: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/62.jpg)
Haifa Verification Conference, October 4, 2010 / 70
Results
Test Total Anomalies
Total Warnings
Correct Warnings
False Warnings
Missed Anomalies
![Page 63: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/63.jpg)
Haifa Verification Conference, October 4, 2010 / 71
Results
Total Anomalies
Total Warnings
Correct Warnings
False Warnings
Missed Anomalies
12 33 10 23 2
Summary for 14 tests performed
3 RwR 3 WrW 6 RwW
2 RwR 3 WrW 5 RwW
70% of total results 5 redundand reads 6 pointer analysis 10 pattern refinement 2 permanent
Due to Unavailable source-code 1 RwR 1 RwW
![Page 64: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/64.jpg)
Haifa Verification Conference, October 4, 2010 / 72
Monitoring and Visualizing Transactional Memory Programs
InForum 2010
![Page 65: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/65.jpg)
Haifa Verification Conference, October 4, 2010 / 73
Message Passing… “how did we used to do? ” Message Passing… “how did we used to do? ”
![Page 66: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/66.jpg)
Haifa Verification Conference, October 4, 2010 / 74
Shared Memory… “how did we used to do? ”
Clear lack of tools
Shared Memory… “how did we used to do? ”
![Page 67: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/67.jpg)
Haifa Verification Conference, October 4, 2010 / 75
Transactional Memory… “how did we used to do? ”
Total lack of tools
Transactional Memory… “how did we used to do? ”
![Page 68: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/68.jpg)
Haifa Verification Conference, October 4, 2010 / 76
Transactional Memory… “how do we do now? ” Transactional Memory… “how do we do now? ”
!"#$%$#&'()*+'",'-.$/")01+-1".'21,
!"#$%& !"#$%' !"#$%(
!"#$%(%'&)
!"#$%'%*+)
!"#$%&%*+)
!"#$%#&'()$*+,'"-*+#',
!
"
#
$
%
&
'
(
)
!*
!!
!"
!#
!$
!%
!&
!'
#.,"#/,*)0*$123,"*)0*",'"(,%
&+''&
!'
*+#"$
#
* "##
#
![Page 69: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/69.jpg)
Haifa Verification Conference, October 4, 2010 / 77
Requirements for Tracing in JTraceView
Keep hardware requirements Different hardware may mask some behaviors and trigger new ones
Log only well defined time-slots When to start to collect events and for how long Huge log files
Keep the global application behavior Same behavior pattern Low and constant overhead No additional synchronizations
![Page 70: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/70.jpg)
Haifa Verification Conference, October 4, 2010 / 78
Keep events in separate buffers
One buffer per thread
Avoids additional synchronizations
Use RDTSC (or similar) registers as a global clock
Clock drifting can be a problem
In many cases it is not
Provides info on real execution time for transactions
Keep global application behavior
Our Proposal for Tracing in JTraceView
![Page 71: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/71.jpg)
Haifa Verification Conference, October 4, 2010 / 79
Specification: Tx_Start (Tx_Read | Tx_Write)*
(Tx_Abort | Tx_Commit)
Run-time: Tx_Start (Tx_Read | Tx_Write)*
(Tx_Abort_Read | Tx_Abort_Write | Tx_Abort_Commit | Tx_Abort_User | Tx_Commit)
Behavior of Memory Transactions
![Page 72: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/72.jpg)
Haifa Verification Conference, October 4, 2010 / 80
Events for TM Tracing
timestamp • When?
eventID
• What?
threadID
• Where?
transID
• Whom? [ ? ]
All Events
Tx_Abort evType
• Why?
Tx_Read | Tx_Write varID
• Memory location
![Page 73: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/73.jpg)
Haifa Verification Conference, October 4, 2010 / 81
Keeping App Behavior
Linked List Red-Black Tree
0
50
100
150
200
250
300
1 2 4 8
10
00x
Op
era
tions/
seco
nd
Number of threads
List 1000 nodes
5% insert, 5% remove, 90% lookup45% insert, 45% remove, 10% lookup
0
5
10
15
20
25
1 2 4 8
10
00
x O
pe
ratio
ns/
seco
nd
Number of threads
List / Atomic Counter / 1000 nodes
5% insert, 5% remove, 90% lookup45% insert, 45% remove, 10% lookup
0
20
40
60
80
100
120
1 2 4 8
10
00
x O
pe
ratio
ns/
seco
nd
Number of threads
List / RDTSC Counter / 1000 nodes
5% insert, 5% remove, 90% lookup45% insert, 45% remove, 10% lookup
0
50
100
150
200
250
300
350
1 2 4 8
10
00
x O
pe
ratio
ns/
seco
nd
Number of threads
Red-Black / Atomic Counter / 1000 nodes
5% insert, 5% remove, 90% lookup45% insert, 45% remove, 10% lookup
0
500
1000
1500
2000
2500
1 2 4 8
10
00
x O
pe
ratio
ns/
seco
nd
Number of threads
Red-Black / RDTSC Counter / 1000 nodes
5% insert, 5% remove, 90% lookup45% insert, 45% remove, 10% lookup
0
1000
2000
3000
4000
5000
6000
1 2 4 8
10
00
x O
pe
ratio
ns/
seco
nd
Number of threads
Red-Black 1000 nodes
5% insert, 5% remove, 90% lookup45% insert, 45% remove, 10% lookup
Unmonitored
JTraceView tracing
Naïve tracing
![Page 74: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/74.jpg)
Haifa Verification Conference, October 4, 2010 / 82
Tracing System Output
Unmanageable by common
mortals
RB-Tree in CTL generates
0.5 KB per processor per milisecond
![Page 75: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/75.jpg)
Haifa Verification Conference, October 4, 2010 / 83
JTraceView: Visualization Tool for TM
JTraceView
Monitoring Layer
Trace file processor
Trace file analyzers
Visualization plugins
![Page 76: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/76.jpg)
Haifa Verification Conference, October 4, 2010 / 84
JTraceView: Visualization Tool for TM
JTraceView
Monitoring Layer
Trace file processor
Trace file analyzers
Visualization plugins
![Page 77: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/77.jpg)
Haifa Verification Conference, October 4, 2010 / 85
!"#$%#&'()$%*+#%',-*+)".*/,"&,$'#0,%
!"#$%& '#%()*
'#%()*+,-.
!"#$%&+//.
!"#$%&'()"*+'#,-#.)(/,*!#)"-
!"#$ %&'("
)*$"+, )*$"+- )*$"+.
0,$(1($2#3*+'#,-#.)(/,-
,
-,
.,
/,
0,
1,
2,
3,
4,
5,
-,,
4
3060/1
.16121
416011
-06101
-,,
Visualization Plugins
!"##$%&'()"*%&+),+)"%-+%-.+/-*.01&
!"##$%&'()*+,-. !"##$%&'()*+,-/ 01"*%&'()*+,-. 01"*%&'()*+,-/
232 /34 432 534 .232 ./34 .432 .534 /232 //34 /432 /534 6232%$#.
232
/34
432
534
.232
./34
.432
.534
/232
//34
/432
/534
6232
6/34
6432
6534
23#).*+"4+5"#
#$%&
!"#$"%%&
!"#$"%%&
' (' )' *' +' ,' -' .' /' 0' ('' ((''(#"
'
)1,''
,1'''
.1,''
('1'''
()1,''
(,1'''
(.1,''
)'1'''
))1,''
),1'''
).1,''
*'1'''
*)1,''
*,1'''
*.1,''
+'1'''
)*#+",-./-0.#
#('&
![Page 78: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/78.jpg)
Haifa Verification Conference, October 4, 2010 / 86
Example
!"#$%#&'()$*+,'"-*+#',
!"#$%&# '%(
)*+#,- )*+#,. )*+#,/
.$/(0(/1#2*!"#$%#&'()$%
-
0
.-
.0
/-
/0
1-
10
2-
20
0-
00
3-
30
4-
40
5-
50
#0,"#3,*)4*$156,"*)4*",'"(,%
16574
52
-60/5
/.
-6///
3
!"#$%#&'()$*+,'"-*+#',
!"#$%&# '%(
)*+#,- )*+#,. )*+#,/
.$/(0(/1#2*!"#$%#&'()$%
-
.
/
0
1
2
3
4
5
6
.-
..
./
.0
.1
.2
.3
.4
#0,"#3,*)4*$156,"*)4*",'"(,%
37443
.4
-70/1
0
-7/00
0
!"#$%#&'()$*+,'"-*+#',
!"#$%&# '%(
)*+#,- )*+#,. )*+#,/
.$/(0(/1#2*!"#$%#&'()$%
-
.
/
0
1
2
3
4
5
6
.-
..
./
.0
#0,"#3,*)4*$156,"*)4*",'"(,%
-72/
6
-7256
.0
-7.13
0
!"#$%#&'()$*+,'"-*+#',
!"#$%&# '%(
)*+#,- )*+#,. )*+#,/
.$/(0(/1#2*!"#$%#&'()$%
-0--
-0/1
-01-
-021
.0--
.0/1
.01-
.021
/0--
#0,"#3,*)4*$156,"*)4*",'"(,%
-0--3
/
-0--3
/
-0--/
.
LL, 8, 20%, 20%, 60%,
50
LL, 8, 20%, 20%, 60%,
10000
RB, 8, 45%, 45%, 10%,
50
RB, 8, 45%, 45%, 10%,
10000
![Page 79: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/79.jpg)
Haifa Verification Conference, October 4, 2010 / 87
STAMP
Stanford Transactional Applications for Multi-Processing
Collection of applications for TM benchmarking
Includes both sequential and parallel code sections
Eight different applications
Six were ported to Java (DeuceSTM framework)
http://stamp.stanford.edu
![Page 80: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/80.jpg)
Haifa Verification Conference, October 4, 2010 / 88
STAMP: Intruder
Signature-based network intrusion detection systems (NIDS) scan network packets for matches against a known set of intrusion signatures
Properties:
Short transactions length
Medium read/write sets
High contention
Medium transaction time
![Page 81: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/81.jpg)
Haifa Verification Conference, October 4, 2010 / 89
Trace Data
Collected in a Sun Fire x4600 8 x AMD Opteron Model 8220 dual-core processor @ 2.8GHz 32 GB DDR2-667 memory Linux Debian lenny
Test Conditions TM algorithm: TL2 Run time: 20 secs Trace File Size: 37MB Total transactions: 20814 Total commit: 14254 Total abort: 6560
![Page 82: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/82.jpg)
Haifa Verification Conference, October 4, 2010 / 90
STAMP: Intruder
Includes three transactional code blocks
Code 0 — atomicGetPacket (streamPtr); Pop data from a queue
Code 1 — atomicProcess (decoderPtr, packetPtr); Assemble packet segments and decode them
Code 2 — atomicGetComplete (decoderPtr, decodedFlowId); Get decoded data of the packet
![Page 83: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/83.jpg)
Haifa Verification Conference, October 4, 2010 / 91
Transactional Code Blocks
![Page 84: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/84.jpg)
Haifa Verification Conference, October 4, 2010 / 92
Read / Write Rates
![Page 85: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/85.jpg)
Haifa Verification Conference, October 4, 2010 / 93
Abort Rate
![Page 86: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/86.jpg)
Haifa Verification Conference, October 4, 2010 / 94
Abort Types
![Page 87: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/87.jpg)
Haifa Verification Conference, October 4, 2010 / 95
Abort — False Conflicts
![Page 88: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/88.jpg)
Haifa Verification Conference, October 4, 2010 / 96
![Page 89: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/89.jpg)
Haifa Verification Conference, October 4, 2010 / 97
Transaction Abort / Retry
![Page 90: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/90.jpg)
Haifa Verification Conference, October 4, 2010 / 98
![Page 91: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/91.jpg)
Haifa Verification Conference, October 4, 2010 / 99
Duration of Transactions
![Page 92: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/92.jpg)
Haifa Verification Conference, October 4, 2010 / 100
Thread timeline
![Page 93: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/93.jpg)
Haifa Verification Conference, October 4, 2010 / 101
Thread timeline
![Page 94: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/94.jpg)
Haifa Verification Conference, October 4, 2010 / 102
Concluding Remarks
Transactional Memory is an appealing paradigm for specifying concurrent programs
We need software development tools specifically targeting TM
Traditional tools do not apply to TM
Transactional Memory programs may also exhibit dataraces
Low-level dataraces: only in weak-isolation High-level dataraces: always
![Page 95: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/95.jpg)
Haifa Verification Conference, October 4, 2010 / 103
Concluding Remarks
It is possible to detect LLDR’s in TM by analyzing “equivalent” lock-based programs
Most HLDR’s occur when a transaction runs between two consecutive transactions of another thread
The three patterns identified capture most of the anomalies referenced in the literature
Good detection rates for a set of well known examples from different sources
![Page 96: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/96.jpg)
Haifa Verification Conference, October 4, 2010 / 104
Concluding Remarks
JTraceView includes a low intrusion monitoring layer
Deals with the very high frequency of events and huge logs
The huge logs can be visually synthesized Statistical charts
Time-space diagram
Time-space view may also help in debugging
![Page 97: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/97.jpg)
Haifa Verification Conference, October 4, 2010 / 105
Want to try it yourself?
Get: TM framework instrumented to generate logs (tracing data) JTraceView to visualize tracing data
From: http://asc.di.fct.unl.pt/trxsys
Check the prototypes page
When: From October 16th
![Page 98: João Lourenço Diogo Sousa Vasco Pessanha - IBM Research · , 2010 / 1 João Lourenço Diogo Sousa Vasco Pessanha Center for Informatics and Information Technologies Computer Science](https://reader034.vdocuments.net/reader034/viewer/2022052610/5c10fa4c09d3f294068b5c9f/html5/thumbnails/98.jpg)
Haifa Verification Conference, October 4, 2010 / 106
The End…
Have fun!