philip w. howard with jonathan walpole, josh triplett, paul e. mckenney
DESCRIPTION
The Ordering Requirements of Relativistic and Reader-Writer Locking Approaches to Shared Data Access. Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney. Outline. The Problem The RWL Solution The RP Solution Other problems and their solutions Multiple Writers - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/1.jpg)
11/2/2011 1
The Ordering Requirements of Relativistic and
Reader-Writer Locking Approaches to Shared Data Access
Philip W. Howard
with
Jonathan Walpole, Josh Triplett, Paul E. McKenney
![Page 2: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/2.jpg)
11/2/2011 2
Outline
• The Problem
• The RWL Solution
• The RP Solution
• Other problems and their solutions
• Multiple Writers
• Performance
![Page 3: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/3.jpg)
11/2/2011 3
The Problem
A B C
A C
![Page 4: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/4.jpg)
11/2/2011 4
The Problem
obtain ref drop ref
unlink reclaim
obtain ref drop ref
Reader 1
Reader 2
Writer
![Page 5: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/5.jpg)
11/2/2011 5
The Problem
void init(stuff *node)
{
node->a = COMPUTE_A;
node->b = COMPUTE_B;
node->initd = TRUE;
}
while (!node->initd)
{}
if (node->a) drop_nuke();
if (node->b) drop_napalm();
![Page 6: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/6.jpg)
11/2/2011 6
The Problem
void init(stuff *node)
{
int value;
value = some_computation;
node->a = COMPUTE_A;
node->b = COMPUTE_B;
node->initd = value;
}
![Page 7: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/7.jpg)
11/2/2011 7
The Problem
void init(stuff *node)
{
node->a = COMPUTE_A;
node->b = COMPUTE_B;
node->initd = TRUE;
}
while (!node->initd)
{}
if (node->a) drop_nuke();
if (node->b) drop_napalm();
![Page 8: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/8.jpg)
11/2/2011 8
The Problem
• Compiler reordering
• CPU reordering
• Memory System reordering
![Page 9: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/9.jpg)
11/2/2011 9
How are these dependencies maintained?
obtain ref drop ref
unlink reclaim
obtain ref drop ref
Reader 1
Reader 2
Writer
![Page 10: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/10.jpg)
11/2/2011 10
How does → work?
Thread 1
A: a=1;
Thread 2
B: if (a)
A → B
else
B → A
![Page 11: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/11.jpg)
11/2/2011 11
How does work?
Thread 1
A: a=1;
mb();
C: c=1;
Thread 2
while (!a)
B: A B
Thread 3
if (c)
D: A D
![Page 12: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/12.jpg)
11/2/2011 12
With Reader-Writer Locks
obtain ref drop ref
unlink reclaim
read-lock read-unlock
write-lock write-unlock
![Page 13: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/13.jpg)
11/2/2011 13
With Reader-Writer Locks
obtain ref drop ref
unlink reclaim
read-lock read-unlock
write-lock
write-unlock
![Page 14: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/14.jpg)
11/2/2011 14
With Reader-Writer Locks
obtain ref drop ref
unlink reclaim
read-lock
read-unlock
write-lock write-unlock
![Page 15: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/15.jpg)
11/2/2011 15
Locking primitives must
• Impose the semantics of the lock
• Prevent compiler, CPU, or Memory system from reordering operations across them
![Page 16: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/16.jpg)
11/2/2011 16
With Relativistic Programming
obtain ref drop ref
start-wait end-wait
start-read end-read
unlink reclaim
![Page 17: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/17.jpg)
11/2/2011 17
With Relativistic Programming
obtain ref drop ref
start-wait end-wait
start-read end-read
unlink reclaim
![Page 18: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/18.jpg)
11/2/2011 18
RP primitives must
• Impose the semantics of wait-for-readers
• Prevent compiler, CPU, or Memory system from reordering operations across them
![Page 19: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/19.jpg)
11/2/2011 19
Outline• The Problem• The RWL Solution• The RP Solution• Other problems and their solutions
• Insert• Move Down• Move Up• General Case
• Multiple Writers• Performance
![Page 20: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/20.jpg)
11/2/2011 20
Insert
C D E
C E
![Page 21: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/21.jpg)
11/2/2011 21
Insert
obtain ref deref
init link
obtain ref deref
Reader 1
Reader 2
Writer
![Page 22: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/22.jpg)
11/2/2011 22
How does work?
Thread 1
A: a=1;
mb();
C: c=1;
Thread 2
while (!a)
B: A B
Thread 3
if (c)
D: A D
![Page 23: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/23.jpg)
11/2/2011 23
Insert
Use rp-publish to perform link operation
![Page 24: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/24.jpg)
11/2/2011 24
Move Down
A
B
C
D
F
G
E
H
A
B
C
D
F
GE
H
![Page 25: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/25.jpg)
11/2/2011 25
Move Down
A
B
C
D
F
G
E
H
A
B
C
D
F
GE
H
F’
A
B
C
D
GE
H
F’
1. init F’2. link F’3. unlink F
![Page 26: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/26.jpg)
11/2/2011 26
Move Downinit(F’) link(F’) unlink(F) reclaim(F)
deref(H) deref(F) deref(D) deref(E)
A
B
C
D
F
G
E
H
A
B
C
D
F
GE
H
F’
A
B
C
D
GE
H
F’
![Page 27: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/27.jpg)
11/2/2011 27
Move Downinit(F’) link(F’) unlink(F) reclaim(F)
deref(H) deref(F) deref(D) deref(F’)
A
B
C
D
F
G
E
H
A
B
C
D
F
GE
H
F’
A
B
C
D
GE
H
F’
deref(E)
![Page 28: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/28.jpg)
11/2/2011 28
Move Downinit(F’) link(F’) unlink(F) reclaim(F)
deref(H) deref(D) deref(F’)
A
B
C
D
F
G
E
H
A
B
C
D
F
GE
H
F’
A
B
C
D
GE
H
F’
deref(E)
![Page 29: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/29.jpg)
11/2/2011 29
RBTree Delete with Swap (Move Up)
D
A
B
E
C
F
null D
A
B
E
C
F
null
C’
A E
D
F
C’
![Page 30: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/30.jpg)
11/2/2011 30
Move Up
init(C’) link(C’) unlink(C) reclaim(C)
deref(F) deref(C’)
D
A
B
E
C
F
null D
A
B
E
C
F
null
C’
A E
D
F
C’
![Page 31: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/31.jpg)
11/2/2011 31
Move Upinit(C’) link(C’) unlink(C) reclaim(C)
deref(F) deref(B) deref(E)
D
A
B
E
C
F
deref(C)
null D
A
B
E
C
F
null
C’
A E
D
F
C’
![Page 32: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/32.jpg)
11/2/2011 32
The General Case
• Mutable vs. Immutable data
![Page 33: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/33.jpg)
11/2/2011 33
The General Case for Writers
• Copy nodes to update immutable data or to make a collection of updates appear atomic
• Use rp-publish to update mutable data
• Use wait-for-readers when updates are in traversal orderA B C
![Page 34: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/34.jpg)
11/2/2011 34
The General Case for Readers
• Use rp-read for reading mutable data
• Only dereference mutable data once
if (node->next->key == key) {
return node->next->value;
}
![Page 35: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/35.jpg)
11/2/2011 35
Outline
• The Problem
• The RWL Solution
• The RP Solution
• Other problems and their solutions
• Multiple Writers
• Performance
![Page 36: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/36.jpg)
11/2/2011 36
Multiple Writers
W
R2
R1
![Page 37: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/37.jpg)
11/2/2011 37
Multiple Writers
W1
Reader
W2
![Page 38: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/38.jpg)
11/2/2011 38
The phone company
Can’t wait to get my new phone
Can’t wait to get my new phone
Can’t wait to ditch thisphone
Can’t wait to ditch thisphone
Where’s my phone book?
Where’s my phone book?
![Page 39: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/39.jpg)
11/2/2011 39
Multiple Writers
W1
Reader
W2wait-for-readers
W1
Reader
W2
![Page 40: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/40.jpg)
11/2/2011 40
RP vs RWL delays
W1
Reader
W2
RP
W1
Reader
W2readerpref
W1
Reader
W2writerpref
W1
Reader
W2
TORP
![Page 41: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/41.jpg)
11/2/2011 41
Trigger Events
RP
RWLR
RWLW
![Page 42: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/42.jpg)
11/2/2011 42
Outline
• The Problem
• The RWL Solution
• The RP Solution
• Other problems and their solutions
• Multiple Writers
• Performance
![Page 43: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/43.jpg)
11/2/2011 43
How does work?
Thread 1
A: a=1;
mb();
C: c=1;
Thread 2
while (!a)
B: A B
Thread 3
if (c)
D: A D
![Page 44: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/44.jpg)
11/2/2011 44
Reader-Writer Locks
read-unlock
write-lock write-unlock
read-lock
read-unlockread-lock
![Page 45: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/45.jpg)
11/2/2011 45
Reader-Writer Locks
Reader
while (writing)
{}
reading=1;
Writer
while (reading)
{}
writing=1;
![Page 46: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/46.jpg)
11/2/2011 46
Relativistic Programming
start-wait end-wait
start-read end-read
start-read end-read
![Page 47: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/47.jpg)
11/2/2011 47
Relativistic Programming
start-read(i)
{
reader[i]=1;
}
end-read(i)
{
reader[i]=0;
}
start-wait()
{}
end-wait(){
for (i) { while (reader[i]) {} }}
![Page 48: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/48.jpg)
11/2/2011 48
Relativistic Programming
start-read(i)
{
reader[i]=Epoch;
}
end-read(i)
{
reader[i]=0;
}
start-wait()
{
Epoch++;
my_epoch = Epoch;
}
end-wait()
{
for (i) {
while (reader[i] &&
reader[i] < my_epoch)
{}
}
}
![Page 49: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/49.jpg)
11/2/2011 49
Performance Benefits to RP
• Less expensive read-side primitives
• Less serialization / more concurrency
• Less waiting
![Page 50: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/50.jpg)
11/2/2011 50
Benchmarked Methods
nolock No synchronization
rp Relativistic Programming
torp Totally Ordered RP
rwlr Reader-Writer Lock Read preference
rwlw Reader-Writer Lock Write preference
![Page 51: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/51.jpg)
11/2/2011 51
Read Performance (size=1)
0
50
100
150
200
250
0 5 10 15 20
Mill
ion
s
Threads
op
era
tion
s/se
c
NOLOCK
RP
RWLW
RWLR
![Page 52: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/52.jpg)
11/2/2011 52
Read Performance (size=1000)
0
0.5
1
1.5
2
2.5
3
3.5
0 5 10 15 20
Mill
ion
s
Threads
Ope
ratio
ns/
sec
NOLOCK
RP
RWLW
RWLR
![Page 53: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/53.jpg)
11/2/2011 53
Update Scalability
0
50000
100000
150000
200000
250000
1 10 100 1000 10000
List Size
Op
erat
ion
s/se
c
RP writes
TORP writes
![Page 54: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/54.jpg)
11/2/2011 54
Update Scalability part 2
0
200000
400000
600000
800000
1000000
1200000
1400000
1600000
1800000
2000000
1 10 100 1000 10000
List Size
Op
erat
ion
s/se
c
RP
TORP
![Page 55: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/55.jpg)
11/2/2011 55
Update Scalability (part 3)
1
10
100
1000
10000
100000
1000000
10000000
100000000
1 10 100 1000 10000 100000 1000000
List Size
Op
era
tio
ns
/se
c
TORP r
RWLW r
TORP
RWLW
![Page 56: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/56.jpg)
11/2/2011 56
Conclusions
• Correctness can be preserved by limiting allowable orderings
• RP read primitives are less expensive than RWL primitives
• RP allows more concurrency and less waiting
![Page 57: Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney](https://reader036.vdocuments.net/reader036/viewer/2022062323/56815861550346895dc5bd9d/html5/thumbnails/57.jpg)
11/2/2011 57
Conclusions
• RP can preserve both correctness and scalability or reads