Download - CSS434 Time & Global States1 CSS434 Time and Global States Textbook Ch11 Professor: Munehiro Fukuda
CSS434 Time & Global States 1
CSS434 Time and Global States CSS434 Time and Global States Textbook Ch11Textbook Ch11
Professor: Munehiro Fukuda
CSS434 Time & Global States 2
Outline Physical clock synchronization
Applications: make Logical clock synchronization
Parallel and distributed simulation Global states and consistent cuts
Distributed garbage collection, deadlock detection, distributed termination detection, and discrete-even simulation
Distributed debugging Checking if a captured snapshot is one of
transitory states we have considered.
CSS434 Time & Global States 3
Why Clock Synchronization
Computer clock: a counter decremented by a crystal oscillation. Single computers: all processes use the same clock. – No
problem Multiple computers: impossible to guarantee that the
crystals in different computers all run at exactly the same frequency.
Synchronization: Absolute (with real time)
Necessary for real-time applications such as on-line reservation systems
Relative (with each other) Required for those applications that need a consistent view
of time across all nodes.
Network
CSS434 Time & Global States 4
Clock SynchronizationPassive Centralized Algorithms – Christian’s
Algorithm
Assumption: processing time has been measured or estimated
Message_delay = (T1 – T0 – processing)/2 New client time = T + message_delay Improvements:
Average multiple measurements Discard outlying measurements
Client
Time serverTime
T0 T1
Time? Time=T
Processing
CSS434 Time & Global States 5
Clock SynchronizationActive Centralized Algorithm – Berkeley
Algorithm
Assumption: processing time has been measured or estimated Server: diff(i) = server_time – (ci_time + message_deley) Client: ci_time = ci_time + diff(i)
Time server
Client 1
Client 2
C1_time
C2_time
Time?
Time?
Diff(1)
Diff(2)
CSS434 Time & Global States 6
Clock SynchronizationDistributed Algorithm – Averaging Algorithm
Assumption: R is large enough to wait for all broadcast messages All nodes broadcast their time periodically Each node computes average. Improvement:
Discard outlying time messages. Exchange their time with their local neighbors.
Node 1
Node 2
Node 3
T0 T0 +R T0 +2R T0 +3R
N3_time=30
N3_time=30N2_time=32
N2_time=32
N1_time=31
N1_time=31
CSS434 Time & Global States 7
Clock SynchronizationNetwork Time Protocol
1
2
3
2
3 3
Note: Arrows denote synchronization control, numbers denote strata.
UDP message
UTC (Coordinated Universal Time)
CSS434 Time & Global States 8
Clock SynchronizationNetwork Time Protocol
Ti
Ti-1Ti -2
Ti- 3
Server B
Server A
Time
m m'
Time
tt’
di = t + t’ = Ti-2 – Ti-3 + Ti – Ti-1
2o = Ti-2 – Ti-3 – t + Ti-1 – Ti + t’o = (Ti-2 – Ti-3 + Ti-1 – Ti)/2 + (t’ – t)/2
Ti-2 = Ti-3 + t + o
o
Ti = Ti-1 + t’ - o
CSS434 Time & Global States 9
Event OrderingHappened-Before Relation
1. Event eki: The kth event of process i
2. Sequence hki : The history of process I through
to the event eki
3. Cause-and-effect e→e’: e proceeds e’.4. Parallel events e e’: e and e’ happen in parallel∥5. Happens-Before Relation:
1. If eki, el
i h∈ i and k < l, then eki → el
i,2. If ei = send(m) and ej = receive(m), then ei → ej,3. If e → e’ and e’ → e”, then e → e”
Most applications need not maintain the real-timesynchronized clock.
CSS434 Time & Global States 10
Event OrderingLogical Clock
1. ee’ LC(e) < LC(e’) for all events2. However, we cannot inferLC(e) < LC(e’) ee’
Example: LC(e21) > LC(e1
3) but e21 || e1
3
LC(ei) := (ei != receive(m)) ? LC + 1 : max(LC, TS(m)) + 1where TS(m) is the timestamp of message m:
P1
P2
P3
e11 e2
1
e12 e2
2
e13 e2
3
LC=1 LC=2
LC=3 LC=4
LC=1 LC=5
m1
m1
CSS434 Time & Global States 11
Event OrderingVector Clock
P1
P2
P3
e11 e2
1
e12 e2
2
e13 e2
3
m1
m1
(1,0,0) (2,0,0)(2,1,0) (2,2,0)
(0,0,1) (2,2,2)
Vi[I] = vi[i] + 1;Pi includes the value t = Vi in every message it sendsVi[j] = max(vi[j], t[j]) for j = 1,2,…,N
1. ee’ V(e) < V(e’)2. V(e) < V(e’) ee’
Example:neither V(e21) V(e1
3) nor V(e21) V(e1
3), and thus e21 || e1
3
CSS434 Time & Global States 12
p2p1
message
garbage object
objectreference
a. Garbage collection
p2p1 wait-for
wait-forb. Deadlock
p2p1
activatepassive passivec. Termination
Global StateApplications necessary to detect a correct global
state
This is not a garbage, becausethe in-transit message points to it
Both p1 and p2 are passive and thus seems ready to finish, but an in-transit message makes p1 active again.
CSS434 Time & Global States 13
Global StateConsistent Cut
Finding C such that (e C) (e’ → e) e’ C∈ ∧ ⇒ ∈
p1p1
p2p2
p3p3
p4p4
ee1111
ee1133
ee1122
ee2222 ee33
22
ee2211 ee22
11
ee2233
ee1144 ee22
44ee33
44
CC C’C’
(send)(send)
(receive)(receive)
CSS434 Time & Global States 14
Global StateDistributed Snapshot – Chandy/Lamport [1985]
A process that wants to take a snapshot sends a snapshot request to the others. Each process records its state upon receiving the first snapshot request. Each process keep recording the messages until receiving a snapshot request
from each of the other process except the one that has originally initiated a snapshot.
P0
P1 P2
P0
P1
P2
Snapshot request
Ordinary message
Message recording
s
mm m s
ss
s
CSS434 Time & Global States 15
Marker (Snapshot request) receiving rule for process pi
On pi’s receipt of a marker (snapshot request) message over channel c:if (pi has not yet recorded its state) it
records its process state now;records the state of c as the empty set;turns on recording of messages arriving over other incoming channels;
else pi records the state of c as the set of messages it has received over c since it saved its state.
end ifMarker (Snapshot request) sending rule for process pi
After pi has recorded its state, for each outgoing channel c: pi sends one marker message over c (before it sends any other message over c).
Global StateDistributed Snapshot – Chandy/Lamport [1985]
CSS434 Time & Global States 16
A Distributed Snapshot Example
p1 p2(empty)<$1000, 0> <$50, 2000>
(empty)
c2
c1
1. Global state S0
2. Global state S1
3. Global state S2
4. Global state S3
p1 p2(Order 10, $100), M<$900, 0> <$50, 2000>
(empty)
c2
c1
p1 p2(Order 10, $100), M<$900, 0> <$50, 1995>
(five widgets)
c2
c1
p1 p2(Order 10, $100)<$900, 5> <$50, 1995>
(empty)
c2
c1
(M = marker message)
Time line
Consistent Cut
Start recording its state
Record its stateMStop recording
CSS434 Time & Global States 17
Samadi’s Algorithm [1985]
p2p2
p1p1
p3p3
p0p0Take snapshotTake snapshot
16161212
1515
tagtag
Report 15Report 15Report 12Report 12
2020
ackack
Report 20Report 20
1. Each process returns an ack whenever receiving a message.2. Once receiving a snapshot message, each process returns a tag instead of an ack until a new
GVT is compute.3. When receiving a snapshot message, each process returns to P0 the minimum time among:
- the minimum timestamp among events that have not yet been processes. - the minimum timestamp among messages that have not yet been acknowledged.- the minimum timestamp among tags it has received.
2020
Done
CSS434 Time & Global States 18
Mattern’s Algorithm [1993]
p2p2
p1p1
p3p3
p4p4(0,0,0,0)(0,0,0,0) (0,1,0,0)(0,1,0,0)
(0,2,-1,0)(0,2,-1,0)
(0,0,0,1)(0,0,0,1)
(0,0,1,1)(0,0,1,1)
(0,0,1,0)(0,0,1,0)
(0,0,0,0)(0,0,0,0)
1st snapshot1st snapshot
2nd snapshot2nd snapshot
1. Process Pi maintains a vector counter: Vi[1..n].2. Pi writes in Vi[j] the number of messages sent to Pj.3. Pi subtract one from Vi[j] when receiving a message from Pj4. During the 1st circulation of a ‘take snapshot’ message, Pi performs:
C[1..n]+=Vi[1..n]; Vi[1..n] = 0Upon completing the 1st circulation, c[I] presents the number of messages in transit to Pi.
5. During the 2nd circulation, Pi wait for performs:C[i] = 0
-1 +1 -1
+1
+1
+1
-1
-1+1 -1
CSS434 Time & Global States 19
An Example: Parallel and Distributed Simulation
1cell/time unit1cell/5 time units
1cell/20 time units
Process 0
Process 3 Process 2
Process 1
CSS434 Time & Global States 20
An Example: Parallel and Distributed Simulation (Cont’d)
Process 0
Process 1
Process 2
Process 3
e2
e1e5 e6
e4e3
e7
e11e10e9e8 e12
e1
e1
attack
attack
Barrier per every simulation cycle
What drawbacks does this method have?
CSS434 Time & Global States 21
An Example: Parallel and Distributed Simulation (Cont’d)
Process 0
Process 1
Process 2
Process 3
e2
e1e5 e6
e4e3
e7
e11e10e9e8 e12
e1
e1
attack
attack
Discrete event simulation with optimistic synchronization
e2 e3
e13 e14
Rollback
Rollback
Old event history kept so as to rollback computation.
When can we garbage collect such history?
CSS434 Time & Global States 22
Time Warp[Jefferson 1985]Optimistic Distributed Simulation• Each process has an input message, an output message, and an event history queue.• When a process receives a message whose timestamp is older than its local time:
1. Roll back its local event execution to that old timestamp. 2. Roll back its receipt of input messages whose timestamp is newer than that old
timestamp.3. Send anti-messages to cancel all emanated messages whose timestamp is newer than that old timestamp.
• GVT (Global Virtual Time): is periodically computed to garbage-collect all the executed events whose timestamp is older than GVT.
152152p2p2
p1p1
p3p3 163163
142142
143143
162162142142141141
141141120120
120120
122122
121121
Arrived lateArrived late
135135
RollbackRollback
Anti-messageAnti-message
LVTLVT
LVTLVT
LVTLVT
CSS434 Time & Global States 23
SPEEDS[Steinman 1992]Breathing Time Buckets
p2p2
p1p1
P1’s LEHP1’s LEH
P2’s LEHP2’s LEHNext GEHNext GEH
(GVT)(GVT)
This is an optimistic distributed simulator, but so aggressive as Time Warp.• Each process broadcasts the oldest local even among those it will execute. This is called a
Local Event Horizon (LEH).• A process must suspend its even processing if it has received an older LEH than the one it
is currently processing.• The oldest LEH among all processes become the next Global Event Horizon (GEH).• Each process may send out all messages and process all events before this new GEH.• Processes which have already processed beyond GEH must roll back their computation to
GEH. No anti-messages are sent out.
CSS434 Time & Global States 24
Vector timestamps and variable values for the execution
m1 m2
p1
p2Physical
time
Cut C1
(1,0) (2,0) (4,3)
(2,1) (2,2) (2,3)
(3,0)x1= 1 x1= 100 x1= 105
x2= 100 x2= 95 x2= 90
x1= 90
Cut C2
Constraints |x1 – x2| <= 50. Before this is violated, a process must send its value toits partner.
CSS434 Time & Global States 25
The lattice of global states for the execution
Sij= global state after i events at process 1 and j events at process 2
S00
S10
S20
S21S30
S31
S32
S22
S23
S33
S43
Level 0
1
2
3
4
5
6
7
CSS434 Time & Global States 26
Evaluating possibly and definitely
F = (φ(S) = );False = (T φ(S) = ) True
?
–
0Level
1
2
3
4
5
F
F
F
F T
F
m1 m2
p1
p2Physical
time
Cut C1
(1,0) (2,0) (4,3)
(2,1) (2,2) (2,3)
(3,0)x1= 1 x1= 100 x1= 105
x2= 100 x2= 95 x2= 90
x1= 90
Cut C2
All included: (2,2) = True
Possibility
Must be True, and thud we don’t care
F
Definitely = T for all linialization(4,3) = T and this is definitely
CSS434 Time & Global States 27
Paper Review by Students Distributed Snapshot
Samadi’s Algorithm Mattern’s Algorithm Discussions: What are pros & cons of these
algorithms? Optimistic Synchronization
SPEEDS Time Warp Discussions: What are pros & cons of these
algorithms in terms of performance, process creation/termination, dynamic memory allocation, and I/O handling?
CSS434 Time & Global States 28
Exercises (No turn-in)1. Textbook p627, Q14.7: An NTP server B receives server A’s message at
16:34:23.480 bearing a timestamp 16:34:13.430 and replies to it. A receives the message at 16:34:15.725, bearing B’s timestamp 16:34:25.7. Estimate the offset between B and A and the accuracy of the estimate.
2. Textbook p628, Q14.14: Two processes P and Q are connected in a ring using two channels, and they constantly rotate a message m. At any one time, there is only one copy of m in the system. Each process’s state consists of the number of times its has received m, and P sends m first. At a certain point, P has the message and its state is 101. Immediately after sending m, P initiates the snapshot algorithm. Explain the operation of the algorithm in this case, given the possible global state(s) reported by it.
3. Textbook p429, Q14.15: The figure below shows events occurring for each of two processes, p1 and p2. Arrows between processes denote message transmission. Draw and label the lattice of consistent states (p1 state, p2 state), beginning with the initial state (0,0).
time
p1
p2