in theory distributed systems in practice, · papers we love nyc. papers we love sf *2 5 aysylu...
TRANSCRIPT
![Page 1: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/1.jpg)
Aysylu GreenbergJune 14, 2016
Distributed Systems in Practice,in Theory
![Page 2: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/2.jpg)
How I got into reading papers as a
practitioner in industry
![Page 3: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/3.jpg)
Computer Science ResearchInDistributed Systems Industry
![Page 4: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/4.jpg)
Operating systems research
![Page 5: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/5.jpg)
Operating systems research
![Page 6: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/6.jpg)
Operating systems research
Concurrency
![Page 7: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/7.jpg)
Operating systems research
Concurrency
Concurrency primitives: mutex & semaphore
![Page 8: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/8.jpg)
Operating systems research
Concurrency
Concurrency primitives: mutex & semaphore
Processes execute at different speeds
![Page 9: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/9.jpg)
Time in distributed systems
https://www.flickr.com/photos/national_archives_of_norway/6263353228
![Page 10: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/10.jpg)
Time in distributed systems
![Page 11: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/11.jpg)
Time in distributed systems
Pipelining
![Page 12: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/12.jpg)
1980
![Page 13: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/13.jpg)
1980
![Page 14: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/14.jpg)
Internet
1980
![Page 15: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/15.jpg)
Internet
Distributed consensus
1980
![Page 16: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/16.jpg)
Internet
Distributed consensus
1980
![Page 17: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/17.jpg)
Internet
Distributed consensus
1980
![Page 18: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/18.jpg)
Paxos
Internet
Distributed consensus
1980
![Page 19: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/19.jpg)
Reconsider large systems
![Page 20: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/20.jpg)
Reconsider large systems
Shared infrastructure
...
![Page 21: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/21.jpg)
CS Research is Timeless
Inform decisions
Mitigate technical risk
![Page 22: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/22.jpg)
* 22
Aysylu Greenberg
@aysylu22
![Page 23: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/23.jpg)
Papers We Love NYC
![Page 24: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/24.jpg)
Papers We Love SF
![Page 25: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/25.jpg)
* 25
Aysylu Greenberg
@aysylu22
![Page 26: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/26.jpg)
Today
● Staged Event-Driven Architecture
![Page 27: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/27.jpg)
Today
● Staged Event-Driven Architecture● Leases
![Page 28: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/28.jpg)
Today
● Staged Event-Driven Architecture● Leases● Inaccurate Computations
![Page 29: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/29.jpg)
Staged Event Driven
Architecture&
Deep Pipelines
2001
![Page 30: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/30.jpg)
Hardware to Data Pipelines
![Page 31: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/31.jpg)
Hardware to Data Pipelines
https://en.wikipedia.org/wiki/Graphics_pipeline
![Page 32: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/32.jpg)
![Page 33: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/33.jpg)
Staged Event Driven Architecture
![Page 34: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/34.jpg)
Staged Event Driven Architecture
+ -
![Page 35: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/35.jpg)
Single-machine pipeline
generalizes to distributed pipelines
Staged Event Driven Architecture
![Page 36: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/36.jpg)
Search Indexing Pipelines
![Page 37: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/37.jpg)
Search Indexing Pipelines
![Page 38: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/38.jpg)
Search Indexing Pipelines
![Page 39: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/39.jpg)
Search Indexing Pipelines
![Page 40: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/40.jpg)
Search Indexing Pipelines
![Page 41: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/41.jpg)
Search Indexing Pipelines
![Page 42: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/42.jpg)
Search Indexing Pipelines
![Page 43: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/43.jpg)
Search Indexing Pipelines
![Page 44: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/44.jpg)
Search Indexing Pipelines
+ -
![Page 45: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/45.jpg)
Leasesas Heart Beat in
Distributed Systems
1989
![Page 46: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/46.jpg)
![Page 47: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/47.jpg)
Leases
● Distributed locking
![Page 48: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/48.jpg)
Leases
● Distributed locking● Lease term tradeoffs
○ short
![Page 49: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/49.jpg)
Leases
● Distributed locking● Lease term tradeoffs
○ short vs long
![Page 50: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/50.jpg)
Leases
● Distributed locking● Lease term tradeoffs
○ short vs long● Use of leases in modern applications
○ Leader election TTL (in etcd)
![Page 51: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/51.jpg)
Leases
● Distributed locking● Lease term tradeoffs
○ short vs long● Use of leases in modern applications
○ Leader election TTL (in etcd)○ Liveness detection
![Page 52: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/52.jpg)
![Page 53: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/53.jpg)
Leases in Build System:Success Scenario
![Page 54: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/54.jpg)
Build my project
Build System
![Page 55: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/55.jpg)
Build my project
Build System
OK
![Page 56: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/56.jpg)
Build my project
Build System
OK
Waiting for the results
![Page 57: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/57.jpg)
Build my project
Build System
OK
Waiting for the results
Build is in progress
![Page 58: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/58.jpg)
Build my project
Build System
OK
Waiting for the results
Build is in progress
Waiting for the results
![Page 59: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/59.jpg)
Build my project
Build System
OK
Waiting for the results
Build is in progress
Waiting for the results
Build is finished
![Page 60: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/60.jpg)
Leases in Build System:Failure Scenario
![Page 61: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/61.jpg)
Leases in Build System
![Page 62: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/62.jpg)
Leases in Build System
![Page 63: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/63.jpg)
Leases in Build System
![Page 64: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/64.jpg)
Leases in Build System
![Page 65: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/65.jpg)
Leases in Build System
![Page 66: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/66.jpg)
Leases in Build System
![Page 67: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/67.jpg)
Using etcd leases for heartbeat$ curl http://server.com/v2/keys/foo -XPUT -d\
value=bar -d ttl=300
![Page 68: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/68.jpg)
{ "action": "set", "node": { "createdIndex": 2, "expiration":"2016-06-14T16:15:00", "key": "/foo", "modifiedIndex": 2, "ttl": 300, "value": "bar" }}
![Page 69: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/69.jpg)
Using etcd leases for heartbeat$ curl http://server.com/v2/keys/foo -XPUT -d \
value=bar -d ttl=300
… 3 minutes later...
![Page 70: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/70.jpg)
Using etcd leases for heartbeat$ curl http://server.com/v2/keys/foo -XPUT -d \
value=bar -d ttl=300
$ curl \
http://server.com/v2/keys/foo?prevValue=bar \
-XPUT -d ttl=300 -d refresh=true -d \
prevExist=true
![Page 71: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/71.jpg)
{ "action": "update", "node": { "createdIndex": 2, "expiration":"2016-06-14T16:18:00", "key": "/foo", "modifiedIndex": 3, "ttl": 300, "value": "bar" } "prevNode": {...}}
![Page 72: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/72.jpg)
{ "action": "update", "node": { "createdIndex": 2, "expiration":"2016-06-14T16:18:00", "key": "/foo", "modifiedIndex": 3, "ttl": 300, "value": "bar" } "prevNode": {...}}
"prevNode": { "createdIndex": 2, "expiration":"2016-06-14T16:15:00", "key": "/foo", "modifiedIndex": 2, "ttl": 120, "value": "bar"}
![Page 73: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/73.jpg)
Leases for heartbeat:How long should the lease term be?
![Page 74: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/74.jpg)
Inaccurate Computations&Serving Search Results
![Page 75: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/75.jpg)
From Accurate to "Good Enough"
![Page 76: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/76.jpg)
[Trade off] Inaccuracy for Performance
![Page 77: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/77.jpg)
![Page 78: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/78.jpg)
![Page 79: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/79.jpg)
![Page 80: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/80.jpg)
[Trade off] Inaccuracy for Resilience
![Page 81: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/81.jpg)
![Page 82: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/82.jpg)
![Page 83: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/83.jpg)
Reduce
Map
Input
Map
Input
Map
Input
![Page 84: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/84.jpg)
Inaccuracy for Resilience
1. Task decomposition
![Page 85: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/85.jpg)
![Page 86: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/86.jpg)
Inaccuracy for Resilience
1. Task decomposition2. Baseline for correctness
![Page 87: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/87.jpg)
![Page 88: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/88.jpg)
Inaccuracy for Resilience
1. Task decomposition2. Baseline for correctness3. Criticality Testing
![Page 89: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/89.jpg)
![Page 90: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/90.jpg)
![Page 91: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/91.jpg)
![Page 92: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/92.jpg)
![Page 93: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/93.jpg)
![Page 94: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/94.jpg)
Inaccuracy for Resilience
1. Task decomposition2. Baseline for correctness3. Criticality Testing4. Distortion and timing models
![Page 95: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/95.jpg)
Distortion Model
![Page 96: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/96.jpg)
Timing Model
![Page 97: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/97.jpg)
[In production]Inaccuracy for Performance & Resilience
![Page 98: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/98.jpg)
Jeff Dean "Building Software Systems at Google and Lessons Learned", Stanford, 2010
![Page 99: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/99.jpg)
![Page 100: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/100.jpg)
![Page 101: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/101.jpg)
[Designing with]Inaccuracy for Performance & Resilience
![Page 102: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/102.jpg)
[Designing with]Inaccuracy for Performance & Resilience
simplified implementation
focus on observabilityapplicable to some problem domains
![Page 103: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/103.jpg)
[Designing with]Inaccuracy for Performance & Resilience
fuzz testing
generative testing
simplified implementation
fault injection testing
focus on observabilityapplicable to some problem domains
![Page 104: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/104.jpg)
References● T. Wurthinger, C. Wimmer et al. "One VM to Rule Them
All"● M. Rinard "Probabilistic Accuracy Bounds for Fault-
Tolerant Computations that Discard Tasks"● F. Corbato, M. Daggett, R. Daley "An Experimental Time-
Sharing System"● E. Dijkstra "Cooperating Sequential Processes"● L. Lamport "Time, Clocks, and the Ordering of Events in a
Distributed System"● http://blinkdb.org/
![Page 105: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/105.jpg)
References● B. Oki, B. Liskov "Viewstamped Replication: A New Primary Copy
Method to Support Highly-Available Distributed Systems"● L. Lamport "The Part-Time Parliament"● M. Welsh, D. Culler, E. Brewer "SEDA: An Architecture for Well-
Conditioned, Scalable Internet Services"● C. Gray, D. Cheriton "Leases: An Efficient Fault-Tolerant
Mechanism for Distributed File Cache Consistency"● S. Agarwal, B. Mozafari et al. "BlinkDB: Queries with Bounded
Errors and Bounded Response Times on Very Large Data"
![Page 106: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/106.jpg)
GratitudeInes SombraDavid GreenbergKaran ParikhMatt WelshErran Berger
![Page 107: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/107.jpg)
Robust & scalable pipelines
![Page 108: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/108.jpg)
Robust & scalable pipelinesLeases for sharing &
heartbeat
![Page 109: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/109.jpg)
Robust & scalable pipelinesLeases for sharing &
heartbeatInaccuracy for resilience &
performance
![Page 110: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/110.jpg)
Robust & scalable pipelinesLeases for sharing &
heartbeatInaccuracy for resilience &
performance
CS research is timeless:use it to mitigate risk
![Page 111: in Theory Distributed Systems in Practice, · Papers We Love NYC. Papers We Love SF *2 5 Aysylu Greenberg @aysylu22. Today Staged Event-Driven Architecture. ... M. Rinard "Probabilistic](https://reader033.vdocuments.net/reader033/viewer/2022042411/5f29ad1816bd21798b479bd1/html5/thumbnails/111.jpg)
Aysylu GreenbergJune 14, 2016
Distributed Systems in Practice,in Theory
@aysylu22