sagas: distributed transactions without locks · sagas: distributed transactions without locks in...

45
Sagas: distributed transactions without locks in Erlang MARK ALLEN - ALERTLOGIC [email protected] @BYTEMEORG

Upload: others

Post on 17-Aug-2020

17 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Sagas:distributedtransactionswithoutlocksinErlangMARKALLEN- ALERTLOGIC

[email protected]

@BYTEMEORG

Page 2: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Sagas

Page 3: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG
Page 4: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG
Page 5: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

TypesofSagas§BackwardRecovery§ForwardRecovery§“RecoveryBlocks”§ParallelSagas

Page 6: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

SagasinErlang

Page 7: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

TheBig IdeaFoldoveralistofclosures…

Page 8: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

TheBig IdeaFoldoveralistofclosures…...unlessthere’sanerror.

Page 9: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

TheBig IdeaFoldoveralistofclosures…...unlessthere’sanerror;Then,foldoveralistofclosures.

Page 10: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3 Forward4 Forward5

Rollback1 Rollback2 Rollback3 Rollback4 Rollback5

Page 11: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3 Forward4 Forward5

Rollback1 Rollback2 Rollback3 Rollback4 Rollback5

Page 12: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3 Forward4 Forward5

Rollback1 Rollback2 Rollback3 Rollback4 Rollback5

Page 13: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3 Forward4 Forward5

Rollback1 Rollback2 Rollback3 Rollback4 Rollback5

Page 14: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3 Forward4 Forward5

Rollback1 Rollback2 Rollback3 Rollback4 Rollback5

Page 15: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3 Forward4 Forward5

Rollback1 Rollback2 Rollback3 Rollback4 Rollback5

Page 16: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3 Forward4 Forward5

Rollback1 Rollback2 Rollback3 Rollback4 Rollback5

Page 17: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3 Forward4 Forward5

Rollback1 Rollback2 Rollback3 Rollback4 Rollback5

BOOM

Page 18: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3

Rollback1 Rollback2 Rollback3

BOOM

Page 19: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3

Rollback1 Rollback2 Rollback3

BOOM

Page 20: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3

Rollback1 Rollback2 Rollback3

BOOM

Page 21: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Flows

Forward1 Forward2 Forward3

Rollback1 Rollback2 Rollback3

BOOM

Page 22: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG
Page 23: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Terminology

§Operation§Step§Transaction

Page 24: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Terminology

Forward1 Forward2 Forward3 Forward4 Forward5

Rollback1 Rollback2 Rollback3 Rollback4 Rollback5

Page 25: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG
Page 26: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

ImplicitAssumptionsandRequirements

Page 27: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

CompensatingClosures

CannotAbort

Page 28: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

IdempotentRequests

Page 29: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Whatif??

Forward1 Forward2 Forward3

Rollback1 Rollback2 Rollback3

BOOM

Page 30: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Whatif??

Forward1 Forward2 Forward3 Forward4

Rollback1 Rollback2 Rollback3

Page 31: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

FuturesinErlang

Page 32: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

WhydoesErlangneedfutures?!

Page 33: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Itdoesn’t!Except…

Page 34: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

AsynchronousSagaOperations

Page 35: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG
Page 36: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Implementation

Page 37: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Semantics?

Page 38: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

PropertyBasedTests

Page 39: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Whatisit?§Writeoneormoreinvariants§Testvaluesareautomaticallygeneratedandtheinvarianttested§Failuresareshrunktothesmallestpossiblefailurecase§Modifycode(ortestcase)andrestarttest§Testsareusuallytimeboxed§Extremelyusefultobuildconfidenceincomplexscenarios

Page 40: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

AcademichistoryQuickCheck:ALightweightToolforRandomTestingofHaskellPrograms

Page 41: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Implementationsfor§Go§C/C++§Clojure§Scala§Haskell(ofcourse)§… lotsofothers…§Erlang/Elixir(watchThomasArtstalkatElixirConf EU2015)

Page 42: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Erlang implementations§Quviq CommercialQuickCheck (http://www.quviq.com)§Free(butnotopensource)“QuickCheck mini”– doesn’tdostatem§PropEr (https://github.com/manopapad/proper)§Triq (https://github.com/triqng/triq)

Page 43: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Generators§Allstandardbasictypes:§ integers,§ floats,§atoms,§binaries,§strings,§ lists

§Listsofbasictypes§Userdefinedgeneratorsusing?LETand?SUCHTHAT

Page 44: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Propositions§?FORALL(Variable,generator_function(),test_function(Variable))

Page 45: Sagas: distributed transactions without locks · Sagas: distributed transactions without locks in Erlang MARK ALLEN -ALERTLOGIC MARK.ALLEN@ALERTLOGIC.COM @BYTEMEORG

Resources§https://github.com/mrallen1/gisla

§https://github.com/mrallen1/criswell

§Sagaspaper:http://www.cs.cornell.edu/andru/cs711/2002fa/reading/sagas.pdf

§Caitie McCaffrey,PapersWeLoveonSagas:https://youtu.be/7dc4Tl5ZHRg?t=27m16s

§Caitie McCaffrey,DistributedSagas:https://speakerdeck.com/caitiem20/distributed-sagas-a-protocol-for-coordinating-microservices

§Theseslides: https://speakerdeck.com/mrallen1/sagas-distributed-transactions-without-locks