reproductible benchmarks - scaling bitcoin · reproductible benchmarks let’s talk tooling not...
TRANSCRIPT
![Page 1: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/1.jpg)
Reproductible benchmarksLet’s talk tooling
Not fancy crypto, but at least it is easy to understand!
Nicolas DORIERCode monkey at DG Lab and Metaco
Main maintainer of NBitcoin and BTCPay Serverhttps://github.com/dgarage/LightningBenchmarks
![Page 2: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/2.jpg)
• Tell you tell me about Lightning performance?
I was asked
![Page 3: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/3.jpg)
• How fast can Alice pay Bob?• How fast can Alice pay Bob through
Carol?• How fast Alices can pay Bob?
I reformulated
![Page 4: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/4.jpg)
• 34 payments / sec• 12 payments / sec• 20 payments / sec (with 4
Bobs)
I responded c-lightning in may 2018
![Page 5: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/5.jpg)
![Page 6: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/6.jpg)
Does it matter?
![Page 7: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/7.jpg)
Does it matter?
No (interesting for developers)
![Page 8: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/8.jpg)
• Lightning network implementers• Services relying on micro
payments
![Page 9: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/9.jpg)
Framework Benchmarking and sharing results predictably easily
Reframing the objective
![Page 10: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/10.jpg)
![Page 11: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/11.jpg)
![Page 12: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/12.jpg)
In the trenches
![Page 13: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/13.jpg)
Run.sh
![Page 14: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/14.jpg)
Setup: Alice pays Bob
![Page 15: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/15.jpg)
Execution: Alice pays Bob
![Page 16: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/16.jpg)
Specify the concurrencyvalues to test
![Page 17: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/17.jpg)
Template used for actors
![Page 18: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/18.jpg)
Generated artifacts
![Page 19: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/19.jpg)
A bug
![Page 20: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/20.jpg)
No bug
![Page 21: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/21.jpg)
![Page 22: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/22.jpg)
![Page 23: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/23.jpg)
Setup: Alice pays Bob via Carol
![Page 24: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/24.jpg)
Execution: Alice pays Bobvia Carol
![Page 25: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/25.jpg)
Vary the number of Carols
![Page 26: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/26.jpg)
![Page 27: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/27.jpg)
![Page 28: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/28.jpg)
Setup: Alices pay Bob
![Page 29: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/29.jpg)
Execution: Alices pay Bob
![Page 30: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/30.jpg)
Vary the number of Alices
![Page 31: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/31.jpg)
![Page 32: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/32.jpg)
Instrumentation
![Page 33: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/33.jpg)
Revisiting the bug
![Page 34: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/34.jpg)
![Page 35: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/35.jpg)
![Page 36: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/36.jpg)
![Page 37: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/37.jpg)
![Page 38: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/38.jpg)
Alice Bob
![Page 39: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/39.jpg)
![Page 40: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/40.jpg)
Careful!
• Containers are running in priviledged mode
![Page 41: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/41.jpg)
TODO
• Depending on a common abstraction to clightning, LND and charge.
• Making Lightning implementations parametrizable
![Page 42: Reproductible benchmarks - Scaling Bitcoin · Reproductible benchmarks Let’s talk tooling Not fancy crypto, but at least it is easy to understand! Nicolas DORIER Code monkey at](https://reader030.vdocuments.net/reader030/viewer/2022041022/5ed2d444924c8a40787fb4a2/html5/thumbnails/42.jpg)
Thank youhttps://github.com/dgarage/LightningBenchmarks