ideas and challenges on testing a experiences testing quagga
TRANSCRIPT
![Page 1: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/1.jpg)
1
Ideas and Challenges on testing a routing protocol Experiences testing Quagga
Martin Winter May 26, 2016
![Page 2: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/2.jpg)
Who is OpenSourceRouting ?
‣ Who is Open Source Routing ? • www.opensourcerouting.org • Project by NetDEF (Network Device Education Foundation) - www.netdef.org
- Non-Profit Company based in California
• Working on Quagga Routing
‣ Who is Martin Winter ? • Co-Founder of NetDEF • Focusing on Testing Quagga • Previously worked for Equipment Vendor & large ISP
2
![Page 3: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/3.jpg)
3
Building Tools Tools used for “building”
![Page 4: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/4.jpg)
Compiler VMs
4
‣ Build in VMs: Simple Reset ‣ Build on specific distro and version • Different versions of libraries used
‣ Different CPU Architectures
![Page 5: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/5.jpg)
Static Analysis: Clang-Analyzer
5
![Page 6: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/6.jpg)
Static Analysis: Clang-Analyzer
6
BUT: Expect a lot of bad warnings…
![Page 7: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/7.jpg)
Static Analysis: Coverity Scan
7
![Page 8: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/8.jpg)
Static Analysis: Coverity Scan
8
Nice: Finds some Copy & Paste errors!
![Page 9: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/9.jpg)
Code Coverage
9
? [this slide is left blank – we don’t do this yet]
![Page 10: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/10.jpg)
10
Testing Tools Tools used for Testing
![Page 11: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/11.jpg)
RFC Compliance Tests
11
Interoperability with Cisco/Juniper ≠ RFC Compliance Test
Hint: • BGP Transitive Attributes • Error Handling
![Page 12: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/12.jpg)
RFC Compliance Test: Ixia ANVL
12
![Page 13: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/13.jpg)
Protocol Fuzzer
13
012301234567890123456789012345678901+-+-+-+-+-+-+-+-+|Version|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|MyAutonomousSystem|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|HoldTime|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|BGPIdentifier|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|OptParmLen|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|||OptionalParameters(variable)|||+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
BGP OPEN Message Format (RFC 4271)
![Page 14: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/14.jpg)
Protocol Fuzzer: Spirent SPS-8000
14
![Page 15: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/15.jpg)
Protocol Performance & Scale
15
Example – Scale using external routes in OSPF
Test traffic sent to an IP on each external route
![Page 16: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/16.jpg)
16
Protocol Performance & Scale
![Page 17: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/17.jpg)
17
Protocol Performance & Scale
Example – Scale using internal routes in OSPF
Test traffic sent to an IP on each link in matrix
![Page 18: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/18.jpg)
18
Protocol Performance & Scale (Result from old Quagga 0.99.22)
![Page 19: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/19.jpg)
19
Protocol Performance & Scale (Result from old Quagga 0.99.22)
![Page 20: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/20.jpg)
20
Protocol Performance & Scale
‣ How to summarize result? (Pass/Fail)
‣ Testing in VM or physical machine?
‣ Tools for testing?
• ExaBGP for BGP
• no tools for OSPF/ISIS/RIP/PIM/….
![Page 21: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/21.jpg)
21
Automate (incl Result Parsing!)
![Page 22: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/22.jpg)
22
Links
• NetDEF CI System https://ci.netdef.org
• Quagga RFC Compliance Reports https://www.opensourcerouting.org/compliance-test-results/
• More details on our testing: https://www.opensourcerouting.org/2016/05/whitepaper-how-opensourcerouting-tests-quagga
![Page 23: Ideas and Challenges on testing a Experiences testing Quagga](https://reader035.vdocuments.net/reader035/viewer/2022081513/62a9916951e91c4be0141993/html5/thumbnails/23.jpg)
23
Questions ?
Martin Winter [email protected] Network Device Education Foundation (NetDEF) www.netdef.org / www.opensourcerouting.org