benchmarking nginx for accuracy and results
TRANSCRIPT
![Page 1: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/1.jpg)
Benchmarking with NGINX
Introduced by Owen Garrett
Presented by Rick Nelson
Nginx, Inc.
![Page 2: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/2.jpg)
About this webinar
When you want to know how many resources to allocate for your NGINX
servers or what the capacity of your current NGINX servers is, you need
to be able to perform proper benchmark testing, but this can be
complicated. In this webinar, you'll learn the considerations that need to
go into planning, configuring and running benchmarks.
![Page 3: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/3.jpg)
Agenda
• Introduction
• Common Pitfalls
• Tips and Techniques
• Demonstration
• Questions
![Page 4: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/4.jpg)
INTRODUCTION
![Page 5: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/5.jpg)
What is NGINX?
Internet
N
Web ServerServe content from disk
Application ServerFastCGI, uWSGI, Passenger…
ProxyCaching, Load Balancing… HTTP traffic
Application Acceleration
SSL and SPDY termination
Performance Monitoring
High Availability
Advanced Features: Bandwidth Management
Content-based Routing
Request Manipulation
Response Rewriting
Authentication
Video Delivery
Mail Proxy
GeoLocation
![Page 6: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/6.jpg)
Why do Benchmarking?
• Stress test
• Capacity planning
• Comparison testing (bake off)
![Page 7: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/7.jpg)
Benchmarking Considerations
• It’s complicated
• What is the goal?
• What kind of test environment do you have?
• What testing tools do you have?
• How well can you simulate production traffic?
![Page 8: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/8.jpg)
What areas are you testing?
• Web server
• Application Server
• Reverse Proxy
• All of the above
![Page 9: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/9.jpg)
What are you testing?
• Can you simulate production traffic?• If not, what are you concerned about:
– Connections– Request rate– Bandwidth– SSL– All of the Above
![Page 10: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/10.jpg)
What are you testing?
• Can you do a full production test?
• If not, do a smaller scale test and extrapolate
• Know your traffic
– Get vs. Post, request/response sizes, etc.
• What you need to test vs. what you are actually testing
![Page 11: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/11.jpg)
What are you testing?
• You may want to test a single variable
– Connections
– Request rate
– Bandwidth
– New SSL handshakes
![Page 12: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/12.jpg)
Test Environment
• You are always testing the whole environment
– Testing Tools
• Load generators
• Web Servers
– Systems under Test (SUT)
• Reverse proxies
• Web servers
![Page 13: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/13.jpg)
Test Environment
• Good rule of thumb for cores needed:
– Load Generators: 2N
– Reverse Proxies: N
– Web Servers: 2N
![Page 14: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/14.jpg)
COMMON PITFALLS
![Page 15: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/15.jpg)
Not Knowing What You Are Testing
• Know your testing tools
• What question does a test answer?
– For example:
• How many requests per second can a SUT handle?
• Can a SUT handle a certain number of requests per second
![Page 16: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/16.jpg)
Real Clients versus Synthetic Clients
Real Clients Synthetic Clients
Latency Low-High Low
Packet Loss Low-High Low
Bandwidth Low-High High
Time between requests Long Short
Idle connections Yes No
![Page 17: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/17.jpg)
Unrealistic Synthetic Clients
• Misleading results– System looks good during benchmark– System has problems with real clients
• Why is this? – Synthetic clients are ideal for the server
• Low latency, low packet loss, busy connections
– Real clients are not• High latency, packet loss, idle connections
![Page 18: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/18.jpg)
Synthetic Clients
• Unrealistic synthetic clients– Apache Bench (ab)
• More realistic synthetic clients– Open Source
• Siege, wrk
– Commercial• Spirent, Ixia, Cloudtest, other cloud services
![Page 19: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/19.jpg)
Misconfiguration
• Many configuration settings can impact tests– Some Linux kernel settings may be too low for heavy loads
– Keepalives
– SSL key sizes make a big difference
– Compression
– Benchmark clients or servers
![Page 20: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/20.jpg)
Tips and Techniques
• Use multiple approaches– Real world simulations for real world results
– Simple tests for baselines and debugging
![Page 21: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/21.jpg)
Tips and Techniques
• If you have found the real limit of a SUT then:
– At least one system resource should be exhausted
• CPU
• Memory
• Bandwidth
• Disk I/O
– If not, then a bottleneck exists elsewhere
![Page 22: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/22.jpg)
Tips and Techniques
• Start with the NGINX defaults
• NGINX directives can impact performance
– accept_mutex, worker_processes, worker_connections, keepalive_timeout, lingering_close, sendfile, keepalive, aio, open_file_cache
![Page 23: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/23.jpg)
Tips and Techniques
• Some 10G drivers don’t use cores effectively
– Some cores at 100%, others have low usage
– Solution: driver dependent
• Scripting
• New driver version
• New Card
![Page 24: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/24.jpg)
Tips and Techniques
• Monitor error files and errors from testing tools
– Errors can return faster
– You may be hitting system limits or errors
• Don’t have load generation on a SUT
• Double check the result figures
![Page 25: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/25.jpg)
Tips and Techniques
• Run all tests multiple times
• If using virtualization, be aware of other loads on the host
• If you don’t understand the results - simplify
![Page 26: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/26.jpg)
Demonstration
Load Generator
Load Generator
Apache NGINX
ab siege
Datacenter
![Page 27: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/27.jpg)
Questions and Answers
![Page 28: Benchmarking NGINX for Accuracy and Results](https://reader034.vdocuments.net/reader034/viewer/2022042817/55a1a3be1a28abd03f8b46eb/html5/thumbnails/28.jpg)
Closing thoughts
• 38% of the busiest websites use NGINX
• Check out the previous webinar on tuning at nginx.com
• Future webinars: nginx.com/webinars
Try NGINX F/OSS (nginx.org) or NGINX Plus (nginx.com)