performance testing back to basics may 2013. who am i?
TRANSCRIPT
Performance Testing
Back to Basics
May 2013
Who Am I?
Who Am I?
Who Are You?
Who Are You?
Performance testers?
Who Are You?
Performance testers Testers/QA?
Who Are You?
Performance testers Testers/QA Developers?
Who Are You?
Performance testers Testers/QA Developers CFR Cluj Supporters?
Today
4 sessions about an hour each Some practical work in groups Any questions/comments speak up Please share your thoughts/experiences
Conversation NOT lecture
Performance Testing
Back to Basics
What is Performance Testing?
?
What is Performance Testing?
performance testing is in general testing performed to determine how a system performs in terms of responsiveness and stability under a particular workload?
Wikipedia 2013
What is Performance Testing?
Load Testing Stress Testing Soak Testing Spike Testing Configuration Testing Isolation Testing
Some History
Traditional Performance Testing
understand the volumes Model the business processes Run experiments (tests) Measure performance indicators (response time,
cpu etc.) Tune if needed Sign off report
Some History
Some History
Some History
From This
To This
Applications have changed
More complex functionality Minimal defined business processes Diverse Public Internet Use of services (homegrown and third party) Cacheing Increased Client functionality (javascript etc.)
Applications have changed
Cloud computing Multiple client platforms (tablet, mobile etc.) Push technology
Traditional Performance Testing
Focus on IT/IS requirements Too difficult to model the entire application Length of time it takes to run a performance test Models have too many variables to be reliable
Problems
New Approaches to Performance
As applications have increased in complexity and application usage has become more diverse we have developed new tools and approaches to help us predict and manage the performance of applications
New Approaches to Performance
Synthetic testing (traditional perf testing)
A/B testing
Instrumentation/Analytics (RUM)
React Fast
Over_Provision
Third Party tools
Performance Testing
Back To Basics
Rank Sites by Performance
News related websites www.nytimes.com news.bbc.co.uk www.huffingtonpost.co.uk tv.msnbc.com www.cnn.com www.dailymail.co.uk
Rank Sites by Performance
News related websites by response time news.bbc.co.uk (0.7) www.dailymail.co.uk (3.3) www.cnn.com (3.5) www.nytimes.com (3.9) tv.msnbc.com (5.5) www.huffingtonpost.co.uk (6.3)
Gomez April 2013
Rank Sites by Performance
News related websites by popularity www.cnn.com tv.msnbc.com www.nytimes.com www.huffingtonpost.co.uk www.dailymail.co.uk news.bbc.co.uk
The Five W's
The Five W's
WHO?
The Five W's
WHO? WHAT?
The Five W's
WHO? WHAT? WHY?
The Five W's
WHO? WHAT? WHY? WHEN?
The Five W's
WHO? WHAT? WHY? WHEN? WHERE?
Who is interested in performance?
IS/IT Department
Who is interested in performance?
IS/IT Department Protection of production Stability Capacity Resource
Who is interested in performance?
Customer
Who is interested in performance?
Customer Stability (of Browser) Freezing Consistency Ease of use (fewest steps to get job done)
Who is interested in performance?
Business
Who is interested in performance?
Business Revenue Unique monthly visitors Conversion rates Site Abandonment rate
Who is interested in performance?
As Application complexity has increased so too has the amount of performance related information available
As a performance tester it is important to be able to interpret the data and be able to present information that is useful and relevant to people.
Test models
Choose 2 key business processes for each site www.amazon.com www.lastminute.com www.digitalspy.co.uk
Test models
Choose 2 key business processes for each site www.amazon.com
Test models
Choose 2 key business processes for each site www.lastminute.com
Test models
Choose 2 key business processes for each site www.digitalspy.co.uk
What to Test?
Information used to prioritize testing
What to Test?
Information used to prioritize testing New technology Heavy usage Spikes in usage Key revenue generators Customer goals (or hot buttons)
What to Test?
As it has become harder to model and test an entire application, being able to identify which areas of an application are key for performance is now more important than ever
Test models
Choose 2 key business processes for each site www.facebook.com www.youtube.com
Why Do we Test?
?
Why Do we Test?
Prevent problems Validate improvements Reproduce problems Aid in decision making
Why Do we Test?
What Does a Successful Test look like?
Setting goals Convert business goals into technical model Volumes (Average, Peak)
Understanding business process
Using Webpagetest
Free tool to allow you to see underlying calls made when you execute a business process
Identify Key calls for performance
Understanding business process
Using Webpagetest
http://www.webperformancetoday.com/2010/07/09/waterfalls-101/
Using Webpagetest
News related websites www.nytimes.com news.bbc.co.uk www.huffingtonpost.co.uk tv.msnbc.com www.cnn.com www.dailymail.co.uk
Understanding business process
Identify Key calls for performance
Based on who is Interested Based on what the key business processes are Based on why you are testing
Why do we Test?
Integrate business goals with technical and customer goals
Be wary of using design/developers as way of understanding what the application really does
When to Test?
Traditionally performance testing was done just prior to release. Reasons for this:
Models take time to build Need for stable baseline to compare against Site must be functional before perf testing
When to Test?
Rise of Agile requires new approaches Split up into smaller component models to
allow testing during development Clustering production allows testing in
production with incremental rollout Test in design
Web Sequence diagrams
Example for Facebook Login
http://www.uml-diagrams.org/sequence-diagrams-examples.html
Where to Test?
Dedicated Performance environment Prod Disaster recovery QA/Dev Environment Preprod Isolated components
Where to Test?
Dont forget the surrounding infrastructure Network Load Balancers Caches CDN Third Party services
How?
Lots of Vendors promising magic. Get demos and proof of concept for your site
Only once you have a good understanding of the basics for your site.
How?
Dont forget the basic free tools available WebPageTest Browser developer tools Curl SoapUI/LoadUI Many others
Questions