intro to perf test
TRANSCRIPT
-
8/8/2019 Intro to Perf Test
1/31
Agenda
Why Performance Test?
What is Performance testing and How do we doPerformance testing?
Intro to Performance Engineering Methodology
Agile S crum methodology - Testing
-
8/8/2019 Intro to Perf Test
2/31
Why P erformance Test?
Speed - Does the applicationrespond quickly enough for theintended users?
Scalabilit y Will the applicationhandle the expected user loadand beyond? (Capacity)
Stabilit y Is the applicationstable under expected and
unexpected user loads?(Robustness)
C onfidence Are you sure thatusers will have a positive
experience on go-live day?
-
8/8/2019 Intro to Perf Test
3/31
-
8/8/2019 Intro to Perf Test
4/31
Scalabilit y
How many users
before it gets slow ? before it stops working?
will it sustain?
do I expect today? do I expect before the nextupgrade?
How much data can it hold?
Database capacity
File S erver capacity
Back-up S erver capacity
Data growth rates
-
8/8/2019 Intro to Perf Test
5/31
Stabilit y
What happens if
there are more users than we expect?
all the users do the same thing? a user gets disconnected?
there is a Denial of S ervice Attack?
the web server goes down?
-
8/8/2019 Intro to Perf Test
6/31
-
8/8/2019 Intro to Perf Test
7/31
Wh at is P erformance Related Testing?
Performance Validation
Performance Testing
Performance Engineering
Detect
Resolve
-
8/8/2019 Intro to Perf Test
8/31
P erformance Validation Performance validation is the process by whichsoftware is tested with the intent of determining if thesoftware meets pre-existing performancerequirements. This process aims to evaluate
compliance.
Primarily used for
determining SL A compliance.
Independent Validation and Verification validating subsequent builds/releases.
-
8/8/2019 Intro to Perf Test
9/31
P erformance Testing Performance testing is the process by which software is
tested to determine the current system performance. This
process aims to gather information about current
performance, but places no value judgments on thefindings.
Primarily used for
determining capacity of existing systems. creating benchmarks for future systems.
evaluating degradation with various loads and/or configurations.
-
8/8/2019 Intro to Perf Test
10/31
P erformance Engineering Performance engineering is the process by whichsoftware is tested and tuned with the intent of realizing the required
performance. This process aims to optimize the most
important application performance trait, user experience.
Primarily used for
new systems with pre-determined requirements. extending the capacity of old systems.
fixing systems that are not meetingrequirements/ SL As.
-
8/8/2019 Intro to Perf Test
11/31
-
8/8/2019 Intro to Perf Test
12/31
C ompare and C ontrast
Engineering:
Is iterative.
Has clear goals, but fuzzy end points.
Includes the effort of tuning the application.
Focuses on multiple scenarios with pre-determined
standards.
Heavily involves the development team.
Occurs concurrently with software development.
-
8/8/2019 Intro to Perf Test
13/31
Intro to PE M et h odolog y
Evaluate S ystem
Develop Test Assets
Baselines and Benchmarks
Analyze Results
Tune
Identify Exploratory Tests
Execute S cheduled Tests
Complete Engagement
-
8/8/2019 Intro to Perf Test
14/31
-
8/8/2019 Intro to Perf Test
15/31
Ev aluate S ystem
Determine performance requirements.
Identify expected and unexpected user activity.
Determine test and/or production architecture.
Identify non-user-initiated (batch) processes.
Identify potential user environments.
Define expected behavior during unexpectedcircumstances.
-
8/8/2019 Intro to Perf Test
16/31
-
8/8/2019 Intro to Perf Test
17/31
D e velop Test Assets
Create S trategy Document.
Develop Risk Mitigation Plan.
Develop Test Data.
Automated test scripts:
Plan
Create
Validate
-
8/8/2019 Intro to Perf Test
18/31
B aseline and B enc h marks
Most important for iterative testing.
Baseline (single user) for initial basis of comparison andbest
case .
Benchmark (15-25% of expected user load) determinesactual
state at loads expected to meet requirements.
-
8/8/2019 Intro to Perf Test
19/31
-
8/8/2019 Intro to Perf Test
20/31
Anal yz e Results
Most important.
Most difficult.
Focuses on:
Have the performance criteria been met?
What are the bottlenecks?
Who is responsible to fix those bottlenecks?
Decisions.
-
8/8/2019 Intro to Perf Test
21/31
-
8/8/2019 Intro to Perf Test
22/31
Tune
Engineering only.
Highly collaborative with development team.
Highly iterative.
U sually, performance engineer supports andvalidates while
developers/admins tune .
-
8/8/2019 Intro to Perf Test
23/31
-
8/8/2019 Intro to Perf Test
24/31
-
8/8/2019 Intro to Perf Test
25/31
-
8/8/2019 Intro to Perf Test
26/31
Ex ecute Sc h eduled Tests
Only after Baseline and/or Benchmark tests.
These tests evaluate compliance with documented
requirements.Often are conducted on multiplehardware/configuration
variations.
-
8/8/2019 Intro to Perf Test
27/31
-
8/8/2019 Intro to Perf Test
28/31
C omplete Engagement
Document:
Actual Results
Tuning S ummary
Known bottlenecks not tuned
Other supporting information
Recommendation
Package Test Assets:
S cripts
Documents
Test data
-
8/8/2019 Intro to Perf Test
29/31
-
8/8/2019 Intro to Perf Test
30/31
Summar y
We test performance to: Evaluate Risk.
Determine system capabilities.
Determine compliance.
Performance Engineering Methodology:
Ensures goals are accomplished.
Defines tasks.
Identifies critical decision points.
S hortens testing lifecycle.
-
8/8/2019 Intro to Perf Test
31/31
Th ank You!!