©company confidential 1 performance testing for tm & d – an overview

©Company confidential 1

Performance Testing for TM & D – An Overview


Introduction & Performance Engineering Coverage on TM&D

Consultation & Performance Engineering Phases Sample Model for S&D Reporting Data collection and Analysis Post test Identification Deliverables

Agenda


Objective

To have a high level understanding ofPerformance Engineering Engagements

for TM & D


Practical conversations on performance

“I think we need Performance Testing, but what is it exactly?” “I think around 300 users will use the system, they will do all

kinds of activity, so can we determine performance ? “Houston, this application is slow … do we need high-end

server?” “What you will do with production data in performance?” “Post Test charts looks nice, let me know the performance is

good or bad” “Does that mean we’re done? Can we release?”

And a latest conversation two days back, one client called me and asked

“ I am not winning the on-line bidding, can I send 20-30 bidding requests

at a time, and increase my bidding chances?”


Introduction

Performance Testing has three dimensions (Keeping the resource as constant):

– Number of Users

– Amount of Data

– Amount of Activity

Any performance related issues can be mapped using these dimensions

Performance is dependent on the following:

– Resources that are used by the application

• Computing Resources, Network Resources and Storage Resources

– Bottlenecks and wait times

– The number of operations involved in performing the computations and the time taken to complete these operations


High Level Scope of Performance Testing

Testing integrated portal accessCustomer data query and updationReport generation (weekly/monthly)

-S&D-ODMS (outlet specific reports)-Key Accounts-Performance and Process

Offline data access and updateReport Publishing in central system

Performance team will identify the other crucial Transactions during System Study Level

Outlet Data Management System

S&D Strategy, Planning and Evaluation

Business Performance and Process Assessment

Key Accounts/HoReCa

TM&D

Performance Coverage on TM & D Systems


Our High Level Process Consultation


Major Steps in Performance Testing

Develop Performance

Strategy/test plan

Develop Scripts,test data

Set up Production mirror image

Test designTest Execution

Confirm PerformanceRequirementsFactor/Metric

Continuous AnalysisFeedback to Dev teamReport issue/Concern

Post Test AnalysisReport Development

Issue Management System

Product Maturity Analysis

Load – Performance – Stress testing

Reliability TestingSecurity testingUsability testing

Compatibility testing


Speed - Does the application respond quickly enough for the intended users? Scalability – Will the application handle the expected user load and beyond? Stability – Is the application stable under expected and unexpected user loads? Confidence – Are you sure that users will have a positive experience on go-live day?

Controller

Load Generation

Vuser host

Analysis

Portal Server/s Database

server

TM&D System Under Test

Solves the resource limitations

Replace Testers with Virtual Users Runs many Vusers on a few machines Controller manages the virtual users Analyzes results with graph & report tools Repeats tests with scripted actions

Tool: Performance Automation


Sample Transaction

Ability for the AE to generate the Forecast for the next quarter on all of the Plan components

15

10

20

30

35404550

35

25

15

105

0

10

20

30

40

50

60

0 2 4 6 8 10

Time in Biz Hour

No

of U

sers

in S

yste

m

User Load

Slow Ramp up, Collect Client side, Server Monitor mean data

Ramp Down, Collect Client side, Server Monitor mean data

Peak Hour Ramp up, Collect Client side, Server Monitor mean data


User Model

For each Transaction, User Model is created based on

Authentication and Authorization of users Feature Usage Criteria/Transaction Type Total Number of Users in System User random activities on a transaction Transactions/unit time Transaction MIX


Online Monitoring

Some common data for online monitoring Error & Exception Response time graph Throughput graph Transaction passed failed Page/component download time graph Keep checking server logs Capture the slow SQLs Server Resource


Performance Measurement

Workload Data

Data Characteristics

Execution Characteristics

– Path characteristics

– Software resource usage

– Processing overhead

Computer System Usage

– Scenario Response Time

– Scenario Throughput

– Key System Resource Usage

– Resource Utilization

– Server throughput

Software resources:

– CPU Usage

– Memory Usage

– SQL Queries

– File I/O

– Messages

– Logging to files or databases

– Calls to Middleware functions

– Calls to software in a different process, thread or processor

– Application Cache and Buffers

– OS Handles like threads, memory, sockets

– network I/O and type of network resource access,

– number of connections to the databases etc


Assignment Deliverables

Pre-Test Performance Strategy for TM&D Performance Modeling for major sections of applications Resource Engagement outline and Effort estimation Delivery Process Methodology Performance Entry Criteria

On-Test

Detailed Test Plan Test Scripts/Templates, Test design Test Configuration Map (Data/Setup) Test Tool, monitoring agent setup and PoC Report (LoadRunner) Performance Exit Criteria Test Results, Review log Post Test Data Risk/Issue log

Post-Test

System/sub-system performance analysis Report Bottleneck identification


Performance findings and Related Tuning

Code optimization Caching strategy Load balancing Distributed Computing Logic SQL Query Profiling Usage of database indexing Removal of Normalization Resource configuration (RAM, Network) Client software version (OS, Browser) Identification of performance pattern …


Most often performance is thought of as something related to “tuning the code” – this is perhaps the single most reason why performance failures occur

80% performance issues belong to architecture and business definition – it is impossible to get more than 10% benefit by tuning code


Your Questions are welcome Thank you

Bangalore TEAM

©company confidential 1 performance testing for tm & d – an overview

Documents