17 productized services cloud performance testing

Productized Services: Cloud Performance Testing

Presented By

© 2012 WIPRO LTD | WWW.WIPRO.COM | INTERNAL1

Presented By

Rupesh Garg,Wipro Technologies.

All About Cloud

What is Performance Testing


Cloud Testing Approach

Cloud Service Providers

Any Questions

Productized Services - Definition


• Allow the market to self-service their final deliverables by means of standardized packages with key parameters and KPIs, which dramatically shortens sales cycles and reduces cost of sales.

• Help in ensuring repeatability and consistency, and also

Benefits of Productized services


• Help in ensuring repeatability and consistency, and also reduce the amount of customer intervention in the process, which in turn improves profitability.

• Allow service providers to achieve efficiencies as deliverables are pre-defined, and enable them to offer differentiated and top-up services at a premium, thereby increasing profitability.

All About Cloud

Cloud?

Cloud computing is a model for enabling convenient, on-demand


Cloud computing is a model for enabling convenient, on-demandnetwork access to a shared pool of configurable computing resourcesthat can be rapidly provisioned and released with minimal managementeffort or service provider interaction.

Cloud?

Deployment

Models

Service

Models

Software as a

Service (SaaS)

Platform as a

Service (PaaS)

Infrastructure as a

Service (IaaS)

Hybrid Clouds

CommunityCommunityCloudCloud

Private Private CloudCloud

Public CloudPublic Cloud


Essential

Characteristics

Common

Characteristics

Resource Pooling

Broad Network Access Rapid Elasticity

Measured Service

On Demand Self-Service

Low Cost Software

Virtualization Service Orientation

Advanced Security

Homogeneity

Massive Scale Resilient Computing

Geographic Distribution


Performance Testing

Performance Testing is used to evaluate following qualities in a system

Reliability

Scalability

Interoperability

Why Performance Testing?


Interoperability

• Causes of inadequate performance

testing

– Lack of requirements

– Lack of understand of the fundamentals of performance testing

Understand Performance Testing


performance testing

– Misinformation provided my tool and simulator vendors

– Limited budget or funding

– Enormity of the task

Evolution of Performance Testing

Load Generator



Load Generator


Firewalls

Load balancers

Routers

Switches

Appliances


Geographic

Locations

Load Generator


Firewalls

Load balancers

Routers

Switches

Appliances


Geographic

Locations

Network

Providers

Load Generator


Firewalls

Load balancers

Routers

Switches

Appliances


Geographic

Locations

Network

Providers

Load Generator


Routes

Latency

Firewalls

Load balancers

Routers

Switches

Appliances


Geographic

Locations

Network

Providers

Load Generator


Routes

Latency

Firewalls

Load balancers

Routers

Switches

Appliances

End-User

Response Time

Cloud Infrastructure Data Business Logic End-user Experience

Performance Testing

SaaS: Performance Testing Approach


Performance Analysis

Performance Management

Load Testing

The cloud brings substantial benefits to load testing, whether thetesting is

carried out by a large corporation with an in-house performance centeror by

a start-up.

o More flexibility: multiple generators, multiple internal resources

Why use the cloud for load testing?


o

etc.

o More savings: you only pay for the power you use, when you really need it.

o More realistic: you can test more realistic scenarios with load being input from different parts of the globe.

o No limits: you can progressively test heavier and heavier loads (the power available is almost limitless) to find the real limits of

• Ability to correlate the load with end-user response times and the impact on the

systems

• We can borrow particular environment as per needs rather than going through

the licensing and setup worries.

• Ability to control the distribution of the load

Using the Cloud for Load Testing


• Ability to use the same systems and configuration for subsequent testing

� With the help of Virtualization, it enables us to test multiple configurations,

which was not possible few years back.

� Perfect coordination of the load injectors

� Know and understand the origination of the load (IP, geography)

Traditional Performance Testing Approach

ControllerLG


SUT

Controller

LG

LG

LG

LG

Cloud World


Performance Testing From Cloud


Performance Testing Approach

Automation Test

Scripts Preparation

Web Tire

App Tire

DB Tire

Test Execution

& Monitoring


Requirement

Gathering

Test Results

Analysis

Typical Performance Testing

Gather Plan Design Execute Analyze

POC

Environment

Setup costs

Capacity Planning

Test Data Management

Load

Injectors

Performance

Modeling


Tool Setup

costs

Cloud Performance Testing

Gather Plan Design Execute Analyze

Load injectors/Test Data


Tool/Environment/

POC setup on the

cloudIAAS/PAAS/SAAS

Load injectors/Test Data /Performance

Modeling/Modeling

Capacity on cloudIAAS/SAAS/PAAS

Cloud Performance Testing Setup

F

i

re

wa

ll

Performance Tester

Performance

Testing and Monitoring Tool

Installed (SAAS)

Data Storage

(PAAS)


F

i

r

ewallApp Under Test

Load Injectors

from Cloud ( IAAS)

Solution Overview

Client System Under Test

Environment

Wipro Performance Team @ Atlanta


Environment

Wipro Performance Team @ Flex

Delivery Center Offshore

IaaS Provider

Performance

Test Tool SaaS

Provider

• Private Cloud

• Know and understand the origination of the load (IP, geography)

• Ability to control the distribution of the load

• Ability to use the same systems and

configuration for subsequent testing

• Perfect coordination of the load injectors

Controlled Cloud Testing


• Perfect coordination of the load injectors

• Ability to correlate the load with end-user

response times and the impact on the

systems

Wipro Cloud load tests


• NEOTYS

• App Perfect

• Gomez

• Platform Lab

• Soasta

• Keynote

• Browser Mob

Cloud Based Testing Services


• Browser Mob

• Load Impact

• Load Strom

• HP - Beta

• sauce LABS

• PushToTest

• LoadStorm™ is a cloud load testing tool that

generates web traffic for you. Please sign-up for

a free account or take a quick tour of the tool.

Load Storm


• SOASTA, Inc. provides services to test websites and web applications.[2][3] It is the leading provider of cloud-based testingservices, and created the industry's first browser-based website testing product. Website tests include load testing, performance testing, functional testing and user interface testing. SOASTA provides cloud website testing with their product CloudTest, which simulates thousands of users visiting a website simultaneously using

http://www.soasta.com/cloudtest/


simulates thousands of users visiting a website simultaneously using the Amazon Elastic Compute Cloud (EC2) service. SOASTA allows customers to use predefined tests or create customized tests to automatically test their web.

• "There was no way we could ever have simulated 500,000 users against our site in an internal test lab," said Christopher Roe, CTO, Qtrax. "By cloud testing we were able to identify connection issues immediately and fix the site prior to production.“

http://www.keynote.com/products/


Gomez:

Test from where your end-users are located by selecting from over 100,000+ load generating consumer-grade desktops

500+ Combinations of Browsers, Operating Systems and Screen Resolutions. Whether you need to test on Windows XP and Internet Explorer 6, Mac OS X 10.5 and Firefox 2.0 or the iPhone OS and Safari 2.0, Gomez’s comprehensive, up-to-date virtual test bed has

Gomez


Safari 2.0, Gomez’s comprehensive, up-to-date virtual test bed has you covered with more than 500 combinations of browsers, operating systems and screen resolutions plus more than 5,000 supported mobile devices.

http://saucelabs.com( Automation)Cross browser testing with Selenium on Cloud


• SAP performance testing

• Cloud load testing service: where the load comes from the cloud

• Stress testing cloud services: where the system under test is in the cloud

• Load testing CRM and ERP implemenations

• Infrastructure performance monitoring and tuning (including the

http://acutest.co.uk/acutest/load-testing


• Infrastructure performance monitoring and tuning (including the network)

• Performance test tools evaluation

• Load testing web applications, websites and web services

• Performance testing multi-tiered solutions that include a combination of applications, IT systems and infrastructure

• IT systems volume testing (including embedded systems)

Productized Costing Space - 1


• "The cloud [test environment] doesn't reflect production environments --applications run on specific [physical] servers, specific virtualization

technology, specific networking and bandwidth, and that is hard to replicate,“

Forrester Research Inc.

• "Then there's the issue of cloud providers not supporting the technology that you use at all, like a specific HP blade server.“

• Integration, in particular, is causing a lot of headaches

Limitations with Cloud Computing


• Integration, in particular, is causing a lot of headaches


Any Questions

• Basics about Cloud Computing:http://en.wikipedia.org/wiki/Cloud_Computing

• SOASTA Performs 500th Test in the Cloudhttp://www.reuters.com/article/pressRelease/idUS118692+27-Jan-2009+MW20090127

• Will Cloud Computing Be the Tipping Point for Change in the Test Industry?http://www.breakingpointsystems.com/community/blog/will-cloud-computing-

be-the-tipping-point-for-change-in-the-test-industry

• Tapping the cloud as a software testing service

References


• Tapping the cloud as a software testing servicehttp://searchcio-

midmarket.techtarget.com/news/article/0,289142,sid183_gci1356175,00.html

• Preparing for testing applications in the cloud

• http://searchsoftwarequality.techtarget.com/tip/0,289483,sid92_gci1357743_mem1,00.

html

• The First Cloud-Based Global Test Platformhttp://www.soasta.com/blog/?p=142

• The Greatest Risk to Cloud Computing ? The Vista Effecthttp://etherealmind.com/2009/03/22/risk-to-cloud-computing/

17 productized services cloud performance testing

Documents