continuous load testing with cloudtest and jenkins

Copyright © 2014 SOASTA, Inc. - All rights reserved.Webinar

andPresent

Automating the hard parts, made easier

2Copyright © 2014 SOASTA, Inc. - All rights reserved.

Continuous Load Testing

TODAY’S PRESENTERS

Mark Prichard: CloudBees, Dir. Product ManagementMike Ostenberg: SOASTA, Dir. Systems EngineeringBrad Johnson: SOASTA, VP Product

TWO COMPANIES, A SINGLE SOLUTION

- the leader in Mobile and Cloud Testing

- the home of Jenkins in the Cloud

Together, we offer a complete cloud-based service:

Continuous Delivery for Web & Mobile


• Why should we load test earlier? • Building tests and preparing the test environment• Connecting your load tests to your CI (Jenkins)• Establishing and managing to a performance

baseline• Q&A

Questions: Submit in the question box during event


“Because it’s hard.”oTools require coding skills we don’t have (C, VB?)oWe don’t have a scalable test environmentoWe don’t have hardware for load generationoIt’s a different tooloThere are too many other tests to do



“Because it takes too long.”oScripting takes foreveroResults are slowoFast setup of environments is impossibleoWe’re dependent on IT or other teamsoWe’re too agile




“Because no one ever told us to.”oThere are no “performance coverage” requirementsoThere isn’t a performance baseline for referenceoThere isn’t a defined performance targetoNo one is managing to a performance metric




“Because no one ever told us to.”oThere are no “performance coverage” requirementsoThere isn’t a performance baseline for referenceoThere isn’t a defined performance targetoNo one is managing to a performance metric

Until Now.


Iteration 1

Automated unit tests

The Issue:

• High expectations from tests run with diminishing time windows

• Code-level performance bugs found late set entire project back

• Separate team and schedule conflict with delivery goals

• Major issues escape to production

Iteration 2

Automated unit & functional tests

Iteration 3

Regression test suite

Iteration n

Regression test suiteLoad & Performance Testing

Deploy

Issues?


Iteration 1Automated unit tests

Deploy

Production perf tests

Iteration 2Automated unit & functional tests

Iteration 3Regression test suite

Incl:

Iteration nRegression test suite

Incl: End to End Production Scale Test

Small perf test Small perf testsPerf smoke tests

Perf smoke tests

The Solution:

• Run incremental performance tests automatically with each iteration

• Establish a trend line for performance early

• Developers find code and architecture issues early

• Performance teams focuse on end-to-end infrastructure and user related testing

• Minimize production escapes and implement production testing

Development/Test team Performance/Ops teamDevOps


I JUST MADE A CODE CHANGE

CloudTest runs load tests via Jenkins

Daily (or more frequent) updates keep teams “performance focused”

Identify bottlenecks

& inform teams, just

like unit and functional

tests in Jenkins

Jenkins launches suitable cloud-based test environment

Manage with a new metric!


Mark Prichard

@mqprichard@mqprichard

o#1 OSS CI server

oWritten in Java

oHighly Extensible

oAbout 8 years old

oEasy to install/use

oExtensible via 1000+ plugins

oWidely adopted

o 70K+ installations

oCloudBees knows Jenkins

oOn-premise, cloud and hybrid

oExpert support

oEnterprise plugins

Have you met Jenkins?

oLet developers focus on developing

o Leave building, packaging, testing, distributing to a computer

oMake the right thing the easiest thing – don’t think about it

oMany, widely-adopted plugins

o Safety in numbers – hugely active community

oDashboard view of the state of your builds

o You don’t have to be an expert at everything

oTest Anywhere

oDeploy on-premise or in the cloud for testing

o Spin up VMs or EC2 Instances as build executors

oWe can do it for you, if you like…

Continuous Integration and Automated Testing

• CloudBees ClickStarts – A complete project in just two clicks!• Bees Shop Demo

• Git repo, database, build and deploy in seconds• Online examples: https://partnerdemo.ci.cloudbees.com

Setting up the Test Envrionment


o Demo:

• Bees Shop Overview

• SOASTA Cloud Test: Setting up a load test up for unattended runs

• Jenkins Performance Data:

• Reviewing Performance Trending Data: Track performance of each build

• Reviewing Performance Pass/Fail Data: Failing

• Setting up Continuous Load Testing

• Install Plot Plugin

• Download GitHub: https://github.com/mostenberg/SOASTAPERFPERL

• Add steps to plot build data for any metrics of interest.

• Publish Junit test results to PASS/FAIL based on performance thresholds: SOASTA/5_PERF_THRESHOLD_RESULTS.xml


Mike Ostenberg


TouchTestContinuous

Testing

CloudTestLoad &

Performance

mPulseReal User Monitoring

• Open Source: online at jenkins-ci.org

• Automatically available on CloudBees DEV@cloud

• Complete control over CloudTest/TouchTest Automation from within your Jenkins builds:

• Totally integrated into Jenkins environment

SOASTA Jenkins Plug-in

• CloudBees ClickStarts – A complete project in just two clicks!• Example configurations all online:

• https://partnerdemo.ci.cloudbees.com/• Bees-Shop-PerformanceTest

Try it out yourself!


Thanks & Q&A

Contact SOASTA:www.soasta.com/cloudtest/[email protected] us:

twitter.com/cloudtesttwitter.com/cloudtest

facebook.com/cloudtestfacebook.com/cloudtest

Knowledge Center•White Papers•Webinar Recordings•Case Studies

CloudLink Community•Support•Tutorials•Video

RESOURCES

Contact [email protected] us:

twitter.com/cloudbeestwitter.com/cloudbees

facebook.com/CloudBeesfacebook.com/CloudBees

www.SOASTA.com www.CloudBees.com

blog.cloudbees.com•Look for Mark’s post on this topic

developer.cloudbees.com•Platform Services•Frameworks•ClickStarts

continuous load testing with cloudtest and jenkins

Technology