‘Tis the SeasonA Holiday 2014 Ecommerce Checklist

Preparing for best case retail success

Holiday 2013 was

BIG

2014 will be

BIGGER

mobile matters

your site needs to

PERFORM

40 % abandon a website that takes more than 3 seconds to load

A 1 second delay in page response can result in a 7% reduction in conversions

79% of shoppers who are dissatisfied with website performance are less likely to buy from the same site again.

If an ecommerce site is making

$10,000 a day, a 1 second delay could cost you

$250,000 in lost sales every year

Start NowPrepare for the holiday rush

‘Tis the SeasonA Holiday 2014 Ecommerce Checklist

Preparing for best case retail success

Elizabeth Scott, TenzingDirector of Technical Services @BethxScott

Brad Johnson, SOASTAVP Product Marketing@bradjohnsonsv

Your Hosts

Infrastructure and Application

Monitoring

Load Testing

Marketing

Agenda

Poll Question

Infrastructure and Application

Assess Your Infrastructure

Current infrastructure performance and capacity baseline:

Server Hardware

Storage

Network Database Application

Assess Your Infrastructure

Know how many users can be on your site: Do a load test

Understand your vendor’s capacity planning process

Develop an emergency capacity plan

Ensure your environment is up to date on latest patch levels

Develop an emergency patch and security plan

Secure Your Environment

Ensure ports on your servers are not open unnecessarily

Limit access

Secure Your Environment

Develop and test your Denial of Service response strategy

Run a vulnerability scan

Secure Your Environment

Tune queries, search results and session limits

Leverage a CDN

Tune Your Application

Find free capacity:

Adjust your caching

Adjust ‘type ahead’ search

Remove/adjust view all options

Adjust other features which are resource hogs

Tune Your Application

Monitoring

Application PerformanceEnd User ExperienceTraffic Levels

Measure, Analyze, Act

•Upstream ConnectionsPayment GatewayShipping and Fulfillment

Measure, Analyze, Act

Plan for the Inevitable

Assess your risk …

WHAT WILL GO DOWN AND WHY?

Plan for the Inevitable

•Create a planWebsite overload plan

Incident response plan

Security/DoS response plan

Plan for the Inevitable

COMMUNICATE

Plan for the Inevitable

Modern Load Testing

True Story

• Division of $6B Company

True Story

• Division of $6B Company• Cyber Monday & last Free Shipping Day

True Story

• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”

True Story

• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”• Crashed on Cyber Monday

True Story

• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”• Crashed on Cyber Monday• No answers on Thursday

True Story

• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”• Crashed on Cyber Monday• No answers on Thursday• Production testing on Friday

reproduced the issue

True Story

Is there anything missing from the

plan?

Prerequisite: Know Your Real Users

Modern Load Testing

Stop guessing about what to test

Most accurate models of real user behavior

Meaningful scale & appropriate concurrency

Control to start, stop & modify tests without delays

Test on your schedule

Modern Load Testing

Test for spikes – using the cloud

Failover Scenarios Success Disasters End to End

Test what you don’t think needs to be tested

Modern Load Testing

Measure, Test and Act in Real Time

Modern Load Testing

Don’t Wait until it’s too late

Compress processes for more test iterations

Traditional Approach

TIME

Test development Infra. setup Fix Repeat?

RetestTest Analyze

Modern Load Testing

Don’t Wait until it’s too late

Compress processes for more test iterations

/

Traditional Approach

TIME

Test development Infra. setup Fix Repeat?

RetestTest Analyze

CloudTest

Modern Load Testing

Don’t Wait until it’s too late

Compress processes for more test iterations

/

Traditional Approach

TIME

Test development Infra. setup Fix Repeat?

RetestTest Analyze

/ / / / Extra time!CloudTest

Modern Load Testing

Use real user data toiterate dev & test plans

Run iterative load tests via Jenkins

Daily (or more frequent) updates keep teams quality focused

Embedded failure details & bottleneck data isolate issues

Manage to a performance

baseline

Mobile? Run functional automation on real devices for functional & performance validation

Spin up suitable, consistent test environments in the

cloud or the lab

I JUST MADE A CODE CHANGE

Jenkins runs any unit tests as usual

Test Continuously – including performance!5. Don’t Wait

s

Memory leaks

Test Scale

Sta

ge /

Tea

m

Dev & Test

Release& Deploy

Network& Ops

CDN file placement

Load Balancer configuration

Network bandwidth

Network configuration

DNS routing

Inadequate server resources

Default configuration settings

Unbalanced web serversAuto-scaling failures

Latency between systems

Slow third-party plug-ins

Garbage collection

Database thread counts

Inefficient database queriesSlow pages

Conflict with other apps

Test Lab Staging Production (100% +++)

Search technology limits

Method-level tuning

Max sockets exceeded

Firewall max capacity

Global latency variance

Security bottlenecks

Continuous tuning for full-scale…and well beyond

Start early, end in the real environment

6. Test in production and at Production scale

• Tests for flash sales with unknown spikes

• mPulse analyzes real user performance

• Continuous tests of mobile and web traffic

“SOASTA was very easy to work with and could accommodate our requirements in a four-week

timeframe.” Gavin Cliffe, OZSale CTO.

OZSale

NO ISSUES

“SOASTA was very easy to work with and could accommodate our requirements in a four-week

timeframe.” Gavin Cliffe, OZSale CTO.

OZSale

160% more visitors

$5M AUD in one sale

50% mobile

Critical Factors for Performance Testing Retail Sites

Test for spikes Test what you think

doesn’t need to be tested

Test expecting failure Analyze & act in real

time

Don’t forget about the user experience

Monitor everything Don’t wait until just

before the holidays Test in production

Marketing

Communicate your campaign plan internally and with stakeholders

Establish a campaign change process

Approval

Communication

Campaign Management

Establish a change freeze on application and infrastructure

Limit publishing and catalogue changes

Campaign Management

Campaign Best Practices

Segment your list into smaller groups

Trickle your emailManage your media content

Email Best Practices

Don’t get too fancyDevelop test cases

for all of your discount codes

Promotions and Discounts

Avoid using discount structures that require a database search

Don’t use new discount structures

Promotions and Discounts

TEST, TEST, TEST!

Promotions and Discounts

Are you interested in a FREE Retail Readiness Performance Test

Contact SOASTA:www.soasta.cominfo@soasta.com

866.344.8766Follow us:

Twitter.com/cloudtestFacebook.com/cloudtest

Retail Readiness Page: soasta.com/retail-readiness/ Case studies: soasta.com/customers/ecommerce/

Webinars future and past: soasta.com/knowledge-center/webinars/

CloudTest Lite is FREE (forever)!soasta.com/FREE

Resources from SOASTA

Thank you

www.SOASTA.com

more than managed hosting. commerce anywhere.www.tenzing.com