‘Tis the SeasonA Holiday 2014 Ecommerce Checklist
Preparing for best case retail success
Holiday 2013 was
BIG
2014 will be
BIGGER
mobile matters
your site needs to
PERFORM
40 % abandon a website that takes more than 3 seconds to load
A 1 second delay in page response can result in a 7% reduction in conversions
79% of shoppers who are dissatisfied with website performance are less likely to buy from the same site again.
If an ecommerce site is making
$10,000 a day, a 1 second delay could cost you
$250,000 in lost sales every year
Start NowPrepare for the holiday rush
‘Tis the SeasonA Holiday 2014 Ecommerce Checklist
Preparing for best case retail success
Elizabeth Scott, TenzingDirector of Technical Services @BethxScott
Brad Johnson, SOASTAVP Product Marketing@bradjohnsonsv
Your Hosts
Infrastructure and Application
Monitoring
Load Testing
Marketing
Agenda
Poll Question
Infrastructure and Application
Assess Your Infrastructure
Current infrastructure performance and capacity baseline:
Server Hardware
Storage
Network Database Application
Assess Your Infrastructure
Know how many users can be on your site: Do a load test
Understand your vendor’s capacity planning process
Develop an emergency capacity plan
Ensure your environment is up to date on latest patch levels
Develop an emergency patch and security plan
Secure Your Environment
Ensure ports on your servers are not open unnecessarily
Limit access
Secure Your Environment
Develop and test your Denial of Service response strategy
Run a vulnerability scan
Secure Your Environment
Tune queries, search results and session limits
Leverage a CDN
Tune Your Application
Find free capacity:
Adjust your caching
Adjust ‘type ahead’ search
Remove/adjust view all options
Adjust other features which are resource hogs
Tune Your Application
Monitoring
Application PerformanceEnd User ExperienceTraffic Levels
Measure, Analyze, Act
•Upstream ConnectionsPayment GatewayShipping and Fulfillment
Measure, Analyze, Act
Plan for the Inevitable
Assess your risk …
WHAT WILL GO DOWN AND WHY?
Plan for the Inevitable
•Create a planWebsite overload plan
Incident response plan
Security/DoS response plan
Plan for the Inevitable
COMMUNICATE
Plan for the Inevitable
Modern Load Testing
True Story
• Division of $6B Company
True Story
• Division of $6B Company• Cyber Monday & last Free Shipping Day
True Story
• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”
True Story
• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”• Crashed on Cyber Monday
True Story
• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”• Crashed on Cyber Monday• No answers on Thursday
True Story
• Division of $6B Company• Cyber Monday & last Free Shipping Day• “We wouldn’t disrupt the plan”• Crashed on Cyber Monday• No answers on Thursday• Production testing on Friday
reproduced the issue
True Story
Is there anything missing from the
plan?
Prerequisite: Know Your Real Users
Modern Load Testing
Stop guessing about what to test
Most accurate models of real user behavior
Meaningful scale & appropriate concurrency
Control to start, stop & modify tests without delays
Test on your schedule
Modern Load Testing
Test for spikes – using the cloud
Failover Scenarios Success Disasters End to End
Test what you don’t think needs to be tested
Modern Load Testing
Measure, Test and Act in Real Time
Modern Load Testing
Don’t Wait until it’s too late
Compress processes for more test iterations
Traditional Approach
TIME
Test development Infra. setup Fix Repeat?
RetestTest Analyze
Modern Load Testing
Don’t Wait until it’s too late
Compress processes for more test iterations
/
Traditional Approach
TIME
Test development Infra. setup Fix Repeat?
RetestTest Analyze
CloudTest
Modern Load Testing
Don’t Wait until it’s too late
Compress processes for more test iterations
/
Traditional Approach
TIME
Test development Infra. setup Fix Repeat?
RetestTest Analyze
/ / / / Extra time!CloudTest
Modern Load Testing
Use real user data toiterate dev & test plans
Run iterative load tests via Jenkins
Daily (or more frequent) updates keep teams quality focused
Embedded failure details & bottleneck data isolate issues
Manage to a performance
baseline
Mobile? Run functional automation on real devices for functional & performance validation
Spin up suitable, consistent test environments in the
cloud or the lab
I JUST MADE A CODE CHANGE
Jenkins runs any unit tests as usual
Test Continuously – including performance!5. Don’t Wait
s
Memory leaks
Test Scale
Sta
ge /
Tea
m
Dev & Test
Release& Deploy
Network& Ops
CDN file placement
Load Balancer configuration
Network bandwidth
Network configuration
DNS routing
Inadequate server resources
Default configuration settings
Unbalanced web serversAuto-scaling failures
Latency between systems
Slow third-party plug-ins
Garbage collection
Database thread counts
Inefficient database queriesSlow pages
Conflict with other apps
Test Lab Staging Production (100% +++)
Search technology limits
Method-level tuning
Max sockets exceeded
Firewall max capacity
Global latency variance
Security bottlenecks
Continuous tuning for full-scale…and well beyond
Start early, end in the real environment
6. Test in production and at Production scale
• Tests for flash sales with unknown spikes
• mPulse analyzes real user performance
• Continuous tests of mobile and web traffic
“SOASTA was very easy to work with and could accommodate our requirements in a four-week
timeframe.” Gavin Cliffe, OZSale CTO.
OZSale
NO ISSUES
“SOASTA was very easy to work with and could accommodate our requirements in a four-week
timeframe.” Gavin Cliffe, OZSale CTO.
OZSale
160% more visitors
$5M AUD in one sale
50% mobile
Critical Factors for Performance Testing Retail Sites
Test for spikes Test what you think
doesn’t need to be tested
Test expecting failure Analyze & act in real
time
Don’t forget about the user experience
Monitor everything Don’t wait until just
before the holidays Test in production
Marketing
Communicate your campaign plan internally and with stakeholders
Establish a campaign change process
Approval
Communication
Campaign Management
Establish a change freeze on application and infrastructure
Limit publishing and catalogue changes
Campaign Management
Campaign Best Practices
Segment your list into smaller groups
Trickle your emailManage your media content
Email Best Practices
Don’t get too fancyDevelop test cases
for all of your discount codes
Promotions and Discounts
Avoid using discount structures that require a database search
Don’t use new discount structures
Promotions and Discounts
TEST, TEST, TEST!
Promotions and Discounts
Are you interested in a FREE Retail Readiness Performance Test
Contact SOASTA:[email protected]
866.344.8766Follow us:
Twitter.com/cloudtestFacebook.com/cloudtest
Retail Readiness Page: soasta.com/retail-readiness/ Case studies: soasta.com/customers/ecommerce/
Webinars future and past: soasta.com/knowledge-center/webinars/
CloudTest Lite is FREE (forever)!soasta.com/FREE
Resources from SOASTA
Thank you
www.SOASTA.com
more than managed hosting. commerce anywhere.www.tenzing.com