Dashing Through Black Friday

Tips to Prepare Your Retail Site for Peak Performance

Keynote Global Testing Services

About Today’s Presentation and Our Speakers

Agenda

1. Lessons from 2010

2. Why Web Load Testing

3. The Cloud

4. Best Practices

5. Q&A

Housekeeping

Session length: 50 minutes

Submit questions at any time

Recording will be made available – watch your email for instructions

Twitter hashtag: #keyndashing

Dave Karow Dave Karow is a Senior Product Manager for Internet Test and Measurement at Keynote Systems. Dave leads the online community for the Keynote Internet Testing Environment (KITE) where his training videos and blogs support thousands of Web development and testing users. Prior to Keynote Systems, Dave held Product Management Positions at BMC Software, Marimba, and Remedy Corporation.

Donald Foss Donald Foss has 15 years of Operations experience and 10 years of Performance Testing experience. He designed the original load testing cluster management system for Velogic, a start-up company acquired by Keynote Systems in 2000. At Keynote, Donald manages the consulting team responsible for delivering Keynote Global Testing Services, which does load-testing and performance tuning work for companies with heavily trafficked Web sites.

Aaron Rudger Aaron Rudger leads evangelism and advocacy for Keynote’s Web Performance and Load Testing solutions. He has more than 15 years of experience in the CRM, IT Management, and eCommerce industries.

2 © 2011 Keynote Systems, Inc.

The Good News & the Bad News

3

Online Retail Revenue 2010

© 2011 Keynote Systems, Inc.

Total Online Holiday Retail Sales Growth ‘09 -> ’10: 12%

The Good News & the Bad News

4

Online Retail Performance 2010

© 2011 Keynote Systems, Inc.

10

11

12

13

14

15

16

17

18

Week 1(11/8)

Week 2(11/15)

Week 3(11/22)

Week 4(11/29)

Week 5(12/6)

Week 6(12/13)

Week 7(12/20)

Week 8(12/27)

Week 9(1/3)

Keynote U.S. Retail Electronics Index

Weekly Average Website Response (seconds)

What is a Real Web Load Test Anyways?

Environment

Amount of load

Customer scenarios and paths

Performance metrics and objectives

Other dimensions?

5 © 2011 Keynote Systems, Inc.

The Real Cost of Outages and Abandonment

6

Lost Value Analysis

© 2011 Keynote Systems, Inc.

$0

$100,000

$200,000

$300,000

$400,000

$500,000

$600,000

$700,000

50,000 100,000 150,000 200,000

Realized Revenue / hr Lost Revenue / hr

Revenue Sweet Spot

You must model abandonment in order to determine the cost of poor performance.

The Real Cost of Outages and Abandonment

7

Quantifying Abandonment

© 2011 Keynote Systems, Inc.

Top Retail Categories by Average Order Size Dollars Lost from Abandonment Per

1,000 Orders

Product category Avg order size 1 Error 2 Errors 3 Errors

Computer Hardware $584.47 $140,273 $450,042 $578,625

Event & Movie Tickets $121.60 $29,184 $93,632 $120,384

Automotive $119.23 $28,615 $91,807 $118,038

Office Supplies $102.47 $24,593 $78,902 $101,445

Consumer Electronics $99.12 $23,789 $76,322 $98,129

Child/Baby Care $86.13 $20,671 $66,320 $85,269

Sporting Goods & Outdoor Activities $74.15 $17,796 $57,096 $73,409

Home & Garden $69.99 $16,798 $53,892 $69,290

Shoes & Athletic Footwear $53.14 $12,754 $40,918 $52,609

Flowers, Greetings, & Specialty Gifts $51.61 $12,386 $39,740 $51,094

Computer Software $46.76 $11,222 $36,005 $46,292

Jewelry & Watches $46.62 $11,189 $35,897 $46,154

Health/Wellness/Beauty $45.94 $11,026 $35,374 $45,481

Apparel & Accessories $44.68 $10,723 $34,404 $44,233

Toys, Games & Hobbies $40.41 $9,698 $31,116 $40,006

Source:Nielsen//NetRatings

The Cloud Changes Everything

8

Or Does It?

© 2011 Keynote Systems, Inc.

Site Demand Elastic Capacity

Web Load Testing Best Practices

Load Testing is an iterative process – it takes time

Use testing tools that model your site accurately

Tests should take into consideration: Familiarity with your site: New users act differently than returning users

Returning users may have objects in local cache

User Latency Tolerance: Some user are more tolerant of performance problems than others. Users may retry up to 3 times

User Tenacity: Some shoppers are more determined than others

Access connection speed: Your users are on different connection speeds (dial-up, DSL, cable, T1)

Test after major releases and infrastructure changes

It’s now September. How many load tests have you run so far this year?

9

Start Early

© 2011 Keynote Systems, Inc.

Web Load Testing Best Practices

Test from multiple geographically disperse areas

Test using well managed systems

Test multiple profiles – no fixed pause times

Test using realistic models that resemble your web logs – no straight lines

10

Use the Right Tools for the Job

© 2011 Keynote Systems, Inc.

Web Load Testing Best Practices

A playbook lists all activities before, during and after the test

Turn off/delay back up jobs

Turn off/delay search indexing

Turn off/delay batch processes

The playbook may evolve. Don’t be afraid to change it.

Most importantly, follow it.

11

Develop a Playbook

© 2011 Keynote Systems, Inc.

Questions?

Follow-up items

• All participants in today’s webinar qualify for a free web load test trial!

• Visit the Keynote Online Retailer Survival Guide: www.keynote.com/retailer

• Subscribe to the Web Performance Watch blog: blogs.keynote.com/the_watch

12

Please submit your questions at this time.

© 2011 Keynote Systems, Inc.