2012 state of mobile ecommerce performance

2012 State of Mobile Ecommerce PerformanceHow fast do leading m-commerce sites load for real users over cellular networks?October 2012

© Copyright Strangeloop Networks Inc. 2012 www.strangeloopnetworks.com | 2

Contents

Executive summary ..................................................................................................................................... 3

Top 10 sites across devices and networks ................................................................................................... 4

Why care about mobile performance and ecommerce? ................................................................................ 5 The mobile-only audience is growing...............................................................................................5

The growth of mobile commerce......................................................................................................6

When it comes to page load times, mobile users have high expectations.......................................6

The impact of slow load times on m-commerce revenues ...............................................................6

Why test on cellular networks? .................................................................................................................... 7 1. Web pages are bigger than ever. .................................................................................................7

2. Latency can vary widely, from as little as 34 milliseconds for LTE to 350 milliseconds or more over 3G. ..................................................................................................7

3. Download speeds can also experience huge variance, from a mere 1 Mbps over 3G to as much as 31 Mbps over LTE. ............................................................................8

M.sites are not a cure-all for mobile performance pains ..................................................................8

Finding 1: The median home page took 11+ seconds to load on both smartphones over 3G. ........................ 9

Finding 2: LTE was 27% faster than 3G. ..................................................................................................... 10

Finding 3: Over LTE, pages loaded 9% faster on the Samsung Galaxy S3 than on the iPhone 5. ................................................................................................. 11

Finding 4: Over 3G, pages loaded 22% faster on the iPad 2 than on the Samsung Galaxy tablet. ................................................................................................ 12

Finding 5: One out of three companies don’t have a mobile-specific site. ................................................... 13

Finding 6: 32% of companies don’t let mobile users view the full site. ........................................................ 14

Finding 7: Companies serve the full site to iPad 2 users, mobile site to Galaxy tablet users. ............................................................................................................. 15

Takeaways ................................................................................................................................................. 16

Methodology ............................................................................................................................................. 17 About the devices tested................................................................................................................17

About Strangeloop Networks ..................................................................................................................... 18


Executive Summary

Prior to this study, there has never been an attempt to measure the performance of ecommerce sites from the perspective of a real user using cellular networks. Given the fact that 3G touches half the world’s population1, and given the emergence of LTE as a network standard that promises to alleviate the current performance woes suffered by mobile users, this is a glaring, if understandable, omission. We say “understandable” because measuring real-world mobile performance presents a host of challenges -- from device and network variance to the lack of tools available for such testing.

Between July and September 2012, Strangeloop Networks developed an approach for reliably testing mobile load times over 3G and LTE. We first tested the full-site home pages and, when available, the mobile-optimized home pages for 200 leading retail sites as they would load on four devices -- iPhone 4, iPhone 5, Samsung Galaxy S smartpone, Samsung Galaxy S3 smartphone, iPad 2, and Samsung Galaxy tablet -- using their native browsers over a 3G connection. We then tested the full-site and mobile-optimized home pages for top 50 ecommerce sites as they would load for an iPhone 5 and Samsung Galaxy S3 over LTE.

We compared these results against benchmark data for desktop performance of the same set of sites, and arrived at some compelling findings.

1: Ericsson: 85% of the world will see 3G/4G by 2017, GigaOM, June 5, 2012.

Key Findings1. The median home page took 11+ seconds to load on both iPhone 4 and Samsung Galaxy S over 3G.

2. LTE was 27% faster than 3G.

3. Over LTE, pages loaded 9% faster on the Samsung Galaxy S3 than on the iPhone 5.

4. Over 3G, pages loaded 22% faster on the iPad 2 than on the Samsung Galaxy tablet.

5. One out of three companies don’t have a mobile-specific site.

6. 32% of companies don’t let mobile users view the full site.

7. Companies serve the full site to iPad 2 users, and the mobile site to Galaxy tablet users.


Top 10 sites across devices and networks

Samsung Galaxy S (3G) Load time (seconds)

1 emusic.com 1.09

2 adpost.com 4.26

3 cars.com 4.80

4 barnesandnoble.com 5.45

5 iherb.com 5.76

6 egames.com 5.81

7 staples.com 5.81

8 overstock.com 6.21

9 abebooks.com 6.39

10 babycenter.com 6.71

iPhone 4 (3G) Load time (seconds)

1 iherb.com 2.68

2 wellsfargo.com 2.91

3 amazon.com 3.84

4 amazon.co.uk 4.21

5 audible.com 4.35

6 ecrater.com 4.49

7 wetseal.com 4.70

8 coldwatercreek.com 5.13

9 adorama.com 5.63

10 overstock.com 5.69

Samsung Galaxy tablet (3G) Load time (seconds)

1 childrensplace.com 0.21

2 cduniverse.com 1.56

3 tickets.com 1.67

4 oldnavy.com 1.70

5 target.com 1.77

6 walgreens.com 1.80

7 musiciansfriend.com 1.89

8 polo.com 1.98

9 hbo.com 1.99

10 zzounds.com 2.00

iPad 2 (3G) Load time (seconds)

1 childrensplace.com 0.21

2 nike.com 0.63

3 weightwatchers.com 0.71

4 oldnavy.com 0.77

5 hbo.com 1.33

6 copart.com 1.47

7 adorama.com 1.47

8 6pm.com 1.49

9 bjs.com 1.53

10 polo.com 1.58


Why care about mobile performance and ecommerce?

As the mobile audience continues to grow, users are increasingly demanding -- expecting sites to load at least as fast on mobile devices as they do on desktop computers.

Mobile users want to shop online, but slow pages are driving them away, and for site owners, the challenge of delivering a fast user experience is an uphill struggle. A typical web page is now 1MB2, and for many mobile users, that page is being served over a 3G network. “Mobile-optimized” sites are only a partial solution, as one out of three smartphone users will choose to view the full site when given the choice.

The mobile-only audience is growingIn the United States, there were an estimated 115 million mobile users online in 2012, and that number is expected to grow to 176.3 million by 2015.3 By 2014, mobile internet usage is expected to overtake desktop usage.4 By 2015, Cisco estimates that there will be 788 million people worldwide who go online exclusively through their mobile devices.5

By 2015, Cisco estimates that there will be 788 million people worldwide

who go online exclusively through their mobile devices.

2: The growing epidemic of page bloat, GigaOM, May 23, 20123: Majority of US smartphone owners use devices to aid shopping, eMarketer, April 12, 20124: The growth of mobile marketing and tagging, Microsoft Tag, March 21, 20115: Global Mobile Data Traffic forecast Update, 2011-2016, Cisco, February 14, 2012


The growth of mobile commerceFour out of five smartphone owners use their devices to shop6, and 25 percent of consumers engage in online shopping exclusively via mobile.7 In 2012, Forrester estimated that m-commerce generated $10 billion in revenue. Forrester also estimates that by 2016, that number will more than triple, growing to $31 billion.8

When it comes to page load times, mobile users have high expectations.A 2012 survey by Keynote found that 64% of smartphone users want a web page to load within four seconds, and 82% expect a page to load within five seconds.9

The impact of slow load times on m-commerce revenuesIn recent years, site owners have developed an awareness of the connection between performance and sales for shoppers visiting via desktop computers; however, it is only very recently that this awareness has extended to mobile performance. One landmark study in the fall of 2012 revealed that a mere 1-second delay in load times led to dramatic losses in bounce rate, page views, conversions, and cart size10:

Bounce rate Page views Conversions Cart size-8.3% -9.4% -3.5% -2.1%

6: From brick-and-mortar to mobile click-and-order: Which retailers are carving out space in the m-commerce market? comScore, September 20127: 25% of mobile users shop online only via smartphone or tablet: study, Luxury Daily, July 9, 20128: Mobile Commerce Forecast: 2011 to 2016, Forrester, June 20119: Mobile consumers expect speed greater than many retailers are providing, Internet Retailer, August 7, 201210: Case study: The impact of HTML delay on mobile business metrics, Web Performance Today, November 2011


Why test on cellular networks?

While many mobile users access the internet via wifi, and as a result have similar bandwidth to a desktop user, more than half of the world’s mobile users access the web over 3G networks. Yet little has been done to date to measure how the web performs from this perspective.

At the other end of the spectrum, LTE is widely hailed as a panacea for poor mobile performance. With the release of the latest round of LTE smartphones, many consumers expect to see a marked improvement in response times. This study is one step toward answering the question: How high speed is “high speed”?

There are three key limiting factors to consider when understanding the performance of web pages over cell networks:

1. Web pages are bigger than ever. The average web page is over 1 MB and contains at least 80 resources (such as images, JavaScript, CSS files, etc.). This has a significant impact on desktop performance. Its impact on mobile performance -- and particularly on 3G performance -- is much more dramatic. This impact will be felt even more keenly over the next three years. At the current rate of growth, pages could surpass 2 MB by 2015.

2. Latency can vary widely, from as little as 34 milliseconds for LTE to 350 milliseconds or more over 3G. This means that every unique page resource is subjected to up to 350 milliseconds of latency. (Latency is the amount of time it takes for the host server to receive and process a request for a page object. The amount of latency depends largely on how far away the user is from the server.)


Latency is outside the control of site owners, who should never underestimate the crippling effects of latency on the usability of their websites. Latency is consistent only in its inconsistency, even when measured at the same location. This is due to a number of variables beyond the amount of data passing through the tower. Factors such as the weather, and even the direction the user is facing, can have a significant impact.

3. Download speeds can also experience huge variance, from a mere 1 Mbps over 3G to as much as 31 Mbps over LTE. It’s interesting to compare this to the average U.S. broadband speed of 15 Mbps,11 and to note that 3G can be up to 15 times slower than broadband, while LTE can be up to twice as fast.

M.sites are not a cure-all for mobile performance painsMany site owners attempt to respond to the combination of high user demands, large web pages, and poor connection speeds by developing smaller, faster, stripped-down “m.sites”; however, these attempts are not completely effective, as up to 35% of mobile users will choose to view the full site when given the option.

These full-site visitors are significantly more likely to spend than m.site visitors. One study found that, for every $7.00 of mobile-generated revenue, $5.50 was generated via full site. Only $1.00 came via m.site, and $0.50 via app.12

11: Report on Consumer Wireline Broadband Performance in the U.S., Federal Communications Commission, July 201212: Case studies from the mobile frontier: the relationship between faster mobile sites and business KPIs, Strangeloop Networks, November 2011


Finding 1: The median home page took 11+ seconds to load on both smartphones over 3G.

At 11.8 seconds for Samsung Galaxy S and 11.5 seconds for iPhone 4, the median home page loaded 40% more slowly on both smartphones than it did on the desktop.

Mobile-specific pages were faster (6.8 seconds for Galaxy S; 5.4 seconds for iPhone 4), but even these numbers lag behind user expectations for sub-4-second page loads.


Not only were most sites slow, many sites were extremely slow. 9% of full sites took 20+ seconds to load on the iPhone 4, compared to 6% for Galaxy S.

What this means: A 2012 survey by Keynote Systems found that 64% of mobile users expect sites to load in 4 seconds or less, and 82% expect load times of 5 seconds or less. Pages that load in 11+ seconds are clearly unable to meet modern expectations.

Finding 2: LTE was 27% faster than 3G.

The average load time13 for pages on 3G was 11.7 seconds, compared to 8.5 seconds for LTE.

What this means: It is no surprise that pages are served more quickly over LTE than over 3G. It is interesting to note, however, that, while pages were faster, this speed improvement was not of the magnitude predicted by some LTE proponents, who have stated that LTE networks are, on average, 10 times faster than their 3G counterparts.14

13: Based on adding the median load times for full-site home pages for Android and iPhone, then averaging the result.14: 4G Explained: What Is LTE?, Mashable, June 12, 2012


Finding 3: Over LTE, pages loaded 9% faster on the Galaxy S3 than on the iPhone 5.

The median full-site page loaded in 8.1 seconds on the Galaxy S3, compared to a median load time of 8.9 seconds on the iPhone 5. This performance gap was consistent across the sites tested.

What this means: While this finding is consistent enough to warrant mention and further inquiry, it is important to note that these results should not be interpreted as a product review. Both the Galaxy S3 and iPhone 5 devices loaded pages significantly faster over LTE than over 3G, which had median load times of 11.8 and 11.5 seconds, respectively. LTE load times approached the desktop median load time of 7.1 seconds -- a promising improvement.


Finding 4: Over 3G, pages loaded 22% faster on the iPad 2 than on the Galaxy tablet.

What this means: As with the previous finding, this should not be considered a product review. While it is possible that the iPad 2 offers superior performance, it may also be possible that site owners give iOS devices and browsers a performance leg up: a site owner who has limited resources for optimizing front-end performance may choose to focus their efforts on optimizing for iOS rather than other platforms.

This finding was strikingly consistent across most of the sites tested, indicating that there is a common root cause behind the fact that pages loaded significantly faster on the iPad 2 than on the Galaxy tablet.


Finding 5: One out of three companies don’t have a mobile-specific site.

Of the 200 sites tested, 68 did not have a mobile site. This could be attributed to a number of causes:• Site owners could be prioritizing

native apps over the mobile web.• Site owners could be applying

responsive design principles to their full sites.

• Site owners could lack resources for building a separate mobile site.

• Site owners could still be in the process of developing their mobile strategy.

What this means: A Google survey of smartphone users found that almost 75% of people say they prefer using a “mobile-friendly” site.15 This prompts the question: what is a “mobile-friendly” site? This has emerged as a controversial issue in the user experience community.

One faction -- led by usability expert Jakob Nielsen16 -- argues for the development of greatly simplified mobile-optimized sites because, in the short term, they yield better usability test results for mobile users than full sites do. Among other things, Nielsen suggests that site owners cut features and content that are not core to the mobile use case.

The counter argument -- championed by user experience expert Karen McGrane17 -- is that creating unique pages for mobile users, known as “forking” content, not only shortchanges users in the short term, it ultimately becomes a headache for site owners to maintain. (While in an ideal world, a site’s content management system would be able to govern the intricacies of serving platform-specific pages, this is not a realistic solution for site owners using older CMSes.) McGrane argues that it’s too early in the evolution of the mobile market to guess what type of content a mobile user wants; users would be better served by receiving full site content in a format that can be viewed easily on a mobile device -- for example by building responsive templates to cover a range of phones and tablets.

15: What Users Want Most from Mobile Sites Today, Google, September 201216: Mobile Site vs. Full Site, Jakob Nielsen, April 10, 201217: A separate mobile website: No forking way, .Net Magazine, May 8, 2012


Finding 6: 32% of companies don’t let mobile users view the full site.

When testing from smartphones, we found that 42 of the 200 URLs will not direct to the full site, and 22 don’t have a ‘view full site’ option.

18: 32% of customers choose to view the full site on their mobile device, Strangeloop Networks, September 201119: Case studies from the mobile frontier: the relationship between faster mobile sites and business KPIs, Strangeloop Networks, November 2011

What this means:A separate Strangeloop study has found that up to 35% of smartphone users will choose to view the full site over the mobile site when given the option18 and mobile visitors who view the full site spend up to 7.5 times more per visit than m.site visitors.19 By not allowing mobile visitors the ability to navigate to the full site, site owners are missing out on significant revenue opportunities.


Finding 7: Companies serve the full site to iPad 2 users, mobile site to Galaxy tablet users.

What this means:Any tablet user who has tried to visit a website, only to be served a stripped-down mobile-optimized page, will attest to how frustrating this experience is. Regardless of the brand of tablet being used, almost all tablet users expect the same experience on their tablet as on their desktop.

One third of the sites tested -- 68 out of 200 -- automatically direct Galaxy tablet users to the mobile-specific site rather than the full site. This was not an issue on the iPad 2.


Takeaways

1. Conduct performance tests across a broad range of parameters -- including devices and networks.This will give a real-world perspective of your site’s mobile performance challenges.

2. Be aware that cellular performance can fluctuate wildly, even at the same location. This holds true for both 3G and LTE networks.

3. If you serve different pages to mobile than you do to desktop visitors, allow people to easily navigate to the full site. Equally important: allow them the ability to return to the mobile site.


Methodology

The test group included 200 leading ecommerce sites, as ranked by Alexa.com. Each site’s home page was tested 30 times per device across six devices -- iPhone 4, iPhone 5, Samsung Galaxy S smartphone, Samsung Galaxy S3 smartphone, iPad 2, and Samsung Galaxy tablet.

The iPhone 4, Galaxy S, iPad 2 and Galaxy Tablet were tested over a 3G connection. The tests were conducted over a two-week period, from July 16-27, 2012.

The iPhone 5 and Galaxy S3 were tested over an LTE connection. The tests were conducted over a two-day period, from September 26-27, 2012.

For all tests, devices were positioned in the same location, in an attempt to mitigate the latency impact caused by location changes. For all tests, devices and radios were at full power and screens were not allowed to lock during testing.

The tests in this study were conducted using a RUM beacon inserted into the page that captured the onload time. The cache and cookies were cleared automatically between each test.

For both smartphones and tablets, we calculated both average and median results across these metrics:

• load time (mobile site)• load time (full site)• resource requests (mobile site)• resource requests (full site)

Sites that did not yield a result were marked as null in the test results.

About the browsers and devices testedWe used the stock browser on all devices.

iPhone 4Version 5.1.1Model MC318LL

iPhone 5Version 5.0Model A1428

iPad 2Version 5.1Model MC773LL

Samsung Galaxy S PhoneAndroid 2.3.6Model Samsung-SGH-I897

Samsung Galaxy S3Android 4.0.4Version SGH-I747M

Samsung Galaxy TabletAndroid Version 4.04Model GT-P7500


Strangeloop makes pages up to 3.5X faster on mobile devices. Automatically.

Talk to one of our Performance Experts and learn how we can help you.

www.strangeloopnetworks.com/talk-to-a-performance-expert

Strangeloop Networks is the leader in providing next-generation web acceleration solutions. Companies like eBay/PayPal, Visa, and Petco rely on Strangeloop to speed up their websites and enterprise applications. As the earliest entrant in the front-end acceleration space, Strangeloop was the first company to market an automated solution – the Strangeloop Site Optimizer – with the ability to tackle front-end performance problems as a service via the cloud. The Strangeloop Mobile Optimizer is the only advanced optimization solution that addresses the unique challenges of delivering faster sites to mobile users. Strangeloop is based in Vancouver, BC. For more information, visit www.strangeloopnetworks.com.

About Strangeloop Networks

2012 state of mobile ecommerce performance

Documents