"rum first" nyc webperf meetup
DESCRIPTION
RUM discussion at NYC Web Performance MeetupTRANSCRIPT
![Page 1: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/1.jpg)
RUM First
![Page 2: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/2.jpg)
Who I Am
Cliff Crocker
VP Product
SOASTA, Inc.
@cliffcrocker
![Page 3: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/3.jpg)
http://www.flickr.com/photos/luc/
Real Users
![Page 4: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/4.jpg)
http://www.flickr.com/photos/spullara/
Happy Users
![Page 5: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/5.jpg)
http://www.flickr.com/photos/sybrenstuvel/
Not SoHappy
![Page 6: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/6.jpg)
RUM 101Real User Measurement (RUM) is a technology for collecting performance metrics directly from the browser of an end user.
• Involves instrumentation of the page with JavaScript
• Measurements are fired across the network to a collection point through a small request object (beacon)
<JS>MySite.com
<beacon>
![Page 7: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/7.jpg)
This helped…
![Page 8: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/8.jpg)
C’mon Safari, be cool…
http://caniuse.com/nav-timing
69.34% Browser Support
![Page 9: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/9.jpg)
Why RUM is Awesome
• Reduces the level of convincing we have to do within our organization that ‘faster is better’
• The only way to measure perceived performance from your end user
• Easy to maintain
• Only getting better…– Check out: http://blog.patrickmeenan.com/2013/07/measuring-
performance-of-user-experience.html– User Timing for RUM and Synthetic
![Page 10: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/10.jpg)
What gives??
No RUM
RUM • According to a study done in 2012* only 1 in 5 of the top ecommerce sites are using some form of RUM
*http://www.webperformancetoday.com/2012/11/13/real-user-monitoring-rum-ecommerce-sites/
![Page 11: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/11.jpg)
Challenges with RUM
1. It’s a “data” problem
2. People don’t like change
3. Hard to “sell” internally
![Page 12: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/12.jpg)
For a modest sized site, in a given day
1-3 Million Page Views
Perf Timing
User Agent
OS/Device
Geography
Page Segmentation
Bandwidth Classification
A/B Bucket
‘Other’
![Page 13: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/13.jpg)
![Page 14: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/14.jpg)
Many have advocated sampling…
![Page 15: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/15.jpg)
1% sample, looking at a period of about 3 hours for a given page segment (Checkout)
Filtered to the United States…
![Page 16: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/16.jpg)
Using a sampling approach
Looking only at California…
…and IE 9 traffic only
![Page 17: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/17.jpg)
Now look at a 100% sample for the same time duration
All the same filters applied – AND looking specifically at ‘Back-End’ time
![Page 18: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/18.jpg)
Sampled at 1%
Not sampled
![Page 19: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/19.jpg)
Many use a batch solution…
![Page 20: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/20.jpg)
• Batch processing large amounts of data has been a traditional approach for RUM
– Great for looking at historical data, trending, etc.
– Big Data is not necessarily Fast Data
• Not ‘Operational’
![Page 21: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/21.jpg)
In order for RUM to be operationally relevant, you need the data in real-time
![Page 22: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/22.jpg)
Synthetic vs. RUM
http://www.flickr.com/photos/myvector/
![Page 23: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/23.jpg)
It is Not Either Or
• Synthetic and RUM coexist in any performance strategy
• Great use cases for both
![Page 24: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/24.jpg)
Synthetic Monitoring
• Pros
– Technology is great
– Real Browsers (IE, Chrome, FF)
– Built in Alerting
– Waterfall analysis
– Screen shots & headers
– Object level detail
• Cons
– False Positives
– Impossible/impractical to measure everything
– Fixed number of browser/OS combinations
– Simulated bandwidth constraints at best
– Too few data points for statistical relevance
![Page 25: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/25.jpg)
RUM
Cast a wide net• Identify key areas of concern• Understand real user impact• Tie this back to $$ (next)
Synthetic
Diagnostic tool• Identify issues in a ‘lab’• Reproduce a problem found with
RUM
http://www.flickr.com/photos/84338444@N00/with/3780079044/
http://www.flickr.com/photos/ezioman/
![Page 26: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/26.jpg)
RUM in the Workplace
![Page 27: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/27.jpg)
Studies show…
![Page 28: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/28.jpg)
However, you have to look at your own data
![Page 29: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/29.jpg)
Understand your KPIs and How to Track Them
• Retail: Order Conversion, Add to Cart Conversion, Revenue
• Media: In Line Display Ad Views, Comments
• Other: Facebook likes/Social Media sharing
![Page 30: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/30.jpg)
Bottom Line, if you want your business to care – then make it relevant for
them (speak their language)
![Page 31: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/31.jpg)
Strategy for Success – RUM First
RUM First
• All the data
• Real-time and historical
Supplement with
Synthetic
• Reduce measurements based on RUM
• Use for adhocdiagnostic
Map to business
KPIs
• YOUR goals
• YOUR data
![Page 32: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/32.jpg)
Free and Open Toolsets
RUM JS Collection Visualization Synthetic
boomerang.js Stats D Graphite WebPagtest
Episodes ShowSlow SpeedCurve (beta is free?)
SOASTA mPulse, Torbit, Google Analytics, Pingdom, New Relic
![Page 33: "Rum First" NYC Webperf Meetup](https://reader035.vdocuments.net/reader035/viewer/2022070322/559433fb1a28aba11a8b463c/html5/thumbnails/33.jpg)
mPulse Litehttp://www.soasta.com/free/