boomerang at foss.in/2010
TRANSCRIPT
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
Measuring the web with boomerang
Philip Tellis / [email protected]
FOSS.IN/2010 – 2010.12.15
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
$ finger philip
Philip [email protected]
@bluesmoongeek - paranoid - speedfreakyahoohttp://bluesmoon.info/
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Less than 20% of page load time is something we can measureand fix during development
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
It’s what we can’t control that bites us
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
browsers
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
plugins
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
OSes
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
viruses
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
antiviruses
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
microwaves
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
baby monitors
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
naughty neighbours
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
file shares
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
governments
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
rodents
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
Too many variations
Try simulating all that in the lab!
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
We need to measure real end-user performance
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
We need to measure real end-user performance from the realend-user’s box
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
While this might work, it isn’t necessarily representative
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
What about JavaScript?
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
boomerang is...
A piece of javascript that you add to your web page where itmeasures and beacons back to you, the end user’s perceivedperformance of your page
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
How?
<script src="boomerang.js" type="text/javascript"></script><script type="text/javascript">BOOMR.init({
user_ip: "<user’s ip address>",beacon_url: "http://mysite.com/beacon.php"
});</script>
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
TimeThe adversaryMeasure twiceboomerang
What does it do?
Measures user’s network throughput and latency to yourserverMeasures the current page’s load timeBeacons these results back to your server
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
LatencyBandwidth/ThroughputLoad timeAccuracy
How does boomerang work?
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
LatencyBandwidth/ThroughputLoad timeAccuracy
Let’s take that one at a time
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
LatencyBandwidth/ThroughputLoad timeAccuracy
Measuring latency
Download a 32 byte gif 10 times in sequenceMeasure the time to download eachDiscard the first measurement because it’s overpricedCalculate the arithmetic mean, standard deviation andmargin of error of the rest
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
LatencyBandwidth/ThroughputLoad timeAccuracy
Wait, did you say overpriced?
The first image might require a DNS lookup and TCPhandshakeSlow start is not an issue since 32 bytes fit in 1 packet
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
LatencyBandwidth/ThroughputLoad timeAccuracy
Measuring throughput
After the latency test is done, we download progressivelylarger imagesStop at the first image that times outRedownload that image a few more timesCalculate the median, standard deviation and margin oferror of the largest images
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
LatencyBandwidth/ThroughputLoad timeAccuracy
Measuring latency before throughput helps here
Those 10 latency images do a lot to widen the TCPwindow sizeThe bandwidth images make much better use of availablebandwidthThe image we end with uses the most bandwidth
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
LatencyBandwidth/ThroughputLoad timeAccuracy
How do we measure page load time?
In the onbeforeunload event, measure the time andstore it in a cookieIn the onload event, check the cookie, and measure thedifference with the current timeWe also make sure that the page that set the cookie is thereferrer of the current page
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
LatencyBandwidth/ThroughputLoad timeAccuracy
What? Two pages?
Yes, this needs two pages and cookies. If those aren’tsupported, we try to use the WebTiming API.
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
LatencyBandwidth/ThroughputLoad timeAccuracy
How accurate is it?
Latency measurements are very accurate (±1%)Bandwidth is to an order of magnitude. For badconnections can be ±30%
Page load time sometimes has outliers, you needpost-filteringThe margin of error tells you how good your data is
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
Github
https://github.com/yahoo/boomerang/
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
Get the code
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
Get the code
git [email protected]:yourname/boomerang.git
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
Submit code
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
boomerang.js
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
plugins
cp plugin.js yourplugin.js
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
The Makefile
make PLUGINS="plugin list" MINIFIER="/minifier/path"
make usage
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
Include it on your page
<script src="boomerang.js" type="text/javascript"></script><script type="text/javascript">BOOMR.init({
user_ip: "<user’s ip address>",beacon_url: "http://mysite.com/beacon.php"
});</script>
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
For most sites, that’s about it
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
You probably want to do more
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
Handle subdomains
Set the site_domain parameter:
BOOMR.init({user_ip: "<user’s ip address>",beacon_url: "http://mysite.com/beacon.php",site_domain: "mysite.com"
});
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
This lets you measure transitions across *.mysite.com
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
Measure more than just load time
<html><head><script>var t_pagestart=new Date().getTime();</script>...<script>var th=new Date().getTime();</script></head><body>...<script>var tj=new Date().getTime();</script>...
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
...<script src="boomerang.js"></script>...var te=new Date().getTime();BOOMR.plugins.RT.setTimer("t_head", th-t_pagestart).
setTimer("t_body", te-th).setTimer("t_js", te-tj);
</script></body></html>
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
This adds the t_head, t_body and t_js fields to the beacon
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
Loading dynamic content
BOOMR.init({user_ip: "<user’s ip address>",beacon_url: "http://mysite.com/beacon.php",auto_run: false
});
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
Loading dynamic content
// Just before download startsBOOMR.plugins.RT.startTimer("t_done");
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
Loading dynamic content
// Just after download finishesBOOMR.plugins.RT.done();
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
BasicMulti-domainIn-page timersAJAX
Much more
http://yahoo.github.com/boomerang/doc/howtos/
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
shouldacouldawoulda
The beacon
GET request to the beacon URL (response ignored)All parameters passed in the query stringExtra timers are passed in as a comma separated list int_other
before_beacon JavaScript event fired just before thebeacon is sent
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
shouldacouldawoulda
What should we do with the data?
Sanity checking to:Remove fake dataRemove abusive dataMaybe just rate limiting
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
shouldacouldawoulda
What can we do with the data?
Statistical analysis to:Remove outliersAggregate based on bandwidth blocksMeasure trends over time and correlate them with codechanges
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
shouldacouldawoulda
Bandwidth blocks
Ref: Analysing Bandwidth & Latency – YUI Blog
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
shouldacouldawoulda
Bandwidth blocks
Data points from some countries may require narrower bands
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
shouldacouldawoulda
Geographic data
Looking at latency from different geographic locations can tellyou where your next mirror should be
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
shouldacouldawoulda
ISPs
Grouping data by ISP can tell you who’s behaving badly
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
shouldacouldawoulda
More data
Write plugins to get more performance dataWe already have a DNS pluginI’m thinking of an IPv6 v/s IPv4 pluginWhat about a full WebTiming plugin?Can we measure connection setup time?
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
shouldacouldawoulda
You decide
Once you have the data, you can do anything with it
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
Thank youhttp://github.com/yahoo/boomerang
http://yahoo.github.com/boomerang/doc/
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
Photo credits
flickr.com/photos/21233184@N02/4389412851
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
Contact me
Philip [email protected]
@bluesmoongeek - paranoid - speedfreakyahoohttp://bluesmoon.info/slideshare.net/bluesmoon
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
IntroductionHow does it work?
Contributing to boomerangUsing boomerang
Data
References
github.com/yahoo/boomerangMore bandwidth doesn’t matter (much) – Mike BelsheAnalysing Bandwidth & Latency – YUI BlogIt’s the latency, stupid – Stuart CheshireThe statistics of web performance
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang