WebPagetest Power UsersPatrick Meenanpmeenan@webpagetest.org@patmeenan

The Basics

● Presentation is available on slideshare○ http://www.slideshare.net/patrickmeenan○ Mostly contains links to everything I’ll be showing

● Video will be uploaded to YouTube○ https://www.youtube.com/user/patmeenan

● I’ll send links and announcements of both to Twitter and Google+○ https://twitter.com/patmeenan - @patmeenan○ https://plus.google.com/u/0/+PatrickMeenan/posts

What we’re covering1. Multiple Runs2. medianMetric3. Visual Comparisons4. Filmstrip URL Manipulation5. Video Comparison View6. Better Screen Shots7. Chrome timelines8. Mobile device vs emulation (data reduction)9. bodies/tcpdump/cloudshark

10. Chrome Tracing11. Images12. Keep UA String13. Resource blocking14. SPOF testing15. Filmstrip comparison debugging/overlays16. Scripting multi-step/authentication17. Scripting request manipulation (headers, DNS, command-line)18. User Timing19. Custom metrics/success criteria20. API21. CI - node wrapper, Travis, Jenkins, Grunt budget22. Bulk tests (url/host substitution)23. Results logging (splunk, logster, etc)24. Benchmarks

Using WebPagetest

The API

Private Instances

Plot Full Results

● Test with > 1 run● Click “Plot Full Results” link below data table

medianMetric

● Defaults to onload (“loadTime”)● Can be any metric by adding &medianMetric=<blah> to URL● Valid metric names are the identifiers returned in the API

Visual Comparisons

● Log in● Run Individual tests with the options you want

○ For best results, label each test run● Go to your Test History● Select the tests you want to compare● Click Compare

Filmstrip URL Manipulation

http://www.webpagetest.org/video/compare.php?tests=140609_KC_P6Z,140609_Z5_NDE

Filmstrip URL Manipulation

http://www.webpagetest.org/video/compare.php?tests=140609_KC_P6Z,140609_Z5_NDE

● Test ID’s for the tests that are being compared● In the order to be displayed● Comma-delimited● Selects the First View run with the median onload time

Each test can be configured

140609_KC_P6Z-l:First+View-r:3-c:0-e:7.3

● -l: (Label)● -r: (Run)● -c: (Cached? 0 = First View, 1 = Repeat View)● -e: (end time in seconds or category)

○ doc (onload)○ full (fully loaded)○ visual (visually complete - first time it gets to 100%)○ all (last visual change)

Video Comparison View

● Get the filmstrips configured the way you want● Click “Create Video”● Click “View as data comparison”

Better Screenshots

● PNG Screen Shots● Control JPEG compression level● Usually API-only, but also available through hidden parameters

pngss= 0/1, enables full-resolution PNG screen shots(not video frames)

iq= <jpeq quality level>, sets compression level for all JPEG images(screen shots and video frames)

http://www.webpagetest.org/?pngss=1&iq=75

Regular Resolution vs Improved

Chrome Timelines

● Advanced Settings -> Chrome -> Timeline○ Always enabled on Android with video capture○ Call stacks are optional (but helpful for debugging)

● Processing Breakdown link○ top of details page○ left of waterfall thumbnail

● Timeline CPU Time below waterfall including in filmstrip● Timeline links to the left of the waterfall (download or view)

CPU Impact - www.amazon.com

Nexus 5

Desktop VM

Moto G

Moto E

200ms ticks

Filmstrip Comparison

CPU Impact - www.google.com/search?q=flowers

Nexus 5

Desktop VM

Moto G

Moto E

200ms ticks

Nexus 7 (2013)

Filmstrip Comparison

Mobile Device vs Emulation - http://www.theguardian.com/uk?view=mobile

Nexus 5

Desktop VM

Moto G

Moto E

Nexus 7 (2013)

500ms ticks

Filmstrip Comparison

Data Reduction Proxy

● Android-specific● Only available with actual devices● Advanced Settings -> Chrome● Sample Test and Comparison

Moar Data!

● Response Bodies○ Desktop-only (or mobile emulation), all browsers○ Stores text resources served to test browser (HTML, CSS, JS)○ Available in “object” tab of waterfall dialog○ Also as download from request detail

● TCP Dump○ Desktop and Android Mobile○ pcap download left of waterfall○ Sharable through cloudshark (view link)

■ Filter “http”■ Find request■ Analysis tools->Follow TCP Stream■ Show only this stream

Chrome Tracing

● Filing a Chrome Performance bug (crbug.com)?● Debugging a performance issue and need more detail?

○ Miss the days of writing raw assembly? yeah, it’s like that● Provides tracing of internal browser activity● Downloadable or viewable directly in the browser

○ WASD to zoom/pan

Images

● View all images link below waterfall in detail view (main test)● Images displayed in the order they were requested● Look for:

○ visible images loading before slideshow/lower images○ things that should be in sprites○ Images with lots of savings○ PNGs that should be JPEGs○ Anything else “odd”○ Decide if you really need to deliver all of those images while loading

● JPEG Analysis view

Preserve Original UA String

● Advanced Settings -> “Preserve Original User Agent String”● Usually Appends PTST <version> to UA string

○ Identify/filter synthetic tests● Sometimes sites do weird things (sometimes on purpose)● AOL blocks ads for example

Resource Blocking - Desktop Only

● Good for what-if testing○ How does the site load without my javascript?○ How does it load without ads?

● Only works for external resources● Substring match● “Fails Fast” - returns an immediate error● Combine with visual comparison for great effect

SPOF Testing

● Good for testing impact of 3rd-party outages● Blackhole FQDN’s only (and entirely)

○ Coming soon, “SPOF all domains except…”● Goes directly to a Filmstrip comparison

○ Comparing Original vs SPOF● SPOF routes given domains to blackhole.webpagetest.org

○ blackhole.webpagetest.org is just a host with a firewall rule to drop all incoming packets

● Non-Responding behaves VERY differently from Fast-Fail● Would your monitoring detect it?

Test Comparison

● Waterfall overlay/slider● Comparison Graphs

Scripting - Multi-step

● Simplest case is multiple distinct pages● logData 0/1

○ logData 0○ navigate www.amazon.com○ logData 1○ navigate http://www.amazon.com/High-Performance-Web-Sites-Essential-ebook/dp/B0028N4WHY/ref=sr_1_2?ie=UTF8&qid=1402339606&sr=8-2&keywords=even+faster+websites

● Test with shared site resources● vs combineSteps

○ combineSteps○ navigate www.amazon.com○ navigate http://www.amazon.com/High-Performance-Web-Sites-Essential-ebook/dp/B0028N4WHY/ref=sr_1_2?ie=UTF8&qid=1402339606&sr=8-2&keywords=even+faster+websites

Scripting - Forms

● Common use is testing pages behind an auth curtain○ Check the “sensitive data” box to have cookies stripped

● Multiple ways to automate:○ Automate login form○ Re-use existing session cookie

● Script Structurenavigate <landing page>xxx (not measured)xxx (not measured)xxxAndWait (measured)

● WPT depends on network activity for end state

Scripting - Form Commands

● setValue● setInnerText● setInnerHtml● selectValue● click, sendClick● sendKeyDown, sendKeyUp, sendKeyPress● submitForm

● exec

exec(AndWait)

● Executes arbitrary javascript in the context of the page● Best to keep it to plain javascript (don’t depend on in-page code)● Debug scripts with local dev tools console before submitting

Script - Get the Weather in Santa Clara

logData 0navigate http://www.weather.com/exec document.getElementById("typeaheadBox").value = 95054logData 1execAndWaitdocument.getElementById("headerSearchForm").submit()

Result - Filmstrip

Scripting - Request Manipulation

● setDns, setDnsName, overrideHost● setCookie● setHeader, addHeader● block

Scripting - Misc

● setViewportSize● sleep

Full scripting docs:https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/scripting

Custom Browser Settings

● Chrome○ Custom command-line options (advanced settings)

■ Specify PAC scripts (proxy)■ Custom host override■ Disable SPDY■ Toggle features

● Firefox○ firefoxPref script command

■ Sets any firefox pref value before launching browser

User Timing

● performance.mark(‘label’)● Exposed as metrics in WebPagetest results

○ In raw data○ As purple vertical lines in waterfall○ Last mark exposed as a unique metric

● Trivial polyfill for older browsers● Validate RUM markers using video capture/filmstrip

Start Render From JS

Light green line in waterfall

Chrome:

var startTime = window.chrome.loadTimes()['requestTime'] ?window.chrome.loadTimes()['requestTime'] :window.chrome.loadTimes()['startLoadTime'];

var render = window.chrome.loadTimes().firstPaintTime - startTime;

IE:var render = window.performance.timing.msFirstPaint;

Custom Metrics

● Runs arbitrary javascript snippet(s) at the end of the test● Defined on a per-test basis or instance-wide

○ New HTTP Archive metrics (DOM depth, iFrame count, etc)● Can be simple one-liners or more complex multi-function

operations● Can override existing metrics● Can return numeric or string values● Lots of uses:

○ Validate expected page content○ Collect RUM beacon results○ Separate results based on A/B or type of ad served○ Count widgets, etc

Custom Metrics - Simple

[iframe-count]return document.getElementsByTagName("iframe").length;

Custom Metrics - Validate Content[result]var result = 499;try {

var copyright = document.getElementsByClassName('copy-text')[0].innerText.slice(-8);result = copyright == "AOL Inc." ? 0 : 498;

} catch(e) {}return result;

Changes test result to:● 499 if the copyright block doesn’t exist● 498 if the copyright block exists but the values don’t match● Leaves it as a success (0) if it exists and matches

API

https://sites.google.com/a/webpagetest.org/docs/advanced-features/webpagetest-restful-apis

1. Submit Test

2. Poll for test status

3. Get test result

API - Submitting a test

● API key required for public instance○ email me (pmeenan@webpagetest.org) for key○ keys are quite limited (~200 “page loads” per day)○ mostly for proof of concept, early dev before deploying private

instance

● Anything you can do with the UI you can do with the API○ If a field isn’t documented, just inspect the HTML on the UI to get the

field name :-)

● Make sure to encode parameters○ urlencode if GET○ form encode if POST

API Response

http://www.webpagetest.org/runtest.php?url=www.aol.com&f=json

{"statusCode":200,"statusText":"Ok","data":{

"testId":"140610_FY_N2F","ownerKey":"04121a1e5b17a59ba9ddd666ae020cf52cee4abe","jsonUrl":"http:\/\/www.webpagetest.org\/jsonResult.php?test=140610_FY_N2F","xmlUrl":"http:\/\/www.webpagetest.org\/xmlResult\/140610_FY_N2F\/","userUrl":"http:\/\/www.webpagetest.org\/result\/140610_FY_N2F\/","summaryCSV":"http:\/\/www.webpagetest.org\/result\/140610_FY_N2F\/page_data.csv","detailCSV":"http:\/\/www.webpagetest.org\/result\/140610_FY_N2F\/requests.csv"

}}

Polling status

● Periodically poll the JSON/XML URL● statusCode = 200 : Test Done

○ Test itself may still have failed● statusCode = 1xx : Test pending/waiting● statusCode = 4xx : Error locating the test

API test result

http://www.webpagetest.org/xmlResult.php?test=140610_FY_N2F&requests=1&breakdown=1

Bulk testing

● Submit a bunch of tests

● Periodically check status

● Do “something interesting” with the results

Bulk Testing

● PHP○ https://github.com/WPO-Foundation/webpagetest/tree/master/bulktest

● Python○ https://github.com/WPO-Foundation/webpagetest/tree/master/batchtool

● HTTP Archive○ http://httparchive.org/○ https://github.com/HTTPArchive/httparchive

Node API Wrapper

Created by Marcel Duran (@marcelduran)

https://github.com/marcelduran/webpagetest-api

$ npm install webpagetest -g

webpagetest test -k <key> http://twitter.com/marcelduran

webpagetest status 140610_F0_T5K

webpagetest results 140610_F0_T5K

webpagetest test http://twitter.com/marcelduran --poll 5 --timeout 60

CI Integration

https://github.com/marcelduran/webpagetest-api/wiki/Test-Specs

Match result to provided specs{ "median": { "firstView": { "requests": 20, "render": 400, "loadTime": 3000, "score_gzip": { "min": 90 } } }}

CI Integration cont.

Uses Mocha with reporters suitable for integration with:● Jenkins● Travis-CI● Drone.io● Just about anything else you can imagine

grunt-perfbudget

Created by Tim Kadlec (@tkadlec)

https://github.com/tkadlec/grunt-perfbudget

Bulk Testing in the UI (Private Instances)

● Submit list of URLs to test● UI options apply to each test● Scripts can be run against each URL

○ %URL% in script gets replaced with test URL○ %HOST% in script gets replaced with host from test URL○ %HOSTR% in script gets replaced with host from URL after redirects

● Aggregate results○ Download as CSV (subset of data)○ Compare tests against each other

Results logging (Private Instances)

● As each test comes in, full result data is written to log files○ Page Data (load time, Speed index, requests, custom metrics, etc)○ Request Data (details for every individual request)

● Each test run or request is on it’s own log file line● JSON-formatted● Config information is included

○ Test URL○ Run #○ First/Repeat View○ Test label○ Location○ Browser○ Connectivity○ Test ID○ URL to test result

Results logging - integration

● Splunk● Logster

○ Logster -> StatsD = Arbitrary metrics trending automatically● Track individual requests across all pages

○ ads JS performance○ CDN response times○ Effectiveness of image compression○ so much more

Benchmarks (Private Instances)

● Still pretty experimental● Recurring tests run automatically

○ Configuration is a bit ugly but flexible● Tests are scheduled, run and aggregated automatically

○ Top-level trended aggregate view○ All-metrics trended aggregate view○ Per-page trended view○ Scatter Plot for a given run○ Filmstrip comparisons

bit.ly/usingwebpagetest

Google Booth

Wednesday 6/25● Lightning Talks

○ 10:45 am - Meet4Speed, Pat Meenan○ 2:40 pm - The PageSpeed Optimization Platform - Joshua Marantz

● Meet the Speaker○ 2:40 pm - WebPagetest Q&A, Pat Meenan

Thursday, 6/26● Lightning Talks

○ 10:45 am The Case for Tunneling Sockets Over HTTP, Wenbo Zhu○ 2:40 pm - Optimize For Bandwidth - the what and why, Matt Atterbury

● Meet the Speaker○ 2:40 pm - Making the Web POSH - Q&A, Jan Maessen

Feedback Please!

http://velocityconf.com/velocity2014/public/content/mobile-app

Thank You!