university of michigan electrical engineering and computer science anatomizing application...
TRANSCRIPT
University of MichiganElectrical Engineering and Computer Science
Anatomizing Application Performance Differences on Smartphones
Junxian Huang, Qiang Xu, Birjodh Tiwana, Z. Morley MaoUniversity of Michigan
Ming Zhang, Victor BahlMicrosoft Research
University of MichiganElectrical Engineering and Computer Science
2
Smartphones are popular Various smartphone platforms
Many applications
High speed network access 3G (UMTS & EVDO) 4G (LTE)
University of MichiganElectrical Engineering and Computer Science
3
Understanding application performance is important
AT&T
T-Mobile
Verizon
University of MichiganElectrical Engineering and Computer Science
4
User coverage vs. collected information
Design tradeoff in smartphone measurement
User coverage
More infoFewer users
Less infoMore users
Collected information
Speedtest.netFCC’s broadband test
Liu, MobiCom 2008
Zhong, HotMetrics 2010
Willkomm, DySPAN 2008
Our study
University of MichiganElectrical Engineering and Computer Science
5
Our contributions 3GTest
Widely deployed Provide new insights into cellular network performance
A systematic methodology to understand application performance Isolate individual factors Enable fair comparisons Identify performance bottlenecks Offer concrete recommendations for improvements
University of MichiganElectrical Engineering and Computer Science
6
Outline Cellular network performance study
3GTest design 3GTest results
Web performance analysis Experimental methodology & setup Web performance results
Streaming video & VoIP performance study Conclusion
University of MichiganElectrical Engineering and Computer Science
7
Key challenges of 3GTest
Adoptability User privacy Running time Ease of use Energy consumption
Data usefulness Metrics relevant to network applications
University of MichiganElectrical Engineering and Computer Science
8
How to make 3GTest easily adoptable? Avoid personal identifiable information
http://www.eecs.umich.edu/3gtest
University of MichiganElectrical Engineering and Computer Science
What information is most relevant for 3GTest?
9
SmartphoneLocal DNS server
Web server
example.com?
192.0.32.10
Get index.html
<html></html>
Ping to 1st hop
End-end ping
University of MichiganElectrical Engineering and Computer Science
10
Representativeness of 3GTest data66K+ users 148K+ runs 84 carriers 147 countries over 10 months
University of MichiganElectrical Engineering and Computer Science
11
Throughput comparison
CDF
T-Mobile:600
kbps
AT&T1Mbps
University of MichiganElectrical Engineering and Computer Science
12
RTT comparison
CDF
T-Mobile450ms
AT&T350ms
University of MichiganElectrical Engineering and Computer Science
13
Retransmission rate comparison
CDF
T-Mobile
AT&T
University of MichiganElectrical Engineering and Computer Science
14
Insights from 3GTest results Local DNS server performance is good
200 ms comparable to ping to 1st hop Carrier network delay dominates end-to-end RTT
End-end ping close to ping to 1st hop Time of day effect
AT&T’s downlink throughput Signal strength correlates with TCP performance
University of MichiganElectrical Engineering and Computer Science
15
Smartphone web browsing is complicated
Smartphone
Local DNS server
Web server
DNS lookup Content customization
TCP handshake TCP data transfer
1
2 4
Server configuration
Client configuration
HTML rendering & JavaScript execution
5Network condition
Server processing
delay3
University of MichiganElectrical Engineering and Computer Science
16
Key challenges of web performance analysis
Multiple factors have impact Network condition Client settings Server configurations Content differences
Lack of low-level information and access No visibility into link layer Difficult to instrument client OS and application No access to public web servers
University of MichiganElectrical Engineering and Computer Science
17
How to isolate impact of individual factor? Vary one factor at a time
Network condition Simulate cellular network with WiFi
Client settings Control TCP concurrency
Server configurations Change compression mode and server parallelism
Content difference Host static website snapshots locally
University of MichiganElectrical Engineering and Computer Science
18
How to identify performance bottlenecks?TCP flow analysis technique to break down page load time
DNS lookup time TCP handshake time TCP idle time TCP transfer time
index.htmllogo.png
Time
University of MichiganElectrical Engineering and Computer Science
19
Controlled experiment setup Platform
Web browsing
Other applications
University of MichiganElectrical Engineering and Computer Science
20
Simple website page load time break down
http://m.ebay.com
TCP idle timeTCP transfer time
University of MichiganElectrical Engineering and Computer Science
21
Content-rich website page load time break down
http://mapquest.com
TCP idle timeTCP transfer time
University of MichiganElectrical Engineering and Computer Science
22
JavaScript execution comparison
PC80X (CPU <10X)
University of MichiganElectrical Engineering and Computer Science
23
Lessons from application performance study Performance bottlenecks of web browsing
Client execution Data download
Higher concurrency significantly improves performance Compression is usually helpful Content customization is beneficial
Data URL scheme Minification Minimal redirections
Video & VoIP Platform specific application behaviors
University of MichiganElectrical Engineering and Computer Science
24
Conclusion 3GTest for measuring cellular network performance
Deployed widely Gain insights into cellular network performance
A systematic methodology to understand application performance Isolate impact of individual factor Identify performance bottlenecks Offer recommendations on performance improvement
University of MichiganElectrical Engineering and Computer Science
25
Visit 3GTest website
http://www.eecs.umich.edu/3gtest
AndroidiPhone Windows Mobile