monitor – analyse – optimisevoice lte 9/oct/2013 v1.0 limited drive tests in rotterdam (no...
TRANSCRIPT
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE
Benchmarking measurements & technical analysis
Dominique Dubois, ElectroRent Stefan Engels, Commsquare
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Commsquare Mission & facts
2
Commsquare helps mobile
operators excel in terms of the
quality of service they deliver
to their customers.
“Network doctors”
Technology excellence
Focus on new technologies
End-to-end view
Independent partner
Founded in 2002
HQ in Belgium
Offices in GR & UK
40 engineers & consultants
30 customers, in BE, NL, DE, LUX, CH, GR, ROM, BUL, TR, etc.
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Commsquare Services & Products
3
Hands-on
training
Benchmarking
Services Products
DAP (data robot)
DataMon (Gn/S11 probe)
Competence areas:
RAT system
End-to-end
SW & IT
Benchmark approach:
Advanced
consultancy
DAP = active monitoring, test robot for data tests DataMon = passive monitoring system Both products are developed by Commsquare Both products support 2G/3G/LTE
In-depth technical analysis
Standard reporting
Measurements
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Introduction
4
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Disclaimer
5
The results in the presentation should not be generalised; the results are just a snapshot (“moment-opname”).
Hence, based on these measurements one can’t claim “operator X has the best LTE network”.
The results presented are a snapshot based on:
Three networks in roll-out undergoing fast changes
A very limited geographical area
A short measurement duration (2 days)
The main goal of this presentation is to demonstrate LTE measurement and analysis capabilities.
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Measurement setup
6
Drive test in Rotterdam & roads
26 & 27 Sep 2013
SwissQual Diversity
Huawei E392 & Win7 (data)
Samsung S3 (voice)
PcTel scanner
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE coverage
7
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE coverage
Total coverage (800+1800MHz)
8
KPN coverage extended, contiguous & indoor
VOD coverage gaps: early-stage roll-out
Deep Indoor: RSRP ≥ -90 dBm Indoor: -90 dBm > RSRP ≥ -100 dBm Coverage: -100 dBm > RSRP ≥ -116 dBm No Coverage: RSRP < -116 dBm
VOD KPN
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE coverage
Coverage strategy (KPN)
9
KPN focus on roll-out in 800MHz band
Very limited 1800 coverage
KPN 1800MHz
KPN 800MHz
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE coverage
Coverage strategy (VOD)
10
800Mhz band: 71 cells
VOD 800MHz
VOD 1800MHz
1800Mhz band: 102 cells
More cells, worse coverage than 800-band
Both 800 and 1800-band roll-out in Rotterdam
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
2G/3G/LTE data
11
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
This is a snapshot for networks in fast change
Limited to Rotterdam & roads during only 2 days
Results are not network-wide relevant
2G/3G/LTE data
Summary KPIs
12
Data Summary KPN T-Mobile Vodafone
FTP download (Mbps) average 20.3 3.9 11.7
FTP upload (Mbps) average 13.8 1.3 7.3
Ping 40 bytes (ms) median 31 79 49
Web Kepler (s) median 1.67 5.31 1.85
Kepler success rate 98.63% 97.60% 96.60%
Dominant Technology LTE HSPA LTE
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
KPN: 100% use of LTE
T-Mobile: HSPA (14Mbps) is dominant technology
VOD: 82% LTE usage
2G/3G/LTE data
Technology breakdown
13
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE data
14
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
This is a snapshot for 2 networks in fast change
Limited to Rotterdam & roads during only 2 days
Results are not network-wide relevant
LTE data
Summary KPIs
15
Data Summary KPN NL Vodafone NL
FTP download (Mbps) average 20.3 13.0
FTP upload (Mbps) average 13.8 8.6
Ping 40 bytes (ms) median 31 46
Web Kepler (s) median 1.67 1.79
Kepler success rate 98.63% 98.20%
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
KPN LTE800 service dominant
LTE1800 only in Delft
VOD LTE1800 service in Rotterdam although LTE800 coverage better
Still UMTS2100 on roads
LTE data
Technology & band usage
16
VOD KPN
Each dot on the map represents an FTP download or upload test
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
KPN: 100% use of LTE
Mainly 10MHz BW LTE800
VOD: 82% LTE usage
Mainly 15MHz BW LTE1800
LTE data
Technology breakdown
17
KPN
97%
3%
26%
74%
0%
20%
40%
60%
80%
100%
10MHz 20MHz 10Mhz 15MHz
800 1800 800 1800
KPN NL Vodafone NL
Band and Bandwidth
VOD
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
VOD 38% below 10Mbps
KPN only 7% below 10Mbps
LTE data
Download performance
18
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
VOD: typical case download <10Mbps Ample resource blocks (50-70)
IP throughput flat below 10Mpbs
Backhaul limitation? Resolving backhaul capacity will improve user experience
LTE data
Download investigation (VOD)
19
Vodafone NL – 7.8 Mbps (avg)
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
VOD 38% below 5Mbps
KPN only 5% below 5Mbps
LTE data
Upload performance
20
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE data
Web browsing
21
1.67s 1.79s
Median page download time - Kepler
KPN
95% in 2s
VOD
80% in 2s
95% in 2.5s
VOD: all browsing sessions >2s have download <10Mpbs
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE data
Ping performance
22
Ping tested with data stick on Win-7 computer
(SwissQual test)
WARNING: ping is not the best delay test method
TCP SYN-ACK is
VOD has wide distribution
31ms 46ms
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
KPN More security but public IP address
VOD Less security but private IP address
LTE data
Security
23
KPN NL Vodafone NL
Authentication Yes No
AS Security Yes Yes
NAS Security Yes No
PDN Address Public, IPv4 Private, IPv4
Max bit-rate for downl (Mbps) 156 102
Max bit-rate for upl (Mbps) 78 51
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
The data results mainly demonstrate 2 networks in a different stage in their roll-out (in the area tested)
KPN network in a more mature stage (in tested area)
More & contiguous coverage (based on LTE800)
Better throughput & web browsing experience
VOD at early commercial stage (in tested area)
Improvement possible in coverage & parameter tuning
Some limitations observed (backhaul?)
Bandwidth of 15MHz (LTE1800) has higher potential
LTE data
Conclusion
24
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE data
Benelux benchmark
25
Data Summary KPN NL Vodafone NL Benelux
FTP download (Mbps) average 20.3 13.0 21.6
FTP upload (Mbps) average 13.8 8.6 12.5
Ping 40 bytes (ms) median 31 46 20
Web Kepler (s) median 1.67 1.76 1.36
Kepler success rate 98.63% 98.20% 99.50%
Comparison with good & more mature network in Benelux, also LTE1800
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE voice
26
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Limited drive tests in Rotterdam (no roads) During 1.5 hours on 27/Sep
100% LTE coverage for both KPN & VOD
Test setup SwissQual Diversity rel 13, Samsung Galaxy S3
Only calls from mobile to fixed line (“speaking clock”)
Test script Call setup (dial)
Disconnect (stop) call after 5 sec
Wait 30s before start of next call
LTE voice
Test setup
27
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
CS-fall-back implementation
During call setup, mobile is pushed to 3G or 2G network
At call end, mobile returns to LTE
Redirection to 3G maintains data; redirection to 2G stops data
LTE voice
CS-fall-back & KPIs
28
UE on 2G or 3G
Call setup time Return to LTE
Idle / UE on LTE Idle / UE on LTE
UE in call
User dial
Ringing tone
End of call
CS-
Fall-
Bac
k
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE voice
Summary
29
CS-FB Summary KPN NL Vodafone NL
Call setup time (s) average 5.63 4.28
CSFB setup success average 100% 100%
Time to return to LTE (s) average 7.7 0.8
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Excellent call setup Both networks achieve 100% success for roughly 150 tests
Most calls set-up on LTE 97% (KPN) and 95% (VOD) respectively
LTE voice
Call setup success rate
30
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Different strategy KPN: fall-back to 3G (UTRAN)
VOD: fall-back to 2G (GERAN) And 1 call to 3G (UTRAN)
LTE voice
CS-FB strategy
31
kj
d
L
VOD
VOD
KPN
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE voice
Call setup time
32
VOD average setup time 4.28s
KPN average setup time 5.68s
Different strategy
KPN redirects to 1st carrier on 3G
VOD redirects to explicit list of ARFCNs on 2G (=a few cells)
VOD
KPN
VOD 4.28s
KPN 5.68s
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE voice
Time to return to LTE
33
KPN: typical performance
Majority returns in 7s
90% returns within 10s
Technical: KPN mobile returns to idle mode in UTRAN; UE then returns to LTE
VOD: very fast return
Fastest in Benelux
99% returns in 1s
Technical: VOD includes redirection info to LTE in the GERAN release message
VOD
KPN
If KPN implements same feature, then it can possibly achieve same
performance
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE voice
Conclusions
34
Call setup on LTE is excellent on both networks
KPN has slower setup & return times Fall-back to 3G network
Data sessions can be maintained during voice call
Faster setup & return possible when introducing features
VOD has fastest call setup time & return to LTE Fall-back to 2G
Fast thanks to features on 2G
At the expense of breaking data sessions during voice call
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
LTE voice
Benelux benchmark
35
Comparison with more mature network in Benelux,
also LTE1800
CS-FB Summary KPN NL Vodafone NL Benelux
Call setup time (s) average 5.63 4.28 6-8
CSFB setup success average 100% 100% ~98%
Time to return to LTE (s) average 7.7 0.8 8-10
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Conclusion
36
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Overall conclusion
37
We have Demonstrated typical LTE benchmarking measurements
Scratched the surface of technical analyses
REMINDER: results are a snapshot and shouldn’t be generalised
LTE networks in Rotterdam and surroundings are in fast roll-out & change:
Data performance is close to international benchmarks for KPN but not for VOD
Voice performance is very good for both, and in particular good for VOD
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE
9/Oct/2013 v1.0
Contact details
Dominique Dubois Stefan Engels (presenter)
[email protected] [email protected]
www.electrorent-europe.com www.commsquare.com
38