3 g benchmarking report jan 2012

63
Udaipur 3G Benchmark Report February 2012 PhiMetrics Technologies Pvt. Ltd.

Upload: emerson-eduardo-rodrigues-pmp

Post on 07-Aug-2015

61 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: 3 g benchmarking report jan 2012

Udaipur 3G Benchmark Report

February 2012

PhiMetrics Technologies Pvt. Ltd.

Page 2: 3 g benchmarking report jan 2012

Section 1

• Benchmarking Equipment and setup

• Data Collection Details

• Voice Testing Algorithm

• Data Testing Algorithm

• Strategic Location Testing

• Benchmarking KPIs: Score Evolution

• Voice & Data KPI weightages

PhiMetrics Benchmarking Methodology

Page 3: 3 g benchmarking report jan 2012

Executive Summary – Udaipur

Udaipur Details

Population 0.6 Million

Area 37 km²

Population Density 16,457 per km²

Udaipur

OperatorsTech

Band of

Operation (MHz)

No. of

3G Cells*

No. of 2G

Cells*Subscribers

Airtel 3G+2G 2100/900 142 20 12.9 Million

Reliance 3G+2G 2100/1800 32 187 5.25 Million

BSNL 3G+2G 2100/900 119 99 5.28 Million

Reliance HSD EVDO 850 15 2.3 Million

Subscriber figures are for Rajasthan Circle taken from the latest TRAI report.

*No of Cells is an indicative no the cells encountered during the Voice drive

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

3

Udaipur , also known as the City of Lakes, is a city, a Municipal Council and the administrative headquarters of the Udaipur district in the state of Rajasthan in western India.

The city lies on the Golden Quadrilateral National Highway (NH) 8, midway between Delhi and Mumbai, located around 700 km from either city.

Demographics of Udaipur

Page 4: 3 g benchmarking report jan 2012

Benchmarking Equipment and Setup

• Nemo Invex is an evolutionary chassis-based benchmarking solution which combines powerful intuitive softwareand scalable military grade hardware design to create superior drive-test systems for wireless networks.

• It is built upon a modular platform for Quality of Service (QoS) wireless network drive testing, RF optimization andcompetitive network benchmarking.

• The system uses specially configured commercial phones attached to controlling devices in isolation chambers toperform various tests such as voice calls, data calls, video calls etc.

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

4

Page 5: 3 g benchmarking report jan 2012

A benchmarking activity, was carried out in Udaipur for over 250Kms across the city. The details of the activity are:

– Total route covered in KMs : approx 250 Kms

– Measurement Period : 30th Jan 2012 to 2nd Feb 2012

– Measurement time during the day : 8 AM to 9 PM

* Except congested area

– Measurement days during the week : Monday to Saturday

– The drive route was created to cover the dense population area , key commercial area, key residential area. The city was divided into 4 clusters/areas. The route consisted of Highways, Main Roads, Other roads and Streets in the composition mentioned below.

• Highway - 20%

• Main Road - 40%

• Other Road - 20%

• Streets - 20%

Data Collection Details

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

4Cluster 1:Area Covered: Pichola, Kalaji Goraji

Cluster 2:Area Covered: Hiran Magari, Vardhman Nagar, Jawahar Nagar, Railway Colony

Cluster 3:Area Covered: Panchwati, Pulan, Sobhagpura, Ahinsapuri

Cluster 4:Area Covered: NH - 76

Page 6: 3 g benchmarking report jan 2012

Test Details Specifications

No of Operators tested Dual Mode (Airtel, Reliance, BSNL)

Voice Test Algorithm

Call Duration 3 mins (180 Secs)

Idle time in between calls 10 Secs

Continuous call for 3 minute duration with 10 seconds ofIdle time. Speech samples (DL and UL) are sent andreceived which provide the MOS score for voice quality.This was repeated for the entire duration of collection

Voice Testing Algorithm

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

6

Page 7: 3 g benchmarking report jan 2012

Test Details Specifications

No of Operators testedDual Mode (Airtel, Reliance,BSNL)

EvDo Network (Reliance)

Data Test Algorithm

Attach and PDP Activation Before every test

Ping 3 times - 32 bytes on www.google.co.in

FTP Download 10 MB (for EvDo) and 15 MB (for Dual Mode) from server

FTP Upload 1 MB (for EvDo) and 1 MB ( for Dual Mode ) to Server

HTTP 10 different websites.

Email send 100 KB Email to SMTP server

Email Receive 100 KB Email from POP3 server

No HTTP Websites

1 www.google.com

2 www.facebook.com

3 www.nimbuzz.com

4 www.airtel.in

5 www.cricinfo.com

6 www.vuclip.com

7 www.youtube.com

8 www.admob.com

9 www.zedge.net

10 www.nokia.com

Data Testing Algorithm

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

7

• After every application test, we have 2 sec Idle time before the next Application is tested.

Page 8: 3 g benchmarking report jan 2012

Strategic Locations Specifications

No of Operators testedDual Mode (Airtel, Reliance, BSNL)

EvDo Network (Reliance)

Data Test Algorithm

WAP Test 10 different WAP sites

FTP Download 10 MB (for EvDo) and 15MB (for Dual Mode) from Server

Email send 100 KB Email to SMTP server

Email Receive 100 KB Email from POP3 server

SMS SMS send to same number (160 char)

MMS MMS send to same number 15 Kb

WAP sites

wap.google.co.in

m.facebook.com

m.orkut.com

wapin.com

m.cricinfo.com

m.vuclip.com

m.youtube.com

m.admob.com

m.zedge.net

m.sonyericsson.com

Strategic Location Testing

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

8

Apart from the above tests during the drive, at 5 strategic locations across the city, additional stationary tests were carried out. These locations are important locations of the city.

1. Airport 4. Suraj Pol

2. Bus Stand 5. Trident Hotel

3. Chetak Circle

Page 9: 3 g benchmarking report jan 2012

Each KPI is then measured on a scale of 0 to 200 on the basis of performance.

Value of the Particular KPI is not reported default score is 100.

The Score is then weighted with the respective weightages to arrive at an overallVoice or a Data Score.

Benchmarking KPIs: Score Evolution

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

9

Page 10: 3 g benchmarking report jan 2012

5 KPI’s are used to arrive at the Voice Quality Score and 15 KPI’s are used to arrive at the Data Quality ScoreFollowing are the details of weightings assigned to different KPIs. These are used for arriving at the final rankingof the operator.

10

Voice & Data Quality Score

*Coverage Calculation:Coverage = A%*10+B%*8+C%*6+D%*1

10

GRADE RSCP IN dBm

A 0 to -74

B -74 to -82

C -82 to -90

D -90 to -98

Page 11: 3 g benchmarking report jan 2012

Section 2

• Overall Voice Quality Score

• Overall Data Quality Score

• SWOT Analysis for Airtel

• Voice Quality Score

• Data Quality Score

Management Summary

Page 12: 3 g benchmarking report jan 2012

MOS 4.06

3.77

3.71

3.66

3.06

Voice Quality Score

12

Call setup time, Accessibility,

Retainability,MOS, Coverage

192

115

88

62

0 9.82

4.68

3.86

2.54

99.35%

98.78%

95.85%

68.00%

100%

3G Coverage 100%

97.00%

85.00%

65.00%

0:00%

VQI CST 2.1

2

1

3

2

1

2

1

CSSR

99.64%

99.05%

96.16%

77.26%

100%

2

13

CCR

2

1

3

3

3

2

1

3

CST= Call Setup Time CSSR= Call Setup Success Rate CCR= Call Completion Rate MOS= Mean Opinion Score

VQI Scoring VCQI CST CSSR CCSR MOS Coverage

Airtel 3G 142.05 162.00 90.1 148.7 167.1 130.9

Reliance 3G 23.81 121.20 7 34.3 0 0

BSNL 3G 72.33 115.50 32.2 0 155.7 76.3

1 Airtel 3G

2 Reliance 3G

3 BSNL 3G

Page 13: 3 g benchmarking report jan 2012

13

Data Quality Score

HTTP

200

140

100

56

0

FTP DL

200

123

87

62

0

FTP UL

200

108

80

62

9

Email

170

147

100

41

0

DCQI

163

107

81

61

13 2

1

3

2

1

3

1

3

3

2

1

3

Data Attach

200

159

100

58

02

13

PDP Context

200

133

100

60

0

2

1

3

PING

0

200

118

100

53

2

1

3

21

2

4

4

4

4

4

4 4

4

1 Airtel 3G

2 Reliance 3G

3 BSNL 3G

4 Reliance EvDo

DQI Scoring DCQI HTTP FTP DL FTP UL EmailData

Attach PDP

ContextPing RTT

Airtel 3G 78.28 8.2 92.64 178.88 89.97 65.12 163.32 54.1

Reliance 3G 48.54 0 79.62 74.82 83.61 0 181.08 0

BSNL 3G 7.25 0 0 0 22.35 77.08 23.24 0

Reliance EvDo 62.29 0 65.28 103.6 58.47 100 100 145.8

Page 14: 3 g benchmarking report jan 2012

Section 3

• Voice Performance Analysis

• Coverage Distribution

• Call Setup Time

• Call Setup Success Rate– Call Setup Failure Analysis

• Call Completion Success Rate– Call Completion Failure Analysis

• Accessibility V/s Retainability: Market Comparison

• Voice Quality: MOS– Airtel MOS Performance

– Airtel EcIo, BLER

– Pilot Pollution Analysis

• Handover Analysis

Detailed Voice Analysis

Page 15: 3 g benchmarking report jan 2012

Call Setup Time:

Airtel has the least call setup time of 2.77 sec.

Call Setup Success Rate:

Airtel has 98.77% Accessibility with just 2 block calls in the entire drive test.

Call Completion Success Rate:

Airtel has 1 drop call and has 99.38% Retainability in the city.

Voice Quality:

Airtel has best speech quality as compared to other operators.

Ranking for Voice KPIs

Voice Performance Analysis

Parameter Best Value Operator

Call Setup Time 2.77 sec Airtel

Accessibility 98.77% Airtel

Retainability 99.38% Airtel

MOS 3.95 Airtel

Coverage 89.64% Airtel

Best Operator Airtel 3G

This report is generated in Phimetrics Proprietary format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

15

Operator Coverage Accessibility Retainability Call Setup Time Voice Quality

Airtel 3G 1 1 1 1 1

Reliance 3G 3 3 2 2 3

BSNL 3G 2 2 3 3 2

Page 16: 3 g benchmarking report jan 2012

Coverage Distribution

Coverage was measured with the help of a digital scanner/phone for all the networks.

Operator % of Samples > -82Coverage Rate using

formulaRank

Airtel 3G 88.32% 89.64% 1

Reliance 3G 49.70% 60.95% 3

BSNL 3G 73.50% 80.27% 2

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

16

• Airtel has good 3G coverage footprint in thecity as compared other operators.

• Reliance coverage was found to benoncontiguous which might be due totemporary failure.

Page 17: 3 g benchmarking report jan 2012

Coverage Plot

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

17

• There were some poor coverage patches(highlighted areas) in the city. Cluster 3 has lowercoverage which includes industrial areas and Udaipur bypass.

Page 18: 3 g benchmarking report jan 2012

18This report is generated in Phimetrics Proprietary format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

Radio Technology Distribution

• Airtel has almost all its voice traffic on 3G, which has resulted in better voice qualitycompared to other operators.

Page 19: 3 g benchmarking report jan 2012

• Airtel has the best call setup time compared toother 3G operators.

Call Setup Time (CST)

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

19

Page 20: 3 g benchmarking report jan 2012

Operator Call Attempts Blocked Calls CSSR Block Rate

Airtel 3G 162 2 98.77% 1.23%

Reliance 3G 165 6 96.36% 3.64%

BSNL 3G 172 5 97.09% 2.91%

Call Setup Success Rate Airtel 3G Reliance 3G BSNL 3G

Total Calls Attempts 162 165 172

Technology Blocked Call Reasons (Radio Cause)

2G SDCCH Drop 1 1 1

2G High RxQual 1

2G Bad Downlink Coverage 2

2G TCH Assignment failure 1

3G RRC Connection failed 2

3G Bad Downlink Coverage 1

3G Missing Neighbour 1

3G High BLER 1

3G Poor EcIo 1

Total 2 6 5

CSSR 98.77% 96.36% 97.09%

Call Setup Success Rate is a measure of subscriber experience associated with the access to network resource. Itconsists of congestion of network or unavailability of network resources.

Call Setup Success Rate

This report is generated in Phimetrics Proprietary format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves he right to use this format for its other customers if required.20

• Airtel has 98.77% Accessibility with2 fail calls.

Page 21: 3 g benchmarking report jan 2012

Call Completion Success Rate is a measure of subscriber experience associated with the continued use ofnetwork resources that have been granted. It consists of radio link drops or abnormal network-initiated releases.

Call Completion Success Rate Airtel 3G Reliance 3G BSNL 3G

Total Calls Established 162 165 172

Technology Dropped Call Reasons

2G DL Interference 2

2G Bad Downlink Coverage 3

3G Missing Neighbour 1

3G Missing Handover 1

3G Low EcIo 1

3G RRC Release received 20

Total 1 5 22

CCSR 99.38% 96.86% 86.83%

Call Completion Success Rate

• Airtel has 99.38% call completionrate with 1 drop call.

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

21

OperatorTotal Call

EstablishedDropped Calls CCSR Drop Rate

Airtel 3G 162 1 99.38% 0.63%

Reliance 3G 165 5 96.86% 3.14%

BSNL 3G 172 22 86.83% 13.17%

Page 22: 3 g benchmarking report jan 2012

Call Setup Failure & Drop Plot

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

22

Fail Call Drop Call

Page 23: 3 g benchmarking report jan 2012

Accessibility V/s Retainability: Market Comparison

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

23

Page 24: 3 g benchmarking report jan 2012

Voice Quality: UL & DL MOS

• Airtel has better DL and UL speech quality amongst all operators.

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

24

Page 25: 3 g benchmarking report jan 2012

Plot DL MOS

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

25

DL MOS

Page 26: 3 g benchmarking report jan 2012

Plot EcNo/ BLER

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

26

Ec/No BLER

Page 27: 3 g benchmarking report jan 2012

Pilot Pollution

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

27

• A scan was performed to detect all PSCs that were within 10dB Max CPICH EcNo

• It was found that there were a lot of instances where up to 7 PSCs were found in certain locations. These could cause substantial Interference. Due to Fast Fading, the instantaneous EcIo could go below the threshold and cause a call to be dropped

• Some of the areas in cluster 1 and 2 have more number of polluting cells.

Airtel

Page 28: 3 g benchmarking report jan 2012

Handover Performance

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

2G handovers

Inter cell Handovers Airtel 3G Reliance 3G BSNL 3G

HO Attempts 25 362 118

HO Success 24 358 109

HO failure 1 4 9

Success Rate 96.00% 98.90% 92.37%

Avg HO Duration (s) 0.116 0.126 0.161

Intracell Handovers Airtel 3G Reliance 3G BSNL 3G

HO Attempts 9 1 19

HO Success 9 1 19

HO failure 0 0 0

Success Rate 100.00% 100.00% 0.00%

Avg HO Duration (s) 0.042 0.088 0.071

3G-3G Handovers

Soft Handovers Airtel 3G Reliance 3G BSNL 3G

HO Attempts 5246 270 2651

HO Success 5246 270 2651

HO failure 0 0 0

Success Rate 100.0% 100.0% 100.0%

IRAT Handovers

UTRAN Handovers Airtel 3G Reliance 3G BSNL 3G

HO Attempts 4 5 71

HO Success 4 5 70

HO failure 0 0 1

Success Rate 100.0% 100.0% 98.6%

Page 29: 3 g benchmarking report jan 2012

Section 3

• Data Performance Analysis

• FTP Performance– FTP Application Throughput Analysis

• HTTP Browsing Analysis– HTTP Performance (Airtel)

• FTP DL/UL and HTTP Failures

• Email Performance

• GPRS Attach and PDP performance

• Data Access Technology Distribution

• Ping Response: Latency Performance

• SMS Analysis

• MMS Analysis

• KPI Overview Table

Detailed Data Benchmarking

Page 30: 3 g benchmarking report jan 2012

Ranking for Data KPIs

• Airtel have good FTP performance with bestdownlink and uplink throughput.

• HTTP browsing performance in terms of browsingsuccess rate.

• Least GPRS attach delay.

• Airtel has best Email send success rate.

• Higher browsing delay.

Data Performance Analysis

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.30

Test GPRS Attach (5%) PDP context activation (5%)

OperatorGPRS attach success rate

(60%)

GPRS attach Delay (40%)

PDP context success (60%)

PDP context Delay (40%)

Airtel 3G 2 1 1 2

Reliance 3G 3 3 1 1

BSNL 3G 1 2 3 3

Test FTP Upload (15%) FTP Download (25%) HTTP (30%) Email Send (5%) Email Retrieve (5%) Ping (10%)

OperatorFTP UL Success (40%)

FTP UL Throughput

(60%)

FTP DL Success (40%)

FTP DL Throughput (60%)

HTTP Success (40%)

HTTP Download time (60%)

Email Send Success (30%)

Email Send time (20%)

Email Retrieve Success (30%)

Email Retrieve

time (20%)Ping RTT

Airtel 3G 1 1 2 1 1 4 1 2 1 1 2

Reliance 3G 3 2 4 2 3 1 1 3 3 3 4

BSNL 3G 4 4 3 4 4 3 4 4 4 4 3

Reliance EvDo 2 3 1 3 2 2 3 1 2 2 1

Page 31: 3 g benchmarking report jan 2012

FTP Statistics

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

31

• File transfer is becoming increasingly andlegitimately popular as a method of transferringmedia files (movies and music).

• The FTP test is used to test the FTP serviceavailability and to measure the applicationthroughput for file download or file upload. Thetest is successful when the desired operation(uploading or downloading) has succeeded.

• Airtel has best downlink and uplink throughput ascompared to other operators.

* BSNL has low FTP throughputs due to lack ofaccess to data plans beyond a certain limit.

Page 32: 3 g benchmarking report jan 2012

HTTP Browsing Analysis

• The web browsing quality of experience is generally associated with the time it takes to locate and download a webpage within a web browser application.

• The HTTP Download test was conducted with 10 different websites. The Downloaded Page size was approx 350Kb.

• Airtel has higher HTTP browsing time for all web sites.

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

32

Page 33: 3 g benchmarking report jan 2012

HTTP Performance

33This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

Page 34: 3 g benchmarking report jan 2012

Airtel FTP DL/UL and HTTP Failures

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

34

FTP DL Failure

HTTP Browsing Failure

FTP UL Failure

Page 35: 3 g benchmarking report jan 2012

Email Send / Retrieve

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

35

• A 100KB email was sent and downloaded viathe POP and SMTP protocols. This is a closesimulation of Users accessing pull mail fromtheir work IDs using their hand phone.

• Airtel has good overall email performancewith least email retrieve delay.

Page 36: 3 g benchmarking report jan 2012

Ping Response: Latency Performance

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

36

• A Latency Test was Conducted with www.google.co.in with 32bytes of data to figure out the round trip time. APing Response signifies the average delay it takes to access a particular service.

• Airtel has high ping latency, as it takes higher number of hops to reach destination server.

Page 37: 3 g benchmarking report jan 2012

Data Access Technology Distribution

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.37

• Airtel has spent maximum amount of time on 3G compared to other operators.

Page 38: 3 g benchmarking report jan 2012

Attach and PDP Context Statistics

38This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

38

• Airtel has good GPRS attach and PDP context activation success rate.

• Airtel has higher PDP context delay compared to Reliance.

Page 39: 3 g benchmarking report jan 2012

39

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

CQI & Modulation Scheme

Operator CQI > 25 HS-DSCH Codes > 8

Airtel 3G 51.89% 43.29%

Reliance 3G 45.59% 21.95%

BSNL 3G 42.33% 39.19%

• Airtel has better channel quality and has been assigned higher number of codes.

Page 40: 3 g benchmarking report jan 2012

SMS & MMS Analysis

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

40

Page 41: 3 g benchmarking report jan 2012

KPI Overview Table

41This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

The attached KPI table gives an overview about the most important performance figures collected during this Dual mode Benchmarking campaign.

Page 42: 3 g benchmarking report jan 2012

Section 4

• Clusterwise Summary – Voice

• Clusterwise Snapshot – Voice

• Clusterwise Summary – Data

• Clusterwise Snapshot – Data

• Cluster Analysis– Cluster 1

– Cluster 2

– Cluster 3

– Cluster 4

Clusterwise Analysis

Page 43: 3 g benchmarking report jan 2012

Clusterwise Summary – Voice

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel3G for other reports. Phimetrics reserves the right to use this format for its other customers if required.

43

Voice Index Cluster 1 Cluster 2 Cluster 3 Cluster 4

Airtel 1 1 1 1

Reliance 3 2 3 3

BSNL 2 3 2 2

Cluster 3

Cluster 1

Cluster 2

Cluster 4

Page 44: 3 g benchmarking report jan 2012

Clusterwise Snapshot – Voice

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel3G for other reports. Phimetrics reserves the right to use this format for its other customers if required.

44

Page 45: 3 g benchmarking report jan 2012

Clusterwise Summary – Data

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel3G for other reports. Phimetrics reserves the right to use this format for its other customers if required.

45

Cluster 3

Cluster 1

Cluster 2

Cluster 4

Data Index Cluster 1 Cluster 2 Cluster 3 Cluster 4

Airtel 3G 1 1 1 1

Reliance 3G 4 3 3 3

BSNL 3G 3 4 4 4

Reliance EvDo 2 2 2 2

Page 46: 3 g benchmarking report jan 2012

Clusterwise Snapshot: Data

46This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel3G for other reports. Phimetrics reserves the right to use this format for its other customers if required.

46

Page 47: 3 g benchmarking report jan 2012

Cluster 1 (Pichola, Kalaji Goraji)

This report is generated in Phimetrics Proprietary format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

47

Best OperatorVoice: Airtel 3GData: Airtel 3G

Focus Area: HTTP Browsing Success Rate

• Airtel has low HTTP success rate.

Good Excellent

MOS

Average Poor

CSSR

CCR

CST

Coverage

1

Page 48: 3 g benchmarking report jan 2012

Best Operator Voice: Airtel 3GData: Airtel 3G

Cluster 2 (Hiran Magari, Vardhman Nagar, Jawahar Nagar, Railway Colony)

This report is generated in Phimetrics Proprietary format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

48

Good Excellent

MOS

Average Poor

CSSR

CCR

CSTFocus Area: Block call and Drop Call

• Airtel has 2 failed calls and 1 drop call in theentire drive and all of them were in thiscluster.

2

Coverage

Page 49: 3 g benchmarking report jan 2012

• Airtel has poor downlink throughput in thiscluster compared to other clusters

• High ping RTT

Best Operator Voice: Airtel 3GData: Airtel 3G

Cluster 3 (Panchwati, Pulan, Sobhagpura, Ahinsapuri)

This report is generated in Phimetrics Proprietary format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

49

Good Excellent

MOS

Average Poor

CSSR

CCR

CST

Focus Area: Ping RTT and Downlink throughput

3Coverage

Page 50: 3 g benchmarking report jan 2012

• Airtel has poor FTP DL success rate

Best Operator Voice: Airtel 3GData: Airtel 3G

Cluster 4 (NH-76)

This report is generated in Phimetrics Proprietary format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

50

Good Excellent

MOS

Average Poor

CSSR

CCR

CSTFocus Area: FTP DL Statistics

Coverage

4

Page 51: 3 g benchmarking report jan 2012

Section 5

Strategic Locations Analysis

• Strategic Location: Overview

• Strategic Location: Voice Analysis

• Strategic Location: Data Analysis– Overall

– Location 1: Airport

– Location 2: Bus Stand

– Location 3: Chetak Circle

– Location 4: Suraj Pol

– Location 5: Trident Hotel

Page 52: 3 g benchmarking report jan 2012

Strategic Location: Overview

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

525 Strategic Locations were tested for Voice (Accessibility testing) and Data Tests

Locations Latitude Longitude

Airport 24.62473 73.88862

Bus Stand 24.57684 73.70061

Chetak Circle 24.59066 73.69103

Suraj Pol 24.57998 73.69714

Trident Hotel 24.58082 73.67027

Page 53: 3 g benchmarking report jan 2012

• Voice Testing was done at each location with many short calls of 30s duration were made

from a stationary position.

• Voice Accessibility was measured.

• MOS could not be measured as the duration of the call was too short for voice clips to be

relayed

Operators Calls Attempted Calls Failed Calls Dropped Call Setup Time

Airtel 3G 75 0 0 2.65

Reliance 3G 74 0 0 2.99

BSNL 3G 75 0 0 3.18

Strategic Location: Voice Analysis

This report is generated in Phimetrics Proprietary format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

53

Failed

Locations Airtel 3G Reliance 3G BSNL 3G

Airport 0 0 0

Bus Stand 0 0 0

Chetak Circle 0 0 0

Suraj Pol 0 0 0

Trident Hotel 0 0 0

Page 54: 3 g benchmarking report jan 2012

Strategic Location : Data Analysis

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

54

Test Airtel 3G Reliance 3G BSNL 3G Reliance EvDo

FTP DL Access 10 12 9 10FTP DL Success 10 0 0 0FTP DL Accessibility 100.00% 75.00% 0% 70.00%

FTP DL Mean Data rate(Kbps) 1637.83 1735.15 67.79 367.88Email Send Access 15 15 15 11Email Send Success % 100.00% 100.00% 86.67% 73.33%Email Send Delay 12.4 24.5 45.3 9.9Email Receive Access 15 15 15 11

Email Receive Success % 100.00% 100.00% 93.33% 73.33%

Email Receive Delay 9.6 11.7 27.9 15.4SMS send Access 50 50 50 -SMS send Success % 100.00% 90.00% 94.00% -SMS send Delay (sec) 2.4 3.7 2.1 -

e2e Delivery Time (sec) 6.4 6.1 3.9

MMS send Access 15 9 6 -MMS send Success % 80.00% 20.00% 0.00% -MMS send Delay 8.54 9.13 0.00 -

e2e MMS Delivery Time(sec) 37.4 20.0 0.0 -

WAP Browsing Attempt 50 30 40 10

WAP Browsing Success 44 18 32 6

WAP Browsing Success Rate 88.00% 60.00% 80.00% 60.00%

WAP Browsing Delay 49.1 11.9 39.7 30.8

Page 55: 3 g benchmarking report jan 2012

Location 1: Airport

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

55

Test Airtel 3G Reliance 3G BSNL 3G Reliance EvDo

FTP DL Access 2 2 1 2FTP DL Success 2 2 0 0FTP DL Accessibility 100.00% 100.00% 0.00% 0.00%

FTP DL Mean Data rate(Kbps) 987.97 93.0 32.35 26.27Email Send Access 3 3 3 3Email Send Success % 100.00% 100.00% 100.00% 100.00%Email Send Delay 13.6 90.4 90.3 17.1Email Receive Access 3 3 3 3

Email Receive Success % 100.00% 100.00% 100.00% 100.00%

Email Receive Delay 22.6 34.1 61.9 52.6SMS send Access 10 10 10 -SMS send Success % 100.00% 70.00% 100.00% -SMS send Delay (sec) 2.4 5.3 2.3 -

e2e Delivery Time (sec) 4.8 10.3 4.5 -

MMS send Access 3 0 0 -MMS send Success % 100.00% 0.00% 0.00% -MMS send Delay 7.99 0.00 0.00 -

e2e MMS Delivery Time(sec) 24.8 0.0 0.0 -

WAP Browsing Attempt 10 10 10 10

WAP Browsing Success 8 4 6 6

WAP Browsing Success Rate 80.00% 40.00% 60.00% 60.00%

WAP Browsing Delay 92.6 36.4 65.2 61.5

Page 56: 3 g benchmarking report jan 2012

Location 2: Bus Stand

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

56

Test Airtel 3G Reliance 3G BSNL 3G Reliance EvDo

FTP DL Access 2 3 2 2FTP DL Success 2 3 0 1FTP DL Accessibility 100.00% 100.00% 0.00% 100.00%

FTP DL Mean Data rate(Kbps) 2970.318 2020.502 124.94 530.78Email Send Access 3 3 3 3Email Send Success % 100.00% 100.00% 100.00% 100.00%Email Send Delay 15.3 7.3 32.1 8.9Email Receive Access 3 3 3 3

Email Receive Success % 100.00% 100.00% 100.00% 100.00%

Email Receive Delay 6.6 5.4 11.4 5.9SMS send Access 10 10 10 -SMS send Success % 100.00% 80.00% 100.00% -SMS send Delay (sec) 2.3 3.5 2.5 -

e2e Delivery Time (sec) 4.6 0.0 4.6 -

MMS send Access 3 3 3 -MMS send Success % 66.67% 0.00% 0.00% -MMS send Delay 7.41 0.00 0.00 -

e2e MMS Delivery Time(sec) 47.1 0.0 0.0 -

WAP Browsing Attempt 10 0 10 -

WAP Browsing Success 8 0 10 -

WAP Browsing Success Rate 80.00% 0% 100.00% -

WAP Browsing Delay 37.3 0.0 43.1 -

Page 57: 3 g benchmarking report jan 2012

Location 3 : Chetak Circle

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

57

Test Airtel 3G Reliance 3G BSNL 3G Reliance EvDo

FTP DL Access 2 3 3 2FTP DL Success 1 3 0 2FTP DL Accessibility 50.00% 100.00% 0.00% 100.00%

FTP DL Mean Data rate(Kbps) 2535.94 2415.13 106.75 331.56Email Send Access 3 3 3 0Email Send Success % 100.00% 100.00% 100.00% 0.00%Email Send Delay 8.4 8.7 9.3 0.0Email Receive Access 3 3 3 0

Email Receive Success % 100.00% 100.00% 100.00% 0.00%

Email Receive Delay 4.9 7.5 5.2 0.0SMS send Access 10 10 10 -SMS send Success % 100.00% 100.00% 70.00% -SMS send Delay (sec) 2.7 3.1 3.0 -

e2e Delivery Time (sec) 17.8 5.4 5.3 -

MMS send Access 3 3 0 -MMS send Success % 100.00% 100.00% 0.00% -MMS send Delay 12.79 9.13 0.00 -

e2e MMS Delivery Time(sec) 20.0 20.0 0.0 -

WAP Browsing Attempt 10 10 0 0

WAP Browsing Success 10 4 0 0

WAP Browsing Success Rate 100.00% 50.00% 0.00% 0.00%

WAP Browsing Delay 4.4 3.7 0.0 0.0

Page 58: 3 g benchmarking report jan 2012

Location 4 : Suraj Pol

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

58

Test Airtel 3G Reliance 3G BSNL 3G Reliance EvDo

FTP DL Access 2 1 1 2FTP DL Success 2 1 0 2FTP DL Accessibility 100.00% 100.00% 0.00% 100.00%

FTP DL Mean Data rate(Kbps) 1228.58 2123.471 0.00 272.81Email Send Access 3 3 3 2Email Send Success % 100.00% 100.00% 33.33% 100.00%Email Send Delay 9.2 8.8 62.7 14.6Email Receive Access 3 3 3 2

Email Receive Success % 100.00% 100.00% 66.67% 100.00%

Email Receive Delay 7.2 6.1 49.6 12.8SMS send Access 10 10 10 -SMS send Success % 100.00% 100.00% 100.00% -SMS send Delay (sec) 2.4 3.3 2.6 -

e2e Delivery Time (sec) 0.0 0.0 4.8 -

MMS send Access 3 0 0 -MMS send Success % 66.67% 0.00% 0.00% -MMS send Delay 7.10 0.00 0.00 -

e2e MMS Delivery Time(sec) 47.8 0.0 0.0 -

WAP Browsing Attempt 10 10 10 -

WAP Browsing Success 10 10 6 -

WAP Browsing Success Rate 100.00% 100.00% 60.00% -

WAP Browsing Delay 74.1 19.2 47.0 -

Page 59: 3 g benchmarking report jan 2012

Location 5 : Trident Hotel

This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

59

Test Airtel 3G Reliance 3G BSNL 3G Reliance EvDo

FTP DL Access 2 3 2 2FTP DL Success 2 0 2 2FTP DL Accessibility 100.00% 0.00% 0.00% 100.00%

FTP DL Mean Data rate(Kbps) 1361.36 0.00 23.308 574.62Email Send Access 3 3 3 3Email Send Success % 100.00% 100.00% 100.00% 100.00%Email Send Delay 15.3 7.3 32.1 8.9Email Receive Access 3 3 3 3

Email Receive Success % 100.00% 100.00% 100.00% 100.00%

Email Receive Delay 6.6 5.4 11.4 5.9SMS send Access 10 10 10 -SMS send Success % 100.00% 100.00% 100.00% -SMS send Delay (sec) 2.3 3.5 2.5 -

e2e Delivery Time (sec) 4.6 0.0 4.6 -

MMS send Access 3 3 3 -MMS send Success % 66.67% 0.00% 0.00% -MMS send Delay 7.41 0.00 0.00 -

e2e MMS Delivery Time(sec) 47.1 0.0 0.0 -

WAP Browsing Attempt 10 0 10 -

WAP Browsing Success 8 0 10 -

WAP Browsing Success Rate 80.00% 0.00% 100.00% -

WAP Browsing Delay 37.3 0.0 43.1 -

Page 60: 3 g benchmarking report jan 2012

Section 6

Summary

• SWOT Analysis for Airtel

• Key findings

Page 61: 3 g benchmarking report jan 2012

SWOT Analysis for Airtel

• High HTTP browsing time• Higher ping RTT• Poor data performance in cluster 3 in terms of downlink

throughputs and RTT• Cluster 3 has poor 3G coverage compared to other clusters

• Airtel has least call setup time.• Airtel has best Accessibility and Voice call Retainability.• Airtel has best DL speech quality.• Airtel has 100% FTP UL success rate and best FTP

Downlink and Uplink throughput.

Strengths Weaknesses

Threats

• Web Browsing experience is lower compared toother 3G Cities• 3G coverage can further be improved by improving it in

cluster 3• Ping latency can be improved by optimizing routing nodes

and delay for each hops

Opportunities

This report is generated in Phimetrics Proprietary format. This report format cannot be duplicated or copied by Airtel for other reports. Phimetrics reserves the right to use this format for its other customers if required.

61

Page 62: 3 g benchmarking report jan 2012

Observations and Recommendations

• Airtel has the best 3G experience for both voice and data, which can becapitalized given the importance of this tourist place.

• Cluster 1 and 2 still have few areas which can be further improved forindoor coverage.

• The Browsing experience can be further improved to widen the gap withits competitors by improving latency.

62This report is generated in Phimetrics Proprietory format. This report format cannot be duplicated or copied by Airtel3G for other reports. Phimetrics reserves the right to use this format for its other customers if required.

Page 63: 3 g benchmarking report jan 2012

Thank You

63

Prepared by: Sanket Jain

Verified by: Hardik Patel

Attested by: Mohit Lohani