lte kpis and acceptance

Slide titleIn CAPITALS

44 pt

Slide subtitle 20 pt

LTE KPI’s and Acceptance


44 pt


Introduction

Slide title 30 pt

Text 18 pt

Bullets level 2-516 pt

›!"# $%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬®¯°±²³´¶·¸¹º»¼½ÀÁÂÃÄÅÆÇÈËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿĀāĂăąĆćĊċČĎďĐđĒĖėĘęĚěĞğĠġĢģĪīĮįİıĶķĹĺĻļĽľŁłŃńŅņŇňŌŐőŒœŔŕŖŗŘřŚśŞşŠšŢţŤťŪūŮůŰűŲųŴŵŶŷŸŹźŻżŽžƒȘșˆˇ˘˙˚˛˜˝ẀẁẃẄẅỲỳ–—‘’‚“”„†‡•…‰‹›⁄€™−≤≥fifl

Why learn about ACCEPTANCE & KPI’s

› Acceptance is part of Business Critical Terms & Conditions

› Control of KPI commitments is control of cash flow and profitability

› Ericsson had 80BSEK locked in Acceptances in 2008

› If all projects finished one day earlier = 500 MSEK/day for Ericsson

Slide title 30 pt

Text 18 pt



Scope and objectives

› Explain how Acceptance & KPI’s are impacting business

› Describe Ericsson view on LTE Acceptance principles and KPI’s

› List LTE KPI approach and point to further information

Objectives

Scope

› Customer and Ericsson view on Acceptance and KPI’s -> business Impact

› KPI Based Acceptance

› Understanding the KPI’s

› LTE KPI’s


44 pt


Overview

Slide title 30 pt

Text 18 pt



LTE ACCeptance principles› HW should be accepted on delivery. The LTE product HW is tested at the factory

according to Ericsson quality procedure. Normal claims and warranty naturally apply.

› HW Activation Codes should be accepted when the codes are made available to the customer in ELIS, the license management system.

› SW should typically be accepted after successful demonstration in the customer lab, following the Demo Object List (DOL). This should include the mechanisms for both HW Activation Codes, and for SW Feature Activation Licenses.

› Installation services should be accepted when the node has been installed and stand-alone tests have been successfully completed.

› Integration services should be accepted when the node has been configured and integrated into the network, and stand-alone tests have been successfully completed.

› Tuning services should be accepted when tuning service has been completed and documented, ideally on a cluster per cluster basis.

Separate payments for HW, SW and Services !

Slide title 30 pt

Text 18 pt



LTE Acceptance tips

Not recommended:› Contractual formulations including delay of fraction of payment of

HW/SW coupled to fulfillment of service

› Contractual formulations including liquidated damages (referring to payment of HW/SW) coupled to fulfillment of service

Wrong acceptance approach has:› Effect of tied capital –increases our Employed capital/Net sales ratio

› Effect of lowered margins – increased costs

If we tie up KPI’s with the HW and SW sales we might end up like this...

Slide title 30 pt

Text 18 pt



CASH FLOW Scenario

Upfront 15%

HW 75%

Delivery

SW75%

Delivery

Services 75%

MonthlyProgress

Final Acceptance 10%

Recommended Terms

10%

75%

15%

PenaltiesDiscounts

!High riskBad Cash flowCost blow out

KPI Based Acceptance Terms

Upfront 15%

HW x%

Delivery

SWy%

Delivery

Services z%

Progress

Final Acceptance 10%

KPITargets

25% plus

50% or less

KPI linkedpayment risk

Slide title 30 pt

Text 18 pt



Execution Acceptance Invoice/Collection

CAPITALEMPLOYED

TIME

UnbilledSales

BilledReceivables

Inventory

ITO days: 83 days Account Receivables, DSO: 115 days

Overdue

Revenue recognition

First invoice sent

Total: >80BSEK

Contract terms & conditions, project closure, and billing, key areas to address

“CASH OUT!”

“CASH IN!”

Operators are using complex acceptance procedure with unrealistic KPIs ...


44 pt


KPI BASED ACCEPTANCE

Slide title 30 pt

Text 18 pt



Why ARE CUSTOMERS DEMANDING KPIS ?

It’s not just about performance – IT’S CONTROL AND COST!

TO CONTROL...

RISKS COSTS

› Internal pressure

› Don’t understand the technology -> LTE !

› Force Ericsson to fix their performance issues

› Avoid risk, shift risk to Ericsson

› Own cash flow issues

› Extra services at no cost

› Penalties, discounts or things for free

› Avoid paying Ericsson

Slide title 30 pt

Text 18 pt



Ericsson Driving Forces

› Profitability, ability to fulfill contract

› Cash flow, timely invoicing

› Separate payment terms for HW, SW, and Services

› Close contract with clear responsibility, scope and acceptance criteria

› Control of contract fulfillment

› Additional sales

Slide title 30 pt

Text 18 pt



Contractual Acceptance directive

1. Acceptance should have a defined scope, procedure and timetable agreed in advance,

2. To be based upon objective and measurable criteria

3. KPI based acceptance should be limited to network and technology consulting part of rollout contracts and limited to KPIs defined in the KPI library at the Acceptance Gateway.

4. If products or parts are put into operation, automatic final acceptance of such parts

Reference: Business Critical Terms&Conditions documents

http://anon.ericsson.se/eridoc/component/eriurl?docno=03402-3096/5Uen&objectId=09004cff82670e79&action=current&format=pdf







Slide title 30 pt

Text 18 pt



ContracTual considerations and mu’s

› Contractual Considerations: The MU shall always strive to follow the Acceptance Principles (Acceptance Approach and the Business Critical Terms and Conditions).

› Restriction: If the MU deviates from the Acceptance Principles, it should clearly state the deviations and estimate the risks and associated impact on the business case for the contract. – Deviations from the Acceptance Principles must be presented to the LTE Deal

Strategy meeting, pre-SDP2. – The MU is responsible to secure required input, Price Governance, material (PPT,

Xls ) – Set up of this meeting is done by contacting Christer Gustafsson or Linda Lindberg

(manager EAB/FRP).

http://cpi.al.sw.ericsson.se/alexserv?ID=17018&DB=23677-lzn7840900_1r4a.alx&FN=1_10056-HSC10550_1Uen.E.html

http://cpi.al.sw.ericsson.se/alexserv?ID=17018&DB=23677-lzn7840900_1r4a.alx&FN=1_10056-HSC10550_1Uen.E.html

http://internal.ericsson.com/wps/portal/yasper/!ut/p/c1/04_SB8K8xLLM9MSSzPy8xBz9CP0os3hjN8MQD3MPIwMD30BnAyNLRzOPsDBDQ3cXM6B8JE55A0djArrDQfbhVmFqgFcebD4-eW9jfT-P_NxU_YLcCIPMgHRFAErM0NQ!/dl2/d1/L0lDU0NTS0tVZ2tra0tDbEVLQ2xFS0NsRS9ZRVVFQUFBSUlDSUFBTU1FSUtJRUFH



http://internal.ericsson.com/wps/portal/yasper/!ut/p/c1/hZDNjoIwAISfxQfYtNAKcoSyUMCi_AiUCwEkivwmuIo8_bp72ZNu5vjNTDIDUvBUn9_qU36thz5vAQepnCFDCKlMRbjZUwmKCNs7DBmCWAZRl891Vy_VEYQggTgLLnBk26lni7Lbh5_NNSgX5hw5JedDYscV_qhaanflIfXzZNQKbaYPlX9J7RzEdl4mhkcE7xApmZEh




Slide title 30 pt

Text 18 pt



Acceptance activities (typical)

QualifySales

Opportunities

CreateProposal

Close Deal Contract

Management Contract

Handshake

RiskFull Track

SDP 4SDP 1 Risk SDP 2 Risk SDP 3

Critical Phase

Work outyour strategy

Prepareoffer/

contract/Strong wanted position

NegotiateRefineAgree

Work out the details –

Keep it simple – Keep costs

down

Execute –Look for ways

to save andclose out +

add on sales!

Ability to influence

Correction Cost

Slide title 30 pt

Text 18 pt



ACR

Strategy

Payments

Satisfaction

KAM

CSR

SystemSolution

ServiceSolution

CFR

Cash Flow

Delivery

Execution

Ensure link betweenTerms of Payment and Acceptance Procedure. (Particularly Cashflow)

Ensure link betweenTerms of Payment and Acceptance Procedure. (Particularly Cashflow)

Align Services (SoW) and Solution with Acceptance Commitment incl. KPIs

Align Services (SoW) and Solution with Acceptance Commitment incl. KPIs

Ensuring KPI Based Acceptance is integrated in total Acceptance Procedure

Ensuring KPI Based Acceptance is integrated in total Acceptance Procedure

Core 3 – ResponsibilityKPI Based Acceptance

• Overall Decision• Ensure Acceptance risks areunderstood by team and pushfor flow between solution and Acceptance• Promote use of KPI Champion

• Overall Decision• Ensure Acceptance risks areunderstood by team and pushfor flow between solution and Acceptance• Promote use of KPI Champion

KPI Based Acceptance Champion KPI Based Acceptance Champion


44 pt


Understanding the KPI’s

Slide title 30 pt

Text 18 pt



Basic facts – KPI’s

› Operator thinking: More KPI’s, Operator is getting more control

› Ericsson thinking: More KPI’s, higher costs, longer procedure, risk for delays in payment and penalties

› LTE is new, we can expect a lot of KPI’s since operators want to protect themselves

› We can be active and suggest KPI’s, rather than wait for them - We know what we are doing

› Right support is provided from the LTE start – look at the Acceptance gateway– What KPI’s and what values to discuss– Acceptance method

Slide title 30 pt

Text 18 pt



WHAT IS IMPACTING KPI VALUES ?

TARGETS

PRE-CONDITIONSHOW TO

MEASURE

CONDITIONS

EXCLUSIONS

PROCESSES

Slide title 30 pt

Text 18 pt



Common mistakeReverse engineering

Often only KPI targets are stated in the contract

TARGETS

PRE-CONDITIONSHOW TO

MEASURE

CONDITIONS

EXCLUSIONS

PROCESSES

Slide title 30 pt

Text 18 pt



Kpi’S AND PROJECT TYPES

› In Large Rollout Projects where we have a lot of risks, unknowns and dependancies and where services are small part - Minimise KPI risks and costs

› In Standalone projects where we have mostly services, and we have control - Use KPIs as business driver

KPIs can result in good business !


44 pt


LTE KPI’s

Slide title 30 pt

Text 18 pt



LTE kpi AcceptanceTypes of Acceptance

› The scope of a contract will determine the type of acceptance used

› Four different types of acceptance are proposed in this document– Product (Supply) Acceptance– Proof of Concept Acceptance– Network (or Cluster) Acceptance– Stability Acceptance

› A network contract may stipulate more than one of these types of acceptance

› It is desirable to minimise the acceptance testing – it can be consuming and costly

KPI’s handling will depend on the type of Acceptance

Slide title 30 pt

Text 18 pt



There are 4 KPI recommended for use in LTE acceptance

Session Setup Success Rate

Session Abnormal Release Rate– ONLY used for drive tests with short calls

or

Minutes per Abnormal Release– Used for stats based KPI & drive tests with long calls

DL User Throughput

UL User Throughput

Recommended KPIFor use in LTE Acceptance

1

2

3

4

Slide title 30 pt

Text 18 pt



Recommended KPIRecommended Location for KPI Measurement

No Recommended KPIControlled

Environment Golden Cluster

Network

(Cluster)

Drive TestCounters / Statistics

1 Session Setup Success Rate X X X X

2a Session Abnormal Release Rate X X X

2b Minutes per Abnormal Release X X

3 DL User Throughput X X X

4 UL User Throughput X X X

Product (Supply) /Proof of Concept

Acceptance

NetworkRollout

Acceptance

NetworkStability

Acceptance

Slide title 30 pt

Text 18 pt



Recommended KPI – exampleSession Setup Success Rate

98.00%

99.0%99.3%

99.50%

97.50%

98.00%98.25%

98.50%

96.00%

96.50%

97.00%

97.50%

98.00%

98.50%

99.00%

99.50%

100.00%

L10A L10B L11 L12 =>

Lab / Controlled

Field / Drive test

Comments and Recommendations for Acceptance:› SSSR should be tested using terminals which have undergone IODT with Ericsson› Short calls are recommended for field test acceptance› Lab and Controlled environment testing is to be undertaken in good radio conditions

(RSRP > -85dBm, SINR > 6dB)› Exclusions should be included for drive testing where the setup failure is caused by

a factor beyond Ericsson’s control (eg. No coverage due to radio network design constraints)

Slide title 30 pt

Text 18 pt



Throughput KPIHigh Level Summary: Acceptance Process

DL Throughput (EPA 5Hz) vs SNR 10MHz

0

10000

20000

30000

40000

50000

60000

70000

80000

-5 0 5 10 15 20 25 30 35 40

SNR (dB)

Th

rou

gh

pu

t (k

bp

s)

Simulation value

Maximum Acceptance Value

Note: Graph is only an example, real values should be calculated using Coverage and Dimensioning

Guideline

› Special methodology for the KPI User throughputs

› Applications, measurement methods, exclusions explained

› Theoretical max value calculations (based on NW configuration) vs. Practical value

› Throughput KPI’s are well documented

› Safe way for Ericsson to take lead in the KPI negotiations

Slide title 30 pt

Text 18 pt



Optional KPIFor use in LTE Acceptance

There are 6 optional KPI for possible use in LTE acceptance:

Simultaneous Connected Users per RBS

Intra LTE HO Success rate

Round Trip Time (RTT)

Terminating Session Success Rate (Paging)

Control Plane Latency (Idle to Active)

Cell Availability

9

8

7

6

5

10

Slide title 30 pt

Text 18 pt



quiz

LTE Acceptance StrategyQuiz

(continue to the next slide to start the Quiz)

PROPERTIESOn passing, 'Finish' button: Goes to Next SlideOn failing, 'Finish' button: Goes to Next SlideAllow user to leave quiz: At any timeUser may view slides after quiz: At any timeUser may attempt quiz: Unlimited times


44 pt


Summary

Slide title 30 pt

Text 18 pt



Summary – LTE Acceptance & KPI’s

› KPI’s are often financial driver, rather than technical› Keep it simple, minimize number of KPI’s› Consult "LTE Acceptance Toolbox" for overall

procedure› For KPI’s listing (Recommended and Optional), and

their definitions consult "Acceptance KPIs - methods and values"

› KPI Target values are last in chain - consequence of pre-conditions

› Special care is given to the throughput related KPI’s – systematic and structured approach

› Negotiate with customer › For more information about KPI Based Acceptance

please contact Graham Allen or Marius Sirbu


44 pt


references

Slide title 30 pt

Text 18 pt



REFERENCES › LTE Acceptance toolbox

› Business Critical Terms & Conditions

› LTE Acceptance approach – HW, SW, Services Acceptance

› KPI Based Acceptance.

› LTE KPI recommendations and values for rollout projects Acceptance KPIs - methods and values

› Network Rollout Projects - KPI Tendering&Contracting

› Price Governance material (PPT, Xls )

Slide title 30 pt

Text 18 pt



lte kpis and acceptance

Documents