mycom osi proptima™ v4 · victus and huawei users can view only radio related nes and statistics...

WIND Hellas

Benefits and Lessons Learned from

MYCOM OSI PrOptima™ V4.2

Kernel upgrade and IP 3.0 solution

20-21 April 2016

Spyros Kokosis

Foteini Erga

Wind Hellas

• Wind Hellas Company Profile and Network

• V4.2 kernel upgrade project

• Drivers

• Project Phases and Risks

• Post-upgrade issues

• Benefits

• Lessons Learned

• New Feature Requests

• Q & A Session

Agenda

COMPANY PROFILE

Network Coverage

% GSM Population Coverage 99%

% UMTS Population Coverage 72%

% Fixed Line Coverage 70%

% LTE Population Coverage 60%

Strategic partnership with Vodafone Hellas for

partial sharing of the 2G / 3G mobile network

since 2014

~ 800 2G and 3G shared sites

6

Radio Transmission IPCircuit Switch

Packet Core Fixed Access Fixed Core VAS

Vendors per Main Technology Area

NOKIA

NOKIA

NOKIA

WH Network

integration in

PrOptima™

CS Domain

PS DomainSGSN

GGSN

MGW

MSS

HLR

RNC

BSCBTS

NodeB

IP/MPLS

OSS

Circuit and Packet Core

VAS

eNodeB

BTS

CS Domain

PS DomainSGSN

GGSN

MGW

MSS

HLR

RNC

BSC

NodeB

IP/MPLS

OSS

Circuit and Packet Core vs. PrOptima™

VAS

eNodeB

LITESPAN

IP

DSLAM

WH Fixed

Transport

IP/MPL

S

INTERNET

BRAS

NGN

IP

LEX

OTE, PLMNAND OTHER OPERATORS

End User

premises

Splitter

H.248

MSAN

Fixed Network

LITESPAN

IP

DSLAM

WH Fixed

Transport

IP/MPL

S

INTERNET

BRAS

NGN

IP

LEX

OTE, PLMNAND OTHER OPERATORS

End User

premises

Splitter

H.248

MSAN

Fixed Network vs. PrOptima™

V4.2 Kernel upgrade

1. Drivers

The main reason for the V4.2 kernel upgrade

Users View Definitions

• Separate the views of users working in Radio Network sharing project (Huawei, Victus engineers)

• Partial data sharing - Confidentiality issues

• Build shared reports between Wind - Victus

The main reason for the IP 3.0 Solution

IP SLA Monitoring

• Verify network SLAs with our customers

• Offer reliable services with the requested capacities

• Metrics to measure:

– Port Speed

– Service Availability

– Port Utilization

– RTT/ Packet Loss/ Jitter

– Delay (round-trip, one-way)

– Packet sequencing

2. Project Phases

“It always takes longer than you expect,

even when you take into account Hofstadter's Law”

Douglas Hofstadter

Project Duration: 8 months

• Preliminary Phase: 2 months

– Replace old HW proptipoller servers (Month1)

– Solaris 11 upgrade on pm servers (Month2)

• Main Phase: 6 months

– Test server V4.2 & IP3.0 upgrade (Month3)

– SIT and UAT (Month4)

– Live server V4.2 Kernel upgrade (Month5)

– IP 3.0 Solution upgrade (Month6)

– Final Acceptance (Month8)

3. Risks

Project Risks

• OS Solaris 11 – prerequisite for V4.2 release - not supported by existing proptipollers HW

• replace HW

• migrate ~ 700 IP elements

• IP release currently installed was too old

• Wind Hellas was the first customer who migrated from IP1.0 to IP3.0

• The 2 step migration procedure never tested on production environment

Project Risks

• Differences between live and test environment regarding the IP network

• 700 IP NEs integrated in the live server

• only 2 test IP devices in staging server

• IP3.0 Solution would be tested on a couple of test IP devices only

Project Risks

Project Risks

• NO roll back procedure neither for V4.2 nor the IP3.0 upgrade

Project Risks

• More than 700 IP nodes should be migrated in the new device platforms concept

4. Post-Upgrade

Issues

“Anything that can go wrong,

will go wrong”Murphy's law

Post Upgrade Issues from the V4.2 upgrade

Upgrade day

• During live server V4.2 kernel upgrade the PrOptipollersstopped polling the IP devices for 6 hours.

Upgrade day

• IP Throughput counters converted from bits per sec to bytes per sec– decreased Throughput values in all IP interfaces

– generation of false PM alarms

– IP3.0 upgrade postponed until issue was resolved

Proved to be an incompatibility issue between V4.2 and IP1.0

Post Upgrade Issues from the V4.2 upgrade

The day after

• Performance degradation

• Delays in reports computation and data importation

Misconfigured parameters

Post Upgrade Issues from the V4.2 upgrade

The day after

• Impact on Pre-aggregated KPIs and Monthly aggregation.

Monthly aggregation script was accidentally left outside of the migration procedure

Misconfigured parameters led to memory allocation issue

Post Upgrade Issues from the V4.2 upgrade

Upgrade day

• The upgrade process took 8 times longer than initially estimated

Client unavailable for 1 business day

# of the IP elements (700) wasn’t taken into account when estimating the migration procedure

Post Upgrade Issues from the IP3.0 upgrade

The day after

• Memory consumption due to a wrong mapping between IP1.0 and IP3.0

Limitation on the daily concurrent users from 30 to 10 users

Post Upgrade Issues from the IP3.0 upgrade

The day after

• Data Discontinuity due to duplicated IP NEs on the NE Palette

• Tilde (~) symbol appeared at the end of the new NE names in the NE Palette

Post Upgrade Issues from the IP3.0 upgrade

5. Benefits

User specific benefits from the IP3.0 release

• Computation Time

Improved user perception especially in IP reports

less congestion during busy hours (morning reports)

User specific benefits from the V4.2 release

• Monthly Aggregation

Flat files for monthly aggregated data

Better computation time for monthly reports vs. the on the fly computation

Increased retention period for monthly data vs. the daily data retention

User specific benefits from the V4.2 release

• Multi Dimensional Analysis

Mixed network areas (2G, 3G and 4G) using the Site and/or Cell Sector new object types

Multi RAT Performance Management

User specific benefits from the V4.2 release

• Ability to import static Network Areas from a CSV file stored on a local PC

Efficiency compared to the manual procedure through the client

No need to contact the administrator

User specific benefits from the V4.2 release

• KPI cut and paste from the client

Easy re-organization of the KPI library through the client

No need to contact the administrator

• Visualization of the raised alarms in a table report

Avg duration per alarm type and Network Object type

# of acknowledged per alarm type and Network Object type

# of cleared per alarm type and Network Object type

# of raised alarm per type and Network Object type

User specific benefits from the V4.2 release

• Automatic switch between pre-agg and on the fly formula

Users don’t have to create different reports for historical reporting

Better computation time

(-) In the reports where Network Areas are used, the system switches to

the pre-agg formula and returns NaN because the Pre-agg KPIs don’t

apply to Nas

(-) Users have to manually disable this feature

User specific benefits from the V4.2 release

• User Views Filtering Victus and Huawei users can view only Radio

related NEs and statistics

User views separation to NOC personnel for the alarms monitoring per Network domain

Enhancements:

• In Alarm monitoring window, the alarms created on Network Areas to be filtered based on the user view definitions.

• When applying a user view definition per MOC, the parent counters should not be visible

• Decrease the log in time required for a user with restricted views

Admin specific benefits from the V4.2 release

BEFORE

AFTER

• Configuration importation per VTI

Autonomous configuration process

Time efficiency

Admin specific benefits from the V4.2 release

• Pre-Aggregated formulas importation from a CSV file

Straightforward and efficient way

Save time especially for bulk requests

Admin specific benefits from the V4.2 release

• Off the shelf support for vendors, models through the device platforms

• Easiest and more accurate way of integrating IP elements

• On demand Device Platform creation

Admin specific benefits from the IP3.0 release

6. Lessons Learned

Lessons learned

Things will always take longer than you expect

When in doubt, choose then validate and if needed iterate

Lessons learned

A HW and SW audit should precede an upgrade

Mirror your live and test environment as much as you can

Lessons learned

Double-check all the configuration settings on the servers

Thoroughly review the migration procedure

Lessons learned

Involve as many users as you can in the UAT / Regression tests

Document all the changes from one release to another

Lessons learned

Mitigate the risk of data loss as much as you can

Remember the lessons of the past

6. Feature Requests

Request for Features

Requests for new Features in the following release:

• Apply Pre-agg KPIs to Network Area dimension

• Enhanced Search & find function in Report explorer

• Improvements on sharing and renaming a Static Network Area

• Cell Sector and Site enhancements

Thank you