the importance of otm post-implementation measurement systems

Imagination at work

The Importance of OTM Post-Implementation Measurement Systems GE Power & Water OTM Functional Team Dave Phillips | Jonathan Coley | Rebecca Hoffman

KPIT

Joe Macri

Power & Water

Imagination at work

© 2015 General Electric Company - All rights reserved

Abstract

This presentation will focus on the measurement systems created by the GE Power & Water OTM functional team as they migrated from their legacy TMS into OTM. Subject matter will include sample metrics showing comparisons of TMS transactional volume, categorization and analysis of support metrics, EDI analytics and automated planning measurements. The functional team will also discuss why these measurements were implemented and how they were developed.

2


Transportation Management System (TMS) migration monitoring

Support ticket analysis

Electronic data interface (EDI) analytics

Go-Live as the beginning of the journey

Agenda

3


Productivity monitoring


Go-Live as the beginning of the journey


Why pick this topic

5

Most SIG topics dealt

with implementation or

application run time

topics.

Few have broached the

subject of measuring to

achieve success once

live.

Select OTM

Provision

Select SI

KICK OFF REQUIREMENTS DESIGN BUILD TEST TRANSITION GO-LIVE

TYPICAL SIG TOPIC RANGE


Metrics evolution

Initial Go Live

Pre ‘12

June ‘12 KPIT Support Assessment

KPIT projects roll out

February ‘14

April ‘14 KPIT OTM and SOA managed

services

Migration to Mavenwire

October ‘14

Some early metrics were

system performance

oriented – Agents, SQL,

Reports and SOA

This, however, did not

account for user

adoption over the legacy

system, user support

issues, veracity of data

from 3rd party sources

6


Metrics development objectives

Knowledge base to find the ‘right’ data

• Example, EDI 997 is often thought of as transmission success

• However, this is acceptance to a trading partner’s translator, not target application

Thorough and effective addressing of issues – true fix, not just a Band-Aid

• Example, using an agent to fix a process issue in OTM when the data which caused the problem comes from a flaw in an integration

7


Transportation Management System(TMS) migration monitoring


Why no FTI

9

Need to join data from both OTM and legacy system

Require transactional reporting – daily and weekly

Technical expertise … Oracle PL/SQL

Flexibility and consistency

Leverage standard well-tuned code

Replicated DB in hosted environment



• Merge legacy and OTM into single dataset • Define use cases by billing code • Keep platform flexible and nimble

Pe

rce

nt

TM

S t

ran

sact

ion

al v

olu

me

Fiscal week

Percentage legacy

Percentage OTM

Total percentage Transportation Management (TMS) usage

10


Total Transportation Management (TMS) usage by volume

• View impact of new integrations • Business-specific factors driving leakage

TM

S t

ran

sac

tio

na

l vo

lum

e

Legacy

OTM

Fiscal week

11


• Detail report included for root cause analysis • Identify router or supplier non-compliance • Determine new use cases driving non-compliance

Fiscal week

Percentage legacy

Percentage OTM

Business-specific OTM migration leakage by mode and region

Pe

rce

nt

TM

S t

ran

sact

ion

al v

olu

me

12


Support Ticket Analysis


0.0%

1.0%

2.0%

3.0%

4.0%

5.0%

6.0%

7.0%

01

_1

5

02

_1

5

03

_1

5

04

_1

5

05

_1

5

06

_1

5

07

_1

5

08

_1

5

09

_1

5

10

_1

5

11

_1

5

12

_1

5

13

_1

5

14

_1

5

15

_1

5

16

_1

5

17

_1

5

18

_1

5

19

_1

5

20

_1

5

21

_1

5

22

_1

5

23

_1

5

24

_1

5

25

_1

5

26

_1

5

27

_1

5

Fiscal week Normalized wo Admin Linear (Normalized wo Admin)

OTM ticket volume normalized by weekly shipment volume without admin

• Monitor stable operations despite new integrations or functionality

14


147

123 120

73 71

31 21 18 16

9 9 8 4

0

20

40

60

80

100

120

140

160

Pareto running 5 week ticket volume categories

• Drive root cause analysis, categorization and FMEA • Deep dive into sub-category level • Keep support costs level despite increasing volume and complexity

15


Electronic Data Interface (EDI) Analytics


Establish EDI connectivity

Define success measurement – 95% Event Update – 24 hour Event Span – 98% Transmission Success

Identify data sources – OTM for Tender offer/Event Updates – FPC for match file details – Middle-ware visibility?

Does ‘received’ (997) really mean success?

Monitor metrics constantly

EDI metrics

17

IF IT DOESN’T GET MEASURED, IT DOESN’T GET DONE …



EDI metrics Pickup and delivery event compliance-Good performance

Compare I_TRANSACTION tender offer to I_TRANSACTION shipment status to produce metrics

18


EDI metrics Pickup and delivery event compliance-Downward trending

Monitor constantly for trends

19


EDI metrics Pickup and delivery event compliance-Span measurement

Span is an indicator of data quality

20


Carrier A Carrier B

Carrier C Carrier D

Carrier E Carrier F Carrier G Carrier H

Carrier I Carrier J

Carrier K Carrier L

Carrier M Carrier N

EDI metrics Pickup and delivery event compliance-Span measurement

21

Span is an indicator of data quality


Freight payment matching Comparing OTM match file publications to FPC shipment matches

If it doesn’t get measured, it doesn’t get done …

22


Productivity Monitoring


No rate Planning failed

Total percent planned – Automation vs. manual

24

Fiscal week

Easily view overall % automation vs. manual Shifts in levels are easily detected

Pe

rce

nta

ge

pla

nn

ed

Auto planned


No rate Planning failed

Total percent planned – Automation vs. manual

25

Fiscal week

Clearly view opportunities falling outside of autoplan Trends can quickly be root-caused

Accurate operations cost estimates

Pe

rce

nta

ge

pla

nn

ed

Legitimate exception Auto planned

the importance of otm post-implementation measurement systems

Technology