measuring team productivity

7/27/2019 Measuring Team Productivity

1/27

Nashville

Measure what is measurable and make measurable what is not so

Galileo Galilei(1564-1642)

O P M

Measuring Team

ProductivityOregon

ProductivityMatrix

using

specifically for Support teams

An experience paper on improving PPQA

team productivity by

Srinath RamaseshWipro Quality Consulting Group


2/27

2

O P M

Fundamentals| The Oregon Productivity Matrix (OPM in short) takes multiple

metrics, uses a eight step process* and subjective weightingsystem to rank/prioritize the metrics. The metrics are thenadded up to give a cumulative composite score

| This single index indicates how well thecompany/organization/team is doing against the performancemetrics

*as we implemented it.


4/274

O P M

Preamble| Productivity is a major concern in an IT industry, especially so for

support teams (or non-coding jobs)z PPQAs (Process and product Quality Assurance)

z Project Management Office

z Administration

| Productivity by definitionz "Goods and services produced per unit of labor and expense

orz The relationship between the inputs required to produce a product or

service, and the value of the output produced

z In a software development activity, this is relatively easy to calculate. We

could measure code productivity as Function pointsor lines of codedeveloped per unit effort or cost

Goods + Services

Labor + Expenses


5/275

O P M

Productivity of Support

Teams| Inputs are fairly self-evident: staff, equipment, costs, etc

z but what of the outputs?

| What exactly do support teams produce?z Multiple deliverables,

z Non periodic deliverables,

z Multiple tasks,

| Measured by multiple parameters

| Example:z An accounts department, for instance, will produce a variety of reports, some

regular and some adhoc They work on several tasks simultaneously, withconstant interruptions, and its impossible to apportion the total input to each

Can Productivity for a support team be clearly established?

Is it possible to measure the value driven by all these activities and deliverables?Is it possible to measure improvement collectively?


6/276

O P M

Its Possible!!!| The productivity of support teams can be measured!!!,

z Changes up and down can be monitored allowing managerialaction to be taken to ensure that improvement takes place

The Hypothesis:Anything that increases the quantity and/or quality of output, without a

corresponding increase in the utilization of resources, creates a gain inproductivity *

Oregon

ProductivityMatrix

Using

Goods + Services

Labor + Expenses

Volume + Value

Efficient Resource Consumption

This presentation covers an experience of implementing a research basedmethodology called Oregon Productivity Matrix with a Process and Product QualityAssurance (PPQA) team in a leading automobile company in North America

=

*Productivity by the Objectives Matrix Oregon Productivity Center


7/277

O P M

Background| The PPQA group at an IT division of a leading Automobile company

assisted projects in process verification and validation activities

| The teams key activities includedz Process validation,

z Issue of phase approvals,

z

Conducting configuration management audits,z Reviewing work products,

z Mentoring projects,

z Facilitating root cause analysis meetings with Project teams

| The Challenge:z With multiple type of activities and multiple measurement parameters, the

teams challenge lay ahead in identifying a single indicator for measuringteams productivity, showing continual improvement, and establishing aperformance measurement system

QA


8/278

O P M

The Solution| The team agreed that the solution was to establish a concrete method

for measuring the PPQA teams productivity, particularly as it relates tothe team's continuous improvement indicators

| Based on a thorough literature search we evaluated that OregonProductivity Matrix (OPM) best fits this scenario

| In addition, we had experiences in the team of successfullyimplementing OPM in an operations management area to indicate asingle performance indicator of all operations


9/279

O P M

OPM History| Developed by the Oregon Productivity Center at Oregon State University in 1986,

(Productivity By the Objectives Matrix (OMX) Handbook-1986)z the OPM was first introduced at Boeing with a pilot team in 1990(Defense and Space

facility, Corinth)

| Similar to Balanced score card* with an exception that the OPM can beimplemented at a shop floor level without getting into strategic perspectives of theorganization

| Has been applied in various industries. Is immensely versatile and adaptable to

various applications in any setting like:z Hardware Network productivity ITz Office productivity (administration)- Servicez Sales productivity - Fabricationz Production control- Food Processingz Purchase-Fabricationz Accounting -Electronics

z Human Resources Bottlingz Planning and Development- Governmentz Police Governmentz Team productivity- Aircraftz Foreclosures- Mortgage Company

*Balanced Scorecard is method developed by Kaplan and Norton that enables organizations to clarifytheir vision and strategy and translate them into action


10/2710

O P M

Advantages of OPM| Flexible

z Goals can be agreed to be adjusted based on continuous improvementz Allows for multiple measurement categories where baseline and current

performance levels are depicted

| Standardizedz A valuable method for standardizing team performance by the use of an index

that can be uniformly interpreted throughout the organization

| Versatilez Can be used in a variety of work environments to establish a performance

measurement system

| Prioritizesz Weights are established for each of the performance areas, thus focusing the

team on the most important areas to improve

| Team orientedz Goals are set by the teams

z Teams are the users and take ownership| Rolled-Up

z May be rolled up to an organizational level with multiple OPMs

| Visualz OPM graphically displays team performance against goals


11/2711

O P M

The OPM GridCategories Point Score Goal

10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00

Goal

Limits Point Weight Total

Performance

Category 1 100.0 99 98 97 96 95 93 91 87 83 82 94% 25

Performance

Category 2 100.0 98 96 94 92 90 88 86 84 82 80 100% 15

Performance

Category 3 100.0 99.5 99 98 97 96 94 92 90 88 85 98% 10

Performance

Category 4 100.0 99 98 98 97 96 95 94 93 92 90 97% 30

PerformanceCategory 5 100.0 99.5 99.0 98.5 98 97 96 95 94 92 90 98% 20

Point ScoreTable

4

Weights3

Goal Limits2

PerformanceCategory

1OPM may be seen as a grid in which the differentPerformance categories(which are monitored using quantitative metrics data) are depicted vertically

along the grid. Goal Limits are identified for each category. Weights each ofthe Performance Category are established to identify importance. A PointScore table is used to earn pointsof performance.


12/27

12

O P M

When Rubber Hits the

Road

(Step 8)Analysis

(Step 1)

Identify Team

Goals

(Step 2)

Define &Validate

Performancecategories

(Step 3)

Set Performance

Goal Limits

(Step 6)

Identify Scores for

Actual Performance

(Step 7)

Arrive at OPM

Index

OPM 2

3

4

5

6

7

8

1

(Step 5)

Establish Performance

indicators(Step 4)

Priori tize Categories

by assigning weights

8


13/27

13

O P M

OPM 2

3

4

5

6

7 1

Identify Team Goals| Identification of the team goals is the first activity towards establishing

any performance measurement systemz Understand expectations from managementz Align the goals with Business goals and objectivesz Brainstorm with team

| What are we trying to achieve?

G1

Achieve consistency in PPQA team performance

G2 Change perception of PPQA from inspectors to mentors

G3 Improve process compliance performance in projects

8


14/27

14

O P M

OPM 2

3

4

5

6

7 1

Define and Validate

Performance Categories| Use the Team goals to arrive at

performance categories for

monitoring| The PPQA team used the Goal

Question Metric (GQM) paradigm * asa method to identify the measurementcategories or performancecategories

| Identify Performance categories thatindicate what the team does

A family of measures thus created can reflect the work of the team in a way no single measure can.

*GQM method was originated by V.Basili and D.Weis

G1 G2 G3

Q1 Q2 Q3 Q4 Q5 Q6 Q7

PerformanceCategory1(Metric1)

PerformanceCategory 2(Metric 2)





Team Goals

Questions derived from goals thatmust be answered in order todetermine if the goals are achieved

Measurement Categories that provide

the most appropriate information foranswering the questions

8


15/27

15

O P M

OPM 2

3

4

5

6

7 1

Define and Validate

Performance Categories1 Performance

Category

Performance

Categories*

Metric

Phase approvals On-Time

Number of approvals issued ontime/Total number ofapprovals issued *100

CM Audits On-Time Number of CM Audits conductedon-time/Total number of CMAudits Planned *100

QA Report Quality % Compliance score based onDefects found from review ofQA Reports in a month

ImprovementEffectiveness

Overall Score Improvement

Index: Total compliancequestions from all

projects/Total applicablequestions in month for allprojects expressed in %age

Meeting Compliance Number of Meetings conducted inthe month /Scheduledmeetings*100

| Validate that these categoriesencompass the range of inputs

used: labor, materials, capital andequipment, and the output factors ofquantity, quality, timeliness and cost

| Validate that each performance

category is positive driven and ismeasured in percentage

z i.e. each metric is measured forcomplianceand not defects or noncompliance

*Note: these are samples and do not cover all categories for a PPQA function

8


16/27

16

O P M

OPM 2

3

4

5

6

7 1

Set Goal Limits1 Performance

Category

2 Goal Limits

Performance Categories

Goal

Limits

Phase approvals On-Time 94%

CM Audits On-Time 100%

PPQA verification Run Quality 98%

Improvement Effectiveness 97%

Meeting Compliance 100%

| Assess Current level of Performance

z Historical data to generate the initial

OPMz Conduct pilot

| Review and analyze with team

| Establish the goal limits for eachperformance category with team

consensus

This step is useful in itself, as people and work groups should have targets for quality of service so that they know the

standards of work that they are expected to achieve.

7

8

1


17/27

17

O P M

OPM 2

3

4

5

6

7 1

Prioritize the Categories| Assigning importance weights (iw)

for each Performance Category based

on the value of the output summing upto 100

| How to assign weights for aperformance Category?

z Take expert judgment based on

experiences and alignment to businessobjective

z Take an simple average based on asurvey

z May be jointly established with theteam and management representativesto ensure alignment to the Businessvalue.

Performance Categories Weight

Phase approvals On-Time 25

CM Audits On-Time 15

PPQA verification Run Quality10

Improvement Effectiveness30

Meeting Compliance 20

100

(Analysis of the following for each category providesthe key to decide the weight )

o Who uses the services or deliverable ?o What is value of this delivery?

o How is performance of each of the services/deliverable affected?

o What is impact of wrong service or deliverable?

1 PerformanceCategory

2 Goal Limits

3 Weights

Weighing by ExpertiseWeighing by CompromiseWeighing by Consensus

Aligning the teams service more closely to business goals and objectives will promote significant productivity gains by

eliminating unnecessary work.

7

8

1


18/27

18

O P M

OPM 2

3

4

5

6

7 1


IndicatorsPoint Score

10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00

100.0 99 98 97 96 95 93 91 87 83 82

100.0 98 96 94 92 90 88 86 84 82 80

100.0 99.5 99 98 97 96 94 92 90 88 85

100.0 99 98 98 97 96 95 94 93 92 90

100.0 99.5 99.0 98.5 98 97 96 95 94 92 90

Point ScoreTable

4

Weights3

Goal Limits2

PerformanceCategory

1

| May be achieved by establishing a point score table

| Point score table as seen in the figure is a range of values fragmented horizontallyin varying percentage fragments between two limits and each percentage indicatinga certain pointearned in a scale of 0 and 10

| The lower limit of this target range is the minimum level of service that can bepermitted or expected called as the minimum expected performance score(meps). The maximum range or the upper limit is fixed at 100% (Optionally, themaximum scale can be the goal score also)

| Intermediate values are determined for scores between these ranges based on past

experiences and adjusted based on pilots

The targets goals levels may be set for each performance area either in terms of in terms of percentage or point

score values.

7

8

1


19/27

19

O P M


Indicators

OPM 2

3

4

5

6

7 1

Categories

10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00

Goal

Limits Point Weight Total

Phase approvals On Time 100.0 99 98 97 96 94 93 91 87 83 82 94% 5 5 25

CM Audits On-Time 100.0 98 96 94 92 90 88 86 84 82 80 100% 10 5 50

PPQA Verification r un Quality 100.0 99.5 99 98 97 96 94 92 90 88 85 98% 7 10 70

Improvement Effectiveness 100.0 99 97 96 95 93 91 89 86 83 80 97% 6 35 210

Meeting Compliance 100.0 99.5 99.0 98.5 98 97 96 95 94 92 90 98% 6 20 120

Performance Category 100.0 98 97 96 95 94 93 91 89 87 85 95% 6 1 6



Performance Category 100.0 99.5 99 98 97 96 95 94 93 92 90 97% 6 20 120

meps 100 625

(iw)

Point Score Goal

GOAL

INDEX

| Establish a grid with performance categories, point score table goal limits and assigned weights

| The goal limits of each performance category corresponds to a point score. The product of point scoreand weight gives the total goal scores.

E.g. :z in the first row: 94% goal limit corresponds to 5pointsz Point score (5) X assigned weight (5) = 25 ( Total goal limit score for category Phase approvals On time)

| The sum of point scores based on the individual goal limits gives a single number that is an aggregate,weighted index called Goal Index

7

8

1

7

8

1


20/27

20

O P M

OPM 26

3

4

5

7 1

OPM 26

3

4

5

7 1

Record Actual Scores

| The actual measurement scores for each of the performance categories are recorded andtranslated as pointsearned from the point score table

z This was done using a specific reporting and consolidation template in a spreadsheet designed to do thenumber crunching of all the measurement attributes and arrive this table.

| The Performance score for each performance category is then a weighted score of the point

| The sum of all performance category point scores gives a single number that is an aggregate,weighted index called OPM index

E.g. :z

in the first row: 83% Actual score of Phase approvals On-Time corresponds to 1 point earned from the

OPMINDEX

Ver 4

Categories

10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 GoalLimits Point Weight Total Actual Poin ts Weight

Perf

Score

Max

Possible

Phase approvals On Time 100.0 99 98 97 96 94 93 91 87 83 82 94% 5 5 25 83% 1 5 5 50

CM Audits On-Time 100.0 98 96 94 92 90 88 86 84 82 80 100% 10 5 50 98% 9 5 45 50

PPQA Verification run Quality 100.0 99.5 99 98 97 96 94 92 90 88 85 98% 7 10 70 97% 6 10 60 100

Improvement Effectiveness 100.0 99 97 96 95 93 91 89 86 83 80 97% 6 35 210 95% 4 35 140 350

Meeting Compliance 100.0 99.5 99.0 98.5 98 97 96 95 94 92 90 98% 6 20 120 91% 1 20 20 200

Performance Category 100.0 98 97 96 95 94 93 91 89 87 85 95% 6 1 6 97% 8 1 8 10



Performance Category 100.0 99.5 99 98 97 96 95 94 93 92 90 97% 6 20 120 88% 0 20 0 200

meps 100 625 100 308 1000(iw)

Actual

OPM Measurement ReportMonth1

Point Score Goal

Performance ScoreActual Scores

tablez Point score (1) X assigned weight (5) = 5 (Total performance score for the category Phase approvals On

time)z Sum of all performance scores =308 = OPM Index for the Month 1

7

8

1


21/27

21

O P M

OPM 2

3

4

5

6

The Moment of Truth

PPQA OPM Dashboard

395 410

485

415

515 515

635 645 650685 685

318285

308

0

100

200

300

400500

600

700

800

9001000

Oct-04 Nov-

04

Dec-

04

Jan-05 Feb-05 Mar-

05

Ap r-05 May-

05

Jun-05 Jul-05 Aug-

05

Sep-

05

Oct-05 Nov-

05

Dec-

05

GOAL LINE

STRETCH GOAL LINE

Calibrate(initial)

Calibrate(yearly)

(Optional)

| The OPM index trend indicates the Productivity line and Goal index gives theGoal Line.

| The OPM indices (productivity line) may be compared to the Goal line tomeasure performance and demonstrate improvement in productivity

7

8

1


22/27

22

O P M

OPM 2

3

4

5

6

Analyze OPM

| Initial ( Pilot Analysis)z Calibrate OPM based on pilot

z Adjust goal limits, point scores or weights

| Monthlyz Team analyses to measure productivity improvements

z Causal Analysis and discussion on improvement areas

z Elimination of assignable causes

| Yearlyz Root cause analysis to identify process improvements

z Calibration and Program corrections

Questions/Symptoms Actions Recommended

Change in scope of activities? Perform GQM

Do we have the right measures? Review performance categories

Performance goal is consistently not achieved Review Goal limits

Consistent 100-percent-of-goal performance on aperformance category

(or becomes the current level of performance)

Review Goal Limits, Point Score, Weights

Re evaluate performance category


23/27

23

O P M Avenues to Improving OPM

| OPM is a process completely driven by the team from selecting the performancemeasurement areas to actually improving them. Implementation successes may

be replicated at other teams in the organization

| The point score table may be embellished with more objective sub- criteria ofperformance (0-10 levels)

| It is possible to combine all the team OPMs into a single OPM for the entireorganization

| Once piloted and proven, build it as a part of your quality system


24/27

24

O P M Lessons Learned

| Increases team ownership for improvement (Process, Product, Productivity)

| Leave it to the practicing team to analyze and improve

| Initially coming to consensus with goals and getting acceptance from all teammembers may be slow, but its worth the time. The process accelerates upon teamagreement and involvement

| Use it as a process to improve the performance areas and not as a end in itself

| Be open to continuous improvement based on root cause analysis

| Productivity is better measured by overall value delivered per unit effort/cost thanby quantity delivered per unit effort/cost


25/27

25

O P M CAUTION !

| DO NOT use OPM to tie the results to any form of reward

system (or worse penalizesystem)z This does not yield good results in long term

z The team may not necessarily focus on the right performancecategories for improvement

| DO NOT use OPM to foster competition amongst teams

z The objective is to improvenot to prove

|

DO NOT use the tool to identify whois the problemz Let the team decide where is the problem and improve it


26/27

26

O P M References

| OSU Archives - Oregon Productivity Center Records (RG 219)

z OMX Handbook

| Productivity by Objectives : By J ames L. Riggs, Glenn H. Felix

| The Use of Performance Metrics as a Strategic Management Tool -Terry R. Collins, Manuel D. Rossetti and J ulie A. Watson paper,University of Arkansas

| Rose, K. (1995) A Performance Measurement Model.. Quality Progress,V28, N2, 63-6

| R.Solingen, E.Berghout: "The Goal/Question/Metric Method" McGraw-Hill Publishing Company, 1999


27/27

O P M

The primary objective of any process improvement activity is to improveand not just prove

Thank you

[email protected]

586-354-4813
mailto:[email protected]:[email protected]

measuring team productivity

Documents