performance evaluation through control charts

8/10/2019 Performance Evaluation Through Control Charts

1/21

1


2/21

2

A ............................................................................................................................................. 3

............................................................................................................................. 4

B ....................................................................................................................................... 4

A ......................................................................................................................... 4

B : .......................................................................................... 6

A ................................................................................................................... 7

? ................................................................. 7

? ......... 7

: ............................................................................................................................... 8

A ..................................................................................10

C C A ................................................................. 11

C C : .......................................................................................................... 12

................................................................................................................................ 13

C C ................................................................ 15

C C : .......................................................................................................... 16

............................................................................................................................................. 19

C : ........................................................ 19

C : ..................................................... 19

................................................................................. 20

...................................................................................................................................... 21


3/21

3

Organizations need to evaluate the performance of their employees at all levels to maintain a

competitive workforce and provide their employees growth opportunities.

Organizations further segregate the employees into different level of performance standards like

excellent, average or poor performers.

This article focus on use of control charts, a tool used for statistical process control, to create

these performance levels objectively, with the help of data gathered out of a project for a specific

duration.

This article also elaborates how the methodology can be extended for different scenarios and

organization requirements.


4/21

4

The main objectives of this study are:

To objectively evaluate the performance of a group of employees, relatively against

each other using control charts.

To conclude statistically, if a group the employees can be segregated in the groups

like excellent, average and poor (or similar), based on their performances, using

control charts.

To review the reason of the employees lower than average and better than average

performance using control charts

[Mean: Simply stated, it is the average of all observations in a given sample. It is one of

the measures of central tendency. Measures of central tendency provide an idea about the

locations of the observations in the sample and the value about which they cluster.

Variance: The variance measures the scatter of the observations from the mean, the

larger the value, the greater the scatter. This is one of the measures of dispersion. In a given

sample, a variance provides an idea on the variability, or scatter of observations around a

given value, usually the mean.

Standard Deviation: The standard deviation also measures the variability of the

observations around the mean. It is the positive square root of variance. It has the same units

as the observations in the sample and thus is easier to interpret.] REF 2


5/21

5

Control Charts:[Control charts, also known as Shewhart charts (after Walter A.

Shewhart) or process-behavior charts, are among one of the tools for Statistical process

control used to make sure if a manufacturing or business process is in a state of statistical

control.] REF 4

[A control chart is a statistical (graphical) tool for monitoring the movement of a

continuing process. The values of quality characteristic are plotted along the vertical axis and

the horizontal axis represents the samples or subgroups from which this characteristic is

derived.] REF 2

[A control chart consists of:

Points representing a statistic (e.g., a mean, range, proportion) of measurements of a

quality characteristic in samples taken from the process at different times [the data]

The mean of this statistic using all the samples is calculated (e.g., the mean of the

means, mean of the ranges, mean of the proportions). A centre line is drawn at the

value of the mean of the statistic

Upper and lower control limits indicate the threshold at which the process output is

considered statistically 'unlikely' and are drawn typically at 3 standard deviation from

the centre line] REF4

Sample Control Chart:

0

2

4

6

8

10

12

1 2 3 4 5 6

C

C (C)

C (C)

C


6/21

6

[The values of the statistic are plotted on control chart, based on the assumption that the

sample has an approximately normal distribution.

The most common basis for analyzing whether a process is out of control is to check for any

sample statistic falling outside the control limits. The reason of such outliers is analyzed and

corrective actions are planned to bring back the process under control.

The causes of variation can be subdivided in to two groups common causes and special

causes.

Variability caused by special cause is something not inbuilt in the process. That is, it is not a

part of the process and does not affect all quality characteristic of all items.

Variability due to common causes is something inbuilt in the process. It is also referred to as

the accepted variation in a process.] REF 2


7/21

7

[Performance management is the process through which the performance of the human

resources in an organization is identified, measured, managed and developed. Basically

organization tries to conclude how well employees perform and then to ultimately improve

that performance level.

Performance appraisal, on the other hand, is the continuous process of evaluating employees

performance. Performance appraisals are reviews of employee performance for a specified

duration. Appraisal is just one component of performance management.] REF [1]

In software development based IT project, the following attributes can be considered for

evaluating the performance of development team.

1) Number of defects injected

2) Schedule adherence

3) Process adherence

4) Total Efforts (may be in hours)

5) Any customized parameters like points/score given against a set of tasks, in a project

can also be used as a measurable parameter. I have used customized parameter in my

study.

Different roles, like testers, consultants, project managers etc. do have some different and

some common parameters, that are considered for their performance evaluation. Hence


8/21


9/21

9

If we plot the control chart with the available data of employees as mentioned above, it will

give us an idea about how scattered, the data is and if it is possible to form different groups of

employees based on their performance. The main benefit is that the outcome of this exercise

will not be based on a personal hunch but on the objectivity of data.

The groups based on performance levels on the control chart are identified as the area falling

between two straight lines parallel to X axis. One of these lines is generally drawn at mean of

sample and rest may be drawn at multiples of standard deviation from the mean.

Please refer to Figure 3 below. Digits 1, 2, 3, 4 and 5 depict the various regions/areas which

may be considered as different groups (levels) of performance.

Generally, organization utilizes this grouping based on performance (any derived score) for

deciding the yearly increment or bonus which is different across the groups (may be none for

lowest performance group) but same within the groups. The employees in different

performance groups are given different ratings/rankings but same in one group.

E.g. Employees in top group may be given a rating 1 (Excellent), the second group may get 2

(good), 3 (average), 4 (below average-poor) and the bottom group may be 5 (unacceptable).

Above image shows 5 different performance zones.


10/21


11/21

11

Developer Points Achieved MR (Moving Range)

Rahul 140

Nisha 65 75.00

Vinod 65 0.00

Mohit 120 55.00

Madhu 55 65.00

Raman 55 0.00

Shivraj 90 35.00

Puneet 130 40.00

Jyoti 130 0.00Pooja 150 20.00

Rohit 85 65.00

Ismail 25 60.00

Lalit 100 75.00

Param 85 15.00

Sachin 220 135.00

Amitabh 100 120.00

Reetika 85 15.00

Yash 55 30.00

Kapil 65 10.00

Kunal 35 30.00

Vikram 200 165.00

Tej 70 130.00

Anurag 95 25.00

Pankaj 150 55.00

Manu 25 125.00

Abhishek 85 60.00

Ajay 85 0.00

Sanjay 6520.00

Umesh 95 30.00

Shruti 85 10.00


12/21

12

The quality characteristic for above sample is the individual points achieved and

plotted against the control limits.

The variability of the process is estimated from Moving Range (MR) found as the

positive difference of two successive observations.

The quality characteristic/statistic that is plotted against the control limits in a control

chart is Points achieved (first control chart in this article) and Defects Injected

(second control chart in this article) as mentioned above.

The control limits are set by finding the mean of sample and then by determining the

3 sigma values which are then plotted as upper (+3 sigma) and lower (-3 sigma)

limits.

Centre line is the mean of the sample taken.

UCL X bar + 3*MR bar/ d2 130.246

LCL X bar - 3*MR bar/ d2 52.657

MR bar MR/ no. of MR values 50.833

Mean ( X bar)Points Achieved/ No.

of Developers91.452

We are inspecting each team member. Hence the sample size is 1.

The score is a variable type of data and hence calls for variable control chat.

The variable chart used with sample size one is X chart i.e. the values for each

developer (sample) is plotted against the mean and 3 standard deviations values.


13/21

13

The time sequence aspect of sampling is not valid in this case. That means, the samples are

not drawn as in manufacturing industry, one by one. All the data for each developer was

maintained and taken together from the records. Hence Moving Range (MR) chart may not

be completely valid in this case and hence not created here.


14/21

14

050

1

00

1

50

2

00

2

50

A

C

C


15/21

15

Developer Number of

Reports(Sample

Size)

Defects NC per Sample

Size(Defects/Display

Converted)

UCL LCL Final LCL Avg

NC=TotalDefects

(NC)/ TotalWeb

Pages

Tej3 1 0.33 1.02

-0.13

0 0.45

Amit5 1 0.20 0.89

0.00

0

Deepti5 1 0.20 0.89

0.00

0

Preet3 1 0.33 1.02

-

0.130

Nidhi2 4 2.00 1.86

-0.97

0

Neha4 1 0.25 0.95

-0.05

0

Tushar2 0 0.00 0.45

0.45

0

Aman3 2 0.67 1.26

-0.37

0

Naman6 1 0.17 0.85

0.04

0.04

Mohit5 1 0.20 0.89

0.00

0

Suresh4 1 0.25 0.95

-0.05

0

Sanjay4 1 0.25 0.95

-0.05

0

Ajay3 2 0.67 1.26

-0.37

0

Kapil4 1 0.25 0.95

-0.05

0

Puneet5 2 0.40 1.08

-0.19

0

Sunil6 3 0.50 1.15

-0.26

0

Nitin 2 3 1.50 1.67 -0.78 0

Yash7 1 0.14 0.82

0.07

0.07

Shruti3 2 0.67 1.26

-0.37

0

UCJaiswal3 2 0.67 1.26

-0.37

0

Vikas6 4 0.67 1.26

-0.37

0

Pankaj2 1 0.50 1.15

-0.26

0

Anurag4 4 1.00 1.45

-

0.550

Saksham 4 1 0.25 0.95 - 0


16/21


17/21


18/21

18

0

.00

0

.50

1

.00

1

.50

2

.00

2

.50

C

(/C)

C

C

AC=(C)/


19/21

19

The performance of developers has been objectively divided in to 3 performance groups

using control charts.

In both charts, some points (developers) are falling outside (or on) the control limits derived

based on sample data. The causes for the same were analyzed. In case of Points chart (X

chart), points falling below LCL, while in case of Defect chart (U Chart) points falling above

UCL, are of major concern (the below average performances in both cases).

One of the reasons for fewer points is that some of the developers may have taken

more leave than other and hence achieved fewer points.

Another reason can be that some of them just joined the organization and hence

needed more time to adjust to company culture and technology.

Some can really be the low performers.

The developers who work hard and are really good may come on the top. It should

also be analyzed if these developers follow any best practices which can be shared

with the team.


20/21

20

The developers may be new and required further training to enhance his technical

skills.

Developer may not be using the standard checklists.

Can multiple parameters/goals be accommodated in the same chart?

o Yes. Assign different weights to different parameters and derive a weighted

score.

o X chart can be created as the weighted score will be a variable data.

o Calculate the mean, define the UCL/LCL and plot the chart.

Can UCL or LCL be set up by organization explicitly (MBO)? What if all developers

in the sample fall below organization standards?

o Yes. As described in OBJECTIVE section, organization can set UCL/LCL,

preferably (not mandatory) at any multiple of Standard deviation from mean

of data, e.g. 1 sigma, 2 sigma, 3 sigma etc and plot the graph. If the employees

in sample fall below the organization set specification limits, they may all be

grouped in to lower than average/poor performance level.

Can we create more than 3 performance groups?

o Yes. As described in OBJECTIVE section, each straight line drawn at a

multiple of standard deviation, parallel to X axis creates two performance

regions, one above and one below. Hence desired number of such lines should

be plotted to get as many performance groups, as are needed.

Can we apply this methodology to industry other than IT also?

o Yes. Only the parameters or attributes for evaluating the performance should

differ in that case.


21/21

21

1. http://www.sagepub.com/upm-data/45674_8.pdf

2. Fundamentals of Quality Control and Improvement (Second Edition) by AmitavaMitra

3. http://en.wikipedia.org/wiki/Control_chart

performance evaluation through control charts

Documents