Issues in Performance MeasurementSubmitted by :Manish Sharma(15733002)Saurabh Meena(15733005)

Introduction• According to Shields and Young,( 1989; Cavalluzzo and Ittner,

( 2004) there are 2 types of Performance issues

Organizational Issues

Technical Issues

Organizational Issues• Organizational culture and beliefs

• Top management Support

• Decision-making authority.

• Training

Technical Issues

• Selecting and interpreting performance measures.

• Information system capabilities.

Six Common Issues• Have outcomes that are extremely difficult to measure • Program is one of many contributors to the desired outcome • Results that will not be achieved for many years• The program relates to deterrence or prevention of specific behavior • Program has multiple purposes and finding can be used for a range of

activities• Purpose of the program is administrative or process oriented

1. Outcomes that are extremely difficult to measure • Some programs have outcomes that are very difficult to measure.For Example

Programs designed to address foreign policy objectivesPrograms related to health and safetyTraining programs in companies

• The scope of problem can be defined byBy focusing on why a program is important what makes it difficult to measure

• Reason for difficulty in measuring:Unclear purpose of programUndefined beneficiaryDifference in view of stakeholder and program managersUnavailability of data• This issue can be reexamined by:

Asking WHY?Why it is important that the program receive funding? Why are program operations important? Why does the program do what it does? If the program were fabulously successful, what problem would it solve? •

How would you know?

For Example: to support an international coalition ask• Why is the success of that coalition important?• What role does the program play in achieving that goal?

2. Program is one of many contributors to the desired outcome

• Various levels of organization contribute to achieving same level of organization.

For example- To measure customer satisfaction Director, Department, Manager, Staff need to be involved in program.

Problems-:• Contribution of different level of organization may have various advantages. It

can also lead to disagreement and can prevent from realizing performance synergies & collaboration.

• This happens when various department and employee fail to control or compile their activities to create peak performance for the organization's program.

• Sometimes Organizations call it “Silo Thinking ”.

What to Do….?• To reduce this, its important to allow data & information flow across the

organization.• Programs should be able to develop activity-specific performance goals that

support the broader outcome.• Program may seek to claim responsibility for the entire outcome and output.

Sometimes small role in overall activity.

3. Results that will not be achieved for many years• In some programs the outcome may come after long time• For Example

Hydrogen TechnologyMars mission of NASA and ISROBig Construction Projects

• The scope of problem can be addressed byDefining the specific short- and medium term steps or milestones to

accomplish the long-term outcome goals Steps should be meaningful, measurable, and linked to the outcome goal. For example: NASA defined spacecraft missions, which provide one level of measures to

assess program effectiveness: mission success. Further, within each Mars mission, the program develops technologies; builds, launches, and operates robotic spacecraft; and performs research using the spacecraft instruments

• Tracking process-oriented measures, such as the extent to which programs make decisions based on competitive review

For Example: Research programs can have many uncertainties, including their expected outcomes. Such programs may rely, in part, on process measures, such as the extent to which the program uses merit-based competitive review in making awards.

• Developing measures to reflect meaningful external validation of the quality and value of the program’s research.

• Demonstrating performance in terms of the broad portfolio of the efforts within the program.

For Example: Expert independent evaluators might also help determine if the process of choosing appropriate long-term investments is fair, open and promises higher expected payoffs in exchange for higher levels of risk.

4. The program relates to deterrence or prevention of specific behavior• Reasons for difficulty

Requires consideration of what would happen in the absence of the deterrence program.

Difficult to isolate the impact of the individual program on behavior that may be affected by multiple other factors.

Example of Program: Coast Guard drug interdiction, Department of Labor/Office of Federal Contract Compliance , Nuclear Regulatory Commission’s Inspection and Performance Assessment Program, Safety program for workers in companies,

• If performance measures reflect a continuum from lower-level outputs to higher level outcome measures

Choose measures that are far enough along the continuum that they tie to the ultimate strategic goal as well as to the program’s activity

some measures may create perverse incentives if they do not reach the correct balance between output and outcome

For Example: A useful measure for the Coast Guard drug interdiction program could be the total volume of drugs entering the United States which can be contrasted with drug seizure rate

• Establishing deterrence targets :For some programs, deterring a majority of the negative outcome is

appropriate.For other programs, most, if not all, of the negative outcome must be

avoided.The target should reflect consideration of the maximization of net benefitsFor Example: For programs in which non-compliance is not life-threatening, and for

which compliance is historically low, short-term targets that demonstrate forward progress toward the acceptable long-range goal may make sense

• Programs where failure is not an optionHere negative outcome can be catastrophic (including programs to

prevent terrorism or nuclear accidents) Traditional outcome measurement might lead to an “all-or-nothing”

goalAs long as the negative outcome is prevented, the program might be

considered successful, regardless of the costs incurred.Proxy measures can be used to determine the good functioning of

deterrence process and should be closely tied to the outcome.For Example: Outcome goals for the Nuclear Regulatory Commission (no

nuclear reactor accidents, no deaths from acute radiation exposures from nuclear reactors, no exposure events at reactors, and no radiological sabotage ) and annual (no more than one precursor event per year, no statistically significant adverse industry trends in safety performance, and no overexposures exceeding applicable regulatory limits.) both are necessary for effectiveness of the program.

5. Program has multiple purposes and finding can be used for a range of activities

• Some programs may be designed to address both multiple objective or support a broad range of activities.

For example-:• Outcomes are measurement of work performed that makes a difference to the

organization, in keeping and achieving the strategic organization & department objective.

• Performance Outcomes are more important than work output, outcomes are benefits or changes the results from the work being performed.

Establish performance goals for Block Grant Level

• Many block grant program provide resources to various levels of organization to focus on specific program area like- job training, marketing, working capital, appraisal program and so on.

• System or board could be developed that uses performance measures and standards t promote “joint accountability” for results. With this program target can be set at the department level, and aggregated up to organization target.

6. Purpose of the program is administrative or process oriented

• Many programs in organizations are administrative or process oriented in nature and present no. of problems when it comes to measuring performance.

• Establish criteria to determine the key business process.For Example-: Use the appropriate measurement for impact on customer, sales,

profits and so on. Benchmarking can be used intermediate outcomes like ROI, Sales and son on.

What to Do…?• Ensure that their performance measurement are measuring the right things so

the management can create desired results or outcomes.• Issue must be balanced between output and outcomes.• For larger administrative efforts, consideration can give ultimate outcomes.

Other Issues • Cost of data collection• Assuring appropriate comparisons to other operations• Data quality• Extrapolating from partial coverage• Matching measures to their purposes• Understanding extraneous influences in the data• Conflicts with other measuring programs - which is "right"• Timeliness of data for measures• Use of measures in allocation of funding• Liability for action (or lack thereof) based on measurement results• Responsibility for measures for which there may be limited

control• Benchmarking and targets

THANK YOU