1 session sponsored by the nasact joint middle management conference a systems approach to...

82
1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April 16, 2012

Upload: cecilia-glenn

Post on 28-Dec-2015

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

1

Session Sponsored by the NASACT Joint Middle Management Conference

A Systems Approach to Performance Measurement and Management

Stephen L. MorganApril 16, 2012

Page 2: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

Today’s Presenter

Stephen L. Morgan, President, EGAPP, Inc., and former Austin City Auditor can be reached at [email protected]

2

Page 3: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

3

A Systems Approach to Performance Measurement and Management

I. Introduction – Performance Accountability System

II. Historical Overview – Where We Have Been in Austin and Beyond

III. Performance Planning

IV. Performance Budgeting

V. Performance Measurement and Reporting

VI. Performance-Based Decision Making

VII. Conclusion – What We Have Learned and Where We Are Going

Page 4: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

4

PLANStrategic & Annual

Planning

PLANStrategic & Annual

Planning

DOPerformance

Budgeting

DOPerformance

Budgeting

CHECKPerformance

Measurement & Reporting

CHECKPerformance

Measurement & Reporting

ACTPerformance-Based

Decision Making

ACTPerformance-Based

Decision Making

I. 1. Introduction: Government Performance Accountability System

Page 5: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

5

BUSINESSPLANNING

• Program/Activity Objectives

• Organizational and Individual Performance Measures

• Structural Alignment

PERFORMANCE BUDGETING

• Performance Targets

• Accounting System

PERFORMANCE MEASUREMENT &

REPORTING

• Individual SSPR Evaluations

• Organizational Performance Assessment

• Performance and Measurement Audits

PERFORMANCE-BASED DECISION-MAKING

• Citizens

• Council

• Managers

• Employees

I. 2. Managing for Results FrameworkCity of Austin

Page 6: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

6

Use existing data whenever possible

Find a balance between too few and too many measures

Audit the data regularly

Modify measures when necessary

Centrally located staff to analyze data and coordinate the system elements

Technological infrastructure to support the system

I. 3. Characteristics of a Successful System

Page 7: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

7

Data forms should have space for explanatory information and detail

Tie measures to budgetary allocation and reward system

Support of top management

Over the long run should affect bottom line performance of the organization

Citizens will be better informed and more participative

I. 3. Characteristics of a Successful System (continued)

Page 8: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

8

1992 – Council Resolution on Performance Measurement and Reporting

1994 – First Performance Measurement & Reporting System Audit

1996 – Second Performance Measurement and Reporting System Audit; Program Budgeting implemented

1998 – Third Performance Measurement and Reporting System Audit

II. 1. Where We’ve Been (in the City of Austin) …

Page 9: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

9

1998 Corporate Managing for Results Initiative Defined

1999 Corporate Partnership Implements CMO Initiative

• Simplify our System• Clarify the Information We Provide• Develop Measures that are Meaningful to our

Employees• Focus on Cost

• Developed a Standard manual--The Resource Guide• Trained over 200 managers• Developed a Single Accounting System• Identified Key Performance Measures for Executive

SSPRs• Corporate Review Team

II. 1. Where We’ve Been…

Page 10: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

10

2002 Fourth Audit of the Performance Management System

2003-2008 Continuous Improvement

• Ongoing Integrated System• Information Used for Operational Management• Measures Are Relevant and Reliable• Budgets Are More Data and Results Driven

• Managers and Supervisors Fully Trained• Performance Measures Supported by More Robust

Technology• Improvements Made to City’s Website and

Stakeholder Access to Performance Information• Citizen and Employee Surveys Provide Data for

Selected Performance Measures

II. 1. Where We’ve Been…

Page 11: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

11

2008-Current

• Website Robust with Capacity to “Drill Down” and Search” through Performance Measures Database

• “Managing for Results” Used as Business Planning and Performance Monitoring Model for More than a Decade--Now Part of City Culture

• Performance Report on Website tracks 115 Key Departmental Measures, of these 21 are Designated Citywide Key or “Dashboard” Measures

• Performance Comparisons Presented in Graphics with Goal/Targets and Measures Tracked Over Five Years

• Performance Report for 2009-2010 Received “Certificate of Excellence” from ICMA in October 2010

• Annual Citizen Surveys Strengthened to Include Focus Groups and Presentations to City Council

• “Best Practice Citizen Centric” External Performance Accountability Report Is Needed

II. 1. Where We Are Now…

Page 12: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

II. 2. Federal Government Performance Management System Continues to Evolve

Government Performance and Results Act of 1993 Executive Order 13450-Improving Government

Program Performance, Nov 13, 2007 OMB 10-24: Performance Improvement Guidance

under GPRA for 2011-2012 Government Performance and Results Act

Modernization of 2010 (signed Jan. 4, 2011)

Page 13: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

II. 2. Federal Agencies with Well Developed Performance Management Systems

Social Security Administration Department of Interior Government Accountability Office Nuclear Regulatory Commission Office of Personnel Management

13

Page 14: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

II. 3. Some Local, State, & Provincial Governments Have Established Performance Management Mandates

Page 15: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

II. 3. State and Local Governments With Well Developed Performance Management

States of Florida, Washington, Texas, Missouri, and Oregon (may have been recognized for individual State departments who are mature and excel in developing and applying performance management systems)

Canadian Provinces are a good model Local governments include Austin, King County,

Phoenix, Bellevue, Charlotte, Portland, Palo Alto, and Tallahassee

Auditors have played key roles in many performance measurement and management initiatives

Page 16: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

16

III. Performance PlanningIII. 1. Establishing programs, activities,

and potential performance expectations

III. 2. Developing annual business/performance plans with performance expectations and measures

III. 3. Reviewing business/performance plans to support improvement and accountability

Page 17: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

III. 1. Service Delivery System (Program Model)

Input

Output

Process

CommunityImpact

IntermediateOutcome

Long-termOutcome

Other ContributingFactors

Page 18: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

18

InputsInputs Processes Processes OutputsOutputs OutcomesOutcomes

Service Efforts Service Accomplishments

Financial Inputs/Outputs = Unit Cost

Outputs/Physical Inputs = Productivity

Inputs/Outcomes = Cost Benefit and Cost Effectiveness

III. 1. Service Delivery System Cause/Effect Relationships

Page 19: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

19

III. 1. Service Delivery System: Auditing Program

Auditing Program or Activity

Inputs• Staff• Funding• Equipment• Facilities/Rent

Processes• Audit Process-

(Survey, fieldwork, & reporting)

Outputs• Reports• Briefings• Presentations

Outcomes• Qualitative –

Policy/system/ management improvements

• Quantitative – Cost savings/ revenue enhancement

• Preventive – Deterrence/ detection

Page 20: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

20

III. 1. Program/Activity Mapping Template

Inputs ProcessOutputs (Services

Delivered)Outcomes (Results)

Page 21: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

21

III. 1. Austin’s Definition of Programs

Activity = Input Process Output OutcomeProgram = group of activities with a common

purpose

Example: Auditing program consists of four separate activities:

Performance AuditsInvestigationsConsulting and AssistanceQuick Response

Page 22: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

22

III. 2. Government Performance Expectations

MISSION PERFORMANCE OBJECTIVES/GOALS/TARGETS

INPUT Economy & Sufficiency

PROCESS Efficiency

OUTPUT Effectiveness

OUTCOME Effectiveness

• Financial: Amount, timing

• Physical: Quantity, quality, timing, price

• Capacity vs. demand

• Productivity

• Unit Costs

• Operating ratios

• Quantity

• Quality: Products, delivery

• Timeliness

• Price or cost

• Mission & Outcome Goal Achievement

• Financial Viability

• Cost-Benefit

• Cost-Effectiveness

Page 23: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

III. 2. Performance Measures

Quantitative indicators of the extent to

which performance expectations (goals,

objectives, and targets) are being

achieved. Compare measures to

expectations.

Page 24: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

III. 2. Inputs (Resources)

Resources used to provide a service

Financial and non-financial resources

Dollars (actual and budgeted)

Full-time equivalents

Direct labor hours

Equipment

$, $, $, $, $, $, $, $

Page 25: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

III. 2. Outputs (Services Delivered) Counts of what was done (often ends with “ed”) Units of service or products provided, or numbers of people

served Synonymous with quantity of workload accomplished (not

potential workload) May be conditioned by quality or timeliness (e.g., units

completed on time, that meet quality standards, with satisfactory customer ratings)

Examples: Number of clients served % clients served within 15 minutes Number of lane-miles resurfaced % lane miles resurfaced to high quality condition Number of potholes filled within 48 hours of notification % fires responded to within 6 minutes

Page 26: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

III. 2. OutcomesFocus on: Why the program exists—direct link to general goals How the program is expected to benefit or impact people

or place (the community, city, region, state or province) What citizens or customers can expect to find changed or

accomplished as a result of the program

Examples: % solid waste sent to landfills % residents rating water service “good” to “very good” Number of trainees who get jobs and stay employed for at least six

months Number of fire related deaths per 10,000 residents % lane-miles of State roads in high quality condition

Page 27: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

Relates inputs to outputs or outcomes

Usually expressed as a cost per unit

Input/output Example: Cost per lane mile paved

Input/Outcome Example: Cost per life saved

Also expressed as productivity:

Outputs/Inputs (units produced/labor hours)

Example: Tons of solid waste collected per employee-year

III. 2. Efficiency

Page 28: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

28

Review Team

• Budget Office, Organizational Development, City Management

Structure• Does it provide for alignment of results?• Does it permit illumination of results and cost information

in a manner useful to decision makers?

Results• Do goals/objectives and measures match?• Was template used for best impact?

Measurability• Are goals/objectives measurable?• Are program and activity measures useful?

III. 3. Austin’s Ongoing Effort to Produce High Quality Performance/Business Plans

Page 29: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

29

III. 3. Overview of the Development of Business Plans

Common Purpose

= Activity B Activity Objective

Common Purpose

= Activity C Activity Objective

Common Purpose

= Activity D Activity Objective

Common Purpose

= Activity E Activity Objective

Performance MeasuresResultOutputEfficiencyDemand

Common Purpose

= Activity A Activity

ObjectiveCommon Purpose

= Program Program Objective

Results =Accomplishment ofKey Result AKey Result BKey Result C

Common Purpose

= Program Program Objective

Results =Accomplishment ofKey Result DKey Result E

GOALS

MISSION

ServiceServiceServiceService

Key Result EPerformance

Measure

Key Result CPerformance

Measure

Key Result BPerformance

Measure

Key Result DPerformance

Measure

Key Result APerformance

Measure

Performance MeasuresResultOutputEfficiencyDemand

Performance MeasuresResultOutputEfficiencyDemand

Performance MeasuresResultOutputEfficiencyDemand

Performance MeasuresResultOutputEfficiencyDemand

ServiceServiceServiceService

ServiceServiceServiceService

ServiceServiceServiceService

ServiceServiceServiceServiceService

EnvironmentalScan

ChangeDynamics

ChangeDynamics

EnvironmentalScan

Page 30: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

30

III. 3. Business Plan Alignment Worksheet with Definitions

Business Plan Element Results

City of Austin Vision: We want Austin to be the most livable community in the country.

Vision: (optional) Describes the desired future state or set of circumstances.

Mission: Comprehensive statement of the Department’s purpose. Identifies Department’s primary customers and identifies the products or services that are provided.

Goals: A broad statement describing the desired outcome for an organization or its programs. Defines the significant results to be achieved over the next 2 – 5 years.

Program: Two or more activities grouped together to form a common purpose to define a program.

Program Objective: Clear statement of the objective of the program.

Program Results Measure(s) Key results this program is expected to achieve.

Activity: A set of services with a common purpose that produce outputs and results for customers.

Activity Objective: Clear statement of the purpose of the activity.

Services that comprise the Activity:

A service is a set of actions that produce a product, output, or result directly with or for customers.

Activity Performance Measures:

Results: The impact that an activity has on customers/citizens

Efficiency: Unit cost of an output

Demand: The amount of services requested or expected by customers of the activity

Output: Units of services provided, products provided, or people served through

the activity

Responsible Employee: Department Executive/Manager responsible for Activity

ALIGNMENT WORKSHEET BY ACTIVITY

Page 31: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

31

III. 3. Sample Business Plan Alignment WorksheetBusiness Plan Element Results

City of Austin Vision: We want Austin to be the most livable community in the country.

Vision: (optional) Our community will be the healthiest in the nation.

Mission: The purpose of the Austin/Travis County HHSD is to work in partnership with the community to promote health, safety, and well being.

Goals: The over-all goal of the Austin/Travis County HHSD is to promote a healthy community which reflects social equity. This over-all goal will be achieved through:

1) Minimizing the public’s exposure to health and environmental hazards.

Program: Environmental Health Services

Program Objective: The purpose of the Environmental Health Services is to provide protection and enforcement service to the public in order to minimize environmental health hazards.

Program Results Measure(s) Average response time to complaints/requestsConfirmed cases of food-borne illnessPercent of customers satisfied with complaint/request processing

Activity: Health and Safety Code Compliance

Activity Objective: The purpose of Health and Safety Code Compliance is to provide inspections, investigations, consultations, and training for the public in order to minimize public exposure to food-borne illness and other environmental health hazards.

Services that comprise the Activity: Inspection servicesInvestigation servicesProvide training to food operation employees

Activity Performance Measures: Results: Confirmed cases of food-borne illness

Efficiency:Cost per food establishment permitAverage inspections/investigations per inspectorCost per food manager trained

Output:Number of complaints/requests completedNumber of food establishment, mobile food vendor inspectionsNumber of temporary food inspections

Responsible Employee: Donald Smith

Page 32: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

32

Program and Activity Objectives: MFR Template Performance Measures: A Family of Measures

The purpose of ________________

is to provide___________________

to ___________________________

so they can __________________

Result Measure…thenResult Measure…then• Outputs: How many?

• Efficiency: At what cost?

• Anticipated Demand

III. 3. Plans: Consistent Process & Product

Page 33: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

33

The purpose of the Combat Operations

(program/activity)

is to provide/produce emergency incident response

(service or product)

to anyone in the service area

(customer)

in order to save lives and minimize

property damage

(planned benefit)

III. 3. Program/Activity Objective Statement (example)

Page 34: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

34

Result: Number of fire deaths per capita

Percent of fires confined to the room or area of origin after arrival of AFD (per census track)

Efficiency: Average cost per call

Output: Number of calls (call volume)

Demand: Number of fire alarms (calls)

expected

III. 3. Performance Measures (example)

Page 35: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

www.Auditor Roles.org 35

Exercise 1: Achieving a Balance Between Relevance and Consistency

1. Why must we allow measures to

change?

2. Why must we insist that measures

stay the same?

Page 36: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

36

IV. 1. Link annual plans and budgets

IV. 2. Establish targets

IV. 3. Collect cost accounting information

IV. Performance Budgeting

Page 37: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

37

Ensure clear linkage between the plan’s programs/activities and the budget’s programs/activities

Ensure congruence between the plan’s goals, objectives, and targets and the budget’s goals, objectives, and targets

IV. 1. Link Annual Performance Plans and Budgets

Page 38: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

38

In the Budget Document• Business/Performance Plan • Activity and Program Pages• Performance Measures: definitions, compared

to targets, trends, and costs

Using the Performance Budget to “Tell Your Story”

• Changing the Conversation• This Result…At This Cost

IV. 1. The Budget – Linking Results, $$$, and People

Page 39: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

39

IV. 2. Establish Targets To Compare to Actuals

Targets for each program and activity measure

Sources of criteria for setting targets• Historical trends and baselines• Program requirements or intent• Customer expectations or demands• Industry or sector standards• Benchmarking within the organization• Benchmarking outside the organization

Page 40: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

40

I.V. 2. Examples of Performance Targets and MeasuresModel Component Target (Expectation) Measure (Actual)

Input Economy In FY 2011, decrease the purchasing office’s personnel allocation by five positions.

Number of purchasing office positions deleted in FY 2011.

Process Efficiency In FY 2011, provide vehicle preventive maintenance services at the unit cost $500 or less per vehicle serviced.

Average vehicle preventive maintenance unit costs in FY 2011.

Output Quality (accuracy) In FY 2011, reduce the restaurant critical inspection error rate by 10 percent.

Percentage reduction in the restaurant critical inspection error rate in FY 2011.

Output Quantity In FY 2011, expand “green energy” electrical services to 1000 additional homes and businesses.

Number of additional homes and businesses in FY 2011 receiving “green energy.”

Output Timeliness In FY 2011, all Level 1 emergency calls will be responded to with a unit on site within six minutes.

Response times (range) to Level 1 emergency calls in FY 2011.

Outcome Effectiveness/ customer satisfaction

In FY 2011, increase convention center customer satisfaction rate from 4.5 to 4.7 on a 5.0 scale.

Change in convention center customer satisfaction rate during FY 2011.

Page 41: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

www.AuditorRoles.org 41

IV. 2. Sources of Performance Expectations

The process for identifying expectations and setting targets should be rigorous.

Exercise 2: List the pros and cons of deriving criteria for performance expectations from the sources discussed previously.

Page 42: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

42

Activity-Based Costing (ABC)

Identify Direct and Indirect Costs

IV. 3. Base program budgets on unit costs that support desired program outputs and outcomes as reflected in targets

Page 43: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

www.AuditorRoles.org 43

IV. 3. Performance BudgetingLong-sought “ideal” of budgeting experts:

Performance-driven budgeting.

Best-case reality:Performance-informed budgeting.

Exercise 3: Why might the best-case reality be “better” than the ideal?

Page 44: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

44

V. 1. Individual Performance Appraisal

V. 2. Organizational Performance Assessment and Reporting

V. 3. Performance Audits and Measurement Certification Audits

V. Performance Measurement and Reporting

Page 45: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

45

Every employee in the organization contributes to the City Vision

Every employee in the department contributes to the Mission of thedepartment.

Every employee in the department contributes to at least one Business Plan Goal/Objective.

The Alignment Worksheets show employees how the Services they provide support specific Activities, Programs, and Goals/Objectives in the Business Plan.

Performance Measures show citizens, City Council and employees how well we are doing.

Every Business Plan Measure must be written into at least one employee’s SSPR including departmental executives.

V. 1. Establishing Accountability Key Pointsof Business Plan Alignment/ SSPR Integration

Page 46: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

46

Alignment Worksheet

Mission Program ObjectiveGoalsProgram –Activity –

Services that comprise Activity Activity Objective

Activity Performance Measures Activity Results Measure

Results:

Efficiency: Description of Services

Demand:

Output: Individual Performance Measure• Same as the Activity Performance Measure• Part of the Activity Performance Measure or,• Contributes to the Activity Performance Measure

Employee SSPR

V. 1. Individual Performance Appraisal

Page 47: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

47

Management

Data Component

Analysis Component

Action Component

PerformanceExpectations

Programand Levels

PerformanceIndicators

IntendedUses

Data Collection Data Processing

Measurement of CurrentPerformance Levels

Comparison of Current Performance with Performance Expectations

DecisionsRegarding Results

Decisions ConcerningPrograms and Levels

Decisions ConcerningMonitoring & Evaluation

V. 2. What is a Performance Monitoring System?

Page 48: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

48

Design monitoring system to track and analyze the selected measures (efficiency, outputs, and outcomes are essential). If expectations are not achieved, determine why and take action.

V. 2. Ensure Performance Measure Definitions/Formulas are Established

Page 49: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

49

V. 2. Ensure the Results of Performance Measures are Available for Analysis and Decision Making

Design a reporting system that is easy to use, accessible to all interested parties, and enables management decisions.

Page 50: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

50

Design reporting formats and decide frequency of reporting. Reports may include:

Daily/Weekly Performance Reports

Quarterly Performance Reports

Annual Performance Reports

Citizen Centric Reports

V. 2. Establish Performance Reporting “Best Practices”

Page 51: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

51

Use performance reports to identify and direct analysis of program performance

Use analysis to identify the causes of inadequate program performance and focus improvements on causes

Also use performance reports to identify high performance programs

V. 2. Use Performance Reports to Improve Performance

Page 52: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

52

V. 2.1. City of Austin Performance Report

Departmental Performance Measures• Total of 115 Measures Grouped into

Public Safety, Community Services, Infrastructure, and Utilities/Enterprise Departments

• Each performance graphic includes: Measure Description, Calculation Method, Results, Assessment of Results, Next Steps, and Contact for More Information

Page 53: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

53

V. 2.1. City of Austin Performance Report

Decisions influenced by:

• Stakeholder/citizen priority or demand

• Stakeholder/citizen satisfaction

• Results shown

• City Council and Management priorities

Page 54: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

V 2.2. Tallahassee Citizen Centric Report

We have a responsibility to inform our citizens about:What we are responsible for doingWhere the money comes from that runs the City and

where it goesWhat we have accomplished with monies received and

expended , andWhat challenges face the City moving forward

We believe informed citizens make for better government

54

Page 55: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

V. 2.2. Purpose of the Citizen Centric Report

• To Demonstrate:

– Transparency

– Accountability

• To Promote

– Dialog

– Two way communication

• To Build Trust

– Over time

– One citizen at a time

55

Page 56: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April
Page 57: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April
Page 58: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April
Page 59: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April
Page 60: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

V. 2.2. Issuance of Citizen Centric Reports and Media Coverage

Four Citizen Centric Reports have been issuedReceived front page newspaper coverageReceived television coverageReport page 3 data verified by City AuditorCitizen groups have received reports and have

been requested to provide audit topic suggestions

Reports are available in hard copy and online

60

Page 61: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

61

V. 3. Conduct Performance and Measurement Audits

Audit departmental and program performance

Audit relevance and reliability of performance measures

Page 62: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

www.AuditorRoles.org 62

V.3.1. Auditing Government Performance

Measure or assess performance during an audit or other study based on authoritative auditing standards. (See Austin, Florida OPPAGA, Kansas City, Phoenix on www.AuditorRoles.org)

— Identify the program’s inputs, processes, outputs, and outcomes

— Develop and implement “ad hoc” performance measurement system

— Using performance expectations as “criteria” and measures as “condition,” analyze program performance

— Identify causes of variances and develop audit recommendations

Page 63: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

Available at:www.theiia.org/bookstore

Page 64: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

V.3.2. Self Assess or Audit Performance Measures Using

Asserted Criteria

Relevance—Measures should be aligned, complete, and useful

Reliability—Each measure and its data should be accurate, valid, and consistent

www.AuditorRoles.org 64

Page 65: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

www.AuditorRoles.org 65

Aligned Linked to mission, goals, objectives

Complete Includes essential aspects of performance

Useful TimelyUnderstandableComparableResponsive to changeMeets broad needs of users

Measures should be:

V.3.2. Test relevance or reliability.Assessing the Relevance of Performance Measures: Asserted Criteria

Page 66: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

www.AuditorRoles.org 66

Accurate Computed correctlyNeither overstated nor understatedAppropriately precise

Valid Corresponds to the phenomena reportedCorrectly definedData & calculation comply with definitionUnbiased

Consistent Consistent with previous periodsControlled by adequate systems

Each measure and its data should be:

V.3.2. Test relevance or reliability. Assessing the Reliability of Performance Measures & Data: Asserted Criteria

Page 67: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

67

VI. Performance-Based Decision Making – Includes Stakeholders (especially citizens), Elected Officials, Managers, and Employees

VI. 1. Using performance information to support decision making

VI. 2. Decision making under greater budget constraints

Page 68: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

68

VI. 1. Performance Information Used for Different Decisions

BudgetaryDecision making

Manage & ImproveOperations

AccountabilityReporting

Page 69: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

69

Assess and adjust program performance service levels, and resources,

improve existing programs and services,

improve internal management systems,

revise performance plans and reports,

initiate new programs and services, and

bottom-line—use performance information to support continuous improvement and public accountability.

VI. 1. Performance Information Supports Decisions to:

Page 70: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

www.AuditorRoles.org 70

V. 2. Strategic Performance Budget Decision Model

Performance Results

LOW

GOOD

HIGH Strategic Successin Achieving

Community or Program Outcomes

Useful Contributor to Government Success

Target forIncreased Funding

Target forFunding Cuts

POOR

Use of Strategic Goals and Performance Results inPrince William County Budget Decisions

Stra

tegi

c Im

port

ance

Page 71: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

www.AuditorRoles.org 71

VI. 2. Strategic Performance Budget Decision Model:

In Reality: Varies with Fiscal Environment

Performance Results

LOW

GOOD

HIGH

POOR

Strategic Successin Achieving

Community or Program Outcomes

UsefulContributor to

GovernmentSuccess

Target forIncreased Funding

Target for Funding Cuts

Use of Strategic Goals and Performance Results in Prince William County Budget Decisions

Less funding available

More funding available

Stra

tegi

c Im

port

ance

Page 72: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

www.AuditorRoles.org 72

Exercise 4Brainstorm ways to encourage government managers and policymakers to use performance information when making decisions:

Political leadersPolicy staffLegislators and legislative staffProgram administratorsService providersConsumers of servicesPress and public

Page 73: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

Conclusion

What did Austin learn and where is the City going?

What are some of the “best practices” from various sources?

73

Page 74: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

74

Supporting our Vision ... Creating A Results Orientation:

Services, Activities and Programs Creating Accountability: Measures and

Indicators Creating Integration: Making it Happen

at the Operational Level

VII. 1.1 Conclusion: Creating a Managing for Results Culture for Austin

Page 75: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

75

“Bottom-Up” approach neglected broad performance areas and alignment

•Department key indicators

Accounting structure is a major hurdle

Definition of “services” not clear

Results orientation difficult when template not used effectively

Poor use of template = poor measures

The “not something I control” syndrome

VII. 1.2 Lessons Learned In Austin

Page 76: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

76

Assessing and Improving the Relevance and Reliability of Reported Measures

• Data Collection Infrastructure• Certification Program• Citizen-Centric Report

Providing Further Training to Re-enforce Cultural Shift in All Departments

• Using Information in Management• Using Information in Operations

Passing and Implementing a “Best Practice” Performance Accountability Ordinance

VII. 1.3 Where is Austin going…

Page 77: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

77

VII. 2.1 Performance Management and Accountability: Best Practices Checklist

Obtain active participation by top-level managers and decision makers

Create a clear vision of why and how performance measures will be used internally and externally

Understand the limits of performance measures what they can and cannot do

Sustain organizational commitment over a long period despite barriers and the potential for bad news.

Integrate the performance measurement and reporting system with organizational planning, service delivery, and decision making systems

Page 78: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

VII. 2.2 Performance Management and Accountability: Best Practices Checklist

Through planning align mission, goals, objectives/targets, and measures

Design goals and objectives/targets that specify a single aspect of performance

Design aggressive yet realistic goals and objectives/targets that encourage progress beyond past performance levels

Involve employees, customers, and stakeholders in developing goals, objectives/targets, and measures

78

Page 79: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

VIII. 2.3 Performance Management and Accountability: Best Practices Checklist

Identify programs and activities to be measured and define them through and input-process-output-outcome model.

Design a “family of measures” for each program which provides key information to support decisions

Periodically evaluate current performance measure; change when needed but try for comparability over time

79

Page 80: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

VII. 2.4 Performance Management and Accountability: Best Practices Checklist

Define each measure and identify data sources and data collection procedures

Produce performance information (including explanatory information) which is clear and useful to all stakeholders (customize)

Educate, encourage, and reward managers for using performance information to make decisions which improve program management and service delivery

80

Page 81: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

Training/Assistance to Get You From Here to There

EGAPP, Inc. provides training in all aspects of performance management and auditing. (Brochure Available)

Auditor Roles Project provides training in assessing/auditing performance management systems and measures. Assistance can also be arranged.

Email to Steve: [email protected]

Page 82: 1 Session Sponsored by the NASACT Joint Middle Management Conference A Systems Approach to Performance Measurement and Management Stephen L. Morgan April

Thank You.

More questions.

More comments.

Thank you, again.

82