benchmarking in economic development programs eric l. rinehart economic development solutions, inc
Post on 19-Dec-2015
218 views
TRANSCRIPT
Benchmarking in Economic Development Programs
Eric L. RinehartEconomic Development Solutions, Inc
Presentation Objectives
Context
Performance Tracking Competitive Benchmarking Report Cards/Scorekeeping
Conclusion Appendices
Presentation Output
To inform and educate ....
Presentation Outcome
To finish my presentation
Before you finish listening
What is Benchmarking?
Current Efforts
Absolute
Relative
Standards
Future Goal
World Class
Past Efforts
How Benchmarking is Practiced Performance Tracking/Monitoring: measuring
the furtherance of organizational objectives
Best Program/Practice Benchmarking: learning best practices of leading performers
Report Cards/Score-Keeping: compilation of indicators tracking community movement to the future
Why Benchmark? - Pressure for Accountability
Public Expectation
improvement in lives return on investment
Organizations best use of effort service to customers
Manager's Operations
efficiency of approach quality of services quantity of services
Oversight Authorities
GPRA: process SEA: elements
Private Sector vs.Public Sector
Needs of Stakeholders: Private: Similarly minded owners/operators Public: Distinct stakeholder perspectives
(good job, right things, actual benefit)
Nature of Results: Private: Direct measurable results in dollars Public: Indirect results; jobs and taxes due to private
investment, sales, innovation, efficiency
Planning Steps Production Function Techniques context Global 'Objective' Summative Eval Formative Evalmission/vision Outcome/Benefit Client Benefit
Public Benefitgoals/scope Output/Results Srvc. Delivered
Plan vs Perform. methodology Process/Activity Quality Mgmt.
Best Practices resources Inputs/Resources Acctng Standard
Performance Tracking Techniques - driven by plans and production
Performance Tracking: Planning
1993 Gov't Performance & Results Act Strategic Planning Performance Plan Program Performance Indicators Rpt
Service Efforts and Accomplishments Specify Measurable Objectives Detail the Strategies Employed Publish Activity Indicators Publish Outcome Indicators
Performance Tracking: Functions
If Inputs: Activities and Resources
And Process: Methods and Procedures
Then Outputs: Direct Results
Then Impact: Expected Outcome
Then Goals: Global Objectives & Strategies
Inputs Level of Resources Accounting Standards
Process Competitive Benchmark Quality Management
Outputs Level of Services Plan vs Performance
Impacts Customer Benefit Public Benefit Cost - Benefit
Performance Tracking: Techniques
Strategic Plan - Agency Goals (& Objectives)
Leadership: Increase the capabilities of individuals involved
Infrastructure: Physical and social infrastructure improvements
Workforce: Develop a skilled and adaptive workforce
Capital: Provide access to capital for business growth and expansion
Technology: Provide access to modern business practices
Market Development: Linkages between customers/ markets
Industry: Retention, expansion and attraction
Performance Plan and Report
Dept. Outcomes/Outputs,for each agency objective
Target Measures,for each outcome area
Plan vs. Actual Report,a baseline amount or percentage and quarterly reports
Tourism Promotion Examples Strategies Domestic and International Advertising
Support Local Marketing Organizations Develop Destination Attractions/Sites
Inputs Advertising Media Purchases Process Tourist/Traveler Inquiry Fulfillment
Visitor Volume at Welcome Centers Outputs Percent of callers traveling in Illinois Outcomes Leisure Traveler Satisfaction
Leisure Travel ExpendituresJobs Created/Retained
Industry Recruitment Examples
Strategies Recruitment/Visit Team Advertising and PromotionState/local Location Incentives
Inputs Accuracy, Currency of DataResponse Time to Inquiries
Process Number of Contacts (Presentations) MadeNumber of Leads (Suspects) Identified
Outputs Number of Businesses AttractedValue of Plant and Equipment Invested
Outcomes Jobs Created/RetainedTaxes Generated
Competitive Benchmarking Process (Best Practices)
Key Results: select critical areas of strategic importanceto the organization
Indicators: select reliable indicators of performance Comparison: compare one's performance to others (or
to standards) using these indicators
Learning: determine where the leaders are and what can be learned by them
Changes: make changes in operations based on what you have learned
Competitive Benchmarking Private Sector Examples
World Class Practices Mgmt, Engineering, Operation
customer responsiveness continuous improvement quality management cycle time reduction cross functional workforce self directed work teams
Performance Measures Productivity, Quality, Efficiency
first pass yields scrap & rework costs customer rejects cycle time customer order lead time on-time deliveries
Benchmarking within Economic Development
Best Practices World Class Practice: Management, Engineering
Operations Practices Performance Measures: Productivity, Quality, &
Efficiency of Performance Best Programs
Formative Analysis: Compare 'architecture' to aid in the design of a program in its early stages.
Summative Evaluation: Compare program results to judge overall worth in relation to alternatives .
Context:- Global Economy- Industry Cluster Focus
Vision:- High Quality Job Creation- Stable Tax Base
Goals:- Business Competitiveness- Community Excellence
Adapted from "Emerging Principles in Economic Development: A Benchmarking Tool", June '95.
Scope:- Public Facilities Capacity- Community Betterment- Business Vitality- Marketplace Capability
Organization: - Regional Collaboration - Partnerships
Process: - Comprehensive Strategy - Strategic, Mission Driven - Customer Orientation - Measurement and Evaluation - Learning
Benchmarking "Principles" of Economic Development
Report Cards - Stated Purposes
the compilation and tracking of 'indicators' to track community progress towards a desired future.
the use of marks or 'indicators' to compare and contrast states or regions.
Report Cards & Score Keeping - Examples
Compendium of Economic Performance Peoria Scope
University of Ill Flash Index Area Report Cards
Business Climate Surveys CfED Report Card of the States
State Progress ReportsOregon BenchmarksMinnesota Milestones
Relevance: (purpose) Measures context only, and not follow-up functions, so they won't help you do your job.
Reliability: (areas for marks) Poor selection of the phenomena to be studied, they are rarely a cogent reflection of society and the economy which is more complex.
Validity: (marks themselves) The use of existing indicators and measures of historical phenomena cause concern that outside or unknown events precipitated the effect noted.
Materiality: (the comparisons) The trend indicators typically used are descriptive and distributive, rather than diagnostic, and therefore are not necessarily informative or enlightening.
Report Cards - Not Ready for Prime Time
Phenomena: State final outcomes expected and therefore to be considered for measurement.
Rationale: Describe the cause and effect model to lay out a logical process to get to the outcomes.
Key Results: Select critical areas of strategic importance. Indicators: Select reliable indicators of performance. Comparison: Compare one's performance against relative
or absolute measures using the indicators. Analysis: Suggest methods for improvement.
Report Cards - Alternative Methods Are Needed
CONCLUSIONS
Embrace 'Benchmarking' Positive Attitudes Key Concept: Relationships
Benchmarking Efforts:Economic Development Application
Best Program Benchmarking: Doing the right things: use formative & summative evaluations.
Performance Monitoring: Doing things right: document procedures & measure their completion.
Report Cards: Right the things done: compile realistic indicators of progress towards the future.
Organizational Environment for Measurement Personal Leadership Openness and Trust
Agreement on Basic Assumptions
Commitment to Evaluation Resources for Measurement Applications of Results
Planning Steps Production Function Techniques market analysis Context Summative Eval Formative Evalmission/vision Outcome/Benefit Client Benefit
Public Benefitgoals/scope Output/Results Srvc. Delivered
Plan vs Perform. methodology Process/Activity Quality Mgmt.
Best Practices resources Inputs Acctng Standard
The Key Concept to Remember - Relationships
Planning Definitions
Strategic Planning Implementation Planning Performance Tracking
Strategic Planning. A systematic process by which an organization identifies and analyzes situations or conditions, formulates conclusions and statements to address the situation, devises alternative strategies, & applies resources to change conditions.
Vision: A value statement in which a preferred state exists or a negative state does not exist.
Mission: Is that which an agency has a calling to do. Approach or Scope: General focus or thrust, the broad limits,
within which planning and programming take place. Goal: The end to which a design tends; its aim. Strategies: Statements of different feasible courses of action
to resolve a situation.
Strategic Planning Definitions
Implementation Planning. The process of determination of a role and definition of objectives; a selection of the programs required to fulfill the role; and the development of time-phased plans for the systematic achievement of those objectives (Harleigh B. Treker, New Understandings of Administration).
Program Activity: A project area as defined by the agency. Objectives: An objective is a specific, measurable event with
standards of achievement that are to be realized. A job or task requiring both time and resources.
Tactics: A job or task requiring both time and resources. Tactics (or activities or services) state 'how', usually in chronological order, a strategy is to be accomplished.
Implementation Plng Definitions
Performance Objective: Target level of performance expressed as a tangible measurable objective.
Output Measure: A defined level of performance of program efforts, specified in quantifiable, measurable terms.
Performance Indicator: Particular value or characteristic used to measure outputs or outcomes.
Baseline of Performance: The initial or starting level of activity or effort, expressed in qualitative or quantitative form.
Plan vs. Performance Measures: A periodic accounting of appropriate program related measurements so as to compare actual program results with established program targets.
Performance Tracking Definitions
Evaluation Techniques
The Organizing Framework Input Evaluation Techniques Process Evaluation Techniques- Numerical Process Evaluation Techniques- Qualitative Output Evaluation Techniques Impact Evaluation Techniques Global Objectives Evaluation Techniques
Inputs: The activities and resources put into an effort (if inputs are adequate, then processes can be performed).
Process: The methods and procedures used to operate the program, (if processes are managed efficiently and effectively, then outputs will be produced).
Outputs: Direct results, often called deliverables or outputs, themselves are not valuable in their own right (if outputs are produced then the purpose will be achieved, assuming the correct outputs have been identified).
Purpose: The expected impact, or outcome, what positive change we expect to occur (if the purpose is achieved, then this will achieve the goal).
The Production Function - The Organizing Framework
INPUT EVALUATION – Inputs are the activities and resources put into an effort (if inputs are available, then some work process can be undertaken). Input measures, such as staffing, work hours dedicated and spending, are routine elements of most budgets.
Level of Resources: Number of staff, amount of resources, number of projects, number of partners.
Accounting Standards: Application of generally accepted accounting principles in the management and control of money.
Employee Involvement: Application and development of the workforce as a critical agency asset.
Input Evaluation Techniques:
PROCESS EVALUATION -- Process measurements consider the methods and procedures used to operate the program, (correctly managed process will produce outputs expected). This type of evaluation helps determine how well the program is operating and identifies opportunities for program improvement.
Performance Goals: Both indicators and a base line to provide immediate information on success or failure. Output indicators define the amount of service units provided, efficiency indicators are used to compute the costs, either in terms of dollars or work hours, per unit of output
Plan versus Performance Measures: Periodic tabulation and comparison of indicators to determine if the program is delivering services at the level expected.
Process Evaluation Techniques - Numerical:
Project Monitoring Tool: A standard operating procedures manual, what business call a business plan, listing legal requirements; administrative (and organizational) qualifications; the customers targeted, the reporting forms to be used; list of services to be performed and so on.
Customer Satisfaction: Survey of how customers perceive the services and their levels of satisfaction in terms of content, format and timeliness.
Competitive Benchmark: Identification of the best practices of each operations (to improve standard operating procedures and achieve economies of scale or scope).
Internal Quality Management: Continuous improvement or standardization of the service or product being delivered.
Process Evaluation Techniques- Qualitative:
OUTPUT EVALUATION -- Outputs or direct results, sometimes called deliverables, are valuable as an intermediate measure (assuming we have identified all the outputs necessary to achieve the purpose)
Project Deliverables: Identify direct products of an operation, i.e., nature of activities; categories of services;
Plan versus Performance: A routine quarterly or annual report on whether planned results are achieved.
Customer Demographics, to identify the types of customer being served and whether the project is serving the customers it should. Demographic profile information such as company size, type of industry, etc. is needed.
Output Evaluation Techniques:
Impact Evaluation Techniques:
IMPACT EVALUATION -- (Purpose) The expected impact, outcome, or results showing what positive change are expected to occur (if the purpose is achieved, then this will contribute to the achievement of the global objectives.)
Customer Benefit: Consider the nature of change or progress within the firms or individuals assisted.
Public Benefit (Impact): How has the program changed lives such as jobs created and retained or new taxes generated. What side effects have occurred?
Cost Benefit: Finally, using REMI or other regional econometric modeling technique, calculate the costs of the program effort in relation to the benefit
Global Objectives Evaluation Techniques:
Formative Analysis: Evaluation which deals with program procedure or quality issues, to determine how to design a program's operations. It is used to fine-tune and improve program strategy and enhance delivery. It requires ongoing input regarding program operation from experts, colleagues and customers.
Summative Evaluation: Looks at whether the program actually made a difference, summing up the overall program usefulness. This type of evaluation requires descriptive questions (what is happening); explanatory questions (why is it happening that way); and evaluative questions (how well is it going).