2008 one award examiner training may 2008 one award today’s session is sponsored by:
TRANSCRIPT
2008 ONE AwardExaminer TrainingMay 2008
ONE Award
Today’s session is sponsored by:
ONE Award
Presenting Sponsors
ONE Award
The ONE Award is aimed at recognizing best practices in non-profit organizations. We seek to identify, promote and reward
our region’s
Organizations of NoteworthyExcellence
ONE Award
Agenda• Overview & Process
• Roles, Responsibilities & Expectations
• Criteria, 2008 Criteria Changes
• Reviewing and Analyzing the Application
• Independent Review
• Consensus Process
• Site Visit Process Overview
• Questions & Comments
ONE Award
Timetable• Applications Due – May 23 at 5:00 PM• Applications Distributed - May 28 • Consensus Review Due – June 23• Site Visit Selection – July 8• Site Visit Training – July 11 or 14 • Site Visits Conducted - July 21- August 1• Site Visit Reports Due - August 15
ONE Award
Timetable (cont.)• Finalists & Winners Determined – late August• Finalists Announced – early September 7 • Award Presentation Luncheon – Oct. 22• Feedback Reports Sent - October 31
ONE Award
Key Roles & Responsibilities
• Examiners
• Selectors
• Award Team
• Sponsors
ONE Award
Examiner Standards• Confidentiality
• Conflict of Interest
• Code of Ethical Conduct
• Signed Statement Required
ONE Award
Understanding the Criteria• Organizational Profile, 4 Categories
• “What” (Approach) questions
• “How” (Deployment) questions
• Objective
• Non-Prescriptive
• Weighted Emphasis on Results
ONE Award
Understanding the Criteria• Organizational Profile• Sets the parameters for examiner review
1. Key Characteristics– Purpose, Vision, Mission, Values
– Programs, Services, Customers, Workforce Profile
– Regulations, Governance, Certification
ONE Award
Understanding the Criteria
2. Competitive Environment– Numbers and types, competitive success factors
Strategic Challenges and Advantages– Mission related, Operations, HR
ONE Award
Understanding the CriteriaCategory 1 - People (150 points)
Leadership, Learning & Motivation
Category 2 - Principles (200 points)
Customer Focus, Governance & Social Responsibility
Category 3 - Process (200 points)
Strategic & Operational Planning, Knowledge Management, Process Management
ONE Award
Understanding the Criteria
Performance Category1. People Results (100 points)
Employee/Volunteer Satisfaction, Learning & Development
2. Principles Results (250 points)
Product/Service Performance, Financial & Market, Governance, Social Responsibility
3. Process Results (100 points)
Operational Performance, Accomplishment of Strategy & Action Plans
ONE Award
Reviewing Smaller Organizations• Size does not affect criteria applicability• Size does provide context (calibrate your
expectations)• Systems likely to be less formal • Don’t have the people or resources to dedicate
to all areas in same way as others• Good governance still an imperative• Ethical behavior is a significant requirement
ONE Award
2008 Criteria Changes
• Italicized in criteria
• Organizational Profile– Key Performance Indicators– Workforce Profile – key segments and their
requirements/expectations– Strategic Challenges/Advantages– Core Competencies
ONE Award
2008 Criteria Changes
• Category 1 - People – 1.1 c – Senior leaders communication– 1.1 e – Organizational improvement, focus
on mission and goals, organizational learning
– 1.3 c – Evaluating training needs
ONE Award
2008 Criteria Changes
• Category 2 - Principles – 2.1 a – Use of voice of the customer to understand
customers’ changing needs
– 2.1 c – Reviewing listening and learning methods to meet changing needs
– 2.2 b – Ensuring that customer access mechanisms meet customers’ needs
– 2.4 b – Evaluation and improvement of governance system – staff and Board, individually and collectively
ONE Award
2008 Criteria Changes• Category 3 - Process
– 3.1 d – Strategic objectives – address strategic advantages and challenges
– 3.2 b – Tracking progress on action plans– 3.3 a – Determining resources needed to
complete action plans– 3.3 c – Emergency/disaster preparation– 3.4 d – Driving process improvement, engaging
workforce members in the process
ONE Award
2008 Criteria Changes
• Category 3 - Process (cont.)– 3.6 a – Determining core competencies,
relationship to creating value for customers– 3.6 b,c – Old 3.7 (2007) combined into 3.6 Key
work processes for creating value, contributing to success and sustainability, supporting value creation
– 3.6 d – Process performance measurement
ONE Award
2008 Criteria Changes
• Category 4 – Performance– 4.7 a – Process performance – key work
processes for creating customer value and support processes
ONE Award
Independent Review
ONE Award
Key Factors Worksheet• Provides the context for individual review• What do they do? • Who are their customers?• Who is their workforce?• What are the key factors (internally and
externally) for the organization?• Key performance indicators• Core competencies, challenges, advantages
ONE Award
Key Factors Worksheet - Sample• Key Characteristics
– Provides mental health counseling to youth/adults in 9 local clinics in 3 county area
– Key customers include youth under 17 years old and their parents/guardians – requirements include transportation, quick access to service, confidentiality, collaborative approach with parents
– Key performance indicators – lives successfully impacted, customer satisfaction, retention through course of treatment, funder loyalty
– 9 member Board, rotating 3 year term, 10 employees, 15 regular clerical volunteers, 300 event volunteers
– Key funders, regulators include…
ONE Award
Key Factors Worksheet - Sample• Competitive Environment
– Competitive success depends upon easy access to services and ability to operate efficiently
– Competes with private practitioners as well as 3 other local agencies who offer similar services
– Competes for funding with numerous small and large local and regional agencies
– Competes with more lucrative private practices for qualified, passionate staff
– Competes with community service agencies for volunteers
ONE Award
Key Factors Worksheet - Sample
• Organizational and Strategic Challenges, Advantages, Core Competencies– Core competencies – individualized counseling
approaches, customer service– Changing requirements from funders place greater
demands on employees– Highly competitive market for qualified counselors– IT systems make information access difficult– Potential county budget cuts threaten survival
ONE Award
Reviewing the Application• Dual customer - applicant and selectors• Bring your knowledge• Not an audit• No advice-giving permitted• Review the Criteria• Review the Glossary of Key Terms• Read the Application completely• Complete the Key Factors Worksheet
ONE Award
Reviewing the Application• Approach – Deployment – Results
• Approach – methods used to address criteria requirements
• Deployment – the extent to which the approach is implemented
• Results – outcomes achieved related to criteria requirements
ONE Award
Reviewing the Application
• Tips for Effective Application Review– Keep the applicant’s Key Factors in mind– Remember the size of the applicant– Don’t assume anything, but give the…– Benefit of the doubt– The applicant wants developmental feedback– Don’t block a winner too soon
ONE Award
Preparing Item Worksheets Item 1.1 - How do senior leaders lead?
Key Process Evidence of systematic approach Evidence of deployment Evidence of systematic process improvement
Setting Vision and Values
Encouraging Frank Two-Way Communication
Senior Leaders – Communicating Key Decisions
Promoting an Environment of Legal and Ethical Behavior
Creating a Sustainable Organization
ONE Award
Preparing Item Worksheets Item 1.1 - How do senior leaders lead?
Key Process Evidence of systematic approach Evidence of deployment Evidence of systematic process improvement
Setting Vision and Values
+ Reviewed and revised by Board
- No indication of timing for review
+ Posted throughout office
+ Shared at staff meetings
- Not clear how Vision and Values shared with volunteers
- No evidence of process review or revision/improvement
Encouraging Frank Two-Way Communication
+ Forums held with employees and volunteers
+ Informal walk-around process, open door policy
+ Forums held four times per year for employees
+ Annual volunteer forum
- Not clear when walk-arounds are conducted or by whom
+ Feedback collected for improvement at forums
- Not clear how feedback used-No evidence of overall review of communications methods
Senior Leaders – Communicating Key Decisions
+ Call ”all organization” meetings as required
- Not clear how volunteers are included + Feedback on communication effectiveness in workforce satisfaction survey
Promoting an Environment of Legal and Ethical Behavior
+ Values include commitment to ethics
- Ethical standards not indicated
-No process to share expectations or reinforce- No indication of sharing expectations to new volunteers
- No evidence of review of approach to promoting desired environment
Creating a Sustainable Organization
Sustainability factors are reviewed yearly with Board
+ Leaders attend futurists conference
+ Leaders meet with employees 2 times each year to discuss key issues
- No evidence of process to review and improve approach to identify and communicate sustainability factors
ONE Award
Scoring Item Worksheets Item 1.1 - Scoring (place an “X” beneath the most appropriate scoring band for both Approach and Deployment)
0-5% 10-25% 30-45% 50-65% 70-85% 90-100%
Approach Not addressed No systematic approach, information is anecdotal
A systematic approach is evident that responds to most of the basic Category requirements
A systematic approach is evident that responds to all requirements of the Category
A mature, effective, systematic approach that responds to all of the requirements of the Category
A mature , effective, systematic approach is evident that is of national role model quality
X
Deployment No deployment is evident
Early stages of deployment
The approach is deployed in most areas
The approach is fully deployed with very few gaps
The approach is fully deployed with no gaps
The approach is fully deployed and has no gaps or weaknesses in any areas
X
ONE Award
Scoring – Categories 1-3Category 1 - Scoring (place an “X” beneath the most appropriate scoring band and choose a percentage score within the band that ends with either 0 or 5)
0-5% 10-25% 30-45% 50-65% 70-85% 90-100%
Approach/Deployment
Not addressed, no deployment is evident
No systematic approach, information is anecdotal. Early stages of deployment
A systematic approach is evident that responds to most of the basic Category requirements. The approach is deployed in most areas
A systematic approach is evident that responds to all requirements of the Category. The approach is fully deployed with very few gaps
A mature, effective, systematic approach that responds to all of the requirements of the Category. The approach is fully deployed with no gaps
A mature , effective, systematic approach is evident that is of national role model quality. The approach is fully deployed and has no gaps or weaknesses in any areas
X
Score for Category 1 - 45% x 150 = 68 points
ONE Award
Preparing Results WorksheetsItem 4.1 – What are your human resource results?
Key Measurement Areas
Key Results Provided/Not Provided
Performance Levels
Performance Trends
Comparative Performance
Indicators of Employee and Volunteers Well-Being, Satisfaction and Dissatisfaction
+ Employee sat.
+ Employee engagement index
- Results not provided for volunteer satisfaction
+/- Employee sat. at 4.5 out of 5
+ 80% of employees engaged in mission – 1st year measured
-Sat. level of 4.5 is constant for past 4 years
- Employee sat. based on survey response rate of 15%
+ Ranks in upper 25% of similar organizations for employee satisfaction
- Response rate for similar organizations is 65%
Indicators of Employee/Volunteer Learning and Development
+ Training hours
+ Conferences attended
+ Certifications achieved
+ 800 hours in 2006
+ 6 people attended 8 conferences
+ 2 employees attained certifications
-Training hours per employee/key volunteers declined each of past 3 years
- Lowest level in 5 years
- No comparative data provided for any measure
ONE Award
Scoring Results Worksheets0-5% 10-25% 30-45% 50-65% 70-85% 90-100%
Performance Levels
No performance results or poor performance
A few results are reported with some good performance in a few results
Good performance levels are reported in many results areas
Good performance levels are reported for most results areas
Good to excellent performance results are reported in most results areas
Excellent, role model performance results are reported in all results areas
X
Performance Trends
No trend data are provided or trends are adverse
Little trend data are reported and some trends are adverse
Early stages of developing trends are evident
No adverse trends exist, no poor performance levels exist for key performance areas
Most strong performance and improvement in performance levels are sustained over time
Excellent improvement trends and consistently excellent performance are reported in most results areas
X
Performance Comparisons
Comparative information is not provided
Little comparative information is provided
Early stages of obtaining comparative information are evident
Some trends and/or current performance levels show good to very good relative performance
Many to most reported trends and current performance levels show areas of leadership and very good relative performance
Evidence of benchmark leadership is demonstrated in most areas
X
Overall Score 0-5% 10-25% 30-45% 50-65% 70-85% 90-100%
Item 4.1
ONE Award
Scoring – Category 4
Scoring Band for People Results – Item 4.1
Overall Score**
0-5% 10-25% 30-45% 50-65% 70-85% 90-100%
X
**(place an “X” beneath the most appropriate scoring band and choose a percentage score within the band that ends with either 0 or 5)
People Results - Score for Item 4.1 - 50% x 250 = 125 points
ONE Award
Consensus Process
ONE Award
Consensus Process• Work in teams• Review worksheets• Develop feedback ready comments based on
worksheets• Score Categories 1-3 and Items 4.1 through 4.7• One scorebook – team score• Submit final feedback report – comments and scores
by June 23
ONE Award
Developing Comments• Strengths
• Opportunities for Improvement (OFIs)
• Criteria-based
• Key factor related
• Complete thoughts, complete sentences
• Actionable, but not prescriptive
ONE Award
Writing Comments• 1-3 sentences per comment
• The applicant is known as “the applicant”
• State in a factual manner
• Make sure comments don’t contradict other area comments
• Don’t comment on how the application is presented or written
• Useful to applicant and selectors
ONE Award
Writing Comments• Comment Format
– Strengths first, followed by OFIs – Refer to item area – 1.1, 2.4. 3.2, 4.3, etc.– Don’t use “would,” “should” or “could”– Professional, polite, positive tone
ONE Award
Developing Comments• Based on independent review worksheets• Categories 1-3 – most important 4-6 comments per
Category• Strengths and Opportunities for Improvement• Possible Comment Areas
– Approach or deployment strength
– Potential role model practice (very important, extremely rare!)
– Gap in approach and/or deployment relative to criteria requirements
– No response to key factor in the Org. Profile
– Example (storytelling) rather than system
ONE Award
Developing Comments• Based on worksheets – strengths and OFIs• 4-6 comments for Item 4.1 (People)• 6-8 comments on Items 4.2 to 4.6 (Principles)• 4-6 comments for Item 4.7 (Process)• Possible Comment Areas
– Favorable/unfavorable performance level and/or trends– Favorable/unfavorable performance in areas that are key to
the organization– Absence of results, comparative data– Favorable/unfavorable comparative performance
ONE Award
Sample Comments• Strength Comment
Item 1.1 – The applicant’s senior leaders and Board set and review organizational vision, mission and values as a component of the annual Strategic Planning Process. Many different communications methods, including the organizational newsletters and monthly staff meetings, are used on an ongoing basis to make sure that these are well understood by all employees and volunteers.
ONE Award
Sample Comments• Strength Comment
Item 2.4 – The applicant ensures fiscal accountability throughout the organization through a series of periodic internal audits conducted by key staff as well as an annual independent financial audit. Audit results are reviewed within 10 days of receipt basis by senior leaders, and the Board reviews these results as part of each quarterly meeting.
ONE Award
Sample Comments• Strength Comment
Item 3.6 – The applicant reviews its core competencies – Service Delivery and Service Design – during the Strategic Planning process and relies on these competencies to strengthen its 3 key service processes – needs assessment, collaborative service design and care delivery. These processes are improved annually utilizing input from customers, employees, volunteers and other key stakeholders, as well as data developed from customer feedback
instruments.
ONE Award
Sample Comments• Strength Comment - Performance
Item 4.1 – The results of the applicant’s annual workforce satisfaction survey indicate that overall employee satisfaction (those giving a score of “5” on a scale of 1 to 5) has increased from 41% in 2003 to 68% in 2007. Satisfaction among volunteers, who play a key role in the applicant’s service delivery process, has risen from 75% to 82% (scores of “5”) in that same time frame.
ONE Award
Sample Comments• Strength Comment - Performance
Item 4.4 – The applicant has reduced dependency on government funding through increased donations and other forms of private support as a source of revenue, addressing a key strategic challenge for the applicant. Private support has risen from 15% of its annual revenue in 2003 to almost 40% in 2007, while growing overall revenue by 30% in that same time period.
ONE Award
Sample Comments• Strength Comment - Performance
Item 4.7 – The applicant has reduced the time it takes to complete the monthly closing of its books from 19 days in 2003 to 9 days in 2007, enabling senior leaders access to key decision data on a more timely basis.
ONE Award
Sample Comments• OFI Comment
Item 1.3 – It is not clear what system or approach, if any, the applicant uses to ensure that knowledge that has been accumulated by long term employees and volunteers is transferred to other employees during their tenure or prior to the potential departure of such knowledgeable workforce members.
ONE Award
Sample Comments• OFI Comment
Item 2.3 – While the applicant expresses that its customers are satisfied, and indicates that its has received numerous letters from satisfied customers, it is not clear what systematic approach the applicant uses to determine the satisfaction of its customers with its services provided on an ongoing basis or how the applicant gathers data on customer experiences to develop actionable data on which to base improvement efforts.
ONE Award
Sample Comments• OFI Comment
Item 3.4 – While the applicant indicates that it conducts monthly performance reviews, both by senior leaders and with its Board, it is not clear how or if the results of those reviews are shared with those individuals responsible for making decisions in the affected performance areas.
ONE Award
Sample Comments• OFI Comment
Item 4.1 – While the applicant indicates that its customer satisfaction results have risen by 300% over the past three years (2004 through 2006), lacking actual data on the specific level of performance achieved, it is difficult to determine whether the applicant is achieving its goal of 95% customer satisfaction.
ONE Award
Sample Comments• OFI Comment
Item 4.6 – While the applicant indicates that it requires that each of its workforce members conduct themselves at the highest levels of ethical behavior, no data are provided to indicate whether the applicant tracks this key performance indicator or whether the applicant is achieving this desired level of performance.
ONE Award
Sample Comments• OFI Comment
Item 4.7 – The applicant indicates that a key factor in its ability to compete effectively in its service is minimization of delivery errors, yet its service delivery results actually reflect an increase in percentage of errors each of the past three years (2005 through 2007) from 6% to 8%. Given such a trend, it is not clear how the applicant will be able to ensure its sustainability in a highly competitive market.
ONE Award
Comments Exercise• Strategic planning is strong
• Applicant states that customers are consistently satisfied
• The applicant’s service delivery process is systematic
• Applicant’s key value creation process successfully create value
ONE Award
Comments Exercise• Item 3.1 -The applicant needs to talk more about
its human resource plans.
• All items - The application does not flow well making it difficult to understand what they are trying to tell me. The use of bullet points rather than complete sentences would be an improvement.
ONE Award
Comments Exercise• Item 4.1 – 4.7 - Applicant indicates results are
strong in most areas• Item 4.1 – 4.7 - Results are not very good for key
factors• Item 4.1 – 4.7 - Comparisons are generally
satisfactory• Item 4.2 -The applicant seems to do pretty well
here
ONE Award
Comments Exercise
• Item 3.1 - The applicant’s strategic planning process is satisfactory, but if the applicant would include more time spent on conducting its SWOT analysis, then the planning process would be a significant strength.
ONE Award
Comments Exercise
• Item 1.1 - The applicant’s leadership team does an outstanding job, and has much experience with meeting organizational challenges. The leadership team really understands its market and its customers so well that they are able to anticipate customer needs and meet them.
ONE Award
Comments Exercise
• Item 3.6 - The applicant’s approach to process management is clearly best practice, and other organizations would benefit by adopting a similar approach.
ONE Award
Comments Exercise• Item 3.4 - In order to improve its performance
review system, the applicant would benefit from adopting a weighted balanced scorecard approach. This would not only make it easier for the applicant to fully understand organizational performance but it would also helps auditors and examiners to better understand their successful practices.
ONE Award
Comments Exercise• Item 1.1 - Leaders in role model organizations
must do an excellent job of walking their talk. The applicant’s leaders seem to do a good job of doing so.
ONE Award
Comments Exercise• Item 3.4 - Comparing data and understanding
performance is critical to improving an organization. If the applicant would designate one individual to spend full time developing a database of reliable and useful comparative data, and build an IT system capable of delivering it to those who need it, they could be a role model organization.
ONE Award
Comments Exercise• Item 3.6 - The applicant works to improve its
processes, but if it adopted a Six Sigma approach, it would really accelerate its improvement. Such improvement would really make funders more willing to support the organization, leading to a higher level of likelihood that the organization would be able to successfully sustain itself.
ONE Award
Comments Exercise• Item 2.4 - The applicant does not tell me enough
about the details of their budgeting process or the internal controls that they have in their accounting system for me to say that it is an effective system.
ONE Award
Comments Exercise• Item 4.1 – 4.7 - Based on the consistent high
levels in many of the applicant’s results, it is clear the applicant has strong leadership, great customer relationships, a strong workforce, and must use a high performing approach to improving its processes (possibly PDSA or another good system). This applicant does great things and is a true role model.
ONE Award
Comments Exercise• Item 1.2 - If I were in charge of the applicant
organization, I would strengthen the human resources management system and increase the investment in training. Otherwise, the applicant will fall behind other organizations in efforts to get maximum performance from their employees and volunteers.
ONE Award
Consensus Process• Team Process
• One Scorebook
• Consensus Comments Only
• Process Tips– Combine and Synergize– Link the Category and Associated Result Area– Develop Team Score Based on Team Scorebook
ONE Award
Site Visit Process• Review Consensus Scorebook• Identify Key Site Visit Issues • Verify and Clarify on Site• Revise Scorebook based on Site Visit Findings• Revise Score based on Site Visit Findings• Customers – Applicant and Selectors• Training week of July 9
