data efficiency in effective communication 6 th annual summit vocational rehabilitation program...

Data efficiency in effective communication

6th Annual Summit Vocational Rehabilitation Program Evaluation and

Quality AssuranceProvidence RI

September 16-17, 2013

Presented by:

• Susan Stoddard, Ph.D., President, InfoUse; RTAC • Scott Sabella, Ph.D., Director, Region 8 TACE • Michael Quinn, Program Evaluation Coordinator,

Alabama Department of Rehabilitation Services• Elizabeth Moody, Strategic Planning Coordinator,

Florida Division of Vocational Rehabilitation

The expanding data challenge

• Expansion of data availability• Better internal data systems• New data analysis and display tools• Advances in data technology• Significant investment in data and systems• How to use data effectively and efficiently?• How to communicate progress and results?

Needed: best data to support…

• Reporting• Evaluating• Planning • Deciding• Auditing• And of course assuring service quality,

outcomes, and continuous improvement• In all of this we rely on our data.

Danger – Information/need mismatch

• Lots of information: 911 and agency MIS; financial transactions ; vendor data; census/ACS and other population information.

• Many internal and external information needs: reporting to RSA and other funding sources; general reporting to the public; internal monitoring and performance assessment, etc.

• Costly, inefficient• How to link these data to agency information needs to

support communication and continuous improvement?

Major data source options (existing)

• Agency data (RSA-911, RSA-2, MIS, case management)

• Public data (census, ACS, disability population, state labor market data, IDEA data)

• Fiscal reports• Case file reviews• Consumer satisfaction surveys• Monitoring reports• State audits

Data Source Options (new)

• Needs assessments (training, client, CRP, stakeholders)

• Surveys (clients, counselors, PWD)• Focus groups• Community hearings• Key informant interviews• Targeted evaluations• Targeted case file reviews

Examples

• Examples of linking the data to agency information needs to support communication and continuous improvement– Alabama– Florida– Utah– CSNAs: Michigan, Iowa, Ohio– Working with SRC

Alabama VRS

Examples of effective data communication using maps

Alabama VRS annual report

Alabama tailored briefing for legislative district

Examples of effective data communication

Florida VR

• External communication method

• Personalized

• VR performance and state population data

• Return on Investment (ROI)

Legislative Profiles

• Available via Sharepoint

• Interactive graphics

• Magic Kingdom themed training

Florida VR Performance Report

• Survey administered every 2 years

• Employee input drives decision-making

• Linkage is evident

Employee Climate Survey + Strategic Plan

Utah State Office of Rehabilitation (USOR)

• Economic Impact Study• Written Benefits Analysis Study

USOR Economic Impact Study

• Relatively simple formula (increase in tax revenue + decrease in public assistance + decrease in Medicaid payments)/state dollars invested

• Multiple uses– Information to agency leadership and staff on impact– State legislature advocacy– State report cards to U.S. Senators and Congressman– National advocacy

USOR Written Benefits Analysis Study

• Are Benefits Planning Assistance and Outreach services associated with improved earnings and employment outcomes• Yes, those who received benefits planning services had

higher wages than the control group• Yes, those who received benefits planning services was

more likely to have a successful closure and more likely to be employed in the quarter after closure

• Utilization• Advocacy for the program• Decision making • Used to support funding benefits planner positions

CSNA examples

• CSNA– Look beyond the “every three year” study…– To what extent is the CSNA integrated into the

continuous feedback and MIS systems? – Ways to make the CSNA investment serve agency

planning and communication needs– What new opportunities need to be created?– CSNA information and formatting… some

possibilities

Michigan CSNA: Outcomes and Attrition

Michigan: Employment rates

Iowa: Outcome rate by impairment, 2007-2010

Iowa: Unique population story

Iowa: Lowest growth rate in US, 1900-2009

5th in % population 65 and over, 2008

Iowa Caseload comparisons

Ohio decreasing referrals, applicants eligibility

Ohio funding context for declines

Ohio: County needs/service gapsTransition

Working with your SRC

• Providing input on outreach to PWD• Providing input on needs for PWD• Indentifying key informants• Developing surveys and communication

strategies• Reviewing and analyzing information• Prioritizing needs• Making recommendations for strategic plan• Making recommendations for the state plan

Parkinson’s Law of Triviality(Bike Shedding)

• “The time spent on any item on an agenda will be in inverse proportion to the sum involved”

• Committee deliberations regarding building an atomic reactor

• Committee deliberations regarding building a bike shed.

Bike Shedding and SRC Example

• Promise grant initiatives ($1,000,000)• Attrition rate ($100,000)• Transportation project ($100,000)• What to serve for lunch at a working lunch

meeting ($600)

• What is the point?

Example: Framing Information for Your SRC?

• Intent: Are you trying to get their input or get items through?

• How much invested in helping them understand complicated issue

• Preservation of language for certain information

• Examples related to daily life

The Important VR Cycles

• State fiscal year• Federal fiscal year• State legislative cycle• CSNA 3 year cycle• State plan submission• Federal reporting (RSA-911, RSA-2)• Federal monitoring

Windows of OpportunityDiscussion

• What are the windows of opportunity for these various cycles?– Preparing for legislative sessions– Federal reporting (program, fiscal)– CSNA– State plan– Federal monitoring

Conclusion: 8 principles to remember

• Know your available data source options and how to use them

• Make use of existing data• Use information for multiple purposes• Data quality: avoid GIGO• Make sure there is a link in process from data to

decision making• Make sure the data gets to the people who can use it,

communicated in a form they understand• Share, borrow and adapt good ideas• Support a continuous improvement system

Contact

• sustoddard@gmail.com

• Scott.Sabella@unco.ecu