assessing customer satisfaction with vocational ...rehabworks.org/rehab/frc_survey09.pdf ·...
TRANSCRIPT
![Page 1: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/1.jpg)
Assessing CustomerSatisfaction with
Vocational Rehabilitation Programs
Florida State University Survey Research LaboratoryCollege of Social Sciences
www.fsu.edu/~survey
Dr. Mary Stutzman, DirectorFSU Survey Research Laboratory
Presentation at the Quarterly Meeting
West Palm Beach , September 14 – 16 2009
![Page 2: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/2.jpg)
2
Why Should the FRC Survey Customers?Provides an “outside” look of customer service.
Customers are sometimes uncomfortable to give feedback directly to the agency giving them services.
Gives customers an opportunity to “speak” to someone outside the Department.
Open-ended comments800 number calls and referrals to FRC
Provides longitudinal data base to assess trends.
Data can be “packaged” for to address specific concerns.
![Page 3: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/3.jpg)
3
Evolution of the Survey Effort
Survey of Customers with Cases ClosedLaunched 2001Questions developed by FRC Evaluation Committee and DVR staffDesigned survey instrument one-page legal front and backSample Design—all cases closed previous month
Customers in Program after 6 monthsLaunched 2002 Questions mirrored Closed Cases SurveySample Design—random sample of 700 per monthCustomers in program after 6 months
![Page 4: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/4.jpg)
4
Customer Surveys OverviewGather information from customers:
Program OutcomesServicesStaff
CLOSED CASES--Surveyed Cases Closed Successful (Status 26)Unsuccessful (Status 80 – 97, 89)
ACTIVE CASES—Surveyed Customers in Program after 6 months
Random sample of 700 per month; 800 beginning July 2008
![Page 5: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/5.jpg)
5
Survey ApproachClosed Cases—Survey all cases closed at the end of month
LetterInstrument• 12 Closed items• 2 Open-ended
Active Cases—Survey sample of 800 cases still active (opened 6 months earlier)
LetterInstrument• 12 Closed items• 2 Open-ended
![Page 6: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/6.jpg)
6
Staffing the Survey Effort
FSU Survey Research LaboratoryAll staff paidStaffed with trained interviewers 7 days a week• can handle PR issues on 800 line and conduct call-
in interviewsMail within 1-week of sample receiptData and open-ended responses entered Analysis of dataProvide reports and presentations on a quarterly and ad hoc basis
![Page 7: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/7.jpg)
7
Design BenefitsSampling Strategy
Power of numbers to analyze by various segments (Areas, Disabilities, etc).Monthly—helps eliminate “seasonal” bias
Response CategoriesAllow “ordinary language” responses for respondentsHelps make reports understandable
OperationsMode choice—mail , Internet, phone800 number and open comments—public relations for Council
ReportingSufficient numbers for Quarterly TrendsSufficient numbers Annual Trends and sub-analyses
![Page 8: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/8.jpg)
8
Respondents are King/Queen
Make convenient2-pages legalKeep short—12 Questions
Accommodate Respondent PreferencesWeb access800 number for phone responseTTY line
![Page 9: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/9.jpg)
9
Privacy Issues: Called ”Project”
![Page 10: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/10.jpg)
10
Closed Cases: Survey Instrument
![Page 11: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/11.jpg)
11
Closed Cass: Survey Instrument
![Page 12: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/12.jpg)
12
Survey Areas
Program OutcomesProgram Choice
ServicesStaff
![Page 13: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/13.jpg)
13
Program Outcomes
Closed Cases
Active Cases
Program OutcomesOverall Satisfaction with Services X XServices Made Life Better X XWould Recommend to A Friend X XEmployment Outcomes and Satisfaction X
![Page 14: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/14.jpg)
14
Program ChoiceProgram Choice Closed
CasesActive Cases
Informed of Alternative Service Providers and Vocational Goals X
Able to Choose Service Providers and Vocational Goals X
Informed of Alternative Service Providers XAble to Choose Service Providers XInformed of Alternative Vocational Goals XAble to Choose Alternative Vocational Goals X
![Page 15: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/15.jpg)
15
Program ServicesProgram Services Closed
CasesActive Cases
Accessibility to VR Offices X XAppropriateness of Services Considering Rehabilitation Goals X
Promptness of Services Provided X XNeed Other Services but Did Not Receive X
Open-Ended Closed Cases
Active Cases
Especially helpful VR associates or services X XWhat VR can do to improve services X XWhy did not get a job X
![Page 16: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/16.jpg)
16
Program Staff
Program Staff Closed Cases
Active Cases
Understand Needs and Feelings X X
Treated with Courtesy and Respect X X
Explained Responsibilities as Client X
Do What Say X
![Page 17: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/17.jpg)
17
CLOSED CASES: APPROACH AND RESPONSE RATE
CUSTOMERS WITH CASES CLOSED: 2007-08 Surveys mailed monthlySurveys mailed to all customers whose cases were closed in the prior month October 2007 to September 2008 mailings18,100 surveys mailedTwo Mailings Analysis based on 5,176 returns
Response Rate = 29%
![Page 18: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/18.jpg)
18
CURRENT CASES: APPROACH AND RESPONSE RATE
CUSTOMERS WITH CASES ACTIVE CASES: 2007-08Surveys mailed monthlySurveys mailed to random sample of 700-800 customers whose case is active 6 months after plan8,700 surveys mailedTwo Mailings Analysis based on 2,441 returnsResponse Rate = 28%
![Page 19: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/19.jpg)
19
Adaptation and Change
Refined and revisited over timeInstrument—”already have a job”Letters—explanation of council and repeated surveysReports—content and format
![Page 20: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/20.jpg)
20
FFY 2007-08
PROGRAM OUTCOMES
Overall Satisfaction --Very or Mostly SatisfiedClosed Cases (Q9) -- 85%Active Cases (Q11) – 79%
Services Made Life Better –Yes, Definitely Helped or Somewhat
Closed Cases (Q8) -- 86%Active Cases (Q10) -- 81%
Recommend to a Friend Yes, Absolutely or Probably Closed Cases (Q10) -- 90%Active Cases (Q12) -- 86%
![Page 21: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/21.jpg)
21
FFY 2007-08
PROGRAM OUTCOMES[Closed Cases - Status 26]
Job Status of Customers (Q12)Nearly one-third of the customers (31%) obtained a job since becoming a clientNearly one-half (60%) already had jobs
Customers (Status 26) who Obtained New Jobs were Satisfied (Q12A)
83% were Very or Mostly Satisfied with their jobs
![Page 22: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/22.jpg)
22
FFY 2007-08
PROGRAM SERVICES
Accessibility Very or Mostly SatisfiedClosed Cases (Q1) – 91%Active Cases (Q1) – 88%
Appropriate ServicesHighly Appropriate or Appropriate
Closed Cases (Q4) -- 88%Active Cases (Q8) -- 86%
Need Other ServicesYes--Definitely or Think So
Closed Cases (Q5)-- 29%
![Page 23: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/23.jpg)
23
FFY 2007-08
PROGRAM STAFF
Understand Needs & FeelingsVery Clearly or Clearly
Closed Cases (Q3) – 87% Active Cases (Q3) – 81%
Promptness Very Promptly or PromptlyClosed Cases (Q7) – 76%Active Cases (Q9) – 67%
Courtesy & Respect Yes (Definitely or Think So)Closed Cases (Q2) – 95%Active Cases (Q2) – 94%
![Page 24: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/24.jpg)
24
FFY 2007-08
PROGRAM STAFF
Do what They Say Will DoYes, definitely-Yes, I think so
Active Cases (Q4) – 84%Explained responsibilities as a clientYes, definitely-Yes, I think so
Active Cases (Q5) – 93%
![Page 25: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/25.jpg)
25
Reporting the Results
![Page 26: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/26.jpg)
26
“Packaging” Results
Design allows for detailed analysesEvolves and changesDifferent products to address different needs
Quarterly Statewide ComparisonsDistrict analysesOpen-ended summariesSpecial Reports
![Page 27: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/27.jpg)
27
Quarterly Comparison
![Page 28: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/28.jpg)
28
District Comparison
![Page 29: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/29.jpg)
29
Quarterly Comparisonover Time
![Page 30: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/30.jpg)
30
Open-Ended Summary
![Page 31: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/31.jpg)
31
Special Report: Open-Ended Detail by Category and Satisfaction
![Page 32: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/32.jpg)
32
Special Report:
Results by Primary Disability by Fiscal Year(2003 through 2005)
![Page 33: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/33.jpg)
33
Major Challenges for Customer Survey Efforts
Participation in SurveysBalancing informational needs of FRC and Agency
![Page 34: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/34.jpg)
34
Response Rates are Declining in the New Millennium
70%
84 03
![Page 35: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/35.jpg)
35
Respondent Behavior Then
![Page 36: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/36.jpg)
36
Respondent Behavior Now
![Page 37: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/37.jpg)
37
What part of “NO” don’t you understand?
COPING WITH REFUSALS
![Page 38: Assessing Customer Satisfaction with Vocational ...rehabworks.org/rehab/FRC_survey09.pdf · Assessing Customer Satisfaction with Vocational Rehabilitation Programs ... • can handle](https://reader031.vdocuments.net/reader031/viewer/2022030213/5aa251c57f8b9ab4208ce204/html5/thumbnails/38.jpg)
38
Balancing Informational Needs
Monitoring dataOperational informationData Packaging—detail vs. overview