from data to wisdom professor christopher weare government transformation conference feb. 2015 1
TRANSCRIPT
Raw Data is of Little UseRespondent Quarter Year Office Mode
# of Cust. Served Staff Transaction Time spent
Overall Rating
HelpfulnessRating
AccomplishedGoal?
FormsUnderstandable
1001 Q1 2012 Mail 1577 3 4 1 4 5 51002 Q2 2014 1 Office 1622 9 4 20 4 3 4 51003 Q3 2014 4 Office 1312 9 8 24 2 5 4 51004 Q4 2013 4 Office 1308 12 8 9 5 2 2 51005 Q1 2013 1 Office 1780 12 1 12 2 3 3 51006 Q2 2013 Mail 1127 3 29 4 5 3 21007 Q3 2014 2 Office 1557 10 7 28 3 1 5 51008 Q4 2013 Internet 858 7 6 5 4 3 31009 Q1 2014 Internet 976 2 16 5 1 1 51010 Q2 2011 Mail 976 7 6 4 2 5 51011 Q3 2012 2 Office 872 17 4 19 1 5 5 51012 Q4 2014 Internet 1523 2 3 4 5 4 51013 Q1 2013 Mail 1711 8 27 4 4 2 21014 Q2 2012 Internet 994 3 27 5 4 1 31015 Q3 2012 Mail 1879 5 6 3 4 2 51016 Q4 2014 Internet 1980 2 5 3 1 3 21017 Q1 2013 1 Office 1467 12 4 29 3 5 3 41018 Q2 2014 3 Office 1006 11 1 22 1 3 5 21019 Q3 2012 Mail 1227 6 27 1 5 1 11020 Q4 2012 Mail 860 8 15 5 1 3 51021 Q1 2011 Internet 892 5 23 3 5 2 51022 Q2 2011 Internet 1086 3 27 4 2 1 11023 Q3 2011 Mail 938 6 19 5 4 2 11024 Q4 2012 Mail 1308 7 22 1 1 4 31025 Q1 2012 3 Office 1189 15 2 27 1 3 3 31026 Q2 2014 4 Office 1105 9 7 5 1 3 1 51027 Q3 2012 Mail 1065 5 11 4 4 5 41028 Q4 2011 1 Office 979 14 3 23 5 2 2 41029 Q1 2013 4 Office 1737 12 1 21 1 3 1 51030 Q2 2014 3 Office 1854 11 2 13 2 2 4 51031 Q3 2013 Internet 1756 5 29 5 1 3 31032 Q4 2014 Internet 981 2 26 2 3 2 21033 Q1 2012 3 Office 1017 15 8 23 3 1 4 11034 Q2 2013 Internet 1497 8 22 1 5 3 41035 Q3 2013 1 Office 1239 12 2 9 3 3 4 11036 Q4 2012 Internet 1522 3 22 1 2 3 41037 Q1 2013 Mail 1149 8 30 1 4 3 5
9
The Road to WisdomData
InformationKnowledge
Wisdom Why and how does data further organizational mission
• Do not fear curiosity!• All data has a story to
tell• Look for– Trends– Outliers– Anomalies– Performance Gaps– Performance Examplars
Agency Performance Dashboard • 4th Q 2014 Performance
10%
10%
20%
30%
40%
50%
60%
70%
80%
Applicant Satisfaction 2014 4Q
% Rating Service Good or Excellent
Target: 70%
Target Achieved
Applicant Satisfaction measures on a 5-point scale how satisfied our customers were completing and submitting a service request
All That Data Going to WasteRespondent Quarter Year Office Mode
# of Cust. Served Staff Transaction Time spent
Overall Rating
HelpfulnessRating
AccomplishedGoal?
FormsUnderstandable
1001 Q1 2012 Mail 1577 3 4 1 4 5 51002 Q2 2014 1 Office 1622 9 4 20 4 3 4 51003 Q3 2014 4 Office 1312 9 8 24 2 5 4 51004 Q4 2013 4 Office 1308 12 8 9 5 2 2 51005 Q1 2013 1 Office 1780 12 1 12 2 3 3 51006 Q2 2013 Mail 1127 3 29 4 5 3 21007 Q3 2014 2 Office 1557 10 7 28 3 1 5 51008 Q4 2013 Internet 858 7 6 5 4 3 31009 Q1 2014 Internet 976 2 16 5 1 1 51010 Q2 2011 Mail 976 7 6 4 2 5 51011 Q3 2012 2 Office 872 17 4 19 1 5 5 51012 Q4 2014 Internet 1523 2 3 4 5 4 51013 Q1 2013 Mail 1711 8 27 4 4 2 21014 Q2 2012 Internet 994 3 27 5 4 1 31015 Q3 2012 Mail 1879 5 6 3 4 2 51016 Q4 2014 Internet 1980 2 5 3 1 3 21017 Q1 2013 1 Office 1467 12 4 29 3 5 3 41018 Q2 2014 3 Office 1006 11 1 22 1 3 5 21019 Q3 2012 Mail 1227 6 27 1 5 1 11020 Q4 2012 Mail 860 8 15 5 1 3 51021 Q1 2011 Internet 892 5 23 3 5 2 51022 Q2 2011 Internet 1086 3 27 4 2 1 11023 Q3 2011 Mail 938 6 19 5 4 2 11024 Q4 2012 Mail 1308 7 22 1 1 4 31025 Q1 2012 3 Office 1189 15 2 27 1 3 3 31026 Q2 2014 4 Office 1105 9 7 5 1 3 1 51027 Q3 2012 Mail 1065 5 11 4 4 5 41028 Q4 2011 1 Office 979 14 3 23 5 2 2 41029 Q1 2013 4 Office 1737 12 1 21 1 3 1 51030 Q2 2014 3 Office 1854 11 2 13 2 2 4 51031 Q3 2013 Internet 1756 5 29 5 1 3 31032 Q4 2014 Internet 981 2 26 2 3 2 21033 Q1 2012 3 Office 1017 15 8 23 3 1 4 11034 Q2 2013 Internet 1497 8 22 1 5 3 41035 Q3 2013 1 Office 1239 12 2 9 3 3 4 11036 Q4 2012 Internet 1522 3 22 1 2 3 41037 Q1 2013 Mail 1149 8 30 1 4 3 5
Compare, Compare, Compare• Time• Geographic subunits• Delivery method• Service populations• Administrative units• To similar governments (other states)• To benchmarks
Are we Improving with Time?
Now we have some useful information concerning WHAT is going in our organization
Building Knowledge from Information
• What is behind this decline in customer satisfaction?
• Tell a story . . .
The rise of the internet and changing expectations
• Draw some expectations
Be Skeptical• Don’t believe everything that you think• Don’t believe everything the data says– Is Manager D’s poor ratings due to poor performance? – Or, is it simple bad luck?
• That is a hypothesis test (yes, a little statistics comes in handy)
• Avoid chasing shadows
If Managers Are Not the Root Cause, Then What?
• Tell another story . . .
Budget cuts have hit office staffing very hard compromising their ability to provide top-notch
service• You have data on staffing and customer load• Division is essential analytic tool:
Staffing Appears Increasingly Strained
2011 4Q 2012 1Q 2012 2Q 2012 3Q 2012 4Q 2013 1Q 2013 2Q 2013 3Q 2013 4Q 2014 1Q 2014 2Q 2014 3Q 2014 4Q0
20
40
60
80
100
120
140
160
Average Customers / Staffer 2011-14
Customer Load Decreases Satisfaction
80 90 100 110 120 130 140 1500.4
0.5
0.6
0.7
0.8
0.9
1
Relationship Between Staffing and Customer Sat-isfaction 2011-2014
Customers Served per Staffer
% Rating Service Good or Excellent
Now we have knowledge of HOW declines in satisfaction are coming about.Understanding cause-effect relationships is the foundation for management.
Leadership
• Judicious application of knowledge• Create a culture of curiosity• Build organizational purpose around
data-based knowledge• If you can't measure it, you can't manage it.”
Former NYC Mayor Michael Bloomberg
• Think capacity building not accountability
32
RecapData
Raw and unintelligible KnowledgeInformation that has
been explained
InformationOrdered data that can
be interpreted
WisdomKnowledge promotingorganizational purposeInquire
Graph DataUse all the Data
Compare, CompareSearch for trends &
anomalies
ExplainTell Stories
Check ImplicationsBe SkepticalDig Deeper
LeadManage Judiciously
Promote Analytic CultureLink Org. Purpose to Data
Capacity over Accountability