Using Data to Bring PeopleTogether

Quality Fair 2010Project and Change Management

Collaborators (PCMC) SessionPeter M. RadcliffeExecutive Director

Office of Planning and AnalysisUniversity of Minnesota

Overview

• Why use data?• Why collaborate?• How can data enhance collaboration?• What are the challenges?• Case study: How has this worked in

practice?• How could you apply this in your own

work?

How do we make good decisions?

Good decisions bring together…• Evidence–What does reality look like?– How do the parts fit together?

• Context– How will stakeholders respond?–What resources are available?–What are our goals?

How do we make bad decisions?

• Evidence is wrong– Data is inaccurate–Model (understanding of what factors lead

to what outcomes) is faulty• Context is wrong– Local culture is different or changing– Resources are unavailable– Contrary to goals of organization

How can data help us?

• More comprehensive than our personalexperience

• More recent than our personalexperience

• Clearer representation of underlyingtrends

• Challenges us to examine our biasesand assumptions

Benefits of collaboration

• Sharing of knowledge and expertise– Not all of the information you need for your

work can be found in your own area– Your team may lack critical skills or

technologies• Improved credibility– Broader collaborations less likely to be

perceived as parochial and self-interested

Challenges to collaboration

• Problem definition–What situation are we trying to change?– Does evidence suggest the problem is real?

• Establishing agreement on outcomes– How do we define success?

• Understanding stakeholders and interests–What does each partner need?– Are interests complimentary, opposed, or

orthogonal

Types and Uses of Data

• Descriptive data–What is the state of the world in this area?

• Metrics–What are our goals and standards of

progress?• Analysis– How do elements connect in this area?– Can show causation or simply correlation

Data, truth, and stories

• Facts may speak for themselves, but theydonʼt say anything interesting on their own

• Data are summaries of a story – they canhelp identify some of the critical points, butthey are not the story themselves

• The full “truth” requires puttingobservations in context

Anecdotes and data

• “The plural of anecdote is data”– Raymond Wolfinger1

• Anecdotes, like data, are summaries ofexperiences which highlight the elements of thestory that are most relevant for the discussion ordeemed most likely to persuade the audience– A data set with a very small “n” (most anecdotes) has

very large error bounds on predictions– A large data set is hard to understand without

statistics to summarize the information, sacrificingsome detail

– Combined, they make a fuller, more engagingargument

1 Nelson W. Polsby. PS, Vol. 17, No. 4. (Autumn, 1984), pp. 778-781. Pg. 779: “Raymond Wolfinger's brilliant aphorism ʻthe plural of anecdote is dataʼ neverinspired a better or more skilled researcher. I e-mailed Wolfinger last year and got the following response from him: "I said 'The plural of anecdote isdata' some time in the 1969-70 academic year while teaching a graduate seminar at Stanford. The occasion was a student's dismissal of a simplefactual statement--by another student or me--as a mere anecdote. The quotation was my rejoinder.“

Trust, Bias, and Integrity

• “Figures won't lie, but liars will figure”– Charles Grosvenor

• Dishonesty with numbers doesnʼt requireoutright fabrication

• Selective use of data sources and definitionscan distort descriptions and analyses

• Best defense is transparency in data sourcesand definitions, and use of institutionalstandards for both whenever possible

• "Grosvenor, Charles H."  The Oxford Dictionary of American Quotation. Hugh Rawson and Margaret Miner. OxfordUniversity Press 2008. Oxford Reference Online. Oxford University Press.  University of Minnesota - TwinCities.  24 January 2010  <http://www.oxfordreference.com/views/ENTRY.html?subview=Main&entry=t251.e757>

Building the relationship

• Collaboration involves intruding on anotherperson or officeʼs business

• Need to understand their goals and needs,as well as your own

• Must demonstrate you are committed totheir success, not just yours

Case Study: Recreational Sports andInstitutional Research

• Recreational sports had information onfacility usage collected from U-Cardsystem

• Had conducted surveys exploring howstudents felt about the facilities and thebenefits students perceived from usingthem

• Lacked ability to show impact oneducational outcomes without assistance

• Approached Institutional Research forhelp

Project Background•Social Interaction•Tintoʼs Theory of Student Departure•Braxton & Hirschy: Communal Potential

Project BackgroundCRF – High Communal Potential

Social InteractionSocial Integration

PersistenceAcademic Success

Analysis and Preliminary Findings•In 2004, Rec Sportsprovided OIR with threeyears of facility usagedata

•OIR connected thatdata with retentioninformation, and found apositive relationship

•Needed a multivariatemodel to rule outspurious relationship

Research Questions

Is CRF usage associated with better academic performanceand increased likelihood of being retained and graduating,above and beyond other important predictors of academicsuccess?

First-term GPA

First-year retention

Graduating within 5 years

Influence of Visits to a CRF onProbability of Returning for a Second

Year

# of First-term Visits to Campus Recreation Facilities

Significance of collaboration

• The story is that campus recreationalfacilities usage is associated with higherretention, even after controlling for manyother factors

• Benefits of partnership– Recreational Sports strengthened their case

for investment in facilities– Institutional Research gained more

understanding of student retention anddemonstrated the value of their modeling inmaking decisions

Roles data played in collaboration

• Initial descriptive data and bivariateanalysis demonstrated showed promise ofeffort and garnered support fromleadership

• Each office needed data and knowledgeonly the other office could provide

• Analysis validated the understanding ofhow facility usage connected to studentsuccess

• Institutional metrics identified outcomes ofinterest

Try this at home!

• What stories do you and your colleaguestell and hear about why things happen inyour area?

• What do those stories tell you about theimportant concepts to measure?

• How could you explore if thoseexperiences can be generalized?

Keep trying…• Who has the information and skills you

need to access and assemble thatinformation?–May be in local systems in your or other

offices, or in institutional records– You may need to invent a way to collect it

• What does your group need to accomplish,and how does that differ from your whatyour collaborators need?

• How would you measure whether each ofyou is succeeding?

Continuing the conversation

• Join the discussion on Moodle– https://moodle.umn.edu/course/view.php?id=11

30–Will post responses to all questions submitted

on the index cards– Ask additional questions on Moodle or

contribute to the discussion

• Contact Peter: radcl002@umn.edu

Appendix

Metrics and Agreement on Goals

• At November 2009 Board of RegentsMeeting, the President, Provost, andExecutive Director of Planning andAnalysis presented a system-wideframework for metrics and key indicators

• Framework provides a structure to whichactivity at all levels of the organization canbe aligned

Goals: Mission and CapacityExtraordinary Education – Recruit, educate, challenge, andgraduate outstanding students who become highly motivatedlifelong learners, leaders, and global citizens.

Breakthrough Research – Explore new ideas andbreakthrough discoveries that address the critical problemsand needs of the state, nation, and world.

Dynamic Outreach and Service – Connect the Universityʼsacademic research and teaching as an engine of positivechange for addressing societyʼs most complex challenges.

Mis

sion

Capa

city

World-Class Faculty and Staff – Engage exceptionalfaculty and staff who are innovative, energetic, and dedicatedto the highest standards of excellence.

ExceptionalStudents

ExceptionalInnovation

ExceptionalO

rganizationExceptional

Faculty and Staff

Outstanding Organization – Be responsible stewards ofresources, focused on service, driven by performance, andknown as the best among peers.

Transforming the U Pillars

26

Strategies and Key Indicators: Education

Recruit highly preparedstudents from diversepopulations

Incoming student preparation

Student diversity

Graduation and retention rates

Placement of graduates

Student engagement

Participation in study abroad andinternational experiences

Student development outcomes (in process)

Student learning outcomes (in process)

Extraordinary Education – Recruit, educate, challenge, andgraduate outstanding students who become highly motivated lifelonglearners, leaders, and global citizens.

Develop lifelong learners,leaders and globalcitizens

Challenge, educate andgraduate students

Goa

lSt

rate

gies

Key

Indi

cato

rs

27

Ensure affordableaccess for students ofall backgrounds

Internal support for scholarships

Average net cost for students

Strategies and Key Indicators: Research

Foster an environment ofcreativity that encouragesevolution of dynamicfields of inquiry

Highly cited research publications

National academy members and otherfaculty awards

Major research awards, research centerawards and centers of excellence

Technology disclosures, licenses andstartups

Breakthrough Research – Explore new ideas and breakthroughdiscoveries that address the critical problems and needs of the state,nation, and world.

Develop innovativestrategies to acceleratethe efficient and effectivetransfer of knowledge forthe public good

Goa

lSt

rate

gies

Key

Indi

cato

rs

Research expenditures and competitiveranking

28

Strategies and Key Indicators: Outreach

Promote and secure theadvancement of the mostchallenged communities

Longitudinal changes in communitieswhere the University is actively engaged(in development)

Active partnerships and assessments ofimpact (in development)

Faculty, staff, and student engagement andcommunity service (in development)

Dynamic Outreach and Service – Connect the Universityʼsacademic research and teaching as an engine of positive change foraddressing societyʼs most complex challenges.

Be a knowledge,information, and humancapital resource for thebetterment of the state,nation, and world

Build communitypartnerships thatenhance the value andimpact of the Universityʼsresearch and teaching

Goa

lSt

rate

gies

Key

Indi

cato

rs

29

Strategies and Key Indicators:Faculty and Staff

Recruit and placetalented and diversefaculty and staff to bestmeet organizationalneeds

Quality of incoming faculty and staff

Faculty and staff diversity

Faculty and staff awards and distinctions

Supervisor and departmental supportsatisfaction

Faculty and staff salary and totalcompensation

Employee engagement index

Employee training and development index (indevelopment)

World-Class Faculty and Staff – Engage exceptional faculty andstaff who are innovative, energetic, and dedicated to the higheststandards of excellence.

Recognize and rewardoutstanding faculty andstaff

Mentor, develop, andtrain faculty and staff tooptimize performance

Goa

lSt

rate

gies

Key

Indi

cato

rs

Engage and retainoutstanding faculty andstaff Employee retention index30

Strategies and Key Indicators: Organization

Ensure the Universityʼsfinancial strength

Bond rating: resources and leverage; liquidityand operating cushion

Private giving and endowment

Carbon footprint

Facilities Condition Needs Index (FCNI)

External awards to units for performance,quality, and innovation (in development)

Research space productivity

Crime and perceptions of safety

Faculty and staff satisfaction with supportservices

Outstanding Organization – Be responsible stewards of resources,focused on service, driven by performance, and known as the bestamong peers.

Be responsible stewards ofresources

Goa

lSt

rate

gies

Key

Indi

cato

rs

Focus on quality service

Foster peer-leadingresearch competitiveness,productivity, and impact

Ensure a safe and secureenvironment for theUniversity community

Promote performance,process improvement, andeffective practice

Research proposals and awards

Workplace injuries

31

Technology commercialization agreements

Integrated Metrics Framework

U-WideStrategies

U-WideKey Indicators

U-WideGoals

32

Unit-LevelGoals

Unit-LevelStrategies

Unit-Level Measures

University-Wide

Unit-Level

Criteria for decision-making

1. Centrality to mission2. Quality, productivity, and impact3. Uniqueness and comparative advantage4. Enhancement of academic synergies5. Demand and resources6. Efficiency and effectiveness7. Development and leveraging of resources

http://www1.umn.edu/systemwide/strategic_positioning/decision.html

Useful URLs

• Institutional Research– http://www.irr.umn.edu/

• Management Reporting– http://www.umreports.umn.edu/

• Enterprise Financial System– https://www1.umn.edu/cco/PeopleSoft/v8/fs.shtml

• Data Warehouse– https://dw.umn.edu/index.asp

More useful URLs

• Accountable to U– http://www.academic.umn.edu/accountability

• OVPR Levels and Trends– http://www.oar.umn.edu/trends/index.cfm

• Office of Measurement Services– http://oms.umn.edu/oms/index.php

• Office of Classroom Management– http://www.classroom.umn.edu