we now have data. what do we do next?. data... “policy and practice” dr. doug christensen...

We now have data.What do we do next?

DATA . . .“Policy and Practice”

Dr. Doug Christensen

Commissioner, Nebraska Department of Education

We Have . . .

• Created a “lofty” place for data

• Making data a “commodity”

– systems are being designed to “create” data

• Data is now a product– can be packaged– can be marketed– can be sold

Commoditizing Data

• Creates new language . . . New questions

• “Data driven” . . .

• What does the data say?

Few Paying Attention To

• Credability of data

• Accuracy of data

• Prinicples of data collection and use

•

•

“He who has the data, rules . . . the _________”

Wheatley• “. . . We increasingly depend on numbers to

know how we are doing for virtually everything.”

• “. . . The measures define what is meaningful rather than letting the greater meaning of the work define the measures.”

• “As the focus narrows, people disconnect from any larger purpose and only do what is required of them.”

Wheatley

• “. . . Dethrone measurement from its godly position, . . .”

• “. . . Offer measurement a new job – that of helpful servant.”

Margaret Wheatley (1999)

The purpose of data is information.

. . . To “inform”

Data informs . . .

• Does not “tell”

• Does not “conclude”

• Does not “drive”

Inform

• The public . . . about value of the public enterprise (how well it is doing)

• The educators . . . to energize and inform the process of continuous improvement

Three Dimensions of Accountability

• Collecting appropriate, valid and useful data

• Reporting data in understandable ways to our public and stakeholders

• Using the data to inform continuous improvement

Accountability is a policy of “information”

. . . data is the tool

Data

Data

• More than test or assessment results• Data includes information about

– processes– inputs– contexts– capacities– –

Major policy question . . .

• Where does data come from?– Outside?– Inside?– Neither?– Both?

Rocket to Moon

• Off course 95% of the time

• Instruments that fed data to rocket engines and navigation systems were on board the rocket– Not remote– Not on the ground

Data about student learning

• Must come from– Assessments of learning (not tests)– Assessments of

• Processes• Inputs• Contexts• Capacities• •

Assessments of Learning

TestsDemonstrationsPerformancesPortfolios (over time)Observations

Test data is far too narrow to inform.

Here is our point . . .

Data from classroom (point of activity) tends to create change by informing the engagement of the key players.

Data from the classroom empowers

• Informs individuals about what doing and how well

• Provides feedback for self direction• Provides feedback to system re supports

needed• • •

Primary goal of data is empowering the learner . . .

• Learn what is expected

• Learn to self determine level beyond expectations

• Learn beyond content

• Learn how to learn and direct own learning

•

•

Data should inform key education decisions and decision-makers . . .

– Who is learning? What?– Who is not learning? What?– What do we do about both?

Data should inform key policy questions

Excellence –

How well . . .?

How much . . .?

Equity –

For whom?

What is a “good” school?

Overall achievement is highSubgroup achievement mirrors the

whole groupBoth trend lines are moving upwardGaps are narrowing

Data Should InformAlignment and Rationality

Programs, Practices

School Aims and Goals

District Aims and Goals

Data should inform . . .

• Strengths

• Areas of concern

• Possible strategies

Policy Pitfalls of Data

• Indicators Outcomes• About process not technology• Data threatens . . .

– People– Conventional wisdom– Current authority–

• Does not save time• Bad data = bad decision-making

Einstein

Not everything that can be counted counts, and not everything that counts can be counted.

We now have data.What do we do next?

Dr. Pat Roschewski

Statewide Assessment Director, NE Department of Education

The Continuous Improvement

Model

The “No Fear” Model

Data,

Data,

Data

Where are we?

What should be our goals?

Organizing For Data Analysis

Who Should be Involved In . . .

Collecting Data Displaying Data Analyzing Data Sharing Results of Data Other Local Uses of Data

Involve AllAll – 100% have a voice

Organizing For Data Analysis

Grouping for Involvement Grade levels Departments Subject Areas Staff, Students, Parents, Support Staff,

Community Members, Board Members

Organizing For Data AnalysisWhen? Pre-service, early out, late starts, mid-year, summer,

common planning times

How Often? Frequency Multiple Sessions

Format Big Picture – District Data – 1st session Building, grade level disaggregated – 2nd session By standard by student – 3rd and subsequent sessions

Understanding Data Basics

Data Sources

Data for Specific Purposes

Clarification of Data Types

Ground Rules

No blaming students No blaming teachers Data is just information Use data for instructional

purposes “De-emotionalize” data

Analyzing Data

What do these data show? Factual Information

Why might this be? Hypotheses

How should we respond? Planning for action

Reading_____________________________________________________________________________________________________________________________________

Math_____________________________________________________________________________________________________________________________________

Question One: What do these data show us?(Factual Information)

1. How many students are involved?

2. How many students met the standards?

3. How many students are in each proficiency level?

4. How great are the differences in grade levels?

5. What stands out in the data?

Question Two: Why Might this be?(Hypotheses)

1. Does the assessment measure what we teach? Why/why not?2. How does the timing of assessment impact the outcomes?3. What trends do we see in the data? Why?4. Skill strengths? Weaknesses?5. What differences are there in grade level or sub-groups? Why?

Reading_____________________________________________________________________________________________________________________________________

Math_____________________________________________________________________________________________________________________________________

Question Three: How should we respond?(Planning for Action)

1. How do we match instruction to skill needs? Do we have the skill as a staff to do that?2. How can we obtain the knowledge of instructional strategies

for all staff?3. In what ways do we offer remediation or acceleration? In classroom,

summer, flexible grouping, curricular adjustment?4. How can we effectively monitor, support, and evaluate classroom

effectiveness?

Reading_____________________________________________________________________________________________________________________________________

Math_____________________________________________________________________________________________________________________________________

We now have data.What do we do next?

David D. Hamm

Superintendent, Plainview Public Schools

Questions to Ponder

Why is it important to be able to produce evidence of what the school achieves for its students?

Is accountability a matter of compliance or responsibility?

When playing golf, which shot is the most important?

A Data ‘Rich’ Environment Board of Education

Administrative Team

Leadership Team

Teachers

Community

Students

How does ‘this’ benefit kids?

Live it every day!!!

It all starts with data ‘mining.’

Define the essential questions that matter most to your school system.

Describe the types of evidence you will need in order to answer the questions.

Identify the measures that will be needed to collect the necessary evidence.

We now have data.What do we do next?

Jan K. Hoegh

Statewide Assessment, NE Department of Education

Good ‘old’ days

Good ‘new’ days

TELL me! SHOW me!

versus