assessment for aais
TRANSCRIPT
Next Gen. Assessment: Assessing & Measuring What Matters Most
Jonathan E. Martin
Head of School
St. Gregory College Prep
www.21k12blog.net
Begin with the end in mind
Backward design
www.21k12blog.net
What gets measured gets done
www.21k12blog.net
The measurement is the message
www.21k12blog.net
6 Suggestions internal
Rethinking in-class assessments
Expanded reporting, particularly with 21st c. skills
Pat Bassett’s Demonstrations of Learning
external
High School Survey of Student Engagement (hssse)
NWEA’s Measurement of Academic Progress (MAP)
College and Work Readiness Assessment (CWRA)
www.21k12blog.net
In Class
Exhibitions and Presentations
Online publications PBL and
demonstrations Making it Real: AP US
Gov’t examples
Testing the right way
Do we still believe in testing?
From Pat Bassett, Schools of the Future prez.
From Pat Bassett & Chapin School
Open Computer Testing Nothing changes if the assessments are still geared to pencil
and paper. Those assessments were developed when there were no computers and the power of Google was not there for students on question of fact. That power gives them the power of argument and allows real thinking: Unleash them. Norman Constantine, Wakefield School.
If you are continuing to base a lot of your assessment on fact-based questions, or even asking higher-level questions but preventing preventing students from accessing the tools of their trade, then it’s time to rethink. You can begin to make use to technology to create richer, more meaningful assessments . Steve Taffee, Castilleja School
Denmark
This is the Danish language exam. One of the teachers stands in front of the class and explains the rules. She tells the candidates they can use the internet to answer any of the four questions. They can access any site they like, even Facebook, but they cannot message each other or email anyone outside the classroom.
The teachers also think the nature of the questions make it harder to cheat in exams. Students are no longer required to regurgitate facts and figures. Instead the emphasis is on their ability to sift through and analyse information.
Minister for education in Denmark, Bertel Haarder, says: “Our exams have to reflect daily life in the classroom and daily life in the classroom has to reflect life in society. The internet is indispensible, including in the exam situation. I’m sure that is would be a matter of very few years when most European countries will be on the same line.”
performance task assessment
It is an empty exercise to assess student learning
without providing a means to adjust teaching
in response to deficiencies revealed
through the information gleaned from that
assessments.
Marc Chun: Taking Teaching to the (Performance) Task
.
www.21k12blog.net
Performance Task Assessment elements
Real-world scenario Authentic, complex
process Higher-order thinking
Authentic performance Transparent evaluation
criteria
www.21k12blog.net
Performance task assessment rubric considering an issue from
multiple perspectives critically examining
evidence valuing claims that are
backed by appropriate and adequate evidence,
reasoning objectively and dispassionately
arriving at informed judgments and decisions.
www.21k12blog.net
Scenario(created by Kristy Miller, University of Evansville)
Prior to the election, a bill is proposed in the state senate that would ban aspartame, an artificial sweetener, from being added to any soft drink or food product. Senator Dulce is opposed to the bill. You are an advisor to State Senator Nathan Dulce, who is running for reelection against County Commissioner Pat Sauer.
Pat Sauer made two arguments in favor of the bill during a recent TV interview: First, Sauer said that because of the strong correlation between the number of people who consume aspartame and headaches, “banning aspartame would improve the health of the state’s citizens.” Second, he said that “aspartame should be replaced with sucralose.”
Documents: Local newspaper article opposing aspartame
Data from the Department of Health about headaches andaspartame
Task: Senator Dulce wants to make sure he votes appropriately on the aspartame bill, so he has asked you to analyze the strengths and/or limitations of each of Sauer’s two main points.
expanding student reporting
assessing and reporting on desired outcomes
www.21k12blog.net
Sources for skill sets
Partnership for 21st c. Skills: creativity, critical thinking, collaboration, communication
Tony Wagner’s Seven Survival Skills
NAIS commission’s list of essential 21st c. capacities
www.21k12blog.net
The “egg”
www.21k12blog.net
St. Gregory’s “egg”
Effort Integrity Leadership Communication skills
Writing Speaking Listening Digital communications
www.21k12blog.net
Egg continued Thinking skills
Inquisitiveness Analytic thinking Synthetic thinking Critical thinking
Innovation skills Creativity and ingenuity Adaptability Initiative Experimentation and risk-
taking Complex, real-world, problem-
solving
www.21k12blog.net
The Egg: a work in progress
Just tacking to report card not successful enough.
Embedding it instead in advisory, conferencing, goal-setting, and self-
assessment.
www.21k12blog.net
Pat Bassett’s call for “demonstrations of learning”Define the skills of your
graduates, and required “exhibitions” or demonstrations of those skills for graduation.
www.21k12blog.net
Demonstrations of Learning: “What you do, not what you
know, is the ultimate test of education.” ~PFB Tweet
1. Conduct a fluent conversation in a foreign language about of piece of writing in that language.
2. Write a cogent and persuasive opinion piece on a matter of public importance.
3. Declaim with passion and from memory a passage that is meaningful, of one’s own or from the culture’s literature or history.
4. Demonstrate a commitment to creating a more sustainable and global future with means that are scalable
5. Invent a machine or program a robot capable of performing a difficult physical task.
Demonstrations of Learning
6. Exercise leadership in arena which you have passion and expertise.
7. Using statistics, assess if a statement by a public figure is demonstrably true.
8. Assess media coverage of a global event from various cultural/national perspectives.
9. Describe a breakthrough for a project-based team on which you participated in which you contributed to overcoming a human-created obstacle.
10. Produce or perform or interpret a work of art.
Wouldn’t it be great if each school in a competitive set defined their lists differently, driven by unique mission, and families chose
schools by their defined outcomes (and demonstrated success at those outcomes?)
www.21k12blog.net
What are/ would be your school’s
demonstrations of learning?
Discuss.
www.21k12blog.net
external assessment: measuring what matters
www.21k12blog.net
Not everything that can be counted counts, and
not everything that counts can be counted.
Albert Einstein
Data must inform, not replace, judgment.
www.21k12blog.net
Balancing the ScorecardOur schools will be
evaluated and measured, whether we like it or not, especially with SAT scores and college lists.
We need to take the initiative, inside schools, to broaden the measurements and balance the scorecard.
www.21k12blog.net
Does your educational mission include:
Meaningful student engagement in learning?
Supportive school culture and strong community?
Personalized learning for breadth of student abilities in core subjects?
Development of higher order thinking skills, and effective problem-solving skills?
www.21k12blog.net
Let’s measure mission:
www.21k12blog.net
To improve accountability.
In accreditation
To Boards
To parents
To funders
www.21k12blog.net
For marketing and communications.
www.21k12blog.net
To improve learning in the 21st century!
www.21k12blog.net
NAIS monograph: July 2010Student Outcomes that Measure the School’s Value Added
“Independent schools are now being challenged to show how their educational model generates what it
does and how it is worth the investment.
“Consider using a tool or tools that best fits your school, mission, and community, knowing that the primary purpose for doing so is institutional assessment and improvement. You will be stronger and more financially viable for doing so.”
www.21k12blog.net
3 assessments featured in Nais monograph
hssse
map
cwra
www.21k12blog.net
High School Survey of Student Engagement
HSSSE
www.21k12blog.net
86
96
92
98
95
67
88
74
88
66
77
95
87
97
89
40
50
60
70
80
90
100
I have a voice in the classroom There is at least one adult at thisschool w ho cares about me
There is at least one adult at thisschool w ho know s me w ell
Adults in this school w ant me tosucceed
My opinions are respected in thisschool
2009 STG Strong & Agree
2009 All Schools Strong & Agree
2010 STG Strong & Agree
What excites and engages you?
51
87
41
51
42
6664
59
64
74
28
65
37 38 37
60
4442
48
55
10
20
30
40
50
60
70
80
90
TeacherLecture
Discussionand Debate
IndividualReading
WritingProjects
ResearchProjects
Group projects Presentations Role Plays Art and DramaActivies
Projectsinvolving
Technology
2010 STG some & verymuch
2010 All Schools Some & very much
Education Secretary Duncan: Beyond the Bubble Tests: The Next Generation of Assessments
I believe the impact of this next generation of
assessments in the classroom will be
dramatic—and that the new assessments will support learning and instructional
practices that teachers have long hungered for
themselves.
www.21k12blog.net
computer adaptive assessment
www.21k12blog.net
Duncan:
One-shot, year-end bubble tests administered on a single day, too often lead to a dummying down of curriculum and
instruction throughout the course of the entire school year.
In short, most of the assessment done in schools today is after the fact and designed to indicate only whether students have
learned. Not enough is being done to assess students’ thinking as they learn to boost and enrich learning, and track
student growth.
For the first time, state assessments will make widespread use of smart technology. They will provide students with realistic, complex performance tasks, immediate feedback, computer
adaptive testing, and incorporate accommodations for a range of students.
www.21k12blog.net
Northwest Evaluation Association
Measures of Academic Progress
St. Gregory teachers discuss on video
the MAP
www.21k12blog.net
Duncan:
One of the biggest frustrations of teachers with existing assessments is that they fail to test higher-order reasoning and writing skills, and
thus fail to show what students know and can do.
For the first time, many teachers will have the state assessments they have longed for– tests of critical thinking skills and complex student learning that are not just fill-in-the-bubble tests of basic skills but support good teaching in the classroom.
The new assessments will better measure the higher-order thinking skills so vital to success in the global economy of the 21st century
and the future of American prosperity. To be on track today for college and careers, students need to show that they can analyze
and solve complex problems, communicate clearly, synthesize information, apply knowledge, and generalize learning to other
settings.
www.21k12blog.net
Council for Aid to Education’s
College and Work Readiness Assessment
CWRA
www.21k12blog.net
St. Gregory students discuss the CWRA
video
www.21k12blog.net
Example Performance Task: Source: Council for Aid to Education, CWRA Institutional ReportYou advise Pat Williams, the
president of DynaTech, a company that makes precision electronic instruments and navigational equipment. Sally Evans, a member of DynaTech’s sales force, recommended that DynaTech buy a small private plane (a SwiftAir 235) that she and other members of the sales force could use to visit customers.
Pat was about to approve the purchase when there was an accident involving a SwiftAir 235. Your document library contains the following materials:
www.21k12blog.net
Documents or Artifacts:
Newspaper article about the accident
Federal Accident Report on in-flight breakups in single-engine planes
Internal Correspondence (Pat’s e-mail to you and Sally’s e-mail to Pat)
Charts relating to SwiftAir’s performance characteristics
Excerpt from magazine article comparing SwiftAir 235 to similar planes
Pictures and descriptions of SwiftAir Models 180 and 235
www.21k12blog.net
Questions:
Do the available data tend to support or refute the claim that the type of wing on the SwiftAir 235 leads to more in-flight breakups?
What is your preliminary recommendation about whether or not DynaTech should buy the plane and what is the basis for this recommendation?
www.21k12blog.net
From CAE:
CWRA tasks require that students integrate: critical thinking,
analytic reasoning, problem solving,
and written communication skills.
The holistic integration of these skills on the CWRA tasks mirrors the requirements of serious thinking and writing tasks faced in life outside of
the classroom.
www.21k12blog.net
Scoring
www.21k12blog.net
St. Gregory seniors
Mean Raw Score Points Increase 128
Percentile Increase, on the College Freshman scale
30 percentile points(67th to 97th)
Median Standard Deviation Increase .92 (compared to .51 at other high schools)
www.21k12blog.net
Q & A
www.21k12blog.net
Thank you.
www.21k12blog.net