Assessing 21st Century Learning
OESIS, February 1, 2013
Exploring Four Approaches and a SideBar:
I Assessing Basic Skills Using Computer Adaptive Testing
II Assessing Higher Order Thinking Skills
III Sidebar: PARCC & SMARTER BALANCED.
IV Assessing Non-Cog
V Putting it Together: PBL, Digital Portfolios, and Demonstrations of Learning.
Kevin Mattingly, dean of faculty at Lawrenceville School (NJ):
• “We talk about 21st century skills and conceptual frameworks, and we say, this is what our kids need!
• How, then, do we find out and evaluate what they know, probing empathetically and fair-mindedly?”
Assessing What Matters
– Technology-based assessments that combine cognitive research and theory about how students think with multimedia, interactivity, and connectivity.
– This can be done within the context of relevant societal issues and problems that people care about in everyday life.
– Systems can be designed to capture students’ inputs and collect evidence of their knowledge and problem-solving abilities.
NETP
My Goal Today:
Explore and examine an array of tools and techniques situated in, suited to, and/or aligned with blended learning which effectively provide assessments of, for, and as 21st century learning.
Commission on Accreditation Criterion 13
The standards require a school to provide evidence of a thoughtful process, respectful
of its mission, for the collection and use in school decision-making of data (both
external and internal) about student learning
Will Richardson
Assessing of Basic Skills Using
Computer Adaptive Testing
Measures of Academic Progress™
(MAP)
MAP Testing is a tool that becomes an extension of our mission, which is to
maximize individual growth.
Pam Shaw, Canton Country Day School (OH)
• Traditional/Conventional Standardized Testing Updated• Computer Adaptive Assessment• Web/Browser based admin, available for online schools• More frequent (3x) than once a year• Reporting on not just achievement but growth• More rapid feedback to teachers• More detailed analysis of student proficiency• Comprehensive curricular resources aligned with
proficiency.
Sample Math Question
• Notice the calculator.
• To use it, click the numbers with your mouse.
Sample Math Question
• Notice the calculator.
• To use it, click the numbers with your mouse.
Sample Reading Question
• Read the passage and select the best answer
• Click the <Go on> button
Sample Reading Question
• Read the passage and select the best answer
• Click the <Go on> button
DesCartes ExampleStudent Scores a RIT range of 171-180 on the Goal Strand of Estimation and Accurate Computations
Teachers using online portal, grabbing student reports and class reports– and using curriculum alignment resources.
Parents and tutors getting these same reports for supplemental instruction as needed.
Blended Assessment of Higher Order Thinking Skills
“CWRA is doing truly groundbreaking work in developing assessments of the skills that matter most in the 21st century.”
The beauty of the CLA/CWRA is that it tests skills all teachers should be accountable for teaching in every class
What is a performance task? Students assume roles in a scenario that is a situated "real world" and contains the types of problems they might need to solve in the future or the present. Often documents must be evaluated for informing answers.
The task requires critical thinking, analytical reasoning and problem solving. Communication skills are used in describing the solution.
St Gregory Students Discuss the CWRA video
Long version: http://www.youtube.com/watch?v=w20K0YY22IA
Short version: http://www.youtube.com/watch?v=Y4qgDXUrc4w
Haverford
OECD TEST FOR SCHOOLS BASED ON PISA
Have you ever wanted to compare your school’s effectiveness with the world’s best national educational systems?
PISA TESTING VIDEO: http://www.youtube.com/watch?v=q1I9tuScLUA
BLOOD DONATION NOTICE
The instruments for taking the blood are sterile and single-use (syringe, tubes, bags).
There is no risk in giving your blood.
Blood donation is essential.
There is no product that can fully substitute for human blood. Blood donation is thus irreplaceable and essential to save lives.
In France, each year, 500,000 patients benefit from a blood transfusion.
Blood donation:
It is the best-known kind of donation, and takes from 45 minutes to 1 hour.
A 450-ml bag is taken as well as some small samples on which tests and checks will be done.
- A man can give his blood five times a year, a woman three times. - Donors can be from 18 to 65 years old.
An 8-week interval is compulsory between each donation.
Question 1An eighteen-year-old woman who has given her blood twice in the last twelve months wants to give blood again. According to “Blood Donation Notice”, on what condition will she be allowed to give blood again?
.................................................................................................................................
.................................................................................................................................
SCORING
Full Credit
Identifies that enough time must have elapsed since her last donation. Depends whether it has been 8 weeks since her last donation or not.
She can if it has been long enough, otherwise she can’t.
Level 2 item – 81.2% of students across OECD can perform tasks at least at this level
Your school’s results are statistically significantly above
Your school’s average score is not significantly different
Your school’s results are statistically significantly belowCompared to Averages
Country or Economy
Reading literacy
Mathematical literacy
Scientific literacy
Shanghai-China
Korea
Finland
Canada
United States
United Kingdom
Germany
Turkey
OECD Average
Reading: 512 Mathematics: 513 Science: 507
Compared to Percentiles (to be developed)
Country or Economy
Reading literacy
Mathematical literacy
Scientific literacy
Shanghai-China
Korea
Finland
Canada
United States
United Kingdom
Germany
Turkey
OECD Average
EXAMPLE: International comparisons of your school’s performance: PISA 2009 Countries and Economies
Section II. Your School’s Results in an international context
School ABCD
Bringing Performance Task in house:
Trans-disciplinary & Collaborative:
“Collaboratively designing and assessing these performance tasks in-house was our very best kind of PD”
Albemarle County PSVirginia Beach PS
Performance Task Academies from CWRA (Worcester Academy)
OCT:Open Computer Testing
What matters is no longer what one knows, but what one can do with what one knows and with information one can access,
evaluate, and apply.
Tony Wagner
For a set of resources including sample tests, www.21k12blog.net/oct
Open Computer Testing
Will Richardson 19 BOLD IDEAS VIDEO: http://vimeo.com/45152953
From BBC News, about Denmark
At five to nine, the room falls silent. One of the teachers stands in front of the class and explains the rules. She tells the candidates they can use the internet to answer any of the four questions. They can access any site they like, even Facebook, but they cannot message each other or email anyone outside the classroom.
Minister for education in Denmark, Bertel Haarder, says: “Our exams have to reflect daily life in the classroom and daily life in the classroom has to reflect life in society.
• We don’t have to memorize everything- which is basically what is going to happen in actual careers you don’t have to memorize everything you need to know you can look it up on the fly.
• By having the internet available you can get a lot more information to use on your exam and get a better understanding.
• You still need to memorize and learn the basics.
• We are still studying to learn how to apply facts on the test.
• And we are studying to determine what the most efficient way to find the information.
Sidebar
“Today is the day that marks the beginning of the development of a new and much-improved generation of assessments for America’s schoolchildren.
Beyond the Bubble Tests: The Next Generation of Assessments – Secretary
Arne Duncan’s Remarks to State Leaders
September 2, 2010
“For the first time, many teachers will have the state assessments they have longed for—tests of critical thinking skills and complex student learning that are not just fill-in-the-bubble tests of basic skills but support good teaching in the classroom.”
September 2, 2010
“In performance-based tasks, which are increasingly common in tests administered by the military and in other fields, students are given a problem and must sift through a portfolio of tools and write analytically about how they would use them to solve the problem.”
September 2, 2010
New York Times
PARCC
Smarter Balanced
PARCC & SMARTER BALANCED PROMISE:
Online ELA & Math assessments, grades 3-8 & HS.
– selected-response, – constructed-response, – technology-enhanced, – and complex performance tasks. (Better in SB, developed by CWRA)
Computer Adaptive Testing: (Smarter Balanced only)
Formative items and tasks for classroom use.
Professional development modules.
Digital library for sharing vetted resources and tools.
Smarter Balanef
SB
SB
As the public schools move their educational assessment “beyond the bubble”
– more authentic/situated scenario based performance tasks – Computer adaptive (SmarterBalanced)– Better evaluation of 21st c. skills and digital literacies, – Richer formative assessment Tools:
• Will they be widely available?
– more integrated teacher PD and collaboration tools
Where do we stand?
Assessing Non-Cognitive Skills and Aptitudes
Wagner’s 7 Survival Skills• critical thinking and problem solving • effective oral and written communication • accessing and analyzing information
-• curiosity and imagination• collaboration across networks and leading by influence
-
• agility and adaptability• initiative and entrepreneurship
• Choate Self-Assessment
Students respond to about 40 statements, such as: • I like to learn for the sake of learning. • Many people fail to get the recognition they
deserve no matter how hard they try. Choate admissions gets a rating on each applicants: self-efficacy, locus of control, and intrinsic motivation.
10% of unique predictive variance…
http://williamsedlacek.info
Character and 21st c. skills report cards
www.21k12blog.net
St. Andrew’s Episcopal (MD) Effort Report card
Assessing Using PBL, Digital Portfolios, and Demonstrations of Learning
Demonstrations of Learning: “What you do, not what you know, is the ultimate test of education.” ~PFB
Tweet
• Conduct a fluent conversation in a foreign language about of piece of writing in that language.
• Invent a machine or program a robot capable of performing a difficult physical task.
• Using statistics, assess whether or not a statement by a public figure is demonstrably true.
• Assess media coverage of a global event from various cultural/national perspectives.
• Demonstrate a commitment to creating a more sustainable future with means that are scalable.
Demonstrations of Learning
Project-Based Learning
Final Product: Content,
Creativity and Craftsmanship
Peer Assessment
Self-Assessment
(Perseverance)
Expert Assessment
Project Organization
Content Knowledge
Tested
Teamwork/Collaboration
PBL HOLISTIC ASSESSMENT
DoL PBL Digital Portfolios
Digital Portfolios• High Tech High Digital Portfolios• New Tech Network Digital Portfolios
Quantifying Portfolio Assessment:
• Percentage completing challenging demonstrations of learning with excellence
• Engaging expert external reviewers of representative sampling of portfolios evaluating according to a school-designed rigorous rubric
