student learning assessments - unesco · outlinei 1 de ning assessment 2 types of assessment 3...

Student Learning Assessments

Dr. Kelvin D. Gregory

Education ConsultantAdjunct at Flinders [email protected]

Dr. Kelvin D. Gregory Student Learning Assessments 1 41

Outline I

1 Defining Assessment

2 Types of assessment

3 Large scale assessmentsConstruct definitionScale characteristicsScale regions

4 Contextual FrameworkContextual FrameworkBroad areasQuestionnaires


The primary purposeThe primary purpose is:

Building capacity for large scale nationalassessments

And driving that is a desire to improve Nepal’s primaryand lower and higher secondary education systems.And we will focus on:

Reasons for national assessments

Writing assessment frameworks and items

Analysis systems


A benchmark interventionWhat if, in addition, you:

Worked with teachers before the start of theteaching block

Clarified the learning topics (and objectives),developed an assessment framework, designedoutcome measures (assessment items and tests)

The Ministry sent benchmark tests, withinstructions and interpretation guides, to schools2-3 weeks before the end of the teaching block

Teachers administered the tests, scored them,and used them as part of the reporting process

Then you repeat the exercise, building onwhat you have learned


Defining Assessment”Assessment is the systematic collection,interpretation and use of information to givea deeper appreciation of what students knowand understand, their skills and personal capabilities, andwhat their learningexperiences enable them to do.”(Northern Ireland Curriculum Guidance for Assessment inthe Primary School)

How is assessment defined in the officialdocuments of Nepal?Do people use this definition?How would students and teachers define assessment?


Dann (2012)”Fundamental ... is the idea that learning, ultimately, isconstructed and controlled bypupils. Without teachers willingness toengage with the curriculum and pupilsdeveloping range of cognitive competenciesand experiences, learning will not proceed.If assessment genuinely seeks to give someindication of pupils levels of learning anddevelopment, in ways which will furtheradvance learning, pupils will need tounderstand and contribute to the process.”(p. 2)How do Nepal’s students contribute tothe assessment process?


Using Assessment Information”Many people want to use assessmentinformation, and they want to use it in manyways.” (Stiggins (2007),p.29)

Who uses assessment information in Nepal?What information do they use?When and how do they use it?Is the information used appropriately andin a timely manner?Can you offer a constructive critique of theuse of assessment information in Nepal?


Formative and SummativeAssessment DefinitionSummative Assessments are givenperiodically to determine at a particular pointin time what students know and do not know.

Formative Assessment is part of theinstructional process. When incorporated intoclassroom practice, it provides the informationneeded to adjust teaching and learning whilethey are happening.


Assessment for learning

enables teachers to use informationabout students’ knowledge, skills andunderstanding to inform their teaching

teachers provide feedback to studentsabout their learning and how to improve


Assessment as learning

involves students in the learning processwhere they monitor their own progress,ask questions and practice skills

students use self-assessment and teacherfeedback to reflect on their learning,consolidate their understanding and worktowards learning goals


Assessment of learning

assists teachers to use evidence ofstudent learning to assess studentachievement against learning goals andstandards


AssessmentIf we want to use assessment as a tool forlearning students need to:

Know where they are going

Know where they are now

Know how to close the gap


Formative and SummativeAssessment

Formative SummativeAssessment for learning Assessment of learningAnalyze strengths andweaknesses

Document achievement

Used to check students’understanding and to planfor subsequent teaching

Provides teachers, studentsand others withinformation about theattainment of knowledgeand skills.

Must be used in a teachingmodel

May be used in subsequentteaching of differentconcepts.

Low stakes but veryimportant

Stakes can range from lowto high.


Defining large scale assessmentsThe British Columbia Government’s largescale assessment policy:

The term large scale assessment refersto any provincial, national orinternational assessment, examination or testthe Ministry directs boards ofeducation to administer.


Large scale assessment purposeThe British Columbia Government’s largescale assessment policy:

A primary purpose of large-scaleprovincial, national and internationalassessments and examinations is toobtain information for the purposes ofpublic accountability, improving programsand certifying that students meetgraduation requirements.


Large scale assessment focusThe British Columbia Government’s largescale assessment policy:

Large scale assessments andexaminations determine students’ knowledgeand skills in particular areasof learning.


Large scale assessment useThe British Columbia Government’s largescale assessment policy:

Assessment information enables educationaldecision-makers at the classroom, district andprovincial levelsto make informed choices related to improvingstudent achievement.


Using large scale studentassessment to improve studentachievementUngerleider (2006) makes the following points

there needs to be broad agreement aboutthe outcomes for all studentsthese must be detailed in the curriculumand supported by teaching materialthe educational community (students,teachers, parents) should be heldaccountable for the outcomesprovide teachers with adequate timeand training to interpret the outcomesDr. Kelvin D. Gregory Student Learning Assessments 18 41

”The validity of any assessment result isconditional on the fit between the purposefor which the assessment was designed andthe use of the results. At present there is noestablished procedure evaluating the validityof the alternative uses to which large scaleassessments might be put. It is not unusualthat an assessment developed for one purposeto be used for another. At present, thedevelopers of the original test leave it tosubsequent users to decide if its use isappropriate.”(Ungerleider (2006), p. 875)


Assessment continuumAssessment involves professional judgementbased upon an image formed by the collectionof information about student performance.

Unstructuredchance meet-ings, conversa-tions

Slightlystructuredquestionnaires,observations,student self-assessment

More struc-turedchecklists, class-room tests,practical work,project work,case studies

Most struc-turedExaminations,standardizedtests, publishedaptitude tests


The ContinuumA continuum attempts to capture in wordswhat it means to make progress or to improvein an area of learning or domain of knowledge.In noncognitive areas we think of the construct changingover time within a continuum.

Proficiency

T ime

Motivation

T ime


Example developmental continuumfor recitation (Grades I and II)

Recitation is poor. Lacks expression.

Cannot recite an entire poem or story without prompt-ing. Pronunciation expression is not appropriate.

Can recite poem, or story with occasional prompting.Expression is not very strong and effective.

Can recite poem, or story with proper speed or expres-sion but makes occasional mistakes in pronuciation orforgets at times.

Can recite poem or a story with proper speed, diction,expression and tone.


Example developmental continuumfor mathematics

2 + 2=

17 + 29=

76 - 29=

What is the difference between the largest and secondlargest numbers that can be made from the followingdigits: 3, 7, 4 and 6.

The larger of two integers is 5/2 the smaller.If twicethe smaller is subtracted from twice the larger, the dif-ference is 12. Find the two numbers.


Traditional subjects


Traditional subject withnoncognitive skills


Introducing standards

We divide the continuum into areas tohelp make sense of it.

I This helps us communicate what we meanby the continuum.

I It simplifies what we are talking about andat the same time we do not exaggerate theaccuracy of what we are measuring.

In cognitive continua, we call these areas”performance standards” .

We describe what the person cando for each standard.

Band 1 Band 2 Band 3 Band 4


Describing the standards

We describe what the person can dofor each performance standard.

We call the process of describing eachstandard ”standard setting”.

The easist way of describing whatstudents can do at each level is to get examples ofstudents’ work.

I Using experts, we might get a large rangeand sample of students’ work, sort it intofour piles of increasing standard.

I Then we describe what students in eachpile can do using terms to describe thecognitive skills.


NAEP Generic Reporting scaleThe USA approach to developing theirachievement reporting scales has been todevelop a generic or general descriptioncovering all scale areas.

Level Description

BasicPartial mastery of prerequisite knowledge andskills that are fundamental for proficient workat each grade.

Proficient

Solid academic performance for each gradeassessed. Students reaching this level havedemonstrated competency over challengingsubject matter, including subject-matterknowledge, application of such knowledge toreal-world situations, and analytical skillsappropriate to the subject matter.

Advanced Superior performance.Dr. Kelvin D. Gregory Student Learning Assessments 28 41

NAEP Generic Reporting scaleAnd then they add subject area specificdescriptions.

Level Description

Basic

Fourth-grade students performing at the Basic level should show someevidence of understanding the mathematical concepts and procedures inthe five NAEP content areas. Fourth-graders performing at the Basiclevel should be able to estimate and use basic facts to perform simplecomputations with whole numbers, show some understanding of fractionsand decimals, and solve some simple real-world problems in all NAEPcontent areas. Students at this level should be able to use though notalways accurately four function calculators, rulers, and geometric shapes.Their written responses will often be minimal and presented withoutsupporting information.

Proficientrefer tohttps://nces.ed.gov/nationsreportcard/mathematics/achieveall.aspx

Advanced

Fourth-grade students performing at the Advanced level should applyintegrated procedural knowledge and conceptual understanding tocomplex and nonroutine real-world problem solving in the five NAEPcontent areas. Fourth-graders performing at the Advanced level should beable to solve complex and nonroutine real-world problems in all NAEPcontent areas. They should display mastery in the use of four-functioncalculators, rulers, and geometric shapes. The students are expected todraw logical conclusions and justify answers and solution processes byexplaining why, as well as how, they were achieved. They should gobeyond the obvious in their interpretations and be able to communicatetheir thoughts clearly and concisely.


The frameworkThe development of cognitive indicator startswith the development of a frameworkgenerally involves the following steps:

widespread participation and reviews by educatorsand education officials;

reviews by steering committees whose members representpolicymakers, practitioners, andmembers of the general public;

involvement of subject supervisors from education agencies;

public hearings; and

reviews by scholars in the field, by the studystaff, and by a policy advisory panel.


Construct definitionCognitive indicators need a definition thatwill define the construct throughout the study.The construct’s meaning should not changeduring this time; it must be stable.


PISA mathematical literacy” Mathematical literacy is an individual’scapacity to formulate, employ, and interpret mathematicsin a variety of contexts. Itincludes reasoning mathematically and usingmathematical concepts, procedures, factsand tools to describe, explain and predict phenomena. Itassists individuals to recognisethe role that mathematics plays in the worldand to make the well-founded judgments anddecisions needed by constructive, engagedand reflective citizens.”


Scale characteristicsGenerally, the numerical characteristics ofthe scale are described. The scale when IRTis used is generally a simple lineartransformation of the mean and standarddeviation obtained from the IRT analyses.


Scale characteristics”PISA results are reported on a scaleconstructed using a generalised form of theRasch model as described by Adams, Wilsonand Wang (1997). For each domain (reading,mathematics and science), a scale isconstructed with a mean score of 500 andstandard deviation of 100 among OECDcountries; accordingly, about two-thirds ofstudents across OECD countries scorebetween 400 and 600 points.”


Then describe regions of the scale

Level Location Items Description

2 409.5

ACIDRAINQ2(460)

At Level 2, students haveadequate scientificknowledge to providepossible explanations infamiliar contexts or drawconclusions based onsimple investigations. Theyare capable of directreasoning and makingliteral interpretations ofthe results of scientificinquiry or technologicalproblem solving.


Contextual FrameworkInternational studies also include acontextual framework. From TIMSS 2011:”Because learning takes place within a context and not inisolation, TIMSS makes every attempt to collect informationabout the important factors that foster improved teaching andlearning in mathematics and science. The questionnairesconcentrate on procedures and practices that have been shownto be effective in increasing achievement in mathematics andscience. In this way, countries can better evaluate their TIMSSresults; in terms of the prevalence of the home or schoolsituation or instructional practice in their country and itsrelationship with student achievement.”


TIMSS Contextual Frameworkbroad areasThe TIMSS contextual framework specifiesfive broad areas:

National and community contexts.

Home contexts.

School contexts.

Classroom contexts.

Student characteristics and attitudestowards learning.

Each of these areas is described using anumber of measurements in the 20 pagecontextual framework.


Contextual FrameworkcomponentsThe TIMSS contextual framework specifiesfour questionnaire types:

Curriculum questionnaire.

Student questionnaire.

Teacher questionnaire.

Head teacher or principal questionnaire.

And describes what information eachquestionnaire is designed to gather and why.


TIMSS school questionnaireThis questionnaire is designed to takeapproximately 30 minutes to complete andhas the following sections:

School enrollment and characteristics.

Instructional time.

Resources and technology.

School emphasis on academic success.

School discipline and safety.

Teachers in the school.

School readiness.

Principal experience and education.


TIMSS teacher questionnaireThis questionnaire is designed to takeapproximately 30 minutes to complete.There are specific versions for specialist mathematics andscience teachers:

Characteristics of the teacher.School emphasis on academic success.Experiences of being a teacher.Experiences with the assessed class.And then questions for mathematics and science

I Experiences teaching the subject.I Homework practices.I Assessment practices.I Preparation to teach.I Use of computers.


TIMSS student questionnaireThis questionnaire is designed to takeapproximately 30 minutes to complete andhas the following sections:

About the student.

About the school.

Mathematics in the school.

Science in the school.


student learning assessments - unesco · outlinei 1 de ning assessment 2 types of assessment 3...

Documents