ASSESMENT IN OPEN AND DISTANCE LEARNING

Objectives:1. To explain the characteristic of

assessment in ODL2. To identify the problem and solution of

assessment in ODL3. To develop the assessment system in ODL

ASSESSMENT ….ASSESSMENT ….

Is a process in gathering, processing, Is a process in gathering, processing, interpreting, and using the interpreting, and using the information of student’s learning information of student’s learning achievement in order to know how achievement in order to know how the goals are attained objectively.the goals are attained objectively.

CHARACTERISTIC OF ODL …. Student and instructor are separated Instruction may be synchronous or

asynchronous Self directed learning

INSTRUCTION IN ODL ….INSTRUCTION IN ODL ….BASIC

COMPETENCIES

LEARNING:TUTORIALS

SELF DIR. LEARNING

ASSESSMENT

INDICATORS

BASIC STEPS IN BASIC STEPS IN ASSESSMENTASSESSMENT

• Determining the purpose of assessmentDetermining the purpose of assessment• Developing the table of specificationDeveloping the table of specification• Selecting appropriate instrumentSelecting appropriate instrument• Preparing relevant instrumentPreparing relevant instrument• Assembling the instrumentAssembling the instrument• Administering the assessmentAdministering the assessment• Appraising the assessmentAppraising the assessment• Using the resultUsing the result

THE PRINCIPLES OF ASSESSMENT IN ODL …

Include the general principles of assessment such as:

* valid * Reliable * Fair Can be managed from distance: * Traditional testing * On line testing

ASSESSMENT ISSUES IN ODL ASSESSMENT ISSUES IN ODL ….….

►Quality of instrumentQuality of instrument►Large scale assessmentLarge scale assessment►On line assessmentOn line assessment►Administering assessmentAdministering assessment►Scoring processScoring process►Final examinationFinal examination

QUALITY OF INSTRUMENT ….QUALITY OF INSTRUMENT ….

VALIDITY AND RELIABILITYVALIDITY AND RELIABILITY

VALID & RELIABELVALID & RELIABEL NOT VALID BUT RELNOT VALID BUT REL NOT VALID & NOT RELNOT VALID & NOT REL

DOMAIN OF COMPETENCY …

TAXONOMY OF EDUCCOMPETENCIES

COGNITIVE

AFFECTIVE

PSYCHOMOTOR

ASSESSMENT’S INSTRUMENT ASSESSMENT’S INSTRUMENT

ESSAY TEST

OBJECTIVEINSTRUMENTS TEST

PERFORMANCEASSESSMENT

PORTFOLIOASSESSMENT

TEST DEVELOPMENTTEST DEVELOPMENT

• Development of test specification

• Writing items based on test specification

• Reviewing test items

• Assembling test

• Reproduction test

• Administering test

• Scoring and grading

TEST SPECIFICATION …TEST SPECIFICATION …

Representative of materialRepresentative of material Type of testType of test Level of cognitiveLevel of cognitive Index of difficulty itemIndex of difficulty item The sum of items and time consumingThe sum of items and time consuming

TABLE OF TEST SPECIFICATIONTABLE OF TEST SPECIFICATION

TOPIC & SUB TOPICLEVEL OF COGNITIVE & INDEX

OF DIFFICULTY ITEMTOTAL

ITEM

HOW TO CONSTRUCT GOOD HOW TO CONSTRUCT GOOD ITEM TEST ?ITEM TEST ?

Use item test specificationUse item test specification Use guide line to construct good item Use guide line to construct good item

(ex.: there is a key answer for (ex.: there is a key answer for objective test or marking scheme for objective test or marking scheme for essay test) essay test)

Item tests should represent specific Item tests should represent specific objectivesobjectives

The items should have highly internal The items should have highly internal consistencyconsistency

EXAMPLE OF TEST ITEMSEXAMPLE OF TEST ITEMS

PREPARE PERFORMANCE OR PORT PREPARE PERFORMANCE OR PORT FOLIO ASSESSMENT ?FOLIO ASSESSMENT ?

Identify competenciesIdentify competencies Prepare the instructional task/ Prepare the instructional task/

student worksheetstudent worksheet Prepare the rubric (marking scheme)Prepare the rubric (marking scheme)

ADMINISTERING OF PERFORMANCE ADMINISTERING OF PERFORMANCE

AND PORTFOLIO ASSESSMENTAND PORTFOLIO ASSESSMENT To motivate studentTo motivate student Discuss, how to do the instructional Discuss, how to do the instructional

task task Give the continuous feedbackGive the continuous feedback

HOW TO ASSESS PERFORMANCE HOW TO ASSESS PERFORMANCE AND PORTFOLIO ?AND PORTFOLIO ?

Use the rubric consistently Use the rubric consistently Give chance to self assessment Give chance to self assessment

based on rubricbased on rubric

EXAMPLE IN ASSESSMENT OF EXAMPLE IN ASSESSMENT OF PERFORMANCE …PERFORMANCE …

Do an experiment with a rope to know the periodic of Do an experiment with a rope to know the periodic of oscillation:oscillation:

1. Length of rope and weight are constant, angle of 1. Length of rope and weight are constant, angle of deviation varies .deviation varies . 2. Length of rope varies, and weight is constant.2. Length of rope varies, and weight is constant. 3. Length of rope is constant, and weight varies.3. Length of rope is constant, and weight varies. Write a report of your experiment and send it by e-mail Write a report of your experiment and send it by e-mail

(due-date is ….) (due-date is ….) Criteria of assessment: Criteria of assessment: 1. Time Precision of report submission (10%)1. Time Precision of report submission (10%) 2. Tidiness of report (10%)2. Tidiness of report (10%) 2. Procedure of experiment (20%)2. Procedure of experiment (20%) 3. Systematic of Displaying Data (10%)3. Systematic of Displaying Data (10%) 5. Data Analysis (30%)5. Data Analysis (30%) 6. Conclusion (20%)6. Conclusion (20%)

THE RUBRIC …THE RUBRIC …

SCORESCORE ASPECTS TO BE ASSESSEDASPECTS TO BE ASSESSED

33

22

11

33

22

11

Report submission is on timeReport submission is on time

1 day delayed1 day delayed

2 days delayed2 days delayed

Report is tidyReport is tidy

Report is less tidyReport is less tidy

Report is not tidyReport is not tidy

THE RUBRIC …THE RUBRIC …

SCORESCORE ASPECTS TO BE ASSESSEDASPECTS TO BE ASSESSED

33

22

11

33

22

11

Experiment procedure is correctExperiment procedure is correct

A little error in experiment procedureA little error in experiment procedure

Too many errors in experiment procedureToo many errors in experiment procedure

Systematic of displaying data is correctSystematic of displaying data is correct

A little mistake in systematic of displaying dataA little mistake in systematic of displaying data

Systematic of displaying data is wrong Systematic of displaying data is wrong

THE RUBRIC …THE RUBRIC …

SCORESCORE ASPECTS TO BE ASSESSEDASPECTS TO BE ASSESSED

33

22

11

33

22

11

Data analysis is very goodData analysis is very good

Data analysis is goodData analysis is good

Data analysis is not goodData analysis is not good

Conclusion is correctConclusion is correct

Conclusion is not too correctConclusion is not too correct

Conclusion is wrongConclusion is wrong

LARGE SCALE LARGE SCALE ASSESSMENT …ASSESSMENT …

• Is an assessment that is administered to a Is an assessment that is administered to a large number of people at one time in large number of people at one time in which the examinees are spread out in which the examinees are spread out in large area large area

ADVANTAGES OF LARGE ADVANTAGES OF LARGE SCALE ASSESSMENT ….SCALE ASSESSMENT ….

It is used to measure learning outcomes It is used to measure learning outcomes effectively, esp. Cognitive domaineffectively, esp. Cognitive domain

It can be managed efficientlyIt can be managed efficiently The outcomes is standardizedThe outcomes is standardized It is fairIt is fair

THE LIMITATIONS OF LARGE THE LIMITATIONS OF LARGE SCALE ASSESSMENT ….SCALE ASSESSMENT ….

It cannot be used to measure learning It cannot be used to measure learning outcomes comprehensivelyoutcomes comprehensively

It depends on well organized item bankIt depends on well organized item bank It needs a large number of human and It needs a large number of human and

financial resources if it is applied manuallyfinancial resources if it is applied manuallyThe security of the item test is very The security of the item test is very

sensitivesensitive

HOW TO MANAGE LARGE HOW TO MANAGE LARGE SCALE ASSESSMENT ….SCALE ASSESSMENT ….

Develop a well organized item bankDevelop a well organized item bank Assembling the test Assembling the test Reproduce test booksReproduce test books Distribute test books to every regional learning Distribute test books to every regional learning

centercenter Test administrationTest administration Scoring processScoring process Send the examination result to every studentSend the examination result to every student

POTENTIAL PROBLEMS IN POTENTIAL PROBLEMS IN LARGE SCALE ASSESSMENT …LARGE SCALE ASSESSMENT …

• Test Quality is low• Not enough test books at some locations• The test administration is uncontrolled• Bad management on score processing • The students do not receive test result

on time

Tips for Successful Large Scale Tips for Successful Large Scale AssessmentAssessment

Develop a well organized item bankDevelop a well organized item bank

Develop standard operation and procedures to Develop standard operation and procedures to control the quality of assessmentcontrol the quality of assessment

Build networking with other institutions in Build networking with other institutions in developing item tests, recruiting proctors, developing item tests, recruiting proctors, delivering item booklets and having places for delivering item booklets and having places for the testthe test

Make sure that every person who involved in Make sure that every person who involved in developing item tests is well-traineddeveloping item tests is well-trained

Tips for Successful Large Tips for Successful Large Scale Assessment (cont)Scale Assessment (cont)

Make sure that all proctors are well-informed Make sure that all proctors are well-informed Be sure that item tests are secureBe sure that item tests are secure Make sure that scoring process works on Make sure that scoring process works on

timetime Make sure that all students have the result Make sure that all students have the result

on timeon time

ON LINE ASSESSMENT IN ODL ON LINE ASSESSMENT IN ODL

Is ICT used for administrating the test ?Is ICT used for administrating the test ? The item bank should have a great number of The item bank should have a great number of

itemsitems Computer adaptive testingComputer adaptive testing To have on line assessment facilityTo have on line assessment facility

THE STRENGTHS AND THE STRENGTHS AND WEAKNESSES OF ON LINE WEAKNESSES OF ON LINE

ASSESSMENT…ASSESSMENT…Strengths:Strengths: AccurateAccurate Effective and efficientEffective and efficient Long lastingLong lasting Flexible in time and placeFlexible in time and placeWeaknesses:Weaknesses: Great investmentGreat investment Limited access to computer for some studentLimited access to computer for some student Difficult to control students’ honesty (???)Difficult to control students’ honesty (???)

Scoring processScoring process

Identity checkIdentity check Mode of process: - ManualMode of process: - Manual

- Computerized- Computerized Score InterpretationScore Interpretation GradingGrading Distribution of test resultDistribution of test result

FINAL ASSIGNMENT OR FINAL EXAMINATION OF A PROGRAM

• Every degree program should be ended with a final assignment or final examination

• Final assignment can be in the form of an academic paper.

• The students can also be tested on their comprehension toward the program by giving them comprehensive examination only