I. Pribela, M. IvanovićNeum, 2009

ContentAutomated assessmentTestovid systemTest generatorModule generatorsConclusion

ContentAutomated assessmentTestovid systemTest generatorModule generatorsConclusion

Automated assessmentAssessment done by computersAssessment fazes

question / assignment selectionquestion / assessment deploymentanswer / solution gatheringanswer / solution gradingstudent / teacher notificationstatistical analysis of results

MotivationContinuous increase of enrolled studentsNo increase of teachers possibleTransfer some work load from teachers to

computers

Advantages (Large groups)Faster assessmentMoving great amount of work load from

teachersLeaving more time for more productive

aspects of teaching

Advantages (Small groups)Enables self-assessmentTeachers can easily note common problems and

difficult topicsQuestion selection can be better adjusted to

individual needsAssessment results are immediateSame questions (assignments) can be repeated

an much times as neededQuestions can contain graphics, sound,

animations, and other multimedia contentIncrease of assessment objectivity

DisadvantagesType of assessments supported

short multiple choice testsfill-in-the-blanks tests with fixed correct

answersAutomated assessment is used more in

natural sciences, and rarely in social sciences, where students write essays

Assessment of programmingProgramming assignments are perfect

example for automated assessmentBesides generic, special systems for assessing

of computer programs were also developed There are two distinct types of computer

program assessment:correctness assessment

does the student program solve given problem?optimality assessment

how efficient the student program is?

Assessing correctnessOldest correctness assessment method is

running student programs against various sets of input data

comparing output form student program with correct oneInput data

created by teacher and given to the studentcreated by the studenthidden from the student

Selecting input datafixedchosen randomly every time

Output datacompared with fixed data corresponding to given inputcompared with output from correct solution

Assessing optimalityEarly systems focused on execution speed

and precision of numeric resultsRecently attention has shifted towards

readability and code style

Assessing code styleMostly based on absolute criteria

identifier lengthline indentation...

Few attempts to use a program model

ContentAutomated assessmentTestovid systemTest generatorModule generatorsConclusion

Testovid systemDesigned for assessment of programmingBatch systemStudents can test solutions by themselvesTeacher can test all assignments at onceGenerates execution reportsUses Apache AntDesigned to fit in Svetovid system

Apache AntApache Ant is a build process automation toolSimilar to make, but written in JavaRequires Java virtual machineMost appropriate for Java projectsUses XML for description of build processesOpen source

ImplementationAs a frame for domain-specific testersOne domain-specific tester contains

Apache Ant file with testing modulesConfiguration file with weights of every moduleAny accompanying files

Configuration filetargets.all = test01, test02, test03, test04, test05, test06,

test01.name = Compilationtest01.score= 1test02.name = Up & Down methodstest02.score= 1test03.name = Up – Upper restrictiontest03.score= 2test04.name = Down – Upper restrictiontest04.score= 2test05.name = Encapsulationtest05.score= 1test06.name = Code styletest06.score= 1

List of modulesName of every

moduleNumber of points for

each module

All marks are binary

Testing module implementation<target name="test06">

<checkstyle file=“Assignment5.java" failureProperty="test06.fail" maxErrors="15"/>

<condition property="test06.advice" value=“There are more than 15 style errors."> <equals arg1="${test06.fail} " arg2="true"/> </condition>

</target>

One module is one Ant target

Result of execution is string that contains message about an error

Advantages and disadvantagesAdvantages

Platform independentFlexible and powerfulDomain-specific testers and modules are

reusableDisadvantages

Considerable time must be invested to create testers

Knowledge of Apache Ant is needed

ContentAutomated assessmentTestovid systemTest generatorModule generatorsConclusion

Test generatorGUI basedPrevious tests can be reusedModular and extensibleNo Apache Ant knowledge needed

GUI based

Previous tests can be reusedOpen and save domain-specific testersCut, copy and paste modulesCan open but not edit manually written

modulesGenerated domain-specific testers are

human-readable and can be manually tweaked

Modular and extensibleSearches for available module generatorsModule generator implements an interface

Provides GUI components for module editingProduces module implementationParses test module implementation

New module generators can be easily added

No Apache Ant knowledge neededAdd new moduleChoose module typeSet nameSet amount of pointsAdjust module

parametersSave domain-specific

tester

ContentAutomated assessmentTestovid systemTest generatorModule generatorsConclusion

Module generatorsCompleted module generators

Input-output tests for JavaCompilation tests for Java

Unfinished module generatorsInput-output test for Modula 2Compilation test for Modula 2Code style compliance test for JavaUnit testing for Java

Input-output tests for Java

Input-output tests for Java

Input-output tests for Java

Compilation tests for Java

ContentAutomated assessmentTestovid systemTest generatorModule generatorsConclusion

ConclusionTestovid systems was poorly utilized

Time consuming to produce testsSome Apache Ant knowledge needed

Test generator will probably improve usageEasier creation of testsFaster learning curve

Better picture after the following semester

Thank you for your attention