e-testing based on service oriented architecture

e-Testing based on e-Testing based on Service Oriented Service Oriented ArchitectureArchitecture

Institute of InformaticsInstitute of Informatics - - FNSFNSUniversityUniversity “Ss“Ss. . Ciryl and Methodious”Ciryl and Methodious” – – Skopje, MacedoniaSkopje, Macedonia

10-th International Computer Assisted Assessment Conference10-th International Computer Assisted Assessment Conference

Goce Armenski, M.ScGoce Armenski, M.Sc([email protected])([email protected])


1.1.

CONTENTCONTENT

ContentContent

- - CCONCLUSIONONCLUSION

- - CCOMMON FRAMEWORKOMMON FRAMEWORK

- - ЕЕТТЕСТЕСТ – – SSYSTEM FOR ELECTRONIC TESTINGYSTEM FOR ELECTRONIC TESTING

- - SSERVICE ORIENTED ARCHITECTUREERVICE ORIENTED ARCHITECTURE

- - IINTRODUCTIONNTRODUCTION

- - ААRCHITECTURERCHITECTURE, , CONCEPTS AND FUNCTIONALITY OF CONCEPTS AND FUNCTIONALITY OF ЕЕТТESTEST

- - AAPPLICATION OF EPPLICATION OF ETTESTEST

- - RRESULTSESULTS

- - SOA SOA IN EIN EAASSESSMENTSSESSMENT


22..

1. 1. INTRODUCTIONINTRODUCTION

IntroductionIntroduction

Changes in the way people live influenced by:Changes in the way people live influenced by:- - GlobalizationGlobalization- - Increased meaning of knowledgeIncreased meaning of knowledge- - Information and communication revolutionInformation and communication revolution

Changes in the educationChanges in the education ( (lot more studentslot more students))- - Lecturing processLecturing process- - Delivery of materialsDelivery of materials- - Knowledge assessmentKnowledge assessment


3.3. IntroductionIntroduction

New education paradigmsNew education paradigmsTeacher Teacher Student centered education Student centered education


4.4. IntroductionIntroduction

Institute of InformaticsInstitute of Informatics ( (FNSFNS))- - every yearevery year 150+ 150+ studentsstudents- 4 - 4 questionsquestions ( (assignmentsassignments) ) x x 150 150 studentsstudents = 600 = 600 questionsquestions- 5 - 5 minmin for marking a questionfor marking a question- 3000 - 3000 minmin / 50 / 50 hourshours

How long the marking will takeHow long the marking will take??????

Ontime feedbackOntime feedback??????

Personalized feedbackPersonalized feedback??????

ObjectivityObjectivity??????

MOTIVATION AND GOALMOTIVATION AND GOAL


5.5. Motivation and goalMotivation and goal

Development of a system for assessment of large number of Development of a system for assessment of large number of students (more that 150) every month, which can be used in students (more that 150) every month, which can be used in distance learning, as well as any other form of knowledge or skill distance learning, as well as any other form of knowledge or skill assessmentassessment..

- Motivation for reports creationMotivation for reports creation Statistical analyses of gathered information Statistical analyses of gathered information

- Motivation for test constructionMotivation for test construction Different tests for every student with same weight Different tests for every student with same weight Lowering the possibility for memorizing Lowering the possibility for memorizing

- Motivation for simple data entryMotivation for simple data entry Possibility for bulk entering of data Possibility for bulk entering of data


6.6.

22. . еТеТESTEST – – CONCEPTS AND FUNCTIONALITYCONCEPTS AND FUNCTIONALITY

еТеТestest – – concepts and functionalityconcepts and functionality

- - Question bankQuestion bank, ,

- - Algorithms for test creationAlgorithms for test creation, ,

- - Systems for data presentationSystems for data presentation,,

- - Result reportsResult reports

QuestionMark, BlackBoardQuestionMark, BlackBoard,, WEB CT, Top Class WEB CT, Top Class, , EduSystem EduSystem

Basic concepts of systems for eTestingBasic concepts of systems for eTesting


7.7. еТеТestest – – concepts and functionalityconcepts and functionality

еТеТest Technologyest Technology

- Web based applicationWeb based application- - Active Server Pages (ASP)Active Server Pages (ASP)- JavaScript- JavaScript- SQL Server 2000- SQL Server 2000- NT Server - NT Server andand Win 2000 Win 2000 compatiblecompatible- IIS 4.0 - IIS 4.0 or neweror newer

Web based sollutionsWeb based sollutions VS desktop based sollutionsVS desktop based sollutions

Web Browser(Netscape 4.x or Internet Explorer 4.x, and above)

Win 2000 Server

Email(SMTP)

IIS 4.0 + ASPSQL ServerAccess

JScr

ipt



еТеТest Architectureest Architecture

Fig. 2 Three tier architecture of the eTesting system

Web Server( )system modules

user interface

application logic

data layer

Database

SQL statements Data



- - Course organizationCourse organization

- - Types of questionsTypes of questions

- - Test creation algorithmTest creation algorithm

- - System for data presentationSystem for data presentation

- - Marking and reportingMarking and reporting

еТест еТест ConceptsConcepts



- learning objectslearning objects- - three structurethree structure

Course organizationCourse organization

Fig. 3 Course organization

LECTURE

PARTA PART PARTB C

SET SET SET SETA1 A2 A3 A4



• Multichoice questionsMultichoice questions ( (choose one of many, choose many of many, yes/no choose one of many, choose many of many, yes/no answersanswers););

• Short entry answerShort entry answer ( (text or numericaltext or numerical););

• Essay answerEssay answer..

Types of questionsTypes of questions

• Questions can have pictures or graphs in the text or offered answersQuestions can have pictures or graphs in the text or offered answers



- dynamic linear testsdynamic linear tests ( (fixed number of questionsfixed number of questions))

Marking and reportingMarking and reporting

Test creation algorithmTest creation algorithm

- results are shown at the end of the testresults are shown at the end of the test- - negative markingnegative marking

Fig. 4 Algorithms for test creation

Linear Dynamiclinear

Testlets Masterymodels

Adaptive

Level of adaptation of the test



Statistical data analysesStatistical data analyses

• Identification of content which is not well presentedIdentification of content which is not well presented;;

• Personalized feedback to studentsPersonalized feedback to students;;

• Identification of week questions which need to be revised Identification of week questions which need to be revised before used againbefore used again;;

• Identifying the individual weaknesses of studentsIdentifying the individual weaknesses of students..


14.14.

44. . APPLICATION OFAPPLICATION OF еТ еТESTEST

Application of eTestApplication of eTest

- - Integration in the process of learningIntegration in the process of learning- - Controlled learningControlled learning

проучиА1

тестирајА1

проучиА2

тестирајА2

проучиА3

тестирајА3

проучиА4

тестирајА4

проучиА

( ,А А А А1 2 3 4, ,

)

тестирајА

( , )А А А А1 2, 3, 4

Fig. 5 Way of passing the learning objects

Successful strategySuccessful strategy????- all questionsall questions- NN questions in a rowquestions in a row- NN right questionsright questions- 3 3 right questions in a rowright questions in a row

Statistical analyses of the user activitiesStatistical analyses of the user activities


15.15.

55. . RESULTSRESULTS

ResultsResults

The use of technology in education is very dependent of The use of technology in education is very dependent of the organizationthe organization - logistics - logistics - social changes - social changes

Synchronization with other systemsSynchronization with other systemsTechnical infrastructureTechnical infrastructure

Practical ImplementationPractical Implementation

• Institute of InformaticsInstitute of Informatics , , FNSFNS (2001) (2001)

• А.А.DD. . MobimakMobimak (2002) (2002)

• UNDPUNDP (2003-2004) (2003-2004)


16.16. ResultsResults

Institute of InformaticsInstitute of Informatics, , FNSFNS ( (januaryjanuary 200 20011))

- 26 courses- 26 courses

- - 1239112391 questionsquestions

- 589 scheduled assessments- 589 scheduled assessments

- 9861 generated tests- 9861 generated tests

Data gatheringData gathering

Does eTesting provides more effective and more objective Does eTesting provides more effective and more objective assessment compared to the traditional forms, and does it help the assessment compared to the traditional forms, and does it help the learning processlearning process



Installed in 4 Faculties from 3 UniversitiesInstalled in 4 Faculties from 3 Universities

- not interoperable- not interoperable

- does not allow searching or using common question bankdoes not allow searching or using common question bank

- does not allow cross-institutional cooperationdoes not allow cross-institutional cooperation

- not fully standards compliantnot fully standards compliant

Problems Facing withProblems Facing with


Possible SolutionPossible Solution




19.19.

6. SERVICE ORIENTED ARCHITECTURE6. SERVICE ORIENTED ARCHITECTURE

Service Oriented ArchitectureService Oriented Architecture


IT Architecture RequirementsIT Architecture Requirements

• SimplicitySimplicity

• Software that reflects the Software that reflects the need of businessneed of business

• Agility and flexibilityAgility and flexibility

• ReusabilityReusability

• Technology independentTechnology independent

• Promote integrationPromote integration

Built to Change


Evolution of the WebEvolution of the Web

Generation 1Static HTML

HTMLHTML

Generation 2Web Applications

HTMLHTML

HTML, XMLHTML, XML

HTML, XMLHTML, XML

Generation 3Web Services

20.20. Service Oriented ArchitectureService Oriented Architecture





An approach for buildingAn approach for building distributeddistributedcomputing systemscomputing systems based on based on

encapsulatingencapsulating business functionsbusiness functions asas servicesservices that can be easily accessed that can be easily accessed

in ain a loosely coupledloosely coupled fashion.fashion.

SERVICE ORIENTED ARCHITECTURESERVICE ORIENTED ARCHITECTURE


SOA: It’s Not a ThingSOA: It’s Not a Thing

• ProductProduct

• A specific technologyA specific technology

• An applicationAn application

• A specific standardA specific standard

• A specific set of rulesA specific set of rules

It’s not a:It’s not a:

An Approach for Building Agile and Flexible An Approach for Building Agile and Flexible Business ApplicationsBusiness Applications




7. SOA in eLearning7. SOA in eLearning

23.23. SOA in eLearningSOA in eLearning


• JISC Technical Framework to support e-LearningJISC Technical Framework to support e-Learning ( (ELFELF))..

• IMS Abstract Framework (IAF)IMS Abstract Framework (IAF)

• LeAPP Learning Architecture ProjectLeAPP Learning Architecture Project

• Carnegie Mellon’s Learning Services ArchitectureCarnegie Mellon’s Learning Services Architecture


e-Learninge-Learning ( (ELFELF))

24.24. SOA in eLearningSOA in eLearning



8. SOA in eAssessment8. SOA in eAssessment

25.25. SOA in eAssessmentSOA in eAssessment


• FREMA.FREMA.

• TENCompetence Assessment ModelTENCompetence Assessment Model


FREMA Core ServicesFREMA Core Services

• Assessment ServicesAssessment Services– AssignAssign– Author itemAuthor item– Author assessmentAuthor assessment– Validate Validate

assessmentassessment– Take assessmentTake assessment– Mark assessmentMark assessment– Moderate Moderate

assessmentassessment– Grade AssessmentGrade Assessment– View FeedbackView Feedback

• Support ServicesSupport Services– ScheduleSchedule– NotifyNotify– TrackTrack– Authorise and Authorise and

AuthenticateAuthenticate

Assessment Domain Definition

Use CasesUse CasesUse Cases

Service Profiles

Gap Analysis

Reference Impl’

Common Usage Patterns




TENCompetence - Phases in the assessment TENCompetence - Phases in the assessment processprocess

Assessment DesignAssessment Design

ItemItemConstructionConstruction

AssessmentAssessmentConstructionConstruction

AssessmentAssessmentDeliveryDelivery

ResponsResponsEvaluationEvaluation

DecisionDecisionMakingMaking




eTest Core ServiceseTest Core Services

• Assessment ServicesAssessment Services

– Item ConstructionItem Construction– Test ConstructionTest Construction– Test DeliveryTest Delivery– Results CollectingResults Collecting– MarkingMarking– Decision MakingDecision Making– Statistical AnalysisStatistical Analysis

• Support ServicesSupport Services

– ScheduleSchedule– Notify and AnnounceNotify and Announce– TrackTrack– AuthenticationAuthentication– User ManagementUser Management




29.29.

SOA SOA Projects in eAssessmentProjects in eAssessment

SOA in eAssessmentSOA in eAssessment

– Research ProjectsResearch Projects• TOIA – Technologies for Interoperable TOIA – Technologies for Interoperable

assessmentassessment

• ASSIS – Assessment SequencingASSIS – Assessment Sequencing

• APIS – assessment Provision through APIS – assessment Provision through InteroperabilityInteroperability

• … … and many othersand many others

– Standards BodiesStandards Bodies• In Assessment – IMS, OSIDs … In Assessment – IMS, OSIDs …

• In Web Services - SOAP, WSDL, WSRF, W3C …In Web Services - SOAP, WSDL, WSRF, W3C …


30.30.

66. . CONCLUSION and FUTURE FORKCONCLUSION and FUTURE FORK

ConclusionConclusion

Service Oriented ArchitectureService Oriented Architecture• Advantages over monolithic applications for TestingAdvantages over monolithic applications for Testing

Review of existing SOA models for eAssessmentReview of existing SOA models for eAssessment

• Design of Common Framework by service identificationDesign of Common Framework by service identification

• Practical realizationPractical realization

Extend the ASSIS project with different test delivery modelsExtend the ASSIS project with different test delivery models

• Dependent on contextDependent on context

• Game based learningGame based learning


Thank you for Thank you for your time!your time!

e-testing based on service oriented architecture

Documents