standing committee meeting nmeict 28/05/2014. developing a joint entrance examination (main) (jee)...

STANDING COMMITTEE MEETING NMEICT28/05/2014

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Shri Vineet Joshi – Chairman CBSE and Executive Council CAER

Mrs. Sarita Manuja - Director CAER

• Shri Vineet Joshi, IAS (Chairman, CBSE & Chairman CAER – Executive Council)

• Ms. Sarita Manuja (Director, Centre for Assessment, Evaluation & Research)

• Prof. James Stephen Tognolini (Senior Vice President, Research and Assessment

Pearson and Visiting Professor, University of Oxford)

• Dr. Sadhana Parashar (Professor and Director - Training and Academics, CBSE)

• Mr. Arjun Bahadur (Deputy Director, Centre for Assessment, Evaluation & Research)

• Dr. Priyanka (Research Officer, Centre for Assessment, Evaluation & Research)

• Dr. Akanksha Bapna (CEO, Evaldesign)

CBSE – CAER CORE PROJECT TEAM

Background

• JEE (Main)

High-stakes examination for which students from across the country spend significant

amounts of time practising to maximise their score

While preparation for examinations is a very useful learning tool, it starts to have a

negative impact on broader learning when the focus is purely on “practising for the

test”

The fact that students are paying significant amounts of money to have this tuition is

also impacting on the “social fabric” of society



Scope and Coverage of Solution

• The CBSE - CAER is proposing to develop an item bank of JEE (Main) items which

students preparing for the JEE can access online. These items will be linked to

learning materials which are designed to help students improve their learning

• All candidates (nationally and internationally) would have access to this powerful

“learning” tool which will addresses some of the concerns outlined in the previous

slide in that the focus is on learning

• It is designed to identify areas of potential weakness; promulgate corrective

strategies that have been identified through an analysis of the assessment items;

and, give further practice examples from similar JEE items which assess the same skill

to make sure that the student has understood the concept

• The intention is to utilise ICT tools to make the learning more interactive



Steps Involved



Develop Table of Specifications

covering the construct of JEE

Identifying and Tagging a pool of 501

Items by Subject Experts and Test

Developers of JEE.

Identifying , describing and storing the rationales behind

the distracters

Incorporating corrective learning strategies for each

incorrect option chosen

Build additional items for practice for each incorrect action to improve learning

Develop an online system for item bank including feedback

and learning components

(with technology partner)

Locate the item bank system on a website to make it accessible

world wide 24/7.

Conduct Load trials and carry out pilot

study

Process Flow



Features of Platform



Features of Platform (Contd)



Technology Used and Hosting Architecture



Programming language - Ruby v2.1.0.

Web Framework - Ruby on Rails v4.1.0

ORM - Active Record

Database - MySQL

Front end - ERB templates, jQuery, SCSS,

Bootstrap

Caching - Redis

Web Server - Nginx

Application Server - Unicorn

Server Operating System - Ubuntu 13.10

Hosting Architecture - Amazon Web Service stack

Deliverables and Timelines



Key deliverables Expected time of delivery

Inception of the Project 1 July 2014

Development of item bank with 501 items 31 Oct 2014

Completion of technology Platform along with Hosting 31 Dec 2014

Pilot run of the Platform 1 Jan 2015

Submission of report of Pilot 30 June 2015

Update of Platform basis Pilot 1 Aug 2015

Cost of the Project



Recurring Heads Annual Cost (INR)

Hosting Cost 3,00,000

Human Resource

Project Manager 12,00,000

Data Analyst 6,00,000

System Administration Cost 12,00,000

Academic Advisors 5,00,000

Total 38,00,000

Cost of the Project



Development Heads Cost (INR)

•Development of Table of Specifications (TOS)•Selection of 501 items coded with metadata described within TOS•Identify 10 items with characteristics similar to the 501 items and tagged as per TOS

14,00,000

Review and Quality Assurance 6,00,000

Building of rationales behind each incorrect option for all 501 itemsTo explain and build Corrective learning strategies for each

24,00,000

Review and Quality Assurance 6,00,000

CBSE CAER Cost of Managing the Development 24,00,000

Total 74,00,000

Cost of the Project (Contd)




Platform Development

Inception Stage (Requirements ; Analysis ; Documentation) 2,80,000

U I Design 2,00,000

Coding and Development 23,00,000

Testing and Integration 11,50,000

Infrastructure and Release 2,50,000

Total (including mandatory costs) 47,00,000

Cost of the Project (Summary)




Recurring Cost (Annual) 38,00,000

Development Cost (Content) 74,00,000

Development Cost (Platform) 47,00,000

Total 1,59,00,000

Unique Features of Solution

• Self learning model which allows learning at own pace and remediation through

pedagogic interventions

• Leveraging ICT to maximize reach and effectiveness of learning

•24/7 access by students world wide

• Can be used for student and teacher development

• Model is based on Social Equity, Access to everyone and improved Quality of

learning

• Large pool of CBSE – CAER domain experts and assessment experts



Realize Objectives of Mission

NMEICT Mission –

A proper balance between content generation, research in critical areas relating to imparting of

education and connectivity for integrating our knowledge with the advancements in other

countries is to be attempted. No Talent of the country should be allowed to go waste

Knowledge modules based on the personalized needs of the learner would need to be delivered

to him /her at the right time with the right content interactively to take care of his/her aspirations

Project Goals and Intention

This project generates content through expert intervention, ensures that all students across the

country have access to high-quality content through this online platform at minimal cost and

increases accessibility of the learning material

It allows for self-paced learning by enabling students to access the question bank and concept

details on their own time, making it highly personalized



CAER – RESEARCH PROJECTS COMPLETED WITH CBSE

• Some Possible Options for Aggregating Subject Examinations at the

Level of Scores for Entry into Indian Tertiary Institutions – JEE

Normalisation Model

• Evaluation of the Validity and Functioning of the Test (The Central

Teacher Eligibility Test, November 2012) - CTET Project

• Study of the Structure of Grade XII Question Papers for Different School

Examination Boards – 21 State Boards

• Evaluation of the Item Functioning of the Test (JEE Main- 2013) – Item

Analysis

CAER – RESEARCH PROJECTS ONGOING WITH CBSE

• Developing On-Line Problem Solving Assessment System for the

students of Grade IX and XI targeting to nurture and assess cross

curricular skills

• Evaluation of the Item Functioning of the Test (JEE Advanced – 2013)

• Examining Grade XII Examination Practices of CBSE and making

recommendations for Improvement

• Comparative Analysis of Difficulty Level of Grade X question

papers of Board based and School Based Summative Assessment (II)

• Evaluation of the Item Functioning of the Test (CTET, 2014)

Thank You!

Centre for Assessment, Evaluation and Research (CAER)4th Floor, 17 Rouse Avenue, New Delhi 110 002

Email: [email protected]: www.indiacaer.org

mailto:[email protected]

http://indiacaer.org/

standing committee meeting nmeict 28/05/2014. developing a joint entrance examination (main) (jee)...

Documents

learning components

broader learning

item bank of jee main

corrective learning

similar jee items

useful learning tool

powerful learning tool

construct of jee