himachal pradesh state assessment - main report
Post on 21-Feb-2018
218 Views
Preview:
TRANSCRIPT
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
1/56
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
2/56
Educational Initiatives Pvt. Ltd.
1
We would like to thank the officials of Education Department and Sarva Shiksha Abhiyan, Himachal Pradesh. The
successful completion of this assessment is attributed to help from many individuals in the department of Himachal
Government.
This study would not have been possible without support from Michael and Susan Dell Foundation (MSDF) and
Learning Links Foundation (LLF). We are highly indebted to them for their guidance and constant support for
successful roll out of State wide assessments.
Special gratitude is due to all the DIET Principals and officers, Research team-SSA and LLF team at Himachal for
their kind co-operation and the great support. We were able to carry out the assessments smoothly because of
the great support provided by the all the Government school teachers who carried out the test administration.
Last but not the least; we are also thankful to the students, and principals of the schools of Himachal thatparticipated in the Assessments.
We sincerely hope that this illustrative study will be useful to the policy makers and schools.
MR. SRIDHAR RAJAGOPALAN MR. VISHNUTEERTH AGNIHOTRI
(Managing Director) (Vice President - Assessments)
Educational Initiatives Pvt. Ltd. Educational Initiatives Pvt. Ltd.
ACKNOWLEDGEMENT
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
3/56
Educational Initiatives Pvt. Ltd.
2
Himachal State Education Department
Sarva Shiksha Abhiyan, Himachal Pradesh
Michael and Susan Dell Foundation
Learning Links Foundation
Educational Initiatives Pvt. Ltd.
Mr. Sridhar Rajagopalan, Managing Director
Mr. Vishnuteerth Agnihotri, Vice-President, Assessments
Ms. Vyjayanthi Sankar, Vice-President, Strategic Relationships
Mr. Venkatesh Reddy, Manager Communications
Ms. Neha Miglani, Research Associate
Ms. Archana Dwivedi, Research Fellow
Ms. Bharti Porwal, Educational Specialist
Ms. Swati Jha, Education Specialist
Mr. Nishchal Shukla, Educational Fellow
Ms. Sidhya Balakrishnan, Jr. Research Associate
Ms. Kaveri Malik, Programme Implementation Manager
Ms. N.V.N.S.S. Durga, Graphic Designer
Mr. Rahul Mathur, Software Development ManagerMr. Jaikishan Keswani, Jr. Software Engineer
Ms. Jayashree T.S., Educational Specialist
Mr. Devpal Shah, Manager- Design and Printing
Mr. Ketan Beherawala, Asst Manager - Accounts
Mr. Drupad Shah, Manager - Logistics
TEAM
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
4/56
Educational Initiatives Pvt. Ltd.
3
EXECUTIVE SUMMARY 4
1.
DESIGN AND IMPLEMENTATION OF THE STUDY 6
1.1 Need For The Study 6
1.2 About the Study 7
1.3 Salient Features Of The Study 8
1.4 Question Paper Design 9
2. MAIN FINDINGS 152.1 Performance Across Subjects and Classes 15
2.2 Comparative Performance Against Indian Private Schools 15
2.3
Learning with Understanding - Performance on Higher Order Questions 162.4 Performance On Practical Competencies 18
2.5 Performance Across Districts 19
2.6 Performance Of Boys And Girls 21
3. INSIGHTS ON SPECIFIC LEARNING AREAS 22
3.1. Misconceptions And Common Errors 22
3.2. Performance In Different Skills 38
4. FIELD AUDITS 42
5. A PEEK INTO THE DISTRICT REPORTS 465.1 A Birds Eye View Of The Districts Performance Across Classes And Subjects 46
5.2 Score Distribution Of Students Across Subjects, Class 47
5.3 Comparison With Other Districts (Composite Index) 48
5.4 Performance On Different Skills 49
5.5 Question-wise Performance Across Classes And Subjects 50
5.6 Subject Level Recommendations 51
6.
APPENDIX 52
Appendix A: Comparison With Asset 52
Appendix B: Difference In Performance In Straight Forward (SF) And Higher Order (HO) Questions 52
Appendix C: District-wise Performance 53
Appendix D: District-wise Composite Performance 54
CONTENT
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
5/56
Educational Initiatives Pvt. Ltd.
4
Why this Study?
Understanding the importance of a comprehensive educational policy, the state of Himachal Pradesh conducted anassessment of learning levels with Educational Initiatives, in partnership with the Michael and Susan Dell
Foundation and support from the Learning Links Foundation. The Learning Assessment was conducted in June
2013 in an attempt to undertake a scientific and systemic study to detect state learning levels. . It seeks to provide
independent third-party assessment of student learning to be used by policymakers and researchers and establish
student learning in terms of knowledge and skills (or competencies) and provide feedback on the learning gaps,
common errors and misconceptions. This report will argue that assessment must promote good education rather
than undermining it.
Design of the study
Coverage:The study was aimed to cover all the 12 districts in Himachal Pradesh. Around 1000 students per class
from each district were sampled and tested for classes 3, 5 and 7. Diagnostic assessment was conducted with
Hindi as the medium of instruction. The study included a detailed scientific process of test development supported
by curriculum checking; development of test items based on the previous studies conducted by EI, fine-tuning and
assembling of the final papers; detailed test adaptation, translation, validation of papers; master trainer workshops,
field audit, data entry, and diagnostic data analysis prior to the release of the report.
Test Format: The learning assessment was conducted for two subjects-Language (Hindi) and Mathematics for
classes 3, 5 and 7. These test papers were designed for a wider audience of MSDF grantees. For each class and
subject, one set of papers was made and adapted into Hindi language. The question papers includedstraightforward questions as well as higher order questions detecting deeper understanding and application of
learnt concepts. The test papers were completely written but the question paper was read aloud once in Class 3
Students ofclass 3 filled in their answers on the question booklets itself. At class 5 and 7 level, the students directly
marked their answers in the given OMR sheets. However, they answered the free response questions in the question
booklets. After the test, the students responses (OMRs as well as question booklets) were sent back to EI where the
class 3 complete question responses and class 5 and 7 free response questions were coded on OMR sheets by the
evaluators at EI.
Test Conduction and Analysis: All schools got the same paper to detect learning levels and to undertake a
comparative analysis of student performance. Further, in order to benchmark achievement of Himachal Pradesh
students with those of Indian private schools (not low-cost schools), sufficient number of test items were used from
ASSET1
1ASSET is a diagnostic test from Educational Initiatives, which is administered every year to more than four lakh students studying in Indian private
schools (not low-cost). Details can be obtained through www.ei-india.com.
.
A team of EI assessment experts conducted training for Principalsof District Institute of Education & Training
(DIET) as well as some District Primary Education Officers (DPEOs), District Education Officers (DEOs) to ensure
standardized test administration. Test conduction was managed by the State of Himachal Pradesh with the help of
LLF. EI teams also carried out field audits during the day of the test in over 3 schools spread across 5 districts.
EXECUTIVE SUMMARY
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
6/56
Educational Initiatives Pvt. Ltd.
5
Based on the student performance on these tests, different types of analyses were carried out using advanced
statistical methods to extract patterns in performances and to understand differences in learning levels across
different groups and patterns in learning. The findings from this analysis have been presented in this report.
MAIN FINDINGS
Based on the analysis of student performance at the school, district as well as the state level, several key findings
have been presented below and detailed in the sections that follow.
1. The state average performance ranges from 48.5% to 59.5% in Hindi and 38.1% to 59.7% in Maths
across the three tested classes. It was observed that the performance was more homogenous within
higher classes, (indicated by lower standard deviation).
2. Students in the State perform lower than the Indian Private School Average (assessed through ASSET),
typically by 5-13 percentage points. Further, this gap between the state and ASSET performance increases
in the higher classes.
3. Lower performance of students was observed in higher order questions involving deeper understanding of
concepts. This indicated that learning with understanding is weak, as the students find it difficult to applylearnt concepts in unfamiliar question formats.
4. The students were observed to be the weakest in questions testing practical competencies such as visual
interpretation, map reading, observation and estimation. Practical competencies are essential for the
students all round development and this is a matter of concern.
5. A comparative analysis of the performance of all the districts revealed that Hamirpur outperformed all
other districts, performing significantly better than the state average. At the same time, the districts of
Chamba, Bilaspur and Sirmaur observed lower performances, significantly below the state average.
6. Students seem to have acquired several misconceptions during the process of learning. These were
observed amongst the students across competencies and subjects. These misconceptions can arise at any
time, and should be tackled at the earliest through strategic pedagogical improvements.
7. The student performance was weak in particular skills of Hindi and Maths. These include intermediate
skills of Fractions, Surface Area and Volume and Integers in Maths and reading comprehension, particularly
in inference based questions and passages based on authentic material in Language.
These findings have been comprehensively discussed in chapters 2 and 3 of this report. The report will present
these findings with relevant statistical methods, representation and sample questions.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
7/56
Educational Initiatives Pvt. Ltd.
6
1.1 NEED FOR THE STUDY
Education is recognized across the world as perhaps the most vital public service of all. However, the
measurement of its impact is still in question. Assessments are essential to ensure that students get the educational
support that they need to succeed, to see the effectiveness of different educational methods, and to ensure that
education budgets are being utilized effectively. Therefore, inevitably, assessments also risk marking teachers,
learners and institutions as successes or failures.
Various Assessments have been in existence as an informal tool for even longer than the actual annual school tests.
The emphasis has been on ways in which an assessment can help to advance learning rather than merely
measuring it. Current developments in education; such as the reinvention of assessment processes, suggest that
the views of different experts on the correct and incorrect ways to assess educational progress are becoming
more influential, and the research projects have helped to build on the evidence needed to improve assessments,
and the educational outcomes which it is meant to focus on.
In India, independent assessments conducted by organisations like NCERT, ASER and Educational Initiatives show
that students in Indias Government schools are not learning well, and not acquiring the expected competencies in
basic literacy and numeracy.
EI has been carrying out research in learning outcomes for several years. It is a pleasure to acknowledge
the Government of Himachal Pradesh and especially all officials of the Education Department for their contribution
in advancing our ideas about appropriate ways of assessing educational outcomes.
Diagnostic scores are of increasing interest in educational testing due to their potential benefits, both remedial and
instructional. Naturally, the number of educational tests that report diagnostic scores is on the rise across the
globe, as are the number of researches based on such scores.
This report provides a critical evaluation of the diagnostic testing attempted by 28,898 students from classes 3, 5
and 7 across the state of Himachal Pradesh. It is demonstrated, using results from operational and simulated data,
that diagnostic scores have to be based on a sufficient number of items and have to be sufficiently distinct from
each other to be worth reporting and that several operationally reported sub-scores are actually not worth
reporting. The format of the state-wide diagnostic testing has taken care of these basic elements.
A diagnostic test helps to identify a student's learning problems, so that teachers can provide specific
instructions to remedy them. Moreover, teachers must often undertake complicated and time-consuming
analyses of students' responses to individual items. Legitimate diagnostic tests permit teachers to use a test's
results without having to devote hours to intricate interpretations.
This report on the Diagnostic Assessment in the state of Himachal Pradesh is a joint initiative undertaken by the
Education Department Govt. of Himachal Pradesh along with Sarva Shiksha Abhiyan (SSA) HP, Learning Links
Foundation (LLF), Michael & Susan Dell Foundation (MSDF) and Educational Initiatives (EI).
1. DESIGN AND IMPLEMENTATION OF THE STUDY
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
8/56
Educational Initiatives Pvt. Ltd.
7
1.2 ABOUT THE STUDY
In its first year of conducting this study, Educational Initiatives conducted diagnostic assessments for 1310 schools
for class 3, 1246 schools for class 5 and 401 schools for class 7 (chosen as sample) across the 12 districts which
fall under the jurisdiction of Education Department, Himachal Pradesh.
The tests were done for classes 3, 5 and 7 in Language (Hindi) and Maths.
Around 1000 students per class from each district were sampled and tested for classes 3, 5 and 7. Only classes
with Hindi as the medium of instruction were covered in the diagnostic assessment. The sampling was performed
by the State based on several criteria. It was ensured that in smaller districts almost all blocks are covered. In
larger districts all blocks that are different in demography were covered to ensure a complete representation of
the district. The clusters inside a block were chosen to represent different demographics of the block. First the
class 7 schools were selected (these are secondary schools). Primary schools attached to secondary schools were
selected post that. It was ensured that 1000 students from class 7 were selected. More primary schools were
selected in same clusters to ensure 1000 students in each grade (3, 5). In some districts the total population is very
less hence the sample size is low.
This report will argue that assessment must, in all cases, promote, rather than undermine, good education.Policy-
makers need to keep the needs of pupils at the fore, and ensure that any evaluation of new developments in an
assessment is carried out with careful consideration of the consequences, both intended and unintended.
It is helpful to make a distinction here between the intended uses of assessment data, and their actual uses.
Assessments are often designed quite differently to ensure their adaptability for different purposes. We must
therefore deliberate on the different ways in which such assessments are used. Paul Newton has identified 22 such
uses2
1. Be part of effective planning
. These are, however, only broad categories. If one considers each purpose in detail, the number of uses for
the data can multiply.
Similarly, the term assessment for learning is often interchangeably used as formative assessment. In 1999, the
Assessment Reform Group defined assessment for learning as the process of seeking and interpreting evidence
for use by learners and their teachers to decide where the learners are in their learning, where they need to go
and how best to get there. ARG also identified ten principles for formative assessment practice, arguing that it
should:
2. Focus on how pupils learn
3.
Be central to classroom practice
4. Be a key professional skill
5. Be sensitive and constructive
6. Foster motivation
7. Promote understanding of goals and criteria
8. Help learners know how to improve
9. Develop the capacity for self-assessment
10.
Recognize all educational achievement
2This list was submitted as part of the evidence from QCA to the inquiry on assessment and testing conducted by the House of Commons Children,
Schools and Families Committee in 2007/08. See: http://www.publications.parliament.uk/pa/cm200708/cmselect/cmchilsch/169/16906.htm#n35
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
9/56
Educational Initiatives Pvt. Ltd.
8
The persistent question thus is what is required to fulfill the aim of formative feedback; i.e., help learners to grow
in capacity to manage their own learning, in order to progress effectively in developing capability in specific subject
domains. As highlighted by the previous definition, learning can hence, be seen as a journey in which learners need
to know where they are at present, where they are going, and how to get from one place to another.
The goals of the diagnostic assessment were to:
Provide independent third-party assessment of student learning to be used by policymakers andresearchers.
Establish student learning in terms of knowledge and skills (or competencies) and provide feedback on the
learning gaps, common errors and misconceptions.
Provide analysis of student performance on specific competencies for classes tested.
Analyze the learning patterns.
Identify (if any) significant differences in performances across districts.
Identify (if any) significant differences between the performance of boys and girls
Diagnose the learning issues at the systemic level and provide pointers for further research.
In short, this diagnostic assessment report is a comprehensive study that aims to provide detailed question-wise
and student-wise data for understanding and monitoring learning across classes.
1.3 SALIENT FEATURES OF THE STUDY
Coverage: This was a sample study covering around 1000
students per class per district from the government schools
across 12 districts of Himachal Pradesh. All the students had
Hindi as the medium of instructions. Details of the coverage ofthe assessment are given in the table alongside.
Scientifically Designed Test Development Cycle:The study
included a detailed scientific process of test development
supported by curriculum checking; development of test items
based on the previous studies conducted by EI, fine-tuning and
assembling of the final papers; detailed test adaptation,
translation, validation of papers; master trainer workshops, field
audit, data entry, and diagnostic data analysis prior to the release
of the report.
Specially Constructed Papers: The test papers for the
students were drawn from EIs tests particularly designed for a
wider pool of audience of the Michael and Susan Dell Foundation.
These tests assess basic competencies as well as deeper
conceptual understanding of the students through straightforward and higher order questions testing
understanding and application of concepts, reasoning ability and critical thinking.
Benchmarking:In order to benchmark achievement of Himachal Pradesh students with those of Indian private schools
(not low-cost schools) sufficient test items were used from ASSET
3
3ASSET is a diagnostic test from Educational Initiatives, which is administered every year to more than four lakh students studying in Indian private
schools (not low-cost). Details can be obtained through www.ei-india.com.
.
District
Planned
Sample
(as shared
by the state)
Actual
Sample*
BILASPUR 2,861 2,225
CHAMBA 3,816 2,904
HAMIRPUR 3,439 2,704
KANGRA 3,270 2,536
KINNAUR 791 319
KULLU 3,272 2,544
LAHUL & SPITI 785 501
MANDI 3,491 2,745
SHIMLA 3,474 2,606
SIRMAUR 3,773 2,699
SOLAN 3,403 2,266
UNA 3,704 2,712
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
10/56
Educational Initiatives Pvt. Ltd.
9
Papers:For each class and subject, one set of papers was made and adapted into Hindi language. All schools got
the same paper set for reasons of testing level and comparison across districts.
Specially Trained Test Administrators and Evaluators: A team of EI assessment experts conducted the
training of Principalsof District Institute of Education & Training (DIET) to roll out the test evaluators in their
respective districts and to ensure standardized test administration. Test dates were decided by the State during
the master trainers workshop after consulting the present DIET principals. It was decided that the master trainers
would impart training to the teachers in their respective blocks and clusters on similar lines.
Conducting the tests: Test conduction was managed by the State of Himachal Pradesh with help from LLF.
Printed papers were dispatched from Shimla to all the districts through DIET offices. The papers were completely
written, however, the papers were read out aloud once for class 3. The teachers of the respective school
invigilated the test after receiving appropriate trainings. The test timings were in line with the school timings of
that area, however almost all the schools conducted the test on the same day.
Field Audits: EI teams carried out field audits during the testing day to check the quality and fairness of test
administration in different schools. On the testing day, 31 schools spread across 5 districts were spot-visited by a
team of 10 members. These 5 districts - Kangra, Kullu, Shimla, Sirmaur, Mandi - were chosen to cover parts of
lower, central and upper Himachal. The team provided feedback to the test implementation and management
team to ensure that the test administration was carried out smoothly. A report on the field audit was prepared by
collating the feedback from all the team members and has been submitted as a part of this report in Chapter 4.
Data Entry:Students ofclass 3 filled in their answers on the question booklets itself. At class 5 and 7 level, the students
directly scored their answers in the given OMR sheets. However, they answered the free response questions in the
question booklets. After the test, the students responses (OMRs as well as question booklets) were sent back to EI
where the class 3 complete question responses and class 5 and 7 free response questions were coded on OMR sheets
by the evaluators at EI.
Analyses:Different types of analyses were carried out on the collected data to extract patterns in performances
and to understand differences in learning levels across different groups. Advanced statistical methods were used to
confirm patterns of learning. Distracter analysis enabled identification of misconceptions and common errors. The
data was studied across different units of analysis: schools, districts, competencies, gender, etc. Tests of statistical
significance were done to compare the performance of districts and girls and boys.
1.4 QUESTION PAPER DESIGN
Most education tests measure the status of students with respect to a cognitive skill or body of knowledge, which
we can refer to as assessed attributes. The test needs to include a sufficient number of items to measure each
assessed attribute so that the teacher can arrive at a reasonably accurate inference about how an individual studentstands with regard to each of these attributes. For instance, if the teacher wants to know whether a student can
multiply pairs of double-digit numbers, one or two items in a test won't provide an accurate estimate. The number
of items required, of course, will depend on the nature of the skill or body of knowledge being measured, but one
item per measured attribute definitely doesn't serve the purpose.
To arrive at a sound inference about student competency, the test should (1) measure a modest number of
significant, high-priority cognitive skills or bodies of knowledge; (2) include enough items for each assessed
attribute to enable teachers to make a reasonably accurate judgment on a test taker's mastery of that attribute; (3)
describe with clarity what the test is assessing; and (4) not be too complicated or time-consuming.
The purpose of this assessment is to provide a detailed diagnostic snapshot on the strengths (students achievements)
and weaknesses (learning gaps). The assessments were developed for Language and Maths for Classes 3, 5 and 7.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
11/56
Educational Initiatives Pvt. Ltd.
10
Paper Format:A simple, easy-to-administer test as well as its results may be better accepted by teachers and
other stakeholders. This will also ensure that the findings of the assessments would be used to improve teaching
and learning. The format of the test papers has been summarized below:
The written test had items that were read and answered by students themselves. However, all the
questions that appear in the written test for Class 3 were read out by the evaluators at the beginning of
the test to familiarise the students with the questions.
The Written test included Multiple Choice Questions, where the students chose an answer from four given
options; and Free Response questions, where the student wrote down his/her answer in a fill-in-blanks format.
The students in class 3 marked their answers on the question papers itself, and these responses were then
recorded on the OMR sheet by a team of evaluation coders at EI. The students of class 5 and 7 recorded
their own responses on the OMR sheet, but Free Response questions were answered on the question
paper itself and similarly these questions were recorded on the OMR later.
The invigilator manual that was provided along with the test papers. This had general and specific
instructions on what needs to be read out to the students for each question in each test paper etc.
Competency Framework:The tests were comprehensive and included a number of competencies to ensure a
more reliable measurement of student achievement. National Curriculum Framework documents, the Minimum
Learning Levels (MLLs), the state textbooks, standard international framework of tests such as TIMSS and PIRLS,
and EIs large-scale benchmarking studies for Government schools, were referred to while finalizing the
competencies for each paper. The tests were thus competency-based such that every question was linked to a
particular competency in the skill framework or the test blueprint.
Types of Questions: The tests included questions to test not just knowledge (recall and procedure) but
understanding and higher order skills. The focus of the questions was on testing for learning with understanding,
i.e., the real understanding of students in concepts they have learnt in their specific classes. A percentage of
questions were also drawn from EIs national level benchmarking test, ASSET.
The following table provides details about the classes, subjects tested, paper length, duration of each paper, the
proportion of free response (FR) and multiple choice questions (MCQs) and the proportion of straight forward
(SF) and higher order (HO) questions:
Subject Class # Questions
# SF Questions # HO Questions
Duration# FRQuestions
# MCQs # FRQuestions
# MCQs
Maths
3 26 -- 13 -- 13 90 mins
5 32 -- 16 -- 16 105 mins
7 36 -- 18 -- 18 105 mins
Language
(Hindi)
3 32 8 8 1 15 105 mins
5 38 2 16 5 15 105 mins
7 44 2 20 3 19 105 mins
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
12/56
Educational Initiatives Pvt. Ltd.
11
Sample Questions
A.
Questions that check for learning that is straightforward or text-bookish
Generally, it is believed that though the students may not be learning with conceptual understanding, they perform well
on questions which test concepts in a format that is familiar to them. This can be tested by placing questions which are
similar to examples or questions of text books. They are described as straight forward or text-bookish questions.
B.
Higher Order Questions - that check for a deeper understanding of concepts:
While students perform well on easy and familiar questions, these questions check for a conceptual understanding
of the subject in question and whether students can apply what they have learnt in different formats.
C.Higher Order Questions - that check for application of concepts
These types of questions checks the students ability to answer the questions based reasoning, visual interpretation, and
estimation. These are the kind of questions which students are faced with in their day-to-day interaction with the world.
MathsClass 3HindiClass 5
MathsClass 7MathsClass 3
MathsClass 7
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
13/56
Educational Initiatives Pvt. Ltd.
12
D. Questions that check for Reading Comprehension
Several questions in the Language papers are based on unseen passages and authentic material seen in daily life such as
tables, labels, notices, etc. These passages test comprehension which is both fact based and implied in the text.
Scoring Process: At EI, test papers for every class are supplemented with a scoring card. The scoring card
provided a question-wise rubric (with scores / codes / answers) for how each response is to be scored. The scoringcard captured common mistakes that children tend to make while answering questions. The data that was collected
by this process was critical for the diagnostic feedback and analyses. This form of scoring is common international
practice in tests such as TIMSS.
MathsClass 5
MathsClass 7
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
14/56
Educational Initiatives Pvt. Ltd.
13
The evaluators from EI for class 3 captured the answer responses of the students in the form of codes as specified
in the scoring card and entered them on to ICR and OMR sheet. The codes thus entered were scanned using
intelligent character recognition (ICR) and optical mark recognition techniques and was converted
into a database of student responses for further analysis. This eliminated human error while transcribing, etc., and
also saved time. While coding responses for multiple choice items, only the option letter ticked was to be
recorded. For free response items, the responses were recorded based on the detailed scoring rubrics. All
scoring cards also had general codes assigned for specific types of responses common to all papers. The details are
given below:
General Codes Assigned Across Papers:
1. For questions with options A, B, C, D: Option ticked (enter option A, B, C, and D in English).
2. Items that have not been attempted at all should be coded as 88.
3. Invalid answers (e.g. question copied, more than one option ticked in multiple choice questions, crossed
out answers, illegible answers) should be coded as 86.
4.
Spellings are not to be checked unless indicated specifically.
5. Instead of ticking, if a child crosses or circles or marks an option in any other way, or rewrites one of the
options, that option should be treated as their choice.
6. Free Response Questions were coded as follows:
a. Correct Answer - 01
b. Partly Correct Answer- codes vary with every question - 11, 21, 31 (depending on the type of FR
question)
c.
Any other answer - 85
Sample Test Items and Their Scoring Rubrics:
Class 7 Maths - MCQ Question
Option ticked (enter option A,B,C,D in English) A
B
C
D
Invalid Answer Code/More Than One Option Ticked 86
Not Attempted 88
MathsClass 7
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
15/56
Educational Initiatives Pvt. Ltd.
14
Class 5 Language- Free Response Question
Meaning comes out clearly; no grammatical errors; sentence complete and related to the given picture 1
Meaning comes out clearly; 1-2 grammatical errors; sentence complete and related to the given picture 11
Meaning not completely clear; some grammatical errors; sentence incomplete but related to the given picture 21
Meaning not clear at all but 1-2 appropriate words related to the given picture are used 31
Sentence not related to the given picture 81
Any other answer 85
Invalid Answer Code/More Than One Option Ticked 86
Not Attempted 88
LanguageClass 5
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
16/56
Educational Initiatives Pvt. Ltd.
15
2.1 PERFORMANCE ACROSS SUBJECTS AND CLASSES
The overall performance for each class and subject is presented in the table below. The state average performance
ranges from 48.5% to 59.5% in Hindi and 38.1% to 59.7% in Maths across the three tested classes. It was
observed that the performance was more homogenous within higher classes, indicated by lower standard
deviation.
Subject ClassN
(Total students)
% Score
(Average)Std. Dev.
Language (Hindi)
3 8,231 59.5 % 21.4
5 9,151 59.0 % 18.4
7 9,516 48.5 % 15.6
Maths
3 8,231 59.7 % 24.3
5 9,151 48.2 % 20.6
7 9,516 38.1 % 18.4
While the overall performance in Hindi has been largely consistent in Classes 3 and 5, the dip in performance in
Hindi class 7 and the decline in overall performance across classes in Maths cannot be explained through an
analysis of the average scores. A comparative analysis may not suffice for this purpose, and a system of
benchmarks against which the scores can be analysed is essential. A question-wise analysis is also essential to
understand what particular skills and competencies the students are not performing well in, and whether the
decline in the overall performance can be traced to weaker conceptual understanding in particular skills.
2.2 COMPARATIVE PERFORMANCE AGAINST INDIAN PRIVATE SCHOOLS
A comparative analysis was undertaken between the performance of Himachal Pradesh Students and the
performance of Indian private schools as represented by ASSET (A diagnostic benchmarking national level test) in
English, Maths, Science, Hindi and Social Science. Himachal Pradesh students typically score lower than Indian
Private Schools.
2. MAIN FINDINGS
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
17/56
Educational Initiatives Pvt. Ltd.
16
Sufficient common questions4
(7 -11 questions in each paper) were used from ASSET to compare the
performance of Himachal Pradesh students with English medium private schools. Gap between Himachal Pradesh
students and students of Indian Private Schools was found to be comparatively high in Class 7 and lower for Class
3 (Hindi and Maths). A closer look at Class 3 questions reveals that Himachal Pradesh students have performed
better than Indian Private School students on certain questions. In Language, for all the classes those questions
wherein students had to infer from the given passages/ stories/ notices, were found to be low performing
compared to those questions which had answers directly stated in the passages/ stories/ notices. The examples
below illustrate the difference seen between Himachal Pradesh schools and Indian private schools on commonASSET questions.
2.3 LEARNING WITH UNDERSTANDING
A high level analysis of questions which tested higher order (HO) skills as opposed to questions which were straight
forward (SF) reveals that students have performed better on SF questions. Broad trend on these lines also indicates that
performance on HO questions goes down as we move higher classes.
425% ASSET items used for the analysis
MathsClass 7MathsClass 3
Sample Question: This question tests students understanding of
operations and the relation between them. 50% of HP students
answered this question correctly compared to 46% of ASSET students.
Sample Question: This question checks if students are able to observe
a pattern and logically think of a general rule being followed in that
pattern. Only 29% of HP students answered this question correctlycompared to 49% of ASSET students.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
18/56
Educational Initiatives Pvt. Ltd.
17
Deeper analysis of question-wise performance of students highlights that learning with understanding is weaker in
higher classes for questions testing conceptual learning. This indicates that the conceptual understanding in
specific topics is not deep enough for the student to understand and attempt questions that are presented in a
non-familiar format or which go beyond the textbook. Rote-based learning among the students indicates that
conceptual understanding is not entrenched enough for the student to progress to higher order questions and
skills which require a strong basic understanding of the concept. This has been explained though the examples
given below.
This question tests the students understanding of basic shapes. Only half of the students of class 5 have been able
to correctly identify the rectangle among the given shapes. About 28% of students have chosen option C, a
rounded rectangle. This indicates that these students are not able to recognise a rectangle in a non-standard
orientation. They may have a mental picture of a rectangle in a standard orientation, and may have chosen the
image that is closest to this picture, irrespective of whether it satisfies all the properties of a rectangle or not.
MathsClass 5
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
19/56
Educational Initiatives Pvt. Ltd.
18
This question tests the understanding of place value system in decimals. Students are expected to realise that 7
tenths and 8 thousandths make 0.708. Only about 24% of students have been able to answer this correctly.
Students may be used to converting conventional expanded forms like 7/10 + 0/100 + 8/ 1000 into decimals. But
here when the conventional order is changed and hundredths are missing, many students may have been confused.
About 34% have chosen option C, 0.8007 and 15% have answered D, 0.87, retaining the order in which the
numbers appear in the question.
MathsClass 7
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
20/56
Educational Initiatives Pvt. Ltd.
19
2.4 PERFORMANCE ON PRACTICAL COMPETENCIES
Practical competencies are defined as visual interpretation, map reading, observation and estimation- all of which
are essential skills for daily life and for the overall development of the student, irrespective of the level of
education that student pursues in the future. Based on the student performance on questions testing these skills, it
was observed that the overall performance is low in these questions. An example of a question testing practical
competencies is given below:
This question checks if students are able to estimate the length of an object relative to another given object. Only
38% students have answered this question correctly. Around 30% students have chosen the wrong answer option
A. They have probably just looked at the height of the scale given in the image and chosen 100 cm as their answer.
For more detailed analysis by skill and question, please refer to Chapter 4 of this report. The skills, questions have
also been analysed at a district level to understand the strengths and weaknesses in competencies for a particular
district. A district wise analysis report for each district is also shared with this report.
MathsClass 5
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
21/56
Educational Initiatives Pvt. Ltd.
20
2.5 PERFORMANCE ACROSS DISTRICTS
2.5.1 District Performance on Composite Performance Index
To have a comprehensive understanding of the states
performance at all levels, a district wise analysis was
performed and relative performance of the districts
were analysed. A Composite Performance Index was
calculated for all districts, which is a simple average of
performance across classes and subjects. This
composite index was calculated for all districts and
their scores on this index were analysed.
At the same time, an analysis only based on an overall
performance may be insufficient as well as misleading;
a particular district may be the top performer, but
may have scores very close to the average. Toincorporate differences of the district scores from the
state average, effect sizes were calculated for each
district through Cohens D5
Based on the district wise performance on this index
and their effect sizes, a few observations have been
presented.
, which would highlight if
the districts performance is significantly higher or
lower than the state average.
Hamirpur outperformed all districts, with a score significantly higher than the state average.
Chamba, Bilaspur and Sirmaur were among the bottom three districts when compared to state average
Kinnaur and Lahul & Spiti removed from the district-wise analysis due to low sample size for both the districts
The remaining districts of Mandi, Kangra, Kullu, Solan, Una and Shimla performed close to the state average
score.
5Cohen's d is defined as the difference between two means divided by a standard deviation for the data
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
22/56
Educational Initiatives Pvt. Ltd.
21
2.5.2 District Performance across classes and subjects
The composite performance index was also calculated separately for each subject to analyse if the performances of
the districts vary between the two subjects. The difference of this score from the state average was also calculated
through the Cohens D.
Based on the district wise performance on the index in Hindi and Maths, observations have been presented below
Hamirpur was the top performer in both subjects, performing significantly better than the state average
Mandi performed significantly better than other districts and the state average in Maths, but performed
close to average in Hindi, leading to a decline in its overall performance.
For a detailed view on each district, please refer to the district wise report (provided for every district) along
with this report.
MathHindi
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
23/56
Educational Initiatives Pvt. Ltd.
22
2.6. PERFORMANCE OF BOYS AND GIRLS
Over 13 thousand girls and a similar number of boys participated in the study. This section of report attempts to
analyse any noticeable difference between the average performance of the boys and girls in the study. The table
below indicates the mean difference in the performance according to gender.
Class SubjectGirl Boys T
Criticvalue
Cohen's
d
Effect Size-
interpretationN Avg SD N Avg SD
3 Hindi 4189 60.5 21.2 3956 58.4 21.4 4.4 0.1 -
5 Hindi 4409 59.6 18.4 4716 58.6 18.3 2.6 0.05 -
7 Hindi 4682 49.4 15.8 4830 47.6 15.3 5.6 0.12 -
3 Maths 4189 60.6 24.3 3956 58.6 24.3 3.7 0.08 -
5 Maths 4409 48.6 20.6 4716 48 20.6 1.4 0.03 -
7 Maths 4682 38 18.4 4830 38.1 18.4 -0.3 -0.01 -
On comparing performances according to gender, it was observed that girls perform better than boys in Hindi
class 3, 5 and 7 and Maths class 3 but the magnitude of difference was marginal for all classes. In Maths class 5 and
7, both boys and girls performed similarly.
Legend
Boys and girls are performing at the same level.
Girls are performing significantly better than boys with
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
24/56
Educational Initiatives Pvt. Ltd.
23
3.1 MISCONCEPTIONS AND COMMON ERRORS
Students develop a cognitive understanding of the world around them through interactions based on their daily
experiences. Teachers and schools help build this understanding. Misconceptions are concepts that students
acquire which are not in line with or do not match the conventional expert view in that topic. These result in
cognitive gaps in their understanding. Identifying the exact nature of student misconceptions is difficult through
regular classroom interactions. Any question in an assessment test that attempts to identify the misconceptions of
the students must force the students to actively use their conceptual understanding. A detailed understanding of
these misconceptions through large scale diagnostic assessments provides the teacher with a starting point to
explore these in the classroom and eventually help to build correct notions or conceptual understanding. Detailed
analysis of the answers given by students in each question in all the subjects tested reveal that students
understanding of the concepts is often faulty as they seem to harbor many mistaken notions. They also seem tocommonly make errors in answering some questions. Identifying the exact nature of student misconceptions is
difficult through regular classroom interactions. Any question in an assessment test that attempts to identify the
misconceptions of the students must force the students. Some misconceptions and common errors found in the
analysis of HP state study include:
The bigger the area of a shape, the larger will be its perimeter
Confusing Part-Whole relationship with Part-Part relationship in Fractions and Ratios
Integers can be added without checking their positive/ negative signs
Using prior knowledge to answer reading comprehension questions
An item wise analysis of all the questions reveals these patterns through a graphical representation known as Item
Response Curve (IRC). A short description on how to read an IRC is given below.
How to read the graph:
These are the item response curves for a
question. The X axis is the total score in the
paper and the Y axis is the percentage ofstudents. The graphs show what percentage of
students scoring different total scores in each
class, chose which answer options for the
question. The legend box shows the total
percentage of students who chose each option
in the paper. In this graph, the percentage of
students choosing option A increases as one
moves from a low total score to a high total
score, while percentage of students choosing
option D increases up to a certain total score
and then starts reducing.
y
x
3. INSIGHTS ON SPECIFIC LEARNING AREAS
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
25/56
Educational Initiatives Pvt. Ltd.
24
3.1.1 Misconceptions and Common Errors in Mathematics
Question Graphs
Class 3 / Question 15
Explanation: This question checks if
students can understand how a paper
would look like when it is folded, cut and
then unfolded. Students are expected to
visualize the shape that would be formedafter the paper is unfolded. Only 30%
students chose the correct answer (D).
However, a lot of students chose option C
(19%) and were probably just looking at
the image in the question and choosing this
option, not realizing that when the paper is
folded into half and cut, double the part of
the paper will be cut. Around 35%
students have not attempted this question.
Class 3 / Question 11
Explanation: This question tests the
understanding of place value. About 50% of
students have answered it correctly as
option A, and 22% have answered it as C.
These students may have mechanically put
down the digits in the order in which they
appear in the question. They may not have
noticed that the question asks for 3 ones
and 7 tens. This could be because they are
used to solving problems in the
conventional order 7 tens and 3 ones and
are not sufficiently exposed to equivalent
alternate forms of writing the expandedforms of a number.
x
y
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
26/56
Educational Initiatives Pvt. Ltd.
25
Question Graphs
Class 5 / Question 3
Explanation: This question checks if
students are able to represent a part of a
group as a fraction. Students are expected
to understand that 10 students out of total
40 (30 girls + 10 boys) are boys and hence
the fraction of boys would be . However
a lot of students have chosen option A
(25%) and option C (39%). Students
choosing option A probably are notconsidering the fact that the total number
of students is 40 and are calculating the
ratio of boys to girls rather than the fraction
of boys in the class. Students choosing
option C are probably making the same
error but are giving the ratio of girls to
boys. These students have not been able to
differentiate a part-whole relationship from
a part-part relationship.
Class 5 / Question 11
Explanation: This question checks if
students can apply fractions to the units of
capacity. Students are expected tounderstand that a litre has 1000 millilitres
and litre would be equal 250 millilitres.
Only 6% students have chosen the correct
answer option A. However, most of the
students (41%) have chosen option B.
These students seem to have the
understanding of fractions but are probably
choosing this option as part of the jug is
filled here. Students choosing option C
(31%) may be matching the 4 of with
400 ml. They do not seem to understandthe concept of fractions.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
27/56
Educational Initiatives Pvt. Ltd.
26
Question Graphs
Class 5 / Question 4
Explanation: This question tests the
understanding of perimeter as the length of
the boundary of a shape. Students are
expected to identify the shape that has the
longest boundary. But almost 31% of
students have answered D. Possibly they
are not able to differentiate between the
terms area and perimeter, and have chosen
the piece that has the largest area. It is alsopossible that they have a notion that the
shape with the largest area has the largest
perimeter too.
Class 5 / Question 5
Explanation: This question tests the
students understanding of basic shapes.
Only half of the students have been able to
correctly identify the rectangle among the
given shapes. About 28% of students have
chosen option C, a rounded rectangle. This
indicates that these students are not able to
recognize a rectangle in a non-standard
orientation. They may have a mentalpicture of a rectangle in a standard
orientation, and may have chosen the image
that is closest to this picture, irrespective of
whether it satisfies all the properties of a
rectangle or not.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
28/56
Educational Initiatives Pvt. Ltd.
27
Question Graphs
Class 5 / Question 13
Explanation: This question checks if the
students can read a pictograph and answer
questions based on it. The students are
expected to understand from the key that
since the picture of one book stands for 2
books, Reena read total 32 books in her
vacation. Only 27% of the students chose
the correct answer. Students choosing
option C (49%) have probably notunderstood the key or not used it at all.
They have just counted the number of
books and chosen this option.
Class 5 / Question 32
Explanation: This question checks if
students are able to estimate the length of
an object relative to another given object.
Only 38% students have answered this
question correctly. Around 30% students
have chosen the wrong answer option A.
They have probably just looked at the
height of the scale given in the image and
chosen 100 cm as their answer.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
29/56
Educational Initiatives Pvt. Ltd.
28
Question Graphs
Class 7 / Question 3
Explanation: This question checks if
students are able to logically solve a
problem based on a daily life context.
Students are expected to understand that
they need 3 more marbles for equal
distribution and look for a number which
would become divisible by 10 on adding 3.
Only 32% students have answered this
question correctly. An equal percentage ofstudents (32%) have chosen the wrong
answer option A. They probably have not
understood the question and have just
added the numbers given in the question.
Class 7 / Question 10
Explanation: This question checks if
students are able to find the sum of given
integers. Only 36% students have
answered this question correctly. A lot of
students (31%) have chosen the wrong
answer option A. They probably haveadded all the 3 numbers and used the
negative sign with the answer. This
indicates that the concept of
integers/addition or subtraction of integers
is not clear to them.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
30/56
Educational Initiatives Pvt. Ltd.
29
Question Graphs
Class 7 / Question 14
Explanation: This question checks if
students are able to find the perimeter of a
given figure. They are expected to
understand that the side of each small
square on the grid is 1 cm and then
calculate the perimeter based on that. Only
23% students have chosen the correct
answer option C. A lot of students (41%)
have chosen option A. They have probablyeither just counted the number of small
squares that are shaded or have found out
the area instead of the perimeter. One
reason for this could be that traditionally
such grids in the textbooks are used to find
out the areas and not perimeters and
students are also confused between the
two concepts area and perimeter.
Class 7 / Question 23
Explanation: This question checks if
students are able to add two fractions withdifferent denominators when given in a real
life context. Only 32% students have
answered this question correctly. A lot of
students (34%) have chosen option D.
They have probably just added the
numerators and denominators separately.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
31/56
Educational Initiatives Pvt. Ltd.
30
Question Graphs
Class 7 / Question 26
Explanation: This question checks if
students are able to simplify a simple
algebraic expression. Students are expected
to know that when the brackets are
opened the sign (- or +) and the digit
outside the brackets get multiplied with
each term inside the brackets. Only 20%
students have answered this question
correctly. Students choosing option A(36%) have probably not multiplied 5 with
the y inside the brackets. Students
choosing option C (20%) have probably
multiplied 5 with the y inside the brackets
but ignored the - sign.
Class 7 / Question 27
Explanation: This question tests the
understanding of place value system in
decimals. Students are expected to realize
that 7 tenths and 8 thousandths make
0.708. Only about 24% of students havebeen able to answer this correctly. Students
may be used to converting conventional
expanded forms like 7/10 + 0/100 + 8/
1000 into decimals. But here when the
conventional order is changed and
hundredths are missing, many students may
have been confused. About 34% have
chosen option C, 0.8007 and 15% have
answered D, 0.87, retaining the order in
which the numbers appear in the question.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
32/56
Educational Initiatives Pvt. Ltd.
31
Question Graphs
Class 7 / Question 33
Explanation: This question checks if
students are able to observe a pattern and
logically think of a general rule being
followed in that pattern. Students are
expected to understand that 2 corners in
each face will not have antennae and hence
a face having n corners will have n-2
antennae. Only 29% students have
answered this question correctly. A lot ofstudents (31%) have chosen the wrong
option C. They probably have thought that
the 50th figure will have 50 antennae but
not considered the fact that the face having
50 corners will not be the 50thfigure.
Class 7 / Question 36
Explanation: This question checks if
students are able to express a part of a
group in terms of ratio. Students are
expected to calculate the number of boys in
the class by subtracting the number of girlsfrom the total number of students and then
give the ratio of number of boys to number
of girls. Only 25% students have answered
this question correctly. A lot of students
(33%) have chosen the wrong answer
option D. They are either just looking at
the numbers given in the question, i.e., 30
and 20, and are choosing their answer as
3:2 or are not careful about what is being
asked in the question.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
33/56
Educational Initiatives Pvt. Ltd.
32
3.1.2 Misconceptions and Common Errors in Language
Question Graphs
Class 3 / Question 13
Explanation: The question tests students
logical analysis of information in a poster.
The poster is about a contest where
students have to send hilarious responses
to the questions asked. The judges will have
to read the responses before announcing
the winner. Students who have understood
this have chosen the correct answer, option
B. Most students (over 50%) have chosen
to skip answering the question probably
because they were looking for direct facts
mentioned in the poster. It is also possible
that due to long length of answer options
students had difficulty reading and
therefore skipped this one. It is likely that
students who chose options A, C, or D
relied on guesswork.
Class 3/ Question 22
Explanation: This question tests students
understanding of words and their meanings
based on the context. In this question, theword tested is snatched () and the
answer is A: took quickly. Some students
have understood this in the context and
have chosen option A. 32% students have
processed the information a little further
and have chosen option C, ate fast, which
is what the goats may do after snatching,
but that is not the act of snatching. These
students may either have not understood
the question or the context, or may have
been short of time.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
34/56
Educational Initiatives Pvt. Ltd.
33
Question Graphs
Class 5 / Question 24
Explanation: This question tests students
understanding of explicit and implicit
information presented in a given text. In
this question, there is a mention of the first
three prizes and then there is a mention of
additional prizes being given away. Students
who have got the correct count after
reading the authentic text have chosen the
correct answer, option D. Students whochose options B or C, (56%), as their
answer have missed out reading the details
in the passage. It is also likely that those
who chose option B have been influenced
by the prior knowledge that there are
generally only three prizes given away.
Class 5 / Question 22
Explanation: This question tests students
ability to identify the main theme in a given
text. In this context, the main activity as
required by the poster is to complete a
dialogue. Students who have been able to
abstract this information have chosen the
correct answer, option D. Many students
(35%) have misunderstood the question
and have chosen option C, use the pen or
buy it. They probably have thought that
the main activity the boy in the comic is
doing, is being asked about. Those choosing
other answers are merely referring to
details in the passage.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
35/56
Educational Initiatives Pvt. Ltd.
34
Question Graphs
Class 5 / Question 20
Explanation: This question aims to test if
student understand implied facts. In the
question, the line is written in a certain
manner to imply that it is not the norm and
is in fact the opposite of what generally
happens. 37% students have incorrectly
chosen option D which is very similar to
the correct answer (A) but is not exactlywhat it means. This shows that they have
either resorted to prior knowledge or have
not understood what the sentence means.
Those choosing other options have tried to
locate facts from the passage to supply as
the answer.
Class 5 / Question 18
Explanation: This question aims to test if
student understand implied facts. In the
question, the meaning of a phrase is being
tested in context of the story. Majority of
students (38%) have wrongly chosen
option B. This could be because of they fact
that they didnt understand the meaning
and chose the answer which seems like a
genuinely good quality generally seen in the
protagonist of a text. Those choosing other
options are basing their responses on either
what they themselved would do, or what
they assumed the protagonist may do in
such a situation.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
36/56
Educational Initiatives Pvt. Ltd.
35
Question Graphs
Class 7 / Question 19
Explanation: In the question, the traits of a
character based on the context in the text
are being tested. Most students (43%) have
failed to understand this and have based
their answer on the text that follows this
context, choosing option A which is an
explicit detail as the answer over an implicit
one. Those choosing options B and C have
either not understood the detail beingtested and have picked a response based on
the overview of the context, or they have
used prior knowledge based on such stories
where the antagonist has negative qualities
and tries to get the protagonist in trouble.
Class 7 / Question 43
Explanation: This question aims to test if
student can recall stated facts. The tested
sentence is very clearly mentioned in the
passage. Only 20% students have been able
to recall this fact correctly and have chosen
the correct answer, option B. Students who
have chosen the wrong answers have done
so either because they were trying to
locate implicit details or because they were
looking at other elements of the situation in
the passage.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
37/56
Educational Initiatives Pvt. Ltd.
36
Question Graphs
Class 7 / Question 28
Explanation: In this question, the
underlying meaning of a given statement is
being tested. Only 21% students have
understood the overall text and have been
able to derive the meaning by choosing the
correct answer, option A. Students
choosing other options are looking for clues
in the preceding or following content.
Class 7 / Question 29
Explanation: This question aims to teststudents understanding of the text and the
references made in certain places. Here,
the focus of the word is on the incident,
option D. Most students (44%) have
chosen option A, referring to the person
with whom the incident happened rather
than the incident itself. They have failed to
understand that the reference is being
made to the case and not the person
involved. Some students have chosen
option B, wrongly assuming that the
reference is being made to the knife lodged
in the patient.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
38/56
Educational Initiatives Pvt. Ltd.
37
Question Graphs
Class 7 / Question 17
Explanation: This question tests students
understanding of the text by being able to
differentiate between facts and opinions.
The given passage is in the form of a story.
The phrase is the writers opinion about
what happened. Students who have chosen
option A (20%) have probably not
understood the phrase and have assumed it
to be a cause and effect question. They
have chosen an answer which tells us thatnothing really happens to the protagonist
despite his errors. 30% students chosen
option C, probably because they have not
understood the phrase tested and have
gone for something which relates to both
Vasco and Malindi.
Class 7 / Question 36
Explanation: This question tests students
ability to draw inferences from a story. In
the given passage, it is Meena who thinks
she is a brave girl as she chooses the brave
thing to do to overcome her fears. 24%
students have chosen option A; this could
either be due to prior knowledge thatparents always think that their children are
brave and are the best. It could also be
because they assumed that she may have
later told her mother that she faced the
situation bravely and her mother must have
called her a brave girl. Students who chose
option B must have done so thinking along
the similar lines as those who chose option
A, only replacing mother with the music
teacher. Students who chose C probably
thought that the teacher thinks is brave as
she overcomes her fears and writes downthe bravest thing she will do. Also, in the
end it is mentioned that the teacher praises
her, though not for the same reason.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
39/56
Educational Initiatives Pvt. Ltd.
38
3.2 PERFORMANCE IN DIFFERENT SKILLS
Analysis was carried out on the performance of the students in the various skills across different classes and
subjects. This analysis enables one to identify patterns in performance across the different skills and understand
which skills are the strongest and which are the weakest. This understanding will pave the way for appropriate
action in the classroom.
3.2.1 Skill-wise Performance in Hindi
Class 3
S.
No.Medium Class Subject Skill
No. of
Questions
Sample
Size
%
Score
1 Hindi 3 LanguageRecognises and writes letters and knows
starting sound of words3 8231 85.3
2 Hindi 3 Language
Reads and writes simple words that are
moderately difficult and have 3-4 letters
and knows names of objects, birds,
animals, etc. seen in daily life
4 8231 81.7
3 Hindi 3 Language
Uses words appropriate to the context
based on their meanings, number and
gender
5 8231 75.6
4 Hindi 3 Language
Reads and writes 3 simple and short
sentences that have less than 5 words in a
sentence
5 8231 52.6
5 Hindi 3 Language
Reads descriptive text, of 3-5 sentences
independently and comprehends beyond
stated facts
12 8231 48.2
6 Hindi 3 Language
Understands written information
presented in various forms as Tables,Notices, Tickets, Posters, Labels, etc seen
in real life
3 8231 34.0
High Performing Skills Low Performing Skills
Class 5
S.
No.Medium Class Subject Skill
No. of
Questions
Sample
Size
%
Score
1 Hindi 5 Language
Reads and writes simple words (that are class
level appropriate) and knows names of
objects, birds, animals, etc seen in daily life
5 9151 81.3
2 Hindi 5 LanguageReads and writes sentences using grammar
concepts8 9151 68.2
3 Hindi 5 LanguageKnows synonyms, antonyms and deduces
word meanings from clues in context4 9151 70.6
4 Hindi 5 LanguageReads descriptive text and comprehends
explicit and implicit details for class level16 9151 53.9
5 Hindi 5 Language
Understands written information presented in
various forms as Tables, Notices, Tickets,
Posters, Labels, etc seen in real life for class
level
5 9151 29.3
High Performing Skills Low Performing Skills
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
40/56
Educational Initiatives Pvt. Ltd.
39
Class 7
S.
No.Medium Class Subject Skill
No. of
QuestionsSample
Size
%
Score
1 Hindi 7 Language
Reads and writes simple words (that are class
level appropriate) and knows names of
objects, birds, animals, etc seen in daily life
4 9516 71.9
2 Hindi 7 LanguageReads and writes sentences using grammar
concepts8 9516 65
3 Hindi 7 LanguageKnows synonyms, antonyms and deducesword meanings from clues in context
3 9516 56.9
4 Hindi 7 LanguageReads descriptive text and comprehends
explicit and implicit details for class level20 9516 38.6
5 Hindi 7 Language
Understands written information presented in
various forms as Tables, Notices, Tickets,
Posters, Labels, etc seen in real life for class
level
9 9516 42.4
High Performing Skills Low Performing Skills
In Hindi language, the skill of Recognizes and writes letters and knows the starting sound of wordsobserved the
highest performance in class 3 while Reads and writes sentences using grammar concepts was the highest
performing in classes 5 and 7
The weakest skills in all classes in Hindi were skills of reading comprehension: Understands written
information presented in various forms as Tables, Notices, Tickets, Posters, Labels, etc. seen in real life and Reads
descriptive text and comprehends explicit and implicit details for class level. Thus understanding both the stories
as well as authentic text material seems to be a challenge.
Students seem to be weak in reading comprehension beyond stated facts. Performance is poorer on
inference based questions as compared to straight forward fact based questions. Examples given below
elucidate the same.
Students should be encouraged to write their own stories, as well as discuss stories that they have read.
Students can be asked to list the different qualities for each character in the story and describe how they feel
the characters will react in different situations and contexts.
HindiClass 7HindiClass 7
Sample Question: Q31 is a straight forward question testing a fact
cited in the passage. Student performance on this question is 65.7%.
Sample Question: Q36 requires students to infer after reading the
entire story, keeping in mind the facts stated. Student performance on
this question is 17.1%
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
41/56
Educational Initiatives Pvt. Ltd.
40
3.2.2 Skill-wise Performance in Maths
Class 3
S.
No.Medium Class Subject Skill
No. of
QuestionsSample
Size
%
Score
1 Hindi 3 Maths Pre-maths skills 3 8231 78.9
2 Hindi 3 Maths Number sense and basic number competency 5 8231 57.5
3 Hindi 3 MathsArithmetic operations: addition and
subtraction4 8231 55.7
4 Hindi 3 Maths Arithmetic operations: multiplication 3 8231 67.3
5 Hindi 3 Maths Basic shapes and geometry 4 8231 59.6
6 Hindi 3 MathsApplication in daily life: money, time,
calendar, length etc.4 8231 52.2
7 Hindi 3 Maths Problem Solving 3 8231 51.8
High Performing Skills Low Performing Skills
Class 5
S.
No.Medium Class Subject Skill
No. of
Questions
Sample
Size
%
Score
1 Hindi 5 Maths Number sense and basic number competency 4 9151 54.0
2 Hindi 5 Maths Four arithmetic operations 5 9151 56.3
3 Hindi 5 Maths Fractions: concepts and applications 4 9151 40.5
4 Hindi 5 MathsApplication in daily life: money, time,
calendar, length etc.5 9151 41.1
5 Hindi 5 Maths Basic shapes and geometry 6 9151 51.5
6 Hindi 5 Maths Data interpretation and analysis 4 9151 54.2
7 Hindi 5 Maths Problem Solving 4 9151 38.3
High Performing Skills Low Performing Skills
Class 7
S.
No.Medium Class Subject Skill
No. of
Questions
Sample
Size
%
Score
1 Hindi 7 Maths Number sense and basic number competency 3 9516 39.1
2 Hindi 7 Maths Factors and Multiples 3 9516 37.8
3 Hindi 7 MathsApplication in daily life and word/visual
problems3 9516 33.7
4 Hindi 7 Maths Area and Perimeter 4 9516 37.3
5 Hindi 7 Maths Geometry: concepts and application 4 9516 45.1
6 Hindi 7 Maths Algebra: concepts and application 3 9516 35.8
7 Hindi 7 MathsFractions, Decimals and Ratios: concepts and
application4 9516 42.2
8 Hindi 7 Problem Solving 4 9516 27.8
9 Hindi 7 Data interpretation and analysis 4 9516 46.5
10 Hindi 7 Integers: concepts and application 4 9516 33.7
High Performing Skills Low Performing Skills
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
42/56
Educational Initiatives Pvt. Ltd.
41
In Maths, the strongest skills were Pre-Maths skillsand Multiplication in class 3,four basic arithmetic operationsin
class 5 andData Interpretation and Analysis and Geometry in class 7.
The weakest skills wereApplications in daily lifeand Problem Solvingacross the three classes.
Transition from basic skills in primary classes to intermediate skills like Integers, Algebra in higher classes seems
to be a challenge
o
Students of class 7 seem to be struggling with Area and Perimeter problems a concept that is
introduced in Class 4
o Integers, a concept introduced in Class 5, also seems particularly weak as many students are
unable to answer straight forward questions
o The transition from concrete to abstract concepts in maths is a challenge and can be systematically
addressed through targeted curriculum efforts
Within a skill, different kinds of questions are covered - some could be straightforward or procedural, some
could be conceptual and some could require higher order thinking. In skills found weak, students are not ableto answer the conceptual and higher order thinking questions. This could be because of the stress laid on the
procedural ways of answering or solving questions. If instead, stress is laid on conceptual understanding,
students are more likely to answer different kinds of questions correctly.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
43/56
Educational Initiatives Pvt. Ltd.
42
EI conducted field audits as a part of this assessment. Over 30 schools were visited during test conduction by 10
Field Audit Resource Persons. The purpose of the audit was to ensure that the execution of the test met certain
standards of quality, maintained a certain level of standardization and to assess whether any issues of bias or
subjectivity can be traced at any of the schools. To ensure that the observations are comprehensive, the auditors
not only observed the Test Administration, but also interacted with the Teachers, Head Teachers and Students to
take feedback and opinions about the test and the testing process.
Each auditor spent an average of 40 minutes in each school and recorded his/her observations on a detailed
checklist. The key observations which the auditors were asked to make in their visits included the adherence to
the processes given in the evaluator manual, whether the teacher had participated in the evaluator training,
infrastructural observations such as issues of disturbance and cluttered classrooms, the responsiveness of the
teachers to the tests and testing processes, issues of cheating and bias through observations on seatingarrangements and teacher involvement, and perception of the teachers and students regarding the test and the
testing process. The responses were then analysed to deduce some key test conduction practices and
observations. Some key observations from the field audits are discussed below:
1. All the schools audited (except one) were ready with the test material and started the test at the scheduled
time. Test duration and timings were followed in all the audited schools. Seating arrangement was proper and
students were sitting comfortably. In most of the audited schools, students from an individual class sat
separately from other classes. No instance of mass assistance observed in the audited schools.
2. Over 80% of the evaluators present during test conduction did not get any formal training from the master
trainers who were trained by EI for this purpose. Though most of them were aware of the broad procedures;
in some schools this caused a little confusion and needed intervention by the Resource Persons in the
classroom from time to time.
3. The processes from the evaluator manual were not consistently followed in all the schools. This can also be
associated with the unfamiliarity of the evaluators with the processes indicated in the Manual.
4. Apart from the evaluator, there were other people present in the classrooms in around 40% of the observed
schools. The presence of external persons during test administration can have certain repercussions of bias
and unnatural environments in the classroom.
5. Over 80% of the school evaluators were aware of the school codes for their schools and helped the students
write the code on their sheets.
6. Over 80% of the classes observed conducted the tests with no serious disturbances.
7. Almost all the schools observed practiced seating arrangements that ensured that the room was not
overcrowded and that the students were seated in rows or a circle; in a way that ensured that the
arrangement minimised the issues of cheating and were deemed appropriate for a testing environment.
8. Most of the evaluators had enough test material for all students and for themselves to conduct the test
according to the procedures. However, in four of the observed schools, testing material was not sufficient for
all the students to test independently. Issues of shortage included the unavailability of the paper for the teacher
4. FIELD AUDITS
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
44/56
Educational Initiatives Pvt. Ltd.
43
to read out the questions for class 3, exclusion of some students due to unavailability of papers as the
enrolment data shared was for previous year.
9. In a few schools, (about 26%) there were a few instances noticed where the teachers were assisting the
students in answering the questions, usually by giving clues.
10. In one school, the test was not conducted during the Resource Persons visit. Since the conduction of the test
was supposed to be at the same time across the schools, this school was probably not informed about the
testing.
11.With respect to the time durations of the tests, most schools followed the time durations prescribed for the
test. Issues of time included delayed commencement of the test and confusion and unfamiliarity about the test
practices such as School Codes and OMRs.
12.Some of the schools reported multi-grade classrooms. While some of the classrooms were effectively
partitioned to ensure minimum disturbance, others were more chaotic and not suitable for a testing
environment. However, no significant patterns were observed with respect to the classroom structure.
13.
Most students shared that they found the test interesting and mentioned a few questions / passages form thetest which were fun to do / read.
14.Some of the students and teachers reported that the testing format was unfamiliar and difficult, while others
commented on the difficulty levels of the Hindi papers and Reading Comprehension sections. This observation
however, varied across the interactions with the students and teachers.
The observations from the field audit visits have been summarized in the table given below.
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
45/56
Educational Initiatives Pvt. Ltd.
44
School Code
Did the teacher
attend the
evaluator
training?
Were
processes
from the
Evaluator
Manual
followed?
Were
there
other
people
present
during
testing
?
Did the
teacher
know
the
school
code?
Did she
help
student
s write
it?
Disturbance
in class?
Was the
Class
overcrowded?
Was the
seating
arrangement
APPROPRIAT
E for testing?
Did the
teacher
have
sufficient
test
material?
Was
time
schedule
and
duration
followed
?
Comments
2110907702 N Y Y Y Y Y N Y NBig hall where classes were being conducted simultaneously,disturbance, friendly teachers, teachers said that the paper was
tough
2110907701 N N Y Y N N Y Y Y Students and teachers both found the tests very difficult
21109079011 N N Y Y N N Y N N Teacher filled OMR sheet himself
2020602602 N N Y Y Y N Y Y Y Teachers on rotation, thought language paper tested facts, givenno formal training, shortage of papers
2020602601 N N N Y N N Y Y Y No formal training for teachers, shortage of papers
2020601402 N N N Y N N Y Y YNo formal training for teachers, shortage of papers, did not find
papers tough
2110904801 N Y N N N N Y Y Y
2110904802 Y Y Y NA N N Y Y Y
2110904501 Y Y N Y N N Y Y Y Teachers and students liked tests
2110904501 N Y N Y N N Y Y Y Teachers and students liked tests
2040304307 N Y Y Y N N Y Y Y
2040301802 N N Y Y N N Y Y YUninformed teachers, paper found difficult, no break b/w tests for
class 3
2040304301 N N Y Y N N Y Y Y Difficult format of test, shortage/excess of papers
2050304201 N N N Y N N Y Y N Papers reached late, no teacher had read the manual
2050307601 N N N Y N N N Y NIssues of time of test-easy but lengthy, papers reached late, noteacher had read the manual
2050404701 Not tested during field audit
2040301102 N N N Y N N Y Y YHindi paper was observed to be tough, there was no extra copyfor teacher to read question for class 3
2040301701 N N N Y N N Y Y YHindi paper was observed to be tough, there was no extra copyfor teacher to read question for class 3
2040301905 N Y N Y N N Y Y Y Hindi paper was tough, test conducted in school ground
2100501201 N Y N Y N N Y Y NNo evaluator guide made available, Reading Comprehension wasfound to be tough
2100501202 Y Y N N N N Y Y N No evaluator guide, positive feedback
2100501401 N Y N Y Y N Y Y Y No evaluator guide, positive feedback
2100501302 N Y N Y N N Y N Y No evaluator guide, positive feedback
2100500902 N Y Y N N N N Y Y Trained teacher was not present, papers up to their level
2100501101 N N Y N N N Y N N Trained teacher was not present, issues in dealing with OMR
2100501001 N N N N Y N Y Y NInstructions not clearly followed, teacher was helping students toomuch
2050103601 N Y N Y N N Y Y Y
2050103801 N N N Y N N Y Y Y Teacher kept leaving class, students scared of test
2050203603 N Y N Y Y N Y Y Y
-
7/24/2019 Himachal Pradesh State Assessment - Main Report
46/56
Educational Initiatives Pvt. Ltd.
45
School Code
Did the teacher
attend the
evaluator
training?
Were
processes
from the
Evaluator
Manual
followed?
Were
there
other
people
present
during
testing
?
Did the
teacher
kno
top related