education gilbert valverde standards, evaluation and accountability
Post on 19-Oct-2014
969 views
DESCRIPTION
TRANSCRIPT
Standards, Evaluation and AccountabilityState-of-Play and Challenges from Latin America
Gilbert A. Valverde, Ph.D.Comparative and International Education Policy ProgramUniversity at Albany – State University of New [email protected]
Working Group on Standards and AssessmentProgram to Promote Educational Reform in Latin America
A World-Wide Shift of Policy Focus• We are experiencing a global shift in education
policy priorities▫ Not only is it important for children to have access
to education.▫ The quality of what transpires within the
classroom also matters.• Prioritizing quality, means focusing on the
content of schooling▫ Global interest in standards, evaluation and
accountability is a signal of this policy shift.
Latin America• Much discussion of the importance of the content of
schooling• Important and persistent problems of quality▫ National, regional, and international testing programs
have thoroughly documented the poverty of educational outcomes in the region.
• The evidence has conclusively demonstrated that the traditional repertoire of policy instruments is not sufficient to address this problem.
• A growing consensus that standards, evaluation and associated policy instruments merit serious consideration.
Goals for this talk
• Discuss how initiatives in standards, assessment and accountability are forwarding important policy conversations on educational quality in Latin America.
• Consider challenges and lessons from the Latin American experience that may be of interest to other parts of the world.
Basic premises
• Educational reform based on principals of promotion of educational quality requires:▫ A clear vision of what is to be accomplished▫ A system to monitor how these goals are being
met.▫ A commitment to act in accordance with the
results of monitoring efforts
Elements of Quality of Vision
• Research and policy experience indicate that accomplishing high levels of quality in educational outcomes requires the articulation of a clear vision of pedagogical objectives that operationalize goals that are▫ rigorous,▫ challenging,▫ have well defined priorities and foci▫ evidence-based
Challenges for Latin America
• A critical examination of the instruments of curriculum policy in the region suggests an urgent need to correct problems of▫ Ambiguity▫ Dispersion and lack of priorities▫ Lack of focus▫ Absence of academic rigor
An example of dispersion and lack of focus
0 5 10 15 20 25 30 35 40
18
17
16
15
14
13
12
11
10
9
8
7
6
5Japan
0 5 10 15 20 25 30 35 40
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Mexicoviejo
viejo / foco
nuevo
nuevo / foco
11
1982
1993
1996
1996
1996
1996
1998
1991
1993
2002
19862005
1996
1991
1990
1996
1996
1998
1997
Source: Sistemas de evaluación de aprendizajes en América Latina. Balances y desafíos. PREAL, 2006.
First National Assessments
RAPID GROWTH•In 1980 – none•In 1989 – 2 (Chile, Costa Rica)•Today – universal
Consequences of rapid growth• Introduction was precipitous, without much preparatory work
with schools or other actors.▫ Many claim that introduction was forced or imposed.
• In most cases the initial measurement model was not at all aligned with the curriculum reforms that often were taking place simultaneously.
• Results were quickly and extensively disseminated, most often accompanied by condemnations of the quality of teaching (which is never measured).
• Important questioning of the technical quality of testing efforts▫ Much suspicion that testing systems are not committed to
technical quality.• Tepid government commitment to testing.▫ Often testing systems do not have budgets until a few months prior
to test administration.▫ Low levels of investment, with unclear commitment to technical
quality.▫ Reticence to act in accordance with test results
Continuing progress• Growing participation in large-scale cross-national
assessments.• Increasing investments in state-of-the-art technologies• Awareness that disappointing outcomes measured by
tests imply need to explore new policy instruments• Setting of more specific improvement targets• Emerging “culture of evaluation”• Capacity building, growing experience and exchange• Openness to revision and improvement of approaches,
methods, analyses and reporting• Teachers growingly interested in potential of properly
utilized assessments for professional development and school improvement purposes
• More interest in returning results to schools and non-traditional educational stakeholders
Nascent efforts in Accountability• If the accountable unit is the school, requires a
census of schools, if the student, a census of students
• No high-stakes testing in the Region• “Low-Stakes” Tests in Costa Rica, El Salvador and
the Dominican Republic▫ In the Dominican Republic, for example, the school-
leaving examination or “Prueba Nacional” contributes only 30% of a student’s grade
▫ Students have 3 or 4 opportunities to pass the exam▫ Few students fail to pass based on the test alone
Schools = primary units of accountability• Chile▫ Yearly publication of league tables ranking all
schools• El Salvador▫ Publication of lists of highest performing schools
• Colombia (Bogotá)▫ “Excellence” awards based on school test scores.
School Choice• Extremely uncommon• Chile is the notable exception
Ministry distributes school-by-school test results for each region, including average scores in each subject area, changes in scores since previous testing round,
and comparisons with schools serving similar
socioeconomic groups Special reports for parents since 2003 Recent research has demonstrated conclusively
that despite the intentions, parents do not make significant use of test results in choosing schools.
New efforts to promote accountability
• Teacher as the unit of accountability▫ Chile: Awards for teachers in the best schools according to the
national test A new voluntary teacher test (including video, teacher
portfolio and content assessment) to compete for “pedagogical excellence” designation
Efforts to put together a required teacher certification test for graduating teachers
▫ Mexico: Teacher bonuses based on complex formula that includes
student test results
However….
Still too early to say whether, in most Latin American countries, institutionalized and permanent mechanisms have been firmly established which enable continuous setting and revising of learning goals and measuring their achievement.
In only few countries, student testing and other kinds of outcomes assessment and reporting are clearly framed or aligned within a set of policies and norms about the kind of utilization which will be given to the resulting information, as part of a well integrated, clear and balanced set of policies for the improvement of learning and for adequate accountability.
New Policy Routes• The most important challenge is to increase mean levels
of achievement in all schools.• Increasing quality in outcomes requires▫ Operationalization of a concrete vision of what can be
considered quality outcomes▫ Assignment of responsibilities▫ Monitoring▫ Commitment to act according to monitoring results.
• A new policy instrument increasingly of interest: standards▫ There are educational standards or projects underway in:
Guatemala, Honduras, Panama, Colombia, Argentina and Chile
▫ Following diverse models from the United States and Australia
Main challenges
• Clarification and social legitimation of assessment purposes and processes
• Clarification of learning goals• Selection of what will be measured and how, according to
purposes • Selection of appropriate institutional framework • Improvement of testing instruments and processes• More and better data analyses • More and better diversified reports• More and better dissemination of results • Appropriate use of results for decision making
Key question looking to the future• Is it possible or desirable to compensate for the early
shortcomings in the introduction of assessment systems by greater efforts in transparency and genuine consultation?
• Can the region learn from failures to lay sufficient policy groundwork in the introduction of assessment systems, in order to avoid pushback against nascent efforts to introduce standards that could lead to similar questioning of their political legitimacy, pedagogical value, and technical soundness?
• Can research and development efforts be made more transparent so that a broad spectrum of actors will be persuaded that governments are sufficiently concerned with the technical quality of standards and the validity and reliability of measures used in their assessment systems?
• Will it be possible to introduce or maintain a concern for benchmarking Latin American standards against “world-class standards” when the likelihood is that Latin American countries will continue to be a the bottom of PISA, TIMSS and other global league tables?
• Can the development of assessment systems in the region be used to introduce a concern for evidenced-based policy making in the region, to temper enthusiasm for theoretical innovations with weak evidentiary bases in the development of curriculum policy?
• Can assessment and standards be brought into alignment with a commitment to act in accordance with evaluation findings in order to develop accountability systems with greater potential for stimulating educational quality?