1.the problem of policy implementation: the case of performance appraisal
DESCRIPTION
1.The Problem of Policy Implementation: the case of performance appraisal. 2.Teacher Learning Through Portfolios. Group 8 Members. Lam Ka Po, Angela Lam Wing Chung, Terence Lok Wai Yi, Begonia Ng Siu Kai. The Problem of Policy Implementation. The case of performance appraisal. - PowerPoint PPT PresentationTRANSCRIPT
1. The Problem of Policy Implementation: the case of performance appraisal
2. Teacher Learning Through Portfolios
Group 8 Members
Lam Ka Po, Angela
Lam Wing Chung, Terence
Lok Wai Yi, Begonia
Ng Siu Kai
The Problem of Policy Implementation
The case of performance appraisal
Aim of the research
• To identify the obstacles which lead to slow
and sporadic policy implementation
• To investigate new perspectives of
implementing policy as solving a problem
• To identify how policy implementation
could be improved
Obstacles of current Appraisal system
• Interest group approach
• Initiatives come from governments and
their associated ministries and departments
• Affected by the political origins of these
initiatives
• Conflict between the government and the
teachers associations
Policy as Solving a Problem
• Offer a conceptualization of performance
appraisal as a complex problem
• Specify the requirements for a quality
solution at both national and local levels
• Prevent satisfying the requirements of
particular interest groups rather than
teachers themselves
Definition of Problem
All the conditions or constraints on the solutio
n (variously weighted) plus the demand that
the solution (an object satisfying the constra
ints) be found
- by Thomas Nickles
Constrains on the solution
• Values– the policy should recognize the professionalism
of teachers
• Practical requirements– it must not cost more than schools can afford
• Goals– it must improve the quality of teaching and
learning
• or any combinations
Integration of constraints
• Particular constraints is likely to be in conflict in complex problems
• Need to integrates the constraints taken as a set by reaching a compromise between different interest groups
• Require shifts in the definition of particular constraints to obtain an adequate solution
Phases in Policy Making
• Intended Policy– What various groups wants
• Actual Policy– The document, legislation and/or report
• Policy-in-use– Regional and school-level reaction
- by Bowe et al.
Scenario Highlights
Principal of Riverview High School
John Burton’s dilemma
Appraisal system• Aim :
to improve teaching and learning
• Safeguard :negative evaluations of individual staff performance would not be reported to the senior management team.
Education Review Officereported:• appraisal system failed
to give the management information on the performance of staff.
Good points of the old system
• Staff were comfortable
• Staff were honest with their chosen appraiser
• Allow staff to identify professional development
goals that would impact positively on children’s
learning
• Staff has the authority to report and to report
what
John Burton’s Dilemma
How to make the appraisal system more structured and accountable
while simultaneously supporting my staff to challenge themselves professionally and to be open about identifying their professional development needs.
A Revised Appraisal System• Guidelines:
– Staff could choose their own appraiser– limited to themes determined by the SMT– observe the work of another staff member in an
area in which they wanted to improve
• Outcomes:– 40% of staff returned the questionnaire on this
revised system– Some staff were critical and some made very
positive comments.– 60% of staff ignored the appraisal guidelines
• Do I have the authority to check up
on those who have not responded to
the questionnaire?• What I can do about those who
had not participated?
Staff’s desire
• non-threatening scheme that emphasized professional development
Management’s concern
• enhance teaching and learning
• introduce a more accountable system that mettheir reportingrequirements
Theory-in-useConstraints
Practices Consequences
Enhancesteaching &learning
Equality of authority
Non-judgmental&non-threatening
SMT suggests what should be appraised& requires classroom visits
Entire processremains confidential
Self-appraisal emphasized
Peer supportencouraged &individuals select own appraiser
Some staff positive,others critical
Participation rates & impact of appraisal unknown
Principal unable to fulfil reporting requirements
The Intervention Process
By
the School of Education, University of Auckland
Invited by
the School Management Team of
Riverview High School
Aims of the Intervention
• to assist Riverview High School to develop an effective performance appraisal system
Methodology : Problem Based
• Examined the way Riverview solved the problem by discovering
– the constraints they set on the solution
– the strategies they used to integrate those
constraints.
Data Collection Methods
• 3 interviews with the senior management staff
• 6 interviews with other staff
• 2 feedback meetings with the senior
management team
• Observations and audio transcripts of both staff
and departmental meetings.
• Review relevant documents, e.g. appraisal
policy
How to implement
• The assistance comprised of 8 workshops
– focusing on the adequacy of current practice
– formulation of a new set of constraints
– developing appraisal practices consistent with
those constraints
– interpersonal skills training
• enabled the SMT to debate the constraints that
led to an integration, rather than opposition
The First Phase
• To identify the values, beliefs and practical
issues that underlay current practice in
order to determine what needed to be
revised and what could be further
developed.
The second Phase
• Formulate a new set of constraints which enabled
the SMT to make explicit their requirements.
• Five nominated constraints were
– made a difference to teaching and learning
– linked to professional development
– respected and supported staff
– held all levels of staff accountable for performance
– practical and efficient
The Third Phase
• Meetings of staff– answer questions
– staff learnt that the problems are difficult
because they had multiple requirements with
considerable tensions between them.– Continued to debate and resolve questions
The Last Phase
• To en-skill the senior management team to articulate the tensions between their proposed constraints and debate their resolution with their staff.
Two full staff meetings
• Both meetings devoted to appraisal
• Showed a shift in the staff’s thinking and the quality of the debate
11 weeks later
First meeting
• The Principal made no mention of the list of constraints as some separate points but an interacting whole
• Staff discussed selected constraints that were of concern to them
• The trained SMT explained that making a difference in the classroom was the key factor but the allocation of staff development was a link only
Second Meeting (11 weeks later)
• Staff began to acknowledge– the tensions between the constraints and to
search for ways to integrate them– they were concerned with the relationship
between appraisal and other school practices– how aspects of the proposal met two or more of
the proposed constraints
• Both staff and SMT deepened their understanding of the integrated nature of the appraised problem
Evaluation of Riverview practice
• Enabled the senior management team to :
– Determine what was desirable
– Determine what was problematical
– Identify the values and beliefs that sustained it
• Enable new policy to be formulated in
addressing current constraints
• Determine what should be retained and
revised
Implications for National Policy
Formulation of a new constraint set at
national level would encourage policy
makers to identify:
• The purpose of appraisal
• The values guiding its introduction
• The practical problems
• What practices are ruled in or ruled out
Skills required for improvement
• Demands new interpersonal skills
• Satisfy a complex constraint set under considerable tensions
• Requires skills to confront and debate the merit of competing perspectives
• Requires co-operations
• Requires the commitment and skills to develop a shared understanding
How policy implementation could be improved ?
• Integrating the constraint set as a whole• Testing proposed constraints by developing
exemplars of practice• Revising either the constraints, the practice
or both• Consulting with practitioners and interest
groups• Formulating new approach – Portfolio ?
Teacher Learning Through Portfolios
Background behind Using Portfolio for Professional Development
• Charles Handy (1996)
– Incremental change becomes discontinuous cha
nge in the Information Age
– Under Incremental change: learning by followi
ng preceder’s footsteps
– Under discontinuous change: learning becomes
the voyage of exploration, of questing and expe
rimenting
Teaching Portfolio
• Can be seen as a voyage of exploration, of questin
g and experimenting
• Definition by Wolf & Dietz (1998):
“A teaching portfolio is a structured collection of tea
cher and student work created across diverse conte
xts of time, framed by reflection and enriched thro
ugh collaboration, that has its ultimate aim the adv
ancement of teacher and student learning.”
Why portfolios are a good idea for PD (I)
• The learning portfolio documents
– not only the teaching experience
– but also, the reflection and enquiry which
produces professional learning
Why portfolios are a good idea for PD (II)
• Groundwater-Smith (1999): “… is underpinned by the idea of the scholarship of practice”
• Gibbons et al (1994): practitioners are “active agents in the definition and solution of problems as well as the evaluation of performance.”
• Grant & Huebner (1998): “… to have significant impact on the development of teacher knowledge and professional practice.”
• Frid & Reid (1999): “…(being) the formation of professional subject – a ‘teaching self’”
Purposes of Portfolios
• Learning portfolio: promote teacher reflection and
ownership over the learning process.
• Assessment portfolio: presents administrators with
information about a teacher’s effectiveness.
• Employment portfolio: provides prospective
employers with information about a teacher’s
suitability for a position.
Process of Portfolios
• It is an extended period of time rather than a specific event.
• There are different processes• To be successful, we need:
a good deal of structured support for the individuals concerned, such as:
– Teacher release time,
– Resources for participating in professional development activities,
– Timetable flexibility for participating meetings,
– Teachers’ willingness to collaborate on the purpose and the process of the portfolio
Content of Portfolios
• Different purposes and processes will require different kind of content
• More than a collection of artifacts for teaching or professional activities:– Retallick & Groundwater-Smith (1996): “…(to be) a compilati
on of evidence which demonstrates the acquisition, development and exercise of knowledge and skills in relation to your work practice.
– Wolf (1996): “… (to be) a set of accomplishments attained over an extended period
– Dietz (1993): “… to be dynamic and changing as the learner experiences discoveries that lead to new directions and activities”
Assessment of Portfolios
• Depending on the purposes,
• For self-reflective professional learning
– Informal assessment (such as self assessment,
Peer evaluation process)
• For other purposes (such as for promotion,
employment):
– Formal assessment (also known as high-stakes
assessment): a higher level of monitoring
Two problem areas for assessment
• Comparability
• Subjectivity
Two problem areas for assessment (I)
• Comparability:
– Difficulty arises from the wide range of
approaches adopted by different persons as
each portfolio is on individual basis.
– Suggestion: stipulating certain core items to be
included
– Problem again: what are these items? Who
makes such decision?
Two problem areas for assessment (II)
• Subjectivity:
– As the evaluation is qualitative rather
quantitative, the question becomes how to make
the evaluation reliable and valid.
– Suggestions:
• A panel of assessors to integrate different views
and to avoid bias,
• A common understanding of the set of
assessment criteria used.
University accreditation: Formal assessment
• More universities are changing their policie
s on credit to allow students to submit a port
folio of their on-job learning for credit purp
oses
• This indicates an acceptance of the portfolio
approach
Summary of advantages by portfolios
• (by Edgerton, Hutchings & Quinlan, 1992)• capture the intellectual substance and ‘situatedness’
of teaching• encourage teachers to take important new roles in the
documentation, observation and review of teaching• …. to use these new roles as powerful tools for impr
ovement; and,• … are forging a new culture of professionalism abou
t teaching
Areas of tensions
• Multi-purpose nature: for self-reflection, for
assessment (comparability and subjectivity)
etc.
• Resources allocated: time allocated to the pr
ocess, teacher’s timetable flexibility etc
• Teacher’s willingness: may largely hinges o
n the above areas.
Local context
• The Hong Kong Institute of Education
conducted the study “Levels of Information
Technology (IT) competency, core course
elements and assessment tools for teacher
training in IT in Education” in 1999
One of the research questions in the study
What are the appropriate methods/tools to
assess the IT competency of teachers?
Recommendation
• It is recommended the portfolio be adopted
as a major form of assessment for teachers’
IT competency in education.
What are the issues raised in relation to this kind of school-based assessment?
Issues were raised in relation to school-based assessment
• Is there sufficient accountability in
school-based assessment?
• Are the principals in good positions to
assess the teachers in terms of using IT in
education?
Is there sufficient accountability in school-based assessment?
• The study suggested that principals are
ultimately accountable for whatever
happens in their schools. The attainment of
their colleagues in using IT as a teaching
tool should not be any different.
Are the principals in good positions to assess the teachers in terms of
using IT in education?
• The study suggested that– principals should be the first ones to attain good
proficiency in applying IT in learning and teaching.
– principals could delegate this job to someone else – a senior staff in the school or an external consultant
Content of portfolio - BIT
• a scheme of work consisting of a slide
show, Web materials to be downloaded
from the Internet as reference link, and the
use of at least one readily available
educational courseware
• rationale for using IT in the lessons: apply
what and how to apply
Content of portfolio – BIT (cont’d)
• a reflective statement of how they have
used IT in their duties as teachers
• self-assessment form completed by the
teachers
Assessment
Portfolio checklist
• to be used by the teachers for self-assessme
nt purposes - before and after the course
• for the school principals to assess the IT co
mpetency in education of their teachers
Other Forms of Assessment ?
• “Assessment by portfolio prepared by individual teachers is only one of possible forms of assessment. Instead of using assessment by portfolio, schools are free to use other forms such as lesson observations with assessment reports completed by school head, or group portfolio compiled by teachers each demonstrating his/her application of the required skills and knowledge in the teaching and learning processes. ”(extracted from the letter of 29.3.2000 to schools)
IT teacher training reference materials (ITERC)
http://www.ited.ed.gov.hk/English/ITinschool/Teacher_training/Teacher_Training.htm
Reflection/ Discussion
• What’s the situation in your school?
– Mode of IT training/ assessment
• Accountability?
• Principals to assess the teachers in terms of using IT
in education?
– Appraisal System