WP4: Outcomes from Trying out Exemplars

Avi Hofstein, Rachel Mamlok-Naaman ,

Ron blonder & Mira Kipnis

The Weizmann Institute of Science

Rehovot 76100, Israel

PARSEL Meeting, Lund, august 2007

Main Goal

“WP4 follows on from WP3 and relates to outcomes from the use of the materials and/or resources

nationally ”.

“Main goal: Determination of outcomes from trying them and finding out whether the materials and/or resources are usable as intended, and can function in the desired approach based on the model. The determination of outcomes will be undertaken by a variety of methods and guided

by the national network partner .…

Main Goal

The national network partner ensures wider dissemination within the country and to the project dissemination team through the website, local seminars, journal articles and videoconferencing. This work package will

be managed by the Weizmann Institute”.

(PARSEL document, 2006)

What are we aiming at?

• Modules (popular, relevant, student / teacher

centred, context-based…)

• Teachers (adaptation and assimilation)

• Students (change of attitudes and beliefs

towards science…)

The PARSEL Evaluation InstrumentsIntroduction and Rationale

This evaluation draft relates to written responses and to

optional interviews for teachers and for students, and its

objectives are to assess the following: •Was the module perceived by the students as relevant to their

lives?•Was the module interesting and useful?•Did the module enhance the students' scientific literacy?•Was the module student centered?•Did the module develop cognitive skills?•How was the module used (formative aspects)?

It has already been proposed and accepted that

the evaluation will come from the teacher and

the students involved in the trying out of

modules. Thus two sets of instruments are

needed. But at the same time, it is important to

recognize that the instruments should not be

extensive, taking away valuable teaching time or

involving the teachers and/or students in extensive out-

of-school tasks.

A further consideration is that the instruments are for the purpose of finding out about the usefulness of the modules at a pan European level. The evaluation is not a research study, although any partner can build on the work of the consortium and, for their purposes, go beyond the project and undertake additional evaluation.

It is proposed that it is reasonable for each

teacher involved in the trying out (hereafter

called a PARSEL teacher) to be asked to

complete 2 instruments only. Furthermore, it is

proposed that each student involved in the trying

out - that is taught by a PARSEL teacher – also

be involved in completing 2 instruments.

4 instruments

For the PARSEL teacher: • A teacher ownership questionnaire )we have another

alternative – suggested by Wolfgang(• An interview at the end of the trying out

For the students:• A pre-questionnaire describing prior teaching / learning

science )we have another alternative – suggested by Claus(

• A post – questionnaire after each module undertaken )we have other 2 alternatives(

• An optional interview at the end of the trying out (to triangulate the quantitative aspects)

PARSEL Evaluation Instruments:Three sections

SECTION 1

Feedback of how the teacher used the module (to be completed by the teacher during or immediately after teaching of the module)

SECTION 2

Pre and Post Semantic differential (SD) questionnaires for students

SECTION 3

Examples of interview questions (items) for teachers and students

SECTION 1. Feedback on how the teachers used the module

The objectives of this section is to obtain feedback from the teacher on how they took ownership of the PARSEL module and in so doing how they modified the various components of the module (if at all), and how they covered each stage. The teachers are asked to complete components as the teaching proceeds and thus the work load is distributed

over the whole teaching time for the module .

SECTION 2A. Student Pre-questionnaire: Students' opinions related to the studying of

science

This draft relates to student responses to a semantic differential questionnaire. Based on literature, a semantic differential questionnaire is a simple and precise instrument to investigate attitudes, beliefs and feelings toward certain issues.

Osgood, C.E., Suci, G.J., & Teenenbaum, P.H. )1957(. The measurement of meaning. Urbana, IL: University of Illinois Press

Main goal of the semantic differential questionnaire

How to strive for scientific literacy whilst making the teaching of science both popular and relevant to students?

• Interest/enjoyment/fun for students• Important/meaningful/useful as perceived by students • Cognitive /NOS /personal /social challenges in student

learning, relevant from a society/career/citizen perspective (currently and in the future)

• Student involvement/varied/simulating approaches (e.g. scenario), enquiry/experimental/student centered / providing feedback to the teaching approach.

Objectives of the semantic differential questionnaire

• Was the module relevant to students' lives? (1, 13, 14, 17)

• Was the module interesting? (3, 5, 16, 18)• Did the module enhance the students'

scientific literacy? (10, 4, 12, 19)• Was the module student centered?

(7, 9, 15, 20)• Did the module develop cognitive skills? (2,

6, 8, 11)

Objectives of the semantic differential questionnaire

To assess:

For items 1-20:

• Did students perceive the teaching in science lessons as being relevant to their lives?

• Was the teaching of science interesting and useful?

• Did the prior teaching enhance the students’ scientific literacy?

• Was the teaching student centered?

• Was the teaching perceived as developing cognitive and affective (interest, motivation) skills?

For items 21-24:• Did the teaching / learning of science provide

opportunities for group work?• Did the teaching encourage inquiry learning?• Include assessment for learning (formative

assessment)?• Include provision for giving a meaning to

science (appreciating the nature of science - NOS)?

SECTION 2B. Student Post-questionnaire, to be administered to the students after the completion of

each module - Objectives

To assess the following: For items 1-20• Did students perceive the teaching in science

lessons as being relevant to their lives? • Was the teaching of the module interesting and

useful?• Did the teaching enhance the students’

scientific literacy?• Was the teaching student centered?• Was the teaching perceived as developing

cognitive skills?

For items 21-24:

• Did the teaching provide opportunities for group work?

• Did the teaching encourage inquiry learning?• Did the teaching include assessment for

learning )formative assessment(?• Did the teaching include provision for giving a

meaning to science )appreciating the nature of science - NOS)?

SECTION 3A. Semi-structured interview questions for teachers who are interviewed by the

PARSEL personnel

This is taken as being interviews with the teachers after teaching at least one module and preferably after teaching a number of modules. This set of interview questions are really only indirectly related to popularity and relevance but do put stress on seeking teacher reactions to the use of the modules. It will be interesting to see how they react to modules which have not been designed in their country and hence relate to another education system.

Purpose of the questions

Question 1: Why did you choose to become a

PARSEL teacher?

To determine whether the teacher has prior

appreciation of the intention of PARSEL

and/or the style of the modules.

Question 2: What were your feelings before, during

and after teaching this module?

A major attempt to get at the teacher reactions to the modules. Unfortunately external pressures (modules not related to the examination questions) can be expected to interfere with a ‘real’ reaction, but nevertheless it is expected to give a good indicator of the ‘worth’ of the module or modules in the eyes of the teacher

Question 3: What was your impression regarding

the students' different reactions to this module?

Allows the teacher to comment on their perception of students’ appreciation of the modules. The inclusion of the term ‘different’ is to push the teacher to recognize that a simple global response is not intended and that a more in-depth response to different student reactions is needed.

Question 4: Will you use this kind of material in the future?

Intends to get a clear response on whether the teacher likes the modules and feels they are worthy of inclusion in the teaching schedule. Teacher responses can still be clouded by the actual teaching situation in the school and pressures from external sources e.g. examinations and hence the black and white response may not be forthcoming.

Question 5: What importance do you give to the wider goals of science education prormoted through these modules?

To determine the ‘readiness’ of the teacher for science teaching that goes beyond the cognitive and gives an indication of how far the teacher is willing to embrace wider education goals. For some teachers, used to following the textbook as the major or sole source of guidance, this question may have no meaning and hence, for them, unanswerable.

Question 6: Do you feel this project promoting similar materials across the whole of Europe and Israel is worthwhile?

To find out the teacher’s interest and reaction to science education outside the country and hence whether they have an interest in knowing more about science education outside the country.

Question 7: Will you recommend the

module to your colleagues?

To validate all the above.

SECTION 3B. Semi-structured interview questions for students who are interviewed by

the PARSEL personnel

This section is optional. We believe that a lot

of data can be gained through interviews.

We found that such data may validate the

quantitative results. However, if it puts too

much pressure on the teachers and on the

PARSEL personnel, it may be skipped.