Teacher Questionnaire

National Research Coordinators Meeting Windsor, June 2008

NRCMeetingWindsor

June 2008

Content of the presentation

• Field trial participation• Main field trial results• Draft questionnaire for the MS• Survey activities questionnaire

(comments on TcQ)• Use of teachers’ data

NRCMeetingWindsor

June 2008

FT participation and results

• Scales: good reliability

• Teachers of all subjects

• Teachers’ response rate

• Schools’ participation – teachers’ participation

NRCMeetingWindsor

June 2008

FT participation

[DPC data]

Teachers in survey 11201

Teachers with quest. data

9316

Schools in survey 718

Schools with quest. data 682

NRCMeetingWindsor

June 2008

FT participation

• 17 countries > 85%

• 6 countries 85-80%

• 8 countries < 80%

(two countries < 50%)

[FT Analysis Report Part II, Table 2]

NRCMeetingWindsor

June 2008

Teacher characteristics• Subject(s) taught (categories from

Talis)– Teachers of all subjects [T. 3 ]

Proposal: only two questions• Gender

– In all but five countries female teachers are more than males. Overall percentage 65%.

– Gender distribution varies across countries (female teachers from 39% to 84%)[T.10]

NRCMeetingWindsor

June 2008

Teacher characteristics

• Age– No category can be omitted

[T 9]

NRCMeetingWindsor

June 2008

Scale reliability

• Confidence in teaching methods CONFTCH (Q11) α .759 [T11]– Minor changes in wording –

comments in translation and adaptation forms

• Participation in school governance TCHPART1 (Q12)

α .829 [T12]– One item dropped (item a)

NRCMeetingWindsor

June 2008

Scale reliability

• Importance given to student opinions TSTUDIMP (Q14) – α .830 [T13]

Proposal: to delete (also in ScQ): not appropriate to grade 8 students

NRCMeetingWindsor

June 2008

Scale reliability

• Student-teacher activities in community - STUDACT (Q15) α .729 [T14]– One item dropped (item a)– To report selected single items instead of

scale scores

• Teacher participation in community activities - TCHACT (Q16) α .794 [T15]– To report selected single items instead of

scale scores

NRCMeetingWindsor

June 2008

Scale reliability

• Student school climate - TSCHCLIM (Q17)

α .836 [T16]

– Two dimensions expected: student sense of belonging to the school; student respect for school’s rules

– One factor only (EFA)

– Items e and h dropped

NRCMeetingWindsor

June 2008

Scale reliability

• Teaching approaches/strategies TCHCONTR - STUDAUTO (Q18)

α .296 and .610 [T17]– Very low reliability– No “content free”– To be dropped

NRCMeetingWindsor

June 2008

Scale reliability• Use of assessment - TCASSES1

(Q19) α .790 [T18]– Item a dropped (low item-total correlations in

many countries)– Only three categories (to collapse “to a little

extent” and “not at all”)

• Student participation in classroom activities/discussion - STCLACT1- STCLDIS (Q20) α .683 and α .720 [T19] after deleting item b– Two dimensional structure confirmed (EFA)

NRCMeetingWindsor

June 2008

Scale reliability

• Classroom climate - TCCLIM (Q21)

α .856 [T20]– Item a dropped

• Classroom discipline - TCDISC (Q22) α .686 [T21]– Low or very poor reliability for many

countries– Item battery to be deleted

NRCMeetingWindsor

June 2008

• Questions on aims of CCE at school (Q23, Q24) [T22]– Format: Q23 to choose the three most

important; Q24 Likert type– Only one question within the format of

Q23– Parallel to the question in ScQ for

comparison (Q17)– Three dimensions: knowledge and

skills, sense of responsibility and active participation

NRCMeetingWindsor

June 2008

• Responsibility of CCE at school (Q25)– <CCE> in item a: NRCs are requested

to give a list of subjects – Item about principals added

• To introduce a question about issues of social tension inside the school (see Q15 in ScQ)

NRCMeetingWindsor

June 2008

International option

• Filter question: NRCs are requested to give a list of subjects of <CCE> to avoid self-selection

• Questions derived from CIVED

• Q27 (sources for CCE): items should be reported as single items for descriptive purposes

NRCMeetingWindsor

June 2008

International option

• Teaching and learning activities TCIVACT-TCIVTRAD (Q28) [T23]– TCIVACT α .760– TCIVTRAD α .646– Two-dimensional structure confirmed

(EFA)– α marginally satisfactory. Values for

TCIVTRAD vary considerably across countries

– Important (from CIVED). Keep in

NRCMeetingWindsor

June 2008

International option

• Confidence in CCE teaching TCIVCONF (Q31)

α .903 [T24]– More appropriate to report on single

items (description of CCE context across countries)

• CCE improvement (Q32) from CIVED– Keep in

NRCMeetingWindsor

June 2008

Survey activities questionnaire

• Comments on teacher survey and on teacher questionnaireA. Comments on administration

procedures (question A7)– Listing of all target grade teachers – Teachers’ year/month of birth– Misunderstanding about teacher

population– Schools/teachers participations– Time of the sampling/administration– Confidentiality/anonimity

NRCMeetingWindsor

June 2008

Survey activities questionnaire

B. Comments on difficulties in reaching student and teacher participation (question F3)

– Comments from 10 countries related to TcQ

– 9 countries report difficulties in obtaining teacher participation

– Principals disagreement– Teachers of different subjects

NRCMeetingWindsor

June 2008

– Length of the questionnaire– Teachers not in the target grade– Some questions were considered

“very indiscrete” – On-line /paper mode

But also• Higher participation than expected• Strategies for overcoming the

difficulties

NRCMeetingWindsor

June 2008

Survey activities questionnaire

• Lenght of the questionnaire

• Clarification of the aims of the teacher survey

• Procedures for presenting the survey to schools

• Tracking forms

NRCMeetingWindsor

June 2008

Proposed TcQ for the MS

• Same sections• Changes based on FT results• Lower number of questions

– Core: 21 instead of 25– International option: 6 questions

(plus filter question)

• Changes in wording according to NRCs comments (NAFs)

• Additional explanatory notes included

NRCMeetingWindsor

June 2008

• International option• List of subjects provided by NRCs• Possibility for an oversampling of

CCE teachers

NRCMeetingWindsor

June 2008

Use of teacher data• Reporting data at the country level

– To provide information about the context of civic and citizenship education from the perspective of school teachers (in general)

• Obtaining aggregated school level data - To provide a picture of how schools implement civic-related teaching practices and about the climate for civic and citizenship education. Aggregated measures can also be used as variables at the student level or in multi-level analysis.

NRCMeetingWindsor

June 2008

Questions or comments?