Using Large-Scale Assessments to Improve Schools

Andrés Sandoval-Hernández – IEA DPCInternational Conference “Improve the school”

Naples, May 14-15, 2015

Contents

• About the IEA DPC

• Introduction to PIRLS and TIMSS

• Benchmarking function– International Reports PIRLS and TIMSS 2011 – Report on relationships between achievement in Math, Science and Reading

• Analytic function– School effectiveness factors for socially advantaged and disadvantaged

students in European countries

• Evaluation tools for teachers– Use of released items

About the IEA DPC

Who we are• International Association for the

Evaluation of Educational Achievement - Data Processing and Research Center

• Located in Hamburg, Germany• Association of countries

What we do• International Large scale assessments

in education • Main studies: TIMSS and PIRLS (2011)• Major activities are comparative

educational studies, research and training

• Almost 70 country members

Introduction to PIRLS and TIMSS

Major postulates of the IEA studies:– The quality of an educational system should be measured

by what students know– What students know is related to the characteristics of the

System– Assessments should be school based and curriculum

driven

What do PIRLS and TIMSS measure?– Achievement in Mathematics, Science and Reading – Student characteristics and learning environment (family,

school, context)

Introduction to PIRLS and TIMSS

Who uses this information?

1. Researchers2. Educational policy makers3. Teachers and principals4. Parents

• Alexander, R. (2012). Moral panic, miracle cures and educational policy: What can we really learn from international comparison? Scottish Educational Review, 44(1), 4–21.

• Areepattamannil, S. (2012). Science self-beliefs and science achievement of adolescents in Gulf Cooperation Council countries. Educational Studies, 38(1), 13–17. Doi:10.1080/03055698.2011.567058

• Areepattamannil, S. (2012). First- and second-generation immigrant adolescents’ multidimensional mathematics and science self-concepts and their achievement in mathematics and science. International Journal of Science & Mathematics Education, 10(3), 695–716.

• Bakker, S. (2012). Introduction of external, independent testing in “new countries”: Successes and defeats of the introduction of modern educational assessment techniques in former soviet and socialist countries. Educational Measurement: Issues & Practice, 31(2), 38–44. Doi:10.1111/j.1745-3992.2012.00234.x

• Chen, Y.-H. (2012). Cognitive diagnosis of mathematics performance between rural and urban students in Taiwan. Assessment in Education: Principles, Policy & Practice, 19(2), 193–209. Doi:10.1080/0969594x.2011.560562

• Cho, I. (2012). The effect of teacher–student gender matching: Evidence from OECD countries. Economics of Education Review, 31(3), 54–67. Doi:10.1016/j.econedurev.2012.02.002

• Choi, K., Choi, T., & Mcaninch, M. (2012). A comparative investigation of the presence of psychological conditions in high achieving eighth graders from TIMSS 2007 mathematics. ZDM, 44(2), 189–199. Doi:10.1007/S11858-012-0401-6

• Dodeen, H., Abdelfattah, F., Shumrani, S., & Hilal, M. A. (2012). The effects of teachers’ qualifications, practices, and perceptions on student achievement in TIMSS mathematics: a comparison of two countries. International Journal of Testing, 12(1), 61–77. Doi:10.1080/15305058.2011.621568

• Gültekin, S., & Demirtaşli, N. Ç. (2012). Çoktan seçmeli, açik uçlu ve karma testlerden sağlanan bilginin madde tepki kuramina dayali olarak karşilaştirilmasi. (in Turkish). Ilkogretim Online, 11(1), 251–263.

• Gür, B. S., Çelik, Z., & Özoğlu, M. (2012). Policy options for turkey: a critique of the interpretation and utilization of PISA results in Turkey. Journal of Education Policy, 27(1), 1–21. Doi:10.1080/02680939.2011.595509

• House, J. D. (2012). Motivational effects of computers and classroom instruction for science learning: an examination of eighth-grade students in the United States and Korea in the TIMSS 2007 Assessment. International Journal of Instructional Media, 39(2), 169–179.

• House, J. D., & Telese, J. A. (2012). Effects of mathematics lesson activities and computer use on algebra achievement of eighth-grade students in the United States and japan: findings from the TIMSS 2007 Assessment. International Journal of Instructional Media, 39(1), 69–81.

• Jianjun Wang. (2012). A comparative study of science achievement and self-concept during a historical transition. International Education Studies, 5(1), 137–146. Doi:10.5539/ies.v5n1p137

• Kwon, O., & Ju, M.-K. (2012). Standards for professionalization of mathematics teachers: policy, curricula, and national teacher employment test in Korea. ZDM, 44(2), 211–222. Doi:10.1007/s11858-012-0384-3

• Lloyd, D., & Pratt, A. (2012). The new OFSTED framework - Implications for teaching and learning. Teaching Business & Economics, 16(2), 28–29.• Männamaa, M., Kikas, E., Peets, K., & Palu, A. (2012). Cognitive correlates of math skills in third-grade students. Educational Psychology, 32(1), 21–

44. Doi:10.1080/01443410.2011.621713• Park, H., Lawson, D., & Williams, H. E. (2012). Relations between technology, parent education, self-confidence, and academic aspiration of Hispanic

immigrant students. Journal of Educational Computing Research, 46(3), 255–265.• Schmidt, W. H. (2012). At the precipice: The story of mathematics education in the United States. Peabody Journal of Education, 87(1), 133–156.

Doi:10.1080/0161956x.2012.642280• Serrano, A. M. (2012). A cross-cultural investigation into how tasks influence seatwork activities in mathematics lessons. Teaching & Teacher

Education, 28(6), 806–817. Doi:10.1016/j.tate.2012.03.001• Spencer, J. A. (2012). Views from the black of the math classroom. Dissent, 59(1), 76–80.

• Wiseman, A. W. (2012). The impact of student poverty on science teaching and learning: a cross-national comparison of the South African case. American Behavioral Scientist, 56(7), 941–960. Doi:10.1177/0002764211408861

• Yoshino, A. (2012). The relationship between self-concept and achievement in TIMSS 2007: a comparison between American and Japanese students. International Review of Education / Internationale Zeitschrift für Erziehungswissenschaft, 58(2), 199–219. Doi:10.1007/s11159-012-9283-7

• Zhu, Y., & Leung, F. (2012). Homework and mathematics achievement in Hong Kong: evidence from the TIMSS 2003. International Journal of Science & Mathematics Education, 10(4), 907–925.

• Bagata, B., Geske, A., & Kiselova, R. (2012). Using the IEA TIMSS tests to compare pupils´ Science Education Achievements at regional and school levels. Journal of Baltic Science Education, 3(1).

• Cavanagh, S. (2012). Complex policy options abound amid international comparisons. Education Week, 31(16), 6–10.• Chen, S. F., Lin, C. Y., Wang, J. R., Lin, S. W., & Kao, H. L. (2012). A cross-grade comparison to examine the context effect on the relationships among family resources, school

climate, learning participation, science attitude, and science achievement based on TIMSS 2003 in Taiwan. International Journal of Science Education, 1-18.• Falch, T., & Fischer, J. A. V. (2012). Public sector decentralization and school performance: International evidence. Economics Letters, 114(3), 276–279.• Sparks, S. D. (2012). Matching up states, countries gives fresh take on performance. Education Week, 31(16), 36–38.• Wang, Z., Osterlind, S., & Bergin, D. (N.D.). Building mathematics achievement models in four countries using TIMSS 2003. International Journal of Science and Mathematics

Education, 1–28. Doi:10.1007/s10763-011-9328-6• Brozo, W. G., Valtin, R., Garbe, C., Sulkunen, S., Shiel, G., & Pandian, A. (2012). A report from the PISA/PIRLS task force: member country highlights from PISA 2009. Reading

Today, 29(6), 11–13.• Caro, D. H., & Lenkeit, J. (2012). An analytical approach to study educational inequalities: 10 hypothesis tests in PIRLS 2006. International Journal of Research & Method in

Education, 35(1), 3–30. Doi:10.1080/1743727x.2012.666718• Thiel, O. (2012). Socio-economic diversity and mathematical competences. European Early Childhood Education Research Journal, 20(1), 61–81.

Doi:10.1080/1350293x.2012.650012• Van Staden, S., & Howie, S. (2012). Reading between the lines: contributing factors that affect grade 5 student reading performance as measured across South Africa’s 11

languages. Educational Research & Evaluation, 18(1), 85–98. Doi:10.1080/13803611.2011.641270• Caro, D. H., & Mirazchiyski, P. (2012). Socioeconomic gradients in eastern European countries: Evidence from PIRLS 2006. European Educational Research Journal, 11(1), 96–110.• Cavanagh, S. (2012). Complex policy options abound amid international comparisons. Education Week, 31(16), 6–10• Blömeke, S., Suhl, U., Kaiser, G., & Döhrmann, M. (2012). Family background, entry selectivity and opportunities to learn: what matters in primary teacher education? An

international comparison of fifteen countries. Teaching & Teacher Education, 28(1), 44–55. Doi:10.1016/j.tate.2011.08.006• Sawchuk, S. (2012). Teacher training has key role to play. Education Week, 31(16), 14.• Döhrmann, M., Kaiser, G., & Blömeke, S. (2012). The conceptualization of mathematics competencies in the International Teacher Education Study TEDS-M. ZDM, 44(3), 325–

340. Doi:10.1007/s11858-012-0432-z• Felbrich, A., Kaiser, G., & Schmotz, C. (2012). The cultural dimension of beliefs: An investigation of future primary teachers’ epistemological beliefs concerning the nature of

mathematics in 15 countries. ZDM, 44(3), 355–366. Doi: 10.1007/s11858-012-0418-x• König, J., & Blömeke, S. (2012). Future teachers’ general pedagogical knowledge from a comparative perspective: Does school experience matter? ZDM, 44(3), 341–354. Doi:

10.1007/s11858-012-0394-1• Blömeke, S., & Delaney, S. (2012). Assessment of teacher knowledge across countries: A review of the state of research. ZDM, 44(3), 223–247. Doi:10.1007/s11858-012-0429-7

• Schmidt, W. H. (2012). At the precipice: the story of mathematics education in the United States. Peabody Journal of Education, 87(1), 133–156. Doi:10.1080/0161956x.2012.642280• Blömeke, S., & Kaiser, G. (2012). Homogeneity or Heterogeneity? Profiles of opportunities to learn in primary teacher education and their relationship to cultural context and outcomes.

ZDM, 44(3), 249–264. Doi:10.1007/s11858-011-0378-6• Cole, Y. (2012). Assessing elemental validity: the transfer and use of mathematical knowledge for teaching measures in Ghana. ZDM, 44(3), 415–426. Doi:10.1007/s11858-012-0380-7• Delaney, S. (2012). A validation study of the use of mathematical knowledge for teaching measures in Ireland. ZDM, 44(3), 427–441. Doi:10.1007/s11858-012-0382-5• Fauskanger, J., Jakobsen, A., Mosvold, R., & Bjuland, R. (2012). Analysis of psychometric properties as part of an iterative adaptation process of MKT items for use in other countries.

ZDM, 44(3), 387–399. Doi:10.1007/s11858-012-0403-4• Senk, S., Tatto, M., Reckase, M., Rowley, G., Peck, R., & Bankov, K. (2012). Knowledge of future primary teachers for teaching mathematics: An international comparative study. ZDM,

44(3), 307–324. Doi: 10.1007/s11858-012-0400-7• Barber, C., & Torney-Purta, J. (2012). Comparing attitudes in the 1999 and 2009 IEA Civic and Citizenship Education Studies: Opportunities and limitations illustrated in five countries.

Journal of Social Science Education, 11(1), 47–63.• Cosgrove, J., & Gilleece, L. (2012). An international perspective on civic participation in irish post-primary schools: Results from ICCS. Irish Educational Studies, 1–19.

Doi:10.1080/03323315.2012.664948• Gilleece, L., & Cosgrove, J. (2012). Student civic participation in school: what makes a difference in Ireland? Education, Citizenship and Social Justice. Doi:10.1177/1746197912448715• Ince, B. (2012). Citizenship education in turkey: inclusive or exclusive. Oxford Review of Education, 38(2), 115–131. Doi:10.1080/03054985.2011.651314• Liljestrand, J. (2012). Citizenship education in discussions concerning political issues. Utbildning & Demokrati, 21(1), 77–95.• Ljunggren, C., & Unemar Öst, I. (2012). Teachers’ dealings with controversial issues – a typology from the 2009 IEA/ICCS study. PCS – Politics, Culture and Socialization, 2(3).• Quaynor, L. J. (2012). Citizenship education in post-conflict contexts: a review of the literature. Education, Citizenship and Social Justice, 7(1), 33–57. Doi:10.1177/1746197911432593• Kennedy, K. J. (2012). Asian students’ citizenship values and their relationship to civic understanding: An exploratory study comparing Thai and Hong Kong students. Research in

Comparative and International Education, 7(2), 248–259.• Cosgrove, J., & Gilleece, L. (2012). An international perspective on civic participation in Irish post-primary schools: Results from ICCS. Irish Educational Studies, 31(4), 377–395.• Gilleece, L., & Cosgrove, J. (2012). Student civic participation in school: What makes a difference in Ireland? Education, Citizenship & Social Justice, 7(3), 225–239.• Macháček, L. (2012). Education of the school youth to democratic citizenship in Slovakia. Mokinių Demokratinio Pilietiškumo Ugdymas Slovakijoje., (107), 110–117.• House, J. D. (2012). Effects of computer engagement and classroom instructional activities on reading achievement of a national sample of Chinese- American elementary- school

students: results from the TIMSS 2006 assessment. International Journal of Instructional Media, 39(4), 345–354.• Johansson, S., Myrberg, E., & Rosén, M. (2012). Teachers and tests: assessing pupils’ reading achievement in primary schools. Educational Research & Evaluation, 18(8), 693–711.• Lee, M., & Hallinger, P. (2012). National contexts influencing principals’ time use and allocation: Economic development, societal culture, and educational system. School Effectiveness &

School Improvement, 23(4), 461–482.• Sparfeldt, J. R., Kimmel, R., Löwenkamp, L., Steingräber, A., & Rost, D. H. (2012). Not read, but nevertheless solved? Three experiments on PIRLS multiple choice reading comprehension

test items. Educational Assessment, 17(4), 214–232.• Abu-Hilal, M. M., Abdelfattah, F. A., Alshumrani, S. A., Abduljabbar, A. S., & Marsh, H. W. (N.D.). Construct validity of self-concept in TIMSS’s student background questionnaire: a test of

separation and conflation of cognitive and affective dimensions of self-concept among Saudi eighth graders. European Journal of Psychology of Education, 1–20. Doi:10.1007/s10212-012-0162-1

• Chen, S.-F., Lin, C.-Y., Wang, J.-R., Lin, S.-W., & Kao, H.-L. (2012). A cross-grade comparison to examine the context effect on the relationships among family resources, school climate, learning participation, science attitude, and science achievement based on TIMSS 2003 in Taiwan. International Journal of Science Education, 34(14), 2089–2106. Doi:10.1080/09500693.2012.701352

• Glynn, S. M. (2012). International Assessment: A RASCH model and teachers’ evaluation of TIMSS science achievement items. Journal of Research in Science Teaching, 49(10), 1321–1344.

• Hansson, Å. (2012). The meaning of mathematics instruction in multilingual classrooms: Analyzing the importance of responsibility for learning. Educational Studies in Mathematics, 81(1), 103–125.

• Hee Kyung Hong. (2012). Trends in mathematics and science performance in 18 countries: Multiple regression analysis of the cohort effects of TIMSS 1995-2007. Tendências no desempenho em matemáticas e ciências em 18 países: análise de regressão múltipla de efeitos de coorte no TIMSS 1995-2007., 20(33), 1–20.

• House, J. D. (2012b). Science achievement of elementary-school students in the United States and Japan in TIMSS 2007: An assessment of the effects of technology engagement and classroom lesson activities. International Journal of Instructional Media, 39(3), 263–274.

• Louis, R., & Mistele, J. (2012). The differences in scores and self-efficacy by student gender in mathematics and science. International Journal of Science & Mathematics Education, 10(5), 1163–1190.

• Mesic, V. (2012). Identifying country-specific cultures of physics education: A differential item functioning approach. International Journal of Science Education, 34(16), 2483–2500.

• Mohammadpour, E. (2012a). Factors accounting for mathematics achievement of Singaporean eighth-graders. Asia-Pacific Education Researcher, 21(3), 507–518.

• Mohammadpour, E. (2012b). A multilevel study on trends in Malaysian secondary school students’ science achievement and associated school and student predictors. Science Education, 96(6), 1013–1046.

• Ng, K. T., Lay, Y. F., Areepattamannil, S., Treagust, D. F., & Chandrasegaran, A. L. (2012). Relationship between affect and achievement in science and mathematics in Malaysia and Singapore. Research in Science & Technological Education, 30(3), 225–237.

• Rutkowski, D., Rutkowski, L., & Plucker, J. A. (2012). Trends in education excellence gaps: A 12-year international perspective via the multilevel model for change. High Ability Studies, 23(2), 143–166.

• Vandecandelaere, M., Speybroeck, S., Vanlaar, G., De Fraine, B., & Van Damme, J. (2012). Learning environment and students’ mathematics attitude. Studies in Educational Evaluation, 38(3/4), 107–120.

• Wang, Z., Osterlind, S. J., & Bergin, D. A. (2012). Building mathematics achievement models in four countries using TIMSS 2003. International Journal of Science and Mathematics Education, 10(5), 1215–1242. Doi:10.1007/s10763-011-9328-6

• Wiseman, A. W., & Anderson, E. (2012). ICT-Integrated Education and National Innovation Systems in the Gulf Cooperation Council (GCC) countries. Computers & Education, 59(2), 607–618.

• Yu, C. H. (2012). Examining the relationships among academic self-concept, instrumental motivation, and TIMSS 2007 science scores: A cross-cultural comparison of five East Asian countries/regions and the United States. Educational Research & Evaluation, 18(8), 713–731.

• Blómeke, S. (2012). Content, Professional Preparation, and Teaching Methods: How Diverse Is Teacher Education across Countries? Contenido, preparación profesional y métodos de enseñanza: ¿Cuan diversa es la educación de los profesores entre países?, 56(4), 684–714.

• Chudgar, A., Luschei, T. F., & Yisu Zhou. (2013). Science and Mathematics Achievement and the Importance of Classroom Composition: Multicounty Analysis Using TIMSS 2007. American Journal of Education, 119(2), 295–316.

• Hopland, A. O. (2013). School Facilities and Student Achievement in Industrial Countries: Evidence from the TIMSS. International Education Studies, 6(3), 162–171.

• Peterson, P. E. (2013). Revelations from the TIMSS. Education Next, 13(2), 5–5.

• Shymansky, J., Annetta, L., Yore, L. D., Wang, T.-L., & Everett, S. (2013). The Impact of a Multiyear Systemic Reform Effort on Rural Elementary School Students’ Science Achievement. School Science & Mathematics, 113(2), 69–79.

• Turgut, G. (2013). International Tests and the U.S. Educational Reforms: Can Success Be Replicated? Clearing House, 86(2), 64–73.• Hao, S., & Johnson, R. L. (2013). Teachers’ classroom assessment practices and fourth-graders’ reading literacy achievements: An international

study. Teaching & Teacher Education, 29, 53–63.• Sandilands, D., Oliveri, M. E., Zumbo, B. D., & Ercikan, K. (2013). Investigating Sources of Differential Item Functioning in International Large-

Scale Assessments Using a Confirmatory Approach. International Journal of Testing, 13(2), 152–174.• Cho,Y., & Kim, T. (2013). Asian Civic Values: A Cross-Cultural Comparison of Three East Asian Societies. Asia-Pacific Education Researcher, 22(1),

21–31.• Geske, A., & Cekse, I. (2013). The Influence of Civic and Citizenship Education Achievements on the Development of Students’ Citizenship

Attitudes. Problems of Education in the 21st Century, 52, 21–34.• Hooghe, M., & Dassonneville, R. (2013). Voters and Candidates of the Future: The Intention of Electoral Participation among Adolescents in 22

European Countries. Young, 21(1), 1–28.• Kaiser, G., & Blömeke, S. (2013). Learning from the Eastern and the Western debate: The case of mathematics teacher education. ZDM, 45(1),

7–19.• Andrews, P. (2013). What Does PISA Performance Tell Us about Mathematics Teaching Quality? Case studies from Finland and Flanders. Oxford

Studies in Comparative Education, 23(1), 99–115.• Daniel, E. G. S. (2013). Asia Pacific science education in a knowledge society. Asia Pacific Journal of Education, 33(2), 170–182.• Drent, M., Meelissen, M. R. M., & Van Der Kleij, F. M. (2013). The contribution of TIMSS to the link between school and classroom factors and

student achievement. Journal of Curriculum Studies, 45(2), 198–224.• Hansson, Å., & Gustafsson, J.-E. (2013). Measurement Invariance of Socioeconomic Status across Migrational Background. Scandinavian

Journal of Educational Research, 57(2), 148–166.• Jen, T.-H., Lee, C.-D., Chien, C.-L., Hsu, Y.-S., & Chen, K.-M. (2013). Perceived Social Relationships and Science Learning Outcomes for Taiwanese

Eighth Graders: Structural Equation Modeling with a Complex Sampling Consideration. International Journal of Science & Mathematics Education, 11(3), 575–600.

• Larson, M. R., & Leinwand, S. (2013). Prepare for More Realistic Testing Results. Mathematics Teaching in the Middle School, 18(9), 524–526.• Machin, S., McNally, S., & Wyness, G. (2013). Educational attainment across the UK nations: performance, inequality and evidence. Educational

Research, 55(2), 139–164.• Mills, J. D., & Holloway, C. E. (2013). The development of statistical literacy skills in the eighth grade: exploring the TIMSS data to evaluate

student achievement and teacher characteristics in the United States. Educational Research & Evaluation, 19(4), 323–345.• Pahlke, E., Hyde, J. S., & Mertz, J. E. (2013). The Effects of Single-Sex Compared With Coeducational Schooling on Mathematics and Science

Achievement: Data from Korea. Journal of Educational Psychology, 105(2), 444–452.

• Peterson, P. E. (2013). Revelations from the TIMSS. Education Next, 13(2), 5–5.• Rutkowski, L., Rutkowski, D., & Engel, L. (2013). Sharp Contrasts at the Boundaries: School Violence and Educational Outcomes Internationally.

Comparative Education Review, 57(2), 232–259.• Tienken, C. H. (2013). Conclusions from PISA and TIMSS Testing. Kappa Delta Pi Record, 49(2), 56–58.• Sandilands, D., Oliveri, M. E., Zumbo, B. D., & Ercikan, K. (2013). Investigating Sources of Differential Item Functioning in International Large-

Scale Assessments Using a Confirmatory Approach. International Journal of Testing, 13(2), 152–174.• Barber, C., Fennelly, K., & Torney-Purta, J. (2013). Nationalism and Support for Immigrants’ Rights Among Adolescents in 25 Countries. Applied

Developmental Science, 17(2), 60–75.• Lauglo, J.013). Do more knowledgeable adolescents have more rationally based civic attitudes? Analysis of 38 countries. Educational

Psychology, 33(3), 255–275.• Schulz, W., Fraillon, J., & Ainley, J. (2013). Measuring young people’s understanding of civics and citizenship in a cross-national study.

Educational Psychology, 33(3), 327–349.• Gustafsson, J.-E. (2013). Causal inference in educational effectiveness research: a comparison of three methods to investigate effects of

homework on student achievement. School Effectiveness & School Improvement, 24(3), 275–295.• Marsh, H. W. &Amp;Rft Au=Abduljabbar. (2013). Factorial, convergent, and discriminant validity of TIMSS math and science motivation

measures: A comparison of Arab and Anglo-Saxon countries. The Journal of Educational Psychology, 105(1), 108–128• Woolard, J. C. (2013). Prelude to the Common Core: Internationally Benchmarking a State’s Math Standards. Educational Policy, 27(4), 615–

644.A three-level multilevel analysis of Singaporean eighth-graders science achievement. Learning & Individual Differences, 26, 212–220.• Blömeke, S., Suhl, U., & Döhrmann, M. (2013). Assessing Strengths and Weaknesses of Teacher Knowledge in Asia, Eastern Europe, and

Western Countries: Differential Item Functioning in TEDS-M. International Journal of Science & Mathematics Education, 11(4), 795–817.• Cañadas, M., Gómez, P., & Rico, L. (2013). Structure of Primary Mathematics Teacher Education Programs in Spain. International Journal of

Science & Mathematics Education, 11(4), 879–894.• Hsieh, F.-J. (2013). Strengthening the Conceptualization of Mathematics Pedagogical Content Knowledge for International Studies: A Taiwanese

Perspective. International Journal of Science & Mathematics Education, 11(4), 923–947.• Hsieh, F.-J., Wong, K., & Wang, T.-Y. (2013). Are Taiwanese and Singaporean Future Teachers Similar in Their Mathematics-Related Teaching

Competencies? International Journal of Science & Mathematics Education, 11(4), 819–846.• Laschke, C. (2013). Effects of Future Mathematics Teachers’ Affective, Cognitive and Socio-Demographic Characteristics on Their Knowledge at

the End of the Teacher Education in Germany and Taiwan. International Journal of Science & Mathematics Education, 11(4), 895–921.• Wang, T.-Y., & Tang, S.-J. (2013). Profiles of Opportunities to Learn for Teds-M Future Secondary Mathematics Teachers. International Journal of

Science & Mathematics Education, 11(4), 847–877.• Zuckerman, G. A., Kovaleva, G. S., & Kuznetsova, M. I. (2013). Between PIRLS and PISA: The advancement of reading literacy in a 10–15-year-

old cohort. Learning & Individual Differences, 26, 64–73.• Nemeržitski, S., Loogma, K., Heinla, E., & Eisenschmidt, E. (2013). Constructing model of teachers’ innovative behaviour in school environment.

Teachers & Teaching, 19(4), 398–418.

How do they use the information? What do they do with it?

Benchmarking function:

International ReportsPIRLS and TIMSS 2011

PIRLS International ReportProgress in International Reading Literacy Study

PIRLS International ReportFurther increases in performance than decreases

- Compared to 2001, ten countries increased their average.

Compared to 2006, thirteen countries increased their average.

Only four countries recorded declines: Bulgaria, Lithuania, the Netherlands and Sweden.

TIMSS International Report (Mathematics 4th grade)Trends in International Mathematics and Science Study

TIMSS International Report (Mathematics 8th grade)Trends in International Mathematics and Science Study

TIMSS International Report More increases than decreases in progress in 4th grade, but not 8th grade

- In 4th grade, of the 17 countries with comparable data, thirteen recorded increases and only three recorded decreases.

- In 8th grade, of the 25 countries with comparable data, nine recorded increases and eleven recorded decreases.

13

TIMSS International Report Increases in Mathematic Achievement (8th grade) in Italy between 1999 and 2011

14

TIMSS International Report Increases in Mathematic Achievement (8th grade) in Singapore and Italy between 1999 and 2011

• Singapore: Increase of 6 Score Points between 1999 and 2011• Italy: Increase of 16 Score Points between 1999 and 2011

1999 2003 2007 20110

100

200

300

400

500

600

700

604 605 593 611

479 484 480 495

Mathematic Achievement from 1999 to 2011

SingaporeItaly

TIMSS International Report (Science 4th grade) Trends in International Mathematics and Science Study

TIMSS International Report (Science 8th grade) Trends in International Mathematics and Science Study

TIMSS International Report More increases than decreases in performance particularly in the 4th grade

- In 4th grade, of the 17 countries with comparable data, eight reported increases and only one reported decreases.

- In 8th grade, of the 25 countries with comparable data, eleven reported increases and six reported decreases.

18

TIMSS International Report Results in Science Achievement (8th grade) in Italy between 1999 and 2011 remain stable

19

TIMSS International Report Increases in Science Achievement (8th grade) in Singapore and Italy between 1999 and 2011

1999 2003 2007 20110

100

200

300

400

500

600

700

568 578 567590

493 491 495 501

Science Achievement from 1999 to 2011

SingaporeItaly

• Singapore: Increase of 22 Score Points between 1999 and 2011• Italy: Increase of 8 Score Points between 1999 and 2011

Benchmarking function:

Report on relationships between achievement in Math, Science and Reading

20

Relationships between achievement in Mathematics, Science and Reading

• The analysis includes only countries that applied PIRLS and TIMSS to the same students (4th grade)

• It is an edited volume with four chapters – Profiles of achievement in mathematics, reading and

science – The impact of reading skill achievement in TIMSS (math

and science) – Effective schools in mathematics, reading and science – The effects of home background on academic achievement

Relationships between achievement in Mathematics, Science and ReadingAchievement Profiles

Relationships between achievement in Mathematics, Science and ReadingAchievement Profiles

Relationships between achievement in Mathematics, Science and ReadingAchievement Profiles

Relationships between achievement in Mathematics, Science and ReadingAchievement Profiles

Analytic function:

A Comparison of School Effectiveness Factors for Socially Advantaged and Disadvantaged Students in ten European Countries in TIMSS-2011

26

Objectives• To what extent an Educational Effectiveness Research model works similarly

for socially advantaged and disadvantaged students?

Methods• Using HLM analyses on the TIMSS 2011 dataset, the fit of the EER model is

evaluated in two sub-samples: 1. Advantaged Students2. Disadvantaged Students

• comparing the results across ten European countries, including Italy

Results• EER model fit better the data of advantaged students in most countries and in

both levels (i.e. student and school level)

A Comparison of School Effectiveness Factors for Socially Advantaged and Disadvantaged Students

Control• Resources for learning at

home • Early reading and math

skills

Analysis

• Security and order atmosphere

• Support for academic success

• Adequate infrastructure • Trained teachers • Well-equipped classrooms • Effective instruction

Relationships between achievement in Mathematics, Science and Reading Effective schools in Mathematics, Science and Reading

29

A Comparison of School Effectiveness Factors for Socially Advantaged and Disadvantaged StudentsSummary of results for all countries

Results for Italy:• Overall, the EER model fits the sample of advantaged

students better (on both levels), but…

Evaluation tools for teachers:

Use of released items

30

Use of released items

• Teachers can use released items in different ways

– To evaluate specific processes, such as processes for Reading Comprehension:

- Make direct inferences - Locate and obtain explicit information - Interpret and integrate ideas and information

– Teachers can give the items to the students and grade them using the corresponding guides

– Teachers can even compare the percentage of students who responded correctly with the international average

For feedback on the teaching-learning process – When the concepts of the class match the items released,

the teacher can use the same procedure to get immediate feedback on their performance

To understand misunderstandings or confusions – The teacher can analyze incorrect or partially correct answers

in its class to understand possible confusion and reinforce a specific topic

To identify individual difficulties – As a basis for planning remedial actions

Use of released items

Use of released itemsExample for schools: Reading comprehension

Enemy Pie Written by Derek Munson, Illustrated by Calahan King

Use of released itemsExample for schools: Mathematics

Use of released itemsExample for schools: Science

Publications:

Mullis, I.V.S., Martin, M.O., Foy, P., & Drucker, K.T. (2012). PIRLS 2011 International Results in Reading. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.

Mullis, I.V.S., Martin, M.O., Foy, P., & Arora, A. (2012). TIMSS 2011 International Results in Mathematics. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.

Martin, M.O., Mullis, I.V.S., Foy, P., & Stanco, G.M. (2012). TIMSS 2011 International Results in Science. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.

Martin, M.O. & Mullis, I.V.S. (Eds.). (2013). TIMSS and PIRLS 2011: Relationships among Reading, Mathematics, and Science Achievement at the Fourth Grade Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.

Sandoval-Hernandez, A., Castejon-Company, A. & Aghakasari, P. (2015). A Comparison of School Effectiveness Factors for Socially Advantaged and Disadvantaged Students in ten European Countries in TIMSS 2011. Šolsko polje 25 (3-4), 61-96.

Nielsen, T. & Angell, C. (2014). The importance of discourse and attitude in learning astronomy: A mixed methods approach to illuminate the results of the TIMSS 2011 survey. NorDiNa 10(1), 16-31.

More Information:

www.iea-dpc.de | www.iea.nl

FB: www.facebook.com/IEARANDA