1

I n f o r m a t i o nabout the PISA Study 2015

3

Objectives of the PISA studyHow well prepared are young people to tackle present and future challenges in their private and professional life, and to engage in lifelong learning?

How effective, and how fair, are the different educational systems?

How do the competencies of 15-year-old students, the various educational systems and the general academic framework develop over the course of time?

Every three years, on behalf of the Organisation for Economic Co-operation and Development (OECD), basic competencies of students aged 15 in the fields of reading, mathematics and science are measured and compared on an international basis. Under the PISA assessment (Programme for International Student Assessment), the young people concerned are not required simply to give an account of knowledge acquired by them in school; instead, they are tested on their ability to apply their knowledge and skills in solving problems set in real-life contexts. In addition, PISA surveys important interdisciplinary competencies such as problem-solving, as well as non-cognitive outcomes of the teaching and learning process (e.g. motivation to learn).

The aim of the programme is to make available to the countries concerned internationally comparable indicators concerning the knowledge and abilities of young people, as well as core aspects of the educational systems and the general academic framework. Those data make it possible to identify strengths, but also potential problem areas, knowledge of which can be used to improve the educational systems.

4

PISA 2015: Participating countries

OECD countries OECD partner countries

Australia AlbaniaAustria AlgeriaBelgium ArgentinaCanada BrazilChile BulgariaCzech Republic Chinese TaipeiDenmark ColombiaEstonia Costa RicaFinland CroatiaFrance Dominican RepublicGermany Republic of MacedoniaGreece GeorgiaHungary Hong Kong-ChinaIceland IndonesiaIreland JordanIsrael KazakhstanItaly KosovoJapan LatviaKorea LebanonLuxembourg LithuaniaMexico Macao-ChinaNetherlands MalaysiaNew Zealand MaltaNorway MoldovaPoland MontenegroPortugal People’s Republic of ChinaSlovak Republic PeruSlovenia QatarSpain RomaniaSweden Russian FederationSwitzerland SingaporeTurkey ThailandUnited Kingdom Trinidad and TobagoUnited States of America Tunisia

United Arab EmiratesUruguayVietnam

5

Principal characteristics of PISA 2015PISA is a cyclical study designed to test, every three years, students‘ knowledge and abilities in the fields of reading, mathematics and science. In each PISA cycle, the focus is placed on one of the three fields. In PISA 2015, the emphasis is on science. Reading skills and mathematics will be examined as minor domains. As part of an international option, Luxembourg will in addition be participating in the measurement of students‘ skills in the field of collaborative problem-solving.

The PISA 2015 tests will, for the first time, be completely computer-based – that is to say, all participating students will work on the tests using a computer. The test software is saved on USB sticks. Each student will be given log-in data enabling him/her to start working on the tests.

As in the past, Luxembourgish students can choose their test language (German or French). Students in English-speaking schools and English-language sections will be tested in English.

PISA 2000

PISA 2003

PISA 2006

PISA 2009

PISA 2012

PISA 2015

Reading MathematicsScience

Reading MathematicsScience

Reading MathematicsScience

Reading MathematicsScience

Reading MathematicsScience

Reading MathematicsScience

Paper and pencil test

Paper and pencil test

Paper and pencil test

Paper and pencil test

Paper and pencil test

Computer-based test

6

Test domains: What does PISA measure? For each test area, a conceptual framework has been developed, describing the composition and content of the test area. The complete PISA framework may be downloaded at http://www.oecd.org/pisa/pisaproducts/pisa2015draftframeworks.htm.

Reading literacyThe term „reading literacy“ is interpreted by PISA as the ability to comprehend and use texts of different kinds from a variety of contexts, to reflect on what one has read and to develop a deeper understanding of the text. This is intended to enable students to enhance their own knowledge and capabilities and to take an active part in life in society.

The PISA framework distinguishes three relevant areas of literacy:

• TextsIn their everyday life, young people and adults encounter a wide range of texts, which differ in their format and their rhetorical structure (e.g. description, narration, expo-sition, argumentation, instruction, etc.). Consequently, the following text formats are used in PISA:

-- continuous-texts: texts written in a continuous manner, such as novels, newspaper articles, essays, short stories and letters

- non-continuous-texts: documents in which information is presented differently from in prose texts, such as lists, forms, itineraries, graphics and diagrams

• AspectsThe aspects of reading delineate the kind of reading approaches involved, i.e. the strategies and intentions with which a reader approaches the text. The PISA tests encompass three different aspects of reading:

- accessing-and-retrieving-information, i.e. searching for, selecting and using one or more items of information in a text

- integrating-and-interpreting, i.e. identifying the meaning of one or more pieces of text, drawing conclusions from them and thereby developing an overall understanding of the text

- reflecting-and-evaluating, i.e. making connections between the text and one‘s own knowledge, ideas and experiences

• SituationsThe texts used in PISA concern a variety of situations close to everyday life in which adults and young people are frequently confronted with similar texts:

- reading in a personal-context, e.g. novels - reading in the public context, e.g. official

reading in an occupational context, e.g. manuals

- reading in an educational context, e.g. school

7

Mathematical literacyThe PISA approach is based on the fundamental idea that, in the course of their school education, students should be enabled to use mathematical skills purposefully in different situations arising in everyday life and in professional life. Thus, for PISA, the concept of mathematical literacy encompasses not only the performance of specific mathematical operations but also the problem-oriented application of mathematics in real life.

The PISA framework distinguishes three relevant areas of mathematical literacy:

• Content - change-and-relationships: all types of

relational and functional connections between mathematical objects, such as the characterisation of relationships by means of equations and inequations

- space-and-shape: all types of flat or spatial configurations, shapes and models, such as the identification of geometric forms and patterns

- quantity: all types of quantification, i.e. the use of numbers for the characterisation and organisation of situations, such as calculating with numbers, dealing with sizes and understanding dimensions

- uncertainty-and-data: all types of phenomena and situations which contain statistical data or in which chance plays a role, such as analysis and interpretation of given data

• ProcessesThree fundamental mathematical processes have been defined which are relevant for the purposes of solving a problem:

- formulating situations mathematically, i.e. giving a mathematical structure to a problem arising from a particular context

- employing-mathematical concepts, facts, procedures and reasoning, in order to arrive at the solution to a mathematical problem

- interpreting, applying-and-evaluating mathematical outcomes, i.e. reflecting on mathematical solutions, outcomes and inferences

The following six general mathematical competencies underlie these three processes:

- devising strategies - mathematising - reasoning and argument - communication - representation - using symbolic, formal and technical

language and operations

• SituationsThese relate to the context in which a mathematical assignment is embedded. Contexts are subdivided into personal, educational, occupational, societal and scientific-situations.

8

Scientific literacyOur society is currently confronted, and will continue in the future to be confronted, by ever greater challenges. These relate inter alia to securing supplies of food and drinking water, controlling infectious diseases, energy supply and climate change. These problems are significant on both an individual and a national and international level, and are closely linked to science and science-based technology. In order to ensure that they are able to tackle these challenges in the future, it is important that students should develop a basic understanding of scientific content, methods and research.

The PISA framework distinguishes three relevant areas of scientific literacy:

• Scientific knowledge - knowledge-of-facts,-concepts-and-content:

knowledge of the natural world in the fields of physics, chemistry, biology, earth sciences and applied science and technology

- procedural-knowledge: knowledge of the application of scientific methods and procedures, i.e. knowledge of test design, measurement errors and control techniques

- epistemic-knowledge: knowledge of the procedures and methods for acquiring knowledge and substantiating that knowledge through data, e.g. evaluating the informative value of results obtained by means of experiments

• Scientific processes - explaining-phenomena-scientifically:

identifying, describing and evaluating explanations for a range of natural and technological phenomena, e.g. the day-night cycle

- evaluating-and-designing-scientific-enquiries: describing and evaluating scientific enquiries and finding possible ways of dealing scientifically with questions, e.g. identifying scientific problems

- interpreting-data-and-evidence-scientifically: analysing and evaluating data, assertions and arguments in a multiplicity of presentations, and drawing appropriate scientific conclusions

• SituationsKnowledge and competencies must, within the framework of PISA, be applied with a view to solving realistic scientific problems (e.g. climate change, energy supply, natural disasters, etc.). The key areas are health- and-disease, natural- resources, environmental-quality, hazards- and- limits of science and technology.

9

Collaborative Problem-Solving In an increasingly global and networked world, more and more tasks are nowadays carried out in teams. It follows that, in the 21st century, teamwork and communication are core competencies.

One of the important factors for the success of teams (e.g. groups of students working on projects, families, companies, organisations, state bodies, etc.) is the fact that individual team members must possess the requisite competencies for efficient collaboration within a group and must be in a position to apply their problem-solving skills in these social situations.

Consequently, PISA 2015 encompasses those problem-solving competencies which are needed when working on projects in schools, in studies and training and, later on, in the workplace. These include skills in areas such as communicating, conflict resolution, team organisation, consensus-building and monitoring the progress of a given project.

The PISA framework distinguishes three core competencies in the field of collaborative problem-solving:

- developing-a-shared-understanding,-i.e. identifying matters of common knowledge, bringing together different points of view amongst group members and developing a common vision of a problem at issue

- taking-appropriate-action-to-solve-a-problem, i.e. recognising which activities lead to the problem being solved and taking the necessary steps

- organising-a-team, i.e. appraising one‘s own role and that of one‘s fellow team members, supervising and where necessary adapting the way in which the group is organised, and providing feedback to other team members

The assignments are designed in such a way that a specific problem is presented (e.g. planning a school party) and the student works together on solving the problem with one or two virtual communication partners.

10

Target group: Who participates in PISA? The target group for the PISA 2015 Main Survey is defined as follows:

- all students born between 1 January 1999 to 31 December 1999, and

- who are in classes 7 and higher.

International sample

In order to obtain representative results, the OECD prescribes the testing of at least 5,400 students from 150 schools in each country. The random sampling is done by means of a two-stage procedure: in the first stage, at least 150 schools out of all the schools in a given country are selected as being representative of the different types of school. Next, 42 students from a participating school are chosen at random.

National feature: census

In small countries such as Luxembourg, as well as Iceland and Malta, which do not have the requisite number of schools and students, different rules apply. In those countries, all youngsters in the specified age group, from all the schools in the country, take part in the PISA test.

Thus, in Luxembourg, the main PISA 2015 test is taking place in all the 44 secondary schools, whether public, private or international. In total, some 5,400 students will be tested.

PISA test procedure

End of the test sessionCollection of test material

Distribution of Parent QuestionnaireStudents dismissed

First part of the cognitive test (60 Min.)

Short break (approx. 5 min)

Second part of the cognitive test (60 Min.)

PISA – break (15 min)

Student Questionnaire (approx. 45 min)

Arrival of the students at the start of the school day

Login and entry of passwordGeneral introduction to the test

ParentsParent Questionnaire

(approx. 35 min)

School PrincipalSchool Questionnaire

(approx. 60 min)

11

Implementation: How is the PISA test carried out?The study will be carried out in all Luxembourg schools between 21 April and 22 May 2015. The precise date will be fixed by the school in agreement with the Ministry of Education, Children and Youth. The test will take up one school morning – approximately 3.5 hours. Each school has nominated a School Co-ordinator, who will be responsible for the organisation of the test within the school. The test will take place in groups of approximately 20 students. External Test Administrators will carry out the test and monitor it to ensure that it proceeds in the correct way.

In the first part of the test, the young people work for two hours on assignments in the fields of science, reading, mathematics and collaborative problem-solving. In this context, different formats are used in relation to the assignments and the answers to be provided:

- Multiple-choice questions - Questions with an open answer format:

the student is required to formulate his/her own answer

- Interactive tasks: carrying out an experiment with the help of a simulation

- Drag-and-drop: putting the parts of the answer in the right order

- Drop-down menus: selecting the correct answer element

In the second part of the test, the youngsters answer a Student- Questionnaire, which takes approximately 45 minutes. This involves, inter alia, their personal and social background, their attitude to science and their learning motivation, their perception of school and teaching, and their experience in handling information and communications technologies.

The school administrative bodies, for their part, receive a School-Questionnaire, relating to, inter alia, the demographic characteristics of the school, its resources, the quality of the school‘s learning environment and collaboration with the parents.

In addition, the parents of the participating students receive a Parent- Questionnaire, in which they are requested to provide details concerning their child‘s previous schooling, their personal attitude to science and their perception of their child‘s school.

The information supplied by the answers given in the questionnaires is highly important: through those data, significant characteristics are gathered regarding the student and his/her school and family home, making it possible to undertake a more in-depth analysis of the data.

12

PISA sample items: Science

Solution Question 3: Answers that indicate that volcanic emissions (e.g. volcanic ash, sulphur dioxide) reflect or absorb solar

radiation (resp. that they block solar radiation from reaching the surface of the Earth).

Solution Question 4: C

Further sample items: http://www.men.public.lu/fr/systeme-educatif/qualite-scolaire/index.html

13

Quality managementThe international consortium lays down strict guidelines for the carrying-out and monitoring of the study. Those guidelines concern in particular:

- development of the test instruments - translations and national adaptations - random sampling - test implementation - coding - data analysis

The PISA quality standards are documented, first, in comprehensive procedure manuals and, second, in planning documents for the implementation. Compliance with the guidelines is checked several times by the international consortium in relation to each country. This ensures that the implementation of the study in all areas is of a very high quality.

Data protectionThe PISA data are evaluated on an anonymous basis: each student is given a code, and his/her name does not feature in the processing of the test results and questionnaires. The data are processed solely using codes; in other words, no result is linked to any name.

In addition, the test results and personal details in the questionnaire are not evaluated in relation to individual students; instead, all results are combined in terms of sizeable groups of students (e.g. according to country, gender or type of school).

14

Commissioning of, and responsibility for, the PISA StudyThe PISA Study is commissioned by the governments of the participating countries. Each country delegates a representative of its government to sit on the PISA Governing Board (PGB), a joint committee. Under the aegis of the OECD, that body is responsible for setting the programme‘s educational policy objectives, and for adherence to the fixed priorities throughout its implementation. The PGB adopts all key decisions.

International consortium

An international consortium, led by the Educational Testing Service (ETS), is in charge of ensuring scientific quality and overseeing the implementation of the PISA Study; this consists of several international organisations and research institutions within the education sector. As well as the ETS, its members include: ACER – Australian Council for Educational Research (Australia), cApStAn – Linguistic Quality Control (Belgium), LIST - Luxembourg Institute of Science and Technology (Luxembourg), DIPF – Deutsches Institut für Internationale Pädagogische Forschung (Germany), Pearson (UK), Statistics Canada (Canada) and Westat (USA).

Organisation at the national level

In each country, a national PISA centre is set up; this maintains close contact with the international consortium. In Luxembourg, the national PISA project centre is established within the Agence- pour- le- développement- de-la- qualité- scolaire (ADQS), a department of the Service-de-Coordination-de-la-Recherche-de-l‘Innovation- pédagogiques- et- technologiques-(SCRIPT) within the Ministry of Education, Children and Youth (MENJE). This is responsible for all phases in the implementation of the project (preparation, execution, coding and data management) and for adherence to the quality standards in Luxembourg. The national PISA results are evaluated, and the national PISA report is drawn up, in collaboration with the Luxembourg-Centre-for-Educational-Testing (LUCET), which forms part of the University of Luxembourg.

Ministry for Education, Children and Youth (MENJE) Service-de-Coordination-de-la-Recherche-et-de-l’Innovation-pédagogiques-et-technologiques-(SCRIPT)-

Agence-pour-le-développement-de-la-qualité-scolaire (ADQS) Jos Bertemes (Director of the SCRIPT), Amina Kafai (Head of ADQS)

PISA Project Manager: Bettina Boehm Project Assistants: Astrid Krug, Christiane Mergen

29, rue Aldringen L-2926 Luxembourg Tel. : (+352) 247-85211

Email: bettina.boehm@men.lu

www.oecd.org/pisa/ or www.men.public.lu