final evaluation of incas and the annual pupil profile report

October 2008

Primary Pupil ProfileFinal Evaluation of InCAS and the Annual Pupil Profile Report

Prepared by Research & Statistics KW2031/0809

1

Final Evaluation Report on the implementation of

the InCAS computer-based assessments and the

Annual Pupil Profile Report in Primary Schools

during the transitional year 2007/2008

Prepared by:

CCEA Research & Statistics Unit

Copyright 2008 CCEA. All rights reserved.

This report is available at: www.nicurriculum.org.uk

This evaluation was conducted from May 2007 through to August 2008. For further

information about this evaluation please contact Karen Wiltshire:

CCEA Research & Statistics Unit

29 Clarendon Road

Clarendon Dock, Belfast

BT1 3BG, or alternatively email

[email protected]


2

Scope of the Report This report is the final of three evaluation reports on the implementation of the revised

assessment and reporting arrangements in Primary schools in Northern Ireland. All

reports are available at www.nicurriculum.org.uk.

The first two reports, released in December 2007 and May 2008, were designed to collect

Year 5 teachers’ views on the;

• Training provided for the InCAS computer-based assessments;

• Administration of InCAS;

• Feedback produced from InCAS; and the

• Usefulness of the InCAS feedback for teachers, parents and pupils.

Year 5 pupils’ opinions on the InCAS assessments were also collected via a questionnaire

that was distributed to all Primary schools in November 2007.

This final report provides a robust evaluation of the views and opinions of all stakeholders

involved in this year’s implementation of

1 InCAS

• Principals • Year 5 Teachers • Year 5 pupils • Parents

2 Annual Pupil Profile Report

• Principals • Year 1 and 5 Teachers • Parents with children in Year 1 and 5 • Year 5 pupils

Such stakeholders have provided CCEA with constructive information on all aspects of

InCAS assessments and the Annual Pupil Profile Report which will be used to inform

planning, development and the next phase of implementation.

A full breakdown of this year’s consultation for this academic year (2007-08) can be found

in the Annex of this report.


3

Acknowledgements

CCEA would like to thank all Principals, teachers, pupils and parents who contributed to

this evaluation. We are extremely grateful to all participants for taking the time to share

their experiences of the InCAS computer-based assessments and the Annual Pupil Profile

Report. This cooperation and involvement allows CCEA to have a detailed and collective

understanding of the implementation of the new assessment and reporting arrangements

in Primary schools in Northern Ireland.

This consultation has provided schools, parents and pupils with an opportunity to inform

future planning and development of these new arrangements.


4

Contents Section Page

1.0 Executive Summary (Contains Conclusion & Recommendations)

5

2.0 Introduction 14 3.0 Method 16 4.0 Results

4.1 Principal 17

4.2 Year 5 Teacher 40

4.3 Year 5 Parent 57

4.4 Year 5 Pupil 70

4.5 Year 1 Teacher 74

4.6 Year 1 Parent 83

5.0 Conclusion 89 6.0 Recommendations 93 7.0 Annex 95


5

1.0 Executive Summary This is the final evaluation report on the first year of the implementation of the InCAS

computer-based assessments in Year 5 and the revised reporting arrangements in Years

1 and 5. The purpose of this evaluation is to provide all stakeholders (Principals, teachers,

pupils and parents) involved in this implementation with an opportunity to present their

views and opinions of these new assessment and reporting arrangements. Information

collected from this evaluation will be used to inform the future planning and development.

Summary of Findings

InCAS Computer-based Assessments

Overall feedback on the InCAS computer-based assessments has been largely positive.

The majority of teachers who participated in evaluations in December 2007 believed pupils

found InCAS user-friendly (94.5%, n = 517). 91% of teachers also deemed the feedback

consistent with their professional judgement (n = 477).

Consultation with stakeholders (Principals, teachers, pupils and parents) in June 2008 has

also been encouraging. Over 70% of Principals (71.7%, n = 220) and Year 5 teachers

(70.9%, n = 251) felt the feedback from the InCAS assessments informed teacher

planning. Furthermore, although not a statutory requirement this transition year a sizeable

number of schools (>24%, n=219) opted to share InCAS feedback with parents. 76%

(n = 136) of Principals and 58% (n = 201) of teachers from such schools deemed this

feedback to be meaningful to parents. 57.5% (n = 1097) of parents who received their

child’s results to InCAS believed it enhanced the information shared at the parents

meetings.

Each phase of evaluation has highlighted a number of challenges schools have

encountered in implementing InCAS assessments for the first time. Many of such

difficulties were a result of teachers using the software for the first time (e.g. difficulties

setting up). However, stakeholders have also identified a number of areas of improvement

within the assessment tool (e.g. volume control); all of which have been shared with

CEM/C2K and modifications made for the 2008/09 academic year.


6

Feedback from the Principal and teacher workshops and parent focus groups raised a

number of issues that had not been expressed as strongly within earlier evaluations.

Despite the fact that the Department of Education funded additional laptops within the

Primary sector a large number of Principals still believe that Primary Schools do not have

the capacity to deliver computer-based assessments, particularly those with composite

classes or those without computer suites. Teachers’ feedback on the reliability of InCAS

has been very high within previous phases of evaluation. However, within all workshops

and focus groups facilitated in June 2008 a sizeable number of Principals, teachers and

parents questioned the reliability and validity of InCAS feedback, contradicting responses

made at the time of delivery.

Capita Profiles 7 Despite the majority of Principals and teachers (>92%) rating the Profiles 7 training highly,

over 40% of both Principals and teachers indicated that it was not adequate preparation

for using the software. Principals were asked by C2K to nominate two representatives

from their school to attend the training. It was evident from the teacher and Principal

workshops that a variety of staff attended, many of whom were not the Year 1 or 5

teachers who were required to complete the Annual Pupil Profile Reports. A large number

of Principals and teachers commented that sub-cover should have been provided for this

training and all teachers planning to use the Profiles 7 to complete reports should have

been allocated a place at the training.

Over 70% of Principals and teachers indicated that their school used the Profiles 7

software to produce Annual Pupil Profile Reports. Over 62% of such Principals and

teachers experienced difficulties with this software. Many respondents felt the software

was not user-friendly and was time consuming. Furthermore, the majority of Principals and

teachers (>68%) deemed the Profiles 7 software to be less manageable than other

methods they have used to compile school reports, such as word documents. Participants

at the Principal and teacher workshops expressed a high level of dissatisfaction with the

Profiles 7 software, stating that it was not user-friendly and had significantly increased the

amount of time teachers take to produce pupil reports.


7

Comment Banks Despite the majority of Principals and teachers indicating that the comment banks were

useful, over 91% of both respondents felt they could be improved upon. A large number of

both Principals and teachers stated at the workshops that comments need to be provided

for all areas of the curriculum. Principals and teachers also stated that they require

exemplary comments for areas in which pupils need to develop.

Annual Pupil Profile Reports

Over 70% of Principals and teachers rated the format of the Annual Pupil Profile Reports

as either excellent or good. The majority of parents with children in Year 1

(96.6%, n = 3467) and Year 5 (86.8%, n = 3007) also rated the format of the Annual Pupil

Profile Report highly.

However, at the workshops and focus groups a large number of attendees (Principals,

teachers and parents) did not rate the content of the Annual Pupil Profile Reports highly.

Such stakeholders felt that grades/marks should have been included within the Annual

Pupil Profile Reports. Furthermore, a large number of Principals, teachers and parents

described the Annual Pupil Profile Reports as “hyper-positive”, bland and impersonal. A

large number of parents with children in Year 1 and 5 stated that it was evident teachers

had used comment banks to write their child’s report.

Attendees at the Principal and teacher workshops suggested that communications

regarding the statutory requirements of the Annual Pupil Profile Report have not been

communicated effectively from the Government agencies. Furthermore, findings from the

parent focus groups indicate that the majority of parents had no knowledge of the purpose

of InCAS computer-based assessments or the Annual Pupil Profile Report.


8

Conclusion This evaluation has identified a number of issues within the Primary Education sector.

Many Primary schools felt pressured from a combination of demands from Government

agencies and parents. In some cases this is exacerbated by the perception that the new

assessment and reporting arrangements are being introduced at a demanding pace.

Feedback collected from earlier phases of evaluation on InCAS has been very positive.

The majority of teachers who participated within previous evaluations felt confident

interpreting feedback and deemed it to be consistent with their professional judgement.

Feedback collected from all stakeholders in June 2008 has been somewhat more

negative. However, problems teachers encountered completing the Annual Pupil Profile

Reports is likely to have escalated their criticisms and concerns about InCAS.

Furthermore the deployment and use of InCAS took place at the same time as a major

transition to a newly contracted C2K managed service in all primary schools. The majority

of concerns raised have been identified in previous evaluations and modifications made for

the 2008/09 academic year. However, one manageability aspect of InCAS that has not

been highlighted as strongly within previous research was that many respondents do not

believe that Primary Schools have the computer capacity to manage the delivery of InCAS

as it involves more year groups. 8,000 additional laptops have been installed in primary

schools across Northern Ireland. These together with the 5,000 assessment laptops

distributed in 2007 mean there are now over 28,000 devices available for use with InCAS.

However, despite this investment respondents stated that teachers with composite classes

and schools without computer suites have found delivering the InCAS assessments

extremely difficult. Many Principals felt that further ICT investment is required for such

schools and teachers need further support to administer InCAS.

Feedback from this evaluation suggests that some Principals and teachers are unclear

about the nature and status of the Annual Pupil Profile Report. The most widely held

misconceptions were that schools were required to use the Profiles 7 software to complete

reports and that the use of comments in the bank were mandatory. As a consequence

some teachers reported high levels of stress. In some cases comments were used well,

with parents reporting a positive reaction to the information they received about their

children. However less careful use of comments from the comment banks resulted in

reports which were considered by parents as overly positive, general and impersonal.


9

Some schools report receiving different messages about requirements of revised

assessment and reporting arrangements from education stakeholders.

The implementation of new electronic assessment and reporting arrangements represents

significant change for many Primary Schools in Northern Ireland. CCEA provided schools

with a variety of support and guidance in the form of; training materials, telephone

helpdesk, a dedicated section on the Northern Ireland Curriculum website, online tutorials

and an InCAS pupil demonstration. Feedback from this year’s consultation indicates that

some schools did not use the materials or follow published instructions. Some teachers

within the teacher focus groups felt that they did not have time to make use of the

resources provided.

Both Principals and teachers expressed high levels of dissatisfaction with the Profiles 7

report writing software. Feedback indicates that teachers and Principals believe this

software has significantly increased the amount of time teachers take to produce Annual

Pupil Profile Reports whilst reducing the quality of information. The majority of Principals

and teachers deemed this software to be extremely time consuming to use, under-

developed and have questioned the degree of piloting prior to this year’s implementation.

Only one participant at the Principal and teacher workshops indicated that they intend to

use this software next year.

This evaluation indicates that parents are not entirely satisfied with the new assessment

and reporting arrangements. However, it is important to note that parent’s feelings have

been heightened by the current uncertainty over transfer arrangements and their lack of

knowledge of InCAS and the revised Annual Report. Some parents indicated that they did

not know that their children were being assessed at the beginning of the school year.

A large number of parents, Principals and teachers expressed concerns that INCAS

assessment outcomes may be used as a method of selection at a future date.

A large number of parents were dissatisfied with the format and content of their child’s

Annual Pupil Profile Report. Although many parents recognised the work required by

teachers to produce this new report there were a number of aspects they criticised.

Parents’ main criticism was that it did not contain grades/marks or a comparison with their

child’s peers which they deemed to be an essential component of any school report.


10

Parents described the reports as “impersonal, vague, over generalised” and as a result

were unclear about how their child was progressing. There was also widespread concern

among parents on how ‘hyper-positive’ the reports were. Many parents felt that the report

did not provide them with a true account of their child’s progress. One parent stated:

“[The Annual Pupil Profile Report] was written in politically correct language which

told me nothing. Please encourage teachers to write what they feel and not what

they think ought to be written. More teacher freedom please”.

Some teachers felt uneasy about writing what might be considered less than positive

comments in the reports.

Teachers also have requested advice on how to share InCAS feedback with parents and

direction on comments they may include within the Focus for Development section of the

Annual Pupil Profile Report.

This report has provided a comprehensive overview of this year’s implementation of the

new assessment and reporting arrangements across Primary Schools in Northern Ireland.

Findings from this evaluation show that embedding such new arrangements has required a

large degree of additional work from teachers and Principals. Schools require further

assistance to ensure that they feel adequately supported and empowered to implement

these new arrangements.


11

Limitations

Parents, Principals and teachers have indicated that they are less than satisfied about the

uncertainty about new transfer arrangements. This has impacted on findings obtained

within this evaluation as many were concerned that either InCAS or the Annual Pupil

Profile Report could or would be used to transfer pupils to post primary schools.

A second limitation of the research is the poor response rate to the Principal and teacher

questionnaires. It may be that those most dissatisfied with InCAS and the Annual Pupil

Profile Report completed and returned questionnaires to CCEA. This may have negatively

skewed results but is a common limitation of research.


12

Recommendations

Before commencing the second roll out of InCAS and the Annual Pupil Profile Report there

are a number of issues documented within this report that should be addressed.

Communication This evaluation suggests that:

• Communications regarding the new assessment and reporting arrangements should be

re-evaluated. All communications on these arrangements should be shared between

all partners (CCEA, DE, C2K and CEM).

• At the start of the next school year CCEA should inform schools which elements of the

Annual Pupil Profile Reports are statutory – it is imperative that such information is

specific to each Year Group.

• CCEA should provide teachers with an ‘at a glance guide’ outlining what elements of

InCAS and the Annual Pupil Profile Report are statutory and flexible.

• CCEA should communicate with parents on the new assessment and reporting

arrangements. InCAS This evaluation suggests that:

• Further work should be carried out to determine whether or not additional computer

capacity may be required to support the use of InCAS within Primary schools.

• CCEA should continue to empower schools to embrace InCAS and develop expertise

and confidence in both administering and analysing feedback produced from InCAS.

CCEA should also reassure schools of the reliability and validity of InCAS.

• Teachers would welcome advice on how to share InCAS feedback with parents.

• Further adaptations to InCAS are needed to meet the needs of pupils with Special

Educational Needs and those with English as an Additional Language.


13

Recommendations continued Profiles 7 Software

This evaluation suggests that:

• Consideration should be given to whether Profiles 7 is the most appropriate report

writing software or if an alternative should be sought and trialled. Alternatively, Profiles

7 should be developed as a matter of urgency to ensure it better meets the needs of

schools completing the Annual Pupil Profile Report.

• All teachers who plan to use the Profiles 7 report writing software should be provided

with appropriate training.

Annual Pupil Profile Report


• Schools should be advised explicitly what the purpose of the Annual Pupil Profile

Reports is and in particular its role (if any) in transfer arrangements.

• Schools require development time to tailor comment banks and ensure all Annual Pupil

Profile Reports are personalised to individual pupils.

• Principals, teachers, pupils and parents would like to see marks/grades to be included

within the Annual Pupil Profile Reports.

• Consideration should be given to moving the ‘Focus for Development’ section towards

the end of the Annual Pupil Profile Report.


14

2.0 Introduction

The implementation of InCAS computer-based assessments and the Annual Pupil Profile

Report commenced at the start of the 2007 academic year. This involves teachers

assessing and reporting pupil progress and achievements. It will be used to support and

inform decisions throughout a pupil’s time at school, providing information for;

• Parents - about helping their child’s learning at home;

• Teachers - as they plan learning and teaching; and

• Pupils - about their achievements and how to improve.

InCAS Computer-based Assessments

InCAS statutory computer-based assessments are completed during the autumn term.

InCAS was developed by the Curriculum, Evaluation and Management Centre at Durham

University and is an adaptive computer-based assessment. Computer-adaptive

assessments are tailored to match a pupil’s ability. The assessment program generates

items depending on the pupil’s response to a previous number of items. The difficulty of

subsequent assessment items presented to the pupil depends upon their responses as

they proceed through the assessment. The adaptability of this assessment method means

that learners are not faced with the de-motivating experience of being unable to answer

many of the items (which is often the case with pencil and paper tests).


15

The InCAS computer-based assessment consists of 11 modules. By the end of the

autumn term 2007, all Primary Schools in Northern Ireland were required to administer two

statutory assessments - Reading and General Math’s to Year 5 pupils to produce age

equivalent scores. The Reading age is achieved on the completion of the Word

Recognition, Word Decoding and Comprehension modules. The General Math’s module

generates the general Math’s Age. Teachers can make full use of the other seven

modules within the InCAS assessment tool (Spelling, Mental Arithmetic, Picture

Vocabulary, Non-Verbal Ability, Attitude to Math’s, Attitude to Reading and Attitude to

School).

The Department of Education determined 2007/2008 a transition year for the new

assessment and reporting arrangements. During this year outcomes from InCAS did not

have to be formally reported to parents. However, during 2008/2009 schools are required

to share the outcomes from the InCAS assessments at a parents meeting in the autumn

term for Years 5 and 6 and by 2009/10 this will be statutory for Years 4-7.

Annual Pupil Profile Reports

This year, revised annual reporting arrangements were introduced for pupils in Years 1

and 5. The purpose of the Annual Pupil Profile Report is to promote consistency in

reporting arrangements across schools in Northern Ireland.

CCEA’s Research & Statistics Unit is responsible for the evaluation of the implementation

of InCAS and the Annual Pupil Profile Reports. The purpose of this evaluation was to

provide all stakeholders (Principals, teachers, pupils and parents) involved in this

implementation with an opportunity to present their views and opinions of these new

assessment and reporting arrangements. Information collected from this evaluation will be

used to inform the future planning and development of the new arrangements.


16

3.0 Method

The following section includes a summary of the methodology used to evaluate the final

phase of the implementation of the InCAS computer-based assessments and the Annual

Pupil Profile Report. A full breakdown of this year’s (2007-08) consultation can be found in

the Annex of this report.

For this final phase of evaluation to be as inclusive as possible, information was collected

from all stakeholders involved in the implementation of the InCAS computer-based

assessments and the Annual Pupil Profile Report. This encompassed Principals,

teachers, pupils and parents. The overall aim of this evaluation was to conduct a robust

and reliable evaluation of these new assessment and reporting arrangements which can

be used to inform and advise upon future implementation years.

Summary of Participants & Research Tools

Questionnaires were the main method of collecting data from stakeholders (Principals,

teachers, pupils and parents). However, a range of qualitative methods were also

employed to further explore and reinforce information collected from the Principal, Year 5

teacher and Year 5 parent questionnaires. In total CCEA’s Research and Statistics Unit

facilitated eight Principal workshops, ten Year 5 teacher workshops and 13 parent focus

groups. Qualitative consultation will be undertaken with Foundation Stage teachers next

year (June 2009).

Response rates to questionnaires and attendance rates at the workshops and focus

groups are detailed below.

Table 1 Response Rates to Questionnaire Attendance rate at Workshop/Focus Gp

Principal 337 111

Year 1 Teacher 281 N/A

Year 5 Teacher 378 201

Year 1 Parent 3,631 N/A

Year 5 Parent 3,586 145

Year 5 Pupil 5,120 N/A


17

4.0 Results

4.1 Results collected from Principal Questionnaires & Workshops

This section presents the results for each question from the Principal questionnaire.

Results are based on completed questionnaires from 339 respondents. Percentages

relate to the number of Principals who answered each question, which may be fewer than

the overall total of 339. A number of Principals reiterated comments throughout their

questionnaire; as a result some comments are repeated throughout this section of the

report.

Qualitative information obtained from eight Principal workshops attended by 111 Principals

have been analysed alongside the feedback collected from the questionnaire. The

purpose of the workshops was to collect in-depth information to reinforce and further

explore that collected from the questionnaire.


18

Manageability of the InCAS computer-based assessments

Figure 1: How Manageable were the InCAS assessments to complete? Figure 1 illustrates that the

majority of Principals felt that

the InCAS assessments were

manageable to complete

(89.1%, n = 287).

However, despite such positive findings, 64.9% of respondents (n = 204) encountered

challenges administering InCAS. The most common challenges Principals commented on

are summarised below.

• Administration/set up was time consuming (33 comments)

• Experienced general technical difficulties (28 comments)

• Difficulties providing adequate supervision/sub-cover (23 comments)

• Availability/allocation of computers (22 comments)

• Computers crashing (20 comments)

• Problems accessing the software (16 comments)

• Difficulties adjusting volume (15 comments)

• Problems with passwords (10 comments)

• Managing the rest of a class while the assessments were administered

(10 comments)

8.7

80.4

9.61.2

0

20

40

60

80

100

% R

espo

nden

ts

Verymanageable

Manageable Notmanageable

Not at allmanageable


19

Workshop findings

Feedback from the Principal workshops reinforced all of the technical and user

difficulties Principals noted within the questionnaire. A large number of participants

within all of the Principal workshops believed that unlike the Post Primary sector,

Primary Schools do not have the computer capacity or ICT support for the

completion of electronic assessments. Many participants stated that teachers with

composite classes and schools without computer suites had particular difficulties

administering InCAS.

An additional 8,000 laptops have been installed in Primary Schools across Northern

Ireland. These together with the 5,000 assessment laptops distributed in 2007 mean

there are now over 28,000 devices available for use with InCAS. However,

attendees at the Principal workshops felt that despite schools receiving additional

laptops they still did not have a sufficient number of workstations to administer

InCAS. Many Principals highlighted that the laptops posed a number of problems for

schools. Principals within one particular workshop highlighted that setting up the

laptops for InCAS was extremely time consuming especially temporary computer

suites made up of a number of laptops. Such Principals stated that this created a

number of accommodation and health and safety issues for schools.

The majority of Principals within all of the Principal workshops believed that schools

had difficulties implementing InCAS. Principals also expressed concerns about the

impact on teaching time as a result of administering InCAS. One Principal stated

that InCAS “should have bedded into the school year and been a natural part of the

school year, but the fact is this would not have worked without the dedicated work of

Principals and teachers who have sacrificed all over the place to administer this.

[This year] other pupils have done without so that Year 5 pupils could complete

InCAS”


20

Principals were asked in the questionnaire to document any managerial decisions made

for the completion of the InCAS assessment with Year 5 pupils. Managerial decisions that

Principals made are listed below.

• Time-tabling (78%, n = 263)

• Allocation of resources (68%, n = 228)

• Deployment of staff (48%, n = 163)

• Substitute cover / supervision for classes (15%, n = 51) Just over 70% of respondents (70.7%, n = 205) indicated that they intend to put

procedures in place to support the administration of InCAS next year. The two most

common procedures amongst Principals include:

(1) Allocation of sub cover/in-house supervision (n = 46); and

(2) Timetabling of computer suites (n = 45). Other procedures are summarised below.

• Utilising ICT skills of staff within school (15 comments)

• Allocate time for teachers to administer and prepare (11 comments)

• Plan to introduce a computer suite (10 comments)

• Purchase more hardware (7 comments)

• Remove pupils in composite classes not completing InCAS (6 comments)

• Ensure tests are conducted in a quiet area (6 comments)

• Plan meetings with all staff involved (5 comments)

A sizeable number of Principals (n = 34) stated that it is impossible for them to set up

procedures without additional funding for resources and staff release.


21

Workshop findings The majority of participants within all eight Principal workshops unanimously agreed that

schools require money to either release teachers to administer InCAS or to employ

someone to supervise pupils completing this electronic assessment.

A large number of Principals expressed concern at the prospect of two year groups

completing InCAS next year. The majority of Principals within two workshops stated that

unless InCAS is improved upon and schools are provided with the necessary support to

administer InCAS they will not permit their teachers to administer this assessment next

year.

Sharing InCAS Feedback with parents

Although not a statutory requirement this year 54.4% of Principals (n = 178) indicated that

their school shared InCAS feedback with parents.

Figure 2: How were results shared with parents?

Figure 2 illustrates that the majority

of schools (94.4%, n = 170) shared

InCAS feedback with parents

verbally.

94.4

2.2 2.8 0.6

0

20

40

60

80

100

% R

espo

nden

ts

Verbally As a chart As a table Otherformat


22

Respondents were asked if they felt the assessment feedback was meaningful to parents,

76% of Principals felt it was (n = 136). Such Principals felt that the feedback was useful to

parents for the following reasons.

• Highlights children’s strengths and weaknesses (9 comments)

• The age equivalent score was easy for parents to understand (6 comments)

• Visual representation was useful i.e. graphs and charts (5 comments)

• Reinforced teachers’ professional judgement (3 comments)

• Useful when used alongside NFER results (3 comments)

Principals who felt InCAS feedback was not meaningful to parents provided the following

reasons for their answer.

• The feedback contained too much jargon (8 comments)

• The feedback was too harsh/blunt for SEN pupils (5 comments)

• InCAS results were unclear to parents (4 comments)

Principals who chose not to share InCAS feedback with parents were asked to outline their

reasons for this decision, these have been summarised below.

• Sharing feedback with parents was not statutory this year (42 comments)

• Felt the results were unreliable (25 comments)

• Teachers were not adequately trained to share feedback with parents (19 comments)

• Teachers wanted to familiarise themselves with feedback this transition year

(16 comments)

• Did not believe parents would understand results (12 comments)

• Did not complete InCAS in time for parent interviews (7 comments)

• Anticipated that feedback would be time consuming to explain (6 comments)


23

Workshop findings

Information collected from all of the Principal workshops regarding the suitability of InCAS

feedback for parents was generally very negative. Many Principals stated that the

meaningfulness of this information varied greatly from parent to parent. The majority of

Principals stated that teachers were not confident interpreting the results and therefore

were reluctant to share results with parents. Many Principals expressed concerns about

the reliability of InCAS in comparison with existing assessments used within school. The

majority of Principals from three of the eight Principal workshops felt that the InCAS results

did not match with standardised test results.

A small number of Principals within all of the workshops believed that the feedback from

InCAS was very “blunt” for pupils with Special Educational Needs. One Principal stated

that when sharing feedback with parents whose child has Special Educational Needs “you

are reporting back that the child is so many years behind – what is that doing to the child

but giving them an educational kicking”.

Within all of the Principal workshops a small number of participants expressed concern

over the potential use of InCAS. Such Principals were dubious that InCAS was only to

inform teacher planning and learning. Many felt that InCAS results could and would be

used for benchmarking against schools and for school inspections – which one Principal

stated will “distort the whole rationale for InCAS as it could be used as a stick to beat

schools”.

From 2008/2009, schools are required to report the age-equivalent scores for Reading and

General Math’s to parents with children in Years 5 and 6. Principals indicated that next

year they plan to share InCAS feedback with parents using either a chart (46.1%, n = 118)

or a table (39.5%, n = 101).


24

Was the InCAS feedback informative to teachers? Principals were asked if they felt the feedback from InCAS informed teacher planning.

Whilst 71.7% of Principals (n = 220) believed the feedback from InCAS informed teacher

planning, 28.3% felt it did not. A small number of Principals who believed the feedback

from InCAS informed teacher planning, commented that it highlighted pupils’ strengths and

weaknesses (n = 29) and informed planning for both individual pupils and the whole class

(n = 16). Comments from Principals who felt InCAS feedback did not aid teacher planning


• Provided no additional information (32 comments)

• Teachers do not have time to utilise the results in planning (10 comments)

• Unexpected results suggest that results may not be reliable (8 comments)

• Results from the General Maths module need to be broken down further (6 comments)

Was the InCAS feedback informative to parents?

58.9% of Principals (n = 162) felt that the feedback from InCAS did not inform parents of

how they can support their child’s learning at home. Such Principals felt that the feedback;

• Required teacher guidance and explanation (16 comments);

• Is difficult for some parents to understand (15 comments);

• Did not provide any new information to parents (14 comments); and

• Requires more detail on how parents can help their child improve (5 comments).


25

Was the InCAS feedback informative to pupils?

Over 50% of Principals felt the results from InCAS did not inform pupils of their

achievements (n = 136) or how they can improve (n = 126). A small number of Principals

(n = 16) commented that pupils require teacher guidance for this information to be

meaningful.

Workshop findings

At all of the Principal workshops Principals’ views on the usefulness of the InCAS

feedback varied greatly. Those who deemed the feedback to be useful particularly liked

the instant feedback and felt it provided effective visual information for teachers and

parents.

Those who did not find the feedback useful believed it was non-specific, too general, not

diagnostic, limited and did not inform teachers or parents of remedial action they should

take.

Profiles 7 Report Writing Training

Two members of staff from every Primary School in Northern Ireland were offered training

in the use of Profiles 7 report writing software by C2K. The majority of Principals

(92.8%, n = 308) who completed the Principal questionnaire indicated that someone from

their school attended this training. A small number of Principals who indicated that no one

from their school attended the Profiles 7 training made the following comments.

• Did not attend training as sub cover was not provided (8 comments)

• Not offered this training (4 comments)

Nearly 70% of Principals (69.4%, n = 218) revealed that they attended the report writing

training themselves.


26

Figure 3: How useful was the Profiles 7 training?

Principals who attended the

Profiles 7 report writing training

offered by C2K were asked to rate

how useful they found it. Figure 3

indicates that the majority of

respondents rated this training as

either very useful or useful (94.4%,

n = 202).

Despite the majority of Principals rating this training as useful, 42.5% (n = 111) of

respondents felt it was not adequate preparation for using the software. Such Principals

stated that;

• All staff need training as it is too difficult for individuals to cascade training to other

teachers (34 comments);

• More training is required as the session was too intensive (28 comments);

• There was too much time between training and using the software (16 comments);

• Further training is required on creating and developing comment banks (9 comments);

and

• Technical difficulties with Profiles 7 were not mentioned at training (7 comments).

Workshop findings

The majority of participants within all of the Principal workshops indicated that someone

from their school attended the Profiles 7 training offered by C2K. The majority of

Principals felt that the C2K training was not sufficient preparation for using the software.

Principals stated that they were overloaded with information which was impossible for

them to remember when they were back in school. Many Principals also felt that there

was too great of a time delay between teachers attending the training and completing the

Annual Pupil Profile Reports. Principals stated that sub cover should have been provided

for this training and that teachers should not be expected to cascade such intensive

training to other teachers.

Online Tutorials

29.9

64.5

3.7 1.9

0

20

40

60

80

100

% R

espo

nden

ts

Very useful Useful Not useful Not at alluseful


27

CCEA provided online tutorials on the Northern Ireland Curriculum website to support

teachers’ use of the Profiles 7 report writing software. Only 15% of Principals (n = 48)

used these tutorials. A small number of respondents who did not use the tutorials

commented that they were not aware of them (n =19) or did not have time to use them (n =

13).

Nearly 90% of Principals who used the online tutorials deemed them to be useful

(89.6%, n = 43). However, only 43.8% of such Principals (n = 21) felt that the tutorials

alone were adequate preparation for using the software.

Workshop findings Findings from the Principal workshops reinforce those collected from the questionnaires.

All attendees at the Principal workshops were unaware of the online tutorials on the

Northern Ireland Curriculum website. However, many Principals stated that even if they

had been aware of them teachers did not have time to use them.

Using the Profiles 7 report writing software 76.7% of Principals (n = 247) indicated that their school used the Profiles 7 report writing

software to complete this year’s Annual Pupil Profile Reports. Principals were asked to

indicate if they thought this software was user-friendly; 65.8% of respondents (n = 169)

believed it was.

However, despite over 65% finding the software user-friendly, 62.3% of Principals

(n = 162) experienced difficulties using it. The most common difficulties Principals

commented on are summarised below.

• Sims crashed (34 comments)

• Encountered difficulties loading Sims to the system (16 comments)

• Unable to access Sims (10 comments)

• Class details were missing or inaccurate (10 comments)

• Editing was difficult (7 comments)


28

Workshop findings Principals within all of the workshops were very critical of the C2K training and time

required for teachers to familiarise themselves with the Profiles 7 software. Principals

were extremely angry and frustrated at the implementation of this software as they did not

feel it was user-friendly or fit for purpose. Three illustrated examples are given below.

“Too complex for a simple objective”

“Is this a cheap solution for the Department and an expensive solution for schools?”

“Costly for schools, teachers and also pupils – took their teachers away to do

something that didn’t give them enough back”

Figure 4: How manageable is Profiles 7 in comparison

to completing reports by other methods?

Figure 4 illustrates that 70.3% of

respondents (n = 196) found

Profiles 7 less manageable to

other methods they have used to

compile school reports.

Respondents were asked to comment on their response, these are summarised below.

• Software is extremely time consuming (85 comments)

• Time taken to complete reports affected work/life balance (77 comments)

• Completing reports in Microsoft Word is more manageable (13 comments)

• Schools require funding to provide non-contact time for teachers (11 comments)

• Reports are impersonal and less informative than previous reports (9 comments)

• Too many detailed areas to comment on (7 comments)

• Time is required for schools to write their own comments (7 comments)

16.5

70.3

13.3

0

20

40

60

80

100

% R

espo

nden

ts

More manageable Less manageable No difference


29

Workshop findings The majority of Principals within all workshops indicated that their school used the Profiles

7 software to complete Year 1 and 5 Annual Pupil Profile reports. Virtually all participants

stated that they did not find this software user-friendly. Many Principals felt that the

Profiles 7 software required a high level of ICT skills.

The majority of Principals were appalled at the amount of technical difficulties their staff

encountered when using Profiles 7. The most common problems commented upon by

participants were; editing and font changing (especially in preview mode) and SIMS

closing down at a specific time of the day. A small number of Principals were perplexed by

the fact that they did not receive a password for this software.

Virtually all of the attendees at the Principal workshops deemed the Profiles 7 software to

be less manageable than other methods they have used to complete school reports. A

large number of Principals felt that a word document would be much more user-friendly

and fit for purpose than the software provided by the Department of Education/C2K. Many

Principals deemed the Profiles 7 software to be under-developed and questioned the

degree of piloting prior to this year’s implementation.

Principals were particularly unhappy that schools were not provided with time

(sub-cover / closure days) to complete the new Annual Pupil Profile Reports and as a

result do not feel supported by CCEA or the Department of Education. A large number of

Principals stated that using the software has been extremely stressful for teachers and had

a detrimental effect on their work/life balance. Many Principals said that they had to keep

their school open until 9pm at night so that Year 1 and 5 teachers could complete reports

using the Profiles 7 software.

C2K provided dongles, which are similar to data keys to provide schools with access to

SIMS at home. It is planned that small Primary Schools will receive one dongle and larger

schools will be provided with two. Principals stated that two dongles per school is

insufficient and it is unfair that schools are expected to buy additional supplies. Many

Principals also stated that dongles are not the solution to writing school reports as

teachers should not be expected to compile reports in their own time. It is important to

note that these devices were never solely intended to be used for the completion of reports

at home.


30

Principals who indicated that their school did not use the Profiles 7 software to complete

this year’s Annual Pupil Profile Reports were asked to document their main reason for

making this decision. The most common were that they were unable to complete reports

at home (n = 32) or because they found the software too time consuming (n = 23). Other

reasons Principals documented are summarised below.

• Experienced technical difficulties with software (14 comments)

• Word template is easier to use (12 comments)

• Inadequate training (5 comments)

• No time to create comment banks (4 comments)

• Difficulties logging on (4 comments)

CCEA Comment Banks

93.5% of Principals (n = 301) indicated that they made use of the CCEA comment banks.

Respondents were asked to rate how useful they found the comment banks. Although

82.7% of respondents (n = 248) felt the comment banks were useful – the majority of

Principals (91.3%, n = 274) felt they could be improved upon. A large number of

respondents provided suggestions on how the comment banks could be improved, these

are listed below.

• Comments are required for all areas of the curriculum (73 comments)

• More comments needed on areas of weakness (47 comments)

• More specific/individual comments (35 comments)

• Greater variety to avoid repetitive reports from year to year (34 comments)

• Make it easier to add/adapt own comments (14 comments)

• Comments reflecting progression and levelling (13 comments)

• Less educational language/jargon (13 comments)

• Punctuation and grammar needs to be corrected within comment banks (7 comments)

• Make comments more appropriate to each Key Stage (7 comments)

• More general comments (6 comments)

• Insertion of <forename> <he/she> (6 comments)


31

Workshop findings Findings from the Principal workshops reinforce those collected from the questionnaire.

Whilst the majority of participants understand the benefits of having comment banks for

teachers to refer to, they were dissatisfied with the quality of them. Virtually all Principals

agreed that the comment banks were too positive, bland, vague and impersonal.

Principals also stated that the comment banks did not cover the new areas of learning in

the curriculum. They felt this was something that teachers should not be expected to

develop as this is a particular area in which teachers require support. Participants also felt

that there was not sufficient choice within the comment banks and that the language used

was not parent friendly.

Annual Pupil Profile Reports Figure 5: Format of the Annual Pupil Profile Reports

Principals were asked to rate the

overall format of both the Foundation

Stage and Key Stage 1 and 2 Annual

Pupil Profile Reports. Figure 5

indicates that over 75% of respondents

rated both the Foundation Stage (n =

241) and the Key Stage 1 and 2 (n =

240) Annual Pupil Profile Reports as

either excellent or good.

A small number of Principals who did not rate the format of the Annual Pupil Profile Report

highly made the following comments.

• Reports are too long and detailed (25 comments)

• Schools previous reporting format was better (10 comments)

• Too many areas to comment on, some of which are overlapping (8 comments)

• Would appreciate opportunity to include comments in other areas such as

play/punctuality (5 comments)

• Disliked the ordering of the boxes (5 comments)

• No grades or comparison to peer groups (5 comments)

6.2 6.5

72.2 71.2

15.416.5

5.9 5.8

0

20

40

60

80

100

% o

f Res

pond

ents

Excellent Good Poor Very poor

Foundation Stage

Key Stage 1 & 2


32

How informative are the Annual Pupil Profile Reports?

Figure 6: How informative are the Annual Pupil

Profile Reports? Figure 6 shows that over 70% of

Principals felt that both the Foundation

Stage (70.1%, n = 213) and the Key

Stage 1 and 2 (71.6%, n = 159) Annual

Pupil Profile Reports were either very

informative or informative. Over 28%

of the remaining respondents did not

deem either version of reports as

informative.

Respondents were asked to comment on their answer. The majority of comments were

from Principals who did not feel the Annual Pupil Profile Reports were informative.

• No better or worse than previous reports (15 comments)

• Reports are impersonal (10 comments)

• Reports give no idea of progression (6 comments)

• Reports are too positive (5 comments)

• Overload of information within reports (4 comments)

• Grades are more informative than qualitative detail within Annual Pupil Profile Report

(4 comments)

• Standardised tests are more informative (3 comments)

7.9 8.1

62.263.5

2220.7

7.97.7

0

20

40

60

80

100

% o

f Res

pond

ents

Veryinformative

Informative Notinformative

Not at allinformative

Foundation Stage

Key Stage 1 & 2


33

Workshop findings

Findings from the Principal workshops on the format and content of the Annual Pupil

Profile Reports were significantly more negative than those collected from the

questionnaire. Virtually all attendees at the Principal workshops described the reports as

“bland, general, vague, too wordy and too positive”. A large number of Principals also felt

that the Annual Pupil Profile Report is not useful to parents as it is too wordy and does not

contain the information parents particularly want – grades/scores.

The majority of Principals deemed their previous school report to be superior to the Annual

Pupil Profile Report – and felt that implementing a standardised report has taken away the

professionalism of the school.

A small number of Principals stated that they were issuing a letter of apology with the

Annual Pupil Profile Report as they were very disappointed with them.


34

Did the Annual Pupil Profile Reports provide Principals with an overall picture of pupil’s performance? Principals were asked to indicate if the Annual Pupil Profile Reports provided them with a

picture of pupils overall performance. Over 64% of Principals agreed that the Foundation

Stage Report (66.3%, n = 201) provided them with a picture of pupils’ overall performance

in Year 1 and the Key Stage 1 and 2 Annual Pupil Profile Report (64.8%, n = 142)

provided them with a picture of pupils’ performance in Year 5.

Principals were asked to comment on their answer. The majority of comments came from

those who were dissatisfied with the new reporting arrangements. Such respondents

commented that the Annual Pupil Profile Reports provided no additional information

(n = 17) and were too positive (n = 14). Other respondents felt that the reports should

include grades and scores from InCAS (n = 14). The remainder of comments are

summarised below.

• States what pupils ‘can do’- no indication of what they cannot do or progression

(12 comments)

• No comparison to peers (11 comments)

• Reports are impersonal (8 comments)

• Only informative due to the teacher’s personal comments (5 comments)

How manageable did teachers find the Annual Pupil Profile Reports to complete? Principals were asked to indicate how manageable Year 1 and 5 teachers found the

Annual Pupil Profile Reports to complete. Just over 50% of Principals felt that Year 1

teachers (50.3%, n = 158) and Year 5 teachers (55.9%, n = 127) found the reports

manageable to complete. A large number of respondents who believed teachers found

the reports unmanageable to complete commented that the reports were very time

consuming to compile (n = 79). Others felt the process was particularly unmanageable

because teachers were unable to complete reports at home (n = 39) and because no

funding was provided for schools to buy sub-cover (n = 20).


35

Workshop findings Throughout all of the workshops Principals stated that this year’s implementation of the

Annual Pupil Profile Reports has had a detrimental effect on teachers’ work life balance.

Principals highlighted that the new Annual Pupil Profile Reports have dramatically

increased the amount of time teachers spend collating and completing reports. Principals

stated that teachers have found completing this new report extremely stressful, particularly

those who used the Profiles 7 software. Many Principals felt they had no choice but to

keep their school open until 9pm at night so that teachers could complete the reports on

the Profiles 7 software. A substantial number of Principals felt that the implementation of

this new reporting arrangement has “abused the good will of teachers”. One Principal

stated that:

“Primary 1 and 5 teachers have undergone training for the revised curriculum this year and if you asked them what has been the major impact of this year, they will say report writing and InCAS, not that they learned how to teach the areas of the revised curriculum .”

Were the Annual Pupil Profile Reports used within any other year groups? Over 50% of Principals (n = >172) indicated that their school used the Annual Pupil Profile

formats to report to parents with children in Years 2, 3, 4 and 6. Over a third of Principals

(34%, n = 115) also used the Annual Pupil Profile Report for pupils in Year 7. Such

Principals stated that they implemented the Annual Pupil Profile Report into other year

groups as they wanted uniformity in both reporting and teacher workload (n = 24).


36

Do the Annual Pupil Profile Reports inform measures to improve whole school performance?

Principals were asked if they thought the Annual Pupil Profile Reports informed measures

to improve whole school performance. Only 27.4% of respondents (n = 62) believed that

the Annual Pupil Profile Report achieved this. Principals who felt the report did not inform

measures to improve overall school performance felt it did not accomplish this for the

following reasons.

• Need grades to inform improvement (15 comments)

• Does not provide any additional information (9 comments)

• Not specific enough to do this (8 comments)

• Reports are too positive (6 comments)

• Difficult to identify whole school pattern from individual reports (6 comments)

• InCAS is a better way to do this (4 comments)

Do the Annual Pupil Profile Reports inform teachers’ planning for learning and teaching? Less than 50% of Principals (n = 102) felt that the Annual Pupil Profile Reports inform

teachers’ planning for learning and teaching. Those who did not believe the report

informed teachers’ planning commented that the report does not provide teachers with any

additional information (n = 9) and that teachers find grades more informative (n = 4).

Workshop findings

All Principals felt that the Annual Pupil Profile Reports did not inform teacher planning.

Principals stated that annual reports are written in June which is the wrong time of year to

inform teachers’ planning. A small number of Principals also stated that the summative

nature of the annual report is not appropriate to inform teachers as they plan learning and

teaching.


37

Do the Annual Pupil Profile Reports inform parents of how they can support their child’s learning at home? 55.1% of Principals (n = 119) believed that the Annual Pupil Profile Report informs parents

of how they can support their child’s learning at home. A small number of such

respondents felt that the Focus for Development section within the report is particularly

useful for parents. Comments from Principals who believed the report does not inform

parents of how they can support their child’s learning are summarised below.

• Report provides no additional information (11 comments)

• Information is limited, and not specific or meaningful (11 comments)

• Reports require face to face consultation (6 comments)

• Reports are too positive - do not highlight weaknesses effectively (6 comments)

Workshop findings Virtually all attendees at the Principal workshops firmly stated that the Annual Pupil Profile

Reports do not inform parents of how they can support their child’s learning at home. A

large number of Principals stated that the report does not provide enough relevant

information to aid improvement. The majority of Principals within all focus groups stated

that the Annual Pupil Profile Reports are too positive and do not provide parents with a

true reflection of their child’s performance at school. Principals also felt that the reports

were not parent friendly as they overload parents with detail containing educational

language. Principals felt that many parents would not understand the content of the

report.

The majority of Principals within two workshops felt that the Annual Pupil Profile Report

should include commentary on pupils’ behaviour.


38

Do the Annual Pupil Profile Reports inform pupils of their achievements and how they can improve?

Over 62% of Principals (62.9%, n = 139) believed that the Annual Pupil Profile Reports

informed pupils of their achievements. A small number of such Principals felt that this

information was clearly documented within the interest and strengths section of the report

(n = 3) and was encouraging to pupils (n = 3). Principals who felt that the report did not

inform pupils of their achievements made the following comments.

• Contains no more information than previous school report (10 comments)

• Pupils did not understand information (6 comments)

• Information was too general (4 comments)

• Year 1 pupils are too young to understand (3 comments)

• Information was not personal enough to each individual child (2 comments)

• Pupils have a better understanding of grades (2 comments)

• Reports are for parents not pupils (2 comments)

Principals were also asked if they thought the Annual Pupil Profile Report informed pupils

of how they can improve. 56.3% of respondents (n = 120) felt that the report advised

pupils of how they can improve. Such respondents believed that the Focus for

Development section clearly informed pupils of how they could improve (n = 10).

Principals who felt the Annual Pupil Profile Report did not inform pupils of how they can

improve stated that the report did not provide pupils with any additional information (n =

17) and would require either the pupil’s teacher or parent to outline how they could

progress (n = 9).

Workshop findings

All Principals at the workshops felt that the Annual Pupil Profile Reports did not inform

pupils of their achievements and how they could improve. A large number of Principals

felt that pupils need either teachers or parents to discuss this with them.


39

Additional Information from the Principal Workshops Throughout all of the Principal workshops there was widespread criticism of the new

assessment and reporting arrangements. Principals felt that schools were not provided

with adequate support to implement InCAS or the Annual Pupil Profile Report. Principals

were particularly vocal about how unhappy and frustrated they were with the

implementation of the Annual Pupil Profile Report.

Principals expressed concern over the increasing requirement of schools to consult with

parents and have requested support to deliver this. Principals were also displeased that

CCEA was seeking parents’ views on the new assessment and reporting arrangements

during this transition year.

A number of Principals stated that this has been the worst year of their teaching career.

Principals asked for reassurance that the strength of feeling within the workshops would

be evident within the final evaluation report. However, the majority of Principals stated that

they were sceptical about what action would be taken following this consultation.


40

4.2 Results collected from Year 5 Teacher Questionnaires & Workshops

This section presents the results for each question from the Year 5 teacher questionnaire.

Results are based on questionnaires received from 378 respondents. Percentages relate

to the number of teachers who answered each question, which may be fewer than the

overall total 378. A number of teachers reiterated comments throughout their


report.

Qualitative information obtained from ten teacher workshops attended by 201 teachers has

been analysed alongside the feedback collected from the questionnaire. The purpose of

the workshops was to provide in-depth information that will reinforce and further explore

data collected from the questionnaire.


41

InCAS Computer-based Assessments Manageability of the InCAS computer-based assessments The majority of teachers (88%, n = 314) stated that they were confident at the prospect of

administering InCAS next year. A large number of such teachers stated that “having

administered [InCAS once they are] confident at doing it again” (n = 49). A small number

of teachers who were not confident at administering InCAS next year said that they found

this assessment time consuming to administer (n = 12) and would require further training

(n = 6).

45% of teachers (n = 154) indicated that they plan to introduce procedures to administer

InCAS next year. The most common procedures teachers plan to implement are

summarised below.

• Layout of class when completing InCAS (19 comments)

• Monitor pupils as they complete InCAS and make use of additional supervision if

available (18 comments)

• Obtain access to more computers / laptops (16 comments)

• Administer InCAS in smaller groups (11 comments)

• Administer InCAS earlier in the term (11 comments)

• Allow more time for administration (9 comments)

• Plan to provide mice for laptops (8 comments)

• Co-ordinate the use of computer suites with other teachers (7 comments)


42

Workshop findings Within all of the teacher workshops teachers’ views on the manageability of the

administration of the InCAS assessments varied greatly. Teachers’ opinions were largely

dependant upon a number of external factors which include: their ICT skills; class size;

availability of computer suites; number of computers within school, staff availability and

contingency plans they had in place.

Virtually all teachers said they encountered challenges administering InCAS. A large

number of challenges were the result of teachers administering InCAS for the first time

(e.g. problems with wireless access and passwords). Other technical challenges specific

to InCAS have been identified within earlier evaluations and will be rectified for the

2008/09 academic year (e.g. volume control).

Feedback from a sizeable number of teachers suggests that they felt they had not planned

or organised the administration of InCAS effectively. As a result, a large number of

teachers stated that they intend to put a number of procedures in place next year.

Procedures teachers plan to put in place next year are listed below.

• Timetable when InCAS will be administered to pupils • Pupils will complete InCAS in quieter rooms to reduce distractions • Teachers plan to make use of the C2K and CCEA helpdesks • Teachers plan to use the pupil demo

A large number of teachers stated that both sub-cover and ICT support are required to

support teachers during the administration of InCAS – particularly for those with large or

composite classes. Feedback from the teacher workshops also suggests that some

schools do not have the computer capacity to deliver InCAS. One teacher stated that

“schools without a computer suite were really at a severe disadvantage”.

Teachers had a number of concerns regarding the roll out of InCAS in 2008/09. Teachers’

main concerns were regarding the lack of computers and the insufficient support and time

teachers have to administer InCAS. A small number of teachers were also concerned that

low test results from the InCAS assessments could damage teachers’ reputations.

A small number of teachers expressed concerns about the impact the InCAS assessments

has had on the curriculum time, as a large amount of time has been spent administering

assessments as opposed to teaching.


43

InCAS feedback Figure 7: Did InCAS inform planning for learning

and teaching? Figure 7 illustrates that over 70% of

Year 5 teachers (n = 251) felt that

the feedback from InCAS informed

their planning for learning and

teaching. Comments from such

respondents are summarised below.

• Highlighted pupils’ strengths and weaknesses (62 comments)

• Confirmed and supported teachers’ professional judgements (42 comments)

• The breakdown of InCAS feedback was useful (27 comments)

• Feedback focused planning and teaching (20 comments)

Teachers who felt the InCAS feedback did not inform their planning for learning and

teaching made the following comments.

• General Maths feedback requires further breakdown (17 comments)

• Did not reflect pupils’ abilities (14 comments)

• Pupils were assessed and grouped prior to receiving InCAS feedback (14 comments)

70.9

29.1

0

20

40

60

80

100

% R

espo

nden

ts

Yes No


44

Figure 8: Did the InCAS feedback inform parents of how they can support their child’s learning?

Nearly 58% of teachers (n = 201) felt

the feedback from InCAS did not

inform parents of how they can

support their child’s learning at

home. However, despite such

negative findings, the main reason

teachers felt that the InCAS

feedback was not useful to parents

was because they chose not to share InCAS results with parents (72 comments). Other

teachers felt the feedback produced from InCAS was not useful to parents as they do not

have enough information about this assessment (16 comments).

Teachers who believed the InCAS feedback informed parents of how they can support

their child’s learning, made the following comments.

• Strengths and weaknesses could be discussed with parents (30 comments)

• Discussed InCAS results at parent / teacher meetings (23 comments)

“ as a result was able to discuss areas for improvement and possible help”

• Highlighted specific areas to focus on (20 comments)

• Reinforced teachers’ professional judgement (10 comments)

Teachers were asked to indicate if the feedback produced from InCAS informed pupils of

their strengths and how they could improve. Only 48% of teachers (n = 169) believed that

the feedback from InCAS informed pupils of their strengths and even fewer teachers

(38.2%, n = 131) believed that the feedback informed pupils of how they could improve.

However, the main reason teachers felt InCAS did not inform pupils of their strengths or

how they could improve was because they did not share the results with pupils (n = 100).

42.257.8

0

20

40

60

80

100

% R

espo

nden

ts

Yes No


45

Workshop findings Information collected from all of the teacher workshops regarding the usefulness of the

InCAS feedback was very mixed. Those who felt the feedback was useful particularly

liked that it highlighted pupils’ strengths and weaknesses. Other aspects of InCAS that

teachers liked included charts and graphs, breakdown of results and the fact that the

feedback informed class groupings. A small number of teachers also stated that the

feedback helped to identify pupils with dyslexia.

“InCAS certainly supported some of my concerns with dyslexic children and I

was able to put three of them into IEPs with dyslexic concerns and now they have been tested by the psychologist”

Many of the teachers who did not find the feedback from the InCAS useful felt that the

content was not in relation to standardised tests and that the feedback from the General

Maths module was not sufficiently broken down. A sizeable number of teachers also

queried the reliability of InCAS – “there were too many discrepancies that impacted on the

staff’s competency and confidence in the system”.

The majority of teachers who participated within the workshops shared InCAS feedback

with parents. A large number of such teachers felt guidance is needed to advise teachers

on how they share feedback with parents. One teacher stated that:

“Teachers want to know how much feedback and what kind of language to

use, some parents challenged teachers on the statistics and where they came from – therefore maybe a bit of advice on what kind of language needs to be involved in feedback is required”

Teachers who did not share feedback with parents generally did not do so because it was

not statutory this year or because they had concerns regarding the reliability of the results

produced from InCAS. A small number of teachers also stated that sharing the age

equivalent scores of pupils whose age was significantly below their actual age would be

very distressing for parents.


46

Profiles 7 Report Writing Training Despite 90.5% of respondents (n = 333) indicating that someone from their school

attended the Profiles 7 training offered by C2K, less than 50% of respondents attended it

themselves (49.5%, n = 172). Respondents indicated who in their school attended this

training.

• Year 5 teacher (113 comments)

• Principal (112 comments)

• ICT Co-ordinator (57 comments)

• Vice Principal (31 comments)

• Year 1 teacher (30 comments)

• Assessment co-ordinator (19 comments)


Teachers who attended the Profiles

7 report writing training offered by

C2K were asked to rate how useful

they found it. Figure 9 indicates that

the majority of respondents rated

this training as either very useful or

useful (92%, n = 161).

Despite the majority of teachers rating this training as useful, 40.9% (n = 72) of

respondents felt it was not adequate preparation for using the software. Comments from

such teachers are summarised below.

• Further training / support is required (10 comments)

• The time span between training and using the software was too long (9 comments)

Problems commenced when attempting to use the system for the first time

(7 comments)

“Unprepared for the errors that came up on the screen which was frustrating”

• More time should have been allocated to get used to the system (5 comments)

25.7

66.3

6.9 1.10

20

40

60

80

100

% R

espo

nden

t

Veryuseful

Useful Not useful Not at alluseful


47

Workshop findings The majority of participants within all of the teacher workshops indicated that someone

from their school attended the Profiles 7 training offered by C2K. The majority of teachers

felt that the C2K training was not sufficient preparation for using the software. Attendees

stated that all teachers should have received this training and that it was unfair to expect

teachers to cascade such technical training back in school. One teacher stated that “the

training was good for people who went but every teacher needed to attend it. Perhaps

more than one session is needed [per school]”. A large number of attendees also stated

that they had difficulties remembering the training when they returned to school.

A substantial number of teachers also highlighted that often the most relevant teachers

who were required to use the Profiles 7 software did not attend the training. The main

reason for this was because schools were not provided with sub-cover for the training and

as a result Principals decided to attend the training themselves or send other members of

staff as they saw fit.

Online Tutorials


teachers’ use of the Profiles 7 report writing software. Only 8.4% of teachers (n = 31) used

these tutorials. A small number of respondents who did not use the tutorials commented

that they were not aware of them (n = 44) or did not have time to use them (n = 10).

Nearly 70% of teachers who used the online tutorials deemed them to be useful

(69.2%, n = 27). However, only 45% of such teachers (n = 18) felt that the tutorials alone

were adequate preparation for using the software.

Workshop findings

Findings from the teacher workshops reinforce those collected from the questionnaires.

The majority of teachers were unaware of the online tutorials on the Northern Ireland

Curriculum website. However, many teachers stated that even if they had been aware of

them they would not have had time to use them.


48

Using the Profiles 7 report writing software 72.5% of Year 5 teachers (n = 264) used the Profiles 7 report writing software to complete

this year’s Annual Pupil Profile Reports. Teachers were asked to indicate if they thought

this software was user-friendly – 59.4% of respondents (n = 161) believed it was.

However, 64.1% of teachers (n = 162) experienced difficulties using the software. The

most common difficulties teachers commented on are summarised below.

• SIMs kept shutting down/freezing/crashing “SIMs kept shutting down in the middle of

writing a profile - very frustrating!” (70 comments)

• Issues saving / deleting / editing (19 comments)

• Problems logging onto SIMs (14 comments)

• SIMs not installed at all / or correctly on the computers (6 comments)

• System very slow (6 comments)

• Class names on system were incorrect (2 comments)

• Problems with passwords (2 comments)

• Spelling and grammatical errors in the comment editor (2 comments)

Figure 10: How manageable is Profiles 7 in comparison to completing reports by other methods?

Figure 10 illustrates that the majority of

teachers (68.8%, n = 185) found

Profiles 7 software less manageable to

other methods they have used to


Respondents were asked to comment on their response, these are summarised below.

• Extremely time consuming (91 comments)

“I lost count of the number of extra unpaid hours needed in school – not to mention

my child minding fees”

• Should be able to access Profiles 7 from home (47 comments)

13.4

68.8

17.80

20

40

60

80

100

% R

espo

nden

t

Moremanageable

Lessmanageable

No difference


49

Workshop findings Attendees at all of the teacher workshops were very critical of the Profiles 7 software. A

large number of teachers expressed high levels of frustration with the Profiles 7 software.

Teachers encountered a large number of problems with the software, particularly with

editing and saving, SIMS shutting down, the system crashing (and losing work as a result),

fonts changing, difficulties adjusting boxes and problems with the school server. A

sizeable number of teachers stated that their school had bought sub cover for a specific

day so that Year 1 and 5 teachers could write reports on the Profiles 7 software; however

they were unable to access SIMS on this specific day and the sub-cover was wasted.

Overall, the majority of teachers found the Profiles 7 software extremely time consuming,

labour intensive, and less manageable than previous methods they have used to complete

reports. Many teachers stated that completing the Annual Pupil Profile Reports had a

detrimental effect on their work/life balance.

“We felt it only worked because we were prepared to make it work, putting in the

exact hours, which isn’t even including time for printing, reprinting and principals looking at them and telling you extra things that had been overlooked. It is scandalous as in some cases 1-2 reports were only being done in one day.”

“We spent hours and hours, used exceptional closure days, days should be

provided by DENI, we shouldn’t have to use our exceptional closure days. Further issue is that we weren’t allowed to take them home.”

“The fact you couldn’t take the reports home was terrible, some teachers were

coming into school at 7am and leaving at 8pm”. A large number of teachers within all of the teacher workshops thought that they were

statutorily required to use the Profiles 7 software to complete this year’s Annual Pupil

Profile Reports. When teachers were made aware that this was not the case they were

extremely angry and critical of communications from government agencies.

“We were under the impression that we had to use SIMS to complete the Pupil

Profile, I found the experience of using SIMS very cumbersome and having to stay late (until 9pm many evenings) and a weekend in order to complete these, an absolute nightmare as we had to have them completed for a stipulated date in mid May!”


50

CCEA Comment Banks

93.7% of teachers (n = 341) indicated that they made use of the CCEA comment banks.

Respondents were asked to rate how useful they found them. Although 77.5% of

respondents (n = 268) felt the comment banks were useful – the majority of teachers

(91.6%, n = 316) felt they could be improved upon. A large number of respondents

provided suggestions on how these could be improved.

• Wider variety of comments required to cover all areas of curriculum (146 comments) “Supply specific banks for all areas of the curriculum. We are all supposed to be

following the revised curriculum therefore all comments should be made available for

staff”

• Too positive – need more negative comments (65 comments)

• Comments need to be more personal and less generalised (53 comments)

• Grammar and vocabulary needs to be improved (16 comments)

• Consult with schools to add to comment banks / amend current one (13 comments)

• Different comments for different year groups / pupil abilities (13 comments)


51

Workshop findings

Findings from the teacher workshops reinforce those collected from the questionnaire.

Whilst the majority of participants foresee the benefits of having comment banks to refer

to, the majority of teachers were dissatisfied with the quality of them. Teachers described

the comment banks as “too positive, too general, repetitive and impersonal”.

Teachers made a number of recommendations and suggestions to improve them. A large

number of teachers stated that the spelling and punctuation within the comment banks is

incorrect and needs to be corrected. Respondents also recommended that comments

should include “beginning to” in addition to the “can do” comments. A sizeable number of

teachers felt that exemplary comments should be provided for the Focus for Development

section of the report. Teachers also highlighted that comment banks for two separate

levels can not be opened/used simultaneously and felt this should be programmed into the

Profiles 7 software for next year.

A large number of teachers incorrectly believed that the comments banks were statutory

and they were required to cut and paste comments into the reports. This is possibly one of

the reasons why teachers felt the reports were so impersonal and did not adequately

reflect pupils’ progression throughout the year. A small number of such teachers stated

that they had more or less written identical reports for every pupil within their class.

Annual Pupil Profile Reports Figure 11: Format of the Annual Pupil Profile Reports

Teachers were asked to rate the overall

format of the Key Stage 1 and 2 Report.

Figure 11 indicates that over 70% of

respondents (71.2%, n = 247) rated the

Key Stage 1 and 2 Report as either

excellent or good.

3.7

67.4

22.2 6.6

0

20

40

60

80

100

% R

espo

nden

t



52

A small number of teachers who did not rate the format of the Annual Pupil Profile Report


• Report is too “wordy” (45 comments)

• There is a lot of overlap between boxes which should be combined (18 comments)

“Focus for Development, The Arts, P.E., The World Around Us and Interests

and Strengths can all be reported on in one box at the end”

• Disliked current format (15 comments)

• Grades should be recorded within the report (9 comments)

• Prefer previous school report (7 comments)

• Focus for Development section would be better placed at the end of the report

(4 comments)

• Interests and Strengths should be combined within the Optional Content box

(3 comments)

Workshop findings Within the majority of the teacher focus groups a small number of teachers felt that the

Focus for Development section should be repositioned from the front page of the report to

the end. Such teachers felt that it was important that the report begins positively and

concludes with constructive advice on areas for development.

Did the Key Stage 1 and 2 Annual Pupil Profile Reports provide you with a picture of the overall performance of individual pupils? 55.2% of teachers (n = 190) believed that the Key Stage 1 and 2 Annual Pupil Profile

Reports provided them with a picture of the overall performance of individual pupils.

44.8% of teachers (n = 154) disagreed and provided the following reasons for their

answer.

• Reports are too positive – require realistic or some negative comments (42 comments)

• Reports are too impersonal / vague (39 comments)

“Too general, not really specific to the individual pupils”

• Grade / results should be included within reports (27 comments)

• Personalised comments from teacher are better than amending or adding to comment

banks (20 comments)


53


Teachers were asked if they thought the Annual Pupil Profile Reports inform measures to

improve whole school performance. Only 20.6% of respondents (n = 74) believed that the

Annual Pupil Profile Report achieved this. Teachers who stated that the Annual Pupil

Profile Reports did not inform measures to improve overall school performance felt it did

not accomplish this for the following reasons.

• Results required (15 comments)

• No more useful than previous school reports (11 comments)

• Too bland, impersonal and difficult to interpret (11 comments)

• Too positive (6 comments)

Do the Annual Pupil Profile Reports inform teachers’ planning for learning and teaching? Only 42.1% of teachers (n = 149) felt that the Annual Pupil Profile Reports informed

teachers planning for learning and teaching. Comments from those who did not believe

the report informed teachers’ planning are summarised below.

• Standard assessment results provide teachers with more useful information (20 comments)

• Prefer format of previous school reports (13 comments)

• Language and terminology used in reports is vague and difficult to interpret (10 comments)

• Teachers use other methods to plan for the future (7 comments) Workshop findings

Participants within all of the teacher workshops unanimously agreed that the Annual Pupil

Profile Report is not informative for teacher planning. The majority of teachers felt that the

report is too vague and too positive. A sizeable number of teachers also stated that the

reports are impersonal, very similar and do not reflect pupils’ ability.


54

Do the Annual Pupil Profile Reports inform parents of how they can support their child’s learning at home? 63% of teachers (n = 218) believed that the Annual Pupil Profile Report informed parents

of how they can support their child’s learning at home. A small number of such

respondents felt that the Focus for Development section within the report is particularly

useful for parents (n = 56). Comments from teachers who believed the report does not

inform parents of how they can support their child’s learning are summarised below.

• The Annual Pupil Profile Reports are no better than previous reports (19 comments)

• Comments within reports are too positive (15 comments)

• Language within reports is impersonal and difficult for parents to interpret (13 comments)

• Grades/scores should be included in the reports - “A lot of parents commented that

they didn’t know where exactly their child was academically. Many would like to know

if they are average, above average or below average. The profile didn’t allow for this

information” (12 comments)

• The report does not highlight areas where children can improve (10 comments)

Workshop findings

The majority of attendees at all of the teacher workshops firmly stated that the Annual

Pupil Profile Reports do not inform parents of how they can support their child’s learning at

home. The main reasons teachers felt that the reports were not informative to parents

was because (1) they do not include scores / grades and (2) because they are not parent

friendly as they contain a lot of “educational jargon”.

The majority of teachers also stated that the Annual Pupil Profile Reports were too positive

and therefore may have given parents a false picture of their child’s ability/progress. One

teacher stated that they had used comment banks to write reports for several pupils who

were working at a level below their year group. This teacher stated that the report read

very positively as the comments she had inserted from the bank of statements were

positive and generally “can do” statements.


55


Nearly 80% of teachers (79.7%, n = 287) believed that the Annual Pupil Profile Reports

informed pupils of their achievements. A small number of such teachers (n = 32)

commented that the Key Stage 1 & 2 Pupil Profile Report is a very positive document that

focuses on the child’s strengths rather than problems.

Teachers who felt that the report did not inform pupils of their achievements made the

following comments.

• Previous school report was more informative (13 comments)

• Language vague and difficult to interpret (13 comments)

• The comments were too positive (8 comments)

• Grades/scores should be included within the reports (6 comments)

• Does not inform pupils of their weakness (6 comments)

• Pupils are already aware of their strengths (3 comments)

Teachers were also asked if they thought the Annual Pupil Profile Report informed pupils

of how they can improve. 69.7% of respondents (n = 242) felt that the report did advise

pupils of how they can improve. Such respondents believed that the Focus for

Development section provided pupils with this information (n = 49).

Workshop findings All attendees at the teacher workshops felt that the Annual Pupil Profile Reports did not

inform pupils of their achievements and how they could improve. A large number of

teachers felt that pupils need grades / marks to inform them of their achievements and

areas in which they need to improve. Other teachers stated that the reports were not pupil

friendly as pupils do not understand the language used within them. A large number of

teachers also felt that the reports were too positive and as a result misleading to parents.


56

Additional Comments

Respondents were given the opportunity to make additional comments at the end of the

questionnaire. The majority of comments highlighted concerns teachers had about the

Annual Pupil Profile Report. A large number of respondents stated that they found

completing the Annual Pupil Profile Reports extremely time consuming (n=117). 23

teachers stated that they would like to see grades/scores included within future reports,

one teacher commented:

“Our parents are very keen for their children to go to grammar school – these

profiles did not tell them how their children had achieved in the year”.

Other teachers indicated that the Annual Pupil Profile Reports were impersonal and vague

(n=21). One teacher commented:

“Children received bland, impersonal reports that left parents uninformed about their

progress in comparison to others”.

13 respondents preferred the format of their schools previous report (n=13).


57

4.3 Results collected from Year 5 Parent Questionnaires & Focus Groups

This section presents the results for each question from the Year 5 parent questionnaire.


to the number of respondents who answered each question, which may be fewer than the

overall total 3586. A number of parents reiterated comments throughout their


report.

Qualitative information obtained from thirteen parent focus groups attended by 145 parents

has been analysed alongside the feedback collected from the questionnaire. The purpose

of this is to provide further in-depth information to reinforce and expand upon data

collected from the questionnaire.


58

InCAS Feedback Parent Teacher Meeting Over 90% of parents (90.6%, n = 3225) attended a parent teacher meeting about their

child this school year. At this meeting, 76.3% of parents (n = 2461) were informed about

the InCAS assessment their child completed in the autumn term. Parents comments

regarding the information they were provided with at this meeting are very mixed. Whilst a

considerable number of parents (n = 130) stated that they were satisfied with the

information they were provided with, many were not (n = 102). A small number of such

parents (n = 13) felt that teachers were unsure about the InCAS assessments and as a

result were unable to fully inform them about InCAS:

“The teachers themselves seemed a little unsure of the aims of the assessment and could only tell me what I already knew - that his Maths and English were age graded by means of a computer test”

“Teachers were very unsure of InCAS what the results would be used for and the

significance of testing”

“A short explanation was given but we were unsure if this was part of a new transfer process or not”

Figure 12: Were you informed of your child’s

results from InCAS?

Parents who attended the parent teacher

meeting were asked if they were

informed of their child’s results from the

InCAS assessment. Figure 12 illustrates

that 55% of parents received their child’s

results from InCAS.

A small number of parents who received their child’s InCAS results commented that they

were pleased with them (n = 53) and felt they informed them of what their child should be

obtaining (n = 43 comments).

5545

0

20

40

60

80

100

% o

f Res

pond

ents

Yes No


59

A number of parents commented that they were not satisfied with the information they

were provided with regarding InCAS for the following reasons.

• Would have liked to discuss the results in more detail (71 comments)

• Had difficulty understanding the results (21 comments)

• Uncertain of the tests validity (18 comments)

• Felt results were meaningless without benchmark or measurement against the rest of

the class (12 comments)

• Results did not reflect child’s ability (7 comments)

Figure 13: How were the results from InCAS Shared with you?

Parents were asked to specify how

teachers shared the InCAS feedback with

them. Figure 13 indicates that 80.3% of

parents (n = 1549) were verbally informed

of their child’s InCAS results. The

remaining 19.7% of parents (n = 379)

received this information in writing.

A sizeable number of parents who were informed of their child’s InCAS results verbally

said that they would have also liked a copy of the results in writing for their “own records”

(n = 83). A small number of parents stated that the verbal information they received at the

parents meeting was not explained and did not contain sufficient detail (n = 31).

Comments were also made from parents who received their child’s InCAS results in

writing. Whilst some parents deemed this information as clear and felt it provided a good

indication of how their child was developing (n = 12) others felt that they were not provided

with enough information (n = 9).

80.3

19.7

0

20

40

60

80

100

% o

f Res

pond

ents

Verbally In Writing


60

Figure 14: Did the results from InCAS enhance the information shared at the parents meeting?

Figure 14 illustrates that over 57% of

parents (57.5%, n = 1097) believed

that the results from InCAS

enhanced the information shared at

the parents meeting. Such parents

commented that the results;

• Identified their child’s strengths and areas they can improve upon (131 comments);

• Were very detailed (26 comments);

• Confirmed what teachers predicted (20 comments);

• Enabled teacher/parent to compare children’s results (6 comments);

• Were a positive addition to the parent meeting (5 comments); and

• Complemented information already provided by the teacher (5 comments).

Two parents commented that the InCAS results:

“gives a more accurate understanding of a child’s performance. It

pinpoints areas of ability and weaknesses in a straightforward manner that all parents can understand”

“gives a better reflection of the child and it is non-biased”

Comments from parents who did not believe the InCAS results enhanced the parents

meeting are summarised below.

• Would like more information on purpose of InCAS assessments (41 comments)

• Results were not explained in great detail (39 comments)

• Computer test does not show a child’s ability/progress (22 comments)

• Would like to know how InCAS is measured/benchmarked (17 comments)

• Would prefer the assessment later in the year so that progress could be measured

(13 comments)

• Unsure of the assessments validity (12 comments)

• Information was difficult to interpret (11 comments)

57.5

42.5

0

20

40

60

80

100

% o

f Res

pond

ents

Yes No


61

A small number of parents were concerned that children with less ICT skills were at a

disadvantage (n = 6). One parent commented that they “have many reservations

surrounding InCAS. I feel computerised testing requires a different skill set to written

assessment. I am particularly concerned about assessment of Mathematics and

Numeracy via InCAS. My child scored very highly but I remain concerned about the

process”.

Focus Group findings Overall findings from the parent focus groups were very negative. The majority of parents

who participated in the focus groups did not feel fully informed about the purpose and use

of InCAS. With the current uncertainty regarding transfer procedures parents were very

concerned that InCAS could potentially be used as a method for transferring pupils to Post

Primary Schools. Such thinking has heightened many parents concerns about InCAS as

they believe there may be a “hidden agenda” with this computer-based assessment.

A large number of parents indicated that they distrust computer-based assessments.

Parents particularly queried where the data from this assessment is stored and what this

information could potentially be used for. A number of parents queried why so much

money was being spent on new assessment arrangements if it is not going to be used for

other purposes.

The majority of participants within the parent focus groups felt that schools were not fully

informed or trained to use InCAS. Parents were also discontent with the amount of time

teachers have spent on training and administering InCAS, which they believed was to the

detriment of pupil learning.

A sizeable number of parents had been alerted to problems that had occurred whilst pupils

were completing InCAS. Such parents were concerned about these and questioned

whether schools have the technology to administer this assessment. Parents were also

concerned about the inconsistencies between schools, stating that schools with a greater

number of teachers with ICT expertise and computers were at an advantage.

A sizeable number of parents questioned the validity of the InCAS feedback and felt that

standardised tests such as NFER had a greater level of reliability and validity. Many of

these parents stated that teachers are not confident with the results produced from InCAS.


62

Focus Group findings continued

The majority of parents were concerned about the level of ICT skills children required to

complete InCAS. Such parents felt that children with less experience of computers were

at an unfair disadvantage. A large number of parents said they would like notice of when

children are completing the assessment to ensure that they have adequate rest prior to the

assessment and understand the importance of the “test”.

A sizeable number of parents who did not receive their child’s scores to InCAS felt that

schools were withholding information from them. A small number of parents who

requested their child’s InCAS results from their school stated that the school was very

reluctant to share this information with them. Such parents stated that this left them

feeling very frustrated and anxious.

Annual Pupil Profile Report 72% of parents (n = 2474) received information from their child’s school about the

introduction of the Annual Pupil Profile Report. A large number of parents indicated that

they received this information in either a letter (n = 176) or a school meeting (n = 103).

Figure 15: Format of the Pupil Profile Report

Parents were asked to rate the format

of the Annual Pupil Profile Report.

Figure 15 illustrates that the majority of

parents rated this report highly (86.8%,

n = 3007). Comments from parents

who rated the report highly are

summarised below.

• A very detailed, comprehensive and informative report (156 comments)

• The report was clear and concise and well presented (133 comments)

• Very pleased with child’s report (74 comments)

• Report provided a good indication of child’s strengths and weaknesses (49 comments)

• Easy to understand and follow (18 comments)

31.9

54.9

9.33.9

0

20

40

60

80

100

% o

f Res

pond

ents



63

A large number of comments were also received from parents who did not rate the Annual

Pupil Profile Report highly, such parents felt that the report;

• Should include grades (181 comments);

• Was impersonal and the language felt prescribed (137 comments)

• Should include a measure/benchmark of how their child is performing in relation to their

peers (74 comments).

• Was not as good as the previous report (55 comments)

• Difficult to interpret/understand (28 comments)

• Was too wordy (27 comments)

• Similar to the previous school report (14 comments)

• Did not advise parents of how they can help their child progress (14 comments)

• Was too positive (10 comments)

• Not very detailed (7 comments)

One parent stated that the report was:

“Uninformative – it seemed to be regurgitation of the curriculum stating what

our child had done and was able to do but not how well, the amount of help required and how her knowledge, understanding and independence rated in comparison to her age and other classmates”

Did you find the information within the Annual Pupil Profile Report easy to read and understand?

The majority of parents indicated that they found their child’s Annual Pupil Profile Report

easy to read (94.8%, n = 3280) and understand (91%, n = 3080). A large number of such

parents (n = 203) said that the report was clearly presented and easy to understand - “the

language used was clear and concise. I would imagine that most parents could

understand it regardless of whether they were from an educational background or not. I

certainly was very happy with it”.


64

A number of parents highlighted aspects of the Annual Pupil Profile Report they disliked.

• Report should include marks/grades to identify the level children are working at (117 comments)

• The language used within the report was too formal (91 comments)

• Report felt impersonal and prescribed (72 comments)

• Report should include benchmark/measurement of how children perform in relation to

their peers (48 comments)

• Preferred previous school report (32 comments)

• Report did not contain enough detailed information on the child (24 comments)

• Report should contain greater detail on how the child should improve (21 comments) Comments from two parents are provided below.

“The terminology used is educationalist. In parts did not appear to be

specific/individual to our child. Do not have a clear idea as to child’s attainment in relation to curriculum or against peers”.

“Yes, it is written in plain English but it is not clear how well or not my child is

doing. It is unclear precisely what knowledge and behaviour P5 children are expected to possess/display and it is therefore impossible to assess my child’s performance from the profile report. For example my child can recognise simple 2D and 3D shapes. Is this what he was taught or were there other shapes taught that he failed to name? Likewise my child can describe some common landscape features and geographical conditions but unclear if “some” was all he was taught and expected to know. My child has improved or made progress in some areas however it is not possible to measure how much he has improved, how he has improved and in specifically what areas because explanation is too general”.

How informative are the Annual Pupil Profile Reports? The majority of parents agreed that the Annual Pupil Profile Report informed them of their

child’s achievements (88.3%, n = 3067) and how their child could improve

(81.7%, n = 2790). One parent described the Annual Pupil Profile Report as “a very well-

rounded report”.


65

Many of the parents who indicated that the Annual Pupil Profile Report did not inform them

of their child’s achievements or areas for improvement provided reasons for their answer.

• Not enough information on areas for progression (131 comments)

• Parents would like report to include pupil grades/scores (113 comments)

• Language in reports was too formal and vague (83 comments)

• Reports should have more detail (61 comments)

• There was no benchmark or measure of child’s progress (49 comments)

• Reports were impersonal – did not provide an individual profile of the child (26 comments)

Figure 16: Focus for Development

Parents were asked if the Focus for

Development section of the Annual Pupil

Profile Report informed them of how they

can support their child’s learning at

home. Figure 16 illustrates that nearly

70% of parents (n = 2345) believed that

this particular section of the report

advised them of how they could aid their

child’s learning at home.

A large number of such parents (n = 242) highlighted that they found this section of the

report particularly useful as it identified specific areas they could focus on and encourage

their child’s learning.

Parents who did not find the Focus for Development section of the report useful stated

that;

• It did not advise them of how they could support their child’s learning (155 comments);

• Information within this section was limited (87 comments);

• The language used was vague and difficult to interpret (56 comments);

• This report did not say anything new about the child (24 comments); and

• Their report did not contain this section (21 comments).

69.3

30.7

0

20

40

60

80

100

% o

f Res

pond

ents

Yes No


66

What did you like most about your child’s Annual Pupil Profile Report?

Parents were asked to indicate what they liked most about their child’s Annual Pupil Profile

Report. The most prevalent responses from parents are summarised below.

• Comprehensive and detailed overview of child’s progress (660 comments)

• The report was clear, concise and easy to read (493 comments)

• Report identifies strengths and weaknesses (242 comments)

• Breakdown of each subject / format of report (209 comments)

• ‘Positive’ language teachers used in the reports (104 comments)

• Allows the teacher to personalise/individualise the report more effectively than previous

reports (88 comments)

• Focus for Development section was useful (55 comments)

Comments from two parents are provided below.

“I found the report to be more informative, easy to read and well

presented. I feel this format of reporting is by far the best I have received to date on my son’s education”

“It provided constructive feedback about my daughter’s progress

indicating all round strengths not just from an academic perspective. It clearly indicated a focus for improvement thus indicating where I can support her development from home”


67

What did you dislike about your child’s Annual Pupil Profile Report?

Parents were asked to indicate what they disliked about their child’s Annual Pupil Profile

Report. The most frequent responses are summarised below.

• Did not include grades/scores to show child’s progress (664 comments)

• Reports seemed impersonal in that they could ‘have been written about any child”

(212 comments)

• The reports did not provide any evidence as to where the child is in relation to

class/peer average (209 comments)

• The language used was formal and difficult for both parents and children to interpret

(202 comments)

• There should be more information provided on areas in which the child could develop

and improve (122 comments)

• Report took too long to read (57 comments)

Comments from two parents are summarised below:

“The report does not give any grades or indication of academic achievement so we can identify [our child’s] strengths and weaknesses. Nor can we see if she is above or below average in her peer group”.

“It was generalised, full of standard sentences drawn from a pool of

unbelievable blandness, designed not to offend or at another level inform”. Focus Group findings

At the time of the focus groups virtually all parents had received their child’s Annual Pupil

Profile Report. Overall findings collected from the parent focus groups are very negative.

The majority of parents did not feel that they had been provided with sufficient information

on the new reporting arrangements. A large number of parents revealed that their main

reason for taking part in the focus group was because they thought the Annual Pupil

Profile Report was a replacement for the transfer test. With the current uncertainty

regarding transfer procedures parents were reluctant to believe that this report would not

be used for transfer purposes. Parents were particularly dubious about the Pupil Profile

Summary that their child will receive in Year 7.


68

Focus Group Findings Continued The majority of attendees at all of the parent focus groups were particularly dissatisfied

with three aspects of the Annual Pupil Profile Report. One of the largest criticisms of this

report was that it did not contain grades. Virtually all parents deemed grades to be an

essential component of a school report. Many parents stated that grades clearly inform

them of their child’s academic performance, behaviour and enable them to monitor their

child’s progress year on year. The majority of parents also stated that they would like a

measure of how their child is performing in comparison to their peers. Parents questioned

why the Department of Education made the decision to exclude grades from school

reports.

The second complaint concerning the Annual Pupil Profile Report was regarding how

impersonal and general the reports were. Parents felt the reports were overloaded with

bland detail that did not inform them of their child’s performance. A large number of

parents stated that it was evident that teachers had used comment banks to complete

reports as the report read like a “cut and paste exercise”. A small number of parents said

that they were aware that practically the same report had been completed for every pupil

within their child’s class.

The third aspect of the Annual Pupil Profile Report that parents were dissatisfied with was

regarding how “hyper-positive” the report was. Many parents stated that they found this

particular aspect of the report unhelpful and misleading. A large number of parents

revealed that they were extremely happy the first time they read the comments within their

child’s report; however they later felt extremely frustrated as the teacher had omitted

crucial information on areas in which their child needs to develop. Many parents stated

that the report is full of positive statements such as “can do”, “beginning to” and that it is

not clear from the report how their child is developing according to their age or to their

individual potential.

Many parents indicated that as the Annual Pupil Profile Report did not include grades, was

vague and contained too much positive commentary they had no choice but to try and

“decode” the report for important pieces of information, especially areas in which their child

needs to improve. As a result, many parents indicated that they are very unhappy with the

Annual Pupil Profile Report they received this year and prefer the schools previous school

report.


69

Additional Comments


questionnaire. Many of these responses have been stated before and incorporated into

the main body of results; however the responses are also summarised below.

Parents raised a number of concerns regarding the Annual Pupil Profile Report. A large

number of parents were dissatisfied that the report did not contain marks/scores (n=286) to

indicate a child’s ability. 103 respondents felt that there was no evidence as to where their

child was in relation to the rest of the class. A significant number of parents also

commented that the language used within the report was too formal and difficult to

interpret (n=100). Parents also described the comments within the report as impersonal

and formulaic (n=72). Parents were aware that teachers had devoted a lot of time to the

reports and were worried that it was at the expense of valuable teaching time (n=75). A

significant number of parents (n=69) felt that the report did not give them enough

information on how their children can improve. 87 respondents said that they preferred the

previous format of their child’s reports.

Parents were also concerned with certain aspects of InCAS. Some parents were

concerned that pupils with lower ICT skills are placed at a disadvantage (n=20). 15

parents felt that the InCAS assessments should be administered later in the school year

and 17 respondents stated that they would like more information on InCAS. One parent

commented that:

“If you tested my child at this stage [June] of P5 rather than the beginning of P5 you

would get a totally different result. I think any computer that dictates your child's

abilities is wrong. Children are people - different in the extreme, brilliant at some

things, not so good at others but to try and put them into tick box categories is

missing the point”.

A large number of parents requested more information on the transfer procedure (111

comments) and on the InCAS computer-based assessment (60 comments).


70

4.4 Results collected from Year 5 Pupil Questionnaires

This section presents the results for each question from the Year 5 pupil questionnaire.

Results are based on questionnaires from 5,120 respondents. Percentages relate to the

number of pupils who answered each question, which may be fewer than the overall total

of 5,120.

Did you read your Annual Pupil Profile Report?

Just over 50% of pupils read their Annual Pupil Profile Report (n = 2739). In addition to

this, nearly 70% of pupils (69%, n = 3538) indicated that their parent read their report to

them and 1.7% of pupils (n = 86) stated that their teacher read it to them.

Figure 17: Did pupils find the Annual Pupil Profile Report easy to understand?

Pupils were asked if they found

their Annual Pupil Profile Report

easy to understand. Figure 17

illustrates that the majority of pupils

(90.9%, n = 4538) found their

report easy to understand.

“I liked this years report better because I understood it better”

“My Pupil Profile Report was much easier to understand than the old report. It

explained things much more deeply, and just got straight to the point”

A large number of pupils said that they liked the layout of the Annual Pupil Profile Report,

which they felt made it easier for them to understand (n = 91).

Stronglyagree

Agree Disagree Stronglydisagree

40.7

2.36.8

50.2

0

20

40

60

80

100

% o

f Res

pond

ents


71

“I prefer this report as it is split into paragraphs and is easier to understand”

“I think that the way the report is set out is easy to understand and shows the areas

I need to improve some of my work”

Figure 18: Did the Annual Pupil Profile report inform pupils of their strengths?

Figure 18 indicates that 96% of

pupils (n = 4784) agreed that their


informed them of their strengths.

A large number of pupils said that they particularly liked that the Annual Pupil Profile

Report had a more positive approach than their previous school report (n = 94). One pupil

said this made them “feel very pleased about [themselves] and encouraged [them] to keep

working hard”.

Stronglyagree

Agree Disagree Stronglydisagree

65.1

12.9

30.9

0

20

40

60

80

100

% o

f Res

pond

ents


72

Figure 19: Did the Annual Pupil Profile Report inform Pupils of how they can improve?

Pupils were asked if they felt their

Annual Pupil Profile Report informed

them of how they can improve. Just

over 85.4% of pupils (85.5%, n = 4239)

agreed that the Annual Pupil Profile

Report informed them of how they can

improve. A large number of pupils

(n = 200) also commented that their

report made them aware of areas in

which they need to improve. One pupil

stated:

“I was really happy about it and happy about what my teacher said about me. There used to be lots of hard words, but now there is not as many and it is like it’s made for kids as well as parents. It made me understand properly the things I need to improve. It was easy to understand.”

Figure 20: Did the Annual Pupil Profile Reports provide pupils with useful information?

Figure 20 highlights that 96% of

pupils strongly agreed or agreed

that their Annual Pupil Profile

Report provided them with useful

information.

44.2 41.2

10.24.4

0

20

40

60

80

100

% o

f Res

pond

ents

Stronglyagree

Agree Disagree StronglyDisagree

Stronglyagree

Agree Disagree StronglyDisagree

65.1

12.9

30.9

0

20

40

60

80

100

% o

f Res

pond

ents


73

A large number of pupils commented that their Annual Pupil Profile Report provided them

with a lot of useful information (n = 90). One pupil said that “the reports for Primary 5 are

great. Usually the reports say about Reading, Maths and English but this time there was

lots more”. A sizeable number of pupils also stated that they preferred the Annual Pupil

Profile Report to their previous school report (n = 76).

“The Pupil Profile Report is better than our old report because there is more detail

added to it”.

Despite such positive findings, a large number of pupils highlighted a number of aspects of

the Annual Pupil Profile Report that they disliked, these are summarised below.

• Would like test results / scores included in the report – “grades help me to understand

where I need to improve stuff like handwriting” (194 comments)

• Wording within the report was difficult to understand – “there were too many big words

explaining the subjects” (181 comments)

• Would like grades within the report (A,B,C / Average) (179 comments)

• Preferred last year’s report (142 comments)

• Report was too long – “there was too much writing and too many pages”

(83 comments)


74

4.5 Results collected from Year 1 Teacher Questionnaires

This section presents the results for each question from the Year 1 teacher questionnaire.


to the number of teachers who answered each question, which may be fewer than the

overall total 261. A number of teachers reiterated comments throughout their


report.

Profiles 7 Report Writing Training Despite 92.4% of Year 1 teachers (n = 232) indicating that someone from their school

attended the Profiles 7 training offered by C2K, less than a third of respondents attended it

themselves (30.8%, n = 74). The most common members of staff respondents who

attended this training are listed below.

• Principal (85 comments)

• P5 teacher (52 comments)

• ICT Co-ordinator (46 comments)

• P1 teacher (39 comments)


Year 1 teachers who attended

the Profiles 7 report writing

training offered by C2K were

asked to rate how useful they

found it. Figure 21 illustrates

that the majority of respondents

rated this training as either very

useful or useful (95%, n = 77).

29.6

65.4

3.7 1.2

0

20

40

60

80

% o

f Res

pond

ents

Very useful Useful Not useful Not at alluseful


75

Despite the majority of teachers rating this training as useful, 54.7% (n = 47) of

respondents felt it was not adequate preparation for using the software. Comments from

such teachers are summarised below.

• Teachers require more practice to familiarise themselves with software (9 comments)

• All teachers require training in the software (3 comments)

• Too much time is required for reading notes and information provided (3 comments)

• Training was not satisfactory – more is needed (3 comments)


76

Online Tutorials


teachers’ use of the Profiles 7 report writing software. Only 4.4% of teachers (n = 11) used

these tutorials. A small number of respondents who did not use the tutorials commented

that they were not aware of them (n =41) others did not have time to use them (n = 7).

Seven of the 11 teachers who used the online tutorials deemed them to be useful.

However, only two teachers felt that the tutorials alone were adequate preparation for

using the software.

Using the Profiles 7 report writing software 70.9% of Year 1 teachers (n = 175) used the Profiles 7 report writing software to complete

this year’s Annual Pupil Profile Reports. Teachers were asked to indicate if they thought

this software was user-friendly – 56.6% of respondents (n = 103) believed it was.

However, 67.2% of teachers (n = 123) experienced difficulties using the software. The

most common difficulties teachers commented on are summarised below.

• Software crashed / shut down which resulted in work being lost (55 comments)

• Software was extremely time consuming (40 comments)

• Difficulty editing profiles and saving changes (34 comments)

• Could not preview profiles before printing them (16 comments)

• Disproportionate amount of time taken to load software / log on (16 comments)

• Software recurrently froze (15 comments)

• Comments in banks were unsuitable (14 comments)

• Unable to access software after 9pm (7 comments)

• Unable to work on reports at home (6 comments)


77

Figure 22: How manageable is Profiles 7 in comparison to completing reports by other methods?

Figure 22 illustrates that the

majority of teachers

(73.2%, n = 161) found the

Profiles 7 software less

manageable than other

methods they have used to


Many respondents highlighted why they found the Profiles 7 software less manageable,

these are summarised below.

• Using the software is confined to school (84 comments)

• Software is very time consuming to use (80 comments)

• Comment banks did not provide for differentiation and were unhelpful (16 comments)

• Previous annual reports were less time consuming and easier to complete

(11 comments)

• Prefer to complete report on Microsoft Word / other software (8 comments)

• School required sub-cover to complete reports on this software (5 comments)

14.8

73.2

120

20

40

60

80

% o

f Res

pond

ents

Moremanageable

Lessmanageable

No difference


78

CCEA Comment Banks

92.8% of teachers (n = 233) made use of the CCEA comment banks. Respondents were

asked to rate how useful they found the comment banks. Although 80.9% of respondents

(n = 190) felt the comment banks were useful – the majority of teachers (92.5%, n = 223)

felt they could be improved upon. A large number of respondents provided suggestions on

how the comment banks could be improved, these are summarised below.

• Comment banks are required for all learning areas (77 comments)

• A wider range of comments are required to differentiate children’s abilities

(52 comments)

• Comment banks need to be more specific / clear (37 comments)

• A greater range of comments are required for Foundation Stage pupils (21 comments)

• Comment banks need to be more personal (13 comments)

• Different comments are required for each Key Stage / Year Group (9 comments)

• Comments were too positive; do not provide a succinct view of the child (6 comments)

• CCEA needs to work alongside schools to develop the comment banks (4 comments)

Annual Pupil Profile Report Figure 23: Format of the Annual Pupil Profile Reports

Teachers were asked to rate

the overall format of the

Foundation Stage Annual

Pupil Profile Report. Figure

23 indicates that over 70% of

respondents (71.9%, n = 171)

rated this report as either

excellent or good.

5.5

66.4

23.5

4.6

0

20

40

60

80

100

% o

f Res

pond

ent



79

A small number of teachers who did not rate the format of the Annual Pupil Profile Report


• Contains too much detail for Year 1 pupils (42 comments)

• Certain sections overlapped and could be amalgamated (14 comments)

• Need to be able to adjust box sizes (7 comments)

• Very impersonal (6 comments)

• Too many boxes (4 comments)

• Parents want levels / scores (4 comments)

• Language is difficult for parents to understand - too much jargon (4 comments)

Did the Foundation Stage Annual Pupil Profile Report provide you with a picture of the overall performance of individual pupils? 65.4% of teachers (n = 159) believed that the Foundation Stage Annual Pupil Profile

Report provided them with a picture of individual pupils performance. Just over a third of

teachers (34.6%, n = 84) disagreed and provided the following reasons for their answer.

• Report was very general (19 comments)

• Report was too positive and could be misleading to parents (18 comments)

• Each report is too similar / impersonal (13 comments)

• Comments had to be adapted to make them specific to individual children

(11 comments)

• Teachers are already aware of how children are performing (10 comments)

• Pupil Profile is no better than original school report (9 comments)

• Does not provide a true overall picture of the child (6 comments)

• Additional comments based on teachers professional judgement were more useful

(6 comments)


80


Teachers were asked if they thought the Annual Pupil Profile Reports inform measures to

improve whole school performance. Only 22.3% of respondents (n = 53) believed that the

Annual Pupil Profile Report achieved this. Teachers who felt the report did not inform

measures to improve overall school performance felt it did not accomplish this for the

following reasons.

• Report focused on what the child could do and did not mention areas for improvement

(7 comments)

• Comments are too general (4 comments)

• Did not reflect whole school performance (3 comments)

• To improve school performance standardised testing is required (2 comments)

• Foundation stage comment bank was too broad (2 comments)

Do the Annual Pupil Profile Reports inform teachers’ planning for learning and teaching? Less that 50% of teachers (48.8%, n = 117) felt that the Annual Pupil Profile Reports

inform teachers planning. Comments from those who did not believe the report informed

teachers’ planning are summarised below.

• Observations are taken throughout the year and are more helpful (9 comments)

• There is no difference from the previous school reports (7 comments)

• Depends on how the teacher uses the comment banks (5 comments)

• Too time consuming for teachers to read (4 comments)

A small number of teachers who felt that the Annual Pupil Profile Report does inform

teacher planning commented that the ‘Focus for Development’ section of the report is

particularly useful for the next year’s teacher (n = 36).


81

Do the Annual Pupil Profile Reports inform parents of how they can support their child’s learning at home? 71.2% of teachers (n = 168) believed that the Annual Pupil Profile Report informs parents

of how they can support their child’s learning at home. A sizeable number of such

respondents felt that the ‘Focus for Development’ section within the report is particularly

useful for parents (n = 62). Comments from teachers who believed the report does not

inform parents of how they can support their child’s learning are summarised below.

• Previous school reports provided the same information (10 comments)

• One to one interviews inform parents of how to support their child (9 comments)

• Depends on level of parental interest (7 comments)

• Reports are not specific enough (7 comments)

• Focus for Development section is too small (7 comments)

• Insufficient information on areas the child needs to improve - too positive (4 comments)

• Comments to give parents more direction on how to help would be useful

(4 comments)


83.9% of teachers (n = 203) believed that the Annual Pupil Profile Reports informed pupils

of their achievements. A small number of such teachers commented that the ‘Interests

and Strengths’ section of the report clearly informed pupils of their achievements (n = 9).

A small number of teachers also stated that it is important to highlight pupils’ strengths

(n = 7).


82

Teachers who felt that the report did not inform pupils of their achievements made the

following comments.

• Pupils are too young to understand this report – it is too complicated (16 comments)

• Parents inform pupils of their strengths (15 comments)

• All reports inform pupils of their strengths (11 comments)

• All comments are worded positively - gives a false impression (9 comments)

• Children are already aware of their strengths - informed throughout the year

(7 comments)

• Report over emphasises strengths leaving little scope for development (6 comments)

Teachers were also asked if they thought the Annual Pupil Profile Report informed pupils

of how they can improve. 71.2% of respondents (n = 163) felt that the report did advise

pupils of how they can improve. A small number of such respondents stated that the

Focus for Development section advised pupils of how they could improve (n = 27).

Additional Comments


questionnaire. The majority of these responses have already been captured in the main

body of the report, however a further summary is provided below.

Teachers raised a number of concerns. The largest concern expressed by respondents

(n = 77) was regarding the large amount of time required to complete the Annual Pupil

Profile Reports. A number of respondents felt preparing reports was an additional source

of stress on teachers (n = 18) and that has reduced planning and teaching time (n = 17).

A number of Year 1 teachers stated that they would like the facility to complete the reports

from home (n = 40) and requested sub cover (n = 27) to allow for this option.

A small number of teachers stated that the reports were impersonal (n = 22) did not give a

clear indication of the difficulties children experienced (n = 15) or effectively detail their

progress (n = 12).


83

4.6 Results collected from Year 1 Parent Questionnaires

This section presents the results for each question from the Year 1 parent questionnaire.


to the number of respondents who answered each question, which may be fewer than the

overall total 3631. A number of parents reiterated comments throughout their


report.


71.5% of parents of Year 1 children (n = 2465) received information from their child’s

school about the introduction of the Annual Pupil Profile Report. A large number of

parents indicated that they received this information from either a school meeting (n = 162)

or a letter (n = 81).

Figure 24: Format of the Pupil Profile Report

Parents were asked to rate the format

of the Foundation Stage Annual Pupil

Profile Report. Figure 24 illustrates

that the majority of parents rated this

report highly (96.6%, n = 3467).

Parents who rated the report highly felt

the report was;

• Clearly laid out (405 comments);

• Detailed and informative (323 comments);

• Covered a wide range of topics that were broken down well (220 comments);

• Easy to read (115 comments);

• Highlighted strengths and weaknesses effectively (66 comments);

• Personal to the child (35 comments); and

• Provided a good breakdown of the year’s activities (32 comments).

56.3

40.3

2.4 1

0

20

40

60

80

100

% o

f Res

pond

ents



84

A large number of comments were also received from parents who did not rate the Annual

Pupil Profile Report highly, such parents felt that the report;

• Contained impersonal/computer generated comments (68 comments);

• Should provide a comparison to their child’s peer group (49 comments);

• Contained too many words that were not informative (34 comments);

• Should include grades (33 comments); and

• Should highlight areas for improvement, not just strengths (28 comments).

Did you find the information within the Annual Pupil Profile Report easy to read and understand?

The majority of parents indicated that they found their child’s Annual Pupil Profile Report

easy to read (98%, n = 3506) and understand (96.5%, n = 3344). A large number of

parents made positive comments regarding their child’s Annual Pupil Profile Report, these


• Report was well set out (170 comments)

• Report included very detailed and useful information (155 comments)

• Language was parent friendly (124 comments)

• Information was accessible as subjects were commented on separately (67 comments)

• All aspects of pupil development were covered (40 comments)

• Gave a good understanding of the child’s progress (39 comments)

A number of parents highlighted aspects of the Annual Pupil Profile Report they disliked.

• Too much educational terminology – difficult to understanding (67 comments)

• Reports were impersonal and too general (57 comments)

• Difficult to take in so much information (46 comments)

• Provided no indication of the child’s stage of development (41 comments)

• Should give a comparison to child’s peers (17 comments)


85

How informative are the Annual Pupil Profile Reports? The majority of parents agreed that the Annual Pupil Profile Report informed them of their

child’s achievements (96.5%, n = 3466) and how their child could improve

(88%, n = 3097). One parents described the report as “a very well rounded report”.

Many of the parents who indicated that the Annual Pupil Profile Report did not inform them

of their child’s achievements or areas for improvement provided reasons for their answer.

• Report was too generic and vague (86 comments)

• Report did not advise how children could improve their weaknesses (72 comments)

• Report highlighted what a child is achieving not what they should be achieving

(55 comments)

• Comments were too positive - no focus on weakness (46 comments)

• Grades would highlight areas in need of improvement (24 comments)

• Focus for Development box does not provide enough information (22 comments) Figure 25: Focus for Development

Figure 25 illustrates that over

81% of parents

(81.1%, n = 2839) believed that

the Focus for Development

section of the Annual Pupil

Profile Report advised them of

how they could support their

child’s learning at home.

A large number of parents stated that this section of the report clearly identified areas

which they should work on with their child at home (n = 534).

Comments from parents who did not find the Focus for Development section of the report

useful are listed below.

81.1

18.9

0

20

40

60

80

100

% o

f Res

pond

ents

Yes No


86

• It did not advise how child’s learning could be supported at home (122 comments)

• Comments were too general and therefore not meaningful (98 comments)

• This section was excluded from the report received (48 comments)

• Information was very brief/limited (32 comments)

• Information seemed to be for school rather than home use (18 comments)

Figure 26: Do you plan to discuss your child’s report with them? The majority of parents

(86.5%, n = 3102) indicated

that they plan to discuss their

child’s Annual Pupil Profile

Report with them. Many of

such parents stated that they

plan to discuss this report

with their child so that they

can;

• Praise their achievements (355 comments);

• Encourage their child’s learning (170 comments);

• Positively inform their child of were they can improve (124 comments);

• Inform their child of their own progress and development (84 comments); and

• Discuss each topic individually (14 comments).

A sizeable number of parents who indicated that they do not plan to discuss their child’s

Annual Pupil Profile Report with them felt that the report was too complex for their child to

understand.

86.5

13.5

0

20

40

60

80

100

% o

f Res

pond

ents

Yes No


87

What did you like about your child’s Annual Pupil Profile Report?

Parents were asked to indicate what they liked most about their child’s Annual Pupil Profile

Report. The most prevalent responses from parents are summarised below.

• Well detailed and informative (719 comments)

• Clear format that is easy to read (680 comments)

• Detailed breakdown of the curriculum areas (603 comments)

• Clearly indicates progress and development (350 comments)

• Advice on home learning and areas of weakness (329 comments)

• Easy to understand (258 comments)

• Highlights child’s abilities and achievements (171 comments)

• Positive language used (151 comments)

• Focuses on areas beyond the academic (108 comments)

• Personal to the child (170 comments)

What did you dislike about your child’s Annual Pupil Profile Report?

Parents were asked to indicate what they disliked about their child’s Annual Pupil Profile

Report. The most frequent responses are summarised below.

• Need comparison to peers - below or above average (204 comments)

• Impersonal and clinical (172 comments)

• Grades would provide better information (132 comments)

• Non-specific and general (119 comments)

• Too long and complex for this age group (82 comments)

• No indication given on how to aid learning at home (79 comments)

• Unclear as to what level a child is achieving (67 comments)

• Comments are overly positive and could be misleading (36 comments)

• Jargon used is difficult to understand (33 comments)


88

Additional Comments Respondents were given the opportunity to make additional comments at the end of the

questionnaire. A large number of respondents felt that the Annual Pupil Profile Report was

detailed and comprehensive (n = 70) and clearly indicated the child’s progress (n = 53). A

number of parents also indicated that they liked the layout of the report (n = 43). Certain

parents felt that the new report format identified strengths and weaknesses (n = 18). A

small number of parents highlighted that they appreciated the time and effort devoted to

the reports (n = 14) with one parent commenting:

“I appreciate all the work the teacher puts into our child’s learning. The report gave

me an understanding of my child’s progress and development throughout the year.

The school provides great friendship and support towards our children and are

there for each parent’s concerns”.

However, parents raised a number of issues. Firstly, a large number of respondents were

dissatisfied that the Annual Pupil Profile Report did not contain grades/scores (n = 163),

which they felt made it difficult to gauge their child’s ability (n = 90). A significant number

of parents found the report impersonal (n = 86) and complex (n = 27) with an overuse of

formal, technical language (n = 29). Some parents (n = 73) were worried that valuable

teaching time was being lost due to emphasis placed on preparing the reports. A small

number of respondents also felt that the report was too positive (n = 26) and did not

provide a balanced picture for parents.

Parents indicated that they would like.

• More specific information is on how they can support learning at home (65 comments)

• More regular meetings with their teachers (28 comments)

• Parents would like more regular reports throughout the school year to help monitor

their child’s progress (23 comments)


89

5.0 Conclusion This evaluation has identified a number of issues within the Primary Education sector.

Many Primary schools felt pressured from a combination of demands from government

agencies and parents. In some cases this is exacerbated by the perception that the new

assessment and reporting arrangements are being introduced at a demanding pace.

Feedback collected from earlier phases of evaluation on InCAS has been very positive.

The majority of teachers who participated within previous evaluations felt confident

interpreting feedback and deemed it to be consistent with their professional judgement.

Feedback collected from all stakeholders in June 2008 has been somewhat more

negative. However, problems teachers encountered completing the Annual Pupil Profile

Reports is likely to have escalated their criticisms and concerns about InCAS.

Furthermore the deployment and use of InCAS took place at the same time as a major

transition to a newly contracted C2K managed service in all primary schools. The majority

of concerns raised have been identified in previous evaluations and rectified for the

2008/09 academic year. However, one manageability aspect of InCAS that has not been

highlighted as strongly within previous research was that many respondents do not believe

that Primary Schools have the computer capacity to manage the delivery of InCAS as it

involves more year groups. 8,000 additional laptops have been installed in primary

schools across Northern Ireland. These together with the 5,000 assessment laptops

distributed in 2007 mean there are now over 28,000 devices available for use with InCAS.

However, despite this investment respondents stated that teachers with composite classes

and schools without computer suites have found delivering the InCAS assessments

extremely difficult. Many Principals felt that further ICT investment is required for such

schools and teachers need further support to administer InCAS.

Feedback from this evaluation suggests that some Principals and teachers are unclear

about the nature and status of the Annual Pupil Profile Report. The most widely held

misconceptions were that schools were required to use the Profiles 7 software to complete

reports and that the use of comments in the bank were mandatory. As a consequence

some teachers reported being stressed. In some cases comments were used well, with

parents reporting a positive reaction to the information they received about their children.

However less careful use of comments from the comment banks resulted in reports which

were considered by parents as overly positive, general and impersonal.


90

Some schools report receiving different messages about requirements of revised

assessment and reporting arrangements from education stakeholders.

The implementation of new electronic assessment and reporting arrangements represents

significant change for many Primary Schools in Northern Ireland. CCEA provided schools

with a variety of support and guidance in the form of; training materials, telephone

helpdesk, a Pupil Profile section on the Northern Ireland Curriculum website, online

tutorials and an InCAS pupil demonstration. Feedback from this year’s consultation

indicates that some schools did not use the materials or follow published instructions.

Some teachers within the teacher focus groups felt that they did not have time to make

use of the resources provided.

Both Principals and teachers expressed high levels of dissatisfaction with the Profiles 7

report writing software. Feedback indicates that teachers and Principals believe this

software has significantly increased the amount of time teachers take to produce Annual

Pupil Profile Reports whilst reducing the quality of information. The majority of Principals

and teachers deemed this software to be extremely time consuming to use, under-

developed and have questioned the degree of piloting prior to this year’s implementation.

Only one participant at the Principal and teacher workshops indicated that they intend to

use this software next year.

This evaluation indicates that parents are not entirely satisfied with the new assessment

and reporting arrangements. However, it is important to note that parent’s feelings have

been heightened by the current uncertainty over school transfer arrangements and their

lack of knowledge of InCAS and the Annual Pupil Profile Report. Some parents indicated

that they did not know that their children were being assessed at the beginning of the

school year.

A large number of parents, Principals and teachers expressed concerns that INCAS

assessment outcomes may be used as a method of selection at a future date.

A large number of parents were dissatisfied with the format and content of their child’s

Annual Pupil Profile Report. Although many parents recognised the work required by

teachers to produce this new report there were a number of aspects they criticised.


91

Parents’ main criticism was that it did not contain grades/marks or a comparison with their

child’s peers which they deemed to be an essential component of any school report.

Parents described the Pupil Profile Reports as “impersonal, vague, over generalised” and

as a result were unclear about how their child was progressing. There was also

widespread concern among parents on how ‘hyper-positive’ the Annual Pupil Profile

Reports were. Many parents felt that the report did not provide them with a true account of

their child’s progress. One parent stated:

“[The Annual Pupil Profile Report] was written in politically correct language which

told me nothing. Please encourage teachers to write what they feel and not what

they think ought to be written. More teacher freedom please”.

Some teachers felt uneasy about writing what might be considered less than positive

comments in the reports.

Teachers also have requested advice on how to share InCAS feedback with parents and

direction on comments they may include within the Focus for Development section of the

Annual Pupil Profile Report.

This report has provided a comprehensive overview of this year’s implementation of the

new assessment and reporting arrangements across Primary Schools in Northern Ireland.

Findings from this evaluation show that embedding such new arrangements has required a

large degree of additional work from teachers and Principals. Schools require further

assistance to ensure that they feel adequately supported and empowered to implement

these new arrangements.


92

Limitations

Parents, Principals and teachers have indicated that they are less than satisfied about the

uncertainty about new transfer arrangements. This has impacted on findings obtained

within this evaluation as many were concerned that either InCAS or the Annual Pupil

Profile Report could or would be used to transfer pupils to Post Primary Schools.

A second limitation of the research is the poor response rate to the Principal and teacher

questionnaires. It may be that those most dissatisfied with InCAS and the Annual Pupil

Profile Report completed and returned questionnaires to CCEA. This may have negatively

skewed results but is a common limitation of research.


93

6.0 Recommendations

Before commencing the second roll out of InCAS and the Annual Pupil Profile Report there

are a number of issues documented within this report that should be addressed.

Communication This evaluation suggests that:

• Communications regarding the new assessment and reporting arrangements should be

re-evaluated. All communications on these arrangements should be shared between

all partners (CCEA, DE, C2K and CEM).

• At the start of the next school year CCEA should inform schools which elements of the

Annual Pupil Profile Reports are statutory – it is imperative that such information is

specific to each Year Group.

• CCEA should provide teachers with an ‘at a glance guide’ outlining what elements of

InCAS and the Annual Pupil Profile Report are statutory and flexible.

• CCEA should communicate with parents on the new assessment and reporting

arrangements. InCAS This evaluation suggests that:

• Further work should be carried out to determine whether or not additional computer

capacity may be required to support the use of InCAS within Primary schools.

• CCEA should continue to empower schools to embrace InCAS and develop expertise

and confidence in both administering and analysing feedback produced from InCAS.

CCEA should also reassure schools of the reliability and validity of InCAS.

• Teachers would welcome advice on how to share InCAS feedback with parents.

• Further adaptations to InCAS are needed to meet the needs of pupils with Special

Educational Needs and those with English as an Additional Language.


94

Recommendations continued Profiles 7 Software


• Consideration should be given to whether Profiles 7 is the most appropriate report

writing software or if an alternative should be sought and trialled. Alternatively, Profiles

7 should be developed as a matter of urgency to ensure it better meets the needs of

schools completing the Annual Pupil Profile Report.

• All teachers who plan to use the Profiles 7 report writing software should be provided

with appropriate training.



• Schools should be advised explicitly what the purpose of the Annual Pupil Profile

Reports is and in particular its role (if any) in transfer arrangements.

• Schools require development time to tailor comment banks and ensure all Annual Pupil

Profile Reports are personalised to individual pupils.

• Principals, teachers, pupils and parents would like to see marks/grades to be included

within the Annual Pupil Profile Reports.

• Consideration should be given to moving the ‘Focus for Development’ section towards

the end of the Annual Pupil Profile Report.


95

7.0 Annex

METHOD OF CONSULTATION The roll out of the Pupil Profile Process commenced this academic year with Years 1 and

5. CCEA’s Research & Statistics Unit has undertaken a full evaluation of the

implementation of InCAS and the Annual Pupil Profile Report. This year’s evaluation has

involved consultation with Principals, Year 1 and 5 teachers, Year 5 pupils, and parents

with children in Year 1 and 5. Consultation has been undertaken at various phases of this

academic year and a range of research methods have been employed to collect feedback.

Details of the consultation are provided below.

Phase 1 Consultation:

May 2007 All Year 5 teachers across Northern Ireland

(approximately 1500) were invited to attend awareness

raising training on delivering the Pupil Profile and using

InCAS computer-based assessments. A questionnaire

was used to collect Year 5 teachers’ views and opinions

of this training. A total of 1395 questionnaires were

completed and returned.

Phase 2 Consultation October - November

2007 Before administering InCAS to pupils, all Year 5 teachers

across Northern Ireland were invited to attend training on

understanding InCAS assessment data. A total of 1425

teachers attended this training. Before the training

commenced 80.2% of teachers (n = 1143) completed a

questionnaire on their previous experiences with InCAS. The

aim of this evaluation was to obtain teachers views and

opinions of InCAS which would be used to inform the future

development of the software for 2008/2009.

On completion of the training 80.8% (n = 1151) of teachers

also completed an evaluation questionnaire based on the

days training on ‘Understanding InCAS Assessment Data’.


96

November - December 2007

Following the teacher training on ‘Understanding InCAS

Assessment Data’, teachers were required to administer the

InCAS assessments to their pupils by 31st December 2007.

Questionnaires were posted to all Year 5 teachers

(approximately 1500) and pupils (approximately 25,000) in

Northern Ireland. Questionnaires were completed and

returned by 37% of Year 5 teachers (n = 556) and 59% of

Year 5 pupils (n = 14,011) in Northern Ireland.

The aim of the questionnaires was to evaluate the success of

the implementation of the InCAS Assessments within Year 5

to date and to inform the development of the tool. Phase 3 Consultation

June 2008 Schools are required to produce an Annual Pupil Profile

Report for all Year 1 and 5 pupils by the end of June 2008.

Throughout June, CCEA’s Research & Statistics Unit

consulted will all key stakeholders involved in this process:

Principals, teachers, pupils and parents. Questionnaires

were the main method of collecting data; however a range of

qualitative methods were also used. A sample of

stakeholders participated in research seminars and focus

groups. Details of these methods are documented below.

Seminars & Focus Groups

Throughout June 2008, CCEA’s Research & Statistics Unit

facilitated; eight Principal workshops, ten teacher workshops

and 13 parent focus groups. The purpose of the workshops

and focus groups was to collect rich in-depth information to

supplement data collected from questionnaires. Two

members of the Research & Statistics Unit facilitated

discussions at these consultations and a representative from

the Pupil Profile Implementation Team was present to answer

any questions that arose throughout the course of the day.

final evaluation of incas and the annual pupil profile report

Documents