Version 1.6

Of interest to everyone in the learning and skills sector,including employers and learners

Framework for Excellence Outputs Guide January 2011

2

0Of interest to everyone in the learning and skills sector, including employers and learners

ContentParagraph number

Introduction 1

Changes made to 2009/10 Framework for Excellence 5

Publication and reporting 9

Queries and complaints 19

Description of the Framework for Excellence in 2009/10 23

Late, missing and insufficient data 27

Performance Indicators

Performance indicator 1: Learner Views 31

Performance indicator 2: Learner Destinations 41

Performance indicator 3: Employer Views 61

Performance indicator 4: Qualification success rates 66

Performance indicator 5: Financial Health 76

Performance indicator 6: Financial Management and Control 95

Annex A: Illustrative examples of revised Framework for Excellence 2009/10 outputs

Annex B: Learner Views

Example of methodology for calculating the score

Annex C: Learner Destinations

i. Destination categories

ii. Scoring methodology

iii. Exemptions and quality threshold overview

Annex D: Employer Views

Scoring methodology

Version 1.6 updated and published on the 10 January 2011.

All hyperlinks were updated and embedded into paragraph copy.

3

Introduction1 The prime purposes of the Framework for Excellence (FfE) are: to provide information

for learners and employers to make informed choices for post-16 education andtraining and to provide consistent management information on key performanceindicators (PIs) for all post-16 providers.

2 Framework for Excellence: Putting the Framework into Practice, (LSC, June 2008),set out our intentions for the implementation of the Framework in 2008/09.Framework for Excellence: Unified Post 16 Performance Assessment, (LSC, July2009), set out the plans for the FfE in 2009/10.

3 Following consultation with main sector representative bodies, changes have beenmade to the way FfE results are calculated and presented to ensure that the datagathered in 2009/10 provides the basis for publishing comparable information toinform learner choice. The changes have affected four of the Performance Indicators(PIs) - Success Rates (SR), Learner Views (LV), Learner Destinations (LD) andEmployer Views (EV) – and ensure that they are fit for purpose and respond to recentchanges in Government policy. The presentation format for the two finance indicatorsFinancial Health (FH) and Financial Management and Control (FMC)) has remainedunchanged, and these PIs have already been communicated to providers by theSkills Funding Agency’s finance teams.

4 The purposes of this document are to:

a. give details of changes made to the calculation and presentation of FfEoutputs for 2009/10

b. help providers to interpret their reports and understand how their scores werecalculated.

Changes made to 2009/10 Framework for Excellence5 The following changes have been made to FfE since the publication of Unified Post-

16 Performance Assessment: FfE Provider Guide 2009/10 (LSC, July 2009):

The removal of grades for the four PIs which will be published: Success Rates,Learner Destinations, Learner Views and Employer Views. They will be shown asungraded scores without correction for contextual factors

a. Scores for each provider will be published alongside the range of scoresobtained for each PI for comparison purposes. As a minimum, thecomparative data will show the lowest score obtained, the median (middlescore) and the highest score. Comparisons against all in-scope providers withvalid scores will be available, as well as data grouped by providers of a similartype. Example reports are shown in Annex A;

b. In the case of SRs there are instances where the range of scores is from 0% -100%. To make information more useful to the public and to comply with dataprotection requirements, where these rates could identify results for individuallearners, the range for SRs will consist of the median (middle score) and the5th and the 95th percentile;

c. Five categories of provider type are used for comparison purposes in 2009/10(as used by Data Service/Statistical First Release):

General FE and Tertiary Colleges

4

Sixth Form Colleges

Specialist Colleges (to include Art & Design/ Land-based /SpecialDesignated Institutions)

Other Public Funded Institutions

Private Sector Public Funded Institutions;

d. The Success Rate indicator has been split into seven sub-categories:

FE long qualifications (excluding A,AS,A2)

FE very short qualifications (<5 weeks)

FE short qualifications (5-24 weeks)

A, AS, A2 qualifications

Apprenticeships (framework completions, overall success rates)

Advanced Apprenticeships (framework completions, overall successrates)

Train to Gain (Full Level 2, Full Level 3 qualifications);

e. The reporting of LV, LD and EV has been amended: where survey resultshave failed to meet the quality thresholds, no score will be given;

f. There has been a change in the scale for scoring Learner Views andEmployer Views for consistency and ease of comparison: these are now bothscored on a scale of 1 to 10;

g. The Department of Education has announced that sixth form colleges will nothave to undertake the Learner Views and Employer Views surveys in 2010/11.Thus, for consistency, the 2009/10 results for these indicators will not bepublished for sixth form colleges although the results will be available toproviders.

6 Supplementary information will be published to show the percentage of learnerscompleting an LSC-funded programme that have progressed into or withinemployment (“the Employment Rate”) or to further learning at the same or higherlevel (“the Learning Rate”). Further supplementary information will include the mostrecent Ofsted inspection grades for Overall Effectiveness and a note to indicate if aprovider has achieved Training Quality Standard (TQS) accreditation.

7 An updated guide to the implementation of the FfE in 2010/11 will be published inOctober as Framework for Excellence Provider Guide 2010/11.

8 For further information on the application of FfE in 2009/10, please see Frameworkfor Excellence: Provider Guide 2009/10 (LSC, September 2009).

Publication and reporting9 In December 2010, the Framework for Excellence outputs (other than Financial

Health and FMC) for 2009/10 will be published on the FfE public comparison website.We will publish the sore for each PI (where available) with supplementary data, forexample data split by age, level and sector subject area, where it is useful and doesnot contravene the Data Protection Act (DPA). Consequently, where publicationwould otherwise enable an individual to be identified, some data will be suppressed.

5

10 We will share the 2009/10 Framework outputs with each provider in October 2010, viathe FfE application on the Provider Gateway to give each provider the opportunity toconsider its own outputs before they are shared widely with the general public.

11 The reports released in October 2010 will be structured to give as much useful dataas possible to providers about their own scores (grades for finance PIs) includinginformation about how the scores/grades were calculated and access todisaggregated data which will be useful for management information and qualityimprovement purposes.

12 Guidance notes in the form of ‘help’ files and access to underlying data are includedin the reports as hyperlinks. Links to the previous year’s tables are also located in theFfE Application.

13 There will be a second release of reports on the Provider Gateway in mid-December2010 to align with the FfE reports which will be available at provider level on thepublic comparison website. These reports will show comparative data for the fourpublished PIs. As a minimum, the comparative data will show the lowest scoreobtained, the median (middle score) and the highest score. Data showingcomparisons against all in-scope providers with valid scores will be available, as wellas data grouped by providers of a similar type. SR reports posted on the ProviderGateway will also show the 5th, 25th, median, 75th and 95th percentile of the range.

14 For Success Rates, interpretative information will be based on the appropriate subsetof FfE providers with greater than one learning aim for the relevant qualification typeand level of breakdown.

15 Examples of provider reports and those that will be available on the publiccomparison website are included in Annex A.

16 It is important that these SR reports are not confused with National Success Rates.National Success Rate Tables differ from the FfE information which is reported andpublished as they focus specifically on the statistical measures derived from theproviders’ Learner Responsive Individualised Learner Record (ILR) with the purposeof providers assessing their performance in relation to the success, retention, andachievement rates of their learners against National Averages. The Framework forExcellence SR information encompasses Learner Responsive and EmployerResponsive SRs. FfE also incorporates a wider scope of Key Performance Indicatorsincluding; the views of learners and employers, and learner destinations, which will bepublished to allow learners and employers to make comparisons between providers.

17 The detail of FfE publication and reporting has been developed in collaboration withnational stakeholders and provider representative bodies in order to ensure thatfuture reporting is consistent across the sector and aligned with the CoalitionGovernment’s programme to empower customer choice in accessing publiclysupported services.

18 In future years, the FfE will take account of the provider-led work on the consistentpresentation of information to inform learner and employer choice that is beingdeveloped by the sector.

Queries and Complaints19 Providers are encouraged to contact their Skills Funding Agency Account Manager or

YPLA regional contact in the first instance if they have any questions about their FfE

6

scores (or finance grades). Any other queries should be directed to the FfE helpdeskby email or telephone 02476 823970.

20 Queries which relate to ILR base data and associated success rates shouldforwarded directly to the Data Service.

21 Any queries must be raised by 15 November 2010 to ensure that we have sufficienttime to deal with them prior to publication. Any scores that are subject to anoutstanding query or complaint will not be published.

22 The Skills Funding Agency aims to manage all queries to provider satisfaction.Formal complaints can be made in accordance with the Agency’s complaintsprocedure available on our website.

Description of the Framework for Excellence in 2009/1023 The FfE consists of six performance indicators:

a. Learner Views

b. Learner Destinations

c. Employer Views

d. Success Rates

e. Financial Health

f. Financial Management and Control

24 Supporting information is published on the Employment Rate, Learning Rate andTQS accreditation where available, as well as the latest Ofsted inspection grade foroverall effectiveness.

25 To take account of the diversity of provider types and missions across the FE system,a set of rules has been developed to determine which indicators apply to which typeof provider. These are outlined in Table 1.

26 Within the broad groupings in Table 1, the extent to which PIs will apply will dependon the individual provider’s characteristics (see Table 2).

Table 1: Performance indicators applying to main provider categories in scope in2009/10

Providergrouping

Lea

rne

rV

iew

s

Lea

rne

rD

est

ina

tio

ns

Em

plo

yer

Vie

ws1

Su

cces

sR

ates

Fin

an

cia

lH

eal

th

FM

C2

Colleges3 Yes Yes Yes Yes Yes Yes

Sixth FormColleges No Yes No Yes Yes Yes

SpecialistColleges4 Yes Yes Yes Yes Yes Yes

Other PublicFunded

InstitutionsYes Yes Yes Yes No Yes

Private SectorPublic Funded

InstitutionsYes Yes Yes Yes Yes Yes

Table 2: Applicability of PIs according to provider characteristics

If the provideris…

Lea

rner

Vie

ws

Lea

rner

De

stin

ati

on

s

Em

plo

ye

rV

iew

s

Su

cce

ssR

ate

s

Fin

an

cia

lH

ealt

h

FM

C

An employer onlytraining its own

staffYes No No Yes Yes Yes

An employertraining employees

of otherorganisations

Yes Yes Yes Yes Yes Yes

Other providers ofEmployer

Responsiveprovision

Yes Yes Yes Yes Yes Yes

A local authorityonly training theirown employees Yes No No Yes No Yes

A local authority Yes Yes Yes Yes No Yes

1 Providers certificated to the TQS on or before 23 March 2010 were not required to complete theEmployer Views survey. TQS accreditation acts as an exemption from the Employer Views survey in2009/10.2 Financial Management and Control3 General FE and Tertiary Colleges4 Including Art & Design, Land-based and Special Designated Institutions

2

If the provideris…

Le

arn

er

Vie

ws

Le

arn

er

Des

tin

atio

ns

Em

plo

yer

Vie

ws

Su

cces

sR

ate

s

Fin

anc

ial

He

alth

FM

C

training employeesof other

organisations

Other independentorganisations5

Yes Yes Yes Yes Yes Yes

Co

mp

letin

gIL

R

No No No Yes7 Yes Yes

AnIndependent

Specialist

Provider

for learners

with learning

difficulties

and/or

disabilities6

Not

com

ple

ting

ILR No No No No Yes Yes

5 Including Special Designated Institutions, other providers of adult learner responsive provision thatare not local authorities and Dance and Drama Academies6 For further details about which PIs apply to independent specialist providers see the Addendum toFramework for Excellence Provider Guide 2009:(http://readingroom.lsc.gov.uk/lsc/National/Addendum_to_Framework_for_Excellence_Provide_Guide.pdf)7 If Employer/ Learner Responsive funding

3

Late, missing and insufficient data27 High quality FfE information will be of benefit to providers themselves, their learners,

customers and other users. Existing data collections which underpin FfE form part ofthe funding agreement with providers. The agreement makes clear what action will betaken where problems are encountered with the supply of data.

28 The data for the Success Rate PI is returned routinely through Skills Funding Agencydata collection systems, therefore it is not expected that there will be any missingscores for this PI as a result of a lack of robust data.

29 In the event that returns for other data collections – the learner and employer surveysand learner destinations – are insufficient to calculate scores, we will include astatement to that effect in the publication of FfE outputs.

30 Throughout 2010/11 we will continue to explore ways in which data returns for eachPI can be maximised. On the whole, this will focus on ways in which awareness of thebenefits of fully engaging with FfE can be raised and best communicated. However,we will reaffirm the contractual obligations that all providers should comply with FfErequirements.

Performance indicators

Performance indicator 1: Learner Views

Applicability

31 This survey applied to learners following programmes with providers in scope for FfEin 2009/10 if they were:

a. aged 16–18 taking at least one LSC-funded learning aim

b. adult learners (aged 19 years or over) with

(i) at least one learning aim funded through the Adult Learner Responsivefunding stream (excluding programmes not subject to formula funding), or

(ii) at least one aim funded through the Employer Responsive funding stream

32 The survey applied to learners, as defined above, including those on programmesdelivered by sub-contractors, where the lead provider was in-scope.

Exemptions

33 Providers with fewer than 10 learners between 1 October 2009 and 12 February 2010were exempt from this performance indicator for reasons of statistical reliability.

34 Sixth Form Colleges are exempt from publication of this PI.

Definition

35 The performance measure is a score derived from the questionnaire on learners’views about:

a. the information, advice and guidance they have received from their provider;

b. the quality of teaching and learning on their programme;

4

c. their overall satisfaction with their learning experience;

d. their satisfaction with the level of support available to them from their provider;and

e. whether they have been treated with respect.

36 Learners also had the opportunity to give feedback on whether their provider isresponsive to their views.

Data source

37 The data were obtained from the survey of learners’ views undertaken between 1October 2009 and 12 February 2010. For further details, please see the Frameworkfor Excellence Provider Guide 2008/09, (LSC, September 2009).

38 The questions and online instructions for the 2009/10 survey can be found on ourwebsite.

Calculating the score

39 To see a worked example of the methodology for calculating the score for the LearnerViews survey see Annex B.

40 The dissemination report for the Learner Views survey will contain a number of blankcells for all FfE providers. These will appear without data in order to ensure thatindividual learners cannot be identified from their responses to the Learner Viewssurvey questions and that learner confidentiality is maintained. These reports will beupdated and released later in 2010 once data has been sufficiently suppressed toensure the responses of individuals are protected. At the same time, comparativedata will also be made available that will enable providers to compare theirperformance with that of similar providers and of all providers as a whole.

Performance indicator 2: Learner Destinations

Applicability

41 This indicator included all learners on a priority programme who had completed atleast one relevant qualification in 2007/08. These learners were investing significanttime and energy in a programme of learning that was designed to affect their lives in asignificant way, and they are defined as follows:

a. all learners aged 16–18 at the start of 2007/08;

b. all learners on Apprenticeships and Advanced Apprenticeships;

c. all learners on Train to Gain programmes;

d. all learners on target-bearing Skills for Life programmes; and

e. all adults on programmes contributing to a full Level 2 or full Level 3qualification, as flagged on the Individualised Learner Record (ILR).

Exemptions

42 This PI did not apply to employers delivering training solely to their own employees.See Table 2 for more information on applicability of PIs and Annex C iii for moreinformation on exemptions.

5

Definition

43 The performance measure for Learner Destinations is the percentage of learners onpriority programmes completing an LSC-funded programme in one academic yearand progressing in the next academic year to one of the destinations defined asfollows:

a. Enrolled in priority learning in 2008/0809 with the same highest level oflearning

b. Progressed to learning with a higher level of highest learning aim

c. Remained in employment or self-employment, with improved job security orenhanced career prospects

d. Entered employment or self-employment in 2008/09, having been in learningprior to 2007/08 where the 2007/08 learning had a positive impact

e. Entered employment or self-employment or training in 2008/09, havingpreviously been outside the labour market

f. Not in employment, education or training, but activity category neutral for thepurposes of the measure

g. Learner not tracked into further learning and ILR field L27 prevented furthercontact

h. Learner included in interview sample but no contact made

i. Learner interviewed but did not meet any of the criteria for a positive outcome

Data source

44 This year’s PI relates to the 2008/09 destinations of learners who completed priorityprovision in 2007/08. Destinations were established through a combination of datamatching (including HEFCE records) and telephone interviews.

Calculating the score

45 To see a worked example of the methodology for calculating the score for the Learnerdestinations survey see Annex C ii). A list of the destination categories and theexemptions and quality thresholds can be found in Annex C i) and Annex C iii)respectively.

Employment Rate

46 The employment rate is the percentage of in-scope learners completing an LSC-funded programme in 2007/08 (“the completion year”) that are projected to haveprogressed to one of the following destinations in 2008/09 (“the destination year”):

a. remained in employment or self-employment in the destination year, withimproved job security or enhanced career prospects;

b. entered employment or self-employment in the destination year, having beenin learning prior to the completion year where the completion year’s learninghad a positive impact; or

c. entered employment, self-employment or training in the destination year,having been outside the labour market prior to the completion year.

6

47 These three employment routes to a positive destination are referred to as “positiveemployment destinations” below.

Calculation

48 The evidence for the Employment Rate comes from telephone interviews. Theseinterviews resulted in learners having a destination that was positive, neutral (e.g. gapyear) or neither positive nor neutral.

49 The employment rate is calculated as:

Number of positive employment destinations 100

All interviewed learners (minus those in neutral activities)

x1

50 Corrective weightings are applied to the sample to ensure that it reflects the balanceof the provider’s learners in 2007/08 and statistical tests are applied to ensure that themargin for error on the results is within acceptable limits.

Learning Rate

51 The learning rate is the percentage of in-scope learners completing an LSC-fundedprogramme in 2007/08 (“the completion year”) that are projected to have progressedto one of the following destinations in 2008/09 (“the destination year”):

a. remained in learning with the same highest learning aim, where at least oneaim was a priority learning aim;

b. remained in publicly-funded education or training with a highest learning aimbelow Level 4 that was higher than the highest learning aim in the completionyear; or

c. remained in publicly-funded learning with a highest learning aim of Level 4 orabove.

52 These three progression routes are referred to as “positive learning destinations”.

Calculation

53 The evidence for the learning rate comes from a combination of telephone interviewsand the matching of learners’ completion year data into the destination year datasetsfor further education and training and higher education. Matched learners all havepositive destinations. The total number of learners with a positive learning destinationis therefore a combination of the matched learners plus an estimate of the number ofpositives in the unmatched group (based on interview results).

54 The interviews resulted in learners having a destination that was positive, neutral (e.g.long-term ill-health) or neither positive nor neutral. From this information, a projectionwas made of the number of these three categories in the whole unmatched group.This projection was weighted according to the age band, level of study and gender ofthe interview respondents to avoid distortion.

55 The learning rate is calculated as:

7

(Matched learners + projected positive learning destinations forunmatched learners) 100

All in-scope learners (minus projected neutral destinations forunmatched learners)

x

1

56 Statistical tests are applied to ensure that the margin for error on the projected resultsis within acceptable limits.

Performance indicator 3: Employer ViewsApplicability

57 The Employer Views PI was applicable to any provider in scope for FfE andcontracted to deliver any volume of Train to Gain or Apprenticeship training, and/orreceived at least £30,000 per annum of employer-responsiveness funding.

Exemptions

58 The following providers were exempt from the Employer Views survey:

a. Providers whose 2009/10 allocation for employer responsive funding was lessthan £30,000

b. Employers who only trained their own employees, including employees ofwholly owned subsidiaries, their parent companies and other companieswholly owned by the same parent

c. Providers in possession of TQS accreditation before 23 March 2010 ( livelisting). For detailed information about the TQS, please see the website.

Definition

59 This is a performance measure of each employer’s views of its provider’sresponsiveness. The measure captures employers’ perceptions on the quality of thetraining provided and the responsiveness of the provider to the employer’s needs.The scores calculated for each provider will enable them to make comparisons abouttheir performance against other providers of the same type and against all providerswho completed the survey and passed the necessary quality thresholds.

Data source

60 Data for assessment comes from the collation of employers’ responses to theEmployer Views survey. Options for responding to the survey included:

a. paper-based questionnaires

b. an online survey

c. automated telephone response

d. a telephone top-up survey for providers considered close to achieving targetnumbers.

Calculating the score

61 To see a worked example of how the employer views survey score is calculated, seeAnnex D.

8

Performance indicator 4: Success Rates

Applicability

62 The Success Rate (SR) performance indicator is applicable to all provider typesdelivering learning aims that contribute to qualification success rates (QSRs).

Exemptions

63 There are no exemptions for this PI.

Definition

64 The SR performance indicator is composed of QSRs for seven qualification types.The SRs for the seven types are derived from base qualification success rates by theData Service to produce QSR tables, and exclude Key Skills and Functional Skills.

65 FfE no longer applies any scoring process to QSRs to produce a score or a grade forpublication.

66 The seven qualification categories are:

a. FE long qualifications, excluding A-levels, AS and A2 courses

b. FE short qualifications (5-24 wks)

c. FE very short qualifications (<5 wks)

d. A-levels, including A2 and AS

e. Apprenticeships (overall success rate for framework completions)

f. Advanced Apprenticeships (overall success rate for framework completions)

g. Train to Gain (full level 2 and full level 3 qualifications only).

67 These QSRs are further broken down according to Tier 1 subject sector areas (SSAs)and age- band. Train to Gain provision is categorised by full Level 2/3 rather than byage-band.

Data source

68 The data sources for this PI are Qualification Success Rates (provided by the DataService) derived from 2008/09 ILR data. These rates, together with a facility foraccessing the constituent data, are available through the Provider Gateway. Themethodology for calculating QSRs can be found on the Data Service website.

Data-submission requirements

69 There were no new data submission requirements. Data for the SR PI are routinelycollected through the ILR.

Data refresh

70 It is intended that the Success Rate scores and comparative data derived from the2009/10 ILR will be published in spring 2011 on the FfE Publication website.

Presentation of success rate information

71 For more information on how the information will be reported and published seeAnnex A.

9

Performance indicator 5: Financial HealthApplicability

72 This PI included:

a. All college providers

b. Non-college providers with LSC funding of £50,000 or more

Exemptions

73 The following providers were exempt from the Financial Health PI:

a. Non-departmental public bodies

b. Other public bodies and agencies

c. NHS Trusts, primary care trusts and strategic health authorities

d. Police and fire authorities

e. Major national charities and voluntary organisations whose main source offunding is not the LSC (for example, Red Cross)

f. Established public listed companies and other registered companies for whichtotal LSC contract values are no more than 5 per cent of annual turnover.

Definition

74 Financial Health is a measure of a provider’s financial status in terms of currentfinancial performance and ability to meet ongoing financial commitments.

75 Financial Health is graded, based on the following three elements:

a. current ratio (solvency)

b. operating surplus or deficit as a percentage of turnover/income (sustainability)

c. borrowing as a percentage of certain reserves and debt (status).

76 For definitions of these elements, please see the Framework for Excellence: ProviderGuide 2009/10 (LSC, September 2009).

or the relevant section of the Financial Planning Handbook.

Data source

77 For 2009/10, the data required to calculate the three elements was sourced asfollows:

a. for colleges – from the Finance Record for the year ending 31 July 2009

b. for other providers in scope – from the latest available statutory financialstatements (full accounts or equivalent, not abbreviated accounts).

78 The Financial Health performance indicator involved no additional data burdens.

Scoring and weighting

79 Each element receives a score, up to a maximum of 100 points. Scores for the threeelements are aggregated, and additional points (maximum of 100) for consistent

10

performance are applied to arrive at a total maximum achievable autoscore of 400points.

Step 1 - Initial scoring

80 For each of the elements a score of 0 to 100 points is awarded, based on performanceas shown in Table 3.

Table 3: Scoring for Financial Health ratios

Score Adjusted current ratio Operating surplus as apercentage of income

Borrowing as apercentage of reserves

and debts

0 <0.2 <-4 >=95 or negative

10 >=0.2 >=-4 <95

20 >=0.4 >=-3 <90

30 >=0.6 >=-2 <85

40 >=0.8 >=-1 <80

50 >=1.0 >=0 <75

60 >=1.2 >=1 <60

70 >=1.4 >=2 <45

80 >=1.6 >=3 <30

90 >=1.8 >=4 <15

100 >=2.0 >=5 =0

Step 2 – Recognition of consistency

81 The scores for the three ratios above are aggregated, and a bonus for consistentperformance is added to the subtotal, as shown in Table 4.

Table 4: Recognition of consistency

Number of ratios scoring Bonus points awarded

Two ratios scoring >= 60 Add 50 points

Three ratios scoring >= 60 Add 100 points

82 An initial grade assessment is made by comparing the aggregated points score withthe assessment.

Step 3 – Grading the Financial Health score (with consistency)

83 The resulting total score out of 400 is graded as shown in Table 5.

Table 5: Assessment criteria for Financial Health key performance area

Provider score Provider grade

310-400 points Outstanding

220-300 points Good

120-210 points Satisfactory

11

<=110 points Inadequate

84 General descriptions associated with the four Financial Health grades are shown inTable 6.

Table 6: Financial Health descriptions

Grade Description

Outstanding A provider that has very robust finances to fulfil its contractual obligations and torespond successfully to opportunities or adverse circumstances.

Good A provider that has sufficiently robust finances to fulfil its contractual obligationsand to respond successfully to most opportunities or adverse circumstances.

Satisfactory A provider that appears to have sufficient finances to fulfil its contractualobligations but also appears likely to have limited capacity to respondsuccessfully to opportunities or adverse circumstances.

Inadequate A provider that is in financial difficulty and very likely to be dependent on thegoodwill of others. There is significant risk of providers in this group not beingable to fulfil contractual obligations because of weak Financial Health.

85 Colleges will continue to carry out a self-assessment of their Financial Health inaccordance with the annual guidance published in relation to their financial returns.Non-college providers may also carry out a self-assessment if they wish, but this is nota requirement. The autoscore and the self-assessment can be subject to moderationby provider financial management teams, on a consistent basis, to take account of theapproved policy items listed in the paragraphs below.

Capital uplift (colleges only)

86 It is recognised that it is common for a college’s Financial Health to deteriorate duringthe build period and early post-completion years of a capital project. For collegesundergoing a capital project at their 31 July year end (that is, where 31 July lies withinthe capital project lifecycle, which is defined as date of first claim to the financial year inwhich the project ends, plus three years), then its Financial Health grade will be upliftedto ‘satisfactory’ rather than an autoscore of ‘inadequate’ in any year, provided that:

87 For 2009/10, the data required to calculate the three elements was sourced as follows:

a. the college is graded ‘outstanding’, ‘good’ or ‘satisfactory’ at the time ofdetailed project approval; and

b. if it will return to at least a grade of ‘satisfactory’ by the third year followingproject completion;

c. it performs at least as well (in the opinion of the LSC or its successor body) asforecast in the project proposal during the intervening years. However, if acollege performs less well than it forecast, its grade will reflect this.

88 Under this approach, there will be a reference point within the capital project proposal;this will reduce to a minimum any need for judgement to be exercised and will providea clear basis for validation.

Moderation criteria

89 Moderation is required only in a small proportion of cases. The criteria are as follows:

12

a. A college or provider may have made its case to (the LSC or successorbodies) seeking moderation (by one grade) where a college or other provideroperates with a revolving credit facility, reducing the reported current ratio.

b. In addition, the LSC or successor body may have moderated a grade on thefollowing grounds:

where a college was in receipt of exceptional financial support in-year,this would normally have led to an ‘inadequate’ grade for Financial Healthbeing reported for that year;

where a college was operating with consent for solvency-relatedborrowing in excess of the limits set out in the financial memorandum, thiswould normally have led to an ‘inadequate’ grade for Financial Healthbeing reported for that year; and

where information other than the latest available audited financialstatements, supported by factual evidence, indicated that the FinancialHealth grade was significantly different from the grade implied by theautoscore, a grade may be moderated. ‘Significantly’ is here defined asbeing sufficiently different to generate an auto grade at least one gradelower. Examples may include (but are not limited to):

i. a court ruling that has financial consequences

ii. the loss of a material contract or area of provision

iii. a contingent liability crystallising

iv. recall of debt by the bank, or

v. cessation of trading.

where company directors received payment from their business viadividends with only a minimum salary being reflected in the accounts. Thepayment of dividends will already be reflected in a reduction in the total ofshareholders funds used in the status ratio and should not normally besubtracted from the profit after tax total used in the sustainability ratio.However, if the payment of dividends substantially reduced the profit totalor indeed results in the provider making a loss, then it would bereasonable to include the lower figure in the profit total used as this wouldhave a material effect on the funds available to the provider to invest intheir business in the future.

90 A grade may have been moderated with reference to group/parent company FinancialHealth and any parent company guarantees. (This is applicable mainly to private sectorproviders, and may result in a grade being moderated or may support an increase ordecrease in the level of contracted activity.)

Performance indicator 6: Financial Management and ControlApplicability

91 All provider types that received LSC funding in 2008/09 were in principle subject to theFinancial Management and Control (FMC) PI.

13

Exemptions

92 Non-college providers that received less than £50,000 annually of LSC funding wereexempt and did not have to complete a Financial Management and Control Evaluation(FMCE).

Definition

93 Providers self-assessed and graded their Financial Management and Controlarrangements using the FMCE document. The LSC provided guidance to helpproviders do this in a consistent manner, and the LSC’s audit teams validated theassessments, taking account of the results of audit work at providers and any otherrelevant, available information.

Data definition and source

94 There were two versions of the FMCE return:

Form A was to be completed by further education colleges and specialistdesignated institutions.

Form B was to be completed by all non-college providers that received morethan £50,000 annually from the LSC.

95 In addition to these two forms, there were ‘short’ versions of each where providerscould confirm their previous year’s assessment if their situations had not changed.

Assessment criteria and the assessment process

96 The self-assessment grade within each provider’s latest FMCE document werereviewed by the LSC’s regional audit teams and validated, subject to strict criteria andin line with the LSC’s audit cycle for 2009/10. The deadline for returning the form was30 November 2009 for the full form and 31 December 2009 for the ‘short’ version.

14

Annex A: Illustrative examples of revised Framework for Excellence2009/10 publication and reporting

Provider reports

1 Success Rates, Learner Destinations, Learner Views and Employer Views are nowshown as ungraded scores without correction for contextual factors

2 Scores for each provider will be published alongside the range of scores obtained foreach PI for comparison purposes. As a minimum, the comparative data will show thelowest score obtained, the median (middle score) and the highest score. Comparisonsagainst all in-scope providers with valid scores will be available, as well as datagrouped by providers of a similar type.

3 The reports released in October 2010 will be structured to give as much useful data aspossible to providers about their own scores (grades for finance PIs) includinginformation about how the scores/grades were calculated and access to disaggregateddata which will be useful for management information and quality improvementpurposes.

4 Guidance notes in the form of ‘help’ files and access to underlying data are included inthe reports as hyperlinks. Links to the previous year’s tables are also located in the FfEApplication.

5 There will be a second release of reports on the Provider Gateway in mid-December2010 to align with the FfE reports which will be available at provider level on the publiccomparison website. These reports will show comparative data for the four publishedPIs. As a minimum, the comparative data will show the lowest score obtained, themedian (middle score) and the highest score. Data showing comparisons against all in-scope providers with valid scores will be available, as well as data grouped by providersof a similar type. SR reports posted on the Provider Gateway will also show the 5th,25th, median, 75th and 95th percentile of the range.

NOTE: Comparison data (tables) will not be available until mid-December 2010 when FfEindicators are published on the comparison website. This is to comply with Official Statistics(OS) guidelines on the pre-release of data. Provider reports on the Provider Gateway will beupdated to include comparative data at this time. The illustrations below show the reports intheir entirety.

15

Success Rates report

This page provides headline figures of your success rates which will be published as part of the Frameworkfor Excellence.

For a breakdown of success rates by age and subject sector area, click the appropriate header.

Success rates: overall summary

Help

AAA Training CollegePrivate Limited Company

123456789123456

2009/10

Provider Name:Provider Type:UKPRN:UPIN:Framework for Excellence Year:

Qualification typeFE longcourses(>24 wks)

1,523Starts or Leavers*

Success rate** (%) 68.9

FE shortcourses(5-24wks)

235

82.9

FE veryshortcourses(<5 wks)

632

89.3

A, AS andA2

1,032

71.4

Apprenticeships

2,321

76.3

Advancedapprenticeships

1,425

54.6

Train toGain (FL2and FL3)

854

60.1

Provider data

Providers of a similar type summary data (values as %)

95.4The top organisation scored

The top 5% scored greater than 93.8

94.6

94.5

92.4

92.1

96.7

94.8

93.3

90.2

87.1

87.5

93.1

91.4

The top 25% scored greater than

The median (middle) success rate 78.1 89.3 89.1 89.7 61.3 73.9 71.1

69.6The bottom 25% scored less than

The bottom 5% scored less than 59.9

75.6

64.6

88.1

87.4

87.1

79.3

56.2

43.4

67.6

59.4

61.5

29.6

31.7The bottom organisation scored 63.3 87.1 75.2 38.2 54.1 27.9

All Providers summary data (values as %)

94.2

89.7

95.2

87.7

87.1

86.9

98.8

89.1

80.6

73.7

76.9

70.1

78.5

73.6

78.2

70.9

66.5

53.9

63.9

51.2

67.1

59.3

58.6

30.9

27.3 31.1 54.1 21.2

83.7 91.7 90.7 92.4 88.6 78.8 82.7

97.7

91.2

97.6

97.1

99.1

98.2

75.7 83.6 82.1

68.9

56.8

62.1

52.9

72.7

69.6

27.7 52.7 50.1

81.6 92.2 97.7

The top organisation scored

The top 5% scored greater than

The top 25% scored greater than

The median (middle) success rate

The bottom 25% scored less than

The bottom 5% scored less than

The bottom organisation scored

* Success rates for learner responsive provision are based on the number of starts, whereas success rates for employerresponsiveness are based on the number of leavers. For a more detailed explanation, please see the Help section.

** Where the number of starts or leavers is between 1 and 9, the Success rate is shown as “-“.

Data comparisons by the seven categories shown above are used for the purposes of provider reports only. Comparisoninformation which is published will show the top, middle and bottom success rates only.

Calculation of percentiles – success rates

6 Percentiles for a given provider type and breakdown have been calculated as follows:

a. All providers of a particular provider type with a statistically robust success rate areidentified, and sorted in order of success rate from lowest to highest

b. These providers are numbered from 1 to the maximum present, in ascending order.

c. Having identified the percentile that needs to be calculated, 1 is subtracted from thenumber of providers identified in the group

d. The percentile is multiplied by this number, and 1 added to determine the twoproviders between which the success rate for the percentile is to be derived.

16

e. This identifies the two providers between which the success rate for that percentileis to be derived, by drawing a straight line between these two points, the successrate is read at the corresponding number calculated in step d above.

7 Additional reports will be available showing comparative data by qualification type, splitby age (full level 2 and full level 3 for Train to Gain) and subject sector area.

17

Learner Destinations report

8 Additional reports will be available showing: the breakdown of learners tracked by age,gender and level of study; the destination of learners by code (codes are available inAnnex C); destination by code split by age, gender and level of study.

9 A full report for Employment Rate and Learning Rate will be available if we have beenable to calculate a score for both.

18

Learner Views report

10 Additional reports will be available showing:

a. the breakdown of responses by age, gender and level of study;

b. the spread of responses per question; the spread of responses per questionsplit by age and highest level of study; and;

c. average score split by age and highest level of study.

11 There will be a separate sheet giving a full breakdown of how your score was calculated(more information on the calculation of Learner Views scores in available in Annex B).

19

12 The dissemination report for the Learner Views survey will contain a number of blankpages for all FfE providers. These pages will appear without data in order to ensure thatindividual learners cannot be identified from their responses to the Learner Viewssurvey questions and that learner confidentiality is maintained. These reports will beupdated and released later in 2010 once data has been sufficiently suppressed toensure the responses of individuals are protected. At the same time, comparative datawill also be made available that will enable providers to compare their performance withthat of similar providers and of all providers as a whole.

20

Employer Views report

13 Additional reports will be available showing: the spread of responses to each question;the spread of responses to each question split by Standard Industry Classification (SIC)code (2003).

21

Presentation of scores for publication

14 Scores and some underlying information will be published on a website that can beaccessed by learners, employers and other interested parties later in 2010. Users willbe able to select the PI for which they wish to see scores and supporting information,and be able to view comparisons between providers.

15 The following shows how ‘top level’ information and comparative data will be displayed.

Publication of success rate data

16 Suppression of some scores and underlying data may be required to comply with theData Protection Act and to ensure statistical robustness. The results will be treated asfollows in these cases:

If there are no learning aims for a particular qualification type, the success rate willbe blank and the associated link to all levels of further breakdown by age, full level2/3 or SSA will be disabled;

A provider’s overall success rate for a given qualification type will show “Notavailable” if either:

22

a. The success rate is considered to be not statistically robust, or

b. We are unable to publish the success rate in order to protect the identity oflearners.

17 For statistically robust success rates that could identify learners if published exactly, weare sometimes able to state that the success rate is either above or below a certainpercentage.

18 For success rates that are broken down by age band, and/or Full Level 2 or Level 3and/or SSA Tier 1 where we are unable to show the exact percentage for any of theabove reasons, the following symbols are used:

- = Unable to display for reasons of statistical robustness

* = Unable to display in order to comply with the Data Protection Act

<xx.x% = SR is below xx.x per cent

>xx.x% = SR is above xx.x per cent

19 Where there are only a very small number of FfE providers of the same provider typewith robust success rates at any level, the comparative information will not be shown asthis would not be statistically robust. Comparative information will still be shown for ‘allProviders’.

23

Annex B: Learner Views

1 A report showing how your learner views score was calculated in more detail isavailable via the Provider Gateway. It explains what data was used to calculate yourscore and at what stages. Below is an illustrative example showing the stages that wereused within the calculation.

24

Annex C : Learner Destinations

i) Destination categories

1 This table shows the different categories that were used when classifying thedestinations of individual learners. A provider specific report showing the number oflearner classified as having a destination within each of the below codes is availablevia the FfE Application on the Provider Gateway.

Code Source Description

Code 1m Matching

Code 1i Interview

Enrolled in priority learning in 2008/0809 with the same highestlevel of learning

Code 2m Matching

Code 2i Interview

Code 2h HEFCE

Progressed to learning with a higher level of highest learningaim

Code 3 Interview Remained in employment or self-employment, with improvedjob security or enhanced career prospects

Code 4 InterviewEntered employment or self-employment in 2008/09, havingbeen in learning prior to 2007/08 where the 2007/08 learninghad a positive impact

Code 5 Interview Entered employment or self-employment or training in 2008/09,having previously been outside the labour market

Code 6 Interview Not in employment, education or training, but activity categoryneutral for the purposes of the measure

Code 7 Administrative code only

Code 8 Matching Learner not tracked into further learning and ILR field L27prevented further contact

Code 9 Interview Learner included in interview sample but no contact made

Code 10 Interview Learner interviewed but did not meet any of the criteria for apositive outcome

25

ii) Learner Destinations scoring methodology

iii) Exemptions and quality thresholds overview

Annex D: Employer Views

Calculating the score

1 Each provider was given a target number of valid responses (i.e. responses to thescoring questions) they needed in order to achieve in a valid overall score. Theminimum number they needed was based on one of two criteria, either;

a the number required to achieve a confidence interval of + or – 5%, at 95%confidence level, with an assumed ‘proxy’ response of 84%; or

b at least 70% of employers were interviewed.

2 The criteria giving the lesser number of interviews requires was used for the purposesof calculating a score.

3 In addition to meeting the above requirements, providers were scored if, with 95%confidence level, the true confidence interval was less than or equal to + or – 0.5.

How your score was calculated

4 This table shows how we calculated the Employer Views score and the steps taken toensure the results for all providers were calculated in a fair and consistent way. Anindividual report for each provider is available via the FfE Application on the ProviderGateway.

33

© Skills Funding Agency

Published by the Skills Funding Agency

Extracts from this publication may be reproduced for non-commercial, educational or training purposes on condition that the source is acknowledged and the findings are not misrepresented.

This publication is available in electronic form on the Skills Funding Agency website:

www.bis.gov.uk/skillsfundingagency

If you require this publication in an alternative format or language, please contact the LSC Help Desk: 0870 900 6800. Skills Funding Agency – P – 100066

Skills Funding Agency Cheylesmore House Quinton Road Coventry CV1 2WT T 0845 377 5000 F 024 7682 3675 www.bis.gov.uk/skillsfundingagency