REF 2014 and the Business & Management community

Mike PiddChair: sub-panel C19 (Business & Mgt Studies)Department of Management ScienceLancaster University Management SchoolUKM.Pidd@lancaster.ac.uk

Overview Purposes of the REF

(the Research Excellence Framework)

REF versus RAE 2008 Panels Staffing

Outputs Environment Impact Timetable

2

You can always blame the REF

REF purposes: official statement

FUNDING: inform research funding allocations by the four UK HE funding bodies (approximately £2 billion per year)

ACCOUNTABILITY: for public funding of research so as to demonstrate its benefits

QUALITY INDICATORS: provide benchmarks and reputational yardsticks

3

REF principles: official statement Expert, peer review panels Equity: all types of research and forms

of output across all disciplines shall be assessed on an equal basis

Equality: HEIs encouraged to submit the work of all their excellent researchers

Transparency: assessment criteria and procedures, and outcomes to be published in full

4

5

RAE 2008: how did it work?

Researchenvironment (20%)

Esteem & impact indicators (10%)

Quality profile

Weighted and aggregated across each submission

e.g.Research incomePhD studentsStaff development

Quality level 4* 3* 2* 1* u/c

% research activity 10 40 35 15 0

Nossex Busines

s School

Research outputs: 4 per person (70%) Peer

review

6

REF 2014: how will it work?

Researchenvironment (15%)

Impact (20%)

Quality profile

Weighted and aggregated across each submission

e.g.Research incomePhD studentsStaff development

Quality level 4* 3* 2* 1* u/c

% research activity 21 38 37 4 0

Nossex Busines

s School

Research outputs: 4 per person (65%) Peer

review

Spot thedifferences

Overview of REF assessment framework

7

Overall quality

Outputs

Maximum of 4 outputs per researcher

Impact

Template and case studies

Environment

Template and data

65% 20% 15%

Key changes since RAE 2008 Includes assessment of impact Removed esteem as a distinct element Structured approach to research environment Fewer UOAs/panels operating more consistently

Only 4 main panels (main panel C for BMS) No separate panel for Accounting & Finance

Strengthened equality and diversity measures Revised eligibility criteria for staff Some UOAs will make (limited) use of citation

data ... But not in BMS (C19)

Increased ‘user’ input on panels; and an integrated role for additional assessors

Publication of overall quality profiles in 1% steps

8

Panels and sub-panels

9

REF 2014: roles of panels MAIN Panel C

Panel criteria & working methods Oversight during assessment process Consistent application of overall assessment

standards Sign off final profiles

SUB-PANEL C19: Business & management studies Limited, specific variations in criteria &

working methods Conduct the assessment Recommend final profiles

Add extra assessors & users during 2012/13

10

Structure of main panel C: REF 2014

11

Janet Finch Chair Manchester/Keele

Cara Aitcheson Sports-related studies Edinburgh

Hastings Donnon Anthropology & dev studies Queens Belfast

Gillian Douglas Law Cardiff

Colin Hay Politics & int studies Sheffield

Peter Neary Economics etc Oxford

Alan Penn Architecture, built env & planning

UCL

Mike Pidd Business & mgt studies Lancaster

Andrew Pollard Education Inst for Education

Keith Richards Geog, env studies & archaeology

Cambridge

John Scott Sociology Plymouth

Peter Taylor Gooby

Social work & social policy Kent

UK academic members (sub-panel Chairs):

Structure of main panel C: REF 2014

12

Trevor Barnes Geography UBC

Frans Berkhout Innovation & sustainability

VU Amsterdam

Francois Borguignon

Economics Paris School of Economics (prev Chief Econ, World Bank)

Paul Finch Architecture & planning

Commission for the Built Environment

Herbert Kritzer Law & politics Univ Minnesota

Jone Pearce Business & mgt studies

UC Irvine

Mark Robson Various Bank of England

Sue Rossiter Education NFER

Martin Walsh Charity sector Oxfam

Paul Wiles Social work & social policy

Oxford (formerly Home Office)

Sharon Witherspoon

Social policy Nuffield Foundation

International and user members

Sub-panel C19: business & mgt studiesMike Pidd1 Lancaster Chair & member of Main Panel C

John Arnold Sheffield Martin Laffin Durham

Jan Bebbington St Andrews Alan Marsden Ex-Arup

David Blackaby2

Swansea Kathrin Moeslin

Erlangen-Nuremberg

Jane Broadbent3 Roehampton Peter Naude Manchester

Robert Blackburn

Kingston Andy Neely Cambridge/Cranfield

Chris Brooks Reading Caroline Oades

ACCA

Colin Eden Strathclyde Richard Thorpe

Leeds

Paul Edwards Birmingham Ian Tonks Bath

Guy Fitzgerald Brunel Caroline Tynan Nottingham

Keith Glaister Sheffield Terry Williams Hull

Mark Jenkins Cranfield Hugh Willmott Cardiff

13

2: Joint with Economics sub-panel 3: Deputy Chair

Staff eligibility & other rules

14

Staff: Eligibility & requirements Category A staff

Academic staff with a contract of at least 0.2 FTE, on payroll of the HEI 31 Oct 2013 primary employment function of either ‘research

only’ or ‘teaching and research’ Category C staff:

Employed by an organisation other than an HEI Have contract or job role including research, Research primarily focused in the submitting unit on

31 Oct 2013 Will contribute to the quality profile but not volume

for funding Research assistants are only eligible by

exception Institutions MUST produce an Equality &

Diversity Code Four outputs per person unless special

circumstances

15

Staff: Individual circumstances

Clearly defined (rules)

Qualified as ECR Part-timers Maternity, paternity or

adoption leave Secondments of career

breaks outside HE

Complex (EDAP) Disability Ill health or injury Mental health conditions Constraints relating to

pregnancy or maternity in addition to maternity leave

Childcare or other caring responsibilities

Gender reassignment Other circumstances

relating to characteristics protected by the Equality Act 2010

Reduced # outputs allowed

16

Early career researchers

Date at which the individual first met the definition of an ECR:

Outputs may be reduced by up to:

Between 1 August 2009 and 31 July 2010 1

Between 1 August 2010 and 31 July 2011 2

After 1 August 2011 3

Part-timers and career breaks

Equiv months absent from work (1 Jan 2008 to 31 Oct 2013):

For part-time staff, average FTE worked (1 Jan 2008 to 31 Oct 2013):

Outputs may be reduced by up to:

14 – 27.99 0.601 – 0.8 1

28 – 48.99 0.301 – 0.6 2

49 or more 0.3 or less 3

Output tariffs: individual circumstances

17

NOTE: % availability need not be same as % used in Volume calculation

Other individual circumstances Maternity, paternity & adoption

Outputs may be reduced by 1 for each period of statutory leave over the REF period.

Also by 1 for each extra period of such leave lasting 4 months or more.

Complex circumstances REF EDAP (Equality & Diversity Advisory

Panel) Recommend any reductions to main panel

Chair, who will decide on any reductions Sub-panels

Must abide by main panel Chair’s decision Will not be told of the circumstances

18

Outputs

19

Eligibility of outputs Must be a product of research (see

REF defn) All forms of output are welcome and

will be treated equally Into public domain between 1/1/08

and 31/12/13 BUT may include pre-published work

(e.g. online first or working paper if (and only if) Pre-published in 2007 but in print form after

1/1/08 Not included in RAE 2008.

20

Co-authored outputs Recognised to be increasing as collaboration

grows in Main Panel C disciplines Where outputs are co-authored/produced,

submitting author expected to have made a substantial contribution

Sub-panels wish to receive the fullest picture of a submitting HEI’s research, hence Co-authored/produced outputs may be submitted

twice in the same UOA by the same HEI MUST be accompanied by a statement explaining

the substantial and distinctive contribution of each of the submitting authors

Still unclear about pre-published, co-authored work if not used by submitting author/institution in RAE 2008.

21

Double weighted outputs May be requested for

outputs of extended scale & scope

MUST have supporting statement

Sub-panels will assess the request separately from its quality

If accepted: counts for 2 items

1 reserve allowed for each double weighing request

22

Environment

23

Vitality: reflects the existence of a thriving, dynamic and fully participatory research culture based on a clearly articulated research strategy, displayed both within the submitting unit and in its wider contributions, and in terms appropriate to the scale and diversity of the research activity that it supports 

Sustainability: understood in terms of the extent to which it is capable in the future of continuing to support and develop such research activity as defined in the quality levels, both within the submitted unit and the discipline more generally

Environment assessment criteria

24

Research environment: 2 aspects1. Narrative (template) with 5 sections

i. Overviewii. Research strategyiii. Staffing strategy & development, including

research studentsiv. Income, infrastructure & facilities v. Collaboration & contribution to the

discipline

2. Quantitative data: based on HESA stats: e.g. doctoral degrees, income ...

25

Impact

26

Impact defined An effect on, change or benefit to the economy,

society, culture, public policy or services, health, the environment or quality of life, beyond academia

Includes an effect, change or benefit to: The activity, attitude, awareness, behaviour,

capacity, opportunity, performance, policy, practice, process or understanding

Of an audience, beneficiary, community, constituency, organisation or individuals

In any geographic location whether locally, regionally, nationally or internationally

Excludes impacts on research or the advancement of academic knowledge within HE; and impacts on teaching or other activities within the submitting HEI

27

Examples of impact: main panel C

28

Impacts on: Examples:

Creativity, culture and society

Enhancements to heritage preservation, conservation and presentationShaping or informing public attitudes and values

Economy, commerce or organisations

Improved products, processes or workplace practicesEnhanced corporate social responsibility policies

EnvironmentChanges in public awareness or behaviourBusiness operations have been changed to achieve environmental objectives

Health and welfare

Development of policy or practice with regard to health services or social care provision

Practitioners and professional services

Influence on professional standards, guidelines or trainingPractitioner debate has been stimulated by research findings

Public policy, law and services

Legislative change or effect on legal practiceInfluence on policy (by government, NGO or private organisation)Impact on democratic participation or access to justice

Research impact submission & criteria

29

Impact templateHow is impact facilitated? Context, approach(es) to impact, strategies & plans, relation to case studies

Impact case

2

Impact case

1

Impact case

3

Impact case

4

Impact case

nCRITERIAReach: understood in terms of the extent and diversity of the communities, environments, individuals, organisations or any other beneficiaries that have benefitted from or been affected (not geographic, but relative to potential domain)Significance: understood in terms of the degree to which the impact has enriched, influenced, informed or changed policies, opportunities, perspectives or practices of communities, individuals or organisations

Maximum of 4 pages per case. Must demonstrate that the unit’s research

made a distinctive contribution to the impact claimed during the REF period 1/1/08 to 31/12/13

Evidence for the impact claimed must be cited May include qualitative, quantitative or material Should be independently verifiable wherever possible

Case studies should not describe activity alone, but should make clear links between activity and impact claimed

Research on which claimed impact is based Must have taken place at the submitting HEI Should meet the definition of 2* (internationally

recognised) Underpinning research published since January 1993

Impact cases30

REF 2014 Timetable31

2011• Panels appointed

(Feb)

• Guidance on submissions (Jul)

• Draft panel criteria for consultation (Jul)

2012• Final panel criteria and methods (Jan)

• HEIs submit codes of practice (Jul)

• Pilot of submission system (Sep)

• Requests for multiple submissions (by Dec)

• Survey of submission intentions (Dec)

2013• Launch REF

submissions system

• Recruit additional assessors

• Staff census date (31 Oct)

• Submissions deadline (29 Nov)

2014• Panels assess

submissions

• Publish outcomes (Dec)

REF 2014: important dates

Date Significance

31/12/13 End of REF period for outputs, impact template & cases and research underpinning impact

29/11/13 Submissions due

31/10/13 Census date for staff

31/7/13 End of REF period for research students, research income & environment

01/01/08 Start of REF period for outputs, environment & impact template & cases

01/01/93 Start of period for research underpinning impact

32

Questions & discussion

Likely sub-panel workloads (based on 2008)

Sub-panel

FTE staff/member

16 Architecture & planning 45

17 Geography & architecture 67

18 Economics & econometrics 47

19 Business & management studies

159

20 Law 100

21 Politics 62

22 Social work & social policy 50

23 Sociology 47

24 Anthropology & development studies

31

25 Education 81

26 Sports-related studies 38

34

Source: REF team

HEFCE QR Funding weighting

4* 3* 2* 1* 0

09/10 based on RAE 2008

7 3 1 0 0

11/12 based on RAE 2008

9 3 0.294 0 0

Based on REF 2014 ? ? ? 0 0

35

Distinct possibility that QR weighting for 2* will be zero after 2014

36

Main

pan

el I: C

hair

David

Otl

ey

34: Economics & econometrics (David Greenaway, Nottingham)

35: Accounting & finance (Andy Stark, MBS)

36: Business & management (Mike Pidd, Lancaster)

37: Library and information mgt(John Feather, Loughborough)

Ray Paul

DavidBlackaby

JaneBroadbent

Structure of main panel I: RAE 2008

Sub-panels did the workMain panel had oversight