Research & Modern Universities

Modern Universities Research Conference27th March 2012

David Sweeney Director (Research, Innovation & Skills)

• Supporting economic recovery and growth

• Building education and research partnerships in the faster growing economies

• Focusing research efforts

• Maintaining our international reach

• Preparing graduates with a ‘global’ outlook

• Maintaining a rich diversity of higher education institutions

Focus on the bigger picture

• Making these explicit to the Government and wider society

• Creating a level playing field between applied and theoretical work but recognising only impact based on excellent research

• Encouraging institutions to achieve the full potential contribution of their research in future

Assessing research quality

To identify and reward the contribution that high quality research has made to the economy and society:

• A strong and innovative national research base is essential to support national prosperity in a globalised knowledge based economy

• Need to strengthen links between undertaking research and developing new products and services

• Our strategic aim is to develop and sustain a dynamic and internationally competitive research sector that makes a major contribution to economic prosperity, national wellbeing and the expansion and dissemination of knowledge.

National policy

Big ideas for the future

“UK Research that will have a profound effect on our future.”

• In 2003 the CBI noted less than 20% of businesses had links with HEIs. In the CBI Education and Skills Survey 2010, 66% of businesses now have links with HEIs.

• Significant contribution to academic culture change towards knowledge exchange: number of academics with positive attitudes to KE has grown from 61% in 2001 to 76% in 2008.

• Benefits to research and teaching: 48% of academics judged that KE gave them new research insights; 38% of academics judged that KE had helped them improve their presentation style.

University – business links

• 11 UK universities in the World Universities Ranking Top 100 (second only to US)

• UK attracts 15% of all international doctoral students (second only to US)

• 3rd in G8 (behind US and Germany) for production of PhD qualifiers

• UK produces more publications and citations per pound spent on research than other G8 nations

• With 1% world population we produce 6.9% of world publications, receive 10.9% of citations and 13.8% of citations with highest impact.

A successful UK research base

• Universities are funded to build and sustain baseline capacity of high quality

• Undertaking research often chosen by the priorities of the researcher – ground-breaking and innovative ‘blue-skies’ research

• Stable base on which to undertake research commissioned by other funders

• Allows exploration of new areas of research, looking at connections between disciplines, support of early-career staff, doctoral students, support of staff between grants and research facilities

• Expenditure at discretion of the university.

Investing QR for success

• The hopes and fears of Governments around the world revolve presently around issues of economic growth.

• Economic growth is not about helping businesses. It is about working with businesses to replenish our jobs lost in the credit crunch and subsequent recession, and to restore living standards.

• The consequences of such economic failures as we have seen over last years can be profound – affecting not only economies, but societies, communities, cultures and political stability. These are very serious matters of deep and pressing concern.

Economic Growth

• The responsibility of restoring economic growth does not fall solely, or even perhaps primarily, to universities. But universities can and are making a contribution.

• The issues are so serious that all who can do something should – and should do more. This applies particularly to universities that are part of the public fabric to the nation.

• It is anticipated that the jobs of the future will not grow back where they have been lost. University research and knowledge exchange can help identify technologies and other innovations that may grow new industries. The graduates and skills developed in higher education can put these new industries to work.

Whose Problem?

• Universities can build on nearly two decades of progress in working with business, but we believe that they will need to do new things in these urgent circumstances:

• Growing demand for their knowledge exchange, to spur greater innovation and hence growth

• Focussing on the vital R&D intensive companies that we have in the UK – retaining them and helping them diversify and grow

• Attracting companies from abroad to set up in the UK for our ideas and talent; and even working with others to help capture more jobs from these companies’ value chains (such as through knowledge exchange through supply chains, leadership and management and skills development)

What Solution (1)

• Universities are funded to build and sustain baseline capacity of high quality

• Devising and spinning-off new technologies, to create new industries and businesses

• Helping exporters through university global links and capabilities

• Working smarter with SMEs. Many are high-tech, but others may have a range of appetites to grow; so universities should be focussing on SMEs where they can find high-growth potential

• Developing the next generation of innovators and entrepreneurs

• Collaborating, with each other and with business, to create clusters

• Being good partners in open innovation of all forms

What Solution (2)

• And above all, using public funding smartly: this includes prioritising and targeting where the greatest impact can be made, including unlocking private sector leverage.

The Challenge

Benefits of research

• Impacts on patient outcomes, health policy and practice, medical technology and the pharmaceutical industry

Clinical medicine

• Impacts on high-tech products and services, public engagement with science and defence and energy policy

Physics

• Impacts on environmental policy, conservation, managing the environmental, utilities, risks and hazards, exploration of resources, public health

Earth systems & environmental sciences

• Impacts on social policy, public services, third sector, practitioners and public debate

Social work & social policy

• Impacts on creative industries, cultural enrichment, civil society, English as a global product, policy development

English language & literature

Societal contribution (1)

• Economic & commercial – creating wealth, for public good and private gain

• Public policy and services – stimulating public sector innovation as a contribution to growth and quality of life

• Society, culture and creativity – enriching and expanding lives, imaginations and sensibilities while challenging cultural values and social assumptions

• Health and welfare – saving lives and enhancing the quality of life.

Societal contribution (2)

• Production – increasing production, yields or quality; reducing waste

• Practitioners and services – changes to professional standards, guidelines or training; influence on workforce planning

• Environment – influencing the policy debate on climate change or other environmental policy issues

• International development – influencing international policy development or international agencies or institutions; quality of life improved in a developing country

• Education – influencing the form or the content of the education of any age group in any part of the world.

• Widespread acceptance of the principle of incorporating impact in the REF, and agreement that the impact assessment should:

• Be based on expert review

• Review historical impacts, not predict future impact

• Focus on the impact of submitted units’ research, not individual researchers

• Be underpinned by high quality research

• Take a wide view of impact, inclusive of all disciplines.

Impact: Initial Consultations

Impact: CriteriaThe criteria for assessing impacts are reach and significance*

Four star Outstanding impacts in terms of their reach and significance

Three star Very considerable impacts in terms of their reach and significance

Two star Considerable impacts in terms of their reach and significance

One star Recognised but modest impacts in terms of their reach and significance

UnclassifiedThe impact is of little or no reach and significance; or the impact was not eligible; or the impact was not underpinned by excellent research produced by the submitted unit

* Each main panel provides a descriptive account of the criteria

• An effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia

• Impact includes an effect, change or benefit to:

• The activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding

• Of an audience, beneficiary, community, constituency, organisation or individuals

• In any geographic location whether locally, regionally, nationally or internationally

• It excludes impacts on research or the advancement of academic knowledge within HE; and impacts on teaching or other activities within the submitting HEI

Impact: Definition for the REF

Impact: Submissions

Impact template (REF3a)

•Sets out the submitted unit’s general approach to enabling impact from its research•One template per submission – with a page limit depending on the number of staff submitted•Covers the period 1 Jan 2008 to 31 Jul 2013•Contributes 20% to the impact sub-profile

Case studies (REF3b)

•Specific examples of impacts that were underpinned by the submitted unit’s research•The number of case studies required depends on the number of staff submitted•Impacts during 1 Jan 2008 to 31 Jul 2013; underpinned by research since 1 Jan 1993•Contributes 80% to the impact sub-profile

• The unit’s approach to enabling impact from its research:

• Context for the approach

• The unit’s approach during 2008-2013

• Strategy and plans for supporting impact

• Relationship to the submitted case studies

• Provides additional information and context for the case studies, and can take account of particular circumstances that may have constrained a unit’s selection of case studies

• To be assessed in terms of the extent to which the unit’s approach is conducive to achieving impact of ‘reach and significance’

Impact: Template (REF3a)

• In each case study, the impact described must:

• Meet the REF definition of impact

• Have occurred between 1 Jan 2008 and 31 July 2013 (can be at any stage of maturity)

• Be underpinned by excellent research (at least 2* quality) produced by the submitting unit between 1 Jan 1993 to 31 Dec 2013

• Submitted case studies need not be representative of activity across the unit: pick the strongest examples

Impact: Case studies (REF3b)

• Each case study is limited to 4 pages and must:

• Describe the underpinning research produced by the submitting unit

• Reference one or more key outputs and provide evidence of the quality of the research

• Explain how the research made a ‘material and distinct’ contribution to the impact (there are many ways in which this may have taken place)

• Explain and provide appropriate evidence of the nature and extent of the impact: Who / what was affected? How were they affected? When?

• Provide independent sources that could be used to verify claims about the impact (on a sample audit basis)

Impact: Case studies (REF3b)

The Research Excellence Framework

Assessment framework,

guidance on submissions

and panel criteria

Presentation outline

• Introduction

• Equality and diversity

• Outputs

• Impact

• Environment

• REF panels

Purpose of the REF

The REF replaces the RAE as the UK-wide framework for assessing research in all disciplines. Its purpose is:

• To inform research funding allocations by the four UK HE funding bodies (approximately £2 billion per year)

• Provide accountability for public funding of research and demonstrate its benefits

• To provide benchmarks and reputational yardsticks

Introduction:

The assessment framework

Overall quality

Outputs

Maximum of 4 outputs per researcher

Impact

Impact template and case studies

Environment

Environment data and template

65% 20% 15%

Introduction:

Guidance and criteria

Comprehensive information and guidance is set out in:

• Assessment framework and guidance on submissions (July 2011):

- Sets out the information required in submissions and the definitions used

• Panel criteria and working methods (Jan 2012):

- Sets out how panels will assess submissions

- Refined following consultation and panel meetings during 2011

Introduction:

Staff selection

• HEIs are responsible for selecting staff whose work is to be included in their REF submissions

• Each HEI is required to develop, document and apply a code of practice on fair and transparent staff selection:

- Must be signed off by the head of the institution and submitted to the REF team by 31 July 2012 at the latest

- The REF Equality and Diversity Advisory Panel will examine these for adherence to the guidance

- They will be published at the end of the assessment process

Equality and diversity:

Codes of practice

• Codes should demonstrate fairness to staff by addressing the principles of:

- Transparency: clearly setting out the procedures for staff selection, and communicating these to all eligible staff

- Consistency: applying consistent procedures across the institution

- Accountability: clearly defining responsibilities for decisions, with appropriate training for those involved

- Inclusivity: promoting an inclusive environment, with robust procedures for staff to disclose individual circumstances

Equality and diversity:

Individual staff circumstances • Up to four outputs must be listed against each member of

staff

• This can be reduced without penalty where an individual’s circumstances have constrained their ability to work productively or produce four outputs during the REF period:

- A wide range of circumstances will be taken into account

- With as much clarity as possible about the permitted reductions

- To be treated consistently across the exercise

• The allowances for maternity, paternity and adoption leave have been revised following consultation

Equality and diversity:

Individual staff circumstances

Clearly defined circumstances•Early Career Researchers•Part-time working, career breaks and secondments outside of HE•Periods of maternity, adoption and additional paternity leave

Complex circumstances•Disability•Ill health or injury•Mental health conditions•Additional constraints related to bringing a child into the family•Other caring responsibilities•Gender reassignment•Other circumstances related to the protected characteristics or employment legislation

Equality and diversity:

Equality and Diversity Advisory Panel• EDAP has been convened to assist the funding bodies

and REF panels in implementing equality and diversity measures in the REF

• EDAP will examine institutions’ codes of practice

• EDAP will advise on all cases of complex individual circumstances and recommend the appropriate reductions to the Main Panel Chairs:

- Sub-panels will be informed of the decisions and will not have access to further information about complex circumstances

- ECU has published worked examples, including EDAP’s rationale

Equality and diversity:

Research outputs• Panels assess the quality of outputs – not individuals

• Outputs may include but are not limited to: printed or electronic publications, materials, devices, images, artefacts, products, buildings, confidential or technical reports, patents, performances, exhibits or events

• All forms of outputs shall be assessed on a fair and equal basis

• Panels will take account of additional information and/or citation data as stated in the ‘panel criteria’

• They will not use journal rankings or impact factors

Outputs:

Co-authorship • A co-authored output may be listed against one or more

individuals that made a substantial contribution to it

• It may be listed against any or all such co-authors returned in different submissions; and a maximum of two such co-authors within the same submission

• In very specific situations the panels require information to establish that the author made a substantial contribution

• Once this is established, panels will assess the quality of the output, not the individual author’s contribution

Outputs:

Double-weighting • Institutions may request ‘double-weighting’ of outputs of

extended scale and scope

• Sub-panels will consider the request for double-weighting separately from assessing the quality of the output

• If a sub-panel accepts a request, the output will count as two outputs in the calculation of the outputs sub-profile

• Institutions may submit a ‘reserve’ that will be assessed only if the double-weighting request is rejected

Outputs:

Citation data• The following sub-panels will make use of citation data:

- Main Panel A: Sub-panels 1-6

- Main Panel B: Sub-panels 7-11

- Main Panel C: Sub-panel 18

• Citation data will be used as a minor component to inform peer-review

• HEIs will be provided access to the data via the REF

submission system

• The funding bodies do not sanction or recommend that HEIs rely on citation data to inform the selection of staff or outputs for their REF submissions

Outputs:

Definition of impact

• Impact is defined broadly for the REF: an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life

• Panels recognise that impacts can be manifest in a wide variety of ways, may take many forms and occur in a wide range of spheres

• Each Main Panel provides examples, which are intended to be helpful to institutions. They are not exhaustive or restrictive lists.

Impact:

Impact submissions

Impact template (REF3a)

•Sets out the submitted unit’s general approach to enabling impact from its research•One template per submission – with a page limit depending on the number of staff submitted•Covers the period 1 Jan 2008 to 31 Jul 2013•Contributes 20% to the impact sub-profile

Case studies (REF3b)

•Specific examples of impacts already achieved, that were underpinned by the submitted unit’s research•The number of case studies required depends on the number of staff submitted (1 plus 1 per 10 FTE)•Impacts during 1 Jan 2008 to 31 Jul 2013; underpinned by research since 1 Jan 1993•Contributes 80% to the impact sub-profile

Impact:

Case studies • Each case study should:

- Clearly describe the underpinning research, who undertook it and when

- Provide a coherent narrative, clearly explaining the relationship between the research and the claimed impact

- Clearly identify the beneficiaries and define the impact

- Provide evidence/indicators relevant to the case being made. Evidence can be qualitative and/or quantitative.

- Provide independent sources of corroboration

- All the material required to make a judgement should be included

Impact:

Underpinning research

• Case studies must be underpinned by research produced by the submitted unit that has made a material and distinct contribution to the impact

• The underpinning research must meet the quality threshold of at least two star

Impact:

Assessment criteria

The criteria for assessing impacts are reach and significance*

Four star Outstanding impacts in terms of their reach and significance

Three star Very considerable impacts in terms of their reach and significance

Two star Considerable impacts in terms of their reach and significance

One star Recognised but modest impacts in terms of their reach and significance

UnclassifiedThe impact is of little or no reach and significance; or the impact was not eligible; or the impact was not underpinned by excellent research produced by the submitted unit

* Each main panel provides a description of the criteria

Impact:

Research environment • Each submission to include a completed template:

- Overview

- Research strategy

- People (including staffing strategy and staff development; and research students)

- Income, infrastructure, and facilities

- Collaboration and contribution to the discipline

• The ‘panel criteria’ request specific types of evidence under each heading

• Data requirements have been streamlined and standardised as far as possible

Environment:

Main and sub-panel roles

Sub-panel responsibilities

•Contributing to the main panel criteria and working methods•Assessing submissions and recommending the outcomes

Main panel responsibilities

•Developing the panel criteria and working methods•Ensuring adherence to the criteria/procedures and consistent application of the overall assessment standards•Signing off the outcomes

REF panels:

There are 36 sub-panels working under the guidance of 4 main panels. Membership is published at www.ref.ac.uk

Sub-panel working methods

• Sub-panels will review their expertise at key stages in the exercise

• Work will be allocated to members/assessors with appropriate expertise

• Each sub-panel will run a calibration exercises for outputs and impacts, guided by the main panels

• All outputs will be examined in sufficient detail to contribute to the formation of the outputs sub-profiles

• Each case study will normally be assessed by at least one academic member and one user member or assessor

REF panels:

Additional assessors

• Both ‘academic’ assessors (to assess outputs) and ‘user’ assessors (to assess impacts) will be appointed

• Assessors will play a full and equal role to panel members, in developing either the outputs or impact sub-profiles. They will be fully briefed, take part in calibration exercises and attend the relevant meetings:

- Some appointments in 2012

- Further appointments in 2013, in the light of the survey of institutions’ submission intentions

REF panels:

Additional assessors will be appointed to extend the breadth and depth of panels’ expertise:

UOA boundaries and interdisciplinary research • UOA boundaries are not rigidly defined. Panels expect

submissions to include work that is interdisciplinary, multidisciplinary or spans boundaries with other UOAs

• Panels are committed to assessing all such work on an equal basis:

- Sub-panels include members with interdisciplinary expertise; assessors will be appointed to extend their breadth and depth of expertise

- The sub-panels prefer to assess all work submitted within their UOAs but may, exceptionally, cross-refer parts of submissions to other sub-panels for advice. (The original sub-panel remains responsible for recommending the outcomes.)

REF panels:

Further information

www.ref.ac.uk (includes all relevant documents)

Enquiries from staff at HEIs should be directed to their nominated institutional contact

(see www.ref.ac.uk for a list)

Other enquiries to info@ref.ac.uk

The REF Framework

Overall excellence profile

Outputs (65%)

Maximum of 4 outputs per researcher

Impact (20%)

Impact Template & Case studies

Environment(15%)

Narrative template + income and student data

• Tested and developed a case study approach to assessing the impact of research

• 5 units of assessment (UOAs)

• 29 UK higher education institutions each submitting to 2 UOAs

• Each submission included:

• An ‘impact statement’ for the submitted unit as a whole

• Case studies illustrating examples of impacts achieved (a total of one case study per 10 research staff)

• Impacts that occurred during 2005-09, underpinned by research since 1993

The impact pilot exercise

• Membership drawn from academia and research users from the private, public and third sectors

• The panels tested the methodology by:

• Assessing the case studies in terms of ‘reach and significance’ of the impacts

• Considering the wider ‘impact statements’

• Producing impact profiles

• Reflecting on the process, identifying issues and making recommendations on how to improve the process

The pilot panels

• Publications on www.ref.ac.uk:

• The findings of the 5 pilot panels

• Feedback from the 29 pilot HEIs (by Technopolis)

• Examples of good practice case studies

• A summary of workshops to explore impact in the arts, humanities and social sciences

• Guidance documents used in the pilot exercise

Pilot reports

Key findings

The process makes explicit the benefits that research in each discipline brings to society

It is possible to assess the impact of research, through expert review of case studies

A number of refinements are needed for full implementation A generic approach is workable, with scope for REF panels to tailor

the criteria as appropriate to their disciplines The weighting should be significant to be taken seriously by all

stakeholders, and needs careful consideration

• Increased links with businesses: in 2003, Richard Lambert noted less than 20% of businesses had links with HEIs. In the CBI Education and Skills Survey 2010, 66% of businesses now have links with HEIs.

• Increased KE income into HE (proxy for economic and social impact). Total income from interactions between UK HEIs and business and community has increased by 35% from £2.28Bn in 2003-04 to £3.09Bn in 2010-11 in real terms. Interactions included take up of HE knowledge, expertise, skills and equipment (IP+).

• Strong return on public investment / leverage: for every £1 of HEIF, between £4.9 and £7.1 of KE income into HE has been generated.

Key achievements of HEIF

• Significant contribution to academic culture change towards KE: number of academics with positive attitudes to KE has grown from 61% in 2001 to 76% in 2008.

• Delivered benefits to research and teaching: 48% of academics judged that KE gave them new research insights; 38% of academics judged that KE had helped them improve their presentation style.

• International comparisons: US, OECD and EU all now referencing UK approach to KE.

Key achievements of HEIF

• HEFCE response:

• From building capacity to focus on rewarding performances – 100%

• Increase maximum award by 50% to £2.85m p/a

• Consulted on a new policy of applying a cut off to allocations at £250k

• HEIF supports a broad range of KE activities that result in economic and social impact

• Knowledge and technology transfer

• Double weight income from SMEs

• Support for enterprise education

• Staff and social enterprise and entrepreneurship

Higher Education Innovation Funding 2011 - 15

• What we count:

• Income from (*including 2 x weighting of SMEs)

• Contract research*• Consultancy*• Equipment and facilities*• IP licensing* (excluding sales of spin off companies)• Regeneration and development funding (UK funds, ERDF,

ESF, etc)• Non credit baring course income (proxy for CPD)• Knowledge transfer partnerships (KTPs)

• The formula

• £150m p/a, max £2.85m, threshold £250k, +/-50% of previous award

• Released against high level HEI strategy

HE Innovation Funding 2011 - 15

Scope of policy

COMMUNITY

PUBLIC SECTOR

CULTURAL LANDSCAPE

BUSINESS

Competitiveness, Growth

Efficiency,Cohesion

Cultural Enrichment & Quality of Life

Resources & Opportunities

PRIVATE SECTOR

SOCIAL & CIVIC ARENA