Assessing regional engagement and knowledge transfer – ranking or benchmarking?

David Charles, EPRC, University of Strathclyde

KT and engagement

• Qualitatively different to assess than teaching and research

• Not same consensus over idea of quality• Not simply in control of university• Does not indicate institutional excellence• Partly dependent on external demand and environment• Subjective assessment depending on perspective

Different forms of KT and RE

• Different paths to KT – research exploitation or informal exchange

• KT as codified vs tacit knowledge – who benefits?• Other forms of engagement – cultural, social,

governance relationships etc

• Varied possible forms of excellence, some easier to measure than others

Ranking

• Comparison across diverse activities• No sensible means of weighting activities• Are we assessing university or regional

environment?• Balance of private and community benefit

Simple exploitation measures

• Patents, licences, spin offs, contract income• Discipline-specific opportunities and partly

demand driven• Example of HEBCIS survey in UK, AUTM in US

and Canada• Different rankings of universities for different

indicators

Benchmarking instead of ranking

• Comprehensive set of indicators• Identify areas of strength and weakness• University and partners to decide on prioritisation• Benchmarking with other universities to learn how

to improve those areas seen as important• Differentiation as an objective to better meet

needs of stakeholders

Engagement embedded in university vision and mission

1 2 3 4 5 Vision and mission does not recognise engagement as a key role for the university

Some reference to the need to engage with the region is placed in the vision or mission, usually in terms of identifying a regional community as being of interest. Vision is developed from a top-down position and is not driving strategy or seen as an influence on staff behaviour.

Engagement is a central element of the vision and mission and is the result of a sophisticated debate within the institution involving staff from various levels of the institution. Engagement is seen as part of the DNA of the university and is considered as important in everything they do.

Rewarding and valuing engagement

1 2 3 4 5 No staff incentives for engagement – positive discrimination against engagement in promotions processes with an emphasis on research.

Formal recognition of engagement in promotions procedures as one of the areas of performance that can be recognised, but little evidence of it having major impacts on behaviour. Little recognition elsewhere in the system. Engagement is tolerated and possibly rewarded where excellence is achieved but not systematically.

Clear and well communicated recognition of engagement in a wide range of staff policies. Engagement is supported through workload and line management and good performance is recognised in promotion and through salary. Resources are available to help staff develop engagement skills including study leave. University recognises scholarship of engagement.

Issues for discussion

• Does it make sense to try and reduce engagement to one or two composite indicators?

• Why do we want to measure engagement, and how does this affect what we try to measure?

• What are the merits of benchmarking approaches that mix output and process indicators?

• Should we focus on mutual learning rather than ranking in this field?