excellence, innovation, evaluation: collaboration x innovation

Excellence, Innovation, Evaluation:A critical view on how poor questions lead to misleading answers

@cameronneylonhttp://orcid.org/0000-0002-0068-716X

“How can excellence in innovation lead to economic growth?”

What do we mean when we use the word…

“excellence”?

Pursuit of excellence: We strive for excellence, seeking to apply the highest standards to benefit…

University of Queensland

The university pursues academic excellencethrough the creation, preservation…

University of San Diego

…to strive for excellence and distinction…

Curtin University

We prize excellence

Cape Peninsula University of Technology

We demand excellence

University of Westminster

…with world-class excellenceacross all of its departments

Princeton

As a world-leading centre of academic excellence we aim…

University of Edinburgh

The university counts the diversity of its community as […] integral to its mission of excellence, and it is committed to diversity and inclusiveness in both educational…

University of Maryland

excellence in…?

excellence in professionally relevant teaching

University of Westminster

excellence in research and education

Indiana University

excellence in research, education and knowledge exchange

Strathclyde University

excellence in teaching, research, and public engagement

Cornell University

excellence in teachingand research

Curtin University

…leadership, innovation and excellence in teaching

Curtin University

and research

…a center of excellence for cultural, athletic and other events

University of Toledo

…internationally recognizedexcellence in informationtechnology services andinfrastructure.

Indiana University

What do we mean when we use the word…

“excellence”?

How can we determine or measure…

“excellence”?

[schools] are recognized for their excellence through national and international peer comparisons.

Indiana University

Pursuing excellence means being satisfied with no less than the highest goals we can envision.

University of Missouri

Pursuing excellence involves being informed by regional, national and global standards, as well as our personal expectations. University of Missouri

“excellence” is the holy grail of academic life

Michèle LamontHow Professors Think

excellence is produced anddefined in a multitude of sites and by an array of actors

Michèle LamontHow Professors Think

“excellence” tells us nothing about how important the science is and everything about who decides.

Jack StilgoeAgainst Excellence

Using the word excellence

is a means of avoidinga difficult conversation

Using the word excellence

is a means of avoiding talking about values

Impact

Research

Economic

Cultural

Education

Health

Environment

Impact

Research

Economic

Cultural

Education

Health

Environment

excellence

Impact

Research

Economic

Cultural

Education

Health

Environment

Impact

Research

Economic

Cultural

Education

Health

Environment

impact

Impact

Research

Economic

Cultural

Education

Health

Environment

quality

Impact

Research

Economic

Cultural

Education

Health

Environment

innovation

Impact

Research

Economic

Cultural

Education

Health

Environment

Impact

evaluation

…what we may be doing […] when we make [a] value judgement of a […] work...

Barbara Herrnstein SmithContingencies of Value

…articulating an estimate of how well that work will serve implicitly defined functions…


…for an implicitly defined audience…


…experiencing the work underimplicitly defined conditions…


Impact

Research

Economic

Cultural

Education

Health

Environment

Impact

There is some good news…

http://www.flickr.com/photos/verzo/8020565592/ CC BY

https://www.flickr.com/photos/see-‐through-‐the-‐eye-‐of-‐g/5392290809 CC BY

https://www.flickr.com/photos/jenny-‐pics/3239638494 CC BY

Citation

Bookmark

Citation

Bookmark

Citation

Social mediaconversation

Bookmark

Citation

Previous co-‐authorship

Conferenceco-‐attendance

Social mediaconversation

5. * Neylon, C. & Wu, S. (2009). Article-Level Metrics and the Evolution of Scientific Impact. PLoS Biol 7, e1000242+

Evidence of Impact: Over 150 citations according to Google Scholar.…

6. * Neylon, C. The Political Economics of Open Access Publishing (blog series). Science in the Open (2015-16).

Evidence of Impact: Over 2500 unique blog views to date…

Impact

“excellence” becomes a performance

Impact

…performance of“excellence”

Impact

…in the place ofexcellent performance

Impact

Change the narrative…

Impact

“soundness”

1.85

1.9

1.95

2

2.05

2.1

2.15

2.2

1996 2001 2006 2011

Global R&D Expenditure (% of GDP)

0

50000

100000

150000

200000

250000

1996 2001 2006 2011

Researchers per million people

evaluation

Impact

PLOS ONE (and arXiv)

Impact

Infrastructure and capacity

Impact

Research

Economic

Cultural

Education

Health

Environment

evaluation

Impact

Research

Economic

Cultural

Education

Health

Environment

Impact

Answers are easy…

Impact

…it’s the questions that are hard

@cameronneylonhttp://orcid.org/0000-0002-0068-716X