Gradgrinding the Social Sciences: The politics of metrics of political science

社会科学的计算化 :政治学度量的政治学

Dr Claire Donovan 克莱尔 · 多诺万 博士

Gradgrinding? 计算化

• Charles Dickens, ‘Hard Times’ (1854)• 查理狄更斯的《艰难时世》（ 185

4 ）

• Thomas Gradgrind = headmaster• 汤玛斯 · 葛莱恩 = 教育家

• Children are ‘little pitchers … to be filled so full of facts’

• 儿童就像“小水罐……等待着被灌满事实”

“‘Quadruped. Graminivorous. Forty teeth, namely twenty-four grinders, four eye-teeth, and twelve incisive. Sheds coat in the spring; in marshy countries sheds hoofs, too. Hoofs hard, requiring to be shod with iron. Age known by mouth Thus (and much more) ...’ ‘Now girl number twenty’, said Mr Gradgrind. ‘You know what a horse is’.”

Can metrics replace peer review?度量方法能替代同行评议吗？

• Butler & McAllister 巴特尔和麦卡利斯特 (2009):– ISI citation data for 2001 UK RAE– 2001 UK RAE 的 ISI 引用数据– political science & international relations– 政治学和国际关系– includes citations to books & chapters (43% of

submitted work)– 包括对书籍和章节的引用（提交工作的 43% ）

Can metrics replace peer review?度量方法能替代同行评议吗？

• Butler & McAllister 巴特尔和麦卡利斯特 (2009):– ‘objective’– 客观性– metrics give similar results to 2001 RAE peer review– 度量方法的结果与 2001 RAE 同行评议结果相似– metrics-based evaluations are ‘a much more efficient and

transparent approach to measuring research quality’– 基于度量的评估方法使“评测研究质量更加有效和有

透明性”

But… 但是– lack of transparency (a black-box)– 缺乏透明性（黑盒子）– only 3 highly-cited books named– 只有 3 本高度引用的书籍– incorrect citation window– 错误的引用窗口

December 2000

Butler & McAllister

December 2006

Book 1 2 118 129

Book 2 38 141 149

Book 3 316 711 724

But… 但是– ISI database needs ‘cleaning’ to find citations to book &

chapters– ISI 数据库需要“清洗”来找到对书籍和章节的引用– they miss co-author of most cited book– 大多数引用书籍的共同作者未注明

• correcting this one error invalidates their conclusions• 纠正这个错误会使他们的结论无效

– a further 877 books, 1,021 chapters and 155 edited books to consider

– 另外有 877 本出版书， 1021 章节和 155 本编辑完成的书需要考虑

The politics of metrics of political science 政治学度量的政治学

• Dispute over metrics and peer review• 度量方法和同行评议之争– metaphor for conflicting epistemological preferences

of UK political scientists– 英国政治学家之间相互冲突的认识论偏好的体现。

• What is ‘light touch’ peer review?• 什么是“轻度” 同行评议？– values data more highly then expertise– 更侧重数据然后是专业知识

Gradgrinding the social sciences 社会科学的计算化

• Metrics-led assessment can make political science departments amount to less than the sum of their parts

• 政治学实际取得的研究成果小于度量方法计算出的总的定量结果– ‘facts’ alone give a fragmented picture– 单独的“事实”给出片面的图像– qualitative description also essential– 定性的描述也很重要

Gradgrinding the social sciences社会科学的计算化

• Metrics-based audits that limit or exclude peer review strip the discipline of its humanism

• 基于度量体系的评估方法限制或禁用同行评议使政治学失去了其人文主义的学科特性。

– diminishes importance of interpretation & expertise– 降低演绎和专业知识的重要性

– ‘objective’ metrics are not value-neutral– “ 客观性”度量方法不是价值中性的

Conclusion结论

• While metrics simplify assessment, they simplify the scope and aspirations of political science through excluding interpretive knowledge

• 当度量方式简化评估时，他们通过禁用演绎知识来简化政治学的范围和目标。

Conclusion结论

• ‘Someday we have to decide whether it is best to have our hearts broken qualitatively or quantitatively’

• 总有一天我们必须决定是否将我们的心脏打开做定性和定量分析。– it is better for UK political science to be a sum of its

parts, and so to have its heart broken both ways– 为了使英国政治学得到实际取得的研究成果，就

要用两种方式打开它的心脏。