Peer Review Can Make Metrics Accountable同行评议可以赋予指标以责任
Kelli Barr凯利 · 巴尔
Department of Philosophy and Religion StudiesCenter for the Study of Interdisciplinarity
University of North Texas哲学与宗教学系跨学科研究中心北德克萨斯大学
Translation by ZHANG Wei
The Question at Hand目前的问题
Are metrics better tools for evaluating research than peer review?在评估研究方面,指标是比同行评议更好的工具吗?
‘Peer Review Can Make Metrics Accountable’International Workshop on Peer ReviewDalian University of TechnologyMonday, May 21 – Wednesday, May 23, 2012
Are metrics better tools for evaluating research than peer review?
Increasingly, academics and administrators are answering with an unqualified “yes专家学者和管理者越来越多地倾向于给出肯定的回答,但这是有问题的。
– H-index, g-index, webometrics H 指数、 g 指数、网络计量学– US News and World Report, Shanghai Rankings 美国新闻与世界报道,上海(交通大学)大学排名– National research evaluations 国家研究评估– Hiring, promotion, and tenure decisions 人才招聘、晋升和终身教职决定
‘Peer Review Can Make Metrics Accountable’International Workshop on Peer ReviewDalian University of TechnologyMonday, May 21 – Wednesday, May 23, 2012
Peer Review Bibliometrics同行评议 文献计量学
Subjective 主观性
Contingent 差异性
Non-transparent 不透明性
Unaccountable 不可解释性
Costly 高成本性
Objective 客观性
Replicable可重复性
Transparent透明性
Accountable可解释性
Low(er) Cost低成本性
Defining the dichotomy using these terms creates a distinction without much of a difference
用这些术语定义的二分法给出了一个其实并没有太多差别的区分
– Bibliometrics are ultimately based on peer review processes 文献计量方法最终要建立在同行评议的程序之上
All metrics instantiate specific values that affect the outcome of the measurements
所有的指标都是某种具体价值标准的体现,都会对评估结果产生影响
‘Peer Review Can Make Metrics Accountable’International Workshop on Peer ReviewDalian University of TechnologyMonday, May 21 – Wednesday, May 23, 2012
http://chronicle.com/article/30-Ways-to-Rate-a-College/124160/
Forbes – concerned with economic, financial, commercial, and business outcomes福布斯—涉及经济、金融、商业、贸易的结果Academic ranking of world universities – concerned with bibliometric (research) outcomes世界大学的学术排名—涉及文献计量学的(研究)结果
‘Peer Review Can Make Metrics Accountable’International Workshop on Peer ReviewDalian University of TechnologyMonday, May 21 – Wednesday, May 23, 2012
Journal impact factor, calculated using Web of Knowledge期刊影响因子——基于 Web of Knowledge 数据库
Sciences Nature 34.48自然科学 《自然》
Cell 31.152《细胞》Science 29.747《科学》
Social Sciences J. of Economic Lit. 6.919社会科学 《经济文献杂志》
Political Analysis 3. 756《政治分析》Ann. Rev. of Sociology 3.702《社会学年评》
Philosophy Am. J. of Bioethics 4.000哲学 《美国生物伦理学杂志》
Phil. and Public Affairs 1.957 《哲学与公共事务》Environmental Values 1.250《环境价值》
Do social science and philosophy journals really have that much less “impact”?社会科学和哲学期刊真的是只有这么点儿影响力吗?
Journal impact factor, calculated using Web of Knowledge
Sciences Nature 34.48
Cell 31.152
Science 29.747
Social Sciences J. of Economic Lit. 6.919
Political Analysis 3. 756
Ann. Rev. of Sociology 3.702
Philosophy Am. J. of Bioethics 4.000
Phil. and Public Affairs 1.957
Environmental Values 1.250
Bibliometrics assume that all research conforms to the model of scientific research 文献计量学假定所有的研究符合自然科学的研究模式。
– Cumulative 累积的– Disciplinary 分科的
‘Peer Review Can Make Metrics Accountable’International Workshop on Peer ReviewDalian University of TechnologyMonday, May 21 – Wednesday, May 23, 2012
Bibliometrics assume that all research conforms to the model of scientific research
– Cumulative – Disciplinary
Bibliometrics highlight certain kinds of research:文献计量学突出强调某些种类的研究:
– Journal publications 期刊发表– Quantitative 定量的– Methodologically focused 关注方法论的– Review papers 评议论文
Implications 启示
‘Peer Review Can Make Metrics Accountable’International Workshop on Peer ReviewDalian University of TechnologyMonday, May 21 – Wednesday, May 23, 2012
Bias toward disciplinary, status quo research对学科的偏见,现状分析
Reinforcing the status quo is not morally neutral 对现状的加强在道德上不是中立的
Appeals to the supposed objectivity, transparency, etc. of metrics are used as evidence of their normative superiority to peer review
对指标的所谓的客观性、透明性等属性的诉求被用来作为它比同行评议具有规范的优越性的证据。
‘Peer Review Can Make Metrics Accountable’International Workshop on Peer ReviewDalian University of TechnologyMonday, May 21 – Wednesday, May 23, 2012
Reflexive evaluation of activity toward goals
对面向目标的行动的自反性评价
Episteme知识
Rational calculation of goals对目标的理性计算
Phronesis实践智慧
Techne技艺
Accountability is about rational justification责任性涉及合理的辩护
• Definitions vary widely with context: 定义会因为语境的不同而产生很大的变化:
– Legal liability法律责任
– Return on investment对投资的回报
– Academic rigor学术的严格性
‘Peer Review Can Make Metrics Accountable’International Workshop on Peer ReviewDalian University of TechnologyMonday, May 21 – Wednesday, May 23, 2012
Accountability is about rational justification责任性涉及合理的辩护
• Definitions vary widely with context:
• Common denominator: evaluating whether or not current tactics meet overall strategic goals
共同点:评估当前的策略是否能够达到总体的战略目标
Accountability is about rational justification责任性涉及合理的辩护
• Definitions vary widely with context: • Common denominator: evaluating whether or not current tactics meet
overall strategic goals
• Accountability is more than just rule following 责任性不仅仅是遵守规则
Conclusions:结论
Neither metrics nor peer review are an unqualified good指标和同行评议都不是最好的方法
The question of the meaning of accountability is also the question of the meaning of responsibility
责任性的意义问题同时也是一个义务的意义问题
If science funding is to be a goal-directed activity, then evaluations of the science funded cannot normatively privilege values such as objectivity for their own sake.
如果科学资助是一个以目标为导向的活动,那么对受资助的科学研究的评估就不能给予诸如为其自身着想的客观性这样的价值以特权。
‘Peer Review Can Make Metrics Accountable’International Workshop on Peer ReviewDalian University of TechnologyMonday, May 21 – Wednesday, May 23, 2012