Saint Matthew strikes again: An agent-based model of peer review and the scientific community structure

Authored by Flaminio Squazzoni, Claudio Gandelli

Date Published: 2012-04

DOI: 10.1016/j.joi.2011.12.005

Sponsors: University of Brescia

Platforms: NetLogo

Model Documentation: Other Narrative Mathematical description

Model Code URLs: Model code not found

Abstract

This paper investigates the impact of referee reliability on the quality and efficiency of peer review. We modeled peer review as a process based on knowledge asymmetries and subject to evaluation bias. We tested various levels of referee reliability and different mechanisms of reviewing effort distribution among agents. We also tested different scientific community structures (cohesive vs. parochial) and competitive science environments (high vs. low competition). We found that referee behavior drastically affects peer review and an equal distribution of the reviewing effort is beneficial only if the scientific community is homogeneous and referee reliability is the rule. We also found that the Matthew effect in the allocation of resources and credit is inherent to a `winner takes all' well functioning science system, more than a consequence of evaluation bias. (C) 2011 Elsevier Ltd. All rights reserved.

Tags

Agent-based model Referees peer review Matthew effect Referee reliability