What Do Blind Evaluations Reveal? How Discrimination Shapes Representation and Quality
Abstract:
Concealing candidate identities during evaluations (“blinding”) is often proposed to combat discrimination, yet its effects on the composition and quality of selected candidates, as well as its underlying mechanisms, remain unclear. I conduct a field experiment at an international academic conference, randomly assigning all 657 submitted papers to two blind and two non-blind reviewers (245 total) and collecting paper quality measures (citations and publication statuses five years later). I find that blinding significantly shrinks gaps in reviewer scores by student status and institution rank, although an institution rank gap persists, with no significant effects by gender. These changes in representation are not at the expense of quality: papers selected under blind review are of comparable quality to those selected non-blind. To understand mechanisms, I run a second field experiment that again implements blind and non-blind review, and elicits reviewer predictions of future submission outcomes. I combine my experiments to estimate a model of reviewer scores that uses blind scores to decompose non-blind disparities into distinct forms of discrimination. I find that the nature of discrimination differs by trait: student score gaps are largely explained by alternative objectives (such as favoring authors whose acceptance benefits others), while institution gaps are attributable to residual drivers such as animus.
Organized by: Sam Norris
