Skip navigation
Link to Back issues listing | Back Issue Listing with content Index | Subject Index

Blinding reviewers

Study
Results
Comment

Just because you are not paranoid doesn't mean that they are not out to get you. People sometimes feel like that if they try to get their papers into certain journals, or abstracts into large and important scientific meetings. At the centre of it all is the peer review process. We accept it is the best we can do, but a study from the US [1] has startling insight into how blinding can affect abstract acceptance.


Study

The study examined submissions for the American Heart Association meetings in 2000 and 2001, where review was open, and 2002-2004, when it was blinded. It was really well done, and the outcome was whether or not a study was recommended for acceptance using standard procedures. Acceptance rates were then judged before and after blining was introduced according to different criteria: US or non-US, English on non-English speaking country, Prestigious or non prestigious institutions, sex of author, US government agency, industry, and by basic or clinical study. Each abstract was evaluated by 8-10 reviewers.


Results

There were over 13,000 abstracts submitted annually, with an average acceptance of 29%. About 40% were from the US, a quarter from highly prestigious institutions. Most (85%) of reviewers were US-based. We might expect higher acceptance from prestigious institutions compared to non-prestigious counterparts, for instance, with open compared with blinded review: if bias were a factor, the gap should narrow with blinding. Table 1 shows that with blinding the gap narrowed by 9%. There were similar changes for abstracts from US government institutions, for US, and for non-industry abstracts. Only sex of author was unaffected.



Table 1: The effects of blinding reviewers on abstract acceptance rates - differences in pairs of separate criteria



Criterion
Change (%)
US govt vs non-government
-12.2
Prestigious vs non-prestigious
-8.9
US vs non-US
-8.5
Not industry vs industy
-7.3
English vs non-English
-4.2



Comment

However balanced we like to think we are, the simple fact is that we are persuaded that research from government agencies, or prestigious institutions, or not coming from industry, is better. When we are deprived of that information, we make different judgements.

Reference:

  1. JS Ross et al. Effect of blinded peer review on abstract acceptance. JAMA 2006 295:1675-1680.

previous or next story