Monday, April 15, 2013

Interactive and Open Peer Reviews Are Better For Science

Science Careers just published a two part article describing an interactive peer review system (part two is here), describing it as a chance to advertise one's work and network on one hand, while considering the novelty of the approach a drawback on the other:
Interactive peer review can offer advantages for researchers. Among the most important: turning a blind, mostly one-way process into a conversation with established researchers. "The primary benefit, particularly for young scholars, is getting their work not just out into circulation but out into active conversation with the people in the field," says Kathleen Fitzpatrick, a visiting professor at New York University in New York City who studies how networked communication technologies affect scholarship. But there are disadvantages, too: Some of these journals have no or low impact factors, for now at least, and more conservative scientists on review committees may not give you much credit for publishing there.
So basically what they're saying is that if you're going to consider publishing some work with a house like Frontiers, you might as well view it as an opportunity to advertise yourself rather than establish a toehold in a big journal.  That approach in itself isn't necessarily bad.

The two parts of the article really portray the collaborative review system as a powerful mechanism for constructive paper writing, which is a useful feature for people early in their careers.  However, there is one feature mentioned near the end that I think is move in a good direction; open peer review and the disclosure of reviewer names.

This is important on several parts.

Firstly, it puts much more power into the the author's hands, as it partly prevents reviewers from being overwhelmingly negative/sarcastic/arrogant/destructive which mostly ends up creating essentially useless reviews. They can't really hide behind anonymity, so are forced to at least try to be useful.

Secondly, and I don't think I've heard anyone discuss this before, is that it actually let's reviewers quantify their reviewing activities as professional contributions.  Most researchers have a section on their CV that includes which journals they've reviewed for, but it's frankly not very useful because a) I can't easily verify that you've been invited as a reviewer (I guess I could ask the editor), b) I have no idea whether you've reviewed more than one paper, ever, and c) I don't know if you're just rejecting everything that comes your way.  If everyone could see how much you're contributed to the scientific community, it might be an one more metric to evaluate productivity.

And lastly, if full peer review comments started to be published by many journals (some already do), it would be nice to read them to identify potential colleagues that are great partners to work with and also the people you would rather avoid.