I’ll give ’em to you right up front: mindfulness, honesty, tact, precision, and respect. Got that? Ok, now on to the rest of this post:
These days everyone loves to hate peer review. It is, the story goes, an old fashioned, evil form of collaboration which has been rendered obsolete by shiny new ways of communicating over the Intarweb. I am part of the problem, since I’m not uncritical of peer review. However, peer review has been so pummeled lately that I am starting to feel sorry for it since, no matter how valid many of the objections against it are, it got caught in a perfect storm of pre-existing antipathy and techno-enthusiasm that could sink pretty much anything. So I want to make a point here about peer review which I feel is under-appreciated: the value it has for improving not articles, but reviewers.
I’ve reviewed dozens of papers by this point, and received comments from many reviewers on my own work. I’ve worked in the video game literature which is large and where I can’t identify peer reviewers. I’ve also worked on Melanesian anthropology, a field so small that the pretense of anonymous review is almost comical. I’ve received remarkably helpful feedback and also ridiculous feedback. Being on the receiving end of peer review has instilled in me a deep desire to do good review, and a deep appreciation of how hard that is to do. Bad peer review is like the comments section of this blog in one of its bad moods: vicious, narcissistic, and extremely abstract. Good peer review, on the other hand, requires one to cultivate a wide range of virtues.
First, peer review teaches you to read closely. As academics we can always decide to not read something, or to dislike it because of its tone, or other reasons. To honestly disagree with a paper for peer review, you really need to read it closely and discern what it actually says — not what you think it says or want it to say given your current set of prejudices. The more you disagree with something, the more carefully you have to scrutinize it.
Secondly, peer review requires you to be honest, frank, and candid
ate about what a work says. A diplomatic person often avoids saying what they think of someone directly to their face. When you can’t avoid speaking directly to someone’s merits, it is just easier to tell people what they want to hear. It’s also very easy to just dump on someone and call their work worthless. But evasion, flattery, and attack are all far different from what is actually needed: rigor, prudence, and honesty.
Good peer reviewers also, third, have to be good writers. There is a veil of secrecy — or worse, the pretense of one — between you and your reader. Any sort of negative feedback always stings (indeed, for some anything but lavish praise stings), and the pain of peer review is amplified by this secrecy. It takes a lot of work to give critical feedback in a way that people will take on board, especially since outright flattery is disallowed. One must strike the right tone.
The fourth good quality of peer review is that you must not only strike the right tone, you must actually have something concrete to say. What specifically is wrong with the paper. How specifically can it be improved? Most academics have good noses and can tell if something stinks. Fewer, however, can figure out what in particular is haywire in a paper. Even fewer can provide suggestions about what sort of remedies might be undertaken. The biggest problem, surprisingly, is that few of us have ever actually sat down and explicitly articulated what actually makes a paper good. We rely on our habitus, which unfortunately (or fortunately?) we can’t just break off a piece of and mail to our collaborators. And less face it, we could stand to be more explicit and precise in our understanding of what makes work valuable — habitus only gets you so far.
Finally, there is one virtue of peer review which seems in particularly short supply: the desire to improve the paper the author has written. Too often, peer reviewers instead complain that the paper is not as they themselves have written it. Sometimes this phenomenon is easily spottable because it is so outrageous: “This paper on X and gender should not be published because the author should have written a paper on X and race”. Often times this lack of respect for the scholarly project of the author hides behind suggested changes which seem to make sense but, if you think about them, are actually unfair. So remember: it’s not their fault that they’re not you. Your job is to understand who they are, and how they and their work could improve.
mindfulness, honesty, tact, precision, and respect: these are not just the virtues of peer review, they are virtues period. Peer review is important because it offers an powerful opportunity to cultivate them. The fact that so many people do not chose to do is, of course, partly caused by the way peer review is architected. But let’s face it: bad peer review happens because good people are hard to find — and that’s something that’s probably true everywhere. I’ve benefitted tremendously from the responsibility to peer review papers because it elicits personal growth. My recommendation? The next time you get lousy feedback from someone, use the incident to strengthen your resolve to become the peer reviewer you wish your paper had gotten.