Wed 27 Feb 2008
Peer Review Round-up, yee haw.
Posted by ckelty under Academia , Briefly Noted , Open Access Open Source , Technology , WebsitesI want to point people to a couple of things related to the intersection of peer review and open access. The first is a recent round-table discussion held at the Center for Studies of Higher Education at UC Berkeley, which brought together a great group of people including Don Kennedy, the editor of Science and Mark Rose (author of Authors and Owners) among others. The minutes are available (link) and they include a number of interesting proposals and diagnoses of the main problems facing scholarly publishing today, including some sharp observations about the financial realities of publishing peer-reviewed work and creative ideas for publishing monographs.
The other, a bit late, but still ongoing is that Noah Wardrip-Fruin over at Grant Text Auto, is experimenting with blog-based, serialized, community peer review. Noah’s book, Expressive Processing (one of a increasingly large number of texts laying claim to the field of “software studies”), is serialized for commentary using comment press (hip hip hooray!) and is being conducted along side “standard” peer review with MIT Press. I think this is a great idea whose time has definitely come, for a couple of reasons. One is that I’m more and more fond of the idea that peer review is best done by communities of people who are not anonymous. Pseudonymy might be a good idea (i.e., I don’t care who “IreadBooks69” is at Amazon but I know that s/he writes great reviews). The other is that community is a just generally a good idea. If the people commenting on Noah’s book feel as though they are contributing, are part of something, and that they get credit for it, or perhaps even get an immediate response, then that beats the heck out of the anonymous, forgotten black hole I routinely send my reviews into. I only hope Noah writes some kind of white paper-ish thing highlighting what works and what doesn’t so that people can repeat the experience.
Ken Wissoker says:
Hi Chris,
You probably won’t be surprised to hear that I prefer the anonymous form of peer review. It allows the possibility that a reviewer could say what they might not say if their identity were known. People say different things in different contexts: in a bar to a friend; in a writing group (where they are supposed to be helpful); in a tenure or promotion letter; and to a press. A lot of what might only be said anonymously turns out to be very valuable to the success of a manuscript. Anonymity might only make this different some of the time—when the reviewer suspects their advice may not be a welcome message—but the preserving the space for that advice is a key part of the process.
The other valuable part—easy to overlook—is reception. I find that when authors know the identity of the reader they move quickly to explain away good advice, using what they know about the reader to narrate the concern as the difference between the reader’s work and their own (as if no one could appreciate work different than their own). In the same case, with feedback from an anonymous “Reader #1” an author is more frequently forced to come to terms with what is or isn’t working in their own prose. Occasionally, I think “if this person knew who was giving this advice, they would just take it” but more often I think the work is more improved by the reviewers being anonymous. A writing group, like a wiki, can really improve a piece of writing, but feels different than peer review, in the way (and I hesitate over this analogy, but kind of think it’s right) that advice from friends might be helpful in a tough situation, but is different than therapy.
Ken
ckelty says:
this is why I mentioned pseudonomy… I think we actually have more than an either/or thing when its possible to log in under a variety of different pseudonyms. I can post comments on this blog under any name I want, if I want the anonymity, but I haven’t yet seen the need. I agree with you, however that in some cases it would certainly be better. I think this is only an argument for being more careful about what one means when one says peer review, and perhaps for making the design of a particular system explicit. For instance, it might be of relevance to potential authors, critics and tenure committees if a press specified the level of peer review a book went through, then one could differentiate between anonymous private (i.e. only reviewers and authors see the reviews), anonymous public (i.e. reviews out in the open), as well as the selection criteria, such as open peer review (anyone who wants to) or “editor-selected” peer review or something. There is also a difference to between pre-revision and post-revision peer review, i.e. the kind of peer review that helps make something better through revision, vs. the kind that merely approves or rejects (which is obviously fuzzier). And so forth.
Ken Wissoker says:
In fact, I know of a couple of cases where pretty famous people gave helpful but fairly critical reviews of manuscripts to me, and then ended up blurbing the books when the author took them some place where they were promised an easier path. It’s possible that those reviewers might have taken advantage of a pseudonymous situation, but I somehow doubt it. I’m not sure that everyone who could volunteer to give someone an anonymous piece of their mind would be helpful either… but maybe I’m just suffering from a duke lacrosse blog hooligan nightmare…