Why the Peer Review Process Works Even When It Doesn’t

[Savage Minds is pleased to publish this essay by Alan Kaiser as part of our Writers’ Workshop series. Alan is a Professor of Archaeology at the University of Evansville. He has published on issues of Roman culture in Spain, urbanism across the Roman empire, and streets as an organizing principle in Roman cities. He is the author of The Urban Dialogue: An Analysis of the Use of Space in the Roman City of Empúries, Spain (British Archaeological Reports, 2000), Roman Urban Street Networks (Routledge, 2009) and Archaeology, Sexism and Scandal: The Long-Suppressed Story of One Woman’s Discoveries and the Man Who Took Credit for Them (Rowman and Littlefield, 2014).]

Eleven editors and more than two dozen anonymous reviewers rejected my scholarly article. It documented the scandal I had uncovered that David Robinson, famed excavator of the Greek site of Olynthos, had plagiarized the work of his forgotten graduate student, Mary Ellingson. My article clearly made a number of people uncomfortable as there is an unspoken rule among American archaeologists working in Greece that it is bad form to criticize our intellectual ancestors in print. In the end I did get the story published as a book but that was only because I found an editor at Rowman and Littlefield who was a former student of mine and who was determined to help. This would seem to be a case study in the problems with the peer review system but that would be the wrong conclusion; this blog post seeks to prove that peer review works despite the flaws in the system.

Mary Ellingson #1

 Mary Ellingson in Athens, June 1931 (Photo courtesy of University of Evansville Archives)

But first the scandal. I stumbled across a scrapbook filled with photos, letters, and news clippings Ellingson had gathered in 1931 recording her archaeological work as a graduate student at Olynthos. Ellingson was part of the excavation staff working under Johns Hopkins University professor Robinson, a man still famous as one of the first classical archaeologists to devote an excavation almost exclusively to uncovering houses and understanding ancient Greek domestic life. Robinson excavated more houses at Olynthos than have been excavated at any other Greek site. Ellingson supervised Greek workmen in the trenches and cataloged terracotta figurines. When she returned to the United States she wrote her master’s thesis on the figurines not only producing a chronological catalog but also an analysis of their use. She completed her dissertation in 1939, which was an expansion of her thesis work to look at figurines from other sites in northern Greece and the Balkans in addition to those from Olynthos. Robinson published her thesis as volume VII in the Excavations at Olynthus series and her dissertation as the first chapter and a half of volume XIV listing himself as the sole author, without mentioning her contribution, and without asking her permission. Even by the standards of his day what he did was flagrant plagiarism. While a few archaeologists at the time published the work of their non-graduate student research assistants as well as their wives without crediting them, I have found no other example of someone publishing the thesis or dissertation of another, perhaps because such a transgression leaves a paper trail that is easy to trace.

I wrote an article on this subject assuming anyone could see the significance of the story to our understanding of the history of archaeology and particularly the place of women in that history. Every major classical archaeology journal in the US, and even some abroad, rejected it. One editor claimed her readers would not find it interesting and so did not send it to reviewers. Some editors and reviewers thought it was not plagiarism in Robinson’s day even though I documented quite clearly that it was. Some argued that it was an isolated incident while others argued such behavior was common; either way their conclusion was that the story was so trivial it was not worthy of publication. No one doubted my research and some even praised my “sleuthing” and “detective work,” but they remained unconvinced on the ultimate issue of publication. One anonymous reviewer stated quite bluntly “…what you are dealing with here is part of the unwritten history of classical archaeology. Best to leave it unwritten.”

It was clear that the story made many people very uncomfortable. Classical archaeology in Greece is unlike any other branch of archaeology in that all Americans who want to work there must spend a year or more at the American School for Classical Studies in Athens. The American School trains archaeologists at two excavations, one in the Athenian Agora and the other at the site of Corinth. The School has a reputation for conservatism; Stephen Dyson has written in his book Ancient Marbles to American Shores (1998, University of Pennsylvania Press) about his worry that the school’s conservative ways produce, “…an archaeological Confucius devoted to the word of the ancestors rather than a classical Lewis Binford, willing to challenge received tradition” (p. 86). I suspect this conservatism is what led to the great discomfort many editors and reviewers felt and the eventual death of the article.

Mary Ellingson #2

Mary Ellingson (left) standing among the excavated house foundations at Olynthos, spring 1931 (Photo courtesy of the University of Evansville Archives)

Fortunately I was able to get the story published in the end as a book rather than an article. By sheer blind luck my book proposal landed on the desk of a former student of mine. She sought out supportive reviewers and who worked hard to see the book through every stage of the process.

The interesting question that arises from this story is whether or not the peer-review process failed in this instance. I don’t think it did. With each rejection I rewrote the article to address concerns the reviewers raised, which improved it a great deal. Ultimately being forced to tell the story in a book rather than an article allowed me to explore the objections raised by the reviewers in more detail and supply much more context than was possible in an article. Writing a book rather than an article also provided me the opportunity to quote extensively from Ellingson’s own letters and writings, allowing her to gain her voice back, which was one of my goals. I don’t hesitate to say that the book is significantly better than the article. The reviews written to date, mostly by people from outside the field of Greek archaeology, confirm that publication as a book was the right choice.

The peer review process is messy. It is frustrating. It is painful. But it works, admittedly sometimes in unintended ways.

 

I am an anthropologist and historian of Tibet, and a professor at the University of Colorado. I conduct research, write, lecture, and teach. At any given time, I am probably working on one of the following projects: Tibet, British empire, and the Pangdatsang family; the CIA as an ethnographic subject; contemporary US empire; the ongoing self-immolations in Tibet; the Chushi Gangdrug resistance army; refugee citizenship in the Tibetan diaspora (Canada, India, Nepal, USA); and, anthropology as theoretical storytelling.

2 thoughts on “Why the Peer Review Process Works Even When It Doesn’t

  1. A nicely stated piece. It actually led me to order the book! I’ve had a similar experience with an article that eventually saw the light of day. While it did not address a scandal such as yours, it did cause reviewers a sense of discomfort. Normally I just make the changes asked for by reviewers. But in this case I held my ground to the principle and tone of the paper while addressing the various criticisms. I now have five full sets of reviewers comments (one for each of the submission attempts) plus the very considered comments by the editors that so very kindly pointed out the errors of my ways 🙂 The paper is now in print and for those interested in seeing what so inflamed an anthropological audience do check it out here:https://www.academia.edu/6241693/Standing_on_the_Shore_with_Sabaan

  2. I’m an American archaeologist who has worked in Greece for two decades. I was a regular member at the American School of Classical Studies at Athens and am on two of its committees. I know of no rule that it’s bad form to criticize past scholars like Robinson. In fact, I read the reviews of your book with great interest, although not unfortunately surprise, and I look forward to reading the book. It’s also not true that you must spend a year at the School to work in Greece; there are plenty of people who haven’t. The American School can be conservative, in good and bad ways, but (1) I’m not sure that Dyson is much of an expert on it as I see no evidence that he’s spent much time there, (2) things are much different now than they were even 10 years ago, and (3) there’s a big difference between the reputation that an institution may have and the political orientations of its constituent members, assuming that these were your reviewers. I suspect that part of the problem (not to deny your point that your anonymous reviewers, whoever they were, were made uncomfortable by the manuscript) is that most journals of Classical archaeology don’t publish much about the discipline’s history.

Comments are closed.