A “Writing Culture” Moment for Psychology?

Psyche: #lfmf bro!

Ha! Just kidding.

The 30 March 2012 issue of the journal Science includes a news piece “Psychology’s Bold Initiative” on a possible moment of introspection for the discipline. Spurred on by some recent high profile academic fraud cases a cohort of scholars are leading a movement aimed at scrutinizing their field.

According to Science, many psychologists now feel that their field has a credibility problem. To my ear this underscores some of the differences between our disciplines and simultaneously calls to mind the critiques of Clifford and Marcus, et al.

The greater concern arises from several recent studies that have broadly critiqued psychological research practices, highlighting lax data collection, analysis, and reporting, and decrying a scientific culture that too heavily favors new and counterintuitive ideas over the confirmation of existing results. Some psychology researchers argue that this has led to too many findings that are striking for their novelty and published in respected journals – but are nonetheless false.

As sociologists and anthropologists are both prone to lament when the mainstream media wants a social scientist they turn to psychology. Even on the topic of human origins it seems the evolutionary psychologists have one up on the human ecologists and bioarchaeologists. Perhaps this envy has contributed to psychology being the punch line in some anthropological circles, but more profound than this are the very different ways the disciplines conduct research and the kinds of knowledge they claim to produce.

Whereas anthropology oft claims the mantle of science with a “but…” psychology appears to have no such hesitation. Rather than deconstruct the authority of science here, I will simply nod in the general direction of France. Psychologists self-identify as scientists. Being that they believe themselves to practice science it follows that one way they may right their vessel is to test the reproducibility of others’ conclusions.

This in and of itself is a radical notion. Reproducibility is one of the core principles of science but the current prestige economy does not reward this sort of work nor does the publication regime offer an outlet for its dissemination. What would be the incentive for expending one’s energies testing reproducibility in an academic culture that gives the highest rewards to new ideas? Just as limiting is the virtually unpublishable status of negative results, which may motivate scientists to identify false positives or structure their research agenda around what is publishable rather than what needs to be known.

A group of 50 psychologists have organized themselves as the Open Science Collaboration with the stated goal of systematically replicating recently published psychological experiments. This is very interesting to me. Instead of worrying about what the limitations of their field might be there’s a group out there setting up an empirical project to test where that limit is.

Jonathan Schooler, author of a study to be tested by the OSC, was quoted in the news story and I found his words to be very revealing.

I think one would want to see a similar effort done in another area before one concluded that low replication rates are unique to psychology. It would really be a shame if a field that was engaging in a careful attempt at evaluating itself were somehow punished for that. It would discourage other fields from doing the same.

Here is the bit that reminded me of Writing Culture. You don’t have to go far in the house of sociology to find scholars who perceive anthropology as having gone off a cliff in the 1980s, a historical moment epitomized by the radical reflexivity of Writing Culture. Even in anthropology its not hard to find old schoolers who think the whole thing has gone to pot and Geertz is the villain. But the kernel of the Writing Culture critique is the same as the OSC movement: its was a call for more empiricism not less.

Correct me if I’m wrong, but reproducibility doesn’t really seem to have a place in contemporary cultural anthropology. On the one hand this makes methodological sense. In ethnography I am my own instrument. The culture I experience is different than the culture you experience even if we’re in the same place at the same time.

But its worth asking again, once more with feeling, how is it that we believe what we read in the journals? I mean, we can disagree about the meaning of an event or whether this idea from Edward Said really goes with that idea from Richard Price. But for the most part if somebody describes Carnival in Trinidad don’t we accept that description and move onto the interpretation?

What the OSC has done is select studies from three high-impact psychology journals published in 2008. “They reasoned that articles published during this time frame are recent enough that most original authors can find and share their materials, yet old enough for the OSC to analyze questions such as whether a study’s reproducibility correlates with how often it has been cited subsequently.” So far the response from study authors has been positive.

What would it look like to refashion this testing of reproducibility on anthropological terms and check up on publications to see if authors really knew what they were talking about? I’m not talking about the rare case of outright fraud, where authors are willfully deceiving readers. But could you pick, say, an essay on Indonesian cross-dressers out of Cultural Anthropology and go to Indonesia and find what they were talking about?

I don’t know, that sounds kind of crazy. Who would pay for it?

Matt Thompson

Matt Thompson is Project Cataloger at The Mariners’ Museum in Newport News, Virginia, and currently working on a CLIR ‘hidden collections’ grant to describe the museum’s collection of early 20th Century photography. He has a doctorate in anthropology from the University of North Carolina and a Masters in information science from the University of Tennessee.

10 thoughts on “A “Writing Culture” Moment for Psychology?

  1. Went in expecting an argument against science based on a caricature of science…

    Then things went much better than expected!

    You hit the nail on the head linking the recent fraud cases to the prestige economy of academia, which in scientific research communities is manifest in a bias toward publishable work and thus the production of false positives.

    Good show!

    A scientifically inclined anthropologist named Benjamin

  2. The culture I experience is different than the culture you experience even if we’re in the same place at the same time.

    True, but not entirely true, and there may be the solution to the problem. Am I wrong to believe that today’s anthropologists are more likely than not to work in places that, if not already studied themselves, are located in regions where a lot of anthropologists have also been working? In places where, as Marcus and Fischer pointed out in Anthropology as Cultural Critique, the anthropologists aren’t the only ones. As Geertz notes in the introduction to Islam Observed, ethnographers work in microscopic situations but their insights, if, indeed, they are insights, are tested in larger conversations, and nowadays those conversations include historians, other social scientists, journalists, and, yes, the natives themselves, in any or all of these roles. Consistency with or explainable difference from what others’ observe is a powerful test of validity.

  3. There have been some attempts at reproducing results in anthropology. There was, for instance, a re-evaluation of Margaret Mead’s claim in Sex and Temperament in Three Savage Societies to have discovered a group of people, the Mundugumor of New Guinea, practicing alternating descent. That’s where boys inherit property from their mothers and pass it on to their daughters, who then pass it on to their sons. Mead claimed that this was indigenously called the “rope” system. It was, and still is, hard to believe that any society would have anything like such a counter-intuitive rule of descent because of the clear potential for conflicts, although Mead provided quite extravagant descriptions of behaviours related to it, like the animosity between a husband and wife in passing on their goods, as if she had witnessed such things.

    Well. I can’t remember who it was – I’m thinking Nancy Scheper-Hughes, but I’m not sure – but somebody went and fact-checked in New Guinea decades later. Mead’s original fieldnotes were also checked. They asked about the word Mead had supplied for the system of alternating descent, and found out that it just meant, “rope”, and not “alternating descent”. It also turned out that the Mundugumor had corporate patrilineal descent groups, both when the facts were checked and when Mead was writing. Mead wrote about them in her fieldnotes.

    Which means, I suppose, that perhaps fieldnotes would be a good place to start verifying.

  4. I used to teach an undergraduate course called Voices from the Third World. Its premise was that the main event of the 20th century was the anti-colonial revolution, whereby peoples coerced into world society by western imperialism fought for their own independent relationship to it. How could anthropologists study and learn from that, something we do very badly? We need access to what their own intellectuals produced, but they were rarely anthropologists. They wrote novels, poetry, history, biography, polticial tracts or made films and plays — Tagore, Gandhi, Fanon, James, Achebe, Marquez and so on.

    The Cambridge students were perplexed. They were used to writing essays based on the idea that everything in The Nuer is true because certified as objective by the profession. This is the muddled premise of scientific ethnography. It means that anthropologists are trained from an early age to suspend disbelief. What they are not trained to do is to exercise critical judgment, as they are in the main humanities.

    So how to gain anthropological knowledge from reading fiction, something you know was “made up”? Yet some ethnographers report snippents of casual conversation as if they were objectove fact and some novelists spend years researching their topic to make sure that don’t instert something counter-factual. If Writing Culture taught us anyhting it was that we can’t afford to suspend critical judgment when it comes to understanding the poetics and politics of ethnography.

    I tried to answer the students’ question by saying that development is about making a better world and how we go about that depends in part on how we make worlds of the imagination. Works of fiction and ethnographies both do that with more or less success. One thing we can learn from studying them both is how.

  5. I’d say we do have a form of replicability testing in anthropology, and it consists, as the Mead example discussed above suggests, in people working in the same area. Unlike the Mead example, the results generally speak well for what we do: I think the quote below, from Annette Weiner’s Trobrianders of Papua New Guinea captures my own experience of reading people who work where I do.

    In 1971, before my first trip to the Trobriands, I thought I understood many things about Trobriand customs and beliefs from having read Malinowski’s exhaustive writings. Once there, however, I found that I had much more to discover about what I thought I already knew. For many months I worked with these discordant realities, always conscious of Malinowski’s shadow, his words, his explanations. Although I found significant differences in areas of importance, I gradually came to understand how he reached certain conclusions. The answers we both received from informants were not so dissimilar, and I could actually trace how Malinowski had analyzed what his informants told him in a way that made sense and was scientifically significant—given what anthropologists generally then recognized about such societies. Sixty years separate our fieldwork, and any comparison of our studies illustrates not so much Malinowski’s mistaken interpretations but the developments in anthropological knowledge and inquiry from his time to mine.
    This important point has been forgotten by those anthropologists who today argue that ethnographic writing can never be more than a kind of fictional account of an author’s experiences. Although Malinowski and I were in the Trobriands at vastly different historical moments and there also are many areas in which our analyses differ, a large part of what we learned in the field was similar. From the vantage point that time gives to me, I can illustrate how our differences, even those that are major, came to be. Taken together, our two studies profoundly exemplify the scientific basis that underlies the collection of ethnographic data. Like all such data, however, whether researched in a laboratory or a village, the more we learn about a subject, the more we can refine and revise earlier assumptions. This is the way all sciences create their own historical developments. Therefore, the lack of agreement between Malinowski’s ethnography and mine must not be taken as an adversarial attack against an opponent. Nor should it be read as an example of the writing of ethnography as “fiction” or “partial truths.” Each of our differences can be traced historically within the discipline of anthropology.

  6. I’d say we do have a form of replicability testing in anthropology, and it consists, as the Mead example discussed above suggests, in people working in the same area.

    Yes. This is precisely the kind of replicability we should be going for. It probably won’t answer the kinds of questions spoken of in the post – Carnival in Trinidad, etc; I should think that varies quite a lot. The number of lineages in a Batak village, their names, and their rules for recruitment, however, are much more concrete and stable. It’s easy to check something like that – about as easy as visiting the USA and finding out that it’s a nation with a President, whose name is Barack Obama.

    I think the Mundugumor example reflects well on anthropology. It doesn’t reflect well on Margaret Mead, though.

    In any case, empiricism is great, but you can have too much – so much that you’ll never be satisfied with any kind of generalisation or abstraction. Writing Culture is very much in that camp.

  7. The relationship between theory and empiricism is an interesting one. When I think about it I tend to begin with a framework proposed by Noam Chomsky. Consider scientific method as an evaluation procedure. The inputs are a body of observations, the data plus, and this is the critical point, at least two theories. The method provides a way of deciding which of the two theories is superior give the data in question. Note what it doesn’t do. It doesn’t identify the one and only Truth of the matter. Neither does it establish that one of the two theories is Right and the other Wrong. All it does is evaluate the two theories and say, given what we know now, the data one theory is better than the other.

    But then the question arises, what criteria to use. The usual candidates are simplicity and precision. Given two theories with the same degree of precision, choose the simpler one. Given two theories that are equally simple, chose the one that is more precise. But what about the case in which the more complex theory is also the more precise? That’s a judgment call; but other things being equal choose precision over simplicity.

    In the history of physics, for example, Copernicus’ solar centric cosmology and Ptolemy’s terra centric cosmology are equally precise. Both predict the movements of the planets with an equal degree of precision. But Copernicus needs fewer epicycles to account for retrograde movements. The palm goes to Compernicus. Then along comes Kepler, who postulates elliptical instead of circular orbits and needs no epicycles at all. Now it is Kepler’s turn for the limelight. Newton’s mechanics explain why the orbits are elliptical and then, noticing that, even given the precision of Newton’s calculations, Mercury doesn’t appear to be where it ought to be, Einstein develops the theory of relativity and explains the bending of light, a new complication, that explains more precisely why we see Mercury where we do. Thus science progresses.

    In anthropology, however, theories are virtually never mathematical laws. Even our most elegant theories are, at best, sketches for stories. That said, the same general principles apply. Given a set of observations, our data, and two competing stories, we are in the position of Hercule Poirot or Miss Marple. We are looking for the simplest story that accounts for all the facts at hand. Given two stories that account for the facts, we prefer the simpler one. But if the simpler story fails to account for facts that a slightly more complicated story explains, we choose that one instead. The real fiction here is the notion that only a certain small, finite set of facts are relevant,
    In the field, we are still struggling to discover what might be relevant and, truth be told, there is always more to be learned.

    Thus we have to keep in mind all the elements of Chomsky’s model and remember not only that a new and better third theory may come along, e.g., Kepler coming along after Copernicus and Ptolemy, but also that new evidence may reverse a previous judgment. In the best of all possible worlds, our new theories don’t replace our old theories. Instead, they offer improvements on them, improvements judged to be improvements either because they are simpler or they account for new evidence.

  8. With Writing Culture the problem is less with theoretical explanations of problems and more with finding out what the problems are. If you’re always trying to avoid the accusation of claiming to have uncovered “cultural truths”, you won’t generalise productively about anything. A Writing Culture enthusiast won’t produce anything like the models of Political Systems of Highland Burma, because even the first step of saying that there are ranked clans linked in marriage alliance would be overstepping the bounds of Writing Culture‘s rigid empiricism, even though those models tell us a lot, and even though they’re hedged with plenty of contingencies.

    If there were an astronomical version of Writing Culture, it would be that Ptolemy, Copernicus, and Kepler are all trying to state too much. Sure, the sun appears to rise today, but maybe it won’t tomorrow.

  9. I hear you Al. Since I am here neither to defend nor to condemn Writing Culture (don’t know enough about it to be fair either way), I’m going to step back from this.

  10. Another example: not sure if it counts as anthropology, but the noted linguist Bernd Heine devoted a whole article (The Mountain People: Some Notes on the Ik of North-Eastern Uganda) to explaining how Colin Turnbull grossly misrepresented, misunderstood, and maligned Ik culture in The Mountain People. He could see many of the same people, and spoke the language much better – but then again, he fortunately never got to see them in the middle of a famine, which would perhaps be required for strict reproducibility…

Comments are closed.