The Dunning-Kruger Effect

Errol Morris has an intriguing series of posts on the Dunning-Kruger Effect on his NY Times blog. The central question “How do we know what we don’t know?” is something central to both Anthropology as a discipline (How do we know what we don’t know about another culture?) as well as teaching (How do we help students come to realize what it is that they don’t know?). For these reasons I found this exchange between Morris and Dunning quite interesting:

DAVID DUNNING: Here’s a thought. The road to self-insight really runs through other people. So it really depends on what sort of feedback you are getting. Is the world telling you good things? Is the world rewarding you in a way that you would expect a competent person to be rewarded? If you watch other people, you often find there are different ways to do things; there are better ways to do things. I’m not as good as I thought I was, but I have something to work on. Now, the sad part about that is — there’s been a replication of this with medical students — people at the bottom, if you show them what other people do, they don’t get it. They don’t realize that what those other people are doing is superior to what they’re doing. And that’s the troubling thing. So for people at the bottom, that social comparison information is a wonderful piece of information, but they may not be in a position to take advantage of it like other people.

ERROL MORRIS: But wait a second. You’re supposed to benefit from feedback. But the people that you’ve picked are dunderheads. And you lack the ability to discriminate between dunderheads and non-dunderheads, between good advice and bad advice, between that which makes sense and that which makes no sense. So the community does you no damn good!

DAVID DUNNING: You know, I think that is an issue. Those among us who are in the 40th percentile, they’re not the best, but they’re not doing too badly. But people at the bottom, you’re going to have to be open-minded and you’re going to have some special hurdles, internal hurdles you have to get over. If people give you conflicting advice, congratulations, you don’t know how to choose. Yes, it is a tricky part of the problem.

I think this is a central problem for teachers trying to get through to the bottom 40% of a class. Often it seems that these students simply don’t do the work. But I think it isn’t so simple. I believe they don’t see the purpose of doing the work. While they understand that there is some information in the assignments that they are missing, they don’t see this information as adding up to new skills, new ways of thinking about the world which might be of benefit to them. For instance, in a class on documentary film, I had one student who was still, after a whole semester of learning about various approaches to discussing films (structure, form, narrative style, etc.) was unable to compare two films. He kept comparing the events portrayed in the films, but didn’t understand what I wanted when I asked him to focus on the films themselves rather than the events they portrayed. In some important way I failed to convince this student that there was anything of value to learn in my class.

As anthropologists we find ourselves in the opposite situation. We are often in the bottom 40% (or worse) in terms of our understanding of the culture we are trying to study (unless you happen to be working in your own culture). And while we differ from the student in the above example in that we know there is something we don’t know and are very motivated to learn it, we still “don’t get” a lot of what our informants try to tell us. I’ve long felt that there is a certain hubris to anthropological research, in the idea that you can spend ten to eighteen months somewhere and then attempt to speak authoritatively about it. The only thing saving us are our collaborators. As Dunning says: “it really depends on what sort of feedback you are getting.” But also on our ability to listen to that feedback, which I think is much harder than we would like to believe.

5 thoughts on “The Dunning-Kruger Effect

  1. Nicely observed, Kerim.

    My first take on the article was that it made a strong argument for the co-evolution of intelligence and sociability, since we humans need other people to see our own blind spots. You’ve put your finger on the simplistic nature of my assumptions. If you can’t sort out what’s right for yourself, what equips you to choose between the opinions you hear from others?

    Then, whoops! I wonder if what we’re moving toward is seeing this as evidence for the necessity of social hierarchy. Humans often find themselves stuck in situations where they don’t know what to do. The natural reaction is to seek advice from someone you trust. But how is that trust created? How are reputations for knowing more than others do about what needs to be done created? Incredibly fundamental stuff.

  2. A problem can be that those who are doing well are neither competent nor smart enough to be competent. They ban together to help each other and blackball the competent. So the competent ones are unsuccessful. Does a majority of dunderheads trump a minority of the competent?

  3. A very good point, Sil. One that does a lot to explain the recent financial disaster, among other things… Things always seem obvious in hindsight, but I imagine the few voices of sanity began to question their own judgement in the face of the dunderhead stampede…

  4. “And while we differ from the student in the above example in that we know there is something we don’t know and are very motivated to learn it, we still “don’t get” a lot of what our informants try to tell us.”

    I think there’s a real fallacy in this statement, and it is something I’ve noticed for a while in anthropology; something like the Dunning-Kruger Effect for anthropologists. It is a form of “domain dependency,” which creates a kind of aphasia is how we view cultures other than our own, yet ignore what it is to live in a culture. It is so amazingly obvious, and yet so ignored, and can be summed up by asking a simple question:

    If you went out tonight (to a bar, park, whatever), and asked a fellow cultural traveler to explain something about your shared culture to you (the reason or purpose for anything), are you likely to get a generally similar explanation or assessment from everyone? Or, are you likely to agree with the people you talk to?

    The answer to that question is obviously, of course not. Here we are, all educated elites with advanced degrees, probably speak more than one language, world traveled, similar culture, etc… and how much do we all agree on the causes or purpose of various phenomena? If we simply took people’s word for things in the field, and regurgitated verbatim what they tell us, then we are nothing but journalists. (This doesn’t even get into the fact that people lie a whole lot, especially to themselves). Unless who we’re talking to is a fellow social scientist and is working with empiric data, a research method, and various theories then we have to assume that the emic data we gather from them is simply that, emic data. It gives us one piece of the puzzle. It is a very necessary part of the puzzle and we can’t do without it, but there’s more to culture than what’s in peoples’ heads. How much information is the average person actually working with in their formation of opinions and beliefs.

    The minute you think you are actually gonna get someone in another culture to give you the keys to understanding that will lead to the development of nomothetic generalizations of what’s going on, is the minute you have to conclude that people (including yourself) are experts in your culture, and that there is no need for things like confirming, empiric data. In your culture you demand evidence, you demand confirmation, you don’t just take someone’s word for something, yet this is what so many anthros do in other people’s culture. This is the Dunning-Kruger Effect at work. Sit down and listen to people in your neighborhood meander on about the nature of religion and god for a few hours, and it’ll cure you of this false assumption.

Comments are closed.