QDA or not QDA?

For years I’ve been asked by students “Which Qualitative Data Analysis software should I use?” I have no effing idea. Despite the fact that I am a Scholar of Teh Internets, I’ve never used QDA software. There are lots of reasons: a) it’s proprietary b) it’s expensive c) none of my advisors or fellow students or any journal editors ever expected me too d) etc. etc.

But recently I reviewed a paper that employed QDA to try to make a point. In my estimation it added exactly nothing to the paper. Conceptual distinctions were fuzzy, terms were assumed to refer to concepts when they may only have been co-occurent in different samples, the distinctions apparently provided by the software were fuzzy at best, at worst totally indistinct, and most annoying of all, the authors could not say what their methodology consisted in, only that they had used software to do something.

Now I could rail against the misplaced scientism and ideological blindness of QDA here, but I do not (want to) think this article was in any way exemplary. Rather, what I want to know is: what are the best articles where QDA has really made a difference? What are the canonical articles? Is there a review article of the best of the best of QDA results? When Atlas.ti costs $1800 a pop, and Nvivo costs $600, doesn’t it seem like there should be a really clear list of all the super advances we have made because of it? Really, shouldn’t the “greatest hits of QDA” be something all anthropologists can easily recount?

ckelty

Christopher M. Kelty is a professor at the University of California, Los Angeles. He has a joint appointment in the Institute for Society and Genetics, the department of Information Studies and the Department of Anthropology. His research focuses on the cultural significance of information technology, especially in science and engineering. He is the author most recently of Two Bits: The Cultural Significance of Free Software (Duke University Press, 2008), as well as numerous articles on open source and free software, including its impact on education, nanotechnology, the life sciences, and issues of peer review and research process in the sciences and in the humanities.

45 thoughts on “QDA or not QDA?

  1. I share your scepticism of QDA but also wanted to try new things for my PhD. I found an open source QDA programme called WEFT QDA that I am playing around with. It was developed by an anthropologist and is very easy to use. And have I mentioned that it is free?

    I am not sure yet whether I will use it all the way through my PhD. At the moment it is fun to play with.

  2. The first point to be made about qualitative data analysis software is that it does NOT analyze qualitative data. Nonetheless, I am currently using a QDA package. I use it for much the same reasons that I use a word processor instead of a typewriter — it makes some tasks easier.

    I am currently trying to understand a large number of hour-long narratives each of which is followed by about an hour-long interview. It’s a lot of text. Now I could cut all of that text up into little pieces, label them all, and sort them in various ways, but it would be a mess and it would be very difficult to control or understand. I find that the QDA allows me to label and retrieve portions of text with some ease. I am working with narratives of alcoholics, so, for example I might label portions of a number of narratives as “drunk driving.” I can then easily retrieve all of these and even try to analyze them myself. The QDA offers a good interface for this kind of activity.

    But, the critical thing is that the software is not a method or a theory on its own. I am not sure one even needs to mention it in describing methods. After all, one doesn’t describe, anymore, the fact that one uses a word processor.

    I also completely agree with your point about the cost of these programs; they are way overpriced. If I had not been able to get my institution to purchase one for me, I would not be using it.

  3. Actually I used to think that QDA software serves well when working on larger projects, maybe even as a team. Until I started trying to work on a larger project as a team. A team that has 6000 m between its members. Then the problems started: like in the case of word processors you have to use the software suite and the the same version. You have to make sure that there is shared access to the files which can be tricky. In the end we just exported the code list, uploaded it to to google docs and started analyzing.

    By the way: I tend to use not a third of the features a QDA suite offers, actually I would just like it to take care of my codings and memos and share them between my collaborators.

  4. I wish I was trained in using qualitative and quantitative software packages during grad school. It would certainly be more appealing to employers than my knowledge of Hmong medical practices. There seems to be a complete lack of interest in preparing anthro grads for the non-academic job market, which the vast majority of us are entering.

  5. Really, shouldn’t the “greatest hits of QDA” be something all anthropologists can easily recount?

    How would you know a QDA program was used in the first place? I know some social scientists have a strange fetish in which they feel the need to identify the software they used in the course of putting an article together (I won’t mention any names but their articles oddly enough don’t mention the brand names of the plumb-bobs, brushes, and trowels used during the data gathering phase of their work) I don’t think mention of software has any place in an article unless the article is about software.

  6. I don’t think mention of software has any place in an article unless the article is about software

    Oh come on. Of course people mention what software they used… all the more so if they are doing quantitative analysis– any hope of replication, or even critique goes out the window if you can’t say how you did what you did. Whether you used Atals or Nvivo (or Stata or R) is probably irrelevant, but your methodology is not.

    What I’m asking is: what’s an example of a great article that couldn’t have been done without QDA software? It seems idiotic to me to assert that QDA software is nothing more than a glorified word processor… if that’s true then it really has no reason to exist. But I don’t believe it.

    @wormtongue: really? How many jobs out there in the so-called real world require use of QDA… I mean I am honestly interetested… who’s using it and for what purpose? And don’t whine… if you didn’t take stats in grad school, it’s not your grad program’s fault. Grad programs in anthropology prepare people to be anthropologists, not non-academic jobholders. I’m tired of being told that we don’t prepare people for the real world. Suck it up.

  7. I think my comparison of QDAs to word processors has been misinterpreted. What I meant to say is that the QDA is simply a tool; it is not a methodology or a theory. CKelty says that if it is just a glorified word processor, it has no reason to exist. Would you also say that if a word processor is just a glorified typewriter, it has no reason to exist, and if a typewriter is just a glorified pen, does it have no reason to exist. The QDA is just a tool like all of the others but one that is focused on making handling and manipulating data easier and more visually comprehensible. You can do the same thing with 3X5 cards or post-it notes, but there is definitely a value to a better tool. (though it may not be worth the additional expense).

  8. well I can reflect on the methodological and theoretical implications of using a word processor instead of a typewriter, and some have written books on the subject… so I guess I don’t know what it means to say something is “just a tool”– there is always a difference. And YOU may think it’s just a tool, but it would be disingenuous to suggest that this is how legions of first year grad students (and many more advanced scholars who write important sounding things about having “coded” their “data”) think of it.

    So I’ll just keep saying this over and over again:

    What is an example of an article or book that shows the power of using QDA?

  9. Oh come on. Of course people mention what software they used… all the more so if they are doing quantitative analysis– any hope of replication, or even critique goes out the window if you can’t say how you did what you did. Whether you used Atals or Nvivo (or Stata or R) is probably irrelevant, but your methodology is not.

    (uh-hum cough) Did someone who seems to think that arithmetic and calculators are the same thing just take a tone with me?

    So I’ll just keep saying this over and over again:

    What is an example of an article or book that shows the power of using QDA?

    I must say, you are being quite belligerent for someone who is asking people to help him answer a question.

  10. Actually, Atlas TI is not that expensive for students: $99 for a student license, which is the same software as the higher prices, it just can’t be paid for by an institution.

    I’ve never used any qualitative software but it would seem to be useful for organizing large quantities of textual data. If I were to help it find, say, every instance of the use of a particular term by across 1000 documents, and I then went on to analyze some of these occurrences and, by chance, a conclusion made it into a publication, I don’t think I’d be compelled to mention the software. Why would I? (Of course, I say this will no knowledge of Atlas TI or how it is commonly used.)

  11. I would advise anyone that is wondering if the cost of a QDA is worth it to them to download a free version and try it out. Atlas.ti can be downloaded from their site for free. This version is fine for small projects, as it only allows 50 codes. Other than that, and a few other limiters, it’s identical.
    I’ve never worked in a rural area, so I’ve used both qualitative and quantitative programs for everything I’ve done. The ease at which you can pull codes up and compare data from different respondents, link codes to pictures and video, and combine interviews from group projects is a real time saver. The main task of qualitative analysis is organizing data in a way that makes it manageable, which is all these programs do. You can get a frequency count for how many times someone says something, how often a word is used, etc…

    It makes team projects much easier, but only if everyone is properly trained. Atlas is tricky. You have to save source docs. to a rich text format, or you won’t be able to combine the work of different people. If someone saved it in another format, they would have to recode everything. There’s also a big issue of intercoder reliability when you’re working with people from fields like marketing, or education. The programs make coding so easy that some people will code things wrongly, because they can.

    Finally, the latest versions of Atlas.ti work seamlessly with SPSS/PSAW, so you can get basic statistical data from the hermeneutic units (interviews). On a more advanced level, you can also now pull out embedded social networks from interview and field note data. Check out this article:

    From Interviews to Social Network Analysis: An Approach for Revealing Social Networks Embedded in Narrative Data. Willie L. Mckether, Julia C. Gluesing and Kenneth Riopelle: Field Methods 2009; 21; 154.

    As far as the money… It might not be a nice thing to say, but the student version, which is the same as the full version, is only about 200 dollars. If you know any students…

  12. “How many jobs out there in the so-called real world require use of QDA… I mean I am honestly interetested… who’s using it and for what purpose?”

    None that people outside of anthropology know about, but then qualitative research is something that few people do well in the “real world.” What passes for ethnography is varied and not regulated.

    To answer your question, you would use this primarily for business and design anthropology. For business consulting, you’d interview a good cross-section of the company or dept., so that you get a good sample from different levels of the bureaucracy, and depts. QDAs then help you to analyze the qualitative data in a way that organizes who says what. It’s a pretty safe bet that if you’re working with a company that communication problems are going to be a core issue, and this helps you see where communication is breaking down. I’m not sure if there are any direct articles about the use of ODAs specifically, but if you know business anth., then you know what I mean. I’ve never see anyone cite using a specific ODA, unless they were talking about it in the journal Field Methods.

    In design anthropology, you would use QDAs to make transcriptions for video taken of people using something, or learning how to use something, etc… The process is very involved and time consuming, and you really have to have a team of people to do it and cross check each other, so I wouldn’t want to add more time by doing in old school. The best article for this is probably:
    Interaction Analysis: Foundations and Practice. Jordan and Henderson. Xerox Palo Alto Research Center.

    I would think that education consulting, border/refugee/immigrant research, environmental anth., and other fields are the same.

  13. (uh-hum cough) Did someone who seems to think that arithmetic and calculators are the same thing just take a tone with me?

    wtf?

    I must say, you are being quite belligerent for someone who is asking people to help him answer a question.

    I have not even begun to be belligerent yet.

    If I were to help it find, say, every instance of the use of a particular term by across 1000 documents, and I then went on to analyze some of these occurrences and, by chance, a conclusion made it into a publication, I don’t think I’d be compelled to mention the software. Why would I?

    First off, you don’t need expensive software to do that. But regardless, I don’t understand this desire to hide the function of the software in what you are doing. If I did a thousand google searches in order to find the ten articles that were relevant to my literature section, I would say “I did a literature search.” If I collected a thousand blog comments, marked every instance of troll-baiting, then gave them all unique IDs that were correlated with URL, ID of the author and various other data, then ran a regression analysis in a stats program to determine who was the worst offender (null hypothesis = MTBradley), then I would CERTAINLY say that in an article. Why is it that using QDA allows one to not specify the method?

    And why aren’t there any articles that are proven testimony to the power of organizing large amounts of “qualitative data”? I know there are. I’m actually asking nicely… I promise 🙂 I will give out gold stars to the people who come up with the best articles.

  14. “Why is it that using QDA allows one to not specify the method?”

    I don’t think that using any QDA allows anyone to do that, rather anthropologists are notorious for not stating the replicable methodology in journal articles. I’ve already given you two articles, but another one hit me just now. It was written by one of my mentors, who is a competent and well regarded business anth.

    An Anthropological Approach to Study of Organizational Change: The Move to Self-Managed Work Teams. Ann Jordan. Practicing Anthropology 21 (4): 1999.

    In it she uses After, but notes that Nudist or Atlas are also useful. I think this is the article you are looking for.

  15. I first used askSam, then switched to nVivo, but to accommodate some colleagues (educational package for a number of licenses), I’ll be switching to Atlas.ti soon.
    Actually, the thing I use the most is Evernote. It’s free (I do pay for the pro version since I come close to the monthly limit), it syncs over numerous platforms, and it is easy to search for items. It works well with teams, for sharing.
    My anthropologist spouse swears by OneNote (which like me she uses for everything). It was free, but she recently paid for an upgrade.

  16. @Rick. those articles are useful, but they aren’t examples of conceptual advances made because of QDA (or any other methodology for that matter). The Ann Jordan article looks great for many purposes, but not that one…

    alternatively, what you and others are pointing out is that there is a distinctive mode of analysis in cultural anthropology, for which QDA is (apparently) a very helpful tool. That methodology, it is true, is not often specified in a “methods section” way…. but it is hardly unstated. In fact, I would argue that it forms the core of the identity of the discipline. My frustration with QDA is that it seems like a way to get around having to stake your claims in an article… at least, I’m still smarting from having had to review an article in which this is the way it was used, for which I called the authors to task.

    There are *lots and lots* of articles about how to use QDA, about how to select which software, about how t collaborate in large teams and how to keep your desk tidy. What I’m looking for is the top 10 articles that are demonstrations of the power of QDA.

  17. Hey, for those who do find QDA software valuable, and want to see what options there are out there, here’s a website with a fairly comprehensive list of available software – including free, opensource, paid, Mac, Windows, and Linux software. I’ve used TAMSAnalyzer for a project where I analyzed several hundred letters to the editor. I found it very helpful for finding patterns in the letters – something which I might have done just by reading them again and again, but which took a lot less time with coded data. I did the same thing, on a whim, with Obama’s nomination speech from the 2008 DNC. I found some interesting things there too.
    I liken QDA software to Bruno Latour’s “second notebook” from Reassembling the Social – ethnographic data that can easily be arranged either by date or by theme. There is a rich tradition in anthropology of using notecards or other methods to do this same thing (as Bill mentioned), but the software does make it a lot easier.
    That said, I’ve also used it in some projects where it didn’t help at all, so if you don’t find it helpful, I can understand that. Just to see if I could fulfill your request for an article or book, and get those gold stars you promised, I went over to AnthroSource and did a few searches. I tried QDA, NVivo, Atlas.ti, “qualitative data analysis”, HyperResearch, and a few others. In all of those searches I only came up with two articles (both of which were “how to” type articles), so either (a) most anthros agree with you that QDA software is useless, or (b) most anthros don’t mention having used QDA software and instead describe it as something else (or (c) the anthrosource search function sucks). Searching for terms like “textual analysis” or coded data gets you more hits, but not a lot more.
    Hope that helps – do I get a gold star for trying? 🙂

  18. wtf?

    Indeed. You said

    Of course people mention what software they used… all the more so if they are doing quantitative analysis– any hope of replication, or even critique goes out the window if you can’t say how you did you did. Whether you used Atals or Nvivo (or Stata or R) is probably irrelevant, but your methodology is not.

    I suppose then that the above is simply muddled, because the clearest reading seems to me to be that you are equating Atals [sic] and statistical technique[s]. I’m not in grade school so I am aware that transparency is foundational to the scientific method.

    But regardless, I don’t understand this desire to hide the function of the software in what you are doing. If I did a thousand google searches in order to find the ten articles that were relevant to my literature section, I would say “I did a literature search.” If I collected a thousand blog comments, marked every instance of troll-baiting, then gave them all unique IDs that were correlated with URL, ID of the author and various other data, then ran a regression analysis in a stats program to determine who was the worst offender (null hypothesis = MTBradley), then I would CERTAINLY say that in an article. Why is it that using QDA allows one to not specify the method?

    The above is more or less the point I was trying to convey in my original reply which you responded to like a patronizing schoolmarm. In the one to two years it takes your article to go from submission to publication a piece of software is probably going to see a new version. A reader looking at it fifteen years from now is probably going to at best know it as legacy software. If there is anyone around to read it fifty years from now they’re going to care wtf-all about the long since deprecated program used to organize the data and do the stats. As you correctly point out, what they will want to know about are the particular statistical techniques utilized.

    There is, by the way, a thing called ‘intercoder agreement’ into which you might care to look. If you get a few other people to look at your dataset they might well point out that I am not even the best choice in this thread for your H0.

    I once attended a seminar meeting at which the professor—unsolicitedly!—referred to you as “brilliant.” You really have taken the shine off that one.

  19. @CK. “Suck it up.” Very mature. Look at any company that does research in the social sciences. SPSS, SAS, Atlas.ti, NVivio, etc. are all quite useful. Your grads might be trained anthropologists, but, let’s face it, hardly any companies are going to drop cash for them to do traditional fieldwork. Being dismissive of non-academic jobs isn’t helpful at all.

  20. Wow. Who’d have thought so much heat could be generated by a discussion of QDA? Anyway, it seems that at the heart of this is a distinction between QDA as tool and QDA as method. As I’ve always understood it, the use of Atlas, NVivo and the like is an over-priced means of avoiding index cards, glue and scissors. As has already been suggested, QDA is thus a tool for analysis first and foremost – it is neither a method (as a means of gathering data) nor a methodology (in the sense of a philosophical framework for collecting this data).

    I’d think this is the way this software is usually used, and explains why there are few papers on it, just as there aren’t a great number on index cards. All you get is a throwaway line about which software package was used. With quantitative data, a similar line describes using SPSS or whatever.

    In order to demonstrate the ‘power’ of QDA, you’d presumably have to revisit old data with the new tool – which isn’t done very often. Furthermore, the inductive nature of much qualitative research means that questions aren’t asked until they emerge through analysis. Consequently, different tool, different questions -but not necessarily different ‘power’. In contrast, different quantitative approach don’t (shouldn’t?) affect the answer. Stats done with SPSS should be just the same as one using STATA, or using a wax tablet and abacus.

    One interesting question to emerge is how much you can credit any methodology with advances in research, particularly qualitative ones whose power derives (in part) from the skill with which they are used, and in which data generally isn’t replicable. If people use QDA as a shortcut for presenting data properly, that may be more to do with the skill of the user than the tool itself.

  21. First, a disclaimer. I have never used QDA software. I am, however, fascinated by this discussion, and the following questions reflect honest ignorance.

    1. Is it fair to say that all of the packages under discussion depend on the researchers to code the data?

    2. If so, how are classic coding issues (validity, reliability) being addressed by those who use QDA?

    3. What is the relationship of these programs to semantic analysis software that works directly with strings of text to identify patterns and concurrences? I think, for example, of Crawdad (http://www.crawdadtech.com/)

  22. “1. Is it fair to say ”

    Yes. Actually, you can have them automatically place a code every time something is mentioned, but I’ve never done that. This software was originally designed for textual analysis, like biblical scholarship and the like.

    “2. If so, how are classic coding issues (validity, reliability) being addressed by those who use QDA?”

    Same by those who don’t.

    “3. What is the relationship of these programs to semantic analysis software ”

    You can do things like build visual networks, similar to the socio-graphs in SNA, with text instead of people to see how they are related. The software doesn’t do it for you though, not the one’s I’ve used.

    “@Rick. those articles are useful, but they aren’t examples of conceptual advances made because of QDA (or any other methodology for that matter). ”

    I’m not sure what you’re looking for here then. I don’t see how any “conceptual advances” have been made by quantitative software either. You can do all statistical analysis with basic calculator and time. It’s all r, variance explained, type stuff. All the conceptual advances have been made by mathematicians, statisticians, or social scientists. Actually, I could make a pretty solid argument, backed up by a lot of literature, that quantitative software packages have done as much harm to science as good. The vast over reliance on post-hoc tests of significance, step-wise regression methods, etc… have lead very weak science.

    We’ve already stated that QDAs don’t let you do anything that you couldn’t do by hand, they just make it a lot easier and faster. However, anyone can play with a statistical package and report numbers, even if they have no idea what they mean, if they properly satisfied statistical assumptions (like homoscedasticity for a regression model), what the confidence interval of a statistic is, etc… Probably about 5-8% or so of quantitative studies as done in a robust way, with all the information needed by others to replicate the study.

    That being said, I think the only methodology which has been conceptually helped with the use of software, and is held to a higher standard of rigor by its community, is social network analysis. One of the reasons for this is that you have to know SNA if you want to use a program. So, while QDAs haven’t lead to staggering conceptual jumps, they haven’t hurt either.

  23. Another question (also reflecting honest ignorance, since I’ve never used QDA software before either): does anyone recommend any particular package (or using QDA at all) for a small-scale, single-researcher project? I’m a grad student in the middle of my first field summer, and I currently just use MSWord, a bunch of multi-colored highlighters, and other scissors-glue-index-card-type coding methods…

  24. Like Fuji Lozada, I am a fan of Evernote. It handles text, photos, and a variety of other media. Text color, size and style can be used for highlighting. Tags and notebooks can be used to organize your material. Automatic backup in the cloud and synchronization across multiple machines are also attractive features (assuming, of course, that you have access to the Net). You can sign up for free and move up to a paid subscription if you need all the bells and whistles.

  25. I take the original posting to ask a question about the analytical power gained, lost, or more importantly, obscured by QDA software. To the extent that each step in the research process (from question formulation to presenting results) contains analytical decisions, which open some vistas but close off others, ckelty’s question is both important and warranted, if a little ham-fisted.

    For a canonical list of articles using QDA software I think you will have to look to a field heavy with grounded theorists as most QDA packages claim some sort of affinity with GT.

  26. I am very interested in using open coding, and I think it is a great idea for grad students no matter how ‘cultural’ they are — any number of note-taking programs can get it done for cheap.

    I think CK’s question raises a sort of category error at work when it comes to what appear, on the surface, to be discussions of ‘qualitative’ versus ‘quantitative’ research strategies. I’m hardly a quanty sort of person but I think the reason that CK feels QDA can’t ‘add value’ to cultural anthropology is that the way cultural anthropology works rests on premises that are basically pretty different from some of the most deeply-held intuitions of our discipline (this is, incidentally, why the Russ Bernards of the world don’t get much traction in the anthropological world, despite the good work they do). Cultural anthropologists seem themselves and their socialization to a lifeworld as a source of insight to be mined, not a bias to be corrected or a source of hypotheses to be validated.

    Equally, dealing with our notes or other material via repeated rereading (here’s a note to John: via the parallel-processing method Andy Abbott describes) is not a flawed system that needs to be formalized in order to be more perfect, it is just different from a more rigorous QDA system. Each method has its benefits, but they are, I think, different methods of proceeding.

  27. “does anyone recommend any particular package (or using QDA at all) for a small-scale, single-researcher project?”

    I’d download the free demo copy of Atlas.ti from their website. It’s the same as the full version, but it just won’t allow large projects that require over 50 codes. They have video tutorials that show you how to use it. Remember to save source documents to rich text format before importing them into the program.

  28. Equally, dealing with our notes or other material via repeated rereading (here’s a note to John: via the parallel-processing method Andy Abbott describes) is not a flawed system that needs to be formalized in order to be more perfect, it is just different from a more rigorous QDA system. Each method has its benefits, but they are, I think, different methods of proceeding.

    An interesting observation. The assumptions that could use some critical reflection are those implicit in calling QDA more rigorous, where rigor is equated with definitions sufficiently clear to count the items in question. That, however, was not what I was thinking of when I asked my questions.

    In my own current research I am using what might be called natively coded data. The relevant categories are those employed by the people I am studying and are used by the natives themselves in compiling lists of the organizations and individuals involved in particular projects. Thus, at least at the basic data level, the usual problems of validation and reliability do not arise. The value of the tools I use lies in their ability to rapidly analyze and create visualizations of data whose volume makes human processing humanly impossible. The software functions, in effect, as a sociological microscope, revealing structures invisible to the naked eye. Using it effectively requires developing techniques functionally equivalent to those biologists use when staining their slides to highlight different types of structures.

    I was wondering, then, if the QDA packages mentioned contain any safeguards against arbitrary codings resulting in flawed analyses.

  29. I think Rex is basically right here that the debate cleaves along the lines of what researchers consider to be the appropriate goal of anthropology and thus its appropriate methods/instruments – i.e. where you sit on the ‘goal gradient’ (which ranges from curiosity to social engineering) will shape your response to the utility of ‘instruments’ (perhaps ranging from phenomenological participant observation to probabilistic survey), and thus the tools you use to interrogate the data produced by those instruments. I was thinking the same thing during the HTS debates – is culture an experience that is lived or a terrain to be mapped? Your response guides your attitude.

    Also, I second Michael Scroggins’s suggestion to look at Grounded Theory – which comes from Sociology … Glaser & Strauss etc. – where I think you will find plenty of claims made about cases where QDA and coding in particular makes a difference. Personally I am not sure it isn’t all some kind of Cult.

    And also also, all this searching for studies where the use of QDA software has made a difference disguises the fact that most of the pressure to use such software is filtering into cultural anthropology from the social scientist labor market (government, NGOs, market research, the Armed Forces) and its attendant bureaucratic conventions, rather than its benefits to the kind of research anthropologists have traditionally done per se.

  30. Cultural anthropologists seem themselves and their socialization to a lifeworld as a source of insight to be mined, not a bias to be corrected or a source of hypotheses to be validated.

    That is very beautifully said, Rex! And I apologize in advance for all the times I am going to repeat this without crediting you. IMHO there is a true value in natural history-style documentation, but the combination of that sort of documentation with what you are referring to as ‘socialization to a lifeworld’ is truly powerful. Particular facts, attitudes, and events are much clearer in light of the gestalt (and of course there is a feedback between them). My big criticism of ethnographic fieldwork as carried out by cultural anthropologists today is that it has eschewed the former and deified the latter.

    You [Rex] have asked more than once on this blog why contemporary cultural anthropologists seem to lack on interest in material culture. I would assert that it has a lot to do with this documentation/socialization imbalance. I think comparison of publications resulting from cultural anthropologists’ ethnographic fieldwork and archaeologists’ ethnographic fieldwork really gets at this.

    Would anyone be willing to agree with me that there is something to be said about the way QDA software is used by cultural anthropologists and the verbocentric turn in the subdiscipline? My limited understanding of the different QDA options is that while they can accommodate the use of visual, material, and (in a ham-handed way, spatial) data that they typically are used for what amounts to text analysis as part of a process that seems to me to be analogous (homologous?) to the Boasians’ text-as-culture conflation.

    I was wondering, then, if the QDA packages mentioned contain any safeguards against arbitrary codings resulting in flawed analyses.

    That particular package is known as Skynet.

  31. I just wanted to point out that there may be confidentiality issues when using somehting like Evernote or other cloud based software for storing and analyzing ethnographic data. If you have sensitive data and/or identifying information in your notes, there’s a chance that the information could be accessed by someone outside of the reseach team. Not to say that it can’t or shouldn’t be used, just that these issues should be taken into consideration and addressed with your IRB if necessary.
    I’ve used Evernote, but it hasn’t worked its way into my regular set of software yet.

  32. The problem with Evernote is that you are limited to tagging at the note level. With Atlas.ti or NVIVO you can tag individual words if you are so inclined. They are an order of magnitude more flexible than Evernote. On OS X I’ve always thought Tinderbox might have some potential, though I have never actually used it.

    The popular QDA vendors aren’t shy about connecting their products with Grounded Theory. If they have grown out of any paradigm it is GT. I doubt many units can be moved by reaching back to Boas.

    There is quite a bit to be said here about the influence of GT (for good and/or bad) but perhaps that should be the subject of a new post.

    The other thread hanging loose here is that QDA software can be used at varying intensities. Using Atlas.ti or NVIVO isn’t a straightforward process like calculating formulas with SPSS. You can use QDA as an elaborate file system with fantastic search tools, to index data, for various levels of coding, or for creating and testing hypothesis.

    Simply saying you use QDA software begs elaboration – Chris is exactly right about this.

  33. Wow, I was convinced this thread was going no where. Thanks for stepping up folks. My apologies for being a dick to MTBradley and Wormtongue (though you have to admit the name is an incitement), I drank all of Rex’s Bathtub “Gin” in the “Hotel Party”. That and I had a bit too much Pure Concentrated Evil before I showed up. Anyways, sorry. My next post will be about what a barbaric form of interaction blogs are and how the only people who really know this are those who comment regularly, and should know better. I’m only on asshole on Savage Minds (and will probably continue to be). Ask anyone. My second next post will be about why that isn’t pathological.

    two general points:

    1) It’s absolutely clear to me now that when I hear “QDA” I should also hear “GT”. But this begs the question: what are the major advances of GT besides “Awareness of Dying.” I know, I know, there’s probably a long list, and actually there are plenty of good people in my own field of STS who are GT Junkies (Straussers mostly, not Glaserians). But do anthropologists really do GT? Is this something that is part of anyone’s graduate curriculum in an explicit way? Or is this just a feature of the general blurring of sociology and anthropology. My point is, just because QDA implies GT does not absolve anyone from saying explicitly what their methodology is, and I still think that includes saying why the software furthers that methodology. I don’t buy the technological determinist claims that there is no difference between an abacus and SPSS. That’s baloney to a cultural anthropologist.

    2) To Rex’s point that there is a category error here, there’s more than one. One is the style of “science” in which hypotheses are tested, and the hypotheses are basically pulled out of the common ass^H^H^H sense of the researchers. This is manifestly not the style of “science” that GT follows, which is to generate hypotheses (they call it theory) through analysis of data. Freshman philosophy of science class divides this according to deductive and inductive, but that’s not helpful because both modes (hypo-testing and hypo-generation) involve both kinds of reasoning. But there is a third mode of “science” which doesn’t seem to be on the table here at all, and that is the experimental mode, in which the use of data, theory and tools are used on the surface to test hypotheses, but at a deeper level to generate surprises. As the saying goes: the right response to real scientific discovery is not “Eureka!” it’s “That’s funny…”
    The only difference in my mind between the style of science involved in cultural anthropology and that in molecular biology is that the latter has evolved a very precise and systematic and *shared* documentation and reporting of that surprise, whereas cultural anthropology has developed a penchant for highly idiosyncratic, virtuoso elaboration of surprise. This is a problem, perhaps. But it is not the same problem as an unwitting over-reliance on software to help you find out what you think you are seeing.

  34. The only difference in my mind between the style of science involved in cultural anthropology and that in molecular biology is that the latter has evolved a very precise and systematic and *shared* documentation and reporting of that surprise, whereas cultural anthropology has developed a penchant for highly idiosyncratic, virtuoso elaboration of surprise.

    An excellent point, but it could, I suggest, be more pointed. To wit, one might take note of the penchant for commonplace, often tedious elaboration of observations that aren’t surprising at all, e.g., the injustice of discrimination based on race, gender, class or religion. Which is not to say that these are not worthy topics—the complaint is about “analysis” that is little more than yet another expression of shock and horror. A bit of “That’s interesting,” followed by analysis of something not observed before would be most welcome.

  35. I don’t know if you could compare molecular biology to any of the social sciences, especially not ethnographic study, but we can work with the gist of the comment.
    It’s important to note that plenty of big names have pointed this out, and have been thoroughly bashed for doing so. Marvin Harris and Russell Bernard have spent much of their careers trying to get more anthropologists to adopt a science loving practice, and haven’t escaped unscathed. There are anthropologists that have continued the science of anthropology, but have done so in the shadow of various non-scientific research strategies. They are consistently accused of “arrogant positivism,” or being, ‘bourgeoisie, overly masculine, and vulgar’.

    John, actually I think there is a strong bias in all the social sciences for new and original research, to the detriment of social science theory.

  36. Mea culpa. Composed in a fit of irritation, my previous comment has led us once again into an old impasse, science versus the humanities. A small voice says to me, “Don’t you remember? One of the things that made anthropology so appealing was that it was both a science and a humanity, a field in which people were willing to explore both the nature and the poetry that, together, make us human.”

    I recall the words of Shibata Tsunefumi, a brilliant creative director with whom I had the pleasure of working at Hakuhodo. He says that when a team is brainstorming, he looks for the moment when, instead of “No, no, no,” people start saying, “In that case….” and start building on each other’s contributions. They stop denying and start improving. That’s when great things happen.

  37. Rick said:

    but we can work with the gist of the comment.

    Oh thank you your majesty.

    Try saying the word “sciences” whenever you say “science” and you may come closer to understanding what I said.

    Also, there’s a subset of Godwin’s Law known only at Savage Minds in which the invocation of Marvin Harris can get you banned for life. You may not know about it, so I’m just re-stating it here.

  38. Tying together some previous threads:
    I too am suspicious of storing fieldwork data in the cloud. There are lots of apps that duplicate enough of Evernote’s functionality — or rather, whose functionality Evernote duplicated! — which are cloud-free by default.

    Kerim, I use Devonthink to ‘code’ fieldnotes — in this case ‘coding’ simply means tagging them with all relevant tags, then rereading them again and collapsing the tags into a few number of more inclusive tags, and so forth until you have three or five that tell you the Main Themes in your work.

    I also recently started using Scrivener and it ROCKS — but for writing up articles, not playing about with data. It’s a great way to move chunks of text around. Also, it is great for managing complex tenure dossiers!

    I am hardly an expert on Grounded Theory, but I don’t actually think that it has much to do with the hardcore QDA people. GT started as both a touchy-feely countercultural thing AND a theory of the absolute scientific validity of your results, since iterating over your notes was said to ‘reveal’ the themes that were Really There. I think most ethnographic sociologists who are interested in coding having taken on board a of 80s theory and recognize that coding is, like observation, fieldnote-taking, and report-writing, always already a transformation of previous meanings. Compare, for instance, Emerson and Shaw on writing fieldnotes with the current GT textbook (currently being maintained by Corbin) with, say, the Miles and Hberman’s manual.

    CK I am a little concerned that you are collapsing the bad use of software with the use of software — surely both you and a QDA enthusiast would agree that “an unwitting over-reliance on software to help you find out what you think you are seeing.”

    As for the oft-raised distinction between ‘social’ and ‘hard’ sciences — I just don’t think dog hunts. It makes much more sense to compare, say, lab and field disciplines, historically particular versus generalizing disciplines, atelier-style versus factory-style disciplines, to understand what is really going in and amongst these disciplines. In particular, jettisoning the notion of ‘science’ versus ‘not-science’ (and somehow, ‘not really knowledge’) helps a lot I think.

  39. “Oh thank you your majesty.”

    No problem, think nothing of it.

    “Also, there’s a subset of Godwin’s Law known only at Savage Minds in which the invocation of Marvin Harris can get you banned for life.”

    No, didn’t know this, but I assumed as much. What happens if I say Marvin Harris 3 times looking at a mirror in the dark?

  40. I’m using NVIVO for the first time (trying would be a better word). I’m a long-time SPSS, LISREL etc. user and hand-coder, and I’m surprised at how clunky and non-user-friendly NVIVO is. I’m glad I have access to an institutional license — I would NOT spend any money on NVIVO. I’ve coded over 200 textual cases with NVIVO, but, as many of you have noted here, it doesn’t do squat to analyze the coded data. It’s like the designers of the program didn’t imagine a situation where users might want to compare attributes on difference nodes. I can’t get it to do what I want, even with reading the entire user manual and consulting with a person on our campus who is supposed to know how to use this software. THANKS for the references to free and other software — I will definitely keep that in mind for future.

  41. Provalis Research has released QDA Miner Lite, a free version of its popular computer assisted qualitative data analysis software, QDA Miner.

    This freeware has been designed to meet the basic needs of researchers and analysts performing qualitative data analysis. Advanced features have been removed from the full version of QDA Miner, such as several text retrieval tools (query-by-example, section retrieval, clustered coding, etc.), analysis features (clustering, multidimensional scaling, crosstab, etc.) and advanced code management tools. All the essential features for importing, coding, annotating documents and images and for retrieving coded segments have been kept, creating an easy-to-use qualitative coding tool ideal for those on tiny budgets (or no budget) or those who wish to teach qualitative research in classes.

    For more information, visit: http://provalisresearch.com/products/qualitative-data-analysis-software/freeware/

Comments are closed.