Tag Archives: Method

Topics concerning ethnographic or anthropological methods or general Methodenstreiten

Fieldnotes 2.0

A couple of weeks ago, on the East Asian Anthropology Listserve, there was a brief flurry of emails in response to a query about what software works best for taking fieldnotes. Apologies for the double-posting, but I thought sharing some of the suggestions with the Savage Minds community would be useful. In terms of fieldnote taking, everyone, like opinions, seems to have their own method. Following the Lifehacker mode of “talk among yourselves,” I’ll list some of the suggestions with some comments and URL’s to check out.

Old School.

First, there is the method that no one (including me) on the EASIANTH listserve brought up – pen and notebooks. While I recommend digitized note-taking to my students, I still tell them that they should always carry around a pen and notebook, for those impromptu jottings and diagrams; old school still works!

Good old MS Word was also recommended by a number of anthropologists, largely because of its wide compatibility. Word documents can always be imported into other analytical software packages, and as David Slater from Sophia University Tokyo pointed out, has a “fields” command that can be used to code fieldnotes.

Fieldnotes 1.0 programs

One up on old school, these programs were made specifically to help organize and analyze qualitative data. I think all these programs (except for Filemaker Pro) can only be run on PC’s.

Anthropac
Atlas ti
Ethnograph
Filemaker Pro. From John McCreery.
Easy to get started with, dead simple to use, available for both Macs and Windows PCs, and now an extraordinary powerful relational database. You can literally start as simple as creating a new database (just like opening a new file in a world processor), adding the fields you want, and start inputting data. Discover that you need a new field, no problem; just add it to the existing fields.
Then, moreover, the sky’s the limit. The program has been evolving since its introduction about the same time as MSWord on Macs, has tens, maybe hundreds, of thousands of users worldwide, and highly robust and helpful user community.
QSR’s nVivo: From Joe Bosco, Chinese University of Hong Kong.
There are many ways to use a program like NVivo, but for me, the main advantage is help in organizing the mass of data we can now collect electronically, and then structuring my work with the texts. The program helps you sort and keep track of files and information, and helps you organize your thoughts and ideas.
The files. NVivo now allows you to import documents in Microsoft Word, RTF, or TXT formats. You can also write note or transcribe interviews directly into NVivo, and it can handle photos and video and websites as external data, with a hyperlink in your NVivo data.
Organizing the data. NVivo allows you to give “attributes” to each case. For example, you can define the attributes of interviewees as male or female, or specify their age, and then later retrieve data based on these attributes (are there any old men who talked about using hair conditioner?). You can think of this as coding the entire case according to attributes of the speaker, if they are interviews.
Coding, or Working With the Data. NVivo offers many ways to code. It is designed primarily to allow you to code as you read along. Ideas come to you, and you can create a code. You can make it a “free code”, which stands alone, or a “tree code” which has relationships to other codes (such as a code for “buys shampoo” under which might be “buys name brands” and “buys cheap brands” as two branches off the trunk.) The program allows you to see all the possible codes in a box off to the side, and you can also see “coding stripes” next to your text, to see how text is coded (though you cannot see all the stripes at once–you probably would not want to since you are likely to have a lot of codes). It is very easy to rename, merge and move codes around. The program can autocode if your data is structured with common headings (e.g. if you asked everyone the same question). It cannot autocode for specific words, though it can easily search for certain words or combinations.
The program has a query function that allows you to search for patterns and test ideas through text searches, word frequency, and “matrix coding”, i.e. comparing results across several groups. All query results can be stored. The program can also produce “reports” which can be as simple as all the text coded for a particular code, or the code that comes from informants with certain attributes. It also gives a “coding summary” which lists the files and codes used, showing your progress in coding.
Each project is saved as one file, making it easy to back up or to move from one computer to another. The program comes with training materials and videos, but I found an introductory class quite helpful in getting jump-started in using the program, though the class was relatively expensive. The program is also not cheap, but it is quite robust and the company does give a lot of support. They even helped upgrade a project file when I could not get it to work with the new version. NVivo 7 works well with Chinese; earlier versions were hit and go, but now you can also enter data in Chinese.

askSam From me.
I myself use this program. AskSam is a free-form database that you can use to organize, code, and search fieldnotes. Although you can import Word documents into askSam, I usually type fieldnotes directly into askSam. I first learned of askSam (and received a free copy) as a participant in a NSF summer methods camp, and have upgraded it each time since then. What makes askSam a good way to manage fieldnotes is that you can both code as you type fieldnotes, or you can later do pretty powerful searches on a large set of material. I also use askSam to manage news clippings – when combined with Surfsaver (a sister program, made by the same company), it is an easy way to save webpages while using Internet Explorer. If I was not already accustomed to askSam, I would consider starting with something like nVivo instead.

Fieldnotes 2.0 programs
These programs are newer programs not really built as analytical tools for social science research, but as general note-taking programs that have the ability to combine different kinds of materials. They have great flexibility (in terms of data input and searching) and can be used to keep track of everything from web clippings to recipes. They also have cool and hip names like Voodoo Pad and Evernote! One very useful compilation of these newer software packages can be found on a website by Allison Alexy; she lists software for both Mac and PC users. The following summaries and links are shamefully (but with her permission) lifted from her website. Do yourself a service, though, and check out her full entry on fieldnoting software.

From Allison Alexy:
Voodoo Pad, by Flying Meat, looks really interesting and if I wasn’t already in the middle of my own system, I think I could be convinced to switch. What I like about it is that it is based on a wiki model — similar to the way that Wikipedia enables users to change it while it keeps a record of what changes were made when and by whom. Down side: it’s only available for macs. Up side: if you’re using a mac, then Voodoo Pad can link very easily with your other programs (address book, iCal, etc.).
This idea — making fieldnotes into the form of an editable website — reminds me a lot of Joseph Hill’s website. Joe is an anthropology grad student who had done fieldwork on religion in Senegal (that’s the short version). As you can see from his website, he used the site as a way to organize his fieldnotes. It’s not just that he posted his notes on a website, but that his notes are the website. His homepage is here, and his page about his dissertation research is here.

For macs there are also:
AquaMind’s NoteTaker — here is a review from MacWorld about version 1.9.4
Hog Bay’s Notebook — I couldn’t stand the layout of this program, but maybe you’ll like it better. I don’t like typing notes on computer screens that are made to look like legal pads.
Circus Ponies NoteBook – a review is here
PersonalWiki — another, less refined, Wiki option

For PC people:
WhizFolders — comes highly recommended.
Scribe — created by the Center for History and New Media.
EverNote — free, which is nice; a review is here

Other favorites or most hated programs?

One final thought — one of the reasons I decided to contiue using Word was I don’t want to be stuck with a mess of a unopenable files if the little software company that designed my program goes belly up. As evil as Microsoft is, the good news about MS programs is that you will probably always be able to open the files.

UPDATE: Sara wrote me saying that a friend uses a program called Soho Notes but she hasn’t yet tried it.

What do you use? Talk among yourselves.

Educate your IRB (a boilerplate experiment)

Educate your IRB (a boilerplate experiment)

1. Virtual versus real ethics: creating alternatives to cynicism and disengagement

Very few anthropologists confuse IRB reviews with the “real” ethical work involved in a field project. Anthropologists of all theoretical stripes understand that participant observation-based fieldwork involves the long-term cultivation of social relationships as both the medium and the substantive content of the work. What is more, we know that this cultivation of social relationships must proceed in important respects on ones informants’ terms—not on the researcher’s terms (as is the case in interview-based and experimental social science). Because participant observers aren’t in control of the research process, the ethical challenges that they face in their projects cannot be known in advance and preplanned except in the most general—therefore ultimately vague and inaccurate—ways.

Because participant observation is a necessarily non-methodical method in the preceding paragraph’s sense, IRBs’ mandated insistence on prospective reviews of research designs set anthropologists up to fudge, circumlocute, and fake their descriptions of project “design”, “subject selection”, “informed consent”, and the rest.

That is, so long as structures of ethical accountability are only imaginable in the form of managerial auditing (using unitary compliance criteria external to the historically elaborated disciplinary standards of good practice), practitioners will be forced to simulate consilience with the regulatory ideal so as to appear compliant, cooperative, and transparent—therefore ethical—to their local IRBs.
Continue reading →

Constructive, creative coping (a complement to IRB critique)

       Following up my previous post, I’d like to open another kind of discussion.  Concerning IRBs, we have a wealth of information about the problems researchers of all sorts face (and more on the way in the form, for example, of ethnographic research on IRB process); there have also recently been a number of important critical interventions in the form, for example, of conferences and professional association position papers, with more to come (in the wake, for example, of the recent NY Times article).   

       Complementing all that, for the immediate short term we also need to build up a stock of creative coping strategies.  I suspect that it would be useful to share what we’ve come up with rather than keeping our innovations local. 

       I am most definitely not suggesting that we devote our attention solely to coping: the November American Ethnologist Forum as a whole — and Katz’s contribution in particular — ought to make that clear.  I’m suggesting that institutional isolation makes everyone weak, whereas cross-institutional sharing of productive interventions enables both students and practitioners of field research, oral history interviewing, and other marginalized research styles to continue doing ethical, critical research even as other efforts are under way to protect and expand those possibilities. 

       Some examples of the creative coping were offered in the November American Ethnologist Forum in Dan Bradburd’s article (on an individual level) and Rick Shweder’s (on an institutional level).  Please write in with your local achievements: whether individual, departmental, or college/university-wide.  In my next post, I’ll describe a local experiment  in what — following my last post — I think of as cross-disciplinary “translation”.     

Consent and the Ethics of the ‘Reseach Relation’

1. Consent and preserving the rights of research participants:

About Chris Kelty’s (03/15/07 this site) suggestive comparisons between copyright and informed consent, and particularly the notion that both “actually diminish rights of subjects regarding the control of information, rather than protecting them”: remember that, even though the IRB system is federal (with a central Office of Human Research Protections apparently coordinating the application of regulations), IRBs are local and control their own procedures. As far as I know, IRB researchers agree that local IRB practice “varies”. The point here is that there is no standard IRB consent form.

My institution’s IRB takes a minimalist approach. Our “Standard Adult Consent” form (which, like many other universities’ IRB materials, is available online) is two pages long, the second page devoted simply to contact information. The main page has a place in which the researcher needs to insert a straightforward (non-jargon) description of the project. Apart from this researcher-generated text, the consent form asserts that the signer affirms only that s/he has been informed about what s/he will do as a research participant. The form includes the mandated language (http://www.hhs.gov/ohrp/humansubjects/assurance/consentckls.htm) that participation is voluntary and that consent can be withdrawn and participation discontinued at any time without penalty. But it adds that “By signing this agreement, I do not waive any legal rights or release Princeton University, its agents, or you from liability for negligence”.

This minimalist approach lives up to regulatory requirements and also affirms a best-case interpretation of what the regulations aimed for: it is clearly aimed to avoid creating an institutional shelter.

In contrast, for example—just to use Chris’s institution—the Rice “Consent to Participate” form simply provides the required assurances about the voluntary nature of participation and rights to withdraw without penalties.

It might be interesting to compare other university forms: so folks, what rights does your local IRB consent form protect or not protect?

2. Disciplinary diversity in the ‘researcher relation’ and its ethics:

I agree completely with Tom Strong’s (03/17/07 this site) sense of the contradiction between naming or authorship (associated with IPR claims and their related local values, like the Papua New Guinean ones he mentions) and confidentiality (associated with both ethnographic ethics and IRB emphases). IRBs presume one kind of researcher/researched relation derived from medical research, in which confidentiality makes sense since participants are vulnerable “subjects”. They are brought into lab settings within which–for the resulting data to be usable–information cannot be shared but must be carefully controlled by the reseacher.

But IRBs these days regulate a very wide variety of fields whose methodologies involve diverse research relations. Fields whose reseach relations tend to be collaborative–involving, for example, the sharing of information and other colleague-like assumptions about intellectual and social agency–are closer to the model that Papua New Guineans—and lmichael’ (03/17/07 this site) Amazonian field communities—prefer. They’re quite different from the medical model of the research relation.

Oral History ethics are perhaps the most elaborate in treating “narrators” as the authors of their narratives: going way beyond even recent anthropological practice. The OH convention is for narrators to hold copyright to their own words and to specify who (if anyone) may have access to the transcripts and recordings.

In this consent discussion, John McCreery (03/18/07 this site) invokes “the spirit of free choice”—and the journalist’s allowing on- or off-record commentary. This position appears consistent with the Belmont principle of “respect” (foundational to the regulations that IRBs enforce). Is it? Why does such a stance tend not to make sense to your average IRB?

My answer is that we’re dealing with incommensurable perspectives on the research relation. John’s (or Tom’s or Michael’s or my, etc.) interlocutors are construed (and construe themselves) as autonomous agents, whereas—following the medical model and its frightening worst cases—IRBs are responsible for “human subjects”: persons construed as “vulnerable” and in need of protection. What is more, bureaucratic rationality demands a disinterested consistency. From that central(izing) vantage, our divergent ethical positions don’t sound principled (they are not heard as issuing from different ethical stances) but simply self-interested (unenlightened, irrational).

Very like audit culture (using Marilyn Strathern’s sense of this term), bureaucratic ethics culture defines the dominant language of best practices. It demands that diverse disciplinary research relations be translated, so to speak, into one ethics language. The burden of translation is, as things presently stand, entirely ours.

As Michael Brown’s story suggests, that doesn’t necessarily mean a surrender of our principles…

[UPDATE: Added links for comments, and gave LMichael credit for a comment previously attributed to Michael Brown. – Ed.]

Responses to comments (continued!)

In my earlier post (“Responses to comments…”), part of my responses somehow disappeared. Here’s the missing bit (and sorry for my post-incompetence!):

#3 on inconsistencies and a sneaky plan to heighten the contradictions: John McCreery raises an excellent question. It would be nice if consistency ruled: all researchers should face the same constraints, but they don’t. The irony here is that consistency is one of the core values of bureaucratic ethics management. Consistency is a recurrent refrain on many local IRBs (“…well, if we allow you guys to do away with written informed consent, then we’d have to allow everyone to…”) and it is a key theme at the national level as well.

Ironies aside, as I understand it, there’s an important, fatal flaw in your deliciously sneaky consistency argument: market researchers don’t depend on federal funds to do their work.

Strictly speaking, IRB oversight is only required for institutions (like most universities and colleges) that accept federal research funds. The federal human subject protections regulations (45 Code of Federal Regulations, Pt. 46) were nicknamed “the Common Rule” in 1991 when 17 federal agencies (like NIH) that fund human subjects research all signed on. Rick Shweder’s contribution to our November 2006 American Ethnologist Forum explains that universities and other institutions that accept federal research funds cannot get those funds unless they sign an “assurance” with the relevant funding agencies, or a general Federal Wide Assurance (FWA): documents that obligate them to have one or more local IRBs to review their employees’ research proposals.

Now, as Shweder’s article also explains, university and college IRBs only need to promise to review federally funded research. However, it seems that most of our institutions have gone beyond this minimum requirement and have checked a box on the FWA form that obligates them to review all research, not just federally funded research! (Folks all over the place are looking in to this situation at their institutions: I recommend that you make friends with someone in your institution’s counsel’s office and look in to it too!) In any case, over the past five or six years of IRB “hypervigilance” (the situation that prompted the AE Forum) boards have been jittery and have tended to review all research regardless of how it is funded, regardless of whether their FWA obligates them to do so or not.

Responding to John’s question about the existence of guidelines parallel to those on which IRBs are founded: I can think of one that, while still being at least partially academic, is interesting nonetheless. Check out the National Academy of Sciences “On Being a Scientist” booklet (available online at http://bob.nap.edu/readingroom/books/obas/ ), which concerns science research ethics very generally. This is another example of the inconsistencies mentioned above: unlike the IRB oversight of research with human participants, these general (mostly non-human participant research) guidelines are completely voluntary even tho the research is very likely to be federally funded!

#4 on the roots of the IRB problems in disciplinary ‘cultures of research’: Another terrific question! My responses to other folks’ questions contain bits and pieces of an answer to this one (as does my AE Forum paper). But a fuller response would be the paper I mention in my comment on #5 (below). My contribution to the Cornell conference was a paper entitled “Comparative ‘Research’: A Modest Proposal Concerning the Regulatory Object” (which I’ll be ready to make available in a few weeks). In my view, the problems go way, way beyond the IRB context and derive exactly from the “cultures of research” of which IRB members and the rest of us are part. My own long-term research has been all about disciplinarity and interdisciplinarity—that is, engagements like those cross-disciplinary discussions concerning methodology you mention. As I explained in my AE paper, IRB discourse is just one of my “fieldsites” (which include other places in which disciplinary practitioners bump up against one another, as well as fractious intra-disciplinary moments of ethical crisis). But it’s a fieldsite in which everyone is implicated and consequently of great interest. My AE paper unpacks the Common Rule “definition of research” a bit; and it also begins to address exactly the issues you identify concerning how ethnographic fieldwork is understood by folks from other disciplines (and vice versa). Check it out.

#5 on the relationship between IRB and intellectual property issues: If Michael Brown is still out there, what do you think about the relationship between IRB surveillance and the management of intellectual property contradictions?

Different institutional mechanisms are at play with respect to intellectual property and IRB controversies. For one thing, the IRB system exists outside of (or prior to) legal mechanisms for dealing with accusations about misrepresentation (libel laws), privacy, and the like. This is a huge issue: several of us refer to it in the AE Forum (and anyone interested in following this might also check outhttp://irbinfo.blogspot.com/ and follow references to Hamburger’s Supreme Court Review paper). In a paper that I wrote for a Cornell Law School-hosted conference on “Bureaucracies of Virtue”, I suggested that we’d be better off (and our informants no worse off) if our work were held to account in the same ways that the work of journalists and other writers are. As I understand the current situation, IRB reviews do not protect us or our institutions from lawsuits (that is, whether or not consent forms are involved, IRB reviews don’t prevent our interlocutors from suing us). As things stand, many critics see IRB reviews as constituting censorship-like prior review (arguably a kind of “prior restraint”, something that the First Amendment protects against).

“Research” as not-Therapy (or: Why “research” is dangerous and must be controlled): An IRB origin tale

In his comment on my earlier post, Kerim wrote that it would be interesting to see how IRB regulations work for “examples which are supposed to be paradigmatic” (medical research). I agree. Doing so really exposes the fundamental impotence of the rules (despite mighty efforts by droves of well-intentioned, clever rule-writers). While ethnographers and biomedical experimenters do face analogous disconnections between regulatory prescriptions and the inevitable surprises of actual research practice, exploring the problems that medical folks in particular face brings us back to the regulatory system’s “origin place”. It brings us back to the problem that the rules were meant to solve but never did.
I’ll summarize the argument—part of the forthcoming PoLAR article, “Comparative ‘Research’” (mentioned in earlier posts)—versions of which also appear less centrally in my 2006 AE paper.
I begin with the observation that there is no such thing as Research-in-general: there is no suite of ideals or traits shared by all the practices to which the term “research” usually refers. Instead, there are diverse, historically specific, socially organized ways of knowing the world, in shifting partial relations with one another, as well as ambivalent relations with particular non- and quasi-research activities (from journalism or travel writing to stage magic or, as we will see below, therapy).
Bureaucratic regulation cannot tolerate such incoherence. To control human-subjects research, variation must be bracketed so as to create a stable object with clear boundaries. Criteria must be found for deciding when instances of “research” begin and end, and for deciding what is and is not “research”.
However, research involving human beings persistently resists reductive objectification in the interest of consistency. This is true not just among the ornery humanistic social studies but also in biomedicine and other unambivalently science-identified fields whose research models fit the Common Rule definition.
So why bother? Why did “research” with human participants come to be viewed as in need of regulation (and therefore agonizing definition)?
The IRB definition of research was designed originally to provide a means of eradicating a potentially lethal ambiguity, specific to medical practice, concerning when “therapy” ends and “research” begins. This danger, publicized in the 1960s and early 1970s, is central to the history of the US federal regulations (for lots of bibliographic leads, see that November AE again).
But the design was flawed: the original ambiguity is still with us.
Human-subjects research regulations (45 CFR, Pt. 46, known as the Common Rule: see Chris Kelty’s Savage Mind 2/8/06, on which I hope to comment soon!)—formally set in place in the late 1970s/early 1980s—derive from earlier NIH guidelines that reflect specifically biomedical background assumptions. These assumptions go something like this:
1. Since it concerns directly improving the wellbeing of particular individuals, medical therapy is appropriately evaluated in terms of individual patient interests.
2. In contrast, since medical research concerns the production of knowledge “generalizable” beyond individual cases, it is appropriately evaluated in terms of the interests of science and society at large.
3. It follows that—since a concern for individuals isn’t inherent to the definition of “research” the way it is to the definition of “therapy”—even though physical risks to persons are associated with both medical therapy and research, the risks to individuals are qualitatively more serious in research
4. Consequently, “research” (in this sense) needs special oversight.
These specific background assumptions are evident in regulatory provisions that, firstly, limit risk/benefit calculi to research “emas distinguished from risks and benefits of therapies/em” and that, secondly, exclude from consideration the “long range effects of applying knowledge” (45 CFR 46.111[a][2]). “Research” is not “therapy”: not the emapplication/em of medical knowledge to individual persons (applications normally enabled when research results are published and made available to practicing physicians). The medical rationale of regulatory distinctions is just as evident in the definition of “human subject”, but I won’t discuss that here.
Now, while Common Rule regulations were applied widely to federally-financed non-biomedical research from the start, the idea of research-as-not-therapy does not translate readily outside of biomedicine. For example, cancer patients participating in clinical trials of new cancer therapies can compare the likelihood of a direct personal benefit from new knowledge against the risk of harm in the research process itself. In contrast, social research rarely promises practical payoffs to its participants. Its benefits tend to be indirect and its harms are located less in the research process than in its products.
Not only is research-as-not-therapy untranslatable across fields, but the regulatory definition did not even do away with the ambiguities within medical practice. Heroic efforts of clarification can be found in works that interpret the Common Rule for IRBs, like the IRB Guidebook (www.hhs.gov/ohrp/irb/irb_guidebook.htm). Nevertheless, to this day it continues to be a frequent topic of debate in IRB circles.
I’ll stop here for now. In the full length treatment of this argument, I offer an extended example of problem solving discussions among experienced biomedical IRB insiders concerning how the medical therapy/research distinction might be operationalized in particular cases. Such discussions—which happen all the time—demonstrate very clearly that it is effectively impossible in practice to distinguish clearly between when “therapy” ends and “research” begins: when innovations in the care of a particular patient with an unusual condition ought to be treated as research (worth publishing, in need of an IRB review). The regulations offer no solution to this problem.
More soon.

Responses to comments on cross-disciplinary dimensions of IRB engagement

This is a response to the first five comments on my last post. These concerned a series of interconnected issues relating to the cross-disciplinary dimensions of IRB engagement:(please disregard the strange font size changes below, which aren’t intentional…) [I’ve removed some of the cruftier html to remove the weirdo fonts -Rex]:

#1 on sociocultural anthropology and ethnographic sociology: I strongly recommend Rosalie Wax’s book Doing Fieldwork (U Chicago 1971, reissued around 1986, and still available on Amazon or your college library). It’s a memoir of fieldwork, and contains a wonderful capsule history that moves between conventional anthropological and sociological sources. Park is an important part of the story, but before that (among other folks) Beatrice and Stanley Webb were working in London, contemporary with Boas and part of Malinowski’s environment. Just in U.S. anthropology, it’s my sense that the various subfields and theoretical styles are unevenly aware of ethnographic sociology (that is, we’re not all equally ignorant!).

I agree that there is a lack of reference to qualitative sociology in the recent generation’s revaluation of work “at home”. But anthropologists have always worked at home; indeed, working at home is cheaper (it doesn’t necessitate securing a research fellowship or grant) and was therefore always common. What has happened over the past generation is that working at home has become not only expedient but also sexy. So one question is: what was the relationship between ethnographic sociology and the long tradition of home style anthropology? Lots of other questions certainly (e.g., for example, how is the anthropology/sociology relationship managed in joint departments?)!

#2 on multidisciplinary projects and IRBs: How IRBs handle multidisciplinary projects is an interesting question. I haven’t seen much commentary in the gargantuan IRB literature on this: so, any stories folks? Tom—do you want to describe the HIV study with respect to IRB approval?

In any case, I very much agree that it’s important to improve our understanding of disciplinary differences: much of my work has focused on this (as my own AE paper suggests). I’ve been particularly interested in the partial connections—the reticulum of similarities and differences—among closely related disciplines like those I sketched (e.g., p 483, 484-5, and esp. 485-6) in that essay.

For example, the IRB literature—definitely including that written by folks who are critical of IRB “mission creep”—is full of generalized references to the problems “qualitative” researchers face when their work is evaluated by IRBs. While it is true that there are significant differences between qualitative and quantitative researchers, this distinction doesn’t begin to address the problems of cross-disciplinary communication between researchers and IRBs (and among IRB members). Consider that thoroughly quantitative survey researchers and thoroughly qualitative, interpretive anthropologists both approach potential informants on the latter’s home ground (where consent forms aren’t the most effective ways to ensure informant consent, where informants have considerable power to stop participating); in contrast, oral historians and interpretive anthropologists—both qualitative—have very different conventions with respect to confidentiality!

#3 on inconsistencies and a sneaky plan to heighten the contradictions:  John McCreery raises an excellent question.  It would be nice if consistency ruled: all researchers should face the same constraints, but they don’t. The irony here is that consistency is one of the core values of bureaucratic ethics management.  Consistency is a recurrent refrain on many local IRBs (“…well, if we allow you guys to do away with written informed consent, then we’d have to allow everyone to…”) and it is a key theme at the national level as well.
 

Ironies aside, as I understand it, there’s an important, fatal flaw in your deliciously sneaky consistency argument:  market researchers don’t depend on federal funds to do their work.
 

Strictly speaking, IRB oversight is only required for institutions (like most universities and colleges) that accept federal research funds.  The federal human subject protections regulations (45 Code of Federal Regulations, Pt. 46) were nicknamed “the Common Rule” in 1991 when 17 federal agencies (like NIH) that fund human subjects research all signed on.  Rick Shweder’s contribution to our November 2006 American Ethnologist Forum explains that universities and other institutions that accept federal research funds cannot get those funds unless they sign an “assurance” with the relevant funding agencies, or a general Federal Wide Assurance (FWA): documents that obligate them to have one or more local IRBs to review their employees’ research proposals.    
 

Now, as Shweder’s article also explains, university and college IRBs only need to promise to review federally funded research.  However, it seems that most of our institutions have gone beyond this minimum requirement and have checked a box on the FWA form that obligates them to review all research, not just federally funded research!  (Folks all over the place are looking in to this situation at their institutions: I recommend that you make friends with someone in your institution’s counsel’s office and look in to it too!)  In any case, over the past five or six years of IRB “hypervigilance” (the situation that prompted the AE Forum) boards have been jittery and have tended to review all research regardless of how it is funded, regardless of whether their FWA obligates them to do so or not.    
 

Responding to John’s question about the existence of guidelines parallel to those on which IRBs are founded: I can think of one that, while still being at least partially academic, is interesting nonetheless.  Check out the National Academy of Sciences “On Being a Scientist” booklet (available online at http://bob.nap.edu/readingroom/books/obas/ ), which concerns science research ethics very generally.  This is another example of the inconsistencies mentioned above:  unlike the IRB oversight of research with human participants, these general (mostly non-human participant research) guidelines are completely voluntary even tho the research is very likely to be federally funded!
      
#4  on the roots of the IRB problems in disciplinary ‘cultures of research’:  Another terrific question!  My responses to other folks’ questions contain bits and pieces of an answer to this one (as does my AE Forum paper).  But a fuller response would be the paper I mention in my comment on #5 (below).  My contribution to the Cornell conference was a paper entitled “Comparative ‘Research’: A Modest Proposal Concerning the Regulatory Object” (which I’ll be ready to make available in a few weeks).  In my view, the problems go way, way beyond the IRB context and derive exactly from the “cultures of research” of which IRB members and the rest of us are part.  My own long-term research has been all about disciplinarity and interdisciplinarity—that is, engagements like those cross-disciplinary discussions concerning methodology you mention.  As I explained in my AE paper, IRB discourse is just one of my “fieldsites” (which include other places in which disciplinary practitioners bump up against one another, as well as fractious intra-disciplinary moments of ethical crisis).  But it’s a fieldsite in which everyone is implicated and consequently of great interest.  My AE paper unpacks the Common Rule “definition of research” a bit; and it also begins to address exactly the issues you identify concerning how ethnographic fieldwork is understood by folks from other disciplines (and vice versa).  Check it out.  
  
#5 on the relationship between IRB and intellectual property issues:  If Michael Brown is still out there, what do you think about the relationship between IRB surveillance and the management of intellectual property contradictions? 
 

Different institutional mechanisms are at play with respect to intellectual property and IRB controversies.  For one thing, the IRB system exists outside of (or prior to) legal mechanisms for dealing with accusations about misrepresentation (libel laws), privacy, and the like.  This is a huge issue: several of us refer to it in the AE Forum (and anyone interested in following this might also check out http://irbinfo.blogspot.com/ and follow references to Hamburger’s Supreme Court Review paper).  In a paper that I wrote for a Cornell Law School-hosted conference on “Bureaucracies of Virtue”, I suggested that we’d be better off (and our informants no worse off) if our work were held to account in the same ways that the work of journalists and other writers are.  As I understand the current situation, IRB reviews do not protect us or our institutions from lawsuits (that is, whether or not consent forms are involved, IRB reviews don’t prevent our interlocutors from suing us).  As things stand, many critics see IRB reviews as constituting censorship-like prior review (arguably a kind of “prior restraint”, something that the First Amendment protects against).                
 

IRBs and the ethnography problem: demarcating ‘research’, locating allies

The point of the November 2006 AE Forum I put together, “Anxious borders between life and work in an age of bureaucratic ethics regulation” (follow the link in Tom Strong’s post introducing me), was to identify the distinctive features of ethnography and of the IRB system (so-called ‘human subjects committees’) that set them up for conflict, and to explore the implications.

Demarcating ‘research’:

A key distinctive feature of ethnographic research is the fact that ethnographers–whether they work in Highland Papua New Guinea or New Jersey–typically embed themselves with their interlocutors: that is, both ideally and often enough in practice, ethnographers live where they work. Anthropologists have long recognized that significant ethical dilemmas derive from the fact that research (ethnographic ‘work’) isn’t demarcated from not-research (the rest of the ethnographer’s ‘life’). But it is a fresh headache on account of intensified IRB oversight. This is because the federal human subjects research regulations (what IRBs are set up to enforce) presume a clear distinction: ‘research’ is, after all, the regulatory object. (I’ll discuss this point in another posting.)

To exaggerate this problem so as to make it more visible, the AE Forum focused discussion on unfunded fieldwork. While ethnographic sociology is typically unfunded–a surprise to most of the anthropologists, I suspect–anthropologists who work “at home” (wherever that may be) also often do so without research grants. This may be especially true of research past the dissertation phase: I’m interested in Savage Minds readers’ experiences. We also focused on the necessarily open-ended, exploratory character of ethnographic discovery practices. While this is also obviously not a new insight, viewed as part of the IRB ‘research’ demarcation problem it helps clarify the distinctiveness of ethnographic work. This is because the federal human subjects research regulations are meant to be applied before ‘research’ begins. The rules are designed for ‘research’ understood as a distinct event, not as an emergent process (as is oftent the case in ethnography, and especially when it’s done at home).

Locating allies:

The focus of the AE Forum was on ethnography in the broadest sense of the term–that is, the research style that anthropology shares with several other fields, notably sociology. This is why I invited Jack Katz (a well known ethnographic sociologist from UCLA who has been doing important critical work in IRBs) to be part of this project.

There’s an important strategic point here. Anthropologists are in the habit of telling themselves that they are the inventors of ethnographic fieldwork and that others who adopt the approach are derivative or in some sense inauthentic. This is just not true. Fieldwork–participant observation particularly–was a co-creation of sociologists and anthropologists, whose methodological histories overlapped heavily during the 19th and early 20th centuries. (I’m happy to say more about this, if anyone is interested; for example, look at Rosalie Wax’s fascinating book, Doing Fieldwork.) While there’s no doubt that fieldwork is positioned differently in present-day anthropology (where it’s the default approach) and sociology (where it’s a marginal or minority approach), I think that recognizing this common history may help anthropologists cultivate allies in their IRB struggles. It’s in our strategic interest to make common cause with ethnographers of all sorts both locally (in our respective institutions, on our local IRBs) and nationally, given the expansion and intensification of ethics regulation.

Thanks, folks

Thanks in advance to Savage Minds participants for this virtual conversation, and thanks in particular to Tom Strong for the invitation. This is an introductory test message, to check whether I understand how this software works…

The pretext for my being here is the November 2006 AE Forum on the special problems ethnographers face in negotiating with IRBs, especially given the expansion and intensification of IRB scrutiny over the past several years.  Over the next few weeks, my aim will be to raise questions and to make a stab at being informative.  I’ve read previous SM posts concerning IRBs and ethnography with interest and will pick up some of the themes raised earlier, but will also move off in other directions.  For example, we have lots (and lots) of horror stories and worst case scenarios, and while we need good critical analysis of all that, we also need reports on survival strategies.  I’ll provide some; consider this an invitation to login with your own.  

Economists and anthropologists on video game violence

Academic bloodsport has been on my mind this week since I have been reading the Charlot/Valeri debate that raged briefly in the pages of Pacific Studies in the late 1980s. It is without at doubt the Ultimate Fighting Championship of debates about Hawai’ian Sacrifice, partially because of the intensity with which it is fought (Valeri, in finest Francophone style, accuses of Charlot of managing to miss “not only the forest, but the trees as well”), partially because of its incredible erudition, and partially because it is, as far as I know, the only major academic debate about Hawai’ian sacrifice to date.

The Charlot/Valeri debate is totally unavailable electronically and so I have no link to it here. However, if you are interested in one academic cutting the work of another into very small pieces I hardily reccomend “Edward Castronova’s smackdown”:http://terranova.blogs.com/terra_nova/2006/07/review_of_carne.html#more of “Carnagey’s work on video game violence”:http://www.public.iastate.edu/~vasser/pubs/06CAB.pdf, which features zingers like “the bugbear of statistical significance is loose among poorly-grounded fields, among which one must now, on the basis of their acceptance of this paper, sorrowfully include experimental social psychology.”
Continue reading →

Ethnography and the IRB

A perennial favorite: the Institutional Review Board and the practice of ethnographic research. I am a member of the Institutional Review Board at my University, ostensibly representing the social and behavioral sciences. Recently the board has undergone a change of leadership and staffing, which means new people and a new round of education about how anthropologists conduct ethnographic research as well as how non-anthropologists do so. For those who don’t know, Institutional Review Boards are a result of the 1974 the National Research Act (Pub. L. 93-348) , which commissioned a report on the use of Human Subjects in Research (known as the Belmont Report) and upon which a code of federal regulations 45 CFR 46 is based, mandating that all research institutions receiving any federal support must oversee research using human subjects. If it isn’t research, or they ain’t human, the IRB is not concerned–otherwise, they must make every effort to 1) know about it and 2) approve it. So far so good.

There are however, a couple of sticking points with respect to ethnnographic research that I have run into repeatedly. One is the requirement of confidentiality. The other is the requirement of written informed consent (and here). (There is also the problem of whether or not there are any reliable standards when it comes to “psychological harm” that subjects might suffer at the hands of unscrupulous researchers–but that is a bugaboo for another post). Now, given the anguish Anthropology has put itself through with respect to colonialism, with respect to complicity in the dispossesion of native people’s land, and most recently with respect Napoleon Chagnon and Patrick Tierney’s Darkness in El Dorado, one would probably be safe in asserting that we are alone among the disciplines who have made the ethical treatment of humans not just a priorty, but virtually impossible to ignore throughout one’s graduate and professional life. We may not have standards of harm that are easily applied across all cases–but we damn well have discussed, in excruciating detail, nearly every possible kind of injustice man can visit on man.

My new IRB overlords, whom I welcome of course, do not see it this way. As far as they are concerned, anthropology appears to occupy a backwater, in which the lack of standardization, the reliance on individual judgment, and the practice of meticulously examining the finest grain of every encounter we have for its justice or injustice are nothing so much as solid evidence that we are in violation of what are clear and reasonable regulations about how scientific research should be conducted.

Take for example, the requirement of written consent. August authorities such as the Anthropological Association of America and the former head (Project Muse required) of the National Science Foundation Anthropology Section have made clear that written documentation may not be appropriate in all cases–the easy cases are those where your informants are non-literate (duh?) or where you are dealing with individuals who have, for instance, a long history of land expropriation due to the unscrupulous signing of documents by ancestors. In these cases they recommend that some other method of obtaining the vaunted “informed consent” of research participants be pursued. But there are a host of less easy cases: those situations where your informants are naturally or historically suspicious of government and its apparatus for instance, or where the presentation of a form would convert a personable and arduously cultivated relationship into a purely formal one. Should the IRB require the presentation of such forms in all cases–even to the extent of rendering some forms of long-practiced ethnographic research impossible? My new IRB Overlords appear to think so.

Or take for instance, the issue of confidentiality. According to the IRB, the only ethical practice is to ensure confidentiality of information, as a default practice, and if necessary, against the wishes of informants. Sure, if the informant is a public figure and the information is public, and the informant wishes to have it made public, then in that case, there seems to be no problem with allowing the researcher to treat the information as public. But if, for instance, an individual of no particular standing, in negotiating whether to participate in your research project, demands that his words be made public and be identified with his real name–do we have the right, or the obligation, to deny him this request? It seems to me that applying this ethical standard across all cases, regardless of the express wishes of individual informants is as suspect as revealing their confidential information. It is, as we should well know, the moral ideal of anglo-american privacy elevated to the status of universal norm. My new IRB overlords call BS on that.

So what I want to know is: experience anyone? I need examples of research in which either the use of oral/non-written documentation of informed consent, or the practice of allowing people the option to be identified has been approved (recently) by IRBs, or articles and documents that describe, or defend such practices. Ultimately, I would like to pen recommendations for use in every such changing of the guard… because otherwise, institution after institution is set to make the practice of ethnographic research as we know it both increasingly formalized and possibly, increasingly impossible.

Doing Research on Subways in Taiwan

Kerim Friedman invited me to guest blog on Savage Minds. I thought about throwing out for discussion some questions that I encountered while doing research on the subway systems in Taiwan.

My current research focuses on the subway systems in the two largest cities in Taiwan: Taipei, the capital and economic-cultural center of Taiwan, whose first subway line was completed in 1996; and Kaohsiung, the country’s hub of heavy industries and one of the world’s largest container ports, whose subway system is now under construction. I use the subway systems as a focal point to understand the regional, national, and global processes that are unfolding in Taiwan. Given that many Asian countries are investing heavily on infrastructure including highways and subways (to boost the country’s global economic competitiveness), my research is not just about Taiwan but carries comparative angles.

In the course of my research — as well as on occasions when I presented my work at professional conferences — I repeatedly faced the question: How do you do research on the subway system in a big city? Indeed, metropolitan Taipei has a population of 6 million, whereas Kaohsiung is a city of 1.5 million people. Over time, I sort of worked out an answer. My involvement with Taipei’s subways was as both a passenger and an ethnographer. That is, the subways constituted the nearly exclusive means of transportation during my stay in Taipei, except for the times when I took a taxi or was driven by friends or families (Research in Kaohsiung is a different story, as the subways are sill under construction). To acquire a broader understanding of the system, I also rode the different routes of Taipei’s subways at different hours of the day as well as on different days of the week, to observe who rode from where, and how and when. The subways also entered in literally every conversation I had with people, both native Taiwanese and foreign-born residents and visitors, in Taipei and elsewhere in Taiwan (and frequently in North America). This fieldwork was blessed with the fact that the subways were, and continue to be, a novelty in the social life of Taiwan; almost everybody had something to say about their personal experience with, or perception and knowledge about, the subways. By extension, with few exceptions, my subway project seemed to generate genuine interests among the people I met, who were often eager to talk to me about the subways. In addition to participant observation, I also had formal interviews with (past and present) government officials who were in charge of the subway construction and of the making and implementation of Taipei City transportation policy prior to the subways, and with civil engineers and urban planners involved in the planning and building of Taiwan’s subway systems (in Taipei and Kaohsiung). In addition, I read intensively literature, popular reports, newspaper and magazine articles on the subways.

But again, at what point can I claim that I have a full understanding of the subway system in Taipei — or Kaohsiung? How many people do I have to interview or talk to in order to say that I have had enough? Or, to put it generally, when can one call it an end when one’s research site is a city with a few million people?

[Anru Lee is an assistant professor at the Department of Anthropology of John Jay College of Criminal Justice, the City University of New York. She is the author of In the Name of Harmony and Prosperity: Labor and Gender Politics in Taiwan’s Economic Restructuring (2004) and a co-editor of Women in the New Taiwan: Gender Roles and Gender Consciousness in a Changing Society (2004).]

Teaching Methodology

I have to admit, despite having studied Anthropology since I was a teenager (my High School offered anthropology), I never really had much in the way of methodology training. And, except for people in applied anthropology programs, I’m afraid that this is the norm rather than the exception. Even my graduate seminar on methodology ended up being a largely theoretical discussion. Perhaps that’s not a bad thing, but lately I’ve noticed that some of the jobs I’d like to apply for expect you to be able to teach a methods course. I thought of this as a good challenge and have been reading up on methodology and looking at examples of other syllabi (see here, here, here, and here) in order to develop my own course.

I often require students to engage in hands-on ethnographic work in my courses. In the visual anthropology course I taught at Haverford last term, students were expected to produce a visual ethnography. But a general methods course presents its own problems. Many of the books I’ve read on the matter just don’t seem to suit themselves to the structure of such a course. For instance, Habermas’ Knowledge and Human Interests would require a course of its own to teach. I had high hopes for Flyvberg, but was ultimately disappointed. Etc. I just can’t seem to find the books or articles that I’d really like to teach. (Perhaps I’ll eventually have to write one of my own…)

My first efforts are up on my wiki here. Please take a look and either offer feedback here, or edit the wiki directly. I’m not very attached to what I’ve done so far. I’ve put a lot of work into it, but it often takes me years and several times teaching a course before I’m happy with a syllabus. Because this is a first effort, I know it will need work. Please focus on the course outline (i.e. the reading list), I’m fairly happy with the general structure of the course.

Thanks for your help!

Face-to-Face

In a comment on my last post Alexandre wrote:

Despite the fact that online communication can be as rich if not richer than face-to-face communication, many anthros favour the latter over the former…

Now Alexandre, a fellow linguistic anthropologist, has some interesting things to say about why anthropologists might prefer face-to-face communication, but I don’t want to discuss those here. Instead, I want to suggest that the whole face-to-face vs. online binary opposition needs to be rethunk.

Let me just discuss shopping – something I do both online and face-to-face. Just the other day we went to the local hardware store. While getting balloons for a party we overheard discussions about the proper way to lay poison for carpenter ants, what the best kind of drill was for a certain task, and we even got advice on which color ribbons would be best to use. We also learned intangible things about life here from people’s body language, their manner of speech, etc. And, at the most basic level, we had an experience that we value – interacting with our fellow human beings.

Now, I think we would all agree that something would have been lost from an online purchase. True, Amazon does offer recommendations, and there are plenty of services that offer all kinds of meta-data that might simulate, or even improve upon the kinds of overheard conversations we might hear in the hardware store, but even after all that we feel that something we value, contact with our fellow human beings, has been lost. Not to mention all that rich ethnographic data we pick up consciously or unconsciously, whether we are trained to do so or not.

But lets look at another shopping experience. Not the village hardware store, but Target. I had at least two face-to-face interactions at Target: asking where to locate an item, and checking out. Both were brief, purely informational, and completely unmemorable. True, they didn’t need to be this way. I’ve seen people go out of their way to make service encounters more personal. I’m just not that type.

Similarly, while the local store might give you interaction with the shopkeeper, you still have no interaction with the producer. The internet now allows you to buy certain items directly from the producer, without a middleman. I haven’t used these services, but I can imagine situations where one might have meaningful interaction with a farmer or craftsman not possible at either a large chain shop or the village store. So while something has been lost from the face-to-face experience of the village store, a whole new type of relationship between consumer and producer has been created.

And, finally, another example from a very different realm. Tonight I’m going to engage in a political protest with people I’ve mostly never met. I’m going to do that right outside the village store I just spoke about. But I didn’t find out about this protest at the village store. I found out about it by putting my zip code into an online form. Here the internet is facilitating new forms of face-to-face communication. It is easy to think of some other examples – like flash mobs, but my favorite is blogging. I’ve already met a lot of real world friends through blogging. And I hope to meet some of my fellow anthropology bloggers at this year’s AAA in Washington, DC.

So, while I think everyone agrees that there is something we value about face-to-face communication which we miss online, there are two things to remember: First, different types of real-world institutions facilitate very different kinds of face-to-face interactions, not all of them positive. And, secondly, in addition to offering alternative modes of interaction (for better or for worse), the internet is also an important tool for facilitating real-world face-to-face interaction.

Personality and ethnography

In graduate seminars, I remember long discussions about the positionality and identity of the researcher and how these would influence not only the interpretations of the results of one’s fieldwork but the results themselves. Depending on the research topic, the information obtained by a young, unmarried female anthropologist, for example, may be quite difference from that obtained by older and married male anthropologist. I think that this is fairly widely recognized in our field at this point.

One related thing that I have often wondered about and at which we only sporadically hinted in our discussions between professors and bright-eyed grad students was the impact of one’s personality on our fieldwork experience and results. We discussed behaviour, yes. We went over questions of how to go about introducing ourselves to potential informants, how to dress in a way that would not offend our hosts, whether or not to have sex with people in the field (yes, Concordia University is very liberated that way) and so forth. One of my fellow students even brought up the possibility that our hosts might be more comfortable talking about their lives if the ethnographer was willing to open up a bit about her own.

Of course, one’s personality and ideas have everything in the world to do with how one would deal with the situations mentioned above. One’s personality and self-concept will very likely determine whether an individual cares if his manner of dressing will offend members of the host group. This will also help determine one’s behaviour in various situations in addition to myriad other factors such as stress, relative comfort level, length of time in the community and so forth. Finally, one’s personality has much to do with how much one is willing to tell about herself.

However, we didn’t go too far into talk of personalities as such. Perhaps we were all a little worried about not fitting the bill if it turned out that, according to our profs, there was a standard personality type that was generally better suited to conduct ethnographic fieldwork.

One interesting that did come out during a one-on-one conversation with my thesis supervisor was that there might be particular personality types that are well suited for particular areas or for particular research topics. As I sat in her office with the pre-fieldwork jitters, she assured me that I had the right kind of personality for working with Natives. Although I could see why she would say that and felt that she was probably trying to make me feel better more than anything else, I wondered how one might be able to assess this. If we assume personality variance within the societies where we conduct research, is it really safe to say that researchers of particular personality types will be more or less successful?

I think it’s safe to say that one’s personality will influence one’s behaviour and therefore one’s results to the extent that personality influences one’s interactions. It certainly influences how well a researcher will get along with people in the field which will in turn affect the kinds of information that they will be willing to discuss. As mentioned above, however, there is no uniform personality in any location and therefore just about any ethnographer will find both people with whom s/he gets along and people with whom s/he doesn’t. Ultimately, then, any researcher, unless they are completely and utterly obnoxious and disrespectful, will probably manage to get along well enough with a large enough number of people to get the information that is required for his research.

The bottom line, I think, is that young ethnographers, such as myself, and ethnographers-to-be need not worry about how well we can live up to the caricatured image of the uppity authoritative scientist. In fact, as tacky as it may sound, I think it’s important for us to be ourselves in the field. What would be interesting to discuss, though, is how personality might affect the fieldwork experience, results and interpretation in ways similar to the various sorts of identities (gender, ethnic background, age, marital status, and so forth) that have been shown to inform our work.