“Traumatic for Both Sides”
Michael Burawoy correctly notes that a revisit by an ethnographer to a research site to learn the reaction of the study participants (what he terms a “valedictory revisit”) is rare and often traumatic:
When the ethnographer returns to the participants, armed with the results of the study, whether in draft or published form [t]he purpose is . . . to ascertain the participants’ responses to the reported research. . . . This is the moment of judgment, when previous relations are reassessed, theory is put to the test, and accounts are reevaluated. It can be traumatic for both sides, and for this reason it is all too rare.24
As Burawoy suggests, it is not uncommon for some study participants to be surprised and very unhappy when they read a researcher’s written portrayal of them.25 When I recruited the families for the original study, I told them I would be writing a book. At the very beginning of the study, when they were signing consent forms, for example, family members often asked if they would be getting a copy of the book. At that point, feeling that it was the least I could do, given their willingness to open up their lives to me, I promised to provide copies. As the study progressed, I began to worry about how the families might react to the book. One impetus for the study was my desire to correct what I considered the superficial and overly romantic view of child rearing that dominated the literature. I wanted to present a realistic view of family life. Still, as I grew emotionally attached to the family members, I worried about the necessity of objectifying them in the writing process. I was concerned that what I planned to write might hurt their feelings.26
When I was still in their homes, sitting around in the kitchen chatting, collecting the original data, I did try to warn the families that the final product probably would not be to their liking. “You know, when someone takes a picture of someone and everyone else thinks it is fine, but the person really doesn’t like it?” I would say. They would nod, somewhat uncertainly. I would then continue, “Well, that is how it will probably be with the book. You probably won’t like it.” Still, I think that the families and I were both surprised by the level of pain, hurt, and dismay—the very kind of reaction Burawoy notes—that the book created for many. One mother said, “I know that you warned us.” But seeing the results in print made the findings painfully real.
This likely negative, even “traumatic,” aspect of research has not been fully integrated into methodological writings, despite striking examples from well-known—even classic—studies. Arthur Vidich was hung in effigy after Small Town in Mass Society was published; communities featured in the early sociological studies such as Yankee City also expressed ire. William Foote Whyte reported the tense reaction, particularly on the part of Doc, to Street Corner Society.27 More recently, a few researchers have vividly described the anger and hurt expressed by ethnographic study participants, but these descriptions have tended to focus on the researcher’s decision not to reveal her or his intention to write a book or the researcher’s decision to conceal key information which would have mattered to study participants.28
Considering the Alternatives
To help mitigate problems study participants experience, some researchers suggest that ethnographers should share the study results prior to publication, so that participants can articulate their concerns.29 This viewpoint is both optimistic and problematic. It raises complex questions regarding the balance between what researchers owe study participants and how much control the researcher ought to retain over what is written.
Some people argue that having participants read draft text and give feedback provides “buy in” on the part of respondents.30 Writers on this subject have also recommended various forms of “member checks.” These range from sharing certain information with participants in order to confirm the accuracy of small details to asking respondents to validate the researcher’s analysis and conclusions. Checks of the former type generally are of little consequence to a study’s main argument or to the relationship with the study participants, particularly if they can be managed in informal, verbal interactions (e.g., stopping by to say hello and checking the information in the midst of a broader conversation about other issues). But checks that involve sharing the written report or a prepublication draft can bring significant perils.
Although the voices of study participants are crucial, they represent only one piece of a complex puzzle. Who should decide the focus of what the researcher writes?31 Study participants are likely to want certain issues to be developed and recast to reflect their own views. If a researcher requests the time and energy of respondents to read text and provide feedback, then the researcher should plan to incorporate that feedback. But academics face constraints from disciplinary standards, editorial boards, and publishers about the shape of the argument or the length of a piece; as a result, researchers can find that they have solicited input that they cannot or do not want to incorporate. They have issued a contradictory invitation to their respondents, saying, essentially, “I know this text may upset you, but I would love for you to take the time to read it and give me your considered opinion. You may have a different understanding of the events; your input is valuable. But, you also need to know that I have to have the final say on the piece. I may, or may not, be able to make the changes you suggest.” Arguably, this kind of highly constrained request for input is yet another tax on study participants; in some circumstances, it could be more an act of tokenism than a sincere request for honest feedback that could radically reshape the publication.
Some researchers have successfully negotiated this process. They have shared their work and argued with participants over the portraits. They have made changes. They have used the prepublication stage to “work through” conflict. Ethnographers such as Hugh Mehan, Tim Black, Mark Warren, and others have managed to make clear their sincere interest in respondents’ feedback, but also make clear that the goal is to correct inaccuracies and not to reshape the written representation.32 Some advocates of prepublication discussions in which subjects provide their interpretation of events feel these sessions offer the potential additional advantage of helping researchers deepen and improve their analyses.
In many instances, researchers who have been most successful in soliciting input from respondents without relinquishing control over the published text were studying organizations and organizational policies. Often, work in these areas draws on stable data collected from written memoranda, published documents, and public records, as well as the more volatile evidence supplied by human subjects. Many researchers who study organizations strive for compromise, “toning down” their analyses to accommodate respondents’ concerns. This approach has costs, however. The portraits often become less sharply focused, particularly with respect to weaknesses or problems in an organization.33
Generally, organizational dynamics are more public, and certainly less personal, than child rearing. In studies of more private settings, it is easy for people to feel criticized, even when that is not the researcher’s intent. Ultimately, many decisions researchers make are dependent on the particular context as well as on the researchers’ own sensibilities. They cannot be mandated. In my case, I worried that showing a draft of the manuscript to the families would make the book hard to complete. I was not primarily concerned about issues of accuracy. The research assistants and I were in the homes frequently, and we took very detailed notes, sought disconfirming evidence, carried out in-depth interviews, and made every effort to make only claims that were buttressed with ample data. But if, as I anticipated, the families did object to how they were portrayed, I would feel deeply conflicted. I would want to stand by my analysis but also please the participants. The result would be paralysis; I would find myself unable to move the book to closure. (Researchers vary in how easy they find writing and/or wrapping up a project. I find both steps challenging.) What if some participants found their portrait so painful that they requested that I drop that chapter? What would I do then? For example, even now, the Yanellis continue to find their portrait quite painful. If I had shown it to them ahead of time, they would have demanded that it be removed from the book or that it be radically restructured to eliminate their sense that they had been made to look like child abusers. If I had taken out the chapter, the book would have been weaker. If I had dropped the discussion of discipline, the central argument of the chapter would have been gutted. If I had proceeded to publish despite the Yanellis’ intense objections, I would have severely damaged our relationship.34