The abstract and slides are available below. Look for the full paper soon.

The Ethics of Twitter Research: A Topology of Disciplines, Methods and Ethics Review Boards

In the five years since its launch, the social networking and microblogging service Twitter has quickly grown to over 300 million users, generating over 300 millions tweets each day. By providing a simple platform for users to explain “what’s happening” in 140 characters or less, Twitter has become the Internet’s de facto public forum to sharing “pretty much anything [users] wanted, be it information, relationships, entertainment, citizen journalism, and beyond” (Dybwad, 2009). This sharing of information, relationships, entertainment, journalism, and beyond has made Twitter a cultural phenomenon.

Beyond the utility Twitter provides its millions of users, it also has emerged as a valuable resource for tapping into the zeitgeist of the Internet and its users. There is cultural and historical value in the information that flows across Twitter’s servers, notes Dylan Casey, a Google product manager: “Tweets and other short-form updates create a history of commentary that can provide valuable insights into what’s happened and how people have reacted” (Singel, 2010). Researchers have been quick to recognize the value in studying Twitter users and activities to gain a better understand of its users, uses, and impacts on society and culture from a variety of perspectives (see, for example, http://www.danah.org/researchBibs/twitter.php). The Library of Congress recognized this importance of Twitter when it announced in 2010 that, “Every public tweet, ever, since Twitter’s inception in March 2006, will be archived digitally at the Library of Congress” (Raymond, 2010, ¶2).

The Library of Congress’s announcement clearly validates the research importance of Twitter, but it also prompted concerns about creating a permanent archive of tweets, and whether such a proposal was properly aligned with users’ understanding of how the platform worked and their privacy expectations. Even in broadcasting the news, the language Wired Magazine chose underscored the apparent transition from a fleeting existence for tweets to a newly instilled sense of permanence when it stated, “While the short form musings of a generation chronicled by Twitter might seem ephemeral, the Library of Congress wants to save them for posterity” (Singel, 2010).

In the wake of the Library of Congress announcement, increased debates over the appropriateness of archiving public Tweets for research purposes have arisen (see, for example, Vieweg, 2010; Zimmer, 2010). Particularly relevant are numerous questions regarding how academic research on Twitter has proceeded thus far, such as: What disciplines are engaging in Twitter research and what amount of scrutiny of research ethics is typical within these fields? What research questions are being investigated, what data is being gathered, and how? Are subjects notified or given the opportunity to opt-out of being studied? How are research ethics boards evaluating such projects?

The goal of this paper is to seek initial answers to these questions by surveying academic research that relies on the collection and use of Twitter data. The body of research articles to be surveyed includes over 200 scholarly articles, dissertations and theses from disciplines ranging from communications, political science, health sciences, economics and computer science, among others. In building this corpus, this project will create a topology of disciplinary approaches to research around Twitter, methods used to collect and analyze Twitter data, and accounts of research ethics boards’ oversight of these projects. Through this analysis, we will gain an insight into the current state of research on Twitter, providing a better understanding of the methodological and ethical challenges before us.

Works Cited

• Dybwad, B. (2009). Twitter Drops “What are You Doing?” Now Asks “What’s Happening?”. Mashable.com. Retrieved February 25, 2012, from http://mashable.com/2009/11/19/twitter-whats-happening/.
• Raymond, M. (2010). How Tweet It Is!: Library Acquires Entire Twitter Archive. Library of Congress Blog. Retrieved February 25, 2012, from http://blogs.loc.gov/loc/2010/04/how-tweet-it-is-library-acquires-entire-twitter-archive/.
• Singel, R. (2010). Library of Congress Archives Twitter History, While Google Searches It. Retrieved February 25, 2012, from http://www.wired.com/epicenter/2010/04/loc-google-twitter/.
• Vieweg, S. (2010). The Ethics of Twitter Research. Paper presented at the CSCW 2010 Workshop on Revisiting Ethics in the Facebook Era.
• Zimmer, M. (2010). Is it Ethical to Harvest Public Twitter Accounts without Consent? MichaelZimmer.org. Retrieved February 25, 2012, from http://michaelzimmer.org/2010/02/12/is-it-ethical-to-harvest-public-twitter-accounts-without-consent/.

Here’s my initial assessment of the privacy concerns, with some information changed to keep the researchers and the project confidential.

 Areas of Concern

The primary concern related to researchers’ use of Facebook for recruiting and communicating with subjects (potential or real) relates to privacy and the possible collection of personally identifiable or other sensitive information.

The use of Facebook Ads results in limited privacy concerns in relation to what the researchers might learn about the subjects. While the research team can target a specific set of users – for example, 18-to-35-year-old women who live in Portland, OR – and would know that anyone visiting their website from that ad fit this general demographic profile, no information about individual Facebook users is shared with the research team. Thus, users would be able to view and click the advertisement, and then visit the research website, without fear of their identity being made known.[1] Further, the act of clicking on ads does not appear on user’s timelines, so friends of that user would not be aware of the user’s action related to the advertisement.

A greater privacy concern related to advertisements relates to what Facebook itself collects. If an advertisement includes text related to drug use, for example, and a users clicks on that ad, Facebook retains a record of that action, and could, potentially, use this action to further develop a profile of the user (beyond what the user voluntarily shares on the platform).[2] Facebook’s potential ability to infer information about the user based on the ads clicked is limited to the information provided in the ad itself, and potentially the URL linked to by the ad.

The use of Facebook Pages introduces greater privacy and confidentiality concerns. If a research team wishes to communicate to interested users or participants through the Page, users will need to “like” the page. This positive action of “liking” a page would appear in the user’s News Feed and be publicly viewable by the users friends. For example, if a user “likes” a page related to a drug use study, her friends would be informed of this action, and the page would be listed among that user’s likes on their profile page. Users can also see a list of friends who have “liked” a page by visiting that page at any time. Users do have the ability of controlling the visibility of their “likes” overall, but not of individual pages.

As with advertisements, user activities on Pages are tracked and potentially used by Facebook to compliment other behavioral and demographic data maintained by the company.

In terms of the [project], these concerns can be summarized as follows:

• Facebook possesses the ability to log user actions related to advertisements, thus any sensitive or controversial content within an ad’s text (such as mention of drug use) could potentially be linked to a user’s account within Facebook’s systems, and Facebook could attempt to infer something about the user based on that action to augment its internal profile of that user. This also applies to user actions on a Facebook Page.
• Facebook possesses the ability to log the URL of external websites users click to from a link embedded in ads or pages. Thus, if a URL contains sensitive or controversial information (for example, http://university.edu/DrugUseStudy), it could potentially be linked to a user’s account within Facebook’s systems, and Facebook could attempt to infer something about the user based on the URL to augment its internal profile of that user.
• User actions (likes, comments, etc) on Facebook pages created for research projects are typically visible on that user’s timeline and profile page, and also viewable by friends who happen to visit the project page. Thus, any sensitive or controversial information on the Page, including its name, could become associated with that user.

Preliminary Recommendations

The primary recommendation is to minimize the ability for Facebook or a user’s friends from associating interactions with project-related advertisements or pages with any sensitive or controversial information. Thus, the following actions can be taken:

• Advertisements should avoid any sensitive or controversial text that could be associated with a user who decides to take action. For example, instead of mentioning a study on “illegal drug use”, the ad can indicate “recreational activities” or similar. By using more generic text, any association made to the user’s account will have minimal harm.
• URLs used in advertisements or pages should also avoid sensitive or controversial text. For example, http://university.edu/RecreationalActivityStudy should be used rather than http://university.edu/DrugUseStudy. To best protect users, the linked pages should also minimize mention of sensitive or controversial text, ideally providing more general information, and then pointing users to other pages that provide more detail.
• Facebook Pages should also avoid sensitive or controversial text in both their name and the content provided within.
• While Facebook Pages are a practical tool for communicating with subjects, users might avoid “liking” a Page due to privacy concerns. Thus, other means of communication must be implemented.

---

[1] The research website likely maintains its own server logs, so a user’s IP address would potentially be collected. This is not related to the use of Facebook, and applies to any visitor to the website under typical circumstance.

[2] Presently, this is mentioned only as a potential concern; additional verification is needed to confirm if Facebook indeed uses “clicked ads” as a means to refine customer profiles.

What do readers think of this preliminary analysis and set of recommendations?

(And like in Pittsburgh, I will begin my talk by invoking this video by Neil Postman, which is what convinced me to get my PhD at NYU.)

I’ve discussed these issues in smaller disciplinary settings, as well as much larger national meetings of IRBs, but this will be my first time having detailed discussions with a university’s particular IRB community.

I’m really looking forward to it.

Internet Research Ethics
First published Fri Jun 22, 2012

There is little research that is not impacted in some way on or through the Internet. The Internet, as a field, a tool, and a venue, has specific and far reaching ethical issues. Internet research ethics is a subdiscipline that fits across many disciplines, ranging from social sciences, arts and humanities, medical/biomedical, and hard sciences. Extant ethical frameworks, including consequentialism, utilitarianism, deontology, virtue ethics, and feminist ethics have contributed to the ways in which ethical issues in Internet research are considered and evaluated.

Conceptually and historically, Internet research ethics is related to computer and information ethics and includes such ethical issues as data privacy and confidentiality, integrity of data, intellectual property issues, and professional standards. Throughout the Internet’s evolution, there has been debate whether there are new ethical dilemmas emerging, or if the existing dilemmas are consistent across research or despite technological influence (Elgesem 2002; Walther 2002; Ess & AoIR 2002). These debates are similar to philosophical debates in computer and information ethics. For example, many years ago, Moor asked “what is special about computers” in order to understand what is ethically unique and the same question applies to Internet research (Moor 1985; Ess & AoIR 2002; King 1996).

Yet, as the Internet has evolved into a more social and communicative tool and venue, the ethical issues have shifted from purely data driven to more human-centered. “On-ground” or face-to face analogies may not be applicable to online research. For example, the concept of the public park has been used as a site where researchers can observe others, but online, the concepts of public and private are much more complex. Thus, some scholars suggest that the specificity of Internet research ethics calls for new regulatory and/or professional and disciplinary guidance. For these reasons, the concept of human subjects research policy and regulation informs this entry, along with disciplinary standards, which will explore the growing areas of ethical and methodological complexity, including personal identifiability, reputational risk and harm, notions of public space and public text, ownership, and longevity of data as they relate to Internet research. Specifically, the emergence of the social web raises issues around subject or participant recruitment practices, tiered informed consent models, and protection of various expectations and forms of privacy in an ever-increasing world of diffused and ubiquitous technologies; anonymity and confidentiality of data in spaces where researchers and their subjects may not fully understand the terms and conditions of those venues or tools; challenges to data integrity as research projects can be outsourced to a mechanical turk or a bot; and jurisdictional issues as more research is processed, stored, and disseminated via cloud computing or in remote server locales, presenting myriad legal complexities given jurisdictional differences in data laws.

As a result, researchers using the Internet as a tool or a space of research—and their research ethics boards (REBs), also known as institutional review boards (IRBs) in the United States or human research ethics committees (HRECs) in other countries such as Australia—have been confronted with a series of new ethical enquiries: What ethical obligations do researchers have to protect the privacy of subjects engaging in activities in “public” Internet spaces? How is confidentiality or anonymity assured online? How is and should informed consent be obtained online? How should research on minors be conducted, and how do you prove a subject is not a minor? Is deception (pretending to be someone you are not, withholding identifiable information, etc) online a norm or a harm? How is “harm” possible to someone existing in an online space?

A growing number of scholars have explored these and related questions (see, for example, Bromseth 2002; Bruckman 2006; Buchanan 2004; Buchanan & Ess 2008; Gullor & Ess 2003; Johns, Chen & Hall 2003; Kitchin 2003, 2008; King 1996; Mann 2003; Markham & Baym 2008; McKee & Porter 2009; Thorseth 2003), scholarly associations have drafted ethical guidelines for Internet research (Ess & Association of Internet Researchers 2002; Kraut et al. 2004), and non-profit scholarly and scientific agencies such as AAAS (Frankel & Siang 1999) have begun to confront the myriad of ethical concerns that Internet research poses to researchers and research ethics boards (REBs).

• 1. Definitions
• 2. Human Subjects Research
• 3. History and Development of IRE as a Discipline
• 4. Specific Ethical Issues in Internet Research
  • 4.1 Privacy
  • 4.2 Recruitment
  • 4.3 Informed Consent
  • 4.4 Cloud Computing and Research Ethics
• 5. Research Ethics Boards Guidelines
• Bibliography
• Academic Tools
• Other Internet Resources
• Related Entries

---

One of the recent key trends in communication and technology research concerns the use so-called “big data” to study social media use. Whether its high profile projects on students’ Facebook use, comprehensive mappings of the blogosphere, or studies of Twitter use during crisis, the potential for collecting and processing large amounts of new forms of data has brought novel approaches and important insights. However, these new forms of data also require us to critically engage with some of the more delicate issues regarding how research on social media use is undertaken. The presentations will deal with how ethical challenges in these contexts can take on different forms, regarding how we as researchers deal with respondents and data providers, as well as how we communicate our ideas to institutional review boards. Bringing together scholars from Australia, Germany, Scandinavia and the US, the panel is well- suited to generate discussion among division members, as well as attendees beyond the CAT division.

My contribution is “New Media, New Ethics: How Social Media-Based Research Demand New Attention to Research Ethics“, where I discuss many of the  growing conceptual gaps that are emerging in relation to Internet research ethics:

Social media tools have opened up vast new means for communication, socialization, expression, and collaboration. They also have provided new avenues for researchers seeking to explore, observe, and measure human opinions, activities and interactions. While scholars, professional societies, and institutional review boards have long-established research ethics frameworks to ensure the rights and welfare of the research subjects are protected, the rapid rise of powerful social media platforms – where individuals increasingly share personal information on platforms with porous and shifting boundaries – provide new challenges to long-held ethical assumptions and guidelines. This talk will present various cases of social media-based research that expose new conceptual gaps in how we think about privacy, anonymity, consent, and harm in the 2.0 era.

My slides are available below, and here are some of the other research and resources I cite in the presentation:

• Internet Research Ethics Digital Library, Resource Center and Commons website
• Ethical decision-making and Internet research: Recommendations from the AoIR ethics working committee
• Buchanan, E., & Ess, C. (2009). Internet research ethics and the institutional review board: current practices and issues. ACM SIGCAS Computers and Society, 39(3), 43-49.
• Carpenter, K & Dittrich, D. “Bridging the Distance: Removing the Technology Buffer and Seeking Consistent Ethical Analysis in Computer Security Research” 1st International Digital Ethics Symposium, Loyola University Chicago Center for Digital Ethics and Policy.
• Moor, J. (1985). What is computer ethics? Metaphilosophy, 16, 266-275.
• Nissenbaum, H. (2009). Privacy in context: Technology, policy, and the integrity of social life. Stanford, CA: Stanford University Press.
• Ohm, P. (2009). Broken promises of privacy: Responding to the surprising failure of anonymization. UCLA Law Review, 57, 1701.
• Soghoian, C. (2012). Enforced community standards for research on users of the Tor anonymity network. Financial Cryptography and Data Security, 7126, 146-153.
• Zimmer, M. (2010). “But the data is already public”: On the ethics of research in Facebook. Ethics and Information Technology, 12(4), 313-325.

The Blackberry Project (formerly known as the Friendship Project) is an ongoing longitudinal study examining teen behavior and sociability, which first recruited its subjects in 2003 (starting with 281 third and fourth graders from 13 Dallas public schools) and relied on yearly laboratory and home observation and surveys for data collection. Then, in 2009, the subjects (now entering 8th grade) were provided with BlackBerry devices with unlimited text and data plans paid for by the investigators. The devices were configured so that the content of all text messages, e-mail messages, and instant messages was saved to a secure server to be mined by the researchers — over 500,000 messages a month are being archived. Preliminary analyses have been published in Developmental Psychology.

The result? Hill puts it best in her headline and opening thoughts:

A Texas University’s Mind-Boggling Database Of Teens’ Daily Text Messages, Emails, and IMs Over Four Years

For the past four years, the University of Texas-Dallas developmental psychology professor has essentially wire-tapped 175 Texas teens,  capturing every text message, email, photo, and IM sent on Blackberries that she provided to them, creating a rich database that now contains millions of funny, explicit, sexual, and inane messages for academic study. Half a million new messages pour into the database every month. This summer, she’s adding Facebook content to the mix as well. The teens sacrificed their privacy for science… and a free smartphone, data plan and unlimited text messaging.

Dr. Underwood’s study has been approved by UT-Dallas’s Institutional Review Board, and she’s also received a Certificate of Confidentiality from the NIH, which are only granted after considerable scrutiny. Each participant is given a unique identification number so that all information that is collected is, according to the project website, “de-personalized”. The research data is stored securely with the help of Ceryx and Global Relay, data security providers who typically work together to store and archive electronic communication data for financial institutions. The archive is password protected and can only be accessed by a small group of selected researchers.

In short, this large-scale and long-term project has undergone considerable review, and appears to be taking privacy and security quite seriously. That said, there remain certain ethical concerns about the research worth discussing.

(Note: my discussion is based on what I can glean from available reports and documents about the study; I’m trying to gather additional information through various channels.)

Consent

Since the Blackberry Project (and its predecessor) focus on studying the activity of minors, gaining informed consent is of particular importance. Participants and parents were required to sign detailed consent forms annual that clearly stated that all electronic communication were be recorded and monitored. (While the consent forms for the earlier Friendship Project are available online, I haven’t been able to locate the consent documents for the Blackberry Project. I’ll request them from Dr. Underwood.) It appears this consent process was repeated annually, which is particularly important as subjects grow and develop, and the content of their text and email messages might change over time (for example, 10th graders might start texting about dangerous or legal activity, which might not have been contemplated when original consent was provided years earlier).

Parental consent for minor subjects is standard procedure. However, I wonder how well a parent actually understands the extent to which adolescents make use of mobile texting, and whether a parent really is equipped to represent (and waive) the privacy interests of their adolescent kids if they fail to recognize both the scale and types of information contained within those text messages. Is parental consent really sufficient when we’re dealing with teenager’s use of social media and personal technology? This is something I’ll need to think about more….

Further, any consent granted only involves the participants themselves and their outgoing messages. But those sending messages to the participants have not consented to having their messages stored and subjected to analysis. Underwood recognizes this problem, but argues it away:

Pioneering researchers studying online communication have argued that electronic communication can be observed without permission in some contexts because the information need not be uniquely identifiable, unless individuals have chosen to make their online user name their actual name (see Subrahmanyam et al., 2006; Whitlock, Powers, & Eckenrode, 2006). In our study, although we did have access to participants’ phone contacts and could see how they labeled individuals there, these were rarely uniquely identifiable, because most adolescents chose to label contacts with first names only or with nicknames.

However, I find this argument a bit thin. Just because some “pioneering researchers” claim it is acceptable to study online messages observed without permission “in some contexts” doesn’t make it necessarily ethical here. Hopefully the IRB pressed hard on this issue.

Undue Influence

Consent is only valid if it doesn’t involve coercion or undue influence. While paying research subjects is commonplace and generally acceptable, the fact that subjects in the Blackberry Project received a free smartphone with fully paid data and texting plans (and a generous 300 minute voice plan) might quality as undue influence. The Office of Human Research Protections defines undue influence when researchers offer an “excessive or inappropriate reward or other overture in order to obtain compliance.” OHRP also notes that “The level of remuneration should not be so high as to cause a prospective subject to accept risks that he or she would not accept in the absence of the remuneration.”

This is where the free Blackberries and service plans might be problematic. Since 11% of the participating families had incomes under $25,000, and 29% under $50,000, the allure of a free, “highly attractive” smartphone, complete with a free and unlimited data plan, might have persuaded some lower-income families to participate who otherwise might have considered the project too risky. If you’re on a tight budget, and your kids keep pestering you for a smartphone, the Blackberry Project might have been a lifesaver, regardless of the risks.

Determining undue influence is a grey area, and, again, I hope that UT-Dallas’s IRB considered this matter with vigor.

Privacy and Anonymity

Underwood has taken great lengths to protect subject privacy, including the use of secure, off-campus data storage platforms and replacing account names with ID numbers within the archive. Yet, considerable privacy concerns remain. There are plenty of cases where simply replacing names with ID numbers fails to provide sufficient anonymity, and the content of the messages themselves might reveal various personal details of the participants and their friends. The researchers indicate they use the participants address books to help “replace phone numbers with whatever the participants used to label their contacts” when compiling transcripts. While some of these labels might be un-identifiable, others might effectively “out” particular people within the dataset.

The Forbes article also notes:

Underwood has gotten calls from investigators around the country who would love access to her database, but she says she doesn’t want to hand over the data unless she can de-identify it or anonymize it. I’m imagining many a privacy scholar shaking his or her head in dismay given how difficult true anonymization is.

Indeed. I’m curious to know what steps toward deidentification or anonymization Underwood intends before sharing the data.

The Forbes piece presses Underwood further about the issue of privacy:

When I asked Underwood if any of the kids (or their parents) had ever expressed concern about the privacy of their communications, and the discomfort they might feel about every single thing they send being archived indefinitely for study, she said it had been a “non-issue.”

“We haven’t really directly asked about it. We don’t do anything to draw attention to our monitoring,” says Underwood. She prefers that teenagers act naturally. Asking them too strongly about how they feel about their privacy might negatively affect the “observing them in the wild” aspect of her study.

This troubles me. Here, a researcher collecting millions of personal messages sent between teens admits to not wanting to directly address privacy with the subjects because it might negatively affect the study. If you bring up the privacy concern, Underwood seems to say, it will just cause them to self-censor. Of course, if her hypothesis is true, that validates the privacy concern itself — the participants might actually care about their privacy, once reminded about it. (Note to researchers: if you find yourself wanting to minimize disclosure of privacy concerns, then you have significant privacy concerns that need to be addressed.)

In sum, the Blackberry Project appears to have been managed properly through the IRB rules and regulations. These open issues speak more to the nature of this kind of research generally, versus about this project specifically. I’m very curious as to how the researchers and the IRB discussed and deliberated these issues, and will provide any updates if I’m able to gain access to more details.

On the opening morning of the conference, I will join John Palfrey and Lydia Shrier for a plenary panel on “Would Margaret Mead Have Blogged? How Social Media has Changed Research”. My slides for that presentation are provided below. I will also be leading a workshop on “Research, the Cloud, and the IRB”, and giving a presentation to the U.S. Department of Energy Human Subjects Working Group.

Internet research ethics is well represented at this year’s AER conference, with a strong track focusing on “Research Involving the Internet & Social Networking”. Among these presentations and workshops are several led by Elizabeth Buchanan. Buchanan will be co-facilitating a pre-conference workshop entitled “What a Tangled Web We Weave: Ethical, Regulatory, and technical Aspects of Internet Research”. She is also participating in the following workshops:  “Research, the Internet, and the IRB: Ethical and Regulatory Issues,” “Research, Social Media and the IRB,” and “Ethical Implications of Pre and Post Enrollment uses of Social Media in Clinical Trials”.

It is a well-written article, quite balanced, and features myself, the T3 principle researcher Jason Kaufman, and fellow Internet research experts Alex Halavais, Fred Stutzman, and Elizabeth Buchanan (I am friends with the latter three, for disclosure). The Chronicle also tracked down a Harvard student presumably within the dataset.

For those looking, my initial blog posts (from 2008) regarding the T3 dataset are here and here, and my full treatment of the dataset release was published here:

• Zimmer, M. (2010). “‘But the data is already public’: on the ethics of research in Facebook,” Ethics & Information Technology, 12(4), 313-325

I don’t want to rehash the entire article or episode, but would like to provide a few annotations:

The article does a nice job pointing out the dual challenges of “Researchers [who] must navigate the shifting privacy standards of social networks and their users”, as well as the “the committees set up to protect research subjects—institutional review boards, or IRB’s—[who] lack experience with Web-based research.”

These are critical revelations that we cannot take lightly. There is much work to be done to ensure researchers of all disciplines and levels recognize and respond to the complexities of engaging in this kind of research online, and that IRBs are sufficiently trained to recognize issues related to Internet research ethics.

To these ends, the Association of Internet Researchers (AoIR) has published an ethics guide (now undergoing revisions) as “as at least a starting point for their inquiries and reflection”, and we’ve held various workshops on the subject. Elizabeth Buchanan and Charles Ess have spearheaded important research on the IRBs’ awareness of Internet-related concerns, and have launched the Internet Research Ethics Digital Library, Resource Center and Commons website as a valuable resource.

And, specific to the article’s mention that I have “pointed to the Harvard case in urging the federal government to do more to educate IRB’s about Web research”, I was privileged to present before the Secretary’s Advisory Committee on Human Research Protections (SACHRP), part of the Office for Human Research Protections in the United States Department of Health and Human Services (HHS). Joined by Elizabeth Buchanan, Montana Miller, and John Palfrey (of Harvard’s Berkman Center, by the way), we discussed emerging ethical issues with Internet-based research and urged the committee to take steps to ensure IRBs and researchers were suitably trained to recognize and address these important ethical issues.

In the context of this entire debate (and some of the original comments left on my blog posts), this passage from the article is quite telling:

But Mr. Kaufman talks openly about another controversial piece of his data gathering: Students were not informed of it. He discussed this with the institutional review board. Alerting students risked “frightening people unnecessarily,” he says.

“We all agreed that it was not necessary, either legally or ethically,” Mr. Kaufman says.

Frankly, I’m troubled by this statement. I will leave it to legal experts to determine if the research violated the consent requirements of the Federal Regulations for the Protection of Human Subjects (45 CFR 46), but from an ethical standpoint, I argue the researchers did have an obligation to respect the intentions of those students who might have restricted their Facebook profiles to only be visible to members of the Harvard community. The researcher’s own codebook acknowledged that the assistants used to access the profile data might have had preferential access to a profile, and that “a given student’s information should not be considered objectively ‘public’ or ‘private’”. This realization should have triggered an ethical concern over whether each students truly intended to have their profile data publicly visible and accessible for downloading.

This is the crux of the issue, and my earlier attempts to learn if and how this apparent waiver of the consent requirement was deliberated by Harvard’s IRB were unsuccessful. Perhaps now we can gain a bit more understanding of why it was deemed that consent wasn’t necessary (and I hope it was a more nuanced decision than simply avoiding “frightening people unnecessarily”).

I agree with the article’s conclusion that the “biggest victim” in this episode is academic scholarship.

The uniqueness of this dataset is of obvious value for sociologists and Internet researchers, and it wasn’t my goal to shut down this research project. It is unfortunate the researchers haven’t been able to find a suitable means of re-releasing the data, but just like the AOL search data release forced us to rethink methods of anonymization before again releasing large datasets of transaction logs, I’m hopeful that this episode can prompt meaningful consideration and debate of our understandings of privacy, anonymity/identifiability, consent, and harm when it comes to Internet-based research.

Finally, I wanted to provide a brief response to the implicit accusation made in the article that I’m a part of some kind of “academic paparazzi”.

I’m not even sure what this means. Perhaps someone thinks I spend my time trolling through other people’s research hoping to find a place where they slip up so I can have a “gotcha” moment? Hardly. I had never written on research ethics until I came across this particular case. I saw a passing mention of the data release on another scholar’s blog, and the ensuing discussion there about how the presumed anonymity of the dataset should be questioned due to its unique data variables. So I started to explore, and my discoveries followed. I’m not out to get anyone, but rather have taken quite a number of proactive steps to help researchers (both the T3 team and more broadly) address these complexities.

The complexities of research ethics and methodology in today’s Internet-based environment is complex, and I’m just starting to scratch the surface. But I don’t take this lightly; I’m a scholar, not a paparazzo.

As I conclude in my full article:

The purpose of this critical analysis of the T3 project is not to place blame or single out these researchers for condemnation, but to use it as a case study to help expose the emerging challenges of engaging in research within online social network settings. …The T3 research project might very well be ushering in ‘‘a new way of doing social science’’, but it is our responsibility scholars to ensure our research methods and processes remain rooted in long- standing ethical practices. Concerns over consent, privacy and anonymity do not disappear simply because subjects participate in online social networks; rather, they become even more important.

I hope that’s the takeaway from all this.

The research paper indicates that the Facebook data was provided to the researchers “in anonymized form by Adam D’Angelo of Facebook.” (D’Angelo was then Facebook’s CTO, and left Facebook in 2008.) Curious as to what precisely was included in the data release, and what steps towards anonymization were taken, I downloaded the data (200 MB zip file) on the morning of February 11.

The data files are separated by institution, and in total include, by my estimation, about 1.2 million user accounts. The content of each institution’s file is described as containing the following:

Each of the school .mat files has an A matrix (sparse) and a “local_info” variable, one row per node: ID, a student/faculty status flag, gender, major, second major/minor (if applicable), dorm/house, year, and high school.

Thus, the datasets include limited demographic information that was posted by users on their individual Facebook pages. The identity of users’ dorm and high schools were obscured by numerical identifiers, but to my surprise, the dataset included each user’s unique Facebook ID number. As a result, while user names and extended profile information were kept out of the data release, a simple query against Facebook’s databases would yield considerable identifiable information for each record. In short, the suggestion that the data has been “anonymized” is seriously flawed.

The consequences of this ease of re-identifying the dataset are numerous.

First, while only limited profile information is within the dataset, there is no indication that any consideration was given to users’ particular privacy settings. Based on the article, all user accounts from each of the 100 networks were provided to the researchers, and as long as the user provided the data to Facebook, it was turned over to the researchers. [Clarification: when I say "all user accounts" we provided, I do not mean full profile information was given to the researchers, just the particular data fields as described above]

Yet, in 2005, users had the ability to restrict access and visibility of their Facebook profile, their demographic data, and their lists of friends (much of this control was taken away in 2009). So, a user might have restricted access to certain information to only people within her network or just her friends, and Facebook’s own privacy policy at the time promised that: “No personal information that you submit to Facebook will be available to any user of the Web Site who does not belong to at least one of the groups specified by you in your privacy settings.” This data release, and the ease by which users could be identified and linked to their data, potentially negates actions taken by users to control access to the data within the files, and seemingly contradicts Facebook’s own privacy policy.

Second, even though the specific data exposure within the dataset is limited, the fact that users can be identified and linked to their in-network social map fosters additional threats to privacy. Previous research (here and here, for example) has shown how “anonymous” datasets can be largely re-identified when there is access to other large sets of data where the subjects are already known. The “Facebook 100″ data, with the Facebook IDs intact to guide identification of users, might be useful in similar efforts.

To recap, the suggestion that the “Facebook 100″ data has been “anonymized” is seriously flawed, and its release might be putting the information of 1.2 million Facebook users at risk.

Interestingly, a few hours after the initial release of the “Facebook 100″ dataset, the researchers Mason Porter announced they were pulling the data due to an unspecified “bug”. Later that evening, the data was again made available with a message indicating that the data files were now fixed.

Again, I was curious, so I downloaded and examined the new dataset. The only change I could see was that now the Facebook ID was removed entirely from the data files, and the order of the records in each file was randomized.

Thus, the “bug” must’ve been that the data was easily re-identifiable, and the “fix” was to take additional steps to anonymize the records. Somone joked on the announcement email list that the “bug” must have something to do with Facebook attorneys, but the Porter’s message re-releasing the data jokes that no lawyers were involved, and that they “really were fixing the data files!”

To me, however, the language used in these explanations was disingenuous. The data, as far as I could tell, had no bugs that prevented its usefulness for social network analysis. No, the problem with the data was that it contained each user’s unique Facebook ID, thus allowing easy identification. The researchers Porter should have been open and honest about why the data was pulled and what they did to correct the situation.

That said, there are still a number of open questions regarding this particular dataset:

To Facebook:

• What kind of internal processes, if any, did D’Angelo follow when releasing the data to these researchers? Was he authorized to do so?
• Was this kind of large data release routine? How many other similar releases have taken place?
• Does Facebook consider releasing this information, with Facebook IDs, in compliance with the privacy policy in effect in 2005? If so, how?

To the research team:

• Was the data received by Facebook already obscured with numerical identifiers replacing student majors, minors, and high schools, or did you add those?
  • UPDATE: I have received word from one of the researchers, Mason Porter, that the data sent to them by Facebook was indeed already obscured with numerical identifiers in the place of actual student major, minor, and high school information.
• Did your IRB review the data used for the research, and approve the subsequent data release?
• Was there any “bug” in the data, or was the attempt to gain greater anonymization of the data the sole reason to pull it from public access?

Obtaining answers to these questions can help us better understand the uniqueness of this situation, and to put better processes and protections in place to prevent similar data releases that falsely believe data is sufficiently anonymized and respecting of users’ privacy expectations.

I hope Facebook and the researchers are willing to engage in a discussion, and I’ll report back on any communication, as allowed.

UPDATE (Feb 15, 6:00pm): I have been in contact with one of the researchers, Mason Porter, who confirmed that the data sent to them by Facebook was indeed already obscured with numerical identifiers in the place of actual student major, minor, and high school information. I’ve inserted this reply into the question above. I have also made a few minor changes to the main text, clarifying that the email messages reporting the “bug” in the data came from Mason alone, and should not be attributed to the entire research team.

UPDATE 2 (Feb 15, 6:10pm): The link to the full, revised dataset (http://people.maths.ox.ac.uk/~porterm/data/facebook100.zip) is no longer active.

UDPATE 3 (Feb 16, 9am): Added a clarification that when I say “all user accounts” were provided to the researchers, I do not mean full profile information was given, just the particular data fields as described above.

UDPATE 4 (Feb 16, 11am): Mason Porter, one of the authors, has posted an explanatory note on his blog indicating that he’s been in contact with the Facebook Data Team, and per their request, “I have taken down the data, and I will be working with them to eventually post a version of the data set with which both they and I are happy.”