Some thoughts on criticisms of citizen science

NPR’s Science Friday had a program on citizen science just last week: Doing Real World Science, but Skipping the Ph.D. They highlight some cool projects, including a couple I hadn’t heard about before. So I recommend listening online or reading the transcript if you didn’t catch it on the air.

One of the guests of the show was Alex Wright who recently wrote an article on citizen science for the New York Times: Managing Scientific Inquiry in a Laboratory the Size of the Web. Both Wright’s article and the Science Friday program brought up the issue of accuracy in citizen science and whether people who take part in these projects can really be called “scientists.” But in the article, one particular quote stuck out for me:

“These people are not doing the work of scientists,” said David Weinberger, a senior researcher at the Berkman Center for the Internet and Society at Harvard, who is writing a book about the changing shape of human knowledge in the online era. “They are doing the work of scientific instruments.”

Though I realize I can’t be sure in what context the above quote was said from the article, it annoyed me. Many citizen science projects in ecology and wildlife related fields involve similar tasks as interns and entry-level lab and field technicians, data collection and entry, albeit simplified. By the above logic, I got a 4-year degree to become largely a scientific instrument. Well, I suppose I have worked with equipment that I’ve been told is worth more than my education, so maybe that’s a compliment. Seriously now, who is going to think they are a genuine astrophysicist because they participated in a Zooniverse project? (Okay, maybe whoever thinks publishing their work in the Journal of Nature and Science is the same thing as publishing in the Journals of Nature + Science; Warning: possibly not suitable for work language, but hilarious).

In addition, citizen scientists can and have gone beyond the basics of what is asked of them in a project and even been contributing authors to scientific papers. I like this quote from the Zooniverse blog in response to the New York Times article:

If you need to run your own projects, or to acquire a publication record to be a ‘citizen scientist’, then consider it an aspirational label. The Zooniverse provides everything you need to do that, although, for now, the barriers are still high. Otherwise, if you contribute to our understanding of the Universe in however minor a fashion, then I’ll call you a scientist, and I look forward to being able to drop the distinction between professional and citizen.

But the issue of accuracy is I think a valid concern. To account for this, many projects are made simple with detailed and easy to understand instructions. They also check for errors in the data just like they would if a paid technician or intern collected the data. After all, both the professional and amateur are capable of human error. So, yes, citizen science projects should take the issue of accuracy into account when designing a project, just like one recent project did.

Researchers at the University of Gloucestershire in the UK looked at the usefulness and accuracy of using existing social networks as a means for untrained people to participate in the citizen science project BeeID. The project’s goal was to map distributions of bee species in the UK through pictures submitted to the BeeID group on Flickr, an interactive photo sharing site.

The instructions on the group’s page were relatively simple: 1) upload a photo of a bee taken in the UK in 2010, 2) add the tag “BEEID2010” to the photo, and 3) add the photo to the Flickr map manually or automatically from GPS data given on a smartphone. The researchers then used a computer program to gather the time and date taken and location data from the photos as long as they were correctly tagged. Researchers and “bee experts” then identified the bees from the photos rather than the project participants. However, the authors acknowledged that in many projects with citizen scientists that are experienced amateur naturalists, the volunteers may be better than the scientists at identification.

At the time of the write-up, the project had received 36 participants and just over 200 bee photos in a period of only 10 weeks. The authors found using Flickr to host a citizen science project and Facebook as a promotion tool can be useful for collecting data, especially with limited funding. As you might have guessed, not all submissions had followed the instructions. In total, 59% of the photos were both correctly tagged and put on the map. However, the project was set up so that submissions not following the instructions could be easily excluded.

While 59% may seem low, in comparison to other photos on Flickr with the “bee” tag, only 13% have any sort of geographic information. A similar Flickr group (but not truly a citizen science project) is the BBC’s Bee Part of It campaign that asks users to add only bee pictures to the group and encourages location data. Of those photos, only 25% (about 500 photos) had been added to the Flickr map. So, when trying to get species presence information over a specific area at low-cost, setting up a group with clear instructions can produce much more meaningful results than “data mining” or just searching for photos.

Could they have gotten more accurate results if they sent out a “real scientist” trained in bee identification and a complete understanding of the protocol? Probably. However, you will find the following sentence in the research paper, “The authors have no support or funding to report.”

Reference

Stafford R, Hart AG, Collins L, Kirkhope CL, Williams RL, et al. (2010) Eu-Social Science: The Role of Internet Social Networks in the Collection of Bee Biodiversity Data. PLoS ONE 5(12): e14381. doi:10.1371/journal.pone.0014381