Ecological Data Sharing:

Current practice and lessons for “scaling up”

Ann Zimmerman

LTER Ecoinfomatics Workshop

October 30, 2003

M.A. Library and Information Science, University of Iowa, 1986

Ph.D., Information and Library Studies, University of Michigan, 2003

Librarian, 1991-Nov. 2003

U.S. Geological SurveyGreat Lakes Science Center

Ann Arbor, MI

Librarian, 1987-1991

U.S. Fish and Wildlife ServiceNorthern Prairie Wildlife

Research CenterJamestown, ND

Data sharing is necessary in order to address many environmental problems.

The destruction of rainforests influences weather patterns in other parts of the world.

Airborne pollutants from one country affect the health of another nation’s water supply.

No one could use my data! They

wouldn’t understand them!

We must have your data to save the

planet!

Why ecological data?

Ecologists work at small spatial and temporal scales

Data sets are small and highly diverseStandard methods are difficult to achieveEcology is a craft scienceThere is a high level of data ownership

Intriguing Questions

Why are some data easier/harder to share than others?

Do standards really facilitate data sharing? If so, when? If so, how?

How do secondary users judge data quality?

Answers are relevant to…

Design of data resourcesStandards development PolicyEducation

Existing Research

The affect of databases on the practice and communication of science

Scientists’ attitudes toward data sharingResearch-related information that scientists

shareExpected returns for sharingData withholding

RQ: What are the experiences of ecologists who use shared data?

How do ecologists locate data and assess their quality?

What are the characteristics of the data they receive?

What information do ecologists depend on to use the data?

What challenges do they face throughout the process?

Qualitative Research Methods

Effective when important variables are unclear and empirical information is scarce

Useful for understanding processes as well as outcomes

Interviewing is a useful method to study past events and when participants cannot be observed

Qualitative Research Limitations

Imprecise measurementVulnerability to biasWeak generalizability of findings

Key Definitions Data

Scientific data

Scientific or technical measurements…and observations or facts that can be represented by numbers…and that can be used as a basis for reasoning or further calculation (NRC, 1997).

Ecologists?

“I’ve never seen so many khaki pants in my life.” “…and beards, this must be the ESA crowd.”

Rahel, Frank J. 1983. The habitus of ecologists: Fear and clothing at Penn State. Bulletin of the Ecological Society of America 64(3): 219-220.

?

Ecologists

Members of ESA, orSelf-identification, orAffiliation or title contains ecolog*

Data Sharing

The voluntary provision of information from one individual or institution to another for purposes of legitimate scientific research (Boruch, 1985)

My study is limited to shared data used for ecological research.

Secondary Use of Data

The use of data collected for one purpose to study a new problem

Includes data gathered to address a specific research question & data used to describe biological or physical phenomena

Data Collection

Method: Semi-structured, in-depth interviews

Primary subjects: 13 ecologists who reused data (selected from 2 key ecological journals)

Secondary subjects: 4 data managers

Data Analysis

Primary data: Interview transcripts

Developed a coding scheme and analyzed data following suggestions from Miles & Huberman*

*Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook, 2nd ed. Thousand Oaks, CA: Sage

Publications. .

Reliability

Reliability as consistency in judgment

*detailed descriptions of selection of subjects, data collection, & data analysis

*reporting of researcher biases, values, & central assumptions

Validity

Validity: accuracy of the information and whether it matches reality

Data triangulation: use of diverse sources to study the same phenomenon

Check results with subjects

Conceptual Framework

Overcoming Distance

Overcoming

D i s t a n c e

Potential Distances:

Cultural, Epistemological, Methodological, or Terminological

Temporal or Spatial

Personal

SocialExchange

Standards Informal

Knowledge

Standards as Distance Spanners:Making Local Knowledge Public

Measurement as a social technology (Porter)*Quantification as a technology of distance

*Standards as a substitute for trust based on personal knowledge

Porter, T. M. (1999). Quantification and the accounting ideal in science. In M. Biagioli (Ed.), The science studies reader (pp. 394-406). New York: Routledge.

Porter, T. M.(1995). Trust in numbers: The pursuit of objectivity in science and public life. Princeton, NJ: Princeton University Press.

Standards Reduce & Amplify

Standard measurements involve a loss of information (reduction).

Reduction turns local knowledge into public knowledge (amplification).

Latour, B. (1999). Circulating reference: Sampling the soil in the Amazon forest. In Pandora’s hope: Essays on the reality of science studies (pp. 24-79). Cambridge, MA: Harvard University Press.

Circulating Reference

The ability of standards to bring the world closer, yet also to push it away

Key Findings

Overcoming Distances in the Secondary Use of Data

Gathering One’s Own Data Helps with Reuse

Ecologists' experiences as collectors of their own data in the field or laboratory plays the most important role in their secondary use of data.

Field-based knowledge

Understanding data

Judging data quality

Data Gathering Provides:

Expertise to understand the critical link between the purpose, the research methods chosen, and the data that result

Ability to recognize data limitationsAbility to visualize potential points of errorA ‘sense’ for data

Research purpose Methods Data

What frog species live here? How many frogs live here?

Charles:

“In some ways it is just very simple. Someone saw an animal on such and such a date at such and such a location. That’s basically it. And you can explain that to six-year-old. The only tricky thing…. and, you know, in some ways it is not that hard conceptually, but I see people making the mistake all the time… What does the absence of a record mean? And the absence of a record doesn’t mean the absence of a species. It may just mean a lack of survey effort. And you see biological reports all the time that people consult the state biodiversity database and say, “Oh, we have no endangered species on this piece of property. It’s okay; go ahead and turn it into a shopping mall.”

Note: All interviewee names are pseudonyms

Susan:

“Well, where the different sources of error can come in-- things like getting water samples, or running the equipment and running the machines that actually analyze water chemistry, and how where you sample within a lake might influence dissolved organic carbon. So, you just get a better idea of all the different things that could influence the final number.”

Visualizing Potential Points of Error

Factors that Influence Research Methods

The scientific questionThe environmentThe taxaPractical considerations such as time,

money, and skill

Nancy:

“When you're in the field, most of what you learn is not the data points you're collecting -- it's just that sense.”

Michael:

“The more you actually go out and do those things the more.... You are sort of more critical of the data.”

Gaining a ‘sense’ for data

Standards of Scientific Practice

Ecologists recognize the informal knowledge they gain in the field, but it is not discussed publicly in the context of “real science”

Formal notions about norms of scientific practice guide the gathering of data for reuse and frame ecologists’ experiences

Hindrances to Sharing & Reuse

Challenge of locating and integrating data collected for many different purposes and at varying temporal and spatial scales

Ecologists’ idiosyncratic methods of organizing data

Re-circulating Reference

Ecologists attempt to reconstruct the original collection of the data they seek to reuse.

Ellen:

“If honestly I could not figure out what they had done, then I just would not use that data point.”

Nathan:

“One person could have a table that has a column of species and density. Another person could have a table that says Species I Density, Species II Density, and Species III Density. Those sorts of schema differences when you scale them up to 10, 20, 30 data sets -- and we would like to get to 100, 200, 1000 data sets -- become extremely limiting in your ability to integrate the data and to utilize them in a particular framework.”

Factors Influencing Data Reuse

Scientific questionsData collection methods Data characteristicsData ownershipPresence of standardsReuse potential IntermediariesEconomic or political value of dataComputational and statistical capacityExistence of formal data sharing systems

Applicability of findings

Data documentation: Develop tools and policies to gather information that data collectors are best suited to provide

“Scaling up”: Recognize the importance of intermediaries

Education: Teach data management, data documentation skills, and ethics and guidelines for secondary data use

Christine:

“In a field like molecular ecology, you grind up a sample, extract the DNA, and sequence it. It's the same thing over and over regardless of the material, and so it's relatively easy to standardize that. Of course, the more I work in molecular ecology, the more I realize that there are many sources of error, many points of decision making, etc. that can and do make standardization difficult.”

Charles:

“The economics data is often much more organized and processed. In economics, typically people are working with a shared data set. There are hundreds of people that work with the current population survey, for example, and you can go and find out, "Well, what are the problems with this data set?" Everyone can tell you, "Oh yeah, ’79 was a really bad year, and there’s a glitch, and you are going to have to reprocess this field if you want to use it. … But ecology data is not like that. Typically it never gets re-analyzed. And so you are on your own and kind of starting from scratch working with, untested and unverified, unvalidated, and unchecked out data most of the time.”

Most scientific data are not simple “measurements”

Taken from a seminary by Paul Avery (University of Florida)

Data Grids for 21st Century Data Intensive Science

SOC seminar – May 8, 2003

Available at: http://www.scienceofcollaboratories.org/NewsEvents/index.php

“…the analysis of proteomics data is currently informal and relies heavily on expert opinion. Databases and software tools developed for the analysis of molecular sequences and microarrays are helpful, but are limited owing to the unique attributes of proteomics data and differing research goals.”

Boguski, Mark S. and Martin W. McIntosh. 2003. Biomedical informatics and proteomics. Nature 422: 233-521.

Ecological Circuitry Collaboratory

“At an individual level, we would like all students in this program to be better able to build, use, and understand models while at the same time have firm grounding in the practices of field- and lab-based empirical science.”

http://www.ecostudies.org/cc/index.html

www.scienceofcollaboratories.org

Project Goals Perform a comparative analysis of collaboratory

projects through invitational workshops that bring together

collaboratory researchers from around the world,

Develop a Collaboratory Knowledge Base technical and social data and detailed findings from existing

collaboratory projects,

Develop general principles and design methods with broad community participation,

Test these principles on existing collaboratories

Bonnie A. NardiSchool of Information and Computer ScienceUniversity of California Irvinenardi@ics.uci.edudarrouzet-nardi.net/bonnie

Research goals:

to provide input to the design of better information & collaboration tools for ecologists

to understand ecologists’ innovative ways of approaching complexity as a cultural response to the difficulties created by the massive scale of ecological phenomena

Work Practices & Information Needsof Ecologists

Special thanks to…

Doctoral committee (Margaret Hedstrom, Chair)

Study participantsScientist friends and colleaguesUSGS Great Lakes Science CenterUM School of InformationUM Rackham Graduate School

To contact me…

Ann Zimmermanasz@umich.edu