LTER 1990-1999LTER 1990-1999Moving from Data to Information Moving from Data to Information

ManagementManagement

John PorterJohn Porter

The World in 1989The World in 1989

All data sharing between LTER sites is on All data sharing between LTER sites is on a case-by-case basis – no policiesa case-by-case basis – no policiesSite data catalogs are Site data catalogs are notnot available availableMost researchers don’t use electronic mailMost researchers don’t use electronic mail– Many mail systems won’t talk to one anotherMany mail systems won’t talk to one another

An “online system” is a “bulletin board” An “online system” is a “bulletin board” type system that requires a modemtype system that requires a modemThere are no standards for metadata There are no standards for metadata content or structurecontent or structure

The World in 1999The World in 1999

LTER sites share most data within 2 years LTER sites share most data within 2 years of collectionof collectionThere is a centralized system for searching There is a centralized system for searching for LTER Datasetsfor LTER DatasetsLTER Climate, Personnel and Bibliographic LTER Climate, Personnel and Bibliographic Databases are fully operationalDatabases are fully operationalElectronic communication (e.g., email) is Electronic communication (e.g., email) is ubiquitousubiquitousThere are published standards for Metadata There are published standards for Metadata contentcontent

How did we get there?How did we get there?

A timeline for progressA timeline for progress

1990 1990 (Snowbird)(Snowbird)

““LTER Core Dataset Catalog” publishedLTER Core Dataset Catalog” published““Guidelines for Site Data Policies” produced Guidelines for Site Data Policies” produced at 1990 DM and All-Scientists’ Meeting at 1990 DM and All-Scientists’ Meeting (“Proprietary Issues”)(“Proprietary Issues”)

LTER Email forwarding system activeLTER Email forwarding system active – (e.g., (e.g., jhp7e@virginia.edujhp7e@virginia.edu -> -> jporter@lternet.edujporter@lternet.edu ) )

– Links different mail systems (e.g. forest service)Links different mail systems (e.g. forest service)

LTER “Connectivity” report submitted to NSFLTER “Connectivity” report submitted to NSF

1991 1991 (San Antonio)(San Antonio)

Working group Working group discussed “Interactive discussed “Interactive Data Access” Data Access” systemssystems

19921992 (Honolulu)(Honolulu)

Mention of “Gopher” online systems at DM Mention of “Gopher” online systems at DM meetingmeetingAll-Site Bibliography proposed (but not All-Site Bibliography proposed (but not funded)funded)10-year plan developed10-year plan developedDemonstration of structured metadata Demonstration of structured metadata using “Attribute-Value” syntaxusing “Attribute-Value” syntax– Started the chain of development on metadata Started the chain of development on metadata

to support automated processingto support automated processing

19921992

The idea of using a common format for metadata exchange was an important outgrowth from the 1992 meeting that ultimately led to EML 2.0 a decade later.

A test shell-script at the VCR/LTER automatically generated a SPSS job based on metadata in the “attribute-value format” from the CPR (now SGS) LTER

1993 1993 (Madison)(Madison)

Symposium “Environmental Information Symposium “Environmental Information Management and Analysis: Ecosystem to Management and Analysis: Ecosystem to Biosphere Scales”, May 20-22, 1993, in Biosphere Scales”, May 20-22, 1993, in Albuquerque – Led to 1994 bookAlbuquerque – Led to 1994 book

Prototype All-Site Bibliography database Prototype All-Site Bibliography database developeddeveloped

8 sites had online Gopher/WAIS systems8 sites had online Gopher/WAIS systems

““Site Review Tips”Site Review Tips”– precursor to site review guidelinesprecursor to site review guidelines

Oct. 1993 – initial distribution of WWW “kit” for Oct. 1993 – initial distribution of WWW “kit” for LTER sitesLTER sites

1994 1994 (Seattle)(Seattle)

Use of WWW servers at LTER sites grows Use of WWW servers at LTER sites grows dramatically dramatically – (first use of URL’s in meeting report)(first use of URL’s in meeting report)

Expanded LTER DM Meeting – bought in other Expanded LTER DM Meeting – bought in other groups for joint meetinggroups for joint meetingLTER Coordinating Committee mandates that LTER Coordinating Committee mandates that each site share at least one dataset onlineeach site share at least one dataset online ““LTER Data & Information System” (later LTER Data & Information System” (later became “Network Information System)” became “Network Information System)” proposedproposedLTER Minimum Metadata Content Standard LTER Minimum Metadata Content Standard formalizedformalized

1994 - Metadata Content 1994 - Metadata Content

1995 1995 (Snowbird)(Snowbird)

Strategic Vision StatementStrategic Vision Statement– First use of “Information Management” instead of “Data First use of “Information Management” instead of “Data

Management”Management”

Continued design of “NIS”Continued design of “NIS”– ““Dream” Information systemDream” Information system

Continued to formalize governance – “Data Continued to formalize governance – “Data Management Task Force” precursor to IMEXECManagement Task Force” precursor to IMEXEC– First electionFirst election

Most LTER sites now have online serversMost LTER sites now have online servers

DataBits starts publication hiatus DataBits starts publication hiatus

1996 – Archibald Biological Sta.1996 – Archibald Biological Sta.

http://www.vcrlter.virginia.edu/nis/im97/rogues.html

1996 1996 (Lake Placid, FL)(Lake Placid, FL)

X-roots Climate Data workshopX-roots Climate Data workshop – May 1996 (seed for CLIMDB)May 1996 (seed for CLIMDB)– V1.0 ClimDB data exchange format developedV1.0 ClimDB data exchange format developed

Mandate to share all data within 2-years of Mandate to share all data within 2-years of collectioncollection

Strategy developed for Data-Table-of-Contents Strategy developed for Data-Table-of-Contents (DTOC) data catalog(DTOC) data catalog

Survey of Site Software & HardwareSurvey of Site Software & Hardware

Participation in Eco-Informa ’96 conferenceParticipation in Eco-Informa ’96 conference

LNO move from Seattle startsLNO move from Seattle starts

1997 1997 (Albuquerque)(Albuquerque)

FLED committee produces Ecological FLED committee produces Ecological Metadata paper in Ecological ApplicationsMetadata paper in Ecological Applications

Prototype CLIMDB & DTOC operationalPrototype CLIMDB & DTOC operational

Revision of LTER Network Data PolicyRevision of LTER Network Data Policy

DIMES workshopDIMES workshop

Data & Information Management for Data & Information Management for the Ecological Sciences (DIMES) the Ecological Sciences (DIMES) workshop – 1997 – Albuquerque, NMworkshop – 1997 – Albuquerque, NMhttp://ecoinformatics.org/pubs/guide/frame.htmhttp://ecoinformatics.org/pubs/guide/frame.htm

Also was Also was important important precursor precursor to 2000 to 2000 bookbook

1998 - Baltimore1998 - Baltimore

1998 1998 (Baltimore)(Baltimore)

LNO completes move from SeattleLNO completes move from Seattle– New Sheriff in TownNew Sheriff in Town

Presentation on XMLPresentation on XML

19981998Operational:•Personnel•DTOC•CLIMDB

Revision:•Bibliography

Prototype:•SiteDB

Planned:•NitrogenDB

1999 1999 (Spokane)(Spokane)

Knowledge Networking for Biodiversity Knowledge Networking for Biodiversity Project fundedProject funded

SiteDB and ClimDB move to LNOSiteDB and ClimDB move to LNO

DataBits publication resumesDataBits publication resumes

EML 1.4 standard publishedEML 1.4 standard published

Active Metadata working groupsActive Metadata working groups

Lessons LearnedLessons Learned

““If at first you don’t succeed, try, try again”If at first you don’t succeed, try, try again”– Initial 1991 Climate Database never really got Initial 1991 Climate Database never really got

off the ground, but subsequent efforts have off the ground, but subsequent efforts have been highly successfulbeen highly successful

– SiteDB had several different homesSiteDB had several different homes

““Many hands make light work” Many hands make light work” – CLIMDB – X-roots -> Climate Committee -> CLIMDB – X-roots -> Climate Committee ->

NTL -> LNO -> ANDNTL -> LNO -> AND– SiteDB – PAL/BNZ -> LNOSiteDB – PAL/BNZ -> LNO

THANKS: People that left the THANKS: People that left the LTER Network 1990-1999LTER Network 1990-1999

Bill Michener – Core Datasets Catalog, Bill Michener – Core Datasets Catalog, DIMESDIMESRudolf Nottrott – Email forwarding system, Rudolf Nottrott – Email forwarding system, LTER Personnel Database, Core Datasets LTER Personnel Database, Core Datasets CatalogCatalogCaroline Bledsoe – All-Site Bibliography v Caroline Bledsoe – All-Site Bibliography v 1.0, X-Roots1.0, X-RootsHarvey Chin – All-Site Bibliography v.1.0, Harvey Chin – All-Site Bibliography v.1.0, setup kits for WWW serverssetup kits for WWW serversTom Kirchner – Attribute-Value formatTom Kirchner – Attribute-Value formatAnd MANY moreAnd MANY more

THANKS – 8/12/1999THANKS – 8/12/1999