1 introduction: context for the week william michener lter network office department of biology...
TRANSCRIPT
![Page 1: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/1.jpg)
1
Introduction: Context for the Week
William MichenerLTER Network Office
Department of BiologyUniversity of New Mexico
![Page 2: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/2.jpg)
2
Ecological InformaticsEcological Informatics
A broad interdisciplinary science A broad interdisciplinary science
thatthat
incorporates bothincorporates both conceptual and conceptual and practicalpractical toolstools
for thefor the
understanding, generation, understanding, generation, processing, and propagationprocessing, and propagation of of ecological data and information.ecological data and information.
![Page 3: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/3.jpg)
3
Ecoinformatics Course Topics
• Introduction and ConceptsIntroduction and Concepts • ActivitiesActivities
– Data documentation (metadata) – Database design and creation
• Ecological and environmental databases• Data models • Querying with SQL
– QA/QC – Website design for dynamic data delivery
• Basic InfrastructureBasic Infrastructure– Hardware/software – Wireless communication – GPS
![Page 4: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/4.jpg)
4
Ecoinformatics• Introduction and ConceptsIntroduction and Concepts – Vanderbilt and
Romanello (Monday)• ActivitiesActivities
– Data documentation (metadata) – Romanello (Monday)
– Database design and creation• Ecological and environmental databases –Porter (Tuesday)• Data models – Porter and McCartney (Tuesday)• Querying with SQL – Porter and McCartney (Wednesday)
– QA/QC – Vanderbilt (Tuesday)– Website design for data delivery – White (Friday)
• Basic InfrastructureBasic Infrastructure– Hardware/software – Murillo (Thursday)– Wireless communication – Vande Castle (Thursday)– GPS – Friggens (Thursday)
![Page 5: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/5.jpg)
5
Ecoinformatics• Introduction and ConceptsIntroduction and Concepts – Vanderbilt and Romanello
(Monday)• ActivitiesActivities
– Data documentation (metadata) – Romanello (Monday)– Database design and creation
• Ecological and environmental databases –Porter (Tuesday)
• Data models – Porter and McCartney (Tuesday)• Implementing a database in MS Access – Vanderbilt &
McCartney (Tuesday)• Querying with SQL – Porter and McCartney (Wednesday)
– QA/QC – Vanderbilt (Tuesday)– Website design for data delivery – White (Friday)
• Basic InfrastructureBasic Infrastructure– Hardware/software – Murillo (Thursday)– Wireless communication – Vande Castle (Thursday)– GPS – Friggens (Thursday)
![Page 6: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/6.jpg)
6
Ecoinformatics• Introduction and ConceptsIntroduction and Concepts – Vanderbilt and
Romanello (Monday)
• ActivitiesActivities– Data documentation (metadata) – Romanello (Monday)– Database design and creation
• Ecological and environmental databases –Porter (Tuesday)• Data models – Porter and McCartney (Tuesday)• Querying with SQL – Porter and McCartney
(Wednesday)– QA/QC – Vanderbilt (Tuesday)– Website design for data delivery – White (Friday)
• Basic InfrastructureBasic Infrastructure– Hardware/software – Murillo (Thursday)– Wireless communication – Vande Castle (Thursday)– GPS – Friggens (Thursday)
![Page 7: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/7.jpg)
7
Ecoinformatics• Introduction and ConceptsIntroduction and Concepts – Vanderbilt and
Romanello (Monday)
• ActivitiesActivities– Data documentation (metadata) – Romanello (Monday)– Database design and creation
• Ecological and environmental databases –Porter (Tuesday)• Data models – Porter and McCartney (Tuesday)• Querying with SQL – Porter and McCartney (Wednesday)
– QA/QC – Vanderbilt (Tuesday)– Website design for data delivery – White (Friday)
• Basic InfrastructureBasic Infrastructure– Hardware/software – Murillo (Thursday)– Wireless communication – Vande Castle (Thursday)– GPS – Friggens (Thursday)
![Page 8: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/8.jpg)
8
Ecoinformatics• Introduction and ConceptsIntroduction and Concepts – Vanderbilt and
Romanello (Monday)
• ActivitiesActivities– Data documentation (metadata) – Romanello (Monday)– Database design and creation
• Ecological and environmental databases –Porter (Tuesday)• Data models – Porter and McCartney (Tuesday)• Querying with SQL – Porter and McCartney (Wednesday)
– QA/QC – Vanderbilt (Tuesday)– Website design for data delivery – White (Friday)
• Basic InfrastructureBasic Infrastructure– Hardware/software – Murillo (Thursday)– Wireless communication – Vande Castle (Thursday)– GPS – Friggens (Thursday)
![Page 9: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/9.jpg)
9
Questions ???
![Page 10: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/10.jpg)
Cyber-Infrastructure Challenges & Ecoinformatics:
An Ecologist’s Perspective
William MichenerLTER Network Office
Department of BiologyUniversity of New Mexico
![Page 11: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/11.jpg)
Today’s Road Map
• Science • Cyber-Infrastructure Challenges• Ecoinformatics
![Page 12: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/12.jpg)
12
Today’s Road Map
• Science • Cyber-Infrastructure Challenges• Ecoinformatics
![Page 13: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/13.jpg)
Most studies use a single Most studies use a single scale of observation --scale of observation --
Commonly 1 mCommonly 1 m22
The literature is biased toward The literature is biased toward single and small scale resultssingle and small scale results
![Page 14: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/14.jpg)
Time (yrs)Time (yrs)
Var
iab
leV
aria
ble
ChangeChange
transition from one stateor condition to another
![Page 15: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/15.jpg)
Space
Space
ParametersParameters
Tim
eTim
e
Thinking Thinking OutsideOutside the “Box” the “Box”
LTERLTER
BiocomplexityBiocomplexity
???? – NEON, CUAHSI, CLEANER, ….???? – NEON, CUAHSI, CLEANER, ….
Increase in breadth and depth of understanding.....Increase in breadth and depth of understanding.....
![Page 16: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/16.jpg)
24 7
11
10
86
5
4
3
2
20
1 918
16
15
17
1413
12
23
22
21
19
LTER26 NSF LTER Sites in the U.S. and the Antarctic: > 1500 Scientists; 6,000+ Data Sets—different themes, methods, units, structure, ….
![Page 17: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/17.jpg)
![Page 18: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/18.jpg)
18
Today’s Road Map
• Science • Cyber-Infrastructure Challenges• Ecoinformatics
![Page 19: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/19.jpg)
19
Knowledge
Data
Information
Phenomena
![Page 20: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/20.jpg)
20
Knowledge
Data
Information
Phenomena
![Page 21: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/21.jpg)
Abstraction of phenomena
Date Site Species Density 10/1/1993 N654 Picea
rubens 13
10/3/1994 N654 Picea rubens
14.5
10/1/1993 N654 Betula papyifera
3
10/31/1993 1 Picea rubens
13.5
10/31/1993 1 Betula papyifera
1.6
11/14/1994 1 Picea rubens
8.4
11/14/1994 1 Betula papyifera
1.8
![Page 22: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/22.jpg)
22
Knowledge
Data
Information
Phenomena
![Page 23: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/23.jpg)
23
YonYon
HitherHither
Hunter-gatherers
![Page 24: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/24.jpg)
24
A Paradigm Shift
Taxon 1Taxon 1
Taxon 2Taxon 2
Taxon 3Taxon 3
Taxon 4Taxon 4
AbioticAbioticfactorsfactors
Integrated, Integrated, InterdisciplinaryInterdisciplinaryDatabasesDatabases
Harvesters
![Page 25: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/25.jpg)
25
Info
rmat
ion
Co
nte
nt
Time
Time of publication
Specific details
General details
Accident
Retirement or career change
Death
(Michener et al. 1997)
Data Entropy
![Page 26: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/26.jpg)
26
Date Site Species Area Count 10/1/1993 N654 PIRU 2 26 10/3/1994 N654 PIRU 2 29 10/1/1993 N654 BEPA 1 3
Date Site picrub betpap 31Oct1993 1 13.5 1.6 14Nov1994 1 8.4 1.8
Date Site Species Density 10/1/1993 N654 Picea
rubens 13
10/3/1994 N654 Picea rubens
14.5
10/1/1993 N654 Betula papyifera
3
10/31/1993 1 Picea rubens
13.5
10/31/1993 1 Betula papyifera
1.6
11/14/1994 1 Picea rubens
8.4
11/14/1994 1 Betula papyifera
1.8
A B
• Schema transform• Coding transform• Taxon Lookup• Semantic transform
Imagine scaling!!
C
Date Site Species Density 10/1/1993 N654 Picea
rubens 13
10/3/1994 N654 Picea rubens
14.5
10/1/1993 N654 Betula papyifera
3
10/31/1993 1 Picea rubens
13.5
10/31/1993 1 Betula papyifera
1.6
11/14/1994 1 Picea rubens
8.4
11/14/1994 1 Betula papyifera
1.8
A B
C
Semantics
![Page 27: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/27.jpg)
27
Semantics—Linking Taxonomic Semantics to Ecological Data
Rhynchospora plumosa s.l.
Elliot 1816
Gray 1834
Kral 1998
Peet 2002?
Chapman1860
R. plumosa
R. plumosa
R. Plumosav. intermedia
R. plumosav. plumosa
R. Plumosav. interrupta
R. plumosa
R. intermedia
R. pineticola
R. plumosav. plumosa
R. plumosav. pinetcola
R. sp. 1
Taxon concepts change over time (and space)Multiple competing concepts coexistNames are re-used for multiple concepts
from R. Peet
Date Species # 1830 R.plumosa 39 1840 R.plumosa 49 1900 R.plumosa 42 1985 R.plumosa 48 1995 R.plumosa 22 2000 R.plumosa 19
A B C0
10
20
30
40
50
60
1/1/00 1/2/00 1/3/00 1/4/00 1/5/00 1/6/00
![Page 28: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/28.jpg)
28
Knowledge
Data
Information
Phenomena
![Page 29: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/29.jpg)
29
Characteristics of Characteristics of Ecological DataEcological Data
Complexity/Metadata RequirementsComplexity/Metadata Requirements
SatelliteImages
DataDataVolumeVolume(per(perdataset)dataset)
LowLow
HighHigh
HighHigh
Soil CoresSoil Cores
PrimaryPrimaryProductivityProductivity
GISGIS
Population DataPopulation Data
BiodiversityBiodiversitySurveysSurveys
Gene Sequences
Business Data
WeatherStations Most EcologicalMost Ecological
DataData
Most Most SoftwareSoftware
![Page 30: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/30.jpg)
30
What Users Really Want…
![Page 31: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/31.jpg)
31
Data Collection
Analysis
Translation
Use By Non-Scientists
Publish For Other Scientists
![Page 32: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/32.jpg)
32
Today’s Road Map
• Science • Cyber-Infrastructure Challenges
• Ecoinformatics
![Page 33: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/33.jpg)
33
A broad interdisciplinary scienceA broad interdisciplinary science
thatthat
incorporates both conceptual and incorporates both conceptual and practical tools practical tools
for thefor the
understanding, generation, understanding, generation, processing, and propagation of processing, and propagation of ecological data and information.ecological data and information.
Ecological InformaticsEcological Informatics
![Page 34: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/34.jpg)
34
Data Designand
Metadata
Data Acquisitionand
Quality Control
Accessand
Archiving
Analysisand
Interpretation
Data Manipulationand
Quality Assurance
ProjectInitiation
Publication
![Page 35: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/35.jpg)
35
Experimental DesignMethods
Data DesignData Forms
Data Entry
Field Computer Entry
ElectronicallyInterfaced Field
EquipmentElectronicallyInterfaced Lab
Equipment
Raw Data File
Quality Assurance Checks
Data Contamination
Data verified?
Data ValidatedArchive Data File
Archival Mass StorageMagnetic Tape / Optical Disk / Printouts
Access Interface
Off-site Storage
Secondary Users
Publication
Synthesis
Investigators
Summary Analyses
Quality Control
Metadata
Research ProgramInvestigators
Studies
yes
no
![Page 36: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/36.jpg)
36
“Ecological Informatics” Activities
• Project / experimental design
• Data design
• Data acquisition
• QA/QC
• Data documentation (metadata)
• Data archival
![Page 37: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/37.jpg)
37
“Ecological Informatics” Activities
• Project / experimental design
• Data design
• Data acquisition
• QA/QC
• Data documentation (metadata)
• Data archival
![Page 38: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/38.jpg)
38
Project / Experimental Design
ExperimentaExperimentallDesignDesign
AnalysesAnalyses
Data / Data / DatabaseDatabaseDesignDesign
![Page 39: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/39.jpg)
39
Project / Experimental Design
• Some Classic References– Green, R.H. 1979. Sampling Design and Statistical Methods for
Environmental Biologists. John Wiley & Sons, Inc., New York.
– Resetarits, Jr., W.J. and J. Bernardo (eds.). 1998. Experimental Ecology. Oxford University Press, New York.
– Scheiner, S.M. and J. Gurevitch (eds.). 1993. Design and Analysis of Ecological Experiments. Chapman & Hall, New York.
– Sokal, R.R. and F.J. Rohlf. 1995. Biometry. W.H. Freeman & Company, New York.
– Underwood, A.J. 1997. Experiments in Ecology: Their Logical Design and Interpretation Using Analysis of Variance. Cambridge University Press, Cambridge, UK.
– Gotelli, N.J. and A.M. Ellison. 2004. A Primer of Ecological Statistics. Sinauer Associates, Sunderland, MA.
![Page 40: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/40.jpg)
40
“Ecological Informatics” Activities
• Project / experimental design
• Data design
• Data acquisition
• QA/QC
• Data documentation (metadata)
• Data archival
![Page 41: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/41.jpg)
41
Data Design
• Conceptualize and implement a logical structure within and among data sets that will facilitate data acquisition, entry, storage, retrieval and manipulation.
![Page 42: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/42.jpg)
42
Data Set Design: Best Practices• Assign descriptive file names• Use consistent and stable file formats• Define the parameters• Use consistent data organization• Perform basic quality assurance• Assign descriptive data set titles• Provide documentation (metadata)
from Cook et al. 2000
![Page 43: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/43.jpg)
43
1. Assign descriptive file names
• File names should be unique and reflect the file contents
• Bad file names– Mydata– 2001_data
• A better file name– Sevilleta_LTER_NM_2001_NPP.asc
• Sevilleta_LTER is the project name• NM is the state abbreviation• 2001 is the calendar year• NPP represents Net Primary Productivity data• asc stands for the file type--ASCII
![Page 44: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/44.jpg)
44
2. Use consistent and stable file formats
• Use ASCII file formats – avoid proprietary formats• Be consistent in formatting
– don’t change or re-arrange columns– include header rows (first row should contain file name, data
set title, author, date, and companion file names)– column headings should describe content of each column,
including one row for parameter names and one for parameter units
– within the ASCII file, delimit fields using commas, pipes (|), tabs, or semicolons (in order of preference)
• Don’t include summary statistics in the data file
![Page 45: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/45.jpg)
45
3. Define the parameters
• Use commonly accepted parameter names that describe the contents (e.g., precip for precipitation)
• Use consistent capitalization (e.g., not temp, Temp, and TEMP in same file)
• Explicitly state units of reported parameters in the data file and the metadata (SI units are recommended)
• Choose a format for each parameter, explain the format in the metadata, and use that format throughout the file– e.g., use yyyymmdd; January 2, 1999 is 19990102
• Use a decimal point or extreme value (-9999) to define missing values
![Page 46: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/46.jpg)
46
4. Use consistent data organization (one good approach)
Station Date Temp Precip
Units YYYYMMDD C mm
HOGI 19961001 12 0
HOGI 19961002 14 3
HOGI 19961003 19 -9999
Note: -9999 is a missing value code for the data set
![Page 47: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/47.jpg)
47
4. Use consistent data organization (a second good approach)
Station Date Parameter
Value Unit
HOGI 19961001 Temp 12 C
HOGI 19961002 Temp 14 C
HOGI 19961001 Precip 0 mm
HOGI 19961002 Precip 3 mm
![Page 48: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/48.jpg)
48
5. Perform basic quality assurance
• Assure that data are delimited and line up in proper columns
• Check that there no missing values for key parameters
• Scan for impossible and anomalous values• Perform and review statistical summaries• Map location data (lat/long) and assess errors• Verify automated data transfers• For manual data transfers, consider double
keying data and comparing 2 data sets
![Page 49: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/49.jpg)
49
6. Assign descriptive data set titles
• Data set titles should ideally describe the type of data, time period, location, and instruments used (e.g., Landsat 7).
• Titles should be restricted to 80 characters.• Data set title should be similar to names of data
files– Good: “Shrub Net Primary Productivity at the Sevilleta
LTER, New Mexico, 2000-2001”– Bad: “Productivity Data”
![Page 50: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/50.jpg)
50
7. Provide documentation (metadata)
• To be discussed in detail
![Page 51: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/51.jpg)
51
Database Types
• File-system based
• Hierarchical
• Relational
• Object-oriented
• Hybrid (e.g., combination of relational and object-oriented schema)
Porter 2000
![Page 52: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/52.jpg)
52
File-system-based Database
Directory
Files
Porter 2000
![Page 53: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/53.jpg)
53
Hierarchical Database
Project
Data sets Investigators
Variables Locations
Codes Methods
Porter 2000
![Page 54: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/54.jpg)
54
Relational Database
Projects
Data setsLocations
Location_idData_id
Location_id
Porter 2000
![Page 55: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/55.jpg)
55
“Ecological Informatics” Activities
• Project / experimental design
• Data design
• Data acquisition
• QA/QC
• Data documentation (metadata)
• Data archival
![Page 56: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/56.jpg)
56
High-quality data depend on:• Proficiency of the data collector(s)• Instrument precision and accuracy• Consistency (e.g., standard methods and
approaches)– Design and ease of data entry
• Sound QA/QC• Comprehensive metadata (e.g.,
documentation of anomalies, etc.)
![Page 57: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/57.jpg)
57
How are data to be acquired?
• Automatic Collection ?
• Tape Recorder
• Data Sheet
• Field entry into hand-held computer
![Page 58: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/58.jpg)
58
“Ecological Informatics” Activities
• Project / experimental design
• Data design
• Data acquisition
• QA/QC
• Data documentation (metadata)
• Data archival
![Page 59: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/59.jpg)
59
Experimental DesignMethods
Data DesignData Forms
Data Entry
Field Computer EntryElectronically
Interfaced FieldEquipment
ElectronicallyInterfaced Lab
Equipment
Raw Data File
Quality Assurance Checks
Data Contamination
Data verified?
Data ValidatedArchive Data File
Archival Mass StorageMagnetic Tape / Optical Disk / Printouts
Access Interface
Off-site Storage
Secondary Users
Publication
Synthesis
Investigators
Summary Analyses
Quality Control
Metadata
Research ProgramInvestigators
Studies
yes
no
Brunt 2000
Generic Data Processing
![Page 60: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/60.jpg)
60
“Ecological Informatics” Activities
• Project / experimental design
• Data design
• Data acquisition
• QA/QC
• Data documentation (metadata)
• Data archival
![Page 61: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/61.jpg)
61
“Ecological Informatics” Activities
• Project / experimental design
• Data design
• Data acquisition
• QA/QC
• Data documentation (metadata)
• Data archival
![Page 62: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/62.jpg)
62
Traditional Fates of Data Post-Publication
• Paper to filing cabinets
• Data to floppy disks or tape
• Data and information lost over time
entropy
![Page 63: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/63.jpg)
63
Data Archive
• A collection of data sets, usually electronic, stored in such a way that a variety of users can locate, acquire, understand and use the data.
• Examples:– ESA’s Ecological Archive– NASA’s DAACs (Distributed Active Archive
Centers)
![Page 64: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/64.jpg)
64Planning
Problem
Analysis and
modeling
Cycles of Research“A Conventional View”
Collection
Publicati
ons Data
![Page 65: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/65.jpg)
65
Cycles of Research“A New View”
PlanningProblem Definition
(Research Objectives)
Analysis and
modeling
Planning
CollectionSelection andextraction
Archive of Data
OriginalObservations
SecondaryObservations
Publicati
ons
![Page 66: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/66.jpg)
66
•Start small and keep it simple – building on simple successes is much easier than failing on large inclusive attempts.
Involve scientists - ecological data management is a scientific endeavor that touches every aspect of the research program. Scientists should be involved in the planning and operation of a data management system.
Support science – data management must be driven by the research and not the other way around, a data management system must produce the products and services that are needed by the community.
Keys to Success
![Page 67: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/67.jpg)
67
Data
Valu
e
Time
SerendipitousDiscovery
Inter-siteSynthesis
Gradual IncreaseIn Data Equity
Methodological Flaws, Instrumentation
Obsolescence
Non-scientific Monitoring
Increasing value of data over time
![Page 68: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/68.jpg)
68
LTER Data Access Policy
1) There are two types of data:
Type I (data that is freely available within 2-3 years) with minimum restrictions and,
Type II (Exceptional data sets that are available only with written permission from the PI/investigator(s)). Implied in this timetable, is the assumption that some data sets require more effort to get on-line and that no "blanket policy" is going to cover all data sets at all sites. However, each site would pursue getting all of their data on-line in the most expedient fashion possible.
2) The number of data sets that are assigned TYPE II status should be rare in occurrence and that the justification for exceptions must be well documented and approved by the lead PI and site data manager. Some examples of Type II data may include: locations of rare or endangered species, data that are covered by copyright laws (e.g. TM and/or SPOT satellite data) or some types of census data involving human subjects.
![Page 69: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/69.jpg)
69
Reasons to Not Share Data:
• Fear of getting scooped• Number of publications will decrease• People will find errors• Someone will misinterpret my data
![Page 70: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/70.jpg)
70
Benefits of Data Sharing• Publicity, accolades, media attention• Renewed or increased funding • Teaching:
• long-term data sets adapted for teaching & texts
• Archival: back-up copy of critical data sets • Research:
• new synthetic studies • peer-reviewed publications
• Document global and regional change• Conservation and resource management:
• species and natural areas protection• new environmental laws
![Page 71: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/71.jpg)
71
Brunt (2000) Ch. 2 in Michener and Brunt (2000)
Porter (2000) Ch. 3 in Michener and Brunt (2000)
Edwards (2000) Ch. 4 in Michener and Brunt (2000)
Michener (2000) Ch. 7 in Michener and Brunt (2000)
Cook, R.B., R.J. Olson, P. Kanciruk, and L.A. Hook. 2000. Best practices for preparing ecological and ground-based data sets to share and archive. (online at http://www.daac.ornl.gov/cgi-bin/MDE/S2K/bestprac.html)
![Page 72: 1 Introduction: Context for the Week William Michener LTER Network Office Department of Biology University of New Mexico](https://reader036.vdocuments.net/reader036/viewer/2022062518/56649e535503460f94b48ceb/html5/thumbnails/72.jpg)
72
Thanks !!!