the ip3 data archive michael allchin ip3 data & information manager
DESCRIPTION
US National Academy of Sciences: 3 pillars of ‘data husbandry’ 1. Integrity 2. Access 3. Stewardship Principles Kleppner et al Ensuring the Integrity, Accessibility, and Stewardship of Research Data in the Digital Age. National Academy of Sciences. ISBNTRANSCRIPT
![Page 1: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/1.jpg)
The IP3 Data Archive
Michael AllchinIP3 Data & Information Manager
![Page 2: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/2.jpg)
AimTo make IP3 data available to the broader scientific community and
general public, in a permanent legacy archive, as required under
the terms of the funding agreement with CFCAS
![Page 3: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/3.jpg)
US National Academy of Sciences:3 pillars of ‘data husbandry’
1. Integrity2. Access3. Stewardship
Principles
Kleppner et al. 2009. Ensuring the Integrity, Accessibility, and Stewardship of Research Data in the Digital Age. National Academy of Sciences.ISBN 978-0-309-13684-6
![Page 4: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/4.jpg)
TasksObtain
![Page 5: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/5.jpg)
TasksObtain
Understand
![Page 6: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/6.jpg)
Data Organisation:Good
![Page 7: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/7.jpg)
Data Organisation:Less Good
![Page 8: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/8.jpg)
TasksObtain
Understand
Validate
![Page 9: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/9.jpg)
Process- Build continuous series in Excel (mostly manual)
![Page 10: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/10.jpg)
Process- Build continuous series in Excel (mostly manual)
- First-pass programmatic (+Mk1 Eyeball) validation (check date progression, interval consistency, watch for estimation formulae, etc)
![Page 11: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/11.jpg)
Process- Build continuous series in Excel (mostly manual)
- First-pass programmatic (+Mk1 Eyeball) validation
(check date progression, interval consistency, watch for estimation formulae, etc)
- Write to data-model in RDBMS (Access MDB)
![Page 12: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/12.jpg)
Process- Build continuous series in Excel (mostly manual)
- First-pass programmatic (+Mk1 Eyeball) validation
(check date progression, interval consistency, watch for estimation formulae, etc)
- Write to data-model in RDBMS (Access MDB)
- Plot and check for consistency / problem areas: resolve or delete!
![Page 13: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/13.jpg)
![Page 14: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/14.jpg)
![Page 15: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/15.jpg)
![Page 16: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/16.jpg)
![Page 17: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/17.jpg)
![Page 18: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/18.jpg)
![Page 19: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/19.jpg)
TasksObtain
Understand
Validate
Archive
![Page 20: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/20.jpg)
Vital Statistics7 Basins: 44 Stations
![Page 21: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/21.jpg)
Vital Statistics424 Individual Datasets
![Page 22: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/22.jpg)
Vital Statistics29.7 Million Values
![Page 23: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/23.jpg)
Vital StatisticsLiDAR for principal research basins
(~89Gb)
![Page 24: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/24.jpg)
One Major ProblemHow to ensure open-ended public accessibility to large volumes of complex and disparate data and associated information, with no
ongoing budget or staff establishment?
![Page 25: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/25.jpg)
Solution: Part 1Go low-tech
write datasets to simply-formatted text files:make available for download from website
(hosted indefinitely by U.Sask.)
Demo 1
![Page 26: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/26.jpg)
To include…- Principal originator (‘Basin Lead’) and co-authors- General contact details- Official citation- Other funding agencies / contributors / support- Disclaimer- ‘Licensing’ text- Basin / Station details- Instrumentation and contextual information (where available)- Notes - Flag key
![Page 27: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/27.jpg)
Solution: Part 2Implement metadatabase on server
to support basic searches
Demo 2
![Page 28: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/28.jpg)
Other Routes: 1Make full database available for
download as Access MDB(with schema)
![Page 29: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/29.jpg)
Other Routes: 2Partner with WE-Hub
cutting-edge environmental data repository:will host clone of IP3 data archive
![Page 30: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/30.jpg)
LessonsOrganise early: adopting standardised
procedures and protocols for gathering, validating, storing and transmitting data will streamline
generation of high-quality datasets, provide better support for
collaborative research, and enhance credibility / defensibility
![Page 31: The IP3 Data Archive Michael Allchin IP3 Data & Information Manager](https://reader034.vdocuments.net/reader034/viewer/2022042800/5a4d1b677f8b9ab0599b0d54/html5/thumbnails/31.jpg)