Making Data discoverable & accessible

Introducing Dept of Parks & Wildlife’s Marine Science information management infrastructure

Florian Mayer, Dept Parks and WildlifePresentation to ANDS round table 18 March 2014

Datadiscoverable & accessible

Informationfrom reproducible,

automated analyses

Knowledgedefensible & transparent

Wisdomto inform policy

Biodiversity Conservation “Conserve, protect, manage native fauna & flora based on best practice science”

Dept Parks & Wildlife Strategic Directions 2013-2014

Research & Monitoring

outcome-focused

The challengeData managementData classification and sensitivityDigital information securityDiscoverabilityAccessibilityComplianceCorporate culture and paradigm shiftIT infrastructure and architectureGovernment agency woes – funding, agency restructure, locked out of academia

Live demoDiscover data- By location- By title- By keyword- By theme

Preview data- Embedded preview- Graphs, Maps

Access data- Download- Programs

The solutionData & metadata catalogueAs used by 200+ govs&NGOs4 years of my workMarines & OIM collaborationWorks for MarinesAvailable to Division & Dept

Code https://bitbucket.org/dpaw/Data catalog - Workflow automation - Stats work bench

Office for Information Management

Scope Department-wide

Mission To enable developers like me to deliver products like this data catalogue

Marine Science Information Management

Scope from Marine Science up to Science & Conservation Division

Environment OIM’s department-wide infrastructure and policies

Mission To deliver information management to Marine Science (serving as template for others)

Datadiscoverable & accessible

Informationfrom reproducible,

automated analyses

Knowledgedefensible & transparent

Wisdomto inform policy

Research & Monitoring

outcome-focused

Biodiversity Conservation“Conserve, protect, manage native fauna & flora based on best practice science” Dept Parks &

Wildlife Strategic Directions 2013-2014

Datadiscoverable & accessible

Informationfrom reproducible,

automated analyses

Knowledgedefensible & transparent

Wisdomto inform policy

Research & Monitoring

outcome-focused

Data APICode repository

R code as web app http://

Source code

Biodiversity Conservation“Conserve, protect, manage native fauna & flora based on best practice science” Dept Parks &

Wildlife Strategic Directions 2013-2014

Datadiscoverable & accessible

Informationfrom reproducible,

automated analyses

Knowledgedefensible & transparent

Wisdomto inform policy

Research & Monitoring

outcome-focused

Code repository

Data + Code + Markup = PDF

Reproducible report

Data API

Sweave

Source code

Collaboration

Biodiversity Conservation“Conserve, protect, manage native fauna & flora based on best practice science” Dept Parks &

Wildlife Strategic Directions 2013-2014

Simulate then observe (rinse&repeat)

Datadiscoverable & accessible

Informationfrom reproducible,

automated analyses

Knowledgedefensible & transparent

Wisdomto inform policy

Research & Monitoring

outcome-focused

Reproducible research

Applying software engineering best-practice to scientific

research

Mayer et al. 2010

Biodiversity Conservation“Conserve, protect, manage native fauna & flora based on best practice science” Dept Parks &

Wildlife Strategic Directions 2013-2014

Internal workings of CKAN @ DPaW

Ubuntu 12.04 LTS VM

~/projects/dpaw_docker/ckan

dpaw_docker code repo

First build creates docker image, copies some CKAN files from image to local file system, creates startup scripts and

persistent files

Modify settings and page templates

Second build overlays modifications into image

/srv/dpaw/ckan/

Docker image scripts and persistent files

Startup container: /srv/dpaw/ckan/startup.sh

Run shell in container: /srv/dpaw/ckan/shell.sh

Database is persistent in /srv/dpaw/ckan/var/lib

http

backup

rsync

clone

Your custom CKAN docker image