gcoos dmac meeting orlando, fl (23-24 feb 2009) secoora data management activities: operational...

GCOOS DMAC Meeting Orlando, FL (23-24 Feb 2009)

SECOORAData Management Activities:

Operational Status Report

GCOOS DMAC MeetingOrlando, FL (23-24 February 2009)


SECOORA Organizational Chart

Board of DirectorsStakeholderAdvisory Council

ExecutiveDirector

BoardDevelopmentCommittee

GovernanceCommittee

Finance and Audit

Committee

Public PolicyCommittee

ProgramCoordinator

Info MgmtCoordinator

Spec. ProjectCoordinator

Science Advisory

Committee

DMAC/ODPCommittee

Education/Outreach

Committee

Ops. andMaintenanceCommittee

DMCC

Stakeholder AdvisoryCouncil Liaison


SECOORA’s Regional DM Approach and Roles

GCOOS

MACOORA

NationalBackbone

Coordinating Node

Info Provider Node

Nodes: USF/FWRI UNC-CH

USC

Sectors: Government, Academia, Private

MACOORA and Domain

National and Database

GCOOS/CaRA and Data Synthesis


SECOORA DMCC ParticipantsName Affiliation

Jeff Donovan (FL) USF

Dan Ramage (SC) USC

Melissa Ide (US) NERRS-CDMO

Xiaoyan Qi (NC) UNC-W

Ivelina Rodenbaugh (FL) FL Fish & Wildlife

Jeremy Cothran (SC) USC

Jesse Cleary (NC) UNC-CH

Sara Haines (NC) UNC-CH

Vembu Subramanian (FL) USF

Chris Calloway (NC) UNC-CH

Susanna King (SC) USC

Dwayne Porter (SC) USC and NERRS

Samuel Walker (SC) SECOORA

Madilyn Fletcher* (SC) ODP and USC

Carroll Hood* (CO) Raytheon, Inc.

Name Affiliation

Lynn Leonard (NC) UNC-W

Jennifer Dorton (NC) UNC-W

Virgil Zetterlind (FL) EarthNC, Inc.

Peter Sheng* (FL) University of Florida

Brock Murch (FL) USF/IMaRS

Chuanmin Hu (FL) USF/IMaRS

Harvey Seim (NC) UNC-CH

Steve Browdy (FL) OMS Tech, Inc.

Charlton Galvarino (SC) Second Creek, LLC

Debra Hernandez (SC) SECOORA

Trent Moore (GA) SKIO

Kathleen O’Keife (FL) FL Fish & Wildlife

Scott Cross (US) NCDDC

Tammy Small (US) NERRS-CDMO

Justin Davis (FL) University of Florida

DMAC-ODP Committee Members: Carroll Hood,* Madilyn Fletcher,* Peter Sheng,* Dwayne Porter, Lynn Leonard, Ken Galluppi, Wade Sheldon, Keqi Zhang, Kathleen O’Keife (* Committee Co-Chairs)


DMAC-ODP Committee: Project Oversight

State Funds ($400K) Percentage Awardees

GA $50,000 12.5% SKIO

FL $154,206 38.6% UM, USF-IMaRS, USF-CMS, FWRI

SC $69,528 17.4% USC

NC $125,905 31.5% UNC-CH, UNC-W

Includes both Data Management and Radar allocations (Total: $400K)


Example SECOORA DM/IM Partners


Regional and National Expectations

• Address Property / Data Ownership Issues

• Establish Data Liability Disclaimer

• Ongoing Metrics for NOAA and OMB Reporting

• Long-Term Funding Support

• Maintain Operational Transparency

• Promote Use of Web Site for Activities/Exchanges

– (e.g., don’t send attachments)

• User-Defined Project Identification/Design

• Scalability and Performance Testing (Ongoing Evaluation)

– (e.g., Load Testing and Allocation)

• Increase Support of Modeling and Proposal-Writing Activities


Success Stories for SECOORA DMAC

• Many In-Kind Contributions (FWRI, USC, USF, Sea Grant, etc.)

• Substantial HW/SW Investments and Hubs

• Formal DMAC Design and Implementation Plan

• Collaborative Approach within Southeast and Beyond

• Numerous Legacy Projects to Leverage

• Some Funding Success and Support within RA Leadership

• Actual Data Streams to Manage

• Formal DMAC Structure within Organization:– Board of Directors

• Executive Director

– DMAC-ODP Program Committee

» Data Management Coordinating Committee (DMCC)


DMAC-ODP: Regional Contributions to IOOS

• Database schema, QA/QC protocols, web services, DIF

• Identify regional DMAC activities where efficiency can be gained through direct collaboration or shared resources

• Continued activities relating to national standards and data interoperability

• National Operations Plan Reviews (Waves, DMAC, HF Radar)

• Documentation of existing products/services; Inventories

• Collaboration with other RAs (GCOOS and CaRA)– DM processes– Remote sensing efforts– Training and application development overlap


Data Access, Management, and Integration


DMAC Technical Requirements Process

• Formal request (internal or external) to goes to DMAC-ODP Committee

• Priorities established and then passed to DMCC for technical requirements evaluation

• Joint meeting for reporting of requirements and identification of necessary resources

• Review for IOOS and/or GEOSS Compliance

• Task assignments and timeline established

• Activity executed and documented


DMCC Communication Strategies

Strategy Issue

Face-to-Face Key objectives/vision

Phone/Web Conf. Specific task assignments

TRAC Tasking Project management; daily

SVN Version control; daily

IRC Programmer’s forum; daily

Email General correspondence


Progress and Ongoing Activities


Data Integration Activities – Regional

• DM Partners:– DM Hubs (Strategic Plan; FL/SC/NC)– Remote Sensing Support (IMaRS)

• GCOOS / SECOORA / NASA – All providing support– QA/QC Protocols (SkIO)– User-Defined Applications (UNC-W)– Integration (USF-CMS)– Biological Data Integration (FWRI)– Communication (UNC-CH)– Database (USC)– Inventories (DMCC)

• Sensors/DM Assets/Data Resources

• EXAMPLE: SECOORA and RENCI– Joint evaluation of a SECOORA data model (Xenia)– Strong regional partner w/ substantial resources– Potential to build relationship (modeling/testing)


Data Integration Activities – Xenia DB

Data Portal OpenLayers Mapping Applications

XeniaXenia Relation Database Table Schema Relation Database Table SchemaMain tables used for storing: organization->platform->sensor->observation dataMain tables used for storing: organization->platform->sensor->observation data


Data Integration Activities – netCDF

Being Submitted for National-Level Standards Approval

SECOORA netCDF Convention and Data DictionarySECOORA netCDF Convention and Data Dictionary


Near Real-Time (NRT) Data Feeds

Data Portal OpenLayers Mapping Applications


NRT Data – Interactive Mapping

OpenLayers Mapping Applications


NRT Applications - New Developments

ObsKML-Based Mapping Applications


Static Observational Data Resources

Data Portal Pre-Formatted Files


Archived Data Products/Resources

Existing Archives and “Timed-Out” Data Streams


Operational Components - Inventories

Sensor Resources Inventory

Benefits of Inventories:

• Being reported daily/weekly/annually

• Reporting on Stability/Reliability

• Resource for Proposal Writing

• Resource for Data Requests

• Recommended Manufacturer

• Enables Notification Mechanisms

• Improves Transparency

• Improves Visibility/Marketability

Proposed Inventories:

• Sensors / Platforms

• DM Infrastructure / Software

• Data / Information Resources

• Budgets / Other Resources


Operational Components - Status

On-Line (Beta) Sensor Status Monitoring/Reporting


Operational Components - Hubs

Three DM Hubs:FL:

USF-IMaRS/CMS;FWRI

NC: UNC-CHSC: USC-Columbia

Ideal Instance Described:HardwareSoftwareServicesRedundancyEfficiency


Google Analytics Web and File Access Reports

Status Monitoring/Reporting

On-Line Sensor Status


Example Link to DM Partners: National

Regionally Relevant Observations from NERRS-CDMO


Example Link to DM Partners: Regional

Regionally Relevant Observations from CaroCOOPS


Example User-Defined Applications

• Marine Weather Portal (UNC-W)– Broker/Partner: National Weather Service– http://forecast.weather.gov/mwp

• Harbor View (GLOS)– Broker/Partner: Wisconsin Sea Grant– http://www.glos.us/hview/hview.php

• Hawaii Beach Safety (UHI)– Broker/Partner: HiOOS, HI Tourism Authority, HI Dept. of Health,

Lifeguards– http://oceansafety.soest.hawaii.edu

• Define for each Project: broker, costs, data needs/sources, interface design, key partners, relevant societal goal, feedback from users, outcomes, stakeholders, value.


User-Defined Application: Radar/S&R Integration

Broker: SECOORA

Costs: ?

Data Needs/Sources: Radar coverage; USCG incident locations

Interface Design: OpenLayers

Key Partners: Radar operators/USCG

Relevant Societal Goal: Hazard mitigation and safe marine operations

Feedback Mechanism: ?

Outcomes: Web mapping application; help determine placement of systems

Stakeholders: Radar operators; regional decision makers

Value: Help determine radar placement and promote use of HF radar network

Applications are a critical mechanism to

publicizing the value of IOOS and DMAC


Ocean Data Partnership


Ocean Data Partnership (ODP) Status

• Evolution and Approach (M. Fletcher, ODP Chair)

• Example ODP Opportunities (S. Walker)

• AOML – Water quality; Expanded drifters; RS data

• FWRI – MARVIN and GAME data sets

• FAU – Ocean energy sensors

• GCRC – Water quality measurements

• Long Bay Focus Group – Water quality; model output

• COSEE-SE BOB/FLO Participants – TBD

• IMaRS – Remote Sensing

• NERRS – MET/WQ data and QA/QC protocols


National Review and Status


National RA Review/Status Workshop (Summary)

• DM/IM was a dominant and consistent theme

• User-defined applications are increasing

• Budget conditions remain unstable

• Quantifiable metrics are increasingly important

• Collaborations were often keys to success

• Google Earth and Google Analytics

• NOAA-IOOS DIF Implementation


Remaining Challenges for DMAC Operations

• Addressing Property / Data Ownership Concerns

• Reconciling/Aligning Fractional Time Allocations

• Balancing Regional Realities with National Progress/Expectations

• Acquiring Long-Term Funding

• Maintaining Operational Transparency

• Promote Use of Web Site for Activities/Exchanges

– (e.g., don’t send attachments)

• User-Defined Project Identification/Design

• Scalability and Performance Testing (Ongoing Evaluation)

• Increased Support of Modeling and Proposal-Writing Activities

• Diverse Regional User/Sector Requirements


The Biggest Challenge: “Vembued” (Again)

gcoos dmac meeting orlando, fl (23-24 feb 2009) secoora data management activities: operational...

Documents