wqx web and fifra data larry theller, abe, purdue university leighanne hahn, oisc march 22, 2012
TRANSCRIPT
WQX Web and FIFRA Data
Larry Theller, ABE, Purdue UniversityLeighanne Hahn, OISC
March 22, 2012
WQX and FIFRA Data
• What is WQX• How is it used• Original concept for data upload• Current “state of the art” data loader• Process flow• Data loader upgrades created for this project• Future Tool (2 year NEIEN project)
WQX as Framework
• The Water Quality Exchange (WQX) is a framework that makes it easier for States, Tribes, and others to submit and share water quality monitoring data over the Internet.
• States, Tribes and other organizations can now submit data directly to the publicly-accessible STORET Data Warehouse using the WQX framework.
• WQX will eventually replace the distributed STORET, the STORET Data Entry Module, Reports Module, and STORET Import Module or SIM as the primary means of submitting water quality monitoring data to EPA.
http://www.epa.gov/storet/wqx/products/WQX_factsheet.pdf
CDX
STORET
NODEWEB
Win 2003
Server
Purdue-OISC WQX NODE Network Design
Red Hat Entp
MySQL
ECN 8-Core Red Hat Server
Ubuntu
MySQL
MySQL
Oracle
Authenticated QUERY to CDX
Development (Test) computer
Main Development Server
Interim Production ServerRollover to:
Permanent Production Server
abeip867
abe2k3s2
Templeton.ecn
laptops
Data Loader Test Query Test
Backup of Prod. database
Proposed, Tested, but Not Implemented
Current WQX Web Design
Staging
CDX
Authenticated upload to staging database
Interim Production Server
Spreadsheet of results
Excel
QC done and
Reformat to XML
OISC
Authenticated XML upload to CDX
1
2
WQX_Web_User_Guide_v2.0.pdf
WQX_Web_User_Guide_v2.0.pdf
http://www.epa.gov/storet/wqx/products/WQX_factsheet.pdf
Project Name Monitoring Locations
2011 Activity - Results
2010Activity - Results
2009 Activity - Results
Sample XML for monitoring location upload.
In this typical spreadsheet layout, the measurements are in columns, and each set of samples from a location is a row.
The columns need to pivot so each measurement becomes one row, and the data collection notes need to be replicated for each measurement.
The pink colored tab is used to export data for upload; The yellow tabs to set-up the input data;
The green tabs provide a linked reference for documented elements in WQX.
New Tool under development
• Load and Flow Duration Curve (LDC)– Upload of flow data from USGS– Upload of water quality data from USGS– Upload of water quality data from WQX
Current Grant: WQX data in Action
23
Load and Flow Duration Curves
The End