nhd update process improvement project u.s. department of the interior u.s. geological survey kevin...
TRANSCRIPT
![Page 1: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/1.jpg)
NHD Update Process Improvement Project
U.S. Department of the Interior
U.S. Geological Survey
Kevin McNinch, Dave Hughes, Paul Kimsey
![Page 2: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/2.jpg)
Current NHD Tools
• NHD Geo Edit v3.3.2– Official release 04/18/2010– ArcGIS 9.3 sp1 & ArcGIS 9.3.1– NHD Model 1.06
![Page 3: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/3.jpg)
Current NHD Tools
• NHD Geo Edit v3.3.3– Official release planned 06/07/2010– ArcGIS 9.3 sp1 & ArcGIS 9.3.1– Compatible with NHD Model 1.07
• Permanent Identifiers & ComID (retired in 1 year)• External Crosswalk table
– Backward compatibility to Model 1.06– Compatible with NHD Model 2.0
• WBD replaces old 1:250k hydrologic units
![Page 4: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/4.jpg)
NHD Update ProcessImprovements
WHY WHAT HOW BENEFITS MID TERM LONG TERM
Short Term 0-1 Years
![Page 5: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/5.jpg)
• WHY do we need to do this in the short term?
NHD Update ProcessImprovements
![Page 6: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/6.jpg)
What do the Stewards Say:
1. Maintenance is too difficult.2. Inconsistent results after editing.3. Need “direct” editing of the NHD data.4. Quick and simple edits need to be easy.5. Software installation and performance is an
issue.
![Page 7: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/7.jpg)
0
Status of Stewardship Today(Sub basins)
![Page 8: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/8.jpg)
The Net Result
After five years of stewardship, the program is not as advanced as it should be.
![Page 9: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/9.jpg)
• WHAT do we need to do to streamline the current process?
NHD Update ProcessImprovements
![Page 10: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/10.jpg)
NHD Update Process Improvement Project
• Charter– 02/01/2010
• Survey– Distributed to all partners– Compiled results from 23 respondents
• Requirements Document– Draft April, 2010
![Page 11: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/11.jpg)
Process Enhancements, Requirements Specifications:
• Ranked in order of High to Low:
• Refactor code for NHD tools and utilities
• Single Installation Package for all NHD Update Process Tools and Utilities
• Merge all current NHD toolbars into one toolbar with logical progression of tasks
![Page 12: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/12.jpg)
Requirements Specifications:
• Utilize ArcSDE Versioning to manage NHD Update Process data transactions
• Allow editing with NHD GeoEdit Tool in geographic coordinate system
• Create queue for QC checks
• Create queued edit tool
![Page 13: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/13.jpg)
Requirements Specifications:
• Make all NHD Update Process tools compatible with ESRI Personal Geodatabases (PGDB) and ESRI File Geodatabases (FGDB)
• Refactor NHD Update Process tools so that they are not reliant on ESRI Task Assistant
![Page 14: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/14.jpg)
Requirements Specifications:
• Incorporate NHD Plus build and refresh QC checks to support NHD Plus creation
• Update process workflow to use ESRI ArcGIS Job Tracking Extension (JTX)
• Improve Metadata Process
![Page 15: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/15.jpg)
Requirements Specifications:
• Reach Code Allocation utility process updates
• Bug Fixes and enhancements from NHD development group list
• Update tools and installation package to work with ArcGIS 10.0 when released
![Page 16: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/16.jpg)
NHD Update Process
• Current process (29 steps)
![Page 17: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/17.jpg)
NHD Stewardship Web site
Extract from database NHD75464.zip
UnZip
1804_orig1804_orig.mdbSave zip file in thisfolder
1804_alb1804_alb.mdbAll edits performed on this mdb
1804_qc1804_qc1.mdbCopy of NHD75464.mdb with transactions applied
1804_geo1804_geo.mdb1804_alb.mdb reprojected to geographic 1804_geo_load
1804_geo_load.mdbUsed for build flow, releveler and final XML extract
Reproject
Reproject
XML Extract
XML2pgdb
Yes
No
Make copy of final 1804_geo.mdb and name it1804_geo_load.mdb
Flow Diagram for Current NHD Maintenance Process
Corrections ?
![Page 18: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/18.jpg)
NHD Stewardship Web site
Extract from database NHD75464.zip
UnZip
1804_orig1804_orig.mdbSave zip file in thisfolder
1804_alb1804_alb.mdbAll edits performed on this mdb
1804_qc1804_qc1.mdbCopy of NHD75464.mdb with transactions applied
1804_geo1804_geo.mdb1804_alb.mdb reprojected to geographic 1804_geo_load
1804_geo_load.mdbUsed for build flow, releveler and final XML extract
Reproject
Reproject
XML Extract
XML2pgdb
Yes
No
Make copy of final 1804_geo.mdb and name it1804_geo_load.mdb
Steps that can be eliminated from the current NHD Maintenance Process
Corrections ?
![Page 19: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/19.jpg)
National Map
Replica Checkout
Flow Diagram for NHD Maintenance Process after Improvements
Replica Checkin
Run QC suite against entire
dataset
Resolve QC errors(Queued Edit tool)
Update using NHDGeoEdit tool
Run QC suite against entire dataset
Resolve QC errors(Queued Edit tool)
![Page 20: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/20.jpg)
• HOW are we going to accomplish this?
NHD Update ProcessImprovements
![Page 21: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/21.jpg)
• Agile Project Management - SCRUM• Requirements are captured as items in a list of
“product backlog”• Iterative development in a series of four-week
“sprints”• Product is designed, coded, and tested during
the sprint• Responsive to changing needs
NHD Update ProcessImprovements
![Page 22: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/22.jpg)
Product Noise Level
Simple
ComplexAnarchy
Complicated
Technology
Re
qu
ire
me
nts
Far fromAgreement
Close toAgreement
Clo
se t
oC
erta
inty
Far
fro
mC
erta
inty
Source: Strategic Management and Organizational Dynamics by Ralph Stacey in Agile Software Development with Scrum by Ken Schwaber and Mike Beedle.
![Page 23: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/23.jpg)
Putting it all together
Image available at www.mountaingoatsoftware.com/scrum
![Page 24: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/24.jpg)
No changes during a sprint
• Plan sprint durations around how long you can commit to keeping change out of the sprint
Change
![Page 25: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/25.jpg)
Scrum framework
•Product owner•ScrumMaster•Team
Roles
•Sprint planning•Sprint review•Daily scrum meeting
Ceremonies
•Product backlog•Sprint backlog•Burndown charts
Artifacts
![Page 26: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/26.jpg)
• Benefits
• Agile development• Frequent delivery• Improved Communication
• Pro-active project management
NHD Update ProcessImprovements
![Page 27: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/27.jpg)
• BENEFITS to the stewardship community
NHD Update ProcessImprovements
- Level of complexity allows for a much broader audience from “expert” to “knowledgeable users”
- Reduces training time - Reduces need for constant technical support
- Eliminates re-training
![Page 28: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/28.jpg)
Where Does NHD Stewardship Go From Here?
• Looking beyond lifecycle of current NHD GeoEdit Tools and NHD Stewardship Processes– Current development will be as reusable as possible
• Begin planning and requirements gathering—create project charter to reflect this work– Completed NHD Architecture Planning Meeting May 13,
2010• Analysis of current process (On-going)• Create NHD Update Process that:
– Addresses a broader spectrum of NHD users– Simplifies
![Page 29: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/29.jpg)
NHD User Base
Huge number of Casual Users
ManyKnowledgeable Users
SomeExperts
![Page 30: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/30.jpg)
NHD Stewardship
Addressed Very Wellwith Edit Tool Complete Edit Capability
Poorly AddressedNo Intermediate ToolBasic Editing
Not Yet AddressedWeb Edit ToolIdentify and Markup EditsIn development
Experts
Casual Users
Knowledgeable Users
![Page 31: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/31.jpg)
• MID TERM 2-3 Years
NHD Update Process
• Web-Based Markup/Editing Tools– Alabama Web Edit Tool(WET)– Requirements gathering group ongoing for
national web markup/editing tool• Detailed requirements gathering and
analysis• Improve web services
![Page 32: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/32.jpg)
• LONG TERM 4+ Years
NHD Update Process
• Evaluating other technologies, trends, and tools for future of Vector Data Stewardship– Open source or COTS tools– Open data standards– Geosyncronization– Volunteered Geographic Information (VGI)
![Page 33: NHD Update Process Improvement Project U.S. Department of the Interior U.S. Geological Survey Kevin McNinch, Dave Hughes, Paul Kimsey](https://reader036.vdocuments.net/reader036/viewer/2022062515/56649c755503460f94928a77/html5/thumbnails/33.jpg)