estimation of accumulated upstream drainage values in braided streams using augmented directed...

Estimation of Accumulated Upstream Estimation of Accumulated Upstream Drainage Values in Braided Streams Drainage Values in Braided Streams Using Augmented Directed GraphsUsing Augmented Directed Graphs

Lawrence Stanislawski, Science Applications International Corporation (SAIC)Lawrence Stanislawski, Science Applications International Corporation (SAIC)Michael Finn, U.S. Geological SurveyMichael Finn, U.S. Geological SurveyMichael Starbuck, U.S. Geological SurveyMichael Starbuck, U.S. Geological SurveyE. Lynn Usery, U.S. Geological SurveyE. Lynn Usery, U.S. Geological SurveyPatrick Turley, U.S. Geological SurveyPatrick Turley, U.S. Geological Survey

2006 AutoCarto Conference, June 25-28. Vancouver, WA

Overview of The National MapOverview of The National Map

National Hydrography Dataset (NHD) – drainage networkNational Hydrography Dataset (NHD) – drainage network

Generalization Process for NHDGeneralization Process for NHD• Pruning of Drainage NetworkPruning of Drainage Network• MethodsMethods

Augmented Directed GraphAugmented Directed GraphBrute Force TracingBrute Force Tracing

• Test ResultsTest Results

SummarySummary


The National MapThe National Map

•Consistent framework for geographic knowledge needed by the Consistent framework for geographic knowledge needed by the Nation Nation •Provides public access to high-quality, geospatial data andProvides public access to high-quality, geospatial data andinformation from multiple partners to help support decision-information from multiple partners to help support decision-making by resource managers and the public. making by resource managers and the public. •Product of a consortium of Federal, State, and local partnersProduct of a consortium of Federal, State, and local partnerswho provide geospatial data to enhance America's ability towho provide geospatial data to enhance America's ability toaccess, integrate, and apply geospatial data at global, national,access, integrate, and apply geospatial data at global, national,and local scales. and local scales. •Includes eight primary data themes:Includes eight primary data themes:

Orthoimagery,Orthoimagery,Elevation,Elevation,Hydrography,Hydrography,Transportation,Transportation,Structures,Structures,Boundaries, Boundaries, Geographic names, Geographic names, Land use/land cover, Land use/land cover,

•Most themes are populated, and content will continue to beMost themes are populated, and content will continue to beadded and updatedadded and updated


The National MapThe National Map

•Created from many data sources and Web Map Services Created from many data sources and Web Map Services (WMS) (WMS)

owned by many organizations.owned by many organizations.•Data sources are electronically stored at various scales and Data sources are electronically stored at various scales and resolutions. Consequently, when data are extracted from several resolutions. Consequently, when data are extracted from several WMS and horizontally integrated in geospatial data applications, WMS and horizontally integrated in geospatial data applications, the differences between data sources become apparent. the differences between data sources become apparent. •The ability to render one or more scales of The ability to render one or more scales of The National MapThe National Map data into an appropriate, or functionally equivalent, data into an appropriate, or functionally equivalent,

representation at a user-specified scale could substantially representation at a user-specified scale could substantially enhance analysis capabilities of these data.enhance analysis capabilities of these data.

Research Topic: development of automated generalization approach that renders a functionally equivalent dataset at a user-specified scale, focusing on the National Hydrography Dataset (NHD) layer of The National Map.


The National MapThe National MapNational Hydrography Dataset (NHD)National Hydrography Dataset (NHD)

National Hydrography Dataset

• Vector data layer of The National Map representing surface waters of the United States.

• Includes a set of surface water reaches

Reach: significant segment of surface water having similar hydrologic characteristics, such as a stretch of river between two confluences, a lake, or a pond.

A unique address, called a reach code, is assigned to each reach, which enables linking of ancillary data to specific features and locations on the NHD.

Reach code from Lower Mississippi subbasin

08010100000413

region-subregion-accounting unit-subbasin-reach number

08010100000696

08010100000413


NHD FeaturesNHD Features

Areal Stream/River

Areal Lake/Pond

Areal Lake/Pond

Linear Streams

Linear Canal/Ditch

Linear Connector

Artificial Paths

The National MapThe National MapNational Hydrography Dataset (NHD)National Hydrography Dataset (NHD)

National Hydrography Dataset

• Divided and distributed at watershed basin and subbasin boundaries.

• Stored in an ArcGIS geographic database (geodatabase) model.

• Three levels of detail (resolutions)

Medium (1:100,000-scale source)

only complete layer

High (1:24,000-scale source)

three-quarters complete

Local (1:12,000 or larger source)Subregions along northern shore of Gulf of Mexico


GeneralizationGeneralization

Develop a generalization strategy that can be implemented on subsets of the NHD.

Strategy should produce a dataset in the NHD model format that maintains:• feature definitions, • reach delineations, • feature relationships, and • flow connections between remaining generalized features.

Extracted dataset should function with NHD applications • less detail • faster processing speed

Extracted level of detail:• user-specified


Development of such a generalization process could eliminate the need to store and maintain all but the highest resolution NHD data layer.

GeneralizationGeneralization

Base data: highest resolution NHD that covers desired area.

Feature pruning – removal of features that are too small for desired output scale.• select a subset of network features• select a subset of area features• remove point features associated with pruned line or area features

Feature simplification• removal of vertices• aggregation, amalgamation, merging, etc.

Focus: network pruning


Generalization: Network Pruning Generalization: Network Pruning ExampleExample

2005 ESRI International User Conference, July 25-29. San Diego, CA.

Gasconade-Osage subregion (1029) falls in the Interior Plains and Interior Highlands physiographic divisions.

Generalization: Network Pruning ExampleGeneralization: Network Pruning Example

Feature pruning – removal of feature that are too small for desired output scale.• select a subset of network features


Pruning test on Gasconade-Osage subregion (1029)Pruning test on Gasconade-Osage subregion (1029)Green: 1:100,000Green: 1:100,000 Blue: 1:500,000Blue: 1:500,000 Red: 1:2,000,000Red: 1:2,000,000

Simple Hydrographic NetworkSimple Hydrographic Network


Braided NetworkBraided Network


Complex Coastal Area Network with CyclesComplex Coastal Area Network with Cycles


Directed GraphsDirected Graphs

Vector Data LayerVector Data Layer• NodesNodes• Directed Edges (Arcs)Directed Edges (Arcs)• Direction represents flowDirection represents flow

Upstream Drainage AreaUpstream Drainage Area

a,(a)

b,(b)

c,(c)

d,(d)

(a+b)

(c+d)

f,(c+d+f)

e,(a+b+e)

(a+b+c+d+e+f)

MethodsMethods

Brute Force Trace – Traces upstream from each node summing up all drainage areas that are encountered.

•Accurate•Slow

Augmented Directed Graph Method – Exploits the Arc Suite polygon data structure within the coverage model, and utilizes a system of tables to arrive at

a faster end result.•Approximately 20x-40x faster•Needs more tests run to discover accuracy


Augmented Directed Graph Augmented Directed Graph Method (One-in-one-out polygon)Method (One-in-one-out polygon)


a,(a)

(a)

(a)

b,(a+b)

c,(c+a) (a+b+a+c-a)

Augmented Directed Graph Augmented Directed Graph Method (One-in-mulitple-out polygon)Method (One-in-mulitple-out polygon)


a,(a)

N1

N2

N3

N4

T1 = Node

Value

N1 a

(a,T1)

b,(a+b,T1)

c,(a+c,T1)

d

e

f(d+e+f,T2)

T2 = Node

Value

N3 d+e+f

T3 = Node

Value

N1N3

ad+e+f

T4 = Node

Value

N1N3

ad+e+f

h

i

(d+e+f+i,T4)

(d+e+f+h,T3)

Augmented Directed Graph Augmented Directed Graph Method (Convergence with tables)Method (Convergence with tables)


z,(a+b+c+z,T1)

y,(a+b+c+y,T2)

T1 = Node Valu

e

N1 a+b+c

T2 = Node Valu

e

N1 a+b+c

N2 ∀ ∀

Node: Sum = Sum – ((Freq – 1) * Value)

T3 =

Freq

Node

Value

2 N1 a+b+c

(a+b+c+z+a+b+c+y,T3)

(a+b+c+z+y,T3)

Node Value

N1 a+b+c

The Cycle ComplexThe Cycle Complex


•Cycles have proven to be the biggest obstacle we have had to overcome

•It is important to notice that all arcs on a cycle will have the same upstream drainage area

•Tables should also be resolved and maintained throughout every edge of the cycle complex in question

•Cycles are the main reason for the fluctuation in speed of the augmented graph method

Future WorkFuture Work


•Convert program to C++•Use ArcObjects to keep the same functionality without reinventing the wheel•ArcObjects allows the programmer to have more control. This will allow the programmer to remove unnecessary steps

•Implement a parallel processing scheme•Begin processing on the sub basin level•This scheme would increase the speed of the process by the number of processors in the cluster with a maximum of the number of sub basins in the United States

Outline of Augmented Directed Graph AlgorithmOutline of Augmented Directed Graph Algorithm


1. Build a polygon to critical node table – table identifies the one-in-one-out and many-out polygons and the associated nodes.

2. For each edge flowing out of a zero in-degree node, set the edge sum equal to the value assigned to the edge, and push the edge sum to the sum on the downstream node.

3. At each convergent node with a complete in-flowing edge sum, process converging tables as needed to reduce the node sum.

4. At each convergent node with a complete in-flowing edge sum, that also closes a one-in-one-out polygon, reduce the node sum by the value assigned to the associated divergent polygon.

5. At each divergent node of a one-in-one-out polygon, assign the node value to the associated one-in-one-out polygon, but reduce the node value by values in the associated node table, if one exists.

6. For each edge flowing out of a node with a completed sum, add the node sum to the edge value and push this value into the sum on the downstream node.

7. Repeat steps 3-6 until all edges have values, or cycles are encountered.

8. If any cycles are found, process each cycle complex separately. For each cycle complex, trace upstream edges from one node in the cycle and assign the sum of the traced edges to one edge in the cycle complex. Repeat steps 3-6 in a special way for all cycle edges.

9. Repeat step 3-8 until all edges have values.

ResultsResults


•Fabricated datasetBrute force same as ADG on all arcs (diff < 0.001 sq km).111 arcs, 1 cycle complexMonotonically Increasing: All tnode values >= edge values.Record FREQUENCY MIN-UPSTR_SQ_KM MAX-UPSTR_SQ_KM 1 111 0.33288 4577.87846

ResultsResults


Pomme De Terre subbasin (10290107) in Gasconade-Osage Region of Missouri (no cycles)Brute force same as ADG on all arcs (diff < 0.001 sq km).Monotonically Increasing: All tnode values >= edge values.Record FREQUENCY MIN-UPSTR_SQ_KM MAX-UPSTR_SQ_KM 1 5580 0.02371 2190.40134

ResultsResults


Lower Potomac. Maryland, Virginia (0207011) (at least 1 cycle)Brute force same as ADG on all arcs (diff < 0.001 sq km).Brute force started ~9 (21:00) pm 06/22/2006 ended 1:47Elapsed time ~3.75 hours (225 minutes) = ~53 edges/minute.Monotonically Increasing: All tnode values >= edge values.Record FREQUENCY MIN-UPSTR_SQ_KM MAX-UPSTR_SQ_KM 1 12039 0.00215 4851.19298


Eastern Louisiana Coastal (3 cycle complexes)ADG less than Brute force by more than 0.001 sq km on two arcsRecord BF_UPSTR_SQKM UPSTR_SQ_KM DIFF 10185 27.5474 27.5747 0.02733 10421 11.3602 11.3875 0.02733Brute force ~ 30 edges/minute).ADG - 65 minutes (~500 edges/minute).Monotonic Increasing: All tnode values >= edge values.Record FREQUENCY MIN-UPSTR_SQ_KM MAX-UPSTR_SQ_KM 1 31149 0.00044 963.73358

SummarySummary

Results demonstrate that the augmented directed graphh algorithm can accurately accumulate values associated with the edges of a graph by accumulating them with the direction of flow. The algorithm has the following properties:

iteratively pushes values downstream from multiple starting points,traversing each graph feature once (except when cycles exist),

tracks duplicate values on divergent edges and removes the duplicates at convergent nodes,

provides monotonically increasing (non-decreasing) values with downstream location on the graph,and

Identifies and handles cycle subsystems in the graph.

From a data processing perspective, the ADG approach appears to be a substantial improvement over existing graph traversal techniques that provide less precise proportioning alternatives for handling duplication at divergences.


SummarySummary

The ADG approach is tailored for data that is properly oriented in the directed graph data structure. Inaccurate results or failed processing are likely when edge features are oriented improperly. Additional testing will be completed to validate this procedure and identify any limitations.

In the future, we will use this approach to generate values for the subsequent pruning process in our generalization strategy for the NHD layer of The National Map. Upstream drainage areas from the ADG algorithm may also be used to estimate stream flow volumes on network features. Alternative uses for ADG processing of network geospatial data are likely to be identified.


ReferencesReferences


Cormen, T., Leiserson, C., Rivest, R., and Stein, C. 2003. Introduction to Algorithms, 2nd Edition. The MIT Press, Cambridge, Ma.

Manber, U. 1989. Introduction to Algorithms: A Creative Approach. Addison-Wesley Publishing Co.

McCracken, D., and W. Salmon. 1987. A second course in computer science with Modula-2. New York, New York: John Wiley and Sons.

McMaster, R., and K. Shea. 1992. Generalization in digital cartography. Association of American Geographers. Washington, D.C.

U.S. Geological Survey. 2000. The National Hydrography Dataset: Concepts and Contents. Accessed May 5, 2006, at URL http://nhd.usgs.gov/chapter1/index.html

http://nhd.usgs.gov/chapter1/index.html

Questions?Questions?

Estimation of Accumulated Upstream Drainage Estimation of Accumulated Upstream Drainage Values in Braided Streams Using Augmented Values in Braided Streams Using Augmented Directed GraphsDirected Graphs

Lawrence Stanislawski, Lawrence Stanislawski, [email protected]@usgs.govMichael Starbuck, U.S. Geological SurveyMichael Starbuck, U.S. Geological SurveyMichael Finn, U.S. Geological SurveyMichael Finn, U.S. Geological SurveyE. Lynn Usery, U.S. Geological SurveyE. Lynn Usery, U.S. Geological SurveyPatrick Turley, U.S. Geological SurveyPatrick Turley, U.S. Geological Survey


estimation of accumulated upstream drainage values in braided streams using augmented directed...

Documents

national map data

data sources

ancillary data

geospatial data applications

geospatial data andinformation

primary data themes

wathe national mapcreated

equivalent dataset