1 1 w. fredrick limp university professor robert harris dba center for advanced spatial technologies...
TRANSCRIPT
11
W. Fredrick LimpUniversity Professor
Robert HarrisDBA
Center for Advanced Spatial TechnologiesUniversity of Arkansas
Spatially Enabling the Spatially Enabling the DBMSDBMS
Design, implementation, Design, implementation, operation and implications for operation and implications for
the futurethe future
22
Outline Review of geospatial data structures and Review of geospatial data structures and
their relationship to “traditional” DBMS their relationship to “traditional” DBMS data typesdata types
Feature and feature-class based spatial Feature and feature-class based spatial data modelsdata models
Data storage and indexing in enterprise Data storage and indexing in enterprise geospatial systemsgeospatial systems
OpenGIS and enterprise systemsOpenGIS and enterprise systems SQL operations in 10GSQL operations in 10G Examples of on-line systemsExamples of on-line systems
33
Initial spatial data model
Paradigm was a mapParadigm was a map Map layer theme fileMap layer theme file Data model corresponded to Data model corresponded to
technology drivertechnology driver Developed in response to “slow” Developed in response to “slow”
display hardware/software display hardware/software performance and need to be performance and need to be responsiveresponsive
44
The attributed graphics model
Graphic files were Graphic files were linked to attributes in linked to attributes in a separate file and a separate file and later DBMS tablelater DBMS table
Allowed user to Allowed user to conduct searches on conduct searches on and get graphic and get graphic element(s) that was element(s) that was associatedassociated
Graphic Info element 1 Road 2 water line 3 valve
1
2
55
Replaced by - blob-based entity spatial model
Software developers take advantage of a Software developers take advantage of a special type of record in database - the special type of record in database - the blobblob– a blob is a chunk of disk space that the DBMS a blob is a chunk of disk space that the DBMS
keeps track of but otherwise just knows keeps track of but otherwise just knows “something is there”“something is there”
Changes view of basic “thing” recordedChanges view of basic “thing” recorded Spatial entities become the “element” Spatial entities become the “element”
– a particular road segment, a particular parcela particular road segment, a particular parcel
66
Database storage in this model
Lot 2
Lot 1
Tract layer
Tract 1 population an attribute Geometric stuff another attribute
94102
94103
234
567
100
100
blob
blob
Columbia
Jefferson
All geometric stuff in a “blob”
77
BLOB problems (1)
BLOB contents are not managed BLOB contents are not managed by DBMSby DBMS– BLOB contents are unknown to DBMSBLOB contents are unknown to DBMS– Managed by 3-rd party softwareManaged by 3-rd party software– All operations external to DBMSAll operations external to DBMS
BLOBs do not reflect known data BLOBs do not reflect known data typestypes
88
BLOB problems (2) Definition of a thingDefinition of a thing
– This is now clear within a single This is now clear within a single themetheme
– but still problem in confusion between but still problem in confusion between themesthemes
District 1
The City
Smith RoadWhat’s this ???
What happens if the road is moved?
99
Theme and element confusion
What if the “road” changes - and it is also What if the “road” changes - and it is also the legal definition of the city limit and the legal definition of the city limit and voting district boundary?voting district boundary?
If the road is an entity then perhaps the If the road is an entity then perhaps the road department updates itroad department updates it– then the city limit and voting district then the city limit and voting district
boundaries are now out-of-dateboundaries are now out-of-date BUT maybe that’s correct - the road BUT maybe that’s correct - the road
changes but the district boundary doesn’t!changes but the district boundary doesn’t!
1010
Spatial operations in this model
How do you find a geographic thingHow do you find a geographic thing– find all old street segments within Alderman find all old street segments within Alderman
Smith’s districtSmith’s district old is an attributeold is an attribute ““district” is a geographic areadistrict” is a geographic area
““IN”, “BESIDE” etc. are viewed as IN”, “BESIDE” etc. are viewed as topological relationshipstopological relationships
Requires specialized “GIS” operations Requires specialized “GIS” operations and possibly specialized data structureand possibly specialized data structure
DBMS simply the data storeDBMS simply the data store– All operations in client or middle-wareAll operations in client or middle-ware
1111
Enterprise systems and interoperability
Spatial data has (to date) been Spatial data has (to date) been stored in many different internal stored in many different internal formatsformats
Common that each software vendor Common that each software vendor has a different spatial data formathas a different spatial data format
Different formats made exchange Different formats made exchange of data difficult (if not impossible)of data difficult (if not impossible)
How to solve this in an enterprise How to solve this in an enterprise context?context?
1212
Solutions Old solutionOld solution
– Everybody got the same software from the same Everybody got the same software from the same vendor vendor OROR
– ORGANIZATIONAL RULE was passed ORGANIZATIONAL RULE was passed – Problems with thisProblems with this
one size doesn't fit allone size doesn't fit all limits technological innovationlimits technological innovation
New solutionNew solution– Open GIS Open GIS (TM) (TM) consortium and interoperability consortium and interoperability
specificationsspecifications– Multiple vendors = competitionMultiple vendors = competition– Lower cost and increased capacityLower cost and increased capacity– Flexibility by unitsFlexibility by units
1313
Definition of OpenGIS®
Open and interoperable geoprocessing:
The ability to share heterogeneous geodata and geoprocessing resources transparently in a network environment.
1414
OpenGIS Specification is an Interface Standard (not a data or transfer)
standard
• OpenGIS-compliant clients can be understood by OpenGIS-compliant servers:– Set or get geodetic datum and projection– Create, modify, copy, delete features in the
database– Select features from the database by query– Execute GIS commands– And many others...
– Adopted by ISO (TC211)
1515
Simple features model
With development led, in large With development led, in large part, by DBMS vendors, a new part, by DBMS vendors, a new spatial data model is createdspatial data model is created
Requires some new algorithms Requires some new algorithms But otherwise can utilize “normal” But otherwise can utilize “normal”
relational and/or object DBMSrelational and/or object DBMS
1616
Basic Data ModelBasic Data Model
P o in t L in e P o lyg on
E le m e n ts
G e o m e tries
S p a tia l L a yer
P o in t L in e P o lyg on
E le m e n ts
G e o m e tries
S p a tia l L a yer
1717
Base Elements
Basic building blocks of a Basic building blocks of a geometrygeometry
Element typesElement types– Point (1)Point (1)– Line (2)Line (2)– Polygon (3)Polygon (3)
Constructed using Constructed using coordinates coordinates
No limit on number of No limit on number of coordinatescoordinates
1818
Extended Elements
PointsLine/Circular Arc StringsPolygonsRectangle optimized for spaceCirclesCompound elements without
areaCompound elements with area
1919
Geometry
Represents a Represents a spatial featurespatial feature
Consists of an Consists of an ordered set of ordered set of primitive elementsprimitive elements
Uniquely identified Uniquely identified with a geometry with a geometry identifieridentifier
Geometry 1Florida, USA
Geometry 2Hawaii, USA
2020
Layers
Consists of Consists of geometries that geometries that share a common share a common set of attributesset of attributes
Stored using a set Stored using a set of tablesof tables
State layer
2121
Conversion from map tile to feature - element in ORDBMS
Map tile by “theme” Enterprise system’s feature classes
2222
Database properties Allows storage of individual elementAllows storage of individual element
– each segment of road is potentially accessibleeach segment of road is potentially accessible– a geometric element serve multiple purposesa geometric element serve multiple purposes
road segmentroad segment city limit segmentcity limit segment voting district segmentvoting district segment
““Automatic” updates of all associated Automatic” updates of all associated “themes” because they are simply DBMS “themes” because they are simply DBMS linkageslinkages
2323
Potential problems with this approach
Zillions of elementsZillions of elements How do you keep track of where they How do you keep track of where they
livelive– rapid access to a record in a DBMS provided rapid access to a record in a DBMS provided
by INDEXby INDEX– index means that all records do not need to index means that all records do not need to
be sequentially accessed and checkedbe sequentially accessed and checked How to index spatial elementsHow to index spatial elements
– attributes are X, Y (and maybe Z)attributes are X, Y (and maybe Z)
2525
Morton Codes (HHcode) for Tiles
.---
32
10
02
00
03
01 10 11
13
3121 30
12
20
22 23 32 33
002
000
003
001
012
010
013
011
033032
031030
023022
021020
102
100
103
101
112
110
113
111
133132
131130
123122
121120
302
300
303
301
312
310
313
311
333332
331330
323322
321320
202
200
203
201
212
210
213
211
233232
231230
223222
221220
2626
Use quad trees to spatially index
002
000
003
001
012
010
013
011
033032
031030
023022
021020
102
100
103
101
112
110
113
111
133132
131130
312
310
313
311
333332
331330
323322
321320
202
200
203
201
233232
231230
223222
221220
30
12
21
GROUP CODE
03 031
03 033
12 12
21 21
23 230
23 231
30 30
32 320
32 321
<layer>_SDOINDEX
SDO_GID SDO_CODE SDO_MAXCODE
1013 T2 T2FFFF…F
1013
1013
T320
T302
T320FF…F
T302FF…F
2727
Spatial indexing implications
Spatial joins (Spatial joins (a a VERY BIG DEAL!VERY BIG DEAL!))– IF same extent is IF same extent is
used to create a used to create a quad tree index quad tree index then the same index then the same index number corresponds number corresponds to the same place to the same place on the earthon the earth
Address layer
Other layer
Geocode returns Index location
2828
R-tree index
Commercially introduced by Commercially introduced by Informix Spatial CartridgeInformix Spatial Cartridge– From Illustra technologyFrom Illustra technology– Data (not space)is partitionedData (not space)is partitioned
Bounding boxes (MBR) around each Bounding boxes (MBR) around each element are createdelement are created
MBR encompassing lower groupsMBR encompassing lower groups Index of MBR “values”Index of MBR “values”
3030
R-trees
Variety of formsVariety of forms Are balanced index structuresAre balanced index structures Appear to be much fasterAppear to be much faster Probably not effective as quad-tree Probably not effective as quad-tree
in a very dynamic data environmentin a very dynamic data environment More readingMore reading
– http://www.geog.ubc.ca/courses/klink/http://www.geog.ubc.ca/courses/klink/gis.notes/ncgia/u37.htmlgis.notes/ncgia/u37.html
3131
Spatial queries
Two stagesTwo stages– Stage 1 - IndexStage 1 - Index
Based on spatial indexBased on spatial index RapidRapid Can dramatically reduce data sizeCan dramatically reduce data size
– Stage 2 - ExactStage 2 - Exact Uses spatial operatorsUses spatial operators Performs calculationPerforms calculation
3232
Query ModelQuery Model
LayerData
PrimaryFilter
INDEX
ReducedData Set
SecondaryFilter
QueryFunctions
ExactResult
Set
Table wherecoordinates are stored
Index retrieves area of interest (window)
Proceduresthat determineexact relationship
3333
Spatial Queries in SQL Based on Egenhofer Relationships Based on Egenhofer Relationships
Between GeometriesBetween Geometries– InsideInside– ContainsContains– CoversCovers– Covered ByCovered By– TouchTouch– Overlap Boundaries IntersectOverlap Boundaries Intersect– Overlap Boundaries DisjointOverlap Boundaries Disjoint– EqualEqual– DisjointDisjoint– AnyInteractAnyInteract
4141
ANYINTERACT AND DETERMINE
ANYINTERACT ANYINTERACT - Given two - Given two geometries, returns true if the two geometries, returns true if the two geometries are not disjointgeometries are not disjoint
DETERMINEDETERMINE - Given two - Given two geometries, returns their geometries, returns their relationship to each other.relationship to each other.
4444
Client or server queries
Many applications that connect Many applications that connect with enterprise systems have with enterprise systems have option for queryoption for query– Server sideServer side– Client sideClient side
Will return to this question laterWill return to this question later
4545
Queries and indexing
PerformancePerformance– Retrieval of ALL valuesRetrieval of ALL values– Retrieval of selected setRetrieval of selected set– Data set sizeData set size
Under 100,00 featuresUnder 100,00 features 100,000 – 1,000,000 features100,000 – 1,000,000 features More than 1,000,000 featuresMore than 1,000,000 features
4646
Relationships
Formal “connections” between Formal “connections” between elements in two tables in a databaseelements in two tables in a database
ExamplesExamples– Transformer is “on” power poleTransformer is “on” power pole
Implies that geographic location of transformer is Implies that geographic location of transformer is dependant on location of pole – move pole move dependant on location of pole – move pole move transformertransformer
– Water meter is at end of water lineWater meter is at end of water line– Bridge is connected to street segmentBridge is connected to street segment– City limit is boundary of voting districtCity limit is boundary of voting district
4747
What happens in relationships
Messaging and/or “triggers”Messaging and/or “triggers” User performs an action on a feature that User performs an action on a feature that
has a relationshiphas a relationship Software recognizes existence of Software recognizes existence of
relationship and executes defined relationship and executes defined operationoperation
May require user input may be automaticMay require user input may be automatic Extensive use of relationships has Extensive use of relationships has
performance implicationsperformance implications Use has design complexitiesUse has design complexities
4848
Transactions and versioning
Enterprise systems provide methods to Enterprise systems provide methods to allow features to be updated while under allow features to be updated while under useuse– User 1 locks individual records are locked for User 1 locks individual records are locked for
edit/update purposesedit/update purposes– User 2 can access data but nor changeUser 2 can access data but nor change– If multiple users are updating rules define If multiple users are updating rules define
how multiple changes are appliedhow multiple changes are applied Versioning allows data configuration to Versioning allows data configuration to
be date definedbe date defined
4949
Metadata Spatial metadata vs applications metadataSpatial metadata vs applications metadata Spatial metadataSpatial metadata
– E.g.E.g. ISO TC 211 19115ISO TC 211 19115 FGDCFGDC
– Originally file orientedOriginally file oriented DateDate SourceSource
– Enterprise systems are dynamic feature/element Enterprise systems are dynamic feature/element structuresstructures
Date = date of last updateDate = date of last update
Movement to element based, dynamic Movement to element based, dynamic metadatametadata