data preparation€¦ · • not all reprojection algorithms are the same ... fme, safe software ()...
TRANSCRIPT
Data Preparation
Warren VickEuropa Technologies Ltd.
What’s your poison?
• We use a variety of methods and technologies• Straw poll
• Hand drawn• Non-GIS software (whole process)• Esri• PB / MapInfo• Open source• Other
• Your choice?
Ken Field, 2017
What is the best GISfor cartography?
• There is no single product or platform• Where possible, use the right tool(s) for the job
Data Preparation
• Crossing the data chasm• Not a task to be underestimated!• Plan• Document
(especially if a repeated process)
Source data Data ready & ideal for use
Effort Results
Disproportionateeffort
What might need to be changed?
• Data Format• Data Type (raster / vector)• Data Projection• Inherent styling• Geometry joins / splits
(Good example in my final presentation)• Topology
Formats
• There are many formats for geospatial data:• Based on published standards• De facto vendor standards
• Data suppliers support limited number of formats• Choose the best format for your software• Some may be better than others:
• Active vs. transfer formats• Know the limitations of your chosen format
• 2Gb file size limits are common
Projection
• Choosing the right projection for a map isvery important
• Nearly all GIS software can reproject on the fly…• … but this is a computational burden• If you have a known destination projection,
it is best to process to it• Not all reprojection algorithms are the same…• … which can lead to misalignment…
Spatial ETL• Extract / Transform / Load• Promoting greater interoperability• Building a bridge over our data chasm• Common tasks:
• Reprojection• Transformations• Restyling
• Other tasks:• Geocoding• Merging / conflict management• Publishing / dissemination
Developing a workflow
• Programming/scripting vs. Visual workflows• Programming/scripting is difficult but becoming
more accessible. e.g. Python• Visual programming is being made available on
many technology platforms• Building blocks (readers, transformers & writers)
with an intuitive drag-and-drop user interface• Examples…
Alistair Aitchisonhttps://alastaira.wordpress.com/2011/04/11/an-unlikely-replacement-for-the-bing-maps-3d-control/
FME, Safe Software (www.safe.com)
Geoprocessing in ArcGIS, Esri (www.esri.com)
Just like programmers creating code,add comments to workflows so that they
may be better understood by others.
Spatial Model Builder, ERDAS Imagine, Hexagon (www.hexagonegeospatial.com)
Graphical ModellerQGIS (www.qgis.org)
Introducing GDAL & OGR
• Open source software (www.gdal.org)• GDAL for raster processing• OGR for vector data processing• Multiple pronunciations
Gee-Dhal vs. Goo-Dull, even Gee-Dull!• Included with QGIS package (www.qgis.org)• Command line tools
OSGeo4W Shell
Introducing GDAL & OGR
• Open source software (www.gdal.org)• GDAL for raster processing• OGR for vector data processing• Multiple pronunciations
Gee-Dhal vs. Goo-Dull, even Gee-Dull!• Included with QGIS package (www.qgis.org)• Command line tools
OSGeo4W Shell
OS Terrain 50
Hillshade Slope Base colour
Using gdaldem command
Using gdaldem command
LIDAR – Digital Surface Model (DSM)
GDAL/OGR Cheat SheetGDAL• Get vector information• Print vector extent• List vector drivers• Convert between vector formats• Print count of features with attributes matching
a given pattern• Read from a zip file• Clip vectors by bounding box• Clip one vector by another• Reproject vector• Add an index to a shapefile• Add a spatial index• Merge features in a vector file by attribute
("dissolve")• Merge features ("dissolve") using a buffer to
avoid slivers• Merge vector files• Get the difference between two vector files
OGR• Get raster information• List raster drivers• Force creation of world file• Convert between raster formats• Reproject raster• Clip raster by bounding box• Crop raster dimensions to vector bounding box• Merge rasters• Stack grayscale bands into a georeferenced RGB• Export a raster for Google Earth• Raster calculation (map algebra)• Create a hillshade from a DEM• Apply color ramp to a DEM
Derek Watkins (http://blog.dwtkns.com)https://github.com/dwtkns/gdal-cheat-sheet
