GEOGRAPHIC SCRIPTING IN GVSIGHALFWAY BETWEEN USER AND DEVELOPER

Geoinformation Research Group,Department of Geography

University of Potsdam

21-25 November 2016

Andrea Antonello

PART 5: RASTER DATA

RASTER DATARaster data can usually be seen as 2 main types: imagery (ex. ortofoto)and physical data (ex. elevation model).

While imagery is something you mostly use as nice background data, withan elevation model there are tons of analyses one can do.

In this document we will mostly refer to raster as the latter case.

Loading raster data in a view is quite simple:

rasters = [ "/home/hydrologis/data/potsdam/dsm_test.asc", "/home/hydrologis/data/potsdam/dtm_test.asc" ] epsg = "EPSG:32632" newview = currentProject().createView("Raster view") newview.setProjection(getCRS(epsg)) newview.showWindow() for raster in rasters: loadRasterFile(raster)

GET RASTER METADATATo work with raster data it is necessary to get some metadata first.

The most used are: rows and cols, boundaries, resolution, bands

dtmLayer = currentView().getLayer("dtm_test") print "Raster information: " print "==============================" print "Columns:", dtmLayer.getWidth() print "Rows:", dtmLayer.getHeight() print "NoData value:", dtmLayer.getNoDataValue().getValue() print "Resolution:", dtmLayer.cellSize print "Bands count: ", dtmLayer.getBandsCount() print "Bounds:" print " north: ", dtmLayer.maxX print " south: ", dtmLayer.minX print " west: ", dtmLayer.minY print " east: ", dtmLayer.maxY

# values can be read through the getData(band, row, column) method print "Random value: ", dtmLayer.getData(0, 10, 10)

THE SIMPLEST RASTER ANALYSISWe now have all the necessary info to loop through the raster data andcalculate max, min, average and valid cells

dtmLayer = currentView().getLayer("dtm_small") count = 0 sum = 0 min = 10000000 # initialize to something high max = 0 # initialize to something low cols = int(dtmLayer.getWidth()) rows = int(dtmLayer.getHeight()) novalue = -9999.0 # now loop over rows and columns for row in range(0, rows): if row % 10 == 0: print row, " of ", rows, " are done" for col in range(0, cols): value = dtmLayer.getData(0, row, col) if value != novalue: # active cells are only the valid ones count += 1 sum += value if value > max: max = value if value < min: min = value

avg = sum/count print "\n\nMax elevation: ", max print "Min elevation: ", min print "Average elevation: ", avg print "Valid cells: %i of %i" % (count, cols*rows)

PUTTING THE RASTER MIN/MAX IN A SHAPEFILELet's put together what we learned until now to extract the position ofthe max and min elevation of the raster and place it in a shapefile. Most ofthe first part is the same as the simple min/max/avg example seen before.

In order to put the points in a shapefile we will need the positions. Asimple way to get them, is to start in the center of the first cell of thecol/row and move pixel by pixel incrementing the X/Y coordinates:

cellSize = dtmLayer.cellSize runningY = dtmLayer.maxY - dtmLayer.cellSize / 2.0 for row in xrange(0, rows): runningX = dtmLayer.minX + dtmLayer.cellSize / 2.0 for col in xrange(0, cols): value = dtmLayer.getData(0, row, col)

# max/min calculation here

runningX += cellSize runningY -= cellSize

PUTTING THE RASTER MIN/MAX IN A SHAPEFILEThe whole piece will look like: dtmLayer = currentView().getLayer("dtm_small") count = 0; sum = 0 min = 10000000; minX = 0; minY = 0 max = 0; maxX = 0; maxY = 0 cols = dtmLayer.getWidth() rows = dtmLayer.getHeight() novalue = -9999.0 cellSize = dtmLayer.cellSize runningY = dtmLayer.maxY - dtmLayer.cellSize / 2.0 for row in xrange(0, rows): runningX = dtmLayer.minX + dtmLayer.cellSize / 2.0 for col in xrange(0, cols): value = dtmLayer.getData(0, row, col) if value != novalue: count += 1 sum += value if value > max: max = value maxX = runningX maxY = runningY if value < min: min = value minX = runningX minY = runningY runningX += cellSize runningY -= cellSize

PUTTING THE RASTER MIN/MAX IN A SHAPEFILENow that we have positions and values, we can create the shapefile:

schema = createFeatureType() schema.append("name", "STRING", 20) schema.append("value", "DOUBLE", 8) schema.append("x", "DOUBLE", 8) schema.append("y", "DOUBLE", 8) schema.append("GEOMETRY", "GEOMETRY") schema.get("GEOMETRY").setGeometryType(POINT, D2) shape = createShape(schema, prefixname="min_max", CRS="EPSG:3003")

and insert the data:

shape.edit() minPoint = createPoint2D(minX, minY) shape.append(name='MIN', value=min, x=minX, y=minY, GEOMETRY=minPoint) maxPoint = createPoint2D(maxX, maxY) shape.append(name='MAX', value=max, x=maxX, y=maxY, GEOMETRY=maxPoint) shape.commit()

currentView().addLayer(shape)

PUTTING THE RASTER MIN/MAX IN A SHAPEFILEand since we are at it, let's add some style:

pointsLegend = shape.getLegend() pointSymbol = simplePointSymbol() pointSymbol.setSize(15); pointSymbol.setColor(Color.BLUE); pointSymbol.setStyle(3); pointsLegend.setDefaultSymbol(pointSymbol) shape.setLegend(pointsLegend)

which should then producesomething like:

CREATING A NEW ARC/INFO ASCII GRIDThe Arc/Info ASCII Grid raster format is quite simple to understand.

The file starts with a header:

NCOLS 355 # number of columns NROWS 361 # number of rows XLLCORNER 1637140.0 # x coordinate of the lower left corner YLLCORNER 5110830.0 # y coordinate of the lower left corner CELLSIZE 10.0 # size of the cell or resolution NODATA_VALUE -9999 # novalue placeholder

and after that, the data - space separated - are inserted row by row.

First, prepare the input and output data:

# open the new file in write mode newRasterPath = "/home/hydrologis/data/potsdam_data/dtm_small_1400_1600.asc" newRasterFile = open(newRasterPath, "w") # get the start raster dtmLayer = currentView().getLayer("dtm_small") cols = dtmLayer.getWidth() rows = dtmLayer.getHeight() novalue = -9999.0

CREATING A NEW ASC RASTERWe have all the information to write the header:

newRasterFile.write("NCOLS " + str(cols)) newRasterFile.write("\nNROWS " + str(rows)) newRasterFile.write("\nXLLCORNER " + str(dtmLayer.minX)) newRasterFile.write("\nYLLCORNER " + str(dtmLayer.minY)) newRasterFile.write("\nCELLSIZE " + str(dtmLayer.cellSize)) newRasterFile.write("\nNODATA_VALUE " + str(novalue))

Then loop through all the values and filter away the ones outside ourrange, i.e. set them to novalue:

for row in xrange(0, rows): newRasterFile.write("\n") for col in xrange(0, cols): value = dtmLayer.getData(0, row, col) if value != novalue and value > 1400 and value < 1600: newRasterFile.write(str(value) + " ") else: newRasterFile.write(str(novalue) + " ") newRasterFile.close() loadRasterFile(newRasterPath)

CREATING A NEW ASC RASTERThe result, overlayed in greyscale on the original dtm, should look like:

NEIGHBOUR OPERATIONS: EXTRACT PITSA pit is a cell in the raster that is lower in elevation than its surroundingcells. Pits are important in hydrology, since "programmatically speaking",the water doesn't flow out of those pits.

Let's start by creating the shapefile that will hold the pit points and put itin editing mode:

schema = createFeatureType() schema.append("value", "DOUBLE", 8) schema.append("GEOMETRY", "GEOMETRY") schema.get("GEOMETRY").setGeometryType(POINT, D2) shape = createShape(schema, prefixname="pits", CRS="EPSG:3003") shape.edit()

and gather the usual information about the raster:

# get necessary info from layer dtmLayer = currentView().getLayer("dtm_small") cols = dtmLayer.getWidth() rows = dtmLayer.getHeight() novalue = -9999.0

NEIGHBOUR OPERATIONS: EXTRACT PITSThe core part loops through the raster values and looks for cells withhigher elevation values around them:

cellSize = dtmLayer.cellSize runningY = dtmLayer.maxY - dtmLayer.cellSize / 2.0 for row in xrange(0, rows): runningX = dtmLayer.minX + dtmLayer.cellSize / 2.0 for col in xrange(0, cols): if row == 0 or row == rows-1 or col == 0 or col == cols-1: continue

v = dtmLayer.getData(0, row, col) if v == novalue: continue v11 = dtmLayer.getData(0, row-1, col-1) v12 = dtmLayer.getData(0, row-1, col) v13 = dtmLayer.getData(0, row-1, col+1) v21 = dtmLayer.getData(0, row, col-1) v23 = dtmLayer.getData(0, row, col+1) v31 = dtmLayer.getData(0, row+1, col-1) v32 = dtmLayer.getData(0, row+1, col) v33 = dtmLayer.getData(0, row+1, col+1)

NEIGHBOUR OPERATIONS: EXTRACT PITSIf the cell is a pit, add it directly to the shapefile:

if v<v11 and v<v12 and v<v13 and v<v21 and v<v23 \ and v<v31 and v<v32 and v<v33: pitPoint = createPoint2D(runningX, runningY) shape.append(value=v, GEOMETRY=pitPoint) runningX += cellSize runningY -= cellSize

shape.commit()

currentView().addLayer(shape)

And never forget to add some style:

pointsLegend = shape.getLegend() pointSymbol = simplePointSymbol() pointSymbol.setSize(6); pointSymbol.setColor(Color.RED); pointSymbol.setStyle(0); pointsLegend.setDefaultSymbol(pointSymbol) shape.setLegend(pointsLegend)

NEIGHBOUR OPERATIONS: EXTRACT PITSThe result should look like:

<license> This work is released under Creative Commons Attribution Share Alike (CC-BY-SA).</license>

<sources> Much of the knowledge needed to create this training material has been produced by the sparkling knights of the <a href="http:www.osgeo.org">Osgeo</a>, <a href="http://tsusiatsoftware.net/">JTS</a>, <a href="http://www.jgrasstools.org">JGrasstools</a> and <a href="http:www.gvsig.org">gvSIG</a> communities. Their websites are filled up with learning material that can be use to grow knowledge beyond the boundaries of this set of tutorials.

Another essential source has been the Wikipedia project.</sources>

<acknowledgments> Particular thanks go to those friends that directly or indirectly helped out in the creation and review of this series of handbooks. Thanks to Antonio Falciano for proofreading the course and Oscar Martinez for the documentation about gvSIG scripting.</acknowledgments>

<footer> This tutorial is brought to you by <a href="http:www.hydrologis.com">HydroloGIS</a>.<footer>