Technical Workshops |

Esri International User Conference San Diego, California

Python – Beyond the Basics David Wynne

Kenneth Smith

July 24 – July 25, 2012

Today’s Workshop

Builds on Python – Getting Started

• Data structures & functions • Script tools • Creating and using classes for parameter values • Accessing data with cursors • Working with geometry

ArcPy

• A cornerstone for automation in ArcGIS - Analysis, conversion, data management, map automation

• The access point to all geoprocessing tools • A package of functions, classes and modules

- Functions that enhance geoprocessing workflows (ListFeatureClasses, Describe, SearchCursor, etc)

- Classes that can be used to create complex objects (SpatialReference, FieldMap objects)

- Modules that provide additional functionality (Mapping, SpatialAnalyst modules)

What’s new for Python at 10.1

• New modules - arcpy.da (data access), - arcpy.na (Network analyst) - arcpy.time

• Python add-ins • Python toolboxes

• Many other enhancements, small and large:

http://esriurl.com/4557

Data structures & functions (Python 201)

Take advantage of key Python data structures

Type Explanation Example List Flexible

ordered sequence

L = ["10 feet", "20 feet", "50 feet"]

Tuple

An immutable sequence (not editable)

T = ("Thurston", "Pierce", "King")

Dictionary

Key/value pairs D = {"ProductName": "desktop",

"InstallDir": "c:\\ArcGIS\\Desktop10.1"}

List comprehension

• Provides a clean and compact way of mapping a list into another list by applying a function to each of the elements of the list

List Comprehension >>> states = ['california', 'alaska', ‘new york'] >>> states2 = [state.title() for state in states] >>> print states2 [‘California', ‘Alaska', ‘New York']

>>> field_names = [f.name for f in arcpy.ListFields(table)] >>> print field_names

Defining Functions

• A simple way to organize and re-use functionality • A bit of code that when executed returns a result

import arcpy def increase_extent(extent, factor): """Increases the extent by the given factor""" XMin = extent.XMin - (factor * extent.XMin) YMin = extent.YMin - (factor * extent.YMin) XMax = extent.XMax + (factor * extent.XMax) YMax = extent.YMax + (factor * extent.YMax) return arcpy.Extent(XMin, YMin, XMax, YMax) oldExtent = arcpy.Describe("boundary").extent newExtent = increase_extent(oldExtent, .1)

Define your function

Return a result

Call the function

Python 201

Demo Data Structures

Creating script tools

Demo The Quadrat Analysis tool

Step 1

Geoprocessing Framework and Python

• Tools can be called from Python • Python code can be made into tools

- With script tools - And at 10.1, Python toolboxes

• Creating tools

- Script tools look & behave like core GP tools - Script tools provide default validation - No UI programming - Documentation of script tools is the same as core tools

Creating a script tool…

• Script tools include input and output arguments - Use GetParameterAsText() or GetParameter() to obtain script

argument values

• Scripts can include messaging - Informative messages during execution of the script - Error messages when a problem arises - Three functions to support tool messaging

- arcpy.AddMessage() - arcpy.AddWarning() - arcpy.AddError()

Tip: Jump Start Your Python Project with a Script Template

###################################################################### ## Name: template.py ## Purpose: ## Author: ## Created: 12/07/2012 ## Copyright: (c) company name ## ArcGIS Version: 10.1 ## Python Version: 2.7 ###################################################################### import os import sys import arcpy def main(*argv): """TODO: Add documentation about this function here""" try: #TODO: Add analysis here pass except arcpy.ExecuteError: print arcpy.GetMessages(2) except Exception as ee: Exception as e: print ee.args[0] # End main function # Ensures this file is usable as a script, script tool, or as an importable module if __name__ == '__main__': argv = tuple(arcpy.GetParameterAsText(i) for i in range(arcpy.GetArgumentCount())) main(*argv)

http://esriurl.com/4558

Jump start your analysis using a template

Demo The Quadrat Analysis script

ArcPy Classes

Classes

• ArcPy supports a series of classes

• Classes are used to create objects - Once created, objects have methods

and properties

• Classes are used most often used for:

- Tool parameters - Working with geometry

Classes continued…

• Many tool parameters are easily defined with a string or a number

• Other parameters are not • Such as a spatial

reference or field map

• Classes can be used define these parameters

Classes as shortcuts

• Use the arcpy classes for parameter values - Extent

- SpatialReference

extent = arcpy.Describe(feature_class).extent arcpy.CreateRandomPoints_management('C:/UC2012/data', 'samplepoints', "", extent, 500)

arcpy.CreateFeatureclass_management( arcpy.env.workspace, 'hydrology', 'POLYGON', spatial_reference=arcpy.SpatialReference(32145))

•Extent class •Search Cursor •Chi-square function

Demo The Quadrat Analysis Tool

Recap

• Development time was fast: - A script template jump started our development - ArcPy provided quick and easy access to tools and

functions - Script tool meant no UI development - Script tools provided default validation

• Took advantage of a function to define and group a useful piece of functionality

• Took advantage of open source libraries (i.e. numpy) • Easy to share and deploy

- No ArcObjects or dlls to register

Accessing Data with Cursors

Cursors

• Cursors provide record-by-record access - Are an important workhorse for many workflows

• The ‘classic’ cursor model works, but… - Not fast enough - Bottleneck for some Python workflows

SearchCursor Read-only access UpdateCursor Update or delete rows InsertCursor Insert rows

Cursors

• arcpy.da cursors use lists and tuples - Row values are accessed by index

• Classic cursors work with row objects - Row values are accessed with setValue/getValue methods

cursor = arcpy.da.InsertCursor(table, ["field1", "field2"]) cursor.insertRow([1, 10])

cursor = arcpy.InsertCursor(table) row = cursor.newRow() row.setValue("field1", 1) row.setValue("field2", 10) cursor.insertRow(row)

Getting better performance

• Maximizing performance

1. Use only those fields you need

2. Use geometry tokens where possible - SHAPE@XY, SHAPE@CENTROID, etc. - Asking for full geometry is expensive

Cursors and with statements

• Historically, needed to be careful with database locking and cursors

• arcpy.da Cursors support with statements - Guarantee close and release of database locks - Regardless of success or failure

with statements

• for loop

• with statement

cursor = None try: cursor = arcpy.da.SearchCursor(table, field) for row in cursor: print row[0] except Exception as err: raise err finally: if cursor: del cursor

with arcpy.da.SearchCursor(table, field) as cursor: for row in cursor: print row[0]

Demo arcpy.da Cursors

Working with Geometry

# Buffer each feature to a new feature class for row in arcpy.da.SearchCursor("C:/data/Roads.shp", ["SHAPE@","NAME"]): geom = row[0] name = row[1] print (geom.length) arcpy.Buffer_analysis(geom, "buffer_{0}".format(name), "100 Feet")

Reading Geometry

• Feature classes have a geometry field - Use SHAPE@ tokens to access

• Returns a geometry object - Has properties that describe the feature - area, length, isMultipart, partCount, pointCount, type, ...

• Geometry objects can often be used in place of feature classes

Reading Feature Geometry

• Geometry has a hierarchy - A feature class is made of features - A feature is made of parts - A part is made of points

• In Python terms - A single part feature looks like this

[pnt, pnt, pnt] - A multipart polygon feature looks like this

[[pnt, pnt, pnt],[pnt, pnt, pnt]] - A single part polygon feature with a hole (inner ring) looks

like

[[pnt, pnt, pnt, None ,pnt, pnt, pnt]]

Reading Feature Geometry

for row in arcpy.da.SearchCursor(polygonFC, ["OID@", "SHAPE@"]): print("OBJECTID: {0}".format(row[0])) for part_count, part in enumerate(row[1]): print "Part count: {0}".format(part_count) for pnt in part: if pnt: print(pnt.X, pnt.Y) else: print("Interior ring")

Loop through each row

Loop through each part in a feature

Loop through each point in a part

For polygons, watch for interior rings

Writing Feature Geometry

• InsertCursor is used to create new features icur = arcpy.da.InsertCursor("D:/data.gdb/roads", field_names) icur.insertRow([row])

• An Update cursor can be used to replace a row’s existing geometry

• Use Point, Array and geometry classes to create geometry

Writing Feature Geometry

# Open an insert cursor for the feature class insert_cursor = arcpy.da.InsertCursor(fc, ["SHAPE@"]) # Create array of point objects array = arcpy.Array([ arcpy.Point(358331, 5273193), arcpy.Point(358337, 5272830)] # Create the geometry object polyline = arcpy.Polyline(array) # Insert the feature insert_cursor.insertRow([polyline]) # Delete cursor del insert_cursor

Geometry operators

• Geometry objects support relational operators at 10

- contains, crosses, disjoint, equals, overlaps, touches, within

• 10.1 adds topological operators - intersect, convexHull, buffer, boundary, clip, difference,

symetricalDifference, union

• 10.1 adds other operators - distanceTo, projectAs, positionAlongLine

Geometry operators

""" Relational operators return Booleans Does line1 cross line2? """ line1.crosses(line2) """ Topological operators return geometry Return the union of two polygons """ polygon1.union(polygon2)

Resources

• Resource Center - http://resources.arcgis.com/communities/python/

• ArcGIS Python Code Gallery - http://arcpy.wordpress.com/

• Good Python References

- “Learning Python” by Mark Lutz - “Core Python” by Wesley J. Chun - “Python Standard Library by Example” by Doug Hellman

• Python at UC 2012 - esriurl.com/uc12python

Evaluation forms: www.esri.com/ucsessionsurveys First Offering ID: 599

Second Offering ID: 689