georeferencing erp drawing files

Georeferencing ERP Drawing Files

WMP Consultant Webinar – 25 August 2011

ArcGIS Skills:

The GeoReferencing Toolbar

Approximate Georeferencing

Using 5 Point GeoReferencing

Computing RMS error


SWFWMD Conventions:

Filenames: ERP_XXXXXX_XXX_TYPE_Page_#.tif ERP_XXXXXX_XXX_TYPE_Page_#.txt

AsBuiltApproved_Plan

Raster Data – A Quick Review

In its simplest form, a raster consists of a matrix of cells (or pixels) organized into rows and columns (or a grid) where each cell contains a value representing information, such as temperature. Rasters are digital aerial photographs, imagery from satellites, digital pictures, or even scanned maps.

In raster datasets, each cell (which is also known as a pixel) has a value. The cell values represent the phenomenon portrayed by the raster dataset such as a category, magnitude, height, or spectral value.


The Effects and Georeferencing Toolbars

Georeferencing is performed in ArcMap with the “Georeferencing Toolbar”.

It is also convenient to use the “Effects Toolbar” during this process.

Note: To activate these toolbars, you need to have a suitable layer in the TOC. Inthis case, you want a raster image (.TIF) in the TOC.



On the “Effects Toolbar”, use the:

Contrast, Brightness , and Transparency

slider tools to adjust the image.

Once positioned, use the:

Swipe, or Blink

tools to evaluate the fit.

Swipe

Blink



Note: The “Fit to Display” and “Auto Adjust” are checked on.

The “Transformation” options arenot available until the image has been Georeferenced!

Three tools on the “Position” dropdown:RotateShiftScale

Add Control Points

View Links Table


Georeferencing – The Quick & Dirty Method

Step 1: View the image to georeference and determine an approximate real-world location. You may need to view the cover page to locate the ERP (or some other method.) Step 2: Add the necessary SWFWMD ArcGIS data to your ArcMap

Document. You may need the Roads, ERPs, STR, etc. layers.Step 3: Add the appropriate year image to your ArcMap Document.

You want to choose an image that POST-DATES your ERP.Step 4: Rotate your ERP image until north points “up”Step 5: In ArcMap, zoom into the real-world location so that your ERP

will “fit your screen”Step 6: Rotate, Shift, and/or Scale the image until it fitsStep 7: Check the fit with the Swipe and/or Blink Effects toolsStep 8: Adjust as neededStep 9: On the Georeferencing Toolbar- “Update Georeferencing” Step 10: On the Georeferencing Toolbar – “Rectify…” (set all parameters)


Georeferencing – The Quick & Dirty Method (Q&D)

For this Example, (ERP_001746_005_AsBuilt_Page_2.tif), I

1- Read the coversheet and found that the ERP was locatedin Sections 6 & 7 – Township 40S – Range 23E

2- Added the STR layer, the Streets, and 2007 image layers to my map3- Viewed the ERP scanned image and rotated it until North was “up”4- Using the selected STR, I zoomed into the general area and5- Selected “Fit to Display” on the Georeference Toolbar6- Adjusted the Transparency on the ERP scan image

and scaled/shifted it into position.7- Checked with the Swipe tool to make sure that things

fit properly,8- Used the “Update Georeferencing”

to write the .TFW file.


Georeferencing – The Quick & Dirty Method (Q&D), con’t.

To finish the Georeferencing,

9- Use the Georeferencing Toolbar to “Rectify…” and10 – Fill-in the “Rectify | Save As” dialog as below.

IMPORTANT:

Cell Size < 0.25 NoData as “blank”Resample TypeOutput DirectoryName as ERP ConventionFormat as TIFF


Georeferencing – The Quick & Dirty Method (Q&D), con’t.

11 – Add “Rectified/georeferenced ERP back into ArcMap and Review



Questions ???

Georeferencing – The Accurate Method: GCPs

We can use the “Quick & Dirty” method of georeferencing for getting the ERP into anapproximate position. However, we need a measure of the accuracy of the rectified ERP. To get this measure, we will use Ground Check Points (GCPs).

In this method, you need to identify at least 5 (five) points on the ERP scan that you canalso indentify on the ortho-rectified aerial image. You can check your ERROR inidentifying points after you select 3 (three) points. The goal is to rectify the ERP scanwith the lowest Root Mean Square (RMS) error as possible.

In general, the imagery has an accuracy of +/- 5’ or better, so

an RMSe < 5’ is expected, andan RMSe > 10’ is not acceptable


Georeferencing – The Accurate Method: GCPs

As with the Q&D Method:

Step 1: View the image to georeference and determine an approximate real-world location. You may need to view the cover page to locate the ERP (or some other location data.) Step 2: Add the necessary SWFWMD ArcGIS data to your ArcMap

Document. You may need the Roads, ERPs, STR, etc. layers.Step 3: Add the appropriate year image to your ArcMap Document.

You want to choose an image that POST-DATES your ERP.

But now:

Step 4: Closely examine the ERP scan for photo-recognizable points. Thesemay be things like:

road centerline intersectionsCenters of Cul-de-sacsMan hole coversFlood control structures

Note: Turn OFF the Auto Adjust option on the Georeference Toolbar


Georeferencing – The Accurate Method: GCPs, con’t.

Step 5: Find those GCPs on the ortho-rectified image and record theirreal-world location coordinates (Northings and Eastings).

Step 6: Zoom to the scanned ERP and using the Add Ground Control tool click the point(s) on the scanned ERP, right-click to openthe dialog box, select “Input X and Y” and type in the real-worldcoordinate for that point.

Step 7: Repeat Step 6 at least two more times. It is important that the first

three GCPs be well distributed throughout the scanned ERP. It isbest to have GCPs near each of the corners and one in the centerof the scanned ERP

Step 8: After at least 3 GCPs are input, select the “Update Display” tomove the scanned ERP into it’s location.


Georeferencing – The Accurate Method: GCPs, con’t.

Step 9: Set the Transparency ( and/or swipe) to determine where toplace additional GCPs, or

Step 10: Open the “View Link Table” to see the RMSeStep 11: If the RMSe is > 10’, add and delete GCPs until a satisfactory

error is obtained, retaining a minimum of 5 GCPs at all times.

Once a satisfactory RMSe is achieved,

Step 12: Update the Georeferencing (as with the Q&D method), and

To finish the Georeferencing,

Step 13- Use the Georeferencing Toolbar to “Rectify…” andStep 14 – Fill-in the “Rectify | Save As” dialog as with the Q&D method.


Georeferencing – The Accurate Method - Example

ERP Approved Plan to be Georeferenced: ERP 004863 000 Page2-Stewart Manor Subdivision

SWFWMD Filename: ERP_004863_000_Approved_Plan_Page_2.tif


Georeferencing – The Accurate Method – Example con’t.

I constructed an ArcMap document with the following layers:ERP Activity AreaSection/Township/RangeNAVTEQ Streets in the SWFWMD

Searched in the “ERP Activity Layer” for ‘Base NBR’ = 4863



Note: In this case, there was a pre-existing polygon in the “ERP Activity Area” layer,and it has a geometry similar to the scanner ERP Drainage Plan. It also thecorrect “Permittee Name” (= Steward Manor HOA Inc.), so we can bereasonably certain that we have identified the correct area.

To locate ERPs not in the ERP Activity Area layer (as applicable):

1- Rely on your personal knowledge of the ERP, or2- Review the cover page(s) which will also be included in your

working directory. These frequently have a location map, or3- Examine the ERP for a PLSS identifier. Then use the

Section/Township/Range Layer to locate the ERP, or4- Find one or more road names on the ERP. Then “select” and

zoom-to those roads on the NAVTEQ Streets in the SWFWMDlayer, or

5- Type the name of the ERP into GoogleMaps (last resort), and pan around to find something familiar.

6- Ask others where in the world this ERP is located!


Having found the real world position of the ERP, I located 6 photo-identifiablepoints on the Drainage Plan

1

2 3

54

6



1

2

3

6

4

5

In ArcMap, mouse over the points and record the

Northings and Eastings:

1- 1,391,630.43 x 510,188.552- 1,391,715.16 x 509,749.723- 1,392,198.19 x 509,758.214- 1,392,153.78 x 510,277.255- 1,392,271.31 x 510,239.516- 1,391,905.28 x 509,982.42



Georeferencing – The Accurate Method – Example con’t.Zoom to the scanned ERPZoom in to the area of a photo-identified pointUse the GCP tool and click on the point, then right click and

select “Input X and Y”Enter Coordinates (X = Easting; Y= Northing), press OKRepeat until all points are entered



In ArcMap:Check that the correct target layer is set, andSelect “Update Display” to move the scanned ERP to the

real-world position.Open the Links Table and examine the RMSe



After examination, it seems that Point #2 has a larger than expected residual,so by removing it, the RMSe decreases and I still have at least 5 points inthe solution.



To complete the process:SAVE the Link Table to: ERP_004864_000_Approved_Plan_Page_2.txtUse the Update GeoreferencingRectify… (using normal parameters)

writing the georeferenced file to:ERP_004863_000_Approved_Plan_Page_2.tif in YOUR

GeoTIFF directory



In ArcMap:

Add the newly georeferenced imageAdjust its transparency (~65%)Check you georeferencing!


What to do when it just simply does not fit!

There are several reasons that after all this processing, the scanned ERP just simply does not fit the image:

1. Spatial rectification error in the imagery2. The ERP was modified in a subsequent revision3. You misinterpreted a GCP coordinate4. You mis-copied/transposed, etc. a GCP5. The page (paper) was stretched over time6. The page was badly warped during scanning

and it goes on and on …

YOU have some control over #5 and #6 !


What to do when it just simply does not fit!YOU control the Transformation used during Rectification

The Transformation controls how the image is stretched:

1st Order – Linear Affine Transformation: shifts and rotates in a linear uniform manner2nd and 3rd Order – fit to a polynomial- allows points to be shifted in a non-uniform

manner.

GOTTCHA: Need more (and more GCPs) for higher order Transformations



The number of GCPs (parameters) needed increases as:

T = Transformation Order, and GCP = number of Links

GCP = (T+1)*(T+2)/ (T)



So…For a 1st order Transformation, you need a minimum of :

3 (but we are using 5)For a 2nd order Transformaiton , you need a minimum of :

(3)*(4)/(2) = 6For a 3rd order Transformation, you need a minimum of:

(4)*(5)/(3) = 10… and the more the better!



Questions ???

Some Georeferencing Dreams and Nightmares!Example 1 – The Ideal Case (almost):

Good, clear corners that match the Parcel lines(ERP_015943_000_AsBuilt_Page_1)



Buildings corners fit well

Other features fit well


Example 2 – The not so Ideal Case: The drawing looks great, but the parcel lines do not match up well

(ERP_003751_000_AsBuilt_Page_1)


Parcels (Scan and ERP Activity Area)do not fit well

But building corners fit!


Example 3 – Nightmare #1: Drawing has great lines, BUT the project was MODIFIED so the site plan doesn’t match!

(ERP_014233_001_Approved_Plan_Page_1)


Site boundary and Parcel lines match well, but…

Where’s the “Wet DetentionPond”?


Example 4 – Nightmare #2: Isolated ponds with no external references (parcels, etc.)!

(ERP_000071_008_AsBuilt_Page_1)


The Drawing does not fit the ERPActivity Area (green lines), and …

The pond limits do not fitthe built pond!


Example 4 con’t.– Nightmare #3: The ERP has multiple parts, spread over multiple sheets, and some have multiple

parts on ONE sheet!(ERP_000071_008_AsBuilt_Page_3)


Good luck finding that pond in this ERP!

If you are “brave” (or just plain foolish) and want totry GeoReferencing any of thesenightmares, we will be happy to

ZIP them up and postto:

http://ftp.sfwmd.state.fl.us/pub/outgoing/GeoreferencingNightmares.zip


http://ftp.sfwmd.state.fl.us/pub/outgoing/


Georeferencing – Metadata

The District will supply a Metadatatemplate .XML file for use with georeferenced ERP images.

The template was designed in ArcGIS10 and can be edited inArcCatalog.

It is the responsibility of the contractor to modify the following:

Title Tags DescriptionCredits

Resource IdentificationData Quality


Georeferencing – Metadata – Resource Identification


Georeferencing – Metadata – Data Quality


ERP Scanning and File Conversion

The District uses a drum scanner to digitize ERP plan sheets into 300 dpi multipage Adobe™ Page Description File (.PDF) formatted files. The .PDF files are read with Adobe™ AcrobatProfession version 9.0 and Exported as Tagged Image File Format (.TIFF) files.

NOTE: For ArcGIS to read the TIF file, the extension MUST be changed from .TIFF to .TIF



The Export function will create one .TIFF file for each page in the original .PDF file. Theuser needs to supply an output location and filename, then choose the Settings…

Note: If there are multiple pages, Adobe will name each page with the name provided as a filename belowand will add the _Page_#.tiff to each!



Choosing the settings as indicated willyield binary (0 & 1) monochrome TIF filesthat when queried with Windows ExplorerProperties | Summary | Advanced, will reveal a BitDepth = 1:


So… When a WMP Consultant returns a georeferenced ERP image to the District,the following files should be included:

1- The ORIGINAL .PDF file (properly named)2- The GEOREFERNCED .TIF file (properly named)3- The GROUND CONTROL .TXT file (properly named)4- The METADATA .XML file (properly named)

EXAMPLE:

ERP_000071_000_AsBuilt.pdf (one may apply to several pages)ERP_000071_000_AsBuilt_Page_2.tifERP_000071_000_AsBuilt_Page_2.txtERP_000071_000_AsBuilt_Page_2.tif.xml

Review of files


Questions ???

Stay tuned for HEPs coming up next!!!

georeferencing erp drawing files

Documents