a geospatial approach to address data etl ... - esri canada · a geospatial approach to address...

16
Presented By: Dilip Sebastian November 15, 2016 A Geospatial approach to Address Data ETL and Address Data Management

Upload: dinhdan

Post on 04-Jun-2018

221 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

P r e s e n t e d B y : D i l i p S e b a s t i a n

N o v e m b e r 1 5 , 2 0 1 6

A Geospatial approach to

Address Data ETL and Address

Data Management

Page 2: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

ABOUT SASKTEL

• SaskTel is the leading Information and Communications

Technology (ICT) provider in Saskatchewan

• Our Mission:

• To provide the best customer experience through our

superior networks, exceptional service, advanced

solutions, and applications

• Headquarters in Regina, SK

• Esri user since 2006

Page 3: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

WHY A NEW DATA MODEL?

• No single comprehensive source of Address data available to satisfy

SaskTel’s business needs.

• Drawbacks of current dataset:

• Lacked information about address type

• Civic Address and Rural Land location information went into the same fields

• Overlapping Address Points for multi-dwelling units

• Non relational

• No data validation

Page 4: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

PROJECT OVERVIEW

• GIS data modelling using ESRI ArcGIS

• From a non-relational data set to relational data modelData Modelling

• ETL Workflows using FME ESRI Edition

• Create Junction tables through spatial analysisETL Process

• ESRI Add-In for Address Creation

• Python scripts for process automationTools & Scripts

• RESTful Services for search & geocoding

• Continual Improvement of internal address datasets

GOLD Source of Address

Page 5: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

ADDRESS AT SASKTEL

• Address is defined by the following:

1. Urban, Rural or First Nations Address

2. Legal Parcel

3. Legal Land Description

Quarter Section

Legal Subdivision

Section 21

Township 16-20

West of 2nd Meridian

Page 6: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

ADDRESS

Addre

ss

Urban

Single Unit

Multi Unit

Single Building

Multi Building

Lot-Block

Rural

Rural Route

First Nations

Quarter Section

Address Data Model.gdb

Address

Buildings

Communities

LegalParcels

LandLocation

SaskTelAttributes

Streets

BlocksHasLots

Community_Domain

LotsHasBuildings

Province_Domain

SubAddress

StreetType_Domain

Page 7: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

COMMUNITY DATASET

• Over 3000 Saskatchewan Communities Geocoded

• Big leap in E911 process

• Unregistered and Registered Communities

co-exist with proper qualifiers.

Google Maps

SaskTel Data

Page 8: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

ETL PROCESS

• Source Data: Old Address Dataset

• Regex Analysis

• Extract and copy Address data into proper fields

• Break down composite data into simple data

• Spatial Analysis

• Generate Business Attributes

• E911 Community

• Land base information

• Related tables for Address and Sub-Address data

• Data Cleansing

Page 9: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

TOOLS & SCRIPTS

• Esri Add-In Tool for Address Creation

• Real-time data validation

• On-the-fly Spatial Analysis

• E911 information

• Land base

• Business Attributes

• Relationship between Address & SubAddress

• Geoprocessing Tools/Scripts

• Batch Geocoding

• Data Formatting

• Process Automation

Page 10: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

ADDRESS CREATION TOOL

• Civic Address

• 1919 Saskatchewan Dr, Regina, SK

• 1060 Dorothy St, BLDG-A, Regina, SK

• Lot-Block Address

• BLK 5 LT 10 Fraser Ave, Craven, SK

• Legal Land Description

• SW15 13-23-W2

• First Nations Address

• HSE-1, Cumberland House IR, SK

Page 11: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

GOLD SOURCE OF ADDRESS

• Decentralized Inventory system to Centralized GIS system

• RESTful services for field service management software

• E911

• Referenced to cleanse and improve address and address related data

• Geocoding Subscriber address data

Page 12: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

ADDRESS & ARCGIS

Address Management

ArcGIS for

Desktop

Custom

Add Ins

Geoprocessing

Scripts

ArcGIS for

Server

Geocoding

Service

Routing

Service

Inventory

Management

Workforce

Management

Page 13: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

SYSTEM ENVIRONMENT

Oracle

Geodatabase

ArcGIS for

Desktop 10.2.1

Address Data

ArcGIS for

Server 10.4

FME ESRI

Edition 2016

Page 14: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

BENEFITS

• Data Quality Improvement

• Improved Productivity

• Increased Market Penetration

• Confident in addressing Customer needs

• Faster Service Provisioning

• Better Customer Service

• Centralized Address Data Management

• Better Analysis Capability

Page 15: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

LESSON LEARNED

• Initially underestimated the complexity of Address Data

• Underestimated the time and effort required for Address Maintenance

• Recommendations:

• Do not delay your Address Management

• Know the advantages that GIS Address Management can provide

Page 16: A Geospatial approach to Address Data ETL ... - Esri Canada · A Geospatial approach to Address Data ETL and Address ... Data Modelling •ETL Workflows using FME ESRI Edition

FUTURE PLANS

• Reconciling with legacy internal systems and drive continual

improvement within the corporation

• Educating internal groups about the advantages of GIS technology