An Insight into the Proposed NTG Address Validation Service

David McDonald

22 March 2012

• Project background & business challenge• Project and technical requirements• Solution architecture• Key components of the proposed solution• Video: Spectrum’s address validation capabilities• The added value – data quality and spatial capabilities • Video: Spectrum Enterprise Designer – workflow example• Summary• Questions?

Agenda

Go into the draw to win a bottle of top notch wine for the best NT Spatial 2012 Tweet. All you have to do is include a #tonkin hashtag for your chance to win!

• Tonkin Consulting and PBS are in discussions with NTLIS to supply & implement a NAMF compliant address validation application to the Northern Territory Government

• The provision of an address geocoding/validation solution to core NTG agencies is a key element of the “Spatially Enabling NT Government” initiative– included in the NTLIS business plan for 2011/12

• The proposed solution will deliver increased efficiencies across all NTG agencies by providing access to a standardised address dataset

Project background

• NTLIS maintains the authoritative source of all address data for NT• Other government agencies also use address data within their own internal

business systems (e.g. licensing, policing, etc.)

• Address data held by other agencies is not validated against the authoritative source maintained by NTLIS when these addresses are entered into their systems

• The NTLIS has an immediate business requirement to geocode addresses for the NTPFES

– Other agencies could use this service to validate and geocode their address information

• Improve data quality and spatial capabilities

• Meet future requirements for other NTG agencies

Business challenge

• NTPFES’ immediate requirements to validate address information include the following:

– dispatch resources to an address (geocode) & deal with poor quality input– have a location but need an address (reverse geocode)– able to locate intersections– cleanse historical address information– confirm a persons identity/address (check against NT & interstate addresses)

• The address validation solution has also been scoped for the needs of other core NTG agencies for requirements such as:

– ability to cleanse historical address information to allow it to be easily mapped and analysed

– postal address validation– improve customer communication and reduce mail costs– improve reporting services through providing spatial context

to various datasets

Project requirements

• The address validation service has been scoped to include a key technical requirements such as:– NT data to be stored and access locally

• more up-to date than G-NAF as it includes local updates (update cycles) • flexibility, cost savings

– Access to interstate address information via OnDemand web services• Access as needed, no maintenance, standards-based, cost per transaction• Provides the ability to undertake intersection searches nationally

– NAMF compliance• In its simplest definition, NAMF provides a set of standards that prescribe how

an address, either postal or physical, should be structured and exchanged

– Solution needs to be accessible via web services• SOAP/ REST capabilities

Technical requirements

• Single solution consisting of a Technology Platform that will manage the flow and processing of all the address data

• Integrates with NTLIS existing infrastructure and will use modules from PBS’ Spectrum Technology Platform

• Implementation services and project management will provide the “glue”

Solution architecture

ILIS(Oracle)

Dictionary BuilderSource Data

NTLIS DMZ

Spectrum OnDemand

Web ServiceClient

Spectrum Server &Dictionary Builder

Apache HTTP Server

NTLIS

• Physical address validation (geocoding) engine technology provided by the Pitney Bowes Spectrum Technology Platform

• Physical address validation within NT utilises NTLIS’ own data within a custom dictionary

• Capability to expose all required functionality via customised web service interfaces exposed via both SOAP and REST

• NAMF compliant web service interface is provided for access to Spectrum via the NAMF interface specification

• Postal validation, geocoding of interstate addresses, geocoding of street intersections use PBS’ Spectrum OnDemand service

– Postal address validation uses an AMAS certified engine to validate the postal address and utilises PAF (Postal Address File)

Key components of the proposed solution

• Spectrum Enterprise Designer– The Enterprise Designer provides the capability to implement flow based

design to process address (and other attribute) data. This allows NTLIS to:• incorporate customised business decisions in to the process of geocoding an address

• access internal repositories (Oracle DB etc) to perform batch processing of address (and other) data

Key components of the proposed solution

• Custom Address Dictionary Deployment– A custom address dictionary builder is included

with the geocoding engine

– NTLIS utilises FME technology to manage spatial data. As such it is proposed that the existing FME technology be utilised to extract the data from Oracle

– Deployment of the newly built dictionary is dependent on NTLIS server architecture. As such it is proposed that Tonkin Consulting and PBS work together with NTLIS IT administration staff to create required scripts

Key components of the proposed solution

Build Dictionary(PBBI)

Stop Spectrum Server(NTLIS)

Backup Dictionary(NTLIS)

Copy New Dictionary

(NTLIS)

Start Spectrum(NTLIS)

Test(PBBI)

Sucess

Finish

Roll back(NTLIS)

Start

Yes

No

Notify Admin(NTLIS)

Extract Address data from Oracle

(NTLIS - FME)

Video: Spectrum’s address validation capabilities

• Data quality and spatial capabilities

The added value

13

UnstructuredUnstructuredXML & StructuredXML & Structured AudioAudio VideoVideo PeoplePeople

Governance

SOXSOX

RepeatableRepeatable

ConfidenceConfidence

ProfilingProfiling

AML / CTF AML / CTF

Best PracticeBest Practice

US Patriot ACTUS Patriot ACT

AuditableAuditable

ComplianceCompliance

Fraud DetectionFraud Detection

EthicsEthics

PrivacyPrivacy

Enrichment

Name AnalysisName Analysis GenderGender

Lifestyle Segm,Lifestyle Segm, Risk AnalysisRisk Analysis

Single View CustSingle View Cust Statistical DataStatistical Data

Spatial X,Y Spatial X,Y Calculate CostsCalculate Costs

Social MediaSocial Media Resolve Tel No.Resolve Tel No.

Do Not Call regDo Not Call reg Patronymic An.Patronymic An.

Location Intelligence

Distance CalcDistance Calc Boundary Calc.Boundary Calc.

Point in PolygonPoint in Polygon Buffer Calc.Buffer Calc.

GeocodingGeocoding Find NearestFind Nearest

RoutingRouting Travel TimeTravel Time

Travel CostTravel Cost Area Calc.Area Calc.

Travel MatrixTravel Matrix Line Calc.Line Calc.

Data Integration

SiebelSiebel SalesforceSalesforce

OracleOracle SAPSAP

CRMCRM ERPERP

RDBMSRDBMS SQLSQL

Flat FilesFlat Files Multi LanguageMulti Language

NAMFNAMF TransliterationTransliteration

Data Quality

De-duplicateDe-duplicate Typing ErrorsTyping Errors

TypeAheadTypeAhead Restrict FieldsRestrict Fields

Valid AddressValid Address AMASAMAS

StandardizeStandardize ConsolidateConsolidate

NormalizeNormalize Missing DataMissing Data

FormattingFormatting Incorrect DataIncorrect Data

AdministrationAdministration

SecuritySecurity

Enterprise DesignerEnterprise Designer

Web ApplicationsWeb Applications Desktop ApplicationsDesktop Applications Batch ProcessingBatch ProcessingWeb ServiceWeb Service

Data Access Web Services

Data Warehouse

RDBMSPackaged

AppsExperian

D&B

Sensis

Acxiom

Spectrum Capabilities – enterprise data quality

Name Address City State Zip Phone Email

Bob Williams 36 Jones Avenue Palmerston NT 0830 08 5555 0000 bob.williams@yahoo.com

Robert Williams 36 Jones Av.Northern Territory

0830 55550000

Burkes, Mike and Ilda 38 Jones av. Plamerston N.T. 0830 618-5322-9550 mburkes@gmail.com

Jason Bourne, Bourne & Cie.

76 King st Palmerston nt +618-5360-5480 61855241329

… … … … … … …

Mis-fielded data

Matching

Records

TyposMixed business and contact names

Multiple Names

Non Standard formats

Missing Data

Incorrect addresses

Spectrum Capabilities – Location Intelligence Module

• Spectrum LI Module Spatial Capabilities:– Find Closest Site– Find Point in Polygon– Find Nearest

• The closest linear distance from a point• The closest travel distance from a point• The shortest travel time from a point

– Get Travel Cost Matrix – Get Travel Boundary – Return Holes– Identify Islands – Get Travel Directions– Find Travel Time – Get Bounding Box – Is Within Boundary

Video: Spectrum Enterprise Designer

• Identify duplicates workflow

• An address validation solution, with an ability to undertake both physical & postal address validation, will enable the NTLIS to better service the NTPFES and other NT Government agencies. – real-time (during address entry) and batch mode services accommodated

• Being standards based and utilising web services provides greater flexibility

• The Spectrum Technology Platform is scalable and can be extended to:– provide improvements to overall data quality

– further enrich existing datasets

• And finally the key benefit of this project is....

Summary

NTG Address Validation Solution

“...to ultimately deliver increased efficienciesacross all agencies by providing easy access

to a standardised address dataset.”

Questions???

David McDonaldAssociate, Spatial Services Manager

TONKIN CONSULTINGLevel 2, 66 Rundle Street, Kent Town SA 5067

08 8237 3100, 08 8237 3110David.McDonald@tonkin.com.au