State of WyomingDepartment of Environmental Quality

Network 101

Nodes – Not as Nasty as Polyps

101 Purpose101 PurposeA basic introduction to the technology

of the Exchange Network for attendees with something other than an IT background. 

◦ What is it and why was it devised? ◦ Why should I care?◦ How does it work?  ◦ What do I have to do to participate?

Minimal techie content included!

101 Topics101 TopicsIntroduction to the NetworkBuilding blocks of the Network

◦The WHAT: XML◦The WHERE: Nodes◦The HOW: Exchanges

Expanding the NetworkQuestions

INTRODUCTION TO INTRODUCTION TO THE NETWORKTHE NETWORK

Network 101

What is the Exchange Network?What is the Exchange Network?

An Internet and standards-based method for

exchanging environmental information between partners.

InternetInternet

State / Tribe

x

x

y z

y z

County

x

x

University

EPA

x

x

y z

y z

Internal System

XML Schema

Network Node

Node Client

What is the Exchange Network?What is the Exchange Network?

An Internet and standards-based method for

exchanging environmental information between partners.

InternetInternet

State / Tribe

x

x

y z

y z

County

x

x

University

EPA

x

x

y z

y z

Internal System

XML Schema

Network Node

Node Client

A little like the POTS (plain old telephone A little like the POTS (plain old telephone system)system)

InternetInternet

x

x

y z

y z

x

x

x

x

y z

y z

An Internet and standards-based method for

exchanging environmental information between partners.

inane

gossip

teenager’s

my

peep

s

Hello?

Wassuppeeps?

I got some new bling

Bet its phat!

With Telemarketers!With Telemarketers!

InternetInternet

State

x

x

y z

y z

x

x

x

x

y z

y z

Benefits of ENBenefits of ENEnables access to timely dataImproved data qualityReduces data management

burden◦Less dual data entry◦Less manual support

Better decisions◦Better data, but also◦Wealth of new data now possible

Computers and robust data management

The Internet and widespread data access

2020thth Century Discoveries Century Discoveries

2020stst Century Limitations Century LimitationsWhat if I need to use the data

from many sites at once?What if I want to produce a map

of those results?What if I want to combine the

data with my collection of related facts?

2020stst Century Exchange Century ExchangeData exchange technologies:

◦Floppy◦FTP / EDI◦E-mail

Data exchange formats:◦Database dumps files◦Flat file◦Spreadsheet document

2020stst Century Exchange Century Exchange2121stst

Data exchange technologies:◦Web services

Data exchange formats:◦XML

History of the NetworkHistory of the Network2900 BC Writing invented1998 AD ECOS and EPA join forces 2000 AD Blueprint of NEIEN created2003 AD Network Specification 1.0

Released2003 AD EPA releases CDX Node2003 AD Washington State release

production Node2004 AD 13 States operational2005 AD 31 States operational2006 AD 41 States and 1 Tribe operational2007 AD National coverage

BUILDING BLOCKS OF BUILDING BLOCKS OF THE NETWORKTHE NETWORK

THE WHAT: XMLTHE WHAT: XML

Network 101

XML in a nutshellXML in a nutshelleXtensible Markup

Language An “XML

Document” contains the Data

An “XML Schema” defines the Structure and the Rules of the data

Schema are developed for each exchange type (e.g., Discharge

Monitoring Report data)

State / Tribe

x

x

y z

y z

County

x

x

University

x y

x y

EPA

x

x

y z

y z

www

Data:Data:Just the factsJust the facts

NODE B

HazWaste

DATA STORE: TSDF

ACME CORP3121, RUMINGTON AVE, DUKE,DEDEW123412312F001 D002 P0324343, DEXTER AVE, DUKE, DEBOCACIOUS ENTERPRISESDEW234317657F001 D002 P0321222, BRUSER AVE, DUKE, DECORNY CANDYDEW145323423F001 D002 P0329898, LEXOR AVE, DUKE, DEFRACTIOUS CORPDEW123324234F001 D002 P0322222, TENBY AVE, DUKE, DEFRIVIOUS INCDEW234234234F001 D002 P032

A filled out form

LEGAL TREATMENT FACILITIES

RCRA ID: DEW123412312

FACILITY NAME: ACME CORP

FACILITY ADDRESS: 3121, RUMINGTON AVE, DUKE,DE

PERMITTED WASTE CODES: F001 D002 P032

RCRA ID: DEW1234317657

FACILITY NAME: BOCACIOUS ENTERPRISES

FACILITY ADDRESS: 4343, DEXTER AVE, DUKE, DE

PERMITTED WASTE CODES: F001 D002 P032

RCRA ID: DEW145323423

FACILITY NAME: CORNY CANDY

FACILITY ADDRESS: 1222, BRUSER AVE, DUKE, DE

PERMITTED WASTE CODES: F001 D002 P032

RCRA ID: DEW123324234

FACILITY NAME: FRACTIOUS CORP

FACILITY ADDRESS: 9898, LEXOR AVE, DUKE, DE

PERMITTED WASTE CODES: F001 D002 P032

RCRA ID: DEW234234234

FACILITY NAME: FRIVIOUS INC

FACILITY ADDRESS: 2222, TENBY AVE, DUKE, DE

PERMITTED WASTE CODES: F001 D002 P032

XML XML Document:Document:

XMLXMLSchema: Schema:

A data entry A data entry form and form and instructionsinstructions

LEGAL TREATMENT FACILITIES FORM

RCRA ID:

FACILITY NAME:

FACILITY ADDRESS:

PERMITTED WASTE CODES:

RCRA ID:

FACILITY NAME:

FACILITY ADDRESS:

PERMITTED WASTE CODES:

RCRA ID:

FACILITY NAME:

FACILITY ADDRESS:

PERMITTED WASTE CODES:

RCRA ID:

FACILITY NAME:

FACILITY ADDRESS:

PERMITTED WASTE CODES:

RCRA ID:

FACILITY NAME:

FACILITY ADDRESS:

PERMITTED WASTE CODES:

CODES MUST START WITH D, F, P, K OR X

CANNOT EXCEED 12 CHARACTERS

AT LEAST ONE CODE REQUIRED, NO LIMIT OF HOW MANY

Simple XML Creation Simple XML Creation DemoDemo

Simple XML Creation Simple XML Creation DemoDemo

Simple XML Creation Simple XML Creation DemoDemo

Simple XML Creation Simple XML Creation DemoDemo

<Tags> are data element

names

Data is between the

tags

Is XML a silver bullet?Is XML a silver bullet?

XML documents get largeCompression helps

Is XML a silver bullet?Is XML a silver bullet?

But agreement is still necessary

To avoid data rejection

BUILDING BLOCKS OF BUILDING BLOCKS OF THE NETWORKTHE NETWORK

THE WHERE: NODESTHE WHERE: NODES

Network 101

What is a Network Node ?What is a Network Node ?

“A simple environmental information Web Service that initiates requests for information, processes authorized queries, and sends/receives the requested information in a standard format.”

Web Service: "a software system

designed to support interoperable

Machine to Machine interaction over a

network.“ W3C

Web Service: "a software system

designed to support interoperable

Machine to Machine interaction over a

network.“ W3C

NodesNodesHardware and software used

to exchange information on the Network

Use the Internet, a set of protocols, and appropriate security to respond to authorized requests for information

Send the requested information in a standard format, XML

Each partner has only one Node

State

x

x

y z

y z

Tribe

x

x

Business

x y

x y

EPA

x

x

y z

y z

www

What does a Node look What does a Node look like?like?

What does a Node look What does a Node look like?like?

Node versus a Node ClientNode versus a Node Client

Makes Requests of other Nodes

Responds to Requests

Comes with a closet

Makes Requests of other Nodes

Responds to spilt liquid

Comes with a monitor

Node

Node Client

BUILDING BLOCKS OF BUILDING BLOCKS OF THE NETWORKTHE NETWORK

THE HOW: EXCHANGESTHE HOW: EXCHANGES

Network 101

Network ExchangeNetwork Exchange

‘Exchange’: the sharing of a specific type of data between two or more Partners. This is often also referred to as a ‘flow’.

For example, sharing of water quality data.

Behind the ScenesBehind the ScenesProvider (STATE)Requestor (EPA)

CDXNODE

AGENCYNODE

ProgramDatabase

QueryProcessor

Data RequestData Request

•AuthenticateAuthenticate

•Perform Perform QueryQuery

Node ManagementNode Management

•Request receivedRequest received

•Account/Service AuthorizedAccount/Service Authorized

•Request loggedRequest logged

Data TransformationData Transformation

•Relational data Relational data retrievedretrieved

•XML transformationXML transformation

•Schema validationSchema validation

AGENCYNODE

Node ManagementNode Management

•Request results loggedRequest results logged

•XML payload returnedXML payload returned

CDXNODE

Data ReceivedData Received

•Response Response loggedlogged

•Payload stagedPayload staged

ProgramDatabase

Data ProcessedData Processed

•Payload Payload validatedvalidated

•Data loadedData loaded

Types of exchangeTypes of exchange

1. Data Synchronization◦ Replicating data

2. Data Publishing◦ Providing data

Data SynchronizationData SynchronizationHazardous

WasteReporter

Internet

EPA’sRCRAInfo

State XHazWaste

DB

InternetNODE

CDX

Data PublishingData Publishing

Hazardous Waste

Reporter

Internet

EPA’sRCRAInfo

HazWasteDB

Internet

Internet

State Y

NODE

CDX

NODE

NODE

Inter-state waste

shipments

?

?

Exchanges in ProductionExchanges in ProductionToday’s Specialscross-program Facility Identification (FRS) Toxics Release Inventory Homeland Emergency

Response

waste RCRAInfo WasteX

health Washington Fish Tissue

Data Exchange Safe Drinking Water

Water quality Pacific NW Water Quality Water Quality Exchange Beach Notification & Monitoring Concentrated Animal Feeding

Operation Discharge Monitoring Reports Electronic Drinking Water

Report Underground Injection Controls

air Air Quality System National Emissions Inventory

XML

XML

`

XML

Air Quality Permitting

Air Ambient Monitoring

Air Emission Inventory

NPDES Permitting (IDEF)

Grants Administration

Facility Identfication

Toxics Release Inventory

Permitting, Compliance

Beaches Notification

Beaches Monitoring

Drinking Water Reporting

Water QualityXML

XML

XML

XML

XML

XML

XML

XML

XML

XML

XML

F/S Identific’n

DOH DB

EIM

C&G Payable

WPLCS

?

AQ SIS

EPCRA

Turbo-Waste

DOH DB

AGBAR

XML

XML

Haz. Waste Gen and Mgmt

Haz. Waste ActivitiesHWTRIn

fo

TIER II

Chemical Inventories

Discharge Monitoring

Node

EPA

xx

xx

yy zz

yy zz

Agency

xx

xx

yy zz

yy zz

Businessxx yy

xx yy

XML

Implementing an Implementing an Exchange Exchange

A Data Exchange is made up ofData services

XML formulation

Data transformation

State

Data

XML

3 Steps to start an 3 Steps to start an Exchange Exchange

1. Map database to data elements

2. Develop XML creator/reader

3. Configure the Node

State

Data

XML

Factors affecting exchange Factors affecting exchange selectionselection

1. Priorities for data sharing with Partners

2. Your data management authority

3. The exchanges that are proven / available

4. The status of agency information systems

EXPANDING THE EXPANDING THE NETWORKNETWORK

Network 101

Network GrowthNetwork Growth

Currently◦ Lots of Nodes

Network GrowthNetwork Growth

Currently◦ Lots of Nodes◦ Mostly regulatory

exchanges

Network GrowthNetwork Growth

Currently◦ Lots of Nodes◦ Mostly regulatory

exchangesSoon

◦ More Nodes◦ More Publishing◦ More Clients

Network EvolutionNetwork Evolution

More Partners are joining

More exchanges are being developed

The universe might expand beyond our “environment”

Node 2.0?Node 2.0?

Technology has matured since 2003Much learnt after 5 years, 60 Nodes

and 20 flowsSo the 1.1 specifications are being

upgradedForecast to be final in July or soThis will result in a better NetworkMigration will not be seamless

Wrapping UpWrapping UpThe Network is all about automating data

sharingXML is just another way to package dataNodes are a little nasty, but so are phones if

you open them upLook for existing solutions first, reuse is

cheaper and avoids multiple “standards”Think outside the box, exchange can be

powerful‘Exchange’ is a state not a single event

The FutureThe Future

This is just the start of the ‘data revolution’. Everything is going to change.

QUESTIONS?QUESTIONS?Network 101