Network Publishing

Node 2.0 and Publishing

• Node 1.1 Focused on Basic Data Submissions

• Data Publishing Should Be the Focus for Node 2.0.

• Data Publishing is Essential to Leveraging all of the Power of our Network SOA .

• Dynamic Data Publishing is the Missing Piece in Delivering Turn-key Network Node Solutions.

Data Publishing As the Model• Promote data exchange automation –

– come and get it

• Facilitate multi-node exchanges

• Content integration is simplified

• Tools are now available to manage and consume services

• Opens the doors to exchanging any relational information in IT systems.

UDDI Publishing Support Services • Universal Description, Discovery and

Integration (UDDI) registry service for cataloging services

•

• Advertise the services you offer

• Discover services offered by other partners

• Interfaces• Browser

• Web services

https://uddi.epacdxnode.net/uddi/bsc/web

ENDS Publishing Support Services

Node

RequestRequest

RequestRequest Parameters

ParametersParameters

ParametersParameters

NodeNode

Schema Style Sheets

Examples

Makes service metadata available as web services.

Network Query tools can leverage this metadata

FormattingFormatting

Building QueriesBuilding Queries

https://discovery.epacdxnode.net/Node.asmx?WSDL

Goals of Dynamic Data Publishing• Empower Data Stewards

• Support connections to any ODBC or JDBC compliant data sources.

• Enable publishing using simple configuration changes

• Create data services without programming.

• Share canned queries and templates

.NET Data Publishing Process

SQLTemplateProcessor

Data Object -

.NET ODBCData Reader

XML TemplateProcessor

Mapping Template: GetTransactionList.xml

<Transaction> <XmlElement xmlName=”UserName”>$userName$</XmlElement> <XmlElement xmlName=”TransactionId”>$transactionId$</XmlElement> <XmlElement xmlName=”DataFlow”>$dataFlow$</XmlElement> <XmlElement xmlName=”Status”>$status$</XmlElement> <XmlElement xmlName=”ErrorMessage”>$errorMessage$</XmlElement> <XmlElement xmlName=”PostTime”>$postTime$</XmlElement> </Transaction>

Db2XML.xml

Data Service: GetTransactionList

Parameter1: $_user = joe@example.comParameter2: $_status = FailedParameter3: $_flowname=FRS

Template SQL - GetTransactionList

select * from Transaction where userName='$_user' And Status =’$_status’ and DataFlow = ‘$_dataFlow’ order by PostTime DESC

SQL - GetTransactionList

select * from Transaction where userName='joe@example.com' And status = ‘Failed’ And DataFlow= ‘FRS’order by PostTime DESC

Data Result Set

Result XML Instance

<TransactionList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://exchangenetwork.net/schema/v1.0/Transaction.xsd"> <Transaction> <UserName>cdx</UserName> <TransactionId>2be717ef-110c-46c7-a4e1-2f9271597da3</TransactionId> <DataFlow>AsyncQuery</DataFlow> <Status>Failed</Status> <ErrorMessage>The requested procedure, GetFacilityByName, is undefined. .</ErrorMessage> <PostTime>2006-08-30 11:40:21</PostTime> </Transaction> <Transaction> <UserName>cdx</UserName> <TransactionId>59bfb6b1-b73f-4c62-ba72-384dddb45b69</TransactionId> <DataFlow>http://neien.org</DataFlow> <Status>Failed</Status> <ErrorMessage>Error downloading document 59bfb6b1-b73f-4c62-ba72-384dddb45b69 from http://epacdxnode.csc.com/xml/cdx_v10.wsdl</ErrorMessage> <PostTime>2004-03-08 22:19:41</PostTime> </Transaction></TransactionList>

Dabases

SQLData Publishing Process

SQLTemplateProcessor

DataObjects

(Dynamic SQL

Processor)

XML TemplateProcessor

Template: Transaction.xml

<TransactionList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://exchangenetwork.net/schema/v1.0/Transaction.xsd">[foreach record] <Transaction> <UserName>[FieldValue(0,3)]</UserName> <TransactionId>[FieldValue(0,1)]</TransactionId> <DataFlow>[FieldValue(0,2)]</DataFlow> <Status>[FieldEncode(0,6)]</Status> <ErrorMessage>[FieldValue(0,9)]</ErrorMessage> <PostTime>[FieldValue(0,10)]</PostTime> </Transaction>[endforeach]</TransactionList>

GetTransactionList

Data Service: GetTransactionList

Parameter1: Status = FailedParameter2: FlowName=FRS

Template SQL - GetTransactionList

select * from Transaction where userName='$_user' $AND(Status, =, _1) $AND(DataFlow, =, _2) order by PostTime DESC

SQL - GetTransactionList

select * from Transaction where userName='joe@example.com' And status = ‘Failed’ And DataFlow= ‘FRS’order by PostTime DESC

Data Result Set

Result XML Instance

<TransactionList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://exchangenetwork.net/schema/v1.0/Transaction.xsd"> <Transaction> <UserName>cdx</UserName> <TransactionId>2be717ef-110c-46c7-a4e1-2f9271597da3</TransactionId> <DataFlow>AsyncQuery</DataFlow> <Status>Failed</Status> <ErrorMessage>The requested procedure, GetFacilityByName, is undefined. .</ErrorMessage> <PostTime>2006-08-30 11:40:21</PostTime> </Transaction> <Transaction> <UserName>cdx</UserName> <TransactionId>59bfb6b1-b73f-4c62-ba72-384dddb45b69</TransactionId> <DataFlow>http://neien.org</DataFlow> <Status>Failed</Status> <ErrorMessage>Error downloading document 59bfb6b1-b73f-4c62-ba72-384dddb45b69 from http://epacdxnode.csc.com/xml/cdx_v10.wsdl</ErrorMessage> <PostTime>2004-03-08 22:19:41</PostTime> </Transaction></TransactionList>

Dabases

SQLDATA Node Publishing• Create SQL Query with wildcard

Parameters

• Create XML Template Map

• Drop Map and Query into Publishing Directory

• POOF!! New Data Service Published

Network Clientcan accesses the new service

Create SQL Query called GetFacilitybyZip

1. 2.

Application Server

FRS Database

Create XML Map

<FacilitySiteList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.epa.gov/enviro/html/frs_demo/FACID_FacilitySiteAll_v2.xsd">[foreach record] <FacilitySiteDetails stateFacilitySystemAcronymName="TestName01" stateFacilityIdentifier="[DatabaseFieldName]"> <FacilitySite> <FacilitySiteName>[DatabaseFieldName]</FacilitySiteName> </FacilitySite> <LocationAddress> <LocationAddressText>[DatabaseFieldName]</LocationAddressText> <SupplementalLocationText>[DatabaseFieldName]</SupplementalLocationText> <LocalityName>[DatabaseFieldName]</LocalityName> <StateUSPSCode>[DatabaseFieldName]</StateUSPSCode> <LocationZIPCode>[DatabaseFieldName]</LocationZIPCode> </LocationAddress> </FacilitySiteDetails>[endforeach]</FacilitySiteList>

Select FacilitySiteName,F.stateFacilityIdentifier,LocationAddressText,SupplementalLocationText,LocalityName,StateUSPSCode,LocationZipCode from FacilitySite F,LocationAddress L Where F.StateFacilityIdentifier = L.StateFacilityIdentifier and StateUSPSCode=QueryParameterorder by F.FacilitySiteName

POOF! - New GetFacilitybyZip service is available

3.

High Speed Service Publishing Model

HSP DLL

Next Generation Node Publishing

• Create Velocity Mapping File

• Drop Mapping Files into Preconfigured Publishing Directory

• Add Service Configuration Information Using the NGN Administration Client

Data Flow Processor

• CGI/AMS

• Supports Configuration Driven Publishing Model

• Leverages iBATIS - an open source framework for mapping SQL queries

• XSLT Used for Output Formatting

Conclusion

• Simplifies the process of publishing down to creating some configuration/mapping files >> no programming<<

• Dramatically decreases the time and cost for publishing data …. So the Network can grow that much faster!

• Supports emergency response type activities etc..

Questions / Comments ?