Exchange DesignBest Practices

Tools for Successful

Flow Design and Implementation

1

Flow Design & Implementation Life Cycle

1. Planning

2. Flow architecture design

3. Determine data elements

4. Draft Schema and submit for initial review

5. Develop flow components (sender/receiver)

6. Implement and test

7. Document and submit for final review and publishing

2

Planning Phase

• Determine goals of the exchange– Often a regulatory reporting requirement

involved (Submit)– Consider data publishing model

• Expose queries • Make receiver “Come and get it”• Still satisfies regulatory requirements

– Consider other uses for the data – make the exchange flexible!

• Ad hoc uses of data (human)• Support warehousing by any partner (machine)

3

Flow Architecture Design Phase

• Determine how to chunk data– Schema and Data Serviceswork hand in hand– Compartmentalize Schema by data module – Enables returning portions of data

• GetFacilityByName(“ACME”)• GetFacilityContactsByFacilityID(“1234”)

4

Flow Architecture Design Phase

• Design feedback mechanisms and fault condition behavior– Many teams only considered the “happy

flow”– Rejected files and partial rejection behavior

should be considered and documented– “GetStatus” responses should be explicitly

defined for each stage of processing

5

Flow Architecture Design Phase

• Determine how to handle Transactional Processing– Ideal scenario – Avoid it!

• Data provider should need no prior knowledge of receiver’s data

• Receiver should know what data to request and how to consume the returned data

• MUCH easier for data providers to implement

– If it can’t be avoided…• Use Header “Payload” element to track

“Insert/Update” and “Delete”• Last resort: Use transaction codes in schema

6

Determine Data Elements

• DON’T design the schema by mirroring an existing database model– Hierarchal structure of schema is best for

maximizing interoperability• Relationships expressed through nesting, not

keys• Key/Keyref requires data provider to do more

elaborate data staging

– Goal is to make the data abstract• That said, evaluate each data source for

candidate elements7

Schema Design and Draft Review

• Schema design: How rigid should it be?

Flexible: • no explicit field lengths, fewer required fields, no

Key/Keyref, embedded code lists• Use the Shared Schema Components!!!

Rigid:• Shorter shelf life (i.e. code lists change)• Usually done to be target system-centric• Limits likelihood that schema will be reused

by other flows or matches data standards8

Flexible Rigid

Schema Design and Draft Review

• Compromise!– Keep schema loose (flexible)– Use schematron to enforce target-specific

rules• EPA CDX hosts a schematron validation• Few resources to help with Schematron • More powerful than good old schema validation• Schematron rules can be modified if business

rules change without revising the schema!

9

Schema Design and Draft Review

• Using the Shared Schema Components

10

Schema Design and Draft Review

IMPORTANT THING #1– Follow the design guidance!

• Review the guidance on the EN web site!• If you have questions, get answers!

IMPORTANT THING #2– Submit for Initial review!

• Exchange Network offers “early engagement” services to help uncover issues with your schema and exchange design before sinking significant resources in flow processing code!

– Major issues have been identified every time! We are here to help!

11

Develop Flow Components

• Build the data staging tables– Good practice: stage the data in a data

model similar to the schema format– Create SQL scripts for staging tables and

publish with the exchange• Build the flow composition/decomposition

components– Use OO design to build parsing routines

• Maximizes the likelihood for reuse!• All flow deliverables (including code) are part of

the public domain12

Implementation and Testing

• Testing will expose problems– Document issues as they arise

• Other implementers will encounter the same issues

– Performance issues with large files– Test iteratively – Be prepared to revise schema and/or data

services if needed– Test between multiple partners– Test end-to-end, including feedback

13

Submit Flow Package for Review

• Prepare flow documentation package– Complete and clean up the FCD, DET, and

schemas– Create sample XML instance documents– Finalize Schema Conformance Report– Review EN guidance before submitting

• Required/Optional materials

• Proper schema packaging– Index.xsd– Folder structure

– Submit for review – (2-6 week turn around)14

Schema Design Rules and Guidance

15***Use other published flows as a reference

Pop Quiz

Question:

A developer is not sure how to design some aspect of a schema or flow. What should he/she do?

Answer:

A.Ignore the problem. It will go away.

B.Just guess and move on.

C.Obfuscate the issue with confusing language in the flow documentation.

D.Review the guidance documents. If no answer if found, ask the NTG for assistance.

16

Schema Design Do’s

DO:• Read and follow the Design Rules and

Conventions (DRC) v1.0 and v1.1• Follow Naming Rules (Schema elements and

types, Namespace, and Data Services)• Follow versioning rules• Use the Shared Schema Components in your

schema• Follow schema file segmentation guidelines• Ask questions• Submit your draft schema for review

17

Flow Configuration Document Do’s

DO:• Thoroughly document all data services

(parameters, order, required/optional, return schema)

• Thoroughly document payload structure (use of Header, zip files, payload operations, arrays of NodeDocuments)

• Assume the reader is not familiar with your flow• Use the FCD template and other FCDs for

reference• If the Header is used, thoroughly document

how and when it is used18

Flow Configuration Document Don’ts

DON’T:• Include non-flow specific stuff (i.e. how to ping,

what the header is)• Forget to remove comments, track changes or

other editing artifacts

19

The Moral of the Story

1. Take advantage of flow development assistance services offered by the NTG!

2. Read, understand, and live the guidance!

3. Sometimes there are no easy answers... Ask questions! Buffer your decisions!

20