aegis.net, inc. - powerful results. delivered. sm osehra summit topic: “why interoperability is so...

AEGIS.net, Inc. - Powerful Results. Delivered.SM

OSEHRA Summit

Topic: “Why Interoperability Is So Damn Hard”

Presenter: Mario HylandSenior Vice President, AEGIS

@Interopguy

1

Powerful Results. Delivered.Powerful Results. Delivered.

September 5, 2014

How Important is Patient Safety?How Important is Patient Safety?


2


AEGIS has been supporting Interoperability Testing within the HealthIT space with VA/HHS (ONC).

HHS/ONC Nationwide Health Information Exchange (NwHIN) (IHE/HL7/OASIS-SAML)

Healtheway Compliance Testing Body (CTB) Certification process for the eHealth Exchange

OSEHRA Open Source Interoperability Work Group with a focus project of the DIL.

NIST HITECH Award HITTI HL7 Conformance Testing Platform

IntroductionIntroduction

3


Definition of Interoperability (let’s agree to use IEEE).

Participants and Systems Not Ready for Certification or Accreditation◦Dependence on Self-Attestation

Understanding the Standards and Specifications◦ Common (or Consensus based) Interpretation.

Conformance vs InteroperabilitySyntactic vs Semantic Interoperability Interoperability vs Continuous Interoperability

Challenges facing Standards Challenges facing Standards AdoptionAdoption

4


Topic: Open Source Development

Objective: Adherence to a Standard can be demonstrated thru implementation & testing

State of Maturity HealthIT State of Maturity HealthIT StandardsStandards

5

Why Interoperability is ChallengingWhy Interoperability is Challenging

Systems Interoperability Challengeso Organizational exchange of informationo Interruption thru Implementationo Common Testing Approach (Happy Path)o Limited Exception Negative Testing (due to

dependence on Peer-to-Peer testing)o Change (yes, there will be change)

Standards will be put under new intense pressure.

6AEGIS.net, Inc. - Powerful Results. Delivered.SM

Robustness PrincipleRobustness Principle

"Be conservative in what you send, be liberal in what you accept"

http://en.wikipedia.org/wiki/Robustness_principle

Conservative – Happy Path Liberal – Negative and Exception Path


How to Mitigate Interoperability How to Mitigate Interoperability RiskRisk We offer a way to Mitigate Interoperability Risko Because – you will face Interoperability Issues

Interoperability Strategieso Standards based Testing (feedback loop)o Test Early and Test Ofteno Infuse Interoperability into the Development

Cycles thru TDD (Test Driven Development)o Embrace the goal of Continuous Interoperability

thru process


How to Integrate TDD into Agile How to Integrate TDD into Agile ProcessProcess



Developers Integration Developers Integration LabLab

10

Developers Integration Lab (DIL)o Self Service Bi-Direction message based

test labo Global accessibility 24x7/365 dayso End-to-End Systems Testingo Beyond Happy Patho 70% Negative testingo Cloud Shared Serviceo Conformance &

Interoperability

DIL ArchitectureDIL Architecture


11

Standards view thru Standards view thru StatisticsStatistics 2013 - 1,387 Implemented Test Cases*:

522 Conformance Rules applying over 36,000 checkpoints. Over 77,391 Test Case Executions starting in April 2013

◦By Healtheway's CTB candidate participants and vendors.

NwHIN Specification 2010 2011

Patient Discovery 170 177

Query for Documents 178 179

Retrieve Document 26 39

Totals 537 576

*ONC S&I Framework ARRA project

The DIL is Actively Exercising Health IT Standards.

1,113 Test Cases for NwHIN specifications

12


DIL StatisticsDIL StatisticsMore than 200+ Organizations Registered on the DIL to

test45 active Healtheway 2014 onboarding candidates (through

Sept.) Healtheway 2013 Onboarding Candidates

Product Vendors 8

HIE Participants 34

Federal Agencies 3

Key Findings:1. Neither vendors or participants are ready when

they started certification testing.2. 2-4+ month learning curve to demonstrate

compliance to baseline specifications and required message conformance/interoperability expectations.

13



The DIL provides objective analysis of health information standards implementation and exchange problem areas:

DIL StatisticsDIL Statistics

DIL execution data record the root cause(s) of conformance/interoperability failures:

• System/product failures• Test case definition defect• Specification lacks sufficient implementation

guidance14

Industry Approach to Industry Approach to Interoperability Interoperability ONC Roadmap to Interoperability (10 Years)

Patient Privacy (Consent Policy & DS4P)o Unstructured vs Structured DataoGranularityoData Providence

Leverage Existing HIT InfrastructureFDASIA Report (Risk Management)Jason Report (API Architecture)

15


OSEHRA Interoperability Work OSEHRA Interoperability Work GroupGroup

GOALS:

Publish thought leadership piecesParticipate in speaking engagementsContribute content to the OSEHRA public comment platformCaptures User Experiences and Technical ChallengesSupport Rapid Integration of Standards Influence technology, public policy, laws and regulationsDevelop resources, tools and test labs

16



Mario HylandSenior Vice [email protected](703) 893-6020 x707(301) 529-5466 (c)

Contact Us -Contact Us -AEGIS.net, Inc.2275 Research Boulevard, Suite 500Rockville, MD 20850www.aegis.net

DUNS: 152858358 CAGE Code: 3YHD9 17


HIE participant learning curves vary widely. One example:


18

Early failures -> Learning - > Refinement -> Success


HIE participant learning curves vary widely. BEST CASE:


19

2 calendar months from DIL registration to CTB test case results submission for CCHIT review/Healtheway CC onboarding approval.


Once past the system configuration learning curve, DIL Test Execution is FAST:


20

12 MINUTES to actually execute the test cases required for Healtheway’s 2013 onboarding program.


Product Vendor learning curve example:


21

Early failures -> Learning - > Refinement -> Success• Vendors should take more time—and make use of the

DIL more extensively—than individual participants.


State HIE - SUTState HIE - SUT

22


DIL Test AccountDIL Test Account

23


DIL Test Scenario/CaseDIL Test Scenario/Case

24


State HIE Patient Info State HIE Patient Info (SUT)(SUT)

25


DIL Test ResultsDIL Test Results

26


DIL Exception ReportingDIL Exception Reporting

27


DIL Message DebuggingDIL Message Debugging

28


DIL Message AnalyzerDIL Message Analyzer

29


DIL Test HistoryDIL Test History

30


DIL Role Based FeaturesDIL Role Based Features

31

Developers Integration Lab (DIL)◦ Test Users / Participants◦ Test Case Developers◦ Test Rules Engine ◦ Community Representatives◦ Certification / Test Validation◦ DIL Administrator


DIL Test Case DeveloperDIL Test Case Developer

32


DIL Test Rules EngineDIL Test Rules Engine

33


DIL Community RepDIL Community Rep

34


DIL Certification / DIL Certification / ValidationValidation

35


DIL AdministrationDIL Administration

36

aegis.net, inc. - powerful results. delivered. sm osehra summit topic: “why interoperability is so...

Documents

interoperability testing

interoperability risk

powerful results

sm slide

peer testing o change

testing feedback loop

tdd test driven development

sm topic