lri validation suite meeting prototype tool demonstration december 20th, 2011
DESCRIPTION
Timeline Demonstration of “proof of concept” validation tool – Tuesday December 20 th, 2011 – Not for use (review in validation suite meetings) – Ballot 1+ IG version Prototype for trial use – Goal is early February (~2 weeks before HIMSS) – Ballot 1++ IG version (locked for Pilots—Today??) Version 1 – 3 months after publication of IG (after ballot 1) – Limited number of test cases Version for MU testing – Fall 2012 (Dependent on final publication of IG) – 3 Months after final publicationTRANSCRIPT
LRI Validation Suite Meeting
Prototype Tool DemonstrationDecember 20th, 2011
Agenda• Administrative• Validation Suite Products Overview• Tool Demonstration – Early Prototype
– Joint effort of NIST and LMCO (ONC)
Timeline• Demonstration of “proof of concept” validation tool
– Tuesday December 20th, 2011– Not for use (review in validation suite meetings)– Ballot 1+ IG version
• Prototype for trial use– Goal is early February (~2 weeks before HIMSS)– Ballot 1++ IG version (locked for Pilots—Today??)
• Version 1– 3 months after publication of IG (after ballot 1)– Limited number of test cases
• Version for MU testing– Fall 2012 (Dependent on final publication of IG)– 3 Months after final publication
Validation Suite Products (Drafts)• Test Case Management Spreadsheet• Test Messages• LIS Test Plan• EHR Test Plan• Testing Methodologies • Vocabulary Spreadsheet (value sets for the IG)
– Including HL7, UCUM, in-scope LOINC, in-scope SNOMED to come– Categorized in-scope LOINC to support testing
• XML Conformance Profile– LRI IG MWBHL7 V2.5.1 XML + CP + CS HL7 V2.7.1 LRI IG XML
• Validation Tool– LIS Mode
• Context-Free• Context-Based (Generated Data Sheets from Test Cases – Scenarios)
– EHR Mode• Generated Data Sheets and Juror Documents from Test Cases – Scenarios
• Provide feedback to implementation guide analysis group
LRI Validation Suite WG Charter OverviewAmbulatory EHR
ONC S&I Framework Test Data
LIS LRI Validation
Tool
EHR LRITest
Harness
NIST LRI Test Tool
NIST LRI Test Tool1. Suite scope limited to the requirements specified in the LRI IG2. Develop test data
1. provided & verified by the S&I Framework community3. LIS LRI Validation Tool
1. Used to validate vendor LIS systems & Test Harness2. Develop a test plan3. Capture and validate LIS LRI messages
4. EHR LRI Test Harness1. Used to validate EHR systems2. Develop a test plan3. Simulation of an LIS system4. Manage, Send, Receive LRI Messages5. Inherently provides CLIA Testing6. Targets
1. S & I Framework EHR Pilots2. Vendor EHR Products 3. MU Certification
LRI ORU R01
LRI ACK R01
S & I FrameworkEHR Pilots
Vendor EHR Products
Ambulatory EHR Systems1. EHR Pilots
1. Draft standard trial use (implementation)2. Verifies LRI IG can be implemented—provide feedback3. Use LRI test harness to perform conformance testing4. Function of the S&I Framework LRI Pilots WG
2. Vendor Product EHRs1. Can use LRI test harness to perform conformance
testing2. LRI Test Harness expected to be used for MU
certification
Part of validation suite deliverables Anticipated users of validation suite products
LRI Validation – High Level Overview• LIS Mode
– Context-Free– Context-Based (Generated Data Sheets from Test
Cases – Scenarios)• EHR Mode
– Generated Data Sheets and Juror Documents from Test Cases – Scenarios
• Conformance Profile Browser• Vocabulary Browser
LIS
LIS Test Tool
Load Cut/paste Send
HL7 V2 Lab Results Message
ValidationReport
Context Free Testing:1. Context free testing will validate a LRI message
created by the LIS2. The context (e.g., the type and results of the
lab test) is unknown to the validation tool3. Therefore not all conformance requirements
of the LRI implementation guide can be assessed
4. However, the validation provides a simple and convenient method for testing message structure and most vocabulary
Process:1. An LRI message is created by the vendor’s LIS2. The message is sent, pasted, or loaded in the
test tool3. A validation is performed
LIS Context-Free Testing
LIS
LIS Test Tool select test case
Test Data Sheet
Manual entry of test data
Load Cut/paste Send
HL7 V2 Lab Results Message
Use Case
Test Case
ValidationReport
EHR transmits Hemoglobin blood test order to the LIS for John Doe along with pertinent demographic and order request data
Lab Results Data SheetJohn DoeDOB: 05/23/1959Gender: MResult: 13.7 g/dL Range 13.2 – 16.2Status: Finaland more…
• A lab test is ordered for a patient• The specimen is collected, and is received and processed in the lab• The lab result is produced and stored in the LIS database• The lab result message is created•The lab result is transmitted to an ambulatory EHR•The lab result is viewed in the ambulatory EHR
Process:1. A technician enters lab results data into LIS
based off the data sheet provided2. The message is sent, pasted, or loaded in the
test tool3. A validation is performed
Context-based Testing:1. Context-based testing will validate a LRI message
created by the LIS2. The context (e.g., the type and results of the lab test)
is known to the validation tool3. Therefore all conformance requirements of the LRI
implementation guide can be assessed
LIS Context-based Testing
EHRLab Message
ACK
Validation
LRI EHR Test HarnessInspection Testing can be performed by:
1. On-site inspection2. Over a webex like technology3. Screen-scraper or screen-capture (include clock)4. Printed Reports
Inspection Testing Techniques1. EHR display screens2. Database access3. Configuration files
Communication
Automated TestingAcknowledgement MessageLimited Utility
LRI Test Message
Validation Report
Test Data
Test Case
Use Case
JurorDocument
ACK
LRI EHR Testing – Test Harness
Conformance Profile• Machine processable version of the LRI implementation guide• HL7 V2.7.1 Standard Conformance Profile
– XML– Chapter 2B– Captures most of the LRI IG requirements
• Message Structure, usage, cardinality, CP, CS, – Key differences from V2.5.1 (and not handled by the MWB)
• Min/max length• C(a/b) – configurable conditionals
– NIST added elements for handling condition predicates and conformance statements (not defined in 2.7.1—NIST will propose for V2.9)
• Support for min/max shortly
• LRI IG MWBHL7 V2.5.1 XML + CP + CS + Length HL7 V2.7.1 LRI IG XML +
Test Case Management Spreadsheet Processor
Message Generation
Engine
Tab Delimited Text File
LRI Test Message
Generation Context File
(XML)
Test Data
Test Cases
Use Case
JurorDocument
(XML)
Test Case Management Spreadsheet
Validation Context File
(XML)
EHR Data Sheet (XML)
LIS Data Sheet (XML)
Message Validation
Engine
Validation Report
LIS Test Plan and LRI Test
Tool
Message Profile (XML)
EHR Test Plan
EHR Test Plan and LRI Test
Tool
LRI Test Tool
Message Model
JurorDocument
(HTML Tables)
Modifies Message Model
Validation Context File
(XML)
Regenerate
Test Case Filter
LRI Test MessageUser
Configuration
Test Case Management Spreadsheet Processing – Priming the Validation Tool
LIS
LIS Test Tool select test case
Test Data Sheet
Manual entry of test data
Load Cut/paste Send
HL7 V2 Lab Results Message
Use Case
Test Case
ValidationReport
EHR transmits Hemoglobin blood test order to the LIS for John Doe along with pertinent demographic and order request data
Lab Results Data SheetJohn DoeDOB: 05/23/1959Gender: MResult: 13.7 g/dL Range 13.2 – 16.2Status: Final
• A lab test is ordered for a patient• The specimen is collected, and is received and processed in the lab• The lab result is produced and stored in the LIS database•The lab result is transmitted to an ambulatory EHR•The lab result is viewed in the ambulatory EHR
1. Local code2. No code—text3. LOINC code
No LOINC given for results
• Test tool inspects OBX.3 for a specific LOINC code• Inspect for 718-7 only or inspect for one of 718-7, 20509-6, 30313-1, 30352-9, 30351-1, and 30350-3?
For Discussion:1. No suggested method given (Should we?)2. With no method given we should expect one
from a set of valid LOINC codes for this order (Lab dependent?)
3. If a specific method is given (is this typical/possible?) then should we look for a specific LOINC code in the sent message?
For Discussion:1. Does the LAB expect a local code for the order in
a electronic order request?2. Use Cases to consider: agreement/no agreement
between LIS and EHR for local codes
It is assumed Lab has capabilities for manual entry
Dat
a E
ntry DB
Message
EHRLab Message
ACK
Validation
LRI EHR Test Harness
CommunicationLRI Test
Message
Validation Report
Test Data
Test Case
Use Case
JurorDocument
ACK
718-7 (LN)
718-7 (LN)
718-7 (LN)S Name Value REF Range
F Hemoglobin 13.7 13.2 – 16.2 g/dL
For Discussion:1. What are the requirements on the EHR with regards to LOINC—
given the LRI IG, CLIA, and meaningful use? 2. What are the EHR display (GUI) requirements for the received lab
results?3. What are the EHR storage requirements for the received lab
results?4. Given that the LOINC code is likely to be translated into the EHR
internal representation, what lab test names need to be displayed in exact form and what lab test names can be displayed in an equivalence text (same concept)?
5. What qualifies as “equivalence—same concept”? How is that determined? Can it be? Expert inspector only? Should displaying of the LOINC text (only) be the requirement—i.e., no mapping?
6. What is the impact of sending only the LOINC code (i.e., no local code) on testing?
Translated into internal
representation
Displayed as internal display text representation