integrating the healthcare enterprise · 2020-03-10 · 43], [rad-44], and [rad-45] transactions to...

Copyright © 2020: IHE International, Inc.

Integrating the Healthcare Enterprise

IHE Radiology 5

Technical Framework Supplement

AI Results 10

(AIR)

Revision 1.0 – Draft for Public Comment 15

Date: March 10, 2020 20 Authors: Radiology Technical Committee Email: [email protected]

Please verify you have the most recent version of this document. See here for Trial 25 Implementation and Final Text versions and here for Public Comment versions.

http://ihe.net/Technical_Frameworks/

http://ihe.net/Public_Comment/

IHE Radiology Technical Framework Supplement – AI Results (AIR) ______________________________________________________________________________

___________________________________________________________________________ Rev. 1.0 – 2020-03-10 2 Copyright © 2020: IHE International, Inc.

Template Rev. 10.4

Foreword This is a supplement to the IHE Radiology Technical Framework V18.0. Each supplement undergoes a process of public comment and trial implementation before being incorporated into 30 the volumes of the Technical Frameworks. This supplement is published on March 10, 2020 for public comment. Comments are invited and may be submitted at http://www.ihe.net/Radiology_Public_Comments. In order to be considered in development of the trial implementation version of the supplement, comments must be received by April 9, 2020. 35 This supplement describes changes to the existing technical framework documents. “Boxed” instructions like the sample below indicate to the Volume Editor how to integrate the relevant section(s) into the relevant Technical Framework volume.

Amend Section X.X by the following:

Where the amendment adds text, make the added text bold underline. Where the amendment 40 removes text, make the removed text bold strikethrough. When entire new sections are added, introduce with editor’s instructions to “add new text” or similar, which for readability are not bolded or underlined. General information about IHE can be found at www.ihe.net. 45 Information about the IHE Radiology domain can be found at ihe.net/IHE_Domains. Information about the organization of IHE Technical Frameworks and Supplements and the process used to create them can be found at http://ihe.net/IHE_Process and http://ihe.net/Profiles. The current version of the IHE Radiology Technical Framework can be found at http://ihe.net/Technical_Frameworks. 50

http://www.ihe.net/Radiology_Public_Comments/

http://www.ihe.net/

https://www.ihe.net/ihe_domains/

http://ihe.net/IHE_Process/

http://ihe.net/Profiles/

http://ihe.net/Technical_Frameworks/



Template Rev. 10.4

CONTENTS Introduction to this Supplement ...................................................................................................... 6 55

Open Issues ................................................................................................................................ 7 Closed Issues ............................................................................................................................ 16 TO DO ...................................................................................................................................... 20

IHE Technical Frameworks General Introduction ........................................................................ 22 9.0 Copyright Licenses ................................................................................................................. 22 60

9.1 Copyright of Base Standards .............................................................................................. 22 9.1.1 DICOM (Digital Imaging and Communications in Medicine) .................................. 22 9.1.2 HL7 (Health Level Seven) ......................................................................................... 22 9.1.3 LOINC (Logical Observation Identifiers Names and Codes) .................................... 22 9.1.4 SNOMED CT (Systematized Nomenclature of Medicine -- Clinical Terms) ........... 23 65

10 Trademark ................................................................................................................................ 23 IHE Technical Frameworks General Introduction Appendices .................................................... 24 Appendix A – Actor Summary Definitions .................................................................................. 24 Appendix B – Transaction Summary Definitions ......................................................................... 24 Appendix D – Glossary ................................................................................................................. 25 70 Volume 1 – Profiles ..................................................................................................................... 26

Domain-specific additions ....................................................................................................... 26 X AI Results (AIR) Profile ........................................................................................................... 26

X.1 AIR Actors, Transactions, and Content Modules ............................................................. 26 X.1.1 Actor Descriptions and Actor Profile Requirements ................................................. 28 75

X.1.1.1 Evidence Creator ............................................................................................... 28 X.1.1.2 Image Manager/Archive .................................................................................... 29 X.1.1.3 Image Display .................................................................................................... 29 X.1.1.4 Imaging Document Consumer ........................................................................... 29

X.2 AIR Actor Options ............................................................................................................ 30 80 X.2.1 Parametric Map Option ............................................................................................. 30

X.3 AIR Required Actor Groupings ........................................................................................ 31 X.4 AIR Overview ................................................................................................................... 31

X.4.1 Concepts .................................................................................................................... 31 X.4.1.1 Result Primitives ................................................................................................ 31 85 X.4.1.2a Root Results ..................................................................................................... 33 X.4.1.2 Result Primitive Encodings ............................................................................... 34

X.4.1.2.1 General Result Encoding Requirements .................................................... 34 X.4.1.2.2 Qualitative Findings ................................................................................... 36 X.4.1.2.3 Measurements ............................................................................................ 37 90 X.4.1.2.4 Locations .................................................................................................... 38 X.4.1.2.5 Segmentations ............................................................................................ 39 X.4.1.2.6 Parametric Maps ........................................................................................ 39 X.4.1.2.7 Tracking IDs .............................................................................................. 40 X.4.1.2.8 Image References ....................................................................................... 40 95



Template Rev. 10.4

X.4.1.3 Result Filtering and Navigation ......................................................................... 40 X.4.1.4 Result Presentation ............................................................................................ 44 X.4.1.5 AI Algorithm Deployment ................................................................................. 45 X.4.1.6 Result Approval & Retention ............................................................................ 45 X.4.1.7 Codesets ............................................................................................................. 46 100 X.4.1.8 Interactive AI ..................................................................................................... 47

X.4.2 Use Cases .................................................................................................................. 48 X.4.2.1 Use Case #1: Store, Retrieve, & Display ........................................................... 48

X.4.2.1.1 Store, Retrieve, & Display Use Case Description ..................................... 48 X.4.2.1.2 Store, Retrieve, & Display Process Flow .................................................. 49 105

X.4.2.2 Use Case #2: Auxiliary Usage ........................................................................... 50 X.4.2.2.1 Auxiliary Usage Use Case Description ..................................................... 50 X.4.2.2.2 Auxiliary Usage Process Flow ................................................................... 50

X.5 AIR Security Considerations ............................................................................................. 51 X.5.1 Security Considerations for Actors ........................................................................... 51 110 X.5.2 Security Considerations for AI Results ..................................................................... 51

X.6 AIR Cross-Profile Considerations ..................................................................................... 51 Appendices to Volume 1x ............................................................................................................. 54 Appendix J – Example AI Result Encodings ................................................................................ 54

J.1 Example Qualitative Finding .............................................................................................. 54 115 J.1.1 <Title> ........................................................................................................................ 54

Volume 2 – Transactions ............................................................................................................ 55 4.Y1 Display Analysis Result [RAD-Y1] ................................................................................ 55

4.Y1.1 Scope ....................................................................................................................... 55 4.Y1.2 Actor Roles .............................................................................................................. 55 120 4.Y1.3 Referenced Standards .............................................................................................. 55 4.Y1.4 Messages ................................................................................................................. 56

4.Y1.4.1 Display Results ................................................................................................ 56 4.Y1.4.1.1 Trigger Events .......................................................................................... 56 4.Y1.4.1.2 Message Semantics .................................................................................. 56 125 4.Y1.4.1.3 Expected Actions (i.e., Display Requirements) ....................................... 56

4.Y1.4.1.3.1 General Result Display Requirements ............................................. 57 4.Y1.4.1.3.2 Display of Qualitative Findings ....................................................... 58 4.Y1.4.1.3.3 Display of Measurements................................................................. 58 4.Y1.4.1.3.4 Display of Locations ........................................................................ 59 130 4.Y1.4.1.3.5 Display of Segmentations ................................................................ 59 4.Y1.4.1.3.6 Display of Parametric Maps ............................................................. 60 4.Y1.4.1.3.7 Display of Tracking IDs ................................................................... 60 4.Y1.4.1.3.8 Display of Image References ........................................................... 61

4.Y1.5 Protocol Requirements ............................................................................................ 61 135 4.Y1.6 Security Considerations ........................................................................................... 61

4.Y1.6.1 Security Audit Considerations ......................................................................... 61 4.Y1.6.2 Display Specific Security Considerations ....................................................... 61



Template Rev. 10.4

Appendices to Volume 2 ............................................................................................................... 62 Volume 3 – Content Modules ..................................................................................................... 63 140

5.1 ITI-20 Record Audit Event ................................................................................................ 63 6.5 DICOM Content...................................................................................................................... 63

6.5.1 AI Result Content Module ......................................................................................... 63 6.5.1.1 Referenced Standards ......................................................................................... 63

145



Template Rev. 10.4

Introduction to this Supplement This AI Results Profile addresses the capture, distribution, and display of medical image analysis results. The central use case involves results generated by artificial intelligence (AI) algorithms. Key considerations include:

• Interoperable Results: 150 The results need to be presented to radiologists in their reading environment. This depends on interoperability between result generation products and radiology reading workstation systems/software.

• Study Integrated Results: Radiologists expect AI-generated results to be presented in the context of the study to 155 which they apply and expect them to supplement (rather than replace) traditional image analysis results and thus a given study will be composed of acquired images, AI results, & traditional clinical data.

• Effective Presentation:

Effective use of results hinges on how they are presented during the busy process of 160 reading the study.

• Convergence of Result Encoding: Many AI results are results that a human could otherwise have produced, and those human results may be used as training data for the AI. AI results might also be used by other AIs in an adversarial network. AI and non-AI results need to be handled together. 165 Convergent encoding of results facilitates this, as well as data pooling and sharing between sites.

• Display Primitives: It is unrealistic to expect radiology displays to implement specific display capabilities for each of the myriad of algorithms being developed. To minimize implementation 170 complexity for displays, and avoid needing different software for each new AI result, compose AI results from a reasonable set of primitives.

This profile will establish baseline data handling and presentation capabilities for an image display product to be “AI-Ready”. Result generation products will be similarly motivated to support these data formats so that their results can be compatible with a variety of displays and 175 site workflows. There are many other radiology applications of AI that are not about processing images. This profile is image-centric. AI Workflow is addressed separately in another IHE Radiology profile under development. 180



Template Rev. 10.4

Open Issues 1 Q. Should we keep the Parametric Map Option or just make it required?

The option collects primitives that may be less used by Evidence Creators and might be considered an implementation burden by some Image Displays.

2 Q. Should probabilistic segmentations be stored as Segmentations or Parametric Maps? While the DICOM Segmentation object supports probabilistic voxels, some frame analysis implementations might find the Parametric Map more logical. If the profile does not point towards one, displays will need to do extra work to support both. Probabilistic Segmentations:

• are well-suited to choosing a threshold then working with the segment

• have values that reflect confidence/applicability of a common statement

• can represent decimal probability values

• are part of the basic AIR Profile Parametric Maps

• are well-suited to considering voxels one by one

• have values that “have a broader meaning”

• can represent floating point probability values

• require advanced support by the display

3 Q. Which Hypothetical behaviors should be made required and/or left in the TI draft? See X.4.1.3 Result Filtering, and 4.Y1.4.1.3 (Display Requirements) Envision how AI Results will be used in interpretation. Are there additional display requirements that would be appropriate? Will AI results be presented before the images, on top of the images, beside the images, after the images? Will there be a need for tabular presentation? A table of multiple timepoints for a certain observation? A table of multiple nodule sizes at one timepoint? Which X.4.1.3 Result Filtering details should the user be able to see (i.e., be required display behaviors)? Mammo CAD required display of the (111071, DCM, “CAD Operating Point”). Should we have a general requirement to make algorithm parameters available to the operator?



Template Rev. 10.4

4 Q. How should algorithms encode confidence in each result? As indicated in X.4.1.3, confidence estimates might be very useful for the interpreting radiologist and might be useful for the filtering and organization rules that radiologists might configure on their displays. Some AI Models may have quantitative estimates of confidence based on good statistical models, others may estimate rough relative confidence, and other Models may not be able to provide any indication of confidence at all. For present/absent-type findings or staging/classification findings, a certainty score may be appropriate. For quantitative results, a 95% confidence interval for the result might be more appropriate. DICOM SR already has some concept codes. Should any of these be added (CP) to findings in the TID 1500 structure?

• DCM 111071 CAD Operating Point

• DCM 111013 Certainty of Impression

• DCM 111011 Certainty of Feature

• DCM 111012 Certainty of Finding

• DCM 111047 Probability of cancer

• DCM 111087 False Markers per Case

• DCM 111086 False Markers per Image

• DCM 111088 Case Sensitivity

• DCM 111090 Case Specificity

• DCM 111091 Image Specificity

• DCM 111089 Lesion Sensitivity The bottom 8 of the above are found in CID 6048: http://dicom.nema.org/medical/dicom/current/output/chtml/part16/sect_CID_6048.html

5 Q. How should algorithms encode criticality of a result? As with “confidence”, criticality (the urgency/severity/actionability) of each result may influence worklist priorities as well as result filtering and organization. This and the RadLex criticality codes are discussed in X.4.1.3.

http://dicom.nema.org/medical/dicom/current/output/chtml/part16/sect_CID_6048.html



Template Rev. 10.4

6 Q. What interest is there in a JSON-based STOW AI Result transaction "Phase II" activity for this profile is being considered to add a transaction that clones [RAD-63] Submit Dose Information (using STOW-RS) with a JSON SR payload based on DICOM Sup219 (currently in Trial Implementation) The transaction seems fairly straightforward (EP 1, CP 1) but would require DICOM work to formally permit Sup219 JSON as a STOW-RS payload. A second step would be to permit Sup219 JSON as a WADO-RS media type.

7 Q. Do we require any filtering and navigation capabilities on the Image Display? Navigation of results in a single study could quickly become complex to the point of being dysfunctional. See discussion in X.4.1.3. Currently none of the described capabilities are required. One approach is to leave the problem hard and depend on the displays doing advanced work based on minimal information from the creators. A second approach is to make the navigation/organization problem easier by having the creators encode more details about the results and their knowledge of the relationships between the results. In some ways it’s a bit like the Enhanced vs Simple MR Images. Do we deploy Simple now and someday try to get people to migrate to Enhanced, or do we go with Enhanced now?

8 Q. What profile name do you prefer?

• AIR – AI Results or Analysis Results

• RADAR – Retrieve And Display AI/Analysis Results

9 Q. Does the concept of “Hanging Protocols” for Results confuse people? Would it be easier to talk about result selectivity rules and Selective Display of Results? See X.4.1.3 Result Filtering.



Template Rev. 10.4

10 Q. Are there specific primitives that should be added (or removed)? Selection of the current primitives was guided by a review performed by DICOM WG-23 of common AI tasks and their primary and secondary results. The adequacy of the primitive set will be evaluated during the Trial Implementation period of the profile. Some changes may be made before the profile goes to Final Text. Additions after that are conceivable but would need to be encapsulated in Named Options.

11 Q. Should we define encoding/display behavior for failed processing in more detail? Based on comments from radiologists, we require the Display to indicate when analysis failed (either completely or partially) and also to distinguish when the algorithm succeeded but there were no findings. Where does it look in the study to find that those things out? For example, in the case of a failed segmentation, should the Evidence Creator create a Root Result SR with failure codes and no Segmentation instance? Would the reason for failure be informative for the radiologist? If so, where should that be recorded? In the case of Mammo CAD, there was text that read “This information shall be obtained from the status values of (111064, DCM, “Summary of Detections”) and (111065, DCM, “Summary of Analyses”).” But that’s not generally applicable.

12 Q. Are there other details for which encoding guidance would be helpful? e.g., populating Participant Sequence, Performed Procedure Code Sequence, etc.

Q. Should we add informative/normative detail about preliminary/partial/verified/etc.? It is covered briefly in X.4.1.6. Think through how displays would expect to filter and manage presentation based on these attributes. Is it clear enough? Are there fields we should mandate the Evidence Creator to populate? Think through how image managers might expose/suppress unverified results, or how they might implement deletion/retention policies. Again, is it clear and will they depend on any particular behaviors by the Evidence Creators?



Template Rev. 10.4

13 Q. Are the [RAD-44] & [RAD-129] (QIDO) Query fields adequate for finding/filtering results? See X.4.1.3 for discussion of how displays/users might navigate available results. Do the query fields seem adequate for displays to be able to get relevant subsets of result objects?

14 Q. Should the algorithm description inside results include whether it is approved or for what? Displays might have different behavior depending on whether the algorithm is approved or not, or what the “approved paradigm of use” is” E.g., a result from an algorithm that has been approved for use in a “second reader” paradigm should not initially be presented to the radiologist. If so, what other paradigm of use codes do we need? Alternatively, displays could be made to consult another source of information using the algorithm ID.

15 Q. Is “Rendering Intent” an appropriate flag to indicate results that are not intended for imaging clinician review? E.g., if an algorithm creates one SR that captures the actual finding(s) for the imaging clinician and another SR that is intended to drive worklist, what flag would the Evidence Creator include to allow the imaging clinician to suppress the worklist SR elements? Counterpoint: we should assume that each display implementation will analyze the entire content of the SR Tree for all the results and make its best guess if each is something the radiologist needs to see or not. The algorithm is probably unable to determine the purpose of the results it is creating.



Template Rev. 10.4

16 Q. Do we need changes to TID 1500 for qualitative findings? The current TID 1500 structure evolved a bit organically over time as new incremental requirements emerged. The result is some variability/inconsistency in where details are coded. In the current sub-templates (TID1501->TID300, TID1410->TID1419, TID1411->TID1419), the measurement HAS CONCEPT MODs like Finding Site as children of the measurement . However, when a TID1501 Measurement Group contains multiple measurements with common modifiers, and when efficiency is important, some details can be coded just once at the group level. But Qualitative Evaluations don’t have the children NUMs do. Should we add them or explain qualitative evaluations always code those details at the group level? There’s also a question about whether a finding that applies broadly to a whole image should be coded as related to a full-image ROI, or disassociated from an image (which raises the question of what the finding applies to). DICOM WG6 is currently working on CP1998 to simplify TID1500 for AI/ML.

17 Q. Is the measurement group in TID 1500 sufficient for establishing relationships between related findings? E.g., can currently put the locations/size/solidity/margin values for a certain nodule in the same measurement group (with a Tracking ID to identify the specific nodule), and then each other module would have its own measurement group. Are there other examples of relationships that would not work with this pattern?



Template Rev. 10.4

18 Q. Should we constrain pre-coord/post-coord of measurements or require displays to handle all conceivable variations? Based on experience with Echo SR, unconstrained structure complexity and varying degrees of pre-coordination/post-coordination were a severe challenge for receiving systems and resulted in low interoperability. X.4.1.2.3 has proposed a “simple” post-coordinated structure. What happens when evidence creators need greater complexity? One approach might be to encourage creators to move to fully precoordinated codes with well-chosen code meaning text. While it may be tedious to make a list of pre-coordinated codes, and requires configuration to handle new codes, an advantage is that it is mechanical and unambiguous. The business logic is a fairly simple “if code=X, then semantics are Y” as opposed to trying to figure out how to parse all the different possible scenarios in the post-coordinated structure (including all the unexpected value combinations “wait, what does it mean to take that measurement in M-mode?”). People may reason it out but it’s harder for software. An alternative is for creators start adding additional post-coordinated context modifiers. Post-coordinated coding feels like it can avoid configuration by encoding the semantics of each post-code and combining, but the experience has not borne that out. Each new code value and each new post-coordinated concept has the potential to affect the meaning of the other post-coordinated concepts and values. To make life maximally interesting for displays and databases, we should allow all the above variations and more. Regardless of our choice, we should plan to monitor this issue during Trial Implementation and early Connectathons and potentially “iterate” the specification.



Template Rev. 10.4

19 Q. How should we encode qualitative findings? Excessive variability will decrease interoperability. A common class of findings are for presence/absence of a condition such as pneumothorax, or pneumonia, or stroke. One question is how to structure the “concept-value statement(s)”, e.g.:

• Finding=Pneumothorax, Presence=Indeterminate

• Pneumothorax Presence=Indeterminate

• Finding=Pneumothorax (presence is implied; can’t code indeterminate)

A related question is whether it is ok to use what are normally value codes to be concept codes, e.g.:

• Pneumothorax=Indeterminate Or in the case of AIW, can we use them as implied tasks, e.g.:

• Scheduled Workitem Code=Detect Pneumothorax

• Scheduled Workitem Code=Pneumothorax (it is implied that I would like you to analyze the images to determine this pathology is present)

• Schedule Workitem Code=Appendix (does this imply detect if the appendix is present, or analyze the appendix for abnormalities, or what?)

Another question is about encoding Finding Site

• Finding Site=Lung, Laterality=Left

• Finding Site=Chest

• <null> (display applications need to be programmed to know where pneumothorax occurs)

In situations where multiple alternatives exist, IHE frequently chooses one to improve interoperability while acknowledging that other choices exist.

20 Q. Do we want to make any additional IOD requirements? See example of Table 4.8.4.1.2.3-1 for Mammography https://www.ihe.net/uploadedFiles/Documents/Radiology/IHE_RAD_TF_Vol2.pdf

https://www.ihe.net/uploadedFiles/Documents/Radiology/IHE_RAD_TF_Vol2.pdf



Template Rev. 10.4

21 Q. Do we need to address in detail the use of Patient Orientation and Spatial Locations Preserved in location display? The MAMMO Profile has the following text:

The Image Display shall also be able to apply the marks to “For Presentation” images whose Source Image Sequence references the SOP Instance UID of the “For Processing” images that are referenced by the Mammography CAD SR SOP Instance, unless the Spatial Locations Preserved (0028,135A) is present in the Source Image Sequence Item and has a value of NO. The Patient Orientation of the images referenced in the Source Image Sequence encoded in (111044, DCM, “Patient Orientation Row”) and (111043, DCM, “Patient Orientation Column”) of the Mammography CAD SR SOP Instance shall be used to transform (flip or rotate) the coordinates of the CAD marks if it differs from the Patient Orientation (0020,0020) of the corresponding “For Presentation” image.

The CXCAD Profile chose to skip For Processing: The Spatial Locations Preserved is not required since “For Processing” images are not supported in this profile

22 Q. Should we permit other Linear/Area/Volume TIDs? Or are they adequately discussed in the referenced Whitepaper by David? Consider TID 1400 Linear Measurement, 1410 Area Measurement, 1402 Volume Measurement, 1404 Numeric Meas, 1406 3D Linear Meas

23 Q. How does a display know whether to display a polyline as a closed polygon? Do we inform creators that the display will not close the figure so they need to encode a last point the same as the first if that is their intent? http://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.10.5.html#sect_C.10.5.1.2

http://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.10.5.html#sect_C.10.5.1.2

http://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.10.5.html#sect_C.10.5.1.2



Template Rev. 10.4

24 Q. Is Contributing Equipment Sequence the best way to record algorithms? (In David’s “successive refinement” (See Figure XXXX.1-1. In Sup219), these are the things on the left side ftp://medical.nema.org/medical/dicom/supps/Frozen/sup219_fz_JSONSR.pdf) It has the advantage that it is present for all IODs and gives the display a uniform place to look for the information. Since it is at the instance level, should we highlight that all the results in the same SR instance should be from the same algorithm instance? And if not, how do we relate a specific result node to the corresponding entry in Contributing Equipment? Reference the UID? Segmentation IOD also has the Segmentation Algorithm Identification Sequence (0062,0007) which includes the Algorithm Identification Macro. That macro is missing Device UID (or equivalent) that would be needed to capture software instances. So an alternative approach would be to work the Algorithm Identification Macro (and/or SR Concept equivalents) into all the other result primitives. And whichever one we choose, should we add a place for PubMed references or is that getting too much into a database that should be external to results?

Closed Issues

Q. What is the scope of the profile? A: Results of image analysis that inform an imaging clinician during reading of the study This focus is a deliberate strategy to avoid scope creep. AI is a large space and there is much fun to be had.



Template Rev. 10.4

Q. How to capture confirmation/contradiction/adjustment of the result by a clinician? A: Out of Scope (but really important) Collecting such data may be critical to monitoring quality of results (particularly in the early stages) and to driving improvement of AI models through expert feedback. This was highlighted by both Luciano and Tessa. The mechanism would likely need to capture such accept/reject/(adjust) input during review and convey it to QA processes (similar model to dual-read discrepancy resolution?) and to the training environment for the AI model. The DICOM Verification Flag (0040,A493) is one possible recording mechanism. Consider addressing this in Phase II, and perhaps coordinating with AIW.

Q. Should we include a Secondary Capture primitive? A. No The goal of the profile is raising the bar to be interoperable/scalable. Including it would undercut the interoperability and shut down use of AI results to drive workflow, provide clinical decision support, populate the patient records, be included in clinical databases, etc. Beside, anyone that wants to store/display Secondary Capture doesn’t need a profile to help them do it. Counterpoint: Should we mention it as a “fall through” in the primitives section? Even if not part of the profile, a default “rendering” involves little/no integration cost for new algorithms, or for dumb displays generally. One might argue that algorithms will understand the nature of their results “best”, although they won’t be able to address the individual preferences of the radiologist and the “analog” result won’t work with databases, clinical decision support, etc., etc., etc.

Q. Is it unreasonable for a display to support the full set of primitives? A. No. It’s a bit of work, but it’s what is needed Without a full set of primitives, we don’t get effective interoperability. You can’t really know which primitives the AI Models will need/use so if “some of them” are unsupported it is disruptive. That said, there is an open issue to have two specific sets with a large set of basic primitives and a small set of advanced primitives.



Template Rev. 10.4

Q. Should Surface Segmentations and RT Structure Sets be included as Named Options? A. No. RT Structure Sets are fairly complex and are out of our primary radiology interpretation scope. Surface Segmentations (mesh-based) are also fairly complex and are not yet widely used in this space. Most segmentations we imagine Evidence Creators doing is contour or voxel oriented. When interest emerges, these could be added as Named Options.

Q. Should we help displays find summary/“entry points” into the collection of results? A. Yes. See X.4.1.2a Root Results An alternative might have been to have a result catalog object for each study that indexes all the results in the study, however continually revising the catalog object each time one of many algorithms stored new results to the study would be challenging, not to mention handling competing updates by when multiple algorithms happen to complete at the same time.

Q. Specify use of existing STOW to store existing JSON (or binary) of SR now? A. Not now. Wait until DICOM WG-27 makes the transaction. When creating the IHE transaction, consider whether it should be specific to AI Results or general for Evidence documents.

Q. Specify WADO-RS request to retrieve an SR, rendered into Sup219 representation? A. Not now. Wait until DICOM WG-27 makes the transaction.

Q. Include “Report snippets”? (text rendering of a result for inclusion in a report) A. No. Out of scope for this profile. Might consider as part of a reporting profile that looks at the reporting process in more detail.



Template Rev. 10.4

Q. Make a new storage transaction for non-image instances? A. No. Historically we’ve elided storage and retrieval of DICOM instances that contain data but are not strictly “images”. We have storage transactions for Images, Creator Images, Evidence Documents (just SR), Presentation States, Key Image Notes, Reports

Q. Is there anything significant to be borrowed from IHE Evidence Documents? A. Not really. We have already borrowed the Store/Q/R transactions. Otherwise, it describes use of SWF and PPWF but we will likely be using AIW. It does make a requirement that the product DCS list all the SOP Classes supported, but that doesn’t really seem to be an issue.

Q. Should SR requirements describe each of the Enhanced, Comprehensive, and Comprehensive 3D IODs A: No. Just Comprehensive 3D. Since “lesser” SR IODs are all valid instances of “higher” IODs, Evidence Creators can simply label all their output as Comprehensive 3D. Since the Image Displays and Consumers should be prepared to handle 3D coordinates, they shouldn’t mind and it removes a source of variability.

Q. Should existing DICOM CAD SR templates be described in this supplement? A. No, not normatively. Section X.6 “AIR Cross Profile Considerations” highlights the existing Mammo CAD and Chest CAD Profiles (although Chest CAD saw no uptake) that specify requirements on the mammo-specific and chest CAD-specific SR templates and associated display requirements. Interested Evidence Creators, Image Displays, and Image managers can support and claim those profiles. One could theoretically encode much of the content of those CAD IODs in TID 1500 and use them in this profile however that is not being specifically called out or analyzed here. If this profile is successful the committee could explore whether the benefits of harmonization/convergence would be worth the re-implementation costs.



Template Rev. 10.4

TO DO 185

Based on post-PC choices for Filtering and Navigation, maybe write a chunk of text like “Filtering Strategies” from REM to show implementers what attributes would be used by the Display to achieve the various filtering described in the Filtering Concepts section. Maybe.

Get a proper Coding Scheme Designator for Radiology Common Data Elements. Currently using 99RDE. Will they become part of LOINC like Radlex?

Get parallel codes for the RID Criticality codes The presumption is that the RID codes were intended to communicate that a clinician has determined a finding is Actionable and it is Urgent, so to communicate that an AI has made such a determination, alternate parallel codes would be needed. Alternatively, we could try make sure an observer type is attached to such codes in CDA, HL7 messages, and DICOM instances and train recipients to look at the observer type.

Diagram the structure of 1500 and show the simplest form of 1500 to use as a pointer to Parametric Map or whatever. (see Harald PPT as a starting point)

Plan Public Comment Outreach Prototyping – Jonathan, Chris L, Harald, AI Group, Tessa Encoding Result Confidence

• Andrei – will check with Director of Research (Yale)

• Kevin – Cleveland Clinic Biostat

• Chris – GE

• ACR – Neil and the model developers

• RSNA – Chris will poke some people.

Add definitions for Newcode0, Newcode1, etc.



Template Rev. 10.4

DICOM CPs for consideration

• Add a code for (Newcode1,99IHE,”Processing Algorithm”) to CID 7005

• Add Device UID (0018,1002) to Contributing Equipment Sequence. Need ability to identify an instance of a “device”. Consider adding UDI too since there is a possibility of FDA engagement and “recall use cases” that UDI was created to address.

• Add a code for “Saliency Image” to CID 7203 Image Derivation for use in parametric maps that are saliency images. Do we also need to reference an SR Finding Node that is the finding it is the saliency for? And is (121322,DCM,”Source image for image processing operation”) the appropriate purpose of reference when referencing the image it overlays on, or do we need something more specific?

• Add a Row 5 description to TID 1500 about what is intended to go in the image library (nothing is required) given that measurements reference the images they are on. Also explain in TID 1600 the intention of the structure with Entry Descriptors above and below the Entry (common and deviations; below overrides above?)

• Allow a 3D point in TID 1411. It currently allows an ellipsoid only. A point that is not exactly in the plane of a given frame (and associated findings) seems like a reasonable use case.

• Add an IMAGE reference below the qualitative evaluations in 1501 so that evaluations that are not measurements (which use TID 300) can also be associated with an image even though there isn’t a ROI (which is how TID 1410 and 1411 get around this)

• Add a code to CID 219 for “Centerline” that could be used for geometric purpose of polylines with Finding Sites like Spine, Abdominal aorta, etc.



Template Rev. 10.4

IHE Technical Frameworks General Introduction The IHE Technical Framework General Introduction is shared by all of the IHE domain technical frameworks. Each technical framework volume contains links to this document where appropriate. 190

9.0 Copyright Licenses IHE International hereby grants to each Member Organization, and to any other user of these documents, an irrevocable, worldwide, perpetual, royalty-free, nontransferable, nonexclusive, non-sublicensable license under its copyrights in any IHE profiles and Technical Framework documents, as well as any additional copyrighted materials that will be owned by IHE 195 International and will be made available for use by Member Organizations, to reproduce and distribute (in any and all print, electronic or other means of reproduction, storage or transmission) such IHE Technical Documents. The licenses covered by this Copyright License are only to those copyrights owned or controlled by IHE International itself. If parts of the Technical Framework are included in products that also 200 include materials owned or controlled by other parties, licenses to use those products are beyond the scope of this IHE document and would have to be obtained from that other party.

9.1 Copyright of Base Standards IHE technical documents refer to and make use of a number of standards developed and published by several standards development organizations. All rights for their respective base 205 standards are reserved by these organizations. This agreement does not supersede any copyright provisions applicable to such base standards. Copyright license information for frequently referenced base standards is provided below.

9.1.1 DICOM (Digital Imaging and Communications in Medicine) DICOM® is the registered trademark of the National Electrical Manufacturers Association for its 210 standards publications relating to digital communications of medical information.

9.1.2 HL7 (Health Level Seven) HL7®, Health Level Seven®, CDA®, FHIR®, and the FHIR [FLAME DESIGN] ® are registered trademarks of Health Level Seven International. Health Level Seven, Inc. has granted permission to IHE to reproduce tables from the HL7 215 standard. The HL7 tables in this document are copyrighted by Health Level Seven, Inc. All rights reserved. Material drawn from these documents is credited where used.

9.1.3 LOINC (Logical Observation Identifiers Names and Codes) LOINC® is registered United States trademarks of Regenstrief Institute, Inc.

http://ihe.net/Technical_Frameworks/#GenIntro



Template Rev. 10.4

9.1.4 SNOMED CT (Systematized Nomenclature of Medicine -- Clinical Terms) 220 Some IHE Profiles incorporate SNOMED® CT, which is used by permission of the International Health Terminology Standards Development Organisation. SNOMED CT© was originally created by the College of American Pathologists. SNOMED CT is a registered trademark of the International Health Terminology Standards Development Organisation, all rights reserved.

10 Trademark 225

IHE® and the IHE logo are trademarks of the Healthcare Information Management Systems Society in the United States and trademarks of IHE Europe in the European Community. They may only be used with the written consent of the IHE International Board Operations Committee, which may be given to a Member Organization in broad terms for any use that is consistent with the IHE mission and operating principles. 230



Template Rev. 10.4

IHE Technical Frameworks General Introduction Appendices The IHE Technical Framework General Introduction Appendices are components shared by all of the IHE domain technical frameworks. Each technical framework volume contains links to these documents where appropriate. 235

Appendix A – Actor Summary Definitions Add the following new or modified actors to the IHE Technical Frameworks General Introduction Appendix A:

240 New (or modified) Actor

Name Definition

Image Display A part of a system that can access imaging evidence objects (images, Presentation States, Key Image Notes, Evidence Documents) through network query/retrieve or reading interchange media and allow the user to view these objects presents medical images and associated imaging data.

The table below lists existing actors that are utilized in this profile.

Complete List of Existing Actors Utilized in this Profile Existing Actor Name Definition

Evidence Creator A system that creates additional evidence objects such as images, presentation states, Key Image Notes, and/or Evidence Documents and transmits them to an Image Archive. It also makes requests for storage commitment to the Image Manager for the data previously submitted. It may also retrieve worklist entries for post-processing steps from the Post-Processing Manager and provide notification of completion of the step, allowing the enterprise to track the status of post-processing work.

Image Manager / Image Archive A system that provides functions related to safe data storage and image data handling. It supplies image availability information to the Department System Scheduler. It is always grouped with an Image Archive to provide long term storage of images, presentation states, Key Image Notes, and Evidence Documents.

Imaging Document Consumer Retrieves imaging data based on references in a retrieved imaging manifest.

Appendix B – Transaction Summary Definitions 245

Add the following new or modified transactions to the IHE Technical Frameworks General Introduction Appendix B:




Template Rev. 10.4

New (or modified) Transaction Name and Number

Definition

Display Analysis Results [RAD-Y1] Presents one or more results of imaging analysis to an operator.

Appendix D – Glossary 250

Add the following new or updated glossary terms to the IHE Technical Frameworks General Introduction Appendix D.

New (or modified)

Glossary Term Definition

AI Result A result of performing clinical analysis in the context of an imaging procedure. The term is used in a broad sense that encompasses results generated by algorithms, such as conventional CAD (Computer-Aided Detection or Diagnosis) as well as those that use current neural network technologies.

AI Model Software that produces an AI Result.



Template Rev. 10.4

Volume 1 – Profiles Domain-specific additions NA 260

Add new Section #

X AI Results (AIR) Profile This AI Results Profile specifies how medical image analysis results can be reliably stored, retrieved, and displayed. The motivating use case involves results generated by artificial 265 intelligence (AI) algorithms, although the profile applies equally to non-AI-based analysis. This profile defines content for data encoding, transactions for moving that content around, and behaviors for basic handling of the content.

X.1 AIR Actors, Transactions, and Content Modules This section defines the actors, transactions, and/or content modules in this profile. General 270 definitions of actors are given in the Technical Frameworks General Introduction Appendix A. IHE Transactions can be found in the Technical Frameworks General Introduction Appendix B. Both appendices are located at http://ihe.net/Technical_Frameworks/#GenIntro Figure X.1-1 shows the actors directly involved in the AIR Profile and the relevant transactions between them. If needed for context, other actors that may be indirectly involved due to their 275 participation in other related profiles are shown in dotted lines. Actors which have a required grouping are shown in conjoined boxes (see Section X.3).




Template Rev. 10.4

Figure X.1-1: AIR Actor Diagram 280

Table X.1-1 lists the transactions for each actor directly involved in the AIR Profile. To claim compliance with this profile, an actor shall support all required transactions (labeled “R”) and may support the optional transactions (labeled “O”).

Table X.1-1: AIR Profile - Actors and Transactions Actors Transactions Initiator or

Responder Optionality Reference

Evidence Creator

Store Evidence Documents [RAD-43] Initiator O (See Section X.1.1.1) RAD TF-2: 4.43 Creator Images Stored [RAD-18] Initiator O (See Section X.1.1.1) RAD TF-2: 4.18 STOW AI Result [RAD-xxx] Initiator (Just to show where

it will fit later) Image Manager/ Archive

Store Evidence Documents [RAD-43] Responder R RAD TF-2: 4.43 Query Evidence Documents [RAD-44] Responder R RAD TF-2: 4.44 Retrieve Evidence Documents [RAD-45]

Responder R RAD TF-2: 4.45

Creator Images Stored [RAD-18] Responder R RAD TF-2: 4.18 Query Images [RAD-14] Responder R RAD TF-2: 4.14

↓ Store Evidence Documents [RAD-43] ↓ Creator Images Stored [RAD-18] ↓ STOW AI Result [RAD-xxx]

Evidence Creator

Image Manager / Image Archive

Image Display

Imaging Document Consumer

Display Analysis Result [RAD-Y1]

← Query Evidence Documents [RAD-44] → Retrieve Evidence Docs [RAD-45] ← Query Images [RAD-14] → Retrieve Images [RAD-16] ← QIDO-RS Query [RAD-129] → WADO-RS Retrieve [RAD-107]

← Query Evidence Documents [RAD-44] → Retrieve Evidence Docs [RAD-45] ← Query Images [RAD-14] → Retrieve Images [RAD-16] ← QIDO-RS Query [RAD-129] → WADO-RS Retrieve [RAD-107]



Template Rev. 10.4

Actors Transactions Initiator or Responder

Optionality Reference

Retrieve Images [RAD-16] Responder R RAD TF-2: 4.16 STOW AI Result [RAD-xxx] Responder (Just to show where

it will fit later) QIDO-RS Query [RAD-129] Responder R RAD TF-2: 4.129 WADO-RS Retrieve [RAD-107] Responder R RAD TF-2: 4.107

Image Display

Query Evidence Documents [RAD-44] Initiator O (See Section X.1.1.3) RAD TF-2: 4.44 Retrieve Evidence Documents [RAD-45]

Initiator O (See Section X.1.1.3) RAD TF-2: 4.45

Query Images [RAD-14] Initiator O (See Section X.1.1.3) RAD TF-2: 4.14 Retrieve Images [RAD-16] Initiator O (See Section X.1.1.3) RAD TF-2: 4.16 QIDO-RS Query [RAD-129] Initiator O (See Section X.1.1.3) RAD TF-2: 4.129 WADO-RS Retrieve [RAD-107] Initiator O (See Section X.1.1.3) RAD TF-2: 4.107 Display Analysis Result [RAD-Y1] Initiator R RAD TF-2: 4.Y1

Imaging Document Consumer

Query Evidence Documents [RAD-44] Initiator O (See Section X.1.1.4) RAD TF-2: 4.44 Retrieve Evidence Documents [RAD-45]

Initiator O (See Section X.1.1.4) RAD TF-2: 4.45

Query Images [RAD-14] Initiator O (See Section X.1.1.4) RAD TF-2: 4.14 Retrieve Images [RAD-16] Initiator O (See Section X.1.1.4) RAD TF-2: 4.16 QIDO-RS Query [RAD-129] Initiator O (See Section X.1.1.4) RAD TF-2: 4.129 WADO-RS Retrieve [RAD-107] Initiator O (See Section X.1.1.4) RAD TF-2: 4.107

285

X.1.1 Actor Descriptions and Actor Profile Requirements Most requirements are documented in RAD TF-2 Transactions. This section documents any additional requirements on profile’s actors.

X.1.1.1 Evidence Creator Evidence Creators represent a source of results. The scope of this profile begins when the 290 Evidence Creator has created a result to store. Methods to manage the Evidence Creator or allow it to obtain the inputs used in processing are not covered here. See the IHE AI Workflow Profile for such details. The actor may be implemented by an analysis or AI software package itself, or it may be a proxy, gateway, or partner system that encodes and transmits results on behalf of such software. 295 The profile does not distinguish between analysis executed locally, in the cloud, hosted in processing servers, or running in standalone workstations. For additional discussion, see Section X.4.1.5 AI Algorithm Deployment. Evidence Creators shall encode their results using the Result Primitives described in Section X.4.1.1. 300



Template Rev. 10.4

Note: Evidence Creators are only required to support the Result Primitives needed to encode their results. This means they might not need to support all Result Primitives.

Evidence Creators shall support the SOP Class(es) corresponding to the Result Primitive(s) they support, as defined in Section X.4.1.2 Result Primitive Encodings. Evidence Creators shall support a corresponding transaction listed in Table X.1-1 for each 305 primitive they produce. Evidence Creators may choose to support both the RESTful version of a transaction and the conventional DIMSE version of that transaction, or they may choose to support one or the other. Evidence Creators shall create at least one result for each case processed regardless of whether there are findings and regardless of whether the analysis is successful. In the case of failed 310 processing, the Evidence Creator shall report the reasons of failure in the result instance. In the case of failure, the Evidence Creator might create a different type of object, for example it might create an SR rather than an empty Segmentation.

X.1.1.2 Image Manager/Archive Image Manager/Archives store results (annotations, measurements, segmentations, etc.) and 315 make them available alongside the modality and human-generated results of a study. Image Managers shall support all the SOP Classes listed in Section X.4.1.2 Result Primitive Encodings.

X.1.1.3 Image Display Image Displays present results together with the other associated images and data relevant to a 320 study. Image Displays shall support the retrieval and display of all Result Primitives described in Section X.4.1.1. This means Image Displays are required to support a set of corresponding transactions listed in Table X.1-1, however Image Displays may choose to support both the RESTful version of a transaction and the conventional DIMSE version of that transaction, or 325 they may choose to support one or the other. Image Displays shall support the SOP Classes listed in Section X.4.1.2 Result Primitive Encodings.

X.1.1.4 Imaging Document Consumer Imaging Document Consumers make use of results in ways other than displaying them. 330 Imaging Document Consumers may include decision support systems, clinical databases, and report creators. An Evidence Creator might implement a grouped Imaging Document Consumer to access existing results as inputs. Although adding results directly to an EMR is out of scope for this profile, it is conceivable that a PACS or interface engine might implement an Imaging Document Consumer that selects, 335 extracts, and transcodes results for insertion into an EMR using an unspecified mechanism.



Template Rev. 10.4

Imaging Document Consumers shall support at least one of the Result Primitives described in Section X.4.1.1.

Note: Imaging Document Consumers are only required to support Result Primitives needed to support the intended functionality of the Image Document Consumer system. This means they might not need to support all Result 340 Primitives.

Imaging Document Consumers shall support the SOP Class(es) corresponding to the Result Primitive(s) they support, as defined in Section X.4.1.2 Result Primitive Encodings. Imaging Document Consumers shall support corresponding query and retrieval transactions listed in Table X.1-1 for each primitive they consume. Imaging Document Consumers may 345 choose to support both the RESTful version of a transaction and the conventional DIMSE version of that transaction, or they may choose to support one or the other.

X.2 AIR Actor Options Options that may be selected for each actor in this profile, if any, are listed in the Table X.2-1. Dependencies between options, when applicable, are specified in notes. 350

Table X.2-1: AI Results – Actors and Options Actor Option Name Reference

Evidence Creator Parametric Map Option Section X.2.1 Image Manager / Archive No options defined -- Image Display Parametric Map Option Section X.2.1 Imaging Document Consumer No options defined --

X.2.1 Parametric Map Option This option isolates more difficult to implement display requirements. The display requirements described in RAD TF-2: 4.Y1.4 are expected to be reasonably implementable by most Image Displays. There are a few capabilities that would be useful for full 355 reading workstations to implement but might not be implemented by more basic viewers. This option communicates which Image Displays support these advanced functions and which Evidence Creators will need to be paired with those advanced Image Displays. An Image Display that supports this option shall support display of the following primitives as described in RAD TF-2: 4.Y1.4.1.3 360

• Parametric Maps An Evidence Creator that supports this option shall be capable of using Parametric Map IODs for storing some of its results.



Template Rev. 10.4

X.3 AIR Required Actor Groupings An actor from this profile (Column 1) shall implement all of the required transactions and/or 365 content modules in this profile in addition to all of the requirements for the grouped actor (Column 2) (Column 3 in alternative 2). Section X.5 describes some optional groupings that may be of interest for security considerations and Section X.6 describes some optional groupings in other related profiles.

Table X.3-1: AI Results - Required Actor Groupings 370 AIR Actor Actor(s) to be grouped

with Reference Content

Bindings Reference

Evidence Creator ITI CT / Time Client ITI TF-1: 7 Image Manager / Archive None Image Display None Imaging Document Consumer None

X.4 AIR Overview

X.4.1 Concepts

X.4.1.1 Result Primitives It is unrealistic to expect radiology displays to implement specific display capabilities for each of 375 the myriad of algorithms that have been developed and will continue to be developed. To make the implementation complexity for displays manageable, and to avoid needing different software for each new AI result, this profile follows the direction set by DICOM WG-23 to determine a reasonable, finite, set of primitive elements from which AI results can be composed. Evidence Creator actors are required to render their results using the defined set of primitives 380 encoded in specific DICOM SOP Classes (See RAD TF-1: X.4.1.2 Result Primitive Encodings) and Image Display actors are required to support basic presentation of those primitives (See RAD TF-2: 4.Y1.4.1 Display Results).

Notes: As with all IHE specified functionality, Evidence Creators that conform to the profile are free to have additional modes that include encoding results any way they like. 385

The current set of primitives is comprised of the following: Qualitative Findings This primitive is a coded concept paired with a coded value. This structure is common in DICOM SR instances. Algorithms might use this for results like flagging presence/absence of a particular finding, 390 categorizing structures in an image, staging pathologies, etc.



Template Rev. 10.4

Examples of SR Concept-Value encoding include:

• (17204006,SCT,"Pneumoperitoneum”) = (272519000,SCT,”Absent”)

• (129715009,SCT,"Breast composition") = (129719003,SCT,”Extremely Dense”) See Section X.4.1.7 for a discussion of codesets for concepts and values. 395 Measurements This primitive is a structure containing a coded concept paired with one or more numerical values and associated details like measurement units, the location of the measurement, etc. This structure is common in DICOM SR instances. Algorithms might use this for results like measured tumor sizes, blood flow rates, etc. 400 Examples include:

• (81827009,SCT,”Diameter”) = "26.43","mm"; where (G-C0E3,SRT,"Finding Site") = (54247002,SCT,”Ascending Aorta”)

• (410668003,SCT,"Length") = "52","mm"; where (121071,DCM,"Finding") = (27925004,SCT,”Nodule”) and (G-C0E3,SRT,"Finding Site") = 405 (23451007,SCT,”Adrenal Gland”)

• (RDE264,99RDE,"Cobb Angle") = "18.4","deg"; where (G-C0E3,SRT,"Finding Site") = (122495006,SCT,”Thoracic spine”)

• (RDE265,99RDE,"Pelvic Tilt") = "3.2","deg" Note: Other details about the Cobb Angle measurement, like the specific vertebra selected to derive the Cobb Angle and 410

whether the top or bottom surface was chosen, might be encoded as additional subordinate results.

Locations This primitive is a spatial location, in the form of coordinates for a point, a line, or a bounding shape like a bounding box or ellipse. The coordinates are typically expressed in an imaging coordinate system in the frame of reference of an imaging dataset. The coordinates may be 415 within a single “slice” (2D) or within an imaged volume (3D). Algorithms might use this for results like the location of a tumor or calcification, a spatial fiducial point, or the tip of an instrument, needle, or tube. Segmentations The primitive is a segmented 2D or 3D region of an imaging dataset. 420 Algorithms might use this for results like identifying pixels/voxels that are part of a lesion, or the boundary of a particular organ, or the extent of an intracranial bleed. Segmentations would also be used for results like tissue classifications (single or multiple) and individual segments may be discontiguous. Parametric Maps 425



Template Rev. 10.4

This primitive is a set of 2D (or 3D) pixels (or voxels) with a spatial correspondence to the source image. For a given parametric map, the values will either be all floating point, all double, or all integer. Algorithms might use this for results such as a “saliency map” (showing which parts of the source image “weighed significantly” on an AI-generated finding like presence of a 430 pneumothorax), or a probability map (showing the probability of a finding like malignant cells being true in each pixel of the image) or quantitative analysis (showing calculated values like brain perfusion, apparent diffusion coefficients (ADC), or an angiogenesis map based on a series of Dynamic Contrast Enhanced images that encode contrast uptake and washout for each pixel).

Note: Tissue classifications may be more appropriately handled as segmentations. 435 Tracking IDs This primitive is an identification of a persistent entity in the form of a unique identifier. Algorithms might use this to identify the same specific tumor in a specific patient in each of a longitudinal set of imaging studies over time. When the same Tracking ID is associated with other primitives like location coordinates, measurements, or qualitative findings (like tumor 440 stage), it facilitates longitudinal analysis or evaluation of change. Image References This primitive is a reference to one or more image instances, in the form of an instance UID, possibly with information to facilitate accessing the instance such as a Study UID or possible sources like a retrieval AE Title, or a retrieval URL. When there is need to be more specific for a 445 multi-frame instance, the reference may include frame information. Algorithms might use this for results such as identifying (with references) other images that the reading radiologist may find relevant to the current image. The relevant images may be prior images of the current patient, images from other patients with comparable pathology, images from a reference normal population, or images from an atlas. 450 Algorithms are also expected to commonly reference the “input” image that was analyzed, however that is expected to be part of the result metadata, rather than the result “payload”.

X.4.1.2a Root Results < This is a new section but most of the semantic content existed previously in Section X.4.1.3. The following Section X.4.1.2 will move to a normative location (X.4.* is not normative) at which 455 point Section X.4.1.2a will become Section X.4.1.2.> As described in Section X.4.1.3, the use of multiple algorithms may produce very large collections of results for a given study. To provide Image Displays with information to help them organize and prioritize the result “entries” listed/communicated to the operator at one time, this profile introduces Root Results that communicate a basic hierarchy for a set of related results. 460 For example, a lung screening algorithm might record a LungRADS™ score as a Root Result, supported by secondary results consisting of multiple detected nodule locations and assessments of the size, solidity, and margin of each detected nodule. Another algorithm might record an SR



Template Rev. 10.4

finding of “pneumonia present” as a root result with a reference to a separate saliency map instance. 465 Root results are SR instances that have a document title of (Newcode0,99IHE,”Analysis Root Result”) allowing consumers like Image Displays to identify them in a study. Root result instances then contain subordinate results (e.g., measurements and locations) and/or reference other instances that contain subordinate results. In the case of the two examples above, an Image Display might initially present the two root 470 results (LungRADS = Category 3) and (Pneumonia present) followed by “drill down”, rather than initially presenting 43 components consisting of location, size, solidity, margin, and LungRADS for each of 8 nodules, an overall LungRADS score, the pneumonia finding, and the pneumonia saliency map reference. Potential behaviors are discussed further in Section X.4.1.3. 475

X.4.1.2 Result Primitive Encodings This section specifies standard encodings of the result primitives introduced in Section X.4.1.1. <This will go somewhere normative (like X.1.1, or a 6.5 DICOM Content Definition for Result Primitives?) but for now this is a convenient location to work on it>

Notes: 1. For extensive specific guidance on topics such as encoding planar measurements and finding-related coordinates in 480 SR, it is strongly recommended that readers refer to: Clunie, D., DICOM SR for communicating planar annotations, An Imaging Data Commons (IDC) White Paper, 2019 https://docs.google.com/document/d/1bR6m7foTCzofoZKeIRN5YreBrkjgMcBfNA7r9wXEGR4/edit?usp=sharing

2. The use of the word “image” in this section is intended to mean both a single-frame image IOD and a specific frame of a multi-frame image IOD. 485

X.4.1.2.1 General Result Encoding Requirements The Evidence Creator is expected to populate much of the contextual metadata (e.g., patient demographics, patient identifiers and issuers, study accession number, etc.) in the result instances based on those values in the medical imaging data being processed, and/or the UPS Workitem (if the Evidence Creator is being driven by AI Workflow for Imaging). 490 Per DICOM, the Evidence Creator will describe itself in each result instance in the attributes of the General Equipment Module. The Evidence Creator shall describe each AI Model/algorithm that was used to generate the results in the Contributing Equipment Sequence (0018,A001). Multiple items may be included. The Evidence Creator shall encode the following details in the Contributing Equipment 495 Sequence:

• Purpose of Reference Code Sequence (0040,A170) shall be (Newcode1,99IHE,”Processing Algorithm”)

• Manufacturer (0008,0070)

• Manufacturer’s Model Name (0008,1090) 500

https://docs.google.com/document/d/1bR6m7foTCzofoZKeIRN5YreBrkjgMcBfNA7r9wXEGR4/edit?usp=sharing



Template Rev. 10.4

• Software Versions (0018,1020)

• Device UID (0018,1002) Notes: 1. The Device UID is intended to capture the state of the AI Model, so each time the AI Model is modified by training

it would be appropriate to update the Device UID.

2. For algorithms, the same value will appear in Manufacturer’s Model Name (0008,1090) and the corresponding 505 (111001, DCM, “Algorithm Name”) content item, if present. Similarly, the same value will appear in Software Versions (0018,1020) and the corresponding (111003, DCM, “Algorithm Version”) content item, if present. The (111001, DCM, “Algorithm Name”) and (111003, DCM, “Algorithm Version”) concepts can be found in TID 4019.

In SR Instances, the Evidence Creator will likely replicate the above information in items in the Author Observer Sequence (0040,A078). 510 Each result shall either be contained in a Root Result instance or contained in an instance referenced from a Root Result instance. See Section X.4.1.2a

Note: Evidence Creators may decide whether their output is best represented with a single root result or multiple. Evidence Creators are not required to reference results from other Evidence Creators.

Root Result instances shall be encoded in a DICOM SR instance using TID 1500 (Measurement 515 Report) as the root template and the document title encoded in the Concept Name of the root container shall be (Newcode0,99IHE,”Analysis Root Result”). Subordinate result instances shall be referenced from their Root Result instance in the Current Requested Procedure Evidence Sequence (0040,A375).

Notes: 1. Subordinate result instances may be referenced from other locations in the Root Result as appropriate. 520 2. For Root Result instances that are only referencing instances of the Parametric Map IOD or a segmentation IOD

without any associated measurements or observations, TID 1501 may be included in TID 1500 without any content other than the Measurement Group container and the tracking IDs.

DICOM SR Guidance Content Date (0008,0023) and Content Time (0008,0033) reflect the datetime that the analysis 525 was executed that generated the result(s). As an inherent part of the SR Content Tree encoding, each Content Item has an Observation UID (0040,A171). Observation UIDs may be used to reference specific findings.

Note: A Tracking Unique Identifier uniquely identifies the entity that is the subject of an observation (e.g., this patient’s aortic valve, or a specific lesion), while an Observation UID uniquely identifies the observation itself (e.g., a diameter 530 measurement of that entity taken today at 2pm). So, a number of observations of the same entity will all have different Observation UIDs but the same Tracking Unique Identifier.

Frame of Reference (FoR) Guidance DICOM instances that encode spatial information typically include a Frame of Reference UID (0020,0052) that identifies the origin and coordinate system of coordinates in the instance 535 (without specifically defining that origin and coordinate system). When two instances share the same Frame of Reference UID, coordinates in the two instances are in the same coordinate space. Evidence Creators are strongly encouraged to use shared Frame of Reference UIDs whenever appropriate to avoid placing the burden on downstream systems of performing otherwise 540



Template Rev. 10.4

unnecessary registrations and transformations. Since the coordinate systems are typically patient-relative, any change of the patient position or use of a different device would result in a new Frame of Reference UID.

Note: Some DICOM Image IODs (like CR images or Ultrasound images) do not necessarily contain a Frame of Reference UID. If DICOM Segmentation IODs are created for such images, the segmentation pixels are required by DICOM to 545 correspond directly to the pixels of the segmented image. In that situation, the segmentation and the image are implicitly in the same Frame of Reference, and if the DICOM Segmentation IOD declares a Frame of Reference UID, the image may then be assumed to be in the same Frame of Reference.

X.4.1.2.2 Qualitative Findings Qualitative findings shall be encoded in a DICOM SR instance using TID 1500 (Measurement 550 Report) as the root template.

• http://dicom.nema.org/medical/dicom/current/output/chtml/part16/chapter_A.html#sect_TID_1500

The value for Procedure Reported (121058, DCM, "Procedure reported") should describe the imaging procedure analyzed, not the algorithm used. 555 Depending on whether the qualitative findings refer to a planar region of an image, a volume, or does not refer to any imaging, the root template will contain, respectively, TID 1410 (Planar ROI Measurement and Qualitative Evaluations), TID 1411 (Volumetric ROI Measurement and Qualitative Evaluations), or TID 1501 (Measurement and Qualitative Evaluations). The CONTAINER for (C0034375, UMLS, "Qualitative Evaluations") in TID 1500 shall not be used. 560



• http://dicom.nema.org/medical/dicom/current/output/chtml/part16/chapter_A.html#sect_565 TID_1501

Notes: 1. A qualitative finding that refers to an entire image is encoded as a planar region that spans the entire image.

2. TID 1411 supports encoding a simple ellipsoid Volume Surface which may be used to “encapsulate” an entity like a tumor without specifically delineating its surface.

Image references, per DICOM TID 1410 and 1411, are mandated for the ROIs that are the basis 570 of the measurements, either directly by referencing an image, or indirectly by referencing a Segmentation instance which then references the image it segments, or by referencing a Series to reference all the constituent images. Image references are currently defined for qualitative evaluations in TID 1501 (just for measurements). 575 If multiple qualitative findings apply to the same entity (see Section X.4.1.6 Tracking IDs), those findings may be encoded in a single invocation of TID 1501, TID 1410, or TID 1411.

http://dicom.nema.org/medical/dicom/current/output/chtml/part16/chapter_A.html#sect_TID_1500










Template Rev. 10.4

A single SR instance may contain multiple invocations of multiple sub-templates (TID 1501, TID 1410, and TID 1411).

X.4.1.2.3 Measurements 580 Measurements shall be encoded in a DICOM SR instance using TID 1500 (Measurement Report) as the root template. Depending on whether the measurements reflect a planar region of an image, a volume, or are measurements that are not tied to a planar region or volume, the root template will contain, respectively, TID 1410 (Planar ROI Measurement and Qualitative Evaluations), TID 1411 585 (Volumetric ROI Measurement and Qualitative Evaluations), or TID 1501 (Measurement and Qualitative Evaluations).

Note: A measurement of an entire image is encoded as a planar region that spans the entire image.

Image references, per DICOM TID 1410 and 1411, are mandated for the ROIs that are the basis of the measurements, either directly by referencing an image, or indirectly by referencing a 590 Segmentation image which then references the image it segments, or by referencing a Series to reference all the constituent images. Image references for measurements in TID 1501 shall be encoded in the invocation of TID 320 “Image or Spatial Coordinates” inside the invocation(s) of TID 300 “Measurements” that encode the measurements inside TID 1501. 595 If multiple measurements apply to the same entity (see Section X.4.1.6 Tracking IDs), they may be encoded in a single invocation of TID 1501, TID 1410, or TID 1411.

Note: For systems that use NCI AIM (Annotation and Image Markup) as an internal representation of annotations such as results or labels, DICOM PS3.21 provides extensive guidance on transcoding AIM observations into DICOM SR.

The anatomy or device that is the subject of each measurement is encoded in (363698007, SCT, 600 "Finding Site"). See Section X.4.1.7 for a discussion of codesets for Finding Site.

Note: Whether using TID 1501 that includes TID 300, or TIDs 1410 and 1411 that include TID 1419, (363698007, SCT, "Finding Site") may be encoded individually as a concept modifier for each measurement, or if all the measurements in the Measurement Group share the same Finding Site, it may be encoded once as a concept modifier of the measurement group. 605

A specific pathology (e.g., (86049000,SCT,"Neoplasm Primary")) being measured at the Finding Site may be encoded in (121071,DCM,"Finding"). Implementations are encouraged to follow the pattern of encoding the anatomical location in Finding Site, the “pathology”, if appropriate, in Finding, and the rest of the semantics into the Concept Name of the measurement. Some examples are shown in Table X.4.1.2.3-1. 610

Table X. 4.1.2.3-1: Measurement Encoding Examples Finding Site Finding Measurement Concept

(54247002,SCT,”Ascending Aorta”) n/a (81827009,SCT,”Diameter”) (122495006,SCT,”Thoracic spine”) n/a (RDE264, 99RDE,“Cobb Angle”) (23451007,SCT,”Adrenal Gland”) (27925004,SCT,”Nodule”) (410668003, SCT,"Length")



Template Rev. 10.4

(64033007,SCT,"Kidney") (86049000,SCT,"Neoplasm Primary") (118565006,SCT,"Volume")

Receiving systems might be less likely (particularly for all the different measurements they may receive) to correctly handle all the different alternative encodings like: Finding Site=Adrenal Gland, Measurement=Nodule Length, or Finding=Adrenal Gland Nodule, Measurement=Length, 615 or Finding Site=Adrenal Gland Nodule, Measurement=Length, or Measurement=Adrenal Gland Nodule Length, etc.; however, these are not prohibited by the profile.

X.4.1.2.4 Locations The SCOORD and SCOORD3D values with which locations are encoded are within the Frame of Reference of the instance which is required to be formally declared in some IODs and is 620 implicit in others. See the Frame Of Reference Guidance in Section X.4.1.2.1 for further details. Point locations shall be encoded in a DICOM SR instance using TID 1500 (Measurement Report) as the root template. If the point location is a 2D point on a single image, the instance will contain an SCOORD (111030, DCM, "Image Region") of Graphic Type POINT in TID 1410 (Planar ROI 625 Measurement and Qualitative Evaluations). If the point location is a 3D point in an image volume, the instance will contain an SCOORD3D (111030, DCM, "Image Region") of Graphic Type POINT in TID 1411 (Volumetric ROI Measurement and Qualitative Evaluations). Line locations shall be encoded in a DICOM SR instance using TID 1500 (Measurement Report) as the root template. 630 Depending on whether the line location is a sequence of 2D points on a single image, or a sequence of 3D points in an image volume, the instance will contain, respectively, an SCOORD (111030, DCM, "Image Region") of Graphic Type POLYLINE in TID 1410 (Planar ROI Measurement and Qualitative Evaluations), or an SCOORD3D (111030, DCM, "Image Region") of Graphic Type POLYLINE in TID 1411 (Volumetric ROI Measurement and Qualitative 635 Evaluations). Planar and volumetric locations may also be encoded as segmentations, as described in Section X.4.1.2.5. The (363698007, SCT, "Finding Site") and (121071,DCM,"Finding") may be populated to communicate the finding that the location represents. If the location represents a device or a 640 piece of anatomy, then just the Finding Site can be populated. Whether the location represents a center point, an outline of a feature, or a rough bounding box/ellipse, can be encoded in (130400,DCM,"Geometric purpose of region") in TID 1410 or 1411. When an anatomical location is known, with or without any spatial coordinates, it is encoded 645 using Finding Site with the associated finding. For example, to localize a detected cerebral hemorrhage to the left temporal lobe without coordinates, a qualitative finding that a cerebral



Template Rev. 10.4

hemorrhage is present would have a Finding Site of (78277001,SCT,”Temporal Lobe”) and a Laterality of (7771000,SCT,”Left”).

X.4.1.2.5 Segmentations 650 Pixel-based or voxel-based segmentations (also referred to as region-based segmentations) shall be encoded as one or more DICOM Segmentation instances. Contour-based segmentations on individual frames shall be encoded as a DICOM SR instance using TID 1500 (Measurement Report) as the root template and either containing TID 1410 (Planar ROI Measurement and Qualitative Evaluations) with an SCOORD (111030, DCM, 655 "Image Region") of Graphic Type POLYLINE or containing TID 1411 (Volumetric ROI Measurement and Qualitative Evaluations) with an SCOORD (111030, DCM, "Image Region") of Graphic Type POLYLINE or an SCOORD3D (111030, DCM, "Image Region") of Graphic Type POLYGON.

Note: Segmentation instances may be referenced in the relevant row of the corresponding template when qualitative findings 660 or measurements are based on the segmentation. See Qualitative Findings and Measurements above.

X.4.1.2.6 Parametric Maps Parametric Maps shall be encoded in a DICOM Parametric Map instance. <Will consider adding specific attribute guidance based on the outcome of the DICOM CP about CID 7203 codes like “Saliency Image”> 665 For Parametric Maps that represent saliency images for other findings:

• That instance containing that other finding shall reference both the Parametric Map instance, and the Image instance to which the finding pertains. o The use of the (130401, DCM, "Visual explanation") concept for the reference to the

Parametric Map is encouraged. 670 For Parametric Maps that are intended to be available to overlay on another underlying image (e.g., the image from which the parametric map was derived), the Parametric Map shall use the Frame of Reference of the image on which it is intended to be overlaid. The Parametric Map shall be encoded with the same orientation as the underlying image, but is permitted to have different pixel spacing, so the Image Display will not have to retrieve registration objects and 675 perform transformations but may have to perform resampling. These requirements are intended to facilitate fused display by the Image Display without requiring sophisticated registration capabilities.

Note: This applies to all saliency maps since they represent “areas of interest” in an underlying image.

Blending Presentation State objects and Advanced Blending Presentation State objects may be 680 created to encode fusion overlay parameters preferences, but the Image Display may or may not support those.



Template Rev. 10.4

X.4.1.2.7 Tracking IDs Tracking IDs for persistent entities shall be encoded in DICOM SR instances using concept (112039, DCM, "Tracking Identifier") for human readable Tracking IDs and concept (112040, 685 DCM, "Tracking Unique Identifier") for globally unique Tracking IDs (UIDs). Such IDs allow correlating multiple different observations of the same entity (such as an organ or lesion). If a measured entity is a known entity (e.g., it is a particular tracked lesion), TID 4108 (Tracking Identifier) shall be included in the invocation of TID 300 for the measurement and the Tracking ID(s) of the measured entity encoded therein. 690

Note: If there is a need to track specific observations (the diameter measurement recorded at 2:59pm on Aug 21 of lesion 45837459), each Content Item in the SR tree may have an Observation UID (and an Observation Datetime) associated with it as described in DICOM PS3.3 Table C.17-6.

The TID templates mandated here for Qualitative Findings and Measurements require that both a Tracking Identifier and a Tracking Unique Identifier be provided to identify the entity that is 695 subject of each Measurement Group.

X.4.1.2.8 Image References Image references that comprise the result itself (for example identifying an image this is most representative of a tumor, or is a relevant prior) shall be encoded as a Qualitative Finding as described in Section X.4.1.2.2 with a reference to the image. 700 Image references that are subordinate to other results (e.g., references to images on which a measurement was taken, or references to the images that were processed to create a segmentation or parametric map) are discussed above in the relevant result primitive section.

X.4.1.3 Result Filtering and Navigation As AI adoption increases, a given study may be subjected to many analyses, potentially from 705 different vendors, leading to many AI-generated results. Consider a chest CT that is acquired and evaluated by algorithms that produce a collection of results, including findings or observations about:

• Cracked ribs,

• Pneumothorax, 710

• Pneumonia,

• Scoliosis,

• Aortic aneurysms,

• Coronary plaque, and

• Lung lesions 715

Further, some results may consist of multiple components; e.g., a finding that a lung nodule is present, a location for the nodule, a segmentation of its boundary, measurements of its



Template Rev. 10.4

dimensions and volume, an overall nodule classification as benign/suspicious/malignant/etc., a Key Object Selection indicating the image slice most representative of the nodule, etc. The software used to review imaging studies (studies which will now include all these analyses, 720 both conventional and AI-based) will need to address the challenge of helping the radiologist be aware of the available information and review it as appropriate, while simultaneously avoiding the addition of extra time and tangible steps that are anathema in the imaging interpretation process.

Note: The display of individual result primitives is directly addressed by this profile in the form of baseline requirements in 725 [RAD-Y1]. Specific discovery or navigation behaviors by the Image Display are currently left to product design and not addressed by this profile; however, the profile will indirectly address the related tasks of filtering and navigating the available results by trying to ensure sufficient relevant metadata is available in the result objects. A “zero-click interface” would certainly be appealing, but this profile will not attempt to design one.

The following “Hypothetical Behavior” sections are intended to support discussion of possible 730 mechanisms that would help imaging clinicians. Those discussions may identify certain key metadata that would support such mechanisms and MAY result in requirements, constraints, or recommendations for Evidence Creators. These hypothetical behaviors are not display requirements in this profile. Hypothetical Behavior: Result “Hanging Protocols” 735 One could imagine an Image Display allowing users to configure rules and behaviors that controlled what results were initially displayed and how they were presented. This is analogous to conventional hanging protocols which use rules to control what images are presented and how they are organized on the available displays. A presumption in this hanging protocol discussion is that none of the results in the study would 740 be inaccessible to the imaging clinician, rather some results would be automatically arranged and presented, while others might require interactions by the imaging clinician to access. Based on experience in mammography CAD, where the rate of false positives presented a significant challenge, some forms of result filtering will be necessary to make this usable. Some conceivable factors that might be incorporated into rules might include: 745

• Whether the result is “normal” or “abnormal” (e.g., might not initially display results that are normal or unremarkable)

• The algorithm “confidence”, perhaps in terms of the sensitivity/specificity of the algorithm, or the positive/negative predictive value, or in terms of some generated metric (e.g., might not display results that do not exceed a certain threshold, and/or might 750 highlight results with a particularly high confidence)

• The criticality of an abnormal result (e.g., might display a result with high criticality, perhaps even if it was below the normal confidence threshold). IHE Results Distribution highlights the RADLEX codes for observation categories (from most to least severe): o RID49480^Category 1 Emergent Actionable Finding^RadLex 755 o RID49481^Category 2 Urgent Actionable Finding^RadLex



Template Rev. 10.4

o RID49482^Category 3 Noncritical Actionable Finding^RadLex o RID50261^Non-actionable^RadLex o RID13173^Normal^RadLex (synonym “Unremarkable”)

• Whether the result has been reviewed/verified/approved by a human (e.g., might display 760 results a resident has “approved/verified/confirmed”, perhaps even if it was below the normal confidence threshold).

• The type of imaging procedure or the anatomical focus (e.g., might suppress cardiac results that are normal when viewing a lung study but displayed them in a cardiac study, or might provide an organ-based summary of findings) 765

• The algorithm make/model/version used (e.g., an imaging clinician might be evaluating a particular algorithm and want to see the results of that algorithm for all studies during the period of evaluation; or an imaging clinician might not have confidence in a particular algorithm and not want those results “cluttering” the display)

• Whether the result is a “root result” (see Section X.4.1.2a Root Results) or a subordinate 770 finding (e.g., might display the root result that pneumothorax is present or a lesion was detected, but suppress the segmentation showing the lesion or pneumothorax location unless requested).

• Whether the algorithm completed successfully (e.g., AI Models that failed or only partially completed might still generate results useful for evaluating its performance, or 775 simply knowing that it was run; results created with a failure status would not normally be displayed). Based on experience in mammography CAD, the following states might be considered: o Successfully processed o Partially processed; incomplete 780 o Processing failed; unsupported data (e.g., wrong body part or modality) o Processing failed; modality model not validated o Processing failed; low confidence

• The relative date/time of multiple results (e.g., current results might be presented next to prior results, and/or differentiated from a current result from re-processing a prior image) 785 o Note that the date/time of the results reflects when they were created, as distinct from

the date/time of the images from which the results were generated

• The approval status of the algorithm (e.g., might not initially display the results of an algorithm approved for use as a “second reader” since that would invalidate its intended mode of use, or during normal clinical reading might not initially display the results of a 790 research algorithm that is not yet approved).



Template Rev. 10.4

• The nature of the current review (e.g., might present the “worst” case of several conflicting results when performing triage, while normal reading workflow might prioritize differently or might present both results.)(Reason for Requested Procedure, or Reason for Admission)? 795

• Relationships between results (e.g., for a primary finding that a tumor is present, the related results that show the segmented surface of the tumor, and the numerical measurements of the dimensions and volume might be displayed together, or the primary finding might be displayed with an indicator that there are secondary/supporting results that can also be displayed. In addition to direct references between results, relationships 800 can also be inferred by results sharing the same tracking id, the same Finding Site, or the same Frame of Reference)

Note: Current DICOM Hanging Protocols do not address such result filtering and organization, however, given sufficient interest, a new DICOM Result Hanging Protocol IOD, or a results extension to the existing Hanging Protocol IOD could be evaluated. This would facilitate centralizing, managing, and exchanging such protocols. 805

Rules might also control how results are grouped or formatted on the display, for example grouping abnormal findings together, or grouping cardiac results separately from lung results, or grouping results that came as a set from a particular algorithm. An additional complexity is that a study might contain conflicting results. Consider a general chest x-ray AI Model that evaluates 6 conditions, which determined that cracked ribs were 810 absent, and another special purpose AI Model (which might be considered to be more sophisticated) which determined that two cracked ribs were present. What if the findings were reversed, with the more specialized model indicating no fractures and the less specialized model indicating fractures were present? Perhaps the result with the higher confidence would be displayed with an indication that conflicting results are present. 815 SR instances from different Evidence Creators will be in different series which may provide additional clues for the Image Display. Hypothetical Behavior: Layers of Detail, Result Summarization It seems likely that displays might leverage component hierarchy by first presenting a summary result or key value to an imaging clinician and offer the ability to explore additional layers of 820 detail as needed, for example to next review the segmentation that underlies a volume measurement, or initially display the above mentioned LungRADS score, but allow the imaging clinician to choose to expose the nodule evaluations it was based on. This exploration might be done to gain greater confidence in the summary result, or to comprehend more details and nuances of the finding(s). 825 While root results capture a hierarchy within a set of results that were generated together, more advanced logic or analysis software might be able to create summary findings that encompass results from different algorithms or prioritize all results for a given study. The Root Results described in Section X.4.1.2a allow a display to use a mechanical filter (on Document Title) to get a first-order set of summary findings. The references in each of those 830 Root Results provides a logical next layer of detail. Ideally, some displays will develop much



Template Rev. 10.4

more sophisticated analysis and logic, or more advanced configurations, and more advanced navigation and display, while the Root Results provide a first simple step up from the flat list of findings. Hypothetical Behavior: AI Model Rendering Intent 835 Mammography CAD allows the analysis algorithm to make decisions that affect the behavior of the Image Display by flagging specific findings as having a Rendering Intent of either Presentation Required, or Presentation Optional. Evidence Creators could also encode an indication of the criticality of an abnormal result (see above) as a way to influence display behaviors. 840 Such an approach could make some display behaviors consistent for the same result across different displays. This supports some “centralization” of some display logic since it is configured/determined at the algorithm, rather than at each individual display. On the other hand, when there are many results, the problem remains of prioritizing/sequencing multiple AI models all indicating presentation required. 845 Hypothetical Behavior: Longitudinal Navigation Longitudinal results (the same measurement or evaluation performed at several points in time) present another axis of result relationships commonly of interest to imaging clinicians. For example, the change in size of a given tumor or the stability of a stenosis grade for a given vessel may be of interest. 850 Tracking IDs provide a mechanism for indicating that the subject of two or more measurements or evaluations is the same entity. Whether such longitudinal results are listed, graphed, scrolled through, or shown side-by-side, are design choices left to Image Display implementations. Some proposed display requirements have been included in RAD TF-2: 4.Y1.4.1.3.7 Display of Tracking IDs. The value of Finding Site may also be useful to correlate findings or results from 855 different timepoints that may be related due to being at the same finding site. Note that finding sites may describe anatomy, devices (stents, pins, biopsy clips, etc.), or, when associated with Finding, a pathology.

X.4.1.4 Result Presentation This profile places a number of basic presentation requirements on the Image Display in the 860 Display Analysis Result [RAD-Y1] transaction. Due to the potential variability of AI Results, even when narrowed to medical imaging interpretation as they are in this profile, it is not possible to prescribe all potentially useful capabilities so details about how results and the associated images are presented is largely left to the implementation. Further, there is significant potential for display products to devise and 865 provide display capabilities that present the results in ways that very effectively support the interpretation process. This profile is not intended to discourage such functions in any way. Image Display implementations might consider some of the display capabilities described in the DICOM Volumetric Presentation States. In those objects, the Graphic Annotation Module



Template Rev. 10.4

includes Tracking UID (0062,0021) which can be used to reference an entity in a segmentation 870 or Structured Report instance to associate it with an annotation, and the Volumetric Graphic Annotation Module includes Referenced Structured Context Sequence (0070,1903) which can be used to reference to a node in a Structured Report instance to associate it with an annotation.

X.4.1.5 AI Algorithm Deployment This profile places requirements on the Evidence Creator, primarily about how results are 875 encoded and submitted for storage. The profile does not constrain the types of technology used by AI algorithms or how AI algorithms are integrated with the Evidence Creator. Some products may choose to implement the Evidence Creator directly in the same software package as the AI algorithm. It is also possible that the Evidence Creator will be implemented by a platform product (such as an AI marketplace) and one or more AI algorithms are then integrated with that 880 platform. Some implementations may progressively assemble an AI result before it finally becomes profile conformant and is stored by the Evidence Creator as one or more instances. For an illustration of this concept, see Figure XXXX.1-1. “Example of Successive Refinement of JSON Payload to Complete SR” in DICOM Supplement 219 “JSON Representation of DICOM Structured Reports” available at: 885 ftp://medical.nema.org/medical/dicom/supps/Frozen/sup219_fz_JSONSR.pdf Or to put it differently, it is permitted, but not required, that the Evidence Creator perform the analysis. The Evidence Creator may receive results from other software and be responsible for ensuring the result encoding conforms to this profile and for sending the results to the Image Manager/Archive. 890

X.4.1.6 Result Approval & Retention Based on experiences with mammography CAD, sites might be expected to establish policies around the approval and retention of AI results. Mammography CAD policies might have been established to address perceived medico-legal risk (although it is not clear that has been borne out). 895 Some site policies may require that AI results be approved before they can be incorporated into the medical record where they are accessible to other clinicians, and/or may require AI result objects be deleted shortly after the study is reported. Others may wish to retain AI results only if they were viewed and/or referenced in the reporting process. Some may retain AI results which are part of a billable activity. 900 Several DICOM mechanisms exist that are relevant to such policies, although the profile does not specifically mandate their use. Within the SR Document General Module (See DICOM PS3.3 http://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.17.2.html), the Verification Flag (0040,A493) may be used to indicate that a Verifying Observer has attested to 905 the content of an SR document.

ftp://medical.nema.org/medical/dicom/supps/Frozen/sup219_fz_JSONSR.pdf

http://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.17.2.html



Template Rev. 10.4

Note: Per DICOM, modifying the Verification Flag requires the creation of a new instance. Displays might then choose to prefer the instance labelled as VERIFIED over others listed in the Predecessor Documents Sequence (0040,A360) of that instance.

For non-SR instances, there is precedent for creating an approval instance to capture approval of 910 other instances. The Protocol Approval IOD captures approvals and assertions about acquisition protocols. The Content Assessment Results IOD captures pass/fail assessments of instances and is initially targeted at pre-treatment radiotherapy checks. Tracking other details, such as whether a result has been viewed, or referenced in a report, is left to the product used for viewing/reporting. 915

X.4.1.7 Codesets While this profile does not mandate the use of particular codesets for most coded observations in the results, agreeing locally on common codesets will be a key pre-requisite of any effective deployment of this profile. Conformance to Regional or National codesets would be a forward-looking step that might yield benefits in the future when evaluating patients who have been at 920 multiple institutions, when applying national clinical guidelines, or when compiling training datasets from diverse sources, etc. Consistent use of common codes can facilitate many automatic, or semi-automatic, functions such as:

• Correlating current observations with prior observations 925

• Inserting findings into reports in dictation systems

• Using findings to drive decision support tools Even more simply, consistent use of codes will make it far more practical to configure Image Displays to behave in sensible ways. Finding Site 930 When encoding values for the Finding Site (G-C0E3,SRT,"Finding Site") of an observation, a variety of recommended codesets may be found in DICOM PS3.16 Context Groups (CID Tables). DICOM codesets are largely constructed from SCT codes (that are free for use worldwide in the context of DICOM), LOINC codes (that are free for use worldwide), and DICOM codes (that are free for use worldwide). 935 Anatomy Many of the same DICOM Context Groups apply when populating the Anatomic Region Sequence (0008,2218) and Primary Anatomic Structure Sequence (0008,2228). These, and Finding Site, will be particularly helpful when organizing results anatomically. Qualitative Finding Concepts and Values 940 For the coded Concept of a Qualitative Evaluation, and for corresponding coded Values, the DICOM PS3.16 Context Groups again provides a starting set of codes.



Template Rev. 10.4

The Reporting And Data Systems curated by the American College of Radiology provides organized sets of finding codes for a variety of evaluations including: BI-RADS (Breast), C-RADS (CT Colonography), CAD-RADS (Coronary Artery Disease), LI-RADS (Liver), Lung-945 RADS (lung nodules), NI-RADS (Head & Neck), O-RADS (Ovarian-Adnexal), PI-RADS (Prostate), TI-RADS (Thyroid). The RadLex terms curated by the Radiological Society of North America at the radlex.org website includes a variety of codes for clinical findings and imaging observations. Another potential source of codes for measurements and qualitative findings are the Common 950 Data Elements for Radiology (RDE), curated by RSNA and ACR at the radelement.org website. As an example, the code RDE436 represents a finding about the “Presence of pulmonary embolism” and is constrained to have a value of Absent, Present, or Indeterminate. Some RDE data elements may have synonyms in terminologies such as SNOMED CT or LOINC with corresponding equivalent codes. 955 In segmentations, a Segmented Property Category Code (0062,0003) of (123037004,SCT,”Anatomical Structure”), might have a corresponding Segmented Property Type Code (0062,000F) containing something like (10200004,SCT,”Liver”) or (66754008,SCT,”Appendix”). A Segmented Property Category Code (0062,0003) of (49755003,SCT,” Morphologically Abnormal Structure”), might have a corresponding 960 Segmented Property Type Code (0062,000F) containing something like (50960005,SCT,”Hemorrhage”), or (27925004,SCT,”Nodule”).

X.4.1.8 Interactive AI Some implementations may involve “interactive” AI Models in the sense that an operator might choose “on the fly” to invoke one analysis or another based on what they observe in the image, 965 or an operator might provide a needed seed point for a segmentation or select a particular region of an image for analysis, or perhaps there might be several iterative invocations of AI Models with operator feedback in between. Regardless, it is expected that the generated results would be conformant with this profile and displayable by conformant Image Displays. In such scenarios, an Evidence Creator would potentially be integrated with the Image Display. 970 Alternatively, the Image Display might be grouped with a Task Requester in the AI Workflow for Imaging Profile to have the analysis performed by an external Task Performer. Interactive AI raises a number of interesting issues. Due to the possibility of bottlenecks in processing and data transfer, some AI may be implemented similar to post-processing labs where it’s all done before reading starts. But as technology advances, more “on-the-fly” or interactive 975 analysis may be possible, where the radiologist might request AI workup of a suspicious feature, perhaps to quantify its characteristics or to get a list of the most likely diagnostic findings. Such questions (particularly relating to managing execution of AI Models) are out of scope of this profile to resolve, but may be addressed in the IHE AI Workflow for Imaging Profile.



Template Rev. 10.4

X.4.2 Use Cases 980 This profile is intentionally focused narrowly on handling AI Results that are generated from images and are applied in the context of “reading” the associated imaging study. Importantly, many questions and challenges that are interesting and worthwhile are Out of Scope, including, but not limited to:

• handling data flow from image acquisition to image analysis, 985

• scheduling and managing AI analysis,

• analyzing imaging data that is not images,

• analyzing non-imaging data,

• “interactive” use of AI

• capturing clinician confirmation/contradiction/adjustment feedback 990

• conveying clinician feedback into algorithm monitoring and/or training

• applying results to clinical processes other than imaging interpretation,

• prioritizing reading worklists,

• performing patient risk stratification,

• submitting AI results to the EMR for use outside radiology, 995

• training and validating AI models,

• tracking/communicating the training/provenance/validation of a model instance,

• mapping primitives to/from FHIR Observation https://www.hl7.org/fhir/observation.html Note: Out of Scope does not mean products that conform to this profile are not permitted to address such needs, but rather

that this profile does not currently address those questions or specify interoperable methods to meet those challenges. 1000

X.4.2.1 Use Case #1: Store, Retrieve, & Display

X.4.2.1.1 Store, Retrieve, & Display Use Case Description Results are produced from the analysis of images and are stored and later retrieved and displayed by an Image Display to support the interpretation of the associated imaging study. An important goal is that the Image Display does not depend on specific knowledge of the AI 1005 Model or the details or internal structure of the result that it produces. This is intended to facilitate the use of a variety of image display products with a variety of AI Model products and architectures while minimizing the integration and customization overhead. Although not shown, the use case expects there to be multiple Evidence Creators and a given study could easily have multiple results. 1010



Template Rev. 10.4

There is a possibility that some results may arrive after reading has been performed. Coordinating some kind of “Ready To Read” signal based on expected and received data is outside the scope of this profile but might be addressed in an AI Workflow Profile.

X.4.2.1.2 Store, Retrieve, & Display Process Flow The Flow shows storage and retrieval of SR instances. As shown in Table X.1-1, but not shown 1015 here, the results could also, or instead, consist of image instances, and could be retrieved using DICOMweb.

Figure X.4.2.1.2-1: Store, Retrieve, & Display Process Flow in AIR Profile

The text in Figure X.4.2.1.2-2 was used to generate the diagram in Figure X.4.2.1.2-1. Readers 1020 will generally find the diagram more informative. The text is included here to facilitate editing.



Template Rev. 10.4

Figure X.4.2.1.2-2: Diagram Pseudocode for Store, Retrieve, & Display Process Flow

X.4.2.2 Use Case #2: Auxiliary Usage

X.4.2.2.1 Auxiliary Usage Use Case Description 1025 Without going into details about the different ways that results may be used, this use case highlights that non-display actors can make use of the same results. This use case also demonstrates query and retrieval using DICOMweb; it could alternatively have used conventional DIMSE DICOM.

X.4.2.2.2 Auxiliary Usage Process Flow 1030

Figure X.4.2.2.2-1: Auxiliary Usage Process Flow in AIR Profile

title Store, Retrieve, & Display participant Evidence\nCreator as EC participant Image\nManager/\nArchive as IM participant Image\nDisplay as ID EC->EC: (Obtain Images) EC->EC: (Generate Results) EC->IM: Store Evidence Documents [RAD-43] ID->ID: (Reviewing Study) ID->IM: Query Evidence Documents [RAD-44] ID->IM: Retrieve Evidence Documents [RAD-45] ID->ID: Display Analysis Result [RAD-Y1]



Template Rev. 10.4

The text in Figure X.4.2.2.2-2 was used to generate the diagram in Figure X.4.2.2.2-1. Readers will generally find the diagram more informative. The text is included here to facilitate editing.

1035 Figure X.4.2.2.2-2: Diagram Pseudocode for Auxiliary Usage Process Flow

X.5 AIR Security Considerations Refer to RAD TF-1x: Appendix F Security Environment Considerations. Personal Healthcare Information (PHI) is present in the DICOM instances being stored, retrieved, processed, and displayed. 1040

X.5.1 Security Considerations for Actors All actors in the AIR Profile should consider grouping with a Secure Application or Secure Node Actor in the ITI Audit Trail and Node Authentication (ATNA) Profile. This profile strongly recommends implementation of the ATNA Record Audit Event [ITI-20] transaction to record when and where AI results are distributed. 1045 The ATNA Profile also requires that all actors implement the Authenticate Node [ITI-19] transaction to further ensure the integrity of transactions. Implementers are advised to take advantage of the authentication and communication encryption capabilities that Authenticate Node [ITI-19] transaction provides between Secure Nodes and to take advantage of TLS. This profile does not particularly add security considerations beyond those already established 1050 for Image Manager/Archives in other profiles.

X.5.2 Security Considerations for AI Results AI Result instances as defined in this profile contain personal demographic information and clinical information. It is appropriate for products implementing the AIR Profile to include appropriate PHI controls. Specifying such mechanisms and features is outside the scope of this 1055 profile.

X.6 AIR Cross-Profile Considerations Since the data created and exchanged in the AI Results Profile are encoded using common DICOM instances, many other Radiology profiles could be used in conjunction with the AI Results Profile: 1060

title Auxiliary Usage participant Image\nManager/\nArchive as IM participant Imaging\nDocument\nConsumer as IDC note over IDC: Meanwhile IDC->IM: QIDO-RS Query [RAD-129] IDC->IM: WADO-RS Retrieve [RAD-107] IDC->IDC: (Populate report,\nDatabase results,\ndrive CDS, etc)



Template Rev. 10.4

• XDS-I and/or XCA-I could be used to exchange AI results within and between enterprises

• IRWF.b and/or IDEP could be used to localize and manage the import of AI results

• PDI could be used to distribute AI results on portable media

• ATNA (Radiology option) should be used to secure the communication of, and record 1065 audit trails for, AI results.

• IOCM could be used to manage changes and quality control of AI results Both the AI Results Profile and the Evidence Documents (ED) Profile make use of the [RAD-43], [RAD-44], and [RAD-45] transactions to store, query, and retrieve DICOM SR instances. Both the AI Results Profile and the Web Image Access (WIA) Profile make use of the [RAD-1070 129] and [RAD-107] transactions to query and retrieve DICOM instances using RESTful DICOMweb. The WIA Profile also documents using the XDS and MHD Profiles to support backend functions, but that is not really relevant to the AI Results Profile. The Mammography Image (MAMMO) Profile introduces requirements on the creation, exchange, and display of mammography images and DICOM SR objects containing CAD 1075 results. While there are direct parallels between the latter and the results in this profile, the MAMMO Profile mandates the specialized SR template (TID 4000) and addresses many mammography-specific issues. Products that create and display mammography results are advised to conform to the MAMMO Profile. The situation is very similar for the Chest X-Ray CAD (CXCAD) Profile. 1080 The Results Distribution (RD) Profile addresses the communication of imaging results, but in that context, “results” refers to diagnostic reports, so there is no direct interaction between the RD Profile and the AIR Profile. AIW-I – AI Workflow for Imaging Profile A Task Performer in AIW-I might be grouped with an Evidence Creator to assign and track 1085 completion of the processing performed by the Evidence Creator (or its constituent AI Models) as well as provide the Evidence Creator with information about how to access it’s input data and allow the Evidence Creator to communicate the list of instances it has created and where they have been stored. The tasks also facilitate controlling the parameters fed to the algorithm, recording execution status (for example, billing may depend on whether a given algorithm was 1090 run even if it didn’t generate any results), and allowing systems (like Reporting Worklist Managers) to subscribe to notifications about AI processing that has been scheduled and/or completed. RRR-WF - Radiology Remote Reading Workflow Profile A Task Performer in RRR-WF might be grouped with an Image Display in a reading workstation 1095 to access a defined reporting task that identifies the study/instances to be reported and where they may be obtained. Since those instances may include the AI Results, this is one of the ways



Template Rev. 10.4

that the Image Display can become aware of the AI Results that it Query/Retrieves in this profile.



Template Rev. 10.4

Appendices to Volume 1x 1100

Appendix J – Example AI Result Encodings These examples were prepared in support of the AI Results (AIR) Profile. See RAD TF-1: X.4.1.2 Result Primitive Encoding for the encoding specification. <If the Result Primitive Encodings ends up in Vol 3, then move this appendix too>. Show examples of encoding of a few "typical" AI results in the described primitives 1105 Need examples for TID1411. The different places images can be referenced may confuse people and the gap in row numbering gap doesn’t help. In 2019e.pdf it fell on a page break so it looked like missing rows. Work in NELSON-like results - volume segments and tracking over 2 year screening intervals looking at doubling time? 1110 Show examples of navigation/presentation of results (if not adequately covered in the use case and Display transaction)

J.1 Example Qualitative Finding

J.1.1 <Title> 1115 Appendix J.1.1 text.



Template Rev. 10.4

Volume 2 – Transactions Add Section 4.Y1

4.Y1 Display Analysis Result [RAD-Y1]

4.Y1.1 Scope 1120 This transaction is used to present image analysis results to someone, such as a radiologist interpreting a study. Although this can be thought of as an “informational transaction” between a display device and an operator, it is not a typical IHE transaction between two devices; the primary focus is on the required behavior of the display rather than messaging between two actors. 1125 The specification is organized around a defined set of result data structures (“primitives”) that may be encountered by the display, and the baseline display behaviors required for each. As with many IHE specifications, the display may have other behaviors in addition to those mandated by IHE. Methods for selecting the data to be displayed is outside the scope of this transaction. 1130

4.Y1.2 Actor Roles The roles in this transaction are defined in the following table and may be played by the actors shown here:

Table 4.Y1.2-1: Actor Roles

Role: Display: Presents results visually to an operator, such as a radiologist.

Actor(s): The following actors may play the role of Display: Image Display

1135 Transaction text specifies behavior for each role. The behavior of specific actors may also be specified when it goes beyond that of the general role.

4.Y1.3 Referenced Standards • DICOM PS3.3: A.35.13 Comprehensive 3D SR IOD

• DICOM PS3.16: TID 1500 Measurement Report 1140

• DICOM PS3.3: A.51 Segmentation IOD

• DICOM PS3.3: A.75 Parametric Map IOD



Template Rev. 10.4

4.Y1.4 Messages

1145 Figure 4.Y1.4-1: Interaction Diagram TODU

4.Y1.4.1 Display Results The Display presents the analysis results to the operator.

Note: The results being presented may or may not have been previously verified by a human and may or may not be accurate. 1150

4.Y1.4.1.1 Trigger Events An operator or an automated function determines that one or more results should be presented.

4.Y1.4.1.2 Message Semantics The results are encoded as described in RAD TF-1: X.4.1.1. The messaging protocol by which the encoded results were transferred to the Display is not 1155 specified. If the Display receives results by a profiled mechanism such as DICOM C-STORE, or DICOMweb WADO-RS, the messaging protocol is specified in the corresponding transaction. If results are accessed by being grouped with another actor such as an Image Manager or Evidence Creator, there is no messaging protocol involved. The representation of the instances (DICOM binary, DICOM XML, DICOM JSON) is also not 1160 specified since it will depend on the representations the Display is capable of receiving.

4.Y1.4.1.3 Expected Actions (i.e., Display Requirements) The behaviors in this section are specified as baseline capabilities. Displays may have additional or alternate capabilities that may be invoked or configured. Displays shall support the capabilities described in this section for result primitives (See RAD 1165 TF-1: X.4.1.1) encoded in instances of the following:

Display

Display Results

“operator”

Display

Display Results



Template Rev. 10.4

• Comprehensive 3D SR IOD

• Segmentation IOD Parametric Map Option Displays that claim support for the Parametric Map Option in the AI Results Profile shall also 1170 support presentation of result primitives encoded in instances of the following:

• Parametric Map IOD

4.Y1.4.1.3.1 General Result Display Requirements The Display:

• shall make the operator aware that results are available for display 1175

• shall be able to display to the operator all result primitives for the current study o Note: a “dump” of the DICOM header or the SR Content Tree is not sufficient

• shall handle multiple results collected together in one instance

• shall handle sets of results that are spread across multiple instances in one or more series

• shall be able to display to the operator all result primitives for prior studies of which the 1180 Display is aware

• shall allow the operator to control which result primitives are displayed (see RAD TF-1: X.4.1.3 for some approaches and considerations) o that control shall include distinguishing based on whether the result primitive has a

(111056, DCM, “Rendering Intent”) value of (111150, DCM, “Presentation 1185 Required”) or (111151, DCM, “Presentation Optional”)

• shall communicate result primitives unambiguously to the operator, including making it clear the specific image/frame in the study to which the result primitive applies o this may involve toggling a GSPS-like overlay on/off

• shall make available for display the following information about the AI Model/algorithm 1190 that generated of each result primitive (identifiable as items from the Contributing Equipment Sequence (0018,A001) with a Purpose of Reference Code Sequence (0040,A170) value of (Newcode1,99IHE,”Processing Algorithm”) ): o Manufacturer (0008,0070) o Manufacturer’s Model Name (0008,1090) 1195 o Software Versions (0018,1020) o Device UID (0018,1002)

• shall make available for display the following information about each result primitive



Template Rev. 10.4

o Content Date (0008,0023) and Content Time (0008,0033) of the primitive instance

• shall indicate when analysis was not attempted or has failed, either entirely, or if some 1200 analyses have succeeded and others failed, as distinct from when analysis has succeeded but there are no findings.

• shall be able to communicate when more than one finding is present in the study for the same Concept (e.g., when two different algorithms make a determination on whether pneumonia is present, or the same algorithm with different parameters), and be able to 1205 present the multiple findings and differentiate between them (e.g., by indicating the different algorithms or parameter sets).

• shall be able to resolve overlapping results (i.e., results where one could obscure the other when rendered together). The method is left to implementation (e.g., allowing the user to cycle through them or choose between them). Overlapping results can occur when the 1210 same measurement is performed multiple times, and when the different observations (ID, length, width, size, character) are made on the same entity.

• shall, if an image reference with concept of (121200, DCM, "Illustration of ROI") is associated with a primitive, be able to display that image.

• shall be able to communicate the list of Root Results (identifiable as SR instances with a 1215 document title of (Newcode0,99IHE,”Analysis Root Result”))

• shall be able to communicate and display the subordinate results (within or referenced by a given Root Result) o the interaction and progressive-disclosure display behaviors for “drilling down” are

left to the implementation of the Display 1220 If the Display is unable to present a result, it is up to the Display whether to notify the operator, make a record in a log file, or fail silently.

4.Y1.4.1.3.2 Display of Qualitative Findings The Display shall be able to display qualitative findings. The specific presentation details of how qualitative findings are displayed to the operator are left 1225 to the implementation of the Display and may involve the use of text, icons, etc.

4.Y1.4.1.3.3 Display of Measurements The Display:

• shall be able to display measurements in the context of the image being measured

• shall be able to display SCOORD and/or SCOORD3D locations associated with spatial 1230 measurements (typically as graphics)



Template Rev. 10.4

4.Y1.4.1.3.4 Display of Locations The Display:

• shall be able to display point locations as a marker on the referenced image

• may be able to display point locations as markers on other images in the same Frame of 1235 Reference

• may be able to present an MPR (Multi-Planar Reformat) view centered on a selected point location

• shall be able to display line locations and region locations as graphics on the referenced image 1240

• shall be able to display the finding site, finding, and/or the concept associated with the location (e.g., “Lesion, Center”)

• shall, if the Display can identify that the locations are CAD marks, o make the operator aware that CAD marks are available for display o indicate whether or not CAD marks are currently activated 1245 o be able to display on the image referenced by o display the most recent locations (by Content Date/Time) first unless otherwise

configured o allow the operator to choose which older locations to display

The form in which the location marks are displayed may influence operator performance, and 1250 hence it may be necessary to display them in a manner prescribed by the vendor that generated the results, which is not encoded in the DICOM object. The form of the location mark rendering is out of the scope of this profile to define.

4.Y1.4.1.3.5 Display of Segmentations The Display: 1255

• shall display each segment in a way that the operator can understand the boundary of the segmented volume

• shall be able to display each segment as either: o a volume if the Display supports 3D rendering in general, or o as an overlay on the image slices to which the segment applies, e.g., with the 1260

segmented region color-coded or by displaying the contours of the segment, o or both.

• shall be able to display the Segment Label (0062,0005) and Segment Description (0062,0006), if available.



Template Rev. 10.4

• shall allow the user to toggle the display of each segment on/off. 1265

• may be able to display a group of related segments in the same view

• may be able to generate a volumetric view of the segmented volume embedded in the images to which they apply

• may be able to apply a Volumetric Presentation State, if present, to the rendering of the segmented volume. 1270

4.Y1.4.1.3.6 Display of Parametric Maps The Display:

• shall be able to display a parametric map alone as a color image

• shall be able to identify corresponding images from which a parametric map was derived as identified in the Source Image Sequence (0008,2112) of the Derivation Image 1275 Sequence (0008,9124)

• shall be able to display a parametric map as an overlay on the corresponding image in a manner that is functionally equivalent to the pipeline defined in DICOM PS3.4 Section N.2.4

• shall allow the operator to control the blending parameters of the display 1280

• shall support resampling of one or both datasets as part of the fusion process

• shall be able to display the LUT Explanation (0028,3003) (found in the Real World Value Mapping Sequence (0040,9096) of the parametric map)

• may be able to display a parametric map as part of an advanced blending transformation in a manner that is functionally equivalent to that described in DICOM PS3.4 Section 1285 N.2.6

• may be able to retrieve and apply Blending Presentation State objects and Advanced Blending Presentation State objects.

4.Y1.4.1.3.7 Display of Tracking IDs The Display: 1290

• shall be able to show the Tracking Identifier for any given result

• shall be able to identify results that relate to the same entity, as indicated by sharing the same value for (112040, DCM, "Tracking Unique Identifier"), or within a single instance sharing the same value for (112039, DCM, "Tracking Identifier")

• shall be able to present results that are related to the same entity as a group, e.g., by 1295 allowing the operator to sequence through them, showing prior and current side-by-side, or listing them in chronological order



Template Rev. 10.4

• may be able to calculate changes when the same measurement for the same entity is present at two different timepoints

• may be able to calculate derived values such as the volume of a segmentation, or the 1300 tumor doubling time based on two timepoints

Note: If such derived values are simply present as additional result primitives, having been pre-calculated by an Evidence Creator, then the Image Display will be able to display them, per the requirements above to display basic findings. The same is true for longitudinal assessments.

4.Y1.4.1.3.8 Display of Image References 1305 The Display:

• shall be able to display the image referenced by any result

4.Y1.5 Protocol Requirements NA

4.Y1.6 Security Considerations 1310 This transaction involves presenting DICOM objects that typically constitute personal health information (PHI) to human observers who are typically clinicians. Typical access controls and audit trails in accordance with local policies would be appropriate.

4.Y1.6.1 Security Audit Considerations The Radiology Audit Trail Option in the IHE ITI Audit Trail and Node Authentication Profile 1315 (ITI TF-1:9) defines audit requirements for IHE Radiology transactions. See RAD TF-3: 5.1.

4.Y1.6.2 Display Specific Security Considerations Since this transaction involves the display of PHI, it may be reasonable for Image Displays to implement typical access controls for patient records, such as logins for operators and role-based access policies. 1320 Since this transaction involves parsing datasets generated by other systems, it may be reasonable for Image Displays to implement basic digital hygiene, such as sanitizing datasets to avoid malicious executable scripts that might be executed by a browser-based viewer.



Template Rev. 10.4

Appendices to Volume 2 1325

No new Volume 2 appendices



Template Rev. 10.4

Volume 3 – Content Modules

Add the following row to RAD TF-3: Table 5.1-2

5.1 ITI-20 Record Audit Event 1330

Table 5.1-2: IHE Radiology transactions and resulting ATNA trigger events IHE Radiology Transaction ATNA Trigger

Event(s) Actor(s) that shall be able to record audit

event Patient Registration [RAD-1] Patient-record-event … … Display Analysis Result [RAD-Y1] Study-used Image Display

Add the following new sections to RAD TF-3.

6.5 DICOM Content This section profiles encodings of DICOM Content. The requirements here are additional to, and 1335 do not contradict, the DICOM Standard.

6.5.1 AI Result Content Module <This may be where the Technical Committee chooses to move the Encoding Requirements from RAD TF-1: X.4.1.2 and the examples from RAD TF-1x: Appendix J. Note: If the examples are here inside a controlled document then updating would require CPs. Referencing example files, 1340 might be a better alternative.> Assumptions and Definitions: It is assumed that….

• Result Primitive – definition.

6.5.1.1 Referenced Standards • DICOM PS3 dicomstandard.org/current 1345