2019 planchallenge man vs machinefilecache.mediaroom.com/...manvsmachine...2.pdf · 2019 varian...

29
©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only ©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only Anthony Magliari MS, CMD Sr Research Scientist Medical Affairs 2019 PlanChallenge Man Vs Machine RapidPlan:Esophagus

Upload: others

Post on 08-Aug-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Anthony Magliari MS, CMD

Sr Research Scientist – Medical Affairs

2019 PlanChallengeMan Vs MachineRapidPlan:Esophagus

Page 2: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

2019 Varian Plan Challenge: Man vs Machine

Goals

• Identify and recognize top Eclipse planners

• Increase awareness of and increase utilization of RapidPlan

• Create a very high quality and finely tuned RapidPlan model

for the community

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 3: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

How to crowd source a RapidPlan model• Create a score card from superset dose constraints from popular

protocol

• Collect CT, structure, plan and dose from users who attempt to get the

highest score possible for their patient

• Use submitted cases to create RapidPlan model(s) and tune

automatically generated optimization objectives for max score• Optional: use RapidPlan created plans as training set cases for final

model

• Replan all submitted cases using final RapidPlan and a defined beam geometry (4arc Halcyon or TrueBeam)

• User’s plan that outscores the RapidPlan model by the largest margin wins

• Explain results and where to download final model

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 4: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

The Protocol: RTOG1010 (esophagus)

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 5: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Scorecard: RTOG1010 superset to 50.4Gy

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 6: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Scorecard: closer look

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 7: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Scorecard: Many Nonlinear Score Functions 1

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 8: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Scorecard: Many Nonlinear Score Functions 2

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 9: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Scorecard: Many Nonlinear Score Functions 3

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 10: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Scorecard: Many Nonlinear Score Functions 4

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 11: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Scorecard: Many Nonlinear Score Functions 5

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 12: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

What is RapidPlan?

A method for estimating DVHs which utilizes Machine Learning

Based on patient geometry

Previous knowledge contained in a set of existing patient cases

Optimization objectives that are agood starting point for the optimization

Based on the estimated DVH and plan prescription

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 13: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

DVH Prediction and Optimization Objectives

DVH prediction is straightforward

• If you have consistently planned, high quality cases: simply eliminate

outliers.

• However, with very careful tuning of the auto-created optimization objectives, RapidPlan can create really high

quality plans—directly.

• One method (used here) to tune auto-created optimization objectives: the Scorecard

The priorities for auto-created Optimization Objectives are not

• Most people create a “good guess” set of auto-created optimization

objectives when they package a RapidPlan model for “a good starting

point”.

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 14: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Recursive Model creation also improves quality

• After creating the initial RapidPlan model

and guessing at an initial set of auto-

created optimizations objectives,

priorities will be modified and the process

will iterate.

• The single highest scoring plan from

each patient created by one of the initial

RapidPlan models, while tuning priorities,

can be used as the training set for a final

RapidPlan model.

• The result is a next generation RapidPlan

model created from plans created by an

initial RapidPlan model.

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 15: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

RapidPlan Optimization Example

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 16: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Example: Esophagus clinical IMRT(45Gy)

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 17: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Example: Esophagus manual VMAT 4arc

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 18: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Example: Esophagus InitalRP1.0 VMAT 4arc

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 19: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Example: Esophagus InitalRP1.0a VMAT 4arc

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 20: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Example: Esophagus Score Cards

122.83 (manual) 133.66 (RPinital1.0) 134.64 (RPinital1.0a) […]

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 21: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Score spreadsheet: Training set for FinalRP

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 22: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Score spreadsheet: Final Result

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 23: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

The winner and honorable mentions

Frank Simac (21st century Oncology)

• Beat the Final RapidPlan model by 11.34

points!

Honorable mentions:

• Nathan Wu (3.04 points)

• Justin Hayes (1.04 points)

• Thomas Costantino (.47 points)

Submitted noncoplanar plans (against the rules):

• Shaomin Zhang (3.41 points)

• Kirby DeLozier (2.13 points)

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 24: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

RapidPlan vs winning plan (Dose)

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 25: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

RapidPlan vs winning plan (Score)121.41 132.49

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 26: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Where to download the Esophagus Model

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Preferred location: Varian MarketPlace

Alternate location: Varian Medical Affairs

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 27: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Where get more informationOn this Esophagus RapidPlan model

• Clinical Description included with model

On creating RapidPlan models

• AAMD website Meetings>Archives>2016In the Knowledge Based Planning Era: The Role of the

Dosimetrist as a Predictive Model Creator/Administrator

Anthony Magliari, MS, CMD

On your submission and how it compares

• https://varian.proknow.com Browse>(new)Dose

On RapidPlan in general and much more

• www.myvarian.com

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 28: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

Conclusion

Special Thanks:

Belinda Thibodaux(!), Lesley Rosa, Ben Nelms, Drew Bullock, Karen Kigin,

Theodore Hong, MD and Andy Su, MD

RapidPlan can perform as well or better than humans

• In this example, only four valid human planners were able to outperform the final

model.

• 39 cases in used in final Esophagus model

You can use a model you helped create in your clinic

• Take pride in knowing some of your knowledge is contained in this model which will

help patients all over the world get great esophagus plans with superior lung and

heart sparing.

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only

Page 29: 2019 PlanChallenge Man Vs Machinefilecache.mediaroom.com/...ManVsMachine...2.pdf · 2019 Varian Plan Challenge: Man vs Machine Goals •Identify and recognize top Eclipse planners

©2019. VARIAN MEDICAL SYSTEMS, INC. For immediate recipient’s use only