2017 engagement evaluation report - pcori

COMET Stakeholder Advisory Board

Submitted: December 22, 2017 2017 Engagement

Evaluation Report

Document prepared by the Center for Medical Technology Policy

COMET: Stakeholder Advisory Board Engagement i

TABLEOFCONTENTSSAB Engagement Activities ...................................................................................................... 1 SAB Recommendations to the COMET Executive Committee ................................................... 2 Stakeholder Engagement Evaluation Interviews ...................................................................... 5 Appendix A: SAB Newsletter ................................................................................................... 6 Appendix B: Survey – SAB Communication Channels ............................................................... 7 Appendix C: Survey – DCIS Language Document .................................................................... 10 Appendix D: Stakeholder Engagement Evaluation Interview Guide ....................................... 15

COMET: Stakeholder Advisory Board Engagement 1

ENGAGEMENTACTIVITIESThe following table summarizes the activities of the COMET Stakeholder Advisory Board in 2017. The

first in‐person meeting was held on January 12, 2017. Additional engagement activities have included

webinars, a newsletter update, and surveys to obtain SAB feedback on specific topics. The surveys and

newsletter are included as appendices to this report.

2017 STAKEHOLDER ADVISORY BOARD ENGAGEMENT ACTIVITIES

Date Activity Goal Participants

1/12/2017 In‐Person Meeting

Introduce SAB members to one another and to the study investigators;

Discuss key issues surrounding DCIS diagnosis and treatment at the decisional dilemma faced by patients and their providers;

Provide the SAB with an update on plans for launching the COMET trial and the role of the website and patient portal in communicating about the trial; and

Solicit recommendations and strategies from the SAB for optimizing trial recruitment.

13 SAB members 3 PLT members

5/2/2017 Newsletter Provide the SAB with an update of COMET activities in the interim period between the in‐person meeting and the June webinar.

n/a

6/13/2017 Survey To assess the communication channels and networks available to SAB members and understand how SAB members may help to effectively communicate key information about the COMET trial.

8 respondents

6/29/2017 Webinar Solicit SAB feedback surrounding three key challenges: o Initiating site activation within an unusual clinical

trial infrastructure (COMET study vs. a multi‐site NCI study)

o Maintaining engagement and enthusiasm at activated sites

o Optimizing communication with activated sites

Discuss strategies for raising awareness about COMET


8/1/2017 Survey Solicit SAB feedback on the DCIS language document developed by the PLT

13 respondents

10/23/2017 Webinar Discuss strategies for raising awareness about COMET

Solicit SAB input on plans for upcoming COMET Investigator Meeting



RECOMMENDATIONSTOTHECOMETEXECUTIVECOMMITTEE

SAB recommendations to the COMET Executive Committee resulted from the following engagement

activities in 2017: 1) January in‐person meeting; 2) June webinar; and 3) October webinar. Specific topics

presented for SAB feedback were identified and discussed with the study team in advance of each

engagement activity to reflect the current study needs and priorities. These included:

Optimizing trial recruitment

Initiating site activation

Maintaining engagement and enthusiasm at activated sites

Optimizing communication with activated sites

Raising awareness about COMET

Recommendations that emerge from SAB engagement activities are summarized and presented to the

COMET Executive Committee for consideration. Recommendations that resulted from the in‐person

meeting and the first webinar were summarized in a matrix format (Excel spreadsheet) and emailed to

the Patient Leadership Team and the Executive Committee for their written response. To simplify and

expedite this process, SAB recommendations are now presented during teleconferences with the

COMET Investigators and PLT members. The Investigator and PLT responses are summarized by CMTP

staff and the matrix is updated accordingly. SAB members are regularly appraised concerning the

disposition of recommendations (i.e., if they have been implemented, and if not, why not).

The following table contains the SAB recommendations from each engagement activity in 2017.

January 2017 In‐Person Meeting

SAB Recommendation Executive Committee Response

Make COMET a “harder sell” on the DCIS website. Increase visibility on the homepage.

We agree with both the SAB and the PLT that the COMET trial should receive increased visibility on the website home page.

Consider direct‐to‐patient marketing

We agree that direct‐to‐patient marketing is an interesting idea and social media may be used to provide information about the COMET trial to potential participants. However, the use of social media as a recruitment tool should not be considered as it would almost certainly contravene specific central/local IRB regulations.

Plan on two visits: the first to discuss regular options and present trial, the second after they take info home and think about it

The Executive Committee agree with the principle that every patient should take whatever time is necessary before deciding whether or not to participate in the COMET trial.


Encourage caregiver attendance at the consultation

The Executive Committee suggest that this should be a matter of individual choice that is left to the discretion of the patient. It is possible that caregiver attendance at the consultation may facilitate patient participation in the COMET trial, but it is equally possible that it may hinder participation.

Provide tailored communication to patients that:

• are at a 5th grade reading level • emphasize equipoise • emphasize the safety of active

surveillance • communicate that DCIS is different • convey benefit of participation in trial

regardless of arm • provide reassurance about study safety

and DSMB oversight • emphasize role as a volunteer and

research partner • stress the altruistic nature of

participation

We agree with the need to provide tailored communication to patients that addresses literacy level concerns but also communicates the importance of the COMET research question and the benefits of contributing to research for future patients

Produce and disseminate videos demonstrating how to communicate with patients about their diagnosis and the trial. Potentially involve SAB members in videos.

We agree that there is a need to ensure the COMET trial is presented by providers in an unbiased and balanced manner. However, we are not sure that a video would be the most effective/efficient mechanism to achieve this (we will discuss with the PLT at our next PI/PLT Strategy Call).

Rather than framing study arms as surgery vs. nothing, present the alternatives as having surgery now vs. having surgery if and when you need it.

The Executive Committee agree that this is a very important concept that should be incorporated into future communication and dissemination materials.

Emphasize the oath to do no harm ‐ "we don't want people to have interventions they don't want or need"

Whilst understanding the rationale behind this suggestion, we believe that interpretation of the Hippocratic Oath in this way may result in an accusation that the COMET study is not being presented in an unbiased and balanced manner ‐ for example, a counter‐argument from critics of active surveillance may be that surgical intervention is required for women with low‐risk DCIS in order to uphold the principle of "do no harm" (i.e. prevent the potential development of invasive breast cancer).


Involve other clinical staff (including nurse navigators) in communicating with patients about the trial

Commencing July 2017, in addition to the Site Initiation Meetings with onboarding sites, there will be a monthly Activated Site Meeting involving site coordinators and other key staff at sites that have commenced recruitment ‐ particularly those who are "gatekeepers" to patients at the interface of the patient/provider encounter. Each activated site also has follow‐up from a PI or Co‐PI (with the PI of the site), from a member of the PLT (with a patient advocate at the site/relevant member of the staff) and from the Project Manager (with the site /study coordinator).

Develop partnerships with:

Mainstream media (generate awareness that not all BC needs immediate treatment)

Google (to prioritize search results) Breast cancer advocacy organizations

We agree with the need for a concerted communication strategy for the COMET study that incorporates all media platforms ‐ particularly social media.

Develop partnerships with: • American Society of Breast Surgeons • Eastern Cooperative Oncology • NRG Oncology • Translational Breast Cancer Research Consortium

The Executive Committee agree with the need to develop effective and fruitful partnerships with all related organizations. The PI and Co‐PIs work continuously to develop such partnerships at each conference/meeting that they attend.

Conduct town hall and virtual town hall meetings with ASBS

We agree with the use of town hall and virtual town hall meetings to disseminate information about the COMET study

Advertise at conferences and on websites of partner organizations

We agree with the need to disseminate information about the COMET study at conferences and on websites of partner organizations

June 2017 Webinar (Executive Committee Response Pending)

Determine what studies are ongoing or planned at sites, and assess how COMET will fit with their

institutional priorities. Some institutions (e.g. Johns Hopkins) maintain a prioritization list that can be

referenced as new studies are being considered. This list helps the institution to assess what is “in the

queue” and to balance expectations and resources.

Focus on activating sites with links to radiology centers (where cases are generally identified) as a potential strategy going forward.

Consider collaborating with NCI Community Oncology Research Program (NCORP) and going through their

steering committee to have the COMET protocol NCORP approved.

Engage with nurse navigators. Many institutions have navigators doing screening for clinical trials, and they

are skilled in understanding the appropriate time to discuss enrollment with patients. They can be engaged

as champions of the study, and it would be useful to determine which COMET sites have nurse navigators

available so that they are engaged early in the activation process.

Work with breast imaging teams who will see patients first, and determine how to get radiologists engaged

and interested. Connect with the imaging core of the ECOG‐ACRIN Cancer Research Group and NRG

Oncology.


EVALUATIONINTERVIEWSWITHSABMEMBERS

As part of the stakeholder engagement evaluation process, CMTP will conduct semi‐structured

interviews with each SAB member in 2018. Each quarter, a sample of approximately six members

(representing a mix of stakeholder affiliations) will be interviewed to assess their experiences and

satisfaction as members of the Stakeholder Advisory Board. These interviews will be conducted by an

experienced CMTP team member who is not regularly involved with COMET activities (thus providing a

more neutral forum for open discussion). A copy of the semi‐structured interview guide is included in

Appendix D.

Conduct outreach to local news stations. Some stations broadcast weekly programs to talk about studies

that are available and take live questions. Organizing broadcasts around events such as breast cancer

awareness month may help with accrual.

Ensure that information about COMET is posted on breastcancer.org

Ensure that case lists are reviewed during rounds at mammography imaging units.

Register COMET with the American Cancer Society Clinical Trials Matching Service

October 2017 Webinar (Executive Committee Response Pending)

Use the hashtag #BCSM (Breast Cancer Social Media) when tweeting about COMET to increase visibility and

gain exposure to a large audience.

Focus on newspapers and magazines rather than targeted communications (women’s magazines are

frequently looking for stories).

Provide pamphlets in radiology offices and mammography centers.

Connect with the institution’s PR department. The site PI can pitch a story (e.g. “Dr. Smith is now enrolling

patients for COMET” and include a blurb about the investigator).

Messages should:

emphasize that DCIS isn’t going to kill you – that message isn’t out there.

emphasize what matters most to patients

Engage radiologists in the earliest stages

Submit a one‐page opinion piece in radiology journal


APPENDIXA:SABNEWSLETTER


APPENDIXB:SURVEY‐SABCOMMUNICATIONCHANNELS


APPENDIXC:SURVEY‐DCISLANGUAGEDOCUMENT


APPENDIXD:INTERVIEWGUIDE‐SABENGAGEMENTEVALUATION

Format: 45‐minute semi‐structured (confidential) phone interview

1. Learn from SAB experiences to optimize the effectiveness and satisfaction of members

2. Obtain feedback regarding lessons learned that could inform future initiatives

I. General/Open‐Ended

a. Thinking back over the past year, how would you describe your overall experience as a member of the

SAB?

Probes:

Does anything stand out in either a positive or negative way?

In what ways has membership on the SAB met or not met your expectations?

II. SAB Role

a. How would you describe the role the SAB is intended to play on the COMET study?

Probes:

Is there anything specific about the SAB’s role that needs to be clarified?

III. SAB Composition

a. What are your thoughts on the ideal composition of the SAB?

Probes:

What size should the group be?

Are there important perspectives that are currently missing from the group?

IV. SAB Meetings

a. Are you satisfied with the meeting schedule?

Probes

Should the group meet more or less frequently?

Is the length of the meetings appropriate?

Is the timing of the meetings convenient for you?

b. Is the meeting time used efficiently?

Probes

Are there better ways to use the time?

Any suggestions for making meetings more productive?

V. SAB Participation

a. How do you feel about your current level of participation in the SAB?

Probes:

Would you like to be more or less involved?

Do you plan to continue participating as a SAB member?

VI. Path Forward

a. Do you have any additional thoughts about the future of the SAB?

Probes:

What are the most important objectives for the group?

2017 engagement evaluation report - pcori

Documents