Identifying opportunities for natureengagement in cancer care practiceand design: protocol for four-roundmodified electronic Delphi

Sarah Blaschke,1,2 Clare C O’Callaghan,1,3,4 Penelope Schofield1,2,5

To cite: Blaschke S,O’Callaghan CC, Schofield P.Identifying opportunities fornature engagement in cancercare practice and design:protocol for four-roundmodified electronic Delphi.BMJ Open 2017;7:e013527.doi:10.1136/bmjopen-2016-013527

▸ Prepublication history forthis paper is available online.To view these files pleasevisit the journal online(http://dx.doi.org/10.1136/bmjopen-2016-013527).

Received 1 August 2016Revised 20 October 2016Accepted 5 January 2017

For numbered affiliations seeend of article.

Correspondence toSarah Blaschke; Sarah.Blaschke@petermac.org

ABSTRACTIntroduction: Opportunities to engage with naturehave shown relevance in experiences of health andrecovery of patients with cancer and are attractinginterest in cancer care practice and design. Suchhealthcare innovations can widen the horizon ofpossible supportive care solutions but requiredeliberate and rigorous investigation to ensureresponsible action is taken and wastage avoided. Thisprotocol outlines a study designed to solicit knowledgefrom relevant experts drawn from a range of healthcarepractitioners, management representatives, designersand researchers to explore levels of opinion consensusfor determining opportunities for, and barriers to,providing helpful nature engagement in cancer caresettings.Methods and analysis: A 4-round modifiedelectronic Delphi methodology will be used to conducta structured, iterative feedback process for queryingand synthesising expert opinion. Round 1 administersan open-ended questionnaire to a panel of selected,relevant experts who will consider the ownrecommendations of patients with cancer for natureengagement (drawn from a preceding investigation)before contributing salient issues (items) withrelevance to the topic. Round 2 circulates anonymisedsummaries of responses back to the experts who verifyand, if they wish, reconsider their own responses.Rounds 3 and 4 determine and rank experts’ top 10items using a 10-point Likert-type scale. Descriptivestatistics (median and mean scores) will be calculatedto indicate the items’ relative importance. Levels ofconsensus will be explored with consensus defined as75% agreement.Ethics and dissemination: Ethics approval for thisstudy was obtained from the Institution’s HumanResearch Ethics Committee (blinded for review). It isanticipated that the results will be published in peer-reviewed journals and presented in a variety of forums.

INTRODUCTIONBackgroundWith the worldwide surge in incidence,cancer will soon impact at least one in threepeople.1 Reducing the burden of cancer and

supporting those affected by cancer hasbecome a healthcare priority. It is knownthat a significant amount of this healthcareburden is preventable; one-third of lives lostto cancer are attributable to behavioural andlifestyle choices and 30% of these cancerdeaths are preventable by attending to keyrisk factors.1 In response, healthcare policymust consider effective clinical care and alle-viate the burden associated with cancer treat-ment and promote positive health behaviourand prevent poor lifestyle choices. Suchhealth-centric strategies focus on patients’own resources to manage health and disease2

and aim to strengthen patients’ capacity tomaintain or regain good health in thecontext of pathogenic biological or psycho-social stressors. To this end, exposure to, and

Strengths and limitations of this study

▪ The aim of the study outlined in this protocol isto determine opportunities for contact withnature in oncology contexts in order to developnovel healthcare design solutions.

▪ Own nature experiences of patients with cancerand recommendations for opportunities toengage with nature in oncology contexts formthe basis for this investigation.

▪ This study represents the first internationalcross-disciplinary collaboration between health-care and healthcare facility design experts tobetter understand the feasibility of appropriateand helpful nature engagement in oncologycontexts.

▪ The electronic Delphi method enables expertsacross disciplines and geographical locations toanonymously contribute valuable knowledge andexperience through a structured and iterativefeedback process.

▪ Participant sample is determined by the willing-ness of healthcare and design experts to partici-pate in questionnaire surveys, which can provechallenging due to common time constraints.

Blaschke S, et al. BMJ Open 2017;7:e013527. doi:10.1136/bmjopen-2016-013527 1

Open Access Protocol

on 10 Novem

ber 2018 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-013527 on 8 March 2017. D

ownloaded from

engagement with, nature presents an often underappre-ciated health resource3 and could be considered anopportunity to broaden health-centric care strategies:“contact with nature may offer an affordable, accessibleand equitable choice on tackling the imminent epi-demic, with both preventive and restorative [public]health strategies”.4 In this context, Nightingale’s seminaland timeless instructions for ‘those who have personalcharge of the health of others’ are still relevant forhealthcare givers and receivers today: “What nursing hasto do …is to put the patient in the best condition fornature to act upon him”5 (p. 133). Preliminary empir-ical evidence from cancer populations show variousbiopsychosocial benefits from contact with nature incancer settings, including improved quality of life,6

increased positive health behaviour such as physicalexercise and fruit and vegetable consumption,7 restoredattention8 and increased social interaction.9

RationaleHealthcare setting design represents an expensiveintersection of healthcare industry and infrastructureas well as potential opportunities for healthcareimprovements10 and increased consumer satisfaction.11

Opportunities to connect with nature are attractinginterest in healthcare setting and service design. Suchhealthcare innovations can widen the horizon of pos-sible solutions to growing healthcare burden butrequire deliberate and rigorous investigation to ensureresponsible action is taken and wastage avoided. Thiscomplex issue involves multiple governing bodies andstakeholders who have the task of innovating cost-efficient and high-quality healthcare that responds tohealth and recovery requirements of patients withcancer.The present study follows from phase 1 qualitative

research into use of nature by patients with cancer andits relevance in their experiences of health and recov-ery, which uncovered their own recommendations forintegrating nature engagement opportunities in health-care. Our preliminary findings report positive health–nature interchanges for patients with cancer andsupport further investigation to strategically determinethe opportunities for, and barriers to, safe delivery ofbeneficial nature engagement in cancer care contexts.To evaluate the feasibility of integrating nature engage-ment opportunities into healthcare, a synthesis ofopinion from a range of experts is needed. This proto-col outlines a study designed to solicit input from rele-vant experts drawn from a range of professional andacademic roles (including cancer-specific experts,where relevant) and explore factors they deem criticallyimportant for the provision of nature-based engage-ment in cancer care settings. To the best of our knowl-edge, no such collection and synthesis of expertopinion on this topic exists across healthcare anddesign disciplines.

AimThe primary aim is to solicit knowledge from relevanthealthcare and design experts in order to explore levelsof opinion consensus about opportunities for, and bar-riers to, providing nature engagement in cancer caresettings.

METHODS AND DESIGNFigure 1 illustrates the study flow according to the modi-fied Delphi methodology adopted in this study, whichstructures an iterative feedback process using a predeter-mined number of four questionnaires (rounds) ratherthan using as many as needed to reach strict consensus.First, an open-ended questionnaire is administered to an‘expert panel’ with the aim to uncover salient issues(items) with relevance to the topic, which are subse-quently verified and finally ranked according to theirpriority reflecting the relative degree of consensusamong the panel. The protocol and related study mate-rials were designed following the SPIRIT 2013Checklist12 where appropriate.

Rounds and timelineFollowing Okoli and Pawlowski’s13 recommendation, thefour-round Delphi will aim to collect rich data, consoli-date ranging expert opinion and indicate levels of con-sensus. Round 1 serves idea generation, round 2 verifiessummaries of responses, round 3 short-list items of prior-ity and round 4 ranks prioritised items. The four ques-tionnaires will be electronically administered via email.All rounds are planned to take 4 weeks:14 2–3 weeks forpanellists to respond (including reminder emails priorto the round closing deadline to maintain a highresponse rate), and 1 week to analyse response data and,based thereon, draft the next questionnaire.

QuestionnairesDelphi is a form of iterative enquiry that builds onongoing data collection. Its primary research tool is aseries of questionnaires built from participants’ stepwiseinput. Questionnaire 1 will be available for distributionat the start of recruitment and questionnaires 2–4 aresubsequently created to reflect content from theongoing data collection. Questionnaire 1, section A firstintroduces the recommendations of patients with cancerdrawn from our preceding investigation. Section B willquery experts’ ideas and perceptions about opportun-ities for nature engagement in the cancer care settingand ask for factors they perceive as barriers to its provi-sion. Questionnaire 1 (item generation) will take notmore than 15 min to complete and questionnaires 2–4(verification and ranking) will take not more than10 min to complete unless the panellists wish to elabor-ate. Questionnaire 1 will be pilot-tested by two to threeresearchers unfamiliar with the Delphi method whowill be asked to provide feedback about theirquestion-and-answer process when completing the

2 Blaschke S, et al. BMJ Open 2017;7:e013527. doi:10.1136/bmjopen-2016-013527

Open Access

on 10 Novem

ber 2018 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-013527 on 8 March 2017. D

ownloaded from

questionnaire.15 This is to ensure questionnaire 1 is com-prehensible to Delphi responders and that the intendedscope and quality of response will be achieved.

AnonymityThe level of anonymity and confidentiality appropriatefor this study is termed ‘quasi-anonymity’,16 whichdenotes that responses will remain anonymised through-out the study and are known only to the researchers.

Since the panel constitutes experts from professionaland academic backgrounds only, there is no need toadopt the common strictures of anonymity requiredwhen involving patients with cancer. Panel members willbe blinded to each other’s responses throughout theDelphi process but can be known to each other as panelmembers. It will be clearly stated that publications willnot reference any personally identifiable participantinformation.

Figure 1 Design of modified four-round Delphi.

Blaschke S, et al. BMJ Open 2017;7:e013527. doi:10.1136/bmjopen-2016-013527 3

Open Access

on 10 Novem

ber 2018 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-013527 on 8 March 2017. D

ownloaded from

PARTICIPANTSSelection of expertsNational and international experts in academic and pro-fessional roles will be selected based on relevant back-grounds who possess both knowledge and experiencerepresentative of the capacity to articulate informedopinion and provide relevant input about the giventopic. Nature engagement in cancer care is a novel topicand thus requires tapping into related expert groups,which diversely address healthcare architecture anddesign and supportive cancer care. Relevant professionalbackgrounds include, for example, oncology and alliedhealthcare practitioners, management representativesworking in healthcare planning and development, andhealthcare setting architects and designers. Academicsand educators may be included if they have taught parti-cipants on health–nature and related healthcare designtopics or have published and presented related researcharticles in academic forums.The identification process uses two strategies. First, the

researchers’ own expert networks will be used and con-sulted for referrals to potential study participants (snow-balling). Second, we will follow Delbecq et al’s17

guidelines for identifying experts for nominal groupstudies, which will increase rigour in recruiting relevantindividuals outside the researchers’ own networks. Thisprocedure has shown to be transferable to Delphistudies13 14 and includes identifying relevant disciplines,sectors, and organisations and retrieving relevant aca-demic and practitioner literature in order to build anexpert list. The following predefined inclusion criteriahave been previously adopted in Delphi panel recruit-ment14 and will supplement the above selection proce-dures: (1) capable of contributing relevant input(knowledge and experience); (2) willingness and suffi-cient time to complete all four rounds; and (3)

sufficient English skills to communicate ideas effectively.Please see the Discussion section for the definition of‘expert’ used in this study.

Sample sizeThe recruitment target is a minimum of 40 expertsaccounting for 10 experts per group (healthcare practi-tioners, management representatives, designers,researchers). This will allow for diversity of views andreveal any divergence of opinion between groups, whilemaintaining a volume of responses that is manageable toprocess. The sample target takes into account that notall participants are expected to complete all four rounds(attrition) and that a minimum of seven panellists (foreach group) are required for reliable outcomes andcomparisons.18 To achieve the minimum sample size, amaximum of 200 experts will be invited to participate.

RecruitmentIdentified experts will receive an email containing an invi-tation to participate, a participation information sheet andquestionnaire 1. Passive consent is given by responding tothe email and returning questionnaire 1. Participation isvoluntary and can be withdrawn at any stage. Participantscan request their demographic information and wherepossible other contributions to be withdrawn; however,due to the study’s iterative process not all contributionscan be withdrawn once included in previous rounds.Reasons for declining will be recorded if provided.

DATA COLLECTION AND ANALYSISProcedureQuestionnaires will be electronically administered viaemail according to Schmidt’s19 sequence detailed infigure 2 below.

Figure 2 Delphi questionnaire

administration process (adapted

from Schmidt et al32).

4 Blaschke S, et al. BMJ Open 2017;7:e013527. doi:10.1136/bmjopen-2016-013527

Open Access

on 10 Novem

ber 2018 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-013527 on 8 March 2017. D

ownloaded from

Phase 1The initial phase constitutes creative brainstorming andaims to elicit a maximum variety of items, before quanti-tatively ranking them.

Questionnaire 1: generation of itemsThis questionnaire will be sent on the same day theexpert accepts participation. Section A constitutes asummary of anonymised recommendations of patientswith cancer and cautions related to nature engagementextracted from the preceding qualitative investigation.Section B asks two basic, open-ended questions request-ing experts to list at least six items (as recommended bySchmidt19) for questions 1 and 2 followed by brief expla-nations of their chosen items. These follow:1. List at least six items relevant to your expertise

describing design features, applications, initiatives orcare practices related to nature engagement, whichhealthcare and design practitioners could feasiblyimplement within the cancer care context.This list seeks to generate a list of design and healthcareopportunities (opportunities list).

2. List at least six important barriers or risk factors that youbelieve affect the provision of nature opportunities incancer care contexts. These can include, for example,physical, psychosocial, economic or political factors.This question seeks to generate a list of barriers and key riskfactors related to the provision of nature opportunities (bar-riers list).Additionally, experts will be asked to offer a brief

explanation of the importance of their suggested item.Space will be provided below each item for free-textdescription.

Analysis (questionnaire 1)All data (items and explanations) will be entered andmanaged in qualitative data analysis software Nvivo V.10for MacIntosh (QSR I. NVivo qualitative data analysissoftware for Macintosh, version 10: QSR InternationalPty Ltd; 2014 2014). The analysis will first remove identi-cal responses, then collate, synthesise and edit remain-ing ideas to achieve consistent terminology of itemsexpressing similar ideas and, finally, logically groupitems into emerging categories. An inter-rater process

will assist interpretative congruity as recommended forthematic analysis.20

Questionnaire 2: validation of categorised itemsThis questionnaire will be designed based on responsesfrom round 1 and aims to strengthen construct validity13

according to the concept of ‘member checking’.20 Allitems generated thus far will be collated into meaningfulcategories, as produced by inter-rater agreement, and willbe recirculated to all experts. Each item is presented witha one-sentence explanation and non-identifiable back-ground information of the panel member who generatedthe item (figure 3). A brief summary of the commentsfrom round 1 is provided. Experts will be asked to:1. Verify correct and fair interpretation of their

responses and that items have been placed in anappropriate category;

2. Verify and, if they wish, refine the categorisations andrecommend additional items.

Analysis (questionnaire 2)Based on responses, items will be further refined andagain subjected to inter-rater discussion.

Phase 2In this phase, panellists will state their priorities and listswill be condensed accordingly.

Questionnaire 3: prioritising itemsQuestionnaire 3 uses a structured format and will list theitems generated thus far in random arrangement tominimise response bias. Each panellist will be asked toselect 10 items (top 10) from each list (opportunitiesand barriers), which s/he deems relevant and critical tothe consideration of nature opportunities in the cancercare setting. Items 1–10 are selected according to theirimportance as judged by the expert who is asked toassign ‘1’ to the most important item, ‘2’ to the secondranked item and so on (figure 4).

Analysis (questionnaire 3)Items selected by the majority of experts will be aggre-gated representing a majority vote. Lists will be reduced

Figure 3 Example of

questionnaire 2 layout.

Blaschke S, et al. BMJ Open 2017;7:e013527. doi:10.1136/bmjopen-2016-013527 5

Open Access

on 10 Novem

ber 2018 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-013527 on 8 March 2017. D

ownloaded from

according to the importance of items calculated basedon the sum of points allocated by each expert to theirtop 10 items, that is, item ‘1’ indicating highest import-ance is coded with 10 points, item ‘2’ coded with 9points and so on. As recommended by Schmidt,19 toavoid burdening panellists with too many items, thetarget size of total items for the final round will be nomore than 20 items for each list (opportunities andbarriers).

Phase 3The aim of this phase is to elicit levels of agreementamong all experts and detect any diverging opinionbetween different expert groups.

Questionnaire 4: ranking itemsQuestionnaire 4 is designed to elicit levels of consensus(not achieve consensus) in the ranking of relevant items.This questionnaire includes aggregated statistical groupresponses generated for each included item thus far: thetotal sum of points assigned to each item by the entirepanel; individual panellists’ own round 3 response and asummary of comments provided thus far (figure 5). Eachpanellist will individually submit a rank ordering of theitems for each of the condensed list (opportunities andbarriers). Each item is presented with a corresponding10-point Likert-type scale (1=not important at all,10=very important) and an option to indicate ‘no judge-ment’ for any given item including space for justification.

Analysis (questionnaire 4)Statistical analyses will be performed using IBM SPSSStatistics V.23 for Macintosh (IBM C. IBM SPSS Statisticsfor Macintosh, version 23. Armonk, NY: IBM Corp;2013). Descriptive statistics (median and mean scores)

will be calculated to indicate items’ relative importance.Descriptives will be calculated for the full sample and byexpert group. The study’s aim is to explore levels of con-sensus rather than achieve consensus. Consensus will bedefined as 75% agreement.21

Finally, if further understanding of qualitativeresponses is required, a small number of one-on-onefollow-up interviews will be conducted with experts toclarify any ambiguity and gain a fuller understanding offinal results. Experts will be informed in the participa-tion information sheet that they may be invited to par-ticipate in a voluntary follow-up interview at studycompletion.

ETHICSEthics for this study was gained from the Institution’sHuman Research Ethics Committee (blinded for review).All consented participants will be assigned a unique iden-tification code. Collected demographic information andcontact details will include name, contact phone number,email address, description of professional role, yearsserved in field of expertise, country of professional resi-dence/affiliation. Participants’ identifiable informationwill be matched with their unique identification code in adigital master file only. All data collected will be storedsafely and securely in locked filing cabinets and inpassword-protected folders on a secure drive (electronicdata) that can be accessed only by the study investigators.Data will be kept for 5 years as per local guidelines.

DISCUSSIONAppropriateness of methodThe Delphi method is an established research tool forcomplex problem solving, which solicits expert opinionthrough a structured, iterative process.19 Its original

Figure 4 Example of

questionnaire 3 layout.

Figure 5 Example of

questionnaire 4 layout.

6 Blaschke S, et al. BMJ Open 2017;7:e013527. doi:10.1136/bmjopen-2016-013527

Open Access

on 10 Novem

ber 2018 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-013527 on 8 March 2017. D

ownloaded from

purpose was to obtain converging consensus about rela-tive priorities in a given topic through a progression ofiterative questionnaires based on controlled feedback22

until statistical census is reached. Since its inception,the Delphi method evolved to address a variety ofresearch problems such as eliciting degrees of agree-ment, delineating differing group attitudes and posi-tions, or understanding the rationales of particularjudgements and opinions.18 23 Delphi variants appliedto such explorative enquiry include, for example, modi-fied, exploratory, ranking and policy Delphis, whichare particularly well suited for investigating areas wherelittle prior knowledge exists;24 where empirical data arelacking;25 and where cursory understanding of groupattitudes and priorities is desired.18 The present studyaims to guide concept development and elicit levels ofconsensus among diverse disciplinary viewpoints inorder to generate new care opportunities related tonature engagement in the cancer care setting. Thebest-suited variant for this purpose is the modifiedelectronic Delphi with a predefined four-round design,which provides the following key advantages: (1) servesthe dual purpose of soliciting broad expert opinion fol-lowed by priority ranking;13 (2) can conclude at a pre-defined number of rounds because strong consensus isnot required when degrees of agreement and groupattitudes are of interest;18 (3) structures a rigorousand rapid feedback-based (online) communicationprocess;26 (4) frees communication from logistical chal-lenges, peer pressure and ‘group-think’ scenarios;27

and (5) cross-pollinates multidisciplinary expertiseachieving broader understanding than would bereached from a single discipline alone.13 The method’sflexibility and ability to easily assemble and coordinateparticipants across disciplines and geographical loca-tions partly explain its growing popularity in medicaland nursing research.16 Mullen18 reports its use inmedical, health service and nursing research for “fore-casting developments in medicine and health technolo-gies”, and “identifying priorities for nursing researchand also priorities for spending and service develop-ments” (p. 49). Relevant to this study are two examplesshowing its application in the cancer context for gather-ing international input for developing pain assessmenttools for palliative care,28 and engaging healthcareexperts from diverse backgrounds with experience insurvivorship care to develop realistic strategies forimproving healthcare for cancer survivors.29

Definition of an ‘expert’An important component of the Delphi method is theidentification of experts. There are no specific standardsfor identifying experts and ‘expertness’ is variouslydefined in different Delphi studies. This presents amajor criticism of the Delphi method22 30 and thereremains little consensus as to what constitutes expertnessand how it is operationally defined.18 22 The dictionarydefinition of an expert, “a person who is very

knowledgeable about or skillful in a particular area,”31

has been found insufficiently instructive for assemblinga Delphi expert panel.22 Consequently, studies haveemployed broader terms to identify and include relevantexperts including ‘informed advocates’,18 ‘informedindividuals’, ‘specialist in their field’ or persons with‘knowledge about a specific subject’.16 Central to theseformulations is the description of individuals whopossess both knowledge and experience representativeof the capacity to articulate informed opinion andprovide relevant input about a given topic, which will bethis study’s working definition of an expert.

Composing the expert panelDelphi studies can use homogeneous or heterogeneousexpert panels depending on the study aim. A heteroge-neous panel of experts can bring a range of disciplinaryviewpoints to the surface and articulate greater complex-ities as well as the boundaries of the topic at hand. Inregard to innovation, Mullen cites that “many innovationsand real breakthroughs…occur from outside a disciplineor specialty” (cited in ref. 18, p. 42) suggesting that cross-pollination of diverse disciplines and backgrounds canproduce insightful and fruitful enquiry. Based on theseprecepts, five groups of diverse yet relevant stakeholdershave been identified in the area of cancer care innov-ation: (1) patients with cancer; (2) healthcare practi-tioners; (3) healthcare management; (4) healthcaresetting designers and (5) researchers. The panel will becomposed of healthcare practitioners, managementrepresentatives, designers and researchers.Patients’ recommendations were drawn from the pre-

ceding qualitative phase 1 study and are presented tothe panel in round 1. The rationale for not recruitingadditional patients with cancer is twofold. First, thepresent study builds on a substantial amount of dataalready collected from qualitative interviews elicitingpatient experiences, suggestions, recommendations andcautions related to nature engagement. Second, of inter-est, are the responses and perceptions of those who bearon decision-making and healthcare policy developmentto ascertain the feasibility and realistic limitations of pro-viding opportunities for nature engagement in thecancer care setting. The strategy of using the own natureexperiences and recommendations of patients withcancer for opportunities to engage with nature in oncol-ogy contexts, to form the basis for this investigation, pro-vides experts with the opportunity of considering theperspectives of patients with cancer when developingtheir own views about opportunities for, and barriers to,providing helpful nature engagement in cancer care set-tings. It is possible that their agreement or disagreementwith patients’ perspectives may affect the study’s findingsand recommendations.

Determining sample sizeDelphi studies have been conducted with varying panelsizes ranging from single digits to low hundreds.18 The

Blaschke S, et al. BMJ Open 2017;7:e013527. doi:10.1136/bmjopen-2016-013527 7

Open Access

on 10 Novem

ber 2018 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-013527 on 8 March 2017. D

ownloaded from

absence of strict guidelines allows individual researchprojects to determine panel sizes according to theirpurpose and limitations.16 However, the most reliableDelphi studies were conducted with fewer than 20 parti-cipants.18 Recommendations suggest populating panelswith 10–18 experts for sufficient input to warrant mean-ingful elicitation of diverse disciplinary viewpoints.13 18

Seven is considered an acceptable minimum panel sizewith accuracy rapidly declining as the number becomessmaller.18 It is understood that the levels of censusamong experts are of more interest than the power offrequencies of response,19 32 which is often misunder-stood when mistaking the Delphi method for a quantita-tive survey.18

Level of anonymityOne of Delphi’s defining features and strengths is theanonymity of responses. Mullen18 states that preservinganonymity in Delphi “removes effects of status, powerfulpersonalities and group pressure” (p. 46–47). Keeneyet al16 note that anonymity “facilitates respondents to beopen and truthful about their views” (p. 197). Varyingdegrees of anonymity have been used in Delphi studies.Some studies have adhered to strict criteria such asanonymising responses to researchers themselves andblinding panellists to one another’s identity.18 There isno agreed level of anonymity or de-identification otherthan preserving “the anonymity of responses…for atleast part of the study”18 (p. 47). Advantages of panel-lists knowing each other’s identities include greatermotivation to engage because of association with prom-inent experts, stimulating exploratory thinking and ideageneration, and introducing greater accountability forconsidered personal responses and the overall Delphistudy’s outcome.18 The present study will make use ofthese advantages and also acknowledge the known factthat complete blinding can be unrealistic becauseexperts might know each other outside the study.16

Summary of strengths and limitationsIn summary, the key strengths of the Delphi method areits flexibility to modify the study procedures (eg,number of rounds) to suit the study context; the abilityto bring together experts from diverse backgrounds andlocations; and participant anonymity to stimulate a freeflow of ideas. The limitations include reliance on expertparticipants who may have limited time to contribute;the lack of a common and robust definition of ‘expert-ness’ in the Delphi literature; and the identification andrecruitment of sufficient suitable experts when consider-ing a low response rate in Delphi studies.

DISSEMINATION PLANParticipants will be sent a summary of results at conclu-sion of the final phase. Presentation of results willinclude the total number of items generated in phase 1and the strength of the items taken into phase 2. Levels

of consensus will be tabled and sufficient raw data pro-vided (eg, number of panellists in each round) tosupport calculation of statistics. A summary of non-identifiable demographics will be presented to validatethe participation of relevant and qualified experts.Based on the findings, it will be possible to revise thetheoretical understandings and practical patient recom-mendations formulated in phase 1. Preliminary expertrecommendations can be drafted for testing in thecancer care context, and propositions can be generatedto inform future research. It is anticipated that theresults of this research project will be published in peer-reviewed journals and presented in a variety of forums,and form part of the principal investigator’s dissertation.

Author affiliations1Department of Cancer Experiences Research, Peter MacCallum CancerCentre, East Melbourne, Victoria, Australia2Faculty of Medicine, Sir Peter MacCallum Department of Oncology, Dentistryand Health Sciences, The University of Melbourne, Parkville, Victoria, Australia3Department of Medicine, St Vincent’s Hospital, The University of Melbourne,Melbourne, Victoria, Australia4Palliative Care Service, Cabrini Health, Melbourne, Victoria, Australia5Faculty of Health Sciences, Department of Psychology, Swinburne Universityof Technology, Melbourne, Victoria, Australia

Correction notice This paper has been corrected since it first publishedonline. The first author’s name has been corrected.

Contributors SB is the principal investigator who conceived the study anddrafted the initial protocol manuscript. SB, CCOC and PS are responsible forthe design of the study and CCOC and PS critically revised the manuscriptand related study materials. All authors have read and approved the finalmanuscript.

Funding SB received a doctoral scholarship during the study period(Australian Postgraduate Award).

Competing interests None declared.

Ethics approval Peter MacCallum Human Research Ethics Committee.

Provenance and peer review Not commissioned; externally peer reviewed.

Open Access This is an Open Access article distributed in accordance withthe Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license,which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, providedthe original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

REFERENCES1. Stewart B, Wild C. World Cancer Report 2014. International Agency

for Research on Cancer. World Health Organization, 2014.2. Antonovsky A. The salutogenic model as a theory to guide health

promotion. Health Promot Int 1996;11:11–18.3. Herzog TR, Chen HC, Primeau JS. Perception of the restorative

potential of natural and other settings. J Environ Psychol2002;22:295–306.

4. Maller C, Townsend M, Pryor A, et al. Healthy nature healthy people:‘contact with nature’ as an upstream health promotion interventionfor populations. Health Promot Int 2006;21:45–54.

5. Nightingale F. Notes on nursing: what it is, and what it is not.New York: Dover Publications Inc, 1969.

6. Rowlands J, Noble S. How does the environment impact onthe quality of life of advanced cancer patients? A qualitativestudy with implications for ward design. J Palliat Med 2008;22:768–74.

7. Blair CK, Madan-Swain A, Locher JL, et al. Harvest for healthgardening intervention feasibility study in cancer survivors. ActaOncol 2013;52:1110–18.

8 Blaschke S, et al. BMJ Open 2017;7:e013527. doi:10.1136/bmjopen-2016-013527

Open Access

on 10 Novem

ber 2018 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-013527 on 8 March 2017. D

ownloaded from

8. Cimprich B, Ronis DL. An environmental intervention to restoreattention in women with newly diagnosed breast cancer. CancerNurs 2003;26:284–92.

9. Sherman SA, Varni JW, Ulrich RS, et al. Post-occupancy evaluationof healing gardens in a pediatric cancer center. Landsc Urban Plan2005;73:167–83.

10. Ulrich RS, Zimring C, Zhu XM, et al. A review of the researchliterature on evidence-based healthcare design. Herd Health EnvironRes Des J 2008;1:61–125.

11. Whitehouse S, Varni JW, Seid M, et al. Evaluating a children’shospital garden environment: utilization and consumer satisfaction.J Environ Psychol 2001;21:301–14.

12. Chan AW, Tetzlaff JM, Gøtzsche PC, et al. SPIRIT 2013 explanationand elaboration: guidance for protocols of clinical trials. BMJ2013;346:e7586.

13. Okoli C, Pawlowski SD. The Delphi method as a research tool: anexample, design considerations and applications. Inf Manage2004;42:15–29.

14. Tetzlaff JM, Moher D, Chan AW. Developing a guideline for clinicaltrial protocol content: Delphi consensus survey. Trials 2012;13:176.

15. Collins D. Pretesting survey instruments: an overview of cognitivemethods. Qual Life Res 2003;12:229–38.

16. Keeney S, Hasson F, McKenna HP. A critical review of the Delphitechnique as a research methodology for nursing. Int J Nurs Stud2001;38:195–200.

17. Delbecq AL, Van de Ven AH, Gustafson DH. Group techniques forprogram planning: a guide to nominal group and Delphi processes.Glenview, IL: Scott, Foresman, 1975.

18. Mullen PM. Delphi: myths and reality. J Health Organ Manag2003;17:37–52.

19. Schmidt RC. Managing Delphi surveys using nonparametricstatistical techniques. Decision Sci 1997;28:763–74.

20. Strauss A, Corbin JM. Basics of qualitative research: groundedtheory procedures and techniques. Sage Publications, Inc, 1990.

21. Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: asystematic review recommends methodologic criteria for reporting ofDelphi studies. J Clin Epidemiol 2014;67:401–9.

22. Baker J, Lovell K, Harris N. How expert are the experts? Anexploration of the concept of ‘expert’ within Delphi panel techniques.Nurse Res 2006;14:59–70.

23. Critcher C, Gladstone B. Utilizing the Delphi technique in policydiscussion: a case study of a privatized utility in Britain. Public Adm1998;76:431–49.

24. Hardy DJ, O’Brien AP, Gaskin CJ, et al. Practical application of theDelphi technique in a bicultural mental health nursing study in NewZealand. J Adv Nurs 2004;46:95–109.

25. McKenna HP. The Delphi technique: a worthwhile researchapproach for nursing? J Adv Nurs 1994;19:1221–5.

26. Linstone HA, Turoff M. The Delphi method: techniques andapplications. MA: Addison-Wesley Reading, 1975.

27. Hasson F, Keeney S. Enhancing rigour in the Delphi techniqueresearch. Technol Forecast Soc 2011;78:1695–704.

28. Biondo PD, Nekolaichuk CL, Stiles C, et al. Applying the Delphiprocess to palliative care tool development: lessons learned.Support Care Cancer 2008;16:935–42.

29. Mertens AC, Cotter KL, Foster BM, et al. Improving health care foradult survivors of childhood cancer: recommendations from a Delphipanel of health policy experts. Health Policy 2004;69:169–78.

30. Crisp J, Pelletier D, Duffield C, et al. The Delphi method? Nurs Res1997;46:116–18.

31. Pearsall J, Hanks P. The New Oxford Dictionary. 2nd edn. Oxford:Oxford English Press, 2003.Expert.

32. Schmidt R, Lyytinen K, Mark Keil PC. Identifying software projectrisks: an international Delphi study. J Manage Inform Syst2001;17:5–36.

Blaschke S, et al. BMJ Open 2017;7:e013527. doi:10.1136/bmjopen-2016-013527 9

Open Access

on 10 Novem

ber 2018 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-013527 on 8 March 2017. D

ownloaded from