RESEARCH ARTICLE Open Access

Target Product Profiles for medical tests asystematic review of current methodsPaola Cocco1 Anam Ayaz-Shah23 Michael Paul Messenger345 Robert Michael West6 and Bethany Shinkins135

Abstract

Background A Target Product Profile (TPP) outlines the necessary characteristics of an innovative product toaddress an unmet clinical need TPPs could be used to better guide manufacturers in the development of lsquofit forpurposersquo tests thus increasing the likelihood that novel tests will progress from bench to bedside However there iscurrently no guidance on how to produce a TPP specifically for medical tests

Methods A systematic review was conducted to summarise the methods currently used to develop TPPs formedical tests the sources used to inform these recommendations and the test characteristics for which targets aremade Database and website searches were conducted in November 2018 TPPs written in English for any medicaltest were included Based on an existing framework test characteristics were clustered into commonly recognisedthemes

Results Forty-four TPPs were identified all of which focused on diagnostic tests for infectious diseases Three coredecision-making phases for developing TPPs were identified scoping drafting and consensus-buildingConsultations with experts and the literature mostly informed the scoping and drafting of TPPs All TPPs providedinformation on unmet clinical need and desirable analytical performance and the majority specified clinical validitycharacteristics Few TPPs described specifications for clinical utility and none included cost-effectiveness

Conclusions We have identified a commonly used framework that could be beneficial for anyone interested indrafting a TPP for a medical test Currently key outcomes such as utility and cost-effectiveness are largelyoverlooked within TPPs though and we foresee this as an area for further improvement

Keywords Medical test Target Product Profile TPP Quality by design Diagnostic Test characteristic

BackgroundDespite significant advances and innovation in the fieldof disease detection the majority of the resulting tech-nologies fail to be translated into tests that are used inclinical practice [1 2] For instance it is estimated thatless than 1 of novel cancer biomarkers actually reachclinical practice [3] One possible driver is that thedevelopment of new tests is often driven by laboratory

discoveries rather than by clinical needs [4] Test manu-facturers have to fulfil extensive evidence requirementsdemonstrating the fitness of their test for clinical prac-tice [5] A poor understanding of unmet clinical needsand the clinical pathway within which a test will sit willoften mean that the clinical and economic benefits can-not be convincingly demonstrated and hence the testmay fail to be adopted into clinical practice [1]Under the lsquoQuality by designrsquo framework a new prod-

uct is designed with the aim of meeting pre-identifiedquality objectives [6] A Target Product Profile (TPP)also known as a Quality Target Product Profile (QTPP)is a strategic document which summarises the necessarycharacteristics of an innovative product to address an

copy The Author(s) 2020 Open Access This article is licensed under a Creative Commons Attribution 40 International Licensewhich permits use sharing adaptation distribution and reproduction in any medium or format as long as you giveappropriate credit to the original author(s) and the source provide a link to the Creative Commons licence and indicate ifchanges were made The images or other third party material in this article are included in the articles Creative Commonslicence unless indicated otherwise in a credit line to the material If material is not included in the articles Creative Commonslicence and your intended use is not permitted by statutory regulation or exceeds the permitted use you will need to obtainpermission directly from the copyright holder To view a copy of this licence visit httpcreativecommonsorglicensesby40The Creative Commons Public Domain Dedication waiver (httpcreativecommonsorgpublicdomainzero10) applies to thedata made available in this article unless otherwise stated in a credit line to the data

Correspondence umpcleedsacuk1Test Evaluation Group Academic Unit of Health Economics Leeds Institutefor Health Sciences University of Leeds Leeds UKFull list of author information is available at the end of the article

Cocco et al BMC Medicine (2020) 18119 httpsdoiorg101186s12916-020-01582-1

unmet clinical need [7] TPPs exemplify the concept oflsquobeginning with the goal in mindrsquo establishing key fea-tures and performance specifications in advance to en-sure that the new product is developed to meet specifichealth-related goals [7 8] TPPs should also be seen aslsquolivingrsquo documents that can be refined and updated asadditional relevant information becomes available [7 9]TPPs could therefore be particularly useful when de-

signing lsquofit for purposersquo medical tests [10] they could beused during the development and manufacturing phaseto ensure that a new test meets pre-established oper-ational and performance requirements in line with un-met clinical need [7] Generating the required evidencefor a new test can take many years and significant in-vestment [11] TPPs therefore have great potential to beused as guiding documents for test developers to avoidlate-stage development failures and reduce researchwasteAlthough we are aware of some examples where TPPs

have been developed for medical tests to our knowledgethere is no formal guidance as to best practice methodsIn the USA guidance for developing TPPs is availablefor new pharmaceutical drugs [7] This guidance issuedby the US Food and Drug Administration (FDA) pro-vides an overview of the purpose and attributes of TPPsand which requirements for a new drug should be in-cluded [7] In this context TPPs are used as voluntarybriefing documents to stimulate discussion between themanufacturer and the FDA throughout the drug devel-opment process [7] The TPP itself outlines specific cri-teria that a new drug should meet [7] However medicaltests differ from pharmaceuticals both in terms of theircharacteristics and in the indirect way in which they im-pact on patient health [5] and therefore this guidance isnot directly transferable to the context of medical testsHere we report a systematic review of the methods

currently used to develop TPPs for medical tests allow-ing us to (1) describe a commonly adopted methodologyframework (2) outline the test characteristics for whichtargets are often set and (3) identify areas requiring fur-ther methodological development

MethodsThe protocol for this review was registered on thePROSPERO database (CRD42018115133) [12]

Search strategyDetails of the full search strategy can be found inAdditional file 1 The following electronic databaseswere searched MEDLINE EMBASE CAB AbstractOnline CINHAL Global Health Scopus and Web ofScience The database search was performed in No-vember 2018 and encompassed a combination of key

terms such as lsquoTPPrsquo lsquoquality by designrsquo lsquoQTPPrsquo andlsquotestrsquoThe grey literature and websites were also searched

using structured methods proposed by Godin et al [13] Acustomised Google search was conducted to identify rele-vant websites and then each of these websites was hand-searched For each website the internal search engine wasused supplemented with hand-searching to identify poten-tially relevant references Duplicates across searches wereremoved This search was also conducted in November2018 For more details please see Additional file 1Table 13 Table 14 and 15All searches were conducted by PC and peer reviewed

by an information specialist

ScreeningTPPs written in English for any type of medical test wereincluded (eg imaging in vitro and in vivo medicaltests) There were no restrictions in terms of publicationdate All publication formats were included except fornewsletters and PowerPoint presentations as these didnot report the methods in sufficient detail to reviewthemEndnote was used to manage references Titles and ab-

stracts of the retrieved references were fully screened byPC based on the inclusion criteria of which a random10 sample was independently screened by BS TPPsthat met the inclusion criteria at this stage or those forwhich it was not possible to determine eligibility basedon title and abstract were then screened based on thefull text For those references where full text was notavailable we contacted authors All full texts of the eli-gible TPPs at this stage were screened independently byPC and BS based on the inclusion criteria The inter-reviewer agreement rate was calculated with Cohenrsquos κstatistic For more details please see Additional file 3Table 31 and 32 Where any disagreements occurred aconsensus-based discussion with the other authors(MM RW) determined whether the reference was eli-gible or not

Data extraction and analysisA data extraction spreadsheet was developed includingbasic descriptive information relating to the TPP (egpublication format disease of interest targeted clinicalsetting funder time horizon) Further to this we ex-tracted data on the methodology used to develop theTPP including details of the input sources (eg expertconsultation review of the literature) the reporteddecision-making process and the stakeholders involvedat each stage of the TPP development As a commondecision-making framework was apparent across theTPPs we summarised the input sources and stake-holders involved for each phase of this process Where

Cocco et al BMC Medicine (2020) 18119 Page 2 of 12

stakeholders and input sources for the drafting phasewere not explicitly reported for we assessed the sourcesincluded in each TPP table and the longer descriptionsof each test characteristicEach TPP was also assessed in terms of the transpar-

ency of reporting the adopted input sources decision-making process and which stakeholder groups wereconsultedAll data extraction was conducted independently by

PC and AAS and in case of disagreement BS MM andRW resolved any differences

Test characteristic clusteringIn addition to the information above the test character-istics reported within each TPP were extracted and de-duplicated Based on an existing evaluation frameworkfor tests (the ACCE framework [14]) two reviewers (BSand MM) independently categorised each of the testcharacteristics under the following outcomes (1) testdefinition (2) analytical performance (3) clinical validity(4) clinical utility (5) regulatory legitimacy and (6) eco-nomic acceptability The category lsquotest definitionrsquo speci-fies the disorder of interest target population and

purpose of the test and thus it overlaps with the con-cept of lsquounmet clinical needrsquo Therefore we renamed theoutcome lsquotest definitionrsquo as lsquounmet clinical needrsquo to bet-ter represent the type of information TPPs provideAnalytical performance describes the ability of a test

to correctly detect and measure a particular analyte (egprecision trueness analytical sensitivity and specificitylimits of detection) [15 16] Clinical validity is defined aslsquothe ability of a device to yield results that are correlatedwith a particular clinical condition or a physiological orpathological process or statersquo [15] whilst clinical utilityrepresents the ability of a test to affect relevant health-related outcomes for patients (eg improvement in qual-ity of life longer lifespan) [17]Some characteristics did not fall within any of the pre-

defined categories Three additional categories weretherefore identified to accommodate these additionalcharacteristics (7) human factors (8) environmental im-pact and (9) infrastructural requirements Human factorsare concerned with the interaction between users anddevices [18] Environmental impact encompasses achange to the environment following an interaction withthe product [19] Infrastructural requirements entail lsquothe

Fig 1 PRISMA flow diagram illustrating literature search results

Cocco et al BMC Medicine (2020) 18119 Page 3 of 12

stock of the basic facilities and equipment needed forrealizing a product or providing a servicersquo [20]

ResultsLiterature searchFull details of the literature search results are reportedin Fig 1 Forty-four TPPs were deemed eligible forinclusion in the systematic review [8ndash10 21ndash61] Inter-reviewer agreement was high at title and abstract (κ =96) and full-text screening (κ = 98) For more detailsplease see Additional file 3 Table 31 and 32

Features of included TPPsThe included 44 TPPs consisted of 23 reports 16 journalarticles 4 published TPP tables (a TPP without anybackground information or context eg [34]) and oneconference poster All TPPs provided guidance on devel-oping medical tests to detect infectious diseases Four-teen of the 44 TPPs focused on neglected tropicaldiseases (32) (eg soil-transmitted helminths Chagasdisease human African trypanosomiasis schistosomiasistrachoma taeniasis cysticercisosis) and on tests for

vector-borne infections (32) (eg Zika virus denguefever hepatitis C malaria E coli) Other types ofinfection included sexually transmitted infections (16n = 7) respiratory infections (14 n = 6) (eg lower re-spiratory tract infection tuberculosis pneumonia) Ebolavirus [57] meningitis [61] and severe febrile illness [8]Seven of the 44 TPPs were funded by Bill and Melinda

Gates Foundation (16) and three TPPs received fund-ing from WHO [8 48 49] The healthcare setting ofinterest was mostly low- and middle-income countriesThe majority of TPPs did not disclose funding sources(64 n = 28)In some TPPs a time horizon was chosen to represent

the timeframe within which achieving the specificationsdescribed in the TPP was considered feasible [22 23 60]In one TPP this was based on a landscape analysis [22]In another expected advancements in technologies andknowledge related to a certain field seemed to justify thetime horizon considered for the TPP [27] Of the 44 TPPsidentified 7 reported the time horizon during which theinformation included in the TPP will be relevant for man-ufacturers (16) Of these 6 TPPs stated a time horizon

Fig 2 Typical activities involved input sources and stakeholders invited for each decision-making phase

Cocco et al BMC Medicine (2020) 18119 Page 4 of 12

of 5 years [22 23 27 28 51 60] whilst the remainingconsidered a time horizon of 10 years [29]

Decision-making stepsA common decision-making framework consisting ofthree distinct phases was apparent across the includedTPPs scoping drafting and consensus-building Figure 2presents the most commonly adopted activities inputsources and engaged stakeholder groupsTable 1 provides a summary of the stakeholders con-

tributing to each phase Some of the included TPPs arenot included in Table 1 as they did not report any infor-mation related to input sources or stakeholder groups[33ndash36 52 53 55 61] A summary of the input sourcesreported to have been used at scoping and draftingphase can be found in Additional file 3 Table 33We will therefore describe the aim of each phase and

breakdown the methodology (activities input sourcesand stakeholders) used within the included TPPs wherereported For specific details on each included TPP seeAdditional file 3 Table 34

Scoping phase methodologyHalf of the TPPs provided some information on thescoping phase (n = 22) The aim of this phase was toprovide an overview of the disease area and the limita-tions associated with existing technologies The clinicalproblems and unmet needs were defined in addition toidentification of which test characteristics to include inthe TPPSome of the key activities undertaken during the scop-

ing phase included reviewing published literature (n = 6)or available data (n = 1) and introductory meetings withstakeholders (n = 4)Some authors reported (n = 4) [22 26 37 50] that

they had conducted a lsquolandscape analysisrsquo providing in-formation on the disease area of interest available diag-nostic technologies and related characteristics andlimitations These were usually based on interviews withstakeholders and reviews of the literature Only Toskinet al [50] conducted a systematic literature reviewreporting the databases searched and key words usedConsultation with experts (68 n = 15) and the litera-

ture (36 n = 8) were the most commonly sought

Table 1 Stakeholders contributing to each phase

Stakeholder groups contributing to each phasea Scoping Drafting Consensus-building

n () n () n ()

Researchers 14 (32) 15 (34) 14 (32)

Industry representatives 11 (25) 11 (25) 14 (32)

International public organisations 9 (20) 12 (27) 12 (27)

Clinicians 7 (16) 8 (18) 8 (18)

Representatives of countries and national disease programs 5 (11) 4 (9) 8 (18)

Policy makers 4 (9) 5 (11) 4 (9)

Laboratory experts 3 (7) 5 (11) 3 (7)

Technicalfunding agencies 2 (5) 3 (7) 4 (9)

Microbiologists 2 (5) 4 (9) 2 (5)

Implementers 2 (5) 2 (5) 5 (11)

Patient advocates 2 (5) 1 (2) 3 (7)

Non-profit sector 1 (2) 2 (5) 4 (9)

Modellers 1 (2) 1 (2) 1 (2)

Health economists 0 (0) 2 (5) 2 (5)

Donors 2 (5) 3 (7) 1 (2)

Market experts 2(5) 1 (2) 1 (2)

Program manager 1 (2) 2 (5) 2 (5)

Scientific associations 1 (2) 3 (7) 2 (5)

Strategists 1 (2) 1 (2) 0 (0)

Expert (unspecific) 1 (2) 3 (7) 3 (7)

Others 2 (5) 2 (5) 1 (2)

The percentages in relation to stakeholder groups do not add up to 100 because more than one stakeholder group usually contributes to the development ofa TPPaPercentages are calculated in relation to the total number of included TPPs (n = 44)

Cocco et al BMC Medicine (2020) 18119 Page 5 of 12

sources of information during the scoping phase (seeAdditional file 3 Table 33 for a full breakdown) Onlyone type of source was considered in 15 TPPs (of which11 was consulting experts) whilst 7 TPPs consideredmore than one sourceDenkinger et al [22] mapped the diagnostic ecosystem

of interest and then performed a survey to gauge stake-holdersrsquo preferences Reipold et al [48] identified themain characteristic categories (eg scope performanceoperational characteristics and pricing) to be included inthe TPPFive TPPs involved a priority-setting exercise which entailed

ranking each identified health need [23 28 32 48 60]During the scoping phase a variety of stakeholders

were engaged (Table 1)

Drafting phase methodologyThe first draft of each TPP was usually prepared by ei-ther an established working group comprising expertsfrom different organisations [9 26 29 32 40 50 58] orauthors of the published TPP There were two caseswhere the TPP was drafted by a completely different or-ganisation [51 57] The TPP was often revised severaltimes and in some cases it was then shortened to en-sure it could be easily communicated to different stake-holders [22 23 28 29 60]Of the 44 included TPPs 33 of them reported which

input sources were considered during the drafting phase(75) (Additional file 3 Table 33) Common inputsources for populating test characteristics were expertconsultations (n = 22) and reviews of the literature (n =22) Some also referred to mathematical models (n = 9)available data (n = 7) guidelines (n = 6) and lsquofield obser-vationsrsquo (n = 5) Only one TPP was informed by pooleddata from a systematic review [50]Twenty-six of the 44 TPPs took into consideration more

than one type of source at the drafting phase as opposedto 7 TPPs which only adopted one (Additional file 3Table 33) Meeting inputs were the most common singlesource (43 n = 3)The stakeholders engaged in the drafting phase are re-

ported in Table 1

Consensus-building phase methodologyInitial agreement with the TPP was often obtained usinga survey of the stakeholders (n = 14) The survey eitherincluded general questions regarding stakeholders viewson the TPP (n = 4) [22 25 27 51] or adopted a Delphi-like approach to provide an initial consensus on variousaspects of the TPP (n = 10) A consensus meeting withstakeholders and experts was typically held (n = 11) anda revised TPP generally agreed upon In some cases anadditional survey was sent to stakeholders on trade-offsbetween test attributes [48] or on rating key parameters

[51 53] For 2 TPPs the final TPP draft was presentedto a broader stakeholder base to validate itThe number of participants invited to the consensus-

building meetings varied (lt 20 participants n = 5 between20 and 50 participants n = 7) One meeting included 100participants [27] For a few of the TPPs the authors alsotook part in the consensus meetings [29 38 58 60]Less than half of the included TPPs reported information

on the activities and stakeholders invited to the consensus-building phase (n = 19) The stakeholders engaged in theconsensus-building phase are reported in Table 1

Transparency in reporting methodsWe also assessed the transparency of the TPPs in termsof reporting their methodology (see Additional file 3Table 35) The decision-making process behind the TPPwas not reported in over a quarter of the included TPPs(n = 16) Further to this many failed to report which in-formation sources were considered to populate the TPP(n = 11) Just under half did not report which stake-holders were involved in the development of the TPP(n = 20) Specifically the name of the organisationsstakeholders were part was only reported in 11 TPPswhilst 9 TPPs mentioned personal details of each stake-holder (20) and 4 TPPs explained why certain stake-holders were invited [26 38 46 58] Sixteen TPPsreported the source of funding (36)There were some TPPs where the methodology was

very clearly reported [26 28 38 60]

Test characteristics included in TPPsAfter removing duplicates 140 different test characteris-tics were reported across the included TPPs Some fea-tures which did not represent test characteristics havebeen excluded such as factors relating specifically to thedisease in question rather than the test For more infor-mation please see Test Characteristics Overview Excelspreadsheet (Availability of data and materials) Figure 3shows the test characteristics most frequently reported(a full list is available in Additional file 2 Table 21)Figure 4 depicts which characteristic categories were

reported in the included TPPs Details on unmet clinicalneed analytical performance and clinical validity ap-peared to be consistently reported however regula-tory requirements environmental footprint and clinicalutility were less frequently considered

DiscussionWe report a systematic review of the methods currentlyused to develop Target Product Profiles for medicaltests Despite TPPs for any medical test being searchedall of the identified TPPs were focused on diagnostictests for infection

Cocco et al BMC Medicine (2020) 18119 Page 6 of 12

Fig 3 Test characteristics frequently reported in all TPPs (n = 44) sorted by categories

Fig 4 Test characteristic categories in absolute number (n) of included TPPs

Cocco et al BMC Medicine (2020) 18119 Page 7 of 12

There was generally a lack of transparency andconsistency in reporting the methods underlying TPPsThis would make it difficult to appraise the recommen-dations within the TPPs ascertain whether the recom-mendations are generalizable to other settings andchallenging to reproduce

Relevancy of TPPs for test manufacturersThe purpose of a TPP is to identify upfront the essen-tial characteristics of a test for it to fulfil a pre-specified unmet clinical need This should in turn in-crease the likelihood that the test will be adopted intoclinical practice and reimbursed [1] A TPP should alsoaccount for contextual aspects that might affect thetestrsquos real-world performance [10] defining infrastruc-tural and technical constraints that impact on the im-plementation into clinical practice This review showsthat TPPs to date have primarily been developed forglobal health applications as the main funding organi-sations are WHO UNICEF and Bill and Melinda GatesFoundation The primary focus on infectious diseasesmay be explained by the remit of the global organisa-tions who fund TPPs WHO included HIVAIDSneglected tropical diseases tuberculosis and malaria aspriority diseases [62] Further to this WHO establishedthe lsquoRampD Blueprintrsquo which aims to promote RampD activ-ities (tests vaccines medicines) during epidemics [63]After having identified pathogen to target first TPPsare usually commissioned to guide the developmentprocess of new healthcare products which will addressthe high-priority pathogen [63]However the development of TPPs should not be limited

to one specific disease area or clinical setting the concept oflsquobeginning with an end in mindrsquo embodied by TPPs couldsupport both international and national health decision-makers This activity should in turn stimulate innovation ofnew tests driven by clinical needs rather than solely by la-boratory discoveries It would also provide manufacturerswith greater clarity around test requirements and confidencein the market for developing innovative tests

Identified limitations in current TPP methodologyIn reviewing current methodology for developing TPPsfor medical tests we have identified three key areaswhere current TPP methodology could be improved (1)oversight of clinical utility (2) a focus on price ratherthan cost-effectiveness and (3) subjectivity of informa-tion sources Here we discuss each limitation and theimplications

Oversight of clinical utilityVery few of the TPPs reported desirable characteristicsrelating to the clinical utility of the test This is not sur-prising given that the majority of research efforts has

focused on generating evidence on the analytical per-formance and diagnostic accuracy of a new test [11] Ahighly accurate test does not necessarily mean that thetest will improve patient health as factors relating todecision-making and the effectiveness of patient man-agement strategies could fall short [64]Assessing the clinical utility of a new test is ex-

tremely challenging Measuring the impact of a teston patient health outcomes is difficult as tests tend toguide patient management decisions rather than dir-ectly impacting on patient health outcomes [5]Therefore estimating the clinical utility of a test re-quires evidence of how the information from a test isincorporated into decision-making and the down-stream effectiveness of those decisions [65] In thecase of a new test this is particularly complicatedgiven the uncertainty around the mechanisms bywhich the test will impact on patient outcomes [65]

Focus on price rather than cost-effectivenessAlthough the minimum and optimal price of the testsfeatured in many of the TPPs none of these was drivenby the trade-off between the overall cost implications ofimplementing the test and the associated patient bene-fits Cost-effectiveness analysis provides a framework tocompare costs and benefits of an intervention againstrelevant comparators including current practice Spe-cifically cost-effectiveness analysis defines whether theintervention being evaluated represents good value formoneyIt is important to consider the cost of the new test in

the context of the benefits that the test may provide Forexample a new test may be relatively expensive but mayalso improve patient health to the extent that the add-itional is justified Conversely a new test may be rela-tively cheap but offer no improvements in patient healthand therefore even the marginal increase in cost is notjustifiedConducting cost-effectiveness analysis at early RampD

stages of new tests can therefore help manufacturers toavoid significant investments in tests that do not havethe potential to be cost-effective [65]These first two limitations are particularly relevant

since decision-makers increasingly demand evidencethat a new test improves patient health and is cost-effective rather than solely evidence of its analyticaland clinical validity [5] Specifically many HealthTechnology Assessment bodies in Europe Australiaand North America consider clinical utility cost andcost-effectiveness in relation to the target populationin addition to analytical performance and clinicalvalidity when assessing new molecular diagnostictests [66]

Cocco et al BMC Medicine (2020) 18119 Page 8 of 12

Subjectivity of input sourcesExpert judgement and evidence identified in publishedliterature were the main sources of information for de-fining desirable characteristics Systematic reviews of theliterature where database searches are reproducible andthe quality of relevant studies are appraised were notconducted to identify relevant evidence at the scopingand drafting phase This is likely to introduce bias andsubjectivity in terms of the evidence used to underpintest characteristic recommendationsAlthough expert judgement is undoubtedly useful

relying solely upon this information source has somelimitations particularly for quantitative estimates Howhumans make probability judgements is highly affectedby many heuristics and systematic biases (eg anchoringavailability overconfidence and insight bias) [67] Specif-ically previous literature has found a poor understand-ing of test accuracy among healthcare professionals [68]as sensitivity and specificity are often misinterpreted andmistaken for predictive values [68]Additionally the quality of expert elicitation heavily

relies on expert selection as it is important to choose ex-perts with good subject knowledge Only 4 TPPs de-scribed how the selection process took place andtherefore the quality of expert judgements might bequestioned Furthermore many TPPs reported literatureas a source for informing TPPs however less than halfof the TPPs cited the references considered This lack oftransparency might hinder the quality and credibility ofsources on which TPPs are based

Study limitationsSince this study is a systematic review of publicly avail-able literature a key study limitation is that we have in-evitably missed any confidential or unpublished TPPsdeveloped in-house by test manufacturers Although theresults of our online searches did not identify any com-panies stating that they have developed TPPs for medicaltests we would not expect to find such information oncompany websites Anecdotally however we have notencountered any formal TPP development activity (bythis we mean definition of desirable test characteristics)within the National Institute for Health Research LeedsIn Vitro Diagnostics Co-operative (NIHR Leeds) MICindustry networkAs there are no guidelines on how TPPs for medical

tests should be developed we did not formally assessrisk of bias We did however appraise the transparencywith which the methodology underpinning each TPPhad been reported Unfortunately it was not possible tofully evaluate the TPP developed by PATH [40] as theonline appendices were not accessible Additionally dueto poor methodological transparency it was difficult toassess with certainty authorship of TPPs and whether

authors of TPPs took part themselves in the consensus-building meetings

Future researchAlthough there is evidence of a common developmentframework this review highlights that there is consider-able variability in the methods employed to draft TPPsand inconsistencies in which test characteristics are de-scribed A key issue in reviewing the methods imple-mented was the lack of transparency in methodologyreportingGuidance on best practice methods for developing

TPPs for medical tests would be highly beneficial Simi-larly to the US FDA guidance on TPPs for drugs a guid-ance document could be developed for TPPs for medicaltests summarising the purpose attributes of TPPs andwhich test characteristics should be includedHowever to inform the development of such guidance

future research should focus firstly on how to systemat-ically identify unmet clinical needs underpinning a cer-tain disease area Monaghan et al [69] developed avaluable checklist for identifying biomarkers based onliterature findings and consultations with experts Webelieve that this checklist could be pertinent for thescoping phase underlying TPP development howeverthis would need further validation in this specificcontextMore research is also required to understand how to

better incorporate the assessment of desirable clinicalutility and cost-effectiveness of innovative tests intoTPPs One possible way forward could be exploring howand if care pathway analysis and early economic model-ling could be integrated into the development of TPPsCare pathway analysis would provide clarity on themechanisms by which a test could impact on down-stream patient outcomes Early economic modellingcould be used to define desirable values for certain testcharacteristics (eg test price diagnostic sensitivity andspecificity) based on cost-effectiveness [67] Thereforeintegrating care pathway analysis and early economicmodelling into TPP development might provide moreevidence-based information to the test developersOutside of the actual methodology for developing

TPPs it would be useful to better understand whethermanufacturers develop tests strictly in line with TPPs orwhether there are any factors which make this infeasibleor challenging Additionally we would be interested inwhich methods manufacturers usually adopt to developTPPs to assess if there are any differences with themethodological framework we highlighted here To thisend interviewing test manufacturers might provide in-teresting insights on the intrinsic value of TPPs for theindustry

Cocco et al BMC Medicine (2020) 18119 Page 9 of 12

Most importantly to ensure that the development ofTPPs becomes widespread practice it would be valuableto explore how TPPs could be integrated into existingregulatory paths for innovation such as the EuropeanUnion Regulation for In Vitro Diagnostics (Regulation2017746) and the US FDA Drug Development and Ap-proval Process It might then be possible to align testcharacteristics featured in TPPs with evidence require-ments which are relevant for market approval decisionsof new medical tests This in return might increase theapplicability of TPPs for the industry

ConclusionsBased on this review we summarised current methodo-logical practice into a framework of value to those inter-ested in developing TPPs for medical testsWe also identified some key weaknesses including the

quality of the information sources underpinning TPPsand failure to consider test characteristics relating toclinical utility and cost-effectivenessThis review thus provides some recommendations for

further methodological research on the development ofTPPs for medical test This work will also help to informthe development of a formal guideline on how to draftTPPs for medical tests

Supplementary informationSupplementary information accompanies this paper at httpsdoiorg101186s12916-020-01582-1

Additional file 1 Search strategies

Additional file 2 Test characteristics

Additional file 3 Results

AbbreviationsACCE Analytic validity Clinical validity Clinical Utility Ethical legal and socialimplications AIDS Acquired immune deficiency syndrome CEA Cost-effectiveness analysis FDA Food and Drug Administration FIND TheFoundation For Innovative New Diagnostics HIV Human immunodeficiencyvirus IDC International Diagnostics Center NIHR National Institute for HealthResearch PRISMA Preferred Reporting Items for Systematic Reviews andMeta-Analysis QTPP Quality Target Product Profile RampD Research andDevelopment TPP Target Product Profile UNICEF United NationsInternational Childrenrsquos Emergency Fund WHO World Health Organization

AcknowledgementsThe authors would like to express their gratitude to Natalie King (Universityof Leeds Information Specialist) for supporting the design of the searchstrategy for this systematic review

Authorsrsquo contributionsAll authors contributed to the conception and design of the work PCperformed the search strategy BS contributed to reference sifting whilst AASconducted the data extraction MM and RW solved any conflicts PC BS MMand RW interpreted the data and drafted and revised the manuscript Theauthors read and approved the final manuscript

FundingThis systematic review was carried out as a part of the full-time School ofMedicine PhD Scholarship awarded to Paola Cocco by the University ofLeeds Additionally this work is supported by the lsquoTackling AMR-A Cross

Council Initiativersquo (grant number MRN0299761) Funding Partners The Bio-technology and Biological Sciences Research Council the Engineering andPhysical Sciences Research Council and the Medical Research Council Thiswork is also supported by the Medical Research Foundations National AMRTraining Programme Bethany Shinkins and Michael Messenger are fundedby NIHR Leeds In Vitro Diagnostic Co-operative and Cancer Research UK viaCanTest Collaborative Anam Ayaz-Shah is funded by Cancer Research UK viaCanTest Collaborative

Availability of data and materialsThe datasets generated and analysed during the current study are availablein the University of Leeds repository httpsdoiorg105518781 Thesedatasets entail an Excel spreadsheet with data extraction and an overview oftest characteristics included in reviewed TPPs

Ethics approval and consent to participateNot applicable

Consent for publicationNot applicable

Competing interestsThe authors declare that they have no competing interests

Author details1Test Evaluation Group Academic Unit of Health Economics Leeds Institutefor Health Sciences University of Leeds Leeds UK 2Academic Unit ofPrimary Care Leeds Institute for Health Sciences University of Leeds LeedsUK 3httpswwwcantestorg 4Centre for Personalised Health and MedicineUniversity of Leeds Leeds UK 5NIHR Leeds In Vitro Diagnostic (IVD)Co-operative Leeds UK 6Leeds Institute for Health Sciences University ofLeeds Leeds UK

Received 4 December 2019 Accepted 1 April 2020

References1 Wang P Kricka L Current and emerging trends in point-of-care technology

and strategies for clinical validation and implementation Clin Chem 2018httpsdoiorg101373clinchem2018287052

2 Drucker E Krapfenbauer K Pitfalls and limitations in translation frombiomarker discovery to clinical utility in predictive and personalisedmedicine EPMA Journal 20134(1)4ndash7

3 Kern S Why your new cancer biomarker may never work recurrent patternsand remarkable diversity in biomarker failures Cancer Res 2012 httpsdoiorg1011580008-5472CAN-12-3232

4 Engel N Wachter K Pai M Gallarda J Boehme C Celentano I et alAddressing the challenges of diagnostics demand and supply insights froman online global health discussion platform BMJ Glob Health 2016 httpsdoiorg101136bmjgh-2016-000132

5 Horvath A Lord S St John A Sandberg S Cobbaert C Lorenz S et al Frombiomarkers to medical tests the changing landscape of test evaluation ClinChim Acta 2014 httpsdoiorg101016jcca201309018

6 US Food and Drug Administration Guidance for industry Q8 (2)pharmaceutical development 2009 httpacademygmp-complianceorgguidemgrfiles9041FNL5b15dPDF Accessed 6 Nov 2018

7 US Food and Drug Administration Guidance for industry and review staffTarget Product Profile mdash a strategic development process tool 2007httpswwwfdagovmedia72566download Accessed 6 Nov 2018

8 WHO FIND MsF A multiplex multi-analyte diagnostic platform 2017httpswwwwhointmedicinesTPP_intro_20171122_forDistributionpdfAccessed 8 Nov 2018

9 PATH Target Product Profile HIV self-test version 41 a white paper on theevaluation of current HIV rapid tests and development of core specificationsfor next-generation HIV tests 2014 httpspathazureedgenetmediadocumentsTS_hiv_self_test_tpppdf Accessed 8 Nov 2018

10 Ebels K Clerk C Crudder C McGray S Magnuson K Tietje K et alIncorporating user needs into product development for improved infectiondetection for malaria elimination programs IEEE 2014 Global HumanitarianTechnology Conference 2014 httpsdoiorg101109GHTC20146970338

Cocco et al BMC Medicine (2020) 18119 Page 10 of 12

stakeholders and input sources for the drafting phasewere not explicitly reported for we assessed the sourcesincluded in each TPP table and the longer descriptionsof each test characteristicEach TPP was also assessed in terms of the transpar-

ency of reporting the adopted input sources decision-making process and which stakeholder groups wereconsultedAll data extraction was conducted independently by

PC and AAS and in case of disagreement BS MM andRW resolved any differences

Test characteristic clusteringIn addition to the information above the test character-istics reported within each TPP were extracted and de-duplicated Based on an existing evaluation frameworkfor tests (the ACCE framework [14]) two reviewers (BSand MM) independently categorised each of the testcharacteristics under the following outcomes (1) testdefinition (2) analytical performance (3) clinical validity(4) clinical utility (5) regulatory legitimacy and (6) eco-nomic acceptability The category lsquotest definitionrsquo speci-fies the disorder of interest target population and

purpose of the test and thus it overlaps with the con-cept of lsquounmet clinical needrsquo Therefore we renamed theoutcome lsquotest definitionrsquo as lsquounmet clinical needrsquo to bet-ter represent the type of information TPPs provideAnalytical performance describes the ability of a test

to correctly detect and measure a particular analyte (egprecision trueness analytical sensitivity and specificitylimits of detection) [15 16] Clinical validity is defined aslsquothe ability of a device to yield results that are correlatedwith a particular clinical condition or a physiological orpathological process or statersquo [15] whilst clinical utilityrepresents the ability of a test to affect relevant health-related outcomes for patients (eg improvement in qual-ity of life longer lifespan) [17]Some characteristics did not fall within any of the pre-

defined categories Three additional categories weretherefore identified to accommodate these additionalcharacteristics (7) human factors (8) environmental im-pact and (9) infrastructural requirements Human factorsare concerned with the interaction between users anddevices [18] Environmental impact encompasses achange to the environment following an interaction withthe product [19] Infrastructural requirements entail lsquothe

Fig 1 PRISMA flow diagram illustrating literature search results

Cocco et al BMC Medicine (2020) 18119 Page 3 of 12

stock of the basic facilities and equipment needed forrealizing a product or providing a servicersquo [20]

ResultsLiterature searchFull details of the literature search results are reportedin Fig 1 Forty-four TPPs were deemed eligible forinclusion in the systematic review [8ndash10 21ndash61] Inter-reviewer agreement was high at title and abstract (κ =96) and full-text screening (κ = 98) For more detailsplease see Additional file 3 Table 31 and 32

Features of included TPPsThe included 44 TPPs consisted of 23 reports 16 journalarticles 4 published TPP tables (a TPP without anybackground information or context eg [34]) and oneconference poster All TPPs provided guidance on devel-oping medical tests to detect infectious diseases Four-teen of the 44 TPPs focused on neglected tropicaldiseases (32) (eg soil-transmitted helminths Chagasdisease human African trypanosomiasis schistosomiasistrachoma taeniasis cysticercisosis) and on tests for

vector-borne infections (32) (eg Zika virus denguefever hepatitis C malaria E coli) Other types ofinfection included sexually transmitted infections (16n = 7) respiratory infections (14 n = 6) (eg lower re-spiratory tract infection tuberculosis pneumonia) Ebolavirus [57] meningitis [61] and severe febrile illness [8]Seven of the 44 TPPs were funded by Bill and Melinda

Gates Foundation (16) and three TPPs received fund-ing from WHO [8 48 49] The healthcare setting ofinterest was mostly low- and middle-income countriesThe majority of TPPs did not disclose funding sources(64 n = 28)In some TPPs a time horizon was chosen to represent

the timeframe within which achieving the specificationsdescribed in the TPP was considered feasible [22 23 60]In one TPP this was based on a landscape analysis [22]In another expected advancements in technologies andknowledge related to a certain field seemed to justify thetime horizon considered for the TPP [27] Of the 44 TPPsidentified 7 reported the time horizon during which theinformation included in the TPP will be relevant for man-ufacturers (16) Of these 6 TPPs stated a time horizon

Fig 2 Typical activities involved input sources and stakeholders invited for each decision-making phase

Cocco et al BMC Medicine (2020) 18119 Page 4 of 12

of 5 years [22 23 27 28 51 60] whilst the remainingconsidered a time horizon of 10 years [29]

Decision-making stepsA common decision-making framework consisting ofthree distinct phases was apparent across the includedTPPs scoping drafting and consensus-building Figure 2presents the most commonly adopted activities inputsources and engaged stakeholder groupsTable 1 provides a summary of the stakeholders con-

tributing to each phase Some of the included TPPs arenot included in Table 1 as they did not report any infor-mation related to input sources or stakeholder groups[33ndash36 52 53 55 61] A summary of the input sourcesreported to have been used at scoping and draftingphase can be found in Additional file 3 Table 33We will therefore describe the aim of each phase and

breakdown the methodology (activities input sourcesand stakeholders) used within the included TPPs wherereported For specific details on each included TPP seeAdditional file 3 Table 34

Scoping phase methodologyHalf of the TPPs provided some information on thescoping phase (n = 22) The aim of this phase was toprovide an overview of the disease area and the limita-tions associated with existing technologies The clinicalproblems and unmet needs were defined in addition toidentification of which test characteristics to include inthe TPPSome of the key activities undertaken during the scop-

ing phase included reviewing published literature (n = 6)or available data (n = 1) and introductory meetings withstakeholders (n = 4)Some authors reported (n = 4) [22 26 37 50] that

they had conducted a lsquolandscape analysisrsquo providing in-formation on the disease area of interest available diag-nostic technologies and related characteristics andlimitations These were usually based on interviews withstakeholders and reviews of the literature Only Toskinet al [50] conducted a systematic literature reviewreporting the databases searched and key words usedConsultation with experts (68 n = 15) and the litera-

ture (36 n = 8) were the most commonly sought

Table 1 Stakeholders contributing to each phase

Stakeholder groups contributing to each phasea Scoping Drafting Consensus-building

n () n () n ()

Researchers 14 (32) 15 (34) 14 (32)

Industry representatives 11 (25) 11 (25) 14 (32)

International public organisations 9 (20) 12 (27) 12 (27)

Clinicians 7 (16) 8 (18) 8 (18)

Representatives of countries and national disease programs 5 (11) 4 (9) 8 (18)

Policy makers 4 (9) 5 (11) 4 (9)

Laboratory experts 3 (7) 5 (11) 3 (7)

Technicalfunding agencies 2 (5) 3 (7) 4 (9)

Microbiologists 2 (5) 4 (9) 2 (5)

Implementers 2 (5) 2 (5) 5 (11)

Patient advocates 2 (5) 1 (2) 3 (7)

Non-profit sector 1 (2) 2 (5) 4 (9)

Modellers 1 (2) 1 (2) 1 (2)

Health economists 0 (0) 2 (5) 2 (5)

Donors 2 (5) 3 (7) 1 (2)

Market experts 2(5) 1 (2) 1 (2)

Program manager 1 (2) 2 (5) 2 (5)

Scientific associations 1 (2) 3 (7) 2 (5)

Strategists 1 (2) 1 (2) 0 (0)

Expert (unspecific) 1 (2) 3 (7) 3 (7)

Others 2 (5) 2 (5) 1 (2)

The percentages in relation to stakeholder groups do not add up to 100 because more than one stakeholder group usually contributes to the development ofa TPPaPercentages are calculated in relation to the total number of included TPPs (n = 44)

Cocco et al BMC Medicine (2020) 18119 Page 5 of 12

sources of information during the scoping phase (seeAdditional file 3 Table 33 for a full breakdown) Onlyone type of source was considered in 15 TPPs (of which11 was consulting experts) whilst 7 TPPs consideredmore than one sourceDenkinger et al [22] mapped the diagnostic ecosystem

of interest and then performed a survey to gauge stake-holdersrsquo preferences Reipold et al [48] identified themain characteristic categories (eg scope performanceoperational characteristics and pricing) to be included inthe TPPFive TPPs involved a priority-setting exercise which entailed

ranking each identified health need [23 28 32 48 60]During the scoping phase a variety of stakeholders

were engaged (Table 1)

Drafting phase methodologyThe first draft of each TPP was usually prepared by ei-ther an established working group comprising expertsfrom different organisations [9 26 29 32 40 50 58] orauthors of the published TPP There were two caseswhere the TPP was drafted by a completely different or-ganisation [51 57] The TPP was often revised severaltimes and in some cases it was then shortened to en-sure it could be easily communicated to different stake-holders [22 23 28 29 60]Of the 44 included TPPs 33 of them reported which

input sources were considered during the drafting phase(75) (Additional file 3 Table 33) Common inputsources for populating test characteristics were expertconsultations (n = 22) and reviews of the literature (n =22) Some also referred to mathematical models (n = 9)available data (n = 7) guidelines (n = 6) and lsquofield obser-vationsrsquo (n = 5) Only one TPP was informed by pooleddata from a systematic review [50]Twenty-six of the 44 TPPs took into consideration more

than one type of source at the drafting phase as opposedto 7 TPPs which only adopted one (Additional file 3Table 33) Meeting inputs were the most common singlesource (43 n = 3)The stakeholders engaged in the drafting phase are re-

ported in Table 1

Consensus-building phase methodologyInitial agreement with the TPP was often obtained usinga survey of the stakeholders (n = 14) The survey eitherincluded general questions regarding stakeholders viewson the TPP (n = 4) [22 25 27 51] or adopted a Delphi-like approach to provide an initial consensus on variousaspects of the TPP (n = 10) A consensus meeting withstakeholders and experts was typically held (n = 11) anda revised TPP generally agreed upon In some cases anadditional survey was sent to stakeholders on trade-offsbetween test attributes [48] or on rating key parameters

[51 53] For 2 TPPs the final TPP draft was presentedto a broader stakeholder base to validate itThe number of participants invited to the consensus-

building meetings varied (lt 20 participants n = 5 between20 and 50 participants n = 7) One meeting included 100participants [27] For a few of the TPPs the authors alsotook part in the consensus meetings [29 38 58 60]Less than half of the included TPPs reported information

on the activities and stakeholders invited to the consensus-building phase (n = 19) The stakeholders engaged in theconsensus-building phase are reported in Table 1

Transparency in reporting methodsWe also assessed the transparency of the TPPs in termsof reporting their methodology (see Additional file 3Table 35) The decision-making process behind the TPPwas not reported in over a quarter of the included TPPs(n = 16) Further to this many failed to report which in-formation sources were considered to populate the TPP(n = 11) Just under half did not report which stake-holders were involved in the development of the TPP(n = 20) Specifically the name of the organisationsstakeholders were part was only reported in 11 TPPswhilst 9 TPPs mentioned personal details of each stake-holder (20) and 4 TPPs explained why certain stake-holders were invited [26 38 46 58] Sixteen TPPsreported the source of funding (36)There were some TPPs where the methodology was

very clearly reported [26 28 38 60]

Test characteristics included in TPPsAfter removing duplicates 140 different test characteris-tics were reported across the included TPPs Some fea-tures which did not represent test characteristics havebeen excluded such as factors relating specifically to thedisease in question rather than the test For more infor-mation please see Test Characteristics Overview Excelspreadsheet (Availability of data and materials) Figure 3shows the test characteristics most frequently reported(a full list is available in Additional file 2 Table 21)Figure 4 depicts which characteristic categories were

reported in the included TPPs Details on unmet clinicalneed analytical performance and clinical validity ap-peared to be consistently reported however regula-tory requirements environmental footprint and clinicalutility were less frequently considered

DiscussionWe report a systematic review of the methods currentlyused to develop Target Product Profiles for medicaltests Despite TPPs for any medical test being searchedall of the identified TPPs were focused on diagnostictests for infection

Cocco et al BMC Medicine (2020) 18119 Page 6 of 12

Fig 3 Test characteristics frequently reported in all TPPs (n = 44) sorted by categories

Fig 4 Test characteristic categories in absolute number (n) of included TPPs

Cocco et al BMC Medicine (2020) 18119 Page 7 of 12

There was generally a lack of transparency andconsistency in reporting the methods underlying TPPsThis would make it difficult to appraise the recommen-dations within the TPPs ascertain whether the recom-mendations are generalizable to other settings andchallenging to reproduce

Relevancy of TPPs for test manufacturersThe purpose of a TPP is to identify upfront the essen-tial characteristics of a test for it to fulfil a pre-specified unmet clinical need This should in turn in-crease the likelihood that the test will be adopted intoclinical practice and reimbursed [1] A TPP should alsoaccount for contextual aspects that might affect thetestrsquos real-world performance [10] defining infrastruc-tural and technical constraints that impact on the im-plementation into clinical practice This review showsthat TPPs to date have primarily been developed forglobal health applications as the main funding organi-sations are WHO UNICEF and Bill and Melinda GatesFoundation The primary focus on infectious diseasesmay be explained by the remit of the global organisa-tions who fund TPPs WHO included HIVAIDSneglected tropical diseases tuberculosis and malaria aspriority diseases [62] Further to this WHO establishedthe lsquoRampD Blueprintrsquo which aims to promote RampD activ-ities (tests vaccines medicines) during epidemics [63]After having identified pathogen to target first TPPsare usually commissioned to guide the developmentprocess of new healthcare products which will addressthe high-priority pathogen [63]However the development of TPPs should not be limited

to one specific disease area or clinical setting the concept oflsquobeginning with an end in mindrsquo embodied by TPPs couldsupport both international and national health decision-makers This activity should in turn stimulate innovation ofnew tests driven by clinical needs rather than solely by la-boratory discoveries It would also provide manufacturerswith greater clarity around test requirements and confidencein the market for developing innovative tests

Identified limitations in current TPP methodologyIn reviewing current methodology for developing TPPsfor medical tests we have identified three key areaswhere current TPP methodology could be improved (1)oversight of clinical utility (2) a focus on price ratherthan cost-effectiveness and (3) subjectivity of informa-tion sources Here we discuss each limitation and theimplications

Oversight of clinical utilityVery few of the TPPs reported desirable characteristicsrelating to the clinical utility of the test This is not sur-prising given that the majority of research efforts has

focused on generating evidence on the analytical per-formance and diagnostic accuracy of a new test [11] Ahighly accurate test does not necessarily mean that thetest will improve patient health as factors relating todecision-making and the effectiveness of patient man-agement strategies could fall short [64]Assessing the clinical utility of a new test is ex-

tremely challenging Measuring the impact of a teston patient health outcomes is difficult as tests tend toguide patient management decisions rather than dir-ectly impacting on patient health outcomes [5]Therefore estimating the clinical utility of a test re-quires evidence of how the information from a test isincorporated into decision-making and the down-stream effectiveness of those decisions [65] In thecase of a new test this is particularly complicatedgiven the uncertainty around the mechanisms bywhich the test will impact on patient outcomes [65]

Focus on price rather than cost-effectivenessAlthough the minimum and optimal price of the testsfeatured in many of the TPPs none of these was drivenby the trade-off between the overall cost implications ofimplementing the test and the associated patient bene-fits Cost-effectiveness analysis provides a framework tocompare costs and benefits of an intervention againstrelevant comparators including current practice Spe-cifically cost-effectiveness analysis defines whether theintervention being evaluated represents good value formoneyIt is important to consider the cost of the new test in

the context of the benefits that the test may provide Forexample a new test may be relatively expensive but mayalso improve patient health to the extent that the add-itional is justified Conversely a new test may be rela-tively cheap but offer no improvements in patient healthand therefore even the marginal increase in cost is notjustifiedConducting cost-effectiveness analysis at early RampD

stages of new tests can therefore help manufacturers toavoid significant investments in tests that do not havethe potential to be cost-effective [65]These first two limitations are particularly relevant

since decision-makers increasingly demand evidencethat a new test improves patient health and is cost-effective rather than solely evidence of its analyticaland clinical validity [5] Specifically many HealthTechnology Assessment bodies in Europe Australiaand North America consider clinical utility cost andcost-effectiveness in relation to the target populationin addition to analytical performance and clinicalvalidity when assessing new molecular diagnostictests [66]

Cocco et al BMC Medicine (2020) 18119 Page 8 of 12

Subjectivity of input sourcesExpert judgement and evidence identified in publishedliterature were the main sources of information for de-fining desirable characteristics Systematic reviews of theliterature where database searches are reproducible andthe quality of relevant studies are appraised were notconducted to identify relevant evidence at the scopingand drafting phase This is likely to introduce bias andsubjectivity in terms of the evidence used to underpintest characteristic recommendationsAlthough expert judgement is undoubtedly useful

relying solely upon this information source has somelimitations particularly for quantitative estimates Howhumans make probability judgements is highly affectedby many heuristics and systematic biases (eg anchoringavailability overconfidence and insight bias) [67] Specif-ically previous literature has found a poor understand-ing of test accuracy among healthcare professionals [68]as sensitivity and specificity are often misinterpreted andmistaken for predictive values [68]Additionally the quality of expert elicitation heavily

relies on expert selection as it is important to choose ex-perts with good subject knowledge Only 4 TPPs de-scribed how the selection process took place andtherefore the quality of expert judgements might bequestioned Furthermore many TPPs reported literatureas a source for informing TPPs however less than halfof the TPPs cited the references considered This lack oftransparency might hinder the quality and credibility ofsources on which TPPs are based

Study limitationsSince this study is a systematic review of publicly avail-able literature a key study limitation is that we have in-evitably missed any confidential or unpublished TPPsdeveloped in-house by test manufacturers Although theresults of our online searches did not identify any com-panies stating that they have developed TPPs for medicaltests we would not expect to find such information oncompany websites Anecdotally however we have notencountered any formal TPP development activity (bythis we mean definition of desirable test characteristics)within the National Institute for Health Research LeedsIn Vitro Diagnostics Co-operative (NIHR Leeds) MICindustry networkAs there are no guidelines on how TPPs for medical

tests should be developed we did not formally assessrisk of bias We did however appraise the transparencywith which the methodology underpinning each TPPhad been reported Unfortunately it was not possible tofully evaluate the TPP developed by PATH [40] as theonline appendices were not accessible Additionally dueto poor methodological transparency it was difficult toassess with certainty authorship of TPPs and whether

authors of TPPs took part themselves in the consensus-building meetings

Future researchAlthough there is evidence of a common developmentframework this review highlights that there is consider-able variability in the methods employed to draft TPPsand inconsistencies in which test characteristics are de-scribed A key issue in reviewing the methods imple-mented was the lack of transparency in methodologyreportingGuidance on best practice methods for developing

TPPs for medical tests would be highly beneficial Simi-larly to the US FDA guidance on TPPs for drugs a guid-ance document could be developed for TPPs for medicaltests summarising the purpose attributes of TPPs andwhich test characteristics should be includedHowever to inform the development of such guidance

future research should focus firstly on how to systemat-ically identify unmet clinical needs underpinning a cer-tain disease area Monaghan et al [69] developed avaluable checklist for identifying biomarkers based onliterature findings and consultations with experts Webelieve that this checklist could be pertinent for thescoping phase underlying TPP development howeverthis would need further validation in this specificcontextMore research is also required to understand how to

better incorporate the assessment of desirable clinicalutility and cost-effectiveness of innovative tests intoTPPs One possible way forward could be exploring howand if care pathway analysis and early economic model-ling could be integrated into the development of TPPsCare pathway analysis would provide clarity on themechanisms by which a test could impact on down-stream patient outcomes Early economic modellingcould be used to define desirable values for certain testcharacteristics (eg test price diagnostic sensitivity andspecificity) based on cost-effectiveness [67] Thereforeintegrating care pathway analysis and early economicmodelling into TPP development might provide moreevidence-based information to the test developersOutside of the actual methodology for developing

TPPs it would be useful to better understand whethermanufacturers develop tests strictly in line with TPPs orwhether there are any factors which make this infeasibleor challenging Additionally we would be interested inwhich methods manufacturers usually adopt to developTPPs to assess if there are any differences with themethodological framework we highlighted here To thisend interviewing test manufacturers might provide in-teresting insights on the intrinsic value of TPPs for theindustry

Cocco et al BMC Medicine (2020) 18119 Page 9 of 12

Most importantly to ensure that the development ofTPPs becomes widespread practice it would be valuableto explore how TPPs could be integrated into existingregulatory paths for innovation such as the EuropeanUnion Regulation for In Vitro Diagnostics (Regulation2017746) and the US FDA Drug Development and Ap-proval Process It might then be possible to align testcharacteristics featured in TPPs with evidence require-ments which are relevant for market approval decisionsof new medical tests This in return might increase theapplicability of TPPs for the industry

ConclusionsBased on this review we summarised current methodo-logical practice into a framework of value to those inter-ested in developing TPPs for medical testsWe also identified some key weaknesses including the

quality of the information sources underpinning TPPsand failure to consider test characteristics relating toclinical utility and cost-effectivenessThis review thus provides some recommendations for

further methodological research on the development ofTPPs for medical test This work will also help to informthe development of a formal guideline on how to draftTPPs for medical tests

Supplementary informationSupplementary information accompanies this paper at httpsdoiorg101186s12916-020-01582-1

Additional file 1 Search strategies

Additional file 2 Test characteristics

Additional file 3 Results

AbbreviationsACCE Analytic validity Clinical validity Clinical Utility Ethical legal and socialimplications AIDS Acquired immune deficiency syndrome CEA Cost-effectiveness analysis FDA Food and Drug Administration FIND TheFoundation For Innovative New Diagnostics HIV Human immunodeficiencyvirus IDC International Diagnostics Center NIHR National Institute for HealthResearch PRISMA Preferred Reporting Items for Systematic Reviews andMeta-Analysis QTPP Quality Target Product Profile RampD Research andDevelopment TPP Target Product Profile UNICEF United NationsInternational Childrenrsquos Emergency Fund WHO World Health Organization

AcknowledgementsThe authors would like to express their gratitude to Natalie King (Universityof Leeds Information Specialist) for supporting the design of the searchstrategy for this systematic review

Authorsrsquo contributionsAll authors contributed to the conception and design of the work PCperformed the search strategy BS contributed to reference sifting whilst AASconducted the data extraction MM and RW solved any conflicts PC BS MMand RW interpreted the data and drafted and revised the manuscript Theauthors read and approved the final manuscript

FundingThis systematic review was carried out as a part of the full-time School ofMedicine PhD Scholarship awarded to Paola Cocco by the University ofLeeds Additionally this work is supported by the lsquoTackling AMR-A Cross

Council Initiativersquo (grant number MRN0299761) Funding Partners The Bio-technology and Biological Sciences Research Council the Engineering andPhysical Sciences Research Council and the Medical Research Council Thiswork is also supported by the Medical Research Foundations National AMRTraining Programme Bethany Shinkins and Michael Messenger are fundedby NIHR Leeds In Vitro Diagnostic Co-operative and Cancer Research UK viaCanTest Collaborative Anam Ayaz-Shah is funded by Cancer Research UK viaCanTest Collaborative

Availability of data and materialsThe datasets generated and analysed during the current study are availablein the University of Leeds repository httpsdoiorg105518781 Thesedatasets entail an Excel spreadsheet with data extraction and an overview oftest characteristics included in reviewed TPPs

Ethics approval and consent to participateNot applicable

Consent for publicationNot applicable

Competing interestsThe authors declare that they have no competing interests

Author details1Test Evaluation Group Academic Unit of Health Economics Leeds Institutefor Health Sciences University of Leeds Leeds UK 2Academic Unit ofPrimary Care Leeds Institute for Health Sciences University of Leeds LeedsUK 3httpswwwcantestorg 4Centre for Personalised Health and MedicineUniversity of Leeds Leeds UK 5NIHR Leeds In Vitro Diagnostic (IVD)Co-operative Leeds UK 6Leeds Institute for Health Sciences University ofLeeds Leeds UK

Received 4 December 2019 Accepted 1 April 2020

References1 Wang P Kricka L Current and emerging trends in point-of-care technology

and strategies for clinical validation and implementation Clin Chem 2018httpsdoiorg101373clinchem2018287052

2 Drucker E Krapfenbauer K Pitfalls and limitations in translation frombiomarker discovery to clinical utility in predictive and personalisedmedicine EPMA Journal 20134(1)4ndash7

3 Kern S Why your new cancer biomarker may never work recurrent patternsand remarkable diversity in biomarker failures Cancer Res 2012 httpsdoiorg1011580008-5472CAN-12-3232

4 Engel N Wachter K Pai M Gallarda J Boehme C Celentano I et alAddressing the challenges of diagnostics demand and supply insights froman online global health discussion platform BMJ Glob Health 2016 httpsdoiorg101136bmjgh-2016-000132

5 Horvath A Lord S St John A Sandberg S Cobbaert C Lorenz S et al Frombiomarkers to medical tests the changing landscape of test evaluation ClinChim Acta 2014 httpsdoiorg101016jcca201309018

6 US Food and Drug Administration Guidance for industry Q8 (2)pharmaceutical development 2009 httpacademygmp-complianceorgguidemgrfiles9041FNL5b15dPDF Accessed 6 Nov 2018

7 US Food and Drug Administration Guidance for industry and review staffTarget Product Profile mdash a strategic development process tool 2007httpswwwfdagovmedia72566download Accessed 6 Nov 2018

8 WHO FIND MsF A multiplex multi-analyte diagnostic platform 2017httpswwwwhointmedicinesTPP_intro_20171122_forDistributionpdfAccessed 8 Nov 2018

9 PATH Target Product Profile HIV self-test version 41 a white paper on theevaluation of current HIV rapid tests and development of core specificationsfor next-generation HIV tests 2014 httpspathazureedgenetmediadocumentsTS_hiv_self_test_tpppdf Accessed 8 Nov 2018

10 Ebels K Clerk C Crudder C McGray S Magnuson K Tietje K et alIncorporating user needs into product development for improved infectiondetection for malaria elimination programs IEEE 2014 Global HumanitarianTechnology Conference 2014 httpsdoiorg101109GHTC20146970338

Cocco et al BMC Medicine (2020) 18119 Page 10 of 12

stock of the basic facilities and equipment needed forrealizing a product or providing a servicersquo [20]

ResultsLiterature searchFull details of the literature search results are reportedin Fig 1 Forty-four TPPs were deemed eligible forinclusion in the systematic review [8ndash10 21ndash61] Inter-reviewer agreement was high at title and abstract (κ =96) and full-text screening (κ = 98) For more detailsplease see Additional file 3 Table 31 and 32

Features of included TPPsThe included 44 TPPs consisted of 23 reports 16 journalarticles 4 published TPP tables (a TPP without anybackground information or context eg [34]) and oneconference poster All TPPs provided guidance on devel-oping medical tests to detect infectious diseases Four-teen of the 44 TPPs focused on neglected tropicaldiseases (32) (eg soil-transmitted helminths Chagasdisease human African trypanosomiasis schistosomiasistrachoma taeniasis cysticercisosis) and on tests for

vector-borne infections (32) (eg Zika virus denguefever hepatitis C malaria E coli) Other types ofinfection included sexually transmitted infections (16n = 7) respiratory infections (14 n = 6) (eg lower re-spiratory tract infection tuberculosis pneumonia) Ebolavirus [57] meningitis [61] and severe febrile illness [8]Seven of the 44 TPPs were funded by Bill and Melinda

Gates Foundation (16) and three TPPs received fund-ing from WHO [8 48 49] The healthcare setting ofinterest was mostly low- and middle-income countriesThe majority of TPPs did not disclose funding sources(64 n = 28)In some TPPs a time horizon was chosen to represent

the timeframe within which achieving the specificationsdescribed in the TPP was considered feasible [22 23 60]In one TPP this was based on a landscape analysis [22]In another expected advancements in technologies andknowledge related to a certain field seemed to justify thetime horizon considered for the TPP [27] Of the 44 TPPsidentified 7 reported the time horizon during which theinformation included in the TPP will be relevant for man-ufacturers (16) Of these 6 TPPs stated a time horizon

Fig 2 Typical activities involved input sources and stakeholders invited for each decision-making phase

Cocco et al BMC Medicine (2020) 18119 Page 4 of 12

of 5 years [22 23 27 28 51 60] whilst the remainingconsidered a time horizon of 10 years [29]

Decision-making stepsA common decision-making framework consisting ofthree distinct phases was apparent across the includedTPPs scoping drafting and consensus-building Figure 2presents the most commonly adopted activities inputsources and engaged stakeholder groupsTable 1 provides a summary of the stakeholders con-

tributing to each phase Some of the included TPPs arenot included in Table 1 as they did not report any infor-mation related to input sources or stakeholder groups[33ndash36 52 53 55 61] A summary of the input sourcesreported to have been used at scoping and draftingphase can be found in Additional file 3 Table 33We will therefore describe the aim of each phase and

breakdown the methodology (activities input sourcesand stakeholders) used within the included TPPs wherereported For specific details on each included TPP seeAdditional file 3 Table 34

Scoping phase methodologyHalf of the TPPs provided some information on thescoping phase (n = 22) The aim of this phase was toprovide an overview of the disease area and the limita-tions associated with existing technologies The clinicalproblems and unmet needs were defined in addition toidentification of which test characteristics to include inthe TPPSome of the key activities undertaken during the scop-

ing phase included reviewing published literature (n = 6)or available data (n = 1) and introductory meetings withstakeholders (n = 4)Some authors reported (n = 4) [22 26 37 50] that

they had conducted a lsquolandscape analysisrsquo providing in-formation on the disease area of interest available diag-nostic technologies and related characteristics andlimitations These were usually based on interviews withstakeholders and reviews of the literature Only Toskinet al [50] conducted a systematic literature reviewreporting the databases searched and key words usedConsultation with experts (68 n = 15) and the litera-

ture (36 n = 8) were the most commonly sought

Table 1 Stakeholders contributing to each phase

Stakeholder groups contributing to each phasea Scoping Drafting Consensus-building

n () n () n ()

Researchers 14 (32) 15 (34) 14 (32)

Industry representatives 11 (25) 11 (25) 14 (32)

International public organisations 9 (20) 12 (27) 12 (27)

Clinicians 7 (16) 8 (18) 8 (18)

Representatives of countries and national disease programs 5 (11) 4 (9) 8 (18)

Policy makers 4 (9) 5 (11) 4 (9)

Laboratory experts 3 (7) 5 (11) 3 (7)

Technicalfunding agencies 2 (5) 3 (7) 4 (9)

Microbiologists 2 (5) 4 (9) 2 (5)

Implementers 2 (5) 2 (5) 5 (11)

Patient advocates 2 (5) 1 (2) 3 (7)

Non-profit sector 1 (2) 2 (5) 4 (9)

Modellers 1 (2) 1 (2) 1 (2)

Health economists 0 (0) 2 (5) 2 (5)

Donors 2 (5) 3 (7) 1 (2)

Market experts 2(5) 1 (2) 1 (2)

Program manager 1 (2) 2 (5) 2 (5)

Scientific associations 1 (2) 3 (7) 2 (5)

Strategists 1 (2) 1 (2) 0 (0)

Expert (unspecific) 1 (2) 3 (7) 3 (7)

Others 2 (5) 2 (5) 1 (2)

The percentages in relation to stakeholder groups do not add up to 100 because more than one stakeholder group usually contributes to the development ofa TPPaPercentages are calculated in relation to the total number of included TPPs (n = 44)

Cocco et al BMC Medicine (2020) 18119 Page 5 of 12

sources of information during the scoping phase (seeAdditional file 3 Table 33 for a full breakdown) Onlyone type of source was considered in 15 TPPs (of which11 was consulting experts) whilst 7 TPPs consideredmore than one sourceDenkinger et al [22] mapped the diagnostic ecosystem

of interest and then performed a survey to gauge stake-holdersrsquo preferences Reipold et al [48] identified themain characteristic categories (eg scope performanceoperational characteristics and pricing) to be included inthe TPPFive TPPs involved a priority-setting exercise which entailed

ranking each identified health need [23 28 32 48 60]During the scoping phase a variety of stakeholders

were engaged (Table 1)

Drafting phase methodologyThe first draft of each TPP was usually prepared by ei-ther an established working group comprising expertsfrom different organisations [9 26 29 32 40 50 58] orauthors of the published TPP There were two caseswhere the TPP was drafted by a completely different or-ganisation [51 57] The TPP was often revised severaltimes and in some cases it was then shortened to en-sure it could be easily communicated to different stake-holders [22 23 28 29 60]Of the 44 included TPPs 33 of them reported which

input sources were considered during the drafting phase(75) (Additional file 3 Table 33) Common inputsources for populating test characteristics were expertconsultations (n = 22) and reviews of the literature (n =22) Some also referred to mathematical models (n = 9)available data (n = 7) guidelines (n = 6) and lsquofield obser-vationsrsquo (n = 5) Only one TPP was informed by pooleddata from a systematic review [50]Twenty-six of the 44 TPPs took into consideration more

than one type of source at the drafting phase as opposedto 7 TPPs which only adopted one (Additional file 3Table 33) Meeting inputs were the most common singlesource (43 n = 3)The stakeholders engaged in the drafting phase are re-

ported in Table 1

Consensus-building phase methodologyInitial agreement with the TPP was often obtained usinga survey of the stakeholders (n = 14) The survey eitherincluded general questions regarding stakeholders viewson the TPP (n = 4) [22 25 27 51] or adopted a Delphi-like approach to provide an initial consensus on variousaspects of the TPP (n = 10) A consensus meeting withstakeholders and experts was typically held (n = 11) anda revised TPP generally agreed upon In some cases anadditional survey was sent to stakeholders on trade-offsbetween test attributes [48] or on rating key parameters

[51 53] For 2 TPPs the final TPP draft was presentedto a broader stakeholder base to validate itThe number of participants invited to the consensus-

building meetings varied (lt 20 participants n = 5 between20 and 50 participants n = 7) One meeting included 100participants [27] For a few of the TPPs the authors alsotook part in the consensus meetings [29 38 58 60]Less than half of the included TPPs reported information

on the activities and stakeholders invited to the consensus-building phase (n = 19) The stakeholders engaged in theconsensus-building phase are reported in Table 1

Transparency in reporting methodsWe also assessed the transparency of the TPPs in termsof reporting their methodology (see Additional file 3Table 35) The decision-making process behind the TPPwas not reported in over a quarter of the included TPPs(n = 16) Further to this many failed to report which in-formation sources were considered to populate the TPP(n = 11) Just under half did not report which stake-holders were involved in the development of the TPP(n = 20) Specifically the name of the organisationsstakeholders were part was only reported in 11 TPPswhilst 9 TPPs mentioned personal details of each stake-holder (20) and 4 TPPs explained why certain stake-holders were invited [26 38 46 58] Sixteen TPPsreported the source of funding (36)There were some TPPs where the methodology was

very clearly reported [26 28 38 60]

Test characteristics included in TPPsAfter removing duplicates 140 different test characteris-tics were reported across the included TPPs Some fea-tures which did not represent test characteristics havebeen excluded such as factors relating specifically to thedisease in question rather than the test For more infor-mation please see Test Characteristics Overview Excelspreadsheet (Availability of data and materials) Figure 3shows the test characteristics most frequently reported(a full list is available in Additional file 2 Table 21)Figure 4 depicts which characteristic categories were

reported in the included TPPs Details on unmet clinicalneed analytical performance and clinical validity ap-peared to be consistently reported however regula-tory requirements environmental footprint and clinicalutility were less frequently considered

DiscussionWe report a systematic review of the methods currentlyused to develop Target Product Profiles for medicaltests Despite TPPs for any medical test being searchedall of the identified TPPs were focused on diagnostictests for infection

Cocco et al BMC Medicine (2020) 18119 Page 6 of 12

Fig 3 Test characteristics frequently reported in all TPPs (n = 44) sorted by categories

Fig 4 Test characteristic categories in absolute number (n) of included TPPs

Cocco et al BMC Medicine (2020) 18119 Page 7 of 12

There was generally a lack of transparency andconsistency in reporting the methods underlying TPPsThis would make it difficult to appraise the recommen-dations within the TPPs ascertain whether the recom-mendations are generalizable to other settings andchallenging to reproduce

Relevancy of TPPs for test manufacturersThe purpose of a TPP is to identify upfront the essen-tial characteristics of a test for it to fulfil a pre-specified unmet clinical need This should in turn in-crease the likelihood that the test will be adopted intoclinical practice and reimbursed [1] A TPP should alsoaccount for contextual aspects that might affect thetestrsquos real-world performance [10] defining infrastruc-tural and technical constraints that impact on the im-plementation into clinical practice This review showsthat TPPs to date have primarily been developed forglobal health applications as the main funding organi-sations are WHO UNICEF and Bill and Melinda GatesFoundation The primary focus on infectious diseasesmay be explained by the remit of the global organisa-tions who fund TPPs WHO included HIVAIDSneglected tropical diseases tuberculosis and malaria aspriority diseases [62] Further to this WHO establishedthe lsquoRampD Blueprintrsquo which aims to promote RampD activ-ities (tests vaccines medicines) during epidemics [63]After having identified pathogen to target first TPPsare usually commissioned to guide the developmentprocess of new healthcare products which will addressthe high-priority pathogen [63]However the development of TPPs should not be limited

to one specific disease area or clinical setting the concept oflsquobeginning with an end in mindrsquo embodied by TPPs couldsupport both international and national health decision-makers This activity should in turn stimulate innovation ofnew tests driven by clinical needs rather than solely by la-boratory discoveries It would also provide manufacturerswith greater clarity around test requirements and confidencein the market for developing innovative tests

Identified limitations in current TPP methodologyIn reviewing current methodology for developing TPPsfor medical tests we have identified three key areaswhere current TPP methodology could be improved (1)oversight of clinical utility (2) a focus on price ratherthan cost-effectiveness and (3) subjectivity of informa-tion sources Here we discuss each limitation and theimplications

Oversight of clinical utilityVery few of the TPPs reported desirable characteristicsrelating to the clinical utility of the test This is not sur-prising given that the majority of research efforts has

focused on generating evidence on the analytical per-formance and diagnostic accuracy of a new test [11] Ahighly accurate test does not necessarily mean that thetest will improve patient health as factors relating todecision-making and the effectiveness of patient man-agement strategies could fall short [64]Assessing the clinical utility of a new test is ex-

tremely challenging Measuring the impact of a teston patient health outcomes is difficult as tests tend toguide patient management decisions rather than dir-ectly impacting on patient health outcomes [5]Therefore estimating the clinical utility of a test re-quires evidence of how the information from a test isincorporated into decision-making and the down-stream effectiveness of those decisions [65] In thecase of a new test this is particularly complicatedgiven the uncertainty around the mechanisms bywhich the test will impact on patient outcomes [65]

Focus on price rather than cost-effectivenessAlthough the minimum and optimal price of the testsfeatured in many of the TPPs none of these was drivenby the trade-off between the overall cost implications ofimplementing the test and the associated patient bene-fits Cost-effectiveness analysis provides a framework tocompare costs and benefits of an intervention againstrelevant comparators including current practice Spe-cifically cost-effectiveness analysis defines whether theintervention being evaluated represents good value formoneyIt is important to consider the cost of the new test in

the context of the benefits that the test may provide Forexample a new test may be relatively expensive but mayalso improve patient health to the extent that the add-itional is justified Conversely a new test may be rela-tively cheap but offer no improvements in patient healthand therefore even the marginal increase in cost is notjustifiedConducting cost-effectiveness analysis at early RampD

stages of new tests can therefore help manufacturers toavoid significant investments in tests that do not havethe potential to be cost-effective [65]These first two limitations are particularly relevant

since decision-makers increasingly demand evidencethat a new test improves patient health and is cost-effective rather than solely evidence of its analyticaland clinical validity [5] Specifically many HealthTechnology Assessment bodies in Europe Australiaand North America consider clinical utility cost andcost-effectiveness in relation to the target populationin addition to analytical performance and clinicalvalidity when assessing new molecular diagnostictests [66]

Cocco et al BMC Medicine (2020) 18119 Page 8 of 12

Subjectivity of input sourcesExpert judgement and evidence identified in publishedliterature were the main sources of information for de-fining desirable characteristics Systematic reviews of theliterature where database searches are reproducible andthe quality of relevant studies are appraised were notconducted to identify relevant evidence at the scopingand drafting phase This is likely to introduce bias andsubjectivity in terms of the evidence used to underpintest characteristic recommendationsAlthough expert judgement is undoubtedly useful

relying solely upon this information source has somelimitations particularly for quantitative estimates Howhumans make probability judgements is highly affectedby many heuristics and systematic biases (eg anchoringavailability overconfidence and insight bias) [67] Specif-ically previous literature has found a poor understand-ing of test accuracy among healthcare professionals [68]as sensitivity and specificity are often misinterpreted andmistaken for predictive values [68]Additionally the quality of expert elicitation heavily

relies on expert selection as it is important to choose ex-perts with good subject knowledge Only 4 TPPs de-scribed how the selection process took place andtherefore the quality of expert judgements might bequestioned Furthermore many TPPs reported literatureas a source for informing TPPs however less than halfof the TPPs cited the references considered This lack oftransparency might hinder the quality and credibility ofsources on which TPPs are based

Study limitationsSince this study is a systematic review of publicly avail-able literature a key study limitation is that we have in-evitably missed any confidential or unpublished TPPsdeveloped in-house by test manufacturers Although theresults of our online searches did not identify any com-panies stating that they have developed TPPs for medicaltests we would not expect to find such information oncompany websites Anecdotally however we have notencountered any formal TPP development activity (bythis we mean definition of desirable test characteristics)within the National Institute for Health Research LeedsIn Vitro Diagnostics Co-operative (NIHR Leeds) MICindustry networkAs there are no guidelines on how TPPs for medical

tests should be developed we did not formally assessrisk of bias We did however appraise the transparencywith which the methodology underpinning each TPPhad been reported Unfortunately it was not possible tofully evaluate the TPP developed by PATH [40] as theonline appendices were not accessible Additionally dueto poor methodological transparency it was difficult toassess with certainty authorship of TPPs and whether

authors of TPPs took part themselves in the consensus-building meetings

Future researchAlthough there is evidence of a common developmentframework this review highlights that there is consider-able variability in the methods employed to draft TPPsand inconsistencies in which test characteristics are de-scribed A key issue in reviewing the methods imple-mented was the lack of transparency in methodologyreportingGuidance on best practice methods for developing

TPPs for medical tests would be highly beneficial Simi-larly to the US FDA guidance on TPPs for drugs a guid-ance document could be developed for TPPs for medicaltests summarising the purpose attributes of TPPs andwhich test characteristics should be includedHowever to inform the development of such guidance

future research should focus firstly on how to systemat-ically identify unmet clinical needs underpinning a cer-tain disease area Monaghan et al [69] developed avaluable checklist for identifying biomarkers based onliterature findings and consultations with experts Webelieve that this checklist could be pertinent for thescoping phase underlying TPP development howeverthis would need further validation in this specificcontextMore research is also required to understand how to

better incorporate the assessment of desirable clinicalutility and cost-effectiveness of innovative tests intoTPPs One possible way forward could be exploring howand if care pathway analysis and early economic model-ling could be integrated into the development of TPPsCare pathway analysis would provide clarity on themechanisms by which a test could impact on down-stream patient outcomes Early economic modellingcould be used to define desirable values for certain testcharacteristics (eg test price diagnostic sensitivity andspecificity) based on cost-effectiveness [67] Thereforeintegrating care pathway analysis and early economicmodelling into TPP development might provide moreevidence-based information to the test developersOutside of the actual methodology for developing

TPPs it would be useful to better understand whethermanufacturers develop tests strictly in line with TPPs orwhether there are any factors which make this infeasibleor challenging Additionally we would be interested inwhich methods manufacturers usually adopt to developTPPs to assess if there are any differences with themethodological framework we highlighted here To thisend interviewing test manufacturers might provide in-teresting insights on the intrinsic value of TPPs for theindustry

Cocco et al BMC Medicine (2020) 18119 Page 9 of 12

Most importantly to ensure that the development ofTPPs becomes widespread practice it would be valuableto explore how TPPs could be integrated into existingregulatory paths for innovation such as the EuropeanUnion Regulation for In Vitro Diagnostics (Regulation2017746) and the US FDA Drug Development and Ap-proval Process It might then be possible to align testcharacteristics featured in TPPs with evidence require-ments which are relevant for market approval decisionsof new medical tests This in return might increase theapplicability of TPPs for the industry

ConclusionsBased on this review we summarised current methodo-logical practice into a framework of value to those inter-ested in developing TPPs for medical testsWe also identified some key weaknesses including the

quality of the information sources underpinning TPPsand failure to consider test characteristics relating toclinical utility and cost-effectivenessThis review thus provides some recommendations for

further methodological research on the development ofTPPs for medical test This work will also help to informthe development of a formal guideline on how to draftTPPs for medical tests

Supplementary informationSupplementary information accompanies this paper at httpsdoiorg101186s12916-020-01582-1

Additional file 1 Search strategies

Additional file 2 Test characteristics

Additional file 3 Results

AbbreviationsACCE Analytic validity Clinical validity Clinical Utility Ethical legal and socialimplications AIDS Acquired immune deficiency syndrome CEA Cost-effectiveness analysis FDA Food and Drug Administration FIND TheFoundation For Innovative New Diagnostics HIV Human immunodeficiencyvirus IDC International Diagnostics Center NIHR National Institute for HealthResearch PRISMA Preferred Reporting Items for Systematic Reviews andMeta-Analysis QTPP Quality Target Product Profile RampD Research andDevelopment TPP Target Product Profile UNICEF United NationsInternational Childrenrsquos Emergency Fund WHO World Health Organization

AcknowledgementsThe authors would like to express their gratitude to Natalie King (Universityof Leeds Information Specialist) for supporting the design of the searchstrategy for this systematic review

Authorsrsquo contributionsAll authors contributed to the conception and design of the work PCperformed the search strategy BS contributed to reference sifting whilst AASconducted the data extraction MM and RW solved any conflicts PC BS MMand RW interpreted the data and drafted and revised the manuscript Theauthors read and approved the final manuscript

FundingThis systematic review was carried out as a part of the full-time School ofMedicine PhD Scholarship awarded to Paola Cocco by the University ofLeeds Additionally this work is supported by the lsquoTackling AMR-A Cross

Council Initiativersquo (grant number MRN0299761) Funding Partners The Bio-technology and Biological Sciences Research Council the Engineering andPhysical Sciences Research Council and the Medical Research Council Thiswork is also supported by the Medical Research Foundations National AMRTraining Programme Bethany Shinkins and Michael Messenger are fundedby NIHR Leeds In Vitro Diagnostic Co-operative and Cancer Research UK viaCanTest Collaborative Anam Ayaz-Shah is funded by Cancer Research UK viaCanTest Collaborative

Availability of data and materialsThe datasets generated and analysed during the current study are availablein the University of Leeds repository httpsdoiorg105518781 Thesedatasets entail an Excel spreadsheet with data extraction and an overview oftest characteristics included in reviewed TPPs

Ethics approval and consent to participateNot applicable

Consent for publicationNot applicable

Competing interestsThe authors declare that they have no competing interests

Author details1Test Evaluation Group Academic Unit of Health Economics Leeds Institutefor Health Sciences University of Leeds Leeds UK 2Academic Unit ofPrimary Care Leeds Institute for Health Sciences University of Leeds LeedsUK 3httpswwwcantestorg 4Centre for Personalised Health and MedicineUniversity of Leeds Leeds UK 5NIHR Leeds In Vitro Diagnostic (IVD)Co-operative Leeds UK 6Leeds Institute for Health Sciences University ofLeeds Leeds UK

Received 4 December 2019 Accepted 1 April 2020

References1 Wang P Kricka L Current and emerging trends in point-of-care technology

and strategies for clinical validation and implementation Clin Chem 2018httpsdoiorg101373clinchem2018287052

2 Drucker E Krapfenbauer K Pitfalls and limitations in translation frombiomarker discovery to clinical utility in predictive and personalisedmedicine EPMA Journal 20134(1)4ndash7

3 Kern S Why your new cancer biomarker may never work recurrent patternsand remarkable diversity in biomarker failures Cancer Res 2012 httpsdoiorg1011580008-5472CAN-12-3232

4 Engel N Wachter K Pai M Gallarda J Boehme C Celentano I et alAddressing the challenges of diagnostics demand and supply insights froman online global health discussion platform BMJ Glob Health 2016 httpsdoiorg101136bmjgh-2016-000132

5 Horvath A Lord S St John A Sandberg S Cobbaert C Lorenz S et al Frombiomarkers to medical tests the changing landscape of test evaluation ClinChim Acta 2014 httpsdoiorg101016jcca201309018

6 US Food and Drug Administration Guidance for industry Q8 (2)pharmaceutical development 2009 httpacademygmp-complianceorgguidemgrfiles9041FNL5b15dPDF Accessed 6 Nov 2018

7 US Food and Drug Administration Guidance for industry and review staffTarget Product Profile mdash a strategic development process tool 2007httpswwwfdagovmedia72566download Accessed 6 Nov 2018

8 WHO FIND MsF A multiplex multi-analyte diagnostic platform 2017httpswwwwhointmedicinesTPP_intro_20171122_forDistributionpdfAccessed 8 Nov 2018

9 PATH Target Product Profile HIV self-test version 41 a white paper on theevaluation of current HIV rapid tests and development of core specificationsfor next-generation HIV tests 2014 httpspathazureedgenetmediadocumentsTS_hiv_self_test_tpppdf Accessed 8 Nov 2018

10 Ebels K Clerk C Crudder C McGray S Magnuson K Tietje K et alIncorporating user needs into product development for improved infectiondetection for malaria elimination programs IEEE 2014 Global HumanitarianTechnology Conference 2014 httpsdoiorg101109GHTC20146970338

Cocco et al BMC Medicine (2020) 18119 Page 10 of 12

of 5 years [22 23 27 28 51 60] whilst the remainingconsidered a time horizon of 10 years [29]

Decision-making stepsA common decision-making framework consisting ofthree distinct phases was apparent across the includedTPPs scoping drafting and consensus-building Figure 2presents the most commonly adopted activities inputsources and engaged stakeholder groupsTable 1 provides a summary of the stakeholders con-

tributing to each phase Some of the included TPPs arenot included in Table 1 as they did not report any infor-mation related to input sources or stakeholder groups[33ndash36 52 53 55 61] A summary of the input sourcesreported to have been used at scoping and draftingphase can be found in Additional file 3 Table 33We will therefore describe the aim of each phase and

breakdown the methodology (activities input sourcesand stakeholders) used within the included TPPs wherereported For specific details on each included TPP seeAdditional file 3 Table 34

Scoping phase methodologyHalf of the TPPs provided some information on thescoping phase (n = 22) The aim of this phase was toprovide an overview of the disease area and the limita-tions associated with existing technologies The clinicalproblems and unmet needs were defined in addition toidentification of which test characteristics to include inthe TPPSome of the key activities undertaken during the scop-

ing phase included reviewing published literature (n = 6)or available data (n = 1) and introductory meetings withstakeholders (n = 4)Some authors reported (n = 4) [22 26 37 50] that

they had conducted a lsquolandscape analysisrsquo providing in-formation on the disease area of interest available diag-nostic technologies and related characteristics andlimitations These were usually based on interviews withstakeholders and reviews of the literature Only Toskinet al [50] conducted a systematic literature reviewreporting the databases searched and key words usedConsultation with experts (68 n = 15) and the litera-

ture (36 n = 8) were the most commonly sought

Table 1 Stakeholders contributing to each phase

Stakeholder groups contributing to each phasea Scoping Drafting Consensus-building

n () n () n ()

Researchers 14 (32) 15 (34) 14 (32)

Industry representatives 11 (25) 11 (25) 14 (32)

International public organisations 9 (20) 12 (27) 12 (27)

Clinicians 7 (16) 8 (18) 8 (18)

Representatives of countries and national disease programs 5 (11) 4 (9) 8 (18)

Policy makers 4 (9) 5 (11) 4 (9)

Laboratory experts 3 (7) 5 (11) 3 (7)

Technicalfunding agencies 2 (5) 3 (7) 4 (9)

Microbiologists 2 (5) 4 (9) 2 (5)

Implementers 2 (5) 2 (5) 5 (11)

Patient advocates 2 (5) 1 (2) 3 (7)

Non-profit sector 1 (2) 2 (5) 4 (9)

Modellers 1 (2) 1 (2) 1 (2)

Health economists 0 (0) 2 (5) 2 (5)

Donors 2 (5) 3 (7) 1 (2)

Market experts 2(5) 1 (2) 1 (2)

Program manager 1 (2) 2 (5) 2 (5)

Scientific associations 1 (2) 3 (7) 2 (5)

Strategists 1 (2) 1 (2) 0 (0)

Expert (unspecific) 1 (2) 3 (7) 3 (7)

Others 2 (5) 2 (5) 1 (2)

The percentages in relation to stakeholder groups do not add up to 100 because more than one stakeholder group usually contributes to the development ofa TPPaPercentages are calculated in relation to the total number of included TPPs (n = 44)

Cocco et al BMC Medicine (2020) 18119 Page 5 of 12

sources of information during the scoping phase (seeAdditional file 3 Table 33 for a full breakdown) Onlyone type of source was considered in 15 TPPs (of which11 was consulting experts) whilst 7 TPPs consideredmore than one sourceDenkinger et al [22] mapped the diagnostic ecosystem

of interest and then performed a survey to gauge stake-holdersrsquo preferences Reipold et al [48] identified themain characteristic categories (eg scope performanceoperational characteristics and pricing) to be included inthe TPPFive TPPs involved a priority-setting exercise which entailed

ranking each identified health need [23 28 32 48 60]During the scoping phase a variety of stakeholders

were engaged (Table 1)

Drafting phase methodologyThe first draft of each TPP was usually prepared by ei-ther an established working group comprising expertsfrom different organisations [9 26 29 32 40 50 58] orauthors of the published TPP There were two caseswhere the TPP was drafted by a completely different or-ganisation [51 57] The TPP was often revised severaltimes and in some cases it was then shortened to en-sure it could be easily communicated to different stake-holders [22 23 28 29 60]Of the 44 included TPPs 33 of them reported which

input sources were considered during the drafting phase(75) (Additional file 3 Table 33) Common inputsources for populating test characteristics were expertconsultations (n = 22) and reviews of the literature (n =22) Some also referred to mathematical models (n = 9)available data (n = 7) guidelines (n = 6) and lsquofield obser-vationsrsquo (n = 5) Only one TPP was informed by pooleddata from a systematic review [50]Twenty-six of the 44 TPPs took into consideration more

than one type of source at the drafting phase as opposedto 7 TPPs which only adopted one (Additional file 3Table 33) Meeting inputs were the most common singlesource (43 n = 3)The stakeholders engaged in the drafting phase are re-

ported in Table 1

Consensus-building phase methodologyInitial agreement with the TPP was often obtained usinga survey of the stakeholders (n = 14) The survey eitherincluded general questions regarding stakeholders viewson the TPP (n = 4) [22 25 27 51] or adopted a Delphi-like approach to provide an initial consensus on variousaspects of the TPP (n = 10) A consensus meeting withstakeholders and experts was typically held (n = 11) anda revised TPP generally agreed upon In some cases anadditional survey was sent to stakeholders on trade-offsbetween test attributes [48] or on rating key parameters

[51 53] For 2 TPPs the final TPP draft was presentedto a broader stakeholder base to validate itThe number of participants invited to the consensus-

building meetings varied (lt 20 participants n = 5 between20 and 50 participants n = 7) One meeting included 100participants [27] For a few of the TPPs the authors alsotook part in the consensus meetings [29 38 58 60]Less than half of the included TPPs reported information

on the activities and stakeholders invited to the consensus-building phase (n = 19) The stakeholders engaged in theconsensus-building phase are reported in Table 1

Transparency in reporting methodsWe also assessed the transparency of the TPPs in termsof reporting their methodology (see Additional file 3Table 35) The decision-making process behind the TPPwas not reported in over a quarter of the included TPPs(n = 16) Further to this many failed to report which in-formation sources were considered to populate the TPP(n = 11) Just under half did not report which stake-holders were involved in the development of the TPP(n = 20) Specifically the name of the organisationsstakeholders were part was only reported in 11 TPPswhilst 9 TPPs mentioned personal details of each stake-holder (20) and 4 TPPs explained why certain stake-holders were invited [26 38 46 58] Sixteen TPPsreported the source of funding (36)There were some TPPs where the methodology was

very clearly reported [26 28 38 60]

Test characteristics included in TPPsAfter removing duplicates 140 different test characteris-tics were reported across the included TPPs Some fea-tures which did not represent test characteristics havebeen excluded such as factors relating specifically to thedisease in question rather than the test For more infor-mation please see Test Characteristics Overview Excelspreadsheet (Availability of data and materials) Figure 3shows the test characteristics most frequently reported(a full list is available in Additional file 2 Table 21)Figure 4 depicts which characteristic categories were

reported in the included TPPs Details on unmet clinicalneed analytical performance and clinical validity ap-peared to be consistently reported however regula-tory requirements environmental footprint and clinicalutility were less frequently considered

DiscussionWe report a systematic review of the methods currentlyused to develop Target Product Profiles for medicaltests Despite TPPs for any medical test being searchedall of the identified TPPs were focused on diagnostictests for infection

Cocco et al BMC Medicine (2020) 18119 Page 6 of 12

Fig 3 Test characteristics frequently reported in all TPPs (n = 44) sorted by categories

Fig 4 Test characteristic categories in absolute number (n) of included TPPs

Cocco et al BMC Medicine (2020) 18119 Page 7 of 12

There was generally a lack of transparency andconsistency in reporting the methods underlying TPPsThis would make it difficult to appraise the recommen-dations within the TPPs ascertain whether the recom-mendations are generalizable to other settings andchallenging to reproduce

Relevancy of TPPs for test manufacturersThe purpose of a TPP is to identify upfront the essen-tial characteristics of a test for it to fulfil a pre-specified unmet clinical need This should in turn in-crease the likelihood that the test will be adopted intoclinical practice and reimbursed [1] A TPP should alsoaccount for contextual aspects that might affect thetestrsquos real-world performance [10] defining infrastruc-tural and technical constraints that impact on the im-plementation into clinical practice This review showsthat TPPs to date have primarily been developed forglobal health applications as the main funding organi-sations are WHO UNICEF and Bill and Melinda GatesFoundation The primary focus on infectious diseasesmay be explained by the remit of the global organisa-tions who fund TPPs WHO included HIVAIDSneglected tropical diseases tuberculosis and malaria aspriority diseases [62] Further to this WHO establishedthe lsquoRampD Blueprintrsquo which aims to promote RampD activ-ities (tests vaccines medicines) during epidemics [63]After having identified pathogen to target first TPPsare usually commissioned to guide the developmentprocess of new healthcare products which will addressthe high-priority pathogen [63]However the development of TPPs should not be limited

to one specific disease area or clinical setting the concept oflsquobeginning with an end in mindrsquo embodied by TPPs couldsupport both international and national health decision-makers This activity should in turn stimulate innovation ofnew tests driven by clinical needs rather than solely by la-boratory discoveries It would also provide manufacturerswith greater clarity around test requirements and confidencein the market for developing innovative tests

Identified limitations in current TPP methodologyIn reviewing current methodology for developing TPPsfor medical tests we have identified three key areaswhere current TPP methodology could be improved (1)oversight of clinical utility (2) a focus on price ratherthan cost-effectiveness and (3) subjectivity of informa-tion sources Here we discuss each limitation and theimplications

Oversight of clinical utilityVery few of the TPPs reported desirable characteristicsrelating to the clinical utility of the test This is not sur-prising given that the majority of research efforts has

focused on generating evidence on the analytical per-formance and diagnostic accuracy of a new test [11] Ahighly accurate test does not necessarily mean that thetest will improve patient health as factors relating todecision-making and the effectiveness of patient man-agement strategies could fall short [64]Assessing the clinical utility of a new test is ex-

tremely challenging Measuring the impact of a teston patient health outcomes is difficult as tests tend toguide patient management decisions rather than dir-ectly impacting on patient health outcomes [5]Therefore estimating the clinical utility of a test re-quires evidence of how the information from a test isincorporated into decision-making and the down-stream effectiveness of those decisions [65] In thecase of a new test this is particularly complicatedgiven the uncertainty around the mechanisms bywhich the test will impact on patient outcomes [65]

Focus on price rather than cost-effectivenessAlthough the minimum and optimal price of the testsfeatured in many of the TPPs none of these was drivenby the trade-off between the overall cost implications ofimplementing the test and the associated patient bene-fits Cost-effectiveness analysis provides a framework tocompare costs and benefits of an intervention againstrelevant comparators including current practice Spe-cifically cost-effectiveness analysis defines whether theintervention being evaluated represents good value formoneyIt is important to consider the cost of the new test in

the context of the benefits that the test may provide Forexample a new test may be relatively expensive but mayalso improve patient health to the extent that the add-itional is justified Conversely a new test may be rela-tively cheap but offer no improvements in patient healthand therefore even the marginal increase in cost is notjustifiedConducting cost-effectiveness analysis at early RampD

stages of new tests can therefore help manufacturers toavoid significant investments in tests that do not havethe potential to be cost-effective [65]These first two limitations are particularly relevant

since decision-makers increasingly demand evidencethat a new test improves patient health and is cost-effective rather than solely evidence of its analyticaland clinical validity [5] Specifically many HealthTechnology Assessment bodies in Europe Australiaand North America consider clinical utility cost andcost-effectiveness in relation to the target populationin addition to analytical performance and clinicalvalidity when assessing new molecular diagnostictests [66]

Cocco et al BMC Medicine (2020) 18119 Page 8 of 12

Subjectivity of input sourcesExpert judgement and evidence identified in publishedliterature were the main sources of information for de-fining desirable characteristics Systematic reviews of theliterature where database searches are reproducible andthe quality of relevant studies are appraised were notconducted to identify relevant evidence at the scopingand drafting phase This is likely to introduce bias andsubjectivity in terms of the evidence used to underpintest characteristic recommendationsAlthough expert judgement is undoubtedly useful

relying solely upon this information source has somelimitations particularly for quantitative estimates Howhumans make probability judgements is highly affectedby many heuristics and systematic biases (eg anchoringavailability overconfidence and insight bias) [67] Specif-ically previous literature has found a poor understand-ing of test accuracy among healthcare professionals [68]as sensitivity and specificity are often misinterpreted andmistaken for predictive values [68]Additionally the quality of expert elicitation heavily

relies on expert selection as it is important to choose ex-perts with good subject knowledge Only 4 TPPs de-scribed how the selection process took place andtherefore the quality of expert judgements might bequestioned Furthermore many TPPs reported literatureas a source for informing TPPs however less than halfof the TPPs cited the references considered This lack oftransparency might hinder the quality and credibility ofsources on which TPPs are based

Study limitationsSince this study is a systematic review of publicly avail-able literature a key study limitation is that we have in-evitably missed any confidential or unpublished TPPsdeveloped in-house by test manufacturers Although theresults of our online searches did not identify any com-panies stating that they have developed TPPs for medicaltests we would not expect to find such information oncompany websites Anecdotally however we have notencountered any formal TPP development activity (bythis we mean definition of desirable test characteristics)within the National Institute for Health Research LeedsIn Vitro Diagnostics Co-operative (NIHR Leeds) MICindustry networkAs there are no guidelines on how TPPs for medical

tests should be developed we did not formally assessrisk of bias We did however appraise the transparencywith which the methodology underpinning each TPPhad been reported Unfortunately it was not possible tofully evaluate the TPP developed by PATH [40] as theonline appendices were not accessible Additionally dueto poor methodological transparency it was difficult toassess with certainty authorship of TPPs and whether

authors of TPPs took part themselves in the consensus-building meetings

Future researchAlthough there is evidence of a common developmentframework this review highlights that there is consider-able variability in the methods employed to draft TPPsand inconsistencies in which test characteristics are de-scribed A key issue in reviewing the methods imple-mented was the lack of transparency in methodologyreportingGuidance on best practice methods for developing

TPPs for medical tests would be highly beneficial Simi-larly to the US FDA guidance on TPPs for drugs a guid-ance document could be developed for TPPs for medicaltests summarising the purpose attributes of TPPs andwhich test characteristics should be includedHowever to inform the development of such guidance

future research should focus firstly on how to systemat-ically identify unmet clinical needs underpinning a cer-tain disease area Monaghan et al [69] developed avaluable checklist for identifying biomarkers based onliterature findings and consultations with experts Webelieve that this checklist could be pertinent for thescoping phase underlying TPP development howeverthis would need further validation in this specificcontextMore research is also required to understand how to

better incorporate the assessment of desirable clinicalutility and cost-effectiveness of innovative tests intoTPPs One possible way forward could be exploring howand if care pathway analysis and early economic model-ling could be integrated into the development of TPPsCare pathway analysis would provide clarity on themechanisms by which a test could impact on down-stream patient outcomes Early economic modellingcould be used to define desirable values for certain testcharacteristics (eg test price diagnostic sensitivity andspecificity) based on cost-effectiveness [67] Thereforeintegrating care pathway analysis and early economicmodelling into TPP development might provide moreevidence-based information to the test developersOutside of the actual methodology for developing

TPPs it would be useful to better understand whethermanufacturers develop tests strictly in line with TPPs orwhether there are any factors which make this infeasibleor challenging Additionally we would be interested inwhich methods manufacturers usually adopt to developTPPs to assess if there are any differences with themethodological framework we highlighted here To thisend interviewing test manufacturers might provide in-teresting insights on the intrinsic value of TPPs for theindustry

Cocco et al BMC Medicine (2020) 18119 Page 9 of 12

Most importantly to ensure that the development ofTPPs becomes widespread practice it would be valuableto explore how TPPs could be integrated into existingregulatory paths for innovation such as the EuropeanUnion Regulation for In Vitro Diagnostics (Regulation2017746) and the US FDA Drug Development and Ap-proval Process It might then be possible to align testcharacteristics featured in TPPs with evidence require-ments which are relevant for market approval decisionsof new medical tests This in return might increase theapplicability of TPPs for the industry

ConclusionsBased on this review we summarised current methodo-logical practice into a framework of value to those inter-ested in developing TPPs for medical testsWe also identified some key weaknesses including the

quality of the information sources underpinning TPPsand failure to consider test characteristics relating toclinical utility and cost-effectivenessThis review thus provides some recommendations for

further methodological research on the development ofTPPs for medical test This work will also help to informthe development of a formal guideline on how to draftTPPs for medical tests

Supplementary informationSupplementary information accompanies this paper at httpsdoiorg101186s12916-020-01582-1

Additional file 1 Search strategies

Additional file 2 Test characteristics

Additional file 3 Results

AbbreviationsACCE Analytic validity Clinical validity Clinical Utility Ethical legal and socialimplications AIDS Acquired immune deficiency syndrome CEA Cost-effectiveness analysis FDA Food and Drug Administration FIND TheFoundation For Innovative New Diagnostics HIV Human immunodeficiencyvirus IDC International Diagnostics Center NIHR National Institute for HealthResearch PRISMA Preferred Reporting Items for Systematic Reviews andMeta-Analysis QTPP Quality Target Product Profile RampD Research andDevelopment TPP Target Product Profile UNICEF United NationsInternational Childrenrsquos Emergency Fund WHO World Health Organization

AcknowledgementsThe authors would like to express their gratitude to Natalie King (Universityof Leeds Information Specialist) for supporting the design of the searchstrategy for this systematic review

Authorsrsquo contributionsAll authors contributed to the conception and design of the work PCperformed the search strategy BS contributed to reference sifting whilst AASconducted the data extraction MM and RW solved any conflicts PC BS MMand RW interpreted the data and drafted and revised the manuscript Theauthors read and approved the final manuscript

FundingThis systematic review was carried out as a part of the full-time School ofMedicine PhD Scholarship awarded to Paola Cocco by the University ofLeeds Additionally this work is supported by the lsquoTackling AMR-A Cross

Council Initiativersquo (grant number MRN0299761) Funding Partners The Bio-technology and Biological Sciences Research Council the Engineering andPhysical Sciences Research Council and the Medical Research Council Thiswork is also supported by the Medical Research Foundations National AMRTraining Programme Bethany Shinkins and Michael Messenger are fundedby NIHR Leeds In Vitro Diagnostic Co-operative and Cancer Research UK viaCanTest Collaborative Anam Ayaz-Shah is funded by Cancer Research UK viaCanTest Collaborative

Availability of data and materialsThe datasets generated and analysed during the current study are availablein the University of Leeds repository httpsdoiorg105518781 Thesedatasets entail an Excel spreadsheet with data extraction and an overview oftest characteristics included in reviewed TPPs

Ethics approval and consent to participateNot applicable

Consent for publicationNot applicable

Competing interestsThe authors declare that they have no competing interests

Author details1Test Evaluation Group Academic Unit of Health Economics Leeds Institutefor Health Sciences University of Leeds Leeds UK 2Academic Unit ofPrimary Care Leeds Institute for Health Sciences University of Leeds LeedsUK 3httpswwwcantestorg 4Centre for Personalised Health and MedicineUniversity of Leeds Leeds UK 5NIHR Leeds In Vitro Diagnostic (IVD)Co-operative Leeds UK 6Leeds Institute for Health Sciences University ofLeeds Leeds UK

Received 4 December 2019 Accepted 1 April 2020

References1 Wang P Kricka L Current and emerging trends in point-of-care technology

and strategies for clinical validation and implementation Clin Chem 2018httpsdoiorg101373clinchem2018287052

2 Drucker E Krapfenbauer K Pitfalls and limitations in translation frombiomarker discovery to clinical utility in predictive and personalisedmedicine EPMA Journal 20134(1)4ndash7

3 Kern S Why your new cancer biomarker may never work recurrent patternsand remarkable diversity in biomarker failures Cancer Res 2012 httpsdoiorg1011580008-5472CAN-12-3232

4 Engel N Wachter K Pai M Gallarda J Boehme C Celentano I et alAddressing the challenges of diagnostics demand and supply insights froman online global health discussion platform BMJ Glob Health 2016 httpsdoiorg101136bmjgh-2016-000132

5 Horvath A Lord S St John A Sandberg S Cobbaert C Lorenz S et al Frombiomarkers to medical tests the changing landscape of test evaluation ClinChim Acta 2014 httpsdoiorg101016jcca201309018

6 US Food and Drug Administration Guidance for industry Q8 (2)pharmaceutical development 2009 httpacademygmp-complianceorgguidemgrfiles9041FNL5b15dPDF Accessed 6 Nov 2018

7 US Food and Drug Administration Guidance for industry and review staffTarget Product Profile mdash a strategic development process tool 2007httpswwwfdagovmedia72566download Accessed 6 Nov 2018

8 WHO FIND MsF A multiplex multi-analyte diagnostic platform 2017httpswwwwhointmedicinesTPP_intro_20171122_forDistributionpdfAccessed 8 Nov 2018

9 PATH Target Product Profile HIV self-test version 41 a white paper on theevaluation of current HIV rapid tests and development of core specificationsfor next-generation HIV tests 2014 httpspathazureedgenetmediadocumentsTS_hiv_self_test_tpppdf Accessed 8 Nov 2018

10 Ebels K Clerk C Crudder C McGray S Magnuson K Tietje K et alIncorporating user needs into product development for improved infectiondetection for malaria elimination programs IEEE 2014 Global HumanitarianTechnology Conference 2014 httpsdoiorg101109GHTC20146970338

Cocco et al BMC Medicine (2020) 18119 Page 10 of 12

sources of information during the scoping phase (seeAdditional file 3 Table 33 for a full breakdown) Onlyone type of source was considered in 15 TPPs (of which11 was consulting experts) whilst 7 TPPs consideredmore than one sourceDenkinger et al [22] mapped the diagnostic ecosystem

of interest and then performed a survey to gauge stake-holdersrsquo preferences Reipold et al [48] identified themain characteristic categories (eg scope performanceoperational characteristics and pricing) to be included inthe TPPFive TPPs involved a priority-setting exercise which entailed

ranking each identified health need [23 28 32 48 60]During the scoping phase a variety of stakeholders

were engaged (Table 1)

Drafting phase methodologyThe first draft of each TPP was usually prepared by ei-ther an established working group comprising expertsfrom different organisations [9 26 29 32 40 50 58] orauthors of the published TPP There were two caseswhere the TPP was drafted by a completely different or-ganisation [51 57] The TPP was often revised severaltimes and in some cases it was then shortened to en-sure it could be easily communicated to different stake-holders [22 23 28 29 60]Of the 44 included TPPs 33 of them reported which

input sources were considered during the drafting phase(75) (Additional file 3 Table 33) Common inputsources for populating test characteristics were expertconsultations (n = 22) and reviews of the literature (n =22) Some also referred to mathematical models (n = 9)available data (n = 7) guidelines (n = 6) and lsquofield obser-vationsrsquo (n = 5) Only one TPP was informed by pooleddata from a systematic review [50]Twenty-six of the 44 TPPs took into consideration more

than one type of source at the drafting phase as opposedto 7 TPPs which only adopted one (Additional file 3Table 33) Meeting inputs were the most common singlesource (43 n = 3)The stakeholders engaged in the drafting phase are re-

ported in Table 1

Consensus-building phase methodologyInitial agreement with the TPP was often obtained usinga survey of the stakeholders (n = 14) The survey eitherincluded general questions regarding stakeholders viewson the TPP (n = 4) [22 25 27 51] or adopted a Delphi-like approach to provide an initial consensus on variousaspects of the TPP (n = 10) A consensus meeting withstakeholders and experts was typically held (n = 11) anda revised TPP generally agreed upon In some cases anadditional survey was sent to stakeholders on trade-offsbetween test attributes [48] or on rating key parameters

[51 53] For 2 TPPs the final TPP draft was presentedto a broader stakeholder base to validate itThe number of participants invited to the consensus-

building meetings varied (lt 20 participants n = 5 between20 and 50 participants n = 7) One meeting included 100participants [27] For a few of the TPPs the authors alsotook part in the consensus meetings [29 38 58 60]Less than half of the included TPPs reported information

on the activities and stakeholders invited to the consensus-building phase (n = 19) The stakeholders engaged in theconsensus-building phase are reported in Table 1

Transparency in reporting methodsWe also assessed the transparency of the TPPs in termsof reporting their methodology (see Additional file 3Table 35) The decision-making process behind the TPPwas not reported in over a quarter of the included TPPs(n = 16) Further to this many failed to report which in-formation sources were considered to populate the TPP(n = 11) Just under half did not report which stake-holders were involved in the development of the TPP(n = 20) Specifically the name of the organisationsstakeholders were part was only reported in 11 TPPswhilst 9 TPPs mentioned personal details of each stake-holder (20) and 4 TPPs explained why certain stake-holders were invited [26 38 46 58] Sixteen TPPsreported the source of funding (36)There were some TPPs where the methodology was

very clearly reported [26 28 38 60]

Test characteristics included in TPPsAfter removing duplicates 140 different test characteris-tics were reported across the included TPPs Some fea-tures which did not represent test characteristics havebeen excluded such as factors relating specifically to thedisease in question rather than the test For more infor-mation please see Test Characteristics Overview Excelspreadsheet (Availability of data and materials) Figure 3shows the test characteristics most frequently reported(a full list is available in Additional file 2 Table 21)Figure 4 depicts which characteristic categories were

reported in the included TPPs Details on unmet clinicalneed analytical performance and clinical validity ap-peared to be consistently reported however regula-tory requirements environmental footprint and clinicalutility were less frequently considered

DiscussionWe report a systematic review of the methods currentlyused to develop Target Product Profiles for medicaltests Despite TPPs for any medical test being searchedall of the identified TPPs were focused on diagnostictests for infection

Cocco et al BMC Medicine (2020) 18119 Page 6 of 12

Fig 3 Test characteristics frequently reported in all TPPs (n = 44) sorted by categories

Fig 4 Test characteristic categories in absolute number (n) of included TPPs

Cocco et al BMC Medicine (2020) 18119 Page 7 of 12

There was generally a lack of transparency andconsistency in reporting the methods underlying TPPsThis would make it difficult to appraise the recommen-dations within the TPPs ascertain whether the recom-mendations are generalizable to other settings andchallenging to reproduce

Relevancy of TPPs for test manufacturersThe purpose of a TPP is to identify upfront the essen-tial characteristics of a test for it to fulfil a pre-specified unmet clinical need This should in turn in-crease the likelihood that the test will be adopted intoclinical practice and reimbursed [1] A TPP should alsoaccount for contextual aspects that might affect thetestrsquos real-world performance [10] defining infrastruc-tural and technical constraints that impact on the im-plementation into clinical practice This review showsthat TPPs to date have primarily been developed forglobal health applications as the main funding organi-sations are WHO UNICEF and Bill and Melinda GatesFoundation The primary focus on infectious diseasesmay be explained by the remit of the global organisa-tions who fund TPPs WHO included HIVAIDSneglected tropical diseases tuberculosis and malaria aspriority diseases [62] Further to this WHO establishedthe lsquoRampD Blueprintrsquo which aims to promote RampD activ-ities (tests vaccines medicines) during epidemics [63]After having identified pathogen to target first TPPsare usually commissioned to guide the developmentprocess of new healthcare products which will addressthe high-priority pathogen [63]However the development of TPPs should not be limited

to one specific disease area or clinical setting the concept oflsquobeginning with an end in mindrsquo embodied by TPPs couldsupport both international and national health decision-makers This activity should in turn stimulate innovation ofnew tests driven by clinical needs rather than solely by la-boratory discoveries It would also provide manufacturerswith greater clarity around test requirements and confidencein the market for developing innovative tests

Identified limitations in current TPP methodologyIn reviewing current methodology for developing TPPsfor medical tests we have identified three key areaswhere current TPP methodology could be improved (1)oversight of clinical utility (2) a focus on price ratherthan cost-effectiveness and (3) subjectivity of informa-tion sources Here we discuss each limitation and theimplications

Oversight of clinical utilityVery few of the TPPs reported desirable characteristicsrelating to the clinical utility of the test This is not sur-prising given that the majority of research efforts has

focused on generating evidence on the analytical per-formance and diagnostic accuracy of a new test [11] Ahighly accurate test does not necessarily mean that thetest will improve patient health as factors relating todecision-making and the effectiveness of patient man-agement strategies could fall short [64]Assessing the clinical utility of a new test is ex-

tremely challenging Measuring the impact of a teston patient health outcomes is difficult as tests tend toguide patient management decisions rather than dir-ectly impacting on patient health outcomes [5]Therefore estimating the clinical utility of a test re-quires evidence of how the information from a test isincorporated into decision-making and the down-stream effectiveness of those decisions [65] In thecase of a new test this is particularly complicatedgiven the uncertainty around the mechanisms bywhich the test will impact on patient outcomes [65]

Focus on price rather than cost-effectivenessAlthough the minimum and optimal price of the testsfeatured in many of the TPPs none of these was drivenby the trade-off between the overall cost implications ofimplementing the test and the associated patient bene-fits Cost-effectiveness analysis provides a framework tocompare costs and benefits of an intervention againstrelevant comparators including current practice Spe-cifically cost-effectiveness analysis defines whether theintervention being evaluated represents good value formoneyIt is important to consider the cost of the new test in

the context of the benefits that the test may provide Forexample a new test may be relatively expensive but mayalso improve patient health to the extent that the add-itional is justified Conversely a new test may be rela-tively cheap but offer no improvements in patient healthand therefore even the marginal increase in cost is notjustifiedConducting cost-effectiveness analysis at early RampD

stages of new tests can therefore help manufacturers toavoid significant investments in tests that do not havethe potential to be cost-effective [65]These first two limitations are particularly relevant

since decision-makers increasingly demand evidencethat a new test improves patient health and is cost-effective rather than solely evidence of its analyticaland clinical validity [5] Specifically many HealthTechnology Assessment bodies in Europe Australiaand North America consider clinical utility cost andcost-effectiveness in relation to the target populationin addition to analytical performance and clinicalvalidity when assessing new molecular diagnostictests [66]

Cocco et al BMC Medicine (2020) 18119 Page 8 of 12

Subjectivity of input sourcesExpert judgement and evidence identified in publishedliterature were the main sources of information for de-fining desirable characteristics Systematic reviews of theliterature where database searches are reproducible andthe quality of relevant studies are appraised were notconducted to identify relevant evidence at the scopingand drafting phase This is likely to introduce bias andsubjectivity in terms of the evidence used to underpintest characteristic recommendationsAlthough expert judgement is undoubtedly useful

relying solely upon this information source has somelimitations particularly for quantitative estimates Howhumans make probability judgements is highly affectedby many heuristics and systematic biases (eg anchoringavailability overconfidence and insight bias) [67] Specif-ically previous literature has found a poor understand-ing of test accuracy among healthcare professionals [68]as sensitivity and specificity are often misinterpreted andmistaken for predictive values [68]Additionally the quality of expert elicitation heavily

relies on expert selection as it is important to choose ex-perts with good subject knowledge Only 4 TPPs de-scribed how the selection process took place andtherefore the quality of expert judgements might bequestioned Furthermore many TPPs reported literatureas a source for informing TPPs however less than halfof the TPPs cited the references considered This lack oftransparency might hinder the quality and credibility ofsources on which TPPs are based

Study limitationsSince this study is a systematic review of publicly avail-able literature a key study limitation is that we have in-evitably missed any confidential or unpublished TPPsdeveloped in-house by test manufacturers Although theresults of our online searches did not identify any com-panies stating that they have developed TPPs for medicaltests we would not expect to find such information oncompany websites Anecdotally however we have notencountered any formal TPP development activity (bythis we mean definition of desirable test characteristics)within the National Institute for Health Research LeedsIn Vitro Diagnostics Co-operative (NIHR Leeds) MICindustry networkAs there are no guidelines on how TPPs for medical

tests should be developed we did not formally assessrisk of bias We did however appraise the transparencywith which the methodology underpinning each TPPhad been reported Unfortunately it was not possible tofully evaluate the TPP developed by PATH [40] as theonline appendices were not accessible Additionally dueto poor methodological transparency it was difficult toassess with certainty authorship of TPPs and whether

authors of TPPs took part themselves in the consensus-building meetings

Future researchAlthough there is evidence of a common developmentframework this review highlights that there is consider-able variability in the methods employed to draft TPPsand inconsistencies in which test characteristics are de-scribed A key issue in reviewing the methods imple-mented was the lack of transparency in methodologyreportingGuidance on best practice methods for developing

TPPs for medical tests would be highly beneficial Simi-larly to the US FDA guidance on TPPs for drugs a guid-ance document could be developed for TPPs for medicaltests summarising the purpose attributes of TPPs andwhich test characteristics should be includedHowever to inform the development of such guidance

future research should focus firstly on how to systemat-ically identify unmet clinical needs underpinning a cer-tain disease area Monaghan et al [69] developed avaluable checklist for identifying biomarkers based onliterature findings and consultations with experts Webelieve that this checklist could be pertinent for thescoping phase underlying TPP development howeverthis would need further validation in this specificcontextMore research is also required to understand how to

better incorporate the assessment of desirable clinicalutility and cost-effectiveness of innovative tests intoTPPs One possible way forward could be exploring howand if care pathway analysis and early economic model-ling could be integrated into the development of TPPsCare pathway analysis would provide clarity on themechanisms by which a test could impact on down-stream patient outcomes Early economic modellingcould be used to define desirable values for certain testcharacteristics (eg test price diagnostic sensitivity andspecificity) based on cost-effectiveness [67] Thereforeintegrating care pathway analysis and early economicmodelling into TPP development might provide moreevidence-based information to the test developersOutside of the actual methodology for developing

TPPs it would be useful to better understand whethermanufacturers develop tests strictly in line with TPPs orwhether there are any factors which make this infeasibleor challenging Additionally we would be interested inwhich methods manufacturers usually adopt to developTPPs to assess if there are any differences with themethodological framework we highlighted here To thisend interviewing test manufacturers might provide in-teresting insights on the intrinsic value of TPPs for theindustry

Cocco et al BMC Medicine (2020) 18119 Page 9 of 12

Most importantly to ensure that the development ofTPPs becomes widespread practice it would be valuableto explore how TPPs could be integrated into existingregulatory paths for innovation such as the EuropeanUnion Regulation for In Vitro Diagnostics (Regulation2017746) and the US FDA Drug Development and Ap-proval Process It might then be possible to align testcharacteristics featured in TPPs with evidence require-ments which are relevant for market approval decisionsof new medical tests This in return might increase theapplicability of TPPs for the industry

ConclusionsBased on this review we summarised current methodo-logical practice into a framework of value to those inter-ested in developing TPPs for medical testsWe also identified some key weaknesses including the

quality of the information sources underpinning TPPsand failure to consider test characteristics relating toclinical utility and cost-effectivenessThis review thus provides some recommendations for

further methodological research on the development ofTPPs for medical test This work will also help to informthe development of a formal guideline on how to draftTPPs for medical tests

Supplementary informationSupplementary information accompanies this paper at httpsdoiorg101186s12916-020-01582-1

Additional file 1 Search strategies

Additional file 2 Test characteristics

Additional file 3 Results

AbbreviationsACCE Analytic validity Clinical validity Clinical Utility Ethical legal and socialimplications AIDS Acquired immune deficiency syndrome CEA Cost-effectiveness analysis FDA Food and Drug Administration FIND TheFoundation For Innovative New Diagnostics HIV Human immunodeficiencyvirus IDC International Diagnostics Center NIHR National Institute for HealthResearch PRISMA Preferred Reporting Items for Systematic Reviews andMeta-Analysis QTPP Quality Target Product Profile RampD Research andDevelopment TPP Target Product Profile UNICEF United NationsInternational Childrenrsquos Emergency Fund WHO World Health Organization

AcknowledgementsThe authors would like to express their gratitude to Natalie King (Universityof Leeds Information Specialist) for supporting the design of the searchstrategy for this systematic review

Authorsrsquo contributionsAll authors contributed to the conception and design of the work PCperformed the search strategy BS contributed to reference sifting whilst AASconducted the data extraction MM and RW solved any conflicts PC BS MMand RW interpreted the data and drafted and revised the manuscript Theauthors read and approved the final manuscript

FundingThis systematic review was carried out as a part of the full-time School ofMedicine PhD Scholarship awarded to Paola Cocco by the University ofLeeds Additionally this work is supported by the lsquoTackling AMR-A Cross

Council Initiativersquo (grant number MRN0299761) Funding Partners The Bio-technology and Biological Sciences Research Council the Engineering andPhysical Sciences Research Council and the Medical Research Council Thiswork is also supported by the Medical Research Foundations National AMRTraining Programme Bethany Shinkins and Michael Messenger are fundedby NIHR Leeds In Vitro Diagnostic Co-operative and Cancer Research UK viaCanTest Collaborative Anam Ayaz-Shah is funded by Cancer Research UK viaCanTest Collaborative

Availability of data and materialsThe datasets generated and analysed during the current study are availablein the University of Leeds repository httpsdoiorg105518781 Thesedatasets entail an Excel spreadsheet with data extraction and an overview oftest characteristics included in reviewed TPPs

Ethics approval and consent to participateNot applicable

Consent for publicationNot applicable

Competing interestsThe authors declare that they have no competing interests

Author details1Test Evaluation Group Academic Unit of Health Economics Leeds Institutefor Health Sciences University of Leeds Leeds UK 2Academic Unit ofPrimary Care Leeds Institute for Health Sciences University of Leeds LeedsUK 3httpswwwcantestorg 4Centre for Personalised Health and MedicineUniversity of Leeds Leeds UK 5NIHR Leeds In Vitro Diagnostic (IVD)Co-operative Leeds UK 6Leeds Institute for Health Sciences University ofLeeds Leeds UK

Received 4 December 2019 Accepted 1 April 2020

References1 Wang P Kricka L Current and emerging trends in point-of-care technology

and strategies for clinical validation and implementation Clin Chem 2018httpsdoiorg101373clinchem2018287052

2 Drucker E Krapfenbauer K Pitfalls and limitations in translation frombiomarker discovery to clinical utility in predictive and personalisedmedicine EPMA Journal 20134(1)4ndash7

3 Kern S Why your new cancer biomarker may never work recurrent patternsand remarkable diversity in biomarker failures Cancer Res 2012 httpsdoiorg1011580008-5472CAN-12-3232

4 Engel N Wachter K Pai M Gallarda J Boehme C Celentano I et alAddressing the challenges of diagnostics demand and supply insights froman online global health discussion platform BMJ Glob Health 2016 httpsdoiorg101136bmjgh-2016-000132

5 Horvath A Lord S St John A Sandberg S Cobbaert C Lorenz S et al Frombiomarkers to medical tests the changing landscape of test evaluation ClinChim Acta 2014 httpsdoiorg101016jcca201309018

6 US Food and Drug Administration Guidance for industry Q8 (2)pharmaceutical development 2009 httpacademygmp-complianceorgguidemgrfiles9041FNL5b15dPDF Accessed 6 Nov 2018

7 US Food and Drug Administration Guidance for industry and review staffTarget Product Profile mdash a strategic development process tool 2007httpswwwfdagovmedia72566download Accessed 6 Nov 2018

8 WHO FIND MsF A multiplex multi-analyte diagnostic platform 2017httpswwwwhointmedicinesTPP_intro_20171122_forDistributionpdfAccessed 8 Nov 2018

9 PATH Target Product Profile HIV self-test version 41 a white paper on theevaluation of current HIV rapid tests and development of core specificationsfor next-generation HIV tests 2014 httpspathazureedgenetmediadocumentsTS_hiv_self_test_tpppdf Accessed 8 Nov 2018

10 Ebels K Clerk C Crudder C McGray S Magnuson K Tietje K et alIncorporating user needs into product development for improved infectiondetection for malaria elimination programs IEEE 2014 Global HumanitarianTechnology Conference 2014 httpsdoiorg101109GHTC20146970338

Cocco et al BMC Medicine (2020) 18119 Page 10 of 12

Fig 3 Test characteristics frequently reported in all TPPs (n = 44) sorted by categories

Fig 4 Test characteristic categories in absolute number (n) of included TPPs

Cocco et al BMC Medicine (2020) 18119 Page 7 of 12

There was generally a lack of transparency andconsistency in reporting the methods underlying TPPsThis would make it difficult to appraise the recommen-dations within the TPPs ascertain whether the recom-mendations are generalizable to other settings andchallenging to reproduce

Relevancy of TPPs for test manufacturersThe purpose of a TPP is to identify upfront the essen-tial characteristics of a test for it to fulfil a pre-specified unmet clinical need This should in turn in-crease the likelihood that the test will be adopted intoclinical practice and reimbursed [1] A TPP should alsoaccount for contextual aspects that might affect thetestrsquos real-world performance [10] defining infrastruc-tural and technical constraints that impact on the im-plementation into clinical practice This review showsthat TPPs to date have primarily been developed forglobal health applications as the main funding organi-sations are WHO UNICEF and Bill and Melinda GatesFoundation The primary focus on infectious diseasesmay be explained by the remit of the global organisa-tions who fund TPPs WHO included HIVAIDSneglected tropical diseases tuberculosis and malaria aspriority diseases [62] Further to this WHO establishedthe lsquoRampD Blueprintrsquo which aims to promote RampD activ-ities (tests vaccines medicines) during epidemics [63]After having identified pathogen to target first TPPsare usually commissioned to guide the developmentprocess of new healthcare products which will addressthe high-priority pathogen [63]However the development of TPPs should not be limited

to one specific disease area or clinical setting the concept oflsquobeginning with an end in mindrsquo embodied by TPPs couldsupport both international and national health decision-makers This activity should in turn stimulate innovation ofnew tests driven by clinical needs rather than solely by la-boratory discoveries It would also provide manufacturerswith greater clarity around test requirements and confidencein the market for developing innovative tests

Identified limitations in current TPP methodologyIn reviewing current methodology for developing TPPsfor medical tests we have identified three key areaswhere current TPP methodology could be improved (1)oversight of clinical utility (2) a focus on price ratherthan cost-effectiveness and (3) subjectivity of informa-tion sources Here we discuss each limitation and theimplications

Oversight of clinical utilityVery few of the TPPs reported desirable characteristicsrelating to the clinical utility of the test This is not sur-prising given that the majority of research efforts has

focused on generating evidence on the analytical per-formance and diagnostic accuracy of a new test [11] Ahighly accurate test does not necessarily mean that thetest will improve patient health as factors relating todecision-making and the effectiveness of patient man-agement strategies could fall short [64]Assessing the clinical utility of a new test is ex-

tremely challenging Measuring the impact of a teston patient health outcomes is difficult as tests tend toguide patient management decisions rather than dir-ectly impacting on patient health outcomes [5]Therefore estimating the clinical utility of a test re-quires evidence of how the information from a test isincorporated into decision-making and the down-stream effectiveness of those decisions [65] In thecase of a new test this is particularly complicatedgiven the uncertainty around the mechanisms bywhich the test will impact on patient outcomes [65]

Focus on price rather than cost-effectivenessAlthough the minimum and optimal price of the testsfeatured in many of the TPPs none of these was drivenby the trade-off between the overall cost implications ofimplementing the test and the associated patient bene-fits Cost-effectiveness analysis provides a framework tocompare costs and benefits of an intervention againstrelevant comparators including current practice Spe-cifically cost-effectiveness analysis defines whether theintervention being evaluated represents good value formoneyIt is important to consider the cost of the new test in

the context of the benefits that the test may provide Forexample a new test may be relatively expensive but mayalso improve patient health to the extent that the add-itional is justified Conversely a new test may be rela-tively cheap but offer no improvements in patient healthand therefore even the marginal increase in cost is notjustifiedConducting cost-effectiveness analysis at early RampD

stages of new tests can therefore help manufacturers toavoid significant investments in tests that do not havethe potential to be cost-effective [65]These first two limitations are particularly relevant

since decision-makers increasingly demand evidencethat a new test improves patient health and is cost-effective rather than solely evidence of its analyticaland clinical validity [5] Specifically many HealthTechnology Assessment bodies in Europe Australiaand North America consider clinical utility cost andcost-effectiveness in relation to the target populationin addition to analytical performance and clinicalvalidity when assessing new molecular diagnostictests [66]

Cocco et al BMC Medicine (2020) 18119 Page 8 of 12

Subjectivity of input sourcesExpert judgement and evidence identified in publishedliterature were the main sources of information for de-fining desirable characteristics Systematic reviews of theliterature where database searches are reproducible andthe quality of relevant studies are appraised were notconducted to identify relevant evidence at the scopingand drafting phase This is likely to introduce bias andsubjectivity in terms of the evidence used to underpintest characteristic recommendationsAlthough expert judgement is undoubtedly useful

relying solely upon this information source has somelimitations particularly for quantitative estimates Howhumans make probability judgements is highly affectedby many heuristics and systematic biases (eg anchoringavailability overconfidence and insight bias) [67] Specif-ically previous literature has found a poor understand-ing of test accuracy among healthcare professionals [68]as sensitivity and specificity are often misinterpreted andmistaken for predictive values [68]Additionally the quality of expert elicitation heavily

relies on expert selection as it is important to choose ex-perts with good subject knowledge Only 4 TPPs de-scribed how the selection process took place andtherefore the quality of expert judgements might bequestioned Furthermore many TPPs reported literatureas a source for informing TPPs however less than halfof the TPPs cited the references considered This lack oftransparency might hinder the quality and credibility ofsources on which TPPs are based

Study limitationsSince this study is a systematic review of publicly avail-able literature a key study limitation is that we have in-evitably missed any confidential or unpublished TPPsdeveloped in-house by test manufacturers Although theresults of our online searches did not identify any com-panies stating that they have developed TPPs for medicaltests we would not expect to find such information oncompany websites Anecdotally however we have notencountered any formal TPP development activity (bythis we mean definition of desirable test characteristics)within the National Institute for Health Research LeedsIn Vitro Diagnostics Co-operative (NIHR Leeds) MICindustry networkAs there are no guidelines on how TPPs for medical

tests should be developed we did not formally assessrisk of bias We did however appraise the transparencywith which the methodology underpinning each TPPhad been reported Unfortunately it was not possible tofully evaluate the TPP developed by PATH [40] as theonline appendices were not accessible Additionally dueto poor methodological transparency it was difficult toassess with certainty authorship of TPPs and whether

authors of TPPs took part themselves in the consensus-building meetings

Future researchAlthough there is evidence of a common developmentframework this review highlights that there is consider-able variability in the methods employed to draft TPPsand inconsistencies in which test characteristics are de-scribed A key issue in reviewing the methods imple-mented was the lack of transparency in methodologyreportingGuidance on best practice methods for developing

TPPs for medical tests would be highly beneficial Simi-larly to the US FDA guidance on TPPs for drugs a guid-ance document could be developed for TPPs for medicaltests summarising the purpose attributes of TPPs andwhich test characteristics should be includedHowever to inform the development of such guidance

future research should focus firstly on how to systemat-ically identify unmet clinical needs underpinning a cer-tain disease area Monaghan et al [69] developed avaluable checklist for identifying biomarkers based onliterature findings and consultations with experts Webelieve that this checklist could be pertinent for thescoping phase underlying TPP development howeverthis would need further validation in this specificcontextMore research is also required to understand how to

better incorporate the assessment of desirable clinicalutility and cost-effectiveness of innovative tests intoTPPs One possible way forward could be exploring howand if care pathway analysis and early economic model-ling could be integrated into the development of TPPsCare pathway analysis would provide clarity on themechanisms by which a test could impact on down-stream patient outcomes Early economic modellingcould be used to define desirable values for certain testcharacteristics (eg test price diagnostic sensitivity andspecificity) based on cost-effectiveness [67] Thereforeintegrating care pathway analysis and early economicmodelling into TPP development might provide moreevidence-based information to the test developersOutside of the actual methodology for developing

TPPs it would be useful to better understand whethermanufacturers develop tests strictly in line with TPPs orwhether there are any factors which make this infeasibleor challenging Additionally we would be interested inwhich methods manufacturers usually adopt to developTPPs to assess if there are any differences with themethodological framework we highlighted here To thisend interviewing test manufacturers might provide in-teresting insights on the intrinsic value of TPPs for theindustry

Cocco et al BMC Medicine (2020) 18119 Page 9 of 12

Most importantly to ensure that the development ofTPPs becomes widespread practice it would be valuableto explore how TPPs could be integrated into existingregulatory paths for innovation such as the EuropeanUnion Regulation for In Vitro Diagnostics (Regulation2017746) and the US FDA Drug Development and Ap-proval Process It might then be possible to align testcharacteristics featured in TPPs with evidence require-ments which are relevant for market approval decisionsof new medical tests This in return might increase theapplicability of TPPs for the industry

ConclusionsBased on this review we summarised current methodo-logical practice into a framework of value to those inter-ested in developing TPPs for medical testsWe also identified some key weaknesses including the

quality of the information sources underpinning TPPsand failure to consider test characteristics relating toclinical utility and cost-effectivenessThis review thus provides some recommendations for

further methodological research on the development ofTPPs for medical test This work will also help to informthe development of a formal guideline on how to draftTPPs for medical tests

Supplementary informationSupplementary information accompanies this paper at httpsdoiorg101186s12916-020-01582-1

Additional file 1 Search strategies

Additional file 2 Test characteristics

Additional file 3 Results

AbbreviationsACCE Analytic validity Clinical validity Clinical Utility Ethical legal and socialimplications AIDS Acquired immune deficiency syndrome CEA Cost-effectiveness analysis FDA Food and Drug Administration FIND TheFoundation For Innovative New Diagnostics HIV Human immunodeficiencyvirus IDC International Diagnostics Center NIHR National Institute for HealthResearch PRISMA Preferred Reporting Items for Systematic Reviews andMeta-Analysis QTPP Quality Target Product Profile RampD Research andDevelopment TPP Target Product Profile UNICEF United NationsInternational Childrenrsquos Emergency Fund WHO World Health Organization

AcknowledgementsThe authors would like to express their gratitude to Natalie King (Universityof Leeds Information Specialist) for supporting the design of the searchstrategy for this systematic review

Authorsrsquo contributionsAll authors contributed to the conception and design of the work PCperformed the search strategy BS contributed to reference sifting whilst AASconducted the data extraction MM and RW solved any conflicts PC BS MMand RW interpreted the data and drafted and revised the manuscript Theauthors read and approved the final manuscript

FundingThis systematic review was carried out as a part of the full-time School ofMedicine PhD Scholarship awarded to Paola Cocco by the University ofLeeds Additionally this work is supported by the lsquoTackling AMR-A Cross

Council Initiativersquo (grant number MRN0299761) Funding Partners The Bio-technology and Biological Sciences Research Council the Engineering andPhysical Sciences Research Council and the Medical Research Council Thiswork is also supported by the Medical Research Foundations National AMRTraining Programme Bethany Shinkins and Michael Messenger are fundedby NIHR Leeds In Vitro Diagnostic Co-operative and Cancer Research UK viaCanTest Collaborative Anam Ayaz-Shah is funded by Cancer Research UK viaCanTest Collaborative

Availability of data and materialsThe datasets generated and analysed during the current study are availablein the University of Leeds repository httpsdoiorg105518781 Thesedatasets entail an Excel spreadsheet with data extraction and an overview oftest characteristics included in reviewed TPPs

Ethics approval and consent to participateNot applicable

Consent for publicationNot applicable

Competing interestsThe authors declare that they have no competing interests

Author details1Test Evaluation Group Academic Unit of Health Economics Leeds Institutefor Health Sciences University of Leeds Leeds UK 2Academic Unit ofPrimary Care Leeds Institute for Health Sciences University of Leeds LeedsUK 3httpswwwcantestorg 4Centre for Personalised Health and MedicineUniversity of Leeds Leeds UK 5NIHR Leeds In Vitro Diagnostic (IVD)Co-operative Leeds UK 6Leeds Institute for Health Sciences University ofLeeds Leeds UK

Received 4 December 2019 Accepted 1 April 2020

References1 Wang P Kricka L Current and emerging trends in point-of-care technology

and strategies for clinical validation and implementation Clin Chem 2018httpsdoiorg101373clinchem2018287052

2 Drucker E Krapfenbauer K Pitfalls and limitations in translation frombiomarker discovery to clinical utility in predictive and personalisedmedicine EPMA Journal 20134(1)4ndash7

3 Kern S Why your new cancer biomarker may never work recurrent patternsand remarkable diversity in biomarker failures Cancer Res 2012 httpsdoiorg1011580008-5472CAN-12-3232

4 Engel N Wachter K Pai M Gallarda J Boehme C Celentano I et alAddressing the challenges of diagnostics demand and supply insights froman online global health discussion platform BMJ Glob Health 2016 httpsdoiorg101136bmjgh-2016-000132

5 Horvath A Lord S St John A Sandberg S Cobbaert C Lorenz S et al Frombiomarkers to medical tests the changing landscape of test evaluation ClinChim Acta 2014 httpsdoiorg101016jcca201309018

6 US Food and Drug Administration Guidance for industry Q8 (2)pharmaceutical development 2009 httpacademygmp-complianceorgguidemgrfiles9041FNL5b15dPDF Accessed 6 Nov 2018

7 US Food and Drug Administration Guidance for industry and review staffTarget Product Profile mdash a strategic development process tool 2007httpswwwfdagovmedia72566download Accessed 6 Nov 2018

8 WHO FIND MsF A multiplex multi-analyte diagnostic platform 2017httpswwwwhointmedicinesTPP_intro_20171122_forDistributionpdfAccessed 8 Nov 2018

9 PATH Target Product Profile HIV self-test version 41 a white paper on theevaluation of current HIV rapid tests and development of core specificationsfor next-generation HIV tests 2014 httpspathazureedgenetmediadocumentsTS_hiv_self_test_tpppdf Accessed 8 Nov 2018

10 Ebels K Clerk C Crudder C McGray S Magnuson K Tietje K et alIncorporating user needs into product development for improved infectiondetection for malaria elimination programs IEEE 2014 Global HumanitarianTechnology Conference 2014 httpsdoiorg101109GHTC20146970338

Cocco et al BMC Medicine (2020) 18119 Page 10 of 12

There was generally a lack of transparency andconsistency in reporting the methods underlying TPPsThis would make it difficult to appraise the recommen-dations within the TPPs ascertain whether the recom-mendations are generalizable to other settings andchallenging to reproduce

Relevancy of TPPs for test manufacturersThe purpose of a TPP is to identify upfront the essen-tial characteristics of a test for it to fulfil a pre-specified unmet clinical need This should in turn in-crease the likelihood that the test will be adopted intoclinical practice and reimbursed [1] A TPP should alsoaccount for contextual aspects that might affect thetestrsquos real-world performance [10] defining infrastruc-tural and technical constraints that impact on the im-plementation into clinical practice This review showsthat TPPs to date have primarily been developed forglobal health applications as the main funding organi-sations are WHO UNICEF and Bill and Melinda GatesFoundation The primary focus on infectious diseasesmay be explained by the remit of the global organisa-tions who fund TPPs WHO included HIVAIDSneglected tropical diseases tuberculosis and malaria aspriority diseases [62] Further to this WHO establishedthe lsquoRampD Blueprintrsquo which aims to promote RampD activ-ities (tests vaccines medicines) during epidemics [63]After having identified pathogen to target first TPPsare usually commissioned to guide the developmentprocess of new healthcare products which will addressthe high-priority pathogen [63]However the development of TPPs should not be limited

to one specific disease area or clinical setting the concept oflsquobeginning with an end in mindrsquo embodied by TPPs couldsupport both international and national health decision-makers This activity should in turn stimulate innovation ofnew tests driven by clinical needs rather than solely by la-boratory discoveries It would also provide manufacturerswith greater clarity around test requirements and confidencein the market for developing innovative tests

Identified limitations in current TPP methodologyIn reviewing current methodology for developing TPPsfor medical tests we have identified three key areaswhere current TPP methodology could be improved (1)oversight of clinical utility (2) a focus on price ratherthan cost-effectiveness and (3) subjectivity of informa-tion sources Here we discuss each limitation and theimplications

Oversight of clinical utilityVery few of the TPPs reported desirable characteristicsrelating to the clinical utility of the test This is not sur-prising given that the majority of research efforts has

focused on generating evidence on the analytical per-formance and diagnostic accuracy of a new test [11] Ahighly accurate test does not necessarily mean that thetest will improve patient health as factors relating todecision-making and the effectiveness of patient man-agement strategies could fall short [64]Assessing the clinical utility of a new test is ex-

tremely challenging Measuring the impact of a teston patient health outcomes is difficult as tests tend toguide patient management decisions rather than dir-ectly impacting on patient health outcomes [5]Therefore estimating the clinical utility of a test re-quires evidence of how the information from a test isincorporated into decision-making and the down-stream effectiveness of those decisions [65] In thecase of a new test this is particularly complicatedgiven the uncertainty around the mechanisms bywhich the test will impact on patient outcomes [65]

Focus on price rather than cost-effectivenessAlthough the minimum and optimal price of the testsfeatured in many of the TPPs none of these was drivenby the trade-off between the overall cost implications ofimplementing the test and the associated patient bene-fits Cost-effectiveness analysis provides a framework tocompare costs and benefits of an intervention againstrelevant comparators including current practice Spe-cifically cost-effectiveness analysis defines whether theintervention being evaluated represents good value formoneyIt is important to consider the cost of the new test in

the context of the benefits that the test may provide Forexample a new test may be relatively expensive but mayalso improve patient health to the extent that the add-itional is justified Conversely a new test may be rela-tively cheap but offer no improvements in patient healthand therefore even the marginal increase in cost is notjustifiedConducting cost-effectiveness analysis at early RampD

stages of new tests can therefore help manufacturers toavoid significant investments in tests that do not havethe potential to be cost-effective [65]These first two limitations are particularly relevant

since decision-makers increasingly demand evidencethat a new test improves patient health and is cost-effective rather than solely evidence of its analyticaland clinical validity [5] Specifically many HealthTechnology Assessment bodies in Europe Australiaand North America consider clinical utility cost andcost-effectiveness in relation to the target populationin addition to analytical performance and clinicalvalidity when assessing new molecular diagnostictests [66]

Cocco et al BMC Medicine (2020) 18119 Page 8 of 12

Subjectivity of input sourcesExpert judgement and evidence identified in publishedliterature were the main sources of information for de-fining desirable characteristics Systematic reviews of theliterature where database searches are reproducible andthe quality of relevant studies are appraised were notconducted to identify relevant evidence at the scopingand drafting phase This is likely to introduce bias andsubjectivity in terms of the evidence used to underpintest characteristic recommendationsAlthough expert judgement is undoubtedly useful

relying solely upon this information source has somelimitations particularly for quantitative estimates Howhumans make probability judgements is highly affectedby many heuristics and systematic biases (eg anchoringavailability overconfidence and insight bias) [67] Specif-ically previous literature has found a poor understand-ing of test accuracy among healthcare professionals [68]as sensitivity and specificity are often misinterpreted andmistaken for predictive values [68]Additionally the quality of expert elicitation heavily

relies on expert selection as it is important to choose ex-perts with good subject knowledge Only 4 TPPs de-scribed how the selection process took place andtherefore the quality of expert judgements might bequestioned Furthermore many TPPs reported literatureas a source for informing TPPs however less than halfof the TPPs cited the references considered This lack oftransparency might hinder the quality and credibility ofsources on which TPPs are based

Study limitationsSince this study is a systematic review of publicly avail-able literature a key study limitation is that we have in-evitably missed any confidential or unpublished TPPsdeveloped in-house by test manufacturers Although theresults of our online searches did not identify any com-panies stating that they have developed TPPs for medicaltests we would not expect to find such information oncompany websites Anecdotally however we have notencountered any formal TPP development activity (bythis we mean definition of desirable test characteristics)within the National Institute for Health Research LeedsIn Vitro Diagnostics Co-operative (NIHR Leeds) MICindustry networkAs there are no guidelines on how TPPs for medical

tests should be developed we did not formally assessrisk of bias We did however appraise the transparencywith which the methodology underpinning each TPPhad been reported Unfortunately it was not possible tofully evaluate the TPP developed by PATH [40] as theonline appendices were not accessible Additionally dueto poor methodological transparency it was difficult toassess with certainty authorship of TPPs and whether

authors of TPPs took part themselves in the consensus-building meetings

Future researchAlthough there is evidence of a common developmentframework this review highlights that there is consider-able variability in the methods employed to draft TPPsand inconsistencies in which test characteristics are de-scribed A key issue in reviewing the methods imple-mented was the lack of transparency in methodologyreportingGuidance on best practice methods for developing

TPPs for medical tests would be highly beneficial Simi-larly to the US FDA guidance on TPPs for drugs a guid-ance document could be developed for TPPs for medicaltests summarising the purpose attributes of TPPs andwhich test characteristics should be includedHowever to inform the development of such guidance

future research should focus firstly on how to systemat-ically identify unmet clinical needs underpinning a cer-tain disease area Monaghan et al [69] developed avaluable checklist for identifying biomarkers based onliterature findings and consultations with experts Webelieve that this checklist could be pertinent for thescoping phase underlying TPP development howeverthis would need further validation in this specificcontextMore research is also required to understand how to

better incorporate the assessment of desirable clinicalutility and cost-effectiveness of innovative tests intoTPPs One possible way forward could be exploring howand if care pathway analysis and early economic model-ling could be integrated into the development of TPPsCare pathway analysis would provide clarity on themechanisms by which a test could impact on down-stream patient outcomes Early economic modellingcould be used to define desirable values for certain testcharacteristics (eg test price diagnostic sensitivity andspecificity) based on cost-effectiveness [67] Thereforeintegrating care pathway analysis and early economicmodelling into TPP development might provide moreevidence-based information to the test developersOutside of the actual methodology for developing

TPPs it would be useful to better understand whethermanufacturers develop tests strictly in line with TPPs orwhether there are any factors which make this infeasibleor challenging Additionally we would be interested inwhich methods manufacturers usually adopt to developTPPs to assess if there are any differences with themethodological framework we highlighted here To thisend interviewing test manufacturers might provide in-teresting insights on the intrinsic value of TPPs for theindustry

Cocco et al BMC Medicine (2020) 18119 Page 9 of 12

Most importantly to ensure that the development ofTPPs becomes widespread practice it would be valuableto explore how TPPs could be integrated into existingregulatory paths for innovation such as the EuropeanUnion Regulation for In Vitro Diagnostics (Regulation2017746) and the US FDA Drug Development and Ap-proval Process It might then be possible to align testcharacteristics featured in TPPs with evidence require-ments which are relevant for market approval decisionsof new medical tests This in return might increase theapplicability of TPPs for the industry

ConclusionsBased on this review we summarised current methodo-logical practice into a framework of value to those inter-ested in developing TPPs for medical testsWe also identified some key weaknesses including the

quality of the information sources underpinning TPPsand failure to consider test characteristics relating toclinical utility and cost-effectivenessThis review thus provides some recommendations for

further methodological research on the development ofTPPs for medical test This work will also help to informthe development of a formal guideline on how to draftTPPs for medical tests

Supplementary informationSupplementary information accompanies this paper at httpsdoiorg101186s12916-020-01582-1

Additional file 1 Search strategies

Additional file 2 Test characteristics

Additional file 3 Results

AbbreviationsACCE Analytic validity Clinical validity Clinical Utility Ethical legal and socialimplications AIDS Acquired immune deficiency syndrome CEA Cost-effectiveness analysis FDA Food and Drug Administration FIND TheFoundation For Innovative New Diagnostics HIV Human immunodeficiencyvirus IDC International Diagnostics Center NIHR National Institute for HealthResearch PRISMA Preferred Reporting Items for Systematic Reviews andMeta-Analysis QTPP Quality Target Product Profile RampD Research andDevelopment TPP Target Product Profile UNICEF United NationsInternational Childrenrsquos Emergency Fund WHO World Health Organization

AcknowledgementsThe authors would like to express their gratitude to Natalie King (Universityof Leeds Information Specialist) for supporting the design of the searchstrategy for this systematic review

Authorsrsquo contributionsAll authors contributed to the conception and design of the work PCperformed the search strategy BS contributed to reference sifting whilst AASconducted the data extraction MM and RW solved any conflicts PC BS MMand RW interpreted the data and drafted and revised the manuscript Theauthors read and approved the final manuscript

FundingThis systematic review was carried out as a part of the full-time School ofMedicine PhD Scholarship awarded to Paola Cocco by the University ofLeeds Additionally this work is supported by the lsquoTackling AMR-A Cross

Council Initiativersquo (grant number MRN0299761) Funding Partners The Bio-technology and Biological Sciences Research Council the Engineering andPhysical Sciences Research Council and the Medical Research Council Thiswork is also supported by the Medical Research Foundations National AMRTraining Programme Bethany Shinkins and Michael Messenger are fundedby NIHR Leeds In Vitro Diagnostic Co-operative and Cancer Research UK viaCanTest Collaborative Anam Ayaz-Shah is funded by Cancer Research UK viaCanTest Collaborative

Availability of data and materialsThe datasets generated and analysed during the current study are availablein the University of Leeds repository httpsdoiorg105518781 Thesedatasets entail an Excel spreadsheet with data extraction and an overview oftest characteristics included in reviewed TPPs

Ethics approval and consent to participateNot applicable

Consent for publicationNot applicable

Competing interestsThe authors declare that they have no competing interests

Author details1Test Evaluation Group Academic Unit of Health Economics Leeds Institutefor Health Sciences University of Leeds Leeds UK 2Academic Unit ofPrimary Care Leeds Institute for Health Sciences University of Leeds LeedsUK 3httpswwwcantestorg 4Centre for Personalised Health and MedicineUniversity of Leeds Leeds UK 5NIHR Leeds In Vitro Diagnostic (IVD)Co-operative Leeds UK 6Leeds Institute for Health Sciences University ofLeeds Leeds UK

Received 4 December 2019 Accepted 1 April 2020

References1 Wang P Kricka L Current and emerging trends in point-of-care technology

and strategies for clinical validation and implementation Clin Chem 2018httpsdoiorg101373clinchem2018287052

2 Drucker E Krapfenbauer K Pitfalls and limitations in translation frombiomarker discovery to clinical utility in predictive and personalisedmedicine EPMA Journal 20134(1)4ndash7

3 Kern S Why your new cancer biomarker may never work recurrent patternsand remarkable diversity in biomarker failures Cancer Res 2012 httpsdoiorg1011580008-5472CAN-12-3232

4 Engel N Wachter K Pai M Gallarda J Boehme C Celentano I et alAddressing the challenges of diagnostics demand and supply insights froman online global health discussion platform BMJ Glob Health 2016 httpsdoiorg101136bmjgh-2016-000132

5 Horvath A Lord S St John A Sandberg S Cobbaert C Lorenz S et al Frombiomarkers to medical tests the changing landscape of test evaluation ClinChim Acta 2014 httpsdoiorg101016jcca201309018

6 US Food and Drug Administration Guidance for industry Q8 (2)pharmaceutical development 2009 httpacademygmp-complianceorgguidemgrfiles9041FNL5b15dPDF Accessed 6 Nov 2018

7 US Food and Drug Administration Guidance for industry and review staffTarget Product Profile mdash a strategic development process tool 2007httpswwwfdagovmedia72566download Accessed 6 Nov 2018

8 WHO FIND MsF A multiplex multi-analyte diagnostic platform 2017httpswwwwhointmedicinesTPP_intro_20171122_forDistributionpdfAccessed 8 Nov 2018

9 PATH Target Product Profile HIV self-test version 41 a white paper on theevaluation of current HIV rapid tests and development of core specificationsfor next-generation HIV tests 2014 httpspathazureedgenetmediadocumentsTS_hiv_self_test_tpppdf Accessed 8 Nov 2018

10 Ebels K Clerk C Crudder C McGray S Magnuson K Tietje K et alIncorporating user needs into product development for improved infectiondetection for malaria elimination programs IEEE 2014 Global HumanitarianTechnology Conference 2014 httpsdoiorg101109GHTC20146970338

Cocco et al BMC Medicine (2020) 18119 Page 10 of 12

Subjectivity of input sourcesExpert judgement and evidence identified in publishedliterature were the main sources of information for de-fining desirable characteristics Systematic reviews of theliterature where database searches are reproducible andthe quality of relevant studies are appraised were notconducted to identify relevant evidence at the scopingand drafting phase This is likely to introduce bias andsubjectivity in terms of the evidence used to underpintest characteristic recommendationsAlthough expert judgement is undoubtedly useful

relying solely upon this information source has somelimitations particularly for quantitative estimates Howhumans make probability judgements is highly affectedby many heuristics and systematic biases (eg anchoringavailability overconfidence and insight bias) [67] Specif-ically previous literature has found a poor understand-ing of test accuracy among healthcare professionals [68]as sensitivity and specificity are often misinterpreted andmistaken for predictive values [68]Additionally the quality of expert elicitation heavily

relies on expert selection as it is important to choose ex-perts with good subject knowledge Only 4 TPPs de-scribed how the selection process took place andtherefore the quality of expert judgements might bequestioned Furthermore many TPPs reported literatureas a source for informing TPPs however less than halfof the TPPs cited the references considered This lack oftransparency might hinder the quality and credibility ofsources on which TPPs are based

Study limitationsSince this study is a systematic review of publicly avail-able literature a key study limitation is that we have in-evitably missed any confidential or unpublished TPPsdeveloped in-house by test manufacturers Although theresults of our online searches did not identify any com-panies stating that they have developed TPPs for medicaltests we would not expect to find such information oncompany websites Anecdotally however we have notencountered any formal TPP development activity (bythis we mean definition of desirable test characteristics)within the National Institute for Health Research LeedsIn Vitro Diagnostics Co-operative (NIHR Leeds) MICindustry networkAs there are no guidelines on how TPPs for medical

tests should be developed we did not formally assessrisk of bias We did however appraise the transparencywith which the methodology underpinning each TPPhad been reported Unfortunately it was not possible tofully evaluate the TPP developed by PATH [40] as theonline appendices were not accessible Additionally dueto poor methodological transparency it was difficult toassess with certainty authorship of TPPs and whether

authors of TPPs took part themselves in the consensus-building meetings

Future researchAlthough there is evidence of a common developmentframework this review highlights that there is consider-able variability in the methods employed to draft TPPsand inconsistencies in which test characteristics are de-scribed A key issue in reviewing the methods imple-mented was the lack of transparency in methodologyreportingGuidance on best practice methods for developing

TPPs for medical tests would be highly beneficial Simi-larly to the US FDA guidance on TPPs for drugs a guid-ance document could be developed for TPPs for medicaltests summarising the purpose attributes of TPPs andwhich test characteristics should be includedHowever to inform the development of such guidance

future research should focus firstly on how to systemat-ically identify unmet clinical needs underpinning a cer-tain disease area Monaghan et al [69] developed avaluable checklist for identifying biomarkers based onliterature findings and consultations with experts Webelieve that this checklist could be pertinent for thescoping phase underlying TPP development howeverthis would need further validation in this specificcontextMore research is also required to understand how to

better incorporate the assessment of desirable clinicalutility and cost-effectiveness of innovative tests intoTPPs One possible way forward could be exploring howand if care pathway analysis and early economic model-ling could be integrated into the development of TPPsCare pathway analysis would provide clarity on themechanisms by which a test could impact on down-stream patient outcomes Early economic modellingcould be used to define desirable values for certain testcharacteristics (eg test price diagnostic sensitivity andspecificity) based on cost-effectiveness [67] Thereforeintegrating care pathway analysis and early economicmodelling into TPP development might provide moreevidence-based information to the test developersOutside of the actual methodology for developing

TPPs it would be useful to better understand whethermanufacturers develop tests strictly in line with TPPs orwhether there are any factors which make this infeasibleor challenging Additionally we would be interested inwhich methods manufacturers usually adopt to developTPPs to assess if there are any differences with themethodological framework we highlighted here To thisend interviewing test manufacturers might provide in-teresting insights on the intrinsic value of TPPs for theindustry

Cocco et al BMC Medicine (2020) 18119 Page 9 of 12

Most importantly to ensure that the development ofTPPs becomes widespread practice it would be valuableto explore how TPPs could be integrated into existingregulatory paths for innovation such as the EuropeanUnion Regulation for In Vitro Diagnostics (Regulation2017746) and the US FDA Drug Development and Ap-proval Process It might then be possible to align testcharacteristics featured in TPPs with evidence require-ments which are relevant for market approval decisionsof new medical tests This in return might increase theapplicability of TPPs for the industry

ConclusionsBased on this review we summarised current methodo-logical practice into a framework of value to those inter-ested in developing TPPs for medical testsWe also identified some key weaknesses including the

quality of the information sources underpinning TPPsand failure to consider test characteristics relating toclinical utility and cost-effectivenessThis review thus provides some recommendations for

further methodological research on the development ofTPPs for medical test This work will also help to informthe development of a formal guideline on how to draftTPPs for medical tests

Supplementary informationSupplementary information accompanies this paper at httpsdoiorg101186s12916-020-01582-1

Additional file 1 Search strategies

Additional file 2 Test characteristics

Additional file 3 Results

AbbreviationsACCE Analytic validity Clinical validity Clinical Utility Ethical legal and socialimplications AIDS Acquired immune deficiency syndrome CEA Cost-effectiveness analysis FDA Food and Drug Administration FIND TheFoundation For Innovative New Diagnostics HIV Human immunodeficiencyvirus IDC International Diagnostics Center NIHR National Institute for HealthResearch PRISMA Preferred Reporting Items for Systematic Reviews andMeta-Analysis QTPP Quality Target Product Profile RampD Research andDevelopment TPP Target Product Profile UNICEF United NationsInternational Childrenrsquos Emergency Fund WHO World Health Organization

AcknowledgementsThe authors would like to express their gratitude to Natalie King (Universityof Leeds Information Specialist) for supporting the design of the searchstrategy for this systematic review

Authorsrsquo contributionsAll authors contributed to the conception and design of the work PCperformed the search strategy BS contributed to reference sifting whilst AASconducted the data extraction MM and RW solved any conflicts PC BS MMand RW interpreted the data and drafted and revised the manuscript Theauthors read and approved the final manuscript

FundingThis systematic review was carried out as a part of the full-time School ofMedicine PhD Scholarship awarded to Paola Cocco by the University ofLeeds Additionally this work is supported by the lsquoTackling AMR-A Cross

Council Initiativersquo (grant number MRN0299761) Funding Partners The Bio-technology and Biological Sciences Research Council the Engineering andPhysical Sciences Research Council and the Medical Research Council Thiswork is also supported by the Medical Research Foundations National AMRTraining Programme Bethany Shinkins and Michael Messenger are fundedby NIHR Leeds In Vitro Diagnostic Co-operative and Cancer Research UK viaCanTest Collaborative Anam Ayaz-Shah is funded by Cancer Research UK viaCanTest Collaborative

Availability of data and materialsThe datasets generated and analysed during the current study are availablein the University of Leeds repository httpsdoiorg105518781 Thesedatasets entail an Excel spreadsheet with data extraction and an overview oftest characteristics included in reviewed TPPs

Ethics approval and consent to participateNot applicable

Consent for publicationNot applicable

Competing interestsThe authors declare that they have no competing interests

Author details1Test Evaluation Group Academic Unit of Health Economics Leeds Institutefor Health Sciences University of Leeds Leeds UK 2Academic Unit ofPrimary Care Leeds Institute for Health Sciences University of Leeds LeedsUK 3httpswwwcantestorg 4Centre for Personalised Health and MedicineUniversity of Leeds Leeds UK 5NIHR Leeds In Vitro Diagnostic (IVD)Co-operative Leeds UK 6Leeds Institute for Health Sciences University ofLeeds Leeds UK

Received 4 December 2019 Accepted 1 April 2020

References1 Wang P Kricka L Current and emerging trends in point-of-care technology

and strategies for clinical validation and implementation Clin Chem 2018httpsdoiorg101373clinchem2018287052

2 Drucker E Krapfenbauer K Pitfalls and limitations in translation frombiomarker discovery to clinical utility in predictive and personalisedmedicine EPMA Journal 20134(1)4ndash7

3 Kern S Why your new cancer biomarker may never work recurrent patternsand remarkable diversity in biomarker failures Cancer Res 2012 httpsdoiorg1011580008-5472CAN-12-3232

4 Engel N Wachter K Pai M Gallarda J Boehme C Celentano I et alAddressing the challenges of diagnostics demand and supply insights froman online global health discussion platform BMJ Glob Health 2016 httpsdoiorg101136bmjgh-2016-000132

5 Horvath A Lord S St John A Sandberg S Cobbaert C Lorenz S et al Frombiomarkers to medical tests the changing landscape of test evaluation ClinChim Acta 2014 httpsdoiorg101016jcca201309018

6 US Food and Drug Administration Guidance for industry Q8 (2)pharmaceutical development 2009 httpacademygmp-complianceorgguidemgrfiles9041FNL5b15dPDF Accessed 6 Nov 2018

7 US Food and Drug Administration Guidance for industry and review staffTarget Product Profile mdash a strategic development process tool 2007httpswwwfdagovmedia72566download Accessed 6 Nov 2018

8 WHO FIND MsF A multiplex multi-analyte diagnostic platform 2017httpswwwwhointmedicinesTPP_intro_20171122_forDistributionpdfAccessed 8 Nov 2018

9 PATH Target Product Profile HIV self-test version 41 a white paper on theevaluation of current HIV rapid tests and development of core specificationsfor next-generation HIV tests 2014 httpspathazureedgenetmediadocumentsTS_hiv_self_test_tpppdf Accessed 8 Nov 2018

10 Ebels K Clerk C Crudder C McGray S Magnuson K Tietje K et alIncorporating user needs into product development for improved infectiondetection for malaria elimination programs IEEE 2014 Global HumanitarianTechnology Conference 2014 httpsdoiorg101109GHTC20146970338

Cocco et al BMC Medicine (2020) 18119 Page 10 of 12

Most importantly to ensure that the development ofTPPs becomes widespread practice it would be valuableto explore how TPPs could be integrated into existingregulatory paths for innovation such as the EuropeanUnion Regulation for In Vitro Diagnostics (Regulation2017746) and the US FDA Drug Development and Ap-proval Process It might then be possible to align testcharacteristics featured in TPPs with evidence require-ments which are relevant for market approval decisionsof new medical tests This in return might increase theapplicability of TPPs for the industry

ConclusionsBased on this review we summarised current methodo-logical practice into a framework of value to those inter-ested in developing TPPs for medical testsWe also identified some key weaknesses including the

quality of the information sources underpinning TPPsand failure to consider test characteristics relating toclinical utility and cost-effectivenessThis review thus provides some recommendations for

further methodological research on the development ofTPPs for medical test This work will also help to informthe development of a formal guideline on how to draftTPPs for medical tests

Supplementary informationSupplementary information accompanies this paper at httpsdoiorg101186s12916-020-01582-1

Additional file 1 Search strategies

Additional file 2 Test characteristics

Additional file 3 Results

AbbreviationsACCE Analytic validity Clinical validity Clinical Utility Ethical legal and socialimplications AIDS Acquired immune deficiency syndrome CEA Cost-effectiveness analysis FDA Food and Drug Administration FIND TheFoundation For Innovative New Diagnostics HIV Human immunodeficiencyvirus IDC International Diagnostics Center NIHR National Institute for HealthResearch PRISMA Preferred Reporting Items for Systematic Reviews andMeta-Analysis QTPP Quality Target Product Profile RampD Research andDevelopment TPP Target Product Profile UNICEF United NationsInternational Childrenrsquos Emergency Fund WHO World Health Organization

AcknowledgementsThe authors would like to express their gratitude to Natalie King (Universityof Leeds Information Specialist) for supporting the design of the searchstrategy for this systematic review

Authorsrsquo contributionsAll authors contributed to the conception and design of the work PCperformed the search strategy BS contributed to reference sifting whilst AASconducted the data extraction MM and RW solved any conflicts PC BS MMand RW interpreted the data and drafted and revised the manuscript Theauthors read and approved the final manuscript

FundingThis systematic review was carried out as a part of the full-time School ofMedicine PhD Scholarship awarded to Paola Cocco by the University ofLeeds Additionally this work is supported by the lsquoTackling AMR-A Cross

Council Initiativersquo (grant number MRN0299761) Funding Partners The Bio-technology and Biological Sciences Research Council the Engineering andPhysical Sciences Research Council and the Medical Research Council Thiswork is also supported by the Medical Research Foundations National AMRTraining Programme Bethany Shinkins and Michael Messenger are fundedby NIHR Leeds In Vitro Diagnostic Co-operative and Cancer Research UK viaCanTest Collaborative Anam Ayaz-Shah is funded by Cancer Research UK viaCanTest Collaborative

Availability of data and materialsThe datasets generated and analysed during the current study are availablein the University of Leeds repository httpsdoiorg105518781 Thesedatasets entail an Excel spreadsheet with data extraction and an overview oftest characteristics included in reviewed TPPs

Ethics approval and consent to participateNot applicable

Consent for publicationNot applicable

Competing interestsThe authors declare that they have no competing interests

Author details1Test Evaluation Group Academic Unit of Health Economics Leeds Institutefor Health Sciences University of Leeds Leeds UK 2Academic Unit ofPrimary Care Leeds Institute for Health Sciences University of Leeds LeedsUK 3httpswwwcantestorg 4Centre for Personalised Health and MedicineUniversity of Leeds Leeds UK 5NIHR Leeds In Vitro Diagnostic (IVD)Co-operative Leeds UK 6Leeds Institute for Health Sciences University ofLeeds Leeds UK

Received 4 December 2019 Accepted 1 April 2020

References1 Wang P Kricka L Current and emerging trends in point-of-care technology

and strategies for clinical validation and implementation Clin Chem 2018httpsdoiorg101373clinchem2018287052

2 Drucker E Krapfenbauer K Pitfalls and limitations in translation frombiomarker discovery to clinical utility in predictive and personalisedmedicine EPMA Journal 20134(1)4ndash7

3 Kern S Why your new cancer biomarker may never work recurrent patternsand remarkable diversity in biomarker failures Cancer Res 2012 httpsdoiorg1011580008-5472CAN-12-3232

4 Engel N Wachter K Pai M Gallarda J Boehme C Celentano I et alAddressing the challenges of diagnostics demand and supply insights froman online global health discussion platform BMJ Glob Health 2016 httpsdoiorg101136bmjgh-2016-000132

5 Horvath A Lord S St John A Sandberg S Cobbaert C Lorenz S et al Frombiomarkers to medical tests the changing landscape of test evaluation ClinChim Acta 2014 httpsdoiorg101016jcca201309018

6 US Food and Drug Administration Guidance for industry Q8 (2)pharmaceutical development 2009 httpacademygmp-complianceorgguidemgrfiles9041FNL5b15dPDF Accessed 6 Nov 2018

7 US Food and Drug Administration Guidance for industry and review staffTarget Product Profile mdash a strategic development process tool 2007httpswwwfdagovmedia72566download Accessed 6 Nov 2018

8 WHO FIND MsF A multiplex multi-analyte diagnostic platform 2017httpswwwwhointmedicinesTPP_intro_20171122_forDistributionpdfAccessed 8 Nov 2018

9 PATH Target Product Profile HIV self-test version 41 a white paper on theevaluation of current HIV rapid tests and development of core specificationsfor next-generation HIV tests 2014 httpspathazureedgenetmediadocumentsTS_hiv_self_test_tpppdf Accessed 8 Nov 2018

10 Ebels K Clerk C Crudder C McGray S Magnuson K Tietje K et alIncorporating user needs into product development for improved infectiondetection for malaria elimination programs IEEE 2014 Global HumanitarianTechnology Conference 2014 httpsdoiorg101109GHTC20146970338

Cocco et al BMC Medicine (2020) 18119 Page 10 of 12

11 Verbakel J Turner P Thompson M Pluumlddemann A Price C Shinkins B et alCommon evidence gaps in point-of-care diagnostic test evaluation areview of horizon scan reports BMJ Open 2017 httpsdoiorg101136bmjopen-2016-015760

12 Cocco P Ayaz-Shah A Shinkins B Messenger M West R Methods adoptedto develop a Target Product Profile (TPP) in the field of medical tests asystematic review 2018 httpswwwcrdyorkacukprosperodisplay_recordphpID=CRD42018115133 Accessed 17 Jan 2019

13 Godin K Stapleton J Kirkpatrick SI Hanning RM Leatherdale ST Applyingsystematic review search methods to the grey literature a case studyexamining guidelines for school-based breakfast programs in Canada SystRev 2015 httpsdoiorg101186s13643-015-0125-0

14 Burke W Zimmern R Moving beyond ACCE an expanded framework forgenetic test evaluation 2007 httpwwwphgfoundationorgdocuments369_1409657043pdf 18 Accessed Feb 2019

15 EUR-Lex Regulation (EU) 2017746 of the European Parliament and of theCouncil of 5 April 2017 on in vitro diagnostic medical devices and repealingDirective 9879EC and Commission Decision 2010227EU (Text with EEArelevance) 2017 httpseur-lexeuropaeulegal-contentENTXTPDFuri=CELEX32017R0746ampfrom=EN Accessed 18 Feb 2019

16 US Department of Health and Human Services US Food and DrugAdministration Center for Drug Evaluation and Research Center forVeterinary Medicine Bioanalytical method validation guidance for industry2018 httpswwwfdagovmedia70858download Accessed 18 Feb 2019

17 Bossuyt P Reitsma J Linnet K Moons K Beyond diagnostic accuracy theclinical utility of diagnostic tests Clin Chem 2012 httpsdoiorg101373clinchem2012182576

18 Association for the Advancement of Medical Instrumentation Humanfactors engineering- design of medical devices 2013 httpsmyaamiorgaamiresourcespreviewfileshe75_1311_previewpdf Accessed 18 Feb 2019

19 Organization IS Terms and definitions in ISO 14001 2015 - where did theyoriginate from 2015 httpscommitteeisoorgfileslivesitestc207sc1filesTerms20and20definitions20in20ISO2014001_201520-20where20did20they20originate20frompdf Accessed 18 Feb 2019

20 Abuhav I ISO 134852016 a complete guide to quality management in themedical device industry editors 2018

21 Chua A Prat I Nuebling C Wood D Moussy F Update on Zika diagnostictests and WHOrsquos related activities PLoS Negl Trop Dis 2017 httpsdoiorg101371journalpntd0005269

22 Denkinger C Dolinger D Schito M Wells W Cobelens F Pai M et al Targetproduct profile of a molecular drug-susceptibility test for use in microscopycenters J Infect Dis 2015 httpsdoiorg101093infdisjiu682

23 Denkinger C Kik S Cirillo D Casenghi M Shinnick T Weyer K et al Definingthe needs for next generation assays for tuberculosis J Infect Dis 2015211S29ndash38

24 DIAMETER Project PATH Target Product Profile point-of-care malariainfection detection test for rapid detection of low-density subclinicalmalaria infections 2014 httpspathazureedgenetmediadocumentsHighly_Sensitive_HRP2_RDT_TPP_Nov15pdf Accessed 8 Nov 2018

25 Ding X Ade M Baird J Cheng Q Cunningham J Dhorda M et al Definingthe next generation of Plasmodium vivax diagnostic tests for control andelimination target product profiles PLoS Negl Trop Dis 2017 httpsdoiorg101371journalpntd0005516

26 Dittrich S Tadesse B Moussy F Chua A Zorzet A Tangden T et al TargetProduct Profile for a diagnostic assay to differentiate between bacterial andnon-bacterial infections to guide antimicrobials use in resource-limitedsettings an expert consensus Am J Trop Med Hyg 2017 httpsdoiorg101371journalpone0161721

27 Donadeu M Fahrion A Olliaro P Abela-Ridder B Target product profiles for thediagnosis of Taenia solium taeniasis neurocysticercosis and porcine cysticercosisPLoS Negl Trop Dis 2017 httpsdoiorg101371journalpntd0005875

28 FIND Forum for Collaborative HIV Research High-priority target productprofile for hepatitis C diagnosis in decentralized settings report of aconsensus meeting 2015 httpswwwfinddxorgwp-contentuploads201903HCV-TPP-Report_FIND-2015pdf Accessed 8 Nov 2018

29 FIND Target Product Profile for tests for recent HIV infection 2017 httpswwwfinddxorgwp-contentuploads201903HIV-Incidence-TPP-FIND-2017pdf Accessed 8 Nov 2018

30 FIND Target Product Profile rapid test for diagnosis of malaria andscreening for hyman African trypanosomiasis (HAT) 2017 httpswww

finddxorgwp-contentuploads201903TPP-5-HAT-malaria-combo-test_02FEB17pdf Accessed 8 Nov 2018

31 FIND Target Product Profile screening test for human Africantrypanosomiasis (HAT) 2017 httpswwwfinddxorgwp-contentuploads201903TPP-1-HAT-screening-test_02FEB17pdf Accessed 8 Nov 2018

32 Gal M Francis N Hood K Villacian J Goossens H Watkins A et al Matchingdiagnostics development to clinical need Target Product Profiledevelopment for a point of care test for community-acquired lowerrespiratory tract infection PLoS One 2018 httpsdoiorg101371journalpone0200531

33 International Diagnostics Centre Target Product Profile point-of-care HIVviral load test 2013 httpsidc-dxorgresourcetarget-product-profile-tpp-point-of-care-hiv-viral-load-test-2013 Accessed 8 Nov 2018

34 International Diagnostics Centre Target Product Profile (TPP) ndash point-of-careCD4 test 2013 httpsidc-dxorgresourcetarget-product-profile-tpp-point-of-care-cd4-test-2013 Accessed 8 Nov 2018

35 International Diagnostics Centre Target Product Profile (TPP) ndash point-of-careEID test 2013 httpsidc-dxorgresourcetarget-product-profile-tpp-point-of-care-eid-test-2013 Accessed 8 Nov 2018

36 International Diagnostics Centre Target Product Profile (TPP) ndash combinedHIVsyphilis test 2014 httpsidc-dxorgresourcetarget-product-profile-tpp-combined-hiv-syphilis-test-2014 Accessed 8 Nov 2018

37 Lim M Brooker S Belizario VJ Gay-Andrieu F Gilleard J Levecke B et alDiagnostic tools for soil-transmitted helminths control and eliminationprograms a pathway for diagnostic product development PLoS Negl TropDis 2018 httpsdoiorg101371journalpntd0006213

38 Nsanzabana C Ariey F Beck H Ding X Kamau E Krishna S et al Molecularassays for antimalarial drug resistance surveillance a target product profilePLoS One 2018 httpsdoiorg101371journalpone0204347

39 Pal S Jasper L Lawrence K Walter M Gilliland T Dauner A et al Assessingthe dengue diagnosis capability gap in the military health system Mil Med2016 httpsdoiorg107205MILMED-D-15-00231

40 PATH Diagnostics for neglected tropical diseases defining the best toolsthrough target product profiles 2015 httpspathazureedgenetmediadocumentsDIAG_ntd_prod_profile_rptpdf Accessed 8 Nov 2018

41 PATH Target Product Profile schistosomiasis surveillance diagnostic 2015httpspathazureedgenetmediadocuments20150115_BMGF_SCH_MDAreduction_NoTechpdf Accessed 8 Nov 2018

42 PATH Target Product Profile schistosomiasis surveillance diagnostic -Schistosoma genus-specific antibody lateral flow test post-eliminationsurveillance 2015 httpspathazureedgenetmediadocuments20150115_BMGF_SCH_postMDA_Abpdf Accessed Nov 2018

43 PATH Target Product Profile schistosomiasis surveillance diagnostic lateralflow test CAA antigen 2015 httpspathazureedgenetmediadocuments20150115_BMGF_SCH_MDAreduction_Agpdf Accessed 8 Nov2018

44 PATH Target Product Profile trachoma surveillance diagnostic - antigenlateral flow test 2015 httpspathazureedgenetmediadocuments20150115_BMGF_Trachoma_MDAreduction_Agpdf Accessed 8 Nov 2018

45 PATH Target Product Profile trachoma surveillance diagnostic ndash post-elimination surveillance antibody lateral flow test 2015 httpspathazureedgenetmediadocuments20150115_BMGF_Trachoma_postMDA_Abpdf Accessed 8 Nov 2018

46 Peck R Wellhausen J Boettcher M Downing L Defining productrequirements for field deployable yellow fever tests 2012 httpwwwajtmhorgdocserverfulltext14761645895_Suppl_1ASTMH_13_Abstracts_1001_1250pdfexpires=1565877435ampid=idampaccname=guestampchecksum=5DBD842D2A8FA98A39884EB2A0713FA3 Full text accesspermission from the author Accessed 8 Nov 2018

47 Porras A Yadon Z Altcheh J Britto C Chaves G Flevaud L et al TargetProduct Profile (TPP) for Chagas disease point-of-care diagnosis andassessment of response to treatment PLoS Negl Trop Dis 2015 httpsdoiorg101371journalpntd0003697

48 Reipold E Easterbrook P Trianni A Panneer N Krakower D Ongarello Set al Optimising diagnosis of viraemic hepatitis C infection thedevelopment of a target product profile BMC Infect Dis 2017 httpsdoiorg101186s12879-017-2770-5

49 Solomon A Engels D Bailey R Blake I Brooker S Chen J et al A diagnosticsplatform for the integrated mapping monitoring and surveillance ofneglected tropical diseases rationale and target product profiles PLoS NeglTrop Dis 2012 httpsdoiorg101371journalpntd0001746

Cocco et al BMC Medicine (2020) 18119 Page 11 of 12

50 Toskin I Murtagh M Peeling R Blondeel K Cordero J Kiarie J Advancingprevention of sexually transmitted infections through point-of-care testingtarget product profiles and landscape analysis Sex Transm Infect 2017httpsdoiorg101136sextrans-2016-053071

51 UNICEF Pneumonia acute respiratory infection diagnostic aid - TargetProduct Profile introduction 2014 httpswwwuniceforgsupplymedia1356fileTarget20product20profile20-20Acute20Respiratory20Infection20Diagnostic20Aid20-20ARIDApdfAccessed 8 Nov 2018

52 UNICEF UNICEF Target Product Profile real time E coli detection (version 14) 2016 httpswwwuniceforgsupplydocumentstarget-product-profilerapid-e-coli-detection-tests Accessed 8 Nov 2018

53 UNICEF UNICEF Target Product Profile rapid E coli detection (version 20)2017 httpswwwuniceforgsupplydocumentstarget-product-profile-rapid-e-coli-detection-tests Accessed 8 Nov 2018

54 UNICEF Zika virus diagnostics Target Product Profiles amp supply update2017 httpswwwuniceforgsupplydocumentstarget-product-profiles-zika-diagnostic-tests Accessed 8 Nov 2018

55 Utzinger J Becker S van Lieshout L van Dam G Knopp S New diagnostictools in schistosomiasis Clin Microbiol Infect 2015 httpsdoiorg101016jcmi201503014

56 WHO Taenia solium taeniasiscysticercosis diagnostic tool Report of astakeholder meeting 2015 httpsappswhointirisbitstreamhandle106652065439789241510516_engpdfsequence=1ampisAllowed=y Accessed8 Nov 2018

57 WHO FIND Target Product Profile for Zaire ebolavirus rapid simple test tobe used in the control of the Ebola outbreak in West Africa 2014 httpswwwwhointmedicinespublicationstarget-product-profilepdfua=1Accessed 8 Nov 2018

58 WHO FIND Development of a Target Product Profile (TPP) and a frameworkfor evaluation for a test for predicting progression from tubercolosisinfection to active disease 2017 httpsappswhointirisbitstreamhandle10665259176WHO-HTM-TB-201718-engpdfjsessionid=BD8E426CA555DE9FEAB589FF2A9B2CCCsequence=1 Accessed 8 Nov 2018

59 WHO FIND Report of a WHO-FIND meeting on diagnostics for Buruli ulcer2018 httpsappswhointirisbitstreamhandle10665274607WHO-CDS-NTD-IDM-201809-engpdfsequence=1ampisAllowed=y Accessed 8 Nov 2018

60 WHO High-priority target product profiles for new tuberculosis diagnosticsreport of a consensus meeting 2014 httpsappswhointirisbitstreamhandle10665135617WHO_HTM_TB_201418_engpdfsequence=1Accessed 8 Nov 2018

61 WHO Specifications for a rapid diagnostic test for meningitis Africanmeningitis belt 2016 httpswwwwhointcsrdiseasemeningococcalTPP-Meningitis-RDT-Apr16pdfua=1 Accessed 8 Nov 2018

62 Kaplan W Wirtz V Mantel-Teeuwisse A Stolk P Duthey B Laing R Prioritymedicines for Europe and the world- 2013 update 2013 2013 httpswwwwhointmedicinesareaspriority_medicinesMasterDocJune28_FINAL_Webpdfua=1 Accessed 13 June 2019

63 WHO An RampD Blueprint for action to prevent epidemics 2016 httpswwwwhointblueprintaboutr_d_blueprint_plan_of_actionpdfua=1 Accessed13 Jun 2019

64 Linnet K Bossuyt P Moons K Reitsma J Quantifying the accuracy of adiagnostic test or marker Clin Chem 2012 httpsdoiorg101373clinchem2012182543

65 Abel L Shinkins B Smith A Sutton A Sagoo G Uchegbu I et al Earlyeconomic evaluation of diagnostic technologies experiences of the NIHRdiagnostic evidence co-operatives Med Decis Mak 201939(7)857ndash66

66 Garfield S Polisena J Spinner D Postulka A Lu C Tiwana S et al HealthTechnology Assessment for molecular diagnostics practices challenges andrecommendations from the Medical Devices and Diagnostics SpecialInterest Group Value Health 2016 httpsdoiorg101016jjval201602012

67 Tversky A Kahneman D Judgment under uncertainty heuristics and biasesScience 1974 httpsdoiorg101126science18541571124

68 Whiting P Davenport C Jameson C Burke M Sterne J Hyde C et al Howwell do health professionals interpret diagnostic information A systematicreview BMJ Open 2015 httpsdoiorg101136bmjopen-2015-008155

69 Monaghan P Lord S St John A Sandberg S Cobbaert C Lennartz L et alBiomarker development targeting unmet clinical needs Clin Chim Acta2016460211ndash9

Publisherrsquos NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations

Cocco et al BMC Medicine (2020) 18119 Page 12 of 12