managing risk in outsourced clinical trials

The State of Clinical Outsourcing Managing Risk in Outsourced Clinical Trials Interactive Discussion Based on Data from Avoca’s 2013 Industry Research DIA Forum June 24, 2013

2

Panel Members

Denise Calaprice-Whitty (Forum Chair) Senior Consultant, The Avoca Group Andy Lee Senior VP, Deputy Head of CSO Platform, Genzyme, a Sanofi Company Mitchell Katz Executive Director, Medical Research Operations, Purdue Pharma L.P. Alistair Macdonald Chief Operating Officer, INC Research

3

Today’s Discussion

Presentation of Data Highlights from Avoca 2013 Industry Survey, and Interactive Discussion

Avoca 2013 Industry Survey Topic: Management of Risk in Outsourced Clinical Trials

•  Part One: Risk Sharing Models •  Part Two: Risk Assessment •  Part Three: Risk Management

4

2013 Avoca Research: Questions Explored

•  Risk-sharing models: What kinds of models are most often used? Under what circumstances? What are the magnitudes of the incentives/penalties? Have they been successful? In what ways? What have been the downsides?

•  Risk assessments: How often are these conducted, and by whom? Are they just qualitative, or also quantitative? On what types of risks do they focus? Do stakeholders generally agree about risks? Have these approaches been successful? In what ways?

•  Risk-based management approaches: To what tasks have such approaches been applied? Have they been successful? In what ways? What have been the downsides? How has this work been viewed by regulators?

5

2013 Avoca Research Respondents

237 Respondents

•  Sponsors: 113 Respondents from 70 companies –  Approximately half in top 20 in terms of revenue

•  66% Pharma •  26% Biotech •  5% Device •  3% Combination/Other

•  Clinical Service Providers: 124 Respondents from 66 companies –  60% in top 20 in terms of revenue

•  86% CROs •  11% Niche •  3% Laboratories

6

Part One: Risk Sharing

7

Types of Risk-sharing Between Sponsors and Providers

•  Provider bonuses for achieving milestones or targets

•  Provider penalties for missing milestones or targets

•  Guarantees of continued work/revenue stream in exchange for provider commitments

•  Provider stake in outcome of study (e.g., company stock)

8

Prevalence of Risk Sharing: Sponsors

0% 5%

10% 15% 20% 25% 30% 35% 40% 45%

Stra

tegi

c pa

rtne

rshi

ps/

allia

nces

(N

=28)

Pref

erre

d pr

ovid

er

rela

tion

ship

s (N

=43)

Tras

acti

onal

re

lati

onsh

ips

(N=3

7)

Bonuses Penalties Guarantees of continued work/revenue Provider stake in outcome

For each of the types of outsourcing relationships listed, which of the risk-sharing models listed has your company used?

9

N=33

Magnitude of Bonuses and Penalties

3% 3%

27%

61%

6% >20% of contract value

11-20% of contract value



<2% of contract value

Sponsor Data: When bonuses are used, what is generally the magnitude of the

maximum possible bonus?

3% 3%

26%

61%

7% >20% of contract value





Sponsor Data: When penalties are used, what is

generally the magnitude of the maximum possible penalty?

N=31

10

Experience with Risk-sharing by Sponsors

54%

35%

14%

41%

56%

41%

6%

9%

45%

0% 20% 40% 60% 80% 100%

Guarantee of continued work/revenue stream in exchange for provider commitments

Provider bonuses for achievement of milestones/targets

Provider penalties for failure to achieve milestones/targets

Primarily positive A mix of positive and negative Primarily negative

N

54

57

58

To date, my experience with each of the below risk-sharing outsourcing models has been…

11

Experience with Risk-sharing by CROs

58%

46%

24%

40%

48%

47%

2%

7%

29%

0% 20% 40% 60% 80% 100%

Guarantee of continued work/revenue stream in exchange for provider commitments

Provider bonuses for achievement of milestones/targets

Provider penalties for failure to achieve milestones/targets

Primarily positive A mix of positive and negative Primarily negative

N

61

51

53

To date, my experience with each of the below risk-sharing outsourcing models has been…

12

Sponsor Data: All Respondents Overall, how satisfied are you with the work that has been done for you by Clinical

Service Providers (including but not limited to CROs)?

5%

50%

60%

71%

100%

50%

30%

24%

10%

0% 20% 40% 60% 80% 100%

>10% of contract value




Very satisfied Generally satisfied Neither satisfied nor dissatisfied Generally dissatisfied Very dissatisfied

N

2

10

21

2

Magnitude of the maximum possible penalty

13

Comments on Experiences

•  “These relationships require absolute clarity in roles and responsibilities and up-front planning assumptions that we have not always had, which has caused difficulties along the way.”

•  “The inclusion of provider bonuses and penalties has had a detrimental affect on study team morale. OK when the target has been reached outright, but lots of 'extra' negotiating when a negative outcome.”

14

Comments on Experiences

•  “In my opinion a CRO doesn't perform better only because they have a bonus or a penalty. The bonus is a way to show them appreciation at the end but never a motivational factor. Aligning with the CRO at the beginning on deliverables and sharing risks and pitfalls before signing the contract will increase the trust level and consequently the commitment.”

15

Comments on Experiences •  “CRO must have complete decision-making authority on the

operational aspects of trial execution (minimal handoffs/decisions by sponsor). CRO must be compensated for a thorough feasibility assessment and allowed to select the suppliers they believe will deliver (without sponsor imposing non-producing investigators in the mix). Also, sponsor company must be willing to provide both the penalty as well as the reward. In my experience, penalties were never applied because there were always mitigating factors (sponsor impacted operational success due to slow response or non-response to needs or due to sponsor mandate of key requirements- such as SOPs, KOLs, etc.).”

16

Part Two: Risk Assessment

17

39%

19%

26%

13% 3%

N=31

Prevalence of Systematic, Formal Risk Assessments in Outsourced Trials

38%

18%

16%

14%

14% For >75% of outsourced clinical trials For 51% - 75% of outsourced clinical trials For 25% - 50% of outsourced clinical trials For 1% - 24% of outsourced clinical trials Never

N=50

Sponsors CROs

Most sponsors and clinical service providers reported that a systematic risk assessment process was used for the majority of

their clinical trials.

18

Sponsor Use of Systematic Risk Assessment Processes: Is there a difference based on size of company?

50%

32%

13%

21%

19%

15%

13%

15%

6%

18%

0% 20% 40% 60% 80% 100%

Yes

No

For >75% of clinical trials For 51% - 75% of clinical trials For 25% - 50% of clinical trials

For 1% - 24% of clinical trials Never

N

16

34

Top 20?

19

N=63

CRO Involvement in Systematic Risk Assessment

32%

47%

16%

5%

Sponsor Data:

For outsourced clinical trials, to what extent is the CRO generally involved in

the systematic risk assessment?

Provider Data:

To what extent is your company generally involved in the systematic risk

assessment?

48%

39%

5% 8%

N=66

20

What Risks are Assessed?

0%

10% 20% 30% 40% 50% 60% 70% 80% 90%

Pati

ent

enro

llmen

t ri

sks

Vend

or p

erfo

rman

ce

risk

s

Dat

a qu

alit

y ri

sks

Oth

er t

imel

ine

risk

s

Site

com

plia

nce

risk

s

Cost

ris

ks

Clin

ical

tri

al s

ubje

ct

safe

ty r

isks

Dru

g/de

vice

sup

ply-

rela

ted

risk

s

Risk

s to

rig

hts

of

clin

ical

tri

al s

ubje

cts/

ethi

cs

Oth

er

Sponsor (N=70) Provider (N=75)

When a systematic risk assessment is performed for projects conducted by your company, which of the following risks are typically formally assessed?

21

N=68

Risk Assessment: Qualitative or Quantitative?

43%

9%

41%

7% Primarily qualitative

Primarily quantitative

Approximately equal mix across trials It depends

Is the risk assessment process primarily qualitative (e.g. risks identified and discussed), or quantitative (e.g. probabilities and consequences of specific risks

estimated quantitatively)?

32%

15%

53%

Primarily qualitative

Primarily quantitative

Approximately equal mix across trials

N=66

Sponsors CROs

22

Part Three: Risk Management

23

How often does your risk assessment process lead to the introduction or

refinement of proactive measures designed to reduce risk?

24

Reducing Risk: Sponsor Data

51%

50%

34%

34%

31%

29%

29%

28%

28%

24%

12%

15%

31%

28%

19%

20%

31%

21%

12%

22%

26%

25%

27%

34%

40%

39%

36%

32%

47%

41%

12%

10%

8%

4%

10%

11%

4%

19%

12%

13%

0% 20% 40% 60% 80% 100%

Review plan for performance data

Capture of performance data (i.e. not clinical trial data)

Changes in numbers of sites selected

Training additions/enhancements

Personnel additions/enhancements

Protocol amendments

Procedural additions/enhancements

Changes in locations of sites selected

Decisions regarding CROs/other vendors used

Addition or removal of specific sites

For >50% of clinical trials For 25% - 50% of clinical trials For 1%-24% of clinical trials Never

N 43

48

49

50

42

44

45

47

49

46

How often does your risk assessment process lead to the introduction or refinement of each of the following proactive measures designed to reduce risk?

25

Reducing Risk: Provider Data How often does your risk assessment process lead to the introduction or

refinement of each of the following proactive measures designed to reduce risk?

47%

45%

38%

34%

31%

23%

21%

18%

16%

14%

22%

19%

32%

36%

35%

33%

27%

33%

19%

13%

26%

30%

30%

23%

33%

35%

48%

43%

56%

50%

4%

6%

7%

2%

9%

5%

8%

9%

24%

0% 20% 40% 60% 80% 100%

Review plan for performance data

Capture of performance data (i.e. not clinical trial data)

Procedural additions/enhancements

Changes in numbers of sites selected

Training additions/enhancements

Changes in locations of sites selected

Addition or removal of specific sites

Personnel additions/enhancements

Protocol amendments

Decisions regarding CROs/other vendors used

For >50% of clinical trials For 25% - 50% of clinical trials For 1%-24% of clinical trials Never

N 46

47

44

44

46

43

44

40

43

38

26

N=50

Use of Risk-related Information to Determine Oversight of CROs

24%

22%

20%

14%

20% For >75% of clinical trials

For 51% - 75% of clinical trials



Never

Sponsor Data

27

Satisfaction with CROs

9% 73%

56%

9%

41%

9%

4%

0% 20% 40% 60% 80% 100%

For >50%

For <50%

Very satisfied Generally satisfied Neither satisfied nor dissatisfied Generally dissatisfied Very dissatisfied

How often do your teams use risk-related

information to determine the level

and/or type of sponsor oversight that you will

employ for your CRO partners?

N

23

27

by Use of Risk-related Information to Determine Oversight

28

Satisfaction with Risk Assessment and Management Performance

Scale of 1 to 5; 1=Very dissatisfied, 5=Very satisfied

29

Opportunities to Enhance Understanding

38%

13%

9%

23%

54%

36%

31%

23%

45%

8%

10%

9%

0% 20% 40% 60% 80% 100%

Executive Management

Middle Management

Project Management

Very strong understanding Good understanding Fair understanding Poor understanding

N

13

39

11

How would you rate your understanding of best practices in risk assessment and management in clinical trials? (Sponsor Data)

While executives generally reported a strong understanding, operational teams may benefit from more training.

30

Are risk-based approaches leading to greater efficiency and higher quality?

14%

32%

38%

34%

23%

6%

25%

28%

0% 20% 40% 60% 80% 100%

more efficient use of resources for your company and/or

your CRO partner?

increased quality?

Yes Sometimes No Don't know/too soon to tell

N

65

65

In general, have your risk assessment and management approaches resulted in…

31

Thank you!

www.theavocagroup.com

32

Definitions Included in Avoca Survey

Risk is defined as "the combination of the probability of occurrence of harm and the severity of that harm" (ISO/IEC Guide 51). RISK ASSESSMENT AND MANAGEMENT (taken from FDA Guidance for Industry Q9: Quality Risk Management). Risk assessment: A systematic process of organizing information to support a risk decision to be made within a risk management process. It consists of the identification of hazards and the analysis and evaluation of risks associated with exposure to those hazards. Risk management: The systematic application of quality management policies, procedures, and practices to the tasks of assessing, controlling, communicating, and reviewing risk.

managing risk in outsourced clinical trials

Health & Medicine