present clean, accurate, and applicable data to ...1).pdf · present clean, accurate, and...

Present Clean, Accurate, and

Applicable Data to Adjudication

Committees

Riaz Mahmood Matt WilsonClinical Trials Coordinator II Clinical Trials Project Leader III

All Rights Reserved, Duke Medicine 2007

Objectives

• Become familiar with case requirements for adjudication (eCRF, source

documents, site narrative)

• Identify the source document types and compulsory/essential source

documents

• Identify the preferred mechanism for source document (eCRF, secure

electronic formats, fax, mail, smartphones)

• Describe the source document collection and processing steps

• Review source documentation “redaction” steps and what constitutes

PHI/PII

• Discuss the advantages and challenges of EHR/EMR

• Future of source document collection

RM/MW


WHY ARE SOURCE

DOCUMENTS

IMPORTANT?

WHAT’S ALL THE

FUSS?

RM


Why Source Documents are Important

• Consistency in Endpoint Evaluation

• True treatment effect - Efficacy

• Quality of data and endpoint adjudications– Systematic, Comprehensive, Unbiased, Blinded, and

Independent

• Good science and part of the regulations

MW


Case Study: RECORD

• Earlier Clinical Trials Suggested

– Rosiglitazone used independently or in conjunction with anti-diabetic

therapy was associated with a higher risk for MI’s and CV deaths

• Conducted from 2001 to 2008

• Results - rosiglitazone was not noninferior to comparators (metformin or

a sulfonylurea) for CV death or CV hospitalization

• September 23, 2010 FDA permits the continued marketing of

rosiglitazone but requires a comprehensive reevaluation of three event

types: Death, Myocardial Infarction, and Stroke.

RM


Case Study: RECORD Re-Adjudication

• FDA Inspection Findings

• 3 sites enrolled 4% of the total enrollment (188 subjects)

– Croatia –57 Subjects

– Sweden –73 Subjects

– Germany –58 Subjects

• Did not always ensure

– Timely processing and adjudication of events (6)

• Reporting of potential endpoints (3)

• Adjudication of events (3)

– Submission of Documentation that was critical for

adjudication (4)

RM


Sample Finding from the FDA Inspection Report

• Subject hospitalized for GI bleed

• After 5 days experiences chest pain and dies

• Cause of death: Non-cardiovascular

• Query - Subject died from an MI

• At Sponsor

– Cardiac biomarkers and ECG not available

• At CRO

– Documentation of ECG and Hospital discharge summary

– Did not support MI

RM


DCRI Re-Adjudication of RECORD

0

20

40

60

80

100

120

140

160

180

200

CV Death, MI,Stroke

CV Death MI Stroke All Death

154

60 64

46

136

181

88

68

50

139

186

88

72

53

139

Study Medication Group

Original Adjudication RECORD Definitions

FDA Definitions

Rosiglitazone Group Comparison of the Original adjudications and re-adjudications with the RECORD and

FDA definitions. Adopted from Mahaffey, K. W., Hafley, G., Dickerson, S., Burns, S., Tourt-Uhlig, S., White,

J., ... & Lopes, R. D. (2013). Results of a reevaluation of cardiovascular outcomes in the RECORD trial.

American Heart Journal, 166(2), 240-249.

RECORD RECORD % FDA FDA %

CV Death, MI, Stroke 27 17.53% 32 20.78%

CV Death 28 46.67% 28 46.67%

MI 4 6.25% 8 12.50%

Stroke 4 8.70% 7 15.22%

All Death 3 2.21% 3 2.21%

• Systematic Event

Identification

• Query Management

• Additional Data Collection

• Similar Results with the

Medication Comparison

Group.

RM


Why the differences

• More Event Triggering

– Systematic Event Identification

• Automated and Manual triggering

• Cases where we received additional SD’s

– Query Management

• 127 Queries issued for Death Events

– 7 (5.5%) cause of death changed

• 70 Queries issued for MI and Stroke Events

– 2 (2.9%)– Adjudication results changed

– 8 (11.4%)– no prior adjudication

• 3rd Party Search for Vital Status

– 16 suspected deaths

RM


Case Study: APPRAISE 2

• Phase III, randomized, double-

blind, placebo-controlled,

parallel arm superiority study

• Evaluated the safety and

efficacy of apixaban compared

with placebo in subjects with

recent ACS and at least 2

additional risk factors for

recurrent ischemic events.

APPRAISE 2 Design and Follow-up. From Alexander, J. H., Lopes, R. D., James, S., Kilaru, R.,

He, Y., Mohan, P., ... & Wallentin, L. (2011). Apixaban with antiplatelet therapy after acute

coronary syndrome. New England Journal of Medicine, 365(8), 699-708.

MW


Case Study: APPRAISE 2

• Trial Stopped Early

– November 15, 2010 DMC

– Apixaban arm showed increases

in major bleeding compared with

the placebo arm

– No signal of efficacy

TIMI Major Bleeding Comparison for Apixaban and Placebo. From Alexander, J. H., Lopes, R. D.,

James, S., Kilaru, R., He, Y., Mohan, P., ... & Wallentin, L. (2011). Apixaban with antiplatelet

therapy after acute coronary syndrome. New England Journal of Medicine, 365(8), 699-708.

MW


APPRAISE 2 Significance

• Consistent evaluation of clinical endpoints

• Efficiency

– Maintaining Currency

– Intracranial Bleeding Identification

• Systematic, Comprehensive, Unbiased,

Blinded, and Independent

• Data Collection

– Well designed

• Informed Trial Decisions

– Timely Adjudications

MW


Duke CEC Guiding Principles - Defining Quality

1. Were all suspected events triggered as planned?

2. Were all events processed per charter procedures?

3. Were all adjudications data based accurately?

4. Were all QC procedures performed and documented with identified

issues addressed?

5. Did any major SOP/WI/HIPAA/GCP/IRB violations occur?

MW


DESIGNING THE

DATA COLLECTION

PROCESS

RM


Guiding Principles: Designing the Data Collection

Process

• eCRF

– Best Practices

– Types

• Source Documents

– Source Docs recommended by Event Type

– Sample Types and Value Provided

– Best Practices

– Regulatory Expectations

RM


Guiding Principles: eCRF Development

• Support the Endpoint Definitions

• Measurable and Objective

• Complements source documents

– Reduces source document

collection

• Pertinent Information

– consistently with the same

questions

• Chronology of care

MW


Important Types of eCRFs

• Hospitalization Page

• Endpoint Pages

– Death, Stroke, Myocardial Infarction, Bleeding, Congestive Heart Failure, Acute

Pancreatitis, and Neoplasms

• Lab Value Pages

– Cardiac Markers, Troponin I/T, CK & CK-MBs

– Labs of Interest: HGB/HCT

• Procedural Pages

– Cardiac Catheterization (CATH)

– Percutaneous Coronary Intervention (PCI)

– Coronary artery bypass graft surgery (CABG)

MW


eCRF Best Practice: Quarterback Page

• Captures Key Components/ Critical

Variables

• Yes/No for each event type

– Dynamically creates event page to

collect further details when event is

answered ‘Yes’

• Through telephone or clinic visit

• Recommend collecting no less

frequently than every 3 months

• Important in later clinical phases

MW


Source Documentation

• Supports and complements eCRF data

• Corroborate and Authenticate

– clinical findings, observations, endpoints, or other

activities in a clinical trial necessary for the evaluation of

the study.

• GCP-Guidelines

– 1.51 Source Data• “All information in original records and certified copies of original records of clinical

findings, observations, or other activities”

– 1.52 Source Documents• “Original documents, data, and records (e.g., hospital records, clinical and office

charts, laboratory notes, memoranda, subjects' diaries or evaluation checklists,

pharmacy dispensing records, recorded data from automated instruments”

RM


Source Documentation Regulations

• (21 CFR 312) Investigational New Drug Application

– “in Phase 2 and 3, to help assure that the quality of the scientific evaluation

of drugs is adequate to permit an evaluation of the drug's effectiveness and

safety” 312.22

– “GCP is defined as a standard for the design, conduct, performance,

monitoring, auditing, recording, analysis, and reporting of clinical trials in a

way that provides assurance that the data and reported results are credible

and accurate and that the rights, safety, and well-being of trial subjects are

protected.” 312.120

• (21 CFR 314) Applications for FDA Approval to Market a New Drug

– “purpose of conducting clinical investigations of a drug is to distinguish the

effect of a drug from other influences, such as spontaneous change in the

course of the disease, placebo effect, or biased observation.” 314.126

RM


Source Documentation Regulations

• Clinical Investigator Recordkeeping and Retention (21 CFR 312.62)

– “An investigator is required to prepare and maintain adequate and

accurate case histories that record all observations and other data

pertinent.”

– “Case histories include the case report forms and ………medical

records including, for example, progress notes of the physician, the

individual's hospital chart(s), and the nurses' notes.”

– Record retention for 2 years following the date of approval of

marketing application

• Medical Devices (21 CFR 812.140)

• Timely reporting of Events (21 CFR 312.64 and 21 CFR 812.150)

– Often Cited in Warning Letters

RM


CEC Best Practices: Insufficient Source Documentation

• Charter defined

– By type what source documents are expected

• If documents missing

– Collaborate with Site Management to collect documents

– Could another document replace or contain results

• Source Document Tracking

• Document insufficient source documents in Adjudication Forms

– Only if reviewer adjudicates event as a No

– “Were sufficient source documents provided for the adjudication of

this event?”

RM


Examples of Compulsory/Essential Source DocumentsClinical Endpoint Source Documents

DeathInvestigator Narrative (eCRF)

Discharge/Death SummaryAutopsy Report (if applicable)

MI/UA

Investigator Narrative (eCRF)

Discharge SummaryProcedure Reports (PCI/CABG)

Event ECGs (pre-event, event, post-event)

Cardiac markers with ULN (eCRF)

CHF

Admission data with dates and narrative

Discharge SummaryDrug therapy

Procedural Reports (if applicable)

Stroke


Discharge Summary

Neurology Consult NotesBrain CT/MRI Scan Report

Bleeding Events


Discharge SummaryRelevant Procedure or Surgical Reports

Hemoglobin/Hematocrit data (eCRF)

Transfusion Records (eCRF)

Stent ThrombosisInvestigator Narrative (eCRF)

Discharge SummaryCoronary Angiography and Procedure Reports (PCI/CABG)

RM


Source Documents – Discharge Summary

Discharge Summary

• Preferred backup to clinical eCRF

narratives for endpoints

• Source documents will be tracked by

CEC staff

RM


Source Documents – Cardiac Markers

Cardiac Markers

• eCRF - Preferred mechanism for

collection

• Trigger Events

• Identify missed events

RM


Source Documents – ECG’s (12 lead)

Support Event Adjudication

• Prior ECG’s

• Event ECG’s

• Post Event ECG’s

– After event has

resolved

RM


Site Event Narratives

• Provides Qualitative information

• Allows sites to enter relevant information not captured elsewhere in

eCRF

• Provides Clinical Context

• CEC Best Practice: Guided Narratives

– Allow for better narrative collection

– Reduce Other Source Documents

– Specific Data points

– Sites prompted to provide information regarding circumstances,

treatment, interventions, diagnostics, and resolution of symptoms or

outcomes (similar to a discharge summary)

MW


Guided Narrative Example - MI

What were the initial (presenting) signs and symptoms (include symptom

duration)?

– Pt presented to ER via ambulance on 03Mar2007 at 12:46hr after onset of

substernal chest pain at 10:30hr that morning.

What interventions / actions / diagnostic procedures / tests were used to identify

the event?

– Pt diaphoretic with BP of 90/60 per EMS. ECG showed 3mm of inferior ST

elevation. Troponin and CK-MB were positive on arrival to ER and are

documented in eCRF.

28

MW


Site Narrative Example – Unobserved Death

What were the relevant medical events leading up to the death of the subject?

• Unobserved Death:

– Neighbor checked on patient at approximately 18:45hr on 08Jul2008. Patient was watching the television with no complaints. When patient's son came to the patient's house at 08:00hr on 09Jul2008, patient was found dead in his recliner with the television still on. No autopsy was done. Patient has history of CHF, MI in April 2007, and status post CABG in May 2007. The patient was last seen at the site on 25Jun2008 during his regularly scheduled visit. At that time, patient complained of increased dyspnea while walking around the neighborhood. Vitals were stable, physical exam was unremarkable and no diagnostic testing was done. The physician's plan was to reassess the patient during the next site visit which had been scheduled for 24Jul2008. PI's thought on proximal cause of death is arrhythmia.

29

MW


Rarely Collected Source Documents

• Usually not considered pertinent information

• Provide little to no context in regards to the

clinical event

• Evaluated by a Core Lab

• Examples

– Actual X-rays Images

– Medical records – Not pertinent to the event

– Intake/Screening forms

– Telephone/Email contact records

– Subject diaries

– Informed consent forms

– Drug disposition records

– Microfiches

RM


ALCOA in Source Documentation and Case Report Forms

• Attributable

– Who wrote it or who did what

• Legible

– Capable of being read and properly redacted

• Contemporaneous

– Was this completed at the time of event or as close to the event as

possible

• Original

– Original recording of the information or copy (paper, electronic)

• Accurate

– Correctly reflects event details (consistency between SD’s and eCRF)

RM


Source Document Collection Expectations: Site Training

• Investigator Meeting/Webinar Training

– CEC Site Manual

• CEC Source Document Processes

• CEC Submission Process

– Secure Electronic Collection

Information

– eCRF Completion Guidelines

– Clinical details related to event

eCRF Completion

Guidelines

MW


COLLECTING THE

DATA

MW


Guiding Principles: Collecting the Data

• Document Tracking

• Redacting

– PHI

– Unblinded info

• Transmittal and Query Process

• Assuring sufficient data present to support

event definitions

MW


Duke CEC Source Document Processing

Source Document Process Flow

Event Triggered

Query to Site for Source Documents

CEC Designee Reviews

Event Case for

completion

Site Collects and Redacts

Source Documents

CEC Designee Prepares

Documents(Blinds and

Labels)

Event sent foradjudication

review

Source Document

Received from Site

RM


Source Document Collection Methods

Collection Method Description Advantages Disadvantages

Secure

Electronic

Systems

FTP Submission of documents via

website

Secure website allows for tracking

of documents

Website maintenance

Additional resources needed for

retrieval of documents

EDC Data and document collection via

electronic data capture systems

Systematic and consistent data

collection

Tracking and management of

source documents

Costs related database maintenance

Fax/RightFax Sent via Fax or Email to Fax Tracking and management of

documents within the system

Additional resources for retrieval and

sorting of documents

Email

Submission of documents via

email

Ease of accessibility

Faster transmission times

Requires secure server

Additional resources for retrieval and

sorting of documents

Mail Rarely do we want to collect

documents through carriers or

CRA’s

Correct documents collected and

sent to CEC

Higher rate of documents received

High probability of losing documents

Higher costs with document

collection

Smartphones Submission of documents using

smartphone

Quick capture of source documents Unsecure network

Data Protection issues

RM

Most

Least


Protected Health Information (PHI)/ Personally

Identifiable Information (PII)

• HIPAA Privacy Rule

– Safeguards individually identifiable health information

• Protected Health Information (PHI)

– Privacy Rule

– Information in the medical record that can be used to identify an individual

• Personally Identifiable Information (PII)

– Used to distinguish or trace an individual's identity, alone or when combined

with other information

MW


Protected Health Information (PHI)/ Personally

Identifiable Information (PII)

• Two methods for de-identification

(HIPAA 164.514)

1. Expert Determination

• Very small risk that information could

identify an individual

2. Safe Harbor

• Removal of the 18 identifiers

Removal of the 18 types of identifiers. Retrieved

from http://www.hhs.gov/ocr/privacy/hipaa/understanding/cover

edentities/De-identification/guidance.html on 10-Mar-2015

RM


First Line of Redaction

• Sites - First line of redaction for PHI/PII

– CEC last line of redaction, prior to the adjudication review (can

include treatment comparisons i.e. INR)

• All PHI/PII needs to be redacted or blinded

– Unless exemption approved by IRB

• Other Considerations:

– Blinding (Treatment) – Charter Defined

– All pages have Site and Subject numbers

– Unit and reference range present

– Date/Time present

MW


Back-up redaction of Source Documentation

• Arrival at CEC - CEC Designee upon source document arrival at CEC

– Reviews for subject identifiers

– Redacted with the Adobe/Nitro redaction tool, as necessary.

• Prior to Adjudication time of event packet/dossier completion

– Ensure that event packets and dossiers are redacted

• If necessary, redaction performed at this step prior to event adjudication review

– Redaction with Adjudication Platform (e.g. MERGE) redaction tool

MW


Electronic Source Document Tracking

• Ability to identify which documents present during event adjudication

• Robust Analysis

– Identify missing documents consistently left out (or sites)

– Demonstrates utilization of alternative documents

– Help overcome challenges in document collection

• Faster Case Processing

• Share documentation with safety for dual reported events

– DCRI Safety Surveillance is eCRF driven no source document collection.

RM


Electronic Source Document Tracking

Example - Bleeding Event

• Documents tracked upon

arrival by CEC Designee

• CEC Designee notates status

as Unobtainable or Not

applicable

RM


Electronic Source Document Tracking Report- Example

75

80

85

90

95

100

LatinAmerica

Europe Middle East NorthAmerica

Africa

83.3385.19

90.2891.5

100

Bleeding by Region with % Discharge Summary Present

RM

N=4 N=2N=170 N=27N=6


CEC Best Practices: Query Management

• Query Tracking

– Faster turn-around on queries

• Query Escalation Plans

• Standardized Query Language

– CEC: Please submit the Pathology Report, Oncology Consult Notes,

Imaging Reports, Surgical Notes, Discharge Summary, and Oncology

Clinic Notes in English to CEC. (Preferred method of document

collection)

– CEC: Please enter information in eCRF death narrative about

circumstances of death (in sleep, while watching TV, etc),

comorbidities/past medical history, whether pt. was having any

complaints at time of death. Contact CRA for assistance. Thank you

MW


PRESENTING THE

DATA

Event Dossier

Review Forms

Patient Profiles

Source Documents

Site Queries/Responses

RM


Guiding Principles: Presenting the Data

• Event Process Flow

• Event Packets/Dossier

• Patient Profile

– Designing

MW

Event Dossier


Duke CEC Event Processing

Event Process Flow

Site enters data in eCRF

Desgineeverifies that all required information and source documents are present

Trigger algorithms

run on eCRF data and

suspected events are identified

Site collects and submits

source documents to

CEC

Event sent foradjudication

review

CEC manages query to site requesting

source documents

If needed, source

documents are translated

RM


Event Packet/Dossier

• Event Packets/ Dossiers present the

relevant clinical information for adjudication

– Clinical Event Adjudication Review Form

– Patient Profile

• Summary of relevant patient data from eCRF

and central labs

– Source Documents

• Medical Records

– Site Queries/Responses

– Archived and provided via electronic

media to sponsor

MW


Patient Profiles

• Generated from eCRF data

• Identifies event to be reviewed, including

event date and time

• Lists all key variables from eCRF in

chronological order

• Site narratives

• Integrated data from Central Labs

• Example of a Patient Profile

RM


Patient Profile Design

• Patient Profiles

– Constructs the situation

from multiple areas of the

eCRF capture

– Enzyme Plots

– Data Change Reports

MW


FUTURE OF

SOURCE

DOCUMENTS

MW


Future of Source Documents in Adjudication

• Impact of EHR

• Post-Marketing Event Verification

MW


Electronic Health Records/ Electronic Medical Records

(EHR/EMR) Guidelines

• Treat electronic record as if paper source

document

• 21 CFR Part 11 Compliant

– need for controls, audit trails, electronic

signatures, and software documentation

• Retain as required under 21 CFR 312.62

• Access to research charts gained via medical

records

RM


Advantages and Challenges of Electronic Health

Records/ Electronic Medical Records (EHR/EMR)

• Advantages

– Greater Accessibility

• Sites can be held to higher standards for due diligence

– Faster collection and Processing times

• Electronic source documents

• Print and Scan

– Reduces Risk for lost documents or pages

RM


Advantages and Challenges of Electronic Health

Records/ Electronic Medical Records (EHR/EMR)

• Challenges

– Not available at all sites (Global Trials)

– Extraneous Documents to Review

• Example Triggered MI Event but receive urine logs and

nursing notes (irrelevant for adjudication)

• Higher Costs due to resources needed for processing and

translation of documents

RM


Post Marketing Event Verification

• Enhanced Event Identification

• Differences in the minimum and the maximum in data collection

• Streamlined Approach

– Fewer Reviewers

– Targeted source document collection

• Adjudicated No due to insufficient source documentation

– Faster Case Processing

– Higher Efficiency

• Concordance Rates

MW


In Conclusion

Proper Event Adjudication

Presenting the Data

Collecting the Data

and Source Documents

Designing the Data

Collection Process

• Presentation of Clean, Accurate, and Applicable Data to

Adjudication Committees is critical

– Consistency in the evaluation of endpoints

– True treatment effect of investigational product

– Confidence in the final endpoint adjudications

• Accomplished by

– Precise and robust collection of eCRF/source

documents

– Comprehensive and applicable event packet/dossier

presentation to adjudicators

RM/MW