Martin Beran – QA

John Roncoroni – FRD

Van Starner – SD

IR-4 National Education Conference Phoenix, AZ - March 1, 2006

Perspectives on QA Audits/Responses

Martin Beran

IR-4 Western Region

Quality Assurance officer

U.C. Davis

Those Darn Findings

• The Pseudo-Antagonistic and Poly- Unsaturated Architecture of QA Findings

• I Have No Idea What QA is Asking Me?

• How Am I Supposed to Respond?

• Who Gets a Copy of My Responses?

QA Obligations

• Researchers Need to Know What the Issue Is

• Findings Should be Clear

• Avoid Accusatory Tones

• Offer Solutions to Problems

These Findings Say it All

• You did not have your SOPs on site at the time of the application. According to the GLP regulations, SOPs should be immediately available. I recommend a copy of your current SOPs be placed in your truck.

One More:

• Part 4E, Test Substance Records - I cannot seem to locate test substance storage records for the period of 2/11 – 3/29/05. Test substance storage records are needed from the date of receipt to the date of the last application. Please insert additional records if available into the FDB.

Findings that are Weak Sauce

• Luke Skywalkers’s training records did not include any reference to GLP training.

• Any impromptu GLP training Skywalker might have had should be added to Part 2B of the FDB. This can be in the form of a copy of his ongoing Jedi training record or a separate sheet of paper with any informal GLP training documented.

Weak Sauce Continued

• Your Crop Destruct statement is insufficient.

• Please explain how and when the treated apples were destroyed. You simply stated that the row was taped off. Please document the measures you took to ensure that the apples did not enter the food chain.

Responses

• Did they answer the question

• Did they provide the necessary information

• Has the FRD carried his corrections over to the FDB

Responses

• I get a lot of these:

• OK

• Noted

• Study director will Address

• How dare you

Note to Study Director

• The plot layout consists of one row of 14 plants. The Protocol states that samples should not be taken from hops on the ends or the edges and they must be taken from at least five hills. Since the researcher has only one row to sample from, this seems to be a Protocol Deviation. Did you mean to write in the Protocol that hops could not be taken from plot edges? This means that plots need to be more than two rows of hop hills. No response needed from the researcher.

More Notes:

• The Chicory samples from the TA01 trial seemed to be quite different in appearance from the –DS05 samples. The Tatooine root samples were at most finger thickness compared to the Death Star samples which were round as a turnip. Even though Amendment #4 mentioned a potential difference in residue, the physical difference in the Tatooine samples was stark enough to be brought to your attention.

Routing FDB Findings

QA

RegionalQA

Coordinator

FieldResearchDirector

RegionalField

CoordinatorHQ

Routing CPI Findings

QA

FieldResearchDirector

RegionalField

Coordinator

HQ

Additional Points

• Sending Corrected Pages Along with the Audit

• The Quality Control Review

• Communicate with the Auditing QA

Response To My Responses?

John RoncoroniIR-4 Western Region

Field Research DirectorU.C. Davis

Finding #1

• The GLP status of the test substance was not know “at the time of receipt,” and there was no record of a communication to the Study Director, who sent an email on July 28. Suggest changing to “No” (not known at time of receipt). Most time there is a GLP assurance in the letter from the supplier, but not in this case.

My Response

• “Technically I was taking the registrant’s word that they would actually send a Certificate of Analysis to Dan Kunkel at HQ - see page 3 part 4. Section 13 of the protocol states that ‘the registrant will provide a Certificate of Analysis to IR-4 headquarters.’ Not necessarily to the FRD.”

Registrant Letter

• “This test substance, which was obtained from production lot no. ABC123, has been found to have an active ingredient concentration of 9.3%. A certificate of quality has been forwarded to the attention of Dr. Dan Kunkel at IR-4 Headquarters.”

Finding #2

Part 6, page 14:

48.4 x 233.823 = 11317.033

Please correct this and the GPA and send

the corrected page back to HQ.

My Response

• The numbers in part 6 page 14 (and actually on the page before also) are:

48.4 x 2333.823 = 112957.03

(reviewer is off by a factor of 10) !

Finding #3

• Part 6 page 28 of 30:Show % deviation from

Protocol rate. This is asked for in instructions for this page.

My Response

• Clarification added:Note: I think that 100.39% indicates that I was 0.39% over the protocol rate and is clear for the reviewer to recognize.

Finding #4

• Part 5 page 14 of 20:The non-applicable ID# has been lined out but there is no error code.

My Response

• This is a field ID that is on a page that is copied – my answer is I have found in the past that there really is no appropriate error code - I will use WE for wrong entry.

Finding #5

• Page 5, pages 7-9:On page 7, there is a notation of the GPS reading used for the water valve used as a permanent landmark. If you will be relying on that reading, versus the valve itself, we should add the GPS unit to the list of GLP exceptions in Part 1C. If the valve has some kind of unique ID, that could be used for future trials.

My Response

• I am relying on the valve as the permanent marker. The valves are far enough apart that the GPS will identify this valve as opposed to others.

Van StarnerIR-4 Headquarters

Study DirectorRutgers Univ.

Current IR-4 Guidance• San Antonio Results Document (9/27/02):

Q: How do we correct raw data that we no longer have in our possession?

A: Make the correction on a copy of the raw data and forward with the responses to HQ QA (updated as per new procedures). Contact the

Study Director if there are any questions about responding to QA findings, or contact the QA who performed the audit with questions of intent.

• IR-4 Advisory #2003-01 (4/25/03): “Responding to FDB QA Findings”

Current IR-4 Guidance• Instructions from QA auditor in audit cover

letter:

“Return the audit with your original responses and any corrections, clarifications or altered copies of your data pages to IR-4 HQ QA. The HQ QA will then forward the responses and any corrected pages to the Study Director and Testing Facility Management. Please keep a copy of the audit for yourself, and send copies of the responses to your Regional Field Research Coordinator. Only one mailing, your original response, needs to be sent to HQ.”

Who is the target for your responses?

QA?Field Coordinators?

Management?EPA?

The Study Director!

Why?

By 40 CFR Part 160.33, the Study Director:

• Represents the single point of study control• Has overall responsibility for the technical conduct of

the study• Is responsible for interpretation, analysis,

documentation, and reporting of study results

So……..• When you receive an audit for review

and response: Don’t take it personally that QA has “dinged” your work, BUT….

Consider how you can best respond to assist the Study Director in

finalizing the study!

And Remember!

Before a final report can be finalized, signed and submitted to EPA, EVERY QA finding must be completely addressed to the satisfaction of the SD, QA and Management!

FINDING: “Meteorological data not complete; last data is for 31 July 2003. Please update meteorological data.”

RESPONSE: None – FRD only signed the bottom of the page of audit findings.

FINDING: “Part 6H – a brief description of procedure used in cleaning spray equipment is asked for here, not the SOP followed. Please provide this information.”

RESPONSE: “Prior to use, equipment is washed with clean potable water. Tanks are triple rinsed and flushed as are the lines, nozzles, strainers, and nozzle screens. Equipment is inspected for leaks or obvious wear prior to calibration and use. Clamps are ………………….”

FINDINGS: “Part 6C – incomplete error documentation; all cross-out entries need to be initialed and dated and the error code added. Lined out spaces need to be initialed and dated on Part 5G, 5H, 7B, 7D. Part 7B – crop destruct; entry needs to be initialed and dated.”

RESPONSE: For all three: “Acknowledged”

FINDING: “Parts 4 and 9 contain several photocopies that lack one or all: Field ID #, date, initials, location of originals”

RESPONSE: “No action needed, thanks for the suggestion.”

OR

“No response necessary from Field Research Director – Study Director should address.”

FINDINGS: “ Please line-out unused parts of Parts 1C, 2A, 4C (page 1 and 3), 5G, and 5H.” A minor error is seen in Part 6, page 27 of 35. Total pass time was 41.5 seconds. 41 seconds has been used in calculations. This has given a lower % deviation of +3.65. It should be +4.76%.”

RESPONSE: For both: “OK”

FINDING: “Part 7A, sample collection/storage; you failed to mention whether row ends were sampled from, and whether five or more hills were sampled from. This needs to be added to the sampling description.”

RESPONSE: “Protocol states: ‘Avoid sampling from plot edges/ends,’ so we complied with the protocol.”

FINDING: “Part 6C, pg. 2 and 4 of 32: please cross out field ID of trial that does not belong in this FDB. In the future, only one field ID per page, please.”

RESPONSE: “Opinion – not written rule. If it’s not written, it doesn’t exist.”

FINDING: “Part 7B. Crop destruction section needs to be filled out with appropriate information.”

RESPONSE: “Acknowledged.”

- Overall, responses to QA audits have been improving significantly, as has the consistency of QA findings, etc.

- There is still room for improvement on both sides, and it’s through training events like these that we can re-emphasize, face-to-face, the need to communicate, provide clear direction, and use a neutral (vs adversarial) tone in order to expedite responses and the entire review process.

Final Thoughts