navigating the research misconduct process kenneth l. busch, ph.d. nsf oig research integrity and...

Navigating the Research Misconduct ProcessKenneth L. Busch, Ph.D.

NSF OIG Research Integrity and Administrative Investigations Division

Who is NSF OIG?– Independent office reporting to the Congress and NSB.– Promote economy, efficiency, and effectiveness.– Prevent and detect fraud, waste, and abuse.– Accomplishes mission through:

• Audits• Investigations

– Criminal and Civil (e.g., false claims, false statements, embezzlement).

– Administrative (e.g., regulatory and policy violations).

Where does research misconduct fit in?OIG is delegated the responsibility for investigating

research misconduct allegations involving NSF programs.

Research Misconduct (RM)

• Federal definition and procedural framework (OSTP Dec. 2000).

• RM means “fabrication, falsification, or plagiarism in proposing or performing research … , reviewing research proposals … or in reporting research funded by [the agency].” (45 C.F.R. 689.1(a))

• Not honest error or differences of opinion.• Must be “reckless, knowing, or intentional” and not careless.• Must be “a significant departure from the accepted practices

of the relevant research community.”

Why care about RM?

• Fairness– An NSF proposal is a request to obtain Federal funds

(ultimately) from the taxpayer. – NSF program officers/reviewers assess the proposers’

ability to carry out the proposed work and the proposers’ understanding of the current state of the field.

– False representation of data and plagiarism misleads reviewers’ assessment of the PI’s (and Co-PI’s capabilities/knowledge.

• Economy, efficiency, and effectiveness– Do not want to pay for research already completed.– Do not want subsequent work to be based on

misrepresented work.

OIG’s Investigative

Scientists

In an investigation-

Good Practices,

Bad Practices, and

Things that make us say “Hmmmm.”

45 CFR Part 689

The Basic Process Allegations

Inquiry & Investigation

Adjudication

Appeal

Final Decision

Define a problem

Ask questions & collect evidence

Draw conclusions

Reconsider

Finalize conclusions

Looks like the scientific method

Looks like the legal process

Take home messages

I. Be prepared-

II. Every case is unique-

III.Document, document, document-

IV. Go where the evidence takes you-

V. Make a balanced, and supportable decision-

Research Misconduct Management Plan

Before Allegation Inquiry Investigation Report & Decision After

Not even a little bit



Policy

• Up-to-date.• Integration with other policies.• Not overly agency specific.• OGC has designated someone specific.• Available online .

• Out-of- date.• Multiple conflicting policies.• Agency specific.• OGC never heard of the policy.• Buried in a subpage in a corner of the

web.

Does your community know your policy? And where to find it? Will they act according to policy?

To whom does your policy apply? Do they know that it applies to them?

Is there buy-in at all levels?

How is it integrated with other policies? (e.g., data management, authorship, IRB, IACUC, financial COI)

Does your General Counsel know you yet?

RIO prevails and handles the allegation under RM policy with notification to NSF.

When faculty don’t know about the policy, they make it up as they go.• PI (junior faculty) raises fabrication allegation against student.• Graduate coordinator gives student choice – leave or be expelled. • Student leaves and PI enlists help of students to investigate.• No formal process started until the Dean found out a week later.• COI and gap in evidence chain-of-custody.

PolicyMid-InvestigationPolicy Change

Shhh . . . Don’t tell NSF• University receives an allegation. • Subject gets a lawyer. • GC wants to handle as personnel

matter and not tell NSF.

Poof! Now we have a

Policy!



IntakeProcess

• Complainant’s role is clear.• Complainant’s role is neutral.• Multichannel intake mechanism.• Intake at neutral level.

• Complainant as adversary.• Lack of anonymity.• High evidentiary burden.• Single intake point.• Intake in zone of conflict.

Can the allegation be anonymous? Must it be written?

Is the complainant an entity (a journal or an agency)? Who receives the allegation? What should the allegation contain? Does the allegation impact research supported internally or externally? Is there any imminent harm or immediate need for action?

Seamless Handoff-Student Academic Integrity Committee adjudicates cheating allegation and passes allegation of plagiarism to RIO to handle federal funder requirements.

Why be so formal about it?• Dean receives RM complaints

from faculty members in one department about another department’s PI.

• He does not begin university process (could impact active awards).

• Keeps it within the college for an informal process with no documentation.

Intake Process

• OIG contacts university about allegation and does onsite inquiry.

• University GC learns for the first time of informal process below.

• University-level investigation process begins.

But he didn’t say the magic word-RIO receives plagiarism allegation from student but immediately classifies it as an authorship dispute without further probing.

Not our problem anymore -Student found to have committed Academic Dishonesty under student policy. RIO in the dark until student has been expelled.



Finding Substance

• Committee role defined.• Committee products defined.• Evidence is secured.

• Single person decision (possible bias).

• Evidence unsecured/lost.

What evidence do you secure? When?

Who conducts your inquiry?

What are the requirements for confidentiality? What is the end product of the inquiry? What are the notification requirements? What is the timeframe for the inquiry?

What do you do with admissions of guilt?

The easy way out -• Dean notifies foreign student by

letter which ends with a pre-written admission and a place for her to sign.

• The student is told if she does not sign, the Dean will begin the inquiry process.

• She signs and leaves the university.

Finding Substance

Something to work with here-Inquiry committee determines the allegations could be proven with further development of the evidentiary record and recommends detailed investigation.

We changed our minds-• Committee recommends investigation. • Subject gets attorney.• Committee reconvenes and notes many other

possible explanations for the data and rescinds recommendation.

• OIG investigates; University out of the loop.

Why bother?Because the subject admitted the wrong-doing, the RIO did not see any need to form a committee or conduct an investigation and proceeded to take care of it himself.

Research Integrity Officer



• All affected funders identified and notified.

• Non-RM allegations identified and passed along.

• Not notifying funders until process is complete.

• Ignoring non-RM allegations and not reporting them.

What evidence do you secure? When?

Who conducts your inquiry?

What are the confidentiality requirements? What is the end product of the inquiry? What are the notification requirements? What is the timeframe for the inquiry?

What do you do with admissions of guilt?

Notifying Funders

Way to go! University RIO notifies OIG that an inquiry committee has determined an investigation is warranted and that the inquiry report is in the mail.

Delays = Complications• University correctly identifies

NSF as a funder in the inquiry report, but does not notify NSF until investigation is wrapping up 2 years later.

• University missed out on opportunity for committee briefing and shared documentation.

• Criminal issues complicated by passage of time.

Notifying Funders

Through other channels• RIO notifies only one federal agency, but has

identified funding from multiple agencies. • OIG learns through blog.

But she said she’s sorry-Subject admitted extensive data manipulation but could not “remember” every place she did. University asks OIG for a multiparty settlement to avoid complete investigation. Subject wants to avoid debarment.

• OIG requests full investigation to fully document the extent of the fabrication and falsification.

• OIG finds link to high impact award; leads to debarment.

• Subject was employed by a government contractor but would not disclose this to OIG.



The Committee

• From relevant community• COIs identified and managed• Tenured• Legal counsel available• Experienced• Policy briefing by OIG case lead

• Not from relevant community• COIs before or during• Untenured • No legal counsel available• Inexperienced• No OIG briefing

What are the confidentiality requirements? What is the timeframe for investigation? How are COIs identified and managed? What are the elements of a finding? Do they differ from NSF’s? What is the process for adding Subjects? How is testimony corroborated?

Standing committees v. Ad Hoc committees

The Committee

• Committees are usually ad hoc and composed of faculty members without any specialized training in investigating.• They are out of their comfort zone.• They are worried about getting sued.• They will gravitate back to their comfort zone

at any opportunity.• They are like juries, except they can ask the

questions. Like juries, they need instructions.

• A RIO (with the help of counsel) should:• guide them back into the investigation zone; • not influence their assessment;• provide the context for consistency.

• OIG has resources you can use.• The OIG Case Lead• The OIG investigation briefing• The OIG website

• Semiannual reports• Case closeout memoranda and research

misconduct reports online.

Can’t you wait?Following a subject interview, a committee member tells the RIO that he has an interest in collaborating with the subject. RIO manages the new COI and tells the OIG case lead.

The Committee

Counsel should avoid representing both the subject

and the university.

Consistency – not always a hobgoblin+ 1 University; + 2 similar allegations of RM;+ 2 different Committees operating in parallel; + 2 very different investigations and

outcomes.= Little to no consistency.

Better Late than Never?• Committee does not schedule its first

meeting until day 175 of 180-day referral period.

Best Made Plans -• University community is small and

contains numerous sets of spouses/partners as well as interdisciplinary collaborations.

• Committee members realize during OIG briefing that they have COIs and cannot serve.

Mum’s the word - • Subject tells colleagues and

reporters about the allegations against him. The case plays out in multiple newspapers.

• Committee members have finished report; maintain confidentiality.

• Institution acknowledges that process completed and that it is now with feds.

The Committee

Ask and Ye shall ReceiveRIO schedules the OIG briefing on the same day as the initial committee meeting as a way of setting the guidelines and policy expectations.

Outside the box-University retains outside committee members to manage potential COIs and add technical expertise.

No Committee is an island -• Committee turns down OIG policy briefing.• RIO is hands off. Committee report is haphazard

and unsupported. No finding of RM.• OIG does not find it accurate, complete, or in

accord with reasonable procedures. • OIG proceeds independently and recommends a

finding and debarment.



Interviews

• Interviews recorded and transcribed• Trust but verify statements• Safeguards for vulnerable persons

• No interview documentation • Statements not corroborated• No safeguards for vulnerable persons

How do you address confidentiality?

How do you balance confidentiality with the number of witnesses called? What are your options for hostile witnesses? Can you compel participation of witnesses? If so, what rights do they have?

How do you handle subjects and witnesses at other institutions?

He said, She said-• Committee members interview subject

and students without notes, recordings, or transcripts.

• Subject later contests committee’s interpretation of his statements.

Go Ahead, Make our Day-Committee records the interviews, introducing all who are present and their roles. Individuals speak one at a time. Questions are open, eliciting free form answers initially. Closed, direct questions follow to clarify points. Committee remembers to ask directly “Did you do it?” Transcriptions are shared with witnesses to ensure accuracy (edits made in track changes). Original and edited versions included in case materials.

Minimal Effort-• Committee relies only on the

evidence provided by OIG in referral letter.

• Committee does not interview the subject or any other witnesses, although they are available.

Interviews

It’s not what you ask, it’s how you ask it-• Committee asks leading or Socratic questions

prompting specific answers from the subject.



Evidence

• Chain of custody.• Forensic images of digital media.• Emails appropriately obtained.• Minimal research impediments.• Trust but verify statements.

• REP violated.• No chain of custody.• Unverifiable digital files. • Unsupervised access.• Inhibition of ongoing research.

What can you secure? DataNotebooksInstrument recordsSamplesEmail(?)

What is the subject’s reasonable expectation of privacy (REP)?

How should you secure it?

Can the research proceed?

Where else may the evidence be?

Evidence

You wanted us to investigate?Committee accepted the software generated similarity reports at face value without manually reviewing the individual sources, some of which were the subject’s own publications.

Looking under the next rockCommittee fails to look at other proposals and publications for additional misconduct.

It’s all interrelatedCommittee fails to look at associated financial COI disclosures when the subject is known to have SBIR or other corporate involvement.

The Devil in the DetailsUniversity forensically analyzed hard drives and found that computer science professor manipulated metadata in proposal drafts to change dates to support his claims.

Border Crossings -Gathering evidence in investigations across international lines has unique challenges.



• Assess the act.• Assess proposals & products.• Develop the evidence.• Connect the dots.• Consistent standards.

• Assess the science.• Ignore proposals or products.• Assess only what OIG provides.• Non-sequitors.• Double standards.

Are you analyzing the science or the misconduct?

What are the elements of a finding, per policy or referral?

What is pattern evidence and where do you look? What can you use it for? Can the IC identify additional Subjects and allegations? How does the IC corroborate testimony?

How do you explain/define intent levels?

Analysis

Reductio ad absurdum-• Under the guidance of counsel, the

committee analyzed each segment of text as discrete units without looking at the totality of the copying.

• Committee used copyright (author centric) instead of plagiarism (reader centric) analysis for material copied from instrument brochure.

Chain-link Plagiarism-Committee found that PI copied from student, and that student copied from sources. Committee found that the actions of both met the definition for plagiarism.

Analysis

Mixed Signals-Committees mix the levels of intent:• “carelessly knowing”• “at least knowingly and most

probably carelessly”• “carelessly, recklessly, and

unintentionally”

Being right doesn’t make it right-Although the subject argued extensively that he ultimately proved his conclusions correct, the committee correctly found that at the time of the misconduct, the data were falsified to support the hypothesis.



Report

• Evidence-based conclusions.• Connect the dots.• Specific recommendations.• Active voice.• Addresses referral questions.• Subject opportunity to comment.

• Assumptions not conclusions.• Non-sequitors.• Vague institutional response. • Passive voice.• No questions answered. • No opportunity for subject to comment.

To whom is the draft distributed for comments? Who is the deciding official? What if individuals have left the institution? How are reputations rehabilitated? What are potential legal implications?

Are the final conclusions reasonable?

Close but no cigar-Deciding official does not make an explicit finding of research misconduct but takes action.

And the Oscar for best report goes to –Report clearly address each element as outlined in university policy and responds to OIG referral questions with specificity, including INTENT. Each conclusion is supported by the evidence and the committee’s rationale is clear. An unambiguous statement of finding (or no finding) is made.

Report

Where’s the Evidence?Report submitted to OIG does not contain all the evidence on which the committee relied.

The other side of the story-Report submitted to OIG but the subject was never offered an opportunity to comment or rebut the finding at the institution.



Notifying OIG

• 45 C.F.R. 689.4(c) (risk to health or safety, civil or criminal violations, etc.)

• Subject retains counsel.• Ancillary allegations.• Deviations from policy.• Any surprises.

• Not notifying OIG until the end, if ever.

• What can the OIG case lead do for me?

• How do you know if there is a potential civil or criminal violation?

• What will NSF or OIG think if we tell them something is wrong?

It’s in the mail –• RIO fails to deliver subject a copy

of the referral packet at the time records are secured (or at any other time during investigation).

And then there were two -• During an interview the witness

disclosed information that made him an additional subject.

• RIO immediately notified OIG.

Notifying OIGThey said what?

• In separate interviews, two subjects each admit voluntarily to knowingly misstating the experimental conditions in a publication.

• A witness also admits to knowingly misstating the conditions.

• The admissions are not part of the originally alleged misconduct.

• OIG learns of the admissions when reading transcripts.

On the same side of the “v” -• RIO receives the subject’s comments

to the report which have a cc to Bob Smith, Esq. of Dewey, Cheatham, and Howe.

• OIG case staff notified immediately. This triggers represented party rules for the attorneys involved.



ImplementActions

• Consult• Debrief • Review policies• Read the OIG report

• Wait • Do nothing• Assume it will never happen again• Don’t read the OIG report

How long do consequences last and who monitors them? Are policy revisions necessary? How do completed cases affect RCR training? How is public disclosure and/or complaint handled?

What if the subject has left the institution?

It’s a work in progress-University officials with oversight responsibility are told of the university decision and of any requirements imposed by NSF (e.g., assurances, data and mentoring plan reviews).

He’s not our problem anymore - • University fails to notify NSF that

it has replaced the PI on and NSF award because the PI was found to have committed RM.

• OIG investigation is ongoing.• NSF must approve replacement

PIs – OIG cannot do this

ImplementActions

Learn, Practice, Repeat-University RCR program uses redacted report as a starting point for case studies in live-instructor training; subject required to be involved in teaching the class. Debunk the myth of “it doesn’t happen here.”

I. Be prepared-Have a comprehensive, up-to-date policy to guide you.

II. Every case is unique-A good policy will get you through twists and detours.

III.Document, document, document-This is the support for your direction and decision.

IV. Go where the evidence takes you-Who, What, Where, When, Why, and How.

V. Make a balanced and supported decision-Make it timely, focused, and clear.

Déjà vu: Take home messages

Contact Information

www.nsf.gov/oig

Hotline:1-800-428-2189E-mail:[email protected]:(703) 292-9158

Mail:4201 Wilson

Boulevard Suite II-705Arlington, VA 22230 ATTN: OIG HOTLINE

navigating the research misconduct process kenneth l. busch, ph.d. nsf oig research integrity and...

Documents