Ellen R. Fisher, Department of Chemistry

Kathy Partin, Office of the VP for Research &

Department of Biomedical SciencesPaul Stretesky, Department of

Sociology

Cultural Competency in an RCR Training Program:

Focus on Misconduct

How this project got started… Faculty member in Chemistry (my day job)

- Providing RCR training to undergrads, grad students

› Developed tools specifically for different groups.1

- Involved in institutional project to expand RCR

What is our constituency?- Land Grant Institution/Research I

› Over $300 M in annual research expenditures› ~1450 faculty (~950 tenure track)

- Student Population› Undergraduate (~21,000) and graduate (~5000)› Vet school (~500)

1E. R. Fisher and N. E. Levinger, J. Chem. Ed. 2008, 85, 796.

Is what’s good for Chemistry good for everyone? There is cultural bias in misconduct

- Non-U.S. Citizens make up ~6% of all scientists in U.S. - Non-U.S. Citizens make up ~31% of all misconduct

cases- Literature: differences in plagiarism attitudes1

We want to be sensitive to cultural differences- Ethnicity, race, gender, disability, citizenship- Discipline

CSU has diverse research programs- Social “soft” sciences- “Hard” sciences and Engineering- Agricultural sciences, natural resources, cross-

disciplinary- Infectious disease 1Marshall & Garry Int. J. Ed. Integrity

2, 26 (2006).

Can we create an institutional-wide RCR program that is “culturally”

sensitive?

What is culturally-sensitive about RCR?

Anecdotal: my experience- Differences between populations (e.g.

undergrads vs. grads)› Level of intellectual maturity› Level of experience

- Differences in sub-disciplines of chemistry› Organic chemistry vs. physical chemistry› Very clear sub-cultures with respect to research

practices

Differences between disciplines- Are the differences real and if so, where do they

exist?- Tailor RCR programs for highest effectiveness

Institutional Project: Research Integrity Office

IRB protocol: pre- and post-assessments of current RCR training efforts- Tuned to areas of difference- Chemistry course and introductory seminars-initial

populations

Some questions- Are there core areas we can identify that are

discipline-independent?- Are there areas that are discipline-specific?- Are there more or less effective methods of teaching

RCR and do they differ between populations?- What are effective methods for assessment of RCR

training?

Pilot Program: Preliminary Data Pre-survey

- Developed from F. Macrina, Research Integrity › Mostly “yes” or “no” questions

- Likert-scale responses› Behavioral – “Not justifiable” – “Always justifiable”› Statements: strongly disagree – strongly agree

- Where does training occur?

Two different disciplines (populations)- Chemistry (CHEM) graduate students

(n=27/28)- Health and Exercise Science (HES) graduate

students (n = 14)

Population 1: The “Culture” of Chemistry

Fairly traditional, laboratory science- Hierarchical structure

› PI (Faculty member)› Postdocs, graduate students, undergraduates

- Mentor-protégé relationships

Sub-disciplines- At least 4 different major subdivisions

› Each has their own culture, norms and practices› More subtle subdivisions

- Cross-disciplinary studies› Many students/research projects now crossing

boundaries

Demographics of CSU Chemistry Department1

Student Community- Total enrollment : ~150 students - >90% Ph.D.- Most (~90%) are “traditional”

Gender- ~2:1 ratio of men to women (grad students)- 7/27 (26%) female regular faculty (5 tenured)

Nation of Origin/Ethnicity- ~16% non U.S. citizen- ~10% underrepresented minority

1Based on institutional data, http://www.ir.colostate.edu/

Population 2: The “Culture” of HES Mixed Research Mission

- Physiology, neurobiology, integrative biology, biochemistry

› “Hard” science› Animal models and human subjects

- Health attitudes and behaviors, social influences› “Softer”-sciences › Psychology and mental health› Human subjects

Shared laboratories/resources- Research teams- Long-term studies

Demographics of CSU HES Department1

Student Community- Total enrollment: ~35 students- ~25% Ph.D. (new program)- Most are “traditional”

Gender- 60% female (grad students)- 3/18 (17%) female regular faculty (2 tenured)

Ethnicity/Race- 100% non-minority, non-Hispanic

1Based on institutional data, http://www.ir.colostate.edu/

Demographics – Survey Respondents

Major population differences- Year in program: MS vs. Ph.D.- Gender

Very little ethnic/racial diversity

Survey Results: Research Misconduct

Fabrication, falsification and plagiarism (FFP)- Data manipulation- Data analysis- Reporting practices- Record keeping

Survey questions- Yes/No questions- Situational or behavioral questions

› Agree/disagree › Justifiable/not justifiable

- Where RCR topics are discussed

Falsifying Data - Behavioral Altering experimental data to make an experiment

look better than it actually was.

Fabricated data – Behavioral• Reporting experimental data that have been

created without actually having conducted the experiment.

Falsification & Data Handling – Yes/No

• Is it ever OK to disregard or modify data points that are part of a data set in order to prove a hypothesis in a research project (e.g. if one point represents an outlier)?

• Is it ever OK to disregard or modify data points that are part of a data set in a laboratory class?

• Do you know how many times any particular experiment must be repeated in order to feel satisfied that the results are trustworthy?

Where Are Students Exposed to RCR Topics?

TopicClas

sPee

rs

In a research group meeting

With advisor/ faculty member

Methods for proper record keeping

Principles for responsible use of animal subjects

Principles for responsible use of human subjects

Importance of honestly reporting what you find

Criteria for authorshipRisks of conflicts of interestThe peer review processResponsibility and strategies for

action after having witnessed research misconduct

Which of the following topics have you discussed in a class, with peers, in a research group meeting or with your research advisor or other faculty member? For each item in the table, check all the boxes that apply.

Where do students learn about data handling?

• Importance of honestly reporting what you find• Methods for proper record keeping

• Some differences exist between populations

• Overall: >80% report discussions in at least 1 venue

Data Analysis: Differences in Populations

Trying a variety of different methods of analysis until one is found that yields a result that is statistically significant.

p = 0.007

Knowledge of Plagiarism

• Do you have first-hand knowledge of scientists plagiarizing the work of someone else?

• Have you ever plagiarized the work of someone else?

Chemistry REU Students

Reporting Misconduct

• Would you report a coworker who you believe has violated scientific integrity standards?

• Would you report a supervisor/advisor who you believe has violated scientific integrity standards?

Reporting Misconduct - Peers• If you witnessed a coworker committing research

misconduct, you would know who would be the appropriate person at CSU to report it to.

p = 0.24

Reporting Misconduct - Supervisors

• If you witnessed a supervisor or principal investigator committing research misconduct, you would know who would be the appropriate person at CSU to report it to.

p = 0.08

Witnessing Misconduct

Responsibilities and strategies for action after having witnessed research misconduct

Authorship and Citation Practices

39% 41%

90%

• Have you ever been an author of a published paper?• Do you believe you have a good understanding of when

and how to credit another person’s words and/or ideas when you use them in your own paper?

• Do rules governing references and citations differ depending on the type of paper you might be writing?

Citations – Do they really understand?

Republishing data without citation of the earlier work.

p = 0.03

Where Are Students Exposed to RCR Topics?

TopicClas

sPee

rs

In a research group meeting

With advisor/ faculty member

Methods for proper record keeping

Principles for responsible use of animal subjects

Principles for responsible use of human subjects

Importance of honestly reporting what you find

Criteria for authorshipRisks of conflicts of interestThe peer review processResponsibility and strategies for

action after having witnessed research misconduct

Which of the following topics have you discussed in a class, with peers, in a research group meeting or with your research advisor or other faculty member? For each item in the table, check all the boxes that apply.

Authors and Authorship Criteria

Nearly 20% of student authors have never had a conversation about authorship criteria

Less than 2/3 of student authors have discussed authorship criteria with a faculty member

Where Are Students Exposed to RCR Topics?

TopicClas

sPee

rs

In a research group meeting

With advisor/ faculty member

Methods for proper record keeping

Principles for responsible use of animal subjects

Principles for responsible use of human subjects

Importance of honestly reporting what you find

Criteria for authorshipRisks of conflicts of interestThe peer review processResponsibility and strategies for

action after having witnessed research misconduct

Which of the following topics have you discussed in a class, with peers, in a research group meeting or with your research advisor or other faculty member? For each item in the table, check all the boxes that apply.

One Major Area of Difference

More than 2/3 Chemistry students report never having had a discussion about animal/human subjects

More than 85% of HES students have discussed human subjects in at least 1 venue (predominantly in classes)

Preliminary Conclusions Some clear differences between the two

populations Have we answered any of our questions?

- Are there core areas we can identify that are discipline-independent?

› Data handling, reporting and record keeping› Others?

- Are there areas that are discipline-specific?› Animal/human subjects› Analysis methods?

- Are there more or less effective methods of teaching RCR and do they differ between populations?

- What are effective methods for assessment of RCR training?

Next Steps Pre-survey:

- More analysis of data- Revise and modify questions

Additional data from other disciplines- Computer science- Psychology (matches with HES)- Engineering programs- Veterinarian Medicine

Develop post-RCR training survey- Administer at least 6 months after training

(annually?)- Focus on awareness

Developing a Culturally-Competent Institutional

RCR ProgramThe Action Plan

CSU’s RCR Efforts Represent a Mosaic

Goal 1Construct a Uniform and Elevated “Floor” of RCR

Training

Implementation of Section 7008 and 7009 of the America COMPETES Act is going to “stimulate” the laying of this floor …

Goal 2: Facilitate the Building of Discipline-Specific RCR

Training

Chemistry

Exercise Sciences

Campus-Wide Initiative Small working group on implementation of

RCR training program (March) Campus-wide workshop (June)

- Undergraduates and graduate students, postdocs

- Faculty and staff- Administration- RCR instructors- Compliance Committees (IRB, IACUC, IBC)- The Teaching & Learning Institute- Business & Finance folks- Ethicists- Nay-sayers

Goals of CSU Workshop Training (Dr. Ken Pimple’s keynote address) Talking (about how RCR training enhances

trainees’ experiences and their future success) Listening (to concerns and solutions) Identifying “local values”

- What is our repertoire of “core” competencies?- Who do we want to teach it to?- How do we want to teach it?- How are we going to verify teaching to meet

mandates?- How are we going to assess its efficacy?

Building consensus on a path forward

Guiding Principles1. The goal of RCR training is not to prevent MiS

and not to make better citizens (although both are admirable goals). The goal is to make better science (and better scientists).

2. An RCR training program, no matter how effective, fails if it also impedes scientific productivity.

3. Cultural competency is much more complicated and important than we first imagined: the critical details of planning an RCR training program are predicated on a careful analysis of what our local “culture” is.

Acknowledgments People

- Prof. Lisa Dysleski (Chemistry)- Prof. Dawn Rickey (Chemistry)- Molly Gutilla (RICRO)

Financial Support- Office of the Vice President for Research- College of Natural Sciences- NSF-REU/Ethics supplement (NSF-0649623)

Data Reporting: pubs vs. grants

• It is more important that data reporting be completely truthful in a publication than in a grant application.

Fabricated data: Authors share blame

• If fabricated data are discovered in a published paper, all coauthors must equally share in the blame.

Fabricated data: Authors share punishment

• If fabricated data are discovered in a published paper, all coauthors must receive the same punishment.