The Whistleblower Effect:

A Quantitative Analysis Demonstrating that Reporting Students for Academic Dishonesty Negatively Impacts Faculty Evaluations

Mihran Aroian

Presentation Outline- Student Judicial Services Overview- Faculty-in-Residence program- Faculty Perceptions- Why Conduct this Research- Results- Recommendations- Questions

SJS Overview

Student Judicial Services

As a part of The University of Texas at Austin’s Office of the Dean of Students, Student Judicial

Services (SJS) fosters moral development on campus through the resolution of academic integrity and conduct-related matters as well as educational

outreach to students, faculty, and staff.

In addition to our educational component, SJS investigates alleged violations of the Institutional

Rules (both academic and non-academic) and implement the disciplinary process with a focus on

student learning and development.

Primary SJS Roles• Outreach/Education• Faculty Consultations• Case Management

• 1,600-1,800 annual referrals (AI and conduct)

• Institutional Rule Revision

Some Statistics 3,300 faculty 2,500 TA’s 50,000 student body 140 academic departments 2011-2013 academic years

320 faculty/TA’s reported cases to SJS 1,155 AI cases over 2 years 50% of all submissions came from (can you

guess how many faculty)? 15 out of 320

SJS Data Administrative disposition versus

Faculty Disposition – 55/45 Top 3 AI violations

Cheating, Plagiarism, Collusion Classification

Freshman (10); Sophomore (20%); Junior (20%); Senior (35%); Grad/Professional (15%)

GPA – even distribution Gender (M/F) – 65/35

Establishing a Faculty-in-Residence

Program at UT Austin

Academic Integrity As Steve Covey says “seek first to

understand, then to be understood” Improving AI is simple

“just as plain as the nose on your face” For example:

What is the shape and color of a Stop Sign? What is the shape and color of a Yield Sign?

Benefits to SJS Dedicated outreach time not available to typical

conduct staff Cultivation of relationships outside of “crisis”

events typical in conduct offices Increased exposure through meetings with

stakeholders previously unidentified by SJS Ally outside of the office to speak with unaffiliated

individuals Addition of a “faculty perspective” to conversations

within the conduct office Discussions at the Macro-level (student conduct

as a field) rather than at the Micro-level (a student’s conduct)

Benefits for Me! Greater appreciation for SJS Understanding the complexity of student

conduct Part of the team Ability to cross boundaries Support SJS with other stakeholders Educate campus stakeholders regarding SJS Involvement with student groups Being part of the solution Satisfying my own needs as an educator

Faculty Perceptions Lack of understanding by faculty as to

what constitutes cheating. Not all faculty engaged in high AI

standards. Faculty unaware of common forms of

cheating. Faculty unwilling to expend effort to

meet with students and file paperwork. Faculty avoid student conflict/tension AI is an afterthought in the class

Faculty Perceptions Lack of consistent message to

students. High level of indifference by many

campus stakeholders. Some faculty do not care because they

are not rewarded for teaching. Faculty need assistance on identifying

cheating and how to deal with students who do cheat.

The Whistleblower Effect:  A Quantitative Analysis Demonstrating that Reporting Students for Academic Dishonesty Negatively Impacts Faculty Evaluations

Authors: Mihran Aroian Raymond Brown

Scenario A Three students submit essentially the

same writing assignment. Student A provided paper to student B & C Student B & C take responsibility Student A believes he is innocent Students upset at instructor Students are found in violation Students are members of the same student

organization as are other students in the class

Scenario B A group presentation of six students

includes significant plagiarized material Initially, nobody takes responsibility One-on-one conversations Students start throwing each other under the

bus Case sent to SJS for investigation Investigation completed the following

semester Two of the six found in violation

Question Will the end-of-semester evaluations

for these instructors increase, decrease, or not be affected by this situation? Scenario A? Scenario B?

Why This Study? Interviewing stakeholders on campus

as the faculty-in-residence Faculty were hesitant to report students Faculty want to avoid conflict/confrontations Belief that reporting students would impede

professional advancement As someone who always reports, I was

shocked! Original hypothesis

Perception by faculty was a false perception and analysis would demonstrate no correlation

Introduction Examine the impact of sanctioning

students for acts of academic misconduct on student evaluations of teaching (SETs). use of student evaluations for faculty

advancement and promotions faculty may be conflicted between enforcing

academic integrity and maximizing their professional advancement with high SET scores.

Method A multi-level modeling design with

8,940 end-of-semester student evaluations

32 faculty members from 17 academic departments was employed to determine if there was a whistleblower effect.

Literature Search Extensive qualitative articles Lack of quantitative analysis Only quantitative studies focused on

grade inflation and higher SETs GPAs have been increasing while SAT

scores have been decreasing

Genesis of Research Design Initial evaluation - faculty against peer

instructors Compare faculty against all faculty

within department Compare faculty against all faculty

within college Final analysis – compare faculty

against themselves

Creation of Sample Compare faculty against themselves Analyze evaluations for course in a

semester when they reported cheating Analyze evaluations for same course in

a semester when they did not report cheating

Evaluation must be for the same class Looked for the closest semester in

temporal proximity - summers excluded Must have reported 3 or more cases

Challenges to Methodology Highest reporting instructors reported

cheating every semester therefore were not included

Cases had to be submitted to SJS prior to 10 days before the end of semester Course Instructor Survey’s are given up to 10

days before the end of the semester

The Course Instructor Survey Five point scale ranging from “strongly

disagree” to “strongly agree” Questions asked on survey

The course was well organized The instructor communicated information

effectively The instructor showed interest in the progress of

the student The tests and assignments were usually graded

and returned promptly The instructor made me feel free to ask

questions, disagree, and express my ideas

Survey Overall rating for the instructor and

course ranging from “very unsatisfactory” to “excellent” on a 5-point scale Overall, this instructor was Overall, this course was

Key criteria for performance evaluation is the overall instructor rating

Limitations of Analysis Results based on data collected at one

university Unable to link evaluations to students Unable to test highest reporting faculty

Results Statistically significant effect for

reporting students (p<.001) Estimated mean rating for classes in

which students were not reported was 3.95

Estimated mean rating for classes in which students were reported was 3.75

Hypothesis - instructors who report students tend to be more demanding of their students in general than those who do not report

Discussion Is it time for a policy review?

Instructors expected to enforce AI Colleges want to instill ethics – beyond college By not enforcing, instructors are implicitly

condoning cheating and undermining accurate student assessment

Evaluations play a critical role for advancement for both tenure-track and non tenure-track faculty Do you harm yourself when you enforce AI?

Student Responsibility UT participates in the National

Assessment of Student Conduct Adjudication Process (NASCAP) Project 21 higher education institutions participate Upon completion of adjudication, students

complete survey When asked the question “What was the

outcome of your case” 6.9% of UT students answered “my case was dismissed”

In reality, only 0.15% of the cases were dismissed

Recommendations Encourage all faculty to communicate

expectations of AI Provide regular communication to

faculty regarding: How students cheat How to deal with cheaters How to confront students Best practices How to reduce cheating

Require students to complete an online AI tutorial each year

Recommendations Encourage faculty to increase

frequency of message Each academic department should

have an AI point person Faculty should regularly update

assignments and exams

Recommendations Include a question about faculty AI

expectations and practices on evals AI should be part of new faculty

orientation Administrators should include

consideration for faculty who report cheating regarding merit review and promotions

http://www.dontmesswithtexas.org/about/the-campaign/ads/1989_willienelson.php