Paper ID #11182

A Mastery Learning Approach to Engineering Homework Assignments

Dr. Jacob Preston Moore, Pennsylvania State University, Mont Alto

Jacob Moore is an Assistant Professor of Mechanical Engineering at Penn State Mont Alto.

Dr. Joseph Ranalli, Pennsylvania State University, Hazleton Campus

Dr. Joseph Ranalli has taught since 2012 as an Assistant Professor at Penn State Hazleton in the Alterna-tive Energy and Power Generation Engineering program. He previously earned a BS from Penn State anda PhD from Virginia Tech, both in Mechanical Engineering. Prior to his current appointment, he served asa postdoctoral research fellow at the National Energy Technology Lab in Morgantown, West Virginia. Dr.Ranalli’s current research interests include development of tools and methods for solar energy resourceassessment and the role of technology in engineering pedagogy.

c©American Society for Engineering Education, 2015

Page 26.64.1

A Mastery Learning Approach to Engineering Homework Assignments

1. Introduction:

In many engineering courses, homework assignments are intended to be active learning

experiences, where students are asked for the first time to grapple in depth with the concepts and

methods discussed in class. For this knowledge building experience, formative feedback that

allows the student to test and refine their own knowledge would be ideal, but the assessment

practices in many courses treat homework assignments as a summative assessment, providing

only a single, final judgment that does not allow the student to refine their solutions at all.

In this paper the authors discuss a method used to help refocus the evaluation of homework

assignments in core engineering theory courses as formative assessment activities, encouraging

students to master the content presented in the assignments. The mastery based approach

encourages students to master the content by focusing on feedback and refinement of student

solutions, rather than on grades. To implement this goal, the mastery grading system used an all

or nothing grading scheme for each problem, but offered instructor feedback and allowed for

unlimited resubmission of assignments without penalty. This rewarded mastery of the problems,

encouraging students to continue working on the problems until they had completely and

correctly answered every aspect of the problem.

Because this method had not been previously implemented in the classroom, the authors sought

to conduct an exploratory study examining the feasibility of such a setup in the engineering

classroom. Through this study the authors sought to answer the following questions:

How will the mastery grading setup affect student homework behaviors and student

performance on homework assignments?

What are students’ opinions of the mastery grading setup?

What time commitments are required from students and from instructors to implement

the mastery grading setup?

To answer these questions, the authors discuss the literature and theory behind the method in

Section 2, the methods used to implement and evaluate the mastery grading setup in Section 3,

the results in Section 4, the conclusions and their implications for practice in Section 5, and

finally the limitations of the work and the potential for future work in Section 6.

2. Literature Review:

2.1 Formative Assessment:

Assessment in education serves to provide feedback on student understanding to both the student

and to the instructor. By providing this feedback in a timely fashion during an instructional unit,

it can serve to help the student identify topics that they do and do not understand and adjust plans

Page 26.64.2

accordingly, and it can serve to help the instructor do the same. When assessment fills this role, it

is known as formative assessment [1]. High quality feedback provided through formative

assessment has long been known to be an essential part of the learning experience [1]–[3].

Another role that assessment can also fill is to verify that at the end of an instructional unit, the

student sufficiently understands the content. When assessment fills this role it is known as

summative assessment [1]. Though summative assessment is also an essential part of the

educational system, it is not designed to aid student learning. Table 1 highlights the common

differences between formative and summative assessments.

Table 1: A comparison of formative and summative assessments

Formative Assessment Summative Assessment

Measures student understanding in order to

adjust instruction to improve student learning

Measures student understanding in order to

determine the level of success in reaching the

learning objectives of the course

Provided in the midst of the instructional unit Provided at the conclusion of the instructional

unit

Usually constitutes only a small portion of the

students final grade if any

Usually constitutes a large portion of the

student’s final grade

Because homework assignments are often viewed as a way for students to learn the content in the

course, rather than a final evaluation of their understanding, the feedback on homework

assignments would ideally serve as a form of formative assessment. In practice though,

homework assignments are often evaluated as a single measure of student understanding at a

single moment in time. This more closely matches the summative assessment approach, when a

formative assessment approach would be better matched to improving student learning.

To make this transition from summative to formative, the focus must be shifted from feedback

that measures success versus failure (i.e. the grade on the assignment) to content based feedback

that identifies how the student can improve their understanding. Though this content based

feedback may be offered to the student in addition to the grade on the assignment, there is often

little immediate incentive for the student to pay attention to this feedback. In many cases it may

be weeks before the student is asked to demonstrate proficiency in the topic area again, even if

they demonstrated major gaps in their knowledge.

2.2 Existing Mastery Based Assessment Systems:

Though no existing literature was found discussing a system that was identical to the mastery

grading system discussed in this work, prior work has been done to offer high quality formative

assessment in classrooms which has informed the design and implementation of the mastery

grading system described in this paper. The literature that is relevant to this research generally

fits into one of three categories: rapid feedback systems (often called clickers), automated

homework assessment systems with resubmission, and resubmission policies with course exams.

Page 26.64.3

Rapid feedback systems ask students to answer multiple choice questions posed in class. The

students’ responses to these questions are collected either via an electronic device or via flash

cards. The instructor can then use the aggregated student responses to adapt instruction to

address any common misconceptions. This method has been shown to improve student learning

[4]–[6], especially if integrated with peer discussion.

Though the method has been shown to be a valuable form of formative assessment offering

instant feedback to the students, the method also has its drawbacks. Because of the nature of the

data collection and aggregation systems, it only works well with multiple choice questions which

may limit the complexity of the questions asked. In addition, even if long and complex questions

are worded as multiple choice questions, having the students successfully solve these problems

will require a significant amount of in class time, as these systems are not designed to work

outside of the classroom.

Another widely used form of formative assessment is an automated homework assessment

system where students are allowed to resubmit. In these systems, students submit electronic

homework assignments to a computer program that automatically evaluates and offers

instantaneous feedback on the student submission. After receiving the feedback, students are

often allowed to redo the problem or at least a similar problem and resubmit that to the program

for evaluation. Only the grade for the last submission counts as part of the homework grade.

When properly implemented, these systems have been shown to have a positive effect on student

learning [7], [8].

As with the clicker-based approach, this method also has a few drawbacks. First, the method

works best with programming assignments that can most easily be assessed via computer

program. Second, the development of an intelligent tutoring system that can offer valuable

feedback to the student takes significant time on the part of the instructor, or requires monetary

investment to access commercial tools.

Finally, though it is less common than the previously discussed methods, some engineering

educators have implemented policies that allow students to retake in class exams where new

scores replace old scores [9], [10]. This system was also implemented with the intention of

making assessment more formative, though the differences in perceptions between in class

exams and out of class homework may make a difference for students. Though the data on

student learning is limited in these studies, they did indicate that students reported giving more

attention to instructor feedback [9] and that the mastery system was more “fair” [10], showing

promise for the methods and echoing similar results found in this study.

Because the focus of this project was on homework assignments conducted outside of the

classroom, the design of the automated assessment systems were used as a starting point, though

the evaluation itself would not be automated. The authors instead focused on emulating the

process of offering feedback to the students and asking students to fix any mistakes and resubmit Page 26.64.4

the assignments. Though the feedback would not be instantaneous, it was hoped that the

feedback and resubmission process would still be valuable as a formative assessment practice.

3. Methods:

3.1 Study Population:

The mastery grading system was implemented in two core engineering theory courses (one

engineering dynamics class and one thermodynamics class) at different college campuses with

two different instructors in order to get a broad view of the implementation. Both classes were

relatively small with five total students in the dynamics class and eighteen in the

thermodynamics class. To have a basis for comparison, data was also collected from two other

engineering science classes (with the same instructor and at the same institution as the mastery

based dynamics class). Each of these classes enrolled six students.

Table 2: Overview of Research Participants

Course

HW Grading

System

Campus Instructor Number of Consenting

Participants

Thermodynamics Mastery Campus A Instructor A 16

Dynamics Mastery Campus B Instructor B 5

Thermodynamics Traditional Campus B Instructor B 6

Strengths of Materials Traditional Campus B Instructor B 5

Students in these sections were told about the purpose of the study and consent forms were

distributed, asking students to either sign to consent or not consent to the study. The consent

forms were collected by a student who placed them in a sealed envelope that was not opened by

the instructor until after final grades were assigned. This was done to eliminate possible bias in

grading of students due to consent in the study or not. Though the mastery grading arrangement

was used for all students in the course as part of the instructor’s regular teaching responsibilities,

data for non-consenting participants was removed at the end of the semester before analysis. Of

all possible participants, twenty one out of twenty three possible students (91%) consented as

part of the mastery grading sections and all eleven students (100%) consented as part of the

control sections. At campus B, many students were enrolled in more than one class with

instructor B (a common occurrence at small campuses), and thus may have been part of both

experimental and control groups. Since student performance in each class is evaluated

independently, each student’s performance in each class was treated as a separate measure.

3.2 Mastery Grading Implementation:

The mastery grading system was implemented in the two classes described above across the

entire spring semester of 2014. The procedure was introduced on the first day of class and also

explained in the course syllabus. The complete procedure used for the mastery grading system

was as follows:

Page 26.64.5

1. Problems from the course textbooks [11]–[13] were assigned to the class and due on a

weekly basis. Most assignments consisted of 3-5 open ended problems. At the assigned

due date the assignments were collected from the instructor.

2. The instructor evaluated the student assignments and provided one of the three following

marks for each problem.

a. Mastered: The problem was answered correctly and completely.

b. Not Mastered: The problem has one or more errors, or analysis was incomplete.

c. Not Attempted: The student made no significant effort in trying to solve the

problem.

In addition to the each of the three possible marks, students receiving the not mastered

mark were also given feedback on any mistakes or gaps in their analysis, guiding them

towards the correct and complete solution.

3. Within one week, these marks were recorded by the instructor, and the assignments with

the marks and the feedback were given back to the student.

4. The student was then given one week to use the feedback to correct any problems marked

“not mastered”. Students would rework those same problems on a separate sheet of paper

and turn in the resubmissions stapled to the front of the original assignment.

5. The instructor would then grade the resubmissions, giving students one of the same three

marks as in the original process, as well as written feedback for any problems still

marked not mastered.

6. Students were allowed to resubmit in this fashion until they had either mastered all the

problems or failed to turn in a resubmission on time. In theory this would allow for

unlimited resubmits, but in practice no student turned in any assignment more than three

times (one original submission plus two resubmissions).

7. At the end of the semester, the student’s homework grade (20% of the total grade in

Dynamics and 15% of the total grade in Thermodynamics) was the number of problems

that the student mastered divided by the total number of problems assigned. This meant

that students received no credit for a problem until it was mastered, but there was also no

penalty for mistakes, so long as the student corrected those mistakes in the resubmission

process.

The overall process was designed to encourage students to work to perfect the solution to each

problem, without penalizing students for the amount of time they took to master each problem.

As the semester progressed, the process was streamlined in two ways.

First, for multi-stage problems, the instructors would mark the initial sections of the problem as

correct. The student was then allowed to redo the problem from the end of that section, rather

than having to rewrite the initial, already correct, portion of the problem. This saved both the

student and the instructor time in the resubmission process, and eliminated what many students

saw as meaningless copying of previous work. Second, when a large portion of the class made a

particular mistake, the instructor would address the issue in the next class, rather than writing the

Page 26.64.6

same comments out on many assignments. This served to save time for the instructor and

allowed for more dialogue on particularly difficult concepts. Both of these changes were

implemented early in the semester in both classes, and it is not believed that either of these

adaptations fundamentally changed the mastery grading system.

3.3 Mastery Grading Evaluation:

To answer the original research questions posed in Section 1, multiple types of data were

collected over the course of the semester.

First, to understand how students used the mastery grading system and to understand the

effect the system had on student grades, student grades were recorded over the course of

the semester, including the grades at each step of the resubmission process.

Second, to understand student opinions of the mastery grading system, semi-structured

interviews were conducted with seven of the consenting participants at the close of the

semester.

Third, Instructor A (who was new to mastery grading), kept a journal of his experiences

and perceptions of the grade for mastery system to help gather instructor opinions of the

system and to help inform other instructors new to mastery grading in the initial

implementation of this system.

Finally, to better understand the time commitments required for the implementation of

this system, the instructors recorded the time spent grading each assignment.

Grade data was collected from all students in the sections involved in the study by the course

instructors. This included data from each step in the resubmission process for homework

assignments, in order to see how students progressed from one step to the next. This was done as

part of the normal instructional process, though non-consenting participants were removed from

the data set and all data was anonymized before analysis was conducted. The data was then

aggregated so that patterns could be looked for in the data.

After final grades for the class were assigned and the consent form packets were opened,

students who consented were contacted to participate in a semi-structured interview to discuss

the mastery grading system. As an incentive for participating in the interview, one student was

randomly chosen to receive a $25 Amazon gift card. The semi-structured interview format

followed an interview guide (Appendix A), but also allowed the interviewer to follow up on

topics that arose spontaneously during the interview.

Finally, the instructors in the two courses took detailed notes on how long they spent grading

each assignment and each stage of the resubmission process for each assignment. This did not

include the development of the problem solutions, which would be the same in both instances,

but it did include any time needed to enter grades into the course management system, as this

varied across approach (more time to reenter grades). This data was collected in log books by

Page 26.64.7

both instructors, noting the time they spent grading and the assignment and the number of

submissions they were grading. The results from these logbooks were then aggregated at the end

of the semester and used to gain insight into how the instructors spent their time grading in the

courses.

4. Results:

4.1 Student Opinions:

For the results we will start by discussing the student opinions of the tool, as this informs some

of the other results. As reported through the interviews, student opinions of the mastery grading

system were all very positive. All seven of the students who participated in the interviews said

they preferred the mastery grading setup over a more traditional homework grading setup. As

justification for the preference for the mastery grading setup, two common themes could be

identified in the interviews.

First, the majority of the students mentioned that they liked the process of focusing on and

correcting what they got wrong initially. They felt that it helped them to learn and that it helped

their course grade in the end because misconceptions could not be ignored.

“It [mastery grading] really put more emphasis on when you didn’t get something in the

homework you would go back and fix it. It just kind of helped learn it more because…

well normally you get your grade and either you are like well I got it or I failed it and

there is nothing I can do about so all I can do is think about what I can study for the test

later.”

Second, many of the students felt the mastery grading system was a more fair way to way to

grade the assignments. Students felt that if they spent the time to learn the material and answer

the problems then they would eventually receive full credit for the assignments. Though some

students were disappointed when small mistakes resulted in a “not mastered” mark on a problem,

they were still motivated to redo the assignment which would eventually get them all the points

for the problem. In the end, the students felt that this system resulted in fewer disputes over

homework grades.

“The system is really fair, it’s not ripping anyone off, and it just lets you get the grade

you earned just a little bit easier, but you learn more at the same time…”

“It [mastery grading] worked well because it’s not going to kill your grade, if you put

the extra time in to redo the questions you got wrong, then you get the points, and when

you do you learn what you didn’t see clearly before.”

Page 26.64.8

4.2 Instructor Opinions:

In addition to recording the student opinions, instructor opinions were recorded on the

implementation of the mastery grading method through the use of the journal kept by Instructor

A over the course of the semester. Instructor B had previously used the mastery grading system

and was helping Instructor A implement the system for the first time in his classes. The journal

was utilized to gather the instructor’s evolving perception of the mastery grading system and to

provide guidance to other instructors who may wish to implement the mastery grading system for

the first time themselves.

Overall, instructor perceptions of the system were positive. The instructor felt that students

generally embraced the new system and there was less tension due to grades on homework.

Additionally, because of the way the mastery grading system was set up, the instructor felt less

pressure to normalize the grades, making sure that the same error always resulted in the same

number of points deducted. For more detailed information on instructor perceptions of the

system and for guidance in implementing the mastery grading system for the first time, see the

companion piece for this paper [14].

4.3 Submission Behaviors:

Next we will discuss the student submission behaviors, examining why students did and did not

utilize the chance to resubmit. It was suspected that different types of students (high achievers,

average students, or struggling students) would utilize the system in different ways. For the most

part however, there was a common factor that many students identified through the interview. If

the student received a “not mastered” mark, they would almost always redo the problem,

regardless of overall performance level.

Some exceptions to this occurred. First, students with poor attendance records and poor records

of turning in the original assignment would also sometimes skip turning in resubmissions. This is

not viewed as a direct result of the mastery grading system, rather a reflection of general poor

participation by a small percentage of students across any system. Second, some students

indicated that if multiple resubmits piled up and important coursework from other classes also

became time consuming, they would sometimes fail to turn in a resubmission. In these cases they

gave priority to other higher stakes assignments from this course and other courses. This piling

up of work was mentioned as a drawback of the system by some students, and more commonly

occurred at the very end of the semester when many classes had major projects due.

4.4 Student Grades:

Students generally felt that the mastery grading system helped their homework grades, and the

data from the actual student grades supports that assumption. Below are two figures showing

comparisons of the homework grades in the classes. The red lines show the average homework

grades with the mastery grading technique (all students across all assignments) after the original

Page 26.64.9

submission, the first resubmission, and the second resubmission. The blue lines show the

homework averages in the classes with traditional grading techniques (across all students and all

assignments). Since there were no resubmissions with the traditional grading technique, grades

are shown as a flat line across all three attempts. Figure 1 compares the homework averages

across the two thermodynamics courses while Figure 2 compares the three classes in the study

taught by the same instructor.

As we can see in Figures 1 and 2, the homework grades with the mastery system started lower

than with the traditional grading system. This makes sense because of the all-or-nothing

approach to grading used with mastery grading, as opposed to the partial credit system used for

traditional grading. By the end of the third resubmission, however, the averages with the mastery

system had been raised above the traditional grading averages. Since the final grades were the

only ones that matter with the mastery grading system, student homework grades were higher

with the mastery grading system in both of these comparisons.

It can also be noticed that the gains seem to be higher between the two thermodynamics classes.

This may point to a confounding factor with the dynamics class. Students interviewed from the

dynamics class indicated that they felt the material in the dynamics class was harder to learn than

0

10

20

30

40

50

60

70

80

90

100

1 2 3

Ave

rage

Ass

ign

me

nt

Gra

de

(O

ut

of

10

0)

Attempt Number

Traditional

Mastery

Figure 1: Comparison of homework grades with the same Class (Thermodynamics) but with two different instructors.

0

10

20

30

40

50

60

70

80

90

100

1 2 3

Ave

rage

Ass

ign

mn

t G

rad

e

(Ou

t o

f 1

00

)

Attempt Number

Thermodynamics (Traditional)

Strengths of Materials (Tradtional)

Dynamics (Mastery)

Figure 1: Comparison of homework grades with the same instructor across three different classes.

Page 26.64.10

the material in the thermodynamics and strengths of materials classes. This difference in

difficulty between courses may explain why the mastery curve is lower for dynamics and why

there are still large gains even for the second resubmission.

4.5 Time Commitments:

Another point that is essential to the feasibility of mastery grading system is the time required of

both the students and the instructors in the course. To evaluate the time commitment of the

students, questions were posed during the interviews to determine if students spent more or less

time on the assignments. To evaluate the time commitment of the instructors, the instructors took

detailed records of the time spent grading assignments and these were compared to time records

from the traditionally graded class sections.

In terms of the time that students spent on the assignments, all interviewed students agreed that

they spent more time with the mastery grading system because of the time needed to redo any

problems that were incorrect. Five of the seven interviewed students agreed that the time they

spent on the original submission was the same as the time they would spend in a traditionally

graded class but that the resubmissions meant they spent more time overall on the problems. One

high achieving student said that they spent less time checking over the original assignment

because they could fix any errors later, and one other student said they spent more time checking

over the original assignment because they did not want to have to redo the problem later.

Students also mentioned that sometimes, when multiple resubmissions piled up in addition to

work from other classes, the workload could become a little hard to manage, but for the most

part students said that the overall workload with the mastery grading system was manageable.

When comparing the time spent by the instructors grading the assignments, the time had to be

normalized to take into account the differing number of students and the differing number of

problems assigned in each class. To help in the comparison, the average time per student per

problem was calculated. This was found by adding up all the time spent grading for a particular

course, and then dividing by the total number of problems submitted to the instructor (including

only original submissions, not resubmissions) times the number of students that submitted that

problem. This gave the average amount of time that the instructor spent grading one problem for

one student (including any time to regrade it). These average times were then used to compare

the time spent on mastery grading to the time spent with traditional grading.

The first and most direct comparison was between the two thermodynamics classes. These

classes had different instructors and were taught at different campuses, but the assigned problems

were kept consistent between the two classes. In this comparison, the traditional grading

instructor spent an average of 101 seconds per student per problem and the mastery grading

instructor spent an average of 52 seconds per student per problem. This means that the mastery

grading system actually took about half as much time per student per problem when comparing

the two systems with the same problem sets but different instructors.

Page 26.64.11

If we compare the three courses with the same instructor (one with mastery grading and two with

traditional grading), we find the opposite of what was found in the multiple instructor

comparison. The mastery grading took an average of 209 seconds per student per problem, while

grading in the traditional courses took an average of 101 seconds and 92 seconds. This means

that the mastery grading took about twice as long as traditional grading with the same instructor

across different courses with different problem sets.

Overall the results from this analysis were inconclusive, as one comparison shows mastery

grading taking significantly more time and one shows mastery grading taking significantly less

time. It is suspected that student performance on problems plays a big role in the time spent

grading those problems. Offering detailed feedback (done with both mastery and traditional

grading here) to those students who made major errors was a time consuming process for

instructors.

A second type of time analysis looked at the time spent grading original submissions versus time

spent grading resubmissions in the mastery grading classes. In this comparison, it was found that

the instructor using mastery grading with thermodynamics spent 86% of his grading time on

original submissions and 14% of his time on resubmissions. The instructor using mastery grading

in dynamics on the other hand spent 69% of his time grading original submissions and 31% of

his time grading resubmissions. Though less time was spent determining how many points

should be deducted with mastery grading, the time spent grading resubmissions would not have

been spent with a traditional grading system. Therefore the best guess of the authors as to the

time commitment of mastery grading uses the average of the two times spent grading

resubmissions. It is estimated that mastery grading takes approximately 23% more time than

traditional grading techniques, though again, the authors feel that more work needs to be done to

better quantify the time commitment required for the instructor in the mastery grading system.

5. Conclusions:

Overall, the mastery grading system was well received by both the students and the instructors in

the courses in this study. Students felt that the system better helped them learn, raised their

grades, and was fairer as a grading system than the traditional one submission homework grading

policy.

Generally students at all levels worked to improve their solutions until they were able to

correctly and completely answer the problems within three tries. In working to redo problems

that they had not completely and correctly answered, students spent more time examining and

then acting on the content of the instructor feedback. This matches the original goal of the

mastery grading system, trying to shift the focus in homework from grades to the content of the

instructor feedback.

In terms of the time commitments, mastery grading does require more time from both the

students and the instructors, though the extra time commitment from the students was generally

Page 26.64.12

described as manageable and the extra time required for the instructor was tentatively estimated

to be only 23% more than traditional grading.

Overall this study shows promise for the mastery grading system, and the authors believe that the

extra time spent with the mastery grading system is more than worth the improvements in

student learning and satisfaction.

6. Limitations and Future Work:

This work represents an exploratory experience with the mastery grading system, and although

there is promise in the mastery grading system, there is still a lot of work to be done.

First and foremost, this study showed that students felt they learned more, but no direct

comparison of student learning was made in this study. More work needs to be done to

objectively measure the learning gains associated with the mastery grading system described in

this work.

Second, with class sizes ranging from five students to eighteen students in this study, the

feasibility of this system in large classes of a hundred students or more is still uncertain. In

addition, the statistical power of the results of the study is limited because of the sample size. A

study looking at how this system might work in larger classes would help to both increase the

statistical power and determine how this system might be scaled up in larger classes.

Third, it was evident in the classes that some students had access to the solution manuals for the

course textbooks. This is not uncommon for engineering students [15]. What is unknown is how

access to solution manuals may impact the mastery grading process or how the mastery grading

process may encourage or discourage students from using these solution manuals.

Finally, there is room to explore the social aspect of the mastery learning system. Students in the

class were encouraged to work together on the homework assignments, and many students

worked with friends to complete the assignments. It would be interesting to examine interactions

between students who had mastered and not mastered problems, to see if any sort of peer

tutoring occurs and to see how the mastery grading system would impact those relationships.

7. References:

[1] D. R. Sadler, “Formative assessment and the design of instructional systems,” Instr. Sci., vol.

18, no. 2, pp. 119–144, Jun. 1989.

[2] E. L. Thorndike, The Psychology of Learning. Teachers College, Columbia University, 1913.

[3] J. D. Bransford, A. Brown, and R. Cocking, How People Learn: Brain, Mind, Experience,

and School, Expanded. Washington, D.C.: National Academy Press, 2000.

Page 26.64.13

[4] J. C. Chen, D. C. Whittinghill, and J. A. Kadlowec, “Classes That Click: Fast, Rich

Feedback to Enhance Student Learning and Satisfaction,” J. Eng. Educ., vol. 99, no. 2, pp.

159–168, 2010.

[5] S. P. Brophy, P. Norris, M. Nichols, and E. D. Jansen, “Development and initial experience

with a laptop-based student assessment system to enhance classroom instruction,” in

American Society of Engineering Education Annual Conference, Nashville, TN, 2003.

[6] S. W. Draper and M. I. Brown, “Increasing interactivity in lectures using an electronic voting

system,” J. Comput. Assist. Learn., vol. 20, no. 2, pp. 81–94, 2004.

[7] L. Malmi and A. Korhonen, “Automatic feedback and resubmissions as learning aid,” in

IEEE International Conference on Advanced Learning Technologies, 2004. Proceedings,

2004, pp. 186–190.

[8] A. Mitrovic, “An intelligent SQL tutor on the web,” Int. J. Artif. Intell. Educ., vol. 13, no. 2,

pp. 173–197, 2003.

[9] S. Sangelkar, O. M. Ashour, R. L. Warley, and Oladipo Onipede Jr., “Mastery Learning in

Engineering: A Case Study in Statics.”

[10] R. L. Armacost and J. Pet-Armacost, “Using Mastery-Based Grading to Facilitate

Learning,” in Frontiers in Education, 2003. FIE 2003 33rd Annual, 2003, vol. 1, pp. T3A–

20–5 Vol.1.

[11] W. F. Riley and L. D. Sturges, Engineering Mechanics: Dynamics, 2 edition. New York:

Wiley, 1995.

[12] W. F. Riley, L. D. Sturges, and D. H. Morris, Mechanics of Materials, 6 edition.

Hoboken, N.J: Wiley, 2006.

[13] S. R. Turns, Thermodynamics: Concepts and Applications. Cambridge University Press,

2006.

[14] J. Ranalli and J. P. Moore, “New Faculty Experiences with Mastery Grading,” presented

at the ASEE Annual Conference, Seattle, WA, 2015.

[15] J. Widmann and K. Shollenberger, “Student Use of Textbook Solution Manuals: Student

and Faculty Perspectives in a Large Mechanical Engineering Department,” in Proceedings of

the 2006 American Society for Engineering Education Annual Conference & Exposition,

Chicago, IL, 2006.

Page 26.64.14

Appendix A: Semi-Structured Interview Guide

Hello and thank you for speaking with me… (explain the purpose of the interview and ask if they are

okay having the conversation recorded)

Background:

Did you enjoy the course?

How much do you feel you learned in the course?

Do you feel that you did well in the course?

Opinions and Usage of the Grade for Mastery Technique:

Tell me about your opinions of the mastery grading technique?

Tell me about your positive experiences with the setup.

Tell me about the negative experiences with the setup.

Explain how your experiences differed from your courses with a more traditional setup.

Do you feel that the setup helped or hindered learning?

Do you feel that the setup helped or hurt your course grade?

Do you feel you spent more or less time on your assignments because of the

system?

How often did you resubmit with this system?

What was the cutoff point for resubmitting (was there ever a point at which you felt it

was not worth it to resubmit)?

Were you more or less likely to pay attention to the feedback on the assignments?

Any other comments you have about the grade for mastery setup?

How would you change the setup if you were teaching the course?

Page 26.64.15