reflective analysis of student learning in a sophomore engineering course

9
PHILLIP C. W ANKAT School of Chemical Engineering Purdue University West Lafayette, IN 47907-1283, USA ABSTRACT Student learning in a specific sophomore engineering course was analyzed using a variety of learning theory explanations. Bimodal grade distributions were observed in most of the examinations. The following theories were applied to these results: knowledge structure, Piaget’s concrete and formal operational stages, Myers- Briggs intuitive versus sensing, Perry’s model of college student development, and deep versus shallow approaches to learning. The deep versus shallow approach to learning theory appears to best ex- plain the results. Problem solving approaches are reviewed in light of the results obtained. Suggestions for improving the course are made, and compared to the results obtained when the course was re-taught. This analysis serves as both an example of application of learning theories and as a review of these theories. I. INTRODUCTION Professors have been encouraged to think reflectively about courses 1, 2 and to use the results of this reflection and of classroom assessments 3 to improve the course and subsequent offerings of the course. The new ABET Criteria 2000 requires assessment to demonstrate course outcomes, and one of the four major objectives of the ABET process is the “improvement of engineering educa- tion.” 4 Many different parts of teaching can be analyzed and reflect- ed upon. What form should these reflections take for engineering classes? Since student learning is the ultimate goal of the course, it is logical that student learning should be one of the primary objectives of any organized reflection of a course. A variety of theories of student learning has appeared in the ed- ucational literature. What appears to be missing in engineering ed- ucation is an extensive example of applying these theories in the re- flective assessment of a course. This reflective assessment can be used to determine how well students are learning and for course im- provement. What form should these reflections take for engineer- ing courses? This paper examines a large sophomore engineering course using a variety of learning theories. Since an inductive approach is known to be more effective for learning new material than a deduc- tive approach, 5 this paper should be particularly useful for readers who are unfamiliar with all of the theories. II. THE COURSE In Spring 1997 I taught Chemical Engineering (CHE) 205, Chemical Engineering Calculations, starting with 72 students at Purdue University. This is normally the first chemical engineering course and at Purdue is usually taken by sophomores. This course and equivalent courses at other universities have a long history of being difficult for many students. 6 The textbook used was Elemen- tary Principles of Chemical Processes 7 which is the most popular textbook for beginning chemical engineers in the US. The course covers mass and energy balances and is in some sense similar to other courses balancing different things (e.g. balancing electrons in circuits or balancing money in accounting). [After this paper was written and submitted, I re-taught CHE 205 in Spring 1998 start- ing with 76 students. Many of the recommended changes were tried in 1998. The 1998 results are added as bracketed items.] I had taught this course 22 years ago at Purdue University and the University of California-Berkeley. After this long of a time, in many ways the assignment was similar to teaching a new course. I used a new textbook and had to adjust myself to teaching sopho- mores again. However, I did have the advantage of a base to com- pare student learning, and it is easier to see changes when one has been away from something for a long period. The course was structured with a large number of opportunities for student practice and a large number of evaluations of the stu- dents. Seven quizzes totaled 40% of the course grade. The students were told that the two lowest quiz grades would be discarded. [There were five quizzes with one discarded in 1998.] After the third and sixth quizzes, longer and more comprehensive tests were given which accounted for 20 and 25% of the course grade, respec- tively. Since all the quizzes and tests were open book and open notes, new examination problems were composed. The eight homework assignments added together were only 5% of the grade. The students were told that failure to work on the homework would be reflected in low grades on the examinations. The final 10% of the course grade was determined by a group project at the end of the semester. [In 1998 quizzes totaled 36%, each exam 25%, homework 5%, recitation 5% and group project 4% of the grade.] A straight grading scale was used: 85 to 100 = A, 75 to 84.99 = B, 65 to 74.99 = C, and 55 to 64.99 = D. The straight scale was used since it reduces competitiveness and convinces the students that there is no penalty for helping each other. One graduate TA and one un- dergraduate TA were assigned to the course. [In 1998 one graduate and two undergraduate TAs were assigned.] The course project required student groups to write a new mass or energy balance problem using data not contained in the text- book. Students were encouraged to be creative in developing their problem. The deliverables were a written problem statement, a written solution, and a 12-minute oral presentation on the problem April 1999 Journal of Engineering Education 195 Reflective Analysis of Student Learning in a Sophomore Engineering Course

Upload: phillip-c

Post on 07-Apr-2017

214 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Reflective Analysis of Student Learning in a Sophomore Engineering Course

PHILLIP C. WANKATSchool of Chemical EngineeringPurdue UniversityWest Lafayette, IN 47907-1283, USA

ABSTRACT

Student learning in a specific sophomore engineering course wasanalyzed using a variety of learning theory explanations. Bimodalgrade distributions were observed in most of the examinations.The following theories were applied to these results: knowledgestructure, Piaget’s concrete and formal operational stages, Myers-Briggs intuitive versus sensing, Perry’s model of college studentdevelopment, and deep versus shallow approaches to learning. Thedeep versus shallow approach to learning theory appears to best ex-plain the results. Problem solving approaches are reviewed in lightof the results obtained. Suggestions for improving the course aremade, and compared to the results obtained when the course wasre-taught. This analysis serves as both an example of application oflearning theories and as a review of these theories.

I. INTRODUCTION

Professors have been encouraged to think reflectively aboutcourses1, 2 and to use the results of this reflection and of classroomassessments3 to improve the course and subsequent offerings of thecourse. The new ABET Criteria 2000 requires assessment todemonstrate course outcomes, and one of the four major objectivesof the ABET process is the “improvement of engineering educa-tion.”4 Many different parts of teaching can be analyzed and reflect-ed upon. What form should these reflections take for engineeringclasses? Since student learning is the ultimate goal of the course, it islogical that student learning should be one of the primary objectivesof any organized reflection of a course.

A variety of theories of student learning has appeared in the ed-ucational literature. What appears to be missing in engineering ed-ucation is an extensive example of applying these theories in the re-flective assessment of a course. This reflective assessment can beused to determine how well students are learning and for course im-provement. What form should these reflections take for engineer-ing courses?

This paper examines a large sophomore engineering courseusing a variety of learning theories. Since an inductive approach isknown to be more effective for learning new material than a deduc-tive approach,5 this paper should be particularly useful for readerswho are unfamiliar with all of the theories.

II. THE COURSE

In Spring 1997 I taught Chemical Engineering (CHE) 205,Chemical Engineering Calculations, starting with 72 students atPurdue University. This is normally the first chemical engineeringcourse and at Purdue is usually taken by sophomores. This courseand equivalent courses at other universities have a long history ofbeing difficult for many students.6 The textbook used was Elemen-tary Principles of Chemical Processes7 which is the most populartextbook for beginning chemical engineers in the US. The coursecovers mass and energy balances and is in some sense similar toother courses balancing different things (e.g. balancing electrons incircuits or balancing money in accounting). [After this paper waswritten and submitted, I re-taught CHE 205 in Spring 1998 start-ing with 76 students. Many of the recommended changes weretried in 1998. The 1998 results are added as bracketed items.]

I had taught this course 22 years ago at Purdue University andthe University of California-Berkeley. After this long of a time, inmany ways the assignment was similar to teaching a new course. Iused a new textbook and had to adjust myself to teaching sopho-mores again. However, I did have the advantage of a base to com-pare student learning, and it is easier to see changes when one hasbeen away from something for a long period.

The course was structured with a large number of opportunitiesfor student practice and a large number of evaluations of the stu-dents. Seven quizzes totaled 40% of the course grade. The studentswere told that the two lowest quiz grades would be discarded.[There were five quizzes with one discarded in 1998.] After thethird and sixth quizzes, longer and more comprehensive tests weregiven which accounted for 20 and 25% of the course grade, respec-tively. Since all the quizzes and tests were open book and opennotes, new examination problems were composed. The eighthomework assignments added together were only 5% of the grade.The students were told that failure to work on the homeworkwould be reflected in low grades on the examinations. The final10% of the course grade was determined by a group project at theend of the semester. [In 1998 quizzes totaled 36%, each exam 25%,homework 5%, recitation 5% and group project 4% of the grade.] Astraight grading scale was used: 85 to 100 = A, 75 to 84.99 = B, 65to 74.99 = C, and 55 to 64.99 = D. The straight scale was used sinceit reduces competitiveness and convinces the students that there isno penalty for helping each other. One graduate TA and one un-dergraduate TA were assigned to the course. [In 1998 one graduateand two undergraduate TAs were assigned.]

The course project required student groups to write a new massor energy balance problem using data not contained in the text-book. Students were encouraged to be creative in developing theirproblem. The deliverables were a written problem statement, awritten solution, and a 12-minute oral presentation on the problem

April 1999 Journal of Engineering Education 195

Reflective Analysis of Student Learning in aSophomore Engineering Course

Page 2: Reflective Analysis of Student Learning in a Sophomore Engineering Course

and the solution. Groups of four students each were selected by theprofessor. Each group had at least one student with co-op experi-ence (this was considered a key element since many students withco-op experience would have an advantage in writing a novel prob-lem), one student with low grades in CHE 205, one student withhigh grades in CHE 205, one extrovert, and one introvert. Groupseither had two women or no women and groups either had two mi-nority students or none. Each group had at most one student whowas not registered in Chemical Engineering. Since there were stu-dents who were not very active in the course, each group had nomore than one student likely to not work on the project. Groupswith a no show were told to finish the project with three people.The last week of classes and the week of finals were devoted to theproject. On the last regular class day of the semester, the twelvegroups who were ready had their project critiqued by another stu-dent group. The written project reports and the oral presentationswere due six days later during the two-hour period reserved for fi-nals. The class was split into two parts with eight groups in eachpart for the oral presentations. The professor heard half the groupsand the graduate TA the other half. The undergraduate TA splither time between the two parts to serve as quality control in thegrading of oral reports.

Every group member was rated by the other group members forparticipation. The student’s participation grade was based 60% onthe professor’s rating and 40% on the ratings of the other groupmembers. [In 1998 I reversed these percentages.] The professor’srating was based on attendance, being on time, and appearing towork on the project. The final project grade for a student was calcu-lated as:

Student grade = (Fractional participation grade)X(Group Pro-ject Grade)

Most of the students enjoyed the project, put a great deal of ef-fort into it, and learned things they had not realized while solvingproblems someone else posed. The project also was an opportunityto emphasize oral and written communication early in the curricu-lum.

The teaching method used was a combination of interactive lec-tures and small group problem solving. The students were encour-aged to work together on homework although they had to hand intheir own paper even if it was identical to three other students’ pa-pers. The professor and the teaching assistants had office hoursspread out from Monday through Friday. Optional help sessionswere held on Sunday nights by the TAs. An in-class help session bythe professor was scheduled before each examination.

After six weeks students were given a 3x5 card and told to an-swer the question, “What can the instructor or the TAs do to helpyou learn?” Most students answered this question thoughtfully.The cards were collected and the results analyzed. The four mostcommon categories of answers were:

1. Do more examples in class, 2. More group work on problems, 3. Nothing - great class as is, and 4. More individual help. [One of the referees made the excel-

lent suggestion to also ask, “What do you think you could doto improve your performance in this class?” This will, hope-fully, cause some self-reflection.]

The teaching style was adjusted to accommodate these requests.Lectures were made almost entirely inductive so that most pointswere illustrated while an example was solved. Lecture notes with

blank spaces to fill in were handed out to the students in advance.When a number had to be looked up, e.g. from the steam tables,the students were asked to do that immediately, in class. This pro-cedure greatly reduced the number of students who could not solveproblems because they did not know where to obtain numerical val-ues. Additional optional help sessions with group work were pro-vided by the TAs. Slightly more time was used in class for groupproblem solving. In addition, the schedule was revised to cover lessmaterial and allow more time for problem solving. Based on thestudents’ comments included with the end of semester course eval-uation, the professor’s willingness to adjust the course to help stu-dents learn was the one thing that impressed them most about theprofessor.

III. GRADE DISTRIBUTIONS

After three weeks, the students took the first quiz on mass bal-ances. This quiz consisted of two problems and was designed to bestraightforward for any student who understood how to solve massbalance problems. The results, shown in Figure 1, are clearly not anormal distribution as one might expect. Similar results were ob-tained on six of the eight other quizzes and tests.

Were these reasonable examinations? The students had 55 min-utes for each quiz. On the five quizzes that produced bimodal dis-tributions, the first students were finished after half an hour and nostudents complained that they had insufficient time. The largenumber of high grades and perfect papers (e.g. 20 on quiz 1 and tenon quiz 7) show that most of the examinations were very solvable inthe time available. It is interesting that one quiz and the first testwhich very few students finished had grade distributions that wereclose to normal. [This experience was repeated in 1998 on one quiz.The second test, which combined test 2 and quiz 7 from 1997 andhence covered a lot of material, was scheduled during a two-hourblock and was not a race. The resulting distribution was not nor-mal.] When speed is an important factor, it interferes with mea-surement of knowledge and understanding for slow students. Thistends to make the distribution look normal and makes the assess-ment less valid.

One would expect grades to predict grades. Did quiz 1 predictthe grades of quiz 7? With the exception of two students, studentswith very low grades (<55) on quiz 1 also received very low grades onquiz 7. [Results were slightly better, 3 of 8 students, improved on thesecond exam in 1998.] Movement in the other direction was muchmore common since a dozen students with high grades (>75) onquiz 1 received a low grade on quiz 7. Although quiz 7 was clearlymore difficult on an absolute scale, results on quiz 1 were predictive.

How do we understand these results? There are a variety of waysto look at these results which will provide us a window on differentlearning theories.

First, consider nonpsychological explanations. Perhaps there re-ally are two separate distributions of students. The sum of two nor-mal distributions does not result in a normal distribution. Since thespring semester includes students who are behind the normalschedule and students who are ahead of schedule, this explanationcould be true. However, a plot of entering grade point averages(GPA) versus the 205 grades (Figure 2) shows the expected positivecorrelation, with a large amount of scatter. Even if there are twodifferent distributions, one wonders why they are different. Other

196 Journal of Engineering Education April 1999

Page 3: Reflective Analysis of Student Learning in a Sophomore Engineering Course

possible explanations for poor performance include insufficient study(lack of effort), “stupid” mistakes, and panic. Although these expla-nations undoubtedly are valid for a few individual students, they donot explain the bimodal nature of the results shown in Figure 1.

Since time spent studying correlates with learning and grades,8

lack of effort would explain low homework grades. For each finalcourse grade the average number of homework assignments turnedin was: A (7.44), B (7.55), C (6.82), D (4.33) and F (4.00). Al-though homework was only 5% of the course grade, effort onhomework (as represented by the number of assignments turned in)correlated well with final course grades. Obviously, the correlationis not perfect - two of the students who failed the course turned inall eight homework assignments, and one student with an A turnedin only three assignments. This data will be shared with the classnext year. A personal observation: it does appear that studentsworked harder 20 years ago. [This observation was not changed bymy experience in 1998.]

An explanation, which is popular with many engineering pro-fessors, is that the students just are not smart enough. That is,through some combination of genetics (as measured by IQ) and en-vironment the students are unable to do university level work. Ifthis is true, why were most of the students’ GPAs quite respectablebefore they took the first engineering course? In addition, studies byJames R. Flynn in New Zealand show that IQ has been increasingconsiderably9. Comparing these students to those of 20 years ago,my observation is that current students are at least as smart andprobably smarter. Thus, “not smart enough” is not the likely causeof difficulties in engineering courses.

IV. LEARNING THEORY EXPLANATIONS

One explanation based on learning theory is an inadequateknowledge structure to solve the problems. Both an appropriateknowledge structure and general problem solving ability are neededto solve problems.10-12 Clearly, the lack of specific knowledge couldhave caused low grades, particularly for quiz 1. The ability to draw acorrect flowsheet appears to require more knowledge than solvingthe problem once the flowsheet has been drawn. However, sinceCHE 205 is not a content-heavy course, this factor can explain thepersistence of low grades in only a few students who failed the firstquiz, and it does not explain the bimodal distributions observed. Aninappropriate knowledge structure may well explain individual in-stances of low scores on tests by students who were otherwise doingwell. Since students must construct their own knowledge structure,12

it is quite possible for students to have major gaps or errors in theirknowledge structures although everything was “covered” in lecture.The lack of a general problem solving structure could well havecaused difficulties and is discussed later.

Are the students who are unable to do well in these quizzes inPiaget’s concrete operational stage?10, 13,14 The quizzes in this courseare clearly written at the formal operational level and require someability with abstractions. Concrete operational students who cannotuse abstract reasoning will have difficulties in engineering courses.However, the number of students in engineering in the concreteoperational stage is certainly less than 10% and is probably signifi-cantly smaller.14 Thus, this theory can explain the difficulties of afew students, but not the entire sample. Note that a much largerpercentage of students in the concrete operational stage will be pre-

sent in beginning science courses or in an engineering-for-nonengineers course.

Different preferred learning styles might be a fertile area to con-sider explaining this data. The Myers-Briggs Type Indicator(MBTI), which is based on Jung’s theory, has been very popular inthe United States to explain differences in normal people. The di-mension of most interest for learning is sensing (S) versus intuitive(N) types.10, 15-17 The sensing individual prefers a straightforward,logical, step-by-step approach to learning. Sensing students oftenlearn from problems, and find theory difficult. Intuitive students,on the other hand, are willing to skip steps and follow hunches.They often learn from theory and want to do a minimal number ofproblems because they think they understand everything. If you areinterested in typing yourself, self-scored psychological tests similarto the MBTI are available free on the Internet athttp://www.ntlf.com/html/pi/9702/psytype.htm and http://www.keirsey.com/cgi-bin/keirsey/newkts.cgi.

The authors of the textbook used in CHE 205 are intuitive, theprofessor is intuitive, and the graduate and undergraduate teachingassistants are intuitive. Although this resulted in a highly cohesiveteam for lectures, help sessions, and grading, sensing students mayhave been at a disadvantage in this course. [In 1998 the graduateTA was sensing which may have helped.] The students in CHE205 were given the MBTI on a voluntary basis. Fifty students tookthe indicator. Nineteen (38%) of these were intuitive, which is lowcompared to the 53% reported for chemical engineering.10, 16 Theresults are reported in Table 1. The number of students in each cat-egory is listed. The expected value for intuitive students in each cat-egory was calculated assuming the distribution of N and S studentsis the same as the overall N/S distribution of 38% /62%

Except for the category, quiz 7 scores below 55, the intuitive stu-dents always did better on quizzes 1 and 7 than the expected values.The final course grades were evenly distributed except for the with-drawals. It appears that there was a mismatch between the instruc-tors’ styles and the sensing students, which did not affect the finalgrades of students who stayed in the course. However, all five stu-dents who withdrew who had been typed were sensing students.This could be a statistical aberration, but is more likely to be signifi-cant. The professor should include more sensing activities such asstep-by-step algorithmic problem solving in his or her teachingrepertoire. At least in theory,17 K-12 teachers are more advanced atindividualizing instruction based on the MBTI than college profes-sors. Although informative, these results do not explain the bi-modal nature of the examination and final grades listed in Table 1.[Since students did not take the MBTI in 1998, no comparisonscan be made.]

Perry’s model of college student development10,18-21 can also beused to study student learning. According to this model, whichconsists of nine positions or stages occupying four general outlooks,people progress from positions 1 and 2, dualistic, right versuswrong orientations to multiplicity (positions 3 and 4) where multi-ple answers are possible. However, the student believes right versuswrong is normal and multiple answers are the exception. Positions5 and 6 involve relativism where the student realizes that the worldis relative (right versus wrong is a special case) because that is theway it is, not because instructors like to make it seem that way. Fi-nally, the student will reach the three stages of commitment to val-ues within relativism. Most sophomores in engineering in theUnited States are in position 2, in transition to position 3, or in po-

April 1999 Journal of Engineering Education 197

Page 4: Reflective Analysis of Student Learning in a Sophomore Engineering Course

198 Journal of Engineering Education April 1999

Figure 1. Grade distribution for first quiz in CHE 205.

Table 1: Comparison of sensing (S) and intuitive (N) student scores in CHE 205 in 1997.

Page 5: Reflective Analysis of Student Learning in a Sophomore Engineering Course

sition 3 with a small number in position 4.21 Moreover, there is verylittle change from the sophomore to the senior year. In CHE 205the problems have a single solution but there are multiple solutionpaths, and usually the paths are equally valid. These problemswould probably be classified as in position 3. Thus, these problemswill slightly stretch the students in position 2 or in transition to po-sition 3. There was very little complaining that the problems wereunfair once the students understood the flowsheet, which impliesthe problems were not two or more levels above the students. Stu-dents in positions 3 and 4 will have no difficulty understandingthese problems. There were many student complaints when theflowsheet was not provided. Apparently, translating from a wordproblem to a flowsheet is much more difficult. This was particularlytrue for some problems in the textbook that required that the flow-sheet be inferred from a variety of statements. Use of more prob-lems requiring the students to develop flowsheets might increasechange on Perry’s scale. Since the levels of the students on Perry’smodel were not measured, it is not possible to carry this analysis anyfurther.

An extremely useful way to consider student learning is to lookat deep versus shallow approaches to learning.22-25 A deep approachto learning using cooperative groups was encouraged as early as1895 by John Dewey.26 Our current understanding of these differ-ent approaches to learning stems from the research in Sweden start-ed by Ference Marton and Roger Saljo in the 1970s. McLeod27

briefly reviews the research on neuroanatomy and brain chemistrywhich shows that a deep approach to learning exaggerates the bio-chemical changes in the brain and can result in lasting changes incognition, attitude and character structure. First year engineeringstudents showed an increase in shallow approaches apparently fu-eled by a high workload and fear of failure.28 Later in the year thestudents became more strategic and increased deep approaches amodest amount.

In a shallow approach to learning, the student focuses on learn-ing isolated tasks often through memorization. The student’s goalis to be able to reproduce the information; the student does notfocus on understanding or determining meaning but instead on su-perficial form. For example, many students have memorized F =ma, but do not know how to apply this equation in novel situationsand do not understand many of the implications of the equation. Ina deep approach to learning students focus on determining themeaning of what they are learning and on learning the connectionsand patterns which make the learning holistic. Everyone has the ca-pability to use either a deep or a shallow approach to learning; how-ever, people usually have a favorite mechanism. Students who pre-fer a shallow approach to learning may find a deep approachdifficult. Students who prefer a deep approach to learning may findbeing forced to use a shallow approach annoying and dissatisfying.In a four-part classification scheme of learning styles based on mo-tivation,22, 25 students who are intrinsically motivated preferred a

April 1999 Journal of Engineering Education 199

Figure 2. Grade in CHE 205 versus grade point average at start of course.

Page 6: Reflective Analysis of Student Learning in a Sophomore Engineering Course

deep approach to learning, extrinsically motivated students pre-ferred a shallow approach to learning, those motivated by grades arestrategic and use the approach which will earn grades, and thosemotivated by peers have a great social life and drop out. The in-structor can push students in the first three groups to use deep ap-proaches, but some extrinsically motivated students will be unableto do this and will become frustrated. Examinations that can be an-swered with shallow learning push the students to employ a shallowapproach to learning. Excessive workload and a very rapid pace alsoencourage a shallow approach to learning.

Many students in science and engineering in the US have foundthat a particular kind of shallow approach to learning is sufficient toachieve satisfactory grades at least in the lower division courses. Inmany courses, the students know that if they can find the “right”equation, insert in numbers, and calculate they will get enoughcredit to pass even if they do not understand the problem. Theready availability of inexpensive, powerful calculators has made thisapproach even more popular with students. A slightly more sophis-ticated version of this “plug-and-chug” approach is to check thatthe values inserted into the equation will give the correct units. Forexample, if an area is needed in a heat transfer problem the studentmay use the cross-sectional area of the pipe instead of the area forheat transfer, and he or she will expect significant partial credit fordoing this.

To a large extent mass and energy balance courses require a deepapproach to learning. Instead of being presented with equations,students are given two general principles - conservation of mass andof energy. The student has to draw a flowsheet, derive the appropri-ate equations for this particular flow sheet and problem, and finallysolve the equations. The shallow approach to learning that workedpreviously does not work well in a mass and energy balance courseunless the instructor teaches the course as a series of algorithms.That is, the instructor creates equations for a variety of cases andunwittingly encourages a shallow approach to learning. Incidental-ly, many students will rate this type of instructor very highly, whichonly reinforces that students are not qualified to judge the appropri-ateness of the content they are studying .2, 10

Students who prefer a shallow approach to learning and resistdeep approaches can be identified by several salient characteristics.They want the correct equation from the book. They find energybalances easier than mass balances [energy balance problems tendto be more algorithmic at this level.] They collect isolated facts buthave difficulty connecting them. In addition, they think if they seeenough solutions, they will learn the material. Unfortunately, if theinstructor is not careful this last belief may be correct. Once theyhave seen and memorized the steps in solving a problem, learnerswho use a shallow approach can often solve similar problems aslong as they have not forgotten the details. Thus, whether a prob-lem requires a deep or shallow approach depends upon the context.A shallow approach to learning is not sufficient in a rapidly chang-ing world.

Since it is my strong belief that we must graduate engineers whoroutinely use a deep approach to learning, a major effort was madein CHE 205 to encourage, reward and as much as possible requirethe students to use a deep approach. The differences between thiscourse and some of their previous courses were explicitly stated.The set up of problems was a major focus of the course. Only abouthalf of the course time not used for examinations was used for lec-tures. The remainder was used for problem solving and discussion

of problem solutions, often in cooperative groups. A fixed (stan-dard) grading scale was used to encourage student cooperation witheach other. A limited amount of student cooperation was even al-lowed on the last two examinations. Considerable individual assis-tance was provided by the teaching assistants and the professor. Ex-aminations were designed to require a deep approach to learningand to be solvable in the time available. Unfortunately, the instruc-tor failed to write examinations that required deep learning on twooccasions. These examinations produced higher averages with dis-tributions, which appeared to be normal. I believe that the resultsshown for CHE 205 can be explained as the difference betweenstudents who used a deep or shallow approach to learning. Themost frustrating part of teaching this course was the extreme diffi-culty in getting some of the students to start using a deep approach.The most rewarding part was the opportunity to work with themany students who clearly grew during the course and improvedtheir ability to use a deep approach. Incidentally, requiring a deepapproach did not adversely affect the student evaluations of thecourse or the instructor.

V. PROBLEM SOLVING

Engineering educators believe that problem solving should be amajor focus of the curriculum. Since CHE 205 is the first course inthe curriculum, problem solving is heavily emphasized throughoutthe course. It is useful to review some recent research on problemsolving.

Problem solving tasks can be separated into two categories: rou-tines (exercises) and problems.29 If a task can be solved completelyby an algorithm known to the solver, it is a routine or exercise, not aproblem. In other words, the solver already knows how to proceedto solve the task. This may be very labor intensive and time con-suming (e.g. think of designing the 100th bridge to cross a superhighway) but the path is clear from the beginning. If the task can-not be solved by a routine method known by the problem solver,then it can be classified as a problem. Problem solving then is,“What you do, when you don’t know what to do.” (G. H. Wheat-ley quoted in reference 29). Most of the activities, which are calledproblem solving, are really the solution of exercises. Tasks are notinherently routines or problems. It is the interaction with the solverthat determines if they are a problem or a routine.

Engineering education has two goals when it comes to “problemsolving.” One is to make the students good solvers of exercises and tobe sure that a large variety of the normal tasks within their disciplinehas been placed into the category of routines. For example, one ofthe successes in CHE 205 was that essentially all the students couldsolve exercises such as linear interpolation or converting from massto mole fraction routinely by the end of the semester. Most couldnot when they started the course. Solving exercises is a perfectly ac-ceptable and valuable goal for engineering education although itclearly is not sufficient. I believe learning to solve exercises is a neces-sary goal since an engineer is much more efficient and economicalsolving a routine than a novel problem. Companies will pay a con-sultant high fees because many of the tasks, which are novel prob-lems within the company, will be routine for the consultant.

Starting with Polya,30 there are a number of “problem-solving”strategies that work very well on exercises. The one I use in all of myclasses10 is a modification of the method developed by Don Woods

200 Journal of Engineering Education April 1999

Page 7: Reflective Analysis of Student Learning in a Sophomore Engineering Course

and his colleagues at McMaster University.31,32 This exercise-solv-ing strategy consists of six steps and a prestep for motivation:

0. I can.1. Define.2. Explore3. Plan.4. Do it.5. Check.6. Generalize.This model is not linear but includes recycling through steps as

many times as necessary. This or similar exercise-solving strategiesare very effective for tasks which the solver has a good idea of howto solve (by definition, exercises). Every engineering student shouldbecome proficient in using a strategy to solve exercises.

Unfortunately, this “problem-solving” approach has not provedto be very useful for solving novel problems (those where the solverdoes not know what to do). Solvers of these problems, even expertproblem solvers, use a strategy which is much more trial-and-error .29 The following paraphrase of Bodner’s “anarchistic” model ofproblem solving29 will give a flavor of what good problem solvers ac-tually do when confronted with a novel problem:

• Read The Problem (RTP)• RTP again• Write down what is, hopefully, relevant information• Draw a picture, make a list, write an equation, or whatever to

help begin to understand• Try something—this may be solution of subexercises that

might be part of the solution• See where this gets you• Draw another picture, make another list, or write additional

equations• Try something else• See where this gets you• RTP• Draw another sketch, etc• Try something else• See where this gets you• Test intermediate results• RTP again• Get frustrated• Write down an answer (any answer)• Check the answer• Start over if you have toWhat makes this description the approach of an expert problem

solver and not a novice?33 The expert problem solver writes thingsdown, draws sketches, constructs a variety of different representa-tions of the problem,34 uses a defined strategy for solving the subex-ercises, monitors progress, checks possible answers, recognizes use-ful steps more quickly, expects the problem to eventually makesense and is looking for this sense (deep approach to learning), iswilling to do multiple step problems and does not give up. Thesemethods were explicitly discussed in CHE 205.

Can the ability of students to solve novel problems be improved?Unfortunately, most attempts have seen, at best, modest success.35

In CHE 205, only two students out of 18 with grades below 55 onthe first quiz were able to significantly improve by the last quiz.Moreover, this may represent an improved knowledge structure,not better problem solving capability. Gains in general problemsolving ability for those who started out as weak were clearly mod-

est. Students who were willing to use a deep approach to learningand who started with satisfactory problem solving skills seemed tobenefit the most from this course. Several students blossomed dur-ing the course. [In 1998, significant efforts were made to have thestudents get a good start. This was successful since only eight stu-dents were below 55 on the first quiz. There was one C, three D’s,and four F’s from this group. The grades for the entire class areshown in Table 1. The withdrawals, F’s, D’s, and C’s are almostidentical both years. [A good start appears to be necessary, but isnot sufficient.] My personal observation is that the average engi-neering student 20 years ago was a better problem solver but not asskilled at calculating as the average engineering student now.

There is hope for the weak problem solvers. Workshops extend-ed over an entire term on various aspects of problem solving signifi-cantly increased the problem solving confidence and ability of stu-dents based on a variety of metrics.35 The successful three-stageprocess started with skill building on problems not connected withengineering. Then the problem solving skills were bridged to engi-neering content by applying the skills to problems in a separate en-gineering course being taken concurrently. At McMaster Universi-ty this concurrent course is mass and energy balances. As part of theworkshops, the students were required to reflect on the applicationof the skills in mass and energy balances and in every day life. Final-ly, the applications of the skills were extended to any type of prob-lem by applying them in later courses. Note that this model is morecomplex than including problem solving explicitly within a discipli-nary course10 or having a separate, stand-alone problem-solvingcourse. Textbooks36, 37 are available which can be used in stand-alone classes or adapted for use in the sophomore course. Adapta-tion of the workshop approach may be hastened in the US by thenew Accreditation Board for Engineering and Technology(ABET) rules which require that engineering programs demon-strate that their students and graduates can solve engineering prob-lems.4 Of course, departments and examiners may interpret this asthe ability to solve exercises.

VI. CHANGES FOR THE NEXT TIME

What did I learn from this analysis and plan to change when Itaught the course in Spring 1998? And how well did these changeswork? First, cover less, but in more depth. There are clearly sometopics in the textbook, which are covered in more depth elsewherein the curriculum, which do not need to be in CHE 205. Of course,picking which topics to include and which to exclude is always atricky judgment call, but there is evidence that courses include morethan necessary.38 [This procedure worked well in 1998 and provid-ed more time; however, even more material should be removed.]Second, spend more time on flowsheets and converting from worddescriptions to flowsheets and then to equations. [This helped, butI now think that step-by-step problems which lead the studentthrough this process are needed.] Third, change the scheduling forthe course. Instead of three fifty-minute lectures a week, scheduletwo 50-minute lectures and a two-hour recitation every week. Aclass of 72 students would be divided into two groups of 36 forrecitation. Recitation would be used entirely for cooperative groupwork on problems. Groups would again be assigned by the instruc-tor. There would be a modest portion of the course grade associatedwith the group problem solving in recitation to strongly encourage

April 1999 Journal of Engineering Education 201

Page 8: Reflective Analysis of Student Learning in a Sophomore Engineering Course

202 Journal of Engineering Education April 1999

attendance and participation. [This was such a success that I amconsidering two, two-hour work periods with mini-lectures.]Fourth, schedule the project midway through the semester. Sincesome students seemed to grasp the importance and power of thematerial through the project, this schedule change would gain thisbenefit much earlier. Earlier scheduling does drastically reduce therange of problems the students can write, and it reduces the infor-mation the professor has about each student for assigning groups.[This schedule change was tried with mixed results. There wasprobably a motivational effect, but on average, the projects were notas significant.] Fifth, continue the following: emphasis on a deepapproach to learning and problem solving, significant help avail-able, inductive approach in lectures, large number of examinationsand homework, straight grading scale, and encouragement to studyin groups. The examinations will continue to have sufficient timefor any student who knows how to solve the problems. However,discard only one quiz since discarding two quizzes is too many. [Allthese procedures were continued. Results on quizzes and tests wereless bimodal, but certainly were not normal distributions. Theywere often almost flat distributions. Only one quiz was discarded in1998, which made the students much more serious about thequizzes. Final grades are compared in Table 1. Only the A/B distri-bution differed. The extra time and depth appeared to benefit A/Bstudents but not F/D/C students.]

Another change, which should be considered, is to have the de-partment schedule a series of problem solving workshops to runconcurrently with CHE 205. This would probably have significantimpact on the students’ ability to solve problems. [Unlike the othermodifications, this modification requires significant resources toimplement and was not implemented.]

VII. ACKNOWLEDGMENT

The assistance of Chris Williams and Cheri Zhou in 1997 andof Eric Stangland, Carol Anthes and Doug Kissner in 1998 inteaching CHE 205 and helping me understand the students’ effortsis gratefully acknowledged. Discussions with Dr. Frank Oreoviczover many years have helped my understanding of students as indi-viduals. Professor Soren Tornkvist convinced me that deep learningwas a critically important idea. Professor George Bodner has ex-tended my understanding of problem solving. Professor BarbaraGlasscock shared her course project with me. Parts of this paperwere prepared as a plenary lecture for the Second InternationalConference on Teaching Science for Technology at Tertiary Level,June 14-17, 1997, Stockholm, Sweden. The reviewers’ commentswere very helpful in improving this paper.

REFERENCES

1. Schön, D., Educating the Reflective Practitioner, Jossey-Bass, SanFrancisco, CA, 1987.

2. Centra, J.A., Reflective Faculty Evaluation, Jossey-Bass, San Francis-co, CA, 1993.

3. Angelo, T.A., and P. Cross, Classroom Assessment Techniques, 2nd.ed., Jossey-Bass, San Francisco, CA, 1993.

4. Anon., “Engineering Criteria 2000: Criteria for Accrediting Pro-grams in Engineering in the United States,” ASEE Prism, vol. 6, No. 7,

March 1997 pp. 41-42.5. Felder, R.M., and Silverman, L.K., “Learning and Teaching Styles

in Engineering Education,” Engineering Education, vol. 78, , 1988, pp.674-681.

6. Felder, R.M., “Stoichiometry Without Tears,” Chemical EngineeringEducation, vol. 24 No. 4, Fall 1990, 188-196 .

7. Felder, R.M. and Rousseau, R.W., Elementary Principles of ChemicalProcesses, 2nd. ed., Wiley, New York, NY, 1986.

8. Astin, A.W., What Matters in College? Four Critical Years Revisited,Jossey-Bass, San Francisco, CA, 1993.

9. American Association for the Advancement of Science, Science forAll Americans, Project 2061, Chapter 13, Oxford, New York, NY, 1990.

10. Wankat, P.C. and F.S. Oreovicz, Teaching Engineering, McGraw-Hill, New York, NY, 1993.

11. Kurfiss, J.G., Critical Thinking: Theory, Research, Practice, and Possi-bilities, ASHE-ERIC Higher Education Report No. 2, Washington,D.C., 1988.

12. Bodner, G.M., “Constructivism: A Theory of Knowledge,” Journalof Chemical Education, vol. 63 No. 10, 1986, pp. 873-878.

13. Phillips, J.L., Jr., Piaget’s Theory: A Primer, W.H. Freeman, SanFrancisco, CA, 1981.

14. Pavelich, M.J., “Integrating Piaget’s Principles of IntellectualGrowth into the Engineering Classroom,” Proceedings 1984 ASEE AnnualConference, ASEE, Washington, D.C., 1984, pp. 719-722.

15. Carskadon, T.G., “Student Personality Factors: PsychologicalType and the Myers-Briggs Type Indicator” in: Prichard, K.W. andSawyer, R.M. (eds) Handbook of College Teaching, Chapter 4, GreenwoodPress, Westport, CT, 1994.

16. McCaulley, M., “The MBTI and Individual Pathways in Engi-neering Design,” Engineering Education, vol. 80, 1990, pp. 537-542.

17. Fairhurst, A.M., and L.L. Fairhurst, Effective Teaching, EffectiveLearning, Davies-Black Publishing, Palo Alto, CA, 1995.

18. Culver, R.S., and J.T. Hackos, “Perry’s Model of Intellectual De-velopment,” Engineering Education, vol. 73, 1982, pp. 221-226.

19. Moore, W.S., “Student and Faculty Epistemology in the CollegeClassroom: The Perry Scheme of Intellectual and Ethical Development”in: Prichard, K.W. and Sawyer, R.M. (eds.), Handbook of College Teaching,Greenwood Press, Westport, CT, 1994, Chapter 3.

20. Perry, W.G., Jr., Forms of Intellectual and Ethical Development in theCollege Years: A Scheme, Holt, Rinehart and Winston, New York, NY,1970.

21. Pavelich, M., and P. Fitch, “Measuring Student’s DevelopmentUsing the Perry Model,” Proceedings, 1988 ASEE Annual Conference,ASEE, Washington, D.C., 1988, pp. 688-672.

22. Ramsden, P., Learning to Teach in Higher Education, Routledge,London, 1992.

23. Marton, F., and R. Saljo, “Approaches to Learning” in: Marton, F.et al. (eds.), The Experience of Learning, Scottish Academic Press, Edin-burgh, 1984.

24. Marton, F., “Describing and Improving Learning” in: Schmeck,R.R. (ed.), Learning Strategies and Learning Styles, Plenum, New York,NY, 1988.

25. Entwistle, N.J., Understanding Classroom Learning, Hodder andStoughton, London, 1987.

26. Ryan, A., John Dewey, Norton, New York, NY, 1995.27. McLeod, A., “Discovery and Facilitating Deep Learning States,”

National Teaching and Learning Forum , vol. 5, No. 6, 1996, pp. 1-7.

28. Meyer, J.H.F., and A.R. Sass, “The Impact of the First Year on the

Page 9: Reflective Analysis of Student Learning in a Sophomore Engineering Course

April 1999 Journal of Engineering Education 203

Learning Behavior of Engineering Students,” International Journal of Engi-neering Education, vol. 9, No. 3, 1993, 209-217.

29. Bodner, G.M., “Toward a Unified Theory of Problem Solving: AView From Chemistry” in: Smith, M.U. and V.L. Patel, (eds.), Toward aUniform Theory of Problem Solving, Lawrence Erlbaum Associates, Hillsdale, 1990.

30. Polya, G., How to Solve It, Princeton Univ. Press, Princeton, NJ,1945.

31. Woods, D. R., et al., “Teaching Problem Solving Skills,” Engineer-ing Education, vol. 66, 1975, pp. 238-243.

32. Chorneyko, D.M., et al., “What Is Problem Solving?” ChemicalEngineering Education, vol. 13, 1979, pp. 132-137.

33. Larkin, J., et al., “Expert and Novice Performance in SolvingPhysics Problems,” Science, vol. 208, 1980, pp. 1335-1342.

34. Bodner, G.M., and D.S. Domin, “The Role of Representation inProblem Solving in Chemistry,” Toward a Cognitive-Science Perspective forScientific Problem Solving, No. 6, National Association for Research in Sci-ence Teaching, 1995, pp. 245-263.

35. Woods, D.R., et al., “Developing Problem Solving Skills: The Mc-Master Problem Solving Program,” Journal of Engineering Education, vol.86, 1997, pp. 75-91.

36. Fogler, H.S., and S.E. LeBlanc, Strategies for Creative ProblemSolving, Prentice-Hill, Englewood Cliffs, NJ, 1995.

37. Lumsdaine, E., and M. Lumsdaine, Creative Problem Solving, 2nd.ed., McGraw-Hill, New York, NY, 1993.

38. Wankat, P.C., et al., “Separations. What to Teach Undergradu-ates,” Chemical Engineering Education, vol. 28, No. 1, Winter, 1994, pp. 12-16.