Novices’ Debugging Behaviors in VB Programming

Mei-Wen Chen Graduate Institute of Information and

Computer Education National Taiwan Normal University

Taipei, Taiwan anniechen@ntnu.edu.tw

Cheng-Chih Wu Graduate Institute of Information and

Computer Education National Taiwan Normal University

Taipei, Taiwan chihwu@ntnu.edu.tw

Yu-Tzu Lin Graduate Institute of Information and

Computer Education National Taiwan Normal University

Taipei, Taiwan linyt@ntnu.edu.tw

Abstract—This study investigates the debugging behaviors of novices in the Visual Basic (VB) programming environment. To understand how novice programmers solve debugging problems, we employed a mixed-method, which used both quantitative and qualitative data , including a questionnaire survey and a debugging-behaviors analysis. Seventy-two students from a vocational high school in Taipei participated in the study. The results indicated that most students believed debugging skill is due to individual aptitude and cannot be developed through learning. Most students did not employ debugging tools provided in Visual Basic environment or even were aware of the existence of the tools. Although most students enjoyed the achievement of resolving bugs, they disliked the challenge of debugging.

Keywords—Novice programmer; Debugging; Visual Basic.

I. INTRODUCTION

Computer programming is an important skill in computer science, by which the hardware/software design can be implemented. Learning programming can foster logical thinking and problem solving ability [1]. In Taiwan, computer science is a required course in senior high schools and programming is one of the major topics in this course. Since Visual Basic was included in the official computer science textbook earlier in Taiwan, most senior high schools, especially vocational schools, still choose Visual Basic as the teaching language recently.

Previous research [2-5] has discussed about programming learning and teaching. Many of them [2, 5] investigated the differences between novice programmers and experts and they all found that novice programmers suffer from a wide range of difficulties and deficits [6-9]. Even though novice programmers knew the syntax and semantics of individual statements, they did not know how to combine these features into valid programs [5] and create correct program plans [2].In Spohrer, Soloway, and Pope’s research [10], they defined program bugs as the differences between the way plans are implemented by novices in their programs and the correct realization of those plans. They then proposed four types of faults in plans: missing, misplaced, malformed, and spurious differences. However, the most challenging thing for novice

programmers is to locate and resolve bugs [7]. Novices usually felt frustrated when they had to find bugs because their knowledge about programming was fragile [8].

To understand why and how program debugging frustrated novices, Many works have devoted in the research of programming debugging [11], including early research for BASIC [9-11] and recent research for JAVA [12-14]. Fitzgerald et al. suggested that students who were good debuggers were good novice programmers [11]. However, most novices could not apply debugging techniques effectively. Some of them even tinkered aimlessly [14]. Therefore, addressing the importance of developing students’tracking and debugging skills is needed [2]. Although programming has been emphasized in computer science education, few courses include enough lectures about debugging [15]. Most computer programming courses did not provide debugging training for students [16]. Some research discussed programmers’ debugging behaviors and found that novice programmers debug programs in a trial-and-error manner without comprehending programs so that they usually cannot resolve errors successfully [12, 13, 17, 18]. Students were used to ask for the teacher’s help without trying to find the causes by themselves. Perkins et al.[9] analyzed students’ debugger behaviors in LOGO and BASIC programming. In their observation, some of the students were not willing to discover possible problems when errors occurred; some others tried to try to find causes of errors; and others tried to modify the program repeatedly without deep thinking. There were three types of debugging strategies employed by students: close tracking of the code, tinkering the code, and breaking problems down.

In existing research of computer science education, there were limited discussions about programming learning in VB. VB provides a visualized programming interface, with which students can write codes without considering too much about syntax. Its real-time debugging tools also assist students to debug in an easier way [19]. Since VB is the most common taught programming language in Taiwan high schools, it is important to understand students’ programming debugging behaviors so that to provide suggestions for educators. In this

2013 Learning and Teaching in Computing and Engineering

978-0-7695-4960-6/13 $26.00 © 2013 IEEE

DOI 10.1109/LaTiCE.2013.38

25

2013 Learning and Teaching in Computing and Engineering

978-0-7695-4960-6/13 $26.00 © 2013 IEEE

DOI 10.1109/LaTiCE.2013.38

25

study, we investigated how novice students debug programs and what their attitudes about program debugging.

II. METHODS

A survey research was conducted to understand students’ debugging behaviors, including observation, questionnaire study, and interview. Seventy-two 12th grade students from a vocational high school in Taipei City participated in the study. All students have learned one-year Introduction of Computer Science and one-year VB programming course. They had fundamental programming concepts and programming experiences such as variables, the conditional statement, the loop statement, subroutines, and arrays. Eight students were picked from a class based on their last semester grades, including two high performance students, three medium performance students, and three low performance students. They are five males and three females. Except the two high performance students, who practiced programming both in and after class, all students learned programming only in class.

In order to explore students’ debugging behavior, we assigned three debugging tasks to eight students, which should be resolved in 90 minutes. The program development environment used in this lecture was Visual Basic 2008. The required knowledge for the debugging tasks included fundamental programming concepts, such as variables, the conditional statement, the loop statement, and subroutines. Both syntactic and semantic errors were contained in the test programs, included missing, misplaced, malformed, and spurious errors. The test programs for debugging were presented in problem statements with sample input and output. The three debugging tasks and the contained errors are listed as Table I.

TABLE I. Tasks errors list. Task name Errors

Task 1 Finding multiples of avalue

missing button code, spurious statement

Task 2 Elements swapping misplaced parameters, malformed subroutines

Task 3 Restaurants ordering malformed checkbox, misplaced if-statement, missing button, spurious statement.

Whole debugging process for each student was screenshotted, which was also monitored and recorded by a researcher. During the observation, notes were taken, which were used to generate questions for the follow-up interviews. Interviews were held immediately after the observations and each interview lasted fifteen minutes. Questions relevant to the debugging behaviors were asked at the interviews. Therefore, the data collected for analysis include: the questionnaire results from all students (the response rate is 91%), the video

and screenshots recording eight student-samples’ debugging behaviors, and the interview data of the same eight students.

The questionnaire included three types of questions: (1) Debugging strategies— to understand what debugging strategies the students used, e.g., employing debugging tools, asking for help, or giving up. (2) Debugging difficulties – to understand what difficulties the students had. (3) Attitude towards debugging – to understand what attitude and feeling the students had about debugging.

Two open-ended questions were also included in the questionnaire: (1) “What do you think about debugging?”(2) “What is the most difficult thing in debugging?”

Students’ behaviors in the debugging sessions were coded into five categories, including Trace, Edit, Test, Help, and Stop (detailed in the following)so that patterns of behaviors could be easily analyzed.

(1) Trace: accessing codes, forms, object attributes, error list, or messages. (2) Edit: revising codes, forms, or object attributes. (3) Test: inputting data and testing the result. (4) Help: using debugging tools and resources or ask for someone’s help. (5) Stop: stopping debugging.

III. RESULTS

A. Use ofDebugging tools The questionnaire results about how students employed

debugging tools were presented in Table II. More than half of the students thought that they applied the “blue waved line” (62%), the “error list” (59%) and the “Immediate window” (53%) to find bugs. The most frequently used strategy is to debug from statements with blue waved lines. Some students thought that they used the “Step Into” function (33%) and the “Breakpoint” function (17%) to find bugs.

Although most students claimed that they could use various debugging functions provided by VB to locate errors in a program, our observation of students’ debugging sessions revealed that they only relied on system error prompt (such as the “error list” and the “blue waved line” in VB), and could not use or even notice the existence of debugging functions (such as “Breakpoint” or “the Step Into”).

2626

TABLE II. Questionnaire results about debugging strategies. Questions Strongly

agree Agree

Neither agree nor

disagree

Disagree Strongly disagree Mean

1. I used the “Error list” to find bugs. 21% 38% 29% 8% 5% 3.64

2. I used the ”blue waved line” to find bugs. 27% 35% 23% 12% 3% 3.71

3. I used the ”Immediate window” to find bugs. 17% 36% 27% 11% 9% 3.41

4. I used “Step Into” to find bugs. 9% 24% 38% 17% 12% 3.02

5. I used “ Breakpoint” to find bugs. 5% 12% 38% 26% 20% 2.56

B. Debugging strategies Students’ debugging strategies were analyzed using the

questionnaire and interview data. As illustrated in Table ,students’ debugging strategies were firstly understood by discussing whether they have debugging plans before modifying the codes. More than half (59%) of the students thought that they corrected the codes without any debugging plan. They were used to correct the codes once they found possible errors.

TABLE III.Questionnaire results about students’ debugging plans.

Questions Frequency Percentage

1. I have considered all debugging steps before modifying the codes. 2 3%

2. I considered some debugging steps and then started modifying the codes. 25 38%

3. I modified the codes without any debugging plan. 39 59%

We then interviewed these eight students for understanding their thoughts about debugging. As presented in Table IV, students could overcome most bugs except malformed and misplaced ones. The reason was that missing errors could be easily detected during compiling, but malformed and misplaced errors could not be identified by the compiler and should be found and resolved by students themselves.

TABLE IV.Number of resolved/total errors for each error type

in three debugging tasks.

Malformed Misplaced Missing Spurious

Task 1 7/8 7/8

Task 2 8/8 3/8

Task 3 4/8 8/8 8/8 6/8

From results of the observation and interviews, it is found that students had four types of debugging behaviors, which are detailed as the following:

1) Reading error messages in the error list The researcher found in the observation that students

seemed to use the error list of VB to debug, so he asked the students whether and how they used the error list to debug. The comment of the student 1 was:

“Yes. I just followed what the error list said. It showed me which term and which line the errors located in, and I would find errors there.”

And the comment of the student 2 was: “I always watched both the error list and the source code

to find the possible errors and the corresponding lines of the source code.”

But when the students were asked whether they grasped the error list said, the student 3 replied that he only could understand part of the error messages. And the student 4 commented:

“Actually I could not understand the error messages provided in the error list very well.”

The last question about the error list of VB is whether error messages help students debug. The student 5 agreed , but he did not grasp their meanings exactly.

We can summarize from the interview that students were

used to read all the error messages in the error list provided by VB and intended to debug according to these error messages. However, they did not understand well about the error messages.

2) Modifying the program by trial and error To prove the finding in the literature suggesting that

novices usually tinkered the code randomly [14], students were asked whether they planned how to carry out the debugging before starting it. However, the student 5 replied no and said that he always modify the code directly without any plan.

This result is in accordance with previous research, which suggested that once students found possible program errors, most of them intended to modify the program by trial and error without a real understanding of what and where went wrong.

2727

3) Commenting suspected codes The researcher observed that most students liked to

commenting suspected codes for debugging. In the interview, this fact was proved. One of the students’ comments is:

“Sometimes I commented out some lines of the code to narrow down the available range for debugging. It would help my debugging.”

Therefore, a common strategy used by the students was to block suspected codes (as reported by the editor/compiler) by tagging them into comment statements in order to make the program run successfully without the reported errors, as shown in Fig. 1.

Figure 1. The source code with comments (the selected text).

4) Using the “Notepad” application to backup codesAn interesting finding in the observation was that some

students opened the Notepad tool of Windows while debugging. The researcher then asked the students to describe why and how they used the Notepad to help debug. One of the student’s replies is:

“I was afraid of forgetting what I wrote and modified. So I opened the Notepad.”

“I copied and pasted my code to the Notepad even if the code was wrong.”

“If I found the current solution could not work, then I could re-consider the previous version of the code.”

While students were trying to correct errors, they were used to copy and paste codes from VB to the Notepad ( as shown in Fig. 2). Whenever the corrected codes did not have expected execution results, they reviewed the backup codes and re-considered their debugging.

Figure 2. Using the “Notepad” to backup the source code.

C. Debugging difficulties In this section, students’ debugging difficulties are

examined. The results are listed in Table 4, which show that more than half (53%) of the students thought that they could find bugs and also knew how to correct it. They (41%) understood error messages in the Error list. Less than a quarter (22%) of the students knew how to use sample inputs to test the program. The results indicate that most students could find where bugs were and knew how to resolve problems, but they did not know how to test and debug the program by using sample input data.

Less than a quarter of the students thought resolving compile errors (23%) and run-time errors (26%) were easy, which implies that most students considered program debugging as a difficult thing.

TABLE V. Results of students’ debugging difficulties.

Questions Strongly agree Agree

Neither agree nor

disagree

Disagree Strongly disagree Mean

1. I understood error messages in the “Error list”.

17% 24% 39% 15% 5% 3.23

2. I could find bugs when the program did not work. 17% 30% 35% 15% 3% 3.42

3. I could find bugs and also knew how to correct them.

20% 33% 29% 17% 2% 3.53

4. I knew how to use sample input data to test the program.

5% 17% 36% 33% 9% 2.74

5. It was easy for me to resolve the “compile error”.

2% 21% 58% 17% 3% 3.02

6. It was easy for me to resolve the” run-time error”.

6% 20% 42% 26% 6% 2.94

2828

According to the questionnaire results, students’ comments about debugging difficulties are concluded as the following:

1) I did not know where errors were. (13/66) 2) I could not find errors. (12/66) 3) I did not know how to correct errors. (8/66)

From what the students commented in the open-ended questions of the questionnaire, it was found that the major difficulty the novices encountered was locating the errors.

“Finding out the bugs was the most difficult thing.” Even though they got the error messages, it was still

challenging for them to locate the bugs. This finding is accordance with the results of pervious research [7, 12].

D. Attitude towards debugging We also examined students’ attitude towards debugging.

The results are presented in Table 5, which show that more than half of the students considered themselves lacking of debugging skills and believed these skills were based individual’s aptitude (65%), which could not be learned (51%). Although students did not like the challenges brought by debugging, they still enjoyed the success of resolving bugs (82%).

TABLE VI.Questionnaire results of students’ attitude towards

debugging.

Questions Strongly agree Agree

Neither agree nor

disagree

Disagree Strongly disagree Mean

1. I like debugging even though it is challenging for me.

6% 17% 45% 21% 11% 2.86

2. After successfully debugging, I will be very happy.

50% 32% 17% 0% 2% 4.29

3. I think my debugging skill is good. 0% 23% 39% 26% 12% 2.73

4. I think the debugging skill is based on individual’s aptitude.

21% 44% 29% 3% 3% 3.77

5. I think the debugging skill can be accumulatedwith experiences and cannot be learned.

12% 39% 26% 18% 5% 3.36

Students’ comments about debugging are concluded in the following:

1) After successfully debugging, I feel a sense of achievement and happy. (23/66)

2) Debugging is difficult. (10/66) 3) Debugging is onerous. (6/66)

The results show that most students disliked debugging and felt it is challenging; however, they did enjoy the success of resolving bugs

IV. CONCLUSION

This research investigates students’ program debugging behaviors and attitude by the questionnaire study, observation, and interview. The results show that most students considered their debugging skills not good enough and believed that the skills were based on individual’s aptitude, which could not be learned. The observation reveals that students did not employ VB debugging tools (the “Step Into” and the “Breakpoint”) even though most of them indicated that they did. Some students even did not know the existence of the VB debugging tools. Most students disliked the challenge from debugging; however, they did enjoy the success of resolving bugs. To engage novices’ in debugging, teachers may consider adopting proper instructional strategies so that students’ senses of accomplishment could be reinforced. This study may be refined in the future by implementing it with a larger sample. Moreover, more programming concepts such as the array and recursion can be included in the debugging problems to understand more about the students’ debugging behaviorsinvolved in solving different problems.

REFERENCES

[1] M. Resnick, J. Maloney, A. Monroy-Hernández, N. Rusk, E. Eastmond, K. Brennan, A. Millner, E. Rosenbaum, J. Silver, B. Silverman, and Y. Kafai, " Scratch: programming for all," Communications of the ACM, vol. 52, iss. 11, pp. 60-67, Nov. 2009.

[2] A. Robins, J Rountree,and N. Rountree, "Learning and Teaching Programming: A Review and Discussion," Computer Science Education, vol. 13, no. 2, pp. 137 - 172, Jun. 2003.

[3] E. Soloway, "Should we teach students to program?," Communications of the ACM, vol. 36, iss. 10, pp. 21-24, Oct. 1993.

[4] R. E. Mayer and A. L. Fay, "A Chain of Cognitive Changes With Learning to Program in Logo," Journal of Educational Psychology, vol. 79, iss. 3, pp. 269-279, 1987.

[5] L. E. Winslow, "Programming pedagogy - Apsychological overview," SIGCSE Bulletin, vol. 28, iss. 3, pp. 17-22, Sep. 1996.

[6] J. C. Spohrer and E. Soloway, "Novice mistakes: are the folk wisdoms correct?," Communications of the ACM, vol. 29, iss. 7, pp. 624-632, July 1986.

[7] E. Lahtinen, K. Ala-Mutka, and H.-M.Järvinen, "A study of the difficulties of novice programmers," SIGCSE Bulletin, vol. 37, iss. 3, pp. 14-18, September 2005.

2929

[8] D. Perkins and F. Martin, "Fragile knowledge and neglected strategies in novice programmers," In E. Soloway & S. Iyengar (Eds), Empirical studies of programmers, pp. 213-229,1986

[9] D. Perkins, D. Perkins, C. Hancock, R. Hobbs, F. Martin, and R. Simmons, "Conditions of learning in novice programmers," Journal of Educational Computing Research, vol. 2, pp. 37-55, 1986.

[10] J. C. Spohrer, E.Soloway, and E.Pope: A Goal-plan Analysis of Buggy Pascal Programs, Human-Computer Interaction, vol. 1, iss. 2, pp. 163-207, 1985.

[11] R. T. Putnam, D. Sleeman, J. A. Baxter, and L. Kuspa, "A summary of misconceptions of high school basic programmers," Journal of Educational Computing Research, vol. 2, pp. 459-472, 1986.

[12] S. Fitzgerald, G. Lewandowski, R. McCauley, L. Murphy, B. Simon, L. Thomas,and C. Zander, "Debugging: Finding, Fixing and Flailing, a Multi-Institutional Study of Novice Debuggers," Computer Science Education, vol. 18, no. 2, pp. 93-116, 2008.

[13] L. Murphy, G. Lewandowski, R. McCauley, B. Simon, L. Thomas, and C. Zander, "Debugging: the good, the bad, and the quirky - a qualitative analysis of novices' strategies," Communications of the ACM, vol. 40, iss. 1,pp. 163-167, March 2008.

[14] B. Simon, D. Bouvier, T-Y. Chen, G. Lewandowski, R. McCartney, and K. Sanders, "Common sense computing (episode 4): debugging," Computer Science Education, vol. 18, no. 2, pp. 117-133, 2008.

[15] R. McCauley, S. Fitzgerald, G. Lewandowski, L. Murphy, B. Simon, L. Thomas, and C. Zander, "Debugging: a review of the literature from an educational perspective," Computer Science Education, vol. 18, no. 2, pp. 67-92, 2008.

[16] R. Chmiel and M. C. Loui, "Debugging: from novice to expert," SIGCSE Bulletin, vol. 36, iss. 1, pp. 17-21, March 2004.

[17] R. Jeffries, "A comparison of the debugging behavior of expert and novice programmers," pp. Paper presented at the American Education Research Association Annual Meeting, New York. , 1982.

[18] M. C. Jadud, "A First Look at Novice Compilation Behaviour Using BlueJ," Computer Science Education, vol. 15, pp. 25-40, 2005.

[19] K. Powers, P. Gross, S. Cooper, M. McNally, K.J. Goldman, V. Proulx, and M. Carlisle, "Tools for teaching introductory programming: what works?," SIGCSE Bulletin, vol. 38, iss. 1, pp. 560-561, March 2006.

3030