introductory computer programming, problem solving and computer assisted assessment
DESCRIPTION
Introductory Computer Programming, Problem Solving and Computer Assisted Assessment. Charlie Daly, DCU John Waldron, TCD. Preview: A Problem. A Programmable Robot. A Problem. Solving a maze. Program a Robot to Solve a maze Analyse a program that is supposed to solve a maze. But. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/1.jpg)
Introductory Computer Programming, Problem Solving and
Computer Assisted Assessment
Charlie Daly, DCUJohn Waldron, TCD
![Page 2: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/2.jpg)
Preview: A Problem
A Programmable Robot
![Page 3: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/3.jpg)
![Page 4: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/4.jpg)
A Problem
• Solving a maze.
• Program a Robot to Solve a maze
• Analyse a program that is supposed to solve a maze
![Page 5: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/5.jpg)
But ...
• Anybody can check if the program works.– Create a maze and check if the robot can solve
it.
• And provide feedback if it doesn’t work– Show what the robot did.
![Page 6: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/6.jpg)
So ...
• Certain problems are very challenging ...
• ... but they can be checked very simply
• ... easy to provide useful feedback to faulty programs
• End of Preview Mathematical proofs similar, easy to check, difficult to conceive
![Page 7: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/7.jpg)
Talk Outline
• Programming Courses are not working.
• Programming ability is difficult to assess
• The Solution: Proper Assessment• Implementation issues (software and peopleware)
• Results
• Conclusions
![Page 8: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/8.jpg)
Programming Courses are not working
• Students in Introductory Programming Courses do not learn to program!
An international multi-institutional study of introductory programming courses
ITiCSE 2001
“Do students in introductory computing courses know how to program at the
expected skill level?”
![Page 9: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/9.jpg)
No!
![Page 10: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/10.jpg)
What is Wrong?
Assessment
"The spirit and style of student assessment defines the de facto curriculum"
“Assessing Students”, Derek Rowntree '77
![Page 11: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/11.jpg)
What is Wrong?
• It is difficult to assess programming ability in a traditional written exam.
• Programming exercises are subject to plagiarism; a serious problem in introductory programming courses.
• If you do not assess something, the students will not learn it.
![Page 12: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/12.jpg)
What's wrong with Exams
• Two sides of programming– language syntax (easy to examine)
– problem solving (hard to examine)
• Unoriginal (repeat) questions• Marks for attempting a question• Assuming insight where none exists
• Objectivity Lecturer doesn't want to fail whole class.
One exam
![Page 13: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/13.jpg)
What's wrong with Assignments
• Most students don't see a problem with using somebody else's code ("if I understand it it's OK")
• Plagiarism is a huge problem.
• Lecturers frequently know it's happening and do nothing.
don't understand there's a difference between writing and understanding
![Page 14: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/14.jpg)
The Bright Shining Lie
• Lecturers think the students know how to program– they passed the exams
• Students think they know how to program– they passed the exams
• Students have an excuse: for students education is about passing exams
![Page 15: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/15.jpg)
Talk Outline
• Programming Courses are not working.
• Programming ability is difficult to assess
• The Solution: Proper Assessment• Implementation issues (software and peopleware)
• Results
• Conclusions
![Page 16: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/16.jpg)
The Solution: Proper assessment
• Come up with original challenging problems.
• Mark properly; only give marks if the solution is completely correct.
• Allow the student to get computer feedback.– (compiler errors, testing) programming is a process
• Make students aware of the assessment!
![Page 17: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/17.jpg)
Context: the course
• Introductory Programming Course– one semester: 12 weeks of lectures
• Students– 400 (300 pure Computing)– no prior programming experience– education ≡ passing exams
• Exams– programming exams week 6 and 12 (Weighting 30%)
– written exam week 16 (Weighting 40%)
![Page 18: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/18.jpg)
RoboProf
• Automated Program marker• WWW interface• Runs a student's program using different
input and check that the program produces the correct output.
• Student is shown the result and (if not correct) may modify and resubmit their program.
more
![Page 19: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/19.jpg)
The Programming Exam
• 3 Questions (in increasing difficulty)• 2 hours (first half hour without computer)• Not open book, but may use a 'cheatsheet'• During the exam the students
– Write a program– Submit it to RoboProf– View feedback– May resubmit without penalty
![Page 20: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/20.jpg)
The Programming Exam
• Students know their result at the end of the exam.– Fewer complaints– Greater insight into their ability– More likely to listen when subsequently shown
a correct solution
• General effect: formative assessment works.
![Page 21: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/21.jpg)
RoboProf Vs Manual Exam
• Standard CAA advantages– Objective– Fast Feedback
• Avoids the problem of the manual marker interpreting the student solution
• Models standard program development process (computer feedback)
• Huge resource requirements (~400 PCs)
![Page 22: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/22.jpg)
Programming Exam
1009080706050403020100
Writ
ten
Exa
m
100
80
60
40
20
0
![Page 23: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/23.jpg)
Programming Exam
100806040200
Writ
ten
Exa
m
100
80
60
40
20
0
![Page 24: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/24.jpg)
RoboProf Vs Manual Exam
• Results: Not much difference.
• Markers only awarded marks for programs that definitely looked like they might work.
• Students knew that the marking system was similar to the programming exams.
![Page 25: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/25.jpg)
Student Impressions
![Page 26: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/26.jpg)
Student Impressions
Problems Solved
Count Mean
0 17 2.563 1 43 2.302 2 81 1.863 3 45 1.578
"I liked the web-based test"
![Page 27: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/27.jpg)
Student Impressions
Problems solved
Count Mean
0 17 3.882 1 43 3.548 2 81 3.138 3 45 2.773
"The marking system was fair"
![Page 28: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/28.jpg)
Comparison with previous years
• There were too many simultaneous changes to the course to draw firm conclusions (programming language, number of students, lecturers, facilities)
• Results of last three years– 1999: 30%– 2000: 20%– 2001: ?
![Page 29: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/29.jpg)
Conclusions
• CAA can evaluate problem solving ability– Using approriate problems
• Programming problems have added advantages– Models the program development process.– Can show errors without showing the solution.– Avoids the inherent problems that humans have
assessing programming problems
![Page 30: Introductory Computer Programming, Problem Solving and Computer Assisted Assessment](https://reader030.vdocuments.net/reader030/viewer/2022033100/568131a1550346895d981112/html5/thumbnails/30.jpg)
Future Work
• Improve Feedback
• Make it adaptable (can be used by other institutions)