Ghent University INTEC
Evaluation of ICT Course Material Evaluation of ICT Course Material in the Context of an Acoustic in the Context of an Acoustic
Design ProjectDesign Project
Presenter: Pieter Vandaeledepartment Information Technology
Ghent University - Belgium
Authors: Pieter Vandaele, Dick Botteldooren, Els Van Zele, Josephina Lenaerts,
Raoul Meuldermans, Marc Van OvermeireGerrit Vermeir
Ghent University INTEC
ProjectProject
““Distributed Development of a Virtual Science MuseumDistributed Development of a Virtual Science Museum
with Guide Applied to Education in Acoustics”with Guide Applied to Education in Acoustics”
• Since 1997Since 1997• partners: 3 universities and 1 institute for higher partners: 3 universities and 1 institute for higher
educationeducation
• cofinanced by the Flemish Ministry of Educationcofinanced by the Flemish Ministry of Education• http://educinno.rug.ac.be/wetenschapsmuseumhttp://educinno.rug.ac.be/wetenschapsmuseum
Ghent University INTEC
Science museumScience museum
Create ICT course material:Create ICT course material: • small learning objects, small learning objects, different levelsdifferent levels
• flexible useflexible use
Demo’s:Demo’s:
•audioaudio
•visualvisual
Ghent University INTEC
Science museumScience museum
Create ICT course material:Create ICT course material: • small learning objects, small learning objects, different levelsdifferent levels
• flexible useflexible use
Theory and Theory and mathematical mathematical derivationsderivations
Ghent University INTEC
Science museumScience museum
Create ICT course material:Create ICT course material: • small learning objects, small learning objects, different levelsdifferent levels
• flexible useflexible use
Interactive Interactive applets and applets and experimentsexperiments
Ghent University INTEC
Science museumScience museum
Create ICT course material:Create ICT course material: • small learning objects, small learning objects, different levelsdifferent levels
• flexible useflexible use
Used to build:Used to build: • written coursenoteswritten coursenotes• projectsprojects• slidesslides• exercicesexercices
Ghent University INTEC
Used to build:Used to build: • written coursenoteswritten coursenotes• projectsprojects• slidesslides• exercicesexercices
Science museumScience museum
Create ICT course material:Create ICT course material: • small learning objects, small learning objects, different levelsdifferent levels
• flexible useflexible use
??• What about effectiveness of material What about effectiveness of material
in improving students’ insight?in improving students’ insight?• What about student appreciation?What about student appreciation?• How do they use the material?How do they use the material?
Student project to evaluateStudent project to evaluate
Ghent University INTEC
EvaluationEvaluation
• Design projectDesign project
• design Helmholtz resonatordesign Helmholtz resonator• students are familiar with basic acoustics students are familiar with basic acoustics • information needed to solve the problem: on the information needed to solve the problem: on the
internetinternet
• first “IT in education”-experience for studentsfirst “IT in education”-experience for students
illustrations, theory and experimental design environment
Ghent University INTEC
EvaluationEvaluation
• Design projectDesign project
Why?Why? – different solution pathsdifferent solution paths
1. Rote experimenter
Theory and equations
Derivations, limits theory
Central formula
Problem
Lab, experiment Operation principle,
illustration
Ghent University INTEC
EvaluationEvaluation
• Design projectDesign project
Why?Why? – different solution pathsdifferent solution paths
1. Rote experimenter
2. Intelligent trial and error
Problem
Theory and equations
Derivations, limits theory
Operation principle, illustration
Lab, experiment
Central formula
1. Rote experimenter
Ghent University INTEC
EvaluationEvaluation
• Design projectDesign project
Why?Why? – different solution pathsdifferent solution paths
1. Rote experimenter
2. Intelligent trial and error
3. Meaningful experimenter
Derivations, limits theory
Operation principle, illustration
Central formula
Lab, experiment
Theory and equations
Problem
1. Rote experimenter
2. Intelligent trial and error
Ghent University INTEC
EvaluationEvaluation
• Design projectDesign project
Why?Why? – different solution pathsdifferent solution paths
1. Rote experimenter
2. Intelligent trial and error
3. Meaningful experimenter
4. Theorist
Derivations, limits theory
Central formula
Operation principle, illustration
Lab, experiment
Problem
Theory and equations
1. Rote experimenter
2. Intelligent trial and error
3. Meaningful experimenter
Ghent University INTEC
EvaluationEvaluation
• Design projectDesign project
Why?Why? – different solution pathsdifferent solution paths
1. Rote experimenter
2. Intelligent trial and error
3. Meaningful experimenter
4. Theorist
5. ExpertDerivations, limits theory
Operation principle, illustration
Lab, experiment
Theory and equations
Problem
Central formula
1. Rote experimenter
2. Intelligent trial and error
3. Meaningful experimenter
4. Theorist
Ghent University INTEC
EvaluationEvaluation
• Design projectDesign project
Why?Why?– collaborationcollaboration– free exploration of material and use new free exploration of material and use new
insights in practical applicationinsights in practical application
– different solution paths different solution paths
Ghent University INTEC
EvaluationEvaluation
• Design projectDesign project• Oral examOral exam
– to defend project– to test insight in operation of Helmholtz resonator
Ghent University INTEC
EvaluationEvaluation
• SurveySurvey– How did they cope with the project?– What material did they use?– How do they perceive their gained insight?– How is their overall appreciation with IT in education?
• Design projectDesign project• Oral examOral exam
Ghent University INTEC
EvaluationEvaluation
• Design projectDesign project• Oral examOral exam• SurveySurvey
Very small statistical population, but someVery small statistical population, but some
interesting resultsinteresting results– project
– IT in education
Ghent University INTEC
Results - projectResults - project
HR AI SP scoreStated importance of applet for experiments 0.24 -0.33 -0.53 0.09Stated importance of equations 0.15 0.26 -0.26 0.19Stated importance of information on derivation and limits of theory -0.39 0.42 0.28* -0.02Stated depth of study 0.27 0.20 0.01 -0.06Year score of student -0.14 -0.30 -0.28 0.07
Ghent University INTEC
Results - projectResults - project
HR AI SP scoreStated importance of applet for experiments 0.24 -0.33 -0.53 0.09Stated importance of equations 0.15 0.26 -0.26 0.19Stated importance of information on derivation and limits of theory -0.39 0.42 0.28* -0.02Stated depth of study 0.27 0.20 0.01 -0.06Year score of student -0.14 -0.30 -0.28 0.07
Ghent University INTEC
Results - projectResults - project
HR AI SP scoreStated importance of applet for experiments 0.24 -0.33 -0.53 0.09Stated importance of equations 0.15 0.26 -0.26 0.19Stated importance of information on derivation and limits of theory -0.39 0.42 0.28* -0.02Stated depth of study 0.27 0.20 0.01 -0.06Year score of student -0.14 -0.30 -0.28 0.07
More detailed study
Ghent University INTEC
Results - projectResults - project
HR AI SP scoreStated importance of applet for experiments 0.24 -0.33 -0.53 0.09Stated importance of equations 0.15 0.26 -0.26 0.19Stated importance of information on derivation and limits of theory -0.39 0.42 0.28* -0.02Stated depth of study 0.27 0.20 0.01 -0.06Year score of student -0.14 -0.30 -0.28 0.07
Ghent University INTEC
Results - projectResults - project
HR AI SP scoreStated importance of applet for experiments 0.24 -0.33 -0.53 0.09Stated importance of equations 0.15 0.26 -0.26 0.19Stated importance of information on derivation and limits of theory -0.39 0.42 0.28* -0.02Stated depth of study 0.27 0.20 0.01 -0.06Year score of student -0.14 -0.30 -0.28 0.07
Legend: Legend: – HR: perception insight in operation Helmholtz resonator – AI: perception insight in ‘acoustic impedance’– SP: perception insight in ‘sound propagation in ducts’– score: score on oral exam question related to project– *: not significant
generalisation
Ghent University INTEC
Results - projectResults - project
HR AI SP scoreStated importance of applet for experiments 0.24 -0.33 -0.53 0.09Stated importance of equations 0.15 0.26 -0.26 0.19Stated importance of information on derivation and limits of theory -0.39 0.42 0.28* -0.02Stated depth of study 0.27 0.20 0.01 -0.06Year score of student -0.14 -0.30 -0.28 0.07
Analysis:Analysis:– focus on applet only insight in problem at hand
Ghent University INTEC
Analysis:Analysis:– focus on applet only insight in problem at hand– focus on equations and derivation theoretical insight
Analysis:Analysis:– focus on applet only insight in problem at hand– focus on equations and derivation theoretical insight
• strong neg. corr.: not reflected in students’ result, perhaps only these students realise the complexity of the matter
• SP only was mentioned in pages concerning limits of the theory
Results - projectResults - project
HR AI SP scoreStated importance of applet for experiments 0.24 -0.33 -0.53 0.09Stated importance of equations 0.15 0.26 -0.26 0.19Stated importance of information on derivation and limits of theory -0.39 0.42 0.28* -0.02Stated depth of study 0.27 0.20 0.01 -0.06Year score of student -0.14 -0.30 -0.28 0.07
Ghent University INTEC
Results - projectResults - project
HR AI SP scoreStated importance of applet for experiments 0.24 -0.33 -0.53 0.09Stated importance of equations 0.15 0.26 -0.26 0.19Stated importance of information on derivation and limits of theory -0.39 0.42 0.28* -0.02Stated depth of study 0.27 0.20 0.01 -0.06Year score of student -0.14 -0.30 -0.28 0.07
– neg. corr. year score & perception insight• better students: unsatisfied
• better students don’t easily state to understand something completely
• they perform well in current system and don’t put much effort in an alternative project
Matter of perception
Analysis:Analysis:
Ghent University INTEC
Results - IT in educationResults - IT in education
– negative correlation: wish more ICT - performance current system
-4
-3
-2
-1
0
1
2
3
4
5
6
500 550 600 650 700 750 800 850 900
yearscore
wis
h t
o g
et m
ore
IC
T i
n e
du
cati
on
Analysis:Analysis:
Ghent University INTEC
Results - IT in educationResults - IT in education
– students like self-assessment tools– possibilities of hypertext to hide unessential information, cross-link
different topics: not estimated very high
Analysis:Analysis:– applets to explore influence of parameters are highly appreciated
Ghent University INTEC
Results - IT in educationResults - IT in education
• more students afraid to waste time• it is more clear what has to be learned • less students fear they’ll have to work more
Analysis:Analysis:– in course context:
Ghent University INTEC
ConclusionsConclusions
• Students appreciated exploration and design problemStudents appreciated exploration and design problem– majority students have the impression they have gained insight– exploration stopped at earlier stage than hoped for (study culture?)
• Unexpected negative correlation: performance Unexpected negative correlation: performance students in current system - appreciation of project / ITstudents in current system - appreciation of project / IT– tuning seems appropriate– appreciation may depend on context in which IT is used
• Structure!Structure!