Download - Case study: evaluation of a tool for searching inside a collection of multimodal e-lectures
Case Study: Evaluation of a Tool for Searching inside a Collection of Multimodal e-
Lectures
Marco RonchettiAngela Fogarolli
Dipartim. Informatica e TelecomunicazioniUniversità di Trento
Italy
Logical Architecture
Synchronous (webcast)
Asynchronous (On line, or download - podcast)
Asynchronous(CD - DVD)
Acquisition setup
QuickTime™ e undecompressore TIFF (LZW)
sono necessari per visualizzare quest'immagine.
Cognitive interface
The main focus is on the projected slide, a clear voice is very important, the video carries additional information
like gestures, and can show the environment just enough
Navigation is possible It must be also possible to use the
blackboard!
LODE: the user interface
New Flash version
QuickTime™ e undecompressore TIFF (LZW)
sono necessari per visualizzare quest'immagine.
Lode - portabilitySupporto della didattica tradizionaleMac – Win - Linux
Player MP4(no slides yet)
Presently attempting a
port to mobile phones
Student feedbackEnthusiastic. We were “forced” to extend the experiment
to a second course!
students who… total 1st exam(45 stud.)
2nd exam (23 stud.)
…used the system to review at least one lecture
63% 51% 87%
…followed the course completely off-line
5% 6% 4%
75% anticipated using the system
often or very often
Advantages from student’s perspective
Better time management: ability to recover lectures lost due to forced
absence (illness, work or other time-frame incompatibility);
ability to better organize their time, deciding not to be present at some lecture (elective absence);
Better understanding review some critical point (cases of poor
understanding of a section due to concentration drop, excessive speed in an explanation or intrinsic difficulty);
ability to check the correctness of notes taken during a lecture;
Other advantages
Miscellaneous perception by the student of a better
service provided by the university; support foreign (italian) students who
might have difficulties with the Italian (English) language;
enrichment of the e-learning portfolio; possibility for the teacher to view
himself; possibility to reuse lectures (across
time, or in different contexts!)
Can we do more?
Scenario: Thousands of recorded lectures. Can we dig in them to extract
material for (less formal) learning?
Cfr. approach by Mike Wald(subtitling for accessibility)
Needle - search results
QuickTime™ e undecompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.
Needle - architecture
ASR
Multimodal search!
Questions
Is it useful? Is it effective?
Scenario100% accurate transcription (manually corrected)Goal: extract some info from a 2 hour lecture in half of
the time…
Assess whether the tool could reduce the learning time and help students in their learning task: we do not aim at claiming or proving that a course delivery mode is better than the other.
Comparing delivery modes (e.g. distance learning with traditional classroom-based instruction) (Sener 2005) for the purpose of establishing the superiority of one delivery mode over another is specious, irrelevant and counterproductive because there is not an uniformity of practice.
Experiment68 students, after 20 hours of a 50 hour Java
course.
Group A: LODE
Group B: NEEDLE
ExperimentPre-Test: 5 questions, 5 minutes
LODE group: 70 minutes for the lecture, plus24 minutes for the post-test (8 questions): total 94
minutes to accomplish the task
NEEDLE group: 48 minutes total for responding to the same 8 questions (50% of the time)
Pre-test results
Test results
AnalysisIn the subset of questions, which require students to have
a complete understanding of a part of the lecture both groups performed in the same way.
In answering those questions that also required some reasoning about concepts, group LODE – who had listened to the entire lecture – performed better.
In the questions where the students where asked to provide the right definition for a concept group NEEDLE performed better.
Further investigationsMultivariate regression:- no significant results
Clustering techniques- Indication that some feature seem to be present -
investigation is still on its way
Conclusion NEEDLE tool is more geared towards
information finding than towards actual learning.
A similar pattern could be reasonably expected in a similar experiment performed by reading some assigned material versus searching quick answer with Google.
However, our finding seems to show that Needle works very well as a tool for extracting information from a video source.