intelligent support for inquiry learning from images: a
Post on 11-Feb-2022
1 Views
Preview:
TRANSCRIPT
Open Research OnlineThe Open University’s repository of research publicationsand other research outputs
Intelligent support for inquiry learning from images: alearning scenario and toolConference or Workshop ItemHow to cite:
Mulholland, Paul; Zdrahal, Zdenek; Abrahamcik, Jan and Wolff, Annika (2009). Intelligent support for inquirylearning from images: a learning scenario and tool. In: The 14th International Conference on Artificial Intelligence inEducation (AIED2009), 6-10 Jul 2009, Brighton, UK, pp. 417–424.
For guidance on citations see FAQs.
c© 2009 The authors and IOS Press
Version: Accepted Manuscript
Link(s) to article on publisher’s website:http://dx.doi.org/doi:10.3233/978-1-60750-028-5-417http://www.iospress.nl/html/9781607500285.php
Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyrightowners. For more information on Open Research Online’s data policy on reuse of materials please consult the policiespage.
oro.open.ac.uk
Intelligent Support for Inquiry Learning
from Images: A Learning Scenario and Tool
Paul Mulholland, Zdenek Zdrahal, Jan Abrahamcik and Annika Wolff
Knowledge Media Institute and Centre for Research in Computing
The Open University, Milton Keynes, MK7 6AA
Abstract. Inquiry learning involves the learner acquiring new concepts and skills
by means of carrying out an investigation. Some previous studies have looked into
how these learning activities can be carried out on source materials, such as web
documents, and the provision of appropriate scaffolding to guide the learning
process. Here we consider how intelligent support can be provided to guide the
learner in analysing source materials and building knowledge from their
interpretation. An important feature of our case study is that the source materials
are images, potentially having greater variation in how they are interpreted and
therefore increasing the need for intelligent support. Intelligent support provided
by our system can identify patterns and inconsistencies in the learner’s
interpretation and knowledge building, and offer recommendations based on
comparison with a reference model. The recommendations derived from a
reference model aim to guide the learner in reviewing and revising their
interpretation of the images and the implications of these for their inferences,
rather than directly providing a “right” or “wrong” evaluation of their answer.
Keywords. Inquiry learning, Learning from images, Machine learning, Decision
Tree learning.
1. Introduction
The increased availability of archive and web content and the established use of web-
based content creation and manipulation tools by young people further stress the
potential for inquiry learning from web-based content within the school curriculum.
The work presented here is being conducted in the SILVER project, funded by the
UK Technology Strategy Board. An objective of SILVER is to develop and test
learning scenarios and supporting software for constructivist learning from web
archives, in particular image archives. Our partners within the project are the
Bridgeman Art Library and Lexara, a software company specialising in the education
and museum sectors.
The software we are developing is intended to support inquiry learning in two
ways. First, visual web-based templates are being developed to guide the overall
learning activity. They are built from reusable modules in the Magic Studio platform.
Magic Studio is an environment developed by Lexara for the development of image-
based interactive components. Magic Studio is therefore used to structure the activity
and provide specific learning tools for tasks such as image exploration, annotation and
organisation.
Second, SILVER-RE (SILVER project Reasoning Engine) is being developed to
interpret learner performance and provide analyses and suggestions for delivery to the
student via Magic Studio. The overall approach taken to the design of SILVER-RE
focuses on identifying and using patterns in the task execution of the learner. This
could involve reasoning why a set of images have been grouped together from the
properties, metadata and labels that have been associated with them. The reasoning
carried out by SILVER-RE is therefore independent of the domain being taught and
does not construct a conventional model of the domain or learner. A learning scenario
making use of Magic Studio is described in section 3 however, the focus of this paper
is SILVER-RE.
2. Related work
There is continued interest in how software can support inquiry learning from
digital resources, in which the learner interprets the resources toward achieving some
higher level goal such as answering a question. Examples include support for teacher
selection and organisation of materials [1] and learner annotation of digital resources
[2]. An important contribution to our understanding of how software can support
humans in learning and reasoning by organising and building structure out of content
comes from work on spatial hypertext including Visual Knowledge Builder [3]. This
work aligns well with knowledge building theories of learning in which people are
encouraged to externalise their understanding and work with knowledge in design-
based activities [4].
Further related research can be seen in systems that automatically organise
collections of content to assist human interpretation using templates or narrative
structures [5, 6, 7]. Other work has looked how Socratic tutors can guide learners in
drawing conclusions from evidence [8]. Given our particular content focus, other
related work has investigated the role of images within learning. This has highlighted
how images can attract attention, provoke curiosity and help bring material to life [9] or
help in making cross-curricula connections [10].
3. Learning scenario
The initial learning scenarios we have been investigating in the project have four
key characteristics. First, there is an overall question or hypothesis for the inquiry task,
such as “How did suffragette campaigning methods change over time and why?”.
Second, each inquiry contains a number of experimental variables, in this example,
types of campaigning methods and time. Third, evidence is interpreted from the images
that essentially form data points within the experiment. Fourth, learning outcomes
related to the domain under study (in this case the suffragette campaign) involve
understanding and explaining patterns across the variables of the experiment. In this
case, understanding how the suffragette campaign changed over time and offering
reasons as to why those changes occurred.
The learning scenarios we have developed so far are aligned with the Key Stage 3
Citizenship curriculum (UK students aged 11 to 14). For the rest of the paper we will
refer to a particular example concerned with understanding the changing roles of
women during the late 19th and early 20
th centuries. Based on a unit of the Citizenship
curriculum, the inquiry activity involves studying a set of images and determining
whether each depicts women in a public or private setting. Additionally, the learner
identifies which roles are being undertaken by women in each picture.
Figure 1 shows the web interface to support the task. The learner selects images
from the bottom of the screen and, by drag-and-drop, indicates whether they are public,
private or both. Clicking on a image provides a larger view and an interface for
associating roles with the image. The interfaces were developed using Magic Studio
and use images from the Bridgeman Art Library.
Figure 1. Magic Studio interfaces to a Suffragettes learning activity.
The role of SILVER-RE is to help the learner identify patterns in the
interpretations they are making and also provide guidance to the learner by comparison
to a reference model.
Tests were conducted in four schools using the Magic Studio interface without
additional intelligent support. The objective was to test the activities and interfaces and
also inform what kind of intelligent support would be most appropriate. This was done
by observing the nature of the advice given by the teacher while conducting the task or
afterwards in a plenary session.
Learners were found to have problems in consistently identifying women’s roles
across the images, particularly for less familiar roles such as governess and nursemaid.
A second issue was a difficulty in appreciating patterns between the public-private
classification of images and their associated roles, such as noticing that certain care-
giving roles (e.g. governess, nursemaid) were depicted more often in the 19th
Century
images. The aim of SILVER-RE was to provide help with these kinds of difficulty by
identifying and reflecting patterns in the learner’s work.
4. SILVER-RE
Reasoning support was developed to provide guidance to the learner mimicking
either the “over the shoulder” prompts a teacher might give to a student working in a
computer lab or the prompts and questions that the teacher might offer during a plenary
discussion. This was done not to replace the teacher, but rather to allow the teacher to
focus on guiding more complex forms of interpretation and analysis that cannot be
delivered by the software.
Technically, an XML interface has been developed between Magic Studio and
SILVER-RE. Magic Studio provides a description of the current state of the learner
activity and the reference model (if provided) and SILVER-RE returns guidance for
presentation to the learner. Magic Studio communicates with SILVER-RE each time
the description of an image changes.
Two types of guidance are provided by the SILVER-RE: help in understanding
patterns in the data, and help in comparing against or moving toward a reference model.
These two types of guidance are described in sections 5.1 and 5.2. SILVER-RE, built
in Java, has its own developer interface which we can use to illustrate the functionality
it provides (see figure 2). The top left of the interface shows the names of the files
loaded in as “working” and “reference” sets. The working set is the current state of the
task as carried out by the student. The reference set is another attempt at the task by
either a teacher or fellow student.
Figure 2. The developer interface to SILVER-RE showing a decision tree representation.
A set of described and classified images can be thought of as a conceptual model
produced in response to the task. The model is expressed as attribute-value pairs
associated with the images and as values of classes to which the images have been
assigned. This is used as input data for a concept learning task [11, 12]. In our example,
women’s roles are represented as attributes and are associated with each image in the
form of attribute-value pairs. The public-private distinction is represented as values of a
class. While, in the example used here, all of the attributes have only yes or no values,
indicating whether of not each role is present in the image, multiple values for an
attribute can also be represented. The decision as to which task data are treated as
classes and which as attributes is related to the design of the inquiry activity and its
learning objectives. In our example, the representation of the task allows investigation
of how patterns in the assigned roles can predict or explain public-private classification
of an image.
The lower left part of figure 2 provides a view of the loaded data file. The left-
most table presents the frequency of each attribute within the image collection. To the
right of the table is a clickable list of images. These are colour-coded to indicate class
membership. The attribute-value pairs of the selected image are shown bottom left. The
functionality of SILVER-RE is described in the next section.
5. Providing guidance to the learner
SILVER-RE provides two kinds of guidance to the learner (see figure 3). The first
type of guidance is derived from patterns identified in the learner’s data as defined by
classes and attributes. This can draw the learner’s attention to consistent patterns in the
data or outlying, inconsistent examples. Second, guidance is derived from a
comparison against image classifications and attributes in a reference model.
This guidance prioritises differences from the reference model in terms of the
attributes used and their association with specific images. An important aspect of the
guidance is that it does not suggest directly changes the learner’s classification. Instead,
guidance is used to influence which attributes are used to annotate the images and then
provoke consideration of consistencies or inconsistencies between the learner’s image
annotation and classification. Therefore, rather than directly suggesting to the student
that the image depicts a public rather than a private setting, guidance focuses on
encouraging review of the annotations of the image (making it more consistent with
images depicting public settings) and then helping the learner to realise that its current
classification as private is inconsistent with the attributes (i.e. women’s roles) they
have associated with it. The aim is therefore to scaffold the learner through the process
of re-evaluating the data and its interpretation rather than just confronting them with a
set answer. Recommendations originating from a comparison with a reference model
can be portrayed as points of difference for discussion or as recommended next steps in
the learner’s development of their model. The recommendations can also be presented
with varying levels of subtlety. This will be discussed later.
Learner model Reference model
Classification Classification
Identification of patterns and
inconsistencies between the
learner classification and
annotations potentially leading to
their modification
Attribute
annotation
Attribute
annotation
Comparison of the learner and
reference models potentially
leading to modification of the
learner's annotations
Reasoning engine
input
Influences on the
learner model
Figure 3. Ways in which SILVER-RE influences the learner’s annotation and classification of images.
5.1. Consistency between annotation and classification
The ID3 decision tree learning algorithm [11, 12] is used to identify patterns
between the attribute annotations and classes. This machine learning algorithm is used
for three reasons. First, it is robust to missing and incorrect data, meaning patterns can
still be identified when the learner’s model is incomplete or inconsistent, which is what
is required. Second, decision trees can be constructed from relatively low numbers of
cases. This is important as students will work with tens rather than hundreds of images
within an inquiry learning task due time and motivational constraints. Third, decision
trees can represent disjunctive concepts (e.g. images in this class have either X or Y)
which is also required for many of the concepts the students will be working with.
The ID3 algorithm makes the optimal split at each point based on the local
information criteria. The decision tree produced by ID3 therefore maximally reduces
entropy at each step but not does necessarily optimize the information criteria across
multiple steps. For example, a non-optimal first level split may produce a decision tree
which after the second step reduces more entropy that a standard locally optimal ID3.
SILVER-RE makes a full search of all possible decision trees up to level three in order
to find the optimal pattern within the model. This involves generating all decision trees
in which each utilised attribute is tried as the root node, then each other attribute is tried
at the next level and so on. Due to the computational expense, an approximation is
made in order to get beyond level three. This involves running the standard ID3
algorithm and then performing a full search incrementally using only attributes that
feature in the ID3 decision tree. This is a reasonable heuristic as the selected attributes
have a proven property to distinguish the classes and the heuristic reduces the search
space. An optimal decision tree for the example model is shown on the right of figure 2.
The slider to the left of this allows depth to be controlled directly in the developer
interface.
SILVER-RE was developed to identify or approximate the optimal decision tree
for a specified level to provide the pattern in the data optimal for the required level of
detail and visual formalism used to present the pattern to the learner. For example, the
learner may be aided in understanding the patterns in their model by viewing a Venn
diagram in which the three sets represent the most distinguishing attributes in their
model and colour and/or symbols are used to represent classes associated with the
image. The optimal decision trees can be used to determine which three attributes
should be used as sets. A Venn diagram presentation using these optimal attributes can
be generated directly from the developer interface.
As well as identifying patterns for the learner, SILVER-RE can also find
inconsistencies or outliers in the current model. For example, if a set of images all had
similar annotations (e.g. a similar set of women’s roles) associated with them but one
was classified as being a public setting and the others as private this would suggest a
potential outlier or inconsistency, as central to the task and provided materials is the
strong relationship between role and the public-private spheres. Resolution would
involve altering either the annotation or classification of the image.
SILVER-RE identifies outliers or inconsistencies as nodes that are deep in the
decision tree relative to other nodes associated with the definition of that class.
Relatively deep nodes are calculated using the mean depth and its standard deviation
for a particular class type. This heuristic assumes that if, for example, all but one of the
images classified as public can be defined in the first few levels of the decision tree and
a large number of levels are needed to incorporate the final case, this case possibly
requires modification to its class membership or annotations. Within educational
scenarios, we assume that generally such outliers will be a consequence of mistakes in
their data. This assumption is made because an important aim of such inquiry activities
is for the learner to build knowledge by generalising from the relatively small set of
presented cases. It is less likely that the teacher would present a scenario with many
single cases requiring unique interpretation and lengthy explanations from which no
generalisation could be made.
This assumption, of course, does not necessarily hold outside of educational
contexts, for example the Duckbilled Platypus being a member of the mammal class.
However, even if such correctly defined outliers were present in an educational
scenario, support in noticing their uniqueness would still be valuable, even though it
should not lead to re-description of the case.
5.2. Comparison against a reference model
Decision tree learning is also used to compare the student’s work with a reference
model. This is done in three steps. First, a set of interesting patterns are identified from
the attributes and classes of the reference model. Second, the student model is
evaluated in terms of how many of these patterns it is consistent or inconsistent with.
Third, differences between the student and reference model are ordered in terms of how
many additional patterns each permits the student’s model to encompass.
This process is carried out in the following way. First, similar to the description in
the previous section, the standard ID3 decision tree is constructed, though this time on
the reference model. All attributes are extracted from this decision tree as examples of
distinguishing attributes within the reference model. This filtered set of attributes is
then used to construct the set of decision trees to a specified depth. As before, this is
done because a breadth-first search of all attributes would be too computationally
expensive. The set of decision trees, including the initial ID3 decision tree are then
used to generate a list of rules predicting the classification of an image based on its
annotation. The rules represent each path from node to leaf in each of the decision trees.
Within our example, a possible rule could be “Private IF nanny AND nursemaid”. The
Quine-McCluskey algorithm [13] is then used to generalize the rules by filtering and
merging them into a simpler, logically equivalent set. This process produces a set of
rules that provides rich characterisation of the patterns in the reference model. As these
are drawn from a set of decision trees they encompass many, if not all, reasonable
perspectives on how the attributes and classes of the images interrelate.
In the second step, the images as described by the student are run through the set of
rules to determine how many of the rules are consistent or inconsistent with the
learner’s description. This provides a metric of how well the student interpretation of
each image aligns with the reference model.
Figure 4. A prioritized list of suggestions derived from comparison with a reference model.
In the final step, ways in which the reference model differs from the student model
are applied to the student model. These can be either differences in annotations (e.g.
image 1 is annotated with nanny in the learner but not the reference model) or the
global use of attributes (e.g. the attribute maid is used in the reference model and
associated with images 1 and 2 but is not used in the student model). Each change is
rated in terms of how many additional rules it aligns with. As the rules are generated
from the reference model, the changes can only increase and not decrease the number
of matching rules.
An example set of suggestions generated from a comparison is shown in figure 4.
In this case, the change that would have the greatest effect (satisfying four additional
rules) is the use of the attribute mayor. Adding the attribute lady’s maid to two specific
images (entitled “Hickory Dickory Dock” and “Dressing for the Ball”) would each
satisfy two additional rules. A number of further changes would each satisfy one extra
rule. This method therefore provides a formal and justifiable way of prioritising the
differences between the two models. The set of identified differences can be treated in
two ways, either as suggestions to the learner to improve their model (if, for example,
the reference model is a set answer) or as points of difference for discussion (if, for
example, the reference model is that of a peer).
The next section describes our ongoing work, including developing new ways of
presenting the guidance to the learner.
6. Ongoing and future work
So far feedback from SILVER-RE takes the form of suggested visualization
formats, identification of potential outliers and suggested changes in terms of attributes
used and image annotations. We are currently exploring ways of specifying outliers and
making suggestions through Magic Studio beyond simple text prompts. These may
vary in terms of their subtlety (i.e. the amount of work the suggestion requires on the
part of the learner) and also their certainty (i.e. whether the suggestions from the
reference model are framed as improvements to their model, or just alternative
perspectives of equal weighting to their own). For example, we are looking at how the
introduction of a so far unused attribute could be facilitated by presenting a range of
images each featuring the attribute and asking the learner to look from similarities. If
the attribute is missed then highlighting image components could be used as additional
scaffolding. Such a technique would also provide a way of gradually extending the
learner’s image collection during the course of the task. We are currently working on
extending SILVER-RE to more generally take into account additional images initially
not used by the learner, and determining when the learner should be encouraged to
bring further images into their collection.
More broadly we wish to investigate how the approach could be applied to new
domains. We believe the approach could be particularly fruitful in contexts that are
more subjective and value-based, where patterns and inconsistencies could potentially
indicate biases in the learner’s perception of which they are not aware.
Acknowledgements. This work was conducted as part of the SILVER project, funded
by the UK Technology Strategy Board.
References
[1] Davis, S., Bogen, P., Cifuentes, L et al, Template-based authoring of educational artifacts. ACM/IEEE-
CS joint conference on Digital libraries. 2006, Chapel Hill, NC, USA.
[2] Quintana, C. and Fishman, B., Supporting science learning and teaching with software-based
scaffolding. American Educational Research Association (AERA), 2006, San Francisco, CA.
[3] Shipman, F., Hsieh, H., Airhart, R., Maloor, P. and Moore, J.M., The Visual Knowledge Builder: A
Second Generation Spatial Hypertext, ACM Hypertext, 2001, pp. 113-122.
[4] Scardamalia, M. and Bereiter, C. Knowledge building. Encyclopedia of Education. 2003, pp. 1370-1373.
[5] Alani, H., Kim, S., Millard, D. et al, Automatic ontology-based knowledge extraction from web
documents. IEEE Intelligent Systems, 2003, 18 (1), 14-21.
[6] Rutledge, L., Alberink, M., Brussee, R., et al, Finding the story - Broader applicability of semantics and
discourse for hypermedia generation. ACM Hypertext, 2003, Nottingham, UK.
[7] Mulholland, P., Collins, T. and Zdrahal, Z., Bletchley Park Text. Journal of Interactive Media in
Education, 2005. URL: http://jime.open.ac.uk
[8] Trella, M., Conejo, R. and Guzman, R., A Web-Based Socratic Tutor for Trees Recognition. Adaptive
Hypermedia, 2000, pp 239-249.
[9] Mowat, E., Teaching and learning with images. VINE, 2002, 32 (3), 5-13.
[10] Littleton, K., Twiner, A., Gillen, J., Mercer, N. and Kleine Staarman, J., Orchestration with the
Interactive Whiteboard. EARLI Conference, 2007, Budapest, Hungary.
[11] Mitchell, T., Machine Learning. 1997, New York, McGraw-Hill.
[12] Quinlan, J. R. Induction of decision trees. Machine Learning, 1986, 1 (1), 81-106.
[13] Hayes, J. P., Introduction to Digital Logic Design. 1993, Reading, MA, Addison-Wesley.
top related