Customized Selection and Integration of Visualization (CVIS) tool for Instructors

Gargi Banerjee, Anura Kenkre, Madhuri Mavinkurve, Sahana Murthy Interdisciplinary program in Educational Technology

Indian Institute of Technology Bombay Mumbai, India

e-mail: gargi@it.iitb.ac.in; anura.kenkre@iitb.ac.in; mavinkurvemk@gmail.com;sahanamurthy@iitb.ac.in@iitb.ac.in

Abstract— Instructors are not using well-designed computer-based visualization as widely as expected. Even among instructors using visualizations in teaching, their integration strategy is not aligned to their objectives with the visualization to bring about effective learning. As a result, the higher level learning outcomes possible with visualizations is often missed. In such context, we present a web-based tool, Customized Visualization Selection and Integration (CVIS) tool that selects visualizations customized to the specified instructional setting. It also guides instructors to implement the appropriate instructional strategy with visualization that is aligned to their instructional objective. The tool has been pilot tested for usefulness and effectiveness with engineering instructors with instructors accepting it as a practical solution towards effective teaching with visualizations.

Keywords- visualization; selection; instructional setting; instructional objective; instructional strategy

I. INTRODUCTION Computer-based visualizations can be described as “the

use of computer supported, interactive, visual representations of data to amplify cognition” [1]. In the current paper, the term ‘visualization’ refers to educational animations and simulations. These visualizations have been proven to be effective teaching-learning resource at the tertiary level. However, their use is not as widespread as expected because of difficulty in selecting the appropriate visualization for the chosen topic [2]. Another problem noted is even when instructors use technological tools like visualizations for teaching; they often fail to achieve effective integration [3]. To address the above problems, different types of search tools like recommender systems [4] have been built to aid in selection of visualizations based on metadata. For the integration problem, existing solutions include creation of learning design (LD) authoring tools that help instructors design an effective learning environment [5]. Though these LD tools provide pedagogical support to instructors, none of them provide guidelines to instructors on which instructional strategy to implement for a specific objective with visualization. Neither do the visualization selection tools address the instructional setting parameter.

Our solution is a web-based tool, CVIS (Customized visualization selection and integration) that introduces a customization layer for engineering instructors. It is not a metadata selection tool but works on the output of existing

recommender systems, customizing the selection to instructional setting. It extends the customization to the integration stage by recommending an instructional strategy aligned to the chosen instructional objective with the visualization. Both these are important factors to consider from instructional design principles. The CVIS tool is based on the CVIS framework that evolved from theoretical concepts of instructional designing, Bloom’s taxonomy, and two grounded theory studies [6], [7] that were our prior work. Our contribution in the current paper is implementation of the CVIS framework into a user-friendly tool that is of practical benefit to Engineering instructors.

II. RELATED WORK We present a comparative table of some of the existing

tools for selection and integration of visualizations below.

TABLE I. SAMPLE COMPARATIVE TABLE OF VISUALIZATION SELECTION & INTEGRATION TOOLS

Tools Target audience

Working Gaps

A. Selection

i) Recommen - der Systems [4]

Instructor & Learner

Apply an algorithm to combine explicit & implicit evaluation data to recommend relevant visualizations

i) Does not include instructional setting among contextualization ii) Evaluation is a challenge due to paucity of publicly available data

A. Selection ii) LOBSTER [8]

University instructor

Search visualizations by topic, digital format, language, content structure and uses clustering mechanism to present the relevant search results

i) No contextualization to setting ii) Instructors evaluated as easier to use than Google search engine

B. Integration

i) Learning Design authoring tool (CADMOS, Learning Designer) [5]

Instructor

Guide to apply pedagogical principles to build sound learning environment

i) Not focused on visualization ii) No guidance for aligning instructional strategy to instructional objective with visualization

C. Selection & Integration: SMART [9]

University Instructor

Checks compatibility between visualization and LD selected by instructor from the options given

i)Does not align LD to instructional objective with visualization ii)Does not consider instructional setting iii) Tool unevaluated

2014 IEEE 14th International Conference on Advanced Learning Technologies

978-1-4799-4038-7/14 $31.00 © 2014 IEEE

DOI 10.1109/ICALT.2014.119

399

III. WORKING OF THE CVIS TOOL The CVIS tool is based on the CVIS (Customized

Visualization Selection and Integration) framework that evolved out of the synthesis of our prior work [8], [9]. The first component of the framework guides instructors in selecting visualizations customized to their instructional setting (lecture, lab or tutorial). The inputs to the framework are instructor specification of instructional setting alongside course name and topic (Fig.1). The tool then matches the instructor inputs to the metadata database for existing visualizations and outputs a visualization that has features appropriate for that setting. The second component of the framework guides instructors in selecting appropriate instructional strategy aligned to the objective with the visualization along with the corresponding learning design. The input to the framework is an instructional objective with the visualization, besides desired time duration of the learning activity (10 mins. /20mins.) and type of activity problem (open-ended/close-ended) (Fig. 1).The tool then consults a lookup table of objectives and corresponding strategies created from analysis of interviews with expert teachers. The output is an instructional strategy conducive to achieving that objective with detailed guidelines on its implementation. For example, if input is an instructional objective like ‘Identify concepts, principles and/or parameters required to solve the problem’, activity duration = 10mins., nature of problem = closed, the tool consults the lookup table and recommends the strategy of ‘Peer Instruction’ as output along with its learning design.

Figure 1: Block Diagram of Working of CVIS Tool

IV. EVALUATION We have done two types of pilot evaluation of the CVIS

tool (usefulness survey and field implementation) with instructors drawn from the target user population of engineering instructors who use visualizations in their teaching. The usefulness survey measured how useful the instructor-users of the tool consider the tool to be. The survey was a 5-point Likert scale survey of 8-items, adapted

from TAM3 (Technology acceptance model) [10] that have widely been used to predict user adoption of new technology tools including educational tools. Analysis of survey responses of 10 instructors revealed high percentage of agreement (90%-100%) on the constructs of perceived usefulness and ease of use for the CVIS tool. Field implementation was done with 2 instructors using the visualization recommended by CVIS for tutorial setting on the topic of ‘Signal Transformation’. The instructional objective was at Apply level: students should be able to execute multiple signal transformations given the input signal. Applying the filter of activity duration, the ‘Think-Pair-Compare’ strategy was recommended for implementation. Post-implementation, the effectiveness of CVIS recommendations was measured through (i) instructor interviews and (ii) 2-group post-test only controlled experiment. The instructors noted a 50% drop in doubts asked while experimental group (following CVIS recommendations) showed significant difference (p=0.01; sample size = 150) on post-test scores.

V. FUTURE WORK As part of future work, we intend to expand the

usefulness survey and field implementation to larger number of instructors. We intend to present CVIS recommended LDs using IMS LD specifications to increase interoperability of recommended LDs. We also plan to expand field deployment of tool with larger number of instructors.

REFERENCES [1] M. Tory and T. Moller, "Rethinking Visualization: A High-Level

Taxonomy", IEEE Symposium on Information Visualization, pp. 151-158, 2004

[2] J. Sinclair, M. Joy, J. Yin-Kim Yau & Hagan, S. (2013). “A practice-oriented review of learning objects.”, IEEE Transaction son Learning Technologies, 6 (2).

[3] P.A. Ertmer, “Teacher pedagogical beliefs: The final frontier in our quest for technology integration?”, Educational technology research and development, 53(4), 25-39, 2005.

[4] J.Z. Li, “Quality, evaluation and recommendation for learning object”, IEEE Conference on Educational and Information Technology (ICEIT), 2, V2-533, 2010.

[5] Y. Mor & C. Brock, "Learning design: reflections upon the current landscape." , Research in Learning Technology, 20, 2012.

[6] A.Kenkre , G. Banerjee, M. Mavinkurve & S. Murthy, “Identifying Learning Object pedagogical features to decide instructional setting”, IEEE International Conference on Technology for Education, 2012.

[7] G. Banerjee, M. Patwardhan & M. Mavinkurve, “Teaching with visualizations in classroom setting: Mapping Instructional Strategies to Instructional Objectives”, IEEE International Conference on Technology for Education, 2013.

[8] C. Curlango-Rosas, G.A. Ponce, G. Lopez-Morteo & M. Mendiola, “Leveraging Google Web Search Technology to Find Web-Based Learning Objects”, Web Congress, 169-176, 2009.

[9] J. Lukasiak, S. Agostinho, S. Bennett, B. Harper, L. Lockyer & B.Powley,“Learning Objects and Learning Designs: An Integrated System for Reusable, Adaptive and Shareable Learning Content”. Assoc. for Learning Technology J., 13(2), 151-169, 2005.

[10] V. Venkatesh & H. Bala, “Technology acceptance model 3 and a research agenda on interventions”, Decision sciences, 39(2), 273-315, 2008.

400