Computers & Education 82 (2015) 292e305

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier .com/locate/compedu

When does higher degree of interaction lead to higher learning invisualizations? Exploring the role of ‘Interactivity Enriching Features’

Mrinal Patwardhan*, Sahana MurthyInter Disciplinary Program (IDP) in Educational Technology, Indian Institute of Technology Bombay, Ground Floor, Mathematics Building, Powai, Mumbai400 076, India

a r t i c l e i n f o

Article history:Received 24 June 2014Received in revised form18 November 2014Accepted 26 November 2014Available online 4 December 2014

Keywords:Interactive learning environmentsMultimedia learningSimulationsAffordanceEngineering education

* Corresponding author. Tel.: þ91 99693 67271.E-mail addresses: mrinalp@iitb.ac.in (M. Patwardh

http://dx.doi.org/10.1016/j.compedu.2014.11.0180360-1315/© 2014 Elsevier Ltd. All rights reserved.

a b s t r a c t

Interactive visualizations are being used extensively for effective teaching and learning. Higher degree ofinteraction in visualizations improves comprehension and leads to deeper learning. However, someresearch studies have reported ambiguous, inconclusive results in terms of learning benefits of inter-active visualizations. The conditional results in such studies suggest some additional features to beinstrumental in assisting learners in deriving benefits of interactivity in visualizations. We refer to thesefeatures as ‘Interactivity Enriching Features’. This study examines how degree of interaction of the userwith the visualization affects learning outcome. The study proposes how interactivity in visualizationscan be enriched by offering apt affordances and evaluates what additional features could make learningfrom interactive visualizations more effective at the same degree of interaction. The study has beencarried out in the context of a course on Signals and Systems in Electrical Engineering on second yearengineering students (N ¼ 134). The subjects were assigned to one of the four conditions: a Non-Interactive Visualization, an Animation, a Simulation, and an Interactivity Enriched Visualization. Thedependent variable was test-score for ‘Understand conceptual knowledge’, ‘Understand proceduralknowledge’ and ‘Apply procedural knowledge’ categories. The research findings indicate that, i) differentdegrees of interaction are required for learning different types of knowledge and ii) interactive visual-ization could not deliver its learning benefits unless it was augmented by ‘Interactivity Enriching Features’in the form of appropriate affordance for variable manipulation, especially for higher learning outcomes.This research study contributes towards the design of educationally effective interactive visualizations.

© 2014 Elsevier Ltd. All rights reserved.

1. Introduction

Use of computer-based interactive visualization is being advocated and suggested extensively as an instructional aid. It is being used by,right from elementary level school students up to university students, in a diverse range of topics. They have been used in the teaching-learning of elementary level science concepts (Barak, Ashkar, & Dori, 2011), as well as complex concepts or processes in engineering andallied courses (Boucheix& Schneider, 2009; Lattu, Meisalob,& Tarhioc, 2003; de los Santos Vidal, Jameson, Iskander, Balcells,& Catten,1996;Wang, Vaughnb, & Min Liu, 2011). Interactive visualizations, in particular, have been extensively used in the learning of science and en-gineering domains. Specifically, in engineering education, they have been recommended to comprehend phenomenon that are dynamicwith respect to time, space or any other variable (Aleksandrova & Nancheva, 2007; Engin, 2006; McManus & Rebentisch, 2008; Kühl,Scheiter, Gerjets, & Gemballa, 2011). Interactive visualizations allow learners to interact with the educational content (Chaturvedi &Osman, 2006; Park, Lee, & Kim, 2009; de los Santos Vidal et al., 1996). The interaction in such visualizations is said to occur when someresponse is elicited from the learner and in turn, the visualization is able to respond to the learner's input. The quality of such interaction hasbeen referred to as interactivity (Sedig & Liang, 2006).

Degree of interaction in interactive visualizations can be varied by varying the amount of learner control. Degree of learner control mayvary from viewing still images up to manipulating content and further up to creation of visualizations. Animation and simulation have been

an), sahanamurthy@iitb.ac.in (S. Murthy).

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305 293

two common variants of interactive visualizations offering varying degree of interaction. While an animation offers pace or direction controlof the animated educational content, a simulation offers opportunities to learner to explore interactions among dynamic variables byallowing manipulation of the educational content (Lin& Atkinson, 2011; Park et al., 2009). The degree of interaction plays an important rolein the learning process (Schulmeister, 2003). In addition to giving control of the information delivery (as in animation) in visualization,when the learner is also allowed to manipulate its content (as in simulation), higher level of thinking is expected to happen (Plass, Homer,&Hayward, 2009) and leads to deeper learning in terms of conceptual and analytical thinking (Stieff & Wilensky, 2003; Wu, Krajcik, &Soloway, 2001).

Interactive visualizations have been known to offer several benefits. By providing opportunities to practise ‘what-if’ scenarios, theinteractivity in the visualizations fosters learners' analytical skills. The learning benefits derived from their use have covered a range ofabilities and skills, such as, explanation ability, comprehension, ability to correlate scientific concepts, learning of conceptual and proceduralknowledge, process of skill acquisition, building of mental models, as well as increased learners' engagement (Kriz & Hegarty, 2007;Rutten,Van Joolingen, & Van der Veen, 2012; Schwan & Riempp, 2004). However, in spite of strong empirical results of learning benefits,it cannot be claimed unanimously that interactive visualizations improve learning, as some studies have not produced convincing evidencefor demonstrating improvement in learning (Hansen, 2002; Domagk, Schwartz, & Plass, 2010; Moreno & Valdez, 2005). Thus, whetherhigher degree of interaction in visualization always leads to effective learning has always been a stimulating research question.

While exploring the relevant literature to answer the above given question, it is observed that, many experimental results related tolearning from interactive visualizations have been conditional and the learning benefits were ensured only in the presence of additionalconditions (Hansen, 2002; Liang, 2006; Lin & Atkinson, 2011; Spanjers, Van Gog, Wouters, & Van Merri€enboer, 2012; Tversky, Morrison, &Betrancourt, 2002). The mixed results from empirical studies suggest that more research is needed to explore, ‘what influences learningfrom interactive visualizations?’ (Rey, 2011). The conditional nature of experimental results suggest that, not just the given degree of learnercontrol, but, some additional features must have been pivotal in assisting learners in deriving learning benefits of interactivity frominteractive visualizations. We propose to refer to such features as ‘Interactivity Enriching Features’, as we anticipate that these features wouldenrich the quality of interactions. These are visualization features as an affordance to users, embedded in interactive visualization. It isexpected that users' physical interaction with these features will lead to the improvement in learning from interactive visualizations.

While affordance refers to all “action possibilities” latent in the environment (Gibson, 1977), in the context of interactive learningenvironment, it refers to the way interactive visualization offers its usage cues and indicates what sorts of operations can be performed on it(Sedig & Sumner, 2006). As affordances relate to both; the degree and quality of interaction, different affordances offered by varying degreeand/or quality of interaction, provide different learning opportunities to learners. The real potential of interactive visualization is achievedwhen these learning opportunities are used intentionally and effectively. To facilitate this aspect, we used two ‘Interactivity EnrichingFeatures’ in the form of affordance. In this study, we demonstrated how the said affordances qualified as ‘Interactivity Enriching Features’ andled to effective learning by effectively utilizing the inherent learning potential of interactive visualizations.

The study has been carried out in the context of a course on Signals and Systems in Electrical Engineering. Signals and Systems is one ofthe foundation courses in the field of Communication and Signal Processing. Numerous interactive visualizations in the form of Java applets,MATLAB/Simulink® models, and LabVIEW models are available and are frequently used as a learning aid (Guan, Zhang, & Zheng, 2009;Kehtarnavaz, Loizou, & Rahman, 2008). Various resources containing interactive visualizations such as SYSTOOL, SSUM, J-DSP Tutor, and‘Interactive learning resources for Signal, Systems and Controls’ (Crutchfield & Rugh, 1997; Rabenstein, 2002; Shaffer, Hamaker, & Picone,1998; Spanias, Chilumula,&Huang, 2006; Sturm,& Gibson, 2005) have been recommended for learning of Signals and Systems. The need tovisualize abstract concepts, to understand multiple representation forms of these concepts, and to apply multiple computational steps(Nelson, Hjalmarson, Wage, & Buck, 2010) are some of the reasons that have made interactive visualizations prevalent in the Signals andSystems teaching community.

One of the motivations for this study was to investigate the highly intuitive and well-accepted notion of ‘higher degree of interactions invisualizations leads to better learning’. This study was set with the purpose of examining whether higher degree of interaction and theaffordances in the form of ‘Interactivity Enriching Features’ lead to effective learning. Firstly, we hypothesized that, higher degree of inter-action in interactive visualizations would lead to better learning of conceptual and procedural knowledge. Another motivation of this studywas to evaluate what additional features could make learning from interactive visualizations more effective at the same degree of inter-action. Thus, we also hypothesized that, at the same degree of interaction, the interactive visualizations with ‘Interactivity Enriching Features’would be more effective for learning as compared to the interactive visualizations without such features. This study reported in this paperinvestigated, how degree of interaction and apt affordance contribute towards effective learning from interactive visualizations, especiallywhile dealing with conceptual and procedural knowledge in the context of a course on Signals and Systems.

2. Review of related work

2.1. Interactivity in visualization

Visualization is “the use of computer supported, interactive, visual representations of data to amplify cognition” (Tory & M€oller, 2002).Interactivity is the process of learner engagement with the content in the visualizations, in which the learners' behaviour depends on theaction of the system, which in turn depends on the reaction of the learner, and so on (Domagk et al., 2010). Interactivity is not merely aninteraction. While interaction refers to various kinds of actions initiated by a learner to interact with the different visualization features,interactivity is the quality or property of such interactions. Lower interactivity implies a behaviourist character of a learner, while higherinteractivity leads to constructivist learning, such as discovery learning. The learning experience from interactive visualizations has beenoperationalized in the form of hierarchical nature of learner control being offered.While referring to this as ‘level of interactivity’, the authorhas defined six hierarchal levels of interactivity offering varying degree of possible interactions (Schulmeister, 2003). From a functional andqualitative perspective, interactivity has been organized in a hierarchical manner by number of researchers, in which, distinct levels ofinteractivity have been proposed by B�etrancourt and Tversky (2000), El Saddik (2001), Gao and Lehman (2003), Hanisch and Straber (2005),Reichert and Hartmann (2004), Schulmeister (2003). While the number of levels in the hierarchy and nomenclature differ in each

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305294

classification, the interactivity is being associated with the learning experience and learners' cognitive processes while interacting withvisualizations. Interactivity levels or degree of interaction, starting from the lowest and subsequently moving towards the highest, reflectthe amount of mental engagement of users, and reflect learner's role from being a passive learner to an active learner.

2.2. Learning impact of interactive visualizations

The various learning benefits from visualizations have been seen in terms of enhancement in the instruction delivery in the classroom,support in terms of pre-laboratory exercises, better comprehension, shortened learning time (Millard, 2000; Pinter, Radosav, & Cisar, 2010;Rutten et al., 2012). The content represented by means of visualization in an interactive manner develops a deeper and clearer under-standing of a topic (Barak et al., 2011; Lengler & Eppler, 2007). With regard to affective domain, visualizations are known to stimulatestudents' interest, motivation and their engagement in the teaching-learning process (Barak et al., 2011; Rutten et al., 2012). While chal-lenging learners' exploratory, interpretational, and sense-making abilities (Card, Mackinlay,& Shneiderman,1999; Chen, 2004; Imhof, 2011;Jonassen, 2006; Liang & Sedig, 2010; Spence, 2007; Thomas & Cook, 2005; Ware, 2004) such interactive tools enforce learners to come upwith their own investigative strategies, thus following a constructivist approach to build knowledge (Liang & Sedig, 2009).

While numerous benefits exist, results of some studies of interactive visualizations are seen to be ambiguous in nature. The desirablelearning benefits from interactive visualizations have been assured only along with certain conditions. For example, students learning frominteractive visualizations have performed better for specific knowledge types (Hansen, 2002) such as conceptual and procedural knowledgebut not for factual knowledge (Kombartzky & Ploetzner, 2007). Students' prior knowledge (Park et al., 2009) and reasoning abilities(Boucheix & Schneider, 2009) played a significant role in learning from interactive visualizations. The widely assumed intrinsic superiorityof animations over static graphics was questioned in an experiment wherein the need to give proper attention to the visuo-spatial char-acteristics of graphics was emphasized (Lowe, 2003). Another study confirmed that an interface design of an interactive visualization had arole to play in enhancing comprehension of a dynamic process (Boucheix & Schneider, 2009). Some of the visualization features, such as;content representation format, temporal aspects of learner control, variable manipulation, cueing and signalling, degree of interaction haveinfluenced learning from visualizations. The general impression about higher degree of learner control leading to higher learning gain wascontradicted in a study wherein higher degree of interaction decreased learners' ability while using the program (Wang, Vaughn, & Liu,2011), suggesting the need for differing level of interactivity while addressing to different levels of cognitive tasks. Theoretical argu-ments such as excessive extraneous cognitive load, more learning demands on learners, split attention theory (Austin, 2009; Kalyuga, 2007;Korakakis, Pavlatou, Palyvos,& Spyrellis, 2009) and expertise reversal effect (Kalyuga, 2005; Kalyuga, Ayres, Chandler,& Sweller, 2003) havebeen suggested as some of the factors that interfere with or impede learning from interactive visualizations.

3. ‘Interactivity Enriching Features’ in this study

To elicit learning benefits of interactivity in visualization, we have proposed two ‘Interactivity Enriching Features’ in this research study.Opportunity to manipulate educational content of interactive visualization dynamically is a very powerful learning tool offered to a

learner. The manner in which learner manipulates the content of visualization dynamically is the core of the learning process. Variablemanipulation of educational content of a visualization, done in a playful manner, without any specific intention, can be detrimental to thelearning process (Guzman, Dormido, & Berenguel, 2010). On this background, restricting and guiding variable manipulation process canchannelize proper utilization of the learning opportunities offered in interactive visualizations. Such restricted actions or constraints, inspite of being restrictions on users, are considered ‘productive constraints’ when they aid in the learning process (Podolefsky, Moore, &Perkins, 2013; Wang et al., 2011) and foster learning by aligning instructor's learning objectives with the delivery of educational contentin interactive visualizations. Thus, affordance of offering ‘Productively Constrained Variable Manipulation’ (PCVM), is considered as one of the‘Interactivity Enriching Features’ in our study.

Another affordance offered by visualization in this research study has been in the form of additional variable for manipulation. Generally,interactive visualization offers manipulation of parameters that influence the phenomenon/concept being depicted in the visualization. Insuch scenarios, users carry out manipulation by controlling the value of typical parameter in the given range as permissible in the givenvisualization. We refer to this kind of manipulation as Parametric Manipulation, as this manipulation leads to variation in the values ofparameter to be controlled. The corresponding variables getting manipulated can be termed as Parametric Variables. Most of the interactivevisualizations offer variable manipulation of this kind. Nevertheless, some of the learning goals, especially catering to procedural knowledgeexpect learner to perform sequence of action steps. As procedural knowledge is the one that exhibits an ability to flexibly use and applyalgorithms and procedures, while dealing with it, decisions about sequencing the steps of procedural task form an important aspect of itslearning. Considering this learning demand of the educational content to be delivered, we have introduced an additional variable to varyaction sequences related to procedural task; this category of variables is being termed as Permutative variables as they allow number ofpermutations of action sequences while executing a procedural task. Thus, affordance of ‘Permutative Variables’ in the form of ‘PermutativeVariable Manipulation’ (PVM) in addition to ‘Parametric variable’ facilitates learners with, not just a control on the quantitative range ofvariable to be manipulated, but also on the sequence in which such multiple variables would get manipulated. The above mentioned‘Interactivity Enriching Features’ were embedded while developing the learning material and its impact while acquiring conceptual andprocedural knowledge was investigated.

4. Research questions and hypotheses

The learning outcomes of Signals and Systems expect students not just to comprehend various concepts from the course, but also toapply them in a meaningful manner while attempting associated procedural tasks. In general, science and engineering curriculum focuseson different types of knowledge. The conceptual and procedural knowledge are two mutually-supportive factors associated with thedevelopment of engineering skills (Taraban, Definis, Brown, Anderson, & Sharma, 2007). Thus, the learning objectives for most the coursesin science and engineering curriculum, at least, aim at understanding the core concepts and applying them in the given context. In order to

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305 295

meet these domain specific requirements, we focussed on ‘understand’ and ‘apply’ cognitive levels for ‘conceptual’ and ‘procedural’knowledge, in conformance with the two-dimensional taxonomy framework as proposed by Anderson (Anderson et al., 2001) to measurelearning effectiveness of interactive visualizations.

The broad level research problem being addressed for this study has been, ‘Given the knowledge types and cognitive levels, what featuresare needed in interactive visualization for effective learning in the context of Signals and Systems?’

To evaluate learning impact of variation in the degree on interaction, our first research question is:

RQ1: Given the type of knowledge and cognitive level, does higher degree of interaction, lead to effective learning?

Our second research question aims at investigating the learning impact of ‘Interactivity Enriching Features’.

RQ2: Given the type of knowledge and cognitive level, how do ‘Interactivity Enriching Features’ affect learning in interactive visualizationwith the same level of interaction?

To answer these research questions, we used the following types of visualizations:(a) a Non-Interactive Visualization (Non-IVZ), (b) anAnimation (ANM), (c) a Simulation (SIM), and (d) Visualization with ‘Interactivity Enriching Features’: Interactivity Enriched Visualization(IE-Viz). The first three visualizations (Non-IVZ, ANM, SIM) had different degree of interaction and learning from them was compared toanswer the first research question. The SIM and IE-Viz, both had same degree of interaction, but, they had different affordances as ‘Inter-activity Enriching Features’. Learning from these two visualizations was compared to answer the second research question.

We had the following hypotheses for our study. Firstly, based on the premise that giving control to learner while working with visu-alization leads to effective learning, we expected that students, learning with visualization offering higher degree of interactionwould learnbetter as compared to students learning with visualization that offered lower degree of interaction. Thus, the hypotheses for RQ1 were:

H1-A) For Conceptual Knowledge at Understand level, students learning with Simulation (SIM)will score higher as compared to studentslearning with Non-Interactive Visualization (Non-IVZ) and also, to students learning with Animation (ANM).H1-B) For Procedural Knowledge at Understand level, students learning with Simulation (SIM) will score higher as compared to studentslearning with Non-Interactive Visualization (Non-IVZ) and also, to students learning with Animation (ANM).H1-C) For Procedural Knowledge at Apply level, students learning with Simulation (SIM) will score higher as compared to studentslearning with Non-Interactive Visualization (Non IVZ) and also, to students learning with Animation (ANM).

Secondly, as one of our motivations for this study, we wanted to assess how the proposed ‘Interactivity Enriching features’ influencelearning from visualizations. Thus, we hypothesized that at the same degree of interaction, visualization with ‘Interactivity Enriching Fea-tures’ offered in the form of affordance (IE-Viz) would lead to more effective learning when compared with the visualization without such‘Interactivity Enriching Features’ (SIM). Thus, we formulated following hypotheses for our second research question.

H2-A) For Conceptual Knowledge, students learning with IE-Viz score higher for ‘understand’ level tasks than students learning withSimulation (SIM).H2-B) For Procedural Knowledge, students learning with IE-Viz score higher for ‘understand’ level tasks than students learning withSimulation (SIM).H2-C) For Procedural Knowledge, students learningwith IE-Viz score higher for ‘apply’ level tasks than students learningwith Simulation(SIM).

The hypotheses mentioned above were tested with the help of a research study focused on the topic ‘Signal Transformation’; a topic thatdeals with the basic transformation operations on signals such as time shifting, time scaling, time reversing and amplitude scaling.

5. Research study

5.1. Materials and method

5.1.1. Participants and experimental designParticipants were students from second year of engineering from colleges affiliated to University of Mumbai (N ¼ 134; 109 males and 25

females). The study was conducted using a 4-group post-test only experimental research design. The post-test scores were used to evaluatethe effectiveness of treatment for various groups. A pre-test was not found to be essential, since students did not have any prior knowledgerelated to the topic andwere exposed to the educational content in the visualizations for the first time. None of the previously learnt coursesfrom curriculum exposed students to the topic on Signal Transformation. Participants were randomly assigned to one of the following fourconditions: (a) Non-interactive visualization (Non-IVZ); N ¼ 41 (b) Animation (ANM group); N ¼ 35 and (c) Simulation (SIM group); N ¼ 23(d) Interactivity Enriched Visualization (IE-Viz group); N ¼ 35. While creating matched-random assignment groups, scores of previoussemester examination were considered for matching the group equivalence. Non-reporting of some of the participants led to unequalsample size among the groups. There was no significant difference found between the performance scores of the students from each group.The mean examination scores were: Non- IVZ (M ¼ 66.906, SD ¼ 7.25), ANM (M ¼ 66.313, SD ¼ 7.24), SIM (M ¼ 67.591, SD ¼ 6.48), IEF-Viz(M ¼ 63.845, SD ¼ 7.95). The one-way ANOVA showed F to be non-significant at p > 0.05 (p ¼ 0.192).

5.1.2. MaterialsThe instructional intervention for the four conditions was as follows:

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305296

(a) The Non-Interactive Visualization (Non-IVZ) group used non-interactive form of the learning material which explained signal trans-formation operations with still images. These operations include single and multiple transformation operations on signals; such asamplitude scaling, time shifting, time scaling and time reversing.

(b) The ANM group studied the same content in the form of animation that offered only play-pause-stop control to learner. It did not offerany opportunity to learner to change educational content of visualization.

(c) The SIM group learnt with an interactive JAVA applet offering dynamic manipulation of variables for interacting with the educationalcontent. The applet offered opportunity to learners to study any one or multiple transformation operations by selecting single ormultiple transformation operations such as amplitude scaling, time shifting, time scaling and time reversing. After selecting a typicaltransformation operation, it also allowed user to set the magnitude of the selected operation by varying the scroll bar. Fig. 1 showssnapshot of the applet screen interface.

(d) Interactivity Enriched Visualization (IE-Viz group): This applet had ‘Interactivity Enriching Features’ (IEFs) embedded into the interactivevisualization. It offered affordance in the form of ‘Productively Constrained Variable Manipulation’ (PCVM) by means of controlling thenumber of variables offered simultaneously for manipulation (Fig. 2). The variables to be manipulated were offered in a controlledmanner. Gradual introduction of variables for manipulation was implemented by offering only one single transformation operationinitially (Tab 1: any one out of amplitude scaling, time shifting, time scaling and time reversing), then two operations (Tab 2:Commutativity of Transformation) and finally followed by all the four transformation operations from the topic on Signal Trans-formation. (Tab 3: Multiple Transformation). Along with this, students were also offered another affordance in the form of ‘PermutativeVariable Manipulation’ (PVM) (in Tab 2: Commutativity of Transformation) to control the action sequencing of the procedural knowledgetask i.e. they could control the sequence in which transformation operations related to procedural knowledge could be carried out.

To test the equivalence of the learning materials in factors other thanwhat was considered in the experiment, such as usability, and lookand feel, the materials were tested with students who had already studied the course on Signals and Systems. They were asked to interactwith the content and were assessed by the instrument, SUS Scale (Brooke, 1996). SUS is a ten-item Likert scale survey with a score rangefrom 0 to 100, that gives a global view of subjective assessments of usability. All the four types of learning materials were tested to establishequivalence of the learning material and were found to be equivalent. Total 70 students participated in this exercise. The mean SUS scalescores were (Non- IVZ (M¼ 76.39, SD ¼ 9.56), ANM (M¼ 77.22, SD¼ 9.92), SIM (M¼ 81.32, SD ¼ 10.83), and IE-Viz (M¼ 80.15, SD¼ 10.59).The one-way ANOVA showed F to be non-significant at p > 0.05 (p ¼ 0.437).

5.1.3. Measures and instrumentsThe assessment instrument was developed to test students' learning in terms of ‘understand’, ‘apply’ cognitive levels and for ‘conceptual’

and ‘procedural’ types of knowledge. Work on the Signals and Systems Concept Inventory (SSCI) (Wage, Buck, Wright, & Welch, 2005) andthe work reported in Hiebert & Lefevre (1986) have reported the necessity to focus on both these types of knowledge as well as on their co-existence. In terms of learning outcomes, students are expected to comprehend various concepts from the course and also to apply them in ameaningful manner while attempting associated procedural tasks. Thus, the ‘understand’ and ‘apply’ cognitive levels as defined in theRevised Bloom's Taxonomy (Krathwohl, 2002) were emphasized in this study. While developing instrument for evaluation of learning

Fig. 1. Screen shot of the Simulation applet developed on Signal Transformation.

Fig. 2. Screen shot of the IEF-Viz applet developed on Signal Transformation.

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305 297

effectiveness of interactive visualizations, we focused not only on the type of knowledge, but also simultaneously on the cognitive level ofthe task to be accomplished. This also fulfilled engineering curriculum's requirement of developing learner's expertise, not just in the giventype of knowledge, but also at the desired cognitive level of that knowledge. To our knowledge, evaluating learning effectiveness ofinteractive visualization by considering not just the knowledge type, but also, along the level of cognitive task has not been reported in theprevious relevant research studies.

In the topic of Signal Transformation, individual transformation operation constitutes conceptual knowledge, while multiple trans-formations to be carried out on a signal in an algorithmic manner is an example of procedural knowledge. With regard to cognitive level ofthe task, the questions related to ‘understand’ cognitive level were in the form of identifying or interpreting the single or multiple trans-formation operations, whereas, at ‘apply’ cognitive level, students were expected to use their comprehension to solve the question byapplying their knowledge in the new situation. The ten assessment questions were distributed across these three categories of ‘understandconceptual knowledge’, ‘understand procedural knowledge’ and ‘apply procedural knowledge’. The normalized scores were comparedacross the groups for the above mentioned categories. Some sample questions have been provided in Appendix A.

5.1.3.1. Content validity by experts. The instrument was developed and peer-reviewed by the researchers of this study in cooperation withfive domain experts who had 20 þ years of teaching experience in the domain of Signals and Systems. Two reviewers also had a formalbackground in Educational Technology research. The review process was carried out in an iterativemanner. The suggestions given on time totime basis were incorporated and instrument was further reviewed till all the reviewers were satisfied with the categorization of thequestions and, and their appropriateness in the context of learning objectives. One of the suggestions received during review process was tokeep signal waveforms simple to avoid students making false mistakes in drawing complicated waveforms while transforming signals. Twoof the initial questions from original assessment tool were dropped on the recommendations of the reviewers as the questions were testingreal world applications of signal transformation concepts, while the learning material was not designed to cater to this learning objective.

5.1.4. Procedure5.1.4.1. Pilot study. Five students who had already studied Signals and Systems took part in a pilot study whose aimwas to determine if thelearning materials, assessment instruments and procedure were suitable and aligned. The pilot study was carried out prior to conductingthe main study to get feedback about various feasibility and usability related issues regarding the learning material, instrument andexperiment procedure. Students gave feedback about the clarity and comprehension of the visualizations and the post-test. The students didnot report any flaw in understanding and interpreting the assessment questions, and mathematical expressions and graphical represen-tation of the waveforms wherever applicable. The pilot experiment conducted also confirmed sufficiency of the time allotment for thetreatment as well as for the problem solving session.

5.1.4.2. Main research study. The study was performed during lab hours where all participants could use individual computer terminals.Participants were informed that the purpose of this experiment was to investigate the effectiveness of interactive computer visualizations inteaching relevant concepts from Signal and Systems. Participants had signed consent forms and were also informed that all the collectedinformation would be kept confidential. Detailed directions on how to use the interactive computer simulation programs were given to theparticipants. All the participants were given 25 minutes of time to interact with the learning material. This was followed by administrationof the test for the next 30 minutes. During the test, participants were not allowed access to the learning material. None of the participants

Table 1Mean scores and standard deviations of the test.

Question category Non-interactive visualization (Non-IVZ)N ¼ 41

Animation (ANM)N ¼ 35

Simulation (SIM)N ¼ 23

Interactivity enrichedvisualization (IE-Viz)N ¼ 35

M SD M SD M SD M SD

Understand Conceptual knowledge 7.97 2.09 7.52 2.04 6.81 2.56 7.24 2.49Understand Procedural knowledge 5.73 3.63 3.43 3.98 3.04 2.92 5.86 3.93Apply Procedural knowledge 3.86 2.99 3.14 2.28 3.91 1.99 5.57 3.08

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305298

demanded more amount of time during learning phase or during test phase. After the experiment, participants were thanked and weregiven participation certificates.

5.2. Result

Table 1 below shows the mean and standard deviations of test scores conducted for the research study.Due to the fact that the test scores violated the assumption of normal distribution, the KruskaleWallis test, a non-parametric equivalent

of a one-way ANOVA, was used to compare the experimental conditions. However, the test scores passed the Levene's test of Homogeneityof Variances, thus confirming equal variances across the groups for all the three categories of test scores (Levene statistic: Understandconceptual knowledge p ¼ 0.601, Understand Procedural Knowledge p ¼ 0.262, and Apply procedural knowledge p ¼ 0.124). Thus, as thehomoscedasticity of data was ensured, the statistical tests selected (KruskaleWallis test and ManneWhitney U test) were suitable for thefurther data analysis for unequal ‘n’ sample size (Glass & Hopkins, 1970).

The KruskaleWallis test gave the following results: Understand conceptual knowledge (c2(3)¼ 3.613, p¼ 0.306), Understand proceduralknowledge (c2(3) ¼ 14.062, p ¼ 0.003), Apply procedural knowledge (c2(3) ¼ 14.667, p ¼ 0.002). For ‘Understand conceptual knowledge’,there was no statistically significant difference between the test scores. However, as the test score for ‘Understand procedural knowledge’and ‘Apply procedural knowledge’ showed statistically significant difference, the ManneWhitney U was used for further analysis of theresults. The result of ManneWhitney U test for the groups were found to be as shown in Table 2.

5.2.1. Result analysis: RQ1The first research question of the study was about impact of degree of interaction on effective learning for the given type of knowledge

and cognitive level. In order to answer this research question, average test score of the students studied fromNon-IVZ, ANM, and SIM groupswere compared as these three groups differed in terms of degree of interaction.

5.2.1.1. Learning impact on understand conceptual knowledge. The KruskaleWallis test on the post-test scores of these three groupsdemonstrated that there was no statistically significant difference between the scores related to Understand Conceptual knowledge acrossthe groups. All the groups; Non-IVZ, ANM, and SIM performed equally well for the assessment questions related to ‘Understand Conceptualknowledge’ category. This was evident from the p value obtained after running KruskaleWallis test (p ¼ 0.306) and also from the individualcomparisons among the groups for ‘Understand Conceptual Knowledge’ category test scores (column 2 of Table 2 shows individual p valuesobtained after running ManneWhitney U test). Thus, hypotheses H1-A and H2-A were not supported by the results obtained. Thisestablished sufficiency of non-interactive visualization (Non-IVZ) (the lowest degree of interaction) for tasks of category ‘UnderstandConceptual knowledge’.

5.2.1.2. Learning impact on understand procedural knowledge. The KruskaleWallis test on the post-test scores of all the three groups andManneWhitney U test reported the following results. While comparing test scores of Non-IVZ, ANM and SIM groups; Animation (ANM) andSimulation (SIM) groups performed equally well (p ¼ 0.931) on the assessment questions related to ‘Understand Procedural knowledge’category. However, therewas statistically significant difference between the scores for Understand Procedural knowledge for i) Non-IVZ andANM groups and ii) Non-IVZ group and SIM groups (p ¼ 0.010 and p ¼ 0.004 respectively). These results did not support hypothesis H1eB.Non-interactive visualization was found to be superior for ‘Understand procedural knowledge’ as compared to ANM and SIM.

5.2.1.3. Learning impact on apply procedural knowledge. The KruskaleWallis test on the post-test scores of all the three groups and Man-neWhitney U test reported the following results. While comparing test scores of Non-IVZ, ANM and SIM groups to answer the first researchquestion, though the Simulation (SIM) group had themaximum test score among the three groups, no statistically significant difference was

Table 2Results of ManneWhitney U test.

Experimental groups Understand conceptual knowledge Understand procedural knowledge Apply procedural knowledge

ManneWhitney U p ManneWhitney U p ManneWhitney U p

Non-IVZ and ANM 632.500 0.321 485.000 0.010 638.500 0.395Non-IVZ and SIM 356.000 0.073 284.000 0.004 433.500 0.582Non-IVZ and IE-Viz 607.500 0.209 699.500 0.840 473.000 0.010ANM and SIM 347.500 0.324 397.500 0.931 315.000 0.145ANM and IE-Viz 582.500 0.698 413.000 0.013 313.500 0.000SIM and IE-Viz 370.000 0.575 242.000 0.006 249.500 0.013

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305 299

found among the test scores of this category. Thus, comparison of Apply Procedural knowledge scores for Non-IVZ, ANM and SIM groupscould not support hypothesis H1-C.

5.2.2. Result analysis: RQ2The second research question of the study was about impact of Interactivity Enriching Features on effective learning from visualizations

at the same level of interaction for the given type of knowledge and cognitive level. In order to answer this research question, average testscore of the students studied from SIM and IE-Viz groups were compared. As explained earlier in the section 5.1.2 of this paper, SIM and IE-Viz had same degree of interaction, but IE-Viz offered affordance in the form of ‘Interactivity Enriching Features’ of ‘Permutative VariableManipulation’ (PVM) and ‘Productively Constrained Variable Manipulation’ (PCVM).

5.2.2.1. Learning impact on understand conceptual knowledge. SIM and IE-Viz performed equally well for the assessment questions related to‘Understand Conceptual knowledge’ category. Thus, hypothesis H2-A was not supported by the results obtained. This result, whenconsidered along with the result obtained in the context of hypothesis H1-A established the sufficiency of non-interactive visualization(Non-IVZ) for tasks of category ‘Understand Conceptual knowledge’.

5.2.2.2. Learning impact on understand procedural knowledge. For ‘Understand Procedural Knowledge’ category questions, IE-Viz group wasfound to be superior and a statistically significant difference was found between the test scores of SIM and IE-Viz group (p ¼ 0.006). Thisresult supported the hypothesis H2-B. When all the four groups test scores were compared, though IE-Viz group score was the highestamong them, we could not find statistically significant difference between Non-IVZ and IE-Viz group test scores. Our interpretation of theresults in this context has been discussed in detail in the discussion section of this paper.

5.2.2.3. Learning impact on apply procedural knowledge. The test score for Apply Procedural knowledge for IE-Viz group was found to behigher and statistically significant as compared to SIM group score. This result supported the hypothesis H1-C indicating supremacy of IE-Vizgroup for effective learning in ‘Apply Procedural Knowledge’ category. The statistically significant difference found between the test scoresof IE-Viz group and each of the Non- IVZ, ANM, SIM groups (p ¼ 0.010, p ¼ 0.000, p ¼ 0.013 respectively) confirmed necessity of IE-Viz foreffective learning of tasks catering to Apply Procedural knowledge.

Please refer to Table 4 provided in the discussion section of this paper for the summary of research questions, experimental results andtheir inferences.

5.3. Understanding students' perception and visualizations exploration approach while learning from interactive learning material

In order to assess students' perception about appropriateness of interactivity level for the given task and how they explore interactivelearning material, twelve students across a range of achievement levels from the target population were interviewed and a screen-capturewas recorded while they interacted with the visualizations. These twelve students were selected by purposive sampling, representing high,medium and low achievers' strata of the population. Due to logistical constraints (such as availability of students during the experimentalstudy period), the number of participants in our study was limited to 12.

5.3.1. ProcedureInitially, students were instructed to interact with the learning material without any external instructions or guidance. While they

interacted with the interactive learning material, the screen capture was recorded using CamStudio™ open source software to observe howstudents explore the interactive learning material.

Followed by this, semi-structured interviews were conducted of all the students individually. The audio recording of the interviews wasdone. In the beginning of the interview, students were asked to comment about the similarities and differences that they noted whileinteracting with the Simulation (SIM) and IE-Viz. Later, students were given three questions from the topic, one each from ‘UnderstandConceptual knowledge’, ‘Understand Procedural knowledge’, ‘Apply Procedural knowledge’ category and they were asked to give theircomments about their perception about which features would make a given type of learning material appropriate for answering thequestion from these categories.

5.3.2. Analysis of recorded screen capturesThe recorded screen captures were analysed to find out the manner in which students explore the interactivity offered by interactive

learningmaterial. The Simulation offeredmanipulation of four parameters related to the content presented. Students could select either oneof them or multiple variables (maximum up to four) simultaneously. For developing in-depth and complete understanding of the content, itwas expected that while exploring, students would not just manipulate all the offered variables individually, but also, different possiblecombinations of these variables by simultaneous selection. The main objective of the analysis of recorded screen captures was to find outwhether students explored all the possible manipulation opportunities offered by the interactive learning environment.

The manner in which students explored the simulation, the path they took while carrying out variable manipulation has been repre-sented in a graphical manner in the form of Simulation Exploration Trajectory Representation as shown in Fig. 3. In this representation,horizontal axis represents every time instance (T1, T2, T3 and so on) at which variable manipulation was carried out by a student, whereasthe vertical axis represents the number of variables selected for manipulation at that time instance (for example, 1/2/3/M, wherein ‘M’

represents multiple, simultaneous selection of all the variables). The typical colour represents trajectory taken by each student whileinteracting with the simulation by manipulating variables.

As could be seen from Simulation Exploration Trajectory Representation, only four students out of twelve carried out multiple variablemanipulation. Seven students manipulated two variables simultaneously and five students manipulated three variables simultaneously.Maximumnumber of students (eleven) did only single variablemanipulation and did not try any of themultiple transformation options. Thehigher concentration of trajectories related to single and double variable manipulation as observed from Simulation Exploration Trajectory

Fig. 3. Simulation exploration trajectory representation.

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305300

Representation revealed that most of the students did not try multiple variable manipulation. This implied that the affordance of variablemanipulation offered by the simulation was not fully exploited by learners and thus learning opportunities offered by interactivity in thesimulation were under-utilized. This may be especially unfavourable for procedural knowledge tasks which involve multiple sequentialoperations. This observation, in away, advocates the need for enforced directions and also the need tomake available all kinds of explorationopportunities in the form of affordance to learners while manipulating variables.

5.3.3. Analysis of interviewsThe semi-structured interviews were conducted to get students' interpretation and perception about the following two aspects. Each

interview lasted for 10e12 minutes. Students were asked about: firstly, what differences and similarities did they notice while interactingwith different interactive learning materials and secondly, which type of learning material they found necessary and sufficient whileanswering the questions pertaining to three different categories; ‘understand conceptual knowledge’, ‘understand procedural knowledge’,‘apply procedural knowledge’. The recorded interviews were transcribed and analysed further using Content Analysis method with a‘sentence’ as the ‘coding unit’. The coding was done keeping in mind the objectives of the questions asked. Accordingly, three categories ofthe codes emerged from the analysis; ‘Feature’, ‘Reason’ and ‘Learning impact’. The comments made by the students fall into following threecategories: i) the identified features of the learning material, ii) the reasoning why a particular feature they find important, iii) theirperception about the learning impact that the feature/s would lead to. Below listed are the typical comments, in verbatim, classified as perthese categories of the code (Table 3). The Exploration approach of data analysis (Cohen, Manion, &Morrison, 2007) was adopted to followpatterns and trends observed in the data.

Following have been the inferences from the data analysis:

- All students were of the opinion that the non-interactive visualization was sufficient for answering question of ‘understand conceptualknowledge’ category and none of them found the necessity of interactive visualization for the same.

- However, while answering the questions related to other two categories, students indicated the necessity of interactive visualization,especially the IE-Viz, in order to understand sequencing part involved in the multiple transformation. The students also reported thatthe interactive visualization helped them in visualizing the transformation process and its sequential operation in a better way.

- When asked about their perception about IEF-Viz, the students preferred to study from it, as it could allow them to study impact ofevery individual operation in a sequential manner. Following are some of the verbatim responses from students that supported thisinference. “ .....This applet (IE-Viz) is better........it is better to study one operation at a time and then go for all operations together.... … itwill give better understanding.....”, “...... this way of dissecting every operation helps in understanding the multiple operations at a laterstage......”.

The objective of this qualitative study was to know students perception about usefulness of the affordances offered in visualizations. Theinterpretations and inferences form this qualitative study were found to be in coherence with the rationale for offering the affordances inthe form of ‘Interactivity Enriching Features’ in visualizations. These features, offered as affordances were perceived to be crucial by students,especially, while applying procedural tasks. In away, this perception of students was corroborated by the result that confirmed supremacy ofIE-Viz for Apply Procedural Knowledge. During interviews, students unanimously accepted sufficiency of Non-interactive visualization for

Table 3Coding categories and corresponding responses.

Student's response (verbatim) Coding categories

“ … this applet allows only single option … ”

” …. It shows one step at a time....”Feature

“ … one at a time and then you go for everything makes strong foundation blocks....”“....visualizing signal transformation becomes easy with this (applet), if one is not able to visualize....”

Reason

“..... incremental learning helps......”“..... PDF version will be enough for basic understanding, simulation explains how to solve problems....”

Learning impact

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305 301

tasks related to ‘Understand Conceptual knowledge’. This perception of students has been confirmed by the results that found average testscores of all the groups to be statistically equivalent while understanding conceptual knowledge. Thus, this qualitative studywas found to beuseful in supporting the results related to ‘the apt degree of interaction for the given task’ and in defending the necessity and usefulness ofaffordances in the form of ‘Interactivity Enriching Features’ in visualizations.

6. Discussion

The goal of this research study was to investigate what features would be needed in interactive visualizations so that interactivity couldallow learners to derive maximum learning benefits from it. The results of our study showed that, especially for higher levels of learning,interactive visualizations could not deliver their learning benefits unless they were supported with appropriate ‘Interactivity EnrichingFeatures’. It also challenged the widely accepted notion of higher degree of learner control contributes towards effective learning. Thus,designing interactivity in accordancewith the task requirement was found to be the key for ensuring its effectiveness. The findings from thisresearch study emerge as guidelines for designing educationally effective interactive visualizations. The summary of results obtained fromthis study has been as given in Table 4 below. Detailed discussion and interpretation of the results follow the table.

6.1. Effect of degree of interaction on learning from visualizations

Our first research question addressed the issue related to appropriateness of the degree of learner control, for the given learning task. Inthis research study, the Non-IVZ, ANM and SIM groups received treatments in terms of differing degree of learner control, i.e. differentinteraction level. Thus, in order to analyse the impact of the varying degree of learner control on effective learning, the learning scores ofthese three groups were compared. In order to systematically approach the aspect of how to select the apt level of interactivity in the case ofmultimedia learning, we have recommended a more appropriate way, based on a two-dimensional taxonomy of learning objectives: that is,cognitive level of the task and type of knowledge. Prior work had reported consideration of either, the type of knowledge (Clark & Chopeta,2004) or the cognitive level (Lahtinen & Ahoniemi, 2005) as criteria for deciding suitable visualization, but to our knowledge, both of thesehave not been considered together prior to this study.

The results obtained from the research study clearly indicated that non-interactive visualization was sufficient for learning tasks fallinginto ‘Understand Conceptual knowledge’ category, as average scores of all the experimental groups were found to be statistically same. Thisresult highlighted that higher degree of interaction need not necessarily lead to higher learning outcome, and the type of knowledge andcognitive level of the learning task need to be taken into consideration.While attempting a given task at a specified cognitive level, learner isexpected to undergo certain amount of cognitive processing. The learning material, that puts additional cognitive overload on the learner,instead of assisting learner, may hamper the learning process. Previous studies in this context (Low& Sweller, 2005; Moreno&Mayer, 1999)have confirmed the undesirable role of cognitive overload in the learning process.

Another result from this research study on this background has been that the Animation (ANM) and Simulation (SIM) were found to beequally effective, but inferior to non-interactive visualization (Non-IVZ) for ‘Understand Procedural knowledge’. Although the directinference from this result has been the superiority of Non-IVZ over Animation (ANM) and Simulation (SIM) for ‘Understand Proceduralknowledge’, it has also thrown light upon a different aspect on interactive visualization; its presentation format, which surfaced as anunplanned ‘Interactivity Enriching Feature’ in this research study. The animation and simulation had different presentation format, however,the usability study conducted for the learning material for Non-IVZ, ANM and SIM group had established their equivalence. Due to theinherent ability of animation and simulation to animate the educational content and show it dynamically, these two types of visualizationshowed the various steps of the procedural tasks in a temporally stacked manner. The non-interactive visualization, lacking the ability ofshowing content dynamically, displayed various steps involved in the procedural task at the same time on a single integrated screen in aspatially distributedmanner, although theywere happening at different time instances. Thus, the factor that influenced learning, apart fromdifferent degree of interaction in visualizations has been, the spatially distributed presentation format of Non-IVZ and the temporallystacked presentation format of ANM and SIM. What might have held back the learning from interactive animation and simulation, has beenits temporally stacked presentation format due to the burden that it put on learners while retaining previously learnt knowledge. Retainingand using the previously learnt knowledge must have demandedmore cognitive processing in addition to the essential cognitive processing

Table 4Result summary.

Research questions Experimental results Inferences form the results

Given the type of knowledge and cognitive level, doeshigher degree of interaction, lead to effectivelearning?

‘Understand Conceptual Knowledge’:-No statistically significant difference was found in theaverage scores of Non-IVZ, ANM, SIM and IE-Viz group.

Lower degree of interaction i.e. non- interactivevisualization is sufficient for effective learning of‘understand conceptual knowledge’.

Given the type of knowledge and cognitive level, howdo ‘Interactivity Enriching Features’ affect learning ininteractive visualization with the same level ofinteraction?

‘Understand Procedural Knowledge’: -Non-IVZ group average score was statistically higherthan the average scores when compared with ANM andSIM group, but IE-Viz group score was found to behigher than Non-IVZ score (though statisticallyinsignificant).

Degree of interaction does not solely contribute toeffective learning while ‘Understanding Proceduralknowledge’. Higher degree of learner control does notlead to improved learning unless accompanied by‘Interactivity Enriching Features’.

‘Apply Procedural Knowledge’:-IE-Viz group score was found to be statistically higherthan SIM group score and also highest among all thefour groups (Non-IVZ, ANM, SIM, and IE-Viz), althoughNon-IVZ, ANM, SIM group scores were found to bestatistically equivalent.

Higher degree of learner control does not lead toimproved learning unless augmented by ‘InteractivityEnriching Features’. Thus, degree of interaction or degreeof learner control does not solely contribute to effectivelearning while ‘Applying Procedural knowledge’.

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305302

and thus, might have attributed towards excessive extraneous cognitive load. We looked at this result differently, through the lenses of oursecond research question, and we foresee the presentation format as one of the influencing factors for effective learning fromvisualizations.It must be noted that, even though the presentation formats were different, it did not affect the results of usability study of learningmaterials. Still, it may be argued as a confounding variable of this study. However, we are tempted to consider it as one another ‘InteractivityEnriching Feature’ due to its potential to influence learning from visualizations along with degree of interaction.

This finding has been crucial and the implications of this result are twofold. Its first contribution is by suggesting that non-interactivevisualizations can be boosted by effective presentation format and can be worked to deliver at par, or perhaps better than interactive vi-sualizations when the interactive nature of visualizations burdens learner with excessive cognitive processing. Another important inter-pretation, rather caution, here is that interactivity offered in interactive visualization would be ‘wasted’ if not supported with theappropriate presentation format. Some of the previous studies (Boucheix & Schneider, 2009; Grunwald & Corsbie-Massay, 2006) havediscussed this aspect of presentation format. In our study, presentation format demonstrated its ability to overrule interactivity in visu-alizations and to becomemore influential in the learning process from interactive visualizations. This aspect will require further explorationandmore experimental research in the future to understand nature of interplay between degree of interaction and appropriate presentationformat in visualizations.

While catering to ‘Apply Procedural knowledge’, the comparative analysis of Non-IVZ, ANM and SIM group average scores exhibited thatthe average score of SIM was higher than the average scores of Non-IVZ and ANM, even though statistically non-significant. Though thisfinding could have been concluded as sufficiency of Non-IVZ for ‘Apply Procedural knowledge’, the statistically significant and the highestscore of IEF-Viz among all the four groups needs detailed elaboration. This issue has been discussed in the following subsection.

6.2. Effect of ‘Interactivity Enriching Features’ on learning from visualizations

The second research question of this study probed into what impact ‘Interactivity Enriching Features’ can have on learning from inter-active visualizations. The core of interactive simulation is the affordance of variablemanipulation. It offers opportunity to learners to interactwith the educational content of visualizations. This is one aspect that sets it apart from non-interactive visualizations and even from ananimation. While comparing learning from all the four experimental groups, the results have clearly showed the highest learning outcomesfrom IE-Viz group for ‘Understand’ and ‘Apply’ procedural knowledge category. Due to the ‘Productively Constrained Variable Manipulation’(PCVM) while selecting variable for manipulation, IE-Viz allowed students to select only one, then only two and gradually all the variablesfor manipulation. This gradual progression in the visualization, from single operation, then two operations and then multiple trans-formation operation helped learners in developing gradual, yet sound knowledge base necessary for attempting the procedural tasks. Thisaspect had also got reflected during interviews of students and their perception about different learning materials. Generally, students havea tendency to interact with simulation by clicking or enabling all the possible options they see. Manipulating variables in this manner neednot necessarily lead to inquiry based learning, but, may rather end up only in a playful interaction. The introduction of productive constraintin the learning material in this manner, not just could ensure meaningful learning, but, also offered complete exploration freedom tolearners. The productive constraint introduced in this manner has the potential to improve learning from simulation environment withoutcompromising the discovery based learning nature of simulation. Additionally, the inclusion of ‘Permutative Variable Manipulation’ (PVM)also offeredmore exploration opportunities to learner, such as swapping action sequence, thus, offering additional affordance to address thedemands of learning objectives.

It must be noted that the ‘Apply Procedural knowledge’ average scores of the three experimental groups, (Non-IVZ, ANM and SIM)were found to be at par, but IE-Viz group's ‘Apply Procedural knowledge’ average score was found to be significantly higher ascompared to the other groups. This clearly indicated that interactive visualization could deliver its learning benefits only after gettingaugmented by ‘Interactivity Enriching Features’. Thus, affordance offered by visualization in the form of ‘Productively ConstrainedVariable Manipulation’ (PCVM) and ‘Permutative Variable Manipulation’ (PVM), certainly exhibits the potential to influence learning frominteractive visualizations, and has proved itself as ‘Interactivity Enriching Features’. These results offer guideline while choosing,designing and operationalizing variable manipulation. For ‘Understand Procedural knowledge’, IE-Viz group scored highest among allthe groups. Further investigation and planned research experiments would be needed to explore what kind of ‘Interactivity EnrichingFeatures’ could overcome the learning obstacle introduced by the temporally stacked presentation format of animation (ANM) andsimulation (SIM).

To sum up, the results obtained for our second research question, have drawn attention to the fact that it is not just the affordance ofvariable manipulation, but ‘what gets manipulated and how’, is more important in interactive visualizations. This aspect should beconsidered very influential in deciding the learning benefits of interactivity.

6.3. Limitations of this study

In this study, we have focussed on the features pertaining to visualization itself. In spite of the fact that learner characteristics also play acrucial role in the effectiveness of learning from interactive visualization (Barak et al., 2011; Park et al., 2009; Yaman, Nerdel, & Bayrhuber,2008), variation in the learner characteristics has not been the focus of this study. Due to the need to accommodate a variety of learners inthe same educational set-up; a typical constraint in university affiliated institutional set-up, our research does not consider specific learnercharacteristics as a variable of the study.

Even though this study focused only for one topic from Signals and Systems, we expect the results to hold good for other topics withsimilar nature. This is due to the fact that, the learning effectiveness of the interactive visualizationwas assessed by using cognitive level andknowledge type of the task based on Anderson's two-dimensional taxonomy framework of educational objectives. This two-dimensionaltaxonomy is being considered to be a generic approach for assessing learning outcome across the knowledge domains. Also, as many en-gineering courses, especially deal with conceptual and procedural knowledge, the results obtained from this study will turn out to be usefulwhile designing interactive visualizations for engineering education.

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305 303

7. Conclusions

This study introduced ‘Interactivity Enriching Features’ to harness the learning potential of interactive visualizations. The researchexperiment in this study identified and investigated the impact of such factors while learning conceptual and procedural knowledge atdifferent cognitive levels in a Signals and Systems course for engineering majors. The results of the study showed that, especially for higherlevels of learning, interactive visualizations could not deliver their learning benefits unless they were augmented with appropriate‘Interactivity Enriching Features’.

We believe and recommend that, this research and its results need to be looked upon as howdesign effortsmust be channelized inmakinginteractive visualizations educationally effective andnot as amedia comparison. ‘What additional factors in the formof ‘Interactivity EnrichingFeatures’ from interactive learning environment can assist learners in the learning process’ has been the central theme of this research study.The ‘Interactivity Enriching Features’ considered in this study have shown their effectiveness in improving learning from interactive visuali-zation. The future direction of research will be to develop a framework based systematic approach that can help in identifying the relevant‘Interactivity Enriching Features’ for the given context. The results of this study should stimulate researchers and educationalists to analyseinteractive visualizations and its interactive nature cautiously and in greater detail. Today, with greater level of acceptance and penetration ofmultimedia learning and its adaptation as teaching-learning strategies, there is a strong need to explore, “what can make interactive visu-alizations educationally effective?” Our research study has come up with some guidelines to address this question.

Acknowledgements

The authors thank and acknowledge the support offered by the Project OSCAR (Open Source Courseware Animation Repository) and theGovernment of India's NationalMission on Education through Information and CommunicationTechnology (NMEICT) for this researchwork.

Appendix A

Knowledge type: Conceptual knowledge and Procedural knowledge.

In the topic of Signal Transformation, individual transformation operation constitute conceptual knowledge, while multiple trans-formations to be carried out on a signal in an algorithmic manner refer to procedural knowledge. Thus, for example, single operation ofadvancing or delaying a signal would expect leaner to obtain conceptual knowledge related to time shifting of a signal, whereas carrying outmultiple transformation operations like time delaying, time scaling and time reversing on the same signal would expect learner to know thesequence of the operations to be carried out to accomplish the procedural task along with relevant conceptual knowledge in the form ofindividual signal transformation operation.

Cognitive Level: ‘Understand’ and ‘Apply’ level.

For ‘understand’ cognitive level, studentswere expected to comprehend the knowledge related conceptual and procedural type. Thus, thequestions related at this cognitive level were in the form of ‘identifying’ or ‘interpreting’ the single or multiple transformation operations.Whereas, at ‘apply’ cognitive level, students were expected to use their comprehension about the topic in order to solve the question byapplying their knowledge in the new situation. Thus, the ‘apply’ level cognitive tasks expected students to carry out necessary calculationseither to write mathematical expression or to ‘plot’ the transformed signal after carrying out single or multiple transformation operations.

Sample questions from the instrument:

‘Understand conceptual knowledge’:Figure A.1 here demonstrates a question of ‘understand conceptual knowledge’ category. As shown in the question, it expects learner to

identify which single transformation operation has transformed signal x(t) to y(t). Here, learner will be able to answer this questioncorrectly if it has ‘comprehended’ ‘conceptual’ knowledge related to time scaling concept.

Fig. A.1. Sample questions for ‘understand’ ‘conceptual knowledge’ level: Signal Transformation.

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305304

‘Understand’ ‘Procedural knowledge’Figure A.2 here demonstrates a question of ‘understand procedural knowledge’ category. As shown in the question, it expects learner to

identify the sequence in which multiple transformations take place while carrying out procedural task of multiple transformation of signalx(4-2t) from signal x(t).

Fig. A.2. Sample questions for ‘understand’ ‘procedural knowledge’ level: Signal Transformation.

‘Apply’ Procedural knowledge:

Fig. A.3. Sample questions for ‘apply’ ‘procedural knowledge’ level: Signal Transformation.

Figure A.3 here demonstrates a question of apply procedural knowledge' category. As shown in the question, it expects learner to applymultiple transformation operations on the given signal and plot the transformed signal. Learner's comprehension of individual singletransformation operation, ability to apply the same in the context of given procedural task and procedural knowledge of the sequence oftransformation operation will enable learner to solve this task of apply procedural category.

References

Aleksandrova, A., & Nancheva, N. (2007). Electromagnetism: interaction of simulation and real lab experiment. International Journal of Information Technologies and Knowledge,1, 44e50.

Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., et al. (2001). In A taxonomy for learning, teaching, and assessing: A revision ofBloom's Taxonomy of Educational Objectives. New York: Longman.

Austin, K. A. (2009). Multimedia learning: cognitive individual differences and display design techniques predict transfer learning with multimedia learning modules.Computers & Education, 53(4), 1339e1354.

Barak, M., Ashkar, T., & Dori, Y. J. (2011). Learning science via animated movies: Its effect on students' thinking and motivation. Computers & Education, 56(3), 839e846.B�etrancourt, M., & Tversky, B. (2000). Effect of computer animation on users' performance: a review. Le Travail Humain: A Bilingual and Multi-Disciplinary Journal in Human

Factors, 63(4), 311e329.Boucheix, J.-M., & Schneider, E. (2009). Static and animated presentations in learning dynamic mechanical systems. Learning and Instruction, 19(2), 112e127.Brooke, J. (1996). SUS: a quick and dirty usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & A. L. McClelland (Eds.), Usability evaluation in Industry (pp. 189e194).

London: Taylor and Francis.Card, S. K., Mackinlay, J. D., & Shneiderman, B. (1999). Readings in information visualization; using vision to think. San Francisco: Morgan Kaufmann.Chaturvedi, S. K., & Osman, A. (2006). Simulation and visualization enhanced engineering education. In Proceedings of an International Mechanical Engineering Education

Conference (pp. 1e8).Chen, C. (2004). Information visualization: Beyond the horizon. London, UK: Springer-Verlag.Clark, R. C., & Chopeta, L. (2004). Graphics for learning: Proven guidelines for Planning, Designing, and Evaluating visuals in training materials. San Francisco: Pfeiffer.Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education (7th ed.). New York: Routledge.Crutchfield, S. G., & Rugh, W. J. (1997). Interactive exercises and demonstrations on the Web for basic signals, systems and control. In , Proceedings of the 36th IEEE Conference

on: Vol. 4. Decision and control (pp. 3811e3815).Domagk, S., Schwartz, R. N., & Plass, J. L. (2010). Computers in Human Behavior Interactivity in multimedia learning: an integrated model. Computers in Human Behavior, 26(5),

1024e1033.El Saddik, A. (2001). Interactive multimedia learning: shared reusable visualization-based modules. NY: Springer-Verlag.Engin. (2006). Modeling and simulation concepts in engineering education: virtual tools. Engineering Education, 14(1), 113e127.Gao, T., & Lehman, J. D. (2003). The effects of different levels of interaction on the achievement and motivational perceptions of college students in a web-based learning

environment. Journal of Interactive Learning Research, 14(4), 367e386.Gibson, J. J. (1977). The theory of affordances. In R. Shaw, & J. Bransford (Eds.), Perceiving, acting and knowing. Hillsdale, NJ: Erlbaum.Glass, G. V., & Hopkins, K. D. (1970). One factor analysis of variance. In Statistical methods in education and psychology (pp. 371e421). Englewood Cliffs, NJ: Prentice-Hall.Grunwald, T., & Corsbie-Massay, C. (2006). Guidelines for cognitively efficient multimedia learning tools: educational strategies, cognitive load, and interface design. Academic

Medicine: Journal of the Association of American Medical Colleges, 81(3), 213e223.Guan, X., Zhang, M., & Zheng, Y. (2009). Matlab simulation in signals & systems using Matlab at different levels. In Education Technology and Computer Science, ETCS '09. First

International Workshop on, 2 pp. 952e955).Guzman, J. L., Dormido, S., & Berenguel, M. (2010). Interactivity in education: an experience in the automatic control field. Computer Applications in Engineering Education, 21,

360e371.

M. Patwardhan, S. Murthy / Computers & Education 82 (2015) 292e305 305

Hanisch, F., & Straßer, W. (2005). How to include visuals and interactivities in an educational computer graphics repository. Computers & Graphics, 29(2), 237e243.Hansen, S. (2002). Designing educationally effective algorithm visualizations. Journal of Visual Languages & Computing, 13(3), 291e317.Hiebert, J., & Lefevre, P. (1986). Procedural and conceptual knowledge in mathematics: an introductory analysis. In J. Hiebert (Ed.), Conceptual and procedural knowledge: The

case of mathematics (pp. 1e27). Hillsdale, NJ: Erlbaum.Imhof, B., Scheiter, K., & Gerjets, P. (2011). Learning about locomotion patterns from visualizations: effects of presentation format and realism. Computers & Education, 57(3),

1961e1970.Jonassen, D. H. (2006). Modeling with technology: Mindtools for conceptual change (3rd ed.). Upper Saddle River, New Jersey: Pearson Education, Inc.Kalyuga, S. (2005). Prior knowledge principle. In R. E. Mayer (Ed.), Cambridge handbook of multimedia learning (pp. 325e337). Cambridge: Cambridge University Press.Kalyuga, S. (2007). Enhancing instructional efficiency of interactive e-learning environments: a cognitive load perspective. Educational Psychology Review, 19, 387e399.Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38(1), 23e31.Kehtarnavaz, N., Loizou, P., & Rahman, M. (2008). An interactive hybrid programming approach to signals and systems laboratory. In Acoustics, Speech and Signal Processing,

ICASSP 2008. IEEE International Conference on (pp. 2633e2636).Kombartzky, U., & Ploetzner, R. (2007). Beneficial effects of learning from animations. In W. Chen, & H. Ogata (Eds.), Learning by effective utilization of technologies: Facilitating

intercultural understanding. Supplementary Proceedings of the 15th International Conference of Computers in Education (pp. 3e4). Amsterdam: IOS Press.Korakakis, G., Pavlatou, E., Palyvos, J., & Spyrellis, N. (2009). 3D visualization types in multimedia applications for science learning: a case study for 8th grade students in

Greece. Computers & Education, 52(2), 390e401.Krathwohl, D. R. (2002). A revision of Bloom's Taxonomy: an overview. Theory Into Practice, 41(4), 212e218.Kriz, S., & Hegarty, M. (2007). Top-down and bottom-up influences on learning from animations. International Journal of Human-Computer Studies, 65(11), 911e930.Kühl, T., Scheiter, K., Gerjets, P., & Gemballa, S. (2011). Can differences in learning strategies explain the benefits of learning from static and dynamic visualizations? Computers

& Education, 56(1), 176e187.Lahtinen, E., & Ahoniemi, T. (2005). Visualizations to support programming on different levels of cognitive development. In Proceedings of The Fifth Koli Calling Conference on

Computer Science Education (pp. 87e94).Lattu, M., Meisalo, V., & Tarhio, J. (2003). A visualisation tool as a demonstration aid. Computers & Education, 41(2), 133e148.Lengler, R., & Eppler, M. J. (2007). Towards a periodic table of visualization methods for management. In IASTED Proceedings of the Conference on Graphics and Visualization in

Engineering (GVE 2007), Clearwater, Florida, USA.Liang, H. (2006). Interactivity of visual mathematical representations: factors affecting learning. Learning, 17, 179e212.Liang, H.-N., & Sedig, K. (2009). Application of frameworks in the analysis and (re) design of interactive visual learning tools. Journal of Interactive Learning Research, 20(2), 215e254.Liang, H.-N., & Sedig, K. (2010). Can interactive visualization tools engage and support pre-university students in exploring non-trivial mathematical concepts? Computers &

Education, 54(4), 972e991.Lin, L., & Atkinson, R. K. (2011). Using animations and visual cueing to support learning of scientific concepts and processes. Computers & Education, 56(3), 650e658.Lowe, R. K. (2003). Animation and learning: selective processing of information in dynamic graphics. Learning and Instruction, 13(2), 157e176.Low, R., & Sweller, J. (2005). The modality principle in multimedia learning. In R. E. Mayer (Ed.), Cambridge handbook of multimedia learning (pp. 147e158). New York:

Cambridge.McManus, H., & Rebentisch, E. (2008). Experiences in simulation-based education in engineering processes. In Frontiers in Education Conference, 2008. FIE 2008. 38th Annual

(pp. S1C 21e26). IEEE.Millard, D. L. (2000). Interactive learning modules for electrical engineering education. In Electronic Components & Technology Conference, 2000. Proceedings. 50th (pp.

1042e1047). IEEE.Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: the role of modality and contiguity. Journal of Educational Psychology, 91(2), 358.Moreno, R., & Valdez, A. (2005). Cognitive load and learning effects of having students organize pictures and words in multimedia environments: the role of student

interactivity and feedback. Educational Technology, Research and Development, 53(3), 35e45.Nelson, J. K., Hjalmarson, M. A., Wage, K. E., & Buck, J. R. (2010). Students' interpretation of the importance and difficulty of concepts in signals and systems. In Frontiers in

Education Conference (FIE), 2010 IEEE (pp. T3G1e6). IEEE.Park, S. I., Lee, G., & Kim, M. (2009). Do students benefit equally from interactive computer simulations regardless of prior knowledge levels? Computers & Education, 52(3),

649e655.Pinter, R., Radosav, D., & Cisar, S. M. (2010). Interactive animation in developing e-learning contents. In MIPRO, 2010 Proceedings of the 33rd International Convention (pp.

1007e1010). IEEE.Plass, J. L., Homer, B. D., & Hayward, E. O. (2009). Design factors for educationally effective animations and simulations. Journal of Computing in Higher Education, 21(1), 31e61.Podolefsky, N. S., Moore, E. B., & Perkins, K. K. (2013). Implicit scaffolding in interactive simulations: Design strategies to support multiple educational goals. arXiv preprint

arXiv:1306.6544.Rabenstein, R. (2002). SYSTOOL e an online learning tool for signals and systems. In ICASSP (pp. 4128e4131). IEEE, ISBN 0-7803-7402-9.Reichert, R., & Hartmann, W. (2004). On the Learning in e-Learning. In World Conference on Educational Multimedia, Hypermedia and Telecommunications (Vol. 2004, pp.

1590e1595). No. 1.Rey, G. D. (2011). Time advice and learning questions in computer simulations. Australasian Journal of Educational Technology, 27(3), 397e410.Rutten, N., Van Joolingen, W. R., & Van der Veen, J. T. (2012). The learning effects of computer simulations in science education. Computers & Education, 58(1), 136e153.de los Santos Vidal, O., Jameson, R. M., Iskander, M. F., Balcells, A., & Catten, J. C. (1996). Interaction and simulation-based multimedia modules for electromagnetics education.

In Frontiers in Education Conference, 1996. FIE'96. 26th Annual Conference, Proceedings of (Vol. 3, pp. 1067e1070). IEEE.Schulmeister, R. (2003). Taxonomy of multimedia component interactivity. A contribution to the current metadata debate. Studies in Communication Sciences. Studi di scienze

della communicazione, 3(1), 61e80.Schwan, S., & Riempp, R. (2004). The cognitive benefits of interactive videos: learning to tie nautical knots. Learning and Instruction, 14(3), 293e305.Sedig, K., & Liang, H.-N. (2006). Interactivity of visual mathematical representations: factors affecting learning and cognitive processes. Journal of Interactive Learning Research,

17(2), 179e212.Sedig, K., & Sumner, M. (2006). Characterizing interaction with visual mathematical representations. International Journal of Computers for Mathematical Learning, 11(1), 1e55.Shaffer, J., Hamaker, J., & Picone, J. (1998). Visualization of signal processing concepts. In Acoustics, Speech and Signal Processing, 1998. Proceedings of the 1998 IEEE International

Conference on (vol. 3, pp. 1853e1856). IEEE.Spanias, A., Chilumula, R., & Huang, C. W. (2006). Work in progress: collaborative signals and systems laboratories at ASU, UWB, UCF, UTD, and URI. In Frontiers in Education

Conference, 36th Annual (pp. 12e13). IEEE.Spanjers, I. A. E., Van Gog, T., Wouters, P., & Van Merri€enboer, J. J. G. (2012). Explaining the segmentation effect in learning from animations: the role of pausing and temporal

cueing. Computers & Education, 59(2), 274e280.Spence, R. (2007). Information visualization: Design for interaction (2nd ed.). Harlow, UK: Pearson Education Limited.Stieff, M., & Wilensky, U. (2003). Connected chemistrydincorporating interactive simulations into the chemistry classroom. Journal of Science Education and Technology, 12(3),

285e302.Sturm, B. L., & Gibson, J. D. (2005). Signals and Systems using MATLAB: an integrated suite of applications for exploring and teaching media signal processing. In Frontiers in

Education, 2005. FIE'05. Proceedings 35th Annual Conference (pp. F2E21e26). IEEE.Taraban, R., Definis, A., Brown, A., Anderson, E. E., & Sharma, M. P. (2007). A paradigm for assessing conceptual and procedural knowledge in engineering students. Journal of

Engineering Education, 96(4), 335e345.Thomas, J. J., & Cook, K. A. (Eds.). (2005). Illuminating the path: The research and development agenda for visual analytics. IEEE Computer Society Press.Tory, M., & M€oller, T. (2002). A model-based visualization taxonomy. School of Computing Science, Simon Fraser University.Tversky, B., Morrison, J. B., & Betrancourt, M. (2002). Animation: can it facilitate? International Journal of Human-Computer Studies, 57(4), 247e262.Wage, K. E., Buck, J. R., Wright, C. H., & Welch, T. B. (2005). The signals and systems concept inventory. Education, IEEE Transactions on, 48(3), 448e461.Wang, P.-Y., Vaughn, B. K., & Liu, M. (2011). The impact of animation interactivity on novices' learning of introductory statistics. Computers & Education, 56(1), 300e311.Ware, C. (2004). Information visualization: Perception for design (2nd ed.). San Francisco, CA: Morgan Kaufmann Publishers.Wu, H. K., Krajcik, J. S., & Soloway, E. (2001). Promoting understanding of chemical representations: students' use of a visualization tool in the classroom. Journal of Research in

Science Teaching, 38(7), 821e842.Yaman, M., Nerdel, C., & Bayrhuber, H. (2008). Computers & Education the effects of instructional support and learner interests when learning using computer simulations.

Computers & Education, 51, 1784e1794.