adaptive feedback in computer-based learning environments: a … · adaptive learning environments...

Review

Adaptive Behavior2017, Vol. 25(5) 217–234� The Author(s) 2017Reprints and permissions:sagepub.co.uk/journalsPermissions.navDOI: 10.1177/1059712317727590journals.sagepub.com/home/adb

Adaptive feedback in computer-basedlearning environments: a review

Andrew Thomas Bimba1, Norisma Idris1, Ahmed Al-Hunaiyyan2,Rohana Binti Mahmud1 and Nor Liyana Bt Mohd Shuib3

AbstractAdaptive support within a learning environment is useful because most learners have different personal characteristicssuch as prior knowledge, learning progress, and learning preferences. This study reviews various implementation ofadaptive feedback, based on the four adaptation characteristics: means, target, goal, and strategy. This review focuses on20 different implementations of feedback in a computer-based learning environment, ranging from multimedia web-basedintelligent tutoring systems, dialog-based intelligent tutoring systems, web-based intelligent e-learning systems, adaptivehypermedia systems, and adaptive learning environment. The main objective of the review is to compare computer-basedlearning environments according to their implementation of feedback and to identify open research questions in adaptivefeedback implementations. The review resulted in categorizing these feedback implementations based on the students’information used for providing feedback, the aspect of the domain or pedagogical knowledge that is adapted to providefeedback based on the students’ characteristics, the pedagogical reason for providing feedback, and the steps taken toprovide feedback with or without students’ participation. Other information such as the common adaptive feedbackmeans, goals, and implementation techniques are identified. This review reveals a distinct relationship between the char-acteristics of feedback, features of adaptive feedback, and computer-based learning models. Other information such asthe common adaptive feedback means, goals, implementation techniques, and open research questions are identified.

KeywordsAdaptation, learning environment, problem-solving, student modeling, learner model

Associate Editor: Tom Froese

1. Introduction

The process of learning involves mistakes and errors.In these situations, students often review course mate-rials and search the Internet or other sources to assistthem in solving their problems (Ghauth & Abdullah,2010). Seeking solution is usually time consuming anddoes not always insinuate a better learning experi-ence. Having a system which generates effective feed-back that guides students to the solution can improvethe learning process (Munoz-Merino et al., 2011).Feedback is frequently provided in a typical class-room setting; however, most of the information ispoorly received because feedback is presented togroups and so often students do not believe such feed-back is relevant to them (Hattie & Gan, 2011).Currently, the gap between students who excel themost and those who excel less is a challenge thatteachers, school administrators, and government offi-cials face frequently (Luckin & Holmes, 2016).

Adaptive learning environments provide personaliza-tion of the instruction process based on different para-meters such as sequence and difficulty of task, type andtime of feedback, learning pace, and others (Brusilovskyet al., 1999; Stoyanov & Kirchner, 2004). One of the keyfeatures in learning support is the personalization offeedback (Advisors, 2013). Adaptive feedback supportwithin a learning environment is useful because mostlearners have different personal characteristics such as

1Department of Artificial Intelligence, University of Malaya, Kuala

Lumpur, Malaysia2Computer & Information Systems Department, College of Business

Studies, The Public Authority for Applied Education & Training (PAAET),

Kuwait City, Kuwait3Department of Information Systems, University of Malaya, Kuala

Lumpur, Malaysia

Corresponding author:

Norisma Idris, Department of Artificial Intelligence, University of Malaya,

50603 Kuala Lumpur, Malaysia.

Email: [email protected]

https://uk.sagepub.com/en-gb/journals-permissions

https://doi.dox.org/10.1177/1059712317727590

https://journals.sagepub.com/home/adb

http://crossmark.crossref.org/dialog/?doi=10.1177%2F1059712317727590&domain=pdf&date_stamp=2017-08-24

prior knowledge, learning progress, and learning prefer-ences. Tailoring feedback according to learner’s charac-teristics and other external parameters is a promisingway to implement adaptation in computer-based learn-ing environment (Narciss et al., 2014). Adaptive feed-back unlike generic feedback is dynamic, as learnerswork through instructions where different learners willreceive different information (Le, 2016). Addressing thisneed, many researchers have proposed variousapproaches to help students in learning (Farid, Ahmad,& Alam, 2015). As a result, they have identified gapsand have been developing various frameworks and edu-cational systems that are able to analyze student learningand provide adaptive feedback.

The main objective of this review is to comparecomputer-based learning environments according totheir implementation of feedback and to identify majoropen research questions in adaptive feedback imple-mentations. Not all the implementations selected haveadaptive feedback as their main design aim. The reasonfor our selection is to provide readers an insight to howadaptive feedback is implemented by comparing awider range of applications.

Previous researchers have conducted reviews ofadaptive feedback systems. Le (2016) analyzed theapproaches used in developing educational systems forprogramming and introduced a classification for adap-tive feedback supported by these systems. Hepplestone,Holden, Irwin, Parkin, and Thorpe (2011) explored var-ious literature supporting the appropriate use of tech-nology for providing feedback to students. Our currentreview follows similar methodologies. However, thisstudy reviews various implementation of feedback,based on the four adaptation characteristics: means,target, goal, and strategy (M. E. Specht, 1998). Basedon our knowledge, there has not been any review ofadaptive feedback implementations according to thefour adaptation characteristics. This classificationscheme provides an overview of the field. It emphasizesthe aspects of the technology, demonstrates openresearch questions, possible research opportunities, andoffers opportunity for researches to identify key charac-teristics, while implementing adaptive feedback systems.The main objective is to compare feedback implementa-tions according to these adaptation characteristics andidentify major open research questions in adaptive feed-back implementations.

The structure of the article is as follows: First, thebackground study on adaptive feedback, explaining thecharacteristics of adaptation and feedback, is discussed inSection 2. In Section 3, we provide the outline of thereview process. The results of the review of adaptive feed-back implementations according to the characteristics ofadaptation and feedback are discussed in Section 4. Wediscuss our findings in Section 5. Future directions andconclusion are presented in Sections 6 and 7, respectively.

2. Background

Brusilovsky (1998) defined systems that model stu-dent’s learning style, prior knowledge, goals, and pre-ference as adaptive, while those systems which useartificial intelligence (AI) techniques to perform therole of an instructor in tutoring and correcting arereferred to as intelligent systems. Learning environ-ments can be either one or a combination of bothadaptive and intelligent elements. According to Chieu(2005), there are five main adaptation techniqueswhich are related to the key components of construc-tive learning environment as follows:

1. Adaptive presentation of learning contents. Thecourse designer should define which learning con-tents are appropriate to a specific learner at anygiven time, for example, simpler situations andexamples for a novice learner than for an expert.

2. Adaptive use of pedagogical devices. The coursedesigner should define which learning activities areappropriate to a specific learner, for instance, sim-pler tasks to a novice learner than to an expert.

3. Adaptive communication support. The coursedesigner should identify which peers are appropriateto help a specific learner, for example, learners withmore advanced mental models help learners withless advanced ones.

4. Adaptive assessment. The course designer shouldidentify which assessment problems and methodsare appropriate to determine the actual performanceof a specific learner, for instance, simpler tests for anovice learner than for an expert.

5. Adaptive problem-solving support. The tutor shouldgive appropriate feedback during the problem-solving process of a specific learner, for example, toshow the learner his or her own difficulties and pro-vide him or her with the way to overcome thosedifficulties.

These adaptation techniques rely on a learner model; anessential component which, among other student rele-vant data, keeps data about the student’s knowledge ofthe subject domain under study.

Adaptive learning involves multiple disciplines suchas Educational Psychology, Cognitive Science, andArtificial Intelligence. This complexity prompted thestructuring of research on adaptivity along the metho-dological questions distinguishing means, target, goal,and strategy (M. E. Specht, 1998):

1. Adaptation means. What information about the lear-ner such as knowledge level, cognitive style, learningstyle, gender, student’s current activity, previousachievements and difficulties, and misconception isknown and used for adaptation?

218 Adaptive Behavior 25(5)

2. Adaptation target. What aspect of the instructionalsystem (pedagogy and domain model) is adaptedbased on the learner model?

3. Adaptation goal. What are the pedagogical reasonsfor the system to adapt to the learner model? Is thesystem aiding inductive or deductive learning; is thesystem adapting to a specific instructional methodbased on the learner model?

4. Adaptation strategy. What are the steps and tech-niques used to adapt the system to the learner model,and how active or reactive are the learners and sys-tem to the adaptation process?

In a learning environment, feedback is seen as theteacher’s (artificial or real) response to the student’saction. There are four main characteristics of feedback:function, timing, schedule, and type (Carter, 1984).Although other researchers (Economides, 2006) sug-gested other characteristics of feedback, we adhere tothe characterization by Carter (1984) because it encom-passes all other characterizations. These characteristicsare briefly explained in Table 1.

For developing an effective adaptive feedback sys-tem, the characteristics of adaptation and feedbackhave to be taken into consideration. The next sectiondiscusses our approach to reviewing implementationsof adaptive feedback based on these characteristics.

3. Materials and method

Scientific journals related to learning, computer tech-nology for education, and artificial intelligence in edu-cation from five main digital libraries were searched,with the aim of reviewing adaptive feedback implemen-tations based on the characteristics of adaptation andfeedback. These libraries include Scopus, Web ofScience, IEEE Xplore, Google Scholar, and ACM. Thelibraries were selected based on their impact evaluation

and wide coverage of peer-reviewed journals in multipleacademic disciplines. In addition to searching thesedatabases, snowballing technique was used to identifysimilar implementations of adaptive feedback systems.Only publications from years 2000 to 2016 were col-lected since most implementations of adaptive feedbackin learning environments were realized during thisperiod. To search for potential articles, keywords suchas feedback, adaptive feedback, intelligent tutoring sys-tem, adaptive learning system, computer-based tutor,pedagogical agents, and computer-assisted learningwere used.

For the searched articles, two criteria were consid-ered: (1) publications from year 2000 to year 2016,which indicated evidence of implementation and scien-tific evaluation of an adaptive feedback system and (2)recent articles with implementations or a clear pro-posed approach. These criteria allow us to considerpublications that have demonstrated practical relevanceand also take into account recently developed adaptivefeedback systems. We also narrowed down our selec-tion based on three views of adaptive feedback systems.First, adaptive learning systems that provide differentinformation to different learners as they work throughinstructions and second, adaptive learning systemswhich generate feedback based on a learner modelwhich distinguishes different learners. Third, wefocused on the proposed adaptive feedback frame-works, with practical implementation strategies.Publications that do not fall within this focus area ormeet the target criteria were excluded.

A total of 1709 articles were found after searchingthrough the five major digital libraries based on key-words as shown in Figure 1. Using EndNote desktopapplication (a software tool for managing articles andcitation), we eliminated the duplicates and selected thearticles that met part of our criteria through relevancesorting. This process resulted in 185 articles excludingthe subject descriptive articles which are mentioned in

Table 1. Characteristics of Feedback.

Characteristicsof feedback

Explanation

Function Feedback can be provided in relation to the instructional goals and objectives. For example, feedback isprovided based on cognitive functions such as promoting information processing, motivational functionssuch as developing and sustaining persistence or provide correct response.

Timing Feedback can be given with respect to timing. It could be in advance, appearing before an action; it couldbe immediate, appearing immediately after an action or delayed, appearing at a longer time after theaction has been made. The feedback is intended to advise, notify, recommend, alert, inform, or motivatethe learner about some concerns.

Scheduling Feedback can also be made available at scheduled instances. For example, when the learner exceeds acertain time threshold, expertise level, after solving certain questions or after every subtopic.

Type There are various feedback types resulting from function, timing, and scheduling. For example,verification feedback, avoidance feedback, correction feedback, informative feedback, cognitive feedback,emotional feedback, scheduled feedback, dynamic feedback, immediate feedback, advanced feedback,delayed feedback, comparative feedback, and isolation feedback.

Bimba et al. 219

the introductory parts. Furthermore, we selected 24 eli-gible articles and added 4 more from snowballingaccording to 20 different implementations of adaptivefeedback. These implementations range from multime-dia web-based ITS, dialog-based ITS, web-based intelli-gent e-learning system, adaptive hypermedia system,theoretical feedback frameworks, and intelligent andadaptive learning environment. The analyzed articlesconsisted of journal articles, conference proceedings,books, and serials. They were examined based on thepublication years, availability, and relevance to theresearch domain.

4. Results

4.1. Classification of adaptive feedbackimplementations

A computer-based learning environment representsknowledge in the form of models. The three key modelsin a computer-based learning environment are the ped-agogical model, domain model, and learner model.Research regarding the design and development ofadaptive learning environments is highly multi-

disciplinary, uniting research from computer scienceand engineering, psychology and psychotherapy, cyber-netics and system dynamics, instructional design, andempirical research on technology enhanced learning(Specht, Kravcik, Klemke, Pesin, & Huttenhain, 2002).While the educational scientists give attention to devel-opment, evaluation, and approval of adaptive instruc-tion algorithms, computer scientists are concernedmore with the development of better algorithms, mod-els (pedagogy, domain, and user), and intelligent adap-tation. The complexity which arises by the union ofthese disciplines initiated the need for structuringimplementations of adaptivity according to the metho-dological questions distinguishing means, target, goal,and strategy (Specht et al., 2002).

Similarly, we adopt this methodology as a classifica-tion scheme to review different implementations ofadaptive feedback in learning environments. Adaptivefeedback implementations can be grouped and ana-lyzed based on adaptation methodology and feedbackcharacteristics. Several adaptive learning systems haveutilized learner’s characteristics to provide adaptivefeedback. In this review, we discuss the following imple-mentations of adaptive feedback.

Figure 1. Review process.


Wayang Outpost is a multimedia web-based intelli-gent tutoring system, designed to help students solvemathematics problems. It promotes meaningful andeffective ways of learning (Arroyo et al., 2003; Arroyoet al., 2014). Gerdes’ tutor is an interactive functionalprogramming tutor, which supports stepwise develop-ment in Haskell programming language (Gerdes,Jeuring, & Heeren, 2012). The E-Tutor, which wasdeveloped at Simon Fraser University in Canada, is aweb-based intelligent computer-assisted language learn-ing (iCALL) system for beginner to advanced levelGerman grammar exercises. It consists of Germangrammar concepts and vocabulary tasks, which is usedby students in North American Universities (Heift &Schulze, 2007). AutoTutor is an intelligent tutoring sys-tem which uses natural language and adaptive dialog tohelp students in understanding concepts in Newtonianphysics, critical thinking, and computer literacy(D’Mello & Graesser, 2012). The intelligent TeachingAssistant for programming (ITAP) is a data-driventutoring system that provides personalized help to stu-dents while working on code-writing problems (Rivers& Koedinger, 2015).

DeepTutor is another dialog-based intelligent tutor-ing system that uses scaffolding to improve student’sknowledge during problem-solving (Rus, Niraula, &Banjade, 2015). ACTIVEMATH is a web-based intelli-gent e-learning system that offers access to variousmathematical learning objects, which supports the con-structivist learning approach (Melis, Moormann,Ullrich, Goguadze, & Libbrecht, 2007). Guru, on theother hand, is an intelligent tutoring system which con-sists of exercises in high school biology, supportingconversation with students and virtual instructionalmaterials (Olney et al., 2012). INSPIRE is an adaptiveeducational hypermedia system which provides mean-ingful tasks to students, based on their preferred wayof learning (Papanikolaou, Grigoriadou, Kornilakis, &Magoulas, 2003). FIT Java Tutor is an intelligent andadaptive learning environment which integrates severalpedagogical approaches to assist students in learningJava programming (Gross & Pinkwart, 2015).

ANDES is an intelligent tutoring system whichencourages students to construct new knowledge inintroductory physics (Gertner & VanLehn, 2000;VanLehn et al., 2005). SQL-Tutor is an intelligenttutoring system which teaches database query languageby helping students learn from their mistakes (Mitrovic,2003; Mitrovic & Ohlsson, 1999; Mitrovic, Ohlsson, &Barrow, 2013). COMPASS uses concept maps as alearning tool which allows students to undertake assess-ment activities (Gouli, Gogoulou, Papanikolaou, &Grigoriadou, 2006). Excel Tutor is an intelligent novicetutor which provides feedback through error detectionand correction skills (Mathan & Koedinger, 2005). Thepedagogical motivation for feedback in Excel Tutor isto guide students in error detection and provide an

opportunity to reason about the causes and conse-quences of the errors.

Adaptive feedback frameworks have also been pro-posed by other researchers. Mason and Bruning’s(2001) theoretical framework enables the creation offeedback based on a variety of conditions such as thecomplexity of task, student’s prior knowledge, stu-dent’s achievement, timing of feedback, and learnercontrol. A conceptual framework for designing infor-mative tutoring feedback forms was put forward byNarciss and Huth (2002). The framework is aimed atderiving general principles for designing informativefeedback based on cognitive task and error analysis.

Other mathematics-based intelligent tutoring sys-tems which provide feedback have been proposed.Animalwatch is an ITS which integrates mathematicsand biological sciences for teaching arithmetics to ele-mentary school students (Arroyo, Beck, Woolf, Beal, &Schultz, 2000). It builds empirical models of the stu-dent’s behavior through an analyses of their interactionwith the mathematics tutor. Tsovaltzi and Fiedler(2003) used a natural language dialog module to imple-ment a mathematical tutoring system for teaching naiveset theory. An integrated learning environment for sec-ondary school mathematics was realized by Bokhove,Koolstra, Boon, and Heck (2007). The aim of thelearning environment is to provide students easilyaccessible practice mathematics problems and intelli-gent feedback when interacting with the content mate-rials. A tool called web-based authoring tool for AlgebraRelated domains (WEAR) assist teachers while author-ing exercises, monitors students during problem-sol-ving, and provides appropriate feedback (Virvou &Moundridou, 2000). WEAR combines knowledge ofauthoring algebraic equations which is applicable inother non-mathematical domains and student errordiagnosis (Virvou & Moundridou, 2001). It adapts theinteraction with students to provide individualizedfeedback (Moundridou & Virvou, 2002). In the nextsections, we concentrate on the key factors that distin-guish these systems in their implementation of adaptivefeedback.

4.1.1. Adaptive feedback means. An adaptive learningsystem alters its behavior based on how a learner inter-acts with it. These alterations are decided based on thelearner’s characteristics which are represented in thelearner model (Lo, Chan, & Yeh, 2012). It involves theaccurate tracking of learner’s activity, monitoring theirindividual characteristics, and providing timely adap-tive feedback according to effective pedagogical princi-ples (Narciss et al., 2014). Tailoring feedback accordingto learner’s characteristics and other external para-meters is a promising way to implement adaptation incomputer-based learning environment (Narciss et al.,2014). Adaptive feedback means, poses the question,

Bimba et al. 221

what information about the learner is known and used forproviding adaptive feedback? These information consistof students’ characteristics such as knowledge level,cognitive style, learning style, and gender.

Wayang Outpost provides adaptive feedback in theform of hints. Two types of hints are provided: (1) acomputational and numeric approach to solve a prob-lem and (2) spatial transformations and visual estima-tions to make the problem easier to solve (Arroyo et al.,2003). The choice of the hint provided by WayangOutpost is based on the learner’s cognitive profile. Thelearner profile is built based on an online assessment todetermine the learner’s math proficiency which includesaccuracy and speed of arithmetic computation and spa-tial ability. Wayang Outpost provides hints that capita-lize on the learner’s cognitive strength when one abilityis clearly better than the other. Otherwise, if both skillsare low, computational help is provided, whereas ifboth are high spatial help is provided (Arroyo et al.,2003). Similar to Wayang Outpost, Gerdes’ tutor pro-vides feedback in the form of hints. However, the hintsand feedback provided by Gerdes’ tutor does not utilizethe characteristics of the learner (such as knowledgelevel and learning style), instead it is generated automat-ically from an organized hierarchy according to the syn-tax tree of the model solution (Gerdes et al., 2012).

The student model in E-Tutor captures the path astudent has taken and the underlying source of theerror. It then provides instructional feedback based onthe students’ prior performance (Heift & Schulze,2007). The novice, intermediate, and expert are thethree types of learners assumed in the student model.These student levels are used to determine the specifi-city of the feedback provided. In AutoTutor, feedback isprovided in the form of a dialog. The dialog moves,pumps, hints, prompts, and assertions are selectedbased on the students’ knowledge (D’Mello & Graesser,2012). It constructs a cognitive model of the students’knowledge level based on the analysis of their typed orspoken responses. ITAP automatically generates feed-back in the form of hints. Using the path constructionalgorithm, it generates hints based on the students’solution strategy as determined in the solution space(Rivers & Koedinger, 2015). DeepTutor utilizes the stu-dents’ knowledge level in order to determine the typeand frequency of feedback (Rus et al., 2015). As theknowledge level of the student increases, less amount offeedback is provided.

In ACTIVEMATH, generic computer algebra sys-tems (CASs) are used to diagnose student’s actions inorder to provide hints, flag feedback (correct/incor-rect), and correct solution (Melis et al., 2007). It doesnot use any learner characteristics in deciding the typeof feedback to be provided. Similarly, Guru providesfeedback incrementally based on student’s knowledgelevel (Olney et al., 2012). INSPIRE supports adaptivenavigation support and adaptive presentation of

learning content only (Papanikolaou et al., 2003). Itdoes not use any student characteristics in providingfeedback. Whenever a learner requires help, examplesand hints are provided based on the theory presented(Papanikolaou et al., 2003). The FIT Java Tutor pro-vides feedback based on the students’ structured solu-tion space, which comprises student solution attemptsand sample solutions (Gross & Pinkwart, 2015). Andesprovides immediate feedback at each stage of problem-solving. The system provides immediate feedback basedon the students’ current knowledge or mental state(Gertner & VanLehn, 2000). Feedback in SQL-Tutor isprovided based on the number of student’s unsuccess-ful solution attempts (Mitrovic & Ohlsson, 1999).

Feedback in COMPASS is generally personalizedbased on identified error in a student’s concept maps,knowledge level, preferences, and interactive behavior(Gouli et al., 2006). Based on the theoretical frameworkproposed by Mason and Bruning (2001), an effectivefeedback design should take into consideration the stu-dent’s achievement level and prior knowledge. WhileNarciss and Huth’s (2002) conceptual framework sug-gests that the necessary information required for pro-viding informative feedback are the student’s learningobjectives, prior knowledge, learning strategies, and pro-cedural and meta-cognitive skills. In Excel Tutor, thesystem just considers the error made by students, itdoes not take into consideration any other personalizedcharacteristics of the student (Mathan & Koedinger,2005).

Adaptive feedback in Animalwatch is providedaccording to the student’s cognitive development andgender (Arroyo et al., 2000). In the process of assistingstudents in learning naive set theory, Tsovaltzi andFiedler (2003) developed a taxonomy of hints which isprovided to the students based on their current and pre-vious answers. Unlike the other approaches, Bokhoveet al. (2007) do not take into consideration any charac-teristics of the student while providing feedback.Instead, it provides intelligent feedback based on expertknowledge, common mistakes, and knowledge aboutthe learning domain. However, in WEAR, feedback isprovided based on the student’s knowledge level(Virvou & Moundridou, 2000).

4.1.2. Adaptive feedback target. In a computer-basedlearning environment, the pedagogical model repre-sents the knowledge and expertise of teaching. Specificknowledge represented in the pedagogical modelincludes effective teaching techniques (deductive andinductive); the various instructional methods (lectures,problem-based learning, inquiry learning, etc.); instruc-tional plan that define phases, roles, and sequence ofactivities (Scheuer, Loll, Pinkwart, & McLaren, 2010);feedback types, depending on a learner’s action; andassessment to inform and measure learning (Luckin &


Holmes, 2016). The domain model represents knowl-edge of the subject been learned. It mainly consists ofconcepts such as how to add, subtract, multiply num-bers; Newton’s law of motion; how to structure anargument; and different approaches to reading (Luckin& Holmes, 2016). Adaptive feedback target is involvedwith the aspect of the instructional system (pedagogyand domain knowledge) that is adapted to providefeedback based on the learner characteristics. Withinvarious instructional methods, there are certain condi-tions that affect the type of feedback provided.

The hint provided by Wayang Outpost could be invarious forms based on three multimedia learning the-ories which include modality principle, contiguity prin-ciple, and animation principle (Arroyo et al., 2014).The modality principle represents words in form ofspeech, the contiguity principle aligns text to corre-sponding graphics while animation principles producean illusion of characters adhering to the basic laws ofphysics. These principles guide the videos which showhow instructors solve maths problems; synchronizedsound, animation, and contiguous explanations ofmaths problem and worked examples. Adaptive feed-back inWayang Outpost is involved with both the peda-gogical (multimedia learning theories) and domainmodels (maths solution in form of worked examples,speech, graphics, and animation), selecting variouscomponents of these models based on the learner char-acteristics (Arroyo et al., 2014). Gerdes’ tutor providesinteractive feedback to students (Gerdes et al., 2012).These interactions give hints to students on the nextstep to take, list of possible ways to proceed, point-outerrors, and provide complete worked-out examples.These information are all part of the instructional mate-rial, but they are not provided based on the student’scharacteristics. Instead, students are provided with anoption to choose what type of hint they desire.

Unlike Gerdes’ tutor, E-Tutor alters its instructionalfeedback within the domain model based on the stu-dent’s proficiency as determined in the student modelby the percentage of previously correct answers (Heift& Schulze, 2007). AutoTutor provides feedback in theform of dialog. The decision to provide a specific formof feedback, either pumps, hints, an answer or aprompt, depends on the information received by thetutor from the student (Nye, Graesser, & Hu, 2014).ITAP extends the Hint Factory (domain knowledge) byautomatically generating hints that are tailored to anindividual’s solution to a problem (Rivers & Koedinger,2015). DeepTutor provides scaffolding and a sequenceof progressive hints, based on the student’s knowledgelevel as articulated in the student model (Rus, Conley,& Graesser, 2014). In ACTIVEMATH, the domain rea-soner generates and provides hints, flag feedbacks, andcorrect solution based on the diagnosis of student’ssolution steps (Melis, Goguadze, Libbrecht, & Ullrich,

2009). Depending on the type of error made by a stu-dent, the Guru tutor assesses the student’s knowledgeand response with a positive feedback, negative feed-back, neutral feedback, or elaborative feedback. Eventhough INSPIRE provides adaptation for navigationsupport and learning content based on student’s learn-ing style, it does not provide various forms of helpaccording to the student’s learning style.

When there is no information about the quality of astudent’s solution (without representative solution), FITJava Tutor provides feedback in form of self-reflectionprompts. At instances where the quality of a student’ssolution is partially known, feedback F1, F2, or a com-bination of F1 and F2 are provided based on previoussuccesses of theses strategies on similar solution quali-ties (Gross, Mokbel, Paassen, Hammer, & Pinkwart,2014). F1 feedback strategy is when the student’s solu-tion differ partially from the correct solution but imple-ments the same problem-solving strategy, the differenceis highlighted without showing the actual solution. F2feedback strategy is when the student’s solution is con-trast with the actual solution but the correct problem-solving strategy is used, the solutions are contrasted toallow the student to compare and find the possible mis-takes. Andes provides flag feedback accompanied byhints and error messages, which the student can decideto consult when they stuck. These hints are generatedusing the solution graph in the domain model, based onthe state of the student model (Gertner & VanLehn,2000). In SQL-Tutor, feedback messages are providedas right/wrong, error flag, hints, partial solution, andcomplete solution (Mitrovic & Ohlsson, 1999). Theseforms of feedback, which are part of the pedagogicalmodule, are provided to the student based on the num-ber of successful solution attempts.

The process of generating effective feedback inCOMPASS depends on the student’s answer duringproblem-solving (Gouli et al., 2006). In COMPASS,feedback is provided based on an answer categorizationscheme. According to the scheme, a student’s answercan be characterized based on completeness, accuracy,and missing out. However, based on (Mason &Bruning, 2001) theoretical framework, less specificfeedback is provided as the learning tasks and student’sknowledge level increases. Students with higherachievement levels can benefit more from feedbackwhich provides general information. Thus, allowingthem to identify their errors and accurately seek thecorrect solutions (Mason & Bruning, 2001). In Narcissand Huth’s (2002) conceptual framework, the aspectsof the instructional content that affects the type of feed-back provided are the instructional goals, learningtasks, issues, and learning problems. This frameworkrecommends a careful alignment of the type of feed-back provided and the characteristics of the instruc-tional content.

Bimba et al. 223

In Excel Tutor, feedback is generated based on anintelligent novice model. The diagnostic capabilities ofthe model supports the provision of context-specificfeedback to students (Mathan & Koedinger, 2005).Similarly, in Animalwatch, hints are classified based onthe degree of hint symbolism and interactivity (Arroyoet al., 2000). However, these hints are provided ran-domly regardless of the student’s interactions.However, a hinting algorithm is used by Tsovaltzi andFiedler (2003) to evaluate the student’s performanceand to provide relevant hint. Different forms of localfeedback are provided by Bokhove et al. (2007). Thefeedback could be a comment regarding the accuracyof a response or an explanation based on the student’sanswer. But, in WEAR, there is no clear aspect of theinstructional system that is adapted in providing feed-back. However, the instructor and student model inWEAR interact with each other to mimic a one-to-onetutoring setting (Moundridou & Virvou, 2002).

4.1.3. Adaptive feedback goal. Feedback can serve differ-ent purposes based on pedagogical principles or a par-ticular learning theory that is been applied. From anobjectivist perspective, feedback is regarded as a rein-forcement, which is aimed at guiding the learner toprogress from a simpler task to a more complex one.The information processing theory suggests that thegoal of feedback is not only to reinforce correctanswers but also to serve as corrective information toallow learners correct their errors (Hattie & Gan,2011). Socioculturalism considers feedback as a recipro-cal dialog, where meaning is reconstructed by peers.The goal of feedback from this view is the consolida-tion, reorganization, and making knowledge explicitthrough exchange of ideas between peers (Pryor &Crossouard, 2008). Visible learning theory views feed-back in the context of student’s learning (alone, withpeers, or adults), at different levels of expertise (novice,proficient, or expert) and level of understanding (sur-face, deep, and conceptual) (Hattie & Gan, 2011). Inan inductive teaching method such as discovery learn-ing, feedback is provided only based on a student’seffort and not as a direct guide for those efforts (Prince& Felder, 2006). In developing adaptive feedback sys-tems, the designer needs to consider the goal of provid-ing feedback. The adaptive feedback goal identifiespedagogical reasons for providing feedback based onthe learner model, thus differentiating various imple-mentations. These characteristics revile the function offeedback.

In Wayang Outpost, adaptive feedback is providedbased on the theory of cognitive apprenticeship, wherea master teaches skills to an apprentice. The main aimof this theory is to encourage learners to accomplishmore difficult problems than they can accomplish with-out a guide. Thus, adaptive feedback in Wayang

Outpost is aimed at providing motivational support andencouraging engagement in the learning process. Helpis provided in the form of similar work examples, whichenable students to solve similar or harder problemsthan the current problem (Arroyo et al., 2014). Thefeedback and hints in Gerdes’ tutor are provided basedon teacher-specified annotations of solutions (Gerdeset al., 2012). There is no pedagogical principle or learn-ing theory involved. The purpose of providing feedbackin E-Tutor is based on the language teaching pedagogy(Heift & Schulze, 2007). This learning theory ensuresthat the amount of feedback provided does not confusethe student (Heift & McFetridge, 1999). AutoTutor pro-vides feedback with the goal of stimulating active con-struction of knowledge based on the constructivistprinciple (D’Mello & Graesser, 2012).

In ITAP, hints are provided as either point hint orbottom-out hint (Rivers & Koedinger, 2015). The pur-pose of providing such feedback does not depend onany pedagogical principle or learning theory.DeepTutor provides different types of hints which aredynamically sequenced based on a constructivist scaf-folding strategy (Rus et al., 2014). Feedback inActiveMath is based on the moderate constructivistapproach. It is aimed at providing a reasonable amountof guidance which allows learners to choose and reflecton their work (Melis et al., 2007). Even though, Guruprovides imcremental feedback which is aimed at tai-loring conversations based on individual student’sknowledge level, it does not base this feedback on anypedagogical principle or learning theory (Olney et al.,2012).

INSPIRE uses the instructional design theory andlearning style theory to provide individualized instruc-tions. However, the hints provided by INSPIRE onlyindicate right or wrong, but do not depend on any ped-agogical principle or learning theory (Papanikolaouet al., 2003). FIT Java Tutor provides feedback which isaimed at guiding students toward self-reflection basedon the example-based learning theory (Gross et al.,2014). Feedback in Andes is aimed at encouraging con-structive learning (Gertner & VanLehn, 2000). Thus,providing little feedback to students unless they requestfor it. The aim of feedback in SQL-Tutor is to promotethe acquisition of new cognitive skills, through the stateconstraint theory (Mitrovic & Ohlsson, 1999). The con-straint theory suggests that acquiring new cognitiveskills is based on the transfer of knowledge from theevaluative to the generative component. Thus, feedbackin SQL-Tutor is provided when students submit solu-tions for evaluation.

The goal of feedback in COMPASS is to supportreflection by guiding the students to reconstruct theirknowledge (Gouli et al., 2006). Mason and Bruning’s(2001) theoretical framework proposes the incorpora-tion of immediate feedback when the goal of instructionis teaching new concepts or the facilitation of concept


acquisition. While delayed feedback should be providedwhen the instructional aim is to develop higher orderskills like abstract reasoning or comprehension (Mason& Bruning, 2001). According to Narciss et al.’s (2014)conceptual framework, feedback can be viewed in thecontext of self-regulated learning, behavioral, and cog-nitive learning theories. Based on these theories, feed-back can either have a goal of tutoring or guiding thelearner, re-enforcement of a concept, or a source ofinformation (Narciss et al., 2014).

The intelligent novice model used in Excel Tutor isaimed at supporting a student in the generative and eva-luative aspects of learning a new skill. It explicitly mod-els these skills and guides students through the processof error detection and correction (Mathan & Koedinger,2005). Tsovaltzi and Fiedler’s (2003) natural languagedialog module provides hints based on the Socratictutoring strategy in order to achieve self-explanation(Tsovaltzi & Fiedler, 2003). While feedback is providedin Animalwatch (Arroyo et al., 2000), Bokhove et al.(2007), and WEAR (Moundridou & Virvou, 2002),there is no clear pedagogical reason for its provision.

4.1.4. Adaptive feedback strategy. Adaptive feedbackstrategies combine several feedback components toassist learners in identifying gaps that exist betweentheir current and desired knowledge state (Narciss,2013). These feedback strategies could be in severalforms which include adaptive bottom-up feedback(where detailed feedback is provided and as proficiencyincreases the feedback changes to general), adaptivetop-down feedback (general feedback is provided first,if there is no improvement then a detailed feedback isprovided), outcome feedback (indicate right or wrong),hints on how to proceed, and location of mistakes(Billings, 2012). Most of the time, a combination ofthese strategies are used to ascertain appropriate feed-back conditions (Narciss, 2013). In some situations, thelearner is given an option to interact with the systemand determine the need for feedback. In implementingthese strategies, several modeling and artificial intelli-gence techniques are used. The adaptive feedback strat-egy looks at the steps taken in providing feedbackbased on changes in student proficiency, how active orreactive are the learners in the feedback process, andthe modeling and artificial intelligence techniques usedin implementing adaptive feedback. The implementa-tion of an adaptive feedback strategy determines thetiming and scheduling of feedback.

The strategy used by Wayang Outpost is through astep-by-step instruction and guidance to a solution.Adaptive feedback is provided only when the learnerrequests for help. There is no explicit process for pro-viding feedback as learner’s proficiency increases(Arroyo et al., 2014). Help is provided when a learnerhas difficulty in one problem, and then a similar

problem is provided, encouraging a transfer of knowl-edge to subsequent problems. The approach used byWayang Outpost to implement this strategy is a data-centric Bayesian Network which produces a probabilitymodel based on student’s previous interaction with thesystem. Help seeking is modeled to see how hint isrelated to skills (Arroyo et al., 2014). The Bayesian net-work has nodes corresponding various hints and skills.Hints in the Gerdes’ tutor is provided in steps (Gerdeset al., 2012). When a student is stuck, they can requestfor help from the tutor. The tutor provides optionsbased on the annotated teacher-specified feedback. If achoice is made, the student can ask for further details ifthe first explanation is not clear. Afterward, the tutorresponds with more details and a bottom-out hint. Toprovide a semantically rich feedback, Gerdes’ tutor usedtechniques such as parsing, rewriting, and programtransformation (Gerdes et al., 2012). In E-Tutor, thefeedback process is iterative. Student’s errors are identi-fied and communicated one at a time. These iterativeprocesses continue until the student gets the correctanswer or decides to submit a solution (Heift, 2010).

Unlike Gerdes’ tutor, E-Tutor does not provide stu-dents with a feedback choice. Instead, feedback is gener-ated based on the correlation between the result of thelinguistic analysis of a student solution and an error-specific feedback message. A parser and head-drivenphase structure grammar (HPSG) are used to determinegrammatically incorrect sentences and associate theerrors detected with feedback messages (Heift, 2016;Heift & Nicholson, 2001). AutoTutor provides feedbackto a student’s initial answer by first providing a shortfeedback, an elaborative feedback, and then an encour-agement (D’Mello & Graesser, 2012). During this pro-cess, the student is actively involved in a conversationwith the tutor. The latent semantic analysis (LSA) algo-rithm in AutoTutor is used to determine the informationwithin a student’s response that matches an expectationin the ideal answer, while a subthreshold expectationselection algorithm determines the prescribed sequentialorder to present expectations (D’Mello & Graesser,2012). When a student makes an error, ITAP provideshints in two levels. The first level (point hint) informsthe student about the type of change required and wherethe change should be. While the second level (bottom-out hint) provides all the information needed to correctthe error (Rivers & Koedinger, 2015). The path con-struction algorithm is used to generate a chain of hintsthat leads to a correct solution state.

DeepTutor provides feedback using the tell-or-elicittactic (Rus et al., 2014). This strategy is based on thescaffolding-modeling-fading theory. DeepTutor imple-ments the theory by eliciting a step, if not compre-hended by the student then it tells. This ensures anactive participation by the student through a two-wayconversation with the tutor. The management of thisdialog is implemented using production rules. In

Bimba et al. 225

ActiveMath, the domain reasoner generates feedbacksuch as flag feedbacks, correct solution, next step hint,correct input, and number of steps to final solution(Melis et al., 2009). These feedback forms are not pre-sented in any sequence. However, students can requestfor hints when needed (Melis et al., 2007). The domainreasoner in ActiveMath is implemented using rule-basedtechniques. The feedback provided by Guru is providedin form of dialog. The response of the tutor on the typeof feedback to be provided is based on an assessment ofthe student’s knowledge. In a case where the studentmakes an error, the incorrect relationship is highlightedand an explanation is provided for the meaning of therelationship; however, if the student has little back-ground knowledge, an extended direct instruction isprovided (Olney, Person, & Graesser, 2011). Guru usesLSA and concept map to align the students’ utteranceswith the domain and students models to determine if aninput is correct or wrong.

In INSPIRE hints and examples are provided to stu-dents to indicate right or wrong or on request.However, these feedbacks are provided without anyconsiderations of a specific type or sequence of presen-tation. The FIT Java Tutor uses a consecutive combina-tion of the F1 and F2 strategy in providing feedback,depending on the learners’ needs and progress. Withthe aim of providing support which is relevant to thestudent’s needs, FIT Java Tutor provides feedback withvarying levels of detail according to the student’s learn-ing progress (Gross et al., 2014). The automated provi-sion of feedback is developed based on clusteredsolution space. ANDES provides help in a sequencebased on three levels. It provides flag feedback in formof a pop up message when the error is likely a slip andnot lack of knowledge, and if it is not recognized as aslip, it is highlighted red. The second level is the what’swrong help, where students can click on a red entry andfind out the reason behind the error. Finally, studentscan request for help when they are not sure of what todo next (VanLehn et al., 2005). During this process, thestudent is actively involved in selecting the sequence ofhints provided. In order to provide immediate feed-back, ANDES uses a context-free parser to detecterrors in student’s input and a solution graph whichcontains relevant solution entries. SQL-Tutor post-pones feedback until the end of problem-solving steps.At the end of problem-solving, the student is presentedwith all the errors, but feedback is given for only oneerror. The feedback is based on the amount of informa-tion they provide. They are in five levels: right/wrong,error flag, hint, partial solution, and complete solution.The levels of feedback are provided based on the stu-dent’s unsuccessful solution attempts (Mitrovic &Ohlsson, 1999). However, the student can request for apartial or complete solution. Violations in a student’ssolution are determined with the aid of constraint-based modeling, relevance, and satisfaction networks.

The steps taken to provide feedback in COMPASSare a gradual provision of various types of feedbacksbased on a four-layered structure and the category of astudent’s answer (Gouli et al., 2006). Feedback inCOMPASS is implemented with the help of conceptmaps used for identifying student’s errors. Mason andBruning’s (2001) theoretical framework suggests a dif-ferent strategy, where the student is provided with aknowledge-of-response feedback and then allowed todecide if they require additional feedback. Mason andBruning (2001) suggest that this strategy will help todevelop the student’s understanding in situations wherethe correct answer was a guess. A proposed guidelinefor selecting and specifying different forms of feedbackis presented by Narciss et al.’s (2014) conceptual frame-work. These guidelines aim at ensuring the studentreceives the appropriate feedback based on the learningtask.

In Excel Tutor, an immediate corrective feedback isprovided to the student at the formulas correction step.However, if the student requests for help in correctingthe error, the system provides a gradual two-step feed-back process. The first step focuses on error detectionand the second step involves error correction (Mathan& Koedinger, 2005). Corrective feedback in Excel Tutoris implemented using production rules associated witherror free and efficient task performance (Mathan &Koedinger, 2005). Whenever a student enters a wronganswer during a tutoring session in Animalwatch, a hintis provided. The first hint provides little amount ofinformation and if the student keeps providing thewrong answer, the system guides them through thewhole process of problem-solving (Arroyo et al., 2000).For implementing feedback in Animalwatch, machinelearning techniques, linear regression, and analysis ofvariance (ANOVA) are used to predicting hint effective-ness. Similarly, Tsovaltzi and Fiedler’s (2003) naturallanguage dialog module supports a gradual provision ofhints based on the number of hints given, the numberof wrong answers, and the category of the student’sanswer. A combination of ontology and productionrules are used to generate feedback in Tsovaltzi andFiedler’s (2003) natural language dialog module. Thereis no identifiable strategy for the provision of feedbackin Bokhove et al. (2007). Student’s errors are only high-lighted in different colors to differentiate correct,incomplete, and wrong answers. Similar to Bokhoveet al. (2007), WEAR (Moundridou & Virvou, 2002) pro-vides individualized feedback to students without anyspecific strategy.

5. Discussion

In this study, 20 different implementations of adaptivefeedback were reviewed and analyzed. These imple-mentations were selected based on their impact and


contribution in computer-based adaptive learningresearch. We present our analysis on these implemen-tations based on the classification criteria for adaptivefeedback, highlighting their levels of adaptive feed-back provision. Subsequently, we compared the vari-ous implementations of adaptive feedback based onadaptive feedback means, target, goal, and strategy;domain of implementation; adaptive feedback imple-mentation techniques; and evaluation method.

5.1. Adaptive feedback implementation categories

In Figure 2, the adaptive feedback implementations arepresented based on the classification scheme discussedin Section 4. We categorized the feedback implementa-tion based on their implementation of adaptive feed-back means, target, goal, and strategy. Figure 2 alsoshows how the adaptive feedback characteristics arealigned to the pedagogical domain and student modelsof a computer-based learning environment. The adap-tive feedback target, strategy, and goal are determinedby concepts in the pedagogical model. However, adap-tive feedback target and strategy can be implementedusing concepts represented in the domain model.Finally, the adaptive feedback means is determined byfactors in the student model.

Similarly, there is a relationship between the charac-teristics of feedback and the features of adaptive feed-back. As shown in Figure 2, the implementation of anadaptive feedback strategy determines the timing andscheduling of feedback. The adaptive feedback goal iden-tifies pedagogical reasons for providing feedback basedon the learner model. These characteristics revile thefunction of feedback. Within various instructional meth-ods, there are certain conditions that affect the type offeedback provided. These reveal a distinct relationshipbetween the characteristics of feedback, features ofadaptive feedback, and computer-based learning models(pedagogy, domain, and student models).

5.2. Comparison of adaptive feedbackimplementations

A detailed comparison of the various implementationsof adaptive feedback is presented in Table 2. The mainobjective is to identify the common ways for imple-menting adaptive feedback means, target, goal, andstrategies; common domains for adaptive feedbackimplementations; adaptive feedback implementationtechniques; and common evaluation techniques. Basedon the results of the comparison, the following conclu-sions were obtained:

Figure 2. Classification of adaptive feedback implementations.

Bimba et al. 227

Tab

le2.

Com

par

ison

ofad

aptive

feed

bac

kim

ple

men

tations.

Feed

bac

kim

ple

men

tation

Dom

ain

Adap

tive

feed

bac

km

eans

Adap

tive

feed

bac

kta

rget

Adap

tive

feed

bac

kgo

al(p

edag

ogi

cal

pri

nci

ple

)

Adap

tive

feed

bac

kst

rate

gyIm

ple

men

tation

tech

niq

ue

Eva

luat

ion

tech

niq

ue

Way

ang

Outp

ost

(Arr

oyo

etal

.,2014)

Mat

hem

atic

sSt

uden

t’sco

gnitiv

epro

file

Multim

edia

lear

nin

gth

eori

es(p

edag

ogi

cal

know

ledge

).W

ork

ed-

exam

ple

s,sp

eech

,gr

aphic

s,an

dan

imat

ion

(dom

ain

know

ledge

)

Bas

edon

theo

ryof

cogn

itiv

eap

pre

ntice

ship

Step

-by-

step

inst

ruct

ion.A

ctiv

ele

arner

Dat

a-ce

ntr

icBay

esia

nN

etw

ork

Pre

-tes

tan

dpost

-te

st

Ger

des

’Tu

tor

(Ger

des

,Je

uri

ng,

&H

eere

n,2012)

Pro

gram

min

gN

/AN

/A(h

ints

and

work

edex

ample

sar

epro

vided

equal

ly,w

ithout

consi

der

ing

studen

ts’

char

acte

rist

ics)

N/A

Leve

lsofdet

ail.

Act

ive

lear

ner

Par

sing,

rew

riting,

and

pro

gram

tran

sform

atio

n

Ques

tionnai

re

E-T

uto

r(H

eift

&Sc

hulz

e,2007).

Langu

age

lear

nin

gSt

uden

t’skn

ow

ledge

leve

lIn

stru

ctio

nal

feed

bac

k(d

om

ain

know

ledge

)B

ased

on

langu

age

teac

hin

gped

agogy

Iter

ativ

eer

ror

det

ection.R

eact

ive

lear

ner

Par

ser

and

hea

d-

dri

ven

phas

est

ruct

ure

gram

mar

(HPSG

)

Anal

ysis

oflo

gdat

ain

the

repo

rtm

anag

er

Auto

Tuto

r(D

’Mel

lo&

Gra

esse

r,2012).

Phy

sics

,co

mpute

rlit

erac

y,an

dcr

itic

alth

inki

ng

Studen

t’skn

ow

ledge

leve

lPum

p,hin

ts,an

swer

s,an

dpro

mpts

(dom

ain

know

ledge

)

Bas

edon

const

ruct

ivis

tpri

nci

ple

Sequen

ceof

feed

bac

ks.A

ctiv

ele

arner

Late

nt

sem

antic

anal

ysis

(LSA

)an

dsu

bth

resh

old

expec

tation

sele

ctio

nal

gori

thm

Bys

tander

Turi

ng

test

.Exper

tco

mpar

ison

ITA

P(R

iver

s&

Koed

inge

r,2015)

Pro

gram

min

gSt

uden

t’sso

lution

stra

tegy

Hin

tfa

ctory

(dom

ain

know

ledge

)N

/ALe

vels

ofdet

ails

.R

eact

ive

lear

ner

Pat

hco

nst

ruct

ion

algo

rith

mD

atas

etan

alys

is

Dee

pTu

tor

(Rus,

Nir

aula

,&

Ban

jade,

2015)

Phy

sics

Studen

t’skn

ow

ledge

leve

lSc

affo

ldin

gan

dse

quen

tial

hin

ts(d

om

ain

know

ledge

)B

ased

on

aco

nst

ruct

ivis

tsc

affo

ldin

gst

rate

gy

Leve

lsofdet

ails

.A

ctiv

ele

arner

Pro

duct

ion

rule

sW

ord

-to-w

ord

sim

ilari

tym

easu

reusi

ng

Mic

roso

ftR

esea

rch

Par

aphra

se(M

SRP)

Corp

us

Act

iveM

ath

(Mel

is,

Moorm

ann,U

llric

h,

Gogu

adze

,&

Libbre

cht,

2007)

Mat

hem

atic

sSt

uden

t’sac

tion

Hin

ts,fla

gfe

edbac

ks,an

dco

rrec

tso

lution

(dom

ain

know

ledge

)

Bas

edon

moder

ate

const

ruct

ivis

tap

pro

ach

No

explic

itst

rate

gyfo

rpre

senting

feed

bac

k.A

ctiv

ele

arner

Rule

-bas

edN

/A

Guru

(Oln

eyet

al.,

2012)

Bio

logy

Studen

t’skn

ow

ledge

leve

lPo

sitive

feed

bac

k,neg

ativ

efe

edbac

k,neu

tral

feed

bac

k,or

elab

ora

tive

feed

bac

k(d

om

ain

know

ledge

)

N/A

Leve

lofdet

ails

.A

ctiv

ele

arner

LSA

and

conce

pt

map

sPre

-tes

tan

dpost

-te

st

(con

tinue

d)


Tab

le2.C

ontinued

Feed

bac

kim

ple

men

tation

Dom

ain

Adap

tive

feed

bac

km

eans

Adap

tive

feed

bac

kta

rget

Adap

tive

feed

bac

kgo

al(p

edag

ogi

cal

pri

nci

ple

)

Adap

tive

feed

bac

kst

rate

gyIm

ple

men

tation

tech

niq

ue

Eva

luat

ion

tech

niq

ue

INSP

IRE

(Pap

anik

ola

ou,

Gri

gori

adou,

Korn

ilaki

s,&

Mag

oula

s,2003)

Com

pute

rar

chitec

ture

N/A

N/A

N/A

Outc

om

efe

edbac

k.A

ctiv

ele

arner

N/A

Ques

tionnai

rean

ddat

atr

acki

ng

inac

tivi

tylo

gs

FIT

Java

Tuto

r(G

ross

&Pin

kwar

t,2015)

Pro

gram

min

gSt

uden

t’sso

lution

stra

tegy

Self-

refle

ctio

npro

mpts

,F1

and

F2fe

edbac

kst

rate

gies

Exam

ple

-bas

edle

arnin

gth

eory

Leve

lofdet

ail

bas

edon

conse

cutive

com

bin

atio

nofF1

and

F2st

rate

gies

.A

ctiv

ele

arner

Clu

ster

edso

lution

spac

e.R

elat

ional

neu

ralga

s(R

NG

)cl

ust

er

AN

DES

(Van

Lehn

etal

.,2005)

Phy

sics

Studen

t’sm

enta

lst

ate

Pro

vides

flag

feed

bac

kac

com

pan

ied

by

hin

tsan

der

ror

mes

sage

sac

cord

ing

toth

eso

lution

grap

h(d

om

ain

model

)

Bas

edon

const

ruct

ivis

tle

arnin

gth

eory

Sequen

ceofhin

tsw

ith

leve

lsof

det

ail.

Act

ive

lear

ner

Conte

xt-

free

par

ser

and

solu

tion

grap

h

Ques

tionnai

re

SQL-

Tuto

r(M

itro

vic,

Ohls

son,

&Bar

row

,2013)

Pro

gram

min

gSt

uden

t’sunsu

cces

sful

solu

tion

atte

mpts

Rig

ht/

wro

ng,

erro

rfla

g,hin

ts,par

tial

solu

tion,an

dco

mple

teso

lution

(ped

agogi

calm

odule

)

Bas

edon

stat

eco

nst

rain

tth

eory

Leve

lsofdet

ail.

Act

ive

lear

ner

Const

rain

t-bas

edm

odel

ing.

Rel

evan

cean

dsa

tisf

acto

rynet

work

s

Ques

tionnai

re

CO

MPA

SS(G

ouli,

Gogo

ulo

u,

Pap

anik

ola

ou,&

Gri

gori

adou,2006)

Intr

oduc

tory

info

rmat

ics

Studen

t’skn

ow

ledge

leve

l,pre

fere

nce

s,an

din

tera

ctiv

ebeh

avio

r

Com

ple

tenes

sofso

lution,

accu

racy

,an

dm

issi

ng

out

(dom

ain

module

)

Support

for

refle

ctio

nG

radual

pro

visi

on

offe

edbac

kbas

edon

four

laye

rs.

Inac

tive

lear

ner

Conce

pt

map

sEm

pir

ical

studie

s

Mas

on

and

Bru

nin

g(2

001)

Dom

ain

indep

enden

tSt

uden

t’sac

hie

vem

ent

leve

lan

dpri

or

know

ledge

Incr

ease

diff

iculty

of

lear

nin

gta

sk(d

om

ain

module

)

Support

for

new

conce

pt

acquis

itio

nan

dab

stra

ctre

asonin

g

Know

ledge

of

resp

onse

feed

bac

k.A

ctiv

ele

arner

N/A

N/A

Nar

ciss

and

Huth

(200

2)

Dom

ain

indep

enden

tSt

uden

t’sle

arnin

gobje

ctiv

es,pri

or

know

ledge

,le

arnin

gst

rate

gies

,pro

cedura

l,an

dm

eta-

cogn

itiv

esk

ills

Inst

ruct

ional

goal

s,le

arnin

gta

sks,

issu

es,an

dpro

ble

ms

(dom

ain

module

)

Self-

regu

late

dle

arnin

g,beh

avio

ral

and

cogn

itiv

ele

arnin

gth

eori

es

Know

ledge

of

corr

ect

resp

onse

and

elab

ora

tive

feed

bac

kbas

edon

spec

ified

guid

elin

es.

Inac

tive

lear

ner

N/A

N/A

Exce

lTu

tor

(Mat

han

&K

oed

inge

r,2005)

Com

pute

rsc

ience

N/A

Inte

llige

nt

Novi

ceM

odel

whic

hre

pre

sents

the

erro

rsm

ade

by

studen

ts(d

om

ain

module

)

Support

sge

ner

ativ

ean

dev

aluat

ive

lear

nin

g

Imm

edia

teco

rrec

tive

feed

bac

k,gr

adual

2-s

tep

pro

cess

.A

ctiv

eLe

arner

Pro

duct

ion

rule

sA

NO

VA

(con

tinue

d)

Bimba et al. 229

� Student’s knowledge level is mostly used for decid-ing what feedback to provide during problem-solving.

� The forms of feedback in the domain knowledge isthe most common aspect of the instructional systemthat changes based on the student’s characteristics.

� The goal of providing adaptive feedback is mostlydue to the constructivist learning theory.

� Students are usually involved actively in the feed-back process.

� Adaptive feedback is usually presented with a cer-tain level of detail, based on the student’s initialresponse.

� LSA and parsers are the most common techniquesused in implementing adaptive feedback.

� The most common techniques used for evaluatingadaptive feedback implementations are throughquestionnaires, pre-test and post-test, and analysisof log data.

Adaptive feedback seems to be more easily implemen-ted in the programming domain as seen from the fullimplementation of adaptive feedback features in FITJava Tutor and SQL-Tutor. This could be as a result ofthe logical and procedural nature of the programmingdomain.

6. Future direction

Adaptive feedback support is necessary in a computer-based learning environment because of the difference instudents’ characteristics. Based on our review, provid-ing full adaptive feedback is yet to be implemented innon-procedural domains. Further research is requiredto tackle this issue. Subsequently, there is a need for anadaptive feedback framework which can accommodatethe various adaptive feedback criteria presented andsupport multiple adaptive feedback means, target, goal,and strategy. This will allow for a better evaluation ofthe following:

� Is there a right combination of adaptive feedbackmeans, target, goal, and strategy which caters for aparticular student?

� Can multiple student characteristics be used effi-ciently for providing efficient feedback?

An area which requires further investigation is the com-plexity of aligning multiple adaptive feedback charac-teristics to a specific student’s needs.

7. Conclusion

Feedback is an effective tool used in typical classroomsettings during teaching. However, the feedback pro-vided is usually to a group of students with differentT

ab

le2.C

ontinued

Feed

bac

kim

ple

men

tation

Dom

ain

Adap

tive

feed

bac

km

eans

Adap

tive

feed

bac

kta

rget

Adap

tive

feed

bac

kgo

al(p

edag

ogi

cal

pri

nci

ple

)

Adap

tive

feed

bac

kst

rate

gyIm

ple

men

tation

tech

niq

ue

Eva

luat

ion

tech

niq

ue

Anim

alw

atch

(Arr

oyo

,Bec

k,W

oolf,

Bea

l,&

Schultz,

2000)

Mat

hem

atic

sSt

uden

t’sco

gnitiv

edev

elopm

ent

and

gender

N/A

N/A

Incr

easi

ng

amount

ofin

form

atio

n.

Inac

tive

lear

ner

Mac

hin

ele

arnin

gA

NO

VA

Tso

valtzi

and

Fied

ler

(200

3)

Mat

hem

atic

sSt

uden

t’scu

rren

tan

dpre

vious

answ

ers

N/A

Socr

atic

tuto

ring

stra

tegy

Gra

dual

pro

visi

on

ofhin

ts.In

active

lear

ner

Onto

logy

and

pro

duc

tion

rule

sN

/A

Bokh

ove

,K

ools

tra,

Boon,a

nd

Hec

k(2

007)

Mat

hem

atic

sN

/ALo

calfe

edbac

kin

dic

atin

gco

rrec

t,in

com

ple

te,an

dw

rong

answ

ers

N/A

N/A

N/A

Shar

able

conte

nt

obje

ctre

fere

nce

model

WEA

R(V

irvo

u&

Moun

dri

dou,2000)

Mat

hem

atic

sSt

uden

t’skn

ow

ledge

leve

lLo

calfe

edbac

kin

dic

atin

gco

rrec

t,in

com

ple

te,an

dw

rong

answ

ers

N/A

N/A

N/A

Em

pir

ical

study

N/A

:not

appl

icab

le.


characteristics. This results in a gap between studentswho excel the most and those who excel less. Providingadaptive feedback that caters for students based ontheir individual characteristics have been implemen-ted in computer-based learning environments witheffective results. This review focuses on 20 differentimplementations of adaptive feedback in computer-based learning environment, ranging from intelligenttutoring system (ITS), multimedia web-based ITS,dialog-based ITS, web-based intelligent e-learningsystem, adaptive hypermedia system, and intelligentand adaptive learning environment. These implemen-tations were carefully selected based on their impactin providing feedback to students. The main objectiveof the review is to compare adaptive feedback systemsaccording to feedback adaptation characteristics andidentify major open research questions in adaptivefeedback implementations.

The review resulted in categorizing these feedbackimplementations based on the students’ informationused for providing feedback (adaptive feedback means),the aspect of the domain or pedagogical knowledge thatis adapted to provide feedback based on the students’characteristics (adaptive feedback target), the pedagogi-cal reason for providing feedback (adaptive feedbackgoal), and the steps taken to provide feedback with orwithout students’ participation (adaptive feedback strat-egy). Other information such as the common adaptivefeedback means, goals, and implementation techniquesare identified. This review reviles a distinct relationshipbetween the characteristics of feedback, features ofadaptive feedback, and computer-based learning models(pedagogy, domain, and student models).

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest withrespect to the research, authorship, and/or publication of thisarticle.

Funding

The author(s) disclosed receipt of the following financial sup-port for the research, authorship, and/or publication of thisarticle: This work was supported by the University of MalayaResearch Grant (RP040B-15AET, 2015).

References

Advisors, E. G. (2013). Learning to adapt: A case for accelerat-

ing adaptive learning in higher education. Education Growth

Advisors. Retrieved from http://tytonpartners.com/library/

accelerating-adaptive-learning-in-higher-educationArroyo, I., Beal, C., Bergman, A., Lindenmuth, M., Marshall,

D., & Woolf, B. P. (2003). Intelligent tutoring for high-

stakes achievement tests. Artificial Intelligence in Educa-

tion: Shaping the Future of Learning Through Intelligent

Technologies, 97, 365.

Arroyo, I., Beck, J. E., Woolf, B. P., Beal, C. R., & Schultz,

K. (2000). Macroadapting animalwatch to gender and cog-nitive differences with respect to hint interactivity and sym-

bolism. In International Conference on Intelligent Tutoring

Systems (Vol. s1839, pp. 574–583).Arroyo, I., Woolf, B. P., Burelson, W., Muldner, K., Rai, D.,

& Tai, M. (2014). A multimedia adaptive tutoring systemfor mathematics that addresses cognition, metacognition

and affect. International Journal of Artificial Intelligence in

Education, 24, 387–426. New York: Springer.Billings, D. (2012). Efficacy of adaptive feedback strategies in

simulation-based training. Military Psychology, 24,114–133.

Bokhove, C., Koolstra, G., Boon, P., & Heck, A. (2007).

Towards an integrated learning environment for mathe-

matics. In 8th International Conference on Technology in

Mathematics Teaching. Czech Republic, 1–4 July 2007, 5pp.

Brusilovsky, P. (1998). Methods and techniques of adaptive

hypermedia. In Adaptive hypertext and hypermedia (pp. 1–

43). Netherlands: Springer.Brusilovsky, P. (1999). Adaptive and intelligent technologies

for web-based education. Ki, 13(4), 19–25.Carter, J. (1984). Instructional learner feedback: A literature

review with implications for software development. Com-

puting Teacher, 12, 53–55. Retrieved from https:

//www.learntechlib.org/p/169781Chieu, V. M. (2005). Constructivist learning: An operational

approach for designing adaptive learning environments sup-

porting cognitive flexibility (Unpublished doctoral disserta-

tion). Universiteit Twente, Enschede, The Netherlands.D’Mello, S., & Graesser, A. (2012). Autotutor and affective

autotutor: Learning by talking with cognitively and emo-

tionally intelligent computers that talk back. ACM Trans-

actions on Interactive Intelligent Systems (Tiis), 2(4),

Article 23.Economides, A. A. (2006). Adaptive feedback characteristics

in cat. International Journal of Instructional Technology

and Distance Learning, 3(8), 15–54.Farid, S., Ahmad, R., & Alam, M. (2015). A hierarchical

model for e-learning implementation challenges using

AHP. Malaysian Journal of Computer Science, 28(3), 166–

188.Gerdes, A., Jeuring, J., & Heeren, B. (2012). An interactive

functional programming tutor. In Proceedings of the 17th

ACM Annual Conference on Innovation and Technology in

Computer Science Education (pp. 250–255). New York,

NY, USA: ACM.Gertner, A. S., & VanLehn, K. (2000). Andes: A coached

problem solving environment for physics. In International

Conference on Intelligent Tutoring Systems (pp. 133–142).

Berlin, Heidelberg: Springer.Ghauth, K. I., & Abdullah, N. A. (2010). An empirical eva-

luation of learner performance in e-learning recommender

systems and an adaptive hypermedia system. Malaysian

Journal of Computer Science, 23, 141–152.Gouli, E., Gogoulou, A., Papanikolaou, K. A., & Grigoriadou,

M. (2006). An adaptive feedback framework to support

reflection, guiding and tutoring. In Advances in web-based

education: Personalized learning environments (pp. 178–202).Hershey, PA, USA: Information Science Publishing.

Bimba et al. 231

Gross, S., Mokbel, B., Paassen, B., Hammer, B., & Pinkwart,

N. (2014). Example-based feedback provision using struc-

tured solution spaces. International Journal of Learning

Technology, 10, 9248–9280.Gross, S., & Pinkwart, N. (2015). Towards an integrative

learning environment for java programming. In 2015 IEEE

15th International Conference on Advanced Learning Tech-

nologies (pp. 24–28).Hualien, China: IEEE.Hattie, J., & Gan, M. (2011). Instruction based on feedback.

In Handbook of research on learning and instruction (pp.

249–271). New York: Routledge.Heift, T. (2010). Developing an intelligent language tutor.

CALICO Journal, 27, 443–459.Heift, T. (2016). Web delivery of adaptive and interactive lan-

guage tutoring: Revisited. International Journal of Artificial

Intelligence in Education, 26, 489–503.Heift, T., & McFetridge, P. (1999). Exploiting the student

model to emphasize language teaching pedagogy in nat-

ural language processing. In Proceedings of a Sympo-

sium on Computer Mediated Language Assessment and

Evaluation in Natural Language Processing (pp. 55–61).

Stroudsburg, PA, USA: Association for Computational

Linguistics.Heift, T., & Nicholson, D. (2001). Web delivery of adaptive

and interactive language tutoring. International Journal of

Artificial Intelligence in Education, 12, 310–325.Heift, T., & Schulze, M. (2007). Errors and intelligence in com-

puter-assisted language learning: Parsers and pedagogues.

New York, USA: Routledge.Hepplestone, S., Holden, G., Irwin, B., Parkin, H., & Thorpe,

L. P. (2011). Using technology to encourage student

engagement with feedback: A literature review. Research in

Learning Technology, 19, 117–127. doi:10.1080/21567069.

2011.586677Le, N.-T. (2016). A classification of adaptive feedback in edu-

cational systems for programming. Systems, 4(2), 22. doi:

10.3390/systems4020022Lo, J. J., Chan, Y. C., & Yeh, S. W. (2012). Designing an

adaptive web-based learning system based on students’

cognitive styles identified online. Computers & Education,

58, 209–222. Retrieved from doi:10.1016/j.compedu.2011.

08.018Luckin, R., & Holmes, W. (2016). Intelligence unleashed: An

argument for AI in education. London: Pearson.Mason, B. J., & Bruning, R. (2001). Providing feedback in com-

puter-based instruction: What the research tells us (Tech.

Rep.). University of Nebraska. Retrieved from https://

www.researchgate.net/publication/247291218_Providing_

Feedback_in_Computer-based_Instruction_What_the_

Research_Tells_UsMathan, S. A., & Koedinger, K. R. (2005). Fostering the

intelligent novice: Learning from errors with metacogni-

tive tutoring. Educational Psychologist, 40, 257–265.Melis, E., Goguadze, G., Libbrecht, P., & Ullrich, C. (2009).

Activemath—A learning platform with semantic web fea-

tures. The Future of Learning, 159–177.Melis, E., Moormann, M., Ullrich, C., Goguadze, G., & Lib-

brecht, P. (2007). How activemath supports moderate con-

structivist mathematics teaching. In 8th International

Conference on Technology in Mathematics Teaching, Hra-

dec Kralove.

Mitrovic, A. (2003). An intelligent SQL tutor on the web.

International Journal of Artificial Intelligence in Education,

13, 173–197.Mitrovic, A., & Ohlsson, S. (1999). Evaluation of a

constraint-based tutor for a database language. Interna-

tional Journal of Artificial Intelligence in Education, 10,

238–256.Mitrovic, A., Ohlsson, S., & Barrow, D. K. (2013). The effect

of positive feedback in a constraint-based intelligent tutor-

ing system. Computers & Education, 60, 264–272.Moundridou, M., & Virvou, M. (2002). WEAR: A web-based

authoring tool for building intelligent tutoring systems. In

Proceedings of the 2nd Helenic Conference on AI, Compa-

nion Volume (pp. 203–214). Thessaloniki, Greece.Munoz-Merino, P. J., Pardo, A., Scheffel, M., Niemann, K.,

Wolpers, M., Leony, D., & Delgado Kloos, C. (2011). An

ontological framework for adaptive feedback to support

students while programming. In Proceedings of the 10th

International Semantic Web Conference (pp. 1–4). Bonn,

Germany.Narciss, S. (2013). Designing and evaluating tutoring feed-

back strategies for digital learning. Digital Education

Review, 23, 7–26.Narciss, S., & Huth, K. (2002). How to design informative

tutoring feedback for multimedia learning. In Instructional

design for multimedia learning (pp. 181–195). Retrieved from

http://www.mendeley.com/catalog/design-informative-

tutoring-feedback-multimedia-learning/Narciss, S., Sosnovsky, S., Schnaubert, L., Andres, E., Eichel-

mann, A., Goguadze, G., & Melis, E. (2014). Exploring

feedback and student characteristics relevant for persona-

lizing feedback strategies. Computers & Education, 71,

56–76. doi:10.1016/j.compedu.2013.09.011Nye, B. D., Graesser, A. C., & Hu, X. (2014). Autotutor and

family: A review of 17 years of natural language tutoring.

International Journal of Artificial Intelligence in Education,

24, 427–469.Olney, A. M., D’Mello, S., Person, N., Cade, W., Hays, P.,

Williams, C., . . . Graesser, A. (2012). Guru: A computer

tutor that models expert human tutors. In Intelligent tutor-

ing systems (pp. 256–261). Berlin, Heidelberg: Springer.Olney, A. M., Person, N. K., & Graesser, A. C. (2011). Guru:

Designing. In Cross-disciplinary advances in applied natural

language processing: Issues and approaches (p. 156) Her-

shey, PA, USA: IGI Global.Papanikolaou, K. A., Grigoriadou, M., Kornilakis, H., &

Magoulas, G. D. (2003). Personalizing the interaction in a

web-based educational hypermedia system: The case of

INSPIRE. User Modeling and User-Adapted Interaction,

13, 213–267.Prince, M. J., & Felder, R. M. (2006). Inductive teaching and

learning methods: Definitions, comparisons, and research

bases. Journal of Engineering Education, 95, 123–138. doi:

10.1002/j.2168-9830.2006.tb00884.xPryor, J., & Crossouard, B. (2008). A socio-cultural theorisa-

tion of formative assessment. Oxford Review of Education,

34(1), 1–20.Rivers, K., & Koedinger, K. R. (2015). Data-driven hint gen-

eration in vast solution spaces: A self-improving python

programming tutor. International Journal of Artificial

Intelligence in Education, 27, 37–64.


https://www.researchgate.net/publication/247291218_Providing_Feedback_in_Computer-based_Instruction_What_the_Research_Tells_Us

Rus, V., Conley, M., & Graesser, A. (2014). The dendrogrammodel of instruction: On instructional strategies and theirimplementation in DeepTutor. Design Recommendations

for Intelligent Tutoring Systems, 311–325.Rus, V., Niraula, N. B., & Banjade, R. (2015). Deeptutor: An

effective, online intelligent tutoring system that promotesdeep learning. In AAAI (pp. 4294–4295). Palo Alto, Cali-fornia: AAAI Press.

Scheuer, O., Loll, F., Pinkwart, N., & McLaren, B. M.(2010). Computer-supported argumentation: A review ofthe state of the art. International Journal of Computer-Sup-

ported Collaborative Learning, 5, 43–102. doi:10.1007/s11412-009-9080-x

Specht, M., Kravcik, M., Klemke, R., Pesin, L., & Hutten-hain, R. (2002). Adaptive learning environment for teach-ing and learning in winds. In International conference on

adaptive hypermedia and adaptive web-based systems (pp.

572–575). Berlin, Heidelberg: Springer-Verlag.Specht, M. E. (1998). Adaptive methods in computer-based

teaching/learning systems. New York, NY: ACM.Stoyanov, S., & Kirchner, P. (2004). Expert concept mapping

method for defining the characteristics of adaptive

E-learning: ALFANET project case. Educational Technol-

ogy Research & Development, 52, 41–54.Tsovaltzi, D., & Fiedler, A. (2003). Enhancement and use of a

mathematical ontology in a tutorial dialog system. In Pro-

ceedings of the IJCAI Workshop on Knowledge Representa-

tion and Automated Reasoning for E-learning Systems (pp.

23–35). Acapulco, Mexico.VanLehn, K., Lynch, C., Schulze, K., Shapiro, J. A., Shelby,

R., Taylor, L., . . . Wintersgill, M. (2005). The Andes phy-

sics tutoring system: Lessons learned. International Journal

of Artificial Intelligence in Education, 15, 147–204.Virvou, M., & Moundridou, M. (2000). Modelling the

instructor in a web-based authoring tool for algebrarelated

ITSs. In International Conference on Intelligent Tutoring

Systems (pp. 635–644). Berlin, Heidelberg: Springer-

Verlag.Virvou, M., & Moundridou, M. (2001). Student and instruc-

tor models: Two kinds of user model and their interaction

in an its authoring tool. In International Conference on

User Modeling (pp. 158–167). Berlin, Heidelberg: Springer-

Verlag.

About the Authors

Andrew Thomas Bimba received a BEng in Electrical and Electronics Engineering in 2006 anda Master’s degree in Computer Science (Artificial Intelligence) in 2014. He is currently a PhDstudent in Computer Science at University of Malaya. His research interest includes cognitiveknowledge base, natural language processing, artificial intelligence in education, machine learn-ing, and computer–human interaction.

Norisma Idris received a BSc degree, Master’s degree, and PhD in Computer Science at theUniversity of Malaya. Her area of interest includes artificial intelligence in education (summari-zation, summary sentence decomposition, heuristic rules, understanding and categorization,essay grading system) and natural language (Malay text processing, text normalization, stem-ming algorithm). She is currently the head of Artificial Intelligence Department at the Universityof Malaya.

Ahmed Al-Hunaiyyan received a BS in BA in 1983 at Kuwait University, an MS degree in MISfrom Aurora University, Illinois, USA, in 1988, and a PhD in Computer Science atHertfordshire University, UK, in 2000. He has lecturing and training experience in multimediaapplications and authoring, database systems and application, management information systems,programming languages, just to mention a few. His research interest includes web-based tutors,multimedia applications, e-learning, human–computer interaction, and knowledge base. He iscurrently an assistant professor in Computer and Information Systems Department, PublicAuthority for Applied Education and Training, Kuwait.

Rohana Binti Mahmud is a senior lecturer at the Department of Artificial Intelligence,University of Malaya. She received a BSc degree at Waikato University, New Zealand, an MScdegree at Universiti Sains Malaysia, and a PhD at University of Manchester, UK. Her area ofexpertise includes natural language (Discourse Structure, Lexical Relation, Malay LanguageText Processing), expert system (Knowledge Base System, Multi-agent Expert System, ExpertTutoring System), and education (AI in Education, Soft skills, Higher Order Thinking Skills).

Bimba et al. 233

Nor Liyana Bt Mohd Shuib is a lecturer at Department of Information System in the Facultyof Computer Science & Information Technology, University of Malaya. She graduated withBCS (Information System) in 2005 from Universiti Teknologi Malaysia and MIT (InformationTechnology) from Universiti Kebangsaan Malaysia, in 2007. She obtained her PhD in 2013 fromUniversiti of Malaya, Malaysia. Her research interests include data mining, education technol-ogy, and information retrieval tools.


adaptive feedback in computer-based learning environments: a … · adaptive learning environments...

Documents