modelo de análisis difuso para clasificar preguntas de ...palabras clave: verbos de taxonomía de...

REVISTA AUS 26.4/ Fouad Jameel Ibrahim AlAzzawi et al.,/ DOI:10.33329/aus.2019.n26.4.10/ www.ausrevista.com/ [email protected]

69

AR

TÍC

UL

O

ABSTRACT/ In this work, a new fuzzy classification algorithm has been developed and evaluated to be used in learning quality management system

to classify exam questions based on Bloom’s taxonomy strategy. An experimental evaluation test has been implemented considering several

classification algorithms, trained and tested on a dataset that contains exam questions extracted from the Moodle system that belongs to private

institutions in the Sultanate of Oman. The proposed fuzzy algorithm has been evaluated with the dominant classification algorithms based on

machine learning models. The obtained results show that Meta classifier ‘Bagging’ outperforms all classification algorithms available in machine

learning with insignificant confidence of 88.7% classification of correct instances, while the developed fuzzy algorithm could achieve significant

confidence of 96.2% classification of correct instances. The proposed fuzzy algorithm outperforms the Bagging algorithm with 7.5% improvement,

in term of linking exam questions to the correct Bloom’s verb categories. The outcome of this work is a Smart Bloom’s Analyzer capable of

providing smart recommendations that possibly improve the assessment method in higher education institutions, a target that comes in-lined with

the framework of Oman Academic Accreditation Authority (OAAA) and learning quality management system in the Sultanate of Oman. Keywords:

Bloom’s Taxonomy verbs, Fuzzy Analysis model, Classification, Machine Learning.RESUMEN/ En este trabajo, se ha desarrollado y evaluado

un nuevo algoritmo de clasificación difusa para ser utilizado en el sistema de gestión de la calidad del aprendizaje para clasificar

las preguntas del examen según la estrategia de taxonomía de Bloom. Se implementó una prueba de evaluación experimental

considerando varios algoritmos de clasificación, entrenados y probados en un conjunto de datos que contiene preguntas de examen

extraídas del sistema Moodle que pertenece a instituciones privadas en el Sultanato de Omán. El algoritmo difuso propuesto se

ha evaluado con los algoritmos de clasificación dominantes basados en modelos de aprendizaje automático. Los resultados

obtenidos muestran que el clasificador Meta "Empaquetamiento" supera a todos los algoritmos de clasificación disponibles en el

aprendizaje automático con una confianza insignificante de 88.7% de clasificación de instancias correctas, mientras que el

algoritmo difuso desarrollado podría lograr una confianza significativa de 96.2% de clasificación de instancias correctas. El

algoritmo difuso propuesto supera al algoritmo de embolsado con una mejora del 7.5%, en términos de vincular las preguntas del

examen con las categorías verbales correctas de Bloom. El resultado de este trabajo es un Smart Bloom's Analyzer capaz de

proporcionar recomendaciones inteligentes que posiblemente mejoren el método de evaluación en las instituciones de educación

superior, un objetivo que se alinea con el marco de la Autoridad de Acreditación Académica de Omán (OAAA) y el sistema de

gestión de la calidad del aprendizaje en El Sultanato de Omán. Palabras clave: verbos de taxonomía de Bloom, modelo de análisis

difuso, clasificación, aprendizaje automático

1. INTRODUCTION

Learning at university level presents great

challenge to educators when it comes to

construct course learning objectives with

related exam questions that reflect learning

outcomes. Research and development in

educational studies have long been addressed

education questions and produce ideas,

1*Fouad Jameel Ibrahim AlAzzawi, 2Boumedyen Shannaq 1Al-Rafidain University College, Baghdad, IRAQ, 2University of Buraimi, Al-Buraimi, Sultanate of Oman, [email protected] [email protected]

Fuzzy Analysis Model for Classifying Exams

Questions in Learning Quality Management

System Based on Bloom’s Taxonomy Verbs

Modelo de análisis difuso para clasificar preguntas

de exámenes en el sistema de gestión de calidad de

aprendizaje basado en los verbos de taxonomía de

Bloom

Publicación /28-08-2019

http://www.ausrevista.com/


70

AR

TÍC

UL

O

methods and systems to manage the learning

process. Rague (2005) stated that the quality

of educational process is in the center of

interest for education managers and

educators. In this regard, the quality of

education can be defined as balanced

correspondence properties and characteristics

of the educational process. In the Sultanate of

Oman, the Government control and

supervision of the quality system is aimed at

ensuring a cohesive national policy for

improving the quality of management,

preparation, and the rational use of national

funds allocated to finance the education

system (OAAA, 2017). National control and

supervision of the quality in education is work

out by national education authorities (OAAA,

2018). Such control and supervision could

mainly require the institution itself to establish

an innovative quality management system that

responds to the requirements of the national

authority (Shannaq, 2018). The goal of this

work is to develop a new classification

algorithm that classifies exam questions

according to Bloom’s Taxonomy verbs using

updated fuzzy model.

To achieve this goal, the following steps were

followed:

1. Study and review the current classification

work in machine learning.

2. Build an effective machine learning model

based on experimental test to find the

dominant classification algorithms available in

the knowledge flow environment in Weka

tools.

3. Propose and realize a new classification

algorithm using fuzzy search method.

4. Conduct experiments to test the algorithm

with the best model obtained in step 2.

Figure 1 illustrates the task of the Quality

Management System (QMS) in Higher

Education Institutions (HEI) to audit the exam

questions based on Bloom’s Taxonomy verb

categories, where ‘?’ indicates the question

matching the Bloom’s verb?

Figure 1 task of QMS in HE to audit the exam

questions

One of the important features of evaluation

the quality of exam questions to fulfill the

requirements of QMS in HEI is to control the

presence of intentionally or unintentionally the

repeated fragments of exam questions that

had better match to Bloom’s verb categories,

which greatly complicates the preparation and

the evaluation process of the exam questions

and becomes very difficult and time consuming

for educators in the absence of additional tools

for the development and maintenance of the

exam questions. Thus, if the QMS doesn’t track

the presence of Bloom’s verb repetitions in

exam questions. Therefore, the exam

questions could be insufficient to improve the

learning outcomes, which may eventually

affect the quality of education. Therefore, the

simplification and partial automation of the

process of locating and refactoring such

repetitions becomes an important task. The

exact text matching using machine learning

and other approaches have been applied to

Bloom’s verbs, so when a Bloom’s verb

appears in the question, the classifier matches

such question to the exact Bloom’s category as

illustrated in figure 1. The proposed fuzzy

algorithm in this work could be consider as an

innovative tool to improve the application of

QMS in HEI. The proposed fuzzy algorithm is

distinguished from other approaches in

considering number of parameters such as the

number of the clone Bloom’s categories,

average number of clones and the average

number of related keywords in a clone for all

Bloom’s categories. In this regard, this work is

prepared to promote an innovative approach

to improve the quality of education through the

application of technology and intelligent

solutions, creating opportunities for enhancing

auditing and traceability.

2. Background and literature review

2.1 The development of Domestic Quality

Management Systems

In recent years, interest in portfolio system

has been noted in Oman’s education. (OAAA,

2017). Sultanate of Oman Quality

Management System (QMS) community

represented by OAAA, have developed a set of

comprehensive documents under the title

“National Education Frameworks” (OAAA,

2018) to be oriented towards the highest level

of quality in education ever achieved in the

region, and the highest standards of

requirements for financial, logistical,

intellectual and information technology



71

AR

TÍC

UL

O

support for the functioning of the educational

system per scholar. OAAA in this regard

developed nine standard indicators to ensure

the quality of education at the registered local

academic institutions. This work is proposed to

improve the quality control of Standard

number 2, namely “Student Learning by

Coursework Programs” and in particular

Criteria number 2.8, namely “Assessment

Methods, Standards and Moderation”. Figure 2

proposed by OAAA are depicted herein to

demonstrate the focus of this work.

Figure 2 Nine standards set by OAAA

Essentially, the assessment of the standard

indicators highlighted in figure 2 should be

based on an analysis of the availability and

effectiveness of the quality assurance system

at the university level, which directly obliges

educational institutions to start creating such a

system. However, though the availability and

effectiveness of such indicators are not clearly

defined until now, which makes it difficult to

carry out an external auditing for the

certification and national accreditation of

academic organizations. Academic institutions

at the Sultanate of Oman are working

extremely to improve the quality of education

based on the standards established by OAAA

(Shannaq, 2018).

2.2 Bloom’s Taxonomy

Bloom’s Taxonomy involves the creation of an

integrated six levels of cognitive learning

objectives and learning assessment to classify

educational learning into levels of difficulty and

specificity. Six lists of verbs cover learning

objectives were proposed to control course

delivery and likewise are used to classify

exams to control their compatibility with the

learning objectives set for students according

to Bloom’s Taxonomy models (Scott, 2003).

The cognitive domain list has been the primary

focus of most world education institutions

including the Sultanate of Oman, and

frequently used to structure course learning

objectives, assessments and activities. Figure

3 shows the terms of question verbs and

samples of question contexts (Huitt, 2011;

QTLMS Unizwa 2017).

Figure 3 Bloom taxonomy verbs



72

AR

TÍC

UL

O

(Anwar, 2017; Siti et al., 2017; Nafa et al.,

2016) presented that planning an assessment

strategy based on Bloom’s Taxonomy, will help

in formulating exams and assignments

compatible with learning objectives set out for

the curriculum courses.

The work presented in this research paper is a

practical software application called Bloom

Analyzer capable of providing smart

recommendations based on Bloom’s

Taxonomy model to improve assessment

method.

3. Classification

Many research papers (Anwar, 2017; Siti et

al., 2017; Nafa et al., 2016) discussed

different possible approaches to the

construction of automatic classification system

for exam questions based on Bloom’s

Taxonomy verbs. Those studies were carried

out within the framework of processing and

classifying exam questions to control the

assessment and ensure their integrity and

compatibility with the six categories of Bloom’s

Taxonomy that were used to construct course

learning objectives. Classifiers are a support

mechanism for adding new experiences to

improve the quality of the educational process.

Ulum, (2016); Kocakaya and Kotluk,(2016);

Omar et al., (2012); Zhang and Lee, (2003)

Have used BayesNet ,J48 ,RandomForest

,NaiveBayes, RandomTree, Stacking, Bagging

and Vote algorithms to build classifiers and

predictors with the help of java API. The

implementation of those algorithms which

belong to the knowledge flow intelligence tools

have been customized to perform all

computational experiments.

3.1 Bloom’s Taxonomy verbs learning

The novelty of this approach is the idea of

converting a text classification task into

learning task with an automatic educator to

build an attribute description of each exam

question under consideration, which is a

Boolean vector of the occurrence of words

(there is a word or not) in a question from a

pre-built exam questions bank extracted from

Moodle used in the institution of higher

education. The task of learning is applied to a

set of input objects (questions) X, where each

object x ∈ X is assigned a value y, called the

output, or answer, belonging to the set of valid

answers Y. Ordered pair “question-answer” (x,

y) where x ∈ X, y ∈ Y is called a precedent. The

relationship between input and output based

on the data in the final set of precedents is

called training sample (Gareth et al., 2018;

Shannaq and Adebiaye 2015; Yusof and Hui, 2010). {(𝑥𝑖 , 𝑦𝑖) | 𝑥𝑖 ∈ 𝑋 , 𝑦𝑖 ∈ 𝑌 , 𝑖 = 1, 𝑁̅̅ ̅̅ ̅}. In

other words, the task is to build a model

(function) f, which having received x as an

input, would predict the value of the answer y.

The process of finding f is called learning or

setting up the model. The main requirement

for a solution is a high generalizing ability, that

is, a trained model must produce accurate

predictions on new (not included in the training

set) precedents. Thus, the optimal solution of

the problem of inductive learning should

satisfy the following conditions:

f * = arg min𝐹 ∈𝐾

∑ 𝐿 (𝑦𝑖 , 𝐹(𝑥𝑖𝑁𝑖=1 ))

optimal solution for inductive learning

L (y , f (x))

non-negative function of loss (penalty)

K

a set of models (𝑥𝑖 , 𝑦𝑖) , i = 1 , 𝑁̅̅ ̅̅ ̅̅ ̅

precedents make up a training set

4. Experiments

An experiment was conducted on exam

questions database, extracted from the Moodle

system. This database included multiple types

of exam questions of information systems

courses, each question marked and classified

based on the six

Bloom’s Taxonomy Action Verbs’ on the

following categories: "Remembering",

"understanding", "Applying", "Analyzing",

"Evaluating", "Creating", (altogether 6

categories are considered).

This work builds a list of keywords based on

Bloom’s Taxonomy verbs and their synonym,

after which Visual C# code have been

developed to read all exam questions and

classify exam questions based on Bloom’s

Taxonomy verbs considering their synonym.

For example:

Exam question: “What is Web-based

multimedia and how it is used today” the C#

code classify this question as a “Remember”.

The same is done for other exam questions.



73

AR

TÍC

UL

O

Figure 4 demonstrates the preprocessing steps for the given questions and classes.

Data 6 classes distribution 6 classes distribution after

filtering

Figure 4 Data set and class preprocessing

To solve the class imbalance problem, this

work used Weka filtering tools to undergo

unsupervised resampling. The 10 folds

distribution have been used into training and

test samples (with the number of objects equal

to 7000 and 1756 respectively). To improve

the classification performance and to

guarantee that all 6 classes will appears in all

folds, Weka supervised filtering tools have

been applied.

Experiment 1

Figure 4.1 demonstrates the knowledge flow

environment to implement a performance

comparison for common selected classifiers.

Figure 4.1 Experiment configuration

Figure 4.1 shows work load dataset, classes

assignment, class fold set, cross validation

fold, training and test groups for 8 classifiers

and link each classifier with performance

evaluator and implement the model for

visualizing the performance chart based on

percent correct classifications. Figure 4.2

shows the plot of ROC curve for the above

setting presented in Figure 4.1.



74

AR

TÍC

UL

O

Figure 4.2 ROC curve compression over 8 classifiers

The Paired T-Tester have been employed to

analyze the experiments. The Roc Curve

presented in figure 4.3 used to perform the

performance comparison over all classifiers

and the percent of correct test have been

selected with significance of 0.05, i.e. any

difference found among the generated

classifiers will be 95% of confidence interval

and figure 4.4 demonstrate details of the

comparison

Figure 4.3 percent_correct indicator among

all classifiers

To assess the quality of the generated

classifications, for each model, the values of

accuracy percent_correct, calculated as it is

shown in figure 4.3. In some cases, the

accuracy values have been determined. It

should be noted that Bagging classifier

performs all other classifier with little

confidence while BayesNet, J48,

RandomForest, NaiveBayes, RandomTree

algorithms receptively are similar

performance. The use of Stacking, and Vote

give the worst results in all aspects. Figure 4.3

shows a test percent_correct total for all

classes (the percentage of correctly classified

exam questions to the total test sample

objects), as well as the average and standard

deviation categorization of each classifier.

From the obtained results it can be seen that

the training of models representing ensembles

of Bagging, BayesNet, J48, RandomForest,

NaiveBayes, RandomTree algorithms

receptively, are significantly exceeds the

Stacking, and Vote, however, Bagging gives

the best indicators for the percent_correct

classification. Thus, the RandomForest

algorithm of decision trees should be also

marked in solving the text classification

problem proved to be a serious competitor to

the meta classifier, i.e. Bagging traditionally

used in problems of this kind, making it

possible to increase the quality of

classification, especially for small categories of

exam questions (Kotsiantis, Tsekouras, &

Pintelas, 2018; Esposito, & Saitta, 2005).

Experiment 2

In his section we propose and realize a new

algorithm using fuzzy search methods. In fact,

comparing the received text with itself, cold

lead to inaccurate repeating fragments using

search algorithms. Initially, the size of the

clones is unknown and could be changeable, as

a result of which a straightforward

implementation of this approach would have

required comparing all fragments of the text to

all sizes. Comparing each other with all

fragments of the same size, there is

complexity of O (n2 / t2), where n is the

number of words in the text (exam questions

in this work), t is the size of the fragment in

terms and quadratic. Let us calculate the

general complexity of the algorithm without

considering the cost of comparing two

fragments of the text between itself: ∑nt = 1 n2

/ t2 = n2 ∗ (1 + 1/2 + ... + 1 / n) 2 = O (n2).

Therefore, regardless of the choice of an

algorithm for comparing these fragments, such

an approach would be extremely time

consuming and unproductive. Thus, in this



75

AR

TÍC

UL

O

work it was decided to build an alphabet from

the text (exam questions), the symbols of

which will be the words found in the text, and

then store the text in the symbols of the new

alphabet. This conversion provides several

advantages: reducing the amount of memory

needed to store text, as well as accelerating

the further operation of the algorithm with

text, since it is possible to work not with string

values of words, but with their position in the

created alphabet, which will allow us to

compare fragments faster. After obtaining the

results of the fuzzy search algorithm, it will be

enough to use the alphabet to return to the

textual representation. However, this

conversion also does not give a significant

increase in the performance of the proposed

algorithm. Therefore, it was decided to change

the way how the fragments were compared,

figure 4.4 shows the steps of the proposed

algorithms.

Figure 4.4 steps of the proposed algorithms

The first step of the proposed algorithm is to

make assumption about the minimum clone

size of interest to the user/auditor. Since we

are talking about working with large exam

questions, we can assume that fuzzy

repetitions of a few words are unlikely to be

the main object of search and refactoring, and

can be easily found by exact search, therefore,

we set a limit on the minimum size of a

repeating fragment. Now the idea of

comparing the text with itself can be modified

using the previous statement. The next step of

the algorithm is to split the text into fragments

of a given size. Then there will be a clear

comparison with each other. Since in this case,

the fragment size is already fixed, a significant

part of the comparisons is discarded. However,

fuzzy clones can have a much larger size than

one such fragment, as a result of which one of

the subsequent stages of the algorithm will be

the expansion of the found clones. With this

approach, the complexity of the algorithm will

be O (n2 / t2), where n is the number of

keywords in the text, t is the minimum clone

size.

The next important step of the algorithm is to

use the hashing to speed up the comparison of

fragments. But, since it is required to search

for inaccurate matches between fragments, it

is necessary to use perceptual hashing, which

will produce similar values on similar

fragments, and will allow us to discard

obviously dissimilar fragments. Rochimah et

al., (2013) proposed a hash function called

Signature used to map each fragment to a

vector of size m. Each ith element of this vector

corresponds to a set of alphabet characters,

and if the fragment contains one of these

symbols, then the ith element will be equal to

one, and zero otherwise. However, in this case,

this use of the function is impractical, since

with significant fragment sizes, their

signatures will always be close to the values

even for completely different fragments.

Therefore, a signature hash function was

proposed, with the sets of values for the

elements of the vector in which there were

symbols of the documentation language, but

now the ith element of the vector will be equal

to one only if the character from the set is the

first symbol of a giving key word from the

words found in the fragment. This will save

memory costs at the same level, without losing

accuracy and the value of the hash function of

the fragment will be the number of H (w) =

∑m − 1i = 0 2i ∗ sign(w)i + 1, where w is a

fragment, sign (w) is a vector signature of size

m by the first characters of words in the

fragment. Figure 4.5 illustrates the signature

of the hash function used for English letters;

the most suitable element was found in this

work is when m = 9.



76

AR

TÍC

UL

O

Figure 4.5 Signature of the hash function

Figure 4.6 shows a diagram for quickly

comparing the values of the hash function of

different fragments.

Figure 4.6 Hashing comparisons of fragments

It was decided to take advantage of the fact

that initially the value was represented as a

vector of zeros and ones. The values of the

hash function of the two fragments are

translated into binary representation, and the

operation of the exclusive ‘OR’ is bitwise used.

In the bitwise representation of the number

obtained by this conversion, the number of

unit bits will denote the difference between the

values of the hash functions of the two

fragments. If this value does not exceed a

certain threshold value specified at the start of

the algorithm, then the second comparison

step is performed.

Figure 4.7 shows the comparison results for

the proposed algorithm and meta classifier

Bagging.

Figure 4.7 comparison results

To verify the correct operation of the

algorithm, as well as to find its limitations,

several sets of test data were generated. A set

of data for checking the correctness of the

algorithm, consisting of several small and large

texts exam questions. Validation Data Test

data consists of several small pieces of text

containing repeating elements. Carrying out

these tests is designed to demonstrate the

operation of the new algorithm on various

types of clones. Figure 4.7 illustrate the

percent of correct classification when the

algorithm is compared to the bagging

classifier.

5. Implementation

The fuzzy algorithm has been used and

customized to build the Smart Information

System and successfully categorizes exam

questions with 96.2% confidence. Figure 5

below shows a sample of the manual work and

figure 5.1 shows the output of the developed

system.

Figure 5.1 sample of the manual work (course specification)



77

AR

TÍC

UL

O

Figure 5.1 Sample of the developed Bloom Analyzer

The Implementation Features of the proposed

algorithm implies the ability to configure and

change some parameters by the user. The user

could configure such parameters as the size of

the fragments into which the text is split, the

maximum permissible editorial distance

between two fragments, and also the proximity

threshold of two hash values for different



78

AR

TÍC

UL

O

fragments. Also, the implementation of the

algorithm provides for the possibility of

parallelization at several stages of the

algorithm: normalizing the text, calculating

hashes of fragments, comparing fragments of

text and expanding them, which will further

accelerate the operation of the algorithm.

Conclusion

This work successfully developed fuzzy

classifier algorithm which is used as catalog for

constructing course exams. The observed

results from the developed Bloom’s Analyzer

suggest the possibility of using the considered

method for the construction of automatic

catalogs in e-learning libraries. We believe

that, Bloom’s Analyzer application will serve

educators to build effective exam questions

and comply with the requirement of QMS.

Analysis of the obtain results to prove the

accuracy of the proposed algorithm have been

conducted on several key tests were carried

out, the successful completion of which means

that the algorithm allows us to find accurate,

so fuzzy repetitions of varying degrees of

variability, and allows us to overcome the

limitations of the machine learning algorithm

to perform the classification task.

References

[1]. Anwar, A. Y. (2012) ‘Bloom’s Taxonomy

Cognitive Levels Data Set’, 25th

International Conference on

Industrial, Engineering & Other

Applications of Applied Intelligent

Systems, Dalian, Liaoning, China.

[2]. Esposito, R. and Saitta, L. (2005)

‘Experimental comparison between

bagging and Monte Carlo ensemble

classification’, Proceedings of the 22nd

international conference on Machine

learning – ICML.

[3]. Gareth, J. Witten, D. Hastie, T. and

Tibshirani, R. (2018) ‘An introduction to

Statistical learning: With applications in

R’ (Springer Texts in Statistics Book 103)

1st ed. 2013, Corr. 7th printing 2017

Edition, Kindle Edition.

[4]. Huitt, W. (2011) ‘Bloom’s et al.

taxonomy of the cognitive domain’,

Educational Psychology Interactive,

Valdosta, GA: Valdosta State University

http://www.edpsycinteractive.org/topics

/cognition/bloom.html.

[5]. Kocakaya, S. and Kotluk, N. (2016)

‘Classifying the Standards Via Revised

Bloom's Taxonomy. A Comparison of Pre-

Service and In- Service Teachers’,

International Journal Of Environmental &

Science Education, 11(18), 11297-

11318.

[6]. Kotsiantis, S. B. Tsekouras, G. E. and

Pintelas P.E. (2018) ‘Bagging Model

Trees for Classification Problems’, LNCS

3746, pp.328-337, Springer-Verlag

Berlin Heidelberg.

[7]. Nafa, F. Othman, S. and Khan, J. (2016)

‘Automatic Concepts Classifications

based on Bloom Taxonomy using Text

Analysis and Naïve Bayes Classifier

Method. CSEDU 8th international

Conference on Computer Supported

Education, 391-396

[8]. OAAA (2017),

http://www.aiaccredits.org.

[9]. Oman Academic Accreditation Authority

(2018), www.oaaa.gov.om.

[10]. Omar, N. Haris, S.S. Hassan, R. Arshad,

H. Rahmat, M. Zainal, N. F. A. and

Zulkifli, R. (2012) ‘Automated analysis of

exam questions according to Bloom's

taxonomy’, Procedia-Social and

Behavioral Sciences, 59(6), 297-303.

[11]. Rochimah S. Dewandono R. D. and

Saputra F. A. (2013) ‘Clone detection

using Rabin-Karp parallel algorithm’ 7th

International Conference on Information

& Communication Technology, Bali.

[12]. Shannaq B. (2018) ‘Smart Information

System for Evaluation the Effectiveness

of the Educational Organization to

succeed the modern university in

entrepreneurial university’, Open Arab

Conference, Higher Education

Development and Quality.

[13]. Shannaq, B. and Adebiaye R.

(2015)‘Analytic-Synthetic Processing of

Information As Smart-Based

Environment for Text Summarization’,

International Journal of Innovative

Research in Science, Engineering and

Technology (An ISO 3297: 2007 Certified

Organization) 4(1).

[14]. Scott T. (2003) ‘BLOOM'S TAXONOMY

APPLIED TO TESTING IN COMPUTER

SCIENCE CLASSES’, Journal of

Computing Sciences in Colleges, 19(1),

267-274.

[15]. Siti, H. Salmah, F. Rina, S. and Mazlina,

M. (2017) ‘Mining exam question based

on Bloom Taxonomy’, SiteSeerx,

http://citeseerx.ist.psu.edu/viewdoc/su

mmary?doi=10.1.1.403.3743.



79

AR

TÍC

UL

O

[16]. Tague, N. R. (2005) ‘The Quality

ToolBox, American Society for Quality’,

Quality Press, 2nd edition.

[17]. QTLMS UNIZWA (2017). Quality

Teaching And Learning Management

System (Qtlms): A Developmental

Student-Centered Pedagogical

Framework. Guideline Book

[18]. Ulum, O. G. (2016) ‘A Descriptive

Content Analysis of the Extent of Bloom'

s Taxonomy in the Reading

Comprehension Questions of the Course

Book Q: Skills for Success 4 Reading and

Writing’, The Qualitative Report 21(9),

1674-1683.

https://nsuworks.nova.edu/tqr/vol21/is

s9/7.

[19]. Yusof, N. and Hui, C. J., (2010)

‘Determination of Bloom's cognitive level

of question items using artificial neural

network’, Proceedings of the 10th

International Conference on Intelligent

Systems Design and Applications (ISDA).

Cairo, Egypt, 866-870.

[20]. Zhang, D. and Lee, W.S. (2003)

‘Question classification using support

vector machines’, Proceedings of the

26th Annual International ACM SIGIR

Conference on Research and

Development in Information Retrieval.

Toronto, Canada, 26-32.


modelo de análisis difuso para clasificar preguntas de ...palabras clave: verbos de taxonomía de...

Documents