11

PhD. – Information SystemThesis Viva

By: Sharif Omar SalemSupervisor: Dr. Khaironi Yatim SharifCo-Supervisor: Dr. Ilham Sentosa

The design and format is done by me, feel free to use the same format. But I am expecting appreciation notification.

2

Research Title

Developing a Hybrid Success Model for Different Content Management Systems in Higher

Education:

A Comparative Analysis of Students’ Perspective on Traditional and SNS systems.

Contents

Subtitle Slide No.

Introduction 4

Theoretical Framework and Proposed Model 12

Research Methodology 18

Findings 30

Hypothesis Discussion 36

Contributions, and Recommendation 41

Publications 46

3

Introduction

4

Research Brief

This academic research Investigated the learners’ outcome and its determinants via experiencing two different treatments.

First treatment by using traditional CMS and second treatment by using FB-based CMS.

A survey based on a developed hybrid eLearning success model is used to collect data.

Findings analyzed to assess the relations in the causal model and to compare the outcome constructs acquired via experiencing the two different systems.

5

Research Motivations

During the last five years; many researchers announced and recommend for further and future research to fill up two gaps:

6

Theoretical gap: The need for a

revised e-Learning system success model

Practical gap: The need for

more understanding of the FB effect

in learning outcome.

7

Theoretical gap: The need for a revised e-Learning system success model

• “A goal of continuing research would be an exploration of how the ISS model would be supplemented in order to more accurately reflect the E-learning environment”.

In 2010; Freeze, Alshare, Lane, & Wen state that

• Factors from the community of inquiry frameworks such as metacognitive, motivational, and behavioral traits of active online students may be a valuable add-on to the eLearning success model

In 2010; Shea & Bidjerano state that

• Considering the perspective of all players of the eLearning system and including additional different factors is important for a proper representation of the system success

In 2012; Bhuasiri et al. state that

• System success dimensions are not technology only, the revolution of web 2.0 and the uniqueness of eLeaning environment especially the different stakeholders guide the researchers to seek for new revised model

Cheng 2012; Lee et al. 2009; Chen 2010; Keramati et al. 2011; Sun et al. 2008; Hassanzadeh et al. 2012; Wang & Chiu 2011

8

Practical gap: The affect of using Facebook in learning outcome.

• Consolidating the Facebook in the learning and teaching process is very important for the students’ education lives and further research is needed to understand this phenomenon

In 2011; Bicen & Cavus

• Future research is needed to investigate more the usability of FB in education

In 2012; LaRue state that

• There is a need for understanding the relation between the learners’ interaction level in the Facebook and their academic success

In 2012; Junco stated that

• Considering the perspective of all players of the eLearning system and including additional different factors is important for a proper representation of the system success

In 2013; Ng & Wong stated that

Problem Statement:

Content Management System (CMS) is widely used in most of the universities worldwide to facilitate higher education stakeholders’ communications.

The challenge is whether Facebook environment “as a CMS system” is favorable and more effective than the traditional CMS system and what determinants/constructs affect the eLearning success.

Recently, few academic studies begin investigating this field of application. Further investigation is needed to fill up the illustrated gaps.

9

Research Objectives:To identify the dimensions of the information system success in the modern e-learning environment, and propose a modified model for e-learning system success.

To implement and test the proposed model when the implemented system is Moodle based.

To implement and test the proposed model when the implemented system is Facebook based.

To compare between the findings of the Moodle-based system analysis and the findings of the Facebook-based system analysis.

10

1

3

2

4

11

Scope of the Study

The research use Facebook as a social network system and Moodle as a traditional CMS system.

The research focus in the application of the system to facilitate higher education.

Respondents for the application of the system are from the LUCT University, Malaysia. Students are participants for the master degree level.

12

Theoretical Framework and Proposed Model

13

Learning System

OutcomeNet Benefits

Information Quality

Service Quality

Intention to Use/ System Use

User Satisfaction

System Quality

Theory 1: Delone and McLean IS success model - 2002

14

Learning System

Outcome

Learning System

Outcome

Teaching Presence

Social Presence

Cognitive Presence

Theory 2: The Community of Inquiry - 2000

15

Learning System Outcome

Community System Success

Sociability

Usability

(System Dimension)

Theory 3: Preece’s sociability and usability framework – 2001

16

Learner’s Outcome

System Sociability

Information Quality

Service Quality

Intention to (Use)

User Satisfaction

Teaching Presence

Learner’s Presence

System Quality

Proposed Hybrid Model for IS Success of Modern LMS

Learner’s Outcome

Learner Outcome

D&M IS Success ModelCoI Theory Preece’s S&U Framework

17

Learning System

Outcome

Learner Outcome

System Sociability

Information Quality

Service Quality

Intention to (Use)

User Satisfaction

Teaching Presence

Learner Presence

System Quality

Research Hypothesis

HA1; HB1HA2; HB2

HA4; HB4

HA3; HB3

HA5; HB5

HA6; HB6

HA7; HB7

HA8; HB8

HA9; HB9

HA12; HB12

HA10; HB10

HA11; HB11

HA15;HB15HA

14; H

B14

HA13

; HB1

3

HA17;HB17

HA18;HB18

HA16;HB16

HC1

HC2

HC3

HA## Hypothesis Set of System 1HB## Hypothesis Set of System 2HC## Hypothesis Set of Comparison

• IU of the FB-based system is different from and higher than the Moodle-based system.• US of the FB-based system is different from and higher than the Moodle-based system.• LO of the FB-based system is different from and higher than the Moodle-based system.

18

Research Methodology

19

Research Design• The study starts up with a theory and ends up with testing

the hypothesis. Deductive Approach

• Statistical analysis based on descriptive measures, variance, covariance techniques are used.Quantitative

Research

• Literature review and systematic reviews techniques are used to build the hybrid model.Qualitative

Research

• This study aims to assess a desired field subjects in two different treatments then compare the outcomes. Field Experiment

Design

• “Counterbalanced Measures Design” technique is performed by assigning participants in different groups and applying treatments to each group in a different order.

Counterbalanced Measure Design

20

Research Design

Scientific R

esearch

Approach

Instrument Tool (Survey)

A questionnaire in English was derived for the study with two major sections.

The first section is for the demographics data, such as age, sex, gender, country, major study, and semester level.

The second section consists of 43 questions covered the independent, mediating and dependent constructs.

The five-point Likert scale with pre-coded numerical scales is used in order to measure the extent of respondent’s view.

21

22

Population and Sample

The population is

all the Module-

based Master students of LUCT ~ 362.

The minimum sample size

for SmartPls = 70.

the effective sample size

based on the statistical

power value = 153.

In reality, the analysed

sample = 231.

Experimental Design (Counterbalanced Measures Design)

This research is Semi-Experimental design use

Field Experiment.

This approach is a mix between the “Between Subjects Design” and

“Repeated Measure Design”

23

Group Pretest TreatmentJan-Feb 2014 Test Treatment

Mar-Apr 2014 Posttest

Group A No Survey Moodle-Based Survey Facebook-Based Survey

Group B No Survey Facebook-Based Survey Moodle-Based Survey

24

Experimental Design (Treatment 1)

Top 5 Most Popular LMS software ranking

25

Experimental Design (Treatment 2)

Active Users of SNS Sites

Data Collection

Data collected for the two samples in

two different empirical

conditions.

Direct collected method is mainly

applied, but under certain conditions

online survey is applied.

Survey distribution and collection was

managed by the researcher and the module lecturer.

Student informed that the survey is

for academic purposes only and it is optional and

confidential.

26

Group TreatmentJan-Feb 2014 Test Treatment

Mar-Apr 2014 Posttest

Moodle Group A Survey Group B Survey

Facebook Group B Survey Group A Survey

Time

Analysis Tools

IBM SPSS v. 20 and SmartPLS 3 is used for statistical analysis.

Analysis Based in PLS-SEM (Variance based Sequential Equation Modeling)

If the research objective is prediction and theory development, then the appropriate

method is PLS SEM.‑27

Instrument Validity

Content Validity

• Experts Panel of five experts were asked to review and comment about the 1st draft survey items. “Measure what is intend to measure”; is the point of view for the feedback from the panel.

Face Validity

• A group of 10 students from different universities in Malaysia surveyed to figure out the goodness of the questionnaire (by observation and discussion)

28

29

Pilot Study - Reliability Test (Cronbach's Coefficient )

IQSQ

SrQTP

SPSS

SUUS

LO

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.74

4

0.72

4

0.74

6

0.72

5

0.70

1

0.75

9

0.72

4

0.83

8

0.73

6

0.80

7

0.74

5

0.75

9

0.74

7

0.72

6

0.74 0.

806

0.88

6

0.69

7

68 bachelor students of LUCT

University - Malaysia for feasibility

of FB based system

40 bachelor students of Palestine

University-Palestine for Moodle

based system

0.7

0.6

30

Findings

31

Data Screening

Registered Students Distributed

Survey Collected Cases Valid Cases

0

50

100

150

200

250

300

350

400

362

310

26585%

22986%

45

36

Moodle-based SystemRetained Waived

Registered Students Distributed

Survey Collected Cases Valid Cases

0

50

100

150

200

250

300

350

400

363

308

26285%

23389%

46

29

Facebook-based SystemRetained Waived

32

Demographic Analysis

Total Gender Age Race Nationality Academic Eng. Prof. Internet Prof.0

20

40

60

80

100

120

100 %231

61.3 (M)51.5 (22-25)

33.1 (Arab)

14.7 (China)

69.5 (MBA)

16.5 (Excellent)

41 (Excellent)

38.7 (F)

35.9 (26-30)

18 (Chinese)

10 (Iran)

19.6 (MA)

44.5 (V. Good)

37.5 (V. Good)

12.1 (31-40)

14.5 (African)

9.1 (Yemen)

10.9 (MSc)

35.5 (Good)

20.6 (Good)

0.4 (>40)

13.2 (Persian)

9.1 (Syria)

3.5 (Poor) 0.9 (Poor)

6.1 (Indian)

8.5 (Malaysia)

5.9 (Malay)

6.7 (Nigeria)

33

Carryover Effect Analysis

Group Phase System

The variance of the learner outcome is explained only by System factor (different systems have a significant affect on LO)

0.05

0.10

P-Va

lue

Source F Sig.System 53.283 .000Phase .096 .757Group 1.997 .158

Dependent Variable: Learner Outcome.

34

Assessing PLS-SEM ResultsValidity

Relations

Internal consistency reliability

Convergent validity

Outer model loadings and significance

Outer Loading

Composite Reliability

“AVE” numbers and Latent Variable Correlations

Variance Inflation Factor (VIF)

• Predictive power (R2) and Predictive relevance (Q2)

• ƒ² effect size • P-Values, T Statistics, and Path Coefficient

Average Variance Extracted (AVE)

Discriminant validity

Collinearity Assessment

Indicator reliability

Hair (2014)

35

Significance of Construct Model Relations

IQ -> IU

IQ -> US

SQ -> IU

SQ -> US

SrQ -> IU

SrQ -> US

SS -> IU

SS -> LO

SS -> US

LP -> IU

LP -> LO

LP -> US

TP -> IU

TP -> LO

TP -> US

US -> IU

US -> LO

IU -> LO

FacebookCoefficient T Value P Values

0.057 0.811 0.209

0.264 3.77 0***

0.156 2.039 0.021*

0.227 2.778 0.003*

0.093 1.28 0.1

-0.04 0.506 0.306

-0.039 0.655 0.256

0.088 1.73 0.042*

-0.012 0.201 0.42

0.298 4.759 0***

0.222 3.552 0***

0.075 1.029 0.152

-0.005 0.074 0.47

0.196 2.886 0.002**

0.355 4.628 0***

0.326 3.839 0***

0.321 5.006 0***

0.145 2.174 0.015*

MoodleCoefficient T Value P Values

0.036 0.599 0.275

0.213 2.987 0.001***

0.142 2.059 0.02*

0.397 5.062 0.000***

0.041 0.653 0.257

-0.047 0.638 0.262

-0.07 1.689 0.046*

0.062 1.365 0.086

-0.01 0.19 0.425

0.209 3.85 0.000***

0.183 3.17 0.001***

0.015 0.234 0.407

0.125 2.036 0.021*

0.207 3.125 0.001***

0.334 4.721 0.000***

0.465 6.993 0.000***

0.263 3.351 0.000***

0.254 3.182 0.001***

36

Hypothesis Discussion

37

Hypothesis Regarding Relations: Moodle-based System

Learner's Outcome 0.704

SS LO Yes* + 0.09 1.37 0.008 0.06TP LO Yes + 0.00 3.13 0.051 0.21LP LO Yes + 0.00 3.17 0.048 0.18US LO Yes + 0.00 3.35 0.068 0.26

IU LO Yes + 0.00 3.18 0.060 0.25

User Satisfactio

n0.673

IQ US Yes + 0.00 2.99 0.049 0.21SQ US Yes + 0.00 5.06 0.152 0.40SrQ US No Null 0.26 0.64 0.002 -0.05SS US No Null 0.43 0.19 0.000 -0.01TP US Yes + 0.00 4.72 0.122 0.33

LP US No Null 0.41 0.23 0.000 0.02

Intention to Use 0.736

IQ IU No Null 0.28 0.60 0.002 0.04SQ IU Yes + 0.02 2.06 0.021 0.14SrQ IU No Null 0.26 0.65 0.002 0.04SS IU No* - 0.05 1.69 0.011 -0.07TP IU Yes + 0.02 2.04 0.020 0.13LP IU Yes + 0.00 3.85 0.070 0.21

US IU Yes + 0.00 6.99 0.268 0.47

Dependent Variable

Predictive Power R2 Hypothesis Sign P Value T Statistics f2 Values Path

Coefficient

38

Hypothesis Regarding Relations: FB-based System

Learner's Outcome 0.657

SS LO Yes + 0.04 1.73 0.013 0.09TP LO Yes + 0.00 2.89 0.046 0.20LP LO Yes + 0.00 3.55 0.060 0.22US LO Yes + 0.00 5.01 0.127 0.32

IU LO Yes + 0.02 2.17 0.027 0.15

User Satisfactio

n0.575

IQ US Yes + 0.00 3.77 0.073 0.26SQ US Yes + 0.00 2.78 0.048 0.23SrQ US No Null 0.31 0.51 0.001 -0.04SS US No Null 0.42 0.20 0.000 -0.01TP US Yes + 0.00 4.63 0.109 0.36

LP US Yes** + 0.15 1.03 0.006 0.08

Intention to Use 0.575

IQ IU No Null 0.21 0.81 0.003 0.06SQ IU Yes + 0.02 2.04 0.022 0.16SrQ IU Yes* + 0.10 1.28 0.007 0.09SS IU No Null 0.26 0.66 0.002 -0.04TP IU No Null 0.47 0.07 0.000 -0.01LP IU Yes + 0.00 4.76 0.092 0.30

US IU Yes + 0.00 3.84 0.106 0.33

Dependent Variable

Predictive Power R2 Hypothesis Sign P Value T Statistics f2 Values Path

Coefficient

39

Re-specified Hybrid Model

Learning System

Outcome

Learner Outcome

System Sociability

Information Quality

Service Quality

Intention to (Use)

User Satisfaction

Teaching Presence

Learner Presence

System Quality

Solid, Approved Relation

Mixed, Semi-Approved Relation

Rejected Relation

Moodle-based

Facebook-based

M

F

F

M

F

40

Hypothesis Regarding Comparison

Intention to Use User Satisfaction Learner Outcome3.3

3.4

3.5

3.6

3.7

3.8

3.9

4

4.1

4.2

4.3

3.6288

3.8035

3.6559

4.0129

4.2575

4.0918

Mean Value

The two-sample t-test shows a significant difference between the mean values of the two data sets for the two

systems with P-value > 0.5

41

Contributions, and Recommendation

CONTRIBUTIONS OF THE STUDY

A model is showing all the constructs for assessing the success of the modern eLearning system and showing interrelationships among these constructs learner outcome in higher education.

An instrument tool (survey) that include all the dimensions within the model. The tool is tested for validity and reliability and can be used by other researchers.

The experiment design process for comparing two information system in a quantitative approach.

Findings and it analysis is important to decision makers in many practitioners.

42

Academic Contributions

A new Hybrid eLearning Success model.

A new survey construct with new definitions and measures.

The experiment design and process of implementation and data collection

A Quantitative comparative analysis between the traditional system and SNS system.

The use of Smart-PLS in acquiring the results.43

RECOMMENDATIONS FOR FUTURE RESEARCH

Further research is needed to assess the model in different conditions.

Replicating the same assessment in other higher education institutes is recommended.

Replicating the same assessment in other levels of study such as Bachelor degree is recommended

44

RECOMMENDATIONS FOR FUTURE RESEARCH

Further research is needed to investigate additional eLearning system success determinants.

Further research is needed to perform another experiment approaches for comparing different systems.

Information technology researchers are recommended to produce an integration approaches for the traditional systems such as Moodle software and the SNS sites such as Facebook.

45

46

Publications

Publications

47

TitleThe role of System Sociability Factor in Modeling Learning Management System Success in University Education.

Status Published; April 2015

Journal/ConferenceInternational Conference on e-Commerce, e-Administration, e-Society, e-Education, and e-Technology (e-CASE & e-Tech 2015)

TitleDEVELOPING A SUCCESS MODEL FOR CONTENT MANAGEMENT SYSTEM IN HIGHER EDUCATION: ANALYSIS FROM STUDENTS’ PERSPECTIVE.

Status In Process. Initial Submission; Expected June 2016

Journal/Conference The Journal of the Association for Information Systems (JAIS)

TitleINVESTIGATION OF A MODIFIED INFORMATION SYSTEM SUCCESS IN UNIVERSITY LEARNING SUCCESS – STUDENTS’ PERSPICTIVE.

Status In Process. Accept Manuscript; Expected Dec. 2015

Journal/Conference Journal of Technology; UTM

Publications

48

TitleFactors Influencing the Learning Management System ( LMS ) Success Among Undergraduate Students in Limkokwing University of Creative Technology , Malaysia.

Status Published; June 2015

Journal/Conference International Journal of Multicultural and Multireligious Understanding

TitleLearning Management System (LMS) Success: An investigation among the university students.

Status Published; Aug. 2015

Journal/ConferenceThe IEEE Conference on e-Learning, e-Management and e-Services (IC3e 2015)

TitleThe effects of school management support on the use of interactive whiteboard (IWB) in high school.

Status Published; 2015

Journal/Conference International Journal of Multicultural and Multireligious Understanding

50

End of Formal VIVA Presentation

51

The next section are not presentedIt is for supporting during discussion

Population and Sampling

52

53

Population

Count of

Moodle-Based System FB-Based System

AllPhase1/Group A

Phase2/Group B

AllPhase1/Group B

Phase2/Group A

Registered Students 362 174 188 363 192 171

54

SmartPLS Minimum Sample Size

Sample size should be at least 10 times the largest number of formative measures of a particular construct, or 10 times the largest number of structural paths points to a single latent construct.

In this study, the minimum sample size is 70.

Hair (2014)

55

Effective Sample Size

55

Cohen (1988)

Dattalo (2008)

G*Power screenshot of the applied setting

56

Actual Respondents

Count of

Moodle-Based System FB-Based System

AllPhase1/Group A

Phase2/Group B

AllPhase1/Group B

Phase2/Group A

Registered Students 362 174 188 363 192 171

Distributed Survey 310 152 158 308 160 148

Collected Cases 265 128 137 262 130 132

Non-Fitted Cases -5     -2    

Uncompleted Cases -11     -9    

Initial Cases for Analysis 249     251    

Unengaged Screening -7     -10    

Univariate Screening -10     -6    

Multivariate Screening -3     -2    

Cleaned Cases for Analysis 229 112 117 233 118 115

Assessing PLS-SEM Results

57

Indicator reliability(Outer Loading)

Outer loading scale used in reflective models to assure the proper loading of measures in its construct.

The acceptable level of outer loading is 0.708 and above.

Levels between 0.4 and 0.7 can be deleted if other indicators reliability enhanced

(Hulland 1999; Hair et al. 2014).

58

59

Indicator reliability (Outer Loading) - MoodleIQ IU LO LP SQ SS SrQ TP US

IQ1 0.783IQ2 0.848IQ3 0.839IQ4 0.839IQ5 0.792IU1 0.773IU2 0.857IU3 0.860IU4 0.815LO1 0.874LO2 0.893LO3 0.861LO4 0.810LO5 0.840LP1 0.844LP2 0.740LP4 0.755LP5 0.685SQ1 0.814SQ2 0.839SQ3 0.800SQ4 0.847SQ5 0.871SS1 0.769SS2 0.811SS3 0.821SS5 0.816SrQ1 0.845SrQ2 0.842SrQ3 0.755SrQ5 0.776TP1 0.826TP2 0.792TP3 0.804TP5 0.860US1 0.893US2 0.882US3 0.905US4 0.859

60

Indicator reliability (Outer Loading) - FacebookIQ IU LO LP SQ SS SrQ TP US

IQ1 0.730IQ2 0.811IQ3 0.812IQ4 0.808IQ5 0.761IU1 0.832IU2 0.876IU3 0.834LO1 0.818LO2 0.821LO3 0.784LO4 0.758LO5 0.821LP1 0.790LP2 0.861LP4 0.763LP5 0.817SQ1 0.754SQ2 0.779SQ3 0.747SQ4 0.815SQ5 0.851SS1 0.711SS2 0.839SS3 0.808SS5 0.774SrQ1 0.820SrQ2 0.785SrQ3 0.796SrQ5 0.768TP1 0.727TP2 0.772TP3 0.801TP4 0.815TP5 0.696US1 0.866US2 0.832US3 0.809US4 0.833

Internal Consistency(Composite Reliability)

To show the consistency of items of the same construct.

Composite reliability should be 0.7 or higher. If it is an exploratory research, 0.6 or higher is

acceptable

(Bagozzi & Yi 1988; Hair et al. 2014)

62

Composite Reliability

IQSQ

SrQTP

LPSS

SUUS

LO

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.88

9

0.89

2

0.87

1

0.87

4

0.88

3

0.86

4

0.88

4

0.90

2

0.89

9

0.91

2

0.92

0.88

0.89

2

0.84

3

0.88

0.89

6

0.93

5

0.93

2

0.7

0.6

Convergent Validity(AVE value)

Average Variance Extracted (AVE) scale used to show that measure inside individual construct is related.

The acceptable level of AVE value is 0.5 and above

(Bagozzi & Yi 1988; Hair et al. 2014).

63

64

Convergent Validity: (AVE) Average Variance Extracted

IQSQ

SrQTP

LPSS

SUUS

LO

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

FBMoodle

0.61

6

0.62

4

0.62

8

0.58

3 0.65

4

0.61

6

0.71

8

0.69

8

0.64

2

0.67

4

0.69

6

0.64

9

0.67

3

0.57

5 0.64

7

0.68

4 0.78

3

0.73

3

0.5

AVE

Discriminant Validity (Fornell-Larcker Criterion Analysis)

“Correlation matrix of AVE values” scale used to show that measures outside a specific construct is not related to it.

Rule of thumb is that the square root of AVE value of a specific construct must be greater than all the other values in the same column or row of the correlation matrix

(Hair et al. 2014; Fornell & Larcker 1981).

65

66

Discriminant Validity (Fornell-Larcker Criterion Analysis)

IQ IU LO LP SQ SS SrQ TP US

IQ 0.821IU 0.679 0.827LO 0.633 0.769 0.856LP 0.664 0.672 0.683 0.758SQ 0.724 0.75 0.689 0.654 0.834SS 0.537 0.444 0.511 0.587 0.498 0.805

SrQ 0.726 0.662 0.618 0.635 0.752 0.532 0.805TP 0.678 0.73 0.739 0.656 0.738 0.522 0.72 0.821US 0.696 0.812 0.76 0.599 0.766 0.459 0.65 0.741 0.885

IQ IU LO LP SQ SS SrQ TP US

IQ 0.785IU 0.587 0.848LO 0.601 0.656 0.801LP 0.603 0.64 0.675 0.809SQ 0.652 0.628 0.635 0.595 0.79SS 0.508 0.429 0.528 0.607 0.428 0.785

SrQ 0.67 0.583 0.613 0.61 0.675 0.57 0.792TP 0.61 0.599 0.693 0.642 0.695 0.535 0.713 0.763US 0.641 0.668 0.715 0.565 0.658 0.432 0.582 0.686 0.835

Forn

ell-L

arck

er

Crite

rion

Anal

ysis

The

Diag

onal

Val

ue m

ust b

e gr

eate

r tha

n al

l th

e ot

her v

alue

s in

the

sam

e co

lum

n or

raw

.

Multicollinearity Analysis: (VIF) Variance Inflation Factor

Each set of exogenous latent variables in the inner model is checked for potential collinearity problem to see if any variables should be eliminated, merged into one, or simply have a higher-order latent variable developed.

The acceptable level of VIF value is above 0.20 and less than 5.00.

(Wong 2013)

67

68

Multicollinearity Analysis: (VIF) Variance Inflation Factor

5.0

0.2

VIF IU LO US

IQ 2.939 2.801

IU 3.647

LO

LP 2.372 2.365 2.372

SQ 3.644 3.164

SS 1.675 1.622 1.675

SrQ 3.052 3.045

TP 3.141 2.826 2.8

US 3.057 3.427

IU LO US

IQ 2.423 2.258

IU 2.23

LO

LP 2.282 2.404 2.269

SQ 2.621 2.5

SS 1.813 1.682 1.813

SrQ 2.784 2.78

TP 3 2.422 2.704

US 2.355 2.352

69

Predictive Power (R2) & Predictive Relevance (Q2)

0.2

0.5

R2W

eak

Mod

erat

eSt

rong

0.75

0.02

0.15

Q2

Smal

lM

ediu

mLa

rge

0.35R Square Q Square

Intention to Use (IU) 0.736 0.495

Learner Outcome (LO) 0.704 0.511

User Satisfaction (US) 0.673 0.516R Square Q Square

Intention to Use (IU) 0.575 0.399

Learner Outcome (LO) 0.657 0.417

User Satisfaction (US) 0.575 0.393

R2 is the “percent of variance explained” by the model in the dependent variables.

Q2 statistic measures the predictive relevance of the model by reproducing the

observed values by the model itself.

70

The Effect Size – ƒ2

The ƒ² effect size measures the change in the R² value when a specified exogenous construct is omitted from the model.

0.02

0.15

ƒ2Sm

all

Med

ium

Larg

e

0.35

IU LO US

IQ 0.002 0.049

IU 0.06

LO

LP 0.07 0.048 0

SQ 0.021 0.152

SS 0.011 0.008 0

SrQ 0.002 0.002

TP 0.019 0.051 0.122

US 0.268 0.0680.00

Extr

aSm

all

IU LO US

IQ 0.003 0.073

IU 0.027

LO

LP 0.092 0.06 0.006

SQ 0.022 0.048

SS 0.002 0.013 0

SrQ 0.007 0.001

TP 0 0.046 0.109

US 0.106 0.127

Significance of Construct Model Relations

Three values are used for the assessment that are significant level or probability estimate value (P value), the significance of path coefficient (T-statistics), and path coefficient.

As Hair (2014), the rule of thumbs for assessing the values is:

P-value could be on three levels 1%, 5% or 10%, but the popular level in psychological studies is 5% or (0.05).

With 5% significance level, T statistic > 1.96 is significant with a two-tailed test and T Statistics >.98 is significant for a one-tailed test.

71

72

Significance of Construct Model Relations

IQ -> IU

IQ -> US

SQ -> IU

SQ -> US

SrQ -> IU

SrQ -> US

SS -> IU

SS -> LO

SS -> US

LP -> IU

LP -> LO

LP -> US

TP -> IU

TP -> LO

TP -> US

US -> IU

US -> LO

IU -> LO

FacebookCoefficient T Value P Values

0.057 0.811 0.209

0.264 3.77 0***

0.156 2.039 0.021*

0.227 2.778 0.003*

0.093 1.28 0.1

-0.04 0.506 0.306

-0.039 0.655 0.256

0.088 1.73 0.042*

-0.012 0.201 0.42

0.298 4.759 0***

0.222 3.552 0***

0.075 1.029 0.152

-0.005 0.074 0.47

0.196 2.886 0.002**

0.355 4.628 0***

0.326 3.839 0***

0.321 5.006 0***

0.145 2.174 0.015*

MoodleCoefficient T Value P Values

0.036 0.599 0.275

0.213 2.987 0.001***

0.142 2.059 0.02*

0.397 5.062 0.000***

0.041 0.653 0.257

-0.047 0.638 0.262

-0.07 1.689 0.046*

0.062 1.365 0.086

-0.01 0.19 0.425

0.209 3.85 0.000***

0.183 3.17 0.001***

0.015 0.234 0.407

0.125 2.036 0.021*

0.207 3.125 0.001***

0.334 4.721 0.000***

0.465 6.993 0.000***

0.263 3.351 0.000***

0.254 3.182 0.001***

Survey

73

74

Questionnaire P1-2

75

Questionnaire P3-4

76

Research Variables and Literature Reference ModelsReferences IQ SQ SrQ SS TP LP IU US LO

IS success model by Delone and McLean (2003) x x x       x x x

ELearning system model by Freeze et al. (2010) x x         x x x

ELearning system model by Wang et al. (2007) x x x       x x x

Hexagonal eLearning assessment model by Ozkan et al. (2009) x x x   x x      

Revised community of inquiry model by Shea & Bidjerano (2010)         x x     x

eLearning acceptance framework by Selim (2007)     x   x x      

Hierarchical model for eLearning CSF.. by Bhuasiri et al. (2012) x x x   x x      

Model of online community attributes&benefit by Kim, Park and Jin (2008)       x         x

Sociability and Usability Framework by Lambropoulos (2005)       x         x

Online Community framework by de Souza & Preece (2004)       x         x

(Garrison et al. 2010)         x x     x

(Arbaugh 2008)         x x   x x

(Daspit & D’Souza 2012)         x x     x

(Lambert & Fisher 2013)         x x     x

(Lee-post 2009) x x x       x x x

(Keramati et al. 2011)         x x     x

(Gao et al. 2010)       x     x   x

(Lin et al. 2007) x x   x       x  

(Phang et al. 2009)       x         x

(Paechter et al. 2010)       x x x      

Constructs’ Definitions

77

Information Quality (IQ)

This study defines information quality as the level of goodness of the information produced by the system and assessed by using different measures differ from system to another based on its nature and functions.

Those measures could include up to date, relevance, accuracy and much more.

78

System Quality (SQ)

This study defined system quality as the level of goodness of the information system features and tools excluding the output and assessed by using different measures differ from system to another based on its nature and functions.

Those measures could include flexibility, response time, system reliability and much more.

79

Service Quality (SrQ)

This study defines service quality as the level of goodness of the personnel support offered by the administrative affairs to the system users.

Those measures could include communication quality, technical competence, and empathy of the personnel staff and others.

80

System Sociability (SS)

This study defined system sociability as the system level of readiness and practice for the interaction services and activities including technology, policies and practice.

Those measures could include system interactive, members’ interaction, policies support, and others.

81

Teaching Presence (TP)

This study defines teaching presence as the level of instructor involvement and participation into the system

including content feeding quality and interaction with the system and members in a synchronous and asynchronous manner.

82

Learner Presence (LP)

This study defines learner presence as the level of learner readiness and participation in the system

including learner confident interaction, confident participation, ability to form an impression and others.

83

Intention to Use (IU)

This study defines intention to use as the level of willingness to use the information system.

Those measures could include frequency of use, dependency, reusability and others.

84

User Satisfaction (US)

This study defines user satisfaction as the level of goodwill achieved after experiencing the system.

Those measures could include system usefulness, adequacy, effectiveness and others.

85

Learner Outcome (LO)

This study defines learner outcome as the level of studying outcome achieved by using the information system.

Those measures could include productivity, performance, better thinking and others.

86

Community of Inquiry Framework

87

Brief of COI

The community of inquiry framework is an instructional design model for eLearning developed by Randy Garrison, Terry Anderson et al. (2000).

This framework is for educational context as it proposes a framework for the use of computer-mediated communication to support the education process.

The framework has three essential elements cognitive presence, social presence, and teaching presence

88

The Community of Inquiry (CoI) bundle is the result of a project ran from 1997 to 2001.

Definitions of COI

Social presence refers to the ability of learners to project their personality into the community of inquiry, means learners can introduce themselves as a real people within the online communication or interaction.

Teaching presence construct outlines task sets such as organization, design, discourse facilitation, and direct instruction and articulates the specific behaviours likely to result in a productive community of inquiry.

Cognitive presence refers to the extent to which the participants in any particular configuration of a community of inquiry are able to construct meaning through sustained communication.

89

Preece’s Sociability and Usability Framework

90

Brief of Preece’s S&U

Preece (2000, 2001) proposed system usability and system sociability as determinants of the online community success. Goals, purposes and functions of the community affect the needs of online communities.

This framework used in many studies where the finding mostly shows a relationship between sociability and social benefits and usability and functional benefit.

91

Definitions of Preece’s S&U

Sociability dimension involves the measures related to the purpose, people, and policies. Purpose factor refers to the interaction and involvement levels of community participants.

Usability dimension covers the measures related to dialog and social interaction support, information design, navigation, and access.

Success definition of the online community differs based on the perspective of whom.

92

Delone and Mclean Information System Success Model

93

Brief of D&M IS Model

D&M IS success model is a result of attempts to provide an integrated scene of IS success that enables comparisons between different studies. It propose a broad and acceptable meaningful of the information system success.

The founders of this famous theory are William H. Delone and Ephraim R. McLean in 1992. Later on, the same authors revised the original theory and proposed an updated model after ten years in response to comments announced by other different researchers.

The updated model proposed six different dimensions of the IS success and provided an identifying, describing, and explaining the relationships among the six dimensions of success.

94

Definitions of D&M IS Model

System quality construct comprises the desirable characteristics of the system itself and includes related measures of the IS itself.

Information quality construct comprises the desirable characteristics of the information system output.

Service quality construct characterizes the quality of the support offered to system users by the IS department and IT support workforce.

The (intention to) use construct characterizes the user utilization level of the desired information system.

User satisfaction construct comprises the user’s level of satisfaction when using the desired information system.

Net benefits construct comprises the extent to which desired information system are contributing to the success of the desired users.95

96