1

Comparing EFL Learners'Reading Comprehension BetweenHypertext and Printed Text

Article Critique 17 Nov 2008

EDS 502 (RESEARCH METHODS IN EDUCATION)

Tseng, M. (2008). Comparing EFL learners' reading comprehension between hypertext and printed text. CALL-EJ Online, 9(2)

2

Critique in a Few Words

1. Summary of the Article2. Critique (+/- points)3. Recommendation4. Implications of the study5. Summary

3

Before summary:

4

Summary of the Article

What are the effects of reading printed text and hypertext on the EFL learners' reading comprehension?

RQ:

Quasi-experimental article: matching only post-test only control group designDesign:

Participants:

Forty-six Taiwanese students equally divided into two groups: the Experimental Group and the Control Group.

Instruments: Four reading texts and a comprehension test and a

questionnaire

Results:The students who read from printed text were more successful, apart from one skill: guessing a word’s meaning from context

5

Summary of the Article (Cont.)

Procedure:

(1) to give students some related vocabulary, and pictures to predict the articles, (2) to give them four articles to read (3) to test their skimming skill, scanning skill, and vocabulary knowledge through reading comprehension questions

6

Summary of the Article (Cont.)

Hypothesis:

There is a difference between reading from a printed text and reading from a computer screen

Variables:

Independent Variable: Reading printed text and hypertext

Dependent Variable: EFL learners' reading comprehension

7

2. Critique of the Article

Threats to internal validity:Implementation:

No information about the course instructors What were the major difficulties of reading hypertext? (Tseng, 2008) Subject Characteristics : No random sampling and random assignment (JOINED?) No detailed information to control confounding variables (age, gender, reading ability, socio-economic status, etc…)Location: Experimental group: Each student has a computerControl group: no computersInstrumentation: Instrument Decay: 9-page reading test.

Attitude of Subjects:Control group: no computers (John Henry effect)

The possible confounding variables or their possible effects on the outcome of the research were not mentioned

8

Validity and reliability of test scores:• No statistical evidence on validity and

reliability• Only raw scores were provided

2. Critique of the Article (Cont.)

9

Questionnaire Results

2. Critique of the Article (Cont.)

Internal validity threats to the outcome of the study No statistics, did not mention validity or reliability

10

2. Critique of the Article (Cont.) External Validity

Quasi-experimental design

No population and ecological generalizability Did not mention target and accessible populations

No random assignment No pre-test

Conclusion: Interpretation from the raw data Contradiction with the literature reviewLiterature review: Advantages of reading from the

computer screen. (non linear reading)Result: This study also confirmed that students made poor performance when they read hypertext (Tseng, 2008).

11

3. Recommendation

1. State confounding variables2. Supply more information about the individuals who

implemented the study3. Try to use random sampling and if possible random

assignment4. Supply more information about the subjects5. Try to keep locations constant, remove such

variables as uncomfortable seating6. Support the instruments with reliability and validity

data7. Not to affect attitudes of the subjects provide

similar opportunities8. Interpretation: Give statistical data analysis9. Mention generalizibility (external validity)10.Choose a stronger design (one of the randomized

designs)11.Literature review should present more research

findings rather than definitions

12

4. Implications of the studyThe pedagogical implications: The importance of selecting web pages for students The importance of setting up computer screens and web pages The importance of teaching students how to read hypertext.

Only three questions without statistical data analysis helped such interpretation:

1. What factors affected reading via computer screens? About 26.4% of participants chose the size of the font, 24.5% of them chose the background color of the web pages, and 47.3% chose the downloading speed.

2. In the background color of web pages, about 35% of participants chose dark background with white words, 60% of them chose white background with black words and 5% chose either ways. It might be speculated that students extended their reading inclination to computer screens because most of the paper was white and printing color was black.

3. Participants were asked whether they preferred to read texts through computer screens or paper. In the Control Group, 18% of them chose screens and 82% of them chose paper whereas in the Experimental Group, 17% of them chose screens and 83% of them chose paper. Entirely, 31% of the entire class chose screen and 69 of them chose paper. For participants, they still preferred to read texts on paper instead of computer screens.

13

While the study has merit, the methods need to be re-evaluated. 

The power of the study needs to be increased by obtaining a larger sample size. 

The numerous potential threats to internal and external validity need to be addressed and minimized where possible. 

It would also be helpful to be given data analysis regarding the validity and reliability of the scores of the instruments. 

Without these, it is impossible to evaluate the potential meaningfulness of this study. 

5. Summary