assessing the accessibility of web 2.0 websites
TRANSCRIPT
ASSESSING THE ACCESSIBILITY OF WEB 2.0 WEBSITES
Authors
Tyson McMillan
University of North Texas, College of Information
918 W Chestnut St, APT 206, Denton, TX 76201
Email: [email protected]
Lin Lin
University of North Texas, College of Information
3940 N. Elm, Rm G150, Denton, TX 76207
Email: [email protected]
This study examines the accessibility of Web 2.0 websites to the visually impaired. Web
accessibility standards are established to maximize the ability of those with impairments to
navigate the web. Various computer tools exist to evaluate web HTML content against existing
accessibility standards. Using a weighted metric-based formula called the Web Accessibility
Barrier (WAB) score, this study adopted an experimental design and compared 88 randomly
selected Web 2.0 websites against 88 randomly selected Web 1.0 websites. The study found
that Web 2.0 websites are significantly less accessible than Web 1.0 websites. Details of the
analysis, evaluation, discussion, and recommendations are included in the paper.
Introduction and Main Problem
Individuals who are visually impaired or impaired in other manners use various tools to
browse web-based content. These tools rely heavily on well-designed HTML code. Web 2.0 is a
popular trend in web design that takes full advantage of the social and interactive features of
the web (Fetscherin & Lattemann, 2007). As Web 2.0 websites increase in popularity, it is
necessary to assess their accessibility to the visually impaired.
The purpose of this study is to assess to what extent Web 2.0 websites are accessible to the
visually impaired by using the Web Accessibility Barrier (WAB) score metric proposed by
Parmanto and Zeng (2005). The WAB metric provides web developers with a picture of the
extent to which his or her web code is compliant. Numerous research has been done on Web
Accessibility or on Web 2.0 separately; however, research on the combined two are limited.
From 1995 to 2007, little research used the WAB metric to measure Web 2.0 site
accessibility (Thomson Scientific, 2007). As a result, there is little research examining the
relationship between Web 2.0 and accessibility issues. This study, therefore, is unique in its
use of the WAB score metric to compare the accessibility of Web 2.0 websites to Web 1.0
websites through a random sampling from both categories.
Background and Related Work
Web accessibility issues and measurements
The Web Content Accessibility Guidelines (WCAG), the Web Accessibility Initiative
(WAI) of the World Wide Web Consortium (W3C), and Section 508 legislation all
seek to maximize the ability of those with impairments to navigate the web (W3C,
2007a; W3C, 2007b; Section 508, 2007). The WAI's purpose is to create
strategies, resources, and guidelines to make the web accessible to people with
disabilities (W3C, 2007a). WCAG are foundation guidelines of the W3C that
explain how to make Web content accessible to the greatest extent possible for
people with disabilities (W3C, 2007b). Section 508 is an extension of the 1998
Rehabilitation Act enacted by Congress to:
1. Eliminate barriers in information technology
2. Ensure availability of new opportunities for people with disabilities, and
3. Encourage development of information technologies that will help achieve
the goals of accessibility.
Section 508 applies to all federal agencies that develop, procure, maintain, or use
electronic and information technologies (Section 508, 2007). Ideally, adherence
to these standards and mandates enables accessibility tools to perform well. Web
code compliance with WCAG 1.0, ideally will reduce the obstacles of those who
use screen readers for accessibility purposes. This is due to the fact that WCAG
1.0 establishes best practices for web coding standards (W3C, 2007a; W3C,
2007b). WCAG 2.0 is now under development to accommodate the rapid
development of newer web technologies (W3C, 2007b). These standards further
are made by experts in the field (W3C, 2007a).
Several automated tools are available to assess compliance with WCAG 1.0
guidelines and WCAG 2.0 is presently under development (W3C, 2007b). Such
accessibility tools include readers like the JAWS® screen reader made by
Freedom Scientific and ZoomText Magnifier/Reader from Ai Squared (Theofanos
and Redishm, 2003). Accordingly, it is often beneficial to evaluate a website's
compliance with these standards by using accessibility evaluation software tools.
Several accessibility evaluation software tools, such as Hi Software's Cynthia
Validator and Watchfire's Bobby validator, exist to assess the adherence of HTML
code to WCAG 1.0 standards. The output of such tools is very thorough and
meaningful; however, these tools often give results on a pass/fail basis as they
relate to the WCAG 1.0 checkpoints. Understandably, these tools tend to fail an
entire page when there is even one violation of WCAG 1.0 guidelines. With the
binary pass/fail results of such tools alone, however, it is difficult to gauge a
website's overall compliance with WCAG 1.0 standards. Is the website for
instance, 90% compliant, or 25% compliant? Accordingly, it would be beneficial to
use a quantitative metric to assess web accessibility, and to obtain results that
are more meaningful. Furthermore, a quantitative metric gives unique insight into
the accessibility of Web 2.0 websites by providing a numerical score that can be
used for comparison purposes.
Web Accessibility Barrier (WAB) Score
The Web Accessibility Barrier (WAB) score metric was proposed by Parmanto and
Zeng (2005). It is a method that enables one to quantitatively identifying
accessibility trends across the 2 studied groups: Web 1.0 websites and Web 2.0
websites. The WAB score formula tests 25 WCAG 1.0 criteria that can be
evaluated automatically and is defined as follows:
wab_score
Where p is the total pages of the website, v is the total violations of a web page,
nv is the number of violations overall, Nv is the number of potential violations, wv
is a weight value in inverse proportion to WCAG priority level, and Np is the total
number of pages checked.
A 0 WABScore indicates a website that passes all 25 checkpoints. Any number
above zero indicates a site moving further away from accessibility criteria. For wv,
the inverse weight was applied for each error in Priority I, II, or III status. Priority I
errors, for instance received a 3 weight, and Priority III errors received a 1 weight
(Parmato and Zeng, 2005). The advantage of the weighted average is that it does
not fail an entire webpage due to one error. Errors are looked at as it relates to the
actual errors and potential errors, thus weighing them in what seems to be a more
fair metric. In other words, one has a more complete picture of the extent to which
one's webpage is compliant.
Web 1.0 and Web 2.0 Website Characteristics
Although there are different arguments regarding the definitions and differences
between Web 1.0 and Web 2.0 websites, several characteristics help generally
distinguish Web 1.0 websites from Web 2.0 websites. A Web 1.0 website is
characterized by simply html-driven, usually static text and flash (Fletscherin &
Lattemann, 2007). Web 2.0 websites characteristically contain elements that
allow users to contribute to the content. Users typically actively participate in and
contribute to the content of such websites. Examples of Web 2.0 include Weblogs,
Wikis (i.e. Wikipedia.org), Podcasts, RSS feeds, and other popular social
networking and tagging websites such as MySpace.com, Facebook.com,
Flickr.com, and the like (Fetscherin & Lattemann, 2007). Static html driven Web
1.0 websites usually do not contain a component that allows users to contribute
to the website's content. The lack of this user contribution component is the major
factor that distinguishes Web 1.0 websites from Web 2.0 websites. A blog website
for instance, is different from a static informational website that does not allow
users to post content.
The Study
Research Questions
This study examines accessibility of Web 2.0 websites and aims to answer the
following questions:
1. Does Web 2.0 website affiliation present challenges in web navigation for
the visually impaired?
2. Is the HTML code in common Web 2.0 applications compliant with web
accessibility standards?
3. Is there a correlation between Web 2.0 affiliation and navigability of a
website?
Methodology
Since early websites were all static websites with no user contribution component, they can
be categorized as Web 1.0. As such, there is a large pool of Web 1.0 websites from which to
choose. For the purposes of this work, it was necessary to select a manageable sample of
Web 1.0 websites. After analysis of several Web 1.0 groups, university websites emerged as a
legitimate Web 1.0 source. Like federal agency websites, university websites tended to
adhere well to Section 508 standards in their design and are more highly regulated than other
Web 1.0 websites due to state guidelines and individual university guidelines (Parmanto and
Zeng, 2005). Also, university websites tended to be slower in adopting Web 2.0 technologies
when compared to other Web 1.0 website groups. The limited use of Web 2.0 technologies on
university websites at the time of this work, and the perceived adherence of university
websites to Section 508 regulations, made such websites an ideal Web 1.0 source.
In this study, a random sample of 88 U.S. university homepages is selected to form the
category of Web 1.0 websites. The university list is based on the U.S. News and World Report
(2007), which listed top schools for 2008. The extent of overall accessibility of this sample of
top universities will be necessary for comparison purposes. In addition, a random sample of
88 Web 2.0 sites is selected from E-Consultant (2007) to form the category of Web 2.0
websites. E-Consultant (2007) listed various Web 2.0 websites by category. This particular
website was selected due to the extensive work of categorization that was done on the part of
the website authors. Table 1 summarizes the sampling:
A-priori analysis with the GPOWER tool for a t-test experiment comparing the two groups is
conducted. Posttest Only Control Group experimental design with Web 2.0 affiliation as the
treatment is utilized, with the WAB score as the method of observation. A controlled
experimental design is used to minimize threats to the internal and external validity
(Campbell and Stanely, 1963, p. 8). Accordingly, Posttest Only Control Group true
experimental design as defined by Campbell and Stanley is adopted (1963, p. 8). T-test
statistical analysis is conducted for an experiment comparing the average WAB scores of
sample Web 2.0 websites to the WAB scores of the sample Web 1.0 websites. Table 2
summarizes the experimental design of the study:
It is important to note that web pages are constantly being changed. Accordingly, data should
be looked at as a snap shot in time of a webpage as it appeared at the time of this work's
evaluation. Use of High Software Accessibility Evaluator was essential to helping obtain the
statistics for each website in this work. The High Software tool evaluates HTML of the
requested URL, and provides output data that can be evaluated via the Web accessibility
barrier method, Web Content Accessibility Guidelines, or other methods. Campbell and
Stanley (1963, p. 26) states that a T-Test is optimal for this Posttest Only Control Group true
experimental design that has been utilized in this work. Thus, after the WAB score was
obtained for each of the websites in the experiment, a T-test was conducted. Of the 88 Web
1.0 websites randomly generated, one website was assigned the wrong URL at the time of
data collection in November 2007, and three Web 2.0 websites no longer existed at the time
of data collection. This is the equivalent of mortality in human-subjects research and thus
accounts for the four missing WAB scores in TABLE 3 below:
Table 4 shows the mean WAB Scores obtained for the sample Web 1.0 websites and the
sample Web 2.0 websites in this work.
Results
Figure 1 demonstrates that WAB scores of Web 1.0 websites reside in a similar cluster,
whereas WAB scores are widely dispersed among Web 2.0 websites.
The following means plot (Figure 2) tells the best story of the significance of the results:
Only 10 out of the 176 or 5.68% of the websites in this work achieved a WAB Score of 0. The
top score exceeded 150,000. The mean WAB score of homepages of the representative Web
1.0 websites is significantly lower than the mean Web 2.0 score of homepages of the
representative sample of Web 2.0 websites. By WAB score standards, Web 1.0 websites are
more accessible than Web 2.0 websites. Illustratively, Web 2.0 websites are on average less
accessible than there Web 1.0 counterparts.
The significant difference between the average mean of Web 1.0 website homepages and
Web 2.0 website homepages, suggests that Web 2.0 websites on average are less accessible
than there Web 1.0 counterparts. In fact, Web 2.0 websites homepages on average were 15
times less accessible than their Web 1.0 counterparts. This further suggests that there is a
positive correlation between accessibility compliance and site navigability for the visually
impaired. This is due to the fact that Web 2.0 websites tended to violate more WCAG 1.0
checkpoints than their Web 1.0 counterparts. These results have great promise for future
exploration on these matters.
The T- test cannot assume equal variances due to the Sig of .000 at the .05 level of
significance. However, our T-Value is char?2.68 with 85.029 degrees of freedom, according to
the Welch test, which assumes unequal variances. Under the conditions of the Welch test,
there is significance between the groups since .009 is less than the .05 level of significance.
This indicates that there is a negative correlation between Web 2.0 affiliation and site
accessibility. Table 5 shows the t-test results:
It is important to note that the results are not significant when assuming Equal Variances in T-
tests (the stronger) test. However, they are significant by the Welch test, which assume
unequal variances. In spite of the t-test information, the WAB score results alone are a very
important contribution.
Analysis and Discussion
In general, Web 1.0 websites were on average more accessible than their Web 2.0
counterparts. This may be due to the fact that the university Web 1.0 websites are in highly
regulated environments. Public universities, for instance, generally must adhere to Section
508 standards. This, among other things, made university sites an optimal control group
instead the other Web 1.0 websites groups available. Web 2.0 websites generally failed
automatic accessibility checkpoints due to their large use of web-based images without
alternative text. Implementing several images without alternative text was the primary
violation that made Web 2.0 websites less accessible than their Web 1.0 counterparts.
The large WAB score difference between Web 1.0 and Web 2.0 alone is an indication that
Web 2.0 websites are on average more inaccessible than their Web 1.0 counterparts. Web
1.0 and 2.0 websites on average had a similar amount of homepage images. Web 1.0 sites
averaged 22.18 homepage images and Web 2.0 averaged 23.27 homepage images.
However, Web 2.0 websites on average had 2.3 times more violations related to images than
their Web 1.0 counterparts. This partially explains why WAB scores for Web 2.0 sites where on
average greater than their Web 1.0 counterparts. Priority I errors get the largest violation
weight of "3" thus Priority I errors have a large impact on overall WAB Score. Web 2.0
websites had more link phrase violations. Therefore, this also affects accessibility of Web 2.0
websites. Priority II Errors get a weight of "2" in the WAB Score, thus, these errors also have a
fairly large impact on overall WAB score. Web 2.0 websites had more links on average, thus
more potential for link violations.
Limitations of the Study
In order to keep the scope manageable, this work only evaluates the home page of each of
the mentioned websites. According to Hackett and Parmanto's (2009) study testing the
homepage only is not enough, therefore future studies should look into each website more
comprehensively. Any future study should take into consider the dynamic and evolving nature
of the web.
Conclusion
This work sought to address several questions as it relates to Web 2.0 website accessibility.
To the questions "Does Web 2.0 website affiliation present challenges in web navigation for
the visually impaired?" and "Is there a correlation between Web 2.0 affiliation and navigability
of a website?", analysis revealed higher on average WAB scores on Web 2.0 website
homepages compared to Web 1.0 website homepages. On average this suggests that more
errors related to accessibility standards, thus more potential obstacles for the visually
impaired users on Web 2.0 websites.
To address the question, "Is the HTML code in common Web 2.0 applications compliant with
web accessibility standards?", only 10 out of the 176 or 5.68% of the websites in this work
achieved a WAB Score of 0. The top score exceeded 150,000. The mean WAB score of the
representative Web 1.0 homepages is significantly lower than the mean score of the
representative Web 2.0 homepages. There is room for improvement in both Web 1.0 websites
and Web 2.0 websites, however, Web 2.0 websites have a greater distance to travel in order
to achieve better accessibility.
Developers can use accessibility tools such as Hi Software's Cynthia Validator and the WAB
Score metric together. The use of the two tools does not have to be mutually exclusive. This
work simply proposes that the WAB score helps provide a more complete picture of the extent
to which one's webpage is accessible. Web 2.0 is a trend that will continue to gain in
popularity. Understanding issues of accessibility as it relates to Web 2.0 will benefit technical
audiences such as web developers, academics audiences such educators and students, and
general web users audiences. From a website developer's standpoint, the WAB Score of
Parmanto and Zeng (2005) offers a numerical gauge of the extent to which one's work is
accessible. The further away a developer's score is from 0, the more work he or she must do
to bring the webpage up to better compliance. As research issues on Web 2.0 become more
popular, educators will be able to teach methods to students that enable them to design more
accessible web-based content while also taking full advantage of the benefits of the
technologies. By following accessibility standards, both visually impaired users, and general
users can take advantage of Web 2.0 websites.
Acknowledgements
The authors would like to acknowledge the work of the individuals of the World Wide Web
Consortium (W3C) who contributed to the Web Content Accessibility Guidelines (WCAG) 1.0
standards . The authors are also grateful to the University of North Texas College of
Information.
References
Archambault, Susan (2002). Independent Samples T Test. Retrieved November 1, 2007, from
http://www.wellesley.edu/Psychology/Psych205/indepttest.html.
Campbell, D.T., and J.C. Stanley (1963). Experimental and Quasi-Experimental Designs for
Research. Houghton Mifflin Compay: Boston.
E-Consultant (2007). 1200+ Web 2.0 Sites in 50+ categories : Web 2.0 Directory :
eConsultant. Retrieved November 1, 2007, from http://web2.econsultant.com/top-web2-
sites-categorized-list.html.
Fetscherin, M., & Lattemann, C. (2007). User acceptance of virtual worlds: An explorative
study about Second Life. Retrieved November 1, 2007, from
http://www.fetscherin.com/UserAcceptanceVirtualWorlds.htm.
Hackett and Parmanto (2009). Homepage not enough when evaluating web site accessibility.
Internet Research 19(1).
Parmanto, B., & Zeng, X. M. (2005). Metric for web accessibility evaluation. Journal of the
American Society for Information Science and Technology, 56(13), 1394-1404.
Section 508 (2007). Retrieved November 1, 2007, from http://www.section508.gov/.
Theofanos, M. F.and Redish, J. (2003). Guidelines for Accessible and Usable Web Sites:
Observing Users Who Work With Screen Readers. Retrieved December 15, 2008, from
http://www.redish.net/content/papers/interactions.html.
Thomson Scientific (2007). ISI Web of knowledge. Retrieved November 1, 2007, from
http://isiwebofknowledge.com.
US News And World Report (2007). USNews.com: America's Best Colleges 2008: National
Universities: Top Schools. Retrieved November 1, 2007, from
http://colleges.usnews.rankingsandreviews.com/usnews/edu/college/rankings/brief/
t1natudoc_brief.php.
W3C (2007a). Web Accessibility Initiative (WAI) - home page. Retrieved November 1, 2007,
from http://www.w3.org/WAI/.
W3C (2007b). Web Content Accessibility Guidelines 1.0. Retrieved November 1, 2007, from
http://www.w3.org/TR/WAI-WEBCONTENT/.