usability evaluation of an e-service in a public website372137/fulltext01.pdf · orebro university...
TRANSCRIPT
OREBRO UNIVERSITY
SWEDISH BUSINESS SCHOOL INFORMATICS
Usability evaluation of an e-Service in a Public
website Project work 2010
Author
Vijay Pratap Paidi
Supervisor
Hannu Larson
Examinator
Anders Avdic
Date
22/4/2010
I
Abstract
Enough research was not done in evaluating a public website which would provide e-services
to all kinds of citizens. A public website would provide e-services to all kinds of people and
also provides the services faster and easier with fewer errors, so it is necessary to provide a
usable website to the users. So usability evaluation of an online payment service in a public
website is described in this paper.
Usability evaluation has been carried out by performing heuristic evaluation which is a
usability inspection method and interview which is a usability testing method. A conceptual
framework was developed to identify the nature of reality and data matrixes were used to
present the data gathered, by performing the methods.
The usability evaluation performed by heuristic evaluation and interviews had identified
usability problems from both non-user and user point of view. Using both the methods, more
number of usability problems were identified which are necessary for the improvement of
online payment service in a public website.
Keywords: Usability evaluation, usability heuristics, usability inspection, usability testing.
II
Acknowledgement
I would like to thank the users of website who participated in the interviews, evaluators who
performed heuristic evaluation to find usability problems in the website and my supervisor
who guided me in writing the paper.
1
Introduction
1 Background
Usability as defined by ISO/IEC 9241 standard is “the extent to which the product can be
used by specified users to achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use”. As the usage of web has increased worldwide, it is
important to maintain the usability characteristics of a website (Liu, 2008).
A public website would provide e-services to all kinds of people and also provides the
services faster and easier with fewer errors, so it is necessary to provide a usable website to
the users (Lili, Stuart, & Jon, 2005).
E-seva is one of the e-governance initiatives taken by the government of Andhra Pradesh in
India; the main theme of e-seva is to provide a one-stop venue for various services by the use
of Information Communication Technologies (ICT’s). It provides different services through
computerized Integrated Citizen Service Centres (ICSC’s). These centres are located in
different parts of the state. Internet services are also provided by e-seva and these internet
services comprises of online payment, downloading of forms, government orders and online
filling of applications on the web. (Prabhu, 2004) These online services are only provided in
Hyderabad, which is the capital city of the state and they are provided through (esevaonline,
2001) which is a public website.
Online transactions and online services are one of the important stages in developing e-
government. This change the way people interact with their government, it saves time and
money for public and government. (Layne & Lee, 2001) E-Government is the use of ICT’s to
improve public services and democratic processes (EU, 2004). So the usability evaluation of
online payment service of e-seva is evaluated in this paper.
1.1 Objective and research question
The objective is to evaluate the usability of online payment service in a public website. The
public website referred to is the e-seva website. The main research question is: How is the
usability of an online payment service in a public website?
1.2 Operationalization of research question
How is the usability evaluation using an inspection method
How is the usability evaluation using a testing method
How to identify different usability problems
1.3 Literature review
There has been previous research done in this field, but most of the research is done in the
private sector rather than the public sector. Usually the investment on a web-based e-
government service would be enormous and the services provided by the government would
be used by all the citizens. Despite the importance of the public e-services little research has
been done in this field. (Lili, Stuart, & Jon, 2005)
2
As a public website would provide services to users with different gender, age, education,
career, income, literacy, etc, the website should follow some criteria and it should be usable
to all kinds of users. (Lili, Stuart, & Jon, 2005)
Usability evaluation was performed by Heuristic evaluation in many papers as this is widely
used and the most efficient one (Otaiza, Rusu, & Silvana, 2010). If a usability evaluation is
carried out by relying on any single evaluation method, it will be misleading and incomplete.
A multidimensional web evaluation strategy can be used to successfully evaluate web-based
e-government service. The usage of several web evaluation methods will achieve
multidimensional web evaluation (Wood, et al., 2003). So in this paper, usability evaluation
is carried out by using one of the usability inspection methods and one of the usability testing
methods.
The literature search was carried out using online databases like ACM digital library, IEEE
explorer and Google Scholar. Keywords like usability, usability evaluation and different
synonyms of the keywords were used to search for the literature. The title and abstract was
checked for every article and if the title or abstract fits the purpose of the paper then it is kept
aside for further reading of the article. The method and conclusion were checked for the
articles which are kept aside in order to have a better understanding about usability
evaluation. In this way the literature search was carried out to find related articles about
previous research done in this field.
1.4 Delimitation
The online payment service is one of the e-service provided by the public website and this
would be used by the whole population of the city, but the results of the study will only
reflect the views of the sample size involved and not the views of whole population.
1.5 Outline of the paper
The method section provides how heuristic evaluation and interviews have been carried out.
The result section presents the data gathered from interviews and heuristic evaluation. The
analysis presents the problems into two categories which are major and minor. The
conclusion presents the summary of the usability evaluation and the discussion presents about
any patterns identified in the paper.
Method
2 Why Usability Evaluation
Usability evaluation is defined as “systematical process of collecting data, in order to have a
better understanding of users and how user groups use the product to perform a specific task
under specified conditions” (Sharp, Rogers, & Preece, 2007). Performing usability evaluation
will help in improving the websites on basis of users demand and would meet the need of
users (Liu, 2008). Usability evaluation methods are commonly divided into usability
inspection method and usability testing method which is described in detail in further
sections. (Otaiza, Rusu, & Silvana, 2010)
3
2.1 Usability Inspection method
This method evaluates usability of a system by inspection. The inspection is carried out by
usability evaluators who identify problems in the design. There are different usability
inspection methods where heuristic evaluation is one among them and is efficient in finding
the usability problems. So, heuristic evaluation was chosen as one of the method for finding
usability problems in the paper (Otaiza, Rusu, & Silvana, 2010). This evaluation was
conducted using the ten usability heuristics developed by (Nielsen J. , Heuristic Evaluation,
2005).
2.2 Usability Testing Method
Usability testing identifies the usability problems by performing empirical testing of the
system with representative users, (Otaiza, Rusu, & Silvana, 2010) in this case the
representative users are the citizens who are the users of online payment service. There are
several testing methods, but interviews were chosen for gathering the empirical data in this
paper. Performing interviews is cheap and simple and it gives the usability problems actually
faced by the users of the e-service.
2.3 Conceptual Framework
Two usability evaluation methods have been chosen to perform the usability evaluation in
this study. (Otaiza, Rusu, & Silvana, 2010) The usability evaluation was carried out for one
e-service of the website to be more specific in finding the problems rather than involving all
the services provided by the website.
The conceptual framework which is shown in figure.1 has been originally derived from
(Otaiza, Rusu, & Silvana, 2010). The framework was then modified according to the need of
the paper. The process of usability evaluation has been taken from (Otaiza, Rusu, & Silvana,
2010) and this is applied to an e-service of a public website in this paper. A public website
provides online services and online payment is one of the e-services provided by the public
website. This online payment process undergoes usability evaluation and this is performed by
heuristic evaluation which is a usability inspection method and interviews which is a usability
testing method. Performing both the methods help in identifying usability problems and these
problems can be provided as a feedback for the designers, which can be used for further
improvement of the website (Nielsen, Overgaard, Pedersen, & Stage, 2005).
4
Public Website Online Services
Online Payment
Usability Evaluation
Usability Inspection
Method
Usability Testing
Heuristic Evaluation
Interviews
Usability Problems
Feedback for Designers
provides
consists
undergoes
performs
performs
uses
uses
Identifies
Identifies
provides
Figure.1Conceptual Framework
2.4 Collection of data
Data has been collected by performing heuristic evaluation which is a usability inspection
method. Heuristic evaluation works better with at least three evaluators, which is the reason
why three evaluators were chosen for this study (Nielsen J. , 2005).
The evaluation was performed by evaluators who have a computer background education and
have completed their graduation. These evaluators are not considered experts in usability
evaluation but have some experience in finding out usability problems by performing
heuristic evaluation. They have been given access to the online payment service by providing
the login information as they were not the actual users of the website.
Heuristic evaluation was performed by these evaluators independently, using the ten usability
heuristics which can be found in Appendix-B and different usability problems were noted
down. Textual data is gathered from the evaluators and then the author registered the data in a
separate document for further interpretation.
Data was also collected by performing interviews which is a usability testing method. The
interview questions were derived from the usability heuristics and modified according to
user’s language. Semi structured interviews were conducted for this study in order to ask any
additional questions according to user’s response (Oates, 2006) and these questions are only
related to user’s perceptions about the interface design of the website. The interviews were
carried out using online phone, field notes were taken during interview and the conversation
was recorded using voice recorder. There were total 8 interviews which were done, but only 7
interviews were recorded as a technical error occurred during one interview. It was hard to
avoid technical error, but as field notes were taken during the interview, the data was not
completely lost. This error occurred while recording the conversation with interviewee 4, so
the data noted in the field notes was interpreted and presented in the results.
The interviewees selected for this study were from different backgrounds and occupation, and
they were selected using convenience approach. The interviewees were users of the public
website and are familiar with internet and online services.
5
The interviewees were asked prior the interview that the conversation would be recorded and
it was recorded only with the permission of the interviewee. The recorded data was then
transcribed and registered in the form of textual data which makes it easy for further
interpretation. The time taken for the interview was also informed prior to the interviewee.
The anonymity of the interviewees was maintained and the registered data was kept
confidential. (Oates, 2006)
2.5 Analysis
The data matrix is used as an instrument for the analysis of the collected data. The idea of
data matrix has been derived from (Oates, 2006), which is used for analyzing qualitative data.
The transcribed data is interpreted and presented in the data matrix for analysis. The data
collected from heuristic evaluation and interviews was presented in two different data
matrixes. The data matrix which is used to present the interview data is shown in Table.1, it
consists of elements as the interview questions and these questions are represented as codes
in the data matrix. These codes are presented against the interviewees and the data gathered
from each interview is presented under different interviewees. The interview questions and
the codes representing the interview questions can be found in Appendix-A.
The data matrix used for presenting the data gathered from heuristic evaluation is shown in
Table.2. This data matrix consists of elements as usability heuristics derived from (Nielsen J.
, Heuristic Evaluation, 2005) and the data is presented against different evaluators. The
usability problems identified are presented under each evaluator against different usability
heuristics.
The data presented in the results is analyzed into two categories which are major and minor.
A usability problem is considered major if they have serious potential for confusing users or
causing them to erroneous use of the system, (for example if a user can’t reset the changes he
made during a process and there is no emergency exit then this would cause loss of
information and user strain, which is a major problem) while minor problems are those which
slow down the system interaction unnecessarily (Nielsen J. , 1992) (for example, irrelevant
information in the website makes difficult for the user to find what he/she needs and
slowdowns the user actions, which is a minor problem )
Result & Analysis
3 Result
The interpreted data from the analysis of the transcribed data is presented in Table 1. The
interpretation was done in order to identify the usability problems from the transcribed data.
The data gathered by performing heuristic evaluation by 3 evaluators is registered as textual
data and this data has been interpreted to identify the usability problems in the textual data.
This interpreted data has been presented in Table.2.
6
Table.1 Interview data
Elements Interviewee#1 Interviewee#2 Interviewee#3 Interviewee#4 Interviewee#5 Interviewee#6 Interviewee#7 Interviewee#8
[A]
-Accountant
-Not a regular
user of
internet
-
Undergraduat
e student
-Regular user
-Software
employee
-Regular user
-Software
employee
-Regular user
-
Undergraduate
Student
-Regular User
-Jr.Engineer
- Regular User
-Software
employee
-Regular user
-
Undergraduat
e Student
-Regular user
[B]
-It is ok -Not clear
-No
hierarchical
design
-It is clear - Alignment
of data can be
improved
-It is clear -No clear
information
-No problem -No clear
information
[C]
-Haven’t
faced any
problems
-Not
clear(mixture
of languages)
-No problem -Font size can
be increased
-Easy to
understand
-Language is
understandable
-It is easy to
understand
-It is more
technical,
difficult to
understand
[D]
-No Back
option and the
home button is
annoying
-No back
option, No
emergency
exit. Cannot
reset changes
-Home button
is problematic
-Home button
is problematic
-No back
option, home
button is
problematic
-No back
option, home
button is
problematic
-Navigation is
difficult in
some pages,
home button is
problematic
during
navigation
-No
emergency exit
in every page
-Cannot undo
and redo
changes
-Home button
is problematic
as it logouts
the user
[E]
-No problem -Tabs in
homepage are
confusing
-Clear
Information
-No proper
information of
the bill
-No problem -Inconsistent
wording in
home page
among tabs,
few data
unclear in
registration
-No problem -It is not user
friendly
[F]
-No User
confirmation
procedures
-Technical
errors during
payment
-No error
prevention
-No error
prevention
-No proper
password
recovery
option
-No user
confirmation
during
payment
process
-No errors -No errors -No errors -No error
prevention
-Cannot reset
changes made
during
payment
process
[G]
-Actions are
easy to
perform
-Problem in
navigation, not
user friendly
-Easy to use
no need to
remember
-Can increase
the user
friendliness of
the website
-Easy to use,
no need to
remember
-Easy to
operate
-No need to
remember
anything
-It is easy to
use, no need to
remember
[H]
-No problem -Works well
with only
internet
explorer,
problem with
choosing pop
down menu
-No problem -No automatic
payment
-Cannot pay
multiple bills
at a time
-Should check
the bank
statement
every time
-It is efficient -Difficulty
with pop down
menu in
homepage
-Can include
more banks,
can increase
services
-It is ok but
can improve
more banks for
online
payments
-Selection of
areas is
confusing and
navigation is
little hard
[I]
-Information is
aligned
properly
-Contains
irrelevant
information
and is better to
divide the
payment
process
-No problem -Not much
irrelevant
information
-Have
irrelevant
information in
the home page
-Little
irrelevant
information
-No problem -Redundant
information is
provided in the
home page
which is
confusing
[J]
-No recover
of errors
during
transaction
-It doesn’t
help to recover
errors rather
gives error
number
-No errors
faced till now
-Haven’t
faced any
errors for
recovery
-No errors
faced
-No errors
faced till now
-No errors
faced till now
-No errors
faced till now
[K]
-Did not find
any help
section
-No help
provided
-No help
provided
-No proper
help section
except a demo
-No help
provided
-No help
provided
-No help
provided
-No help
section
provided
7
Table.2 Data from Heuristic evaluation
Elements Evaluator#1 Evaluator#2 Evaluator#3
Visibility of system
status
-No feedback within reasonable time
during the web process
-The system status is mentioned
during the navigation of the website
-No proper feedback of the website
-No proper alignment of data
Match between
System and real
world
-The website uses acronyms which
might not be understandable for everyone
-The language is easy but in some
cases has a mixture of languages
which might be confusing
-The language uses acronyms which
might not be easily understood
-Font size can be increased
User Control and
freedom
-The system does not support undo or
redo changes
-The home button makes the usage of the
website problematic
-Not completely user friendly
-Lack of undo and redo changes
-No emergency exit
-Home button is problematic for
navigation as it makes the user logout
-No emergency exit
-No undo redo changes
-Pressing home button makes the
user logout
Consistency and
standards
-It does not provide information about
the services
-The structure and layout is not coherent
-The website can be more informative
about the services provided
-Similar words which confuses the
user in the homepage
-No proper information of the bill
Error prevention
-Does not prevent errors during payment
process
-Should always check the bank statement
for the confirmation of payment
-It can lead to errors during the
payment process. It does not show the
amount to pay
-It does not provide with user
confirmation during the payment
process
-Cannot login by pressing enter after
entering the username and password
-No error prevention
-Gives errors with different browsers
during login
-Services not clearly defined which
might lead to error
Recognition rather
than recall
-The user can be confused during
navigation due to different menus
provided in the website
-Navigation is hard during adding
different services
-It is easy to navigate no need to
remember
Flexibility and
efficiency of use
-The website appearance changes in
other browsers
-The website works well only with
internet explorer and does not work
well with other browsers
-The options under pop down menu in
the home page of the website is hard
to click sometimes
-Cannot pay multiple bills at a time
Aesthetic and
minimalist design
-Irrelevant information is provided which
is disturbing in the website
-User confirmation is lacking during the
transaction process
-It provides with some irrelevant
information which leads to confusion
-The tab buttons not properly aligned
in the website
-Irrelevant information diminishes
the user action
Help users, diagnose,
and recover from errors
-There is no help to recover from errors,
the errors are not informative
-Few errors provide numbers and do not
give information to avoid the error
-No recover of errors, error
-It doesn’t mention that the website
only works well with internet explorer
-Provides error numbers which does
not help recover errors by the user
- It doesn’t help in recovering errors
-Gives error codes which are of no
use to the user
-No proper password recover option
Help and
documentation
-No help provided in the website, no
descriptions of the services provided
-The trouble shooting is poorly designed
-There is no help section provided in
the website but provides a demo
which has to be downloaded and
played by the user which is a poor for
m of help
-No help section provided in using
the website
8
3.1 Analysis
Usability evaluation was performed by heuristic evaluation and interviews. All the major
usability problems which are identified by heuristic evaluation have been confirmed by the
users through interviews.
The usability problems identified by performing interviews depends on the interviewee, as
interviewees 1,3,5 and 7 only found few usability problems. All the interviewees were regular
users of internet except interviewee1. Interviewee 1 is not a regular user and uses internet for
only payment of bills. Three interviewees were students and the rest were employees in
different areas. There was no real difference between the problems identified by students and
employees. The problems identified by interviews identified more problems than heuristic
evaluation, in payment process of the website. The identification of usability problems was
different with each interviewee, but all the interviewees faced similar problems in user
control and help sections of the website.
The problems identified by heuristic evaluation are performed by 3 evaluators. All the three
evaluators have identified usability problems in all the ten usability heuristics. These
evaluators identified more usability problems than interviews, in Flexibility and efficiency of
use, Help and documentation and Visibility of System status.
The total number of usability problems found by performing both the methods is 44 and out
of them, major problems identified are 19 and minor problems identified are 25. The usability
problems which are identified as major and minor are presented in Table 3. There were many
similar problems identified in testing and inspection methods but few additional problems
were also identified by both the methods.
There is no help section in the website, even though the usage of the website is not hard, it is
necessary to provide a proper help section. The navigation of the website is also not
convenient and there is no prevention of errors (for instance, the website does not provide the
amount to pay which can lead to errors). The appearance of information in the website can be
improved; these are few problems which are identified by both the methods. By using two
usability evaluation methods, more number of usability problems was identified.
As shown in figure.2 more number of major problems are identified in User control and
freedom, Consistency and Standards, and Flexibility and efficiency of use. These problems
have been identified by using the responses from both the methods.
9
Table.3 Usability problems
Type of problem Description of the problem
No back option provided during navigation
Pressing the home button logouts the user
No User confirmation procedures of the payment process
Sometimes technical errors occur during payment
No proper recover of errors during transaction
No emergency exit during navigation
Cannot reset changes
Works well with only internet explorer
Problem with choosing pop down menu
It doesn’t help to recover errors rather gives error number
Major No proper information of the bill
No proper password recovery for this website
Should check the bank statement every time
Can include more banks and increase number of services
Selection of area for payment is confusing and navigation is little hard
It does not provide information about the services
Does not prevent errors during payment process
The website appearance changes in other browsers
No help section provided
No hierarchical design of the information
Tab menus in homepage are confusing
Lack of user friendliness
Contains irrelevant information and is better to divide the payment process
Alignment of data can be improved
Font size can be increased
No automatic payment of bills for every month
Cannot pay multiple bills at a time
Navigation is difficult in some pages
Inconsistent wording in home page among tab menus
Language is more technical, difficult to understand
No error prevention during login
Minor Redundant information is provided in the home page which is confusing
No feedback within reasonable time during the web process
The website uses acronyms which might not be understandable for everyone
The structure and layout is not coherent
The system status is not mentioned during the navigation of the website
Cannot login by using keyboard after entering the username and password
Navigation is hard during adding different services
The tab menus not properly aligned in the website
The website doesn’t say that it only works well with internet explorer
A demo is provided in the website which is a poor form of help
Gives errors with different browsers during login
Information is not clear during registration and adding of services
The trouble shooting is poorly designed
10
A graph has been illustrated to present the number of major and minor usability problems
identified, for each usability heuristics in figure 2. It has number of problems as x-axis and
usability heuristics as y-axis, where n is 44.
Figure 2: Graph presenting major and minor problems for each heuristic
Conclusion & Discussion
4 Conclusion
Main research question:
How is the usability of an online payment service in a public website?
The usability evaluation performed by heuristic evaluation and interviews had identified
usability problems from both non-user and user point of view. Using both the methods, more
number of usability problems were identified which are necessary for the improvement of
online payment service in a public website.
How is the usability evaluation using an inspection method?
The evaluators had identified usability problems in all usability heuristics and the problems
which are found more in heuristic evaluation than interviews were in Flexibility and
efficiency of use, Help and documentation and Visibility of System status of the website.
0 1 2 3 4 5 6 7
Visibility of system status
Match between system and …
User control and freedom
Consistency and standards
Error prevention
Recognition rather than recall
Flexibility and efficiency of use
Aesthetic and minimalist design
Help users recognize, …
Help and documentation
Number of usability problems
Usa
bil
ity h
eu
ris
tics
Major and Minor problems
Minor problems
Major problems
n=44
11
How is the usability evaluation using a testing method?
Interviews were performed which is a usability testing method, and the problems identified
depends on the kind of interviewee but the occupation of the user did not play any role in
identifying the usability problems. The problems which are found more in interviews than
heuristic evaluation were in the payment process of the e-service.
How to identify different usability problems?
The major usability problems identified by heuristic evaluation were confirmed by the users
who participated in interviews. There were a large number of similar problems identified by
both the methods but few additional problems have been identified by both the methods. The
use of both methods helps in identifying more usability problems than any single evaluation
method.
4.1 Discussion
The problems identified from a usability evaluation can be used as a feedback for the
designers of the website, so the usability problems identified have been divided into major
and minor categories. The major problems are considered the first priority problems while the
minor problems are the second priority problems; this makes it easy for the designers to
identify different usability problems.
The evaluators in heuristic evaluation deliberately search for problems in the website’s
usability while the users of the website who participated in the interviews do not intentionally
search for problems; they only identify the problems faced during their usage of the website.
The interviews give users perception of usability problems. This gives a two-dimensional
way of finding usability problems. There were few problems found more in interviews than
in heuristic evaluation and in the same way there were few problems identified more in
heuristic evaluation than in interviews. Thus there were different usability problems found by
each usability evaluation methods.
The usage of two or more usability evaluation methods was proposed for the usability
evaluation of public websites and this case confirms that more information about usability
problems can be achieved even by the use of two evaluation methods. This also shows that
more number of usability problems can be identified using two usability evaluation methods
than any single evaluation method. This kind of usability evaluation has been carried out
before for transactional web applications, but it was applied to a public website in this paper.
The heuristic evaluation should be used as the usability inspection method as it is an efficient
one in identifying the usability problems but different methods can be used from the usability
testing methods other than the interviews. Further research can be performed on various other
public websites using two or more usability evaluation methods.
12
References
Literature
Prabhu, C. (2004). E-GOVERNANCE: Concepts and Case Studies. New Delhi: Prentice Hall of India.
Oates, B. J. (2006). Researching Information Systems and Computing. London: Sage.
Sharp, H., Rogers, Y., & Preece, J. (2007). Interaction design: Beyond human computer interaction.
John Wiley & Sons Ltd.
Articles
Layne, K., & Lee, J. (2001). Developing fully functional E-government: A four stage model.
Government Information Quarterly , 122-136.
Lili, W., Stuart, B., & Jon, G. (2005). Evaluating Web-based e-government services with a citizen-
centric approach. Hawaii International Conference on System Sciences (pp. 1-10). IEEE.
Liu, F. (2008). Usability Evaluation on Websites. IEEE , 141-144.
Nielsen, C. M., Overgaard, M., Pedersen, M. B., & Stage, J. (2005). Feedback from Usability
Evaluation to User Interface Design: Are Usability Reports Any Good? In Human-Computer
Interaction - INTERACT (pp. 391-404). Springer Berlin.
Nielsen, J. (1992). Finding Usability Problems Through Heuristic Evaluation. ACM , 373-380.
Otaiza, R., Rusu, C., & Silvana, R. (2010). Evaluating the Usability of Transactional Web Sites.
Advances in Computer-Human Interactions (pp. 32 - 37). IEEE Xplore.
Wood, F. B., Siegel, E. R., Lacroix, E.-M., Lyon, B. J., Benson, D. A., Cid, V., et al. (2003). A
Practical Approach to E-Government Web Evaluation. IEEE , 22-28.
Websites
esevaonline. (2001). Retrieved 04 02, 2010, from eseva: www.esevaonline.com
EU. (2004). ICT for Government and Public Services. Retrieved 04 15, 2010, from Europe's
Information Society: http://ec.europa.eu/information_society/activities/egovernment/index_en.htm
Nielsen, J. (2005). Heuristic Evaluation. Retrieved 04 08, 2010, from
http://www.useit.com/papers/heuristic/
13
Appendix-A
[A] What is your occupation and how often do you use Internet?
[B] Does the website keep you informed about your position in the web process?
[C] Is the language in the website easy to understand?
[D] Does the website provide you to undo and redo your changes?
[E] Does the website provide you with clear information?
[F] Does the website help you in recognizing your data and reduce errors?
[G] Is the website easy to use and does it reduce your memory load
[H] Is the website adaptable and efficient to use?
[I] Does the website provide you with irrelevant information during your process?
[J] Does it help you recover from errors?
[K] Does it provide you with help, in using the website?
14
Appendix-B
These are the general principles for user interface design and were developed by (Nielsen J. ,
Heuristic Evaluation, 2005)
S.No Usability Heuristics Description
1 Visibility of system status The system should always keep users informed about what is
going on, through appropriate feedback within reasonable time
2 Match between system and
the real world
The system should speak the users' language, with words,
phrases and concepts familiar to the user, rather than system-
oriented terms. Follow real-world conventions, making
information appear in a natural and logical order
3 User control and freedom
Users often choose system functions by mistake and will need
a clearly marked "emergency exit" to leave the unwanted state
without having to go through an extended dialogue. Support
undo and redo.
4 Consistency and standards
Users should not have to wonder whether different words,
situations, or actions mean the same thing. Follow platform
conventions
5 Error prevention
Even better than good error messages is a careful design
which prevents a problem from occurring in the first place.
Either eliminate error-prone conditions or check for them
and present users with a confirmation option before they
commit to the action.
6 Recognition rather than recall
Minimize the user's memory load by making objects, actions,
and options visible. The user should not have to remember
information from one part of the dialogue to another.
Instructions for use of the system should be visible or easily
retrievable whenever appropriate
7 Flexibility and efficiency of
use
Accelerators -- unseen by the novice user -- may often speed
up the interaction for the expert user such that the system can
cater to both inexperienced and experienced users. Allow
users to tailor frequent actions.
8 Aesthetic and minimalist
design
Dialogues should not contain information which is irrelevant
or rarely needed. Every extra unit of information in a dialogue
competes with the relevant units of information and
diminishes their relative visibility.
9
Help users recognize,
diagnose, and recover from
errors
Error messages should be expressed in plain language (no
codes), precisely indicate the problem, and constructively
suggest a solution.
10 Help and documentation
Even though it is better if the system can be used without
documentation, it may be necessary to provide help and
documentation. Any such information should be easy to
Search, focused on the user's task, list concrete steps to be
carried out, and not be too large.