3461 midterm material midterm is short answer everything up through last week’s lecture things...
TRANSCRIPT
3461
Midterm Material
Midterm is short answer Everything up through last week’s lecture Things to study:
Slides Programs Javadoc
No need to memorize all methods of Swing classes. Familiarity with the most common ones may be tested
3461
Midterm Material
Here is an example of a possible midterm question (however this particular question is not on your exam!)
[6 marks] What three different types of layouts for multiple windows onscreen were presented in class? Briefly describe each type.
3461
Evaluation Paradigms
“Quick and Dirty” evaluation Usability Testing Field studies Predictive evaluation
3461
“Quick and Dirty” Evaluation
User-centered, highly practical approach Used when quick feedback about a design is
needed Can be conducted in a lab or the user’s natural
environment Users are expected to behave naturally Evaluators take minimum control Sketches, quotes, descriptive reports are fed back
into the design process
3461
Usability Testing
Applied approach based on experimentation Used when a prototype or a product is available Takes place in a lab Users carry out set tasks Evaluators are strongly in control Users’ opinions collected by questionnaire or
interview Reports of performance measures, errors etc. are
fed back into the design process
3461
Field Studies
Often used early in design to check that users’ needs are met or to assess problems or design opportunities
Conducted in the user’s natural environment Evaluators try to develop relationships with users Qualitative descriptions that include quotes,
sketches, anecdotes are produced
3461
Choose paradigm and techniques
Practical and ethical issues might be considered
Factors: Cost Timeframe Available equipment or expertise
Compromises may have to be made
3461
Decide on ethical issues
Studies involving humans must uphold a certain code
Privacy of subjects must be protected Personal records must be kept confidential Exact description of the experiment must be
submitted for approval
3461
Evaluate the data
Should quantitative data be treated statistically? How to analyze qualitative data? Issues to consider:
Reliability (consistency) Validity Biases Scope Ecological validity
3461
We’ll take a closer look at…
Two predictive evaluation techniques: Heuristic evaluation Cognitive walkthroughs
A usability testing technique User testing
3461
Heuristic Evaluation
Heuristic evaluation is a technique in which experts, guided by a set of usability principles known as heuristics, evaluate whether user interface elements conform to the principles.
Developed by Jakob Nielsen Heuristics bear a close resemblance to design
principles and guidelines Interesting article on Heuristic Evaluation:
http://www.useit.com/papers/heuristic/heuristic_evaluation.html
3461
List of Heuristics
Visibility of system status Match between system and the real world User control and freedom Consistency and standards Help users recognize, diagnose, and recover
from errors
3461
List of Heuristics (cont.)
Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help and documentation
3461
Heuristic for Websites
Avoid orphan pages Avoid long pages that force scrolling Provide navigation support, such a site map that is
always present Avoid narrow, deep, hierarchical menus Avoid non-standard link colours Provide consistent look and feel for navigation Avoid complex URLs Avoid long download times
3461
Cognitive walkthroughs
“Cognitive walkthroughs involve simulating a user’s problem-solving process at each step in the human-computer dialog, checking to see if the user’s goals and memory for actions can be assumed to lead to the next correct action” (Nielsen and Mack, 1994).
Focus is on evaluating design for ease of learning
3461
Cognitive walkthrough steps
Characteristics of typical users are identified Designers and evaluators meet, walk through the
action sequences for each task and try to answer the following questions: Will the correct action be evident to the user? Will the user notice that the correct action is available? Will the user know from the feedback whether they made
a correct choice?
3461
Usability Testing is NOT...
“What type of feedback did you gather from your usability participants?”
“I showed my program to three different people and they all said it looked really, really good.”
3461
It doesn’t matter how useful a program could be if the interface is unusable.
“Not even the most brilliantly conceived and ingenious computer system can do all that it was designed to do- or even a small part of what it was designed to do- unless the brilliance of its operation and purpose is matched by the cunning simplicity of its user interface. (Hicks and Essigner, 1991)
There is a distinction between useful and usable.
3461
Usability Testing- Definition
Usability testing is a method by which users of a product are asked to perform certain tasks in an effort to measure the product's ease-of-use, task time, and the user's perception of the experience. Changes are made to the application or site based on the findings of the usability tests. Usability test participants are encouraged to think aloud and voice their every opinion. Usability testing is best used in conjunction with a user-centered design process, a method by which a product is designed according to the needs and specifications of users.
Adopted from http://searchwebservices.techtarget.com/sDefinition/0,,sid26_gci214526,00.html
3461
Why is Usability Testing Important? (2 of 3) It is impossible to predict usability from
appearance, just like it is impossible to judge a person’s personality on appearance.
Casual “feel good” feedback is inadequate. Formal testing is often the only way problems
are identified pre-release. Problems found once a product is released are usually not fixed unless they are really severe.
3461
Why is Usability Testing Important? (3 of 3) User, Designer, Programmer each have
different models. The designer’s intuition is not always correct. Design standards and guidelines are not
sufficient. Usability testing leads to competitive
advantages and reduced support costs.
3461
The REAL Components of Usability
Relevance - is how well a system serves the users needs.
Efficiency - states how efficiently the users can carry out their tasks using the system.
Attitude - is the users subjective feelings towards the system.
Learnability - is how easy the system is to learn initially and how well the users remember the skills over time
http://www.affectus.se/artiklar/usabilityinsp/
3461
Relevance
number of good and bad features recalled by users
number of available commands not invoked by users
number of available commands invoked by users
number of times users needs to work around a problem
percent of task completed
http://www.affectus.se/artiklar/usabilityinsp/
3461
Efficiency (1 of 2)
time to complete a task percent of task completed percent of task completed per unit time
(speed metric) time spent in errors number of commands used frequency of help and documentation use time spent using help or documentation
http://www.affectus.se/artiklar/usabilityinsp/
3461
Efficiency (2 of 2)
number of repetitions of failed commands number of runs of successes and of failures number of times interface misleads user number of times user needs to work around a
problem number of times the help facilities solve the
users problem
http://www.affectus.se/artiklar/usabilityinsp/
3461
Attitude
percent of favorable/unfavorable user commands number of good and bad features recalled by users number of users preferring the system number of times user loses control over the system number of times the user is disrupted from a work
task number of times user expresses frustration or
satisfaction
http://www.affectus.se/artiklar/usabilityinsp/
3461
Learnability (1 of 2)
ratio of success to failures (over time) time spent in errors percent or number of errors number of commands used frequency of help and documentation use time spent using help or documentation number of repetitions of failed commands number of runs of successes and of failures
http://www.affectus.se/artiklar/usabilityinsp/
3461
Learnability (2 of 2)
number of available commands not invoked by users
numbers of features or commands that can be remembered after a test
the proportion of users using efficient strategies compared to those using less efficient strategies
number of logical errors made
http://www.affectus.se/artiklar/usabilityinsp/
3461
Nielson’s Usability Severity Ratings
0 - Not a usability problem at all
1 - Cosmetic problem only - need not be fixed unless extra time is available on project
2 - Minor usability problem - fixing this should be given low priority
3 - Major usability problem - important to fix, so should be given high priority
4 - Usability catastrophe - imperative to fix this before product can be released
3461
Usability Frequency Ratings
0 - Problem did not occur
1 - This problem occurs rarely- only once/under very unusual circumstances
2 - This problem occurs occasionally - under less commonly occurring conditions
3 - This problem occurs often during common tasks
4 - Problem occurs every time under all tested conditions
3461
What information to provide?
Give a brief explanation that the participant’s involvement is to solicit user feedback. Any problems are the fault of the software.
In real-world situations explain confidentiality agreement, liability legalities, and that participant is free to leave at any time (and still get paid).
Provide instructions as to the user’s task but not explanations of the software.
3461
When should usability testing be conducted?
Ideally, as early as possible (from the prototyping stage) and then repeatedly throughout the development process.
“Test early and often.”
3461
Low Fidelity Prototyping
Hand sketches and scenarios (storyboards, scene by scene) where the focus is on the design, not on the interface mechanics.
Usability testing can be conducted using hand sketches before any code has been written.
Can be used early in the development process Fast to modify and iterate Can be used to define requirements
3461
What to User Test?
Possibilities include: Conformance with a requirement Conformance with guidelines for good design Identification of design problems Ease of system learning Retention of learning over time Speed of task completion Error rates Subjective user satisfaction
Galitz, W. O., (2002) The Essential Guide to User Interface Design, 2nd Edition, Wiley Computer Publishing, New York, NY.
3461
What questions to ask during a usability session? Depends on which phase of the development
cycle- what can be changed? Conceptual model? Layout? Fonts?
There should be a list of questions of the major design issues that is prepared in advance. There should be specific questions that the usability testing is designed to answer. Usability testing has specific objectives.
3461
What is the Goal of Usability Testing?
Usability testing should be designed to determine if the software is meeting the
Qualitative Usability Goals Quantitative Usability Goals
Adapted from Mayhew, Deborah J. (199) The Usability Engineering Lifecycle
3461
Qualitative Usability Goals- Examples
The design must support users working in a high-interrupt environment, with lots of context information on screen to remind users where they are when they get distracted.
The design must support very infrequent users of a very complex task. Thus, it must be self-explanatory, easy to learn and remember.
Adapted from Mayhew, Deborah J. (199) The Usability Engineering Lifecycle
3461
Quantitative Usability Goals - Examples
Experienced users (defined as users who have performed the task five times in a training session) should take no longer than 15 seconds minutes on average to address an email.
Novice users (defined as first-time users) should take no longer than three minutes to complete the registration input form.
Adapted from Mayhew, Deborah J. (199) The Usability Engineering Lifecycle
3461
When to ask questions?
If you are worried about interrupting the task flow of your participant, then ask the question after the completion of the task.
If you are more worried about the participant forgetting their current thought process than interrupting, then ask right away.
3461
Heuristic Evaluation
A heuristic evaluation is a detailed evaluation of a system by interface design specialists to identify problems.
Required reading: http://www.useit.com/papers/heuristic/heuristic_evaluation.html
3461
Thinking Aloud Protocol
During a usability test, instruct participants to verbalize their thoughts aloud protocol. The usability testers prompt participants by asking direct questions about the software, in order to understand their mental model of the system and the tasks, and where they have trouble in understanding and using the system.
3461
Co-discovery Method
During a usability test, two participants perform tasks together while being observed. In order to increase the amount of communication to gain insight to their thought process, one participant is assigned the mouse and the other the keyboard. They are to help each other in the same manner as they would if they were working together to accomplish a common goal using the product.
3461
Usability Testing Sequence of Events (1 of 2)1. Know your Purpose. There should specific
questions which need answering
2. Find Representative Users. This is a non-trivial task, whose difficulty is often overlooked in texts on usability. The quality of the participants make or break the success of the usability session.
3. Watch & Learn. Don’t lead the user to give the answers you want to hear. Be prepared to learn new things.
3461
Usability Testing Sequence of Events (2 of 2)4. Report the Data. The data collected and
observations made must be communicated to the developers and to management.
5. Back to the Drawing Board. It is not enough just test- time must put aside to correct the problems found by tests.
Adapted from http://keith.instone.org/howtotest/introduction.html