Richard Griffiths - Lecture Notes
Introduction to Usability Evaluation
Introduction: Evaluation in User Centred Design ProcessesEvaluation is an important part of any user-centred HCI design process (e.g., GUIDE ), where it is used to:
There are both formative and summative aspects to evaluation.
- To identify usability problems.
- To assess whether the GUI design satisfies usability requirements.
- To evaluate whether the GUI design will be usable in practice by its intended users.
1 is carried out throughout the process to help form the design.
2 and 3 occur towards the end, and become summative in that they assess the success of the whole design exercise.
Methods and Techniques
- analytic evaluation
- expert evaluation
- observational evaluation
- survey evaluation
- experimental evaluation
- no need to build prototype
- no need to arrange user testing
- can be very time consuming
- often require specialist psychological knowledge
- don't tell us anything about errors or learning behaviour
Expert mustn't be part of the design team
Need interface description, task description, user model
Structured vs. unstructured reporting
Predefined categorizations, e.g. Heuristic Evaluation
- efficient, quick, rich source of comments
- often source of solutions as well as problems
- experts have biases
- experts are not real users
Direct observation (but Hawthorne effect)
Video recording with playback, participative or not
Verbal protocols: think aloud, question asking
Wizard of Oz
- real users
- interference with performance
- post-task rationalization
Example: anywhere where evidence is needed of e.g. recall of commands, planning, understanding of operations, messages, level of performance of system and user, etc.
- often labour intensive
InterviewsStructured and sequenced form filling
Unstructured, topics to cover, but no fixed sequence
- can obtain in-depth response from user
- can enable new issues to emerge
- time consuming (expensive)
- requires training and skill to carry out
Closed questions --> rating scale
Multipoint scales, including Likert Scale
Semantic differential scale
- cheap to administer to a large number of users (but need to motivate user to return it)
- easy to analyse - unless unstructured responses are allowed
- time and skill required to develop the questionnaire (or commercial set can be used - expense)
- will only uncover what is looked for
Can the hypothesis be stated in a way that can be tested?
Statistical analysis to check reliability of results
- reliable results
- need specialist knowledge
- resources needed to set up experiment
- can't be used for every design decision
ReferencesRedmond-Pyle, David & Moore, Alan Graphical User Interface Design and Evaluation: A Practical Process Prentice Hall 1995
This page is maintained by Richard Griffiths and does not necessarily reflect the official position of the University of Brighton.