When it comes to evaluating the usability of an interface, user testing is often considered unnecessary if an expert has already reviewed it. Since people rarely behave the way you expect, an expert can find major usability problems, but usability tests always reveal surprising issues.
Usability testing and expert reviews are both useful and tend to have different findings, therefore it’s usually recommended to combine the two in order to get the most comprehensive analysis of the interface.
Why an expert review can’t substitute usability testing
- Users’ behavior is often hard to predict, even for professionals. According to Stephanie Rosenbaum, “real users always surprise us. They often have problems we don’t expect, and they sometimes breeze through where we expect them to bog down.”
- Since experts are rarely members of the target group, they sometimes “miss the real problems that cause users to fail tasks. This can be especially true when the target audience has a particular skill set.” See a comprehensive comparison of the two methods.
- A usability test can reveal much more than how usable a site is – it can also demonstrate the “users’ emotional response to the brand, statement of business purpose, graphics, long- and short-term messaging, competitive position, sales path, and more”, Tedd Follansbee explains at UX Matters.
- They have different purposes: expert reviews focus on general usability standards while a usability test is better at finding issues related to special domain knowledge and actual user tasks. Often expert reviews should come before usability tests.
- Empirical data is always better than guessing, Jakob Nielsen says. He believes that any empirical fact “improves the probability of making correct UI design decisions.” His article Guesses vs. Data highlights a few case studies showing that even experts are very inefficient when it comes to predicting how users will react to a given interface.
- Plasq co-founder Keith Lang says that thinking that experienced designers do not have to test their products is one of the greatest misconceptions in web design. According to him, even the most successful and experienced designers confirm that “user testing is the absolute key”.
- When a design decision has to be accepted by many stakeholders, an expert review might seem another disputable opinion while data based on usability test results – often including metrics – is seldom questioned.
Expert reviews have their benefits and purposes, too:
- When a quick overview of the interface is required, “expert reviews are especially useful for finding violations of usability standards and best practices”, Jim Ross advises.
- Expert reviews are a common usability method because they’re relatively cheap and quick. Inspecting a site and writing a report might only take a few days and, in addition, many still think that a usability test needs a big budget.
- It’s always recommended to do an expert review before usability testing an interface, not only to avoid exposing the users to obvious usability mistakes, but also to “determine what to focus on during testing. You can do an expert review to find the obvious problems, allowing usability testing to find and validate the more important problems”, a UXmatters article suggests.
- The scope of a review can sometimes be wider than that of a test. According to Jim Ross, “an expert review can be more thorough and evaluate more parts of a user interface than in usability testing, finding a greater number of problems, because testing is usually limited in time and scope, focusing on certain tasks and parts of an interface.”
Why is it hard (even for experts) to predict how users will react?
- …because people are not rational,
- …because people are not like you,
- …because people sometimes will not use your product the way you imagine,
- …and because even people themselves cannot tell you what they’ll like.