To drive customer engagement, a product or feature must both provide value and be easy to use. We covered demand testing in our post: “How to Validate Demand with a Prototype.” This post is about Usability testing.
While observing testing, we have noticed two common mistakes that we want to share in the interest of learning.
Common Usability Testing Mistakes:
Below are two different test structures with the same goal. Try to determine the two differences before I tell you.
The two key differences:
Did you guess the key 2 differences between the tests?
Read below to check…
Difference 1: Phrasing of the “task”
The first big difference between the two tests is how the task is phrased. It’s a subtle difference, but the impact is significant.
- In Test 1, the task phrasing is very prescriptive. We even use the exact names that are in the product.
- In Test 2, the task phrasing is use case driven, ie “Share your thoughts on a project.”
Why is this important?
The reason the phrasing of a task is important is because new users come to your product to solve a problem that they have. The solution that you provide and how you label the path to that solution is what you are testing with a Usability test. However, if during your test, you “lead the witness” by being overly prescriptive with your task, the results will be tainted. Test 1’s task phrasing basically tells the user where to click. Thus, it would be unclear if a person is able to complete the task because the UX is intuitive, or because they just looked for buttons with the label “notepad” and “add note” since we used those words.
On the contrary, the task in Test 2 is phrased how a user would normally think..ie. a problem to be solved. In this case, the problem to be solved is “adding my thoughts on a project.” Therefore, successful navigation of the flow and completion of the task by a test respondent can give us confidence that the feature UX is intuitive.
Conclusion: Task phrasing for a usability test should be use case driven. Avoid being overly prescriptive when you phrase a task, or you will “lead the witness.” Tweet this
Difference 2: The entry page
In our fictional app, the “add note” feature is on the “notepad” page. But since most of our users in our app spend time in the “current project”, we really need to test whether navigating from the “current project” page to the “notepad” page to add notes is intuitive to our customers.
This is especially important because the user task is to “share your thoughts on a project” as we noted above. By starting on the main page, we can test if the user intuitively thinks to navigate to the “notepad” when she wants to complete the use case of “sharing thoughts.”
If the entry point of a usability test starts the user on the page with the activity (as we did in Test 1), then you are not testing the user’s ability to navigate to the feature properly. You are simply testing if a user can do an activity on that specific page.
Conclusion: A good usability test not only measures the ability of a user to complete a task, but also tests a user’s ability to complete common navigation flows to that task. Tweet this
Here is how I would structure a usability test on this feature:
- Who: Since this is a new feature within an established product, I would test on actual customers.
- How many: 5 customers per start page (see below)
- When: At the prototype phase
- Where: I would do this remotely to see if my customers can naturally do this before I took the step to schedule an in-person usability test.
- Test intro: “We are planning to add a feature to the ToDo app to let you to share your thoughts on a project. Here is a prototype of that feature. Please use this prototype to share your thoughts on this project with your teammates.”
- Test start page: I would run a usability test that started from the Current Projects page, the Archived projects page and the Timeline page as those are the three core navigation flows that my customers will have. I would NOT run a test starting on the Notepad page.