How Many Questions Can I Ask During a User Test?

I get asked all the time, “how many questions is a good number to ask in a user test?”

The answer is actually time based, not quantity based.

Lean ser Testing

The 5 minute rule:

Test respondents start to lose focus after 5 minutes¹. If you are running the usability test in person, you can prompt the test respondent with a new task to keep their focus. But if the test is run remotely, then the 5 minute rule is important.

Why is this important?

If your remote user test is longer than than 5 minutes, the test respondents will begin Satisficing², which means thinking less about the responses. Tweet this

How to get more bang for the buck:

The 5 minute clock starts once the test starts, NOT once the questions start. So if interacting with a prototype/site takes 3 minutes, then you only have 2 minutes for follow-up questions. The good news is, you can control the quantity of questions that fit into time that window with simple tactics. For example:

Do this:
Option 1
Do NOT Do This:
Zrzut ekranu 2014-10-06 15.56.22

Questioning Best Practices:

  1. Make your questions short
  2. Limit the response options to only the critical choices
  3. Limit the text of the response options
  4. If doing remote user testing, test one specific use case at a time. Do not try to jam 3-5 use cases into a single test unless you are doing the test in person.

 

Conclusion:

The longer a test lasts, the less focus a test respondent has, the less accurate the qualitative responses are. The best remote user tests are under 5 minutes, and have short, specific follow-up questions. Tweet this


¹There are several articles that explain this point. Here is one from Constant Contact: http://support2.constantcontact.com/articles/FAQ/1966

²Source: https://www.surveymonkey.com/blog/en/blog/2011/02/14/survey_completion_times/


No Comments

Leave a Comment

Only registerd members can post a comment , Login / Register