Perfecting your user testing: How to get the result you need, rather than the one you want

What do you need to know?

When embarking on a user testing programme, be it for a new or existing project, you first need to decide what you need to know.

This may sound obvious, but you need to have this objective in mind throughout the entire process to make sure that the results that you're going to get are going to be useful. This could be as simple as making sure that rebranding choices are still going to work with your target audience, or that users are easily able to purchase one of your products. Some user testing will focus on easily measurable, tangible and objective factors, whilst others will focus on more subjective concepts - your testing process needs to reflect that and what you want to achieve.

If you don't take that time to do this and go straight to testing, what we have found is that whilst you will of course get data off the back of it, but this may not be actionable, relevant or useful.

Tailor the tests to your objectives

There are many different types of testing, from surveys and focus groups, through to to face-to-face user testing sessions. There are pros and cons to each of them, and factors such as the turnaround time, quality of the results and of course, cost will all play a role in your decision making process.

If what you're trying to find out is feedback, such as customer satisfaction or willingness to recommend your product, you will need a large set of respondents and this may therefore suit a survey-based approach. This is what we call quantitative data and it's great for finding that out such easily measurable data. It dones, however, lack the ability to dig further into the answers that come from this approach.

On the other hand, if you're testing something like a checkout process of your store; a process that is much more involved for your users, then you may want to use a qualitative testing process like remote testing or face to face interviews.

With these qualitative testing methods, you tend to require a much smaller set of users to do your testing but you also need to spend a lot of time taking these users through the process and analysing the data that comes back from the session. You also need to be careful to recruit the right people to take part in the test.

Don’t contaminate your results with leading scenarios

With any type of user testing, you'll need to make sure that you come up with a list of questions or tasks that will help you answer that first initial question of "what do I need to know?".

This can be more difficult when creating tasks for user testing sessions, as you also need to make sure you don't give anything away in the task which may contaminate your results. It's extremely important that you don't ask leading questions or that you don't create tests that direct your test subjects towards a particular action.

Our advice to help with this is to try and think of actionable tasks for them to take, but present them to them as scenarios.

An example of this would be "you want to return an item", rather than "find and go to the return to policy page". One thing that you'll want to be aware of though is how many tasks or questions you're going to ask them to go through. One thing that we found is that when this is more exhaustive, you may find the engagement from your participants drop - particuarly if it's an online survey.

Find the right people

At this point you have decided what you want to achieve from user testing, what type of tests you want to do and you will have your questions and tasks all prepared. Now you need the people to do the testing.

If you are planning on doing an on-site survey, that's going to be your audience pretty much chosen for you. However, if you're going to do any other type of testing where you have to recruit people, you need to make sure that you get them from the right audience. For example, if your site or app is target purely at people in the medical field the opinion of a teacher or a plumber isn't going to be as helpful.

Not every test is that specific of course. However, when you need a certain type of person we recommend working with a recruitment agency to help source the right person for the test you are conducting.

The criteria doesn't need to just be about what industry the test subject works in. It can be based around almost any demographic, behavioural or attitudinal factor. The more specific you are, the higher the recuitment costs are likely to be, but this could be a worthwhile investment to ensure that your testing best reflects the eventual end user.

Ask questions, don’t answer them

This can be the hardest part of user testing and in many cases, it is where user testing can really start to provide results that don’t always reflect the reality. We've already talked about how you need to avoid giving away instructions in your tasks and questions, but this can be a lot harder when performing the test itself.

Now if the user is struggling with a given task, you must resist the urge to give them pointers no matter how tempting it is.

You should have users think out loud throughout the entire process. This will give you the most information about what they're thinking throughout the task and even if they're going down an incorrect route, it can actually give you some of the most useful information as you find out things that are wrong with your site or service.

A technique that we found useful in these situations is to try and answer questions with other questions. An example of this is "should I go here?". Instead of answering that directly, why not try answering "what makes you think that you should go there?".

And don't be afraid if there are moments of silence, especially on more difficult tasks. Give the user a moment let them prepare and then ask them what they are currently thinking.