Thinking inside the right box: Developing tasks for usability test participants

One question I get in workshops on usability testing is How do I get participants to do the tasks I want them to do?

On further discussion, we find (the attendee and I) that this question is really asking two things:

• How do I use usability testing to exercise the design?
• How do I motivate users to try things I want them to try?

Thinking outside-in versus thinking inside-out

Teams get to a point where they have to make decisions or settle disagreements about which direction to go with a design. The natural – and good – thing is to go to the users and collect a little data. You have this thing you want to test. This is thinking from the inside out, from the point of view of the system or design you’re working on.

Users have goals they want to reach. So, you have to think from their point of view – that is, from the outside, looking in.

It’s easy to get caught up in asking test participants to try particular design features without fitting that trying-out into a realistic situation for the participant. Teams do it all the time. Here’s example of inside-out thinking in setting up a task:

Task for the participant from the designer’s point of view: We’ve added a map to our search so you can see where our product outlets are. Here. Try it out and tell us what you think.

You watch and listen, but what happens? The data is about a reaction to something that is out of context of use. Here’s a response from a test I did in a similar situation:


Participant response:

That’s cool. I like the idea of having a map. This one looks good. But I don’t know that city, so I don’t know what these locations are in relation to. Hmm. And look, the little numbers in the bubbles show me… something. What are they numbered from? What makes one number 1 and another one number 10? When I hover over those bubbles, it shows me more information, but I can’t see the other locations on the map now.

What do you do now?

Compare that situation to this.

Task scenario for the participant from a user’s point of view: Ever had problems with your cell phone? Okay, imagine that you’re in a city other than the one you live in. You’re there visiting your family (insert appropriate occasion here), so you don’t know where the stores are where you could take your phone to be fixed or exchanged. But you do have access to the Web. What do you do now?

Participant response:

Man, I’ve had that happen. First, I went to the site for the company I get service through. This isn’t my computer so I don’t have bookmarks set up to go to my account. Okay, so I type in the main site address and then I look for some way to find retail locations. I’m on the site now, but I’m not seeing what I’m looking for. Do I have to log in first? No, that would be stupid. They wouldn’t make me do that, would they? Where’s the link for stores? I swear I’ve used it before. Huh. Oh, here it is at the top. It’s a tiny link. Click. I’m there. Great. I’m going to enter my mother’s zip code to see where the stores are near her. Woo! I got a map. Cool. I can instantly see where there are locations within range. I don’t know the neighboring towns very well, though, so I’ll have to zoom in at some point. Hmmm. What’s the address of the nearest store? I need directions now…

See how much richer that data is? Let us deconstruct what’s going on here.

The task scenario sets up a situation that the participant can relate to and you hope he’s motivated to do, and leaves it open-ended. (You can adjust the scenario to fit the participants’ experiences and motivations and still get consistent data across participants.) This way, you can see much more natural behavior, thought processes, and performance. And you get some seriously cool stuff along the way.

First, you learn that the participant goes to the site enough to bookmark it in the browser. You also got help with your information architecture, in trigger words for labels: “retail locations” and “stores.” The participant is telling you the vocabulary he uses to articulate the task goal. You want to use those words in your interface and search terms.

Next, you observe that the way to get to store locations on the site isn’t immediately obvious because the participant doesn’t see it right away and wonders if he has to log into an account. This may be an artifact of the task, or it may be a design issue. Over a few sessions, you should be able to tell whether the position, size, design, or proximity of the widget should be changed.

Then, he enters a zip code to get to a map. That works. The participant’s interaction with the map tells you that he grocked it right away. At a glance, he got what he needed and it made sense. Yay for you!

Now he’s using the map in his context to reach his goal. You can use what happens next to further refine your design.

It’s okay to start out thinking about tasks by localizing test scenarios to certain areas of the design as long as you turn them around to look at the localized design problems from the larger point of view of the user – from the outside, in.