Showing posts with label planning. Show all posts
Showing posts with label planning. Show all posts

Monday, February 28, 2011

Involving older adults in design of the user experience: Inclusive design

Despite the reality of differences due to aging, research has also shown that in many cases, we do not need a separate design for people who are age 50+. We need better design for everyone.

Everyone performs better on web sites where the interaction matches users' goals; where navigation and information are grouped well; where navigation elements are consistent and follow conventions; where writing is clear, straightforward, in the active voice, and so on. And, much of what makes up good design for younger people helps older adults as well.

For example, we know that most users, regardless of age, are more successful finding information in broad, shallow information architectures than they are with deep, narrow hierarchies. When web sites make their sites easier to use for older adults, all of their users perform better in usability studies. The key is involving older adults in user research and usability testing throughout design and development.

There are some important considerations in working with older adults in studies. Remembering the points below will ensure that you and your participants have a good experience and you get the data you need to inform design decisions.


Finding participants: Understanding older adults before you recruit

In many places in the world, older adults outnumber people in other age groups. The question is, how do you find the right people to take part in studies? They can be difficult to get to.

We found that these approaches did not work well:

  • Community web sites, message boards, or chat sessions. The oldest old tend not to take part in these groups, so posting ads in those places is not a fruitful way to find participants.
  • Senior centers and community colleges. These are places that offer classes in using computers. If you want computer and web novices for a study, they might be good places to find appropriate participants. They are not good sources if you want to observe people with enough web experience to see them working at a web site without teaching them.
  • Flyers at a senior center, when they did not make clear that we were recruiting for a study. Many older people are much more cautious and skeptical than younger people. They are often fearful of being cheated or "taken." For example, we had put up flyers at a senior center from which we got no response; later we learned that people thought we might be trying to sell them something.
  • Cold calling from a database. This is probably again because older people are afraid that they may be scammed into buying something.


These ideas did work well:

  • Calling with a personal connection. If we could say that a mutual acquaintance had suggested the contact, potential participants were much more receptive to hearing about the study and considering taking part. It is important to establish credibility and trust with the potential participant.
  • Being careful in the initial call to say where we had gotten the contact information and that we weren't selling anything.


Recruiting older adults

Recruiting participants in their 70s and 80s is more difficult than recruiting participants in their 50s. The oldest candidates are less receptive to strangers phoning them, and they don't check email as frequently.

Recruiting by phone. Phoning is important. Plan to phone potential participants at least once (or have your recruiters do so). You need to quickly establish credibility and trustworthiness, to assure potential participants that you are not selling anything, and to establish a connection by letting them know where you got their names. When you can do that, potential participants are often glad to hear from a real person. It is also easier for them to determine legitimacy and to ask questions about the study on the phone. They'll use your answers to help them decide whether they want to take part.

Another reason for phoning potential participants is so you can judge their English language skills and whether they are hearing impaired. (You may well want to include limited English speakers and hearing impaired users in your study; if you do, you want to be aware of these specifics about the participants before they come.)

Recruiting by email. Email can be very efficient for younger participants; it's less so with adults in their later 60s and 70s and older. Give yourself more time for these older participants; they generally don't check their email more than a couple of times per week. This happens for a variety of reasons: They don't feel the need to check mail frequently. They use a computer at a senior centre or a library. They have limited time available through their Internet service provider.

As suspicious as older adults are of telemarketers, they are also vigilant about spam. If your email address is unknown to them, without an appropriately descriptive subject line, they may delete it. Always put on a very clear subject line.



Screening older adults

Screening older adults demands specificity. Many older users when asked the question "what do you do online?" answer "email." They often don't think about practical activities such as banking or bill paying online as "using the web."

Many older users are also not as familiar with the language of the web as younger users are. They don't distinguish between the Internet and the web. They don't always know the difference between the web browser and the web page.

We found that self-reported data about frequency of use and numbers of hours spent online were not good indicators of proficiency, either. For example, we had one participant who spent 60 hours per week online. We didn't find out until the session started that her sole use of the web was playing games on four web sites that her friend had set up as separate shortcuts on her desktop.

So, asking a variety of specific questions to gauge potential participants' familiarity with the web can help the person recruiting participants for a study make judgments about how suited the person might be for the study. Even if you're looking for a mix of proficiency levels, you still have to be able to determine where in the range a potential participant fits.



Tech savviness matrices
An assessment we’ve found to work well asks about frequency of use over a broad range of types of interactions older people can take part in on the web. Here are some example assessment grids we have used.





Scheduling sessions with older adults

Scheduling sessions with older participants can present some logistical challenges that you might not think about in studies involving younger participants.

They arrive early. Because many older people are retired (or at least have ample free time), they almost always arrive for their sessions early – up to an hour early. Be sure to have someone to greet them and set up a comfortable place for them to wait.

They bring their spouses. Older participants often bring their spouses or a friend with them. They may have travelled some distance to get to the session; they may have planned activities for after their session; or they simply may not like driving alone. Have magazines, a phone, and a comfortable chair available for the spouse or friend.

They do best in the morning. Even though people in their later 60s, 70s, and 80s are vital and energetic, they usually have more—and better—attention to give earlier in the day. Try to schedule people who are in their late 60s, 70s and 80s in the morning and save any afternoon sessions for participants in their 50s or early 60s. We don't recommend running evening sessions.

They don't like driving in rush hour. If you are holding sessions in a central place (rather than meeting participants in their homes or workplaces), schedule the sessions outside of peak traffic times, if possible.


Reminding older adults of important points before they come

Reminders about one-to-one sessions. Participants can become nervous and uncomfortable if they realize after arriving that they will be the only participant in the session. Usability studies are still fairly new to the general population. Recruiting firms often recruit for focus groups, and participants who come through these firms often assume that they are coming to a focus group.

Reminders about videotaping and observations. Although a good practice is to ask for permission to record and to have people observing the sessions when you recruit, people tend to forget that. Make sure that the person who calls the participant to confirm the session also tells the participant that:

  • "You will be videotaped and observed by people you won't be able to see during the session."
  • "This is a one-on-one session. You will be the only participant in the study room with a moderator."

Special reminders for older adults.
 
  •  Computer glasses. Many participants will have special glasses for using the computer. So another important reminder is, "Don't forget your computer glasses!"
  • Eat first. Also, for long sessions— anything longer than 45 minutes— ask participants to make sure they eat before they arrive. Because many participants expect to take part in focus groups rather than individual sessions, they also expect to be fed. If you have snacks available, try to have fruit and nuts or other relatively healthy food. Many older participants are diabetic.



Working with older adults during sessions

Many older participants won't know what to expect coming into a usability study session. Be clear in setting their expectations and be firm but polite about keeping the session focused on what you're trying to find out.

Make participants comfortable. Be respectful without being patronizing. You can be a neutral moderator but still be polite. "Please" and "thank you" are important. Many older adults expect more statements of politeness like these than younger participants do.

Older participants also deserve extra consideration, politeness, and detailed information about the session. They will feel more comfortable if they know what to expect up front:

  • Clearly explain the session plan, timing, and what they can expect.
  • Warn participants that you'll interrupt them and that you may stop them before they have completed tasks.
  • Schedule breaks for long sessions (and tell them they can take breaks whenever they need to).
  • Have them practice thinking aloud.
  • Consider including a practice task to help participants understand how the session will work.
  • Take account of beliefs that participants may have learned or created about how to work with computers.
  • Remember that older participants often are not versed in computer and web terminology, so avoid using this jargon when working with them.
  • Be extra patient with older participants; wait longer than you normally might to prompt; consider giving participants permission to ask for hints.
  •  If participants stop talking, consider letting them continue that way; try reflecting on the task later.
  • Teach participants something at the end of the session.

Keep them on track tactfully. Most of the participants we've had in sessions are interesting, charming, and very talkative. Many older participants have a lot of stories to tell. Their stories say a lot about who they are, and where they have been— and often provide a context for interpreting data.

But it may be easy for participants to get off track during the session, and while it may feel awkward or mean sometimes, it is the moderator's job to keep the participant focused on the task, talking about it, and getting data for the study. This is the main reason for warning participants in the introduction to the session that you may interrupt them and that you may stop tasks before they've completed them.

Listen for their beliefs about computers and the web. Many people who are in their late 60s and older never used computers at work. This means they have no previous experience from which to make inferences about how a computer or an application might work. Many learn how to use computers and the Internet through friends, family, and neighbours. They inherit the superstitions and myths that those people have developed to help themselves work around problems. Then the older adults bring these myths into sessions with them, and you'll hear about them as task-solving strategies and workarounds. It's important to capture these; they are part of the users' reality and we have to deal with these beliefs when we design web sites.

Be careful of the words you use; avoid computer jargon. Older computer users rarely know much about computer-related terminology, so you should avoid using these terms during your sessions. Older participants often don't know the names of widgets such as drop-down boxes or cascading menus. Most of our participants also had little knowledge of web-related terminology. For example, they weren't sure about terms such as "link," "URL," and "login." Many were unclear about the meanings of "online community" and "message boards." "Browsing" wasn't always meaningful in the context of a feature called "browse by topic." The word "emoticon" and the concept behind it were completely foreign to most of our participants. This means that you must pay close attention to what participants do and point at on the screen or device.

Give them time. Older participants almost always take longer to do tasks than younger participants. And, although they seem to struggle, the oldest participants also expect using the computer to be difficult. Plan for tasks to take much longer for older participants than they would for younger participants—up to 25 percent longer in our experience.

Help participants understand the time constraints of the session by explaining the session format in your introduction. Also, wait longer to prompt than you normally might. You might also consider giving participants permission to ask for hints when you introduce the session.

If necessary, hold the think-aloud and ask participants to reflect later. A classic technique in usability testing is to ask participants to think aloud while they work through tasks toward a goal. When tasks become complex or difficult, participants may stop talking. Use your best judgment about nudging them to tell you what they're thinking. For some participants with short-term memory loss or other cognitive impairment (such as that caused by pain medication), your asking for their thoughts may interrupt their task enough that it causes them to make errors. In those cases, you may get more usable data without the think aloud protocol by asking participants to reflect later.

Don't lead even when you want to. If, as a session moderator, you have a soft spot in your heart at all for participants, working with older participants will exercise that spot a lot. You may be tempted to give hints; worse, you may lead them in ways you don't intend. Be patient and firm but polite while keeping to your agenda.

When appropriate, teach something at the end. If the session has been difficult for the participant, or, if there is some small thing that would make using the computer or the web easier, take a little time at the end of the session to teach the participant something. For example, show participants how to change the text size in their browsers and shortcuts for copying and pasting and printing.



Including older adults in user research and usability studies: Older, wiser, wired

Older adults don’t behave differently from younger people online. Just thinking about old age as a collection of disabilities is old business. The new world of designing for older adults is about creating web sites and other technology that is useful and desirable as well as accessible to the broadest range of users. Older adults as a cohort are living longer than their parents because they’re healthier, and many will be affluent because they’ve been saving up for a lifetime – this means they have time, money, and motivation to be online.

Neither a monolithic view of older adults nor an entirely separate design for older adults is necessary. Younger designers developing web sites for older adults need to learn more about older adults’ life experiences. For example, many older adults don’t perceive themselves as old. And so, all technology design – not just designs for older adults – should involve users.

Thursday, August 19, 2010

Researcher as director: scripts and stage direction


For most teams, the moderator of user research sessions is the main researcher. Depending on the comfort level of the team, the moderator might be a different person from session to session in the same study. (I often will moderate the first few sessions of a study and then hand the moderating over to the first person on the design team who feels ready to take over.)

To make that work, it's a good practice to create some kind of checklist for the sessions, just to make sure that the team's priorities are addressed. For a field study or a formative usability test, a checklist might be all a team needs. But if the team is working on sussing out nuanced behaviors or solving subtle problems, we might want a bit more structure.

A couple of the teams I work with ensure that everything is lined up and that *anyone* on the team could conduct the sessions by creating detailed scripts that include stage direction. Here are a couple of samples:

























Whether the team is switching up moderators or it's the same person conducting all the sessions, creating a script for the session that includes logistics is a good idea:

  • think through all the logistics, ideally, together with the team
  • make sure the sessions are conducted consistently, from one to the next
  • back up the main researcher in case something drastic happens -- someone else could easily fill in


Logistics rehearsal
When you walk through, step by step, what's supposed to happen during a session, it helps everyone visualize the steps, pacing, and who should be doing what. My client teams use the stage direction in the script as a check to make sure everything is being covered to reach the objectives of the sessions. It's also a good way to review what tools, data, and props you might need.

Estimate timing
Teams often ask me about timing. When they get through a draft of a script that includes stage directions, they get a pretty solid feeling pretty quickly for what is going to take how long. From this they can assign timing estimates and make decisions about whether they want participants to keep going on a task after the estimated time is reached or redirect to the next task.

Mapping out location flow
It's easy to overlook the physical or geographic flow - what a director would call blocking - of a session. Where does the participant start the session? In a waiting room, at her desk, or somewhere else? Will you change locations within a room or building during the session? How do you get from one place to the next?


Consistency and rigor
Including stage directions in a script for a user research session can help reviewer-stakeholders understand what to expect. More importantly, the stage directions act as reminders to the moderator so she's doing the same things with and saying the same things to every participant in the study. This means nothing gets left out deliberately and nothing gets added that wasn't agreed on ahead of time. (For example, the team could identify some area to observe for and put a prompt in the script for the moderator to ask follow-up questions that are not specifically scripted, depending on what the participant does.)


Insurance
Any really good project manager is going to have a Plan B. With a script that includes detailed stage directions, anyone who has been involved in the planning of a study should be able to pick up the script and moderate a session. The people I worked with at Tec-Ed called this "the bus test" (as in, If you get hit by a bus we still have to do this work).

Some teams I work with want to spread out and run simultaneous sessions. The stage directions can help ensure consistency across moderators. (Rehearse and refine if you're really going to do this.)

Finally, when it comes time to write the report about the insights the team gained, the script -- with its stage directions -- can help answer the questions that often come asking why things were done the way they were done or why the data says what it says.


Stage it
Each person in a session is an actor, whether participant or observer. The moderator is the director. If the script for a study includes instructions for all the actors in the session as well as the director in addition to documenting what words to say, everyone involved will give a great performance.

Saturday, August 9, 2008

Getting ready for sessions: Don’t forget…

There are a bunch of things to do to get ready for any test besides designing the test and recruiting participants.
  • make sure you know the design well enough to know what should happen as the participant uses it

  • copy any materials you need for taking notes

  • copy of all the forms and questionnaires for participants, including honorarium receipts

  • organize the forms in some way that makes sense for you. (I like a stand-up accordion file folder, in which I sort a set of forms for each participant into each slot. I stand up the unused sets and then when they’ve been filled out, they go back in on their sides.)

  • check in with Accounting or whoever on money for honoraria or goodies for give-aways

  • get a status report from the recruiter

  • double-check the participant mix

  • make sure you have contact information for each participant

  • check that you have all the equipment, software, or whatever that you need for the participant to be able to do tasks

  • run through the test a couple of times yourself

  • double-check the equipment you’re going to use (I use a digital audio recorder, so I need memory sticks for that, along with rechargeable batteries)

  • charge all the batteries

  • double-check the location

Which gets us to where you’re going to do the sessions. But let’s talk about that later.

Tuesday, July 29, 2008

Where usability testing fits into your research strategy

What, you don’t have a research strategy? Let’s think about the future here.

It’s not uncommon – and not bad – to be working in the present, reacting to the ever-growing demand for usability testing in your organization. “Ever-growing” is good. But when Jared Spool asked me to do a podcast with him recently to talk about what I think makes the difference between a good user experience team and a great user experience team, it got me thinking.

The recipe, based on my observations in dozens of corporations, comes down to these three main ingredients:
  • Vision
  • Strategy
  • Involvement

Vision is an overused word, but here I mean that you and your team have visualized the ideal customer experience -- no limits, no constraints. Imagine the best possible interactions a customer could have with your organization at every touch point. Write it down.

Strategy means that you have a plan for reaching the vision. Over the long term, you can learn about and take into account customers’ contexts and goals while matching those up to the goals and objectives of the business.

Involvement calls all interested people in the business together (and that really should be everyone from management to design to development to support and anyone else in the organization) to embrace the vision and carry out the strategy across disciplines.

But I haven’t said much about usability testing yet. Where does it fit in? Everywhere. Part of my strategy would be to teach as many people in the organization to do usability testing as possible. You probably can’t do all the testing that is wanted (let alone needed). If you teach others to do it and coach them along the way, the customer ultimately benefits as the organization gains a closer, smarter understanding of the customer experience and can make evidence-based decisions about how to get to the ideal experience it shares a vision of.

Friday, May 30, 2008

Observers are your friends

Research that you do alone ends up in only your head. No matter how good the report, slide deck, or highlights video, not all the knowledge gets transferred to your teammates. This isn’t your fault. It just is.

So what to do? Enlist as many people on your team as possible to help you by observing your usability testing sessions. You can even give your observers jobs, such as time keeper if you’re measuring time on task. Or, if you are recording sessions, it could be an observer’s job to start and stop the recordings and to label and store them properly.

The key is to involve the other people on the team – even managers – so they can
  • Help you
  • Learn from participants
  • Share insights with you and other observers
  • Buy in
  • Reach consensus on what the issues are and how to solve them

Who should observe: Everyone
Ideally, everyone on the design and development team should observe sessions. Every designer, every programmer, every manager on the project should watch as real people use their designs.

Each of the observers should watch as many sessions as possible. (Okay, you could settle for two sessions.) Seeing only one session is just a snapshot; what happens in that session may not be similar to what happens in the other sessions. If the observer who saw only one participant is in an influential position in the organization, that one set of observations may outweigh others when it comes time to get some consensus from observers about what the issues are. (Hint: You don’t really want that.)


Training observers: Ground rules are essential
Observers can watch from a separate space – most labs are set up this way – or they can be present in the testing room with you and the participant during each session. Either way, you should brief them on
  • What to look for
  • How they might see it
  • How to behave with participants
  • What to do (and not do) with their observations
It probably isn’t enough just to hand out a list of rules if this is the first usability test these people will watch. Consider holding a short training session to explain how the usability test will be conducted and what they can expect in watching sessions. Be plain (but diplomatic, of course) about your expectations of your observers.

Here’s one of my favorite sets of guidelines for observers who will be in the testing room with you and the participant.


If observers will not be in the testing room with you and the participant but rather will be watching from a separate viewing room, ask observers to resist redesigning or finding solutions until the end of the study (or until the day’s debrief, whichever works best in your situation). This is another good reason to give each observer a little job – it keeps them occupied at different things that demand the attention that might otherwise be given to premature redesign.


Working with observers: Reaching consensus

No small part of working with observers is understanding what their separate stakes in the test are. When you know this, you can facilitate discussion among observers effectively. If you don’t know, you’re probably working harder than you need to to gain consensus.


Consensus is important. Reaching consensus among observers means that

  • There’s a shared vision of where the design should go. This is good for the team and the design.
  • Everyone uses the same descriptions for issues. Having a common language for talking about the usability issues among departments or groups will help the diverse group visualize the user experience together.
  • Nuances and exceptions have been discussed and agreed on. Anything that isn’t optimally feasible to fix can be negotiated among design and development team members in a daily debrief or final design direction meeting.
  • There should be no surprises when you publish the report. Because team members observed sessions and discussed the issues together, insights and outcomes in the report should reflect the agreed direction from those observations and group discussions.

Feeling lonely? Gather up some observers for your next usability test from your design and development teams. Make them feel welcome, give them duies, and work with them to understand the issues. They’ll thank you for it.

Tuesday, May 20, 2008

The importance of rehearsal

You have designed a study. Everyone seems to be buying in. Scheduling participants is working out and the mix looks good. What’s left to be done except just doing the sessions? Three things:

  1. Practice.
  2. Practice.
  3. Practice.

There are three rounds of practice that I do before I do a “real” session. Jeez, I can hear you say, why would I need to practice so much? Why would you, Dana, who have been doing usability testing for so many years, need to practice so much? I do it for a couple of reasons:

  • It gives me multiple opportunities to clarify the intent of the test, the tasks, and the data measures.
  • I can focus on observing the participant in each regular session because any kinks have been worked out.

Walk through the script and gather tools and materials
The first is to walk through my test plan and script. I read the script aloud even though I’m by myself. While I’m doing that, I do two things: adjust the wording to sound more natural, and gather tools and materials I’ll need to do the sessions.

Do a dress rehearsal or dry run
For the second round of practice, I do a dry run of the now refined script with someone I know filling the role of the participant. We do everything you would normally do in a session, from greeting and filling out forms, to doing tasks, to closing the session. I might occasionally stop the session to adjust the script or to make notes about what to do differently next time. I might even ask the participant (usually a friend, neighbor, or colleague) questions about whether the test is making sense. It’s a combination of dress rehearsal and “logic and accuracy” test to get the sequence down and to make sure you’ve got all the necessary pieces.

Pilot the protocol
Finally, there’s the pilot test session. In this pilot, I work with a “real” participant – someone who was screened and scheduled along with all of the other participants. I conduct the session in the same way I intend to conduct all of the following sessions. The twist this time is that observers from the design team should be present. At the end of the session, I debrief with them about the protocol.

Don’t waste good participant data
There have been times when I’ve been rushed by a client or was just too cavalier about going into a usability test and did not rehearse. I paid for it by having rough sessions that I couldn’t use all the data from. Every time it’s a reminder that preparation and practice are as important to getting good data as a good test design is.

Monday, May 5, 2008

Should you test in a lab or in the field?

I haven’t been in a usability test lab for about a year. Ironically, since I was writing a book about usability testing, much of my work was field research to learn about particular audiences and their tasks.

And, though my usual position about labs is that exploratory usability testing is probably better done in the user’s environment, I’m excited about getting back into the lab.


Good reasons to test in a lab
I’m doing these upcoming tests in a lab facility because

  • The testing is quantitative and summative. That is, I’m doing very specific counts of errors and failures that are strictly defined, so I want to control other aspects of the test such as the computer setup.

  • I don’t want to interact much with the participants. I only want to direct participants when to start their tasks. Otherwise, I will intervene in the session only at prescribed points, so I will direct the session from a different room from where the participants are working.

  • I may have observers, but I won’t know until the last minute. Though I prefer it if observers arrive before the session starts and stay through a whole session, at a facility they can come and go because they can observe from a separate room.

Good reasons to test in the field
I recently did a usability study in the field. Why?

  • I wanted to learn about the user’s environment (rather than controlling it). In the exploratory study I’m thinking of, I got the best of both worlds: usability testing data in a realistic situation. I learned about lighting levels, surrounding noise, and what the participant’s desk setup was like. But I also got to observe relationships and interactions the participant had with others, typical interruptions (and recovery from those), and how the thing I was testing fit into the person’s work.

  • It was convenient for the participants. They don’t have travel to the testing site. The interruption of their typical day is minimized.

  • The sessions were informal enough that observers could be present in the room (after they had been properly trained). In fact, people from neighboring cubes often chimed in comments or questions because they’d overheard what we were talking about. I took this to be a good thing because I learned about that communication dynamic, but those eavesdroppers often contributed information that was useful to me in my study.

In a future post, I'll talk about what to look for in a lab facility if you're renting one and how to find one.

Tuesday, June 5, 2007

Why create a test design?

I get a lot of clients who are in a hurry. They get to a point in their product cycle that they’re supposed to have done some usability activity to exit the development phase they are in and now find they have to scramble to pull it together. How long can it take to arrange and execute a discount usability test, anyway?

Well, to do a usability test right, it does take a few steps. How much time those steps take depends on your situation. Every step in the process is useful.


The steps of a usability test
Jeff Rubin and I think there are these steps to the process for conducting a usability test:
  1. Develop a test plan
  2. Set up the testing environment and plan logistics
  3. Find and select participants
  4. Prepare test materials
  5. Conduct the sessions
  6. Debrief participants and observers
  7. Analyze data and observations
  8. Create findings and recommendations

Notice that “develop a test plan” and “prepare test materials” are different steps.

It might seem like a shortcut to go directly to scripting the test session without designing the test. But the test plan is a necessary step.


Test plan or test design?
There’s a planning aspect to this deliverable. Why are you testing? Where will you test? What are the basic characteristics of the participants? What’s the timing for the test? For the tasks? What other logistics are involved in making this particular test happen? Do you need bogus data to play with, userids, or other props?

To some of us, a test design would be about experiment design. Will you test a hypothesis or is this an exploratory test? What are your research questions? What task scenarios will get you to the answers? Will you compare anything? If so, is it between subjects or within subjects? Will the moderator sit in the testing room or not? What data will you collect and what are you measuring?

It all goes together.



Why not just script the session without writing a plan?
Having a plan that you’ve thought through is always useful. You can use the test plan to get buy-in from stakeholders, too. As a representation of what the study will be, it’s understanding the blueprints and renderings before you give the building contractor approval to start building.

With a test plan, you also have a tool for documenting requirements (a frozen test environment, anyone?) for the test and a set of unambiguous details that define the scope of the test. Here, in a test plan, you define the approach to the research questions. In a session script, you operationalize the research questions. Writing a test plan helps you know what you’re going to collect data about and what you’re going to report on, as well as what the general content of the report will be.



Writing a test plan (or design, or whatever you want to call it) will give you a framework for the test in which a session script will fit. All the other deliverables of a usability test stem from the test plan. If you don’t have a plan, you risk using inappropriate participants and getting unreliable data.