Institutionalization of UX: A Step-by-Step Guide to a User Experience Practice, Second Edition (2014)

Part II. Setup

Chapter 7. Tools, Templates, and Testing Facilities

If you throw usability staff into the organization without the right equipment, they are going to be slow and inefficient, and impractical in their approach.

Get appropriate tools (e.g., lab equipment and prototyping software), templates (e.g., reusable questionnaires), and testing facilities. These items form an essential toolkit—the core infrastructure for routine usability work.

Your toolkit can enable usability designers to efficiently complete the user experience design methodology. To determine the toolkit you need, review your methodology.

A well-trained staff in a room with nothing but paper can out-design a poorly trained staff equipped with a state-of-the-art facility. While in China, I was once given a tour of a half-million-dollar usability testing facility. It was impressive—very advanced looking. Later, in quiet conversations with the staff, it became clear that the team had no idea what to do with the facility. They had equipment, but no software to support the testing, and no methods or skills.

The main value of facilities, tools, and templates derives from time savings. Instead of creating a testing form from scratch every time a test is needed, a user experience design engineer can take an existing form and modify it for a client’s specific test in approximately 20 minutes. By comparison, creating the concept and forms for a test from scratch may take days or even weeks. Clearly, it is advantageous to hire good staff members and supply them with the tools that can make a difference.

This chapter outlines the tools you need, the templates that are helpful, and the usability testing facilities that will help your staff be most efficient and effective. Of course, by the time this book is published, some of the tools and templates described here may be outdated. New developments are taking place all the time. For example, you might hear that usability testing labs have recently moved from “marginally useful in special circumstances” to being “a practical part of almost every test.” Or you might learn that remote testing, which isn’t used often today, is becoming far more practical and, therefore, much more widely used. Remote testing is usability testing performed at a distance; the participant and the facilitator are not in the room together (and may not even be on the same continent), yet the facilitator can monitor what the participant is doing and saying. Unmoderated remote tests are tests that are run automatically. The participant interacts with a scripted program that collects performance data and responses. Because the contents of toolkits will likely change, a skeptical attitude about these tools is useful—if a tool does not really make a difference in the design, spend your usability testing budget elsewhere.

Introduction to Your Toolkit

Your methodology points to the facilities, tools, and templates you need. For example, if the methodology specifies that a test of branding occurs at a certain point, you will want to have templates for reusable questionnaires and a standard template for the final report.

If you update your methodology, you may need to update the corresponding tools, templates, and facilities as well. Conversely, the introduction of new facilities, tools, and templates might lead you to change your methodology. Online prototyping has become easier, for example, so you might move it to an earlier point in the design cycle. Likewise, as remote testing becomes more feasible and useful, you might add it to your methodology and develop new tools and templates to fit it. Be careful about implementing these kinds of changes, however: some “amazing” breakthroughs are actually not particularly useful. Consider the new, crowdsourcing solutions that let many people review your design. While there are some known advantages to crowdsourcing supporting innovation, having a load of armatures be reviewed out of context may not be a valid methodology.

The following sections cover the infrastructure you should consider implementing at your company. They also explore scenarios and priorities for each facility.

Testing Facilities

Depending on the circumstances, testing facilities can range from a simple office setting or hotel room to a full-blown usability testing lab. Of course, a full usability testing lab is not necessary to conduct usability testing. If office space is at a premium, the office of one of the usability team members can be used for testing. There may not be a one-way mirror, special equipment, or videotaping in such a site; instead, you may have only a few chairs, a desk, and a computer at your disposal. Skilled staff members, however, can still successfully create and run the tests. Similarly, it is quite acceptable to use a conference room to run tests, but it is critical that the room be reasonably quiet and free of visual and auditory interruptions. For this reason, it is best practice not to use participants’ workspaces for testing. You can observe them there, but workspaces are not good places to run tests.

There are a number of reasons for establishing a formal, dedicated usability testing facility. Notably, designating a space for testing shows a commitment to testing within the organization. While it is nice to have a room or perhaps a suite with that label, the practical value of a usability testing facility is less significant than the political statement it makes. Of course, the facility becomes an albatross if it is not regularly used—or evolves into a storage space.

While running tests in storage closets can still yield quite good results, it is best to have a testing environment that makes the participants and the facilitator feel comfortable and important. If you can make the test a relaxed experience, you will get more accurate and complete results. At the same time, you do not want the evaluation environment to feel too formal. Facilities that feel imposing and clinical should be avoided. Usability engineers call the people who engage in testing participants instead of subjects for a reason; no one likes to feel like a lab rat!

Facilitating a test is a very demanding activity. It takes focus, and it’s difficult, if not impossible, for one person to simultaneously keep the test process running, observe the nuances of the results, and record data. There is no additional energy or time left to greet participants, provide the initial forms, and hand out the compensation once the testing is complete. Consequently, it is very useful to have additional staff available to handle these functions. Professional testing facilities, for example, have support staff.

In some cases, you will need a facility geographically separated from your offices. You might decide to do testing in a number of cities intermittently, or you might even need to complete testing in these different cities quite often. In this scenario, it makes sense to establish a relationship with a testing facility in each location. These testing facilities are generally set up for marketing studies, but they work well for usability testing. It is also possible to use a conference room in a hotel, but testing facilities come with valuable amenities such as a greeter, a one-way mirror, built-in sound and video, and usually a more comfortable atmosphere.

Whether you sign a contract with a professional testing facility or build your own testing space, there are a few other advantages associated with use of a professional testing facility versus use of a simple conference room for usability testing. Figures 7-1 and 7-2 show the appearance of a typical professional testing facility. Your facility may have a one-way mirror. Most people can tell when you have a one-way mirror, so if your facility has one in place, you should be straightforward about it. With proper briefing, the mirror works very well. Developers, business owners, and marketing and usability staff can come to the site and observe the participants without disturbing the test. They can discuss what they see and send in their questions to the test facilitator. Alternatively, instead of a one-way mirror, you can use video feeds to adjacent rooms to allow others to observe without disturbing the test.

Figure 7-1: Observer’s side of a professional testing facility using a one-way mirror¹

1. Photo courtesy of the Bureau of Labor Statistics.

Figure 7-2: User’s side of a professional testing facility using a one-way mirror²

2. Photo courtesy of the Bureau of Labor Statistics.

Recording of Testing Sessions

There is value in recording data-gathering and testing sessions. Professional facilities have video capability, and the new portable labs support video as part of their software.

Two types of recording usage may be employed. One common practice is to produce a full video recording of the session for the record. A continuous tape is made of the test, and you end up with many hours of tape. If someone later says, “I don’t believe the user actually did that,” you can offer to let him or her see the appropriate portion of the tape. In other cases, a much shorter highlights tape is culled from the full videotaping sessions. This edited video, 5–10 minutes long, shows key findings of the usability testing through the voices and actions of the participants themselves. Carefully selected examples on well-edited highlights tapes often can depoliticize the usability test findings: the finding is no longer the “opinion” of the tester, but rather the voice of the participant. Highlights tapes effectively grip the audience’s attention when used as part of the final presentation (which, from a practical standpoint, cannot be done with the full, unedited video of the session). This is a very effective practice—there is nothing like showing video of the users in action.

In the past, recording sessions were prohibitively expensive, but the “shoebox” usability equipment available today (see Figure 7-3) has made the cost much more reasonable. Such equipment includes a TV camera, microphone, monitor, and a remote marker to make it easy to find interesting tape segments. There is actually no tape—just a high-capacity hard drive to save the data—so it is also far easier to edit and present the results. This ease of use, combined with its reasonable cost, makes the shoebox lab a practical alternative to traditional equipment.

Figure 7-3: Shoebox usability equipment

Most labs are using applications and hardware that make video editing increasingly efficient. With these new facilities, you can easily place parts of the video record in the report (see the sample of a test presentation video in Figure 7-4). The lab software lets you record the user’s facial expressions and the activity on the screen. You can then add detail to reports with clear instances of user actions and reactions.

Figure 7-4: Example of a video record from a usability test

A few labs use an eye-tracking device, which identifies where the user’s eye is fixating. You can gain a lot of information from this device. For example, you can see users scanning around the page because they are lost or scanning an image because they cannot tell if it is selectable.

Eye-tracking devices are very useful for research purposes. For example, studies have shown that people start scanning in the main area of a webpage and initially ignore the logo, tabs, and left-hand navigation [Schroeder 1998], and that people’s eyes are drawn first to areas that have saturated colors (pure bright colors), darker areas, and areas of visual complexity [Najjar 1990].

You do not need an eye-tracking device to run an excellent usability test. A good facilitator can see where the user is looking and can supply you with very similar data. An eye-tracking device is expensive and requires setup time, so you probably won’t use it for routine usability tests. It may come in handy in a remote usability test, however, because the facilitator will not be physically present with the participant.

Modeling Tools and Software

Most of the important usability work can be completed with a simple Microsoft Office suite and software for graphics work. It may help to have a flowcharting package, but that’s about all the software you need. You also need to be able to use a word processor to document meetings and descriptions, and to have some kind of tool to mock up screens and pages. Which tool you use is not as critical as making sure that the usability staff members are comfortable with that tool and do not get distracted or waste time writing “code” to make the screen mock-ups work. Some people prefer a graphics program such as Adobe Photoshop, but Microsoft PowerPoint or other presentation tools work just as well. Some usability staff already have facility with tools such as Microsoft Visio. Whatever tool your staff members already know how to use that allows them to quickly mock up screens and pages is the best tool to use.