Institutionalization of UX: A Step-by-Step Guide to a User Experience Practice, Second Edition (2014)

Part I. Startup

Chapter 2. Selecting a Usability Consultant

Outside consultants are important to usability institutionalization—they can do things no insider can do.

Retain a usability consultancy early in the process to provide infrastructure and jump-start the institutionalization process.

There are many usability consultancies, and there are huge differences in their capability, resources, and fit with your culture. Pick the right consultancy for your organization.

A good consultant will guide your strategy, set up your infrastructure, help develop your staff and internal organization, and smoothly transition to a role that supports the internal group.

A few companies invest in a solid internal usability group during the early phases of user experience institutionalization, but most companies start with a usability consultancy. The consultancy brings an integrated and experienced team of people and a complete set of resources. They can help your organization establish a well-tuned strategy and do most of the work of setting up the infrastructure. Consultants are skilled in supporting change management issues. As outside experts, their advice is often more easily accepted than internally made recommendations. Also, in the unlikely event that the whole initiative fails, consultants are easy and inexpensive to fire. For all these reasons, it makes sense to start with a consultancy.

In my experience at HFI, the cost of an effective initial setup for addressing usability concerns within a large company could be in the range of $800,000 to $1.4 million just for consultancy resources. This level of investment is required to cover the minimum set of activities, documents, and deliverables needed to get usability going.

You should select a usability consultancy that has a critical mass of staff, processes, tools, and specialists who can help with the startup tasks and the creation of the infrastructure. You want to select a consultancy that comes equipped with a full infrastructure because you do not want to create a methodology and toolkit from scratch. Also, be sure to select a firm with a good set of training courses so you won’t need to build and maintain a suite of courses for your company—this can be a very expensive and time-consuming endeavor.

The consultancy’s role changes over time. In the Startup and Setup phases, the consultancy guides the strategy and establishes the infrastructure. The firm may help with the design of one or two showcase projects. In the Organization phase, the consultancy helps with recruiting and supporting the initial projects for which internal staff members are not yet in place, experienced, and confident. During the Long-Term Operations phase, the consultancy’s role changes yet again. The consultancy should continue to provide training and supplemental support where needed. This supplemental support may take the form of a very high-level consultant who performs audits and provides a second opinion, or it may take the form of specialists who assist the organization with uncommon technologies. The consultancy may also provide lower-level staff to help fill the ranks of usability practitioners. One particular advantage associated with some consultancies is the provision of offshore usability support. Done properly, the quality and low cost of an offshore group can make it possible to support all the projects that need usability work.

It is important to select the right consultancy, and this chapter will help you by describing what you should look for. Table 2-1 summarizes the selection criteria and suggests weightings for these items.

Table 2-1: Weighted Criteria for Selecting a Usability Consultancy

While it is unlikely that any consultancy will score 100% on these criteria, you should try to find a company that comes close to this ideal. We have included a point system to reflect that some items are more important than others. This rating system should hold up well across different organizations, but you may want to adjust the weightings to match your needs and priorities.

Staffing

You should make sure that your consultancy has professional usability engineering staff. (Chapter 12, Staffing, describes the different types of usability engineering skills.)

Some of the consultancy’s staff should have advanced degrees in usability engineering. These degrees can go by different names, including the following:

• Software Ergonomics

• Human Factors Engineering

• Engineering Psychology

• Usability Engineering

• User Experience Design

The staff members may also have backgrounds in cognitive psychology or sensation and perception.

Make sure that the staff members are oriented toward practical design work rather than research. If some staff members publish a lot, make sure that their firm also employs people who do the practical work. They should be familiar with the current research, but they don’t have to author it all.

At least some of the staff members should have substantial experience in the field. It typically takes 10 years to develop solid competence. Although some of the staff may have less experience than this level, a consultancy in which most of the members have only one or two years in the field is likely not experienced enough to guide your organization.

Without skilled staff, there will be little value added by the consultancy, no matter how good its infrastructure is. The people within the consultancy must be at the vanguard of the field and become role models for future growth within your organization.

Completeness of Solution

Some companies can provide a few pieces, but not all, of your usability solution—some may perform only usability testing or training, while others may offer expertise in projects or do high-level change management consulting. You can try to piece together a solution, but the parts will never be perfect matches. Certainly, all good usability consultants have a similar philosophy and work from similar principles. However, even small discrepancies in the interpretation of research or nomenclature can reduce credibility and confuse internal stakeholders. Therefore, you should select a consultancy that can provide a complete and integrated solution, including the following elements:

• Support for institutionalization:

Strategic consulting—consulting on the strategies for usability within the organization

Expert reviews—evaluation of existing technology products, such as applications or websites, to see which usability problems they contain

Introductory training—training on the basics of usability

Detailed skills-level training—training on more detailed skills for in-house usability professionals

Methodological standards—standards for a user-centered design process

Design standards—standards for the visual and interaction decisions for projects

Templates and tools to support the standards—tools to make standards easy to implement

Recruiting of usability practitioners—help and advice in finding qualified practitioners

• User-centered design on specific projects:

High-level user interface design—consulting expertise in high-level design for critical and showcase projects

Graphic design—graphic design expertise that dovetails with usability

Detailed design and functional specifications—user requirements from a usability point of view

Usability testing—consulting and help with the design and implementation of standard usability test protocols

• Ongoing support:

New research updates—a strategy for keeping the staff up-to-date on the latest work and research

Usability audits—periodic audits of standards and processes

Mentoring—mentoring of both executives and practitioners to advance their expertise

These criteria are very important because the effort required to manage a fragmented solution is significant. Even simple problems such as discordant terminology can slow the process. (For example, is the description of the user’s workflow called task design, scenario creation, orstoryboards?)

Domain Expertise

Organizations often overestimate the importance of specific domain expertise when selecting usability consultants. What is more important is that the consultant can augment his or her general domain knowledge with exceptional usability expertise.

The consultant should have general expertise in your type of applications and high-level domain expertise—that is, the consultant should understand the general domain of your business. Extensive specific domain expertise in the unique focus of your organization is not necessary. If your organization is in the financial industry, for example, it is important that the consultant have experience with financial products. If your organization produces software for chemical engineers, your consultant should have experience with scientific applications. He or she does not need to have extensive knowledge about your particular type of chemical product or financial offering.

Consider how you might choose an accounting firm. It would certainly be helpful if your accounting firm had worked in your industry—after all, your industry has some unique conventions and special needs, so a certain level of familiarity with these issues could save days or weeks of time that would otherwise be spent on investigation and learning. But when it comes right down to it, the accounting domain is pretty similar across industries. If this were not true, there would be no large accounting firms; instead, all accounting would be done by specialty firms focused on narrow vertical markets. You might be better off with a firm that knows your industry less but has a better staff or a superior process.

Like accounting, usability work is relatively similar across domains. Consultants with domain knowledge can save some time and offer some special insights, but having this level of expertise should not be a critical factor when choosing a consultancy.

Methodology

Select a consultancy that follows a complete and systematic user-centered development process. This means that the process is fully documented and has specified activities and deliverables that can be modified as needed to meet each new set of project demands. It is also important that the methodology you choose be appropriate and comprehensive.

Many user-centered design methods are available. HFI has developed its own user-centered development process, which is outlined in Chapter 4. The Usability Engineering Lifecycle: The Practitioner’s Handbook for User Interface Design by Deborah Mayhew [1999] describes another such process. A brief look at the evolving software usability methodologies would include the following:¹

1. This list is summarized from Cost-Justifying Usability [Bias and Mayhew 1994], which presents information on these authoritative methodology texts in more detail. Please consult that source for full bibliographic information for the books listed here.

• Gould and Lewis (1985)—published one of the earliest usability methodologies

• Manter and Teorey (1988)—outlined a usability methodology that was integrated into a standard software development life cycle

• Nielsen (1992)—expanded earlier usability principles to 10 detailed steps that should be integrated into an overall development plan

• Schneiderman (1992)—introduced eight interactive stages for usability considerations across any interactive systems development process

• Mayhew (1992)—advanced usability methodology by offering more detailed specifications for integrating iterative usability techniques into the software development cycle

There is also some useful ISO work, such as “Ergonomics of Human–System Interaction: Specification for the Process Assessment of Human–System Issues” (ISO/TS 18152, first published in 2010). In any case, there is no need to try to write your own methodology. It is faster, cheaper, and better simply to customize an existing method.

Tools and Templates

User-centered design requires a lot of work, but the amount of effort can be vastly reduced if the consultant has a set of tools and templates that support the process. Templates such as questionnaires for interviews, screeners for usability testing, templates for documenting expert reviews, and usability testing reports are critical to ensure that deliverables are easy to create and standardize. Without these templates in place, the consultant will spend a great deal of time reinventing everything from scratch—an expensive process that will extend the project timeline.

In addition to the templates, there is significant value in having a set of tools. The consultancy might have remote testing software, a portable or fixed usability testing lab, and software for tasks such as cluster analysis of card sort data. Working with tools and templates makes consultants more efficient and the deliverables and results more consistent. But it is even more important that you obtain these tools and templates for your own organization so that your company can use them in the future. It makes more sense to use a commercial off-the-shelf solution and have it customized to meet your needs than to invest in creating these resources from scratch. You can certainly create your own templates over time, but having a set to use initially saves months of effort.

Object-Oriented Approach

If you were looking for a systems integrator, you would never consider one that did not have an object-oriented approach to systems development. You would not hire a company that just coded flat files for everything, burying all the work and making reuse impossible. Unfortunately, as of this writing, most user experience design companies tend to just create slide decks, or perhaps documents. After a few years, you are likely to end up with terabytes of data, worth millions of dollars—lost forever in a document management system. A simple question such as “What do we know about high-net-worth customers using a bank branch?” becomes nearly impossible to answer, because the search to answer it yields hundreds of documents that need to be read slowly to determine whether they are even relevant.

Best practice today calls for creating slide decks that tell a story as part of a project, while breaking out most of the content into separate objects (e.g., user profile, scenario). At that point, UX practitioners throughout the company can grow and share a set of ecosystem objects that relate to your domain. Instead of jumbling everything together in a flat set of slides, the organization should separate its user profiles, scenarios, environments, artifacts, needs and opportunities, projects, specifications, standards, and methods.

It should be easy to know or quickly identify every project that usability-tested a given type of user. Otherwise, you will be paying to re-research and re-design the same elements. Being able to pull in and work with UX objects is far more powerful and efficient than building everything from scratch.

User-Centered Size and Stability

Many usability consulting firms have only a handful of practitioners—perhaps even just a single practitioner—and possibly some additional contractors. These companies can have significant limitations. If you are working on a usability institutionalization program in a large company, your company’s needs may overtax the capabilities of the quality staff in a small organization. Even if your own organization is small, a one- or two-person consultancy will be quickly overwhelmed by the swings in demands during even a single project.

If a consulting firm has eight staff members and you hire all eight staff members for five months, at the end of the project you might find that the firm disappears because it cannot immediately find new work to absorb all of the resources released at the end of your project. That means the same consultancy will not be there to help later when your organization has further needs.

Needless to say, it is a serious problem to have a consulting company fold in the middle of your strategy. The time you have spent getting its staff members up to speed on your organization will be lost. If the consulting firm cannot meet your fluctuating needs, you will have to slow down or modify your process to accommodate its limited capacity and capabilities.

A large firm can provide a team of people to support your projects and won’t collapse if you don’t need that support for a while. Larger firms are also generally more stable. The fact that these companies are well established means that they will be more likely to be there for you over the long haul, and they are more likely to have specialists who can meet your needs and facilities that can support your effort.

Corporate Cultural Match

It is not easy to achieve a perfect fit between corporate cultures. You want to be sure that the consultancy’s technical staff will fit with your company’s environment. In some situations, it helps to have a strong figure who can walk into a confusing situation and provide order. In other circumstances, you may need a usability expert who can work with your team and smoothly create consensus. It is important that the consultants have some flexibility in style. It may be even more important that they can provide a variety of staff members with different personalities and working styles, so that the ones who best fit your corporate culture can work with your organization.

Many other special characteristics of organizations need to be considered. In some cases, uncomplimentary cultures present some advantages. For example, a very systematic and controlled culture might need a consultant who can break the mold, be artistic, and provide a different perspective. Conversely, if your organization lacks a systematic process, it helps to have a consultant who can inspire that structure. In most cases, however, similarity in cultures is more helpful and often leads to a more comfortable long-term relationship. If you have a scientific- and engineering-oriented culture, pick a usability firm that has a similar approach. While the consultant will work closely with you for a relatively short period, there will also be a less concentrated, longer-term relationship. Being comfortable with your consultant’s organization has value.

Specializations

Hiring a consultancy that has staff with different specializations in the field offers a significant advantage. People who have focused deeply on a given topic can answer questions in an instant.

There are many different types of specialists. Some usability professionals have experience in meeting the exact challenges of specific populations:

• Cross-cultural and multilingual design

• Design for the visually impaired

• Design for children

Some have experience in specific technologies:

• Mobile and embedded applications

• Gesture interfaces

• Telematics

• Voice response

• Web and responsive design

• Workstations

Others have experience with specific technical areas and topics:

• Search strategies

• Natural language dialogs

• Representation of large networks

Some staff may have particular experience in different parts of the user-centered design process:

• Interviewing

• UX strategy

• Innovation

• Persuasion engineering

• Complex structural design

• Usability testing

• Graphics

• Prototyping

UX staff also may specialize in certain types of situations and businesses:

• E-commerce

• Intranets

• Shrink-wrapped software

• Emerging markets

• Bottom of the pyramid (poor people)

Having consultants with a wide range of experiences at your disposal allows you to draw on solutions from many contexts. For example, someone with cross-cultural design expertise would take one look and notice that your command abbreviation “S” may mean “save” in English, but “exit” (sortie) in French.

Organizational Structure

Look for a consulting firm with the necessary staff to fit your organization’s needs. With a small organization, it may not be difficult to manage all the usability work that is occurring simultaneously. When a larger organization is the client, the consultancy may also need to offer a business manager to handle contractual relationships, an accounting contact to handle payment issues, and a separate legal person to handle nondisclosure and contractual issues. For large-scale efforts, you have to deal with more people than just the usability consultants.

There will always be a primary usability lead and perhaps some additional usability staff on the team. In addition, it helps tremendously to have an escalation channel separate from the usability lead. When only the usability lead serves as a contact, it can be challenging or awkward to communicate when difficulties arise in the consulting engagement—especially if the usability lead has created the problem!

Change Management Ability

The consultant must provide high-level strategic guidance in the organizational process. Organizations must often realign their operations to support a user-centered design process and they almost always have to manage resistance to change. The consultant will actually provide assistance in change management. Change management refers to making changes in a planned and systematic way so as to make the transition to new processes easy and effective. This kind of guidance should include pointing out pitfalls, helping to identify and manage specific areas and people in need of special attention, and opening communication channels. The consultant should directly support the organizational process and have many examples, case histories, and research to substantiate his or her position.

Select a consultant who will help you meet your essential political goals and will defend your best interests, even at his or her own expense—one who will “stand in front of a freight train” for you. The consultant should have the ability to talk as a peer to anyone, at any level of the organization.

For some organizations, change management may be an important but not critical factor in the success of the usability implementation. For other organizations with a history of change management issues—or with highly charged changes already under way—effective change management might be a “make or break” factor in implementing usability.

Quality Control and Feedback

Some organizations rely on the skills of a single practitioner to ensure the quality of usability process deliverables. In such a case, there may be no standard process, review, or oversight. This is like the early days of programming when the work was done in a garage by a master programmer. If he or she was good enough, the code would work, but there was usually no concept of a separate and formal quality assurance process. The only concept of process improvement in this scenario was trial and error.

Look for a consultancy that has good methods to ensure quality. Is there a method of certifying staff quality and knowledge? Is there a review process for deliverables? Is there a process for gathering feedback about the quality of the design and results and improving from lessons learned?

Make sure the consultancy has, at a minimum, a systematic user-centered methodology and actually follows it. Such adherence ensures a more reliable application of design. This methodology is just the minimum, however. You should seek out a consultancy that engages in ongoing process improvement. Provisions for quality assurance and process improvement indicate a more mature organization that will provide you with far better service and support.

Ongoing Training for the Consultancy’s Staff

Earlier in the chapter, we mentioned that the consultancy should provide training for your organization’s employees, but it is also important to know how current the firm keeps its staff. The usability field is constantly adding new principles and methods. If the consultancy does not gather and disseminate these insights, you will not benefit from current best practices.

For example, usability specialists once thought that the optimal limit on menu size was either 7 (plus or minus 2) or 10 items [Miller 1956]. More recent research suggests that the optimal menu size is more like 16–36 items, as long as they are presented in groups of not more than 10 items each [Paap and Roske-Hofstrand 1986]. Insights such as these, which are often presented at critical conferences, can be valuable to your projects, so you should be sure that at least some of the consulting firm’s staff attend these meetings. Chapter 8 lists some of these top conferences in the field. In addition, a continuous stream of useful books, articles, blogs, and e-communications can provide up-to-the-minute information about the UX field.

Selecting a consultancy that keeps its staff up-to-date on the best practices in usability and user experience ensures that those staff members can bring those practices to your organization’s emerging user-centered design program.

Summary

A good usability consultancy is invaluable in helping your organization transition to an efficient and thorough user-centered design process. It is worth spending additional time screening consulting firms early in your institutionalization effort to make sure that you choose one with the most appropriate capabilities for your organization.

The next chapter provides insights into how you can create a practical strategy for your efforts that includes the activities and resources you have begun to formulate.