Measuring Findability and Navigation - Analytics for Product Development - Customer Analytics For Dummies (2015)

Customer Analytics For Dummies (2015)

Part IV

Analytics for Product Development

Chapter 15

Measuring Findability and Navigation

In This Chapter

arrow Valuing findability

arrow Measuring findability

If customers can’t find what they’re looking for on your website — say your navigation is poor or your search function isn’t working — customers will move on and find what they’re looking for with another company. All your effort in regards to pricing, features, marketing, and packaging won’t matter; the end result will be a lost opportunity. Your customers have to be able to find things quickly and easily.

In this chapter, I show you how to determine what customers are trying to find, then how to measure their capability to find the items (findability). I also give you ways to improve your findability and then measure whether changes you’ve made actually improve your findability.

remember While this chapter mostly talks about websites, findability is more that just your website. It pertains to wherever customers interact with your company. That could be software or mobile apps they’re navigating, or how you display products in your bricks-and-mortar store. You can apply the information in this chapter to every area of your company.

Finding Your Areas of Findability

If you’ve ever looked for a product on a website, tried to change the settings on your iPhone, hunted for food in a supermarket, or searched for a movie on cable TV, you have some idea about findability.

Findability is the percent of items that a customer can find successfully, how quickly items are found, and how much difficulty customers have in locating an item.

Customers expect quick and easy access to products and information. How do customers find items on your website? Take a critical eye to these areas of your website:

· Search boxes: Search functions are vital to visitors finding what they want.

· Categories: If you categorize your items and label each category appropriately, then visitors can find what they’re looking for.

tip You can offer categories in the form of links that visitors can click through or offer drop-down lists that visitors can drill down to find what they’re looking for. Categories (called navigation) are usually listed on the left or right side of the page for quick access.

· Breadcrumb links: Breadcrumb links are useful navigation tools so visitors don’t get lost in your website. These links show visitors the path they’ve taken to get to the page they’re currently viewing. These are generally links at the top or bottom of the page that matches the navigation, and visitors can click to move back to where they’ve been in the path.

· Suggested items: If you already know a customer is interested in a product — based on a search term or past browsing (or buying) history — you can also offer a list of similar products. Something in that list might be exactly what your customer is looking for.

Figure 15-1 shows how one popular company, Amazon, uses these navigation features to enhance its findability.

remember The capability to find the right item or function can mean the difference between a potential customer completing a sale or moving on to another website. You have to make sure that your findability is as good as it can be in order to prevent losing those customers.

9781118937594-fg1501

Figure 15-1: Popular areas of navigation increase your findability rate.

Identifying What Customers Want

The first step in understanding findability is to know what your customers are looking for. If you have a diverse customer base, you need to identify a target segment and persona to use for your findability study. Check out Chapters 4 and 5 to split your customer base into segments and then create a persona for each segment.

tip If you have a diverse customer base that buys different products, you’ll need to conduct your findability study for each target segment.

Identify a good cross-section of products or pieces of information that your target segment would likely look for on the website. Although you might have thousands of products, pages, and content, you don’t need to be comprehensive; just pick a representative sample.

tip Narrow the list to no more than 30 items. You should only ask participants in a study to spend no more than around 30 minutes. Anything more and participants lose focus and drop out, making data collection more costly.

Here are some ideas on how to identify those items:

· Search logs: See what users are searching for on the website’s internal search function. If customers are searching for a product, then it’s a good indication that they’re looking for that product.

· Your website analytics program: Analytics programs like Google Analytics provide key words and traffic logs to see which pages are most visited and what external search words customers are using.

· Purchase records: Products that are purchased more frequently will have the most traffic, and even small improvements in findability can result in substantially higher revenue.

· Comments and complaints: An easy place to find problems with your categorization system is to see if customers have complained about not being able to locate items.

· Your existing customers: Ask customers to pick their top five products or things they look for when browsing or purchasing. You can also ask them which product, if any, they recall looking for and whether they were able to find it.

You can ask website visitors to take a 5-minute survey using a website intercept. Questions to ask could be who they were, why they were visiting, and to pick the top five most important pieces of information they want to locate on the website.

· New and known issues: Sometimes you’ll know that items can’t be found. In other cases, you have a new product that you’ll want to know if customers can find.

For example, maybe you have a seasonal item that you only offer during certain times of the year. You know ahead of time that visitors might not be able to find the item or not know that you carry it (because it’s seasonal), and you can figure out ways to make it more findable.

tip People usually search for products by category (for example, barbeque grills) and by brand (for example, Weber Grill). Look at existing customer data to see how customers search for items. If they use brands or generic categories when searching, you’ll want to use the same terms because you’re measuring findability.

Prepping for a Findability Test

Before you can launch your findability study, you have to do some things to prepare for it. You need to know what your existing findability is — after all, you need to know where you’re at now to know what changes you need to make your findability and navigation better in the future. Setting up your existing navigation in a tree test can aid in your findability study. These tasks set you up for a successful findability study.

Finding your baseline

You need to find what your existing findability rate is — baseline findability — for items before you start making changes. You need to understand how well customers can find the items using your existing navigation structure. It’s important to know how findable items are now so that when you make changes to labels, names, and navigation structures you have something to compare them to. Make sure you aren’t making it harder for customers to find products!

For a baseline findability study, you need participants who represent your customer base to try to locate a typical set of items using your current website navigation. The resulting metrics will become your baseline findability.

remember You may think that customers always use the search box on your website. But research I’ve conducted shows that around half of customers start with a “browse-first” approach on most websites.

Although the fonts, layout, colors, and overall look and feel of a website play a major role in helping or hindering customers from finding items, isolate problems with the taxonomy — the order, labels, structure, and hierarchy of categories that contain the products or information — as a separate issue from navigation. After you have a good navigation structure, you can then test the design and know that the navigation is likely not the culprit causing problems.

Designing the study

For electronic product searches (websites, software, mobile apps), the best way to test findability is to replicate your existing navigation structure into a tree test. It’s called a tree test because it looks like a tree trunk and branches, much like old versions of Windows Explorer.

tip While you can conduct a tree test using pencil and paper, the most efficient way to test a navigation structure is using software. Software can streamline the process by automating a lot of the functions and doing the hard work for you. You can find software from companies like UserZoom (www.userzoom.com) and Optimal Workshop (www.optimalworkshop.com).

The following steps show you how to conduct a tree test using UserZoom:

1. Replicate your website navigation structure.

This should be the top two to three levels of your navigation structure. Figure 15-2 shows the navigation structure for the National Multiple Sclerosis Society replicated in UserZoom.

9781118937594-fg1502

Figure 15-2: A navigation structure in UserZoom mirroring the actual navigation from the website.

2. Add the items you want to test, and the search terms visitors typically use to find those items.

tip To understand what items visitors were looking for, review the page views and search logs for what pages and terms were the most popular, ask the web and content teams for what kind of content people typically want, and conduct a survey on the website. If you aren’t sure what your customers are looking for, see the earlier section “Identifying What Customers Want.”

3. Identify the correct paths users should take to find each item.

As part of the study setup, identify which locations (the leaves in the tree) are correct paths users can take to find the information. The software tracks the paths users take and compares them to the paths you’ve identified as correct. Figure 15-3 shows the navigation tree with the correct paths identified.

9781118937594-fg1503

Figure 15-3: Paths that users should take to find items are mapped.

Looking at your findability metrics

Before you launch your findability study, you need to know which data UserZoom collects for your baseline metrics. I discuss the most common metrics in the following sections.

Findability

Your key metric is findability or findability rate: whether the customer successfully finds the product or piece of information successfully. This is just like the completion rate in a usability study (see Chapter 14). Did the customer find the intended item (yes/no)? This answer gets coded as 1 for Found and 0 for Not Found. For example, if 70 out of 100 participants found the item, then the findability rate is 70%.

tip An item can live in multiple places, so double-check “right answers.” Remember: Participants and real customers will often look in many different places.

Time to find

If you ask users to locate an item, they assume the item can be found. Otherwise, why would you ask them? This is one of the inherent biases in findability testing. If users eventually find the item but it takes them a long time, then you have a findability issue. The tree-testing software records how long it takes participants to locate (or fail to locate) an item.

The median time on task is an excellent way to understand the average time it takes participants. To also gain perspective of how consistent the experience is, look at the variability of found item times. Divide the standard deviation of the task time by the mean time. High values indicate high variability (this metric is called the coefficient of variation).

Task Difficulty

If you ask participants how easy or difficult it was to locate the item after they finish looking for it, you can use a rating scale to measure the perceived difficulty. A seven-point rating scale called the Single Ease Question (SEQ) (discussed in Chapter 9) gauges how hard it is for users to locate items. So if there are 12 items to locate, you have responses to 12 items.

Scores below about a 6 on the seven-point scale are below average in ease (and therefore are more difficult).

Open-Ended responses

If participants rate an item as relatively difficult to locate, you can ask them to describe in their own words the problem they encountered. These remarks become excellent ideas for improving the navigation and findability. They help provide the “why” behind the numbers.

Card sorting

If you want to test how clear your category labels are, ask participants to sort your items into their own categories and then label them (a technique called card sorting). Check responses against the way you have your categories labeled, which can help you understand the thought process your customers use in finding products.


National Multiple Sclerosis Society

The National Multiple Sclerosis Society (NMSS) wanted to redesign its website. Members and visitors loved the wealth of information that the NMSS maintained online, but had trouble finding information when they needed it. Instead of just starting with a brand new website with a new navigation structure, the NMSS conducted a findability study in UserZoom. First, it measured the baseline findability.

To understand what items visitors were looking for, page views and search logs for what pages and terms were the most popular were reviewed. The web and content teams were asked to create a list of information that people typically want. In total, people were looking for 64 pieces of information, including information about newly approved medications, information about specific symptoms, and ways of sharing information via social media.

To reduce the items for website visitors to find, the NMSS needed input from qualified participants. It’s difficult to find participants in the MS community from a panel agency, so the society used a website intercept launched from its website (www.nmss.org). For two weeks, website visitors were asked to answer a five-minute survey about who they were and why they were visiting, and to pick the top five most important pieces of information they wanted to locate on the website. In total, data was collected from 1,300 website visitors using their desktop computers or mobile devices (tablets or smartphones).

Ranked in order, the top ten items were selected for the findability study. The terms the visitors used in the tasks were added into the UserZoom software for testing.

The discovery: 77% of participants found the location for information around new treatments, a reasonably high findability rate, as shown in the first figure. In comparison, only 44% found the information about vitamin D as a treatment, and an even smaller number of participants, only 9%, located information around financial assistance programs. Both these tasks have high difficulty ratings as well, which isn’t surprising: the NMSS already knew the items were difficult for participants to find. But it did find out that although the New Treatments task had relatively high findability at 77%, participants thought it was about average in difficulty, as shown in the second figure. To find out how these problems were fixed, see the “Improving Findability” section.

9781118937594-sb1501


Conducting Your Findability Study

When you have your tree test set up the way you want it, you can add profile and demographic questions to the beginning or end of your study. You can also ask participants which items they found the most difficult to locate. Figure 15-4 shows a schematic for a typical tree test.

9781118937594-fg1504

Figure 15-4: The hierarchy of tree-testing metrics.

warning Be careful about what you ask your participants so they don’t feel like you’re invading their privacy. If you ask for income, age, or other personal questions, consider making these questions optional if it makes a difference between someone continuing your study and opting out.

The total time it takes to collect responses for a tree test depends on your sample size and how difficult it is to recruit participants. Typically, you need 50 to 300 participants to take a tree test and you collect the data within a week or two.

Determining sample size

The ideal sample size for running a baseline findability study is based on identifying the tolerable margin of error around the estimate (see Chapter 2 for a reminder on the margin of error). The following minitable, which is derived using the margin of error from a 50% binary (yes/no) response question, shows the approximate sample size needed to achieve a specific margin of error.

Margin of Error

90% Confidence (+/-)

24%

10

20%

15

15%

28

14%

32

13%

38

12%

45

11%

54

10%

65

9%

81

8%

103

7%

136

6%

186

5%

268

4%

421

3%

749

For example, if 103 customers completed a tree-test study and 50% found an item, you can be 90% confident the percentage of all customers locating the item in the tree test would be between 42% and 58%.

Recruiting users

Like a usability study or survey, you need to find qualified participants who represent your customer base, and then have them attempt to locate items. Generally, you can find customers to participate in a findability study in three places:

· Customer lists: If you keep lists of current customer contact information and have permission to contact them, then you can simply ask, and usually, you compensate these customers to participate in a study.

· Panel agencies: Companies such as Op4G (http://op4g.com), Toluna (www.toluna-group.com), and Research Now (www.researchnow.com) have active panels of millions of people with a range of profiles that mimic the U.S. or international populations. You provide these companies with the type of customer you want and then pay per completed study.

Panel agencies recruit physicians, engineers, IT administrators, CFOs, attorneys, and small-business owners to participate in findability studies. You can also recruit based on habits, such as people who have a Costco membership or who have recently purchased a car.

· Website intercepts: For testing website navigation, one of the best places to find qualified participants is from customers actively visiting the website. Using a pop-up intercept from companies like UserZoom (www.userzoom.com) and Ethnio (http://ethn.io), a simple line of code will pop up and ask website visitors to participate in a study. It will even redirect them to an online study, allowing for virtual real-time data collection.

Analyzing the results

With the data collected, summarize the findability metrics by task: findability, task difficulty, and task time.

1. Calculate the findability rate as your first (or gateway) metric.

If users can’t even locate the item, then it’s a findability issue. While it’s unrealistic to expect 100% of participants to find each item, you should aim for findability rates in the high 80%-90% range. Figure 15-5 shows findability rates for products on Target’s website. Around half the products had reasonable findability rates, with a few, including the Wildkin Kaleidoscope Backpack, had very low findability rates.

remember A high findability rate doesn’t mean participants think an item is necessarily easy to locate. Participants may still take a long time or found the number of steps or choices too difficult. Finding an item is necessary but not sufficient to achieve high findability.

2. Examine how difficult it was to find the item by looking at the average score to the difficulty question.

For 7-point rating scales, the average score should be above a 6 — that’s the average level of difficulty. Figure 15-6 shows the perceived difficulty of finding the same items. Even though nine items had findability rates above 50%, only four items had difficulty ratings that were above average. These lower-rated items are good candidates for you to improve their findability.

3. Calculate the median time to find the item.

For the participants who did find the item successfully, compute the median time to find the item. The median is the middle value of a set of times. It can be found using the Excel function =Median(). While the actual time it should take to find an item depends on the complexity of the tree, aim for times to be less than 30 seconds.

9781118937594-fg1505

Figure 15-5: Some items rate high findability, while others are on the low end.

9781118937594-fg1506

Figure 15-6: Items that rank below average in difficulty are areas to improve.

Improving Findability

When you’ve collected all your baseline data, it’s time to do something about improving the findability. Here’s how to dissect the information gleaned from your tree-testing software:

1. Look for items that were hardest to find.

2. Look for items that were rated more difficult (even if they were found).

3. Look for items where users took too long to locate.

Anything that takes over a minute to find is too long; ideally items should take less than 30 seconds to find.

4. Examine the open-ended responses participants provided to see why items are difficult to find.

In the following sections, I offer insights into how to fix the issues you find.

Cross-linking products

Tree-testing software tells you not only whether customers found the item, but also the paths they took, as well as the paths taken by customers who didn’t successfully find the items. Look to see where customers are going and if possible, add a cross-link or recategorize your products or information.

remember Cross-linking is especially useful for websites and software where you don’t need to physically place items.

For example, in a baseline findability study with the National Multiple Sclerosis Society’s website, only 12% of participants found the correct location for financial assistance information — one of the top reasons people visit the website.

Figure 15-7 shows the output from part of the tree test. The correct location was “Society Programs and Services,” as indicated with the checkmark and box. The first percentage shows that 12% of participants at some point considered this option, and the second percentage, 9%, shows the percentage of participants who ultimately selected this as the location. In looking at the options immediately above the Society and Program Services, a substantial percentage (44%) selected Insurance and Money Matters as the place they’d expect to find information on financial assistance programs. Adding a cross-link from Insurance and Money Matters or moving the content solves the problem.

9781118937594-fg1507

Figure 15-7: In this example, 44% of participants looked in a category that didn’t have the right information.

Regrouping categories

One of the biggest problems you might have with findability is the category names you use. You might use jargon or terms that are familiar with you, but less so with your customers. Your customers can have problems navigating your website if your categories aren’t labeled properly — for them, not you.

Tree-testing software can show you what your customers expect category names to be. A complementary method, called card sorting, helps identify labeling problems. With card sorting, users take your products and sort them into categories and then give their categories names. Look at those customer-provided names and consider changing or adding to your categories to aid in better navigation.

Rephrasing the tasks

Sometimes, low findability rates can be caused by participants not fully understanding what they are looking for. For example, gift cards are popular purchases on retail websites. In a target.com tree test, the most difficult item to find was the “Iconic Puppy Gift Card $5-$1,000.” Participants were confused when looking for the item. In reading the comments, participants didn’t know if the item they were looking for was a gift card or a $5 puppy!

If you get a lot of similar comments, try rephrasing your tasks to be sure participants understand what to look for.

Measuring findability after changes

Now it’s time to find out whether the changes you’ve made to your navigation actually improve the findability and navigation of your website. Here’s where having the baseline data pays off. The way to know if you made improvements or made things worse is by comparing the findability of the same items after you’ve implemented changes.

· Findability: Did your users find more of your items using your new navigation? Aim for findability rates above 70% (but any improvement is good!).

For example, in a tree test on a retail website, an item of children’s furniture had an initial low findability rate of 29%, but it improved to 74% after changes were implemented. The baseline test showed users split across categories; the improvement came from adding links to the correct department from those categories.

Figure 15-8 shows the 90% confidence intervals for the percent of users who found the item before and after the change. The difference was both statistically significant (p <.01) and practically significant (the findability rate more than doubled!). See the appendix for what p-values are and how to interpret them when comparing two percentages.

· Difficulty: Did users find it less difficult to find the items? You should find overall that the difficulty rate went from harder than average to easier than average.

Figure 15-9 shows the perceived difficulty of finding the children’s furniture using the rating scale of difficulty. Higher scores indicate an easier experience finding the item, compared to hundreds of other tasks. Before the findability fixes, the item scored easier than 37% of all tasks. After the change, the perceived ease score was easier than 59% of all tasks in the database.

remember As with measuring the user experience in general, measuring findability in particular involves multiple metrics and multiple methods to show a quantifiable better navigation.

9781118937594-fg1508

Figure 15-8: Percent of users who located an item before and after improving cross-navigation links.

9781118937594-fg1509

Figure 15-9: Percentile rank of perceived difficulty using the Single Ease Question.

Figure 15-10 shows that only 1% of participants were able to locate a prom dress on a department store website using the old category “Guys and Juniors.” After changing the category to “Teens,” based on the input from participants in a card-sorting study, the findability rate improved to 74%!

· Task time: When users located the item both before and after the change, were they able to find it in the same amount of time or quicker after changes were made based on the first set of results? Figure 15-11 shows the median task time for three items found on a ski resort website. Customers were able to find vacation package deals faster on the improved navigation structure compared to the old. In the other two tasks, the times were not statistically different, suggesting changes that benefited one task didn’t harm the others.

9781118937594-fg1510

Figure 15-10: After improving labels and item placement, items in this website were much easier to find.

9781118937594-fg1511

Figure 15-11: Improved findability minimizes the time users take to locate an item.