You Have to Walk Before You Can Test

Oh no. What is this guy doing? Where is he going? Why is he poking around in his account settings? We asked him to find new drivers for his golf bag. There’s a giant search box right there. It’s right THERE. I can’t even watch this anymore. How much time is left? Three minutes? Great. We’re not even going to get to talk about the new search feature.

Well, that was a waste of time. The next user test will be more helpful. We’ll actually get to the new screens we designed.

Except the next guy doesn’t get it either. And the woman after that? She doesn’t search the way you want her to at all. They are all doing it wrong.

This kind of thing happens all the time during usability testing. You never get to step three, four, or five of your new feature because the users can’t even find step one. Even though this can be very deflating as a designer and as a squad it’s important to remember that this is a win. It’s a win because you learned something and you learned it early enough to do something about it.

The Feature Test That Wasn’t

Here at Hudl we are building a new Playbook feature. Our assumption is that physically drawing a football play with all 22 players, routes, and blocks is time-consuming and difficult. In the first solution we tested, a coach could just tell us what play he wanted and we’d draw most of it out for him.

See the two form fields at the top? Our test users didn’t either.

We built some fancy type-ahead forms and predictive visual elements. All the coach had to do was start typing the name of a play and boom, players and lines would magically appear on the field below.

Except that never happened. Not one of the coaches we tested this design with touched the form fields at the top of the screen. Not one. Every single coach immediately started manip­u­lat­ing the play drawing tool — you know, the giant, sexy UI element in the middle of the screen just begging to be played with.

So this is a good thing?

Why did we consider these tests a win? The majority of our test users never even noticed the very clever predictive form fields that we now realize we almost went out of our way to hide at the top of the screen. Well, it was a win because even though no one got to step five we learned a hell of a lot about what’s wrong with step one.

We learned that coaches want to draw first and ask questions later. In a quick and dirty second iteration we learned that even with some not-so-gentle prodding coaches will still try to manipulate the diagram first.

We took it in even further. We handcuffed the rest of the interface until the coach gave us something to go on. In short, we made the easy way build a play diagram the only way to build a play diagram.

Any guesses what you should do here?

We started to get back data and anecdotes that were much more relevant after that. Coaches could start to see the benefit of what we were trying to build. They became more enthu­si­as­tic and forthcoming.

The next time you run a usability test and small roadblocks are keeping you from getting to your destination, try to remember that those blocks are important. That’s where you should focus because those roadblocks will teach you about your users and how they want to use your product.