Usability — Focusing on Users, Not Processes

In the Beginning

When we began user studies with our coaches, our initial focus was on the process of the study itself. We tried hard to ensure we wouldn’t lead our coaches with any of our own assumptions. We were very formal. We stuck to reading from scripts, fussed with tons of video cameras, and were almost robotic in our approach.

There was so much going on in trying to keep the study ​“on track” that we weren’t able to engage in any #RealTalk with our coaches about the actual problems we were trying to solve with the products they were testing.
This formal environment took a lot of time to set up and sometimes made coaches uncom­fort­able. Coaches don’t do their work in labs; they live on the field and in tiny offices full of gear and stacked playbooks and DVDs.

The New Beginning

After reading ​“Rocket Surgery Made Easy” and talking with others interested in user research, we realized we could get more out of our studies if they were simpler and more frequent. So we stopped writing and reading scripts and started talking with our users like the real people they are. Suddenly, we began to uncover tons of product research gold.
Active users love to talk. They love telling us what sucks and what they love about our products. Of course, we’re still careful not to lead them with our questions. We’re cautious in the transition between having them execute a task and asking follow-ups to understand their goals and mental models.
Our focus in the usability studies has shifted toward the user and and away from perfecting the process. We found that if you keep things simple and practice more regularly, you get more out of each study.

Any Type of Study is Worth it

Starting out, don’t worry about what format your study takes. What matters is that you actually talk to your users. Just do it. Do whatever works best for you and your team, but remember you won’t figure that out until you try different formats and learn which insights each one generates. Here are the formats that we use most often. (Obviously this isn’t everything.) The most important thing is to always talk to your user:

• Dogfooding. It sounds strange, but all we do is grab any warm body in the office and have them try to use the feature we are working on.These sessions provide another set of eyes on whatever it is we’re building and provide a ton of value. Finding problems early has saved us a lot of time.
• Call a user and put them on speak­er­phone so the entire team can listen to the con­ver­sa­tion. Ask them how they use your product and what sucks about it at every level (small details to big-picture issues).
• GoToMeeting/​Google Hangout. These tools allow us to share screens so we can let the user ​“show and tell.” It’s great if you want to watch the user interact with a feature on their own computer/​browser, and you can record the session for later reference. These services also allow our remote workers to join in on the study with little hassle.
• Visit the user where they use your product. It’s amazing how much you can learn just by being in their environment. Our devs and designers have powerful computers with huge monitors. How does your product look on a user’s 13-inch laptop from 2008, or on a phone out in the sun?
• UserTesting​.com is great for general public facing studies. We write and submit a simple task scenario built with their online tools. Thirty minutes later, we get videos from six different users trying to complete the scenario and tasks we outlined. It’s a powerful tool to see random users playing with your product for the first time. We use it to see whether or not our product ideas ​“just make sense” to people outside of our teams.

Be Willing to Learn Anything

You’ll get a lot more out of the studies if you let your users talk about the things they are familiar with. Many times we have gone into a study planning to learn about a specific feature, and the user will start a discussion about another feature in our product and how painful it is.

Be willing to talk about the features they want to talk about. You might learn how something as simple as delaying a hover state on a button can help a user’s process, when it’s such an easy fix for your team.

Get Everyone Involved

This is critical. Allowing your team to see a user struggle with a feature or product they’re working on creates a healthy empathy. As a team we get caught up adding feature after feature, and forget who exactly we’re building for.

Building up that empathy will force you to fix the important-but-smaller parts of your product. It’s much easier for the entire team to maintain that user-centered mindset on new and past features if they’re involved with every study.

Make one person responsible for leading and driving the study, but remember to get the project manager, designer, developer and QA in the room whenever possible. It’s easy for a person to feel intimidated if there are multiple people asking a wide range of questions right from the start, so don’t allow it until the main part of the study is over and it becomes time to fully engage in con­ver­sa­tion. This allows your user to stay in the flow of the basic study and get comfortable with what is expected of them before feeling overwhelmed with an entire group of people asking questions.

Keep it Short

We keep user studies to 30 minutes. Figure out what you want to get out of the study and go after that first. If they really want to talk more, just schedule another meeting. This keeps everyone engaged and motivated to keep doing regular studies. If it takes multiple hours every time you schedule a study, your teams may start to avoid them, even if they feel like they’re getting a lot out of them.

Take Actionable Items Away

What’s the point of doing a study if you don’t go over what you’ve learned? Plan 5 -10 minutes to talk with your team immediately after the study. This is when things are fresh in your head, and you can avoid tons of doc­u­men­ta­tion or com­mu­ni­ca­tion later.

Decide as a team on what action items you’ll tackle as the result of what you witnessed. This is also a good time to make sure everyone agrees with the takeaways and areas to address. You don’t want to refer to the study a week later and someone else on the team disagrees with what actually happened in the study.

Get Started and Don’t Be Afraid

It really is all about just doing it. Reach out to your users. If they’re using your product actively, you’ll be surprised by how many will be willing to help make it better. Then just talk to them about your product — what they like, what they don’t like — and stay focused on the root of the problem. Don’t let the idea of a big, formal usability study scare you away from doing the simple stuff that can be just as valuable.