User testing is a crucial piece of the puzzle that is user experience design. Various testing methods help experts understand the audience and their actions. According to Rogue Wave Software, 50% of an average developer’s time is spent on fixing issues that could have been avoided in the first place.
How could they be avoided, you might ask? Well, with the help of user testing, of course.
Understanding Different Usability Testing Methods
Thanks to different usability testing methods, product teams can better understand how the users interact with the product or service, as well as what problems they might face during these interactions. This helps them anticipate a lot – and with it, avoid many problems along the way.
Nowadays, there are many categories of usability testing methods. According to Maze, an amazing tool for creation of tests, usability methods can be generally categorized as:
- Quantitative vs. qualitative usability testing
- Moderated vs. unmoderated usability testing
Tools like Maze allow designers to easily craft the tests, therefore reducing the time spent and maximizing the results. You can use them to collect actionable insights through user feedback, all with the goal to create better user experiences.
The main distinction between qualitative and quantitative testing is widely known. It’s based on how data is collected. When you’re performing qualitative testing, you’re collecting data about attitudes and behaviors by observing the users. In contrast, quantitative data testing uses an indirect way to accumulate this data, usually accomplished with testing tools like user comments and product reviews.
In this article, you’ll learn the distinction between the two types in the second categorization. But, before you jump into performing usability testing, you should learn where to store and keep the data.
Storing data from usability testing
No matter which option you choose for usability testing, you will end up juggling quite the amount of data afterward. To keep it safe and make it easily usable by all interested parties, you should make sure that it is put in the right place.
The solution? You need quality knowledge base software.
According to Atlassian, a knowledge base is: “a self-serve online library of information about a product, service, department, or topic”.
This makes it the ideal place to store and keeps your usability testing data. Thankfully, there are numerous amazing options for free knowledge base software like Slite and ZenDesk. You can use these to keep data safe, but also make it accessible to everyone who needs it, as well as use it to improve your UX design.
Comparing moderated and unmoderated testing
UX design has evolved a lot in the past decade. This brought us a better understanding of how we can create greater products. Testing the usability is essential for that goal.
Still, even though designers are aware of the need for testing, it is very confusing to pick a way to do this. There are many options for carrying out usability testing. If you’re stuck trying to pick one, this article is for you.
There are pros and cons for both moderated and unmoderated usability testing, which is why this article briefly describes them both to help you decide which one is best for you.
Moderated usability testing
Moderated usability testing is a technique used to evaluate users’ behaviors and thoughts. It involves users and a trained moderator (or facilitator). Such testing is mostly used in a corporate setting, a UX lab, or remotely with the help of technology.
Usability testing tools help the moderator understand how the user interacts with the product or service. It also allows them to better connect with the users. Since this is real-time testing, moderators have better insight into the thoughts of participants as they take action.
The moderator is an expert in the product under evaluation. This person handles various things including the administration of tasks, recording of comments and behaviors, guidance of participants, etc. Moreover, the moderator needs to communicate with the participants, answer their questions if needed, and reply to their feedback. All of this is done in real-time, regardless of if it’s done remotely or in person.
Moderated testing can be done remotely or in person. In the first case, participants use technology to share their screen with the moderator. They might be asked to speak their thoughts out loud as they complete the given task, allowing the moderator to follow their train of thought.
When it is done in person, everyone is in the same location i.e. the same room. People communicate with the moderator directly and in real-time, providing him with insight into their thoughts and asking questions if needed.
Advantages and Disadvantages
- + Improvisation. Moderators can improvise when performing the testing. They can give additional tasks or remove unnecessary tasks when they deem this to be necessary.
- + Flexibility. Moderators can ask questions while participants do the test to better understand their behaviors. They can also request that participants share their thoughts whilst working on the test.
- + More insight. Since this is done in real-time and the moderator can discuss the thoughts of participants, the testing yields more detailed data for evaluation.
- – Tons of data to store/evaluate. The documentation of real-time results requires technical equipment and some preparation, especially if the team wants to get data in the form of audio and video recordings.
- – Time-consuming. This is a time-consuming testing option. It requires a lot of preparation, traveling to a location, as well as the actual presence of the team as participants complete their tasks.
- – Costly. Compared to unmoderated testing, this type can be much more expensive. When performing it, one has to consider expenses for the location where the testing will be held, transportation costs, equipment costs, etc.
Unmoderated usability testing
Unmoderated testing lacks one big element of moderated testing – the facilitator or moderator. Such testing is completed by its participants in their selected environment. Because it doesn’t require special equipment, excludes the role of a facilitator, and is done remotely, unmoderated testing is quicker and cheaper (in most cases). This makes it ideal for designers with low budgets and tight deadlines.
As the name tells you, unmoderated usability testing is not overseen and guided by a moderator. This means that there isn’t support in real-time, and the participants are not allowed to ask questions while performing their tasks.
Due to its nature, unmoderated usability testing is used for different purposes. Compared to moderated testing that makes a bigger picture, tasks without moderators are used to test very specific elements of the product.
There are two things that you must understand when trying to decide between the two. These are:
- Unmoderated testing is not recommended on low-fidelity prototypes or wireframes. Users can be confused by these and they’ll need some guidance to provide accurate results or complete the tasks.
Participants in unmoderated testing are more likely to be less engaged and invested. This can lead to less realistic results.
Advantages and Disadvantages
- + Self-moderating. The test participants are given a more direct role in their decisions. They moderate themselves and their decisions are in no way affected by the moderator.
- + Flexibility. This type of testing can be performed from any location and at any time. Participants can use an environment they feel comfortable in, on almost every device they have handy. Designers don’t have to be present and can see the results when they want.
- + Saves time. Compared to moderated testing that takes some time to prepare and execute, this type is implemented much faster.
- + Wider reach. Putting people in the same place can be complicated if they are located far from you. This removes the barrier and gives researchers a wider scope of participants to choose from.
- – No real-time questions. Designers cannot ask questions to understand the thought flow of their participants in real-time.
- – No interventions. There’s less flexibility in affecting this process. Since designers and mediators are not present, they cannot intervene if things go wrong, add or remove tasks when they deem it necessary, etc.
Which one should you use?
Experts in the UX field frequently debate this point – which of the two is better to use? The answer to this is not final. The right type of usability testing depends on many factors including the budget, interests, product, stage of the product, etc.
Generally speaking, you should make the choice based on your priorities. For some designers, the limited budget makes moderated testing on-site impossible. Their only option is unmoderated testing.
In other cases, even if the designer has the budget for moderate, they can benefit from unmoderated testing more. It all depends on their needs.
If you decide to go without a moderator, you can fully leverage the power of technology. You can get people from the other side of the world to participate in your testing without issues such as transportation or time.
But, in this case, some naturality is lost. There’s absolute silence – without a chance to jump in and ask the participants some questions that can bring a lot of insight.
On one side, participants can benefit from the space that’s given with unmoderated testing. Asking them questions can interrupt them and affect their actions. On the other side, they can benefit from a moderator who clears things up, and they can also process their thoughts better when they discuss them with others.
Some things are missed without the moderator. Which ones? Well, when people answer questions and perform tasks, they do more than write down words or say things. There are gestures and body language, which give insights that can never be received through unmoderated testing.
As you can see, both these types of testing have a big list of advantages – and downsides. This is why they are both so widely practiced and designers have used them to progress their strategies.
Usability testing, in general, is a practice that helps designers uncover the barriers in their design process, as well as improves the user experience. Different types of testing bring different results. Choosing the right approach is essential for you to meet your goals.
Let’s make a small, general comparison.
Moderated usability testing allows you to interact directly with the participants (your target audience). It gives you a chance to gain a more profound understanding of their goals, actions, and pain points. The downside is that this takes longer, costs much more, and limits the width of your test sample.
Unmoderated usability testing is much quicker to implement, much cheaper, and simpler for both sides. For you, this means more flexibility in checking the results and finding more participants. For participants, this means working on the tasks without interruptions and at the time they set for themselves. The downside is that the results are more limited in scope and that you have to make sure that everything is absolutely clear since users are left alone to handle it.
Recommendations and final thoughts
The best recommendation one can give you in terms of which one to choose is – choose both if possible. These are best implemented at their optimal time. For example, unmoderated testing is usually more effective when it’s done at the starting points when you need more generalized knowledge.
As long as you are testing, you are working to improve your product and its success. Trying both options will give you different insights and provide you with data that you can use to boost your success.
Your decision should also be led by your experience. How is this?
If you have less or no experience with user testing, you should try unmoderated testing first. This equals less preparation and gives the users a chance to moderate themselves. It’s simply more practical and prevents you from making a great error as a moderator – unintentionally affecting the users’ decision or action.
It’s also their experience that matters in the process. Do you want people who’ve participated in such tests or people that haven’t?
When you make this decision, you can decide on the type of testing to perform. Some want to get the results from more experienced participants who know how to complete such tasks. Others believe that the results are more honest and accurate if they use inexperienced participants. In most cases with unmoderated testing, people have already participated in similar tests.
What does it all come down to?
It all comes down to your decision. So, take your time, consider your options, and remember – it’s always wise to test.
