Exploring the Types of UX Research Methods
A UX study is an opportunity for teams – product, design, accessibility and beyond – to get a look into the actual experience an end user might have, or is already having with your designs, products, and services. It’s not always easy to understand the user experience with its inherent landscape of complexity. This is why various types of UX research methods have arisen over the years. Fortunately, organizations can enlist the help of a third-party firm that specializes in UX studies and let them round up real users that can provide real UX insights.
In this blog, I cover the basics of the types of UX research methods that exist, how they differ from one another and what they provide organizations that are looking to improve their user experiences. I’ll share details on both qualitative and quantitative study types, and how examining actual data and data trends can remove the guesswork for organizations trying to create intuitive and easy-to-use experiences.
Generative and evaluative qualitative UX studies
Generative research
Sometimes referred to as discovery or exploratory research, this type of UX research allows us to develop a deeper understanding of users, their motivations, behaviors, typical workflows and pain points. This method can be used before a design is ever created to help inform what should be developed and what might best solve the users’ problems. We often recommend this methodology to our clients in the AI space. When the product and design teams understand users’ pain points with their current workflows, then they can better pinpoint where, when, and how AI might help solve those problems and avoid building a shiny, new toy that no one wants to play with.
Evaluative research
This UX research type helps you uncover what is or isn’t working well with a design. You may have conducted generative research to get insights into how to best create these designs, but you’ll want to test the designs with your users early and often. For these studies, users may be asked to complete tasks using the designs and then provide their reactions. We can conduct this type of study on an existing design in production, a prototype of a user flow, or even a design on a napkin.
A strong UX research partner like Applause will analyze the study results, share feedback, and provide recommendations to help product and development teams better address unearthed user issues.
Customer journeys (CJs) fall under UX research, as well. These efforts tend to focus on specific user flows. For example, a CJ might help you uncover how smooth a checkout experience is, or the process of placing an order online and then going to a store to pick it up (BOPIS). Unlike UX studies, with Applause, CJs also involve checking for functional testing issues that may impact the UX, whether it’s a broken feature, or a functioning part of a feature that can be improved to reduce user friction.
Qualitative UX research methods
There are two types of qualitative UX research methods that we often use at Applause: remote unmoderated and remote moderated UX studies. For a broader view of UX testing, access our User Experience Testing: Questions Answered blog.
Remote unmoderated UX studies
Teams want insight into their users’ overall experience with their designs. It’s important for them to understand how people typically use their product, and in doing so, users’ main pain points or things that they love.
These studies are usually more evaluative in nature; participants typically review a design and may take up to an hour to complete the task-based study. Participants complete this study by themselves, on their own devices and on their own time. As the name indicates, there is no moderator involved.
Remote unmoderated UX studies can be executed several ways. One way is through an open-ended survey that participants complete at their own pace. Another way involves having participants video record themselves while they go through various tasks. At Applause, we conduct these types of unmoderated UX study types on a regular basis; they are one of our most popular offerings.
The typical size of these research studies is 15 people: 5 people for each condition. So, for example, if you’re testing the onboarding flow for a new streaming service, you might have five participants onboarding on iOS mobile phones, five on Android and five on the Web. However, you might add other groups of five – a set on a Roku set-top device, another on Amazon Fire TV, others on Apple TV, etc. These numbers are based on UX best practices over the years, where the maximum return on investment (ROI) for UX studies occurred with 3-5 participants. Still, there are times where we diverge from these standards depending on the needs of our clients, and where customization may be required.
We also run diary studies. Just like keeping a personal diary where you record key parts of your day for reflection at a later time, diary study participants record their experience throughout the day as they use a particular product or service. They might do this as they are shopping at a grocery store or using a new device. We may receive information from these participants on a daily or weekly basis, shedding light on their experiences over time.
Remote moderated UX research studies
Remote moderated UX research studies typically involve a one-on-one interview with a participant and researcher. For these, we may have a design and have the participant go through a flow, or give feedback on a static image or even some copy. We’re able to do this at any stage of the process – from designs on a napkin to prototypes to fully functional websites or apps.
For these types of UX research, we generally have 10 total participants. These sessions are usually conducted over the course of a week and typically take about an hour. However, we can do them much faster if needed.
After the sessions are complete, we analyze the data; this may take a week or two depending on the size of the study. In total, including planning, these take about four to six weeks for a typical study and anywhere from six to eight weeks for a larger one.
What are quantitative research studies?
The other major type of UX research methodology, quantitative research, focuses on larger sample sizes. With this type of study, researchers often use surveys with closed-ended, ratings-style questions to gather data from hundreds or thousands of participants. On average, these studies may take only 5-15 minutes to complete, significantly less time than participants would spend completing a qualitative study.
The findings of these studies tell us ‘what’ is happening and what issues occur within the users’ overall experience, whereas qualitative studies will help us understand the ‘why’ behind the issue and also give insight into ‘how’ we might fix it to improve the users’ experiences.
Many of our quantitative study customers want competitive benchmarking of their Net Promoter Score (NPS) or Customer Satisfaction Score (CSAT) to learn how they measure up against competitors. We may repeat these benchmarks every six months or once per year. Quantitative’s superpower is that the findings give us confidence that what we found in the survey would hold true no matter who we asked within that target population.
We often prefer to run quantitative studies ahead of qualitative studies as the findings may provide us with insights into specific trouble areas for a greater number of people (the ‘what’). We can then conduct a follow-up qualitative study with 10-15 participants to dive more deeply into those issues, uncovering why those issues are happening and how the pain points might best be alleviated.
For example, a quantitative study may show that users in a particular country do not like using a voice assistant. This might cause the organization concern that they may need to reconsider launching within that market. In this case, we would recommend following up with a qualitative UX study to uncover why the target population doesn’t like using the voice assistant and how potential customers in this market might be convinced to use a voice assistant in the future.
Who are the researchers?
At Applause, we have a great team of UX researchers spread around the world and ready for studies as needed. They are highly trained in behavioral research and/or data science, and well-prepared to execute on the various types of UX research methods. Many have degrees in human-computer interaction (HCI), psychology, anthropology, or sociology. A good number of our team members have master’s degrees, and we also have a few PhDs. While our highest concentration of researchers is within North America, we have researchers in Europe, Israel and many other countries across the globe.
Having in-market researchers provides a big advantage to our customers, as many companies based in North America want to expand their reach. While English may be spoken in the target market, there can be vast cultural differences. For example, a customer may ask us to do a study in North America, India, and Germany. We may have a North American-based researcher lead the study, but then we’ll bring in global in-market researchers to work alongside the lead researcher. The in-market researchers don’t just provide UX findings, but most importantly, they also provide us with cultural insights that the North American researchers might miss.
What kinds of things are found while executing different types of UX research?
I’ve seen a wide array of UX findings over the years. Some are very common, like users having trouble finding a button that is placed below the fold on a web page. But one of the things I enjoy most is discovering things for our clients that are both something they weren’t aware of, and that helps inform and reshape their strategy and next steps. It’s very, very satisfying to uncover these insights and exemplifies the need for on-going UX research, as technologies and social trends continually change.
Case Study: A grocery shopping study
We executed a grocery shopping study for a global e-commerce company that was looking for insights into customers’ experiences when ordering via smart speakers. The company wanted to gain a better understanding of shoppers in India and we uncovered some very interesting results.
First, Indian users were shying away from using the speakers because the speakers were doing a poor job of understanding their various accents. But beyond the issue of understanding local accents, we found that cultural habits around shopping had a significant impact on use.
Shoppers in India were more likely to shop in more diverse ways than shoppers in North America: Because the participants we interviewed were largely vegetarian and would often consume fresh fruits and vegetables in their diets, they tended to shop much more frequently during a week than what we had observed in the U.S.
Rather than shopping at only one primary store, the participants shopped at a variety of locations during a typical week — for example, in a larger supermarket as well as with a small, independent vendor in their community. And, more multigenerational households might require more shopping. Much more communication would need to happen across the household to coordinate shopping, as there may be multiple people shopping for the extended family at any given time.
You can see that the company had to think a bit differently about the use of smart speakers within this particular cultural context. And, while the client knew that accents might be an issue, they did not realize how significant the issue was. After this study was completed, we suggested that the company conduct related functional testing that would help as they progressed the product for this market. Our research enabled the company’s product team to reset their expectations, reapproach their thinking and consider how to best improve the use of smart speakers within this market.
Localization efforts are key to establishing great user experiences. Its importance here cannot be overemphasized.
Case Study: A quantitative study for a global cryptocurrency platform
We worked with a global crypto firm that had a long list of questions and tasks pertaining to multiple countries. Their team really wanted to understand how they were viewed within countries outside of North America and where their weaknesses may lie. We mapped out a quantitative study plan, as a qualitative research study would have been a massive project and not in the best interest of our client.
When we had initial meetings about this UX study, the client thought they had customer retention issues in Germany and the UK. After we did the research, we found minor issues in these two countries, but found that the biggest issues were actually occurring in France. This finding gave us a specific target to begin further research: Why is retention an issue in France? Was there a cultural reason? Was there a specific competitor in the country that impacted retention? Our qualitative researchers then set out to conduct this research to dig into the ‘why’ behind these noted issues and ‘how’ this company might improve the overall experience for their French customers.
So, sometimes we validate a suspicion on the part of the client, and other times we find other issues. In this case, both happened.
Functional testing and UX testing are closely tied together
When you work on digital experiences, you quickly understand that great UX depends on testing with real people that match your customer demographic. If you want a frictionless and pleasing customer experience, you must build it using real user feedback.
We have a really interesting blended offering here at Applause that we call User Feedback Analysis. This offering applies a UX lens to functional testing, providing directional UX feedback to uncover UX and/or design issues prior to release, or at any time throughout the product life cycle.
For user feedback analysis studies, we work with clients who are interested in bug hunts. They’re mostly looking for functional issues, but they may have several UX-related questions too. Our researchers work alongside our functional team, customizing a brief set of questions based on the goals that the client may have.
As our functional testers look for bugs, they also look for experience-related problem areas, such as design- or copy-related issues. Our researcher analyzes the data, which we present to the client as a blend of functional and UX insights. The insights we uncover help provide the client with direction. Are there UX issues reported? Is more investigation needed? If so, what type, UX or functional? The special sauce here is that we have a UX researcher at the helm, providing these insights and next step recommendations.
Customization to meet your UX needs
Applause is here as a trusted partner, dedicated to consulting with our clients to meet their diverse needs with precision and expertise. Our ability to strategize and implement the most effective UX research methodologies — be it qualitative, quantitative or a seamless integration of UX and functional testing — sets us apart. Our approach is not one-size-fits-all, it’s tailored to the specific needs of each client. Our commitment is to deliver unparalleled value and insights, ensuring your success is our success.
14 min