UX research is powerful—but slow. Finding participants, analyzing responses, and making sense of data can take weeks.
What if AI could speed up your research without compromising quality?
Here are five ways AI is transforming how we do user research and two pitfalls to watch for.
1. Card Sorting at Scale
Card sorting is a great exercise to understand how users group and make connections between information. Done well, it leads to more intuitive navigation, clearer structures, and a user-first information architecture instead of one that reflects an organization’s internal silos.
The Challenge:
For an effective card sort, you need 15 participants, sometimes up to 30-50 to identify meaningful patterns.
- Users can also only handle a certain number of cards to sort. Otherwise, they get tired and overwhelmed.
- Recruiting and running a robust card sort study can take weeks, making it difficult to iterate quickly.
How AI Can Help:
Large Language Models (LLMs) like ChatGPT can generate initial categorizations that supplement real-user data. Because LLMs are trained on vast amounts of human-generated text, they can simulate how people naturally organize and relate concepts.
This means that asking ChatGPT to complete a card sort will get a pretty good grouping and overall intuitive labels. I’ve seen a strong overlap in our user-generated groups and our AI-generated groups. Using AI allows me to test more cards (even up to 70-80) when we’re limited to ~50 cards with users. I also generate directional insights faster.
How to Use AI for Card Sorting:
- Ask an LLM to sort a larger dataset of cards (70-80)—something impractical for human participants.
- The AI-generated categories often align closely with user-generated groupings, validating its utility.
- AI speeds up early-stage analysis, helping me identify key themes before running full user sessions.
The output isn’t perfect, and the more unique your audience segment is, the less useful general grouping is, but overall, adding AI to supplement my user-driven card sorts has made my analysis more comprehensive and faster.
Example Prompt for Card Sorting
[TIP] I want you to take the perspective of a PERSONA. You should follow my instructions as if you were fulfilling this role. For context, I’m seeing how you would complete this task, which is a card sort task. Consider the following list of items and how you would best group them. Create as many groups as you deem appropriate, and give them descriptive names. Then, give me the groups and how you would categorize each item.
2. Write Usability Scripts Faster
One way I use AI almost daily is to get a first draft of my interview and usability scripts. While I haven’t had success using ChatGPT for screener questions (I always get overly simplified and leading questions) I love prompting for an initial usability guide. This works well for generative testing, where my guide is a framework, and probing questions
How AI Can Help:
Part of why I’ve had success is because I create and use a custom GPT for my usability guides.
By leveraging custom-trained AI models, I’ve built a question bank segmented by industry and audience (B2B & B2C). Instead of writing scripts manually, I upload this dataset into ChatGPT, allowing it to generate tailored usability guides based on best practices.
I save time, and ChatGPT is relying on best practices. I will continue to build my question and task repository and add additional examples for evaluative tasks. One of my next projects is to customize the GPT for moderated vs. unmoderated sessions, to keep reducing my own edit time.
Example AI Prompt for a Custom GPT to Write a Usability Guide
[TIP] Read the files I uploaded. Use these for formatting, tone, style, and required information. Ask the user for information about the goals, audience, and product. After that, act as an expert UX Researcher. Create a usability guide, following the formatting and styles in the Example Usability Guide for a moderated session that would last about an hour.
3. Transcripts and Summaries in Minutes
After you have your sessions recorded, generating transcripts is much faster using AI.
How AI Can Help:
AI tools like Zoom Assist and Otter.ai go can summarize themes, pull impactful quotes, and even evaluate sentiment.
Being able to search in the transcript, or pull first impressions has made it easier to focus on the insights. I’m able to spend less time scrubbing through videos and more time on the analysis.
Remember to always get permission for AI tools from participants and clients. Research your transcription tool and understand their privacy policies and data management before using these in your sessions.
4. Instant Feedback and Coaching
As a seasoned researcher, I’ve moderated hundreds of sessions. I’m always looking to improve and level up my skills, and using ChatGPT (see #3) as my own personal coach has helped me level up.
How AI Can Help:
Ask for feedback on a recent session, which I’ve found helpful to identify areas I can probe more in-depth and phrasing suggestions. I’ll always read articles and join webinars to improve my approach, but the personalized suggestions from ChatGPT have given me actionable and personalized coaching. Seeing feedback this way also helps me improve as a mentor to others.
Example AI Prompt to Improve Your Moderator Skills
[TIP] I recently moderated a usability testing session for a SaaS platform. The study aimed to evaluate how participants understand navigation and feature relationships.
Here’s the transcript of the session. Please review it and provide feedback on:
- My moderation skills, including neutrality, question phrasing, and participant engagement.
- Missed opportunities for deeper probing or clarifying responses.
- Suggestions for improving transitions, task instructions, and rapport-building.
- Insights from the session that I might have overlooked.
5. Building an Insight Repository
As Seer has scaled our UX research, sharing findings and nuance can be difficult.
I sometimes go back to previous projects for relevant insights and to understand any shifts in user perceptions. With AI’s ability to recognize themes and interpret a large amount of data at once, I intend to build an insight repository where all research activities and insights are stored. This is a growing space, and I expect the way these repositories are built to change with new models and features.
In the future, I see a world where we have a custom insights model, and researchers, designers, product managers, and executives can ask a question and get a research-backed answer custom to their users.
UX Research will be more impactful and valuable the more we can share insights and build on what we’ve already done.
It’s not all positive…AI also has some pitfalls in user research
Unfortunately, while there are many positives to using AI, there are a few places I’ve seen issues. Here are two things to keep an eye out for as AI is used more and more.
1. Participant AI Usage
When recruiting with platforms like UserInterviews, I’ve noticed an increase in users adding lists, AI-generated answers, and other tell-tale signs that they are trying to game the system to participate in a study they aren’t qualified for.
Watching out for fraud has always been a key part of recruitment, but now, even if users have the right answers, researchers need to keep a close eye on answers.
In an even more alarming situation, I conducted a session with a participant I can confidently say was reading from AI when answering my questions. I could see the device’s reflection on the screen as I asked questions and the participant looked over to read the ‘correct’ answer.
This has been a hot topic for job interviews, with candidates using listening software and reading off answers. As bad actors get more sophisticated, researchers need to be on the lookout for this as well.
Here are a few methods to keep sessions relevant:
- Require cameras to be on to see participants engagement and focus
- Be careful about leading questions or giving too much context or jargon
- Keep probing questions succinct and don’t be afraid to ask for more details
In my case, I was clear that if the participant wasn’t able to give more specific answers and real-life experiences I would end the session. I didn’t accuse them, because I can’t prove anything, but my objectivity was gone and I wasn’t getting valuable insights, so I couldn’t continue the session in good conscience.
At the end of the day, new features from testing platforms like UserTesting and UserZoom may need to be developed if this issue becomes widespread.
2. Over-reliance on What Users Say
I’ve outlined multiple methods to increase the speed of insight and analysis from user testing.
AI is a great tool for summaries and themes, but at the end of the day, it focuses on what users are saying.
As a UX expert, you need to be focused on what users are doing.
Watching behavior, understanding sentiment, and seeing facial reactions can be more telling than verbal feedback.
Don’t let speed and convenience overshadow the main focus of research or your products will be driven by what users say, and huge areas of both friction and delight will be overlooked.
[TIP] As Margaret Mead famously wrote, “What people say, what people do, and what they say they do are entirely different things.”
As a UX professional, I’ve seen how AI can improve my processes and make my user research more impactful. I’m not afraid to incorporate it, but it is a tool, with positives and negatives.
Start freeing your time up to focus on strategy by using AI in your processes, but keep in mind that you are the expert and pair AI with critical thinking and intentional application.
Speed up your research with insights from the Seer team delivered straight to your inbox. Subscribe below.
