“There are no shortcuts. If you’re working on finding a shortcut—the easy way—you’re not working hard enough on the fundamentals. You get away with it for a spell, but there is no substitute for the basics. And the first basic is good old-fashioned hard work.” Coach John Wooden.
John Wooden is known for winning 10 NCAA national championships in 12 years as head coach of the UCLA Bruins. The man clearly knew a thing or two about how a basketball team is meant to work—and he knew how to implement that knowledge into the game itself.
I’m a fan of the way Wooden dished out his no-frills wisdom. As someone who’s worked in the UX field for nearly two decades—and who’s been watching the recent fervor and excitement that’s being drummed up by AI and other emergent technologies—I agree with his sentiments above.
When it comes to user research, there are certain places where taking shortcuts just isn’t advisable and never will be. But there are other places where the field of UX can and will innovate and modernize, both on the technological and process planes.
All about AI: How it’s Impacting UX
Undoubtedly, AI is shaking things up considerably. But its relevance is best understood as a tool, not a be-all-end-all.
“AI” as a “we can replace users and researchers and USERS type concept” is and will continue to be distracting in UX and product design. For most organizations, the idea that AI tools will replace researchers conducting research is simply impossible. AI as a tool has some really cool ways of r automating time-consuming/ repetitive parts of the research process, such as generating research questions or gleaning preliminary, cursory insights from acquired data. But despite AI’s usefulness in finding clever (or more to it, time-saving) ways to use feedback data for research purposes, it’s not up to the task of contextualizing the information that’s been gained or knowing how to ask the right types of questions.
AI tools' effectiveness in UX largely depends on the quality of the data being fed to the tool. It sounds painfully obvious, but it bears repeating: accurate data feeds are necessary if data is going to be used in any practical and measurable way. An example of a real-time feed could be a chatbot interaction that logs customer feedback, which AI can then analyze to identify common usability issues that users are facing, which then directs researchers to areas where they should focus their research.
- A note about data dependency and limitations
- The ethical waters surrounding AI usage in research are murky at best. Many companies strongly resist having any of their information fed to the large language models that power these tools for fear of how it’s being used or who could access it.
- For researchers, this creates a bit of an impasse: if you don’t have new information going into the system you’re using to generate research questions with/glean insights from, then you’re basically working with old or stale information. LLMs lack the depth of information needed to properly contextualize any prompts given by a researcher who might be looking for a comparison analysis. A big part of why they lack this info is because companies don't want their information to be fed into LLMs in the first place. Getting a clean 1:1 comparison isn’t possible, and this lack of comprehensive data will likely hobble any “analysis” a tool like GenAI spits out.
- This being the case, the best thing researchers can do is learn how to use these tools to save time, but they should not rely on them as sources of accurate and comprehensive data—at least, not yet.
Shifts in researcher skill sets
Modern UX-Research roles are evolving. Today UX researchers are expected to know not only how to collect data, they’re also required to know how to derive strategic insights from it (with the ultimate goal of influencing product directions). This consolidation of what should be separate roles creates a tall order that many junior to mid-level researchers aren’t prepared to fill as they enter the field, and this isn’t exactly advantageous to the future of the discipline.
To combat this, research teams should advocate for themselves by insisting they have a research/design lead who can help define and communicate the overall strategy. Since strategy is very time-consuming work, it's almost impossible for those same persons to also be responsible for conducting the tactical research. Yes, we should spend more time educating researchers on strategic thinking and helping them understand their business goals or needs. Still, we can't expect a team of 2-3 researchers (not even at the senior or principal level) to be strategic in isolation, and they would need time to be able to do that well.
Mentorship programs should be established where experienced UX professionals can guide novices through the complexities of the role. These mentors can provide practical advice, share experiences, and offer feedback on projects, helping junior researchers develop a more nuanced understanding of the field.
Challenges (and changes) in research methodologies
Humans are around 60% water. Water flows downhill. Put another way — humans like the path of least resistance (and the best bang for our buck!). What’s my point? All the fervor around the purported benefits of AI and automation has bled into conversations between researchers and stakeholders, making long-existent conflicts of interest loom that much larger. Stakeholders pressure researchers to deliver insights quickly—leading to surface-level insights, broad assumptions towards large categories of people, and rushed analyses—feeling confident that the new technologies will allow for greater efficiency without losing quality.
Gaming UX is a great example of these diverging interests between stakeholders and researchers. Here, questions often revolve around general usability rather than specific elements like immersion and continuity of the user journey, which are crucial for a satisfactory gaming experience.
Going back to the John Wooden quote, the rub here is that there isn’t any substitute for the basics. Yet, UX professionals are being pressed to abandon the basics in some cases, with many finding that time and budget constraints significantly impact the quality of their outputs.
The solution to this is threefold: education, advocacy, and structured methodology.
First, educating stakeholders about the intricate processes involved in UX research and the potential pitfalls of rushing these processes is crucial. This includes demonstrating how deep, methodical insights often lead to more effective, sustainable, and strategic product improvements than superficial analyses.
Second, UX professionals must advocate for the importance of foundational research practices that prioritize thoroughness over speed. This might involve pushing back on timelines or demanding adequate resources to ensure quality isn't sacrificed for expedience. Easier said than done, but necessary. Good research should always begin and end on the basis of mutual respect.
Finally, implementing a structured methodology that integrates new technologies and tools without compromising on the core principles of UX research can help maintain quality. This could include phased research stages, iterative testing, and milestone reviews to ensure that every aspect of user experience, particularly in complex fields like gaming, is comprehensively understood and optimized.
The more things seem to change, the more they stay the same
The future of UX is exciting to consider. Changes always bring with them some growing pains, and the introduction of new technologies into research methods is no exception. The best way to integrate these new tools in a sustainable way is to stay true to the core principles and best practices in UX research.
While AI can augment the efficiency and scope of research methodologies, it can’t replace the critical human insight essential for the deep understanding of user behaviors and needs. Effective UX research still relies on a meticulous approach to data collection, a rigorous analysis process, and, most importantly, a nuanced interpretation of that data that AI currently cannot achieve independently. Keep your basics solid so that when you do take shortcuts, you don’t wind up lost.