Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
Enthusiasm for AI tools, especially large language models like ChatGPT, is everywhere, but what does it actually look like to deliver large-scale user-facing experiences using these tools in a production environment? Clearly they're powerful, but what do they need to make them work reliably and at scale? In this session, Sarah provides a perspective on some of the information architecture and user experience infrastructure organizations need to effectively leverage AI. She also shares three AI experiences currently live on Microsoft Learn: An interactive assistant that helps users post high-quality questions to a community forum A tool that dynamically creates learning plans based on goals the user shares A training assistant that clarifies, defines, and guides learners while they study Through lessons learned from shipping these experiences over the last two years, UXers, IAs, and PMs will come away with a better sense of what they might need to make these hyped-up technologies work in real life.
Key Insights
-
•
AI applications must not default to a generic everything chat bot; context and task specificity are crucial.
-
•
Building on a strong infrastructure reduces complexity in AI application development.
-
•
Evaluation should be a major component of AI development efforts; success cannot be gauged on anecdotal evidence.
-
•
Simplicity in tasks leads to higher success rates in AI applications; complex tasks are more challenging to manage.
-
•
User journeys should guide the development of AI features rather than an assumption they can handle everything.
-
•
Understanding task context can lower the ambiguity footprint of AI applications, thus improving usability.
-
•
Improving AI applications often requires organizations to first strengthen their foundational capabilities.
-
•
Visibility of AI interfaces matters; users must understand when and how AI is impacting their experience.
-
•
Dynamic context adds complexity, but can enhance the efficacy of AI tools when managed well.
-
•
A strong foundation in evaluation practices helps teams make informed decisions about AI model effectiveness.
Notable Quotes
"AI isn't the answer to all problems; it's about finding the right context for its use."
"We're not building the perfect model; we're improving the user experience within defined parameters."
"A chatbot is not a silver bullet; understanding user journeys is fundamentally important."
"The ambiguity footprint helps map the complexity of AI applications and the associated risks."
"It's critical to evaluate AI outputs; otherwise, we operate based on gut feelings."
"Ambiguity in AI development is inevitable, but we can manage it to improve our applications."
"When launching AI solutions, build complexity gradually to avoid overwhelming challenges."
"The stakes get higher with sensitive information, making context crucial in development."
"User engagement is more predictable when AI tools complement rather than interfere with tasks."
"Rigorous evaluation processes are as important in AI as they are in traditional product management."
More Videos
"Designing for people and process is just as important as focusing on the technology itself."
Alexandra SchmidtEnterprise UX Playbook
December 1, 2022
"Our design system is meant to enable designers and developers to focus on solving core problems."
Dave Malouf Amy ThibodeauPanel: Design Systems and Documentation
November 7, 2017
"UX research is inherently future oriented; no one is making products for the past."
Nicole AleongFuture Orientations to Everyday Life: Futures Anthropology as a Methodology
March 26, 2024
"We need to protect against dehumanization at all costs."
John TaschekMaking People the X-Factor in the Enterprise
June 15, 2018
"It's a marathon, not a sprint; pace yourself accordingly."
Sarah Auslander Betsy Ramaccia Gordon RossInsights Panel
November 18, 2022
"Introduce a wild card; someone thinking on the fringes can spark up the thinking."
David Cronin Uday Gajendar Peter Morville Kendra ShimmellDiscussion
May 13, 2015
"One of the hallmarks of a healthy team is how well members feel cared for."
Kristin SkinnerTheme 2: Introduction and Provocation
January 8, 2024
"We can't drive the kind of learning we're most interested in like a project plan."
Jen BriselliLearning Is The Engine: Designing & Adapting in a World We Can’t Predict
April 16, 2025
"Major companies know that solving accessibility issues will lead to market leadership."
Sam ProulxTo Boldly Go: The New Frontiers of Accessibility
September 9, 2022
