In this episode, Product Manager Sibo Ding tells us how natural language understanding (NLU) capability improves performance and user experience for Virtual Agent, AI Search, and other ServiceNow user interface features. This episode covers:
What is NLU?
How does ServiceNow use NLU?
How does ServiceNow NLU work?
How does a company add NLU to its ServiceNow products?
What’s in store for ServiceNow NLU in the future?

For more information, see:
Product documentation: Natural Language Understanding
ServiceNow Community: Virtual Agent & NLU

Your feedback helps us serve you better – please leave us a comment.

Subscribe today!
Apple Podcasts
Google Podcasts
Spotify
Stitcher
TuneIn
RSS

In this episode, Product Manager <a href="https://www.servicenow.com/community/user/viewprofilepage/user-id/301755">Sibo Ding</a> tells us how natural language understanding (NLU) capability improves performance and user experience for Virtual Agent, AI Search, and other ServiceNow user interface features. This episode covers:
<ul>
<li>What is NLU?</li>
<li>How does ServiceNow use NLU?</li>
<li>How does ServiceNow NLU work?</li>
<li>How does a company add NLU to its ServiceNow products?</li>
<li>What’s in store for ServiceNow NLU in the future?</li>
</ul>
For more information, see:
<ul>
<li>Product documentation: <a href="https://docs.servicenow.com/bundle/utah-intelligent-experiences/page/administer/natural-language-understanding/concept/natural-language-understanding.html">Natural Language Understanding</a></li>
<li>ServiceNow Community: <a href="https://www.servicenow.com/community/virtual-agent-nlu/ct-p/virtual-agent-natural-language">Virtual Agent &amp; NLU</a></li>
</ul>
Your feedback helps us serve you better – please leave us a comment.
Subscribe today!
<ul>
<li><a href="https://podcasts.apple.com/us/podcast/servicenow-techbytes/id1038560176">Apple Podcasts</a></li>
<li><a href="https://www.google.com/podcasts?feed=aHR0cHM6Ly93d3cub21ueWNvbnRlbnQuY29tL2QvcGxheWxpc3QvOTIyZTBmMTktMmQ4NC00NDg1LTgzNWMtYTgzZjAwMDM2YjAwLzczOWVjNGJlLWUzYjAtNDAyYy05MzliLWE4NDcwMTU1ZjEwOC8xODc0YWU0YS1hNWQwLTRlNjItOGE5OC1hODQ3MDE1NWYxMTEvcG9kY2FzdC5yc3M%3d">Google Podcasts</a></li>
<li><a href="https://open.spotify.com/show/7upVTXh4RNk1iClfSepDp0">Spotify</a></li>
<li><a href="https://www.stitcher.com/podcast/servicenow-community-podcasts/servicenow-techbytes?refid=stpr">Stitcher</a></li>
<li><a href="https://tunein.com/podcasts/Technology-Podcasts/ServiceNow-TechBytes-p1134179/">TuneIn</a></li>
<li><a href="https://omny.fm/shows/servicenow-techbytes/playlists/podcast.rss">RSS</a></li>
</ul>

ServiceNow Insights is your insider’s guide to the world of ServiceNow. Join us as we unpack the latest products, innovations, and updates with in-depth discussions led by the creators behind the platform. Hear firsthand about the challenges they've overcome, the solutions they've pioneered, and the exciting future they envision. Whether you're a seasoned admin, agent, platform owner, or just discovering the power of ServiceNow,  and offers valuable insights to help you stay informed and leverage the full potential of the platform.

ServiceNow Insights&nbsp;is your insider&rsquo;s guide to the world of ServiceNow. Join us as we unpack the latest products, innovations, and updates with in-depth discussions led by the creators behind the platform. Hear firsthand about the challenges they've overcome, the solutions they've pioneered, and the exciting future they envision. Whether you're a seasoned admin, agent, platform owner, or just discovering the power of ServiceNow,&nbsp;&nbsp;and offers valuable insights to help you stay informed and leverage the full potential of the platform.

ServiceNow TechBytes is a Podcast Program hosted …

ServiceNow Insights 

ServiceNow Podcast Program is a collection of podcast shows hosted by various ServiceNow experts and professionals covering different focus areas and a broad variety of topics. Come listen to our experts talk about what's going on at ServiceNow. Join the conversation!

Visit the Now Community forums

ServiceNow Podcast Program is a collection of podcast shows hosted by various ServiceNow experts and professionals covering different focus areas and a broad variety of topics. Come listen to our experts talk about what's going on at ServiceNow. Join the conversation!
<a href="https://community.servicenow.com/community">Visit the Now Community forums</a>

ServiceNow Podcasts

ServiceNow Insights Podcast

Episode 66: Natural language understanding

Voice AI agent evaluation — why it's fundamentally harder than text, how cascade failures derail conversations invisibly, and ServiceNow's open-source framework to establish industry evaluation standards. Featuring real audio examples showing authentication failures, leaked reasoning, and latency problems.

WHAT WE COVER 

TARA BOGAVELLI — Research Engineer, ServiceNow 
Leading the open-source voice agent evaluation framework. Explains why existing benchmarks don't measure what matters and what ServiceNow is releasing to establish industry standards.

KATRINA STANKIEWICZ — Staff Machine Learning Engineer, ServiceNow 
Cascade model architecture expert. Breaks down STT → LLM → TTS failure modes, named entity transcription challenges, and real audio example analysis.

GABRIELLE GAUTHIER MELANÇON — Staff Applied Research Scientist, ServiceNow 
Multi-language evaluation specialist. Reveals why Large Audio Language Models lag behind, the native speaker requirement, and bot-to-bot simulation methodology. 

CHAPTERS
0:00 Introduction — The evaluation gap 
1:11 ServiceNow's Open-Source Framework Announcement — Tara Bogavelli 
2:43 Meet the Researchers 
3:43 Voice-Specific Challenges — Tara Bogavelli 
5:03 Cascade Architecture: STT → LLM → TTS — Katrina Stankiewicz 
7:57 The Named Entity Problem — Katrina Stankiewicz 
10:06 Evaluation Metrics: Accuracy vs Experience — Gabrielle Gauthier Melançon 
11:23 Bot-to-Bot Testing at Scale — Gabrielle Gauthier Melançon 
14:30 The LALM Gap: Why Audio AI Judges Struggle — Tara Bogavelli
16:57 Real Audio Example: Flight Rebooking Gone Wrong 
21:58 Breaking Down the Failures — Katrina Stankiewicz 28:30 Wrap-Up & Resources

KEY INSIGHTS

The Cascade Failure Problem: STT → LLM → TTS errors propagate invisibly Named Entity Transcription: The #1 enterprise blocker—names, confirmation codes, emails break authentication Accuracy vs Experience: Perfect task completion means nothing if users hang up due to poor experience LALM Gap: Large Audio Language Models lag behind text LLMs—human evaluators remain essential Latency Kills Conversations: Five-second pauses make users think the call dropped, breaking the experience even when tasks complete Open-Source Framework: ServiceNow releasing evaluation tools, metrics, and bot-to-bot simulation methodology for the industry.

LEARN MORE

Website: https://servicenow.github.io/eva/ GitHub: 
https://github.com/servicenow/eva Blog Post: 
https://huggingface.co/blog/ServiceNow-AI/eva Dataset: https://huggingface.co/datasets/ServiceNow-AI/eva

ABOUT

Hosted by Bobby Brill. ServiceNow Insights podcast explores AI research, real-world applications, and the people building the future of work. #VoiceAI #AIEvaluation #ServiceNow #MachineLearning #OpenSource #ConversationalAI #STT #TTS #LLM #VoiceAgents #AIResearch #Podcast

Voice AI agent evaluation &mdash; why it's fundamentally harder than text, how cascade failures derail conversations invisibly, and ServiceNow's open-source framework to establish industry evaluation standards. Featuring real audio examples showing authentication failures, leaked reasoning, and latency problems.
WHAT WE COVER&nbsp;
TARA BOGAVELLI &mdash; Research Engineer, ServiceNow Leading the open-source voice agent evaluation framework. Explains why existing benchmarks don't measure what matters and what ServiceNow is releasing to establish industry standards.
KATRINA STANKIEWICZ &mdash; Staff Machine Learning Engineer, ServiceNow Cascade model architecture expert. Breaks down STT &rarr; LLM &rarr; TTS failure modes, named entity transcription challenges, and real audio example analysis.
GABRIELLE GAUTHIER MELAN&Ccedil;ON &mdash; Staff Applied Research Scientist, ServiceNow Multi-language evaluation specialist. Reveals why Large Audio Language Models lag behind, the native speaker requirement, and bot-to-bot simulation methodology.&nbsp;
CHAPTERS 0:00 Introduction &mdash; The evaluation gap 1:11 ServiceNow's Open-Source Framework Announcement &mdash; Tara Bogavelli 2:43 Meet the Researchers 3:43 Voice-Specific Challenges &mdash; Tara Bogavelli 5:03 Cascade Architecture: STT &rarr; LLM &rarr; TTS &mdash; Katrina Stankiewicz 7:57 The Named Entity Problem &mdash; Katrina Stankiewicz 10:06 Evaluation Metrics: Accuracy vs Experience &mdash; Gabrielle Gauthier Melan&ccedil;on 11:23 Bot-to-Bot Testing at Scale &mdash; Gabrielle Gauthier Melan&ccedil;on&nbsp; 14:30 The LALM Gap: Why Audio AI Judges Struggle &mdash; Tara Bogavelli 16:57 Real Audio Example: Flight Rebooking Gone Wrong 21:58 Breaking Down the Failures &mdash; Katrina Stankiewicz 28:30 Wrap-Up &amp; Resources
KEY INSIGHTS
The Cascade Failure Problem: STT &rarr; LLM &rarr; TTS errors propagate invisibly Named Entity Transcription: The #1 enterprise blocker&mdash;names, confirmation codes, emails break authentication Accuracy vs Experience: Perfect task completion means nothing if users hang up due to poor experience LALM Gap: Large Audio Language Models lag behind text LLMs&mdash;human evaluators remain essential Latency Kills Conversations: Five-second pauses make users think the call dropped, breaking the experience even when tasks complete Open-Source Framework: ServiceNow releasing evaluation tools, metrics, and bot-to-bot simulation methodology for the industry.
LEARN MORE
Website: https://servicenow.github.io/eva/ GitHub: https://github.com/servicenow/eva Blog Post: https://huggingface.co/blog/ServiceNow-AI/eva Dataset: https://huggingface.co/datasets/ServiceNow-AI/eva
ABOUT
Hosted by Bobby Brill. ServiceNow Insights podcast explores AI research, real-world applications, and the people building the future of work. #VoiceAI #AIEvaluation #ServiceNow #MachineLearning #OpenSource #ConversationalAI #STT #TTS #LLM #VoiceAgents #AIResearch #Podcast

EVA - A Framework for Evaluating Voice Agents by ServiceNow

AI governance at scale — what it means, how to do it, and what regulations you need to know now. Host Bobby Brill brings together five ServiceNow experts across two conversations for a complete 20-minute briefing on governing AI in the enterprise.

━━━━━━━━━━━━━━━━━━━━━━━━
WHAT WE COVER
━━━━━━━━━━━━━━━━━━━━━━━━

RAVI KRISHNAMURTHY — VP, AI Platform, ServiceNow
Why hidden AI is one of the biggest unmanaged risks in the enterprise — and why governance is an accelerator, not a brake.

PETER WEIGT — Responsible AI, ServiceNow
The innovation paradox: how AI Control Tower makes governance a team sport and breaks down the silos that slow AI deployment down.

SAMPADA CHAVAN — AI Control Tower, ServiceNow
How AI Control Tower was built, what the discovery problem really looks like, and why compliance must be baked into the AI lifecycle — not bolted on at the end.

ANDREA LAFOUNTAIN — AI Legal, ServiceNow
The three regulatory frameworks every enterprise needs to know: EU AI Act, Colorado AI Act, and NIST. Plus: the compliance strategy that scales across all of them.

NAVDEEP GILL — Responsible AI, ServiceNow
The math on enterprise AI compliance — why it's exponential — and how AI Control Tower's automated discovery keeps you ahead of it.

━━━━━━━━━━━━━━━━━━━━━━━━
CHAPTERS
━━━━━━━━━━━━━━━━━━━━━━━━

0:00 Introduction
1:23 The Hidden AI Problem — Ravi Krishnamurthy & Sampada Chavan
5:33 AI Control Tower in Practice — Peter Weigt & Sampada Chavan
7:37 The Regulatory Landscape — Andrea LaFountain & Navdeep Gill
14:38 Compliance in Action & Key Deadlines
17:05 Wrap-Up

━━━━━━━━━━━━━━━━━━━━━━━━
KEY DATES TO KNOW
━━━━━━━━━━━━━━━━━━━━━━━━

EU AI Act enforcement: August 2026
Colorado AI Act enforcement: June 2026
NIST AI RMF: Voluntary framework, increasingly referenced by regulators

━━━━━━━━━━━━━━━━━━━━━━━━
LEARN MORE
━━━━━━━━━━━━━━━━━━━━━━━━

ServiceNow AI Control Tower: https://www.servicenow.com
NIST AI Risk Management Framework: https://www.nist.gov/artificial-intelligence

━━━━━━━━━━━━━━━━━━━━━━━━
ABOUT THIS PODCAST
━━━━━━━━━━━━━━━━━━━━━━━━

Hosted by Bobby Brill. A ServiceNow podcast exploring the people, technology, and ideas shaping the future of work.

#AIGovernance #ServiceNow #AIControlTower #ResponsibleAI #EUAIAct #EnterpriseAI #AICompliance #FutureOfWork #NowAssist #Podcast

AI governance at scale &mdash; what it means, how to do it, and what regulations you need to know now. Host Bobby Brill brings together five ServiceNow experts across two conversations for a complete 20-minute briefing on governing AI in the enterprise. ━━━━━━━━━━━━━━━━━━━━━━━━ WHAT WE COVER ━━━━━━━━━━━━━━━━━━━━━━━━ RAVI KRISHNAMURTHY &mdash; VP, AI Platform, ServiceNow Why hidden AI is one of the biggest unmanaged risks in the enterprise &mdash; and why governance is an accelerator, not a brake. PETER WEIGT &mdash; Responsible AI, ServiceNow The innovation paradox: how AI Control Tower makes governance a team sport and breaks down the silos that slow AI deployment down. SAMPADA CHAVAN &mdash; AI Control Tower, ServiceNow How AI Control Tower was built, what the discovery problem really looks like, and why compliance must be baked into the AI lifecycle &mdash; not bolted on at the end. ANDREA LAFOUNTAIN &mdash; AI Legal, ServiceNow The three regulatory frameworks every enterprise needs to know: EU AI Act, Colorado AI Act, and NIST. Plus: the compliance strategy that scales across all of them. NAVDEEP GILL &mdash; Responsible AI, ServiceNow The math on enterprise AI compliance &mdash; why it's exponential &mdash; and how AI Control Tower's automated discovery keeps you ahead of it. ━━━━━━━━━━━━━━━━━━━━━━━━ CHAPTERS ━━━━━━━━━━━━━━━━━━━━━━━━ 0:00 Introduction 1:23 The Hidden AI Problem &mdash; Ravi Krishnamurthy &amp; Sampada Chavan 5:33 AI Control Tower in Practice &mdash; Peter Weigt &amp; Sampada Chavan 7:37 The Regulatory Landscape &mdash; Andrea LaFountain &amp; Navdeep Gill 14:38 Compliance in Action &amp; Key Deadlines 17:05 Wrap-Up ━━━━━━━━━━━━━━━━━━━━━━━━ KEY DATES TO KNOW ━━━━━━━━━━━━━━━━━━━━━━━━ EU AI Act enforcement: August 2026 Colorado AI Act enforcement: June 2026 NIST AI RMF: Voluntary framework, increasingly referenced by regulators ━━━━━━━━━━━━━━━━━━━━━━━━ LEARN MORE ━━━━━━━━━━━━━━━━━━━━━━━━ ServiceNow AI Control Tower: https://www.servicenow.com NIST AI Risk Management Framework: https://www.nist.gov/artificial-intelligence ━━━━━━━━━━━━━━━━━━━━━━━━ ABOUT THIS PODCAST ━━━━━━━━━━━━━━━━━━━━━━━━ Hosted by Bobby Brill. A ServiceNow podcast exploring the people, technology, and ideas shaping the future of work. #AIGovernance #ServiceNow #AIControlTower #ResponsibleAI #EUAIAct #EnterpriseAI #AICompliance #FutureOfWork #NowAssist #Podcast

AI Control Tower - Governing AI at Scale with ServiceNow

What is Agentic AI — and what can it actually do for your business? In this episode of the podcast, host Bobby Brill brings together three conversations with the people building Agentic AI at ServiceNow into one 20-minute briefing.

Whether you're leading a team, managing a platform, or just trying to understand what your company is investing in - this episode gives you the full picture.

━━━━━━━━━━━━━━━━━━━━━━━━

WHAT WE COVER

━━━━━━━━━━━━━━━━━━━━━━━━

RITA CASTILLO — VP of Design, Platform & AI

The clearest explainer of Agentic AI you'll find. Rita walks through the full evolution from scripted automation to agentic systems, what makes agentic different, and how the orchestrator-and-agent hierarchy works at ServiceNow.

AMRUTHA RAMESH — Visiting Researcher, ServiceNow

Amrutha built Agent Ada — a real AI agent designed to turn enterprise data into actionable insights. She explains how Ada understands your goal, adapts to who's asking, and delivers analysis your team can actually use.

ISSAM LARADJI & ANITHA RAGHAVAN — Research Scientist & Product Manager, ServiceNow

How do you know if your AI's insights are trustworthy? Issam and Anitha built InsightBench — a benchmark that uses AI to evaluate AI — to answer exactly that question.

━━━━━━━━━━━━━━━━━━━━━━━━

CHAPTERS

0:00  Introduction
1:30  What Is Agentic AI? — Rita Castillo, VP of AI Design
6:00  How Agents Work as a Team
10:30  Meet Agent Ada — Amrutha Ramesh, Visiting Researcher
16:00  Can You Trust Your AI? — Issam Laradji & Anitha Raghavan
21:30  Wrap-Up

━━━━━━━━━━━━━━━━━━━━━━━━

GUESTS

━━━━━━━━━━━━━━━━━━━━━━━━

Rita Castillo — VP, AI Design at ServiceNow

Amrutha Ramesh — Visiting Researcher, ServiceNow

Issam Laradji — Research Scientist, ServiceNow

Anitha Raghavan — Product Manager, ServiceNow

━━━━━━━━━━━━━━━━━━━━━━━━

LEARN MORE

━━━━━━━━━━━━━━━━━━━━━━━━

ServiceNow Agentic AI: https://www.servicenow.com

More episodes:

Narrative Analytics with Agent Ada: https://youtu.be/k1P9o2glq90

InsightBench & AgentPoirot: https://youtu.be/0_ikQQ82qAk

Understanding Agentic AI with Rita Castillo: https://youtu.be/JyFdcQChYLs

━━━━━━━━━━━━━━━━━━━━━━━━

ABOUT THIS PODCAST

━━━━━━━━━━━━━━━━━━━━━━━━

Hosted by Bobby Brill. ServiceNow INsights podcast explores the people, technology, and ideas shaping the future of work.

#AgenticAI #ServiceNow #AIAgents #ArtificialIntelligence #FutureOfWork #EnterpriseAI #DataAnalytics #NowAssist #MachineLearning #Podcast

What&nbsp;is&nbsp;Agentic&nbsp;AI&nbsp;&mdash;&nbsp;and&nbsp;what&nbsp;can&nbsp;it&nbsp;actually&nbsp;do&nbsp;for&nbsp;your&nbsp;business?&nbsp;In&nbsp;this&nbsp;episode&nbsp;of&nbsp;the&nbsp;podcast,&nbsp;host&nbsp;Bobby&nbsp;Brill&nbsp;brings&nbsp;together&nbsp;three&nbsp;conversations&nbsp;with&nbsp;the&nbsp;people&nbsp;building&nbsp;Agentic&nbsp;AI&nbsp;at&nbsp;ServiceNow&nbsp;into&nbsp;one&nbsp;20-minute&nbsp;briefing.
Whether you're leading a team, managing a platform, or just trying to understand what your company is investing in - this episode gives you the full picture.
━━━━━━━━━━━━━━━━━━━━━━━━
WHAT&nbsp;WE&nbsp;COVER
━━━━━━━━━━━━━━━━━━━━━━━━
RITA&nbsp;CASTILLO&nbsp;&mdash;&nbsp;VP&nbsp;of&nbsp;Design,&nbsp;Platform&nbsp;&amp;&nbsp;AI
The clearest explainer of Agentic AI you'll find. Rita walks through the full evolution from scripted automation to agentic systems, what makes agentic different, and how the orchestrator-and-agent hierarchy works at ServiceNow.
AMRUTHA&nbsp;RAMESH&nbsp;&mdash;&nbsp;Visiting&nbsp;Researcher,&nbsp;ServiceNow
Amrutha built Agent Ada &mdash; a real AI agent designed to turn enterprise data into actionable insights. She explains how Ada understands your goal, adapts to who's asking, and delivers analysis your team can actually use.
ISSAM&nbsp;LARADJI&nbsp;&amp;&nbsp;ANITHA&nbsp;RAGHAVAN&nbsp;&mdash;&nbsp;Research&nbsp;Scientist&nbsp;&amp;&nbsp;Product&nbsp;Manager,&nbsp;ServiceNow
How do you know if your AI's insights are trustworthy? Issam and Anitha built InsightBench &mdash; a benchmark that uses AI to evaluate AI &mdash; to answer exactly that question.
━━━━━━━━━━━━━━━━━━━━━━━━
CHAPTERS
0:00&nbsp;&nbsp;Introduction 1:30&nbsp;&nbsp;What Is Agentic AI? &mdash; Rita Castillo, VP of AI Design 6:00&nbsp;&nbsp;How Agents Work as a Team 10:30&nbsp;&nbsp;Meet Agent Ada &mdash; Amrutha Ramesh, Visiting Researcher 16:00&nbsp;&nbsp;Can You Trust Your AI? &mdash; Issam Laradji &amp; Anitha Raghavan 21:30&nbsp;&nbsp;Wrap-Up
━━━━━━━━━━━━━━━━━━━━━━━━
GUESTS
━━━━━━━━━━━━━━━━━━━━━━━━
Rita Castillo &mdash; VP, AI Design at ServiceNow
Amrutha&nbsp;Ramesh&nbsp;&mdash;&nbsp;Visiting&nbsp;Researcher,&nbsp;ServiceNow
Issam&nbsp;Laradji&nbsp;&mdash;&nbsp;Research&nbsp;Scientist,&nbsp;ServiceNow
Anitha&nbsp;Raghavan&nbsp;&mdash;&nbsp;Product&nbsp;Manager,&nbsp;ServiceNow
━━━━━━━━━━━━━━━━━━━━━━━━
LEARN&nbsp;MORE
━━━━━━━━━━━━━━━━━━━━━━━━
ServiceNow Agentic AI: https://www.servicenow.com
More&nbsp;episodes:
Narrative&nbsp;Analytics&nbsp;with&nbsp;Agent&nbsp;Ada:&nbsp;https://youtu.be/k1P9o2glq90
InsightBench&nbsp;&amp;&nbsp;AgentPoirot:&nbsp;https://youtu.be/0_ikQQ82qAk
Understanding&nbsp;Agentic&nbsp;AI&nbsp;with&nbsp;Rita&nbsp;Castillo:&nbsp;https://youtu.be/JyFdcQChYLs
━━━━━━━━━━━━━━━━━━━━━━━━
ABOUT&nbsp;THIS&nbsp;PODCAST
━━━━━━━━━━━━━━━━━━━━━━━━
Hosted by Bobby Brill. ServiceNow INsights podcast explores the people, technology, and ideas shaping the future of work.
#AgenticAI #ServiceNow #AIAgents #ArtificialIntelligence #FutureOfWork #EnterpriseAI #DataAnalytics #NowAssist #MachineLearning #Podcast

Episode 66: Natural language understanding

In 2 playlist(s)

ServiceNow Insights Podcast

Social links

Follow podcast

Recent clips