Deployed: The AI Product Podcast

• The Secrets To Being A Great AI PM: Evals, Quality & More | Apollo.io AI Product Lead, Tyler Phillips

  On this episode we talk with Tyler Phillips, AI Product Lead at Apollo.io, about what it really takes to build successful AI products in sales. Tyler shares surprisingly candid insights about the "unsexy" but critical work that great AI PMs do - from spending hours evaluating AI outputs to building systematic quality frameworks. Learn why it can help for domain experts should write prompts instead of engineers, how Apollo measures AI quality across their products, and the practical strategies they've developed after some early missteps. Whether you're an aspiring AI PM or an engineering leader trying to structure your team effectively, this conversation offers useful lessons on what actually drives AI product success. 

  --------  

  42:52
• Building Box's AI Platform: Enterprise Lessons in Scaling LLMs | Ben Kus, CTO of Box

  On this episode of Deployed: The AI Product Podcast, we sit down with Ben Kus, CTO of Box, to unpack how they built a secure, scalable AI platform within their enterprise content management system. Ben shares candid insights from Box's journey integrating AI capabilities while maintaining enterprise-grade security for sensitive customer data. Learn practical strategies for evaluating AI quality without accessing customer data, building internal platforms that engineering teams want to use, and designing architecture that can evolve with rapid AI advances. Whether you're integrating LLMs into an established product or building new AI features, this conversation offers valuable lessons on balancing innovation with enterprise requirements. Join Ian, Co-Founder and CEO of Freeplay, for this in-depth discussion on scaling AI responsibly in the enterprise.

  --------  

  44:15
• Building Enterprise-Grade AI Agents: Lessons from Sierra's Arya Asemanfar

  In this episode, we sit down with Arya Asemanfar, Product and Engineering leader at Sierra, to explore the future of AI-powered enterprise agents. Arya breaks down how Sierra is transforming customer experiences by building agents that go beyond traditional chatbots—taking meaningful actions, embodying brand voice, and scaling reliably for unique business needs.Learn how Sierra approaches agent development with their innovative “Agent OS,” solves challenges like tool hallucination, and co-designs solutions with customers. Arya also shares actionable insights for product and engineering leaders, from building better evaluation systems to designing intuitive feedback loops.Topics covered in this episode:What Are Enterprise Agents?The difference between traditional chatbots and AI-powered agents that take meaningful actions.Examples of how Sierra’s agents transform customer service interactions (e.g., returns, exchanges).Building Scalable AI AgentsThe importance of robust architecture for reliability and performance.How Sierra’s “Agent OS” supports the full lifecycle of agent development—from design to monitoring.Addressing tool hallucination and ensuring agents only take actionable, reliable steps.Using evaluation frameworks to test and refine agent behavior at scale.Collaborating with CustomersWhy each agent is treated as a unique product tailored to a customer’s brand, policies, and processes.The co-design process and how it ensures agents align with business needs.Actionable Advice for Teams Building AI AgentsStart with concrete customer examples to develop intuition and build better agents.Implement evaluation systems early to accelerate learning and iteration.Invest in tools and processes to streamline human feedback and testing.The Future of AI and Product DesignEmerging paradigms that will transform how users interact with software.Predictions for how AI will shape new user experiences beyond automation.

  --------  

  43:15
• How Help Scout Moved From Traditional SaaS To An AI-Native Product Company: A Conversation with Nick Francis & Luis Morales of Help Scout

  In this episode of Deployed: The AI Product Podcast, we talk with Nick Francis, CEO and Co-founder, and Luis Morales, VP of Engineering at Help Scout, about their journey integrating AI into their established (13 years in market!) customer support platform.Nick and Luis share candid insights on:• How they shifted from skepticism about AI to embracing it as a tool for delighting customers• Their process for experimenting with AI features, from summarization to automated email drafts• Challenges in prompt engineering and building effective RAG systems• Creating a new role for an AI product quality expert with deep customer knowledge• Transitioning to an experimentation-driven engineering culture• Making the bold decision to completely overhaul their business model and pricingThe conversation covers both high-level strategy and tactical details of Help Scout's AI development process. Nick and Luis offer valuable perspectives for any company looking to thoughtfully integrate AI capabilities into existing products.For product and engineering leaders navigating the complexities of AI adoption, this episode provides practical advice on balancing innovation with customer needs. Tune in to learn how Help Scout is leveraging AI to enhance the customer support experience while staying true to their values.

  --------  

  52:16
• Insights from Building AI Systems At Google Scale: In Conversation With Kyle Nesbit

  Kyle Nesbit, a longtime Googler and AI expert, joins us on Deployed to share lessons from his 17+ years at the forefront of distributed systems, machine learning, and AI-driven product innovation. Kyle has helped build foundational technologies like BigQuery and worked on early large language model (LLM) development at Google, giving him a unique perspective on how teams can successfully transition from traditional engineering to modern AI-focused workflows.In this episode, we explore:The challenges and opportunities of transitioning traditional engineering teams to LLM developmentWhy starting with evaluation metrics is the foundation for successPractical strategies for iterative improvement and guardrail designHow to balance product priorities and quality trade-offs when scaling AI systemsThe real story behind AI demos and how to communicate progress effectivelyFoundational issues in data discovery and access—and why solving them matters more than chasing trends

  --------  

  52:45

Flere Teknologi podcasts

Trendige Teknologi podcasts

Om Deployed: The AI Product Podcast