Ep097: Specialized Agents & Agentic Orchestration - New Relic and the Future of Observability

New Relic's Head of AI and ML Innovation, Camden Swita discusses their four-cornered AI strategy and envisions a future of "agentic orchestration" with specialized agents.

Topics Include:

• Introduction of Camden Swita, Head of AI at New Relic.
• New Relic invented the observability space for monitoring applications.
• Started with Java workloads monitoring and APM.
• Evolved into full-stack observability with infrastructure and browser monitoring.
• Uses advanced query language (NRQL) with time series database.
• AI strategy focuses on AI ops for automation.
• First cornerstone: Intelligent detection capabilities with machine learning.
• Second cornerstone: Incident response with generative AI assistance.
• Third cornerstone: Problem management with root cause analysis.
• Fourth cornerstone: Knowledge management to improve future detection.
• Initially overwhelmed by "ocean of possibilities" with LLMs.
• Needed narrow scope and guardrails for measurable progress.
• Natural language to NRQL translation proved immensely complex.
• Selecting from thousands of possible events caused accuracy issues.
• Shifted from "one tool" approach to many specialized tools.
• Created routing layer to select right tool for each job.
• Evaluation of NRQL is challenging even when syntactically correct.
• Implemented multi-stage validation with user confirmation step.
• AWS partnership involves fine-tuning models for NRQL translation.
• Using Bedrock to select appropriate models for different tasks.
• Initially advised prototyping on biggest, best available models.
• Now recommends considering specialized, targeted models from start.
• Agent development platforms have improved significantly since beginning.
• Future focus: "Agentic orchestration" with specialized agents.
• Envisions agents communicating through APIs without human prompts.
• Integration with AWS tools like Amazon Q.
• Industry possibly plateauing in large language model improvements.
• Increasing focus on inference-time compute in newer models.
• Context and quality prompts remain crucial despite model advances.
• Potential pros and cons to inference-time compute approach.

Participants:

• Camden Swita – Head of AI & ML Innovation, Product Management, New Relic

See how Amazon Web Services gives you the freedom to migrate, innovate, and scale your software company at https://aws.amazon/isv/