Kubernetes in the Era of GPUs

Haseeb Budhani (@haseebbudhani, CEO @rafaysystemsinc) discusses the evolution from traditional DevOps to platform engineering and what "Enterprise Ready" Kubernetes looks like in 2025. We explore AI workloads running on Kubernetes and how modern orchestration solutions can transform teams from bottlenecks into enablers. We also cover the security considerations for GPU-enabled AI workloads and balancing developer self-service capabilities with proper governance and control.

SHOW: 950

SHOW TRANSCRIPT: The Cloudcast #950 Transcript

SHOW VIDEO: https://youtube.com/@TheCloudcastNET 

NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST: "CLOUDCAST BASICS"

SPONSORS:

• [DoIT] Visit doit.com (that’s d-o-i-t.com) to unlock intent-aware FinOps at scale with DoiT Cloud Intelligence.
• [VASION] Vasion Print eliminates the need for print servers by enabling secure, cloud-based printing from any device, anywhere. Get a custom demo to see the difference for yourself.

SHOW NOTES:

• Rafay website

Topic 1 - Welcome to the show, Haseeb. Give everyone a quick introduction.

Topic 2 - Let’s start by talking about the evolution of Kubernetes as a platform. You’ve said and we’ve talked about on this show for some time how Kubernetes is more of a platform to run platforms. We’ve also seen trends in the industry and shifts in what it means to be DevOps or Platform Engineering in recent years. You've positioned Rafay as a Kubernetes Operations Platform that's now evolved into a Cloud Automation Platform. How do you define the difference between Kubernetes management and true platform engineering?

Topic 3 - What does “Enterprise Ready” Kubernetes look like in 2025?

Topic 4 - Let’s flip over to AI/ML and GPUs with Kubernetes for a bit. Many developers and data scientists aren’t aware of the underlying platform they run on. I saw a stat recently that about 95% of AI runs on Kubernetes, either on-prem or in the cloud. Despite this, Platform teams are often stuck doing manual GPU provisioning, which doesn't scale with AI adoption. How do modern GPU orchestration solutions change the platform team's role?

Topic 5 - With GPU workloads often handling sensitive data and AI models, security becomes even more critical. How should organizations approach security and compliance in their GPU-enabled Kubernetes operations?

Topic 6 - "Most developers don't want to write YAML or manage clusters — they just want to ship software." How do you balance giving developers the self-service capabilities they want while maintaining the control and governance that platform teams need?

FEEDBACK?

• Email: show at the cloudcast dot net
• Bluesky: @cloudcastpod.bsky.social
• Twitter/X: @cloudcastpod
• Instagram: @cloudcastpod
• TikTok: @cloudcastpod