December 17, 2024 • 41 mins
Andy Suderman, CTO at Fairwinds, CNCF Ambassador, and Kubernetes Policy Working Group Co-Chair, joins us to decode the complexities of Kubernetes adoption. From deployment challenges to policy management best practices, Andy shares real-world insights drawn from his experience helping organizations navigate container orchestration. We explore the nuanced differences between cloud providers' Kubernetes offerings, discuss the critical role of policy in securing and optimizing clusters, and get his perspective on where Kubernetes is headed in the coming years.
Where to Find Andy
- LinkedIn: https://www.linkedin.com/in/sudermanjr/
- BlueSky: https://bsky.app/profile/sudermanjr.bsky.social
Show Links
- CNCF: https://www.cncf.io/
- CNCF Slack: https://slack.cncf.io/
- Fairwinds: https://www.fairwinds.com/
- Fairwinds GitHub: https://github.com/FairwindsOps
- Fairwinds YouTube: https://www.youtube.com/@fairwinds1174
- Kubernetes Slack: https://kubernetes.slack.com/
- Policy Working Group: https://github.com/kubernetes/community/tree/master/wg-policy
Follow, Like, and Subscribe!
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Andy (00:00):
Kubernetes is not secure or configured perfectly by
default.
I can get something running inKubernetes very quickly and the
defaults are not any of thosethings.
They're not secure, they're notefficient, they're not
necessarily reliable, and whatpolicy can do is turn your
Kubernetes environment intosomething that is secure,
reliable and efficient bydefault.
William (00:41):
Welcome to another episode of the Cloud Gambit
podcast, where we weave complexnarratives of technology into
captivating stories withcaptivating guests.
I'm your host, William, andwith me my co-host, the
Shakespeare of servers, the JaneAusten of analytics, the
Hemingway of cloud native,Yvonne Sharp.
How is the plot unfolding foryou today, on this beautiful
(01:03):
Friday?
Eyvonne (01:05):
It has been a week full of drama in my world.
Just kids and extracurricularsand unexpected school closings
and all kinds of fun in ourworld.
So we are enduring.
William (01:21):
I continue to do a lot of driving every day, driving
kids around to everything, whichspend a lot of time in the car.
So, other than that, nothingnew.
On my side With us today,though, we have Andy Suderman,
cto at Fairwinds.
Welcome to the podcast, andy.
How are you doing?
I'm doing well.
Thanks for having me.
(01:41):
Yeah, so for those who may notbe familiar with your work,
could you tell us a little bitabout what you do day to day?
Andy (01:49):
Yeah, so Fairwinds is a managed Kubernetes as a service
provider.
So we build on top of majorcloud provider Kubernetes
implementations and providestable platforms for people to
run their applications.
Top of that, we build software,we have open source, do a lot
of things.
We've been in the cloud nativespace for about eight years now,
(02:10):
so just shy of the full timethat Kubernetes has been around.
William (02:18):
Right on.
So how did you start out intechnology in general?
None of us started out inKubernetes, that's for sure.
Andy (02:24):
Well, actually some of us have, because now we're getting
the newer generation of folks.
I've hired a few folks that sayI've never done anything but
Kubernetes and it just continuesto blow my mind when that
happens and I feel old.
So I got my start very young.
My dad worked in IT and hebrought a couple old computers
(02:45):
home I think we were running.
We had a start.
The computers would boot up inWindows like 3.1, and then we'd
have to like start the UI forWindows 95.
Like it was this weird setup,but anyway we built a network in
our house using the IPXprotocol not even TCP, ip to
play StarCraft.
So my intro into technology wasplaying StarCraft across an IPX
(03:09):
network.
William (03:11):
That's awesome, that's the coolest and best way to
learn, because then you havethis.
It's just awesome.
It's like an awesome scienceexperiment, like why you're
having a lot of fun.
Andy (03:23):
Yeah, yeah, so that was me having a lot of fun, yeah yeah.
So that was me at a very youngage and then just kind of kept
that love all the way through.
You know, I worked in like thecomputer lab in high school and
then I went to college formechanical engineering, which is
completely out of the realm ofcomputers Not completely, but
mostly and when I got out ofschool I realized there were a
(03:44):
lot more jobs in software andtechnology than there were in
mechanical engineering and I wasstill pretty good at software
and mostly operations typethings.
So I've always been on the opsside.
I worked for a little firmdoing IT for them for a little
while and then dove fully intothe software world.
Eyvonne (04:02):
And can we just pause for a minute and appreciate the
value of working for a small ITservices firm early in career?
I did that as well and you getexposed to such a broad gambit
part of the pun of problems andyou get to understand users and
(04:24):
their challenges and I justthink it's a great way to start
for anybody who wants to work intechnology.
William (04:34):
Yeah, I would agree.
The larger the company, thelarger.
The more teams you have, themore you get into a focus area.
But the smaller you are, themore you get touched.
You get your hands in multiplecookie jars and you get a really
wide range of experience andthen you can actually figure out
what it is that you want tospecialize in.
Andy (04:52):
You can really narrow that down, yeah definitely, at the
time I was responsible foreverything from desktop support
to printers, to servers, and soI really I did get to see all of
it.
William (05:06):
Yeah, so how did that translate into you specializing
in Kubernetes specifically?
Andy (05:12):
That's a good question.
So I went to work for a companyin Denver called ReadyTalk they
used to build web conferencingsoftware so I got hired as a
sysadmin there maintainingservers in a data center as much
by hand as with automation.
So I got hired as a sysadminthere maintaining servers in a
data center as much by hand aswith automation so about 50% PUP
at 50% SSH and learned a lotreally fast about managing
(05:35):
larger environments and workingwith developers directly and
hosting actual software insteadof just running internal IT for
a company.
And during that time we wereacquired and we had a team of
about 11 people.
It was like two sysadmins, liketwo estates, two or three
(05:56):
application infrastructureengineers all the different
titles we had back then and allkind of working towards the same
goal of deploying andautomating, deploying the
software.
And the company that purchasedus did a massive layoff and so
we went from a team of 11 to ateam of five.
(06:17):
And so then I started torealize we have to learn this
like DevOps thing, right, torealize we have to learn this
like devops thing, right?
Um, like how do, how do we domore automation but also push
more back towards the developerso that they can be more
self-service, and so that was myfirst foray into like devops,
and about that time I wasintroduced to kubernetes.
(06:38):
Um.
So we, uh, being in denver, Iwas able to go to the Boulder
Kubernetes meetup at the oldDeus office and Chris Nova gave
a presentation on COPs, or KOPsas you can, as a lot of folks
say.
So I saw that demo and I waslike we need this technology.
(07:02):
It can do all of the thingsthat we've been struggling to do
in the data center We've beenstruggling to get working, you
know, high availability,automatic failover, automatic
scaling that we've beenattempting to do with this.
Like we had this awfulhomegrown Zen VM set up that we
all hated, and anyway, I justsaw it and it blew my mind and I
(07:22):
was like we have to do this,and that was probably about
eight, nine, eight years ago orso, so right in the early days,
and just kind of went all infrom there, brought it back to
the team.
The team was all in, startedbuilding things on Kubernetes,
did that for another year or so,and then that's when I moved
over to ReactOps.
William (07:42):
That's awesome, awesome .
Yeah, I've had.
So I think you mentionedearlier that fairwinds.
Did you say that you all reallyfocused on the cloud native
provided?
Uh, cube services like the eksis and aks is a little world, is
that right?
Andy (07:58):
yeah, we do now.
Obviously we got our startbefore eks existed, before eks,
so we did other things back then, but nowadays that's our focus.
William (08:08):
Right on.
Yeah, I have some experiencewith those.
Well, at least the top three,the AWS, azure and GCP Less
Azure, I guess, and more EKS andGKE.
But one thing that still standsout in my mind when I was in
the enterprise and that wasday-to-day life was like oh,
this is not easy stuff, like Iremember in the beginning days I
wanted to like pull my hair outa lot.
(08:29):
Um, there was, and there's, agreat amount of like technical
complexity.
But a lot of times, if you havethe right engineering and you
can make the right hires andbuild the right talent,
technical complexity, you canreally work through that.
But with Kubernetes and eveneven the cloud managed services,
it's, it's a it's a differentway of doing so many different
(08:51):
things.
There's like organizationalhurdles, like there's a lot of
strategic considerations,especially if you're not
Greenfield, like you're notbuilding something new, but
you're transitioning, like maybeyou're migrating, transitioning
data center applications toKubernetes and Cloud Native.
Do you have any, or can youtalk about some of the
(09:13):
challenges you see with thiswhen adopting Kubernetes even
today?
Andy (09:19):
Yeah, definitely, definitely.
So you know, going back to myroots at this web conferencing
company, obviously we raneverything in the data center.
We wanted to use Kubernetes andreally the only way to get
started was to find somethingGreenfield.
So the company that acquired uswas looking to build this
particular piece of the stackand it needed to be more
(09:42):
regionally located to the users,to be more regionally located
to the users, and so this was myopportunity to say hey, maybe
Kubernetes is the right choicehere because we can deploy
quickly, we can update easilyacross all these various regions
that we need to deploy to, andso we built out this brand new
project in Kubernetes.
But then explaining that,evangelizing that within the
(10:03):
company and getting the folksthat had been working on this
similar technology it's like SIPbased you know web audio
technology that's been aroundfor 30 years Trying to convince
these engineers that have beendoing this on you know bare
metal for 20 years that weshould deploy to Kubernetes is a
hurdle.
(10:25):
And it's not a technologyproblem, in my opinion, it's a
people problem, it's a cultureproblem.
You have to be able to sell theidea.
You have to be able to bringpeople around you and get them
behind it and build kind of thisgroundswell movement from the
bottom up, at least within thatcompany.
That worked.
But that's really, it'sovercoming fear and uncertainty.
(10:48):
It's similar to the move to thecloud, move to Kubernetes.
Eyvonne (10:53):
People are just scared of it, and so getting them to
not be scared, I would love tohear more if there were specific
either talk tracks orconversations that stand out in
your mind that helped move thosefolks toward a new technology
stack, because I find that oftenthat's the greatest hurdle to
(11:15):
overcome is getting enough ofthe organization on board with
the potential to actually makethe change real.
It's not so much the technology, like you said, so I'd love to
hear more about specificallywhat helped to make that
transition.
Andy (11:34):
Yeah, I don't know that we ever successfully made it at
that company.
So I can jump further into myhistory over the last six or so
years of working with you knowsmall to large companies in
deploying their Kubernetes andit's a much broader experience
because we are managed serviceproviders.
And I think really you know itdepends on your audience when
(11:55):
you're talking to the highlytechnical folks, you're talking
to your you know staff engineers, principal engineers or even
your senior engineers about whywe should switch technologies,
really showing the technologicalbenefits.
How, like, really for me, liketalking about just going back to
the data center days we wouldspend a month every year
rotating certificates.
(12:16):
Right, we would get a new cert,right, we'd get it issued via
the cert provider and it wouldtake us literally like a month
to like copy the cert out to allof these servers and restart
all the Java, like all theTomcats, and get them to pick up
this new cert.
Eyvonne (12:34):
And so I just saw some PTSD flash through William's
eyes.
He left us there for a minuteand went to another place.
William (12:45):
Yeah, definitely some PTSD in multiple scenarios there
.
I understand that pain yeah,yeah.
Andy (12:50):
So then you know you build a poc, you spent you spin up
cert manager, right, you get itto pull in a cert and you show
them how, all of a sudden, we'vejust taken a month of effort
off of our plate for the nextyear, right, and that you know
it might take us a month tomigrate all of our stuff to use
CertManager, but that's forever.
Now we're done.
(13:11):
Now we can focus on otherthings that are more important
to us.
Those technical demos and thoseproofs of concept of these are
the places where we have pain.
This is how we solve that withthis.
New technology is a huge helpwith the technical folks with.
William (13:28):
You know any new, new technology that's getting
adopted, even if it's even ifit's new for you but it's, you
know, not new for the rest ofthe world.
You know you tend to step on afew landmines, here and there it
happens.
Have you seen just top-of-mindmistakes I guess common mistakes
(13:56):
that organizations can makewhen they go hardcore into
Kubernetes when they don't haveexperience there prior.
Andy (14:02):
Yes, I love talking to those people because they make
great customers.
Um, so some of the things thatwe see, you know there's two
sides of it.
There's one, which is veryprevalent today, which is
massive over provisioning right.
They haven't thought about thecontainerized world.
They've taken the settings fromtheir old vms or even their
(14:25):
laptops and just applied it youknow, resource request limits to
their pods and now they justhave this massive cloud
environment.
It's just running away on.
That's super common.
You hear that story all thetime these days.
Um, but some of the moreinteresting ones are folks not
understanding some of theunderlying complexities of
things like probes and autoscaling, and so they've gotten
(14:47):
themselves into a state wherethey're just like sitting there
manually scaling a deployment upand down, like every morning
somebody will run a kubectlcommand and scale up, you know,
like a deployment.
We're seeing less and less ofthis these days.
This is a lot of.
This was more in the early days.
Another fun one that I heardone company that we talked to
(15:08):
decided to.
They wanted to only run onecluster, so they had this like
800, 900 node cluster that wasdev stage test prod all in one
cluster and they'd written thismassive internal tool to deploy
to this cluster and theyrealized eventually that they'd
(15:30):
gotten themselves into a hole,like they could not update
anything without downtime andprod, like they had massive
issues and they were stuck inthis tech debt hole because this
massive internal tool thatthey'd written to do their
deployments couldn't deploy tomultiple clusters.
So they could spin up moreclusters and they could put
stuff in them, but they couldn'tactually deploy their app to it
and they had like a six monthbacklog of stuff to fix to get
(15:53):
to multiple clusters.
And it was just like that.
I feel for y'all, like that'stough.
Eyvonne (16:01):
Well, it's the kubernetes.
Monolith right so you're?
You're taking two differentparadigms and trying to munch
them together, and then neitherof them work right.
William (16:13):
That is such a good one that you point out though that
is actually one of my worstexperiences with doing this, and
not even if you take thatmonolithic cluster approach and
you don't even separate it outper environment.
You can't even differentiatebetween like prod and non-prod.
That feeds into like all sortsof funny, like security
compliance, just differentthings that you want to avoid at
(16:35):
all costs.
But it brings up a goodquestion, because when I was
working for an organization, atone point where Kubernetes was
new to us, we had some very.
It was at the point in thejourney of cloud where
developers had a lot of power.
They had all the power and theyalso had a lot of budget, so
(16:56):
they were pushing for each.
It just feels funny saying thisnow, but each development team
was basically getting their ownEKS cluster.
So when you say overprovisioning, we were like over
provisioned on steroids and wehad to rope that sucker back in
after a year of blowing outspend.
It was.
It was quite challenging.
(17:17):
So I guess my question is youknow there is no one size fits
all, I understand that, but doyou have any like guidelines or
recommendations or things you'veseen as far as like?
Like we had maybe an MVP.
We got our vehicle product here.
When is it time, like what aresome impending events or things
(17:37):
to tell us that we need a newcluster in an environment?
Andy (17:42):
interesting, interesting, I mean for the majority of folks
that we work with, one prod,one non-prod is enough.
But there are things.
There are some clients, like Imean most of the companies that
we work with, they're medium tolarge.
Some of the larger ones willhave, you know, multiple
production clusters.
But the medium-sized companieshave not reached a scale where
(18:05):
one cluster can't cut it forprod, where you start to run
into other scenarios DR, datalocality, right.
You have business reasons thatrequire you to have another
production cluster or anotherstack in another place.
You know there's also somespecific technological
requirements.
Like I talked about earlier.
(18:26):
We worked with a gaming company.
You want to have your gameservers as close to your end
users, to your players, aspossible, and so we were running
many clusters in many regionsand so lots of different
topologies there.
But then if you've gotten to asophisticated enough point where
you can switch back and forthbetween clusters, maybe you
(18:47):
don't need another one.
But maybe blue green kubernetesupgrades is in the cards for
you because you have the abilityto actually route traffic.
Move to another cluster, yourdata, you've solved your data
locality problem and that'swhere I think folks can really
benefit from multiple productionclusters.
William (19:09):
Right, okay, yeah, that makes a lot of sense and I
don't know if this is a goodquestion or not, but it just
popped in my head.
I don't mean to put you on thespot, but I think there's a lot
of fans out there for each ofthe cloud providers, kubernetes,
services, and I've worked inall three.
But I tend to be partial to EKSjust because it's where I have
(19:32):
more experience.
But I do know that when I setup GKE, it seemed much easier.
The whole process of onboardingand operationalizing everything
within the company I wasworking for at the time, across
a few teams.
It was pretty substantial.
It was much easier, much lesswork involved than EKS.
(19:53):
So do you have any experienceswith kind of like what's easier
or a company would use one overthe other?
I don't know if this is a goodquestion or not, but I'm curious
.
Andy (20:05):
In the beginning we only worked with AWS and GKE, with a
brief stint with Oli Cloud.
I won't talk about that.
But so on AWS we were runningCOPS and then in Google we were
running GKE, because EKS didn'texist yet, and I will say I
think this still continues tothis day.
If you want the newest, mostcutting edge stuff and features
(20:30):
of Kubernetes and you want toupgrade quickly and you want to
use all of the newer APIs asthey come out, even in beta in
some cases, gke is the place tobe, because there's still a
large chunk of Kubernetesdevelopment being done within
Google for GKE and for theircustomers.
(20:50):
In fact, one of our oldemployees, a good friend of mine
, rob Scott, works at Google onthe Gateway API and that's going
to get implemented fully in GKElong before it does anywhere
else, unless you're usingsomething else open source that
you're adding onto your cluster.
So it's less work to get tothose things GKE does and can
(21:16):
make things a lot easier reallyquickly.
I agree I think a lot of folksdefault to AWS because they're
comfortable with it and EKS hasbecome a very good product.
It did not start out that way.
In the early days it was alittle rough around the edges,
but nowadays we use it to greatsuccess and very well.
(21:37):
So the other thing I thinkabout a lot is networking.
Between the two, googlenetworking and AWS networking
are very, very different and ifyou want to learn like truly
cloud native networking and dosome really interesting stuff
that may feel like magic atsometimes, gcp is awesome right.
(22:01):
You can do some cool stuff overthere that you just can't do in
AWS right now.
So I don't know, I kind of feellike you just generally that
kind of cutting edge, newer,truly cloud native experience in
Google versus the.
You know I still have, you know, worked in a data center.
I know like all the data centernetworking and how to build a
data center.
(22:22):
Aws is going to be the closestanalog to that in the cloud and
it will feel very comfortable toyou.
William (22:29):
Yeah, I can definitely salute that because AWS is just.
It's so hierarchical with theway that you build and nest
resources, especially withnetworking, like you're locked
in, subnets are in scope.
At the availability zone levelYou're really rigid on okay,
this is a regional construct,this is an availability zone
construct, but Google sort oflike blurs those lines and makes
(22:52):
stuff happen under the hood andthings just happen to work and
you don't have to think about asmany of those things on the
network side, which is easy,easy sometimes, but then when
you want to get in and dig intosome details for certain reasons
, it can be a little morechallenging.
Andy (23:07):
Absolutely yeah.
If you want a pod in onecluster to talk directly to
another pod in another clusterin a completely different region
, use GCP, because that's goingto be a nightmare in AWS, but
GCP will do it out of the box ifyou click the right button.
William (23:24):
Yvonne loves to hear this.
Eyvonne (23:26):
I do.
William (23:27):
The GCP hype machine.
Eyvonne (23:29):
Do I speak up or do I just let them keep going?
No, yeah, we love theflexibility of GCP networking,
global VPCs, all that good stuff.
We stitch stuff together underthe hood in ways that other
folks don't.
It's an engineering firstmindset, you know from the place
where Kubernetes was born.
(23:50):
So we just have some technologylegs up there, especially in
this space.
Andy (23:56):
So yeah, as a business leader, I have to like raise an
eyebrow on occasion at Googleright, because they are a little
riskier from like hey, is thisproduct going to exist in 12
months, kind of thing.
But I don't think GKE is goinganywhere, so I'm pretty safe
with GKE.
Eyvonne (24:13):
Yeah, like the GCP network and GKE are foundational
solutions and we've got biggerproblems if those things go away
.
Right?
William (24:23):
Yeah, I remember going through some of those talks when
we adopted GCP at a in anorganization and I think one of
the things we and you know sortof came to the conclusion of is
yeah, I mean, some products aregoing to go away.
But if you look at, like, whatGoogle's core business model is,
if you look at the ads, youknow the ad business and you
know look at the cloud computingbusiness and how it's grown and
(24:46):
like the amount of investmentthey've put in, I mean I think I
could be totally wrong, but Ithink that's a huge part of the
future and that means, like thecore services of their, their
cloud offering, the standardstuff like vp, subnet, some of
the platform as a service,databases, banner, all those
(25:07):
things, and then GKE like thethings that everybody's using
that are pretty core to runningGCP are probably going to be
around for the long haul.
Eyvonne (25:18):
Well, and we can't go a podcast without mentioning AI,
right.
So the ability to run AIinfrastructure and attach that
to your you know Kubernetesenvironment to be able to do
inference or whatever.
Like all that's there too.
So yeah, I couldn't help myself, sorry.
William (25:39):
Yeah, it's a feature, it's not a product, it's a
feature, it's not a product.
Yes, yes, I love it.
So you're co-chair of theKubernetes Policy Working Group,
a part of the CNCF.
(26:02):
Yvonne and I have actually beenreally interested in just sort
of the work that the CNCF andthe Linux Foundation has been
doing, especially with a lot ofthe licensing changes like new
products I love, like I loveOpenToku and I still use and
love Terraform.
But just seeing kind of how theCNCF is helping out and really
like instead of I'm going totread carefully here but I guess
(26:24):
instead of dividing community,just bringing community together
, it's pretty awesome.
But can you tell us a littlebit about what this group that
you co-chair focuses on?
Yeah, for sure.
Andy (26:40):
So the group was started primarily by Jim Buguadia from
Nirmata, who's the creators ofKiberno.
We have an open source policyengine.
Kiberno is also an open sourcepolicy engine.
I have to give them kudosbecause they built a really
great product and the.
You know they looked around kindof a CNCF landscape and
(27:02):
realized there was no group sortof talking about how to
implement and use policy inKubernetes.
Right, because really you know,in the whole of technology,
kubernetes is a relativelyimmature technology that's
changing obviously rapidly.
But you know, you compare it toother things that have been
around for 30 years.
It's relatively new.
(27:22):
And so policy was when thisstarted probably I don't know
when the actual startup was.
It's probably like four, three,four years ago.
Policy was brand new on thescene.
Opa had been around for a littlewhile but OPA or OPA, nobody
had been really using it, nobodywanted to write Rego.
And so they said you know, wekind of we want to get this
(27:45):
group together to talk about, togive advice to the community,
to put out information about thebest ways to implement and use
policy, why you should implementand use policy and things like
that.
So that's been really ourcharter.
I can't speak it from memory,unfortunately, but it's probably
(28:05):
not a good thing as a co-chair,but we spend a lot of time just
working with the community,putting out white papers,
talking to folks about theirpolicy implementations, how that
ties into governance, riskcompliance and really trying to
be tool agnostic and talk aboutthe how and the why of policy as
(28:27):
a whole.
William (28:28):
That's great.
Why so like?
Just for the audience that'snot as familiar with like policy
management with Kubernetes.
Why is policy management sodaggone important in Kubernetes
in the first place?
Andy (28:41):
I mean there's a lot of reasons, but I think the primary
one that I like to talk aboutbecause I think it hits home for
most engineers is at least mostops engineers is that
Kubernetes is not secure orconfigured perfectly by default.
I can get something running inKubernetes very quickly.
(29:04):
Something running in Kubernetesvery quickly, it might not
scale, it might not bin packwell, it might not be secure
right, because I can just throwsome stuff at Kubernetes and the
defaults are not any of thosethings.
They're not secure, they're notefficient, they're not
necessarily reliable.
So you really have to know whatyou're doing to get to that
point and what policy can do isturn your Kubernetes environment
(29:27):
into something that is secure,reliable and efficient by
default, because you'reenforcing all of those rules
about how you use Kubernetes upfront rather than trying to go
back and fix them afterwards.
So that's kind of the dream ofpolicy for me.
William (29:44):
Love it.
Any cool initiatives orprojects like in focus with the
working group or anything in thefuture that we get a sneak peek
of, or is it just kind of whatyou already outlined?
Andy (29:58):
I mean, all of our meetings are completely public,
so there's nothing hiding inthere that I can reveal.
I think some of the moreinteresting things we're working
on can reveal.
I think some of the moreinteresting things we're working
on.
I know Poonam Lamba from Ithink I said her name right.
I apologize if I didn't.
I don't know her last name, butfrom Google is working with the
(30:18):
rest of the group a white paperon what they're calling shift
down security.
So we have shift left security,but we're talking about
shifting it down in the stack.
So I think that's interesting.
I haven't had a whole lot oftime to review it, but it sounds
like a very interesting paper.
I'm currently working slowly onputting together a policy
(30:40):
survey to understand better howfolks are using policy right now
, because I think we've movedaway from the days where nobody
knows what policy is, but Ithink we're still in the time
where very few people areactually utilizing it to great
effect.
And then, on top of that, wehave a few members of the
working group who are alsomembers of other SIGs and TAGs
(31:03):
and things that are working veryhard on the compliance angle,
and so I know the folks atDefense Unicorns are doing a lot
of really cool stuff.
I think there's even a newchannel for Lula, which is a
tool for basically tying yourcompliance to your policy,
because policy enforcement isnot compliance, but it can be
(31:23):
used to feed compliance, and sothey're working on using some
actual like automated tools todo compliance and tie it to your
policy engine.
Eyvonne (31:34):
Just you know, your friendly PSA Policy enforcement
is not compliance, is notsecurity.
Yes, those are all verydifferent domains.
They do inform and intersectand overlap in some interesting
places, but they are distinctand it's important to know and
understand those distinctions.
Andy (31:54):
Absolutely.
Compliance can never.
I've been saying lately,compliance can never create good
security, but good security canmake compliance easier.
Eyvonne (32:05):
That's right.
That's a great way to say it,and you know you can be
compliant and follow the processand have a rule created that's
permit IP any, any, that wouldcomply with your policies
because it was approved by allthe right people and implemented
according to your standards.
William (32:26):
That is also secure secure I wonder how many times
that's been done I've.
I've sort of stood in the gapand kept it from happening a
time or two yeah, and, as andysaid, like these meetings are
public and also the, the cncfhas an amazing slack channel
that is anybody can join, as I'mcreeping in there all the time
(32:48):
looking at stuff.
It's great.
So if you want to know reallywhat's going on, not just with
this working group, but lots ofother work that the CNCF is
doing, join the Slack channel.
Add some of the channels inthere, the groups, and just kind
of peek around.
It's good stuff.
Andy (33:08):
Yeah, both the CNCF Slack and the Kubernetes Slack.
The organizational structure isa little bit confusing, but
tags are CNCF, sigs areKubernetes and both of them have
working groups Very confusing,but both Slacks are great.
Be careful how many channelsyou add, because it will create
(33:29):
a lot of noise.
William (33:31):
yeah, it does the time to wax philosophical?
I guess a little bit, but youknow, things are always changing
in tech.
We know that the only the onlything that's consistent is
change, and sometimes it's a bitoverwhelming.
Kubernetes is no stranger tothis, as it changes very rapidly
.
Is Kubernetes sort of going,you know, when you started
(33:54):
working in the Kubernetes spaceand you started learning, is it
sort of going in the directionthat you thought it would?
Do you think that, like, wheredo you see it headed from here
on out?
What do you see Kubernetesbecoming in the next five years?
Andy (34:10):
I get asked this question so often.
William (34:13):
It's like the unanswerable question.
Andy (34:15):
It is too.
It's impossible.
I know we do this thing everyyear, like our marketing folks
love putting together like whatare your predictions for the
next year?
And I always my boss at thetime is like I hate doing these.
It's like I feel like such acharlatan, just like spouting
off this, this knowledge that is, you know, we, we have no way,
(34:37):
but I do have thoughts, ofcourse.
So, like the first question youasked, which is is it going the
way that I thought it wouldfrom eight years ago?
And I think absolutely yes.
It's definitely seeing much moreadoption.
The cloud providers are allheavily involved with it.
(35:00):
A Kubernetes offering right,your DigitalOceans, your Linodes
, your Sivos built almostentirely on the idea of having a
Kubernetes offering all thesecloud providers.
So I think that's super cool.
The one thing I think isinteresting is that adoption has
been maybe slower than I hadanticipated, I think partly just
(35:23):
because of the hype.
Early on, we were getting thesereports that were saying, like
every major corporation is usingKubernetes and I said, well,
sure, but it's like five peoplewithin the whole organization
that are using Kubernetes andthe rest of it's all still
running on whatever they wererunning before.
That's slowly starting tochange, but it's very slow and
(35:44):
of course the adoption of anynewer technology is going to be
slow.
There's that to answer yourfirst question.
The second answer much harder,but I think we're starting to
see echoes of it already.
I think Kubernetes will becomesort of the underpinning of a
(36:05):
lot of various offerings.
So you're seeing more and moreofferings built on top of
Kubernetes, that sort ofabstract away some of the
complexities for you.
I've been talking to some folksat like North Flank and a
couple other places that arebuilding these like developer
platforms for deploying yourapps that are built entirely on
Kubernetes and then maybe havethe option to like take over the
(36:28):
full management of Kubernetes.
And then we're also seeingthings like Cloud Run, fargate
not necessarily Kubernetes underthe hood, but very
Kubernetes-like right Knativestuff like that, where
containers and some of theconcepts from Kubernetes are
(36:50):
still ubiquitous, and so I thinkthis way of running
infrastructure will remain intothe future, whether we actually
use raw Kubernetes in five, 10years, maybe not.
Maybe there will come alonganother ubiquitous abstraction
layer, or so you know, somefolks will continue to use just
(37:12):
pure kubernetes and a lot offolks will use these abstraction
layers, but I don't think it'sgoing anywhere totally agree
with that one.
William (37:19):
I think it's here to stay so much, especially startup
software these days.
I don't want to say it'sexclusively built on Kubernetes,
but I mean I would love, I mean, if there was some numbers out
there of startups, like in thepast two years, that are
building exclusively onKubernetes.
I bet that number is like veryhigh, Just because the
(37:40):
distributed nature, distributedscale of you know the outcome.
Andy (37:45):
It would be interesting.
I you know I went to.
I did a talk at Boulder StartupWeek about infrastructure for
startups, and I think a lot ofthe sentiment in the room was
people were scared to useKubernetes because of the
complexity, because of the size,and I think the general
recommendation that I've seenfrom experts is don't use
(38:08):
Kubernetes.
Use something either built ontop of Kubernetes or that allows
you to move to Kubernetes whenyou need that complexity and
scale.
And so I think it would beinteresting to see that survey,
because I wonder how many peoplewould say, yes, I'm running on
Kubernetes or no, I builtentirely on Fargate or Cloud Run
or you know some other.
You know container as a servicetype thing, but still fully
(38:32):
containerized, using some of theconcepts that would allow me to
utilize Kubernetes down theroad.
Eyvonne (38:37):
When it makes sense.
If you're a smart startup andyour goal is, like time to value
, like you've got to get thething built and out there.
Scale is a concern, but it's asecondary concern, absolutely.
And so if you can deploy in aserverless platform, get
something up and running, doproofs of concepts, be able to
(39:00):
start onboarding customers,that's a huge win.
But there are scale andflexibility limitations with
serverless platforms thatanybody's going to hit once they
reach a certain size.
So it makes sense to deployinitially in a more managed
environment and graduate up asyour needs dictateated.
(39:23):
But I think ultimately, atscale, anybody who's running a
significantly complex enough,demanding enough application is
going to end up there eventually.
William (39:37):
Absolutely yeah.
What I'm hearing is basicallylike the right solution for the
right problem.
So don't use a dump truck whenyou could use a pickup truck or
when you're building yourstartup, just like, okay, don't
just jump immediately into aservice mesh until you have
service mesh problems out therethat you need to solve in
(40:00):
general, because that adds evenmore complexity to the scenario.
Andy (40:07):
I'm writing a talk right now called Kubernetes.
As a Big Hammer Is Everything aNail, and it will be exactly
about this topic.
William (40:14):
I love that I have something to add to that.
We'll be on the lookout.
Is it going to be recorded?
Andy (40:20):
It'll be at Kubernetes Community Day in DC, so I'm sure
it will be recorded.
William (40:28):
Awesome, right on.
I guess we've hit sort of theunless you have anything else.
Do you have anything else youwant to throw in there?
Yvonne, I'm good.
Awesome.
This has been an excellentconversation.
I love conversations like this,just talking about the real you
know the meat on the bone asfar as the technology and
digging in and just kind offiguring out what's out there,
(40:48):
what's happening, and especiallybeing able to talk to someone
that's an expert like yourself.
So this has been an awesome,awesome conversation.
We really appreciate your time.
Where can the audience find youat?
Do you have?
Andy (41:01):
any social media, Mostly on LinkedIn.
These days I tend to avoid someof the others.
I think I have a Mastodonaccount that I check on occasion
.
I'm generally SudermanJRSudermanJr just about everywhere
, or Andy Suderman, and I'mdefinitely active in the
Kubernetes, the CNCF Slack.
(41:22):
We also have a community Slackfor our open source at Fairwinds
.
You can find it on any of ouropen source repos.
Slack is definitely the bestway to get a hold of me.
Eyvonne (41:33):
I'm in it 24-7.
Andy (41:34):
it feels like Awesome.
William (41:38):
I'll put all those in the show notes.
Thank you so much, thank you.
Andy (41:42):
It's been a pleasure.
Kubernetes is not secure or configured perfectly by
default.
I can get something running inKubernetes very quickly and the
defaults are not any of thosethings.
They're not secure, they're notefficient, they're not
necessarily reliable, and whatpolicy can do is turn your
Kubernetes environment intosomething that is secure,
reliable and efficient bydefault.
William (00:41):
Welcome to another episode of the Cloud Gambit
podcast, where we weave complexnarratives of technology into
captivating stories withcaptivating guests.
I'm your host, William, andwith me my co-host, the
Shakespeare of servers, the JaneAusten of analytics, the
Hemingway of cloud native,Yvonne Sharp.
How is the plot unfolding foryou today, on this beautiful
(01:03):
Friday?
Eyvonne (01:05):
It has been a week full of drama in my world.
Just kids and extracurricularsand unexpected school closings
and all kinds of fun in ourworld.
So we are enduring.
William (01:21):
I continue to do a lot of driving every day, driving
kids around to everything, whichspend a lot of time in the car.
So, other than that, nothingnew.
On my side With us today,though, we have Andy Suderman,
cto at Fairwinds.
Welcome to the podcast, andy.
How are you doing?
I'm doing well.
Thanks for having me.
(01:41):
Yeah, so for those who may notbe familiar with your work,
could you tell us a little bitabout what you do day to day?
Andy (01:49):
Yeah, so Fairwinds is a managed Kubernetes as a service
provider.
So we build on top of majorcloud provider Kubernetes
implementations and providestable platforms for people to
run their applications.
Top of that, we build software,we have open source, do a lot
of things.
We've been in the cloud nativespace for about eight years now,
(02:10):
so just shy of the full timethat Kubernetes has been around.
William (02:18):
Right on.
So how did you start out intechnology in general?
None of us started out inKubernetes, that's for sure.
Andy (02:24):
Well, actually some of us have, because now we're getting
the newer generation of folks.
I've hired a few folks that sayI've never done anything but
Kubernetes and it just continuesto blow my mind when that
happens and I feel old.
So I got my start very young.
My dad worked in IT and hebrought a couple old computers
(02:45):
home I think we were running.
We had a start.
The computers would boot up inWindows like 3.1, and then we'd
have to like start the UI forWindows 95.
Like it was this weird setup,but anyway we built a network in
our house using the IPXprotocol not even TCP, ip to
play StarCraft.
So my intro into technology wasplaying StarCraft across an IPX
(03:09):
network.
William (03:11):
That's awesome, that's the coolest and best way to
learn, because then you havethis.
It's just awesome.
It's like an awesome scienceexperiment, like why you're
having a lot of fun.
Andy (03:23):
Yeah, yeah, so that was me having a lot of fun, yeah yeah.
So that was me at a very youngage and then just kind of kept
that love all the way through.
You know, I worked in like thecomputer lab in high school and
then I went to college formechanical engineering, which is
completely out of the realm ofcomputers Not completely, but
mostly and when I got out ofschool I realized there were a
(03:44):
lot more jobs in software andtechnology than there were in
mechanical engineering and I wasstill pretty good at software
and mostly operations typethings.
So I've always been on the opsside.
I worked for a little firmdoing IT for them for a little
while and then dove fully intothe software world.
Eyvonne (04:02):
And can we just pause for a minute and appreciate the
value of working for a small ITservices firm early in career?
I did that as well and you getexposed to such a broad gambit
part of the pun of problems andyou get to understand users and
(04:24):
their challenges and I justthink it's a great way to start
for anybody who wants to work intechnology.
William (04:34):
Yeah, I would agree.
The larger the company, thelarger.
The more teams you have, themore you get into a focus area.
But the smaller you are, themore you get touched.
You get your hands in multiplecookie jars and you get a really
wide range of experience andthen you can actually figure out
what it is that you want tospecialize in.
Andy (04:52):
You can really narrow that down, yeah definitely, at the
time I was responsible foreverything from desktop support
to printers, to servers, and soI really I did get to see all of
it.
William (05:06):
Yeah, so how did that translate into you specializing
in Kubernetes specifically?
Andy (05:12):
That's a good question.
So I went to work for a companyin Denver called ReadyTalk they
used to build web conferencingsoftware so I got hired as a
sysadmin there maintainingservers in a data center as much
by hand as with automation.
So I got hired as a sysadminthere maintaining servers in a
data center as much by hand aswith automation so about 50% PUP
at 50% SSH and learned a lotreally fast about managing
(05:35):
larger environments and workingwith developers directly and
hosting actual software insteadof just running internal IT for
a company.
And during that time we wereacquired and we had a team of
about 11 people.
It was like two sysadmins, liketwo estates, two or three
(05:56):
application infrastructureengineers all the different
titles we had back then and allkind of working towards the same
goal of deploying andautomating, deploying the
software.
And the company that purchasedus did a massive layoff and so
we went from a team of 11 to ateam of five.
(06:17):
And so then I started torealize we have to learn this
like DevOps thing, right, torealize we have to learn this
like devops thing, right?
Um, like how do, how do we domore automation but also push
more back towards the developerso that they can be more
self-service, and so that was myfirst foray into like devops,
and about that time I wasintroduced to kubernetes.
(06:38):
Um.
So we, uh, being in denver, Iwas able to go to the Boulder
Kubernetes meetup at the oldDeus office and Chris Nova gave
a presentation on COPs, or KOPsas you can, as a lot of folks
say.
So I saw that demo and I waslike we need this technology.
(07:02):
It can do all of the thingsthat we've been struggling to do
in the data center We've beenstruggling to get working, you
know, high availability,automatic failover, automatic
scaling that we've beenattempting to do with this.
Like we had this awfulhomegrown Zen VM set up that we
all hated, and anyway, I justsaw it and it blew my mind and I
(07:22):
was like we have to do this,and that was probably about
eight, nine, eight years ago orso, so right in the early days,
and just kind of went all infrom there, brought it back to
the team.
The team was all in, startedbuilding things on Kubernetes,
did that for another year or so,and then that's when I moved
over to ReactOps.
William (07:42):
That's awesome, awesome .
Yeah, I've had.
So I think you mentionedearlier that fairwinds.
Did you say that you all reallyfocused on the cloud native
provided?
Uh, cube services like the eksis and aks is a little world, is
that right?
Andy (07:58):
yeah, we do now.
Obviously we got our startbefore eks existed, before eks,
so we did other things back then, but nowadays that's our focus.
William (08:08):
Right on.
Yeah, I have some experiencewith those.
Well, at least the top three,the AWS, azure and GCP Less
Azure, I guess, and more EKS andGKE.
But one thing that still standsout in my mind when I was in
the enterprise and that wasday-to-day life was like oh,
this is not easy stuff, like Iremember in the beginning days I
wanted to like pull my hair outa lot.
(08:29):
Um, there was, and there's, agreat amount of like technical
complexity.
But a lot of times, if you havethe right engineering and you
can make the right hires andbuild the right talent,
technical complexity, you canreally work through that.
But with Kubernetes and eveneven the cloud managed services,
it's, it's a it's a differentway of doing so many different
(08:51):
things.
There's like organizationalhurdles, like there's a lot of
strategic considerations,especially if you're not
Greenfield, like you're notbuilding something new, but
you're transitioning, like maybeyou're migrating, transitioning
data center applications toKubernetes and Cloud Native.
Do you have any, or can youtalk about some of the
(09:13):
challenges you see with thiswhen adopting Kubernetes even
today?
Andy (09:19):
Yeah, definitely, definitely.
So you know, going back to myroots at this web conferencing
company, obviously we raneverything in the data center.
We wanted to use Kubernetes andreally the only way to get
started was to find somethingGreenfield.
So the company that acquired uswas looking to build this
particular piece of the stackand it needed to be more
(09:42):
regionally located to the users,to be more regionally located
to the users, and so this was myopportunity to say hey, maybe
Kubernetes is the right choicehere because we can deploy
quickly, we can update easilyacross all these various regions
that we need to deploy to, andso we built out this brand new
project in Kubernetes.
But then explaining that,evangelizing that within the
(10:03):
company and getting the folksthat had been working on this
similar technology it's like SIPbased you know web audio
technology that's been aroundfor 30 years Trying to convince
these engineers that have beendoing this on you know bare
metal for 20 years that weshould deploy to Kubernetes is a
hurdle.
(10:25):
And it's not a technologyproblem, in my opinion, it's a
people problem, it's a cultureproblem.
You have to be able to sell theidea.
You have to be able to bringpeople around you and get them
behind it and build kind of thisgroundswell movement from the
bottom up, at least within thatcompany.
That worked.
But that's really, it'sovercoming fear and uncertainty.
(10:48):
It's similar to the move to thecloud, move to Kubernetes.
Eyvonne (10:53):
People are just scared of it, and so getting them to
not be scared, I would love tohear more if there were specific
either talk tracks orconversations that stand out in
your mind that helped move thosefolks toward a new technology
stack, because I find that oftenthat's the greatest hurdle to
(11:15):
overcome is getting enough ofthe organization on board with
the potential to actually makethe change real.
It's not so much the technology, like you said, so I'd love to
hear more about specificallywhat helped to make that
transition.
Andy (11:34):
Yeah, I don't know that we ever successfully made it at
that company.
So I can jump further into myhistory over the last six or so
years of working with you knowsmall to large companies in
deploying their Kubernetes andit's a much broader experience
because we are managed serviceproviders.
And I think really you know itdepends on your audience when
(11:55):
you're talking to the highlytechnical folks, you're talking
to your you know staff engineers, principal engineers or even
your senior engineers about whywe should switch technologies,
really showing the technologicalbenefits.
How, like, really for me, liketalking about just going back to
the data center days we wouldspend a month every year
rotating certificates.
(12:16):
Right, we would get a new cert,right, we'd get it issued via
the cert provider and it wouldtake us literally like a month
to like copy the cert out to allof these servers and restart
all the Java, like all theTomcats, and get them to pick up
this new cert.
Eyvonne (12:34):
And so I just saw some PTSD flash through William's
eyes.
He left us there for a minuteand went to another place.
William (12:45):
Yeah, definitely some PTSD in multiple scenarios there
.
I understand that pain yeah,yeah.
Andy (12:50):
So then you know you build a poc, you spent you spin up
cert manager, right, you get itto pull in a cert and you show
them how, all of a sudden, we'vejust taken a month of effort
off of our plate for the nextyear, right, and that you know
it might take us a month tomigrate all of our stuff to use
CertManager, but that's forever.
Now we're done.
(13:11):
Now we can focus on otherthings that are more important
to us.
Those technical demos and thoseproofs of concept of these are
the places where we have pain.
This is how we solve that withthis.
New technology is a huge helpwith the technical folks with.
William (13:28):
You know any new, new technology that's getting
adopted, even if it's even ifit's new for you but it's, you
know, not new for the rest ofthe world.
You know you tend to step on afew landmines, here and there it
happens.
Have you seen just top-of-mindmistakes I guess common mistakes
(13:56):
that organizations can makewhen they go hardcore into
Kubernetes when they don't haveexperience there prior.
Andy (14:02):
Yes, I love talking to those people because they make
great customers.
Um, so some of the things thatwe see, you know there's two
sides of it.
There's one, which is veryprevalent today, which is
massive over provisioning right.
They haven't thought about thecontainerized world.
They've taken the settings fromtheir old vms or even their
(14:25):
laptops and just applied it youknow, resource request limits to
their pods and now they justhave this massive cloud
environment.
It's just running away on.
That's super common.
You hear that story all thetime these days.
Um, but some of the moreinteresting ones are folks not
understanding some of theunderlying complexities of
things like probes and autoscaling, and so they've gotten
(14:47):
themselves into a state wherethey're just like sitting there
manually scaling a deployment upand down, like every morning
somebody will run a kubectlcommand and scale up, you know,
like a deployment.
We're seeing less and less ofthis these days.
This is a lot of.
This was more in the early days.
Another fun one that I heardone company that we talked to
(15:08):
decided to.
They wanted to only run onecluster, so they had this like
800, 900 node cluster that wasdev stage test prod all in one
cluster and they'd written thismassive internal tool to deploy
to this cluster and theyrealized eventually that they'd
(15:30):
gotten themselves into a hole,like they could not update
anything without downtime andprod, like they had massive
issues and they were stuck inthis tech debt hole because this
massive internal tool thatthey'd written to do their
deployments couldn't deploy tomultiple clusters.
So they could spin up moreclusters and they could put
stuff in them, but they couldn'tactually deploy their app to it
and they had like a six monthbacklog of stuff to fix to get
(15:53):
to multiple clusters.
And it was just like that.
I feel for y'all, like that'stough.
Eyvonne (16:01):
Well, it's the kubernetes.
Monolith right so you're?
You're taking two differentparadigms and trying to munch
them together, and then neitherof them work right.
William (16:13):
That is such a good one that you point out though that
is actually one of my worstexperiences with doing this, and
not even if you take thatmonolithic cluster approach and
you don't even separate it outper environment.
You can't even differentiatebetween like prod and non-prod.
That feeds into like all sortsof funny, like security
compliance, just differentthings that you want to avoid at
(16:35):
all costs.
But it brings up a goodquestion, because when I was
working for an organization, atone point where Kubernetes was
new to us, we had some very.
It was at the point in thejourney of cloud where
developers had a lot of power.
They had all the power and theyalso had a lot of budget, so
(16:56):
they were pushing for each.
It just feels funny saying thisnow, but each development team
was basically getting their ownEKS cluster.
So when you say overprovisioning, we were like over
provisioned on steroids and wehad to rope that sucker back in
after a year of blowing outspend.
It was.
It was quite challenging.
(17:17):
So I guess my question is youknow there is no one size fits
all, I understand that, but doyou have any like guidelines or
recommendations or things you'veseen as far as like?
Like we had maybe an MVP.
We got our vehicle product here.
When is it time, like what aresome impending events or things
(17:37):
to tell us that we need a newcluster in an environment?
Andy (17:42):
interesting, interesting, I mean for the majority of folks
that we work with, one prod,one non-prod is enough.
But there are things.
There are some clients, like Imean most of the companies that
we work with, they're medium tolarge.
Some of the larger ones willhave, you know, multiple
production clusters.
But the medium-sized companieshave not reached a scale where
(18:05):
one cluster can't cut it forprod, where you start to run
into other scenarios DR, datalocality, right.
You have business reasons thatrequire you to have another
production cluster or anotherstack in another place.
You know there's also somespecific technological
requirements.
Like I talked about earlier.
(18:26):
We worked with a gaming company.
You want to have your gameservers as close to your end
users, to your players, aspossible, and so we were running
many clusters in many regionsand so lots of different
topologies there.
But then if you've gotten to asophisticated enough point where
you can switch back and forthbetween clusters, maybe you
(18:47):
don't need another one.
But maybe blue green kubernetesupgrades is in the cards for
you because you have the abilityto actually route traffic.
Move to another cluster, yourdata, you've solved your data
locality problem and that'swhere I think folks can really
benefit from multiple productionclusters.
William (19:09):
Right, okay, yeah, that makes a lot of sense and I
don't know if this is a goodquestion or not, but it just
popped in my head.
I don't mean to put you on thespot, but I think there's a lot
of fans out there for each ofthe cloud providers, kubernetes,
services, and I've worked inall three.
But I tend to be partial to EKSjust because it's where I have
(19:32):
more experience.
But I do know that when I setup GKE, it seemed much easier.
The whole process of onboardingand operationalizing everything
within the company I wasworking for at the time, across
a few teams.
It was pretty substantial.
It was much easier, much lesswork involved than EKS.
(19:53):
So do you have any experienceswith kind of like what's easier
or a company would use one overthe other?
I don't know if this is a goodquestion or not, but I'm curious
.
Andy (20:05):
In the beginning we only worked with AWS and GKE, with a
brief stint with Oli Cloud.
I won't talk about that.
But so on AWS we were runningCOPS and then in Google we were
running GKE, because EKS didn'texist yet, and I will say I
think this still continues tothis day.
If you want the newest, mostcutting edge stuff and features
(20:30):
of Kubernetes and you want toupgrade quickly and you want to
use all of the newer APIs asthey come out, even in beta in
some cases, gke is the place tobe, because there's still a
large chunk of Kubernetesdevelopment being done within
Google for GKE and for theircustomers.
(20:50):
In fact, one of our oldemployees, a good friend of mine
, rob Scott, works at Google onthe Gateway API and that's going
to get implemented fully in GKElong before it does anywhere
else, unless you're usingsomething else open source that
you're adding onto your cluster.
So it's less work to get tothose things GKE does and can
(21:16):
make things a lot easier reallyquickly.
I agree I think a lot of folksdefault to AWS because they're
comfortable with it and EKS hasbecome a very good product.
It did not start out that way.
In the early days it was alittle rough around the edges,
but nowadays we use it to greatsuccess and very well.
(21:37):
So the other thing I thinkabout a lot is networking.
Between the two, googlenetworking and AWS networking
are very, very different and ifyou want to learn like truly
cloud native networking and dosome really interesting stuff
that may feel like magic atsometimes, gcp is awesome right.
(22:01):
You can do some cool stuff overthere that you just can't do in
AWS right now.
So I don't know, I kind of feellike you just generally that
kind of cutting edge, newer,truly cloud native experience in
Google versus the.
You know I still have, you know, worked in a data center.
I know like all the data centernetworking and how to build a
data center.
(22:22):
Aws is going to be the closestanalog to that in the cloud and
it will feel very comfortable toyou.
William (22:29):
Yeah, I can definitely salute that because AWS is just.
It's so hierarchical with theway that you build and nest
resources, especially withnetworking, like you're locked
in, subnets are in scope.
At the availability zone levelYou're really rigid on okay,
this is a regional construct,this is an availability zone
construct, but Google sort oflike blurs those lines and makes
(22:52):
stuff happen under the hood andthings just happen to work and
you don't have to think about asmany of those things on the
network side, which is easy,easy sometimes, but then when
you want to get in and dig intosome details for certain reasons
, it can be a little morechallenging.
Andy (23:07):
Absolutely yeah.
If you want a pod in onecluster to talk directly to
another pod in another clusterin a completely different region
, use GCP, because that's goingto be a nightmare in AWS, but
GCP will do it out of the box ifyou click the right button.
William (23:24):
Yvonne loves to hear this.
Eyvonne (23:26):
I do.
William (23:27):
The GCP hype machine.
Eyvonne (23:29):
Do I speak up or do I just let them keep going?
No, yeah, we love theflexibility of GCP networking,
global VPCs, all that good stuff.
We stitch stuff together underthe hood in ways that other
folks don't.
It's an engineering firstmindset, you know from the place
where Kubernetes was born.
(23:50):
So we just have some technologylegs up there, especially in
this space.
Andy (23:56):
So yeah, as a business leader, I have to like raise an
eyebrow on occasion at Googleright, because they are a little
riskier from like hey, is thisproduct going to exist in 12
months, kind of thing.
But I don't think GKE is goinganywhere, so I'm pretty safe
with GKE.
Eyvonne (24:13):
Yeah, like the GCP network and GKE are foundational
solutions and we've got biggerproblems if those things go away
.
Right?
William (24:23):
Yeah, I remember going through some of those talks when
we adopted GCP at a in anorganization and I think one of
the things we and you know sortof came to the conclusion of is
yeah, I mean, some products aregoing to go away.
But if you look at, like, whatGoogle's core business model is,
if you look at the ads, youknow the ad business and you
know look at the cloud computingbusiness and how it's grown and
(24:46):
like the amount of investmentthey've put in, I mean I think I
could be totally wrong, but Ithink that's a huge part of the
future and that means, like thecore services of their, their
cloud offering, the standardstuff like vp, subnet, some of
the platform as a service,databases, banner, all those
(25:07):
things, and then GKE like thethings that everybody's using
that are pretty core to runningGCP are probably going to be
around for the long haul.
Eyvonne (25:18):
Well, and we can't go a podcast without mentioning AI,
right.
So the ability to run AIinfrastructure and attach that
to your you know Kubernetesenvironment to be able to do
inference or whatever.
Like all that's there too.
So yeah, I couldn't help myself, sorry.
William (25:39):
Yeah, it's a feature, it's not a product, it's a
feature, it's not a product.
Yes, yes, I love it.
So you're co-chair of theKubernetes Policy Working Group,
a part of the CNCF.
(26:02):
Yvonne and I have actually beenreally interested in just sort
of the work that the CNCF andthe Linux Foundation has been
doing, especially with a lot ofthe licensing changes like new
products I love, like I loveOpenToku and I still use and
love Terraform.
But just seeing kind of how theCNCF is helping out and really
like instead of I'm going totread carefully here but I guess
(26:24):
instead of dividing community,just bringing community together
, it's pretty awesome.
But can you tell us a littlebit about what this group that
you co-chair focuses on?
Yeah, for sure.
Andy (26:40):
So the group was started primarily by Jim Buguadia from
Nirmata, who's the creators ofKiberno.
We have an open source policyengine.
Kiberno is also an open sourcepolicy engine.
I have to give them kudosbecause they built a really
great product and the.
You know they looked around kindof a CNCF landscape and
(27:02):
realized there was no group sortof talking about how to
implement and use policy inKubernetes.
Right, because really you know,in the whole of technology,
kubernetes is a relativelyimmature technology that's
changing obviously rapidly.
But you know, you compare it toother things that have been
around for 30 years.
It's relatively new.
(27:22):
And so policy was when thisstarted probably I don't know
when the actual startup was.
It's probably like four, three,four years ago.
Policy was brand new on thescene.
Opa had been around for a littlewhile but OPA or OPA, nobody
had been really using it, nobodywanted to write Rego.
And so they said you know, wekind of we want to get this
(27:45):
group together to talk about, togive advice to the community,
to put out information about thebest ways to implement and use
policy, why you should implementand use policy and things like
that.
So that's been really ourcharter.
I can't speak it from memory,unfortunately, but it's probably
(28:05):
not a good thing as a co-chair,but we spend a lot of time just
working with the community,putting out white papers,
talking to folks about theirpolicy implementations, how that
ties into governance, riskcompliance and really trying to
be tool agnostic and talk aboutthe how and the why of policy as
(28:27):
a whole.
William (28:28):
That's great.
Why so like?
Just for the audience that'snot as familiar with like policy
management with Kubernetes.
Why is policy management sodaggone important in Kubernetes
in the first place?
Andy (28:41):
I mean there's a lot of reasons, but I think the primary
one that I like to talk aboutbecause I think it hits home for
most engineers is at least mostops engineers is that
Kubernetes is not secure orconfigured perfectly by default.
I can get something running inKubernetes very quickly.
(29:04):
Something running in Kubernetesvery quickly, it might not
scale, it might not bin packwell, it might not be secure
right, because I can just throwsome stuff at Kubernetes and the
defaults are not any of thosethings.
They're not secure, they're notefficient, they're not
necessarily reliable.
So you really have to know whatyou're doing to get to that
point and what policy can do isturn your Kubernetes environment
(29:27):
into something that is secure,reliable and efficient by
default, because you'reenforcing all of those rules
about how you use Kubernetes upfront rather than trying to go
back and fix them afterwards.
So that's kind of the dream ofpolicy for me.
William (29:44):
Love it.
Any cool initiatives orprojects like in focus with the
working group or anything in thefuture that we get a sneak peek
of, or is it just kind of whatyou already outlined?
Andy (29:58):
I mean, all of our meetings are completely public,
so there's nothing hiding inthere that I can reveal.
I think some of the moreinteresting things we're working
on can reveal.
I think some of the moreinteresting things we're working
on.
I know Poonam Lamba from Ithink I said her name right.
I apologize if I didn't.
I don't know her last name, butfrom Google is working with the
(30:18):
rest of the group a white paperon what they're calling shift
down security.
So we have shift left security,but we're talking about
shifting it down in the stack.
So I think that's interesting.
I haven't had a whole lot oftime to review it, but it sounds
like a very interesting paper.
I'm currently working slowly onputting together a policy
(30:40):
survey to understand better howfolks are using policy right now
, because I think we've movedaway from the days where nobody
knows what policy is, but Ithink we're still in the time
where very few people areactually utilizing it to great
effect.
And then, on top of that, wehave a few members of the
working group who are alsomembers of other SIGs and TAGs
(31:03):
and things that are working veryhard on the compliance angle,
and so I know the folks atDefense Unicorns are doing a lot
of really cool stuff.
I think there's even a newchannel for Lula, which is a
tool for basically tying yourcompliance to your policy,
because policy enforcement isnot compliance, but it can be
(31:23):
used to feed compliance, and sothey're working on using some
actual like automated tools todo compliance and tie it to your
policy engine.
Eyvonne (31:34):
Just you know, your friendly PSA Policy enforcement
is not compliance, is notsecurity.
Yes, those are all verydifferent domains.
They do inform and intersectand overlap in some interesting
places, but they are distinctand it's important to know and
understand those distinctions.
Andy (31:54):
Absolutely.
Compliance can never.
I've been saying lately,compliance can never create good
security, but good security canmake compliance easier.
Eyvonne (32:05):
That's right.
That's a great way to say it,and you know you can be
compliant and follow the processand have a rule created that's
permit IP any, any, that wouldcomply with your policies
because it was approved by allthe right people and implemented
according to your standards.
William (32:26):
That is also secure secure I wonder how many times
that's been done I've.
I've sort of stood in the gapand kept it from happening a
time or two yeah, and, as andysaid, like these meetings are
public and also the, the cncfhas an amazing slack channel
that is anybody can join, as I'mcreeping in there all the time
(32:48):
looking at stuff.
It's great.
So if you want to know reallywhat's going on, not just with
this working group, but lots ofother work that the CNCF is
doing, join the Slack channel.
Add some of the channels inthere, the groups, and just kind
of peek around.
It's good stuff.
Andy (33:08):
Yeah, both the CNCF Slack and the Kubernetes Slack.
The organizational structure isa little bit confusing, but
tags are CNCF, sigs areKubernetes and both of them have
working groups Very confusing,but both Slacks are great.
Be careful how many channelsyou add, because it will create
(33:29):
a lot of noise.
William (33:31):
yeah, it does the time to wax philosophical?
I guess a little bit, but youknow, things are always changing
in tech.
We know that the only the onlything that's consistent is
change, and sometimes it's a bitoverwhelming.
Kubernetes is no stranger tothis, as it changes very rapidly
.
Is Kubernetes sort of going,you know, when you started
(33:54):
working in the Kubernetes spaceand you started learning, is it
sort of going in the directionthat you thought it would?
Do you think that, like, wheredo you see it headed from here
on out?
What do you see Kubernetesbecoming in the next five years?
Andy (34:10):
I get asked this question so often.
William (34:13):
It's like the unanswerable question.
Andy (34:15):
It is too.
It's impossible.
I know we do this thing everyyear, like our marketing folks
love putting together like whatare your predictions for the
next year?
And I always my boss at thetime is like I hate doing these.
It's like I feel like such acharlatan, just like spouting
off this, this knowledge that is, you know, we, we have no way,
(34:37):
but I do have thoughts, ofcourse.
So, like the first question youasked, which is is it going the
way that I thought it wouldfrom eight years ago?
And I think absolutely yes.
It's definitely seeing much moreadoption.
The cloud providers are allheavily involved with it.
(35:00):
A Kubernetes offering right,your DigitalOceans, your Linodes
, your Sivos built almostentirely on the idea of having a
Kubernetes offering all thesecloud providers.
So I think that's super cool.
The one thing I think isinteresting is that adoption has
been maybe slower than I hadanticipated, I think partly just
(35:23):
because of the hype.
Early on, we were getting thesereports that were saying, like
every major corporation is usingKubernetes and I said, well,
sure, but it's like five peoplewithin the whole organization
that are using Kubernetes andthe rest of it's all still
running on whatever they wererunning before.
That's slowly starting tochange, but it's very slow and
(35:44):
of course the adoption of anynewer technology is going to be
slow.
There's that to answer yourfirst question.
The second answer much harder,but I think we're starting to
see echoes of it already.
I think Kubernetes will becomesort of the underpinning of a
(36:05):
lot of various offerings.
So you're seeing more and moreofferings built on top of
Kubernetes, that sort ofabstract away some of the
complexities for you.
I've been talking to some folksat like North Flank and a
couple other places that arebuilding these like developer
platforms for deploying yourapps that are built entirely on
Kubernetes and then maybe havethe option to like take over the
(36:28):
full management of Kubernetes.
And then we're also seeingthings like Cloud Run, fargate
not necessarily Kubernetes underthe hood, but very
Kubernetes-like right Knativestuff like that, where
containers and some of theconcepts from Kubernetes are
(36:50):
still ubiquitous, and so I thinkthis way of running
infrastructure will remain intothe future, whether we actually
use raw Kubernetes in five, 10years, maybe not.
Maybe there will come alonganother ubiquitous abstraction
layer, or so you know, somefolks will continue to use just
(37:12):
pure kubernetes and a lot offolks will use these abstraction
layers, but I don't think it'sgoing anywhere totally agree
with that one.
William (37:19):
I think it's here to stay so much, especially startup
software these days.
I don't want to say it'sexclusively built on Kubernetes,
but I mean I would love, I mean, if there was some numbers out
there of startups, like in thepast two years, that are
building exclusively onKubernetes.
I bet that number is like veryhigh, Just because the
(37:40):
distributed nature, distributedscale of you know the outcome.
Andy (37:45):
It would be interesting.
I you know I went to.
I did a talk at Boulder StartupWeek about infrastructure for
startups, and I think a lot ofthe sentiment in the room was
people were scared to useKubernetes because of the
complexity, because of the size,and I think the general
recommendation that I've seenfrom experts is don't use
(38:08):
Kubernetes.
Use something either built ontop of Kubernetes or that allows
you to move to Kubernetes whenyou need that complexity and
scale.
And so I think it would beinteresting to see that survey,
because I wonder how many peoplewould say, yes, I'm running on
Kubernetes or no, I builtentirely on Fargate or Cloud Run
or you know some other.
You know container as a servicetype thing, but still fully
(38:32):
containerized, using some of theconcepts that would allow me to
utilize Kubernetes down theroad.
Eyvonne (38:37):
When it makes sense.
If you're a smart startup andyour goal is, like time to value
, like you've got to get thething built and out there.
Scale is a concern, but it's asecondary concern, absolutely.
And so if you can deploy in aserverless platform, get
something up and running, doproofs of concepts, be able to
(39:00):
start onboarding customers,that's a huge win.
But there are scale andflexibility limitations with
serverless platforms thatanybody's going to hit once they
reach a certain size.
So it makes sense to deployinitially in a more managed
environment and graduate up asyour needs dictateated.
(39:23):
But I think ultimately, atscale, anybody who's running a
significantly complex enough,demanding enough application is
going to end up there eventually.
William (39:37):
Absolutely yeah.
What I'm hearing is basicallylike the right solution for the
right problem.
So don't use a dump truck whenyou could use a pickup truck or
when you're building yourstartup, just like, okay, don't
just jump immediately into aservice mesh until you have
service mesh problems out therethat you need to solve in
(40:00):
general, because that adds evenmore complexity to the scenario.
Andy (40:07):
I'm writing a talk right now called Kubernetes.
As a Big Hammer Is Everything aNail, and it will be exactly
about this topic.
William (40:14):
I love that I have something to add to that.
We'll be on the lookout.
Is it going to be recorded?
Andy (40:20):
It'll be at Kubernetes Community Day in DC, so I'm sure
it will be recorded.
William (40:28):
Awesome, right on.
I guess we've hit sort of theunless you have anything else.
Do you have anything else youwant to throw in there?
Yvonne, I'm good.
Awesome.
This has been an excellentconversation.
I love conversations like this,just talking about the real you
know the meat on the bone asfar as the technology and
digging in and just kind offiguring out what's out there,
(40:48):
what's happening, and especiallybeing able to talk to someone
that's an expert like yourself.
So this has been an awesome,awesome conversation.
We really appreciate your time.
Where can the audience find youat?
Do you have?
Andy (41:01):
any social media, Mostly on LinkedIn.
These days I tend to avoid someof the others.
I think I have a Mastodonaccount that I check on occasion
.
I'm generally SudermanJRSudermanJr just about everywhere
, or Andy Suderman, and I'mdefinitely active in the
Kubernetes, the CNCF Slack.
(41:22):
We also have a community Slackfor our open source at Fairwinds
.
You can find it on any of ouropen source repos.
Slack is definitely the bestway to get a hold of me.
Eyvonne (41:33):
I'm in it 24-7.
Andy (41:34):
it feels like Awesome.
William (41:38):
I'll put all those in the show notes.
Thank you so much, thank you.
Andy (41:42):
It's been a pleasure.
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com