Mastering Infrastructure as Code: Lessons from Matt Gowie's Consultancy Experience - DevOps 235 - Adventures in DevOps

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:01):
All right, what's going on, Let's do another episode of
Adventures in DevOps Born. How's it going.

Speaker 2 (00:07):
Uh, it's a pretty good and Uh. I actually have
a fact we're all prepared this week. Uh, and that
is we've gotten upgrade to our website, adventures in DevOps
dot com. So if you're listening to the podcast via
a different source and you probably don't never see it,
you know, here's an opportunity. Uh, you know, don't pay
too much attention to it. But I think it's much
better than where we were at before.

Speaker 1 (00:29):
Awesome, that's cool.

Speaker 3 (00:30):
And do you make the new website. I had a
part of the.

Speaker 1 (00:34):
Dev chat, had a hand in it. Hi, Jillian, welcome back.

Speaker 3 (00:42):
Hello, thank you for having me back.

Speaker 1 (00:44):
Yeah, how's it been going.

Speaker 3 (00:46):
It's good. It's good.

Speaker 4 (00:47):
We had like the snow apocalypse here last week, but
this week we're good. We're back to just snow right on.

Speaker 3 (00:55):
Yeah.

Speaker 1 (00:56):
This is my first year living up north and I've
been so happy with the fact that I bought a snowblower.
That's been so much fun. But I don't think.

Speaker 3 (01:07):
About getting one for the driveway. But I haven't.

Speaker 4 (01:09):
I haven't quite done it yet because there's guys that
come to plow so.

Speaker 3 (01:12):
I don't know, I don't know.

Speaker 1 (01:15):
Yeah, highly recommended. It's just a great feeling, just chugging
along behind it, blasting the snow. So, Matt Gowie, welcome.
Happy to have you on a show.

Speaker 5 (01:27):
Yeah, thanks for having me. Nice to meet y'all be here.

Speaker 1 (01:31):
Yeah. So, Matt, you are you own master Point consulting company, right,
so give us a little bit of the background on that.

Speaker 6 (01:43):
Yeah, I've run Masterpoints since the end of twenty sixteen.
For a long time it was me as a solo consultant,
and then around very beginning of twenty twenty two, I
kind of started to build a team. So we're a
boutique consulting shop. We're entirely focused on infrastructure as code nowadays.
What that means is particularly terror form and open TOFU

(02:06):
because those are the market winners in infrastructure as code today.
And then we do a little bit of work with
Polumi and yeah, we help our clients automate, migrate, implement
best practices, and really help them build the right workflows
on infrastructure so that they don't have that as a
bottleneck and they can help their application engineers move fast.

Speaker 2 (02:29):
Yeah, I mean that's like almost a decade, right, it's close.

Speaker 1 (02:33):
You must have seen some.

Speaker 2 (02:34):
Wild changes that have occurred in the space over that time.

Speaker 5 (02:38):
Yeah, it's been fun.

Speaker 6 (02:41):
I got into you know, I always had a background
in DevOps. I started in startups where you just a
you wear fifteen different hats and you do whatever you
can to help the team move forward. But I didn't
get into infrastructure as code specifically. I was really big
an answerable until like twenty seventeen and twenty eighteen maybe,

(03:04):
and then I around zero goot eleven is when I
got into terrorforms specifically, and I was like light bulb,
got got to get into this more.

Speaker 5 (03:13):
This is the way to do things for sure.

Speaker 1 (03:16):
So I know that, like consultant is like one of
the common career paths for people in our industry, and
the big challenge there is always how do you find clients?
You know? And is this a consulting arrangement where I'm
technically a full time employee but I don't get any
of the benefits. So how how does it look from

(03:41):
your perspective, like what do you look for in a client?
How do you approach this whole problem?

Speaker 6 (03:48):
Yeah, I think being solo is very different than running
an agency.

Speaker 5 (03:53):
But I think one truism, one you know.

Speaker 6 (03:59):
Fact of life that I've found is that you know,
people do business with who they know and who they trust.

Speaker 5 (04:06):
So referrals are always king within consulting.

Speaker 6 (04:10):
You know, you'll get a referral from a previous client
or a colleague who knows you well and can speak
to Hey, Matt and his team do really good work,
and that client that referral, that prospect is ninety percent
more likely to you know, sign on the dotted line
and trust that, hey, we're going to do a great
job for them, then somebody who finds us through SEO

(04:34):
or I've never done outbound email marketing because I don't
believe in that as a viable business. But you know,
that type of stuff is not the approach that I take.
I take a very like human approach to it, where
I go out and network a bunch.

Speaker 5 (04:48):
I try and.

Speaker 6 (04:50):
Be a good person within my community, and that is
the means by which I like really work to try
and bring in business for my coany. For some people, Hey,
if you're just getting into consulting, maybe that sounds awful
to you. Maybe you just want to be heads down
writing code. You can go and join a consulting shop.
We're hiring every once in a while, so reach out.

Speaker 1 (05:12):
For sure. I think that's one of the big surprises
for people who take on the consulting route is understanding
that you are leaving your one job to take on
three jobs, because now you have Job number one is
a consultant where you're generating the billable hours. But then

(05:33):
job number two is you're an accountant because now you
have to do all the taxes and the legal paperwork
and the bookkeeping. And job number three is you are
in marketing because you always have to be recruiting and
finding your next job once this one wraps up.

Speaker 6 (05:49):
Yeah.

Speaker 2 (05:49):
I mean, I'm with Matt that I don't believe in
the code calling. I don't think those emails work, and
yet I still see I probably get five to six
a day from random people on LinkedIn saying, hey, you know,
can we sell you our services whatever it is. They
don't even know what my company is doing, and yet
they're already like, oh, we know, we can help you.

Speaker 5 (06:12):
It's just constant. I think if you have any C suite.

Speaker 6 (06:16):
Level title in your name, I think we all get
a insane amount of of both spam email and you know,
the the LinkedIn messages and it's just it's just piles up.

Speaker 5 (06:32):
It's kind of insane.

Speaker 6 (06:33):
And I still don't understand how people do it because
I can't think of a worse Like I, you know,
get ten thousand emails here, you know, maybe that's high,
but like I've never responded to one, So.

Speaker 2 (06:47):
Yeah, you're you're missing out your opportunity here. I respond
to every single one of them with a vengeance, like
like I'm curious, like I really want to know, like
is this person, you know, do they have a competency,
you know, are they looking for something special? I really
care about my like LinkedIn network, so I'm always looking
to see, you know, making a connection with an initial
person is it worthwhile for them and for me? And

(07:10):
so like I'll try to dive into that. And some
of them, you know, actually do turn it around and
you know, are able to talk about the subject or
the topic. But a lot of times they're just the
partner portal expert or marketing manager or account manager and
they have no idea of the thing that they're even
selling in the first place. And I'm like why, like

(07:31):
why did you think this was going to go like like,
just walk me through your process here. Step one, connect
with me on LinkedIn, h step two dot do dot
step three profit like like, I don't know how as
it is.

Speaker 5 (07:43):
That's pretty good.

Speaker 4 (07:44):
I would like to invite you to have a conversation
with like some teenagers. It goes, it goes exactly the same,
exactly the same.

Speaker 1 (07:55):
So do you find that a lot of your you know,
you talked about your your network and everything's coming to
you or your your most likely sources through referrals. Is
that within your community or is that online? Does that
include digital connections?

Speaker 5 (08:12):
Yeah, so yes it does.

Speaker 6 (08:15):
I mean, of course I think that you know, we're
myself and a number of members on my team are
big contributors maintainers of our own open source as well
as like one of the larger open source infrastructure is
code terrorform libraries. Uh, so we have a bunch of
connections in that world. I have CTO online communities I'm

(08:40):
involved in. I have a bunch of like volunteer with
other extrajournal organizations that I'm involved in, and I think
we try and pretty be pretty involved in like different slacks,
and I think that things come from those all the time.
So Yeah, it's not just you have to be going
out and shaking hands and kissing babies. It's also some

(09:01):
level of online Uh, you know, presence is really important.
I personally am posting on LinkedIn three times a week.
I have a lot of really good, uh colleagues, people
that are I like a lot and highly respect their opinion.
They are, you know, people I've met through LinkedIn and
just like posting content. So I think that there's a

(09:24):
lot of different avenues how that networking happens, but.

Speaker 5 (09:27):
It all, you know, kind of leads to a similar
uh and result.

Speaker 2 (09:32):
Let's let's plug your your terror form open source library.

Speaker 1 (09:36):
Like what is that?

Speaker 6 (09:38):
So we are maintainers of cloud posse. Uh have you
folks ever heard of them?

Speaker 4 (09:43):
I have, So you're that person I steal your code
like all the time.

Speaker 5 (09:47):
See, they're the best.

Speaker 4 (09:49):
When I originally got it on the show before I
didn't know this, this is like the best super event
for a Tuesday morning.

Speaker 6 (09:55):
Ever.

Speaker 5 (09:56):
Yeah, they're they're not my company.

Speaker 6 (09:58):
Eric's a good and I really respect the hell out
of what that guy's doing. But when I initially got
into Infrastructure's code, they were the community that I found personally,
and I was just like, Hey, these folks are doing
this sensibly, like this is one infrastructure is not Infrastructure
is a commodity, right, Like, we're all shipping the same

(10:20):
load balancer, we're shipping the same certificate, we're shipping the
same postgress instance. Why do we need to re Yeah.

Speaker 2 (10:31):
I have no idea what.

Speaker 3 (10:32):
You're talking about.

Speaker 6 (10:37):
Anyway, they they do it right. And when I found
their modules, I was really excited. I was contributing a
bunch and then as part of that, I ended up
on their like maintainer teams. So we're helping them get
prs tested and merged and complying with all the best
practices that those folks have set out. And I have
multiple my team have now joined that that we're organization,

(11:00):
And yeah, that's the I really believe in open source
infrastructure's code. I think there's like interesting discussions there. Some
people do not Some people think that we should be
copy and pasting and writing little snowflakes all over the place.
But that's, you know the difference in opinion that happens.

Speaker 2 (11:19):
I actually want to get into that because one of
my questions that I had thought of that I really
wanted to ask you is what is the real impact
of And I know someone's going to be kicking me
for bringing this topic up at only whatever like the
ten minute point, and that's realistically any sort of LLM
integration here, because like I really do feel like that, Okay,
well laugh it out that there's a lot like this

(11:43):
is exactly the sort of thing that I think LMS
are very good at generating. But at the same time
it's the worst place to have those little mistakes that
will cost you, you know, huge production incidents.

Speaker 6 (11:56):
Yeah, so I will say that we don't typically a
ton of infrastructure as code. You know, we'll do it
for new resources that we haven't seen maybe sometimes, but
for the most part, you know, I'm I have a
very serious talk with my team that like, hey, you
need to understand how all these things come together. This
can't be l generate, you know, get commit and ship it,

(12:19):
because yeah, there's a lot of nuances in that one
flag that might be a security loophole that might cause
you to over provision something along those lines.

Speaker 3 (12:30):
And.

Speaker 6 (12:32):
I don't know, we see it a lot where people
are generating a ton of code and then they don't
really have a good handle on Hey, what all this
is doing.

Speaker 5 (12:41):
And I think that's that's usually where i'm.

Speaker 6 (12:43):
You know, a proponent of open source, because hey, you
could probably get that same use case that you are
looking for.

Speaker 5 (12:50):
You know, you're trying.

Speaker 6 (12:51):
To deploy a bunch of data dog monitors, and instead
of generating code around them and creating a you know, yeah,
get the job done for today, but for day two operations,
is it better to have a small open source module
that provides that same monitor, that provides a spec, provides

(13:11):
you something that you can upgrade in the future, provide
something that's already tested and already secure. There's a lot
of benefits to at least doing.

Speaker 5 (13:19):
The search beforehand.

Speaker 6 (13:21):
To go for open source, I think then just saying hey,
we can generate the hell out of this code.

Speaker 2 (13:27):
Yeah, are the open source modules showing up in the
LM results or is it pure like underlying HCl.

Speaker 6 (13:35):
Not that I've seen, And you know, we haven't done
a good test of Hey, let's use an LM that
we've given like a lot of really good instructions to
read through the three hundred open source modules that we
typically use. But I think that that's a good question.
I haven't seen anybody who's like using a trained model

(13:56):
to say, hey, these are our approved module libraries.

Speaker 2 (13:59):
I mean, it wasn't even just that, but because I
remember there was like a pretty huge scandal with Pulumi
where they tried to do something similar to this and
the LM that they were featuring on their website just
like could not provide more wrong information. Uh So that's
totally a data point there.

Speaker 6 (14:16):
Yeah, that's interesting, always trying to do some something new
and I applaud them for innovation.

Speaker 5 (14:26):
You're gonna get it wrong sometimes, right, you gotta try.

Speaker 1 (14:28):
It's good for sure, fell Fast. So whenever you start
working with a new client, do you have like a
particular vertical or size of company that you work really
well with and have isolated on that intentionally?

Speaker 6 (14:45):
Yeah, we still love startups, and you know, it's sad
that the VC market is.

Speaker 5 (14:50):
No good, but I still still love startups.

Speaker 6 (14:54):
We just have moved further up funnel in terms of
we usually talked to the series C and above, you know,
the folks who are well established and they're trying to scale,
because that's really where a lot of our expertise comes
in Handy is hey, you have an organization that has
grown organically you've been trying to grow fast and you've

(15:14):
made some mistakes and we can kind of come in
and dig you out of your hole. So yeah, that
realm is where we really like to engage is the
mid market to like late stage series of funding startups.

Speaker 5 (15:30):
We have an anchor client who's a large enterprise.

Speaker 6 (15:32):
They're you know, a Fortune five hundred car manufacturer. But
really I love those those later stage startup clients because
they have the most fun problems. They usually have great
engineers that are just like, we need to be pointed
in the right direction, and hey, we can do that,
so it's fun.

Speaker 2 (15:52):
Are you seeing the same holes at each of your
clients or is it like everyone have their own special
set of problems.

Speaker 6 (16:00):
I think that there's a you know, uh, there's always
some combination of similar problems, you know. There there's a
really common problem that we've written a bunch about on
our blog, and we we have another blog post coming
out really soon.

Speaker 5 (16:16):
That's you know, the terroorlyth problem. Have y'all ever heard
of that term? This one's new for me. Okay.

Speaker 6 (16:21):
It's basically, when you build a infrastructure as code root
module that's too big, so it's storing too many resources
in state and then it becomes slow topply, it has
blast radius issues, and you you know, can't really do
role based access control in your team because hey, you're
managing your network alongside your database, alongside your application cluster.

(16:44):
So we've seen that a ton and you know that
comes in all different shapes and sizes. It's very unique
to the organization. We we know how to break those
up really well. So that's that's an easy one. We'll
have a post on that on the blog very soon
that talks about how teams can break it up themselves
give it away for free. You know. The other thing

(17:06):
I think is there's still a lot of organizations who
are applying locally and they don't they haven't figured out automation.
And then on the opposite side of the coin, there's
a bunch of organizations who have built their own automation
around infrastructure as code and they've found out the hard
way that that's not a good route, and they, you know,
are really you know, continuing to toil in the details

(17:27):
of like, hey, how do we you know, make this
scalable for our organization.

Speaker 5 (17:32):
And it's it's kind of a bad rabbit hole.

Speaker 6 (17:34):
We always suggest, hey, either go again open source or
pick one of the really awesome vendors in the space.
But yeah, those are those are some of the like
high level problems. There's a lot of like nuanced, like
code level details that I think we see a lot
that we have strong opinions on. But yeah, those are
the ten thousand foot ones.

Speaker 2 (17:54):
See I saw will laughing here sort of like the
villain at the end of a movie that is having
some sort of s like logical breakdown, and I just
I have to I have to wonder what was going
through his head at that moment.

Speaker 1 (18:07):
Just reliving some past trauma. I think.

Speaker 4 (18:12):
Those Yeah, this whole show is just like PTSD induced.

Speaker 1 (18:17):
You know, no, I can totally relate to that, especially
in the stage of company that you're you're focused on,
you know, the late late stage series startup, because early
on I've been guilty of this many times. You know,
early in the startup stage, you think about your past
experiences and you're like, I'm not going to do that again.

(18:39):
We're going with I see from the very beginning, and
so you start building all of these processes and controls,
and the flaw with that is you don't actually have
a product for your company. Yet, you know, because for
every startup there's always the there's the product that you launched,
and then there's the product that you're customers actually wanted.

(19:02):
And never in the history of startups have those two
been the same thing. So you make all of these
process and design and architectural decisions around this product that
now no longer exists. And it takes someone like yourself, Matt,
from an outside perspective to come in and say, you know,

(19:24):
you're holding on to these little nuggets here hoping that
their gold, and they're not. Just let them go.

Speaker 5 (19:34):
Sound advice.

Speaker 2 (19:35):
I think it's a huge problem there though, which is
that those processes are sort of coupled to the culture
of your organization, so like you almost need to like
actually start a new company to get rid like actually
eliminate all of that tech deat In some ways, you've
built it up, You've built up expectations on how that works.
Like maybe you are career ladder is somehow hooked to

(19:58):
the types of task and responsible. I mean, no one's
got it that bad, but there's a lot that needs
to be thought about other than just like you can't
just click a couple of buttons and delete their code.

Speaker 1 (20:08):
I call it the polished turd syndrome, because you start
with this turd and you just keep polishing it and
polishing it, and then eventually it starts to get some
shine to it, and so now you're really emotionally invested
in making this turd nice and shiny, and you need
someone to come in and say, dude, it's it's just
a turd.

Speaker 3 (20:27):
It go.

Speaker 2 (20:27):
They have to want to listen, though, And I think
that's part of the problem is that often you see
like the maybe operations division or sales or marketing, or like,
we're actually making a lot of money from that thing.
It's not like as much money as we should be making,
but we're making, you know, a million ten million a year,
and we just need to keep the lights on, and
they don't understand how much toil and complexity is actually

(20:51):
involved to just keeping the lights on for that. And
so I actually think I've seen this way more often
than the other side, where an engineering team doesn't want
to own it, they don't want to do anything with it,
except someone keeps on telling them, oh, yeah, you know,
you have to keep it working, but you're not allowed
to spend any time doing that.

Speaker 5 (21:06):
I've seen that, and I think that, you know, a
part of what you folks are talking about.

Speaker 6 (21:11):
I mean, yeah, at the level, at the stage of
company that we usually engage with, there's always some level
of like some cost fallacy, right, there's some group within
the organization who's wrapped up in the idea that, hey,
we've invested so much in this tooling, why why would
we get rid of it?

Speaker 5 (21:30):
What do you mean? And yeah, it does.

Speaker 6 (21:32):
It does sometimes take a third party like my company,
that comes in and says, hey, here's the reasoning. You know,
we're the outside eyes. We're giving you this this perspective
because honestly, your workflow is not great. You're not, you know,
doing yourself favors here by continuing to polish this, so

(21:53):
to speak. So, yeah, I agree with what you folks
are getting at. It is it's always comes down to
people problems in one way or another. And I think
that luckily I've been consulting long enough where I like
can navigate those well.

Speaker 5 (22:09):
Uh, and yeah, it's fun.

Speaker 2 (22:11):
Who who are the decision makers that end up signing
off on needing to pull in a third party for you?

Speaker 6 (22:17):
It usually comes down to the director level or above,
so it's usually director of Engineering, Director of DevOps infrastructure,
the VP, the CTO, it's somewhere along that chain. Sometimes
it starts with an engineer, where the engineer will be like.

Speaker 5 (22:34):
Hey, we have this horrible problem.

Speaker 6 (22:35):
We're not dealing with it like we gotta we gotta
do something, and that it can go bottom up.

Speaker 5 (22:40):
But it's not typical.

Speaker 6 (22:42):
It's more a yeah, some somebody in engineering leadership realizes
they got to make a change, uh, and they're reaching
out to to kind of talk through, Hey, what does
this look like? How can we, you know, actually benefit So.

Speaker 1 (22:57):
Yeah, how often I think this is like a non
technical component of the job. I'm interested in your your
take on it. How often is it that the problem
that you're really trying to solve is the cost of scalability,
where like, the company's doing well and they're scaling and
they're growing, but the infrastructure costs are are growing quicker

(23:22):
than revenue.

Speaker 6 (23:25):
So I don't think it's the actual cloud costs that
it's not. That's not actually typically a problem. It's the
maintenance cost for engineers and the amount of time that
they're spending to feed the system that they've built in
one way or another, and that's the thing that's.

Speaker 5 (23:44):
Slowing them down.

Speaker 6 (23:45):
Basically, they've made some bad decisions along the line that
is causing their organization to overall have them be a bottleneck.
Whether that's that's their infrastructure workflow as a whole. Maybe
it's because they you know, need to hire an entire
security team to make sure that things are not you know,
going to cause a you know, critical CBD and you

(24:07):
know day day five of new application launch or something.

Speaker 5 (24:12):
Those are the problems we more typically see.

Speaker 6 (24:14):
So it's less actual, hey, our cloud costs are growing astronomically,
and more hey, we're we're spending too much time here,
Like why is this? Why is my team telling you
need three more engineers? That's that seems like madness when
you know we're not we're not really doing doing all

(24:35):
that much crazy stuff here. We should be more, we
should be faster, we should be more streamlined.

Speaker 5 (24:39):
Yeah.

Speaker 2 (24:40):
Maybe I actually sort of want to flip this question
around because I really like it and I think there's
a different perspective here that really gets me thinking. Is
you're not you're not solving technical problems, are you? Like
I I get the sense that you know you're coming in.
It's not that the organization is missing two more engineers
to Oh, if we only had these two people who
understood terrior form better, we would solve all our problems.

(25:02):
I mean, is that really it? I got the sense
of probably you could do that. And they may be
missing technology, but they're probably missing something like a better
uh strategy for on the personal level, like there how
they're approaching problems, or how they're prioritizing work or something
like that. And maybe before I ask the next question,
I'll give you a moment.

Speaker 5 (25:22):
Yeah.

Speaker 6 (25:23):
I think it's interesting because it's multifaceted, right. It's sometimes
they've I like the this is my new joke where uh,
you know what, why do Masterpoint clients reach out to us? Oh,
because they applied themselves into a corner. I don't know
if it's a good joke you folks on second person,

(25:44):
I'm trying trying it out on but yeah, they back themselves.

Speaker 5 (25:48):
Into a corner.

Speaker 6 (25:49):
They they you know, they go about things in from
an infrastructure as code perspective. We're we're still young, right,
it's still a really early day.

Speaker 5 (26:00):
We might be ten years old.

Speaker 6 (26:01):
But Hashikorp when they came out with terrorform and terrorform
I see as like that's what mainstreamed infrastructure is code.
They came out with that in twenty fourteen, twenty fifteen,
and it was really early, and it was just it
was way more configured than it was code. And it's
slowly grown and it's gotten better. And the reality is

(26:25):
they didn't put out many best practices.

Speaker 5 (26:27):
So you have a ton of organizations, a.

Speaker 6 (26:30):
Ton who have just gone and built whatever the hell
they wanted and it has turned into a mess and
that can be a such a operational detriment to their
organization that we're coming in and we're providing some of
those best practices on how to like do things streamlined
that it is a technical problem and they do need expertise.

Speaker 5 (26:52):
The other side of it.

Speaker 6 (26:54):
Is kind of what we were talking about before, you know,
the cultural you know, hey, some cost bowel see trying
to guard like trying to direct them into direction that's
going to be you know, operationally efficient.

Speaker 5 (27:10):
There's a bunch of other things too, but like that's.

Speaker 6 (27:13):
What it usually comes down to, is like those two
sides of the coin. Yeah, does that make sense to
answer your question?

Speaker 5 (27:20):
Yeah, Yeah, for sure.

Speaker 2 (27:21):
I mean, there's some fundamental problem here, and I really
like the perspective of they didn't think enough about what
the future was going to look like for them, and
they may not have the right people on the team
to I mean, you don't fix a problem with the
same initial conditions still in play. You need to change
something to have a different outcome. I feel like you

(27:43):
keep on bringing up potential perspectives that master Point have
that may be highly controversial, and I want to get
those on the record in a way, and like one
of them is like one terraform repository for your whole organization,
or do you like merge it with the individual application code,

(28:03):
so like code and infrastructure as code next to each
other in the same repo or one centralized location.

Speaker 5 (28:10):
That's one we don't have as much of a strong
opinion on.

Speaker 6 (28:13):
I will say that we do have a strong opinion
on there should be an infrastructure mono repo for your organization.
I think that that's just like there's beyond just environmental infrastructure.
There's a lot of infrastructure that is global. Right if
you're managing GitHub, if you're managing if you're doing observability

(28:34):
as code, and you're managing monitors and SLOs and dashboards
and things like that within a tool like data Dog
or Honeycomb. You want those to be in a centralized location.
They shouldn't be scattered into fifteen different repositories. But even
when it comes down to environmental infrastructure, so hey we

(28:55):
have a VPC, we've got a load balancer, we've got
a Postcrest database.

Speaker 5 (29:03):
If a client.

Speaker 6 (29:03):
Wants to do a hey they want their let's say
they're shipping on Kubernetes, they want their Kubernetes infrastructure to
live alongside the application that it is, you know, the
repository that it's managed in. That's totally fine, but we
would say, hey, put your put the environmental infrastructure that
actually supports that application in the infrastructure mono repo and

(29:27):
really have it be a single point over off in
its own land so that you can like scalably duplicate
that repo so as you add more services, as you
add more components to the overall system, then then there's
it makes sense there, and then you don't need to
have those engine application engineers doing pullar quest to some
infrastructure mount repo that they may or may not have

(29:50):
any clue about what's going on.

Speaker 2 (29:52):
So he I mean, it's it's always great when the
answer is you know, it depends, right, you know that's.

Speaker 5 (29:56):
It always depends.

Speaker 6 (29:57):
Yeah, yeah, particularly as a consult always.

Speaker 1 (30:03):
Jillie, what was that?

Speaker 3 (30:05):
That is everybody's favorite answer? Is it depends?

Speaker 5 (30:07):
Yeah.

Speaker 2 (30:08):
I mean there are some things that you know, I
just sort of want the canonical, like it's always going
to be this, and you know one of them is
a TF workspaces.

Speaker 1 (30:17):
What is that?

Speaker 5 (30:18):
Is that?

Speaker 6 (30:18):
Yes or no?

Speaker 3 (30:19):
I don't like that spaces I like I like directories.

Speaker 4 (30:22):
I want to be able to run Tree on a
directory like you know, kick it old school here workspaces.
Nobody ever remembers to change the workspaces. Know, everybody else
can have an opinion.

Speaker 5 (30:32):
So that is this is it?

Speaker 6 (30:35):
This is like the you've you've now nailed the one
which is this is the biggest divide within infrastructure as
code is. We have people who look at infrastructure as
this is configured that should be copy and pasted, and
we have people that look at infrastructure as this is
more like code that should be you know, there should
be one instance of this thing and then we should

(30:56):
deploy that instance. Many times we're of the camp I
disagree with Jillian, but I, you know, do respect that. Hey,
there's a thousand different ways to do this, and people
have their own way.

Speaker 3 (31:08):
It's okay. I like it when we can find on
the show, it makes it.

Speaker 6 (31:11):
Yeah, yeah, I'm glad that even immediately came in. But yeah,
we used to have workspaces. A lot of people rag
on workspaces. I think that you can do them wrong.
They are kind of a very light abstraction layer.

Speaker 3 (31:28):
There's a really cool that tends to be my problem.

Speaker 6 (31:33):
That is a thing that'll plenty of people do is
forget that they exist. There is a really great issue
in the opa tofu GitHub organization right now where Martin Atkins,
one of the biggest original implementers UH and biggest contributors
to terrorform core for for many years now.

Speaker 5 (31:54):
He is now switched over to the open tofu team.

Speaker 6 (31:56):
And one of the things that he did within his
first like month or so was put up an ish
you about deprecating workspaces, and now that issue has Yeah,
Gillian's laughing.

Speaker 4 (32:05):
But like, well, I'm not like opposed to workspaces in theory.
It's just that every time I've tried to use them
and practice me or somebody else forgets they exist and
like and then that's a problem.

Speaker 2 (32:16):
It could be worse though, they could be having separate
GET branches for each environment.

Speaker 4 (32:23):
Okay, that's totally reasonable.

Speaker 3 (32:28):
What are you talking about? No, I'm sorry, I know,
I know that's not reasonable.

Speaker 2 (32:32):
There's only one main line in a Git repository, and
I will die on this hill.

Speaker 5 (32:37):
I will die on that hill as well.

Speaker 6 (32:39):
Particularly, an infrastructure is code, Particularly an infrastructure is code
because you're you're not dealing with twelve factor apps in
infrastructure is code, right, you don't have a separation of
the configure that drives that infrastructure is code like you
do with a application that could be in a gifflow model.
So you're then if you want to make a change
to a production configt sating, you have to do it

(33:02):
in the proud branch. And that might mean that you're
you just get into branch nonsense. And I just posted
this recently of like my favorite, one of my favorite
sayings is you know, more branches, more problems.

Speaker 5 (33:15):
I think it's way worse, sick, I think.

Speaker 6 (33:19):
It's way worse in infrastructure is code world. If you
are doing a branch based workflow.

Speaker 2 (33:24):
I mean just doesn't make any sense from against dowdpoint,
because these are become separate trees. And if these are
separate trees, why are they even in the same repository.
You're not gonna really do anything effective. I mean someone
may say, well, you can cherry pick some diffs from
one to another one, but I mean, realistically.

Speaker 5 (33:40):
That's a nightmare. That's yes.

Speaker 1 (33:43):
Is that your billing model, Matt, Your your hourly rate
is based on the number of branches they have in
their repo?

Speaker 6 (33:51):
Oh, I hope we don't have a client that's a
done of branch based workflow.

Speaker 5 (33:55):
The amount of work.

Speaker 6 (33:56):
Yeah, maybe our hourly rate doubles, Yeah, would be good.

Speaker 5 (34:01):
Maybe I haven't had to do it yet.

Speaker 6 (34:02):
But well you've only heard about those those clients or
heard about those setups.

Speaker 5 (34:09):
Yeah, not yet.

Speaker 2 (34:11):
So another one is you mentioned early on in the
episode public versus Private modules, And at least from my understanding,
I think this is one of those areas where if
your core competency isn't building open to fool HCl models,
you're probably going to get it wrong. And I believe

(34:31):
it's a huge challenge to undo that mistake and using
the either raw resources or what's come before you out
there in the world is way more valuable than what
you'll ever be able to do. But you know, I
want to hear I want to hear the experts perspective
on this.

Speaker 6 (34:48):
Yeah, we view it as, Hey, these the open source world.
As long as you're correctly evaluating your the open source
modules that you are using your you're getting one a community,
you're getting tested code, you're getting best practices in terms
of naming, tagging, and potentially security or not potentially, but

(35:12):
you know, more likely security. And then you're getting something
that when you go to, let's say, add a new feature,
you can always check if that module has been updated
recently by somebody else's doing that same exact feature.

Speaker 5 (35:29):
Maybe they're adding IPv six support, don't.

Speaker 6 (35:33):
We don't support IPv six for all of our clients, right,
I know that some of the modules that we use do,
And if we had built those modules ourselves in house
for our clients, then they would have to go in
and find all.

Speaker 5 (35:45):
The places to add IPv six support.

Speaker 1 (35:47):
Right.

Speaker 6 (35:47):
So what we typically tell clients is, hey, if you're
if you want to encode conventions for your larger organization,
you want to make sure that Hey, the numberumber of
variables and the you know, the best practices that you
see or fit wrap open source with your own small

(36:07):
layer on top of that child module. You can do
a smaller child module that just consumes another child module,
and that bottom layer child module can be a open
source and you can really I think, still get a
lot of the benefits and create a lot of the
flexibility that you want for your organization.

Speaker 5 (36:26):
So that's usually what our recommendation is.

Speaker 1 (36:30):
So what's your background before DevOps, Matt? Were you in
infrastructure or software development?

Speaker 6 (36:38):
Software development? I? Yeah, did CS in university. Luckily went
to a co op school, so I started working at
startups when I was still in school.

Speaker 5 (36:51):
I did two stints at two awesome startups.

Speaker 6 (36:54):
They taught me a thousand things not to do, which
was great, and one of the those startups hired me
before I even left, and I moved to Philadelphia and
was doing full stack. You know. I started mobile development,
did a bunch of back end rails work, did a
bunch of front end stuff. I was really big an
amber JS back in the day, and then got into

(37:17):
infrastructure and AWS specifically during that second startup experience.

Speaker 1 (37:23):
Right on cool The reason I asked that is because
of your approach to this. Like throughout my career, I've
seen like two philosophies in doing this, and I think
they they sort of resonate with what you were just
talking about. Like you have people who have written code
in the past and they understand, you know, like abstraction
and rappers and things like that. Then you have like

(37:45):
your your old school knuckle dragger it guys who would
you know, drag the rack in and if you needed
more network port, you would drag another network switch in
and you would do it in IRAQ and you would
crawl under the floor to wire it up and stuff.
And I think depending on what your early part of
your career looks like greatly influences how you solve ic problems.

Speaker 5 (38:10):
Yeah. Yeah, I strongly agree.

Speaker 6 (38:14):
And I think it's one of the things that I
get confused about within the infrastructure as code space.

Speaker 5 (38:21):
Why are we all doing it so differently? And I
think it's an interesting.

Speaker 6 (38:24):
Space because we do have the CIS admin folks who
were just awesome at Linux and they you know, could
one line bash themselves out of.

Speaker 5 (38:34):
Any hole that they were in. And you know, we.

Speaker 6 (38:39):
Have people that are coming from application engineering that are
you know, coming from Hey, I'm going to pick up
modern practices and I learned Python, but now I'm switching
over into into writing code or writing infrastructure because that's
what the organization needed today.

Speaker 5 (38:54):
And I think that it's this weird.

Speaker 6 (38:56):
It's not a traditional software path, so you know, you
have all these different ways of thinking about it, but
when it comes down to it, a lot of the
time I asked the question, why do we not treat
infrastructure as code?

Speaker 1 (39:11):
As code?

Speaker 5 (39:12):
Like that?

Speaker 6 (39:13):
That is like a core principle, a core like issue
that I have with a lot of people's setups, and
it seems.

Speaker 1 (39:23):
Obvious when you phrase it that way.

Speaker 5 (39:25):
Yeah.

Speaker 2 (39:26):
Well, I think that there was a fight in one
of the online communities I'm in whether or not configuration
that lives in a YAMO file counts as programming, Like
if you go and change that, and you know, for me,
it's just a DSL, and every DSL is programming language.
Whether or not it's turning complete, it's sort of irrelevant.
And I'm sure someone's going to be like, YAML is

(39:47):
turning complete. I don't know that it is, but I'm
sure someone will make that comment, in which case, yeah,
for sure, but I mean you're changing something, you know
what you're impacting here, and you know I do. I
do think that the there is this fundamental disagreement on
how people really think about the world, and their internal
values and their own perspectives influence how they think about

(40:09):
the infrastructure that they write or what they create.

Speaker 1 (40:14):
The psychology of DevOps.

Speaker 4 (40:18):
Yeah, people problems, like that's right. We have a lot
more people problems than we do technical problems.

Speaker 5 (40:24):
Yeah, it's never a technical problem.

Speaker 4 (40:27):
I mean it work with like the teching people, I
would just I would get so annoyed so fast because
the people that I work with they don't like really
care about like the nitty gritty of what I'm doing.

Speaker 3 (40:38):
They prefer not to know.

Speaker 4 (40:40):
They would prefer if they never had to deal with
me at all, is like what they really they can't
do that, Like they want me in and out as
quickly as possible and have no idea what I'm doing,
and that's that's like their wish list.

Speaker 3 (40:51):
So I can't imagine working with people who I don't know.

Speaker 4 (40:54):
I think I had like one client nitpick at me
about like a load balance, the type of load balance
so that I was using and I was just like, oh,
I'm not doing this again.

Speaker 5 (41:05):
Yeah, I hear you.

Speaker 1 (41:06):
Yeah.

Speaker 6 (41:07):
I think, particularly at the larger organization level, a lot
of people want you know, infrastructure and DevOps is not
a driver for success, right, It's a requirement. It needs
to be part of the like pyramid that builds up
to you know, profit at the end of the day.

(41:28):
But people want it to be a really small piece
and they want it to be quiet, and they you know,
want to build that upper application piece that actually drives dollars.
And it's a frustrating place to be in in terms
of psychology at DevOps, like as you just mentioned, well, like.

Speaker 5 (41:46):
That is when I when.

Speaker 6 (41:49):
I talk to a lot of DevOps engineers, platform engineers,
whatever you want to call them, it's like, you know,
they're always just struggling with They have a ton of work,
they have a huge backlog, they don't have a bunch
of it, like, they don't have a big enough team.

Speaker 5 (42:02):
And I think that's really consistent.

Speaker 2 (42:07):
I think part of it comes from the fact someone
was saying this recently to me that they think the
next innovation and DevOps will be where the DevOps teams
and the platform and the platform engineering teams and the
product engineering teams will come and work together. And I'm like,
I don't I think you misunderstood what DevOps means. If
you think that's that's a future we haven't reached yet.

(42:30):
But I think you know, organizations definitely, you know, it's
swopped out whatever the what they were calling release engineering
or infrastructure management, they just gave it a new name.
And so of course we're going to see those problems
in organizations where they are stuck there, where they don't
see a solution, where they don't see anything as different,
and they need someone to come in and help them.

(42:52):
Got I got another controversial one. Here we go, cross plan.

Speaker 5 (42:57):
Oh have you read our blog posts?

Speaker 1 (43:00):
I have not.

Speaker 5 (43:01):
Okay, we've got a great blog post on it.

Speaker 6 (43:04):
I I shared out all the time because it's I don't.

Speaker 5 (43:09):
Know, it just feels it's a shame.

Speaker 4 (43:11):
I so.

Speaker 6 (43:13):
Back when I kind of started to build my team
and started to transition from solo to agency, I had
been really stoked on Kuber Duddies for a long time,
and you know, really, uh, I thought it was the
Swiss Army knife to solve all the problems. I think
about that less now.

Speaker 1 (43:35):
Episode over.

Speaker 6 (43:36):
Sorry, I still think it's a great tool, don't get
me wrong.

Speaker 5 (43:41):
So anyway, I you know.

Speaker 6 (43:44):
I was thinking cross Plaine was the next thing. I
was like really excited. I had my senior engineer Veronica
on my team. We we did her and I like
collaborated on a long term proof of concept to build
a bunch of infrastructure in cross plane, and I was
just bullheaded. I was really really excited. And then just

(44:07):
as we got through this proof of concept, it just
thing after thing. We're just too painful. It's just not
There was things that were missing that I was like, wait,
we can't do that. There aren't data sources. And they
have some of those things now, but I still think
that overall there's a huge chicken and egg problem, like
where do you get that Kubernetes cluster? So you have

(44:29):
to kind of solve that differently all the time, And
there is a bunch of problems around ergonomics of that tool.

Speaker 5 (44:37):
And I just talked to somebody, somebody who worked at
far Winds.

Speaker 6 (44:43):
She you know, they had gone all in on cross
plane and then they ended up pulling it out and
going back.

Speaker 5 (44:48):
To terra form. Uh. And our blog post kind of
shares all our thoughts on that.

Speaker 6 (44:52):
But yeah, cross Plaine, is it was exciting as an idea,
I don't think it.

Speaker 5 (44:56):
I don't think it delivered on the promise. Sadly.

Speaker 2 (44:58):
Yeah, I mean, I'm really with you here, I don't.
I don't know what the best way of summing it
up is, Like, imagine if Kubernetes deployed all of your infrastructure,
and you.

Speaker 1 (45:08):
Know, I don't.

Speaker 2 (45:09):
I don't love that because I just don't like these
two things coupled together. Uh, And I feel like it's
taking one very complex thing and throwing yet like a
second aspect to it where people are already over using
Kubernetes in a lot of places where it may or
may not need to actually be utilized, and then to
throw this on top. I'm really glad there is an

(45:29):
article out there that discusses these because I definitely would
have wanted to throw it at some of my past
clients and customers who share it.

Speaker 6 (45:35):
You know.

Speaker 2 (45:36):
I think there's one of these problems though, where if
you find yourself in an organization that has those problematic
patterns in place, how do you fix.

Speaker 5 (45:45):
That problematic patterns in places?

Speaker 2 (45:47):
And like, like, you know, you come in an organization
and they are using cross Plane and you're like, okay,
you know, they're probably I've been called in because they
know that there are issues, and you can look at
it and be like, okay, I bet one of the
issues is how is the fact' using cross play you
didn't think about what the implication of that was going
to be. You're pretty much stuck on Kubernetes, you know,
full on there. And it's the same teams in organizations

(46:09):
that don't have Kubernetes experience but somehow of cross plane experience,
and they also don't have infrastructure experience outside of Kubernetes
or outside of cross PLAYE.

Speaker 5 (46:19):
I just I fear. I fear those organizations.

Speaker 6 (46:22):
Yeah, and I think there isn't a great answer there. Right,
It's like, hey, either you continue down that path and
you upscal you you try and make it as little
as painful as possible. I think one of the things
is like probably there's a new pricing model with that
Upbound has introduced where cross Plane is now if you

(46:44):
want access to their providers like the them, you need
to pay at least one thousand dollars a month. I
think that's like newer versions of their providers like maybe
three or six months or something like that, which was
kind of mind boggling to me. So I think you
need to be you know, you need to accept that. Hey,
if you've made that poor technology decision, you you either
need to learn you need to migrate away from it,

(47:07):
or you need to like go further into it. You know,
you need to lean into it, which would if you
have the expertise, maybe you can make that work. I
think that it's still probably going to be painful, but
I think you could probably continue to polish to bring
bring it back to our earlier conversation. I like.

Speaker 2 (47:25):
I like that perspective. I think that's a good one.
It's that you're in a problematic spot. You there is
no there's no world where you don't spend more resources.

Speaker 6 (47:34):
Uh.

Speaker 2 (47:34):
It doesn't just magically get better. And you can either
go deeper on it, uh, you know, level up your
team's experience, utilize the technology the way it's supposed to,
or pick a better pick a better answer. And I
think a lot of people don't want to hear that answer.

Speaker 6 (47:48):
Yeah, we had a client they had built their huge wrapper,
a typeescript rapper. They were they were a typeescript shop.
They had a huge mono repo. It was kind of
a thing of beauty. I think they were in the
one hundred thousand plus pull requests count.

Speaker 4 (48:04):
But the.

Speaker 6 (48:06):
Thing they had done was they built a big typeescript
wrapper around terror.

Speaker 5 (48:10):
Form and.

Speaker 6 (48:13):
It got complex. They were doing a ton of code generation,
they were doing a ton of stuff that I was like, guys,
come on, and they knew it too.

Speaker 5 (48:21):
They knew it was painful to them. They like they
were like, hey, what do we do with this? Do
we keep going?

Speaker 6 (48:26):
And our advice, you know, after we did an audit
for them and then kind of just did some guidance
sessions for a few months later, and our advice was like, no,
you need to pull everything out, Like we need to
get away from this because you're just going to keep
building complexity and your engineers that you're hiring are not
going to know that complexity. They're going to come in

(48:47):
and say, hey, I know Terrorfarm, I can do this,
and then no, they don't that because there's so many
devils in the details. You've built so many layers of abstraction.
A lot of the time is keep it simple. Stupid
is a beautiful saying. To continue to repeat. So yeah, yeah, a.

Speaker 3 (49:06):
Lot of silly things. But like, I don't know.

Speaker 4 (49:09):
The thing that I like about terraform is it's just
a fancy makepile, so I can't imagine throwing anything else
on top of that besides making it like it get template,
like I do that a lot, where it's it's like
my template freepo, or I'll use like cookie cutter or
something to generate the tf bars pile.

Speaker 5 (49:24):
But that's well, that's so I sort of get it.

Speaker 2 (49:28):
And I think where it came from for me is
that originally things like terror gront needed to exist because
of lack of workspace support, or lack of environmental variable support,
or lack of good loop support within And I hesitate
to call it terraform because now we have open TOFU
and I'm now I need something that groups all the
HCl language support together so I don't have to pick

(49:50):
one word over the other one. I mean, I just
want to see open TOFU Terraform's done for me, agree,
So and maybe I want to get your opinion on
that too, because I think that's that's a good perspective
are I sort of get it, And actually with terrorform,
with TF. Now you have the CDK as well, which
is at least blessed version, but at least for me,

(50:10):
I prefer the thing that's more declarative. Like I feel
like that's sort of the point, is declarative infrastructure rather
than programmatic infrastructure creation, because we had that with things
like Puppet and Chef and it did not it did
not go over well. But so I'll ask you, Matt,
you know open TOFU or terraform.

Speaker 5 (50:30):
We're if you see any of my content, if you
see our blog posts.

Speaker 6 (50:35):
We're big open TOFU folks. I have talked it open
tofu North America. We've migrated five clients to open Tofu
six and we're I honestly believe in the whole thing,
not only just from the open source perspective and the
fact that Hashi Corp. Did a rug pull, but because

(50:58):
they're they're innovating, may be a bit better, right, Like
they have new features that I'm like pretty excited about.
And I like that they have a slack community and
people are really active in it. I like that they
are supporting their their community and being really on.

Speaker 5 (51:16):
Top of it. There's a lot of really good engineers
on that team.

Speaker 6 (51:21):
I've gotten to, you know, go out and have have
a beer with with Christian, who's the team lead, and
I really like the guy. So I think that outside
of even the license change, I would say I'd be
going towards open TOFU. But if you're on terraform today,
the biggest thing I always say this, it comes down

(51:41):
to optionality. When you want to go and automate your terrorform,
you either have on terraform, you have the option of
open source tools like Atlantis. You have the option of
writing earned pipes, which we highly recommend or highly don't recommend,
recommend against.

Speaker 5 (51:59):
And then finally you have tear from Cloud. Tear from
Cloud not a bad product.

Speaker 6 (52:04):
It does what it needs to do, but it is
five to ten times more expensive than its competitors.

Speaker 5 (52:09):
And that is the big rub.

Speaker 6 (52:11):
Right, you have a product in the space and it's
the only product you can choose and pay for and say, hey,
we have a vendor that helps us manage the complexity
of all of our infrastructure, but it's five to ten
times more expensive than everybody else. Like that's it's a
really hard pill to swallow. So I think that open
TOFU gives you a way out of that, and that's

(52:32):
one of the main reasons that we tell people to
go that route.

Speaker 2 (52:36):
See, I have my fingers crossed that will finally get
a switch for every single resource that allows us to
turn it on or off without abusing the count variable.

Speaker 6 (52:47):
Interesting, why do you dislike count? Well, do you just
want like an enabled flag?

Speaker 5 (52:53):
Yeah?

Speaker 2 (52:53):
I do just want an enabled flag. The number one
reason I dislike it is, besides all the linting problems,
is it converts your resources from a single into an
array with a single object, and then if you want
to turn it off, you can't like remove the count
once it's true, Like, you can't pull that out and
just have everything work.

Speaker 5 (53:13):
I get what you're saying.

Speaker 6 (53:14):
So there is now the moved block which allows you
to like change the path of something within the state file,
which can save you there.

Speaker 5 (53:23):
You know, it can make it so that, hey, you.

Speaker 6 (53:24):
Can add count to a resource even if it didn't
have it before, and not have it destroy and then
recreate that resource still pain.

Speaker 5 (53:31):
I get what you're saying.

Speaker 6 (53:32):
Maybe there should be some you know, smarts that gets
built into that, but not really.

Speaker 2 (53:38):
Want Yeah, I mean you really you really want your
tools to enable a pit of success and I feel
like this is one of the things that is for
sure a pedo failure and tricks people up. And yeah,
there's ways around it, for sure. But the last thing
I want to do is like you're, oh, yeah, let
me put an extra ticket on our board for every
single infrastructure change, just so someone can go back and

(53:59):
write to move block and then delete the move block
when they don't need it anymore.

Speaker 6 (54:02):
It's yeah, no one's gonna do that work, I hear you,
And I wonder if they could build that.

Speaker 5 (54:09):
I don't know. I'm going to look up if there's
an issue in the open tofu for this.

Speaker 2 (54:13):
I will put my thumbs up on it. I'm not
going to do that tough for development myself, but I'm
happy to use my very expensive thumbs up button to
so expensive.

Speaker 5 (54:24):
Yeah yeah.

Speaker 6 (54:25):
I think one of the great things about open tofu
is that they are being very community driven in terms
of what they work on, so those thumbs up matter
a lot. Where they have a board an issue that's
like the top of their issue list that just lists
all the issues that have gotten a certain number of
thumbs up, and they're saying, hey, we're going to work
on the top one and that is that's really cool

(54:48):
because hey, we have a say, I think a lot
of the a lot of the problems with terraform too.
We're around we as a community where you know, banging
the gavel like, hey, we need this thing, we need
the thing, we need this thing, and just weren't getting
anywhere with it, and open Tofu has just kind of
flipped that.

Speaker 1 (55:05):
So for sure, Yeah, no, I think that the key
driver behind that is Hashi Core had to satisfy the board,
you know, whereas open Tofu has to satisfy the end users.

Speaker 5 (55:24):
Yeah.

Speaker 6 (55:26):
Yeah, it's a shame that Hashikor went public.

Speaker 5 (55:32):
I think they had some great leadership.

Speaker 6 (55:34):
And they did what companies do, which is you get
to a certain point and everybody wants to make money
off of your hard work.

Speaker 5 (55:43):
And I don't blame them for that, I really don't.

Speaker 6 (55:46):
But I think being a public company is brutal and
you know, it's just rough.

Speaker 5 (55:54):
It's a shame.

Speaker 1 (55:56):
Now that is. Having spent my career in startups, I
can't imagine in twenty twenty five why anyone would take
their company public if you really believe in the mission
of your company and the end users who are supporting
your product. If that's your focus, there's I don't see
a valid argument to take the company public.

Speaker 2 (56:19):
I have to imagine that it's not usually from like
totally private to IPO like. It's usually through the VC chain,
which is all about extracting the most money out of
that thing and scamming the most number of people in
that pyramid scheme until you know you can get it
out to the public, and then from there the shareholders

(56:39):
are very myopic, focused on just the next quarter and
don't realize that having huge impacts on how the business
actually works and the perception of the brand has a
long lasting impact to the bottom line. I do have
I do have a question of my own. Maybe it's
just something that you've thought about a little bit. One

(57:00):
of the problems that we have in our space. So
for our product, it's logging and access control that we
provide for our customers, and there's a white labeling experience.
Now there's a whole part of the infrastructure which is shared,
but then there is a bunch of pieces which are
per customer, per account for each one of our customers,
and sometimes they have more than one, and we're in

(57:22):
this weird area where we don't know whether or not
infrastructure as code makes sense for that, and whether or
not we should be rolling out either. I mean, for
this we're actually using CloudFormation templates and AWS, but we
could just easily switch over to open TOFU whether or
not that even makes sense, or whether or not running
through this list of resources that a new accounts need
is a programmatic process, or whether or not it should

(57:43):
be declarative and infrastructure. And I know we're not the
only company with this problem. We're not a special snowflake here,
So I don't know if this is something you've seen
before and have some wisdom to share.

Speaker 6 (57:54):
Yeah, I have some wisdom we have. Our most recent
case study was with a company called power Digital. For
each of their customers, they're basically spinning up a small
data warehouse in Snowflake that connected data wus and a
new GitHub repository.

Speaker 5 (58:14):
And like did a bunch of things.

Speaker 6 (58:16):
And they had five hundred customers, right, so they were
doing this constantly.

Speaker 5 (58:22):
And really what.

Speaker 6 (58:23):
It comes down to is you you do probably want
that in infrastructure is code because you want to manage
the life cycle of that, right, you might change that
that architecture for that client. Infrastructure that you're like just
stamping out every time you get a new one. You
might change a little thing, right, You might you might
do something to it, you know, new tomorrow, and you

(58:46):
want to roll that out across everyone. Infrastructure is code
is great for that. You also might, you know, if
there's any cost associated with those resources, you might want
to destroy that infrastructure when that client goes away or
customer goes away. I think that infrastructure is code makes sense.
The point problem is that you just want that to
be a highly automated, low touch workflow. Uh. And that

(59:09):
is the point that becomes a rub, is that you
need to kind of come at it from this perspective
of all right, we need to be we need to
have our infrastructure as code to be on such rails
that it needs to get indicate and it needs to
apply automatically, and it needs to do all of that
very seamlessly so that we're not needing to.

Speaker 5 (59:26):
Think about it too much.

Speaker 6 (59:28):
We built that for our other clients, So if you
want to talk one on one afterwards, I more than
happy to give you all the the info on how
that that worked.

Speaker 5 (59:35):
But yeah, I think that.

Speaker 6 (59:39):
Infrastructure should be a thing that we create provision, we
can change if.

Speaker 5 (59:43):
We need to, and we can destroy it if we
need to.

Speaker 6 (59:46):
And if we just do that with you know, calling
the a ws STK or we.

Speaker 5 (59:52):
You know, do something.

Speaker 6 (59:53):
That that's the A w U S CLI bash script
or whatever it is. Those types of things, they can
be a they feel very programmatic and they feel like
a really good solution at the time, but then you
don't have as much control in the long term.

Speaker 5 (01:00:10):
So yeah, that's my thought.

Speaker 1 (01:00:13):
Cool, Yeah, I want to switch topics just a little bit,
or not topics, but switch trajectory. Maybe it's a better
word for someone who's listening to the podcast. Considering the
consultant versus career DevOps approach, what's your your bullet lists

(01:00:34):
of pros and cons of each?

Speaker 6 (01:00:40):
The pros list is high, is long in terms of
just like you know, being a consultant, particularly owning my
own business is very very advantageous to the rest of
my life in terms of I set my own schedule,
I decide who I work with.

Speaker 5 (01:00:59):
You know, I get to build a team, which is
really nice and it's you know, they're they're my.

Speaker 6 (01:01:04):
People, and you know, I enjoy helping them grow as
engineers and you know in their career. There's there's a
lot of things on the pros list. What's more interesting
probably is the cons list. The the cons I think
come from you know, there's always going to be some
level of like feast or famine. You know, I've been
doing this for a long time and I still you know,

(01:01:26):
find myself in are we gonna get a new client
next corner? And usually it all works out right, Like
you know, there's some you know, serendipitous occasion. I've never
you know, had to let anybody go because we didn't
have enough work. I've never had to you know, be
uh out of work for for many months at a time.

(01:01:50):
You know, I've had it well. But like, hey, if
you're just starting and you don't have a network, you
might go for a certain six months without you know,
picking up a client and actually having work. So I
think that understanding that you're you're on your own and
you know your your livelihood can depend on that is
its own like source of stress, and I think that

(01:02:14):
there's something there.

Speaker 5 (01:02:15):
I think you also really need to be able to
talk to people problems.

Speaker 6 (01:02:18):
A lot of the times I'm talking to clients and
letting them know that I'm I'm there as an individual
to help them, and I can see that they're stressed.
I can see that we are are trying to solve
some like deeper emotional need and you need to have
those like soft skills at a deep level to help

(01:02:39):
them navigate the right decision and get that stress off
their plate. And I think those are two things that
a lot of people that are like really excited about consulting,
they don't think about those two things.

Speaker 5 (01:02:53):
Consultants are not the best engineers. I'll tell you that.
I think that the.

Speaker 6 (01:02:59):
You know, I definitely wouldn't consider myself a you know,
a wizard code er. I think I can write cleaning code,
and that's like part of a craft that I really love.
But I definitely have worked with a lot of smarter
people in my career, and they're off being you know,

(01:03:19):
senior engineers or above elsewhere at companies that just tell them, hey,
we need to build this thing. They're not trying to
solve those people problems. They're not trying to navigate those
the intricacies of hey, this client you know, consultant relationship.

Speaker 5 (01:03:36):
So I don't know, does that answer your question?

Speaker 1 (01:03:39):
It does? Yeah, And I think It highlights one of
the things that there's almost like a translator skill required
to be a successful consultant, because your clients typically don't
come to you describing a technical problem. They come to
you describing some impact that's hattening to their business, and

(01:04:00):
you have to be able to translate that talk with
a mass follow up questions and then translate that into
a technical problem that you can solve.

Speaker 5 (01:04:10):
For sure. Yeah, a lot of it is like can
I repeat that back to you?

Speaker 6 (01:04:15):
Like this is what I'm hearing, right, And a lot
of things come back to that. You have to be
able to read between those lines and kind of understand
at a root level, like what's the actual issue here?
They're telling you one thing, but it's something else, and
that's a skill.

Speaker 1 (01:04:32):
Yeah, yeah, can I repeat that back to you? Has
probably got to be one of the most valuable phrases
in humanity.

Speaker 6 (01:04:40):
Yeah, let's make sure we're on the same page here.
That's the way it goes.

Speaker 2 (01:04:44):
I think there's a maybe an additional connection here, which
you brought up early on who your decisions makers are.
If you're a consultant, you're selling your services to someone
they have to have money, like an engineer probably isn't
going to make the decision on paying you to come
in and help, which means you're talking to the like
you said, directors of technology and higher and what problems
do they think they have?

Speaker 1 (01:05:05):
Right?

Speaker 2 (01:05:06):
Uh, and they're not like, oh, well, you know, we
have some terriform modules that don't work, right, you know,
it's probably not what they're coming and saying.

Speaker 5 (01:05:12):
No matter.

Speaker 1 (01:05:14):
Yeah.

Speaker 6 (01:05:15):
And and the way we approach that is that, hey,
I want to we want to come in and we
want to solve both problems. Right, we want to solve
the leadership's problems that are typically around scale workflow decreasing
you know, engineering costs or maintainability costs.

Speaker 5 (01:05:31):
We also want to solve the.

Speaker 6 (01:05:32):
Ergonomic problems that the you know, actual people who are
writing the infrastructure's code or the application engineers or are
dealing with.

Speaker 5 (01:05:39):
So what we do is, well, typically.

Speaker 6 (01:05:41):
We have an audit and you know, more and more
we're we're we're selling that audit as like are our
way to really help understand an organization and get them
the right prescriptive guidance that they need. And as part
of that we and we you know, interview engineering leadership.
We also engineer interview a bunch of the infrastructure engineers

(01:06:02):
and application engineers. We make sure that we're kind of
holistic in approaching the problem, not just from what we're
being told, but making sure we're uncovering what else is there,
because we don't want to, you know, leave any turn
like rock unturned.

Speaker 5 (01:06:17):
It's important right on.

Speaker 1 (01:06:18):
Just feel like a good time to move on to picks?

Speaker 2 (01:06:20):
Yeah, I think so let's do it all right?

Speaker 1 (01:06:22):
Cool, Jillian, You've been out for a couple of weeks,
so I can only assume that you have been diligently
researching your next pick. So I'm excited to hear what
you got this time.

Speaker 4 (01:06:33):
I'm just going to keep going with the shameless self
promotion until I get more clients.

Speaker 3 (01:06:37):
This is what I'm going to be doing.

Speaker 4 (01:06:38):
You know, that's a bit on topic of the show,
so you know, if you've been listening to the show lately,
you know, I really like AI and LMS and all
of those kind of tools. I do have a service
to get all of that set up for you on
your own AWS account. This is mostly geared towards data
science companies, because if you're not a data science company,
I don't really know what to do with you. Frankly,

(01:07:00):
if you need kind of a junior, maybe grad student
level research assistant to go querying your papers, querying structured
and unstructured data sets. We've got more data sets being
added every day so far. The top one is open
targets for drug discovery. But I've had a whole bunch
of single cell spatial TRANSCRIPTO mix. Like there's just people

(01:07:21):
are starting to do some pretty like cool and wild
things with it, which is exciting. So if you're interested
in that, you can go to my website. It's a
dabbleodevops dot com slash ai and you'll see that there's an.

Speaker 3 (01:07:33):
LM data discovery tool this week.

Speaker 4 (01:07:35):
The page is in fact up last time it may
or may not have been, I'm not sure, but this
is okay.

Speaker 3 (01:07:41):
There it exists.

Speaker 1 (01:07:43):
Open research for drug discovery. Sounds like my time in
high school.

Speaker 3 (01:07:49):
Yeah, yeah, all right, that's good.

Speaker 4 (01:07:52):
It's a little bit too real for me to say,
Will and my and my teenagers, so we're just we're
right on over that.

Speaker 3 (01:07:59):
You can't do that this morning.

Speaker 2 (01:08:02):
I think I think that part may actually have to
be cut from the from the episode before.

Speaker 4 (01:08:10):
I'm not sure if like anything gets cut from these episodes.

Speaker 3 (01:08:13):
I don't know.

Speaker 4 (01:08:13):
I've always wondered if the things that we can say
and still have sponsors or do we just have sponsors
at all for this show?

Speaker 2 (01:08:22):
Okay, we definitely have to cut that part, so I'm
going to market the clip here at this point.

Speaker 6 (01:08:28):
And it was good though, I once it clicked, I
got it as that was a great one.

Speaker 1 (01:08:37):
It's like a it was a joke grenade where you
pull the pin and throw it out and it's three
to five seconds before it actually lands. Yeah, all right, Warren,
where'd you bring for a pick this week?

Speaker 5 (01:08:47):
Yeah?

Speaker 2 (01:08:47):
Uh so, I'm gonna be a show for our conference.
I absolutely love DevOps Days. I think it's one of
the best set of conferences anywhere in the world. They're
volunteer run and my CEO will be giving the keynotes
speak talk at devop Days Zurch this week. It's all
about just some thinking at authors and it's a it's
actually a great talk.

Speaker 1 (01:09:09):
Right on nice Yeah, I agree with you on devop Stays.
Devop Stays Amsterdam is probably one of the best I've
been to because those guys they just go out of
their way so that when you leave all of your
swag reminds you that you were in Amsterdam.

Speaker 5 (01:09:29):
You know.

Speaker 1 (01:09:29):
It's very very like cultural and historic and authentic and
super cool and.

Speaker 5 (01:09:35):
Thoughtful, very cool.

Speaker 6 (01:09:37):
Yeah, and I'll double plus one that with saying that
devop Stays Denver has been there's a really good community
behind it, really good group of folks, and their talks
are awesome and.

Speaker 5 (01:09:50):
It's just great community.

Speaker 6 (01:09:52):
If people have not been to their local chapters devop Stays,
they're they're missing.

Speaker 5 (01:09:56):
Out right on.

Speaker 1 (01:09:58):
All right, Matt, would you bring for a.

Speaker 6 (01:10:01):
I have a book I had I had a hard
time picking, but I am obsessed with this series called
Dungeon Crawler Carl h It is a fantasy sci fi series.

Speaker 5 (01:10:15):
They're on book seven now.

Speaker 6 (01:10:18):
And you might scoff at the name and you might
think that that's not for you, And I will tell
you that I'm nine for ten on friends that I've
recommended it to and had them go wow, I now
have a new favorite book that you know.

Speaker 5 (01:10:32):
It's really consistent. Yeah, yeah, that guy out of my life.

Speaker 6 (01:10:40):
Fantastic, really witty, really funny. There's a talking cat. You
will enjoy it if you read it.

Speaker 1 (01:10:47):
Just the name Dungeon Crawler Carl sounds. It sounds like
the hero from an eighties video game. That's such a
cool name.

Speaker 5 (01:10:55):
Yeah, it's Goofy.

Speaker 6 (01:10:57):
I think that a lot of people have an issue
with the name, but you read it, you'll enjoy it.

Speaker 2 (01:11:04):
It makes me think of this like the hardest video
game I've ever played, and that's not Dark Souls or
Ninja Guide, and it's something called Leicester the Unlikely for
Super Nintendo.

Speaker 4 (01:11:13):
It's almost like.

Speaker 2 (01:11:14):
An eighties game. You are literally playing just a regular
human who has to navigate quite challenging set of circumstances.
Like imagine you're in a fantasy world and you don't
have any superpowers and you can't jump high and if
you fall off a rock you will die.

Speaker 5 (01:11:29):
That is this game.

Speaker 2 (01:11:31):
And you get abducted by cannibals and have to like
steal keys and unwhittle ropes in order to get out,
and it's it's quite the challenge because there is no
help at all while you're playing, so you will die
over and over again.

Speaker 1 (01:11:44):
Nice.

Speaker 5 (01:11:45):
Those games were just brutal. Yeah.

Speaker 6 (01:11:48):
I used to have a I forget what it was,
one of the handhelds back in you know, when I
was a kid. I feel like it was made by
Sega Games anyway. Yeah, the game gear, Yeah, I had
the game gear.

Speaker 5 (01:12:01):
I had save.

Speaker 6 (01:12:02):
I would play a game for ten hours and I
couldn't save it and I was like, oh my.

Speaker 5 (01:12:07):
God, I drove me insane. I think that things were
a little bit different back then.

Speaker 3 (01:12:14):
With no save.

Speaker 4 (01:12:15):
But see, this is why you just play cute c
SIM games where you can't die, Like if you guys
are just playing Disney Dream like BALI, this is not
a problem.

Speaker 2 (01:12:23):
That's pick next week.

Speaker 3 (01:12:26):
I think I really picked it, but I probably probably.

Speaker 5 (01:12:28):
Because I want to hear about it.

Speaker 3 (01:12:30):
It has like my favorite Disney ships.

Speaker 4 (01:12:32):
It's Malipicent and Hades, and I just I love that pairing.

Speaker 3 (01:12:36):
It's so great.

Speaker 1 (01:12:41):
Well, so my pick is going to be kind of
letdown after that conversation because I'm picking seat covers.

Speaker 3 (01:12:48):
So I like covers.

Speaker 5 (01:12:51):
I like seat covers.

Speaker 1 (01:12:52):
Yeah, so I just got a new set of seat
covers from a company called Sheer Comfort. S h e
A are so like shearing a sheet but shere comfort.
And they've got so many different choices for seat covers.
And you know, the seats in my truck they were
getting like mud and dirt and stuff. On them, and

(01:13:12):
I thought, I just can't do this to the seats,
so I bought these seat covers. They're super cool, really
really well made, pretty easy to put on, look great
once they're on, and then it's got all like the
nice features, like they're specific to my model of trucks.
So like I've got a Ford truck, So it's got

(01:13:32):
these little loop pandles that you have to pull to
get the seats to fold down, so it's got the
cutouts for that so that that works natively, and it's
got you know, it's built so that the side restraint
airbags still work, which may or may be cool at
some point in my life. But a lot of little
features like that and just really well built. So yeah, sure,

(01:13:55):
comfort seat covers. If you're looking for a set of
seat covers. And then they know they sent with the box,
they sent like a product catalog, which kind of shocked me,
and it's a huge product catalog. So they also make
in addition to seat covers, they make like full I
don't even know what you call them. If you wanted

(01:14:16):
a blanket for your car truck, they have like shaped
covers for those, or if you have to put a
cover on your RV. They make a full custom fit
cover to fit your RV. Just a lot of things
that I didn't even know existed that I found out
because they included the catalog.

Speaker 2 (01:14:33):
I think, will your pick next week? Is like you're
already thinking about the McMaster car product, right, Yeah.

Speaker 1 (01:14:39):
For sure, this is the master the McMaster car catalog
of seat covers and car wraps.

Speaker 3 (01:14:49):
But are they well, if it's like sheer, like sharing
a sheep.

Speaker 1 (01:14:53):
That is one of the options you can get the
sheepskin seat covers, you know, go straight back to the
eighties and put them in your Camaro with the T tops.
I didn't go that route. I went with a it's
like almost like a neoprene thing that's not going to
show any mud or dirt or coffee that I spill

(01:15:13):
on it.

Speaker 4 (01:15:14):
I didn't know people had like well seat covers, And
now that's a new existential dread for me to have
that anytime I get into somebody's car, I'm gonna wonder,
oh no, is this wall?

Speaker 3 (01:15:23):
And am I going to die?

Speaker 1 (01:15:25):
But well, I think you're pretty safe as long as
whoever you're with isn't wearing like a mullet and a handlebar,
mustache and a camaro. You're probably gonna be okay, that's true.

Speaker 3 (01:15:35):
I can probably self select for these things.

Speaker 1 (01:15:37):
Yeah, I think so. I think it's pretty safe. All right.
Now that I've finished offending everyone on the list, I
think we've got ourselves an episode. Matt, thanks for joining us. Man,
it's been great talking to you.

Speaker 4 (01:15:52):
Yeah.

Speaker 5 (01:15:52):
It's a really good conversation. Folks appreciate it. Yeah, good questions,
good topics.

Speaker 1 (01:15:57):
Yeah. So be sure and check out Matt's website master
points masterpoint dot io.

Speaker 5 (01:16:02):
Is that right, Yes it is.

Speaker 1 (01:16:03):
Yeah. So if you need consulting services or if you are,
you know, wanting to try your career out, check it
out and see if he's hiring. Arren Chillian. Thank you
both for being on the show with me today. Thank you,
and for everyone listening. Thanks for listening, and I'll see
everyone next week.

All Episodes

Mastering Infrastructure as Code: Lessons from Matt Gowie's Consultancy Experience - DevOps 235

Episode Transcript

Popular Podcasts

My Favorite Murder with Karen Kilgariff and Georgia Hardstark

24/7 News: The Latest

Dateline NBC

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}Mastering Infrastructure as Code: Lessons from Matt Gowie's Consultancy Experience - DevOps 235