November 26, 2025 • 53 mins
What does it take to make more environmentally sustainable software? Carl and Richard talk to Tom Herkhove about Microsoft's efforts to make carbon footprint and emissions visible for applications. Tom talks about the Azure API Management interface as a great starting point, and the ability to shift workloads to low-emission data centers as needed. The conversation also digs into wasted cycles, like automatically fired CI/CD pipelines whose results are never reviewed. It all begins with measuring - what action you take from there is up to you!
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:01):
How'd you like to listen to dot net rocks with
no ads? Easy? Become a patron for just five dollars
a month you get access to a private RSS feed
where all the shows have no ads. Twenty dollars a
month will get you that and a special dot net
Rocks patron mug. Sign up now at Patreon dot dot
NetRocks dot com. Hey guess what it's dot net Rocks
(00:37):
episode nineteen hundred and seventy eight. I'm Carl Franklin and
am Richard Camp and Richard, you're a down under?
Speaker 2 (00:44):
I am?
Speaker 1 (00:44):
I am.
Speaker 2 (00:44):
This is the last week of the month long tour
with the grand baby. And you know, the whole idea was,
while she's not moving around too much, just a little package,
it'll be a good time to take her and show
her off to the extended family. Of course, in this
time she's learned to crawl, so now she's a toddler
and starting to get bumps and bruises. Just you know,
remember that time your kids get all.
Speaker 1 (01:04):
Your baby get gets all banged up. That's happening. How
cool is that? It's been really fun. The only thing
she might regret is not being able to remember any
of it's the best part.
Speaker 2 (01:13):
Right when she's twelve years old, like you've already been
to Australia.
Speaker 1 (01:19):
Yeah, she won't remember the trauma of being dropped on
her head on the plane, you know, nothing like that.
Speaker 2 (01:26):
Well, we've got a few more days left. I'm actually
up near Brisbane.
Speaker 1 (01:31):
We're in the.
Speaker 2 (01:31):
Gold Coast, an area called kulan Gata, right by the beach.
It's beautiful here. I love the Gold Coast boys.
Speaker 1 (01:37):
It hot holy man. Yeah, all right, So let's start
right off with the year nineteen seventy eight, right, dude,
what do you got? Well, some of the major events.
Harvey Milk was elected the first openly gay person elected
the public office in California, so you know, things are
becoming a little more socially liberal. The Jonestown massacre on
(02:00):
November eighteenth was terrible, crazy. We're nine hundred members of
the People's Temple died in a mass suicide murder in Jonestown, Guyana,
orchestrated by Jimmy Jones in the good News Camp David Accords.
In September, President Jimmy Carter facilitated a peace agreement between
Israel and Egypt. There were some air flight crashes Air
(02:22):
India Flight eight fifty five Boeing seven forty seven coasts
crashed off the coast of Bombay. Two hundred and thirteen
people perished. The northeastern US blizzard of nineteen seventy eight.
You weren't around for this, Richard, because I was on
the West coast territory. Do you remember it? I do
(02:43):
I remember it? So I would have been ten eleven, Yeah,
And we had the day off from school and there
was literally four and a half five feet of snow
right packed up against the door, wow, and all over.
And so my brother and our friend Todd, and I
(03:04):
and Chris Conrandy from down the street, we set up
out to make Skylab, which is what they called it.
Just this huge edifice of snow with snowballs could roll
down and around and everything. We just had a great time,
all right.
Speaker 2 (03:22):
Yeah, sure, And this is right right when Skylab was
going to about to re enter, right, they'd love it
the next year, yep.
Speaker 1 (03:28):
So we were hip little kids even then. The FIFA
World Cup nineteen seventy eight FIFA World Cup was held
in Argentina, with the host nation winning the tournament. Garfield
debuted in the comic strip World Jim Davis made its
first appearance in nineteen seventy eight films Grease, National Lampoon's
(03:50):
Animal House. Of course, those were great. In science, you
can talk about these things. You'll probably talk about the
first human cloning claims.
Speaker 2 (04:00):
Right, yeah, well I'm not going to talk a lot
about it, but yes, that's when those experiments going on.
Speaker 1 (04:06):
Nobel Peace Prize was awarded to Anwar Sadat and monocom
Big and for their efforts in the Camp David Accords.
So what was happening in computing in space.
Speaker 2 (04:15):
Let's do the space stuff first. Last year and seventy seven,
the Shuttle Enterprise was deployed and they were doing their
first captive tests and landing tests. In seventy eight, they
fly the Enterprise by seven forty seven to Alabama going
to do vibration testing for launching. There was the intent
to actually fly Enterprise into space at that time, although,
(04:36):
as we'll find out a couple of years later further on,
there's so many changes that happened from all these tests
that it's too expensive to retrofit Enterprise to be a
flable vehicle. And they'll take an earlier test Article STA
zero nine to nine and that'll and refit in and
it'll become challenger. Otherwise, in space, there was Venus missions
(04:57):
in nineteen seventy eight, both American and so the Pioneer
Venus project launched in May, arise in December. This is
the orbiter, so it actually goes into orbit around Venus
and actually operated until nineteen ninety two doing measurements and
so forth. But there was a second part to it
called the multi Probe, which actually launched later in August,
(05:18):
but arrived earlier because it didn't need to go into orbit,
so it used a higher energy trajectory to get there.
Speaker 1 (05:25):
What did we not know about Venus that we felt
we needed to know by sending those probes.
Speaker 2 (05:30):
We I mean, we didn't have We didn't know the
composition of the atmosphere. We didn't know it all the
way down. That's why putting the landers down to measure
as you went down, we were just trying to figure
out what was going on there, Like even understanding how
hot it was took time. You know, the the blanket
effects of CO two weren't well known yet, so this
(05:51):
was part of you know, the CO two absorbs in
for red light and so that's why it'll concentrate heat.
But also this is like it's down on the surface there,
like what can we see? And so there was a
whole series of missions. They kept going like this is
even the end of it. So on the American side
they put down this multiprobe actually had four probes on it,
(06:11):
a big one that was about seven hundred pounds, called
the Large Probe, cleverly named, and then three small probes
that were all about two hundred pounds. The big one
had a parachute, so it was more like the Venera landers.
It transmitted information for an hour as it was descending
through the atmosphere, but as soon as it impacted the
surface it stopped transmitting. The three small probes were just
(06:34):
arrow shells. They had no parachutes or anything, so they
were free falling granted through a very thick atmosphere. And
one of them, the one they called Day because it
landed on the Day side, Venus continued to function. It
survived the landing and transmitted for over an hour from
the surface.
Speaker 1 (06:50):
So that means there was some kind of solid surface and.
Speaker 2 (06:54):
Without it there is right, it's mostly old volcanic flows.
The Venia missions this were eleven and twelve. So the
Soviets flew to Venus many, many times, and they will continue.
They'll be a thirteen oh fourteen. And so both of
an air missions went off flawlessly. They both landed on
Venus cleanly. Eleven transmitted for ninety five minutes, twelve transmitted
(07:18):
for one hundred and ten. And I'd like to point
out it's not that they only lasted that long. It's
that their transmitters are only so strong, and so the
orbiter has to relay the data back to Earth.
Speaker 1 (07:30):
Oh wow, and it goes.
Speaker 2 (07:32):
Out of range after about an hour and a half,
I see, and then nothing can pick it up anyway,
although generally speaking within it within twenty four hours or
so forth, things are broken from the pressure of the
heat and the amount of sulfuric acid in the atmosphere.
Speaker 1 (07:47):
Yeah, there is that little detail, little detail.
Speaker 2 (07:50):
This is also the year that Sharon, the moon of Pluto,
is identified by some astronomers just looking at it with telescopes.
It's the moon is massive, of course, it's about almost
an eighth the size of Pluto itself, so really really big,
and of course not great resolutions. We wouldn't get a
(08:11):
good picture of Sharon until the new Horizon spacecraft gets
there in twenty fifteen. Science wise, the RSA algorithms that's RIVS,
Shamir and Aldeman is publicly described in nineteen seventy eight.
So that's craz I think we still depend on.
Speaker 1 (08:26):
To this day. Still is it to this day? Yeah?
Speaker 2 (08:29):
And speaking of computing, this is a crazy year for computing.
So this is the introduction of Microsoft's Altair Basic.
Speaker 1 (08:36):
Yeah.
Speaker 2 (08:36):
Right, nineteen seventy seven was when the Altar came out,
but it's now it's fully released.
Speaker 1 (08:40):
Bill Gates through OUTE that right, Yeah, that's right.
Speaker 2 (08:42):
The Apple two distrive, the five and a quarter drive
gets released for the Apple two and they start making
the twoe. The Intel eighty eighty.
Speaker 3 (08:49):
Six processor kind of an important one, yeah, kind of important.
The sixteenth and sixteenth definitive sixteen bit processor. Yeah, here's
when you don't know the Intel twenty eight to sixteen.
Twenty eight sixteen the first electronically reirasable and reprogrammable ROM.
Speaker 1 (09:06):
Wow.
Speaker 2 (09:07):
So up until then when you when we finally made ROM,
they had to be erased by UV light, so you
would take the chip out of the computer, you would
shine a bright UV light on it to erase it,
and then you'd reprogram it and put it back in
the machine. This was an electronically reprogrammable one E PROM Yeah.
The first e proms. Carter also brought the first computer
into the White House, was in HB three thousand, the
(09:30):
introduction of laser disc Oh yeah, I remember those, remember the.
Speaker 1 (09:34):
Big laser discs. Yeah, and you know what its original
name was.
Speaker 2 (09:37):
You're gonna love this because it is nineteen seventy eight
disco vision.
Speaker 1 (09:41):
Oh no, no, And the first and the first disc
released is Jaws. Yeah, that makes sense.
Speaker 2 (09:50):
Yeah, I want to talk the word Star for CPM. Okay,
that's what it comes out, and it doesn't become super popular,
then it'll become super popular in nineteen eighty one because
word Star cuts a deal with the Osbourne people to
bundle it with the Osbourne one bundling even back then,
Wow and the deck vacks eleven seven eighty And I
(10:11):
wouldn't normally talk who a lot about an individual computer,
but there's so much of the architecture of this machine,
even though it was a mini computer, that sort of
describes how PCs will be built going forward, because it
was really they really worked hard on the instruction set
to maintain backward compatibility so that newer models could build
on the same instruction set pretty much indefinitely. And much
(10:33):
as that machine didn't survive long term, although it was
a phenomenal machine in its era, it really sort of
outlined the ideas of maintaining backward compatibility for better or worse. Yeah,
one last item. This is when Vince Surf and his
team define and test TCPIP.
Speaker 1 (10:49):
Yeah.
Speaker 2 (10:50):
Wow, the stuff we're using right now as we make
this show nineteen seventy eight.
Speaker 1 (10:53):
The TCPIP stack. That's it's the holy grail of computing
communication back then.
Speaker 2 (11:00):
Yeah, and UDP as well. It was all part of
the definition. UNIP is part of the stack. Yeah, and
that's we're using UDP now obviously. Yeah, we do what
we're doing wicked fast.
Speaker 1 (11:10):
All right, Well, uh, what's next? I think we do better?
No framework, right, let's do it. Let's do it. Roll
the music all right, man, what do you go? You're
in Australia right now. I am Our friend Simon Crop
is from Australia and he's up to something else. Now
(11:33):
it's often is yeah, this is uh, he just posts
in our Slack channel. Hey, I wrote a source generator.
Oh cool, of course you did, said, that was my reply. Yeah.
So this is called project files source generator. It's a
C sharp source generator that provides strongly typed compile time
(11:54):
access to project files marked with copied output directory in
the CS projectile. Wow. So a type safe API for
accessing files that are copied to the project's output directory,
eliminating magic strings, and providing intelligence and support for filepaths. Cool.
Now you might think that this is oh yeah, that's
(12:15):
kind of a neat feature. But you know, the more
I dug into the documentation here, the more you know
he highlights these features, and you're like, oh, how could
I have you know, lived without these strongly typed access
to project files. Right? That means you know this dot
dot dot file name compile time safety, so typos and
(12:36):
filepass become compilation errors. I love it. Intel Sense support
so you can discover available files through ID autocomplete. Automatic
synchronization regenerates when project files change, which is a great
feature of a cogenerator. Hierarchical structure that mirrors the project's
directory structure. Support for the glob pattern including star star,
(13:01):
recursive patterns. Smart naming converts file directory names to valid
see sharp identifiers, so you know, file names with a
space in the middle, that kind of stuff get converted
to a dash or whatever you say. Just really smart
thinking about source generation. Yeah, so there you go. Hats
(13:21):
off to Simon as usual.
Speaker 2 (13:23):
And you know, I've been calling LM's lately source generators too,
just re entrant.
Speaker 1 (13:27):
Ones, yeah, and nondeterminist there.
Speaker 2 (13:29):
Yeah, but it's interesting to see a proper source generator,
you know, in this day and age as well.
Speaker 1 (13:34):
Yeah, it's good on ya, Simon. Yeah, cool stuff.
Speaker 2 (13:36):
Who's talking to us today? Richard and I grab a
comment top of show nineteen seventy four. That's one we
did with Den Delamarski or we talked about kid hubspec
Kid because you know, all the AI.
Speaker 1 (13:46):
I realized there's so much we didn't talk to Dan
about that we could have. We could have gone on
for another hour at least.
Speaker 2 (13:53):
Easily, and it probably you know, and we're likely going
to maybe not with Den, but you know, one of
the things I've been doing on this tour talking to
a lot of different developers about how their development workflows
are changing using these new tools and just thinking large scale.
This comment comes from Brad who said during the episode
of the State of the developers will become shepherds of AI.
Speaker 1 (14:12):
Love the term. My question is that these.
Speaker 2 (14:15):
Companies don't hire younger developers to learn who will shepherd
the AI when the current developers retire or leave. And
I think Rad's referring to this idea that we don't
need junior developers anymore because the llms are going to
do it, which I think is passed for how long.
I think it's patently false. Yeah, right, Like it's just
not the case. I think what junior developers need to
know and what every developer needs to know is starting
(14:37):
to change with these tools, and arguably the junior developers
will be more responsive to this because you know, in
the end, our job wasn't the right code. It was
to provide solutions to customers. And we've been teaching junior developers, well,
that's you do that by writing code, and I think
we're just gonna be teaching them something different, you know,
(14:57):
as these new tools start to merge. It's it's pretty
raw stuff, but it's a great shocking point about what
the life cycle a developer looks like with these tools. Yeah,
involved as well, right, and and who you know what
it really means to shepherd AI going forward and just
want to talk about here. I'll leave it for the show.
Speaker 1 (15:15):
Brad.
Speaker 2 (15:15):
Thank you so much for your comment, and a copy
of Music go By is on its way to you.
If you'd like a copy of Music go Buy, write
a comment on the website at dot NetRocks dot com
or on the facebooks. We publish every show there and
if you comment there and a read on the show,
we'll send you copy Music go By.
Speaker 1 (15:28):
I told you that I've been working on the next
version of Music to code By, the next track, right,
mm hmm. Yeah, so it's pretty good. I think I
sent you a little sample. I think you have and
I don't know that I've had a chance to open it.
Traveling hither and you're on right now? Yeah, yeah, okay, Yeah,
it's pretty cool. It's different, you know, And that's the
that's the thing. I try to make every track a
little bit different because well, you know, you're a bit older,
(15:48):
you want to switch it up. I think you're you know,
you've been doing this for a while, Fred, so I
mentioned things have changed, you know, yeah, and waiting your
approach sure, I'll give it a listen music to co
by dot net if you want to get those tracks,
twenty three of them to be exact in waveflak or
in MP three format. Okay, so let's bring back Tom Karkov.
(16:09):
He's been on the show several times. Tom's a senior
software engineer at Microsoft working on the data plane for
Azure API Management, helping customers successfully scale their API landscapes.
He has been working in the cloud native space for
more than seven years. As of now, auto scaling is
in his DNA and is an alumni maintainer of Kubernet's
(16:33):
Event Driven Auto Scaling or KEDA kDa. Then that's a
CNCF graduation project that makes application auto scaling on Kubernetes
dead simple. Prior to joining Microsoft, Tom was recognized as
a Microsoft Azure MVP an advisor from twenty fourteen to
(16:53):
twenty twenty one, a CNCF Ambassador from twenty twenty to
twenty twenty three, and was one of the first gid
up stars. He is famous for turning coffee into scalable
and secure cloud systems and writes about his adventures on
blog dot Tom Karkov be and That's k e r
K h O V E. Welcome back, Tom, Thank you
(17:16):
very much for having me. You bet good to have
you back.
Speaker 4 (17:19):
Friend, always fun.
Speaker 1 (17:20):
Any comments on anything you've heard so far.
Speaker 4 (17:23):
Yeah, AI definitely is changing things. It's also making it
easier to learn and to explore. And also I had
a small accident a half a year ago, so it
was not really able to type with one hand.
Speaker 1 (17:40):
Oh no, But it was.
Speaker 4 (17:42):
Interesting to see how you can use AI or good
old speech to text and how powerful it is nowadays
to help you mind blowing.
Speaker 1 (17:52):
Are you all recovered?
Speaker 4 (17:53):
Semi?
Speaker 1 (17:54):
So?
Speaker 4 (17:54):
I can move almost my finger a bit, but not
fully so pressing control is a bit hard, but there's
more things in life.
Speaker 1 (18:03):
Do you play an instrument?
Speaker 4 (18:05):
Thankfully?
Speaker 1 (18:05):
Not thankfully, but you could always play the kazoo.
Speaker 4 (18:13):
My wife does not approve, but.
Speaker 1 (18:17):
I appreciate that you started.
Speaker 2 (18:19):
You'd use this as an opportunity experiment with the voice interfaces,
so you're writing code by voice.
Speaker 4 (18:24):
So I used to use a speech to text to
write my emails and it didn't make any mistakes, so
that was very interesting. I tried to use Copilot and
Visual Studio back then, but there was no voice option,
but VIAS code had it back in the day, so
I just gave it prompts to a voice. And it
also made me realize how important it is to think
(18:46):
about these kind of features so that everybody can use
these tools.
Speaker 2 (18:50):
Yeah, you're going to be an accessibility advocate now I
think you've had You've now had the pain.
Speaker 1 (18:55):
Yes, yes, that's really interesting.
Speaker 4 (18:58):
Now I know how it is to be able to
do it. So that was interesting.
Speaker 2 (19:02):
Yeah, we we. I think we three value our hands
just about more than anything. Yes, like you think about
what we do all day. My god, I need my
hands exactly totally.
Speaker 1 (19:14):
Yeah.
Speaker 2 (19:15):
But even been the API guy for forever, right, and
I only see that as getting more Importantase. I can't
tell you how many times we have this conversation about
if you're not using as your API management, you're making
a mistake, like you expose that API to the world.
There's all kinds of punishing things that are going to
happen to you unless you have a tool you know
meant for that problem.
Speaker 4 (19:36):
Yes, clearly I'm biased, but before I joined Microsoft, I
was a consultant and it was kind of one of
the default tools in my tool chain because sometimes people think, oh,
I don't need this, but later on they would notice that,
oh I could have used it, but now it's not
in place, so now it's a problem to shove it in.
Is it challenging to retrofit and theory? No, but like
(19:59):
you need to talk to different endpoints. If you don't
have DNS properly set up, that's kind of breaking change
for your customers, et cetera, et cetera. So if you
don't think about those things upfront, that can be problematic.
But yeah, I agree. It helps you govern all your
APIs and to secure them and keep things under control,
(20:19):
which is usually a problem if you're growing like fast
as a company.
Speaker 1 (20:24):
Yeah, for sure. Some things people don't think of when
they expose an API layer is denial of service attacks
against your API. Yes, you know, because if you have
an endpoint out there that's answering the door no matter
who comes to the door, and an army shows up,
you've got a problem.
Speaker 4 (20:39):
Yeaheah, it's fine until it isn't. Yeah, then go figure
out how to solve it. If you don't have a
solution like this.
Speaker 1 (20:45):
Even when that DDoS attack is an over diligent developer
who's written some bad software.
Speaker 4 (20:51):
Oh well, I've seen things before I joined, Like people
get to know what a health endpoint is, so they
eagerly start adopting it and then they create their own
videos attack and they cannot turn it off, so they
need to figure out which machinef to shut down to
avoid bringing their customers down.
Speaker 2 (21:12):
That's an interesting one and that's one of the things
that as your API is really good for is that
you can hey, take that one out. You know, in fact,
you can set you set a policy places it's over there,
just shut that one off and let me know.
Speaker 1 (21:24):
Yeah.
Speaker 4 (21:24):
Ballacies are a great way to rate limit at a
validation of the payloads at security logging, KPI tracking, et cetera.
And the second thing we have is like API analytics.
So if you were to have one customer that's very
aggressive and you need to figure out like who is it,
(21:44):
that report would basically tell you, oh, it's Richard or Carl,
we need to reach out or probably make access.
Speaker 1 (21:54):
Yeah.
Speaker 4 (21:55):
So and now we have as of built I think
last year, we have an AI gateway and that one
is getting a lot of traction. So you can also
put it in front of your AI models and workloads.
Speaker 1 (22:09):
You mean, if you have exposed an ll M via
an endpoint. Why would you do that?
Speaker 4 (22:16):
Yeah? For example, if you have asure open AI, but
you need to kind of rate limit the tokens for
that service or balance across accounts, you can now do
that through our gateway as well.
Speaker 1 (22:30):
So there's still security there.
Speaker 4 (22:32):
Yes, there's security. There's rate limiting, monitoring, et cetera. So
that's growing a lot as well. So, like I can imagine,
we keep joking we used to be an API gateway,
but now the piece between brackets because we're also in
AI gateway Now.
Speaker 2 (22:50):
Yeah, I mean it's just a subsection in the documentation
for the Azure API management but it's I bet it's
got a lot of demand on it right now. Lots
of people are trying to figure out what the product
is here.
Speaker 4 (23:01):
Yeah, it's it's been fun lately. I wouldn't say quiet,
but interesting.
Speaker 1 (23:06):
Interesting, we've been busy.
Speaker 2 (23:08):
What I mean, what makes the is the the tricky
part of the AI gateway. Now you're measuring tokens like that,
that's the issue. I want to rate limit on tokens?
Speaker 4 (23:17):
Yes, correct, Yeah, so measuring tokens uh to rate limit
but also kind of have metrics. Who's using how many
tokens were to have those insights, Yeah.
Speaker 2 (23:29):
All the same things I want when with my a
PI utilization it's like, you know, this person is making
x many calls per second, or yes, they're doing the
most expensive calls like that, that kind of thing, just
sort of having that shape of what the utilization looks
like per customer exactly. Yea, yeah, all right, because yeah,
if you're if you're if those tokens roll uphill and
(23:51):
you're paying for them and then you know, flat rating
to the customer, this is a good way to bankrupt
your company a big old hurry exactly.
Speaker 4 (23:58):
So that's that's why we try to help protect yourself
from running out of money.
Speaker 2 (24:03):
Yeah, finding out all of a sudden you yeah, I
don't think. I think, yeah, I suddenly get a big
bill at the end of the month. It's like, guess
what your customers did.
Speaker 1 (24:13):
Exactly. That's what probably also a good place to log
the prompts, right, Yes, so you can do that. You
could say, hey, we ran out of token, so let
me see what everybody's sending, and then they're like, you know,
what's the nearest burger king or whatever, Hey, you don't
use it for that.
Speaker 4 (24:31):
Or how often do people say thank you?
Speaker 1 (24:34):
Yeah? Right. There was a whole article about how much
thank you was costing in tokens.
Speaker 4 (24:39):
Yes, it's burning a lot of compute just to process
thank you, which is being polite in case the robots
start to wake up.
Speaker 1 (24:47):
But I don't know money. I think it helps us
when we're polite to the models.
Speaker 2 (24:53):
Yes, I got it. I got to tell you. I've
read some prompt chains for coding where it's like, dude,
if this wasn't software, this is an HR violation. Why
are you communicating to this tool so obnoxiously?
Speaker 1 (25:08):
Yeah? Right, I told you that. My My wife's favorite
thing to tell she who starts with a you know,
the echo, but is shut up, not stop. She just
says shut up and it and it works makes her
feel good. I guess.
Speaker 4 (25:27):
I don't know.
Speaker 1 (25:29):
Google.
Speaker 4 (25:30):
Google responds interesting. So once was an in traffic jam
and then Google asked me how can I help you?
And I said, oh, go away, like not so politely,
and it said, oh, I know I'm not a human,
but you still need.
Speaker 1 (25:43):
To be polite, but be nice.
Speaker 4 (25:45):
It's like there was like, okay, interesting, good point.
Speaker 1 (25:48):
Yeah do I do? I? Well, like I said, I
think it makes us humans feel better about ourselves when
we're just polite as a habit. I agree. Yeah, No,
I think that, you know, negativity is contagious, Yes, you know, ye,
and so I think think it's a thing.
Speaker 4 (26:06):
I guess it comes. It will also improve you, Like,
these agents work better the more context you give it,
and if you're more polite, you will naturally give more
context and explain more rather than being rude. That's not
going to be useful anyway.
Speaker 2 (26:21):
Yeah, route tends to be terse where politeness tends to
be more or moose, and that additional information should get
better results exactly. Yeah, that's an interesting point on that
as well. It's just like, hey, it makes the room nicer. Sure,
I'm just thinking about a bullpen of developers, you know,
a dozen people working together and they're all cursing at
software at the same time.
Speaker 1 (26:44):
What kind of room is that. I don't know if
I want that room, I want a better room than that.
Speaker 4 (26:48):
That's why they invented the term war room.
Speaker 5 (26:50):
Yeah, yeah, well there is, there are real war rooms,
you know, and I think I think we just borrow
that in software from the military.
Speaker 1 (27:02):
We had a warm room at a place that I
worked in California in the nineties. It was before Carl
and Gary's went up. So, but it was my idea.
It was this is a really weird experience. I basically said, hey,
let's make this extra office that nobody's using the warm room,
and if somebody has a problem, you go in there
(27:25):
and try to solve it with them, and you know,
shut the door, and then when you come out, you
don't come out until you have a solution. So one
developer said, Okay, Carl, let's go into the warm room.
I have a problem. I said, what, And he goes,
do you have an agenda? Like what? Oh? Man, that's weird.
(27:51):
Be careful what you ask for. Yeah, I know.
Speaker 2 (27:53):
And nothing's free, man, it comes to consequences the case.
Speaker 1 (27:57):
Yeah, why don't we take a break? Yeah, that's good idea.
So we'll be right back after these very important messages.
Did you know that you can work with AWS directly
from your ide AWS provides toolkits for Visual Studio, Visual
Studio code, and jet brainswrider. Learn more at AWS dot
(28:18):
Amazon dot com, slash net slash tools. Hey Carl, here,
you probably know. Text Control is a powerful library for
document editing and PDF generation. But did you know they're
also a strong supporter of the developer community. It's part
of their mission to build and support a strong developer
(28:39):
community by being present, listening to users, and sharing knowledge
at conferences across Europe and the United States. So if
you're heading to a conference soon, check if text Control
will be there and stop buy to say hi. You
can find their full conference calendar at dubdbdb dot textcontrol
dot com and make sure you we thank them for
(29:00):
supporting dot net Rocks. Now we're back. It's on at
Rocks Amateur Campbell. It's called Franklin.
Speaker 2 (29:07):
Hey, talking to our friend Tom, who spends part of
his life in API management. But now you're more focused
on sustainability. But I guess API managements about sustainability as well, right.
Speaker 4 (29:17):
Yeah, So we've been spending some time on this because,
like we just mentioned, like we front all these APIs
and now these AI models as well, and we kind
of have a good position to help customers who want
to reduce their emissions. Interesting, like, we're at the heart
(29:39):
of these platforms and we know where the traffic is
going to route, so we have a very good opportunity
to make sure that the traffic that gets routed to
the back end can for example, be routing to a
region where it is daytime versus night, which means right,
if they're more solar. If we map them to the
(30:01):
Azure data centers, for example, we know what the regular
emission is in those regions, so we can say, oh,
this one now runs dirty energy. Let's shift the traffic
to another region where we know it's screen energy. And
that gives you the benefit of if they use auto
scaling on these back ends, they can really shift their
whole traffic there so they can scale in and reduce
(30:25):
their emissions.
Speaker 2 (30:26):
And it's an interesting truth that data centers have more
energy available during the day, presumably largely due to solar
so arguably could even be cheaper. You know, we could
be chasing the lowest cost workloads around as well. I've
always thought that API management was a great point for
measuring like cost of transaction, but you're also talking about
emissions per transaction as well. Is Microsoft, We've seen a
(30:50):
lot of work being done and Carl you've been a
part of that with the new version of Polly where
they're re engineering software A to scale enormously, so just
to be more aware of what that utilization looks like
at scale, because they're looking at millions billions of utilizations
possibly per second, and that does every cycle costs money,
(31:11):
every cycle lost money. Man, So yeah, intriated, all these
things work together.
Speaker 1 (31:17):
Stein of Energy, Richard, do you want to give us
a little what am I trying to say a little
preview of the end of year energy geek out? Anything
that you'd like to talk about in terms of energy
methods or generating energy from you know, to power these
AI data centers. That might be a good idea. Well,
(31:39):
I'm there's a whole section.
Speaker 2 (31:41):
You know, I work off my notes for the end
of year geek outs for months, so I have a
whole section about what all the tech companies are doing
to deal with the power requirements for AID data centers.
It's extraordinary. And you know, we've talked about this before
and I don't want to get too far into it now,
but power companies are all sure conservatives. They don't tend
to build new technologies. That's not what power companies do.
(32:03):
They're in the reliability business. Tech companies like building new things.
So an interesting debate here about whether it makes sense
for tech companies to actually mature power technologies, mostly because
the three spending money on them that'll be available to
the greater public in the long term. So yeah, stay tuned.
Speaker 1 (32:24):
We have a nuclear power station in my town. I'm
at the north end of the town. It's at the
south of the southern tip, right on the water. And
this we heard rumors floating that somebody wanted to op
I think it might even be Microsoft might have wanted
to open a data center or build a data center
right next to the power nuclear power plan. Sure interesting,
(32:47):
you know, and that's really thinking ahead because if you're
going to use wads and wads of energy, you might
as well be as close to the source as possible.
Speaker 2 (32:55):
And it reminds us that data centers in many ways
are the new heavy dust street. You know, it's super
normal to build an aluminum smelter or a steel plant
beside the power generation source because you use so much
of it. So you know, this new heavy energy building
it alongside its power makes a lot of sense. But
that's another show, and.
Speaker 1 (33:16):
Another show I don't want to do rail Us and
another show that's coming up soon. Yes, yes, the end
of the year is upon us.
Speaker 4 (33:24):
So it's interesting though how much progress they've made on
the data center side to be more efficient. Like, I'm
not an expert on that, but like we now have
AI data centers they announce that ignite it are fully
closed water systems where they pour in like households of
(33:47):
water that they consume usually in the closed system, and
it stays in there for years. Right, because it's closed loop,
you're not like losing any water.
Speaker 2 (33:58):
Yeah, they're reusing water over over again, she was more complicated, doue.
Speaker 4 (34:02):
Yeah, they're getting rid of concrete by kind of reinventing
types of wood to make it more sustainable. It's really interesting,
mind blowing, Yeah, if you dive into that area.
Speaker 1 (34:14):
Yeah.
Speaker 2 (34:15):
And the advantage here, of course, is that tech is
not a low margin business, so you can afford to
spend more money knowing you're going to earn out over
time with these data centers. But I think this all
starts with being able to measure utilization. Yes, it's crazy
to think about this in context of an API, but
it makes perfect sense.
Speaker 1 (34:32):
It is the logical unit of measure.
Speaker 4 (34:34):
Yeah, so we don't expose the kind of the carbon
emission directly, but if you want to measure things in
azure there's a service called Azure Carbon Optimization which shows
you the emission by Azure compute workload and it gives
you kind of a recommendation and it says, for example,
(34:54):
it's also good for cost saving. Oh, you're using this
vmsas instance, with this vmskew, this is your emission for
this service, but you're utilizing only like twenty percent. We
recommend you using this skew or that skew or all
these kind of things. So that is good for the
kind of reporting side. Interesting, where are we over provisioned?
(35:17):
And then what we're trying to help with is to
kind of route your requests to the regions where it's
more efficient. But there's also a second aspect, like we
have all these policies that you can run kind of
this logic on your gateway before going to the back end.
So we've introduced capability that in your policies you can check, oh,
(35:40):
the emission is high. So let me not log all
these big payloads which require more compute on the gateway
or puts pressure on service bus or event hubs in
that region. Let's just skip this section and reduce the
kind of the overhead in this region. If it's not
required so that the gateways can also scale in.
Speaker 2 (36:01):
So it's just a recognizing there are optional workloads that
could be time shifted, the location shifted, yes, to change that.
Speaker 4 (36:08):
Like auditing metrics. These kind of things are not always
mission critical. So I always compare it with what teams
did during COVID right, they were overloaded, they did not
have the CPUs, so we did it small things like
remove the Thomas typing indication, which costed a lot of CPU.
(36:28):
And there's a blog from I think Mark Rosinovich that
explains the impact from disabling that feature on the compute
that required. Wow, it's kind of our flavor of this
for your APIs Yeah.
Speaker 2 (36:42):
Yeah, No, I think it's interesting to think in terms
of you've got a company with a green initiative like this,
and you can start working through your software to say
where can we make an optimization. There's a case study
begging here to say, well, which which of these has
the largest impact?
Speaker 1 (37:02):
Where do you even start? But I think the number one,
of course is measure is once you measure, the obvious
ones will fall out. Yeah.
Speaker 4 (37:09):
Measure and adopt auto scaling would be another one. I
would say, like a lot of people are over provisioned
or maybe like you don't need to be over provisioned
all the time. Like most cases, customers can just start
scaling out during the business week, but in the weekend
they can scale in because nobody's in it.
Speaker 2 (37:29):
Yeah, if you still got the old reflexes, which I'd
have to admit I still do. Like we always when
we're building out our own infrastructure, you provision for peak,
which is to say, you over provision for everything else, Yes,
so that you can get through you know where we're
recording this right around the time Black Friday is coming,
so talk about peak, right, and so they you know
(37:51):
the idea that really elasticity works both ways, not just
scaling up but scaling down. Yes, but autoscale is not
a product per se, right, Like there's a lot are
things that go into auto scale.
Speaker 4 (38:01):
Yes, that is true. There's more things you can do though,
Like it doesn't always have to be like this intense.
Let's say, like architecture platforms like this, because of course,
if you shift traffic, the kind of the new region
where all the traffic is going needs to be able
(38:23):
to handle it, because otherwise you're going to create your
own outage. So that's why if not a single back
end is kind of healthy, we'll just keep routing it
there anyway. But as a developer, what's one of the
most common thing that happens. I check in my code,
I have a CI trigger, etc. In a lot of cases,
(38:44):
people have a lot of pipelines starting out, but nobody
looks at them. Right, that's a lot of compute as
a company or burn So maybe revised. If you really
need all of those CI pipelines, or make them on the.
Speaker 1 (38:57):
Nd maybe you only activate the pipeline on Friday or
something exactly.
Speaker 4 (39:02):
So if it's not like that urgent, maybe don't do it.
Speaker 2 (39:06):
Yeah, you guys are countering like a decade. Now I'll
go faster, go faster, Go faster, go faster. Right, it
depends if you're using these pipelines. Fine, but I've seen
a lot of people that once create a pipeline, then
never look at it anymore.
Speaker 1 (39:22):
Question right, Well, I kind of think that was the
lore of doing that. It's like, yeah, automate as much
of that work as possible so that you don't have
to worry about it. But now we're worrying about it
because it's costing a lot of cycles and money that
you know comes down to money.
Speaker 4 (39:40):
It is a trade of like is it useful or
is it something that we actually don't really need anymore
or run less often?
Speaker 1 (39:49):
Maybe remember we used to have build Friday's Richard, yep,
you know, back when we thought a build master was
a good idea. Oh yeah, build server. We had to
build server. Yeah, but you know we're happy with just
checking in code and everybody's synced up and all that stuff.
And when we want to have a release, you just
(40:11):
publish it and it's yeah, we can get that down
to just one command now right.
Speaker 2 (40:16):
Yes, well so, but to Tom's point, you also get
to the situation where everybody's pushing builds constantly and then
somebody's deciding to actually put it out, and so in between,
all those in between builds were kind of unnecessary. So
you know, the advantage of automation is that it's also
then tunable. So it would be that hard to go
(40:37):
back in there from a macro perspective and say, how
many of these builds actually get used?
Speaker 1 (40:42):
Can we delay? You know, when does it make sense
actually push? And then with the consequence of that, I.
Speaker 4 (40:47):
Think CI is one aspect. There's also the situation where
you have scheduled pipelines where the tests every hour or
every two hours, but how often do you really look
at these tests? Like maybe daily is enough?
Speaker 1 (41:02):
Right? How important do you see agency in DevOps be
becoming a partner so that you could give it a
set of rules that it could monitor and make its
own decisions as to when to build based on those rules.
Because that's really what you're doing when you when you're
figuring out strategies and and doing it yourself. So I
(41:25):
mean when you see that.
Speaker 4 (41:27):
I think that that's something that will come as a
next step. Maybe not as the next step, but eventually
one Like we push all these cis how many devs
actually check if it regresses the tests, then everything accumulates
and then you have x failing tests. Now go figure
(41:47):
out what what what's the problem or who to figure out?
Who to solve this problem? I think that's where agents
will become a theme where they kind of auto three
IG identified the issue. Yeah, picking the people alto rovert
or whatever. That's going to help a lot.
Speaker 2 (42:05):
On run as I just put out a show on
the Azure software Reliability Engineering agents, which that's literally what
their job is is to pick up events in it
that occur and a deployed piece of software and analyze
them even to the point of making co change recommendations.
Speaker 1 (42:23):
This is all in the back end in Azure. It's
all in Azure. Yeah.
Speaker 2 (42:26):
Yeah, these agents, I mean, it's the still miles to go.
These are early versions, but it does speak to I
think exactly what you're describing to them that you will
have from you know, you know, as a guy who
spent time on a Saturday trying to keep an e
commerce side alive. The fact that you can have an
error and these tools will at least do the initial
(42:47):
homework right that by the time you're looking at it,
it's already done. The pull of this was the state
of the machine, This was the current load levels. Like
all of the infos, we at least have a shape
of what was this problem.
Speaker 1 (42:57):
Yeah, and at least they could recommend a few courses
of it and you didn't even have it to it.
But you're making decision and that's yeah, that's where we
are today. I like that. I like being the decider.
I'm the decider.
Speaker 4 (43:09):
So you're talking about it as youre s a reagent.
I presume that's right. Yes, that is a very that's
something we're onboarding as well. Like a lot of times
it just reboot a machine and monitor its progress or whatever.
It's perfect for all these kind of things.
Speaker 1 (43:28):
Yeah, for sure.
Speaker 4 (43:28):
Reducing all that toil, it's really great.
Speaker 2 (43:31):
Yeah, if the answer was just reboot the machine and
the problem will go away. The fact that you can
have a tool that can get to that chain and
do it for you, yes, and so by the time
you know about it, it's already happened and recovered itself.
Speaker 4 (43:43):
Yeah, and then it will start aggregating all these symptoms
and kind of unfold the bigger problem. It's like, why
do we need to reboot all these machines. Then it
will kind of summarize all the symptoms and say, probably
your problem lies there, now go fix it.
Speaker 2 (43:56):
Yeah, memorily here, or do dispaced there, that kind of thing.
Speaker 1 (44:02):
You know, It's always funny when the software acts and
then notifies you.
Speaker 2 (44:05):
Like back in the old SQL server hot failover days,
like you would just get a notification databases failed over,
and it was chilling because it's like, well, why did
you fail over?
Speaker 1 (44:20):
You know you're going to fail again. Reminds me of
the early days of Azure Web Services when you because
they didn't understand how to deal with the memory leaks
or things like that, there was no stuff built in
on the server. The recommend mended course of action was
(44:41):
to automatically restart the service on a schedule, just you know,
and that was a way that you could out of sight,
out of mind memory leaks or other problems. Yep, you know,
turn it off, turn it back on. The universal fix.
All right. How much is security in the back of
your mind or the forefront of your mind is to
(45:02):
work your way through these things APIs and AI in
all of that.
Speaker 4 (45:08):
That's a good question, Carl. So in the past half
a year we've seen the sprawl of MCP servers spinning
up its very early days on the kind of the
authentication specification for this. But that's actually one of the
aspects that we can help with with Azure Api Management
(45:28):
is you can expose your APIs as MCP servers, and
we have Azure Api Center, which is basically your internal
MCP catalog or API catalog that can help you keep
track of all of these things. And the beauty is
that because you expose them to Azure Api Management, you
(45:48):
can reuse those policies and say I want to have authentication,
I want to have weight limit, et cetera. So you
still can use these MCP servers inside your organization, but
you can still have kind of your enterprise policies on top. Yeah,
(46:10):
that policies and control on top that you have as
a standard in your company to make sure it doesn't
become a liability.
Speaker 1 (46:17):
Yeah, and that's all good stuff. What I'm thinking about
is giving an MCP server, which is really an agent, right,
giving it agency to act on your behalf things that
you know, maybe some people don't want it acting that
way on your behalf. Maybe that could pose a problem,
(46:40):
especially with over hyperactive, hyperactively helpful lms like you know,
Claudes on It for example, loves to do things that
you didn't ask it to, and if it had if
it had the power to send emails or to read emails,
(47:01):
it could infect a system with now where like, there's
so many problems when you give an agent agency to
do things that might seem very convenient for you but
could open up security holes.
Speaker 4 (47:17):
I think authorization is a very important point here, Like
even if it would still have access, is it still
authorized to perform the action right, right, and as you
expose those APIs, like one of the things we allow
you to do is like maybe you have an API
(47:37):
that says get an order, create an order, and cancel
an order, and then the back office of your store.
But when you expose the MCP server, you have the
option to only expose get order, maybe create order, and
the rest. It doesn't like, even if it would have
access to it, we're not going to expose the API
(47:58):
even so it's not able to haul it right. And
I think that's the second aspect, is that we used
to have all these open eyes spects with the operations.
Now with this tooling, which is not a human person,
we need to be more careful what are we even
giving it access to.
Speaker 1 (48:14):
For example, giving a SEQL server MCP the ability to
generate queries and run queries.
Speaker 4 (48:23):
That is very dangerous.
Speaker 1 (48:25):
Yikes. I mean, think about that, generate t sql and
then execute it. Unless you say, the only thing you
can the only command you can create is select.
Speaker 4 (48:37):
Is drop table.
Speaker 1 (48:39):
If you say select is the only thing you can
do and you can't do anything that's going to act
you know, drop table or delete or anything like that.
You know, but you're still still untrustworthy, I think.
Speaker 4 (48:53):
Very dangerous, and also enough material for the LM to
have trained on to learn what not to do, you
would hope, So that is definitely yeah. I think that
comes back to the point I was making, like, don't
just expose everything. Think carefully, do reviews of what you're exposing.
Because it's not a human anymore. It's something that you
(49:13):
will ask And like you mentioned, some models are eager
to help, but that means you, like, maybe they're going
to do things you don't want.
Speaker 2 (49:23):
Anyway, I do appreciate that, you know, getting back to
as your API management. It's like you have to go
down the ENTRA pipeline to even use this, right down
to sort of the B two C model where your
customers have to have identities and so forth. You're forcing
us into the pit of security success. You want to
use these tools, you've got to follow zero trust rules.
You've got to properly authenticate everybody. It's that authorization and
(49:45):
policy in place, and it's a paint in the butt.
It's part of the challenge of getting started with these things.
It's also way beneficial when stuff goes wrong.
Speaker 4 (49:56):
Yes, and I was not part of the screw, so
I cannot kind of get into the details. But and
ignited last week when we're recording this, foundry was a
big thing. And as part of Foundry or AI gateways integrated,
which means every agent now hasn't dedicated entra id identity. Right,
(50:19):
So we used to be humans who had like a
personal email address with an identity. Now agents have the same.
So that kind of gives you the next step of
putting these guardrails around these agents to make sure they're not.
Speaker 2 (50:34):
And I hate to can't anthropomorphize on that because we've
also been giving applications identities as well for exactly the
same reason, yesh, right, But.
Speaker 4 (50:42):
For agents it was it was more it was harder
to achieve this and it was not really like dedicated
to the model model, like you had to kind of
configure it that way. But in this case, it's like
fully dedicated to this agent, let's say. But again I
did not work on that, so I need to defer
that to my colleagues. But I think that's a good
(51:03):
step forward for security as well.
Speaker 1 (51:05):
Tom, is there anything that we missed, anything else you
want to talk about before we wrap it up?
Speaker 4 (51:09):
No, So if somebody is interested. We have a blog
post which we can maybe onto the show notes where
you can sign up to use these sustainability capabilities. There's
a ton of announcements from Mignite. We also shift some
of the AI capabilities which you can find on the blogs.
But if you have any questions, just reach out to
(51:31):
me on LinkedIn. Happy to excellent answer.
Speaker 1 (51:33):
Your question, Tom. Thanks, It's always interesting when we talk
and this was no different. Thanks very much.
Speaker 4 (51:39):
Thank you very much. Have a good day many all.
Speaker 1 (51:41):
Right, we'll talk to you next time on dot net rocks.
(52:04):
Dot net rocks is brought to you by Franklin's Net
and produced by Pop Studios, a full service audio, video
and post production facility located physically in New London, Connecticut,
and of course in the cloud online at pwop dot com.
Visit our website at d O T N E t
R O c k S dot com for RSS feeds, downloads,
(52:27):
mobile apps, comments, and access to the full archives going
back to show number one, recorded in September two thousand
and two. And make sure you check out our sponsors.
They keep us in business. Now go write some code,
see you next time. You got Javans
How'd you like to listen to dot net rocks with
no ads? Easy? Become a patron for just five dollars
a month you get access to a private RSS feed
where all the shows have no ads. Twenty dollars a
month will get you that and a special dot net
Rocks patron mug. Sign up now at Patreon dot dot
NetRocks dot com. Hey guess what it's dot net Rocks
(00:37):
episode nineteen hundred and seventy eight. I'm Carl Franklin and
am Richard Camp and Richard, you're a down under?
Speaker 2 (00:44):
I am?
Speaker 1 (00:44):
I am.
Speaker 2 (00:44):
This is the last week of the month long tour
with the grand baby. And you know, the whole idea was,
while she's not moving around too much, just a little package,
it'll be a good time to take her and show
her off to the extended family. Of course, in this
time she's learned to crawl, so now she's a toddler
and starting to get bumps and bruises. Just you know,
remember that time your kids get all.
Speaker 1 (01:04):
Your baby get gets all banged up. That's happening. How
cool is that? It's been really fun. The only thing
she might regret is not being able to remember any
of it's the best part.
Speaker 2 (01:13):
Right when she's twelve years old, like you've already been
to Australia.
Speaker 1 (01:19):
Yeah, she won't remember the trauma of being dropped on
her head on the plane, you know, nothing like that.
Speaker 2 (01:26):
Well, we've got a few more days left. I'm actually
up near Brisbane.
Speaker 1 (01:31):
We're in the.
Speaker 2 (01:31):
Gold Coast, an area called kulan Gata, right by the beach.
It's beautiful here. I love the Gold Coast boys.
Speaker 1 (01:37):
It hot holy man. Yeah, all right, So let's start
right off with the year nineteen seventy eight, right, dude,
what do you got? Well, some of the major events.
Harvey Milk was elected the first openly gay person elected
the public office in California, so you know, things are
becoming a little more socially liberal. The Jonestown massacre on
(02:00):
November eighteenth was terrible, crazy. We're nine hundred members of
the People's Temple died in a mass suicide murder in Jonestown, Guyana,
orchestrated by Jimmy Jones in the good News Camp David Accords.
In September, President Jimmy Carter facilitated a peace agreement between
Israel and Egypt. There were some air flight crashes Air
(02:22):
India Flight eight fifty five Boeing seven forty seven coasts
crashed off the coast of Bombay. Two hundred and thirteen
people perished. The northeastern US blizzard of nineteen seventy eight.
You weren't around for this, Richard, because I was on
the West coast territory. Do you remember it? I do
(02:43):
I remember it? So I would have been ten eleven, Yeah,
And we had the day off from school and there
was literally four and a half five feet of snow
right packed up against the door, wow, and all over.
And so my brother and our friend Todd, and I
(03:04):
and Chris Conrandy from down the street, we set up
out to make Skylab, which is what they called it.
Just this huge edifice of snow with snowballs could roll
down and around and everything. We just had a great time,
all right.
Speaker 2 (03:22):
Yeah, sure, And this is right right when Skylab was
going to about to re enter, right, they'd love it
the next year, yep.
Speaker 1 (03:28):
So we were hip little kids even then. The FIFA
World Cup nineteen seventy eight FIFA World Cup was held
in Argentina, with the host nation winning the tournament. Garfield
debuted in the comic strip World Jim Davis made its
first appearance in nineteen seventy eight films Grease, National Lampoon's
(03:50):
Animal House. Of course, those were great. In science, you
can talk about these things. You'll probably talk about the
first human cloning claims.
Speaker 2 (04:00):
Right, yeah, well I'm not going to talk a lot
about it, but yes, that's when those experiments going on.
Speaker 1 (04:06):
Nobel Peace Prize was awarded to Anwar Sadat and monocom
Big and for their efforts in the Camp David Accords.
So what was happening in computing in space.
Speaker 2 (04:15):
Let's do the space stuff first. Last year and seventy seven,
the Shuttle Enterprise was deployed and they were doing their
first captive tests and landing tests. In seventy eight, they
fly the Enterprise by seven forty seven to Alabama going
to do vibration testing for launching. There was the intent
to actually fly Enterprise into space at that time, although,
(04:36):
as we'll find out a couple of years later further on,
there's so many changes that happened from all these tests
that it's too expensive to retrofit Enterprise to be a
flable vehicle. And they'll take an earlier test Article STA
zero nine to nine and that'll and refit in and
it'll become challenger. Otherwise, in space, there was Venus missions
(04:57):
in nineteen seventy eight, both American and so the Pioneer
Venus project launched in May, arise in December. This is
the orbiter, so it actually goes into orbit around Venus
and actually operated until nineteen ninety two doing measurements and
so forth. But there was a second part to it
called the multi Probe, which actually launched later in August,
(05:18):
but arrived earlier because it didn't need to go into orbit,
so it used a higher energy trajectory to get there.
Speaker 1 (05:25):
What did we not know about Venus that we felt
we needed to know by sending those probes.
Speaker 2 (05:30):
We I mean, we didn't have We didn't know the
composition of the atmosphere. We didn't know it all the
way down. That's why putting the landers down to measure
as you went down, we were just trying to figure
out what was going on there, Like even understanding how
hot it was took time. You know, the the blanket
effects of CO two weren't well known yet, so this
(05:51):
was part of you know, the CO two absorbs in
for red light and so that's why it'll concentrate heat.
But also this is like it's down on the surface there,
like what can we see? And so there was a
whole series of missions. They kept going like this is
even the end of it. So on the American side
they put down this multiprobe actually had four probes on it,
(06:11):
a big one that was about seven hundred pounds, called
the Large Probe, cleverly named, and then three small probes
that were all about two hundred pounds. The big one
had a parachute, so it was more like the Venera landers.
It transmitted information for an hour as it was descending
through the atmosphere, but as soon as it impacted the
surface it stopped transmitting. The three small probes were just
(06:34):
arrow shells. They had no parachutes or anything, so they
were free falling granted through a very thick atmosphere. And
one of them, the one they called Day because it
landed on the Day side, Venus continued to function. It
survived the landing and transmitted for over an hour from
the surface.
Speaker 1 (06:50):
So that means there was some kind of solid surface and.
Speaker 2 (06:54):
Without it there is right, it's mostly old volcanic flows.
The Venia missions this were eleven and twelve. So the
Soviets flew to Venus many, many times, and they will continue.
They'll be a thirteen oh fourteen. And so both of
an air missions went off flawlessly. They both landed on
Venus cleanly. Eleven transmitted for ninety five minutes, twelve transmitted
(07:18):
for one hundred and ten. And I'd like to point
out it's not that they only lasted that long. It's
that their transmitters are only so strong, and so the
orbiter has to relay the data back to Earth.
Speaker 1 (07:30):
Oh wow, and it goes.
Speaker 2 (07:32):
Out of range after about an hour and a half,
I see, and then nothing can pick it up anyway,
although generally speaking within it within twenty four hours or
so forth, things are broken from the pressure of the
heat and the amount of sulfuric acid in the atmosphere.
Speaker 1 (07:47):
Yeah, there is that little detail, little detail.
Speaker 2 (07:50):
This is also the year that Sharon, the moon of Pluto,
is identified by some astronomers just looking at it with telescopes.
It's the moon is massive, of course, it's about almost
an eighth the size of Pluto itself, so really really big,
and of course not great resolutions. We wouldn't get a
(08:11):
good picture of Sharon until the new Horizon spacecraft gets
there in twenty fifteen. Science wise, the RSA algorithms that's RIVS,
Shamir and Aldeman is publicly described in nineteen seventy eight.
So that's craz I think we still depend on.
Speaker 1 (08:26):
To this day. Still is it to this day? Yeah?
Speaker 2 (08:29):
And speaking of computing, this is a crazy year for computing.
So this is the introduction of Microsoft's Altair Basic.
Speaker 1 (08:36):
Yeah.
Speaker 2 (08:36):
Right, nineteen seventy seven was when the Altar came out,
but it's now it's fully released.
Speaker 1 (08:40):
Bill Gates through OUTE that right, Yeah, that's right.
Speaker 2 (08:42):
The Apple two distrive, the five and a quarter drive
gets released for the Apple two and they start making
the twoe. The Intel eighty eighty.
Speaker 3 (08:49):
Six processor kind of an important one, yeah, kind of important.
The sixteenth and sixteenth definitive sixteen bit processor. Yeah, here's
when you don't know the Intel twenty eight to sixteen.
Twenty eight sixteen the first electronically reirasable and reprogrammable ROM.
Speaker 1 (09:06):
Wow.
Speaker 2 (09:07):
So up until then when you when we finally made ROM,
they had to be erased by UV light, so you
would take the chip out of the computer, you would
shine a bright UV light on it to erase it,
and then you'd reprogram it and put it back in
the machine. This was an electronically reprogrammable one E PROM Yeah.
The first e proms. Carter also brought the first computer
into the White House, was in HB three thousand, the
(09:30):
introduction of laser disc Oh yeah, I remember those, remember the.
Speaker 1 (09:34):
Big laser discs. Yeah, and you know what its original
name was.
Speaker 2 (09:37):
You're gonna love this because it is nineteen seventy eight
disco vision.
Speaker 1 (09:41):
Oh no, no, And the first and the first disc
released is Jaws. Yeah, that makes sense.
Speaker 2 (09:50):
Yeah, I want to talk the word Star for CPM. Okay,
that's what it comes out, and it doesn't become super popular,
then it'll become super popular in nineteen eighty one because
word Star cuts a deal with the Osbourne people to
bundle it with the Osbourne one bundling even back then,
Wow and the deck vacks eleven seven eighty And I
(10:11):
wouldn't normally talk who a lot about an individual computer,
but there's so much of the architecture of this machine,
even though it was a mini computer, that sort of
describes how PCs will be built going forward, because it
was really they really worked hard on the instruction set
to maintain backward compatibility so that newer models could build
on the same instruction set pretty much indefinitely. And much
(10:33):
as that machine didn't survive long term, although it was
a phenomenal machine in its era, it really sort of
outlined the ideas of maintaining backward compatibility for better or worse. Yeah,
one last item. This is when Vince Surf and his
team define and test TCPIP.
Speaker 1 (10:49):
Yeah.
Speaker 2 (10:50):
Wow, the stuff we're using right now as we make
this show nineteen seventy eight.
Speaker 1 (10:53):
The TCPIP stack. That's it's the holy grail of computing
communication back then.
Speaker 2 (11:00):
Yeah, and UDP as well. It was all part of
the definition. UNIP is part of the stack. Yeah, and
that's we're using UDP now obviously. Yeah, we do what
we're doing wicked fast.
Speaker 1 (11:10):
All right, Well, uh, what's next? I think we do better?
No framework, right, let's do it. Let's do it. Roll
the music all right, man, what do you go? You're
in Australia right now. I am Our friend Simon Crop
is from Australia and he's up to something else. Now
(11:33):
it's often is yeah, this is uh, he just posts
in our Slack channel. Hey, I wrote a source generator.
Oh cool, of course you did, said, that was my reply. Yeah.
So this is called project files source generator. It's a
C sharp source generator that provides strongly typed compile time
(11:54):
access to project files marked with copied output directory in
the CS projectile. Wow. So a type safe API for
accessing files that are copied to the project's output directory,
eliminating magic strings, and providing intelligence and support for filepaths. Cool.
Now you might think that this is oh yeah, that's
(12:15):
kind of a neat feature. But you know, the more
I dug into the documentation here, the more you know
he highlights these features, and you're like, oh, how could
I have you know, lived without these strongly typed access
to project files. Right? That means you know this dot
dot dot file name compile time safety, so typos and
(12:36):
filepass become compilation errors. I love it. Intel Sense support
so you can discover available files through ID autocomplete. Automatic
synchronization regenerates when project files change, which is a great
feature of a cogenerator. Hierarchical structure that mirrors the project's
directory structure. Support for the glob pattern including star star,
(13:01):
recursive patterns. Smart naming converts file directory names to valid
see sharp identifiers, so you know, file names with a
space in the middle, that kind of stuff get converted
to a dash or whatever you say. Just really smart
thinking about source generation. Yeah, so there you go. Hats
(13:21):
off to Simon as usual.
Speaker 2 (13:23):
And you know, I've been calling LM's lately source generators too,
just re entrant.
Speaker 1 (13:27):
Ones, yeah, and nondeterminist there.
Speaker 2 (13:29):
Yeah, but it's interesting to see a proper source generator,
you know, in this day and age as well.
Speaker 1 (13:34):
Yeah, it's good on ya, Simon. Yeah, cool stuff.
Speaker 2 (13:36):
Who's talking to us today? Richard and I grab a
comment top of show nineteen seventy four. That's one we
did with Den Delamarski or we talked about kid hubspec
Kid because you know, all the AI.
Speaker 1 (13:46):
I realized there's so much we didn't talk to Dan
about that we could have. We could have gone on
for another hour at least.
Speaker 2 (13:53):
Easily, and it probably you know, and we're likely going
to maybe not with Den, but you know, one of
the things I've been doing on this tour talking to
a lot of different developers about how their development workflows
are changing using these new tools and just thinking large scale.
This comment comes from Brad who said during the episode
of the State of the developers will become shepherds of AI.
Speaker 1 (14:12):
Love the term. My question is that these.
Speaker 2 (14:15):
Companies don't hire younger developers to learn who will shepherd
the AI when the current developers retire or leave. And
I think Rad's referring to this idea that we don't
need junior developers anymore because the llms are going to
do it, which I think is passed for how long.
I think it's patently false. Yeah, right, Like it's just
not the case. I think what junior developers need to
know and what every developer needs to know is starting
(14:37):
to change with these tools, and arguably the junior developers
will be more responsive to this because you know, in
the end, our job wasn't the right code. It was
to provide solutions to customers. And we've been teaching junior developers, well,
that's you do that by writing code, and I think
we're just gonna be teaching them something different, you know,
(14:57):
as these new tools start to merge. It's it's pretty
raw stuff, but it's a great shocking point about what
the life cycle a developer looks like with these tools. Yeah,
involved as well, right, and and who you know what
it really means to shepherd AI going forward and just
want to talk about here. I'll leave it for the show.
Speaker 1 (15:15):
Brad.
Speaker 2 (15:15):
Thank you so much for your comment, and a copy
of Music go By is on its way to you.
If you'd like a copy of Music go Buy, write
a comment on the website at dot NetRocks dot com
or on the facebooks. We publish every show there and
if you comment there and a read on the show,
we'll send you copy Music go By.
Speaker 1 (15:28):
I told you that I've been working on the next
version of Music to code By, the next track, right,
mm hmm. Yeah, so it's pretty good. I think I
sent you a little sample. I think you have and
I don't know that I've had a chance to open it.
Traveling hither and you're on right now? Yeah, yeah, okay, Yeah,
it's pretty cool. It's different, you know, And that's the
that's the thing. I try to make every track a
little bit different because well, you know, you're a bit older,
(15:48):
you want to switch it up. I think you're you know,
you've been doing this for a while, Fred, so I
mentioned things have changed, you know, yeah, and waiting your
approach sure, I'll give it a listen music to co
by dot net if you want to get those tracks,
twenty three of them to be exact in waveflak or
in MP three format. Okay, so let's bring back Tom Karkov.
(16:09):
He's been on the show several times. Tom's a senior
software engineer at Microsoft working on the data plane for
Azure API Management, helping customers successfully scale their API landscapes.
He has been working in the cloud native space for
more than seven years. As of now, auto scaling is
in his DNA and is an alumni maintainer of Kubernet's
(16:33):
Event Driven Auto Scaling or KEDA kDa. Then that's a
CNCF graduation project that makes application auto scaling on Kubernetes
dead simple. Prior to joining Microsoft, Tom was recognized as
a Microsoft Azure MVP an advisor from twenty fourteen to
(16:53):
twenty twenty one, a CNCF Ambassador from twenty twenty to
twenty twenty three, and was one of the first gid
up stars. He is famous for turning coffee into scalable
and secure cloud systems and writes about his adventures on
blog dot Tom Karkov be and That's k e r
K h O V E. Welcome back, Tom, Thank you
(17:16):
very much for having me. You bet good to have
you back.
Speaker 4 (17:19):
Friend, always fun.
Speaker 1 (17:20):
Any comments on anything you've heard so far.
Speaker 4 (17:23):
Yeah, AI definitely is changing things. It's also making it
easier to learn and to explore. And also I had
a small accident a half a year ago, so it
was not really able to type with one hand.
Speaker 1 (17:40):
Oh no, But it was.
Speaker 4 (17:42):
Interesting to see how you can use AI or good
old speech to text and how powerful it is nowadays
to help you mind blowing.
Speaker 1 (17:52):
Are you all recovered?
Speaker 4 (17:53):
Semi?
Speaker 1 (17:54):
So?
Speaker 4 (17:54):
I can move almost my finger a bit, but not
fully so pressing control is a bit hard, but there's
more things in life.
Speaker 1 (18:03):
Do you play an instrument?
Speaker 4 (18:05):
Thankfully?
Speaker 1 (18:05):
Not thankfully, but you could always play the kazoo.
Speaker 4 (18:13):
My wife does not approve, but.
Speaker 1 (18:17):
I appreciate that you started.
Speaker 2 (18:19):
You'd use this as an opportunity experiment with the voice interfaces,
so you're writing code by voice.
Speaker 4 (18:24):
So I used to use a speech to text to
write my emails and it didn't make any mistakes, so
that was very interesting. I tried to use Copilot and
Visual Studio back then, but there was no voice option,
but VIAS code had it back in the day, so
I just gave it prompts to a voice. And it
also made me realize how important it is to think
(18:46):
about these kind of features so that everybody can use
these tools.
Speaker 2 (18:50):
Yeah, you're going to be an accessibility advocate now I
think you've had You've now had the pain.
Speaker 1 (18:55):
Yes, yes, that's really interesting.
Speaker 4 (18:58):
Now I know how it is to be able to
do it. So that was interesting.
Speaker 2 (19:02):
Yeah, we we. I think we three value our hands
just about more than anything. Yes, like you think about
what we do all day. My god, I need my
hands exactly totally.
Speaker 1 (19:14):
Yeah.
Speaker 2 (19:15):
But even been the API guy for forever, right, and
I only see that as getting more Importantase. I can't
tell you how many times we have this conversation about
if you're not using as your API management, you're making
a mistake, like you expose that API to the world.
There's all kinds of punishing things that are going to
happen to you unless you have a tool you know
meant for that problem.
Speaker 4 (19:36):
Yes, clearly I'm biased, but before I joined Microsoft, I
was a consultant and it was kind of one of
the default tools in my tool chain because sometimes people think, oh,
I don't need this, but later on they would notice that,
oh I could have used it, but now it's not
in place, so now it's a problem to shove it in.
Is it challenging to retrofit and theory? No, but like
(19:59):
you need to talk to different endpoints. If you don't
have DNS properly set up, that's kind of breaking change
for your customers, et cetera, et cetera. So if you
don't think about those things upfront, that can be problematic.
But yeah, I agree. It helps you govern all your
APIs and to secure them and keep things under control,
(20:19):
which is usually a problem if you're growing like fast
as a company.
Speaker 1 (20:24):
Yeah, for sure. Some things people don't think of when
they expose an API layer is denial of service attacks
against your API. Yes, you know, because if you have
an endpoint out there that's answering the door no matter
who comes to the door, and an army shows up,
you've got a problem.
Speaker 4 (20:39):
Yeaheah, it's fine until it isn't. Yeah, then go figure
out how to solve it. If you don't have a
solution like this.
Speaker 1 (20:45):
Even when that DDoS attack is an over diligent developer
who's written some bad software.
Speaker 4 (20:51):
Oh well, I've seen things before I joined, Like people
get to know what a health endpoint is, so they
eagerly start adopting it and then they create their own
videos attack and they cannot turn it off, so they
need to figure out which machinef to shut down to
avoid bringing their customers down.
Speaker 2 (21:12):
That's an interesting one and that's one of the things
that as your API is really good for is that
you can hey, take that one out. You know, in fact,
you can set you set a policy places it's over there,
just shut that one off and let me know.
Speaker 1 (21:24):
Yeah.
Speaker 4 (21:24):
Ballacies are a great way to rate limit at a
validation of the payloads at security logging, KPI tracking, et cetera.
And the second thing we have is like API analytics.
So if you were to have one customer that's very
aggressive and you need to figure out like who is it,
(21:44):
that report would basically tell you, oh, it's Richard or Carl,
we need to reach out or probably make access.
Speaker 1 (21:54):
Yeah.
Speaker 4 (21:55):
So and now we have as of built I think
last year, we have an AI gateway and that one
is getting a lot of traction. So you can also
put it in front of your AI models and workloads.
Speaker 1 (22:09):
You mean, if you have exposed an ll M via
an endpoint. Why would you do that?
Speaker 4 (22:16):
Yeah? For example, if you have asure open AI, but
you need to kind of rate limit the tokens for
that service or balance across accounts, you can now do
that through our gateway as well.
Speaker 1 (22:30):
So there's still security there.
Speaker 4 (22:32):
Yes, there's security. There's rate limiting, monitoring, et cetera. So
that's growing a lot as well. So, like I can imagine,
we keep joking we used to be an API gateway,
but now the piece between brackets because we're also in
AI gateway Now.
Speaker 2 (22:50):
Yeah, I mean it's just a subsection in the documentation
for the Azure API management but it's I bet it's
got a lot of demand on it right now. Lots
of people are trying to figure out what the product
is here.
Speaker 4 (23:01):
Yeah, it's it's been fun lately. I wouldn't say quiet,
but interesting.
Speaker 1 (23:06):
Interesting, we've been busy.
Speaker 2 (23:08):
What I mean, what makes the is the the tricky
part of the AI gateway. Now you're measuring tokens like that,
that's the issue. I want to rate limit on tokens?
Speaker 4 (23:17):
Yes, correct, Yeah, so measuring tokens uh to rate limit
but also kind of have metrics. Who's using how many
tokens were to have those insights, Yeah.
Speaker 2 (23:29):
All the same things I want when with my a
PI utilization it's like, you know, this person is making
x many calls per second, or yes, they're doing the
most expensive calls like that, that kind of thing, just
sort of having that shape of what the utilization looks
like per customer exactly. Yea, yeah, all right, because yeah,
if you're if you're if those tokens roll uphill and
(23:51):
you're paying for them and then you know, flat rating
to the customer, this is a good way to bankrupt
your company a big old hurry exactly.
Speaker 4 (23:58):
So that's that's why we try to help protect yourself
from running out of money.
Speaker 2 (24:03):
Yeah, finding out all of a sudden you yeah, I
don't think. I think, yeah, I suddenly get a big
bill at the end of the month. It's like, guess
what your customers did.
Speaker 1 (24:13):
Exactly. That's what probably also a good place to log
the prompts, right, Yes, so you can do that. You
could say, hey, we ran out of token, so let
me see what everybody's sending, and then they're like, you know,
what's the nearest burger king or whatever, Hey, you don't
use it for that.
Speaker 4 (24:31):
Or how often do people say thank you?
Speaker 1 (24:34):
Yeah? Right. There was a whole article about how much
thank you was costing in tokens.
Speaker 4 (24:39):
Yes, it's burning a lot of compute just to process
thank you, which is being polite in case the robots
start to wake up.
Speaker 1 (24:47):
But I don't know money. I think it helps us
when we're polite to the models.
Speaker 2 (24:53):
Yes, I got it. I got to tell you. I've
read some prompt chains for coding where it's like, dude,
if this wasn't software, this is an HR violation. Why
are you communicating to this tool so obnoxiously?
Speaker 1 (25:08):
Yeah? Right, I told you that. My My wife's favorite
thing to tell she who starts with a you know,
the echo, but is shut up, not stop. She just
says shut up and it and it works makes her
feel good. I guess.
Speaker 4 (25:27):
I don't know.
Speaker 1 (25:29):
Google.
Speaker 4 (25:30):
Google responds interesting. So once was an in traffic jam
and then Google asked me how can I help you?
And I said, oh, go away, like not so politely,
and it said, oh, I know I'm not a human,
but you still need.
Speaker 1 (25:43):
To be polite, but be nice.
Speaker 4 (25:45):
It's like there was like, okay, interesting, good point.
Speaker 1 (25:48):
Yeah do I do? I? Well, like I said, I
think it makes us humans feel better about ourselves when
we're just polite as a habit. I agree. Yeah, No,
I think that, you know, negativity is contagious, Yes, you know, ye,
and so I think think it's a thing.
Speaker 4 (26:06):
I guess it comes. It will also improve you, Like,
these agents work better the more context you give it,
and if you're more polite, you will naturally give more
context and explain more rather than being rude. That's not
going to be useful anyway.
Speaker 2 (26:21):
Yeah, route tends to be terse where politeness tends to
be more or moose, and that additional information should get
better results exactly. Yeah, that's an interesting point on that
as well. It's just like, hey, it makes the room nicer. Sure,
I'm just thinking about a bullpen of developers, you know,
a dozen people working together and they're all cursing at
software at the same time.
Speaker 1 (26:44):
What kind of room is that. I don't know if
I want that room, I want a better room than that.
Speaker 4 (26:48):
That's why they invented the term war room.
Speaker 5 (26:50):
Yeah, yeah, well there is, there are real war rooms,
you know, and I think I think we just borrow
that in software from the military.
Speaker 1 (27:02):
We had a warm room at a place that I
worked in California in the nineties. It was before Carl
and Gary's went up. So, but it was my idea.
It was this is a really weird experience. I basically said, hey,
let's make this extra office that nobody's using the warm room,
and if somebody has a problem, you go in there
(27:25):
and try to solve it with them, and you know,
shut the door, and then when you come out, you
don't come out until you have a solution. So one
developer said, Okay, Carl, let's go into the warm room.
I have a problem. I said, what, And he goes,
do you have an agenda? Like what? Oh? Man, that's weird.
(27:51):
Be careful what you ask for. Yeah, I know.
Speaker 2 (27:53):
And nothing's free, man, it comes to consequences the case.
Speaker 1 (27:57):
Yeah, why don't we take a break? Yeah, that's good idea.
So we'll be right back after these very important messages.
Did you know that you can work with AWS directly
from your ide AWS provides toolkits for Visual Studio, Visual
Studio code, and jet brainswrider. Learn more at AWS dot
(28:18):
Amazon dot com, slash net slash tools. Hey Carl, here,
you probably know. Text Control is a powerful library for
document editing and PDF generation. But did you know they're
also a strong supporter of the developer community. It's part
of their mission to build and support a strong developer
(28:39):
community by being present, listening to users, and sharing knowledge
at conferences across Europe and the United States. So if
you're heading to a conference soon, check if text Control
will be there and stop buy to say hi. You
can find their full conference calendar at dubdbdb dot textcontrol
dot com and make sure you we thank them for
(29:00):
supporting dot net Rocks. Now we're back. It's on at
Rocks Amateur Campbell. It's called Franklin.
Speaker 2 (29:07):
Hey, talking to our friend Tom, who spends part of
his life in API management. But now you're more focused
on sustainability. But I guess API managements about sustainability as well, right.
Speaker 4 (29:17):
Yeah, So we've been spending some time on this because,
like we just mentioned, like we front all these APIs
and now these AI models as well, and we kind
of have a good position to help customers who want
to reduce their emissions. Interesting, like, we're at the heart
(29:39):
of these platforms and we know where the traffic is
going to route, so we have a very good opportunity
to make sure that the traffic that gets routed to
the back end can for example, be routing to a
region where it is daytime versus night, which means right,
if they're more solar. If we map them to the
(30:01):
Azure data centers, for example, we know what the regular
emission is in those regions, so we can say, oh,
this one now runs dirty energy. Let's shift the traffic
to another region where we know it's screen energy. And
that gives you the benefit of if they use auto
scaling on these back ends, they can really shift their
whole traffic there so they can scale in and reduce
(30:25):
their emissions.
Speaker 2 (30:26):
And it's an interesting truth that data centers have more
energy available during the day, presumably largely due to solar
so arguably could even be cheaper. You know, we could
be chasing the lowest cost workloads around as well. I've
always thought that API management was a great point for
measuring like cost of transaction, but you're also talking about
emissions per transaction as well. Is Microsoft, We've seen a
(30:50):
lot of work being done and Carl you've been a
part of that with the new version of Polly where
they're re engineering software A to scale enormously, so just
to be more aware of what that utilization looks like
at scale, because they're looking at millions billions of utilizations
possibly per second, and that does every cycle costs money,
(31:11):
every cycle lost money. Man, So yeah, intriated, all these
things work together.
Speaker 1 (31:17):
Stein of Energy, Richard, do you want to give us
a little what am I trying to say a little
preview of the end of year energy geek out? Anything
that you'd like to talk about in terms of energy
methods or generating energy from you know, to power these
AI data centers. That might be a good idea. Well,
(31:39):
I'm there's a whole section.
Speaker 2 (31:41):
You know, I work off my notes for the end
of year geek outs for months, so I have a
whole section about what all the tech companies are doing
to deal with the power requirements for AID data centers.
It's extraordinary. And you know, we've talked about this before
and I don't want to get too far into it now,
but power companies are all sure conservatives. They don't tend
to build new technologies. That's not what power companies do.
(32:03):
They're in the reliability business. Tech companies like building new things.
So an interesting debate here about whether it makes sense
for tech companies to actually mature power technologies, mostly because
the three spending money on them that'll be available to
the greater public in the long term. So yeah, stay tuned.
Speaker 1 (32:24):
We have a nuclear power station in my town. I'm
at the north end of the town. It's at the
south of the southern tip, right on the water. And
this we heard rumors floating that somebody wanted to op
I think it might even be Microsoft might have wanted
to open a data center or build a data center
right next to the power nuclear power plan. Sure interesting,
(32:47):
you know, and that's really thinking ahead because if you're
going to use wads and wads of energy, you might
as well be as close to the source as possible.
Speaker 2 (32:55):
And it reminds us that data centers in many ways
are the new heavy dust street. You know, it's super
normal to build an aluminum smelter or a steel plant
beside the power generation source because you use so much
of it. So you know, this new heavy energy building
it alongside its power makes a lot of sense. But
that's another show, and.
Speaker 1 (33:16):
Another show I don't want to do rail Us and
another show that's coming up soon. Yes, yes, the end
of the year is upon us.
Speaker 4 (33:24):
So it's interesting though how much progress they've made on
the data center side to be more efficient. Like, I'm
not an expert on that, but like we now have
AI data centers they announce that ignite it are fully
closed water systems where they pour in like households of
(33:47):
water that they consume usually in the closed system, and
it stays in there for years. Right, because it's closed loop,
you're not like losing any water.
Speaker 2 (33:58):
Yeah, they're reusing water over over again, she was more complicated, doue.
Speaker 4 (34:02):
Yeah, they're getting rid of concrete by kind of reinventing
types of wood to make it more sustainable. It's really interesting,
mind blowing, Yeah, if you dive into that area.
Speaker 1 (34:14):
Yeah.
Speaker 2 (34:15):
And the advantage here, of course, is that tech is
not a low margin business, so you can afford to
spend more money knowing you're going to earn out over
time with these data centers. But I think this all
starts with being able to measure utilization. Yes, it's crazy
to think about this in context of an API, but
it makes perfect sense.
Speaker 1 (34:32):
It is the logical unit of measure.
Speaker 4 (34:34):
Yeah, so we don't expose the kind of the carbon
emission directly, but if you want to measure things in
azure there's a service called Azure Carbon Optimization which shows
you the emission by Azure compute workload and it gives
you kind of a recommendation and it says, for example,
(34:54):
it's also good for cost saving. Oh, you're using this
vmsas instance, with this vmskew, this is your emission for
this service, but you're utilizing only like twenty percent. We
recommend you using this skew or that skew or all
these kind of things. So that is good for the
kind of reporting side. Interesting, where are we over provisioned?
(35:17):
And then what we're trying to help with is to
kind of route your requests to the regions where it's
more efficient. But there's also a second aspect, like we
have all these policies that you can run kind of
this logic on your gateway before going to the back end.
So we've introduced capability that in your policies you can check, oh,
(35:40):
the emission is high. So let me not log all
these big payloads which require more compute on the gateway
or puts pressure on service bus or event hubs in
that region. Let's just skip this section and reduce the
kind of the overhead in this region. If it's not
required so that the gateways can also scale in.
Speaker 2 (36:01):
So it's just a recognizing there are optional workloads that
could be time shifted, the location shifted, yes, to change that.
Speaker 4 (36:08):
Like auditing metrics. These kind of things are not always
mission critical. So I always compare it with what teams
did during COVID right, they were overloaded, they did not
have the CPUs, so we did it small things like
remove the Thomas typing indication, which costed a lot of CPU.
(36:28):
And there's a blog from I think Mark Rosinovich that
explains the impact from disabling that feature on the compute
that required. Wow, it's kind of our flavor of this
for your APIs Yeah.
Speaker 2 (36:42):
Yeah, No, I think it's interesting to think in terms
of you've got a company with a green initiative like this,
and you can start working through your software to say
where can we make an optimization. There's a case study
begging here to say, well, which which of these has
the largest impact?
Speaker 1 (37:02):
Where do you even start? But I think the number one,
of course is measure is once you measure, the obvious
ones will fall out. Yeah.
Speaker 4 (37:09):
Measure and adopt auto scaling would be another one. I
would say, like a lot of people are over provisioned
or maybe like you don't need to be over provisioned
all the time. Like most cases, customers can just start
scaling out during the business week, but in the weekend
they can scale in because nobody's in it.
Speaker 2 (37:29):
Yeah, if you still got the old reflexes, which I'd
have to admit I still do. Like we always when
we're building out our own infrastructure, you provision for peak,
which is to say, you over provision for everything else, Yes,
so that you can get through you know where we're
recording this right around the time Black Friday is coming,
so talk about peak, right, and so they you know
(37:51):
the idea that really elasticity works both ways, not just
scaling up but scaling down. Yes, but autoscale is not
a product per se, right, Like there's a lot are
things that go into auto scale.
Speaker 4 (38:01):
Yes, that is true. There's more things you can do though,
Like it doesn't always have to be like this intense.
Let's say, like architecture platforms like this, because of course,
if you shift traffic, the kind of the new region
where all the traffic is going needs to be able
(38:23):
to handle it, because otherwise you're going to create your
own outage. So that's why if not a single back
end is kind of healthy, we'll just keep routing it
there anyway. But as a developer, what's one of the
most common thing that happens. I check in my code,
I have a CI trigger, etc. In a lot of cases,
(38:44):
people have a lot of pipelines starting out, but nobody
looks at them. Right, that's a lot of compute as
a company or burn So maybe revised. If you really
need all of those CI pipelines, or make them on the.
Speaker 1 (38:57):
Nd maybe you only activate the pipeline on Friday or
something exactly.
Speaker 4 (39:02):
So if it's not like that urgent, maybe don't do it.
Speaker 2 (39:06):
Yeah, you guys are countering like a decade. Now I'll
go faster, go faster, Go faster, go faster. Right, it
depends if you're using these pipelines. Fine, but I've seen
a lot of people that once create a pipeline, then
never look at it anymore.
Speaker 1 (39:22):
Question right, Well, I kind of think that was the
lore of doing that. It's like, yeah, automate as much
of that work as possible so that you don't have
to worry about it. But now we're worrying about it
because it's costing a lot of cycles and money that
you know comes down to money.
Speaker 4 (39:40):
It is a trade of like is it useful or
is it something that we actually don't really need anymore
or run less often?
Speaker 1 (39:49):
Maybe remember we used to have build Friday's Richard, yep,
you know, back when we thought a build master was
a good idea. Oh yeah, build server. We had to
build server. Yeah, but you know we're happy with just
checking in code and everybody's synced up and all that stuff.
And when we want to have a release, you just
(40:11):
publish it and it's yeah, we can get that down
to just one command now right.
Speaker 2 (40:16):
Yes, well so, but to Tom's point, you also get
to the situation where everybody's pushing builds constantly and then
somebody's deciding to actually put it out, and so in between,
all those in between builds were kind of unnecessary. So
you know, the advantage of automation is that it's also
then tunable. So it would be that hard to go
(40:37):
back in there from a macro perspective and say, how
many of these builds actually get used?
Speaker 1 (40:42):
Can we delay? You know, when does it make sense
actually push? And then with the consequence of that, I.
Speaker 4 (40:47):
Think CI is one aspect. There's also the situation where
you have scheduled pipelines where the tests every hour or
every two hours, but how often do you really look
at these tests? Like maybe daily is enough?
Speaker 1 (41:02):
Right? How important do you see agency in DevOps be
becoming a partner so that you could give it a
set of rules that it could monitor and make its
own decisions as to when to build based on those rules.
Because that's really what you're doing when you when you're
figuring out strategies and and doing it yourself. So I
(41:25):
mean when you see that.
Speaker 4 (41:27):
I think that that's something that will come as a
next step. Maybe not as the next step, but eventually
one Like we push all these cis how many devs
actually check if it regresses the tests, then everything accumulates
and then you have x failing tests. Now go figure
(41:47):
out what what what's the problem or who to figure out?
Who to solve this problem? I think that's where agents
will become a theme where they kind of auto three
IG identified the issue. Yeah, picking the people alto rovert
or whatever. That's going to help a lot.
Speaker 2 (42:05):
On run as I just put out a show on
the Azure software Reliability Engineering agents, which that's literally what
their job is is to pick up events in it
that occur and a deployed piece of software and analyze
them even to the point of making co change recommendations.
Speaker 1 (42:23):
This is all in the back end in Azure. It's
all in Azure. Yeah.
Speaker 2 (42:26):
Yeah, these agents, I mean, it's the still miles to go.
These are early versions, but it does speak to I
think exactly what you're describing to them that you will
have from you know, you know, as a guy who
spent time on a Saturday trying to keep an e
commerce side alive. The fact that you can have an
error and these tools will at least do the initial
(42:47):
homework right that by the time you're looking at it,
it's already done. The pull of this was the state
of the machine, This was the current load levels. Like
all of the infos, we at least have a shape
of what was this problem.
Speaker 1 (42:57):
Yeah, and at least they could recommend a few courses
of it and you didn't even have it to it.
But you're making decision and that's yeah, that's where we
are today. I like that. I like being the decider.
I'm the decider.
Speaker 4 (43:09):
So you're talking about it as youre s a reagent.
I presume that's right. Yes, that is a very that's
something we're onboarding as well. Like a lot of times
it just reboot a machine and monitor its progress or whatever.
It's perfect for all these kind of things.
Speaker 1 (43:28):
Yeah, for sure.
Speaker 4 (43:28):
Reducing all that toil, it's really great.
Speaker 2 (43:31):
Yeah, if the answer was just reboot the machine and
the problem will go away. The fact that you can
have a tool that can get to that chain and
do it for you, yes, and so by the time
you know about it, it's already happened and recovered itself.
Speaker 4 (43:43):
Yeah, and then it will start aggregating all these symptoms
and kind of unfold the bigger problem. It's like, why
do we need to reboot all these machines. Then it
will kind of summarize all the symptoms and say, probably
your problem lies there, now go fix it.
Speaker 2 (43:56):
Yeah, memorily here, or do dispaced there, that kind of thing.
Speaker 1 (44:02):
You know, It's always funny when the software acts and
then notifies you.
Speaker 2 (44:05):
Like back in the old SQL server hot failover days,
like you would just get a notification databases failed over,
and it was chilling because it's like, well, why did
you fail over?
Speaker 1 (44:20):
You know you're going to fail again. Reminds me of
the early days of Azure Web Services when you because
they didn't understand how to deal with the memory leaks
or things like that, there was no stuff built in
on the server. The recommend mended course of action was
(44:41):
to automatically restart the service on a schedule, just you know,
and that was a way that you could out of sight,
out of mind memory leaks or other problems. Yep, you know,
turn it off, turn it back on. The universal fix.
All right. How much is security in the back of
your mind or the forefront of your mind is to
(45:02):
work your way through these things APIs and AI in
all of that.
Speaker 4 (45:08):
That's a good question, Carl. So in the past half
a year we've seen the sprawl of MCP servers spinning
up its very early days on the kind of the
authentication specification for this. But that's actually one of the
aspects that we can help with with Azure Api Management
(45:28):
is you can expose your APIs as MCP servers, and
we have Azure Api Center, which is basically your internal
MCP catalog or API catalog that can help you keep
track of all of these things. And the beauty is
that because you expose them to Azure Api Management, you
(45:48):
can reuse those policies and say I want to have authentication,
I want to have weight limit, et cetera. So you
still can use these MCP servers inside your organization, but
you can still have kind of your enterprise policies on top. Yeah,
(46:10):
that policies and control on top that you have as
a standard in your company to make sure it doesn't
become a liability.
Speaker 1 (46:17):
Yeah, and that's all good stuff. What I'm thinking about
is giving an MCP server, which is really an agent, right,
giving it agency to act on your behalf things that
you know, maybe some people don't want it acting that
way on your behalf. Maybe that could pose a problem,
(46:40):
especially with over hyperactive, hyperactively helpful lms like you know,
Claudes on It for example, loves to do things that
you didn't ask it to, and if it had if
it had the power to send emails or to read emails,
(47:01):
it could infect a system with now where like, there's
so many problems when you give an agent agency to
do things that might seem very convenient for you but
could open up security holes.
Speaker 4 (47:17):
I think authorization is a very important point here, Like
even if it would still have access, is it still
authorized to perform the action right, right, and as you
expose those APIs, like one of the things we allow
you to do is like maybe you have an API
(47:37):
that says get an order, create an order, and cancel
an order, and then the back office of your store.
But when you expose the MCP server, you have the
option to only expose get order, maybe create order, and
the rest. It doesn't like, even if it would have
access to it, we're not going to expose the API
(47:58):
even so it's not able to haul it right. And
I think that's the second aspect, is that we used
to have all these open eyes spects with the operations.
Now with this tooling, which is not a human person,
we need to be more careful what are we even
giving it access to.
Speaker 1 (48:14):
For example, giving a SEQL server MCP the ability to
generate queries and run queries.
Speaker 4 (48:23):
That is very dangerous.
Speaker 1 (48:25):
Yikes. I mean, think about that, generate t sql and
then execute it. Unless you say, the only thing you
can the only command you can create is select.
Speaker 4 (48:37):
Is drop table.
Speaker 1 (48:39):
If you say select is the only thing you can
do and you can't do anything that's going to act
you know, drop table or delete or anything like that.
You know, but you're still still untrustworthy, I think.
Speaker 4 (48:53):
Very dangerous, and also enough material for the LM to
have trained on to learn what not to do, you
would hope, So that is definitely yeah. I think that
comes back to the point I was making, like, don't
just expose everything. Think carefully, do reviews of what you're exposing.
Because it's not a human anymore. It's something that you
(49:13):
will ask And like you mentioned, some models are eager
to help, but that means you, like, maybe they're going
to do things you don't want.
Speaker 2 (49:23):
Anyway, I do appreciate that, you know, getting back to
as your API management. It's like you have to go
down the ENTRA pipeline to even use this, right down
to sort of the B two C model where your
customers have to have identities and so forth. You're forcing
us into the pit of security success. You want to
use these tools, you've got to follow zero trust rules.
You've got to properly authenticate everybody. It's that authorization and
(49:45):
policy in place, and it's a paint in the butt.
It's part of the challenge of getting started with these things.
It's also way beneficial when stuff goes wrong.
Speaker 4 (49:56):
Yes, and I was not part of the screw, so
I cannot kind of get into the details. But and
ignited last week when we're recording this, foundry was a
big thing. And as part of Foundry or AI gateways integrated,
which means every agent now hasn't dedicated entra id identity. Right,
(50:19):
So we used to be humans who had like a
personal email address with an identity. Now agents have the same.
So that kind of gives you the next step of
putting these guardrails around these agents to make sure they're not.
Speaker 2 (50:34):
And I hate to can't anthropomorphize on that because we've
also been giving applications identities as well for exactly the
same reason, yesh, right, But.
Speaker 4 (50:42):
For agents it was it was more it was harder
to achieve this and it was not really like dedicated
to the model model, like you had to kind of
configure it that way. But in this case, it's like
fully dedicated to this agent, let's say. But again I
did not work on that, so I need to defer
that to my colleagues. But I think that's a good
(51:03):
step forward for security as well.
Speaker 1 (51:05):
Tom, is there anything that we missed, anything else you
want to talk about before we wrap it up?
Speaker 4 (51:09):
No, So if somebody is interested. We have a blog
post which we can maybe onto the show notes where
you can sign up to use these sustainability capabilities. There's
a ton of announcements from Mignite. We also shift some
of the AI capabilities which you can find on the blogs.
But if you have any questions, just reach out to
(51:31):
me on LinkedIn. Happy to excellent answer.
Speaker 1 (51:33):
Your question, Tom. Thanks, It's always interesting when we talk
and this was no different. Thanks very much.
Speaker 4 (51:39):
Thank you very much. Have a good day many all.
Speaker 1 (51:41):
Right, we'll talk to you next time on dot net rocks.
(52:04):
Dot net rocks is brought to you by Franklin's Net
and produced by Pop Studios, a full service audio, video
and post production facility located physically in New London, Connecticut,
and of course in the cloud online at pwop dot com.
Visit our website at d O T N E t
R O c k S dot com for RSS feeds, downloads,
(52:27):
mobile apps, comments, and access to the full archives going
back to show number one, recorded in September two thousand
and two. And make sure you check out our sponsors.
They keep us in business. Now go write some code,
see you next time. You got Javans
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Burden
The Burden is a documentary series that takes listeners into the hidden places where justice is done (and undone). It dives deep into the lives of heroes and villains. And it focuses a spotlight on those who triumph even when the odds are against them. Season 5 - The Burden: Death & Deceit in Alliance On April Fools Day 1999, 26-year-old Yvonne Layne was found murdered in her Alliance, Ohio home. David Thorne, her ex-boyfriend and father of one of her children, was instantly a suspect. Another young man admitted to the murder, and David breathed a sigh of relief, until the confessed murderer fingered David; “He paid me to do it.” David was sentenced to life without parole. Two decades later, Pulitzer winner and podcast host, Maggie Freleng (Bone Valley Season 3: Graves County, Wrongful Conviction, Suave) launched a “live” investigation into David's conviction alongside Jason Baldwin (himself wrongfully convicted as a member of the West Memphis Three). Maggie had come to believe that the entire investigation of David was botched by the tiny local police department, or worse, covered up the real killer. Was Maggie correct? Was David’s claim of innocence credible? In Death and Deceit in Alliance, Maggie recounts the case that launched her career, and ultimately, “broke” her.” The results will shock the listener and reduce Maggie to tears and self-doubt. This is not your typical wrongful conviction story. In fact, it turns the genre on its head. It asks the question: What if our champions are foolish? Season 4 - The Burden: Get the Money and Run “Trying to murder my father, this was the thing that put me on the path.” That’s Joe Loya and that path was bank robbery. Bank, bank, bank, bank, bank. In season 4 of The Burden: Get the Money and Run, we hear from Joe who was once the most prolific bank robber in Southern California, and beyond. He used disguises, body doubles, proxies. He leaped over counters, grabbed the money and ran. Even as the FBI was closing in. It was a showdown between a daring bank robber, and a patient FBI agent. Joe was no ordinary bank robber. He was bright, articulate, charismatic, and driven by a dark rage that he summoned up at will. In seven episodes, Joe tells all: the what, the how… and the why. Including why he tried to murder his father. Season 3 - The Burden: Avenger Miriam Lewin is one of Argentina’s leading journalists today. At 19 years old, she was kidnapped off the streets of Buenos Aires for her political activism and thrown into a concentration camp. Thousands of her fellow inmates were executed, tossed alive from a cargo plane into the ocean. Miriam, along with a handful of others, will survive the camp. Then as a journalist, she will wage a decades long campaign to bring her tormentors to justice. Avenger is about one woman’s triumphant battle against unbelievable odds to survive torture, claim justice for the crimes done against her and others like her, and change the future of her country. Season 2 - The Burden: Empire on Blood Empire on Blood is set in the Bronx, NY, in the early 90s, when two young drug dealers ruled an intersection known as “The Corner on Blood.” The boss, Calvin Buari, lived large. He and a protege swore they would build an empire on blood. Then the relationship frayed and the protege accused Calvin of a double homicide which he claimed he didn’t do. But did he? Award-winning journalist Steve Fishman spent seven years to answer that question. This is the story of one man’s last chance to overturn his life sentence. He may prevail, but someone’s gotta pay. The Burden: Empire on Blood is the director’s cut of the true crime classic which reached #1 on the charts when it was first released half a dozen years ago. Season 1 - The Burden In the 1990s, Detective Louis N. Scarcella was legendary. In a city overrun by violent crime, he cracked the toughest cases and put away the worst criminals. “The Hulk” was his nickname. Then the story changed. Scarcella ran into a group of convicted murderers who all say they are innocent. They turned themselves into jailhouse-lawyers and in prison founded a lway firm. When they realized Scarcella helped put many of them away, they set their sights on taking him down. And with the help of a NY Times reporter they have a chance. For years, Scarcella insisted he did nothing wrong. But that’s all he’d say. Until we tracked Scarcella to a sauna in a Russian bathhouse, where he started to talk..and talk and talk. “The guilty have gone free,” he whispered. And then agreed to take us into the belly of the beast. Welcome to The Burden.