What Can AI Agents Do For Network Automation?

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Tim McConnaughy (00:14):
Hello and welcome back to another episode
of the Cables to Cloud podcast.
I am your host this week, tim,and with me, as always, is my
co-host, chris Kilometers.
He's living in a land downunder.

Chris Miles (00:26):
Officially changed the name.
Yeah.

And with us we have a very special returning
guest, a good friend of us, goodfriend of the podcast, duan
Lightfoot.
How are you doing, duan?

Du'An Lightfoot (00:36):
Hey, what's up Tim, what's up Chris?
And everyone out there inpodcast land thanks for having
me on.

Excellent.
So, just in case you know, wehave to just real quick, because
of course we don't know ifpeople missed the last episode
that we had you on Just do areal quick intro yourself and
we'll just roll right into thetopic today.

Yeah, my name is DeJuan Lightfoot.
I'm a developer advocate at AWS.
I'm not representing AWS, but Iam speaking my views of what
I've learned over my 20 yearcareer in tech, from being a
system administrator, networkengineer, developer advocate,
infrastructure automationengineer.
I guess I'm a kind ofgeneralist to learn wherever the

(01:14):
trend takes me.

Yep, and you're in.
Your big thing now actually isyou're like all in on the AI
stuff, aren't you?

yes, yes, for the last, I guess um chat gbt
came out in 2022 and I've beenusing it ever since january 2022
, that is oh wow, yeah, yeah,right, the, the crossing, the
chasm guy, you're like the, thefirst guy I'm.
I'm definitely not the firstguy, because you know, um gpt2
came out, I want to say like2019 or 2020, something like

(01:44):
that.
So people are doing vi for awhile now got you.

So yeah, but anyway.
So dewan you, we've invited youback.
I mean, well, first of all, wejust love having you on the
podcast, so I appreciate youmaking the time, uh, but the
other thing is that you fairlyrecently, within the last few
months, uh attended autocon,which uh is an automation focus
convention.
Uh, is it?
Is it caught?
Is?
Is it Denver where they have it?

(02:07):
I'm trying to remember.
Is it Colorado?
Somewhere in Colorado, right?

Yeah, it was in Denver, Colorado.
Autocon was a cool conference.
It was different than mostconferences I've been to.
It was a smaller conference,had one track and it was really
focused on being highly engagedin actual network automation For
those that are in the communityactually building things,
writing scripts, deployinginfrastructure configuration

(02:32):
with code.
They were all there.
Well, about 500 people werethere from the industry speaking
on topics connecting building.
I mean it was a good time.

That's cool when you say it's small.

Oh yeah, sorry, go ahead.
I was going to say yeah, yeah,I think I was talking to a
friend of the pod, will collins,uh, after he had just got back
from there and he said it was.
He said it was the bestconference he's ever been to, um
, and he, says like just becauseit was, I think, just because
it was kept relatively small andmore contained, and you can
have these like he said thehallway conversations were like
phenomenal in that environment.

So yeah, yeah, it's really intimate, right.
You go to a lot of conferencesand they can kind of be
overwhelming.
So many people, so many thingsto do.
You won't see everybody.
This one you really get achance to hang out with people
that you've seen online, peoplethat you know throughout your
career that may have gone andthey're in different companies.
I met all types of people fromall different walks of life and
it was a really good experience.

So I mean how.
I mean how, when you sayintimate convention, like paint
a picture for the audience Likewhat are we talking?
Like 500 people, like 300people.

Yeah, it was 500 people in a single hotel,
right A conference room in ahotel.
So whenever the keynotes wouldhappen or talks would happen
since it was only one trackeverybody would converse in the
same room, listen to the talkand then we all take a break
together.
And, yeah, you pretty much hadto meet all the people you
probably wanted to meet at theconference.

That's cool.
Yeah, I don't think there'senough of.
I mean, there's like littleregional events, I think, all
over the country at any giventime, but's it's hard to hit
that, um, that uh sizing problem.
Like you know, once it gets toobig, you know, of course
everybody goes to it, it'sreally popular, you got sponsors
, they can make it big, but youdefinitely lose a lot of the uh,

(04:16):
the intimacy.
And then you know if it's toosmall, of course, then like
nobody, nobody makes the trip,nobody shows up.
You know, like it's it's a hard, it's a hard target to hit,
right.

Yeah, normally when I leave a conference I'm
tired, like I come back and Ineed some downtime.
This one I was still rechargedand ready to get back to work,
so it was really good.

That's awesome and so, like the whole
convention obviously is focusedon on some form of automation.
Now, it's not.
It's not just networkautomation, right, or is it
mostly focused on networkautomation?

network automation right, or is it
mostly focused on networkautomation?
Yeah, it's mostly focused onnetwork automation.
Now there were some talks thatspoke about um kind of
networking in the industry,where we stand and how it
applies to network automation,but the conference itself was
pretty much around networkautomation yeah, so actually I
get this is a good question then.

so, given that you're you're deep in the ai
trenches now and you went to anetwork automation convention, I
mean, and and it's been for afew years for me since I've
really played around withnetwork automation and the crew
that's been kind of pushingnetwork automation, are you know
, for the people there and maybeit's a biased, maybe it's a
biased group, right, but arethere still people thinking like
, oh you know, automation isgoing to take my network

(05:21):
engineering job?

Or is it you think it's finally starting to
come around where people arereally doing it?
Yeah, this group is different,right?
Because this group is actuallybuilding things, so you will
have to talk to people thatprobably weren't at the
conference, that's what I wasthinking, too, when I was
thinking of asking the question.

I'm like ah, it's kind of a self-selection
bias there, isn't it?

Yeah, if you're thinking ai is going to take
your job, you're probably notattending the ai conference,
right?

yeah, the ai conversation is different and we
I guess we can kind of starttalking about ai now.
Yeah, the way I look at it,right, it's been an evolution.
Um, network automation has beenhappening before the python
hype.
Yep, um, network engineers havebeen um using scripts, whether
they're EEM scripts or whetherthey're Perl scripts, whether

(06:07):
there's Bash scripts, some typeof scripting, powershell, you
name it.
We've been doing networkautomation for years.
The thing that I see has kind ofevolved is for one,
containerization.
So, rather than running the codeon your laptop, now we're able
to centralize it and run itsomewhere right and then we're
able to automate it and scheduleit and have it kick off, and

(06:29):
then we're actually able tointegrate APIs.
So now we can have these agentsor workers that go out and grab
information or data from ourdevices, load them into our
database and then leverage anAPI to grab that data and
display it to all the engineerson the team.
And all the engineers do nothave to actually physically log
into every device, they justtalk to some web interface or

(06:51):
some API, right?
So when we started talking aboutAI, I look at the next
evolution is OK, I built allthese tools to get interface to
get routes, to add routes, to doall of these things in the
network, to get interface to getroutes, to add routes, to do
all of these things in thenetwork.
But rather than me having toput these scripts together, can
I use AI to kind of makedecisions on what happened in my

(07:13):
network?
We're getting deep down thatrabbit hole when it comes to
agents, but I just think, whenit comes to AI, since large
language models like ChatGBTunderstand language, now they
can kind of understand what'shappening in our network and
help us make more intelligentdecisions and also free up time
so we can do more higher end,complex tasks in the network

(07:34):
yeah, I think I recall um aspecific slide in your
presentation that you had atautocon, um specifically kind of
laying, kind of setting thescene a little bit about you
know, around the AI hype, whatis myth versus reality.

So could you take a couple minutes and kind of
touch on what you think are themajor myths versus realities
that we're faced with with thesenetwork agents?

Yeah, I think, first, that AI will replace
network engineers.
I think that's a myth when youlook at, let's say, the term AI
agents, what?
What that is is a system.
You have the LLM as the brainof that system, but the actual
agent is the code and the toolsthat network engineers are
already building.
So we still have to understandthe network we and more, better

(08:22):
in the future but we still neednetwork engineers that
understand the network,understand the infrastructure,
to be able to think the models.
When you give it a tool, let'ssay get interface, well, how
does it know how to reason aboutthat?
Well, somebody needs to writethe descriptions and help the
model be able to understand thesteps in the infrastructure, to
know what tools are availablefor it to be able to get

(08:44):
interface or to make a change ortake a series of actions on
your network.
So no, it will not replacenetwork engineers anytime soon,
could?
it help us at tier one.
Yes, it may could do that right.
Tier two possibly could do somethings there, but when we
started looking at thearchitecture deep in the weeds
networking we still need networkengineers.

Yeah, I mean like until AI can truly
interpret a diagram and they'regetting it's getting there right
.
I've seen some diagrams whereit can understand like, hey,
this line indicates that thesetwo devices are linked together,
and like that means somethingyou know it kind of reasoned
through that.
I've seen some of that.
Like that means something youknow, kind of reasoned through
that.
I've seen some of that.
But yeah, so on the agents,actually the agentic side of it,

(09:27):
do you think that you know thenetwork engineers of the future,
that kind of like, are going tobe using AI almost like a well,
just as another tooling set?
Really, do you think it's goingto just okay, trying to think of
how to say this?
Do you think it's going to bemore like people are going to go
to chat, gpt or cloud orwhatever and be like, hey, I'm a

(09:48):
network engineer, I need togather something from you know
some Git interface or you knowwhatever from a thousand devices
.
You know what's the best way todo that and then like work with
the agent to create the toolingfor that.
Or do you think it's going tobe more like and maybe it's a
little common to make L&B right,or do you think it's going to
be more like you're going tobuild the agent, put it all
together, build in the agentickind of reasoning and all of

(10:12):
that to use multiple tools, andthen just kind of oh okay, and
just walk away and kick it off.

Yeah, I think the first part.
A lot of companies are alreadyusing some type of large
language model for RAG retrieval, augmented generation.
I have these wikis, I have thisdocumentation, I have these
things about my network andthese guys to help engineers.
Now can you retrieve it andhelp answer questions for my
team?
Right, they have some sort ofsimple system like that.

(10:37):
So I think we're kind ofalready there at that point.
Now to your point of usingagents for our workflows.
I wonder if customizing agentsper organization, how that will
work, because when you think ofnetwork automation, that's hard
right For a lot of teams.
So if you layer AI on that,that introduces another level of

(10:59):
complexity, so that may takesome time.
But will it help us write code?
Sure, it's already doing that.
Will it help us retrieveinformation and really
understand our network a littlebetter?
Yes, it's already doing that.
Will it help us analyze routesor ACLs?
So can it help us actuallywrite configuration?
So it's doing a lot of thesepieces.

(11:23):
For the question that you asked,now what would a real solution
look like being implemented inan environment?
I think we're going to getthere gradually.
We may have a tool thatactually comes out the end of
the year where everybody's like,yeah, this is it.
This is what I see Right,because if I look at like Netbox
, you already have your networkstate, they have all this Right,

(11:43):
right.
You can build an agent on topof that fairly easy.
That one agent is not going tobe the same for every
environment, so to build asolution that works in every
environment that's going to bekind of hard you know, because
it had John Capobianco on hererecently just kind of giving us

(12:04):
kind of a breakdown on whatagents are and how they work.

And you know, obviously the agent needs to be
given a set of tooling in orderto you know, use and find out
information about the networkthat we're talking about.
Right, and you know, like yousaid, it sounds like some of
this tooling might be commonthings that engineers and
architects are already using ona daily basis, which is cool,

(12:27):
but it seems I mean, from myperspective, it sounded like a
lot of that has to be API based.
And you know, I think thenetwork realm has not been the
one, that is, you know, leadingthe pack in terms of the best
API support across the board.

Let's pause there.
I don't think it's necessarilyAPI-based right.
I think it's more so when youthink of a tool, think of a.
Let's take away the API.
Just think Python function.
Right, let's say you write somePython code that adds one plus
two or multiplies three plusfour, where you have two
integers that are being passedinto the function and then some
arithmetic is happening withinthat function and then the

(13:05):
solution is returned Right.
Well, that's actually afunction for an agent.
You can have a math agent thatsolves math problems, right,
there's no API involved in that.
So if you have aninfrastructure where you're
making SSH or you're usingsomething like NetMeco NetMeco,
yeah, right.
As long as your data is usingsomething like Pydanic, where

(13:26):
it's actually structured, youcan actually do that.
You don't necessarily need anAPI, you know.
You just need a tool that takesinputs.
You have Python function,because the actual agent is the
system that handles the.
Let's take a step back.
An LLM out the box, it cananswer a question and return a
response, but to get the weather, to check the date, to check
your interfaces, it needs sometype of function or tool that it

(13:48):
knows about.
Now it can't call that tool,but it can say hey, in my
configuration I know, I havethis tool and I know how to use
it.
So I asked the question getinterfaces?
It's going to say, okay, I knowI have a tool.
I can't tell you what theinterface status is, but I know
I need to use this tool.
Let me reply back Tool usage.
I need to use this tool.

(14:09):
And now, within your Python code, this is where the agentic
workflow actually happens,because in your Python code or
some type of orchestration tool,you actually handle that
response from the agent to say,ok, the agent needs a tool.
Now let me kick off this code,go get interfaces, get the
results and reply back with auser message to the agent or the

(14:29):
LLM to say, okay, here's theresponse, the tool result from
the interfaces.
Now the LLM will analyze thatresult and say, oh, okay, now I
can answer your question basedon these results.

It's like an expansion on RAG, these results.
It's like a.
It's like an expansion on rag,like you know, except that it's
like up to date, like trulyyou've run a tool, like you said
, and actually fed that resultof the rag kind of back into the
llm, to the only difference theonly, the only difference from
like the way rag is implementednormally, or let's say
traditional rag, right, raghappens with before it goes to

(15:05):
the llm, right, so that's rightbefore it goes to the llm.

When it comes to agents or tool usage, you
actually send your initialprompt to the llm and the llm is
able to reason or decide whattool to use and then it will
start and let you know.
Here's the tool I need to use,and your Python code needs to be
able to handle that response,to be able to get the data or
the information that the modelneeds to then be able to give

(15:31):
you an accurate response.

So it's like half and half right.
Whatever the reply is or thatyou get back from the LLM you're
writing handle like handlingcode that'll say oh well, I'll
do this based on what you'retelling me you need to do.
Basically.

Right, and that's why, when we talk about
implementing this into, do youhave to troubleshoot your
network automation?
But you also have totroubleshoot the track and trace
or the tracing or loggingwithin those calls to your LLM,
because the LLM you had tomonitor the request going into
the model, right, the promptgoing into the model, the
response is coming back from themodel and then the
orchestration that's happeningin your agent.
That's like four differentsteps or workflows is where you

(16:28):
are at the log and troubleshootand anything can go wrong at any
given point.

It's like a pipeline kind of I mean there's
a lot more you know.

Yep, it is yeah it's like you just need to
service chain all this stufftogether to make sure it happens
in the right operation.
Right, and there's tools forthat right.

So LaneChain provides a library for that.
There's several others I know.
On AWS we have Amazon BedrockAgents.
That kind of helps youorchestrate and we handle that
for you.
So a lot of that logging the,building out the agent functions
or tools you can actually dothat with Lambda functions.

I was going to say it's.

Lambda right To your agents.

Actually, I actually do that with lambda
functions and I'm gonna say it'slambda, right to your agents.
Actually, I haven't seen this,so I'm kind of curious now.
So the front end for bedrockagents does it actually take you
to like to lambda, or is itjust like a wrapper around?

like a gooey wrapper.
Okay, right, so you actually gointo lambda.
When you actually create yourown functions, that the tools
for your agent, you actually gointo lambda, put your python
code there and then you actuallybuild this.
You can build it out on aconsole or you can actually use
I think it's JSON schema, don'tget me wrong but to actually
tell your agent what the objectsor the tools or the functions
that it has.

(17:35):
You can do this in a console.
But there's also a schema thatwe provide that you can actually
update and give it to youragent so that you don't have to
go in the console and click allaround.

Right.
One question that I do have isit's like a prevalent issue with
AI that I think is partiallybeing, you know, handled over
the last few months, but there'sstill probably a lot of people
out there that are like, oh, Idon't want to use AI because I
don't trust it, becausehallucinations happen and I just

(18:06):
can't trust the data that itprovides to me.
Right?
So from your perspective, inthis scenario where maybe you
are doing things where it'sgiving you information about
your network, what kind ofverification process comes into
that, and how do you avoid, orat least do some fact checking,
with things like hallucinations?

yeah, before pushing like big major.
You know what is it?

uh, automation lets you fuck up at scale, right
um, so hallucinations aren't abad thing, and the reason why I
say that is because if you lookat a large language model, the
way they're trained, they'retrained on all the data of the
internet.
That's called pre-training, sothe data is clean, sanitized,
put together in documents andthen you train on these
documents and then you have amodel that can spit out some

(18:53):
document based that looks likeit, looks like some words, looks
like some language because itunderstands it, or labelers that
basically ask questions that weall ask.
They write these questions andthen they write responses that
the model should give, and sothey do this for a ton of
different questions, right?
And then, once they have thesequestions, then the model is

(19:16):
trained again to be a helpfulassistant, right?
Interesting and so it has all ofthis information where it says,
ok, I'm able to.
I have all this data, Iunderstand the questions that
are asked and I'm able to makepredictions to help kind to to

(19:37):
get some type of accurateanswers that I'm confident about
Because, due to themathematical, mathematical
probability in the model, I'mable to predict the next token
in the response.
Probability in the model.
I'm able to predict the nexttoken in the response.
Right, because it uses tokens,because it doesn't actually
understand text and itunderstands tokens or embeddings
or whatnot.
When this happens, thehallucinations they are the
model kind of guessing on how torespond.

(19:59):
Now, sometimes they do getthings wrong.
We don't really understand that.
But ways to check against thatone guard rails.
You put guard rails around amodel.
Those are it's like a firewallthat goes around the model.
You check the inputs, you checkthe outputs, and so you kind of
do that around the model whenit responds.
We have something on AWS calledautomated reasoning and so if

(20:20):
you're looking for, let's say, aresponse from the model and you
are able to determine let's sayYou're looking for IP addresses
or you're looking for somethingthat's structured and something
that you can actually testagainst Well, you can use
automated reasoning to ensurethat the response is what you
expect and the reason why themodel responded this way.
Also, you can actually parseyour responses into the model

(20:44):
and out of the model's text.
Right, ok, so you can actuallywrite tools to troubleshoot and
test against the responsesbefore it even returns your user
, because you're still usingcode at the end of the day,
right, and when it, when, whenthose, when, those solutions
happen, you should log thoseresponses so you can go back and
see, okay, what was thequestion to ask.
When we're writing questions tothe model, there's something

(21:05):
called prompt engineering.
I'm sure you all heard of it,right?

yeah.

Yeah, of course you can ask a basic question,
but if that basic questiondoesn't contain accurate
responses I mean accurate umcontent it's not posed correctly
, it doesn't have enoughinformation, it's not direct,
there's a lot of fluff in it.
It wasn't, it wasn't able touse rag correctly.
There's a lot of things thatcould go wrong in that.

(21:28):
So when you say a hallucination, we need to find out what
actually happened with thatprompt.
And then, does it align withthe response?
Right, right.
And so when we're building ourapplications because
applications are being builtwhat you're trying to do is
evaluate the response before itgets to your end user.
Okay, right.
So you may not have one model,you may have a collection of

(21:50):
models, yeah, that are testingand validating responses before
they even get to the user.
Right, yeah.
And then one more thing You'reactually going to evaluate your
models before you put those inproduction.
So if you have, let's say, anagent or some type of workflow
that's built now, you're goingto want to test this 100 times,
200 times, to see how theprompts, or more, to see how the

(22:12):
prompts are being responded.
What is the accuracy?
You're going to test this.
This is just like you test anyother code.
You're going to test this aswell.

Because you mentioned guardrails and
validation, how could you giveme, give us an example of what,
a how you would build aguardrail and like what that
like?
Are you just writing as part ofyour prompt a guardrail like
validate?
This response meets thiscriteria, or like I'm?

I'm struggling with the visualizing that yeah,
so picture a large languagemodel, picture a box around a
large language model.
Protects it, what goes in it,and protects what comes out of
it, just like a firewall, right?
And so if you were, let's useamazon bear, bear Rock, amazon
Bear Rock, converse API or anyother APIs you can actually
create a guardrail.
And let's say this guardrail issupposed to look for usernames,

(22:58):
passwords, phone numbers,account numbers, credit cards,
like.

PII stuff.

PII stuff.
Right, You're going to inspectthe prompt to ensure that no PII
is going into the model.
If it does, you can tell it tomatch those numbers right and
then, when it comes out from themodel, you can tell it to match
those numbers, or what youcould do.
Let's say someone has to ask aquestion that you don't want it
to ask.
Maybe they want to ask aboutsports betting or something that

(23:25):
the model should not beanswering.
Well, before the question evengoes to the model, guardrails
can check the question andensure that that question is
blocked before it even goes tothe model, which is going to
help save on cost.

Right, because you're not running the.

you're not running the query, you're not lighting
up the model, the LLM also aretied to a, you know, a
confidence score.
Right, a scoring mechanism tosay how likely the likelihood
that the response is accurate,right?
Can the guardrail kind of comeinto play on that kind of

(24:01):
outward direction and say like,don't provide responses that are
below a certain confidencescore?
Is that where you'd use aguardrail, or would that be
somewhere else?

You can do that and I believe there are tools
for that.
There are tools for that onamazon bedrock.
That, specifically I'm.
I would have to get back to youon that if we have it.
But yes, you can do that.
Um, but what are you scoringagainst?

I think I read a blog, I think we read a blog
about that.
Actually, uh, this was monthsago, but I think I read that
that was actually a new featurethat they had added to bedrock
actually was the confidencescore.
Yeah, uh, ability, actually Ithink it is.
I didn't mean to hit you on the, I didn't mean to ask you.
You know, like hit you in thegut, uh out of nowhere, but uh,
because that's a.
We certainly kind of meanderedthere a little bit so you

(24:44):
probably weren't ready, but I do.
I do sort of remember readingan amazon blog about, about
bedrock adding uh, confidencescore or something, if I
remember correctly.

So there's a name for it, and we call it
contextual grounding.

That's it.
That was it.
Yeah, that's what it's called.

So when you configure a guardrail, you can
check.
There's like several differentthings.
You can check for contentfilter, deny topics, sensitive
information, word filters,contextual grounding.
That's why I wasn't clear on it, because it's called contextual
grounding and that's just acheckbox that you check when you
configure it.
You just turn it on and itactually does it.

Okay, okay.
So that's the type of check youwould run if you were building,
say, an agent that's gonna kickoff some tooling to like go
make changes on your network orsomething like that.

Is that accurate?
Well, my question was just tomake sure that is even something
that you would put into aguardrail, or does that come in
somewhere else in the kind ofsequence?

But yes, that's doable via the guardrail.
Yep, it's in the guardrail.

I mean, but what you said is accurate, right
, if you build a tool, andbuilding an AI tool is the same
as building any other code orwhatnot.
It's just you're talking to anLLM instead of writing Python
code directly or something right?
So you would have to run it.
The part I'm still unclearabout and this is because I just
haven't done it is, let's say Ibuilt an agent and I wanted the
agent to do X, y, z, it doesn'treally matter whatever that

(26:01):
workflow looks like and I run itand it'd say it doesn't do the
workflow correctly, like howwould you troubleshoot that?
Would you go back to the prompt?
Troubleshoot that?
Would you go back to the prompt?
Like I mean, let's just approve, let's just say that it's spat
out nonsense, like ip addresses.
Then you assume, like the ip,the tooling for handling the ip
is wrong.
I'm sorry, I didn't mean tonail you down on this, man, I'm
just.

I'm just curious, because we kind of
you've now activated my trapcard, I guess, because now you
got me thinking so, just like,python code is involved, right,
so you python code is going tobe involved, or some type of
code, right, you can usejavascript, you can use whatever
, but let's just stick withpython because that's what most
networking is used.
When it comes to what you'retalking about, you're going to
have logging and with everyapplication you build, there's

(26:41):
going to be some type of logging.
Now, when we're communicatingwith the llm, in order for your
llm, which is the brain of youragent, it's, it's not right.
The, the llm itself is not theagent.
The orchestration that you haveis what makes the agent.
When you actually build outyour API calls to your LLM.

(27:02):
There's two things you need.
You need a system message.
The system messages tell theagent the role that it has.
Right, you're a networkengineer, the tools that it has
right, right, the arguments thatthose tools accept, right.
So let's say, your, your agent,is a weatherman, right?
Or weather woman?
Well, that agent is going toneed, is going to need, to know

(27:22):
that it's, so it has access tothe weather, right?
right and access to get weather.
Also need to know what citiesor states or locations right, so
you need to-.

Right.
What are the inputs?

What are the inputs right?
So you're going to talk to youragent and tell it in the system
message all of these things soit can really be able to reason
and understand when to use atool and when not to use a tool.
And what parameters do I needwhen I need this tool?
Because I may need to ask theuser okay, you want me to get
the interface?

(27:53):
What is the IP address?
So in the system message yougive it all of that information,
right, and then from there theprompt is actually the user
inputting their questions thatthey have Right, right.
But the system message iswhat's going to be the, I would
say, the secret sauce to makingyour agent have the
understanding that it needs, oryour LLM having the

(28:15):
understanding it needs for thetools and functions that it has,
the capabilities to.

Yeah, it's like a preloaded prompt or like a
role definition, like kind ofthat the user doesn't have to do
right, the user only has to askthe question.
But you've already, I see.

So we're troubleshooting right.
You're logging the prompts thatthe users are um sending to the
model.
Well, since you're loggingthose prompts, what you could
actually do is take one of theprompts, send it to your agent
and see how it responds rightyour application to see what's
being called, what's not beingcalled.
Did it use the tool?
Did it not use the tool?
What's the data coming backfrom the tool?
And then, how did it processand respond once it had the data

(28:53):
from the tool?

Yeah, perfect.

Chances are, is that it could be something
wrong with the.
There's something called a toolconfig, right?
The tool config is like theparameters and the understanding
that the model has for theinputs and the actual functions
and tools that it has.
And then you have yourfunctions, the Python functions
themselves.
They need to come back andstructure JSON to be able to

(29:15):
come back to the tool.
All of this needs to be codedcorrectly in Troubleshot, right?
So these are this is prettymuch how you would troubleshoot
it, and there's tools to do this.
You can do this all throughcode as well.

Perfect sense to me yeah.

I mean it sounds like you know potentially I
guess it depends on your usecase right and what your actual
network implementation lookslike but it sounds to me like
there there's there's the needfor a lot like a lot of work
upfront, like there's a lot ofcoding that you need to do
upfront to kind of get thisstuff in place um before um
before it's actually, you know,usable, depending on you know,

(29:48):
like you said, if I'm assuming,if the network stack that you
use supports an API that isrelatively structured and has
good documentation, you canprobably just feed that into the
agent.
As far you know, and at leastthat's from what I got from our
talk with John, you can kind offeed all that API documentation
into the agent and it knows touse that tooling.
But you know, when it comes towriting your own Python and

(30:12):
things like that, how much doyou think a network agent
specifically requires kind ofmore upfront lifting compared to
another activity that you usean agent for?

So networking compared to whether agent or?
Yeah, exactly.

I mean like, yeah, I guess that's a very open-ended
question.
Very open-ended the use for anagent can be anything under the
sun.

I guess it can be anything under the sun.

it can be anything simple, yeah so I I
think, um, when I like to lookat this, when it comes to
network automation, I alwayslooked at it a little different
than a lot of network engineers,and I think it was because of
my time at a particular companywe aggregate our data to a
central location, and now thisis before netbox, so now the
netbox is around, they kind ofaggregate you yeah, put it there

(30:54):
right.

We aggregate our data to a central location.

And now this is before NetBox.
So now the NetBox is around,they kind of aggregate to there,
right, and then you can accesseverything from there.
So the first thing you probablywant to do is aggregate your
data, rather than going out andhaving your agents or LLMs
making API calls to your devices, which that doesn't sound very
safe right.

(31:17):
I mean, it can happen, butthat's probably not a way we
want to use it, right.
Right, we want to make surethat it has read-only access
right to a particular databaseor to a particular API,
something where it can just readthe data and then start there.
That would be the simplestthing, right?
And then seeing what you canaccomplish with that right.

So I think you're kind of like really emphasizing
the value of having a CMDB insome capacity about the network,
rather than pulling the networkdirectly, having that
intermediary device or mechanismthat's going to have all that
state data so that you can kindof really simplify what the AI
needs to reach out to in thefirst place.

Yeah, but we also have to think about, like
the I would say, the scale ofthe network.
That's something that's alwaysgot to come into mind, because
if you've got 10 devices, howmuch value are you going to get
out of all that?
You know what I mean.
So that's something we alsohave to think about.
But if you're doing things atscale, yeah, aggregating your
data and being able tocentralize it, it's going to

(32:14):
help and save you a tremendousamount of time and headache.

It'd be like that meme with the guy with the giant
spoon and it's like my you knowthree three three-tier web app
with cdn and it's like, it'slike a tiny little ball, yeah
yeah, I mean.

So you actually , I mean you wrote an agent,
right, like the networkwhisperer code is is an agent.
So if anybody wants to see whatDuan's talking about and
actually get down in the nittygritty of what he's talking
about, like and we're going tohave it in the show notes, of
course, right, Um, the code'sthere, like straight up.
Just go take a look at it andyou'll between how much of the

(32:51):
Python is there versus whatwe're involving the LLMs with
and stuff right.

I think I'm going to redo it, because in
order to run that code, you needaccess to AWS.
So what I think I want to doand if I can do it, if it's
still available, I'm going torewrite it, maybe containerize
it, put a front end on it.
But the back end will probablyconnect to Cisco DevNet.
Okay, put a front end on it, butthe back end will probably
connect to like um cisco devnetright, grab the information from
there and then everyone couldhave access to it and actually

(33:18):
see an agent and be able toreverse engineer it and try it
out for themselves.
I think I'll do that in thenext day or two and I'll get it
to you and it'll be in the shownotes oh, brilliant.

Uh, I mean, that's what john did too, I mean
, and we're we're big fans ofthe idea of containerizing this
whole thing, because you know,especially for network engineers
who don't already have all thehey, you got to install Python
virtual environment, like that'sthe old DevNet, struggle right
about getting people spun up.
If you could just hand them acontainer and be like it's all
preloaded, dude, just do thepart that you care about.

(33:48):
You know that's great, allright, just do the part that you
care about.
You know that's that's great,though.
All right.
Um, let's see, is it?
No?
I just, uh, I'm trying to thinkis there anything we?
I mean there's so much,honestly, we could probably go
god, freaking forever on this.
This is so interesting stuff.
I, I don't know.
I find this very interestingand I I also I'm glad that
you're here and I'm glad thatwe're friends and I can ask you
all these questions, becauseit's hard, man, it's hard to

(34:09):
even go to the llms and ask themthese questions.

You know, like I think part of it is, um
knowing the right, the questionsto ask.
Yeah, I've been kind of doingit and I understand, you know
the thing, the struggles thatyou all have, you know in
networking, and then alsounderstand the llm side, since I
do it so much.
You know, um, but there'sthings that you all see today in
networking that I don't see.
So we got to kind of help eachother.

(34:33):
You know what I mean, yep.

We all got to help each other.
That is absolutely the case.

Yeah.
I think maybe to close out,there was a great piece in your
talk kind of talking about theroad ahead and what the
strategies are for adopting AIin this new era that we're in
Right.
So I mean specifically maybeyou can kind of go over some of
those points, what you wouldrecommend to the listeners on
where they should get started.

Yeah, I think network automation starting
there with Python, is networking, of course, networking
foundations and getting familiarwith networking.
I would say layer one, layertwo, getting familiar with
Python.
Layer three will be APIs,because that's that networking
layer, of course, right, gettingfamiliar with that.
I would say layer four would bekind of connecting it all

(35:16):
together, start making code anddoing some automation, and then
that session layer is going tobe that transition to
understanding LLMs, right, beingable to understand how to write
prompts, understanding how RAGworks, understanding the
different types of models, right.
And then from there, when westarted looking at presentation

(35:37):
and application, we're kind ofbuilding it all together.
Maybe you're using somethinglike Streamlit and you're
starting your own type of, Iwould say, agent or agent to
workflow by saying, okay, I havethis Python code, I'm able to
automate this.
Now let me see if I could ask achat bot to be able to run
these tools and provide me aresponse.
And then I think, if you kindof get that layered approach,

(35:58):
then your wheels will getturning and you'll start
building out your own solutionsand being able to see where you
can actually take this, becauseit's kind of hard to see it
until you actually get yourhands on it and try some things
out, right.
But if you don't reallyunderstand how to code, I

(36:18):
recommend okay, use AI to helpyou learn how to code.
I use AI for so many things.
It's the part when I'm drivinghome.
I don't listen to music, I'mtalking to AI trying to learn
stock options or learn somethingyou know because it can answer
questions.
And then I'm already studyingthese things and now, after I'm
studying, I could talk to AI andask it to clarify, and if it
doesn't sound right, then I cansay are you sure?

(36:40):
And then it's going to say,yeah, I don't know You're
absolutely right.

I got this all wrong.

And then I know I need to do research.
But then I find out what thelimitations are and where the
gaps are, and it helps me learneven more, because I'm realizing
that it can't help me here andI got to dive deeper.

Yeah, it's a big feedback loop.
I guess that's a really goodpoint, right.
The more you learn from AI, butalso just from interacting with
it, you start to find the edges, like you know where, the edges
of the coloring book that youcan't go outside, or where it
really is good at, and andthat's a whole skill in and of
itself, beyond just simplyprompt engineering.
So, yeah, it's a good, goodcall.

That's crazy.
So you're just doing this.
While you're driving, you'rejust talking to an agent yeah
yeah, that's crazy man.
Like what are you using to dothat?

Um, yeah, I use the chat GB gbt app so I pay
for probably about five ai tools.
Right, but they are.
They help me learn.
Um yeah some of them are betterthan others, but I use one for
code.
I use multiple for code, right?

yeah, um, and it's helped me grow oh, I did
want to ask you one more thingbefore we, before we close out,
and I it's important what do youthink about specialized
language, like small languagemodels, like, do you think we're
going to see more smalllanguage models, more focused
small language models, kind ofthe bridge between getting all
the way down to agents and, youknow, all the way up to LLMs, to

(38:05):
maybe save resources and getmore specialized or not.

Yeah, I think when you start looking at these
applications, a lot oforganizations want to save on
cost, and so, if you look at asmaller model, it's going to be
lightweight, it's going to befast.
You got the 1B and 7B modelsfrom like Lama, which is meta.
You got models from Alibabathat are pretty lightweight and
small.
You have hosted models on cloudwith like haiku, which is

(38:32):
pretty fast, and I think umopenai just released their 03
mini.
That's pretty fast and prettysmart, and so when we look at
these models, um, a lot ofpeople go after the larger
models, right, the largelanguage models, which a smaller
model is a large language modelas well, except it doesn't have
as many parameters right right?
um, you can run that on yourlaptop, right, because it's

(38:54):
lightweight.
The capabilities on the largemodels have been distilled down
to the smaller models, so theyare optimized in this
lightweight compact box Right Tohave some of the same
performance as the larger models, but they may not have the
depth and breadth of thoselarger models.

Not as much information, yeah.

Right, but they understand language.
So the thing that you could doto get around that provide
context.
So when we talk about a lot ofthe tasks that we want to
perform, a lot of it is aboutthe context that you wanted to
understand your data and performan action off the data.
It's not necessarily what'sinside the brain of the model,

(39:34):
it's about the data that youhave and if it has access to
that data and you're structuringyour products appropriately, it
can actually get you somereally good performance for a
lot cheaper cost and be a lotfaster in your application.
Yeah, that's really important.

I think in the future, go ahead, future, go
ahead.
No, I was just saying I thinkthat's so important now because
that's the cost of doing this isstill out of the reach of a lot
of organizations to like runthis stuff locally, so I think
that's the way forward.
Sorry, I didn't mean to cut youoff there.

No, I think if you're asking questions with the
model out the box trying to seeits capabilities, I don't think
that's the right approach.
I think more so right in aprompt that actually has the
context that you want to provideit, like what do you want it to
solve?
Give it the information itneeds to solve the problem and
then evaluate the model righthow does it perform, what is the
consistency, what is theaccuracy on providing the

(40:23):
responses right.
And then, once you determinethat, it's like okay, this works
for me, Because what I run nowI'm running all smaller, smaller
models, like everything that Iwant to do.
I can do with a lightweightmodel, except for right code oh,
is it just not?

the?
The skill, the, the reasoningof like how to create code just
isn't in the small, or just thethe depth of of code knowledge
is just not there in the smallercode knowledge for the things
that I do.

Um, gotcha, I go to like I use cloud sonnet
3.5 to.
That's been the best model whenit comes to code.
But if I just need something tounderstand something really
quick, there's a couple ofapplications I use for that, and
on my local laptop I runsomething called Ollama which
allows you to run large languagemodel locally on your laptop,

(41:08):
and so I run Ollama 3 on mylaptop.
I also run DeepSeek on mylaptop.
Well, I don't run DeepSeek onmy laptop, I run that on my
desktop because I have an NVIDIAcard which is a 2060.

I was going to say be careful, they might go to
jail.
2060.
I was going to say be careful,they might go to jail.

Might go to jail Hawley.

He put together legislation trying to get people
to go to jail for a maximum of20 years.
So no worries For runningDeepSeek, they're trying to
introduce this dude's.

Just, he's one of those performative guys, but
anyway, yeah, he just tried tointroduce legislation.
Yeah, if you run deep seek, ifyou download deep seek, it
wasn't even run it like if youdownload deep seek, you can go
to jail or something like that.
I don't know that it's it'stypical government, like how you
guys even plan on enforcingthis type of.

I wonder what that means, because there's like
, again, there's deep, there'sdeep.
Seek the steel models wherethey've been trained off of,
like Lama Right Off of GPT, ortrained off of QM.
Yeah, you know.
So it's like I don't know.

It's typical.
The government is trying itsbest, to you know, to be
relevant in this stuff.
But anyway they just don't knowwhat they're talking about.

Last thing I'll say about the kind of expansion of
these smaller models, and thisis probably my just kind of dumb
way of thinking about this, butthe thing that worries about me
, or worries me about this is ifwe start to see a vast
implementation of these smallermodels.
If I think about me as a person,when I'm going to make a
decision, when I'm going to givea recommendation or process any

(42:41):
data, I need a lot of contextto make a decision, like I need
to know the whole story, blah,blah, blah.
Even if I think about theconcept of maybe, like, maybe
you wanted a networking specificthis is this is probably a dumb
example, but if you, if youwanted a networking specific
model that was just focused oncomputer networking and like it
knew everything about networkingin and out, blah, blah, blah,

(43:08):
and you got it to make decisionswithout also understanding the
context of how applications workand how applications actually
function, like are we, are we,are we going to start
snowballing and like, like,accelerate towards mediocrity in
that, in that context, withouthaving the full context of the
situation?
And you know what?
Like I don't know.
I like this is just somethingI'm spinning around in my head,
I don't know if it makes anysense, but that's uh seems like
that would worry me a bit Ithink if you needed something

(43:29):
that specialized, maybefine-tune the model yeah, I've
heard of fine-tuning.

I mean fine-tuning.
I guess I was talking to andrewabout that.
I really uh, you know he'srunning the gni boot camp or
whatnot.
He was showing me how you canfine tune.
It's interesting, interestingidea.

So fine tuning would be.
You have this model, it'spretty capable, right, but it
doesn't have domain specificinformation or intellectual
property that's related toanswering your questions and you
really wanted to specialize inthis, and a context window just
isn't enough, ragging isn'tgetting it done and you don't
have the money to train your ownmodel from scratch.

(44:03):
So what fine tuning does is itsays okay, question, answer,
response.
You fine tune the model to beable to recognize the questions
that you ask and how to respondwhen those questions are asked,
right?
So basically, you're taking allof your data and you're
basically putting it in questionand answer or question and
response format and then youretrain the model based off of

(44:24):
that information.
So now, some of the knowledgethat it didn't need is going to
kick that out and add newknowledge based off of what you
have.
But it still understandslanguage, it still has that
capability, but it specializesin the information that you have
.
Now, fine tuning is veryspecialized.
It's not something that's OK.
Let me take these questions,let me take these is very
specialized.
It's not something that's okay.
Let me take these questions,let me take these responses, and

(44:45):
then it's going to work.

No, this is something that you have to
iterate on and that to actuallyget it right yeah, there's,
there's, so there's so much ofthere's so much that goes into
the uh, the underpinning, uhmathematical stuff for the deep
learning and machine learning,and all that really fools this
but.

But just one more piece.
Think of it this way right, ifwe do use the smaller models and
they're not able to get you anaccurate response?
You can actually create asystem that says, okay, if I
don't know information, I needto go to my big brother and get
the information right.
You can actually create it,like all of this is through code
.
Yeah, all of this is throughcode.
Like you can save on cost byletting the smaller model handle

(45:22):
the not as complex tasks andthen defer to the larger models
for those really more in-depth,complex tasks.

That's a really good point.
Man, that's a really good point.
You know I've worked withAndrew Brown on the Gen AI boot
camp stuff and we were workingon the.
We had a project we're workingon which we'll see later on in
the bootcamp, and that's exactlyhow it worked.
Right, we tried to handle someof the stuff locally and then I
didn't even know you could do it, but of course I should have
thought of course it's just coderight At the end of the day,

(45:58):
right, you just make an API call, feed it a prompt and then get
the response.

You know.
It's something simple becauseyou can tell a model.
If you don't know, say I don'tknow, right, and if you don't
know, call this function.
Call Big Brother, right and youcall.
Big Brother and pass in thephone.
Yeah, phone a friend, originalprompt.
Phone a friend Right and passin the prompt to Big Brother.

Big brother answers it and then helps little
brother respond yeah, andyou're only paying for the, the
tokens, those, those veryspecialized tokens that you ran
on on the big brother, right?
So that's that's really good.
Point too, man, that's reallygood, all right.
Oh, man, we have a lot to thinkabout.
Uh, this has been so awesome.
We're definitely gonna have tohave you, uh, back again.
I'm sorry in advance becauseyou're gonna, you're gonna, but

(46:42):
no man.
Man, you're so good atexplaining this.
I mean, obviously that's yourjob, so no surprise there.
But it's always great havingyou on man, and this has been so
helpful and I hope that it'sgoing to help a lot of our
listeners understand their placekind of in the hierarchy to
start playing with this stuff.
So I appreciate it Anything toadd?

Chris, Nothing to add.
I will say one thing that Ididn't know prior to meeting
DeJuan at reInvent last year.
Dude, you're tall as hell, bro.
I didn't know you were thattall.
Yeah, you always fool me.
Every time I see you, you'resitting down, so I didn't know.
I mean he was towering over me.
You ever dunked a basketball?
Can you dunk?

nah man, no, I could direct grab the rim when I
was in shape.

Hey, touch a rim still something I've never done
out of basic training.

I think I dunked like one time and then.
I thought, I drank a beer andwas down here from there awesome
alright, we'll wrap up thereagain.

Thank you to all the listeners for listening
or for watching.
If you're watching us onYouTube maybe both, maybe you
want to watch this twice forsome reason.
If, hopefully, if you like whatyou heard, saw, then please
subscribe to us on yourpodcatcher.
Watch us on YouTube.
Check out DeJuan's everything.
We'll make sure all his stuffis in the show notes, but
especially the Otacon stuff andthe code that he's going to

(48:03):
rewrite by the time you see it.
So we'll see you on anotherepisode.
Thanks for joining us.

Hi everyone.
It's Chris and this has beenthe Cables to Clouds podcast.
Thanks for tuning in today.
If you enjoyed our show, pleasesubscribe to us in your
favorite podcatcher, as well assubscribe and turn on
notifications for our YouTubechannel to be notified of all
our new episodes.
Follow us on socials at Cablesto Clouds.
You can also visit our websitefor all of the show notes at

(48:33):
CablesToCloudscom.
Thanks again for listening andsee you next time.

All Episodes

Episode Transcript

Popular Podcasts

Stuff You Should Know

Dateline NBC

On Purpose with Jay Shetty

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}What Can AI Agents Do For Network Automation?

Episode Transcript

Popular Podcasts

.css-r6mb8g{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:1;overflow:hidden;}Stuff You Should Know

Dateline NBC

On Purpose with Jay Shetty

All Episodes

What Can AI Agents Do For Network Automation?

Stuff You Should Know