Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Andreas Welsch (00:00):
Episode is going
to be a little different on
(00:01):
short notice.
Hey Kiumarse, thank you so muchfor joining.
Maybe if you can talk to ouraudience a little bit about
yourself, who you are and whatyou do as we now pivot more
towards the topic, how can youpick the best AI agent framework
for your business?
Where are you?
What do you do?
Kiumarse Zamanian (00:19):
Hey, thanks
very much Andreas.
And really I've enjoyed yourpodcast and great book and also
the LinkedIn classes you have.
Congratulations on all those.
So I'm actually in the SanFrancisco Bay area.
I've been in Silicon Valley forabout 30 years.
I did my PhD at Carnegie MellonUniversity, focusing on data
(00:41):
management and some AI, and thenmoved to Silicon Valley and
worked in about eight companiesso far, focused on analytics and
interopability.
And for example, I worked forAutodesk, Informatica, Yahoo,
Responses, which got acquired byOracle.
And then after Oracle, I joinedWalmart Connect.
(01:03):
And most recently I've beenworking in the Gen AI area since
kind of early 2023 in the agentareas and exploring ways that we
can leverage these agents toautomate and make people more
productive and also making surethat these things are doing the
(01:23):
right thing.
Yeah, I'm really excited abouttalking about a lot of those
concepts with you.
And I've been evaluating a wholebunch of these AI agent
frameworks lately.
And I just wrote a little paper,I'll share that after the call.
Andreas Welsch (01:37):
Yeah,
absolutely.
And I think that gets us rightto the meat of the topic, right?
There are so many differentframeworks already out there,
some by large commercialvendors, some by communities, by
startups.
How do you even know which oneyou should pick?
What have you found and whatwould you recommend?
Where should people start?
Kiumarse Zamanian (02:00):
I think it's
really important to understand
first of all, what are theseagents and what are they
supposed to do and what kind ofa technology stack is really
required to support these AIagents?
A little bit of quickbackground.
Agents have been when I was atschool, graduate school, we had
these agents that were based onrules that were using heuristics
(02:21):
and there were some workflowsthat were manual in terms of how
these agents are supposed tocommunicate with one another.
What differentiates this newgeneration of agents is that
they're really leveraging theselanguage models to be more
autonomous and to be able to, infact, plan, learn, correct
themselves, and also be able toreally orchestrate how to make
(02:44):
the decisions.
Even though the human is stillin the loop, I think what we
call human in the loop now, Ithink the goal is to try to have
the human on the loop.
So to have these agents actuallycollaborate with one another
relatively seamlessly andindependently.
So think of it as as you seelike a project manager trying to
(03:06):
get something, want to set up ameeting for you to brief your
clients about the latestfeatures in a particular
product.
Then you basically task yourteam and say, This event will
happen.
This is what we're going todiscuss.
You may not even tell them whatwe're going to discuss.
You're just going to give themsome kind of high level
objective.
And then this team willbasically go figure out among
themselves how to organize thatmeeting, what kind of content
(03:28):
they're going to present.
That's like putting the slidestogether.
Maybe there's going to be somesort of a venue that they need
to make sure they havereservations, to have food
coming, all kinds of things thatneed to happen, right?
Individually, we were able to doresearch using ChatGPT or Gemini
to do these tasks individually.
But the human was really themain interface with these large
(03:50):
language models.
You basically, what they call azero prompt.
You basically issue a prompt,you get a response back, and
then you issue another prompt.
And most of the time thesethings may contain the context,
etc.
But now you're actuallydelegating these tasks to
agents.
That among themselves need todecide how to break particular
objective into multiple tasksand find out if there's an agent
(04:13):
that is actually specialized ina particular task.
If so, delegated to thatparticular agent, or maybe
there's some fun function thatyou need to call to basically
get that information.
A lot of times you may need realtime information about flight
information or hotelreservations or food, et cetera,
that large language model willnot be able to help these
functions and tools also builtin.
(04:34):
The way to look at it is thestack that is required.
To build these agentorchestration really is composed
of really six layers.
At the bottom, you have thefoundation models that we're all
familiar with.
The ones from OpenAI andMicrosoft, AWS etc, So these are
basically a lot of big companiesare spending millions of dollars
(04:58):
to train these large languagemodels, right?
Along with those, you may evenhave a smaller specialized for a
specific area, right?
So this is your bottom layer.
That's like where you're, a lotof your learning has happened
and inference will happen inthat, at that level.
Then you have got you got tobuild memory and tools that will
support your agents.
Just having a chatbot sitting ontop of an LLM it might be
(05:21):
sufficient for just basicallygetting some simple answers, but
if you want to have context andbe able to maintain what type of
information has been passed backand forth between these
different agents.
You need to have memorymanagement.
You need to be able to maintainstate.
So this becomes very similar tohow we develop programs.
And when you write software,there's all kinds of state
(05:42):
management, et cetera, involved.
So that layer sits on top of thefoundation walls.
And then on top of that, youbasically have these agent
frameworks that are able toutilize.
The memory and state and thelarge language model, et cetera.
So these frameworks basicallysit on top and are able to help
you to create agents and haveagents communicate with one.
(06:04):
We've seen many examples thathalf last year or so in this
area, particularly fromMicrosoft AutoGen and LangChain,
we're all familiar with.
And then after that, LangGraphand LangFlow.
So these, are basically,frameworks that help you build
agents and have them talk withone another.
But then you, once you actuallybuild these agents, you didn't
actually host these agents.
(06:26):
So you basically expose them topeople who are going to use them
either through APIs or throughUIs or some way that basically
they become services that areavailable, right?
So this becomes service forhosting these agents, et cetera.
Then once you actually havethese hosted, you will have some
domain specific agents, right?
So you might actually build anagent that is very much
(06:49):
specialized in dealing withmarketing campaign management.
Or you have another one that isvery specific to maybe some
specific retail or travel orhealth, or you name it, you will
have these very domain specificagents.
Now you have this kind of awhole host of agents, some of
them might be platform specific,some of them might be working
(07:09):
only within a Microsoft AutogGenenvironment, some of them in
Google, etc.
Then you need to make sure theseagents are collaborating with
one another.
So again, the analogy withhaving people.
They have special skills andthey have this special expertise
to do things.
These people need to collaboratewith one another just as we are
collaborating now.
These agents need to be able tocommunicate, pass information
(07:30):
back and forth, be able toutilize standards.
So that's where you get to thesemulti agent orchestration
frameworks that are justemerging that will really help
you grab agents and have themcommunicate so that they can not
only be able to use largelanguage models and small
language models to basicallyfind information for you, but
(07:54):
also be able to call functions,delegate tasks between each
other and make decisions andlearn what they have self
correct and be able to doplanning, et cetera.
There are a lot of activity inthis area of the multi agent
orchestration frameworks thatare just happening at a recent.
The particularly with big playjust announced, AWS just
(08:16):
announced theirs.
Crew AI has certainly moved intothis area.
And some of the other playersthat are core focused on the
domain specific like cognizant aneural AI platform, they have
very specific areas.
So I think it's a long answer toactually evaluate how we're
going to use these orchestrationframeworks and be able to find
(08:41):
out which ones really you haveto identify the problem that you
want to solve, just likeanything else that you want to
find the solution.
Is this going to be very domainspecific?
Is this going to be veryspecific to it?
Do you need to build somethingmuch more enterprise level that
goes across organizations, etcetera?
So I think from that perspectivecertainly having this kind of an
openness and the different modelsupport, you want to have
(09:04):
something at least at theenterprise level that is can
scale.
You can actually have opennessthat you can choose different
components that are appropriatefor meeting your needs.
Second thing you want to makesure that you've got the right
mechanisms to have control andgovernance, right?
This is extremely importantbecause you don't want these
(09:24):
agents to just be a hundredpercent on their own or be
reliant on humans you want to beable to delegate the automation
to these agents and put theright guardrails and governance
rules in place so that you canautomatically, or some agents
actually are specifically taskedwith monitoring what other
agents are doing, whatinformation they're passing to
(09:45):
one another, etc.
Governance is extremelyimportant, particularly in some
areas that may deal with healthcare privacy, as we all know, is
extremely important beingretrieved.
Maybe one group may have accessto certain information, etc.
So that all needs to be builtin.
And it's extremely important tohave a framework that does give
you the flexibility to definethese rules and governance
(10:08):
rules.
And sometimes some of these likethe as I mentioned, some of the
domain specific frameworks, likethe one from Cognizant, actually
comes in with a lot of thecontrols built in and the
governance built in, becausethey're very much focused on
very specific domains, but ifyou're dealing with something
like it, LangChain or Crew AI orAutoGen, et cetera they do have
some governance capabilities,but that's an area that's
(10:30):
extremely important and it'sevolving.
Then control certainly goes handin hand with governance rules.
Which agent is in control ofwhat, and how do you actually
bring a human into the picture?
I was reading some news thatapparently there were some
agents that were developed to godo some, I think, reservation or
do some research or something,and they were self checking to
(10:53):
see if they actually found theright information or not.
And they were continuouslyfailing the check and the they
were just being creative.
Checking with the large languagemodel to see what the next thing
that they should, they werescraping websites, finding email
addresses and start sendingemails to people, which was
really crazy, right?
So these agents were just goingrogue and doing all kinds of
crazy things, right?
(11:14):
So you need to be very carefulabout that in terms of control.
Let me just pause there.
I've got a few more items tocover.
Andreas Welsch (11:20):
Yeah.
Let me jump in real quickbecause I think there's a you
said in, in many ways, theseagents we can think of them as
human colleagues or as anequivalent in in the sense that
helps us think about how we caninteract with them.
And then my question is we allsign a code of conduct when we
(11:42):
join a company or join a largecompany.
There's some kind of anonboarding, there are some
ethical standards, someexpectations, there's some
professional standards if youthink of finance and IFRS and
others.
How do agents first of allacquire that knowledge to know
that, hey, these are theparameters of my role, I need to
know what IFRS and only actwithin, and then to also act
(12:05):
ethically in alignment with,first of all, but also general
principles of how we dobusiness, so they optimize for
the right things, right?
Additional revenue, and I canincrease the margin, but I might
do that as an agent at theexpense of my customer
satisfaction.
Exactly.
Yes.
What have you seen to make surethat these agents actually act
(12:29):
ethically?
Kiumarse Zamanian (12:32):
Yeah, that's
an excellent point, Andreas.
I think ethics is certainlysomething that could be
formalized in certain rules, butI think it's sometimes depending
on different, maybe within acompany, you have certain very
specific rules certain culturesI know at Walmart, there was a
tone of voice when the chatbotstalk with the customers, it had
(12:55):
to actually comply with the toneand like the way that the
Walmart All right.
Customers need to be treated.
I'm sure, in health care, youhave similar things.
People's ethnicity anddemographics and various
different things that ethicscomes into play.
As I said, if these things couldactually be put into some rules
(13:18):
that could be checked by certainagents as one way sometimes you
actually have some of thesemodels if there's a model.
That is fine tuned, or you havea small language, you can in
fact build in some of thosebasically the conversations that
you've had.
For example, let's say there's alaw firm that is developing some
kind of a chatbot that willmaybe help the associates or
(13:41):
maybe the clients with certaininformation.
And they want to make sure thattheir information they're
providing is not biased or someHR agency, right?
If you actually take this modeland then train it on very
specific dialogues or type ofdocuments that have been used
that are safe.
So you basically has beentrained to actually know how to
(14:02):
interact using the right ethics,et cetera.
But you want to be very carefulthat you want to have some
guardrails, very specificguardrails, that when a response
actually is generated an agentshould check against certain
rules, right?
Does it meet certain criteria,et cetera?
And as long as those rules arecodified, then these agents
certainly can now do that.
So I think that's where youreally want to make sure that
(14:23):
you've got a framework thatallows you to build in those
kind of Governance rules andalso ability to maybe to call
out to a certain APIs andthings.
Maybe you have services that youhave developed already.
There are lots of companies thatare actually focusing on just
governance and ethics and thingsthat there might be services are
available that these tools cancall to check, right?
(14:43):
So there, these are case by caseand you got to make sure that
your framework supports thesekinds of different scenarios and
it's flexible enough.
Andreas Welsch (14:52):
That's awesome.
And I would love to come back toanother point that you made
earlier that eventually we willhave different teams of these
agents and virtual teams andmaybe to some extent, even
virtual organizations, fromstarting with individual tasks
to departmental or teams,departmental, interdepartmental,
(15:13):
marketing talks to finance, whatis my budget?
Can I get a little more for thiscampaign?
Here's why I think we canachieve better ROI or drive more
customer demand to evendifferent companies having a
sales agent and procurementagent that figure out, do you
have the product?
Can you give me a discount?
When can I get it?
These kinds of things, even ifthat is still a little further
(15:34):
out.
Those last two phases ofinterdepartmental intercompany.
What are you seeing in terms oforchestration, in terms of
getting these different agentsto talk to each other?
Because I think also the realthe reality in large
organizations is that theremight be specialized agents by
one vendor for more customerexperience topics that might be
(15:56):
agents specializing on yourfinance agents on, your HR
products, wherever these vendorsbuild in these capabilities,
plus, like you mentioned, theCrew AI, AutoGen, other
frameworks that are built in ontop.
Are there any kinds ofstandards?
Is there an ecosystem that yousee that either exists or that
needs to exist for, that tohappen?
Kiumarse Zamanian (16:18):
Yeah, that's
an excellent question.
And I think that area is justevolving.
I come from the enterpriseintegration.
I come from the times that theobject model from Microsoft and
there was CORBA from objectmanagement group.
And a lot of people werestruggling building object
models that were standardizedand came and dominated.
(16:38):
And with CALM that gives youactually a fairly flexible way
of.
Defining these objects and beingable to integrate.
And also at Oracle, it was a bigdeal to have these, all these
different applications to beintegrated with.
Now they're facing the samechallenges there.
And certainly with the mobileindustry, a lot of apps being
developed and how would theseapps actually live in an
(16:59):
ecosystem that they certainlyhave standards based on if
they're using Android or ifthey're using iOS, et cetera.
So there's some interruption.
I think there's a standard fromOpenAI is a JSON schema standard
in terms of how you're able topass messages to a large
(17:19):
language models, and certainlysome of the communications are
also getting to some extentstandardized, right?
Just like anything else, no HTTPprotocol all these other
protocols that emerge forinternet.
I think we're going to see someinteroperability protocols
emerging for agents tocommunicate with one another.
(17:39):
At the end of the day theseagents are basically intelligent
abstractions of softwareabstractions that are able to do
certain things certainly they'reable to communicate with the
memory and things that arewithin the framework itself.
Now we're talking about whathappens if you actually have an
agent community, right?
So you publish your agent.
Let's say I in fact, there's a Ithink there's a website that's
(18:02):
called age agents.ai or there'sactually a place that you can go
and develop agents and publishthem, right?
So it's becoming the ecosystemof agents.
You know, you can go and grab asmany agents as you want and then
you have an orchestrationplatform that you put these
agents together to accomplishsomething much more extensive.
So for these agents to be ableto communicate with one another
(18:22):
and plug and play to fit themtogether like a jigsaw puzzle
you're going to need to havesome interoperability standard.
I think those are emerging rightnow as we speak, but there's
going to be, as there are majorplayers in this and they want to
have dominance in this area,right?
So I think Microsoft is going toprobably push for its own agents
so is AWS, so is Google.
Crew AI is trying to be a littlebit more like a Switzerland and
(18:45):
making sure that its agents arecommunicating.
People that have the tendency tofocus on a particular vendor
especially in large corporationsbecause enterprises, they don't
want to take any risks dealingwith unproven maybe
orchestration framework.
There's going to be some timefor some of these smaller
players to gain ground, but Ithink we definitely need some
(19:06):
industry initiatives to be ableto define these interoperability
standards between agents.
Andreas Welsch (19:12):
Thank you for
sharing that.
Now, folks, we're coming upclose to the end of the show.
I was wondering if you cansummarize the key takeaway for
our audience today.
What's the number one thingpeople should take away when
thinking about agents andevaluating what fits best to
their business?
Kiumarse Zamanian (19:28):
So I think
the best thing is to really
understand what these agents aregood for, right?
I think this, we're basicallylooking at the kind of viewing
AI more from a building systemsperspective now.
So for people who actually arefamiliar with building systems
and putting things togetherthink of these things as no
abstractions that are able toaccomplish things.
(19:48):
And the main, obviously the maindifference here is that you're
really relying on these languagemodels that the helping you with
the inference or helping youwith the planning the advantages
that you don't really have to doa lot of times do any
programming.
There are frameworks thatactually there's a lot of them
drag and drop low code or nocode at all that you can build
these things.
Really understand what is itthat you start experimenting
(20:11):
with these there's some toolsout there that you don't have
any, you don't need anyprogramming experience and
they're actually, you can go tosome of these companies websites
and start experimenting.
I think experiment what theseagent frameworks are good for,
what you want to accomplish, andcome to it from a project, maybe
(20:32):
development, project managementperspective break down your
tasks start experimentingdefining your agents, giving the
persona, give them the the tasksthat they need to be
specifically good at the toolsthey need, maybe APIs, et
cetera.
So you like a, yeah, high leveldesign of a project and then
breaking it down into thesethings experimentation.
(20:53):
And then I think more thananything, learning about what's
going on.
I think I'm reading quite a bitabout this.
There's some really good peopleon LinkedIn in this space that
is really important to follow.
But more importantly, I thinkit's just get your fingernails
dirty a little bit if you wantto play around, as I said,
sometimes you don't even need tohave any programming experience
to just basically learn what'sgoing on and try things out.
(21:15):
So I'm really going to go big onagents a framework that supports
it's open, it's easy to use,particularly if you're not a
programmer.
It does have the very usableinterface.
It's user friendly.
It's easy to install, easy tomaintain.
Some of these tools sometimesrequire a fair bit of technical
expertise to even install them.
(21:37):
Depending on how much knowledgeyou have.
Pick something that you'recomfortable with and then start
experimenting.
And scalability is veryimportant.
I think it's one thing, but youwant to put something in
production that's going to havethousands of people using it or
managing gigabytes of data.
Then the scalability comes intoplay and and ecosystem openness
(21:58):
and flexibility.
All these are important.
As I mentioned, I have a paperthat I've written that I'd be
happy to share after this callwith the participants and others
that I've discussed these atlength.
Andreas Welsch (22:09):
Alright,
Kiumarse, thank you so much for
being with us today, for sharingyour expertise with us, and for
those in the audience, forlearning with us.
Kiumarse Zamanian (22:15):
Thank you so
much.
It was great to be here.
Thank you.
Andreas Welsch (22:17):
This is the last
episode of the year.
I can't believe that the yearhas gone by so quickly.
Next year we'll start seasonnumber four of What's the BUZZ?
And we already have someexcellent guests lined up.
I'll share you more about thatin a couple of weeks.
If you do celebrate holidays inthe next couple of weeks, Happy
Holidays and a Happy New Year!