Ep35: Vector Databases + Building Effective Agents

In this episode, there are two topics I'm looking forward to diving deeper into: vector databases and AI agents. I'm particularly interested in understanding how vector databases work, how they work with data, and their role in AI applications.

Then I share my thoughts on Anthropic’s article about Building AI Agents, which discusses their varying definitions—from simple workflows to fully autonomous systems—and provides practical examples. The article highlights the importance of starting simple, adding complexity only as needed.

Episode Transcript

(00:05):
You are listening to the BreaktimeTech Talks podcast, a bite sized tech
podcast for busy developers, where we'llbriefly cover technical topics, new
snippets, and more in short time blocks.
I'm your host, Jennifer Reif, anavid developer and problem solver
with special interest in data,learning, and all things technology.
There are a few things sitting on myradar for future content that I hope to

(00:28):
spend some time exploring and learning.
Two of those things are vector databasesand AI agents, and both of these kind
of dovetail really nicely together.
We'll talk a little bit more aboutthat, but vector databases are a key
component of AI stacks, and I'd like tobetter understand how they work, both
their strengths and their weaknesses.
While AI Agents are about usingLLMs or large language models in

(00:53):
certain decision making scenarios.
I read a really great article fromAnthropic that goes into depth
on the many different use casesand architectures for AI agents.
To kick us off though, I wanted to startthis conversation with vector databases.
Vector databases really took off whenthe beginning of the wave of AI started,

(01:13):
mostly around the beginning of 2023.
So, ChatGPT released in theend or November of 2022, and
then you really started to seeRetrieval Augmented Generation,
or RAG, come into play in 2023.
And this is really where vectordatabases started to excel, and really
just hit hit the market big time.

(01:34):
As a database person, I know thatthey do store data differently
than a graph, obviously, as withkind of any type of database,
document databases, key valuestores, relational databases,
graph databases, so on.
And I also know that vectordatabases are optimized for very
fast retrieval of vectors, whichare numeric representations of data.
If you want to go into a littlebit more detail on what vectors and

(01:56):
embeddings are, I do talk about that
in episode 10 of this podcast.
So check that out if you want toget a background or a refresher on
what embeddings and vectors are.
Then vector databases storethose vector representations.
As with a lot of database vendorsat the time, Neo4j also implemented
vector support in the database tostore and query vector data as well.

(02:20):
That was not something that graphdatabases, relational databases,
document databases, other typeswere naturally fit to store.
But most database companies at the timestarted adding vector support so that
then they could basically add thesecapabilities on as well and utilize
vectors and run similarity searches.
But my question is, how doesa native implementation differ

(02:42):
from a non native implementation?
These vendors that tacked onvector support to their databases,
how is a native implementationdifferent, better, worse, et cetera.
What are their strengths and theirweaknesses with a firsthand experience?
I have heard that native canbe stronger in certain cases,
which makes valid sense, right?
I wouldn't want to put a graphinto a non graph database.

(03:04):
They are designed for some veryspecific strengths and then
also have some weaknesses too.
But I really want to know what arethose strengths of a native vector
database, and then does that actuallyhold true for the data sets that
I typically work with and the usecases I like to test out or explore.
When is it actually better to havea vector database versus a graph

(03:25):
database or another type of database?
What use cases do they work well for?
What data sets, what questionsand problems and analysis do they
handle really well and othersthat they don't quite so much?
These are the questions that Ireally hope to invest some time in
answering by working hands-on with thetechnology, hopefully sometime soon.
Again, I have a little bit ofbackground, but I hope to dive

(03:47):
deeper and actually play with thetechnologies, pull some data in,
query it, explore it, maybe connectit up to some gen AI applications
and see where that takes me.
The piece of content dovetails actuallyreally nicely with this topic of vector
databases because it's Anthropic'sarticle called Building Effective Agents.

(04:08):
Now the article goes into a definitionand says that the definition of
agents varies quite a bit, whichactually is what I'm aware of as well.
It's what I've been exposed to.
Depending on who you're talking to,you'll get answers and different
definitions of what an agent actually is.
These vary from very autonomous,almost non touched or non intervention

(04:32):
systems, to something that's...has alot of interventions and gate checks
and kind of other processes or othersystems or applications that pop in
and check or combine different answersand tasks that the LLM has completed.
Anthropic also backs this kindof concept or what I know up that
says that, agents, the definitionof an agent varies quite a bit.

(04:55):
Some will define it as fully autonomoussystems where others, they'll define it
as more prescriptive sets of steps withpotential interactions from other systems,
for instance, like human intervention.
The article also talks a little bitabout when and when not to use agents.
They recommend the simplest solutionand then only adding complexity as

(05:16):
needed, which I am a huge fan of.
Start simple first and thenadd on that complexity only as
needed and only if you need it.
So I really loved this approach.
They followed up with that a fewdifferent times throughout the article.
Start simple and only buildin the complexity that you
need and when you have to.
Then they go into detail on thedifference between workflows and agents.

(05:38):
They separate the idea of morefully autonomous systems and more
prescriptive sets of steps withlayers of interventions, that
being workflows, and then the moreautonomous stuff being agents.
And they give lots of different examples.
They step through a tonof different use cases.
They're very thorough, very comprehensive,and I really thought this was

(05:59):
extremely helpful to see all thedifferent ways that you can use this
general concept of agents, but asthey've separated workflows and agents
in a variety of different scenarios.
So lots of examples moving frommost intervention heavy workflows to
the most autonomous agentic systemsand kind of everything in between.

(06:20):
They start off with workflows with thosethat have the most intervention, things
like retrieval augmented generationtype of systems where you have a lot of
an LLM completes a task, you might havea verification or a gate check before
that response goes out, and then youmight have an LLM completing a few
different tasks and then bringing allthose together and checking those and

(06:43):
then summarizing them towards the end.
So again, multiple steps that the LLMis doing, but some other interactions
or intermediary steps in between.
And then that kind of moves, once youget past all the workflows and there
are a bunch of them, then you moveinto kind of the agents, which are a
little bit more autonomous, a littlebit more where the LLM is making not

(07:04):
only completing the tasks, but it'smaking decisions about those tasks and
what it needs to do to complete those.
And it goes into several moredetails, lots more examples
on the most intervention
of agentic systems to the mostautonomous of agentic systems.
Agents are, at the base level, somekind of LLM decision making, so there's

(07:28):
some kind of decision making processgoing in to the LLM's answers. And
that's what agentic systems mean,and this is exactly what Anthropic's
article backs up. But again, they'redividing that the least autonomous
segments or examples, use cases are goingto be a little bit more like workflows,

(07:48):
much more process oriented, multiplesteps and so on, where the most autonomous
send it something and let it do abunch of things and spit out a response
where there's very little intervention.
Those are going to bemore agentic systems.
Again, goes into that detail supportseverything that I've read so far.
It clears my verificationchecks, if you will.

(08:09):
And I thought this was reallyinteresting, extremely thorough.
It is probably going to be about a 10or 15 minute read, so it does go into
a lot of detail, but it doesn't takeup too much of your time either and
really gives a fantastic overview of indepth knowledge and all the different
cases where agentic systems are used.
As always, I'm constantlylearning and exploring.

(08:32):
Vector databases and agentic systemsare something I hope to learn more
about and explore for myself to actuallyrealize the value and the way that they
fit into the larger tech ecosystem,
and of course, in Gen AI. Thanksfor listening and happy coding.