April 11, 2025 • 10 mins
Fresh from the Arc of AI conference, I’m unpacking the biggest insights that stuck with me—ranging from the extremes of AI’s capabilities to the deeper implications for how we build and maintain our tech systems. I’ll also share a new blog post and code repo I published on loading data into Pinecone, some next-gen tools I’m eyeing, and thoughts on a great article from the Redis blog about why vector databases aren’t enough. Navigate the evolving landscape of LLMs, generative AI, and modern infrastructure with me in this episode.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:05):
You are listening to the BreaktimeTech Talks podcast, a bite-sized tech
podcast for busy developers where we'llbriefly cover technical topics, new
snippets, and more in short time blocks.
I'm your host, Jennifer Reif, anavid developer and problem solver
with special interest in datalearning and all things technology.
(00:25):
After a conference last week,I've had some time to digest
everything that I learned.
In this episode, I'll highlight a fewthings that surfaced for me from the
AI event, and then I'll talk about somecontent that I got published this week and
some next tools that I want to explore.
Then keeping with our AI theme, Icame across an article that talks
about why tech solutions needmore than just a vector database.
(00:48):
The first thing I wanna kick thispodcast off with though is a shout out
to the Arc of AI conference last week.
This was a fabulous event withsome really inquisitive minds
and really thoughtful questions.
I really enjoyed a lot ofthe things that surfaced.
There were some things that I had to thinkabout or some really deep and intriguing
questions and topics that I came across.
(01:11):
I'm sifting through all of thatinformation now and hopefully will
have, some content or tools or thingsthat I explore next based on that.
First though, is that I wanna doa shout out to the organizers.
They did an amazing job with the schedule.
The sessions were a perfect length,there were good breaks in between.
And then of course, the layout ofeverything, the food and the location
(01:32):
was all spectacular too, which always,makes a conference even more fun.
They've already scheduled the eventfor 2026, so if you're interested
in getting in on the fun for nextyear, I would start planning early.
You can already start.
The first topic that came outof this event, though, for
me, was the extremes of AI.
A lot of us see and understandthat AI seems like a magic solution
(01:57):
to a ton of problems, right?
We wanna toss AI at everything andjust see where it's super powerful
and where it kind of falls short.
And those are things thatwe're continuing to learn.
We've already figured a fewof those things out, right?
But we still see AI as the endall, be all magic solution.
And then on the flip side of that,we also see that AI has a lot
of added complexity in a lot ofways and some limitations still.
(02:20):
There's things that it's really doesnot handle so well, or solutions that
we try to stick it into that it'sjust not designed or meant to be in.
I think what came out of this conferenceis that we have to remember that
LLMs are a series of probabilities.
They've been trained on a huge corpus ofdata, and they mostly retain and predict
(02:42):
the next bit of word or solution orwhatever it is, whatever type of model
you're working with, whether that's textor images or some other type of modality.
And so when we're talking to andworking with an LLM, it's really all
about figuring out the probabilitiesand guiding it into the right area.
Now, I mentioned in a previous episode.
(03:05):
As a side note, completely unrelatedtopic, that someone, when they are
playing a game, they wanna pick thehighest probability plays or moves
right, in order to win the game.
You go through, you're like, oh,I've got these, like if you're
playing chess for instance.
Oh, I have these several options thatI can pick from for my next move.
Which one has the highestprobability of putting me in a
(03:27):
winning situation at the end?
We do that with sports or chess orboard games or whatever it is we tackle,
and why shouldn't we do this whenworking with tech solutions, especially
thinking like working with an LLM.
How do I get this LLM to hit theproper path of probabilities to
give me the solution that I'mlooking for on the other side?
(03:47):
And I found this crossover a littleinteresting and kind of fun as well.
The next thing is, that is somewhatrelated, is that traditionally we've seen
technical systems as being deterministic,they're ones and zeros, right and wrong.
If they're not one thing, they're another.
Where now, in this world of AI,we're starting to see that generative
(04:11):
AI is non-deterministic, whichmeans that you can't always predict
and have a consistent outcome.
LLMs are based on probabilities.
There're going to be non-deterministic.
There's gonna be some variation in theend result or solution that you get.
And this is really a shift in theway we've designed our systems,
(04:31):
integrated our systems, thoughtabout our systems from the past.
Integrating an LLM now requires a newapproach to topics like architecture
and testing, and so many other facetsof our technical systems that we're
starting to see some of this, andthere's realization in this area.
But it really hit me that AI impactsso much more than just cool solutions.
(04:56):
It really impacts our entireinfrastructure of software design and
implementation and maintenance afterwards.
After all of this, at the conferencelast week, I started to put together
some of the content that I've beenwanting to publish for a while.
The first piece of this was justgetting data loaded to Pinecone, so
(05:17):
I put up a code repository as wellas an accompanying blog post this
week that is finally published.
So that should be available.
It walks through the whole journeyof how I ended up getting data into
Pinecone, which was a much longerand more complicated journey than
what I anticipated or had hoped.
But, I came across some good realizationsalong the way, and hopefully some of
the tidbits that I came across willhelp somebody else if you are looking
(05:40):
to load data into Pinecone as well.
There were also were some other toolsthat I'm starting to look at now, after
coming out of the AI event last weekand seeing some of the latest tech and
things that are going on in the industry.
I really would like to look atsome of these coding tools that
seem to be all the rage right now.
And explore more on the thingsthey can do, their strengths, their
(06:02):
weaknesses, the way that I caninfluence my daily workflow as well
for implementing some of these things.
And then looking a littlebit at MCP as well.
You've probably heard thisterm if you're kind of keeping
up with the AI space at all.
They are also super popular right now andI really just wanna understand why that
is and how they're useful and what thesetypes of things mean for integrating them
(06:25):
into our tech systems and our daily work.
The article that I wanted tohighlight this week is one from
the Redis blog that is called "YouNeed More Than A Vector Database".
I really loved this article.
It had raised some really great pointsand things that actually overlapped
really quite well with the Arc of AIevent I was at last week and some of
(06:45):
the topics we've already discussed.
So I'll walk through this.
We're gonna keep with ourAI theme for this episode.
And then of course, there's a littlebit of plug for Redis AI at the end.
This is a Redis blog after all.
They're gonna have a littlebit of a marketing plug there.
But again, some of the points raisedhere are super interesting and I think
super valid and something that weneed to consider or talk more about.
The first in the article is that thistechnology for vector databases and
(07:10):
vector similarity search has existed fora while, but the latest AI wave has really
made them vital, fast, and easy to use.
However, the author makes the casethat most of our use cases still
need a combination of vector pluslexical (or keyword) search to
make them flexible and useful.
And if you think about this, mosttechnologies don't do one particular
(07:33):
thing in one thing only, right?
You have to build in some kind ofwell-rounded set of functionalities.
This exists for any database.
It exists for any technology toolor implementation or architecture.
You need it to be able to handlemultiple different things,
flexible keyword searching, fuzzysearching, all that sort of stuff.
Even just for the lexical side of things.
(07:54):
Vector really is no different.
It's one component in a vastarray of functionalities that
need to be covered when you'redoing data searches in general.
Then the author goes into the factthat vectors and embeddings sound
fancy, but here's the definitionhe gave that I really loved.
"Vectors are numericalfingerprints of data".
(08:14):
And the author says that wetend to overcomplicate things.
We wanna make things sound morecomplicated than they actually are.
But I really loved the idea that weshould think of vectors as a unique
fingerprint for a piece of data.
I thought that was really cool.
The author goes into a littlebit about the vector math and the
high level concepts there, buthe goes on to say that the vector
(08:36):
math piece isn't that difficult.
What's difficult is what comesafter all of that, right?
Vector databases make similarity searcheasy and fast, but there are other
challenges that need to be overcomefor more complete and robust solutions.
The challenging bit is in theimplementation, the availability,
the resiliency, the data load,the real times transactions,
(08:56):
the operations types of things,
Vector databases are justone piece of that puzzle.
The author points out there's some thingsin the testing and evaluation processes
that need to adapt in order to handlethese new systems, which was something
that I actually just talked aboutthat came out of the Arc of AI event.
We have to start incorporating new typesof processes and steps and checks and
(09:20):
balances into our technology systemsin order to implement and integrate
these LLMs and generative AI solutions.
I really thought that there were someincredibly valuable points made in this
article, and we need to think bigger thanjust a particular database or a similarity
search functionality when we look atintegrating these new types of systems.
Of course there are thingsthat are being progressed on.
(09:43):
There are some changes being made.
There's some good workgoing on in the space.
But as with anything, we have toremember that it's not a silver bullet.
This week allowed me some time tosift through all the great ideas
and thoughts that bubbled up fromlast week's arc of AI conference.
I was also able to catch up on somecontent about my journey through loading
data into Pinecone with Spring AI.
(10:05):
We rounded out the episode steppingthrough the Redis blog post about why
we need more than a vector database.
As always, thanks forlistening and happy coding.
You are listening to the BreaktimeTech Talks podcast, a bite-sized tech
podcast for busy developers where we'llbriefly cover technical topics, new
snippets, and more in short time blocks.
I'm your host, Jennifer Reif, anavid developer and problem solver
with special interest in datalearning and all things technology.
(00:25):
After a conference last week,I've had some time to digest
everything that I learned.
In this episode, I'll highlight a fewthings that surfaced for me from the
AI event, and then I'll talk about somecontent that I got published this week and
some next tools that I want to explore.
Then keeping with our AI theme, Icame across an article that talks
about why tech solutions needmore than just a vector database.
(00:48):
The first thing I wanna kick thispodcast off with though is a shout out
to the Arc of AI conference last week.
This was a fabulous event withsome really inquisitive minds
and really thoughtful questions.
I really enjoyed a lot ofthe things that surfaced.
There were some things that I had to thinkabout or some really deep and intriguing
questions and topics that I came across.
(01:11):
I'm sifting through all of thatinformation now and hopefully will
have, some content or tools or thingsthat I explore next based on that.
First though, is that I wanna doa shout out to the organizers.
They did an amazing job with the schedule.
The sessions were a perfect length,there were good breaks in between.
And then of course, the layout ofeverything, the food and the location
(01:32):
was all spectacular too, which always,makes a conference even more fun.
They've already scheduled the eventfor 2026, so if you're interested
in getting in on the fun for nextyear, I would start planning early.
You can already start.
The first topic that came outof this event, though, for
me, was the extremes of AI.
A lot of us see and understandthat AI seems like a magic solution
(01:57):
to a ton of problems, right?
We wanna toss AI at everything andjust see where it's super powerful
and where it kind of falls short.
And those are things thatwe're continuing to learn.
We've already figured a fewof those things out, right?
But we still see AI as the endall, be all magic solution.
And then on the flip side of that,we also see that AI has a lot
of added complexity in a lot ofways and some limitations still.
(02:20):
There's things that it's really doesnot handle so well, or solutions that
we try to stick it into that it'sjust not designed or meant to be in.
I think what came out of this conferenceis that we have to remember that
LLMs are a series of probabilities.
They've been trained on a huge corpus ofdata, and they mostly retain and predict
(02:42):
the next bit of word or solution orwhatever it is, whatever type of model
you're working with, whether that's textor images or some other type of modality.
And so when we're talking to andworking with an LLM, it's really all
about figuring out the probabilitiesand guiding it into the right area.
Now, I mentioned in a previous episode.
(03:05):
As a side note, completely unrelatedtopic, that someone, when they are
playing a game, they wanna pick thehighest probability plays or moves
right, in order to win the game.
You go through, you're like, oh,I've got these, like if you're
playing chess for instance.
Oh, I have these several options thatI can pick from for my next move.
Which one has the highestprobability of putting me in a
(03:27):
winning situation at the end?
We do that with sports or chess orboard games or whatever it is we tackle,
and why shouldn't we do this whenworking with tech solutions, especially
thinking like working with an LLM.
How do I get this LLM to hit theproper path of probabilities to
give me the solution that I'mlooking for on the other side?
(03:47):
And I found this crossover a littleinteresting and kind of fun as well.
The next thing is, that is somewhatrelated, is that traditionally we've seen
technical systems as being deterministic,they're ones and zeros, right and wrong.
If they're not one thing, they're another.
Where now, in this world of AI,we're starting to see that generative
(04:11):
AI is non-deterministic, whichmeans that you can't always predict
and have a consistent outcome.
LLMs are based on probabilities.
There're going to be non-deterministic.
There's gonna be some variation in theend result or solution that you get.
And this is really a shift in theway we've designed our systems,
(04:31):
integrated our systems, thoughtabout our systems from the past.
Integrating an LLM now requires a newapproach to topics like architecture
and testing, and so many other facetsof our technical systems that we're
starting to see some of this, andthere's realization in this area.
But it really hit me that AI impactsso much more than just cool solutions.
(04:56):
It really impacts our entireinfrastructure of software design and
implementation and maintenance afterwards.
After all of this, at the conferencelast week, I started to put together
some of the content that I've beenwanting to publish for a while.
The first piece of this was justgetting data loaded to Pinecone, so
(05:17):
I put up a code repository as wellas an accompanying blog post this
week that is finally published.
So that should be available.
It walks through the whole journeyof how I ended up getting data into
Pinecone, which was a much longerand more complicated journey than
what I anticipated or had hoped.
But, I came across some good realizationsalong the way, and hopefully some of
the tidbits that I came across willhelp somebody else if you are looking
(05:40):
to load data into Pinecone as well.
There were also were some other toolsthat I'm starting to look at now, after
coming out of the AI event last weekand seeing some of the latest tech and
things that are going on in the industry.
I really would like to look atsome of these coding tools that
seem to be all the rage right now.
And explore more on the thingsthey can do, their strengths, their
(06:02):
weaknesses, the way that I caninfluence my daily workflow as well
for implementing some of these things.
And then looking a littlebit at MCP as well.
You've probably heard thisterm if you're kind of keeping
up with the AI space at all.
They are also super popular right now andI really just wanna understand why that
is and how they're useful and what thesetypes of things mean for integrating them
(06:25):
into our tech systems and our daily work.
The article that I wanted tohighlight this week is one from
the Redis blog that is called "YouNeed More Than A Vector Database".
I really loved this article.
It had raised some really great pointsand things that actually overlapped
really quite well with the Arc of AIevent I was at last week and some of
(06:45):
the topics we've already discussed.
So I'll walk through this.
We're gonna keep with ourAI theme for this episode.
And then of course, there's a littlebit of plug for Redis AI at the end.
This is a Redis blog after all.
They're gonna have a littlebit of a marketing plug there.
But again, some of the points raisedhere are super interesting and I think
super valid and something that weneed to consider or talk more about.
The first in the article is that thistechnology for vector databases and
(07:10):
vector similarity search has existed fora while, but the latest AI wave has really
made them vital, fast, and easy to use.
However, the author makes the casethat most of our use cases still
need a combination of vector pluslexical (or keyword) search to
make them flexible and useful.
And if you think about this, mosttechnologies don't do one particular
(07:33):
thing in one thing only, right?
You have to build in some kind ofwell-rounded set of functionalities.
This exists for any database.
It exists for any technology toolor implementation or architecture.
You need it to be able to handlemultiple different things,
flexible keyword searching, fuzzysearching, all that sort of stuff.
Even just for the lexical side of things.
(07:54):
Vector really is no different.
It's one component in a vastarray of functionalities that
need to be covered when you'redoing data searches in general.
Then the author goes into the factthat vectors and embeddings sound
fancy, but here's the definitionhe gave that I really loved.
"Vectors are numericalfingerprints of data".
(08:14):
And the author says that wetend to overcomplicate things.
We wanna make things sound morecomplicated than they actually are.
But I really loved the idea that weshould think of vectors as a unique
fingerprint for a piece of data.
I thought that was really cool.
The author goes into a littlebit about the vector math and the
high level concepts there, buthe goes on to say that the vector
(08:36):
math piece isn't that difficult.
What's difficult is what comesafter all of that, right?
Vector databases make similarity searcheasy and fast, but there are other
challenges that need to be overcomefor more complete and robust solutions.
The challenging bit is in theimplementation, the availability,
the resiliency, the data load,the real times transactions,
(08:56):
the operations types of things,
Vector databases are justone piece of that puzzle.
The author points out there's some thingsin the testing and evaluation processes
that need to adapt in order to handlethese new systems, which was something
that I actually just talked aboutthat came out of the Arc of AI event.
We have to start incorporating new typesof processes and steps and checks and
(09:20):
balances into our technology systemsin order to implement and integrate
these LLMs and generative AI solutions.
I really thought that there were someincredibly valuable points made in this
article, and we need to think bigger thanjust a particular database or a similarity
search functionality when we look atintegrating these new types of systems.
Of course there are thingsthat are being progressed on.
(09:43):
There are some changes being made.
There's some good workgoing on in the space.
But as with anything, we have toremember that it's not a silver bullet.
This week allowed me some time tosift through all the great ideas
and thoughts that bubbled up fromlast week's arc of AI conference.
I was also able to catch up on somecontent about my journey through loading
data into Pinecone with Spring AI.
(10:05):
We rounded out the episode steppingthrough the Redis blog post about why
we need more than a vector database.
As always, thanks forlistening and happy coding.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.