What does it take to go from leading Kafka development at Confluent to becoming a key figure in the PostgreSQL world? Join us as we talk with Gwen Shapira, co-founder and chief product officer at Nile, about her transition from cloud-native technologies to the vibrant PostgreSQL community. Gwen shares her journey, including the shift from conferences like O'Reilly Strata to PostgresConf and JavaScript events, and how the Postgres community is evolving with tools like Discord that keep it both grounded and dynamic.
We dive into the latest developments in PostgreSQL, like hypothetical indexes that enable performance tuning without affecting live environments, and the growing importance of SSL for secure database connections in cloud settings. Plus, we explore the potential of integrating PostgreSQL with Apache Arrow and Parquet, signaling new possibilities for data processing and storage.
At the intersection of AI and PostgreSQL, we examine how companies are using vector embeddings in Postgres to meet modern AI demands, balancing specialized vector stores with integrated solutions. Gwen also shares insights from her work at Nile, highlighting how PostgreSQL’s flexibility supports SaaS applications across diverse customer needs, making it a top choice for enterprises of all sizes.
Follow Gwen on:
What's New In Data is a data thought leadership series hosted by John Kutay who leads data and products at Striim. What's New In Data hosts industry practitioners to discuss latest trends, common patterns for real world data patterns, and analytics success stories.
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:05):
I'm You Hello, everybody.
Thank you for tuning in to thisepisode of what's new in data.
I'm really excited about our guests today.
We have Gwen Shapira, who's co founderand chief product officer at Nile.
(00:26):
She was previously an engineering leaderfor cloud native Kafka at Confluent.
Gwen, how are you doing today?
Really happy to be here.
We've been chatting about thison off for a while and I'm glad
to finally be talking to you.
Absolutely.
Yeah.
Gwen, we've gone back and forthover the years communicating over
(00:48):
all the latest, greatest tech in,the enterprise and open source.
The first time I saw you.
Give a talk was six years ago,probably about exactly six years ago
at O'Reilly Strata event in New York.
You had hundreds of people in theaudience listening to you talk
about microservices and the futureof ETL with open source and Kafka.
(01:10):
So great to be catching up withyou here on this podcast now.
Really great to be catching up with you.
And I have to say, I miss O'ReillyStrata a lot as a conference.
There are other good dataconferences these days, but I
feel like Strata was special.
It wasn't a vendor specific.
It was really unique.
It was unique.
(01:31):
It was very cool.
It was in Javits Center, whichI always thought was a really
electrifying, exciting place to be.
The practitioners love.
Going there to talk about what theybuilt and it wasn't super vendor heavy
the vendor participation was alwaysreally good and pointed because you
had to talk about how you add valueinto the open source ecosystems because
(01:53):
it was it was always very tied to theHadoop infrastructure in the beginning.
So it always had that.
So I agree.
I do miss that conference.
What are some of yourfavorite conferences today?
Yeah, I have to say my conferenceshas changed a lot because I'm now
like part of the Postgres ecosystem.
(02:15):
Pg dev conf up in Vancouver a fewmonths back was really amazing.
And then, and because a lotof our customers and users
are JavaScript developers.
I've started going to JavaScriptconferences and the ecosystem around
that, which is also very differentand the entire vibe is very different.
(02:40):
So yeah, I don't know if the conferenceI go to these days are the most relevant
for the Data people in your audiencea postgres Conf is definitely a great
experience if you are in that ecosystemAnd the other thing is of course like the
classics the qcon and goto like the bigData or architecture conferences that they
(03:08):
always have a data track, but they talkabout software architectures in a larger
way and This is where I got my excitementabout microservices, for example.
Absolutely.
And you've done great work as aengineering leader for cloud hosted open
source infrastructure for a long time.
So not surprising to hear that now you'rebranching into JavaScript conferences
(03:34):
and Postgres conferences right now.
Nile, the company that you cofounded is doing some incredible
work in the Postgres community.
Tell me a bit about the current stateof Postgres, what people are talking
about there, what's being announced,what are people building with it?
Yeah, so this is really interesting.
So one thing that everyone knows, butit's hard to internalize until you
(03:58):
join the community, is that Postgreshas been around for a very long time.
If you look at the source code, youcan find a lot of if you do get blame,
this line was added 25 years ago, 30years ago, before, definitely before
I was working on anything at all.
(04:22):
So it, and a lot of the people whomade those commits 25 years ago,
they're still reviewing patches.
To this day, they are workingemployed to the company.
Like they're not even retired andthey're just still part of the community.
So you have a real sense of tradition.
(04:43):
I would say in the community and you also.
Can see how the community istrying to keep itself modern life.
So they have a discord now.
It is this is probably a big changefor people who've been working with
the same community and same people for30 years, long before discord existed.
And so it's really fun to seecommunity maintain a lot of
(05:07):
traditions and also evolve.
It's also really interesting to see.
If you look at any particular.
Contribution any particularchange that people are getting in.
It looks really slow.
It, I think a bug fix that everyoneagrees has to be fixed, but is non
(05:30):
trivial to implement can take three,four months to get in a feature that
may be slightly more controversial.
It can be years.
And so you look at the PR and yousay, Oh God, there's this feature and
Oracle had it forever, and they've beenjust discussing it for three years.
(05:51):
Will it ever get in?
But I think in a way it misses thebigger picture because when you look
at releases and Postgres has reallygood process you know exactly to the
day a year in advance years in advancewhen each release is going to happen
And in the few months running up tothe release, you know what features are
(06:15):
very likely going to make it or not.
So it's probably the most predictable opensource project I've ever seen in my life.
Apparently they wereonly late for release.
Once in a few days because they found avery critical bug in something that was
very hard to leave out of the release,something along those lines and it
(06:36):
took them extra few days to completelyrewrite something in order to make it in.
But in, in general, it's been excellent.
It's extremely reliable.
And then when you look at everyrelease, there is always good features.
Now, Postgres is very largeand not everyone needs every
new features that makes it in.
So for example, one of the mostexciting features for this new
(07:00):
release, this is Postgres 17.
So it was released like a few weeks back.
It's still very new.
So the most exciting featuresare that people got crazy over.
is an optimization for queries thatinvolve basically filtering on a list.
So where value in the columnis in and a list of few values.
(07:27):
It used to be that an index hadto be scanned every time for
each value in the list, whichcan get bad if the list is full.
Giant.
And now it can do onescan and find everything.
So this is a cool optimization.
Everyone is very excited about it.
But of course, if you look at the largenumber of workloads, not everyone uses
(07:48):
in, or even if you use in, not it doesn'tlike would you really go through the
upgrade just to get an in optimization?
Is that like the biggestbottleneck in your application?
Whether or not you feel pressure to getthose new features really depends on
your specific workload, which is why yousometimes see people running years old,
(08:12):
like five, six years old, a Postgresin production with no problems at all.
And if you use partitions, don't do it.
Partitions are still, got a lotbetter in the last few releases.
But, yeah, like a lot of workloads.
Will work well on Postgres 12.
(08:33):
I wouldn't recommend running Kafkafrom five years ago in production.
And I think some newer projectsevolve much faster, but Postgres is
depending on what you do, you can befine with older versions, I would say.
Absolutely.
And that's so much of.
(08:53):
Development that goes into databasesis just continued optimizations.
And there's a lot of very sophisticatedwork that goes into optimizing
information storage and retrievaland different types of indexes, and
even using like probabilistic datastructures and things like that.
It's a lot of development, which,and then at the same time, it has to.
Remain resilient.
(09:14):
It has to be backwards compatible.
All these thingsin Postgres is amazing.
Sotwo ways that Postgres testing is amazing.
First, there was the very famousbackdoors that almost made it to SSH.
I think a few months back, this wasdiscovered by one of the core Postgres
(09:35):
engineers, Andrea's friend, on whilehe was testing performance of Postgres.
And he noticed that his tests weregetting slow because SSH ing to the
machine, SSH process was taking moreCPU and was Taking slightly longer.
And this looked weird tohim, so he investigated.
So it's clear that there is a lotof detail orientedness that goes
(09:59):
into, there cannot be any performanceregression in Postgres ever.
And the other cool thing, peoplethese days talk a lot about the
deterministic simulation testing.
And.
Postgres had something very similarto test transaction isolation for
(10:20):
many, years, where they basically,you write some scenarios, the
transactions do things, and thenthey shuffle them, and they make
sure that all the shuffling stays.
Still maintain the same invariance andyou get the same results, no matter how
you move things around, which basicallytests that you have transaction isolation.
And and then at Nile, we actually used thefact that those tests exist to test some
(10:44):
of our transactional guarantees becausewe do some distributed transactions
that are not part of normal Postgres,but we could still reuse their tests.
It's pretty cool that, it wasjust so much ahead of its time.
Yeah, absolutely incredible.
I think that's why it's taking alead in community adoption as the
(11:08):
open source database of choice fora lot of both data infrastructure
companies and then companies areoffering databases as a service to
enterprise and it's super well adopted.
There's no debating that.
So it's great to hear the your perspectivefrom in the weeds of working with
(11:28):
the Postgres development community.
And yeah it's, great context to know thatthere's so much yes, there's rich history.
Databases have to be, in my opinion,developed over 20 to 30 years.
And there's a popular hacker newspost about Oracle's code base it's
this massive 25 million lines ofcode C and there's former Oracle
(11:53):
developers allegedly who chimed in andcommented saying getting one feature
in there just took such a long time.
And I think that's the only.
Way to have a database.
If someone comes in and says, Hey,I built a database from scratch in
the last six months, and with AI orsomething like that they, they code, they
generated it with, that's a big red flag,And now I want to try Zoho, go through
chatGPT and ask him to generate adatabase for me and see what happens.
(12:20):
Yes.
Yes.
The if, chatGPT is smart,it'll decline your request.
Or something like that.
Exactly,like it'll say, no, this
is really complicated.
Yeah.
Then we'll know that AGI is here, butif it actually tries to generate it in
Python or something there's problems.
Yeah.
Sam Altman has some work to do.
But that, that, that's greatto to hear the latest there.
(12:42):
And yeah, curious to hear whatelse is going on with Postgres.
Yeah.
So the other thing is because as yousaid, the community is very careful about
adopting features into core Postgres.
A lot of the interesting, excitingthings are in the ecosystem
with people publishing whatare called Postgres extensions.
(13:08):
For all kinds of scenarios.
So while I was at the Postgresconference, I chatted to a really
nice person who is building a Postgresextension that adds support for Arrow.
And there are multiple extensions withA support for per key file formats.
(13:29):
So if you put those together,suddenly you, there is some big
data operations that just becamea lot more efficient in Postgres.
So this is pretty exciting and noone had to wait on Postgres to decide
whether they want or need a new fileformat or a new memory format or
(13:49):
anything along those lines, because youcan just do so much with extensions.
And then you have extensions that havebeen around for a long time and are
still generating a lot of excitement.
So one, one of them is that I just keptbeing mentioned as an example in the
conference in all kinds of situations.
(14:10):
It's a HypoPG.
It basically allows you to createhypothetical indexes without having to
Spend the time on resources of actuallycreating and also risking production
repercussions for actually having them.
And then just check, how will thisquery behave if I had this index or more
(14:32):
likely, you have a workload that mixes,inserts, updates, selects, all that.
How will it be, will this workloadbe faster or slower if I had.
Those other indexes.
So it's, yeah, I think, I don't knowdatabase people are slightly obsessed
with performance and optimization.
It's a big part of what we care about.
(14:55):
So this is one of the thingsthat we keep coming back to.
There are cool optimizations going onthe network layer optimizations for
making SSL more efficient, which is.
Really important.
It used to be with Postgres.
I don't know if it exactly predatedSSL, but Not only did it originally
(15:17):
not support SSL for its first fewyears because it was really one of
the earliest databases that existed.
For a long time, nobody reallythought SSL was all that important.
It all ran in the data center.
Do I really need SSL betweenevery machine and my data center?
And I think the security positioningof most companies in the last 10
(15:41):
years shifted to a point where,yes, of course you're going
to run SSL between everything.
And also, of course, with all the Postgresas a service, a lot of time you actually
access Postgres beyond firewall walls.
So you obviously need a much higher levelof security and nobody's actually willing
(16:02):
to trade off security for performance.
We want all of it.
So having better SSL support inthe protocol and more efficient
one is obviously important.
Yeah.
That's a great point you brought up andyeah, I'd love to dive into each one
of those, but most if we're just goingLast in first out the point about SSL.
(16:24):
Yes.
And I very important that security teamsare now standardizing that for making
sure that Postgres connections are alwayssecure over networks, I've worked with
enterprise security teams that just don'tallow for instance, like inbound network
(16:45):
connections to an on premise data center,because they know that data centers are
You know, maybe 20, 30, 40 years old.
And for all they know, there's a bunchof databases with one, two, three, four,
five is the password and no, no networkleveling and like none of that stuff.
No patching for no security patchesapplied for the last five years.
For sure.
So now everyone's assuming,okay you're going to get.
(17:09):
Tons of requests coming in overnetworks from cloud or maybe another
data center, most likely cloud.
Now, it's not super controversialto have your enterprise data center
in the cloud as controversialas it was 5 to 10 years ago.
Yeah, absolutely critical that becomesa first classes and very performant.
(17:29):
For, databases.
And the other interesting one, whichI want to actually learn more about,
cause I'm always curious about this isyou mentioned there was an extension
for Apache arrow which is and for thelisteners, it's an in memory column or
Format that's used in a lot of, what's themost popular usage of arrow I'm, thinking
(17:53):
is probably the most popular.
Yes, absolutely.
It makes operations on in memory columnardata structure, super fast adopted
by a lot of very popular open sourceand enterprise data lake platforms.
And now that's becomingan extension for Postgres.
So how would that work exactly?
(18:14):
Is it actually going to absorb arrowformat into the Postgres buffer cache?
Or is it going to be somethingthat sits outside of Postgres?
I believe it'soutside the buffer cache.
So Postgres has the buffer caches, theshared memory area, and then every.
It's called background process,but it's essentially a process that
(18:36):
represents your connection to a Postgres.
It has its own memory area for thingsthat are private, if I'm doing a large
sort and I want to do it in memory, it'smy sort, it's nobody else's, it will get,
those bytes into my area of the memory.
And so my understanding is that thearrow is going to be in the background
(19:00):
process memory not in something shared.
And my understanding is that it isSomething that you basically configure on
specific tables and in specific scenarios.
It's not something that you just getevery table in your arrow because
it doesn't really make sense.
It's optimized for specific typesof data, specific tasks and so on.
(19:27):
It really remains to be seen when I metsomeone and we just talked about it.
He said he has an extension.
I admit, I haven't tested it.
I don't know.
When is it cool and how cool is it?
Soif they're doing it and you heard
about it, it's probably cool.
But that's my bar forcool projects right now.
And the other one you mentioned wasParquet, which is also Super interesting
(19:51):
parquet, of course, being a file formatand used by things like iceberg Apache
iceberg and Delta, and also becoming astandard for bottomless cloud storage.
You can put tons of parquet files on S3or GCS your cloud of choice or on premise.
(20:14):
So is the idea there thatPostgres can query parquet files?
Yeah, this is exactly the goal.
And so I think maybe a year back,I talked to someone he was building
an application and he wanted touse Nile and his application was.
(20:37):
Basically required processing a lot ofdocuments behind like offline, essentially
you would get PDFs of catalogs fromall sorts of vendors like furnitures
and all kinds of things like that.
And would basically would process itand extract information out of that.
(21:01):
And he will use the Databricksfor this information extraction.
But then he wanted the datain Postgres to serve it.
Now, Databricks, when it justsaves data natively, it has
its own, I think, Delta format.
But it can write Parquetfiles very easily.
And so he was like, okay, howdo I get the data from those
Parquet files that my parquet.
(21:22):
Parquet.
Databricks process gives me intoNiles so I can actually serve it to my
customers and run some queries on it.
So we did some research together becauseI did not know either, and we found a
lot of different ways of doing that.
So there is, and extensionscan be the Wild West.
(21:42):
And as you can imagine, Postgresis popular, Parquet is popular.
There will be a lot of different ways.
Some of them are more suited forETL, which is what we ended up doing.
Working with, but some would basicallyopen the connection and allow
you to directly query a project.
So it's, this is really cool, right?
(22:03):
Yeah, absolutely.
It's, it is really becominga form of interoperability
between the core components ofcloud and SAS infrastructure.
And it's great to hear that Nile, Usersand customers are ahead of the curve
there and how they're adopting it.
So that's great.
(22:23):
I do see that as being a parquet andpostgres as being very foundational
components there and pieces of thestack and a big part of The drive and
investment going into an investmentboth from I'm talking not so much
like venture capitalists, but moreso enterprise investment now is
(22:44):
modernizing for AI use cases, right?
Generative AI is a very popular,trend right now, but I do think that
based on what I'm seeing, the cons,the conversations I'm having, it's
top of mind for many enterprisesto, to implement and fuse into their
operations and customer experiences.
How, is Postgres supporting AI workloads?
(23:07):
Yeah, so basically.
AI workloads is a very wide term,there's so many different models.
Obviously, what comes to everyone's mindswhen we talk about AI is the language
models, but they're definitely very farfrom being the only models out there.
And there's a lot of models around theConverting images to images, just as an
(23:33):
example, there are not language modelsare transformers off a different sort.
But everyone thinks about language models,and then within language models, one of
the biggest subsets, especially for theenterprise is what is known as RAG, and
that's retrieval argumented generation.
The idea is that if you ask him aquestion without a lot of context it.
(24:01):
Is quite likely to hallucinate or answersomething that is out of band, not
really relevant for what you need is.
But if you provide itrelevant information, it can
combine the fact that it.
Has a lot of knowledge about the worldfrom studying the entire internet.
It understands nuances of thelanguage, both in input and in output.
(24:23):
But it now also has somerelevant information for your
question, for your use case.
Now the problem is how do wefind relevant information to
give the AI with my question?
And this is where vectorembeddings come in.
Vector embeddings is a different typeof AI model, essentially, that knows
how to convert text to some kind ofset of a lot of numbers that represent
(24:48):
something about the meaning of the text.
And then if you take two textsthat turn into vector and you say,
How close are they to each other?
It's vectors.
You have a distance function.
How close are they to each other?
It represents how much the meaningis similar, which is a form of how
relevant they are to each other.
(25:10):
And semantic meaning is different fromthe old text search rules to know based on
word frequency and dictionaries, becauseit captures something about the meaning
of a word in a context, in the domain.
So function and procedure arevery similar if you're talking
about a programming language.
They're very different from each otherin almost any other context in the world.
(25:33):
If you talk to some, to a lawyerand tell him that function and
procedure is the same thing, hewill think you are out of your mind.
Obviously procedures is somethingvery specific in law and functions
are completely unrelated.
So you get this ability to Basicallyvectorize the user question and
(25:53):
use that to find documents thathave similar semantic meaning
to it, meaning more relevant.
And then you feed it toAI and get an answer.
So databases have now need to beable to not just query, select by
a value in a column or something,they have to do vector search.
(26:14):
Find me the Most similar vector tothis one known as nearest neighbor
search because you imagine thismulti dimensional space and you're
looking for the vectors that areclosest to the one I just gave you.
So it's the nearest neighbor search.
This is also, this is whatvector stores specialize in.
And there are fivegazillion of them out there.
(26:35):
For obvious reasons, because everyone isgoing to need one, especially enterprises.
And if it's faster, it's generally better.
So you're going to want totake a slice of this pie.
Now, there is a lot of specializedvector stores, and Postgres has,
just like it can do Parquet, ithas an extension called pgVector
(27:00):
that specializes in vector search.
Now, Postgres is not the only relationaldatabase, or even non relational database.
It's not the only thing that suddenlygrew an extension that does vectors.
I think MySQL, Oracle, MongoDB,Datastacks, Databricks, I don't
(27:21):
even know who else is doing it.
Everyone is doing vectors.
You cannot really have a databasewithout vectors these days.
Absolutely, yeah.
A person who needs to buildthis architecture basically
faces two decisions.
The first is, do I want a specializedvector store, or do I want a vector store
in, that is part of another database?
(27:43):
And I honestly, I cannotactually advise a lot on that.
I haven't tested a lot ofthe other vector stores.
I would say the default thing to keepusing your, the tech stack you're already
familiar and comfortable with untilproven otherwise is not a bad idea.
(28:04):
So if you already have MongoDB,try using MongoDB, see how it goes.
If you have Postgres, try using Postgres.
See how's that work.
If you have two, let's say you usePostgres for your OLTP and BigQuery
for your data warehouse, then youstart thinking about what kinds of
(28:26):
data do I need to combine together.
So a lot of time, the use of vectorsand AI is part of an OLTP flow.
It is part of your real time application,user logs in, looks at some of their data.
Let's say that you are Stripe, andthey look at their latest revenue
(28:50):
statements, and now they have a question.
Something seems off.
And Then they ask, you have AIcapabilities, because that's what
everyone is about these days.
So the customer now asks, can youplease explain this data point to me?
Now, what is this data point?
Because this is going to be the mostrelevant context, and you're going to
(29:11):
want to include it in the question, andyou also want to find other relevant
information based on exactly what is this.
You're already likely connectedto an OLTP database, and You know
what was the last thing that youretrieved that the user is seeing.
To build in, RAG into that, Is Verystraightforward because your vector
(29:40):
will exist right there in the databasewith the rest of the data that the
customer is currently looking at.
If the AI comes back with an answer thatref references them other data, it's
very easy to pull it out and includeit in the response it's already there.
Now, if you put the vectorsin the data warehouse.
First you need the online application tosuddenly connect to the data warehouse
(30:03):
and then tell the data warehouse people tonot take the data warehouse down anymore.
Definitely not every night betweenthose hours that they're used to
taking it down for data loadingor whatever they do at night.
So it gets a bit, In my mind,it gets a bit more complicated
because data warehouses are justmanaged in this different way.
(30:25):
They don't always have the freshest data.
This is a bit of a concern.
If your data, if you havea real time data warehouse.
Then it's obviously going to, you maywant to experiment like you could do both.
And the other way to look atthings is really that you may
(30:49):
want to do some preparation.
And for example, index the vectorsor optimize them in some way.
on the data warehouse and then copythings over to an oil depot system.
And I'm not sure this is something that isvery easy these days, but it's something
that I'm experimenting with becausethere is, vectors are really weird.
(31:11):
There is parts of the flows thatare a lot like data warehouses.
Like doing the ingest processlooks a lot like a data warehouse.
Creating indexes can be veryheavy and the data warehouse
machines may be a better fit.
On the other hand, this interactive,I'm chatting and I need to continuously
pull stuff up, is very much OLTP ish.
(31:34):
So I really think we need to finda way to use the data warehouse for
what it's good for, use OLTP for whatit's good for, and Get the right data
structure to the right place over time.
Yeah, incredibly valuable advice, Gwen.
And that's, and thank you specificallyfor the free consulting, because
(31:58):
I use Postgres and BigQuery, sothat's directly applicable to me.
Thank you.
No, I'm joking.
But it is a great point that, yes, ayear, maybe two years ago, when we were,
Early in generative AI with languagemodels there was this craze to adopt a new
stack, a new vector database whatever itwas, a lot were spinning up at the time.
(32:23):
Which they're probably it's, greattechnology and probably it's specialized
go to markets for that motion, whichhas its own value for teams that don't
change things super fast, but do haveto Innovate when the time is right
and already have large investments andthings like Postgres or whatever that
database is, yes, it makes the mostsense to test what the, with the new
(32:46):
features that came out to support you,whether it's a vector extension or your
data warehouse provider offers vectorcolumns and seeing what goes out there.
Cause yeah when I researched researches18 months ago, it sounded like you needed
a totally new sack, but now with allthe announcements literally in the last
month, it looks like my data warehouse, mydatabase and my BI tool all support chat
(33:11):
GPT on my data type experiences, right?
So it's it's such a big problem with somuch attention on it that it's not like
no sequel 15 years ago where the largedatabase providers didn't really invest
a lot in the competing in no sequel.
So like new players like MongoDBor even Lucene based, Inverted
index, like elastic search couldcome into the market and grow.
(33:34):
So that, that, that's, what's,that's, what's interesting about it.
And the other thing that's, especiallywhen you're talking about there's
really two categories of it.
One of which is customer facing.
Chat experiences, which you reallywant to make sure it doesn't
hallucinate and rag is critical tomake sure it has all that context.
But that's that's one part of it.
(33:55):
The other is internal workloads where,yeah, you could probably tolerate
a little bit more hallucination,but these language models.
Can really be tripped up byinconsequential information, which
means that the way you actually indexthis text really matters, right?
So if you're single view of the customeror whatever that knowledge needs to
(34:19):
be is actually spread across let'ssay 200 normalized tables in different
relations and you're representingcustomer names with like I IDs and some
tables and some other values and others.
And you have to do a bunch of joins.
You also have to think aboutwhere is the ETL happening?
Exactly.
Yes, 100%.
(34:39):
And we're already seeing alot of ingestion pipelines
and ETLs specifically for AI.
And because having AI in the name andpitch deck guarantees funding these
days, you see a lot of new kind of ETLor workflows for AIs that, yeah, it's
(35:07):
I think a lot of the ETL incumbents.
Maybe you're not moving fast enough,or maybe, I don't know about Striim,
maybe you are moving fast enough, butis ATL ETL is an old practice, right?
So obviously the next generation alreadybeing real time and all this, but, there
(35:27):
is IBM data stage going ages, back.
I don't know if they're moving fast enoughto get power to be part of this new.
Vectorized ETL.
Absolutely.
It's the whole landscape is shiftingunderneath data and infrastructure
(35:49):
providers, how are you goingto best support AI workloads?
And so I did want to ask you what'sthe you gave a very Comprehensive
advice and great a great answer interms of the, state of AI for, Postgres
specifically what are some of theadvantages of doing, rag and vector
(36:10):
embeddings and using the Postgres vectorextension compared to other stacks?
Yeah.
So basically I would say thebiggest thing is that you are using
Postgres and I do think Postgres is.
The perfect database for buildingapplications on top of, and
obviously I'm looking mostly at B2B,those multi tenant applications.
(36:32):
But, if you think about Postgres, it'sjust has such strong support for such a
wide variety of workloads that for someonewho is Just building an application the
first few years before you have time tohire a data engineering team to build
your ETLs and data warehouse and all that.
(36:55):
It can do data warehouse workloads.
It can do document, like JSONdocument workloads like MongoDB.
It may not be as amazingas it is as MongoDB.
But it's one database that can dothat and relational and text search.
And if you don't like the built intext search, those 500 extensions
(37:15):
with better text search, it just, itcan do so much and it can do vectors.
So if you like, if you're looking fora very capable database, it also does
vectors, Postgres is really good for that.
And then I've been watching PG vector.
Extremely closely and it's so nice tosee it also releases like clockwork
(37:39):
as part of the postgres community.
So we're having a release everysix months and the improvements
and features that go into postgres.
Any iteration is just so impressive.
And again, those very detailed benchmarksand it moves so fast like you think
(38:00):
every half year is not that fast for AI.
But when I talk to people using it,they literally don't even know about the
cool stuff that happened back in April.
And we're already looking atthe next release that gets
even better on top of that.
So one of the cool optimizationsthat I really love there is
the support for quantization.
(38:21):
Quantization is one of those AI topicsthat people, not enough people know
about, and it's such a game changer.
So AI, everything about AI isthose large vectors and matrices.
And those are extremelylarge, almost by definition.
The more numbers you have, themore data you can represent.
But it turned out that everynumber in those very large
(38:44):
vectors is floating point.
And if you use, and it turns out thatyou don't actually need all 32 bits of
a floating point to get a good result.
A lot of time, 16 bits, even 8 bits,as long as those are the significant
bits, They're actually good enough.
So now you can get same resultwith half or quarter of the memory.
(39:08):
It basically means that you don'tneed the latest, greatest, impossible
to obtain super expensive GPUs.
It can run just fine on my back M1 orsomething very cheap that I can read
from the GPU for rent vendors out there.
And this also applies to embeddings.
(39:32):
PgVector allows you to, instead ofusing a full vector, it has a half vec
type, which is exactly how it sounds.
It's 16 bit floats instead of 32.
That's it.
Half the cost, double the speed,and you did absolutely nothing
except in search and replace vectorwith half vec in your script.
(39:57):
So it's it's pretty cool stuff thatjust I feel like the problem is
not that people don't use pgVector.
It's that they use it, butdon't know how to get the most
out of it, maybe the problem.
Absolutely.
And this is really what it comes downto is there's such a large ecosystem on
top of Postgres, ultimately deliveringin an application on built on top of a
(40:21):
language model will require like thatextra work, that extra optimization.
And like you said, this is overallprobabilistic operation, right?
When you do the similaritysearch on, on, on vectors.
And when you think about some ofthe most, Popular data structures
for probabilistic operations.
(40:43):
You think of hyper log, which isused for like carnality estimates.
And this is something that also likeexists, on Postgres, not exactly
in the, AI context, but just as anexample of ways to, that the Postgres
community has optimized performanceand it's just a recurring theme with
Postgres that, you know, whateverthat technical gap that needs to be
(41:07):
filled, it's going to be addressedby the community and the leaders.
And you'll, always be in good companyin terms of those who adopt it and
scale it and with, there'll be highcosts with generative AI applications.
You don't want to pay tons of extramoney on like database licensing,
for example, in some contexts.
(41:28):
And then, being able to do thatwith Postgres gives you not only the
community behind it but just such arich set of great SaaS providers to
work with, too, that also standardizeon Postgres and have Postgres compatible
APIs and very friendly to Postgres.
(41:50):
With that being said, I'd love to hearabout what you're working on at Nile.
Yeah, thank you.
Of course.
Basically as you've heard, I'm reallyexcited about Postgres and I'm really
excited about the ideas that you'rebasically, you don't have to think about
the database in terms of a database.
(42:12):
For this type of data versus theother type of data, you are mostly
thinking about, okay, I'm building thisapplication, what do my customers need?
And then you know that Postgresis going to give you all of it.
If you need vectors, if you needrelational, if you need JSON, if
you need to join all three in someway to give what you need to your
(42:34):
customers, you can do all of it.
With Postgres.
So for us, that was this, thebeginning, like you can build
anything you want on Postgres.
It is a great basis for an application.
And then we started, when we started Nile,we talked to a lot of people building
software as a service, as because you'vebeen kind enough to talk with us and help
(42:57):
us out in our large discovery effort.
And we talked about whatproblems people run into in
different stages of development.
Building not just an application, butalso a business on top of the application.
And we realized that veryearly in the process, you
have to make one big decision.
(43:18):
Are you going to use the samePostgres for all your customers?
Or, and just cram them into big tablesthat have a tenant ID column in them?
Or you're going to have a databaseper customer and isolate them
more, be able to Maybe give eachone better service because you can
(43:38):
back and restore them individually.
You can change thingsfor them individually.
And you have to make thisdecision so early in the game.
It is so hard to changeif you got it wrong.
And if you think about it, it's so tiedto how your business is going to work.
If you're going to sell to largercompanies, enterprises, Of course,
(43:59):
you, it makes sense to have adatabase for each one of them.
If you're going to have 50, 000 smallcustomers within the first year, then
of course it makes sense to storethem all in one large set of tables.
But how do you know when you justgo and design your first data model?
And what do you do if you get it wrong?
(44:20):
For us, this was the missingbit in the architecture.
The fact that you have, you'renot just having a random database.
This database has data, not foryou, but for your customers.
You need to Give them to figure outa model that gives them the right
service, but also has the rightcost and manageability for you.
(44:44):
And then you start, when you startthinking about it, it's not your
data, it's the data of your customers.
How do you authenticate?
How do you manage access to it?
The current model where you have singleuser connecting to your customer database,
and you only do any kind of filtering andaccess controls in the application, As a
data person seemed absolutely terrifying,like people really the entire world.
(45:08):
This is how you're shipping applications.
I trust you with my data, andthis is how you secure it.
It sounded very scary.
So we wanted a better system atNile, and We were a bit naive.
We were like, okay,Postgres is very hackable.
You can write all those cool extensions.
(45:29):
How hard can it possibly be to writea model where it will look like you're
inserting everything into the same table?
It will still, you have oneconnection, you do insert into
and you put the tenant ID.
But behind the scenes, we actuallydetect which tenants it's for.
We route it to the right database.
(45:50):
When you run a query, if it's forone tenant, we route it correctly.
If it's for multiple tenants, wecollect all the results and give
you back a collective result.
So we basically set on the pathof re architecting Postgres,
which day one sounds like,how hard could it possibly be?
And then two years later,it's okay, we are still here.
(46:12):
Starting on an interesting journey,but some of the pieces are now in
place and we can see where it's going.
And yeah, that was, basicallywhat we ended up with.
Very cool.
I yeah.
Niall doing some absolutelyincredible stuff.
(46:32):
And I love how.
It's very, works great with Postgres.
If you're already a sophisticatedPostgres user, it takes away so much of
the boiler plate away of, and not thesegeneralized problems of how do you deal
with multi tenancy and applications and,creating isolation for certain users.
(46:54):
And I, I really love the templates thatyou have on your website too, that just
show how fast it is to deploy this app.
With the infrastructure that I alreadyuse that's very valuable, cause I don't
want to spin up all new types of I don'twant a new type of database and, even
worse, you don't, nobody wantsa new programming language,
(47:16):
absolutely.
Yeah.
Yeah.
It's great to just provide a fast pathfor users, not only to deploy, but also
scale and, in a, framework that's, that,that really does allow you to standardize
and grow with the infrastructurethat, that you already have.
I think that's, what's going to continueto be resilient because if you look
(47:39):
at the databases people are using.
10 years ago it's, yes, there'snew entrance and niche type of
databases, but the big players arestill very active Postgres being, of
course the largest open source one.
So yeah, very excited about your work.
(48:00):
I've been following your talks for along time, Sorry to the listeners for
the history lesson going back to Strata,but hopefully some of you were there
and able to appreciate it as well.
But Gwen, that's great.
Where can people followalong with your work?
And so there's a few places.
First of all, I always have andwill continue to blog a lot.
(48:21):
So these days it's on the Nile blog.
And no, I post, I'm very excitedabout AI, if you haven't noticed.
So I, and especially multi tenantAI, where you actually get to isolate
the tenants So I post a lot aboutthat and optimizations and so on.
And also some Postgresbasics from time to time.
(48:43):
And then I am less active onTwitter than I used to be.
But, and it's no longercalled Twitter anyway.
But you can still find me on the Excel.
And, and then yeah, I alsoam available on LinkedIn.
I normally accept connectionsand post interesting things and
(49:07):
respond to people who ping me.
Oh, and Niall has a Discord.
So if you want a live chat, I am there.
Because a lot of our users are there,I'm very responsive on our Discord.
That's incredible.
So many great ways to follow alongwith Gwen and stay in touch with her.
I totally recommend it for everyonewho wants to keep their skills nice and
(49:27):
sharp and updated and especially withawesome open source infrastructure where
Gwen's done a lot of work through bothher work at Confluent and now at Nile.
So Gwen, thank you so muchfor joining this episode.
I'm really excited to follow, continuefollowing along with you and thank
you to the listeners for tuning in.
(49:48):
Thank you so much.
It's been a pleasure.
I'm You
I'm You Hello, everybody.
Thank you for tuning in to thisepisode of what's new in data.
I'm really excited about our guests today.
We have Gwen Shapira, who's co founderand chief product officer at Nile.
(00:26):
She was previously an engineering leaderfor cloud native Kafka at Confluent.
Gwen, how are you doing today?
Really happy to be here.
We've been chatting about thison off for a while and I'm glad
to finally be talking to you.
Absolutely.
Yeah.
Gwen, we've gone back and forthover the years communicating over
(00:48):
all the latest, greatest tech in,the enterprise and open source.
The first time I saw you.
Give a talk was six years ago,probably about exactly six years ago
at O'Reilly Strata event in New York.
You had hundreds of people in theaudience listening to you talk
about microservices and the futureof ETL with open source and Kafka.
(01:10):
So great to be catching up withyou here on this podcast now.
Really great to be catching up with you.
And I have to say, I miss O'ReillyStrata a lot as a conference.
There are other good dataconferences these days, but I
feel like Strata was special.
It wasn't a vendor specific.
It was really unique.
It was unique.
(01:31):
It was very cool.
It was in Javits Center, whichI always thought was a really
electrifying, exciting place to be.
The practitioners love.
Going there to talk about what theybuilt and it wasn't super vendor heavy
the vendor participation was alwaysreally good and pointed because you
had to talk about how you add valueinto the open source ecosystems because
(01:53):
it was it was always very tied to theHadoop infrastructure in the beginning.
So it always had that.
So I agree.
I do miss that conference.
What are some of yourfavorite conferences today?
Yeah, I have to say my conferenceshas changed a lot because I'm now
like part of the Postgres ecosystem.
(02:15):
Pg dev conf up in Vancouver a fewmonths back was really amazing.
And then, and because a lotof our customers and users
are JavaScript developers.
I've started going to JavaScriptconferences and the ecosystem around
that, which is also very differentand the entire vibe is very different.
(02:40):
So yeah, I don't know if the conferenceI go to these days are the most relevant
for the Data people in your audiencea postgres Conf is definitely a great
experience if you are in that ecosystemAnd the other thing is of course like the
classics the qcon and goto like the bigData or architecture conferences that they
(03:08):
always have a data track, but they talkabout software architectures in a larger
way and This is where I got my excitementabout microservices, for example.
Absolutely.
And you've done great work as aengineering leader for cloud hosted open
source infrastructure for a long time.
So not surprising to hear that now you'rebranching into JavaScript conferences
(03:34):
and Postgres conferences right now.
Nile, the company that you cofounded is doing some incredible
work in the Postgres community.
Tell me a bit about the current stateof Postgres, what people are talking
about there, what's being announced,what are people building with it?
Yeah, so this is really interesting.
So one thing that everyone knows, butit's hard to internalize until you
(03:58):
join the community, is that Postgreshas been around for a very long time.
If you look at the source code, youcan find a lot of if you do get blame,
this line was added 25 years ago, 30years ago, before, definitely before
I was working on anything at all.
(04:22):
So it, and a lot of the people whomade those commits 25 years ago,
they're still reviewing patches.
To this day, they are workingemployed to the company.
Like they're not even retired andthey're just still part of the community.
So you have a real sense of tradition.
(04:43):
I would say in the community and you also.
Can see how the community istrying to keep itself modern life.
So they have a discord now.
It is this is probably a big changefor people who've been working with
the same community and same people for30 years, long before discord existed.
And so it's really fun to seecommunity maintain a lot of
(05:07):
traditions and also evolve.
It's also really interesting to see.
If you look at any particular.
Contribution any particularchange that people are getting in.
It looks really slow.
It, I think a bug fix that everyoneagrees has to be fixed, but is non
(05:30):
trivial to implement can take three,four months to get in a feature that
may be slightly more controversial.
It can be years.
And so you look at the PR and yousay, Oh God, there's this feature and
Oracle had it forever, and they've beenjust discussing it for three years.
(05:51):
Will it ever get in?
But I think in a way it misses thebigger picture because when you look
at releases and Postgres has reallygood process you know exactly to the
day a year in advance years in advancewhen each release is going to happen
And in the few months running up tothe release, you know what features are
(06:15):
very likely going to make it or not.
So it's probably the most predictable opensource project I've ever seen in my life.
Apparently they wereonly late for release.
Once in a few days because they found avery critical bug in something that was
very hard to leave out of the release,something along those lines and it
(06:36):
took them extra few days to completelyrewrite something in order to make it in.
But in, in general, it's been excellent.
It's extremely reliable.
And then when you look at everyrelease, there is always good features.
Now, Postgres is very largeand not everyone needs every
new features that makes it in.
So for example, one of the mostexciting features for this new
(07:00):
release, this is Postgres 17.
So it was released like a few weeks back.
It's still very new.
So the most exciting featuresare that people got crazy over.
is an optimization for queries thatinvolve basically filtering on a list.
So where value in the columnis in and a list of few values.
(07:27):
It used to be that an index hadto be scanned every time for
each value in the list, whichcan get bad if the list is full.
Giant.
And now it can do onescan and find everything.
So this is a cool optimization.
Everyone is very excited about it.
But of course, if you look at the largenumber of workloads, not everyone uses
(07:48):
in, or even if you use in, not it doesn'tlike would you really go through the
upgrade just to get an in optimization?
Is that like the biggestbottleneck in your application?
Whether or not you feel pressure to getthose new features really depends on
your specific workload, which is why yousometimes see people running years old,
(08:12):
like five, six years old, a Postgresin production with no problems at all.
And if you use partitions, don't do it.
Partitions are still, got a lotbetter in the last few releases.
But, yeah, like a lot of workloads.
Will work well on Postgres 12.
(08:33):
I wouldn't recommend running Kafkafrom five years ago in production.
And I think some newer projectsevolve much faster, but Postgres is
depending on what you do, you can befine with older versions, I would say.
Absolutely.
And that's so much of.
(08:53):
Development that goes into databasesis just continued optimizations.
And there's a lot of very sophisticatedwork that goes into optimizing
information storage and retrievaland different types of indexes, and
even using like probabilistic datastructures and things like that.
It's a lot of development, which,and then at the same time, it has to.
Remain resilient.
(09:14):
It has to be backwards compatible.
All these thingsin Postgres is amazing.
Sotwo ways that Postgres testing is amazing.
First, there was the very famousbackdoors that almost made it to SSH.
I think a few months back, this wasdiscovered by one of the core Postgres
(09:35):
engineers, Andrea's friend, on whilehe was testing performance of Postgres.
And he noticed that his tests weregetting slow because SSH ing to the
machine, SSH process was taking moreCPU and was Taking slightly longer.
And this looked weird tohim, so he investigated.
So it's clear that there is a lotof detail orientedness that goes
(09:59):
into, there cannot be any performanceregression in Postgres ever.
And the other cool thing, peoplethese days talk a lot about the
deterministic simulation testing.
And.
Postgres had something very similarto test transaction isolation for
(10:20):
many, years, where they basically,you write some scenarios, the
transactions do things, and thenthey shuffle them, and they make
sure that all the shuffling stays.
Still maintain the same invariance andyou get the same results, no matter how
you move things around, which basicallytests that you have transaction isolation.
And and then at Nile, we actually used thefact that those tests exist to test some
(10:44):
of our transactional guarantees becausewe do some distributed transactions
that are not part of normal Postgres,but we could still reuse their tests.
It's pretty cool that, it wasjust so much ahead of its time.
Yeah, absolutely incredible.
I think that's why it's taking alead in community adoption as the
(11:08):
open source database of choice fora lot of both data infrastructure
companies and then companies areoffering databases as a service to
enterprise and it's super well adopted.
There's no debating that.
So it's great to hear the your perspectivefrom in the weeds of working with
(11:28):
the Postgres development community.
And yeah it's, great context to know thatthere's so much yes, there's rich history.
Databases have to be, in my opinion,developed over 20 to 30 years.
And there's a popular hacker newspost about Oracle's code base it's
this massive 25 million lines ofcode C and there's former Oracle
(11:53):
developers allegedly who chimed in andcommented saying getting one feature
in there just took such a long time.
And I think that's the only.
Way to have a database.
If someone comes in and says, Hey,I built a database from scratch in
the last six months, and with AI orsomething like that they, they code, they
generated it with, that's a big red flag,And now I want to try Zoho, go through
chatGPT and ask him to generate adatabase for me and see what happens.
(12:20):
Yes.
Yes.
The if, chatGPT is smart,it'll decline your request.
Or something like that.
Exactly,like it'll say, no, this
is really complicated.
Yeah.
Then we'll know that AGI is here, butif it actually tries to generate it in
Python or something there's problems.
Yeah.
Sam Altman has some work to do.
But that, that, that's greatto to hear the latest there.
(12:42):
And yeah, curious to hear whatelse is going on with Postgres.
Yeah.
So the other thing is because as yousaid, the community is very careful about
adopting features into core Postgres.
A lot of the interesting, excitingthings are in the ecosystem
with people publishing whatare called Postgres extensions.
(13:08):
For all kinds of scenarios.
So while I was at the Postgresconference, I chatted to a really
nice person who is building a Postgresextension that adds support for Arrow.
And there are multiple extensions withA support for per key file formats.
(13:29):
So if you put those together,suddenly you, there is some big
data operations that just becamea lot more efficient in Postgres.
So this is pretty exciting and noone had to wait on Postgres to decide
whether they want or need a new fileformat or a new memory format or
(13:49):
anything along those lines, because youcan just do so much with extensions.
And then you have extensions that havebeen around for a long time and are
still generating a lot of excitement.
So one, one of them is that I just keptbeing mentioned as an example in the
conference in all kinds of situations.
(14:10):
It's a HypoPG.
It basically allows you to createhypothetical indexes without having to
Spend the time on resources of actuallycreating and also risking production
repercussions for actually having them.
And then just check, how will thisquery behave if I had this index or more
(14:32):
likely, you have a workload that mixes,inserts, updates, selects, all that.
How will it be, will this workloadbe faster or slower if I had.
Those other indexes.
So it's, yeah, I think, I don't knowdatabase people are slightly obsessed
with performance and optimization.
It's a big part of what we care about.
(14:55):
So this is one of the thingsthat we keep coming back to.
There are cool optimizations going onthe network layer optimizations for
making SSL more efficient, which is.
Really important.
It used to be with Postgres.
I don't know if it exactly predatedSSL, but Not only did it originally
(15:17):
not support SSL for its first fewyears because it was really one of
the earliest databases that existed.
For a long time, nobody reallythought SSL was all that important.
It all ran in the data center.
Do I really need SSL betweenevery machine and my data center?
And I think the security positioningof most companies in the last 10
(15:41):
years shifted to a point where,yes, of course you're going
to run SSL between everything.
And also, of course, with all the Postgresas a service, a lot of time you actually
access Postgres beyond firewall walls.
So you obviously need a much higher levelof security and nobody's actually willing
(16:02):
to trade off security for performance.
We want all of it.
So having better SSL support inthe protocol and more efficient
one is obviously important.
Yeah.
That's a great point you brought up andyeah, I'd love to dive into each one
of those, but most if we're just goingLast in first out the point about SSL.
(16:24):
Yes.
And I very important that security teamsare now standardizing that for making
sure that Postgres connections are alwayssecure over networks, I've worked with
enterprise security teams that just don'tallow for instance, like inbound network
(16:45):
connections to an on premise data center,because they know that data centers are
You know, maybe 20, 30, 40 years old.
And for all they know, there's a bunchof databases with one, two, three, four,
five is the password and no, no networkleveling and like none of that stuff.
No patching for no security patchesapplied for the last five years.
For sure.
So now everyone's assuming,okay you're going to get.
(17:09):
Tons of requests coming in overnetworks from cloud or maybe another
data center, most likely cloud.
Now, it's not super controversialto have your enterprise data center
in the cloud as controversialas it was 5 to 10 years ago.
Yeah, absolutely critical that becomesa first classes and very performant.
(17:29):
For, databases.
And the other interesting one, whichI want to actually learn more about,
cause I'm always curious about this isyou mentioned there was an extension
for Apache arrow which is and for thelisteners, it's an in memory column or
Format that's used in a lot of, what's themost popular usage of arrow I'm, thinking
(17:53):
is probably the most popular.
Yes, absolutely.
It makes operations on in memory columnardata structure, super fast adopted
by a lot of very popular open sourceand enterprise data lake platforms.
And now that's becomingan extension for Postgres.
So how would that work exactly?
(18:14):
Is it actually going to absorb arrowformat into the Postgres buffer cache?
Or is it going to be somethingthat sits outside of Postgres?
I believe it'soutside the buffer cache.
So Postgres has the buffer caches, theshared memory area, and then every.
It's called background process,but it's essentially a process that
(18:36):
represents your connection to a Postgres.
It has its own memory area for thingsthat are private, if I'm doing a large
sort and I want to do it in memory, it'smy sort, it's nobody else's, it will get,
those bytes into my area of the memory.
And so my understanding is that thearrow is going to be in the background
(19:00):
process memory not in something shared.
And my understanding is that it isSomething that you basically configure on
specific tables and in specific scenarios.
It's not something that you just getevery table in your arrow because
it doesn't really make sense.
It's optimized for specific typesof data, specific tasks and so on.
(19:27):
It really remains to be seen when I metsomeone and we just talked about it.
He said he has an extension.
I admit, I haven't tested it.
I don't know.
When is it cool and how cool is it?
Soif they're doing it and you heard
about it, it's probably cool.
But that's my bar forcool projects right now.
And the other one you mentioned wasParquet, which is also Super interesting
(19:51):
parquet, of course, being a file formatand used by things like iceberg Apache
iceberg and Delta, and also becoming astandard for bottomless cloud storage.
You can put tons of parquet files on S3or GCS your cloud of choice or on premise.
(20:14):
So is the idea there thatPostgres can query parquet files?
Yeah, this is exactly the goal.
And so I think maybe a year back,I talked to someone he was building
an application and he wanted touse Nile and his application was.
(20:37):
Basically required processing a lot ofdocuments behind like offline, essentially
you would get PDFs of catalogs fromall sorts of vendors like furnitures
and all kinds of things like that.
And would basically would process itand extract information out of that.
(21:01):
And he will use the Databricksfor this information extraction.
But then he wanted the datain Postgres to serve it.
Now, Databricks, when it justsaves data natively, it has
its own, I think, Delta format.
But it can write Parquetfiles very easily.
And so he was like, okay, howdo I get the data from those
Parquet files that my parquet.
(21:22):
Parquet.
Databricks process gives me intoNiles so I can actually serve it to my
customers and run some queries on it.
So we did some research together becauseI did not know either, and we found a
lot of different ways of doing that.
So there is, and extensionscan be the Wild West.
(21:42):
And as you can imagine, Postgresis popular, Parquet is popular.
There will be a lot of different ways.
Some of them are more suited forETL, which is what we ended up doing.
Working with, but some would basicallyopen the connection and allow
you to directly query a project.
So it's, this is really cool, right?
(22:03):
Yeah, absolutely.
It's, it is really becominga form of interoperability
between the core components ofcloud and SAS infrastructure.
And it's great to hear that Nile, Usersand customers are ahead of the curve
there and how they're adopting it.
So that's great.
(22:23):
I do see that as being a parquet andpostgres as being very foundational
components there and pieces of thestack and a big part of The drive and
investment going into an investmentboth from I'm talking not so much
like venture capitalists, but moreso enterprise investment now is
(22:44):
modernizing for AI use cases, right?
Generative AI is a very popular,trend right now, but I do think that
based on what I'm seeing, the cons,the conversations I'm having, it's
top of mind for many enterprisesto, to implement and fuse into their
operations and customer experiences.
How, is Postgres supporting AI workloads?
(23:07):
Yeah, so basically.
AI workloads is a very wide term,there's so many different models.
Obviously, what comes to everyone's mindswhen we talk about AI is the language
models, but they're definitely very farfrom being the only models out there.
And there's a lot of models around theConverting images to images, just as an
(23:33):
example, there are not language modelsare transformers off a different sort.
But everyone thinks about language models,and then within language models, one of
the biggest subsets, especially for theenterprise is what is known as RAG, and
that's retrieval argumented generation.
The idea is that if you ask him aquestion without a lot of context it.
(24:01):
Is quite likely to hallucinate or answersomething that is out of band, not
really relevant for what you need is.
But if you provide itrelevant information, it can
combine the fact that it.
Has a lot of knowledge about the worldfrom studying the entire internet.
It understands nuances of thelanguage, both in input and in output.
(24:23):
But it now also has somerelevant information for your
question, for your use case.
Now the problem is how do wefind relevant information to
give the AI with my question?
And this is where vectorembeddings come in.
Vector embeddings is a different typeof AI model, essentially, that knows
how to convert text to some kind ofset of a lot of numbers that represent
(24:48):
something about the meaning of the text.
And then if you take two textsthat turn into vector and you say,
How close are they to each other?
It's vectors.
You have a distance function.
How close are they to each other?
It represents how much the meaningis similar, which is a form of how
relevant they are to each other.
(25:10):
And semantic meaning is different fromthe old text search rules to know based on
word frequency and dictionaries, becauseit captures something about the meaning
of a word in a context, in the domain.
So function and procedure arevery similar if you're talking
about a programming language.
They're very different from each otherin almost any other context in the world.
(25:33):
If you talk to some, to a lawyerand tell him that function and
procedure is the same thing, hewill think you are out of your mind.
Obviously procedures is somethingvery specific in law and functions
are completely unrelated.
So you get this ability to Basicallyvectorize the user question and
(25:53):
use that to find documents thathave similar semantic meaning
to it, meaning more relevant.
And then you feed it toAI and get an answer.
So databases have now need to beable to not just query, select by
a value in a column or something,they have to do vector search.
(26:14):
Find me the Most similar vector tothis one known as nearest neighbor
search because you imagine thismulti dimensional space and you're
looking for the vectors that areclosest to the one I just gave you.
So it's the nearest neighbor search.
This is also, this is whatvector stores specialize in.
And there are fivegazillion of them out there.
(26:35):
For obvious reasons, because everyone isgoing to need one, especially enterprises.
And if it's faster, it's generally better.
So you're going to want totake a slice of this pie.
Now, there is a lot of specializedvector stores, and Postgres has,
just like it can do Parquet, ithas an extension called pgVector
(27:00):
that specializes in vector search.
Now, Postgres is not the only relationaldatabase, or even non relational database.
It's not the only thing that suddenlygrew an extension that does vectors.
I think MySQL, Oracle, MongoDB,Datastacks, Databricks, I don't
(27:21):
even know who else is doing it.
Everyone is doing vectors.
You cannot really have a databasewithout vectors these days.
Absolutely, yeah.
A person who needs to buildthis architecture basically
faces two decisions.
The first is, do I want a specializedvector store, or do I want a vector store
in, that is part of another database?
(27:43):
And I honestly, I cannotactually advise a lot on that.
I haven't tested a lot ofthe other vector stores.
I would say the default thing to keepusing your, the tech stack you're already
familiar and comfortable with untilproven otherwise is not a bad idea.
(28:04):
So if you already have MongoDB,try using MongoDB, see how it goes.
If you have Postgres, try using Postgres.
See how's that work.
If you have two, let's say you usePostgres for your OLTP and BigQuery
for your data warehouse, then youstart thinking about what kinds of
(28:26):
data do I need to combine together.
So a lot of time, the use of vectorsand AI is part of an OLTP flow.
It is part of your real time application,user logs in, looks at some of their data.
Let's say that you are Stripe, andthey look at their latest revenue
(28:50):
statements, and now they have a question.
Something seems off.
And Then they ask, you have AIcapabilities, because that's what
everyone is about these days.
So the customer now asks, can youplease explain this data point to me?
Now, what is this data point?
Because this is going to be the mostrelevant context, and you're going to
(29:11):
want to include it in the question, andyou also want to find other relevant
information based on exactly what is this.
You're already likely connectedto an OLTP database, and You know
what was the last thing that youretrieved that the user is seeing.
To build in, RAG into that, Is Verystraightforward because your vector
(29:40):
will exist right there in the databasewith the rest of the data that the
customer is currently looking at.
If the AI comes back with an answer thatref references them other data, it's
very easy to pull it out and includeit in the response it's already there.
Now, if you put the vectorsin the data warehouse.
First you need the online application tosuddenly connect to the data warehouse
(30:03):
and then tell the data warehouse people tonot take the data warehouse down anymore.
Definitely not every night betweenthose hours that they're used to
taking it down for data loadingor whatever they do at night.
So it gets a bit, In my mind,it gets a bit more complicated
because data warehouses are justmanaged in this different way.
(30:25):
They don't always have the freshest data.
This is a bit of a concern.
If your data, if you havea real time data warehouse.
Then it's obviously going to, you maywant to experiment like you could do both.
And the other way to look atthings is really that you may
(30:49):
want to do some preparation.
And for example, index the vectorsor optimize them in some way.
on the data warehouse and then copythings over to an oil depot system.
And I'm not sure this is something that isvery easy these days, but it's something
that I'm experimenting with becausethere is, vectors are really weird.
(31:11):
There is parts of the flows thatare a lot like data warehouses.
Like doing the ingest processlooks a lot like a data warehouse.
Creating indexes can be veryheavy and the data warehouse
machines may be a better fit.
On the other hand, this interactive,I'm chatting and I need to continuously
pull stuff up, is very much OLTP ish.
(31:34):
So I really think we need to finda way to use the data warehouse for
what it's good for, use OLTP for whatit's good for, and Get the right data
structure to the right place over time.
Yeah, incredibly valuable advice, Gwen.
And that's, and thank you specificallyfor the free consulting, because
(31:58):
I use Postgres and BigQuery, sothat's directly applicable to me.
Thank you.
No, I'm joking.
But it is a great point that, yes, ayear, maybe two years ago, when we were,
Early in generative AI with languagemodels there was this craze to adopt a new
stack, a new vector database whatever itwas, a lot were spinning up at the time.
(32:23):
Which they're probably it's, greattechnology and probably it's specialized
go to markets for that motion, whichhas its own value for teams that don't
change things super fast, but do haveto Innovate when the time is right
and already have large investments andthings like Postgres or whatever that
database is, yes, it makes the mostsense to test what the, with the new
(32:46):
features that came out to support you,whether it's a vector extension or your
data warehouse provider offers vectorcolumns and seeing what goes out there.
Cause yeah when I researched researches18 months ago, it sounded like you needed
a totally new sack, but now with allthe announcements literally in the last
month, it looks like my data warehouse, mydatabase and my BI tool all support chat
(33:11):
GPT on my data type experiences, right?
So it's it's such a big problem with somuch attention on it that it's not like
no sequel 15 years ago where the largedatabase providers didn't really invest
a lot in the competing in no sequel.
So like new players like MongoDBor even Lucene based, Inverted
index, like elastic search couldcome into the market and grow.
(33:34):
So that, that, that's, what's,that's, what's interesting about it.
And the other thing that's, especiallywhen you're talking about there's
really two categories of it.
One of which is customer facing.
Chat experiences, which you reallywant to make sure it doesn't
hallucinate and rag is critical tomake sure it has all that context.
But that's that's one part of it.
(33:55):
The other is internal workloads where,yeah, you could probably tolerate
a little bit more hallucination,but these language models.
Can really be tripped up byinconsequential information, which
means that the way you actually indexthis text really matters, right?
So if you're single view of the customeror whatever that knowledge needs to
(34:19):
be is actually spread across let'ssay 200 normalized tables in different
relations and you're representingcustomer names with like I IDs and some
tables and some other values and others.
And you have to do a bunch of joins.
You also have to think aboutwhere is the ETL happening?
Exactly.
Yes, 100%.
(34:39):
And we're already seeing alot of ingestion pipelines
and ETLs specifically for AI.
And because having AI in the name andpitch deck guarantees funding these
days, you see a lot of new kind of ETLor workflows for AIs that, yeah, it's
(35:07):
I think a lot of the ETL incumbents.
Maybe you're not moving fast enough,or maybe, I don't know about Striim,
maybe you are moving fast enough, butis ATL ETL is an old practice, right?
So obviously the next generation alreadybeing real time and all this, but, there
(35:27):
is IBM data stage going ages, back.
I don't know if they're moving fast enoughto get power to be part of this new.
Vectorized ETL.
Absolutely.
It's the whole landscape is shiftingunderneath data and infrastructure
(35:49):
providers, how are you goingto best support AI workloads?
And so I did want to ask you what'sthe you gave a very Comprehensive
advice and great a great answer interms of the, state of AI for, Postgres
specifically what are some of theadvantages of doing, rag and vector
(36:10):
embeddings and using the Postgres vectorextension compared to other stacks?
Yeah.
So basically I would say thebiggest thing is that you are using
Postgres and I do think Postgres is.
The perfect database for buildingapplications on top of, and
obviously I'm looking mostly at B2B,those multi tenant applications.
(36:32):
But, if you think about Postgres, it'sjust has such strong support for such a
wide variety of workloads that for someonewho is Just building an application the
first few years before you have time tohire a data engineering team to build
your ETLs and data warehouse and all that.
(36:55):
It can do data warehouse workloads.
It can do document, like JSONdocument workloads like MongoDB.
It may not be as amazingas it is as MongoDB.
But it's one database that can dothat and relational and text search.
And if you don't like the built intext search, those 500 extensions
(37:15):
with better text search, it just, itcan do so much and it can do vectors.
So if you like, if you're looking fora very capable database, it also does
vectors, Postgres is really good for that.
And then I've been watching PG vector.
Extremely closely and it's so nice tosee it also releases like clockwork
(37:39):
as part of the postgres community.
So we're having a release everysix months and the improvements
and features that go into postgres.
Any iteration is just so impressive.
And again, those very detailed benchmarksand it moves so fast like you think
(38:00):
every half year is not that fast for AI.
But when I talk to people using it,they literally don't even know about the
cool stuff that happened back in April.
And we're already looking atthe next release that gets
even better on top of that.
So one of the cool optimizationsthat I really love there is
the support for quantization.
(38:21):
Quantization is one of those AI topicsthat people, not enough people know
about, and it's such a game changer.
So AI, everything about AI isthose large vectors and matrices.
And those are extremelylarge, almost by definition.
The more numbers you have, themore data you can represent.
But it turned out that everynumber in those very large
(38:44):
vectors is floating point.
And if you use, and it turns out thatyou don't actually need all 32 bits of
a floating point to get a good result.
A lot of time, 16 bits, even 8 bits,as long as those are the significant
bits, They're actually good enough.
So now you can get same resultwith half or quarter of the memory.
(39:08):
It basically means that you don'tneed the latest, greatest, impossible
to obtain super expensive GPUs.
It can run just fine on my back M1 orsomething very cheap that I can read
from the GPU for rent vendors out there.
And this also applies to embeddings.
(39:32):
PgVector allows you to, instead ofusing a full vector, it has a half vec
type, which is exactly how it sounds.
It's 16 bit floats instead of 32.
That's it.
Half the cost, double the speed,and you did absolutely nothing
except in search and replace vectorwith half vec in your script.
(39:57):
So it's it's pretty cool stuff thatjust I feel like the problem is
not that people don't use pgVector.
It's that they use it, butdon't know how to get the most
out of it, maybe the problem.
Absolutely.
And this is really what it comes downto is there's such a large ecosystem on
top of Postgres, ultimately deliveringin an application on built on top of a
(40:21):
language model will require like thatextra work, that extra optimization.
And like you said, this is overallprobabilistic operation, right?
When you do the similaritysearch on, on, on vectors.
And when you think about some ofthe most, Popular data structures
for probabilistic operations.
(40:43):
You think of hyper log, which isused for like carnality estimates.
And this is something that also likeexists, on Postgres, not exactly
in the, AI context, but just as anexample of ways to, that the Postgres
community has optimized performanceand it's just a recurring theme with
Postgres that, you know, whateverthat technical gap that needs to be
(41:07):
filled, it's going to be addressedby the community and the leaders.
And you'll, always be in good companyin terms of those who adopt it and
scale it and with, there'll be highcosts with generative AI applications.
You don't want to pay tons of extramoney on like database licensing,
for example, in some contexts.
(41:28):
And then, being able to do thatwith Postgres gives you not only the
community behind it but just such arich set of great SaaS providers to
work with, too, that also standardizeon Postgres and have Postgres compatible
APIs and very friendly to Postgres.
(41:50):
With that being said, I'd love to hearabout what you're working on at Nile.
Yeah, thank you.
Of course.
Basically as you've heard, I'm reallyexcited about Postgres and I'm really
excited about the ideas that you'rebasically, you don't have to think about
the database in terms of a database.
(42:12):
For this type of data versus theother type of data, you are mostly
thinking about, okay, I'm building thisapplication, what do my customers need?
And then you know that Postgresis going to give you all of it.
If you need vectors, if you needrelational, if you need JSON, if
you need to join all three in someway to give what you need to your
(42:34):
customers, you can do all of it.
With Postgres.
So for us, that was this, thebeginning, like you can build
anything you want on Postgres.
It is a great basis for an application.
And then we started, when we started Nile,we talked to a lot of people building
software as a service, as because you'vebeen kind enough to talk with us and help
(42:57):
us out in our large discovery effort.
And we talked about whatproblems people run into in
different stages of development.
Building not just an application, butalso a business on top of the application.
And we realized that veryearly in the process, you
have to make one big decision.
(43:18):
Are you going to use the samePostgres for all your customers?
Or, and just cram them into big tablesthat have a tenant ID column in them?
Or you're going to have a databaseper customer and isolate them
more, be able to Maybe give eachone better service because you can
(43:38):
back and restore them individually.
You can change thingsfor them individually.
And you have to make thisdecision so early in the game.
It is so hard to changeif you got it wrong.
And if you think about it, it's so tiedto how your business is going to work.
If you're going to sell to largercompanies, enterprises, Of course,
(43:59):
you, it makes sense to have adatabase for each one of them.
If you're going to have 50, 000 smallcustomers within the first year, then
of course it makes sense to storethem all in one large set of tables.
But how do you know when you justgo and design your first data model?
And what do you do if you get it wrong?
(44:20):
For us, this was the missingbit in the architecture.
The fact that you have, you'renot just having a random database.
This database has data, not foryou, but for your customers.
You need to Give them to figure outa model that gives them the right
service, but also has the rightcost and manageability for you.
(44:44):
And then you start, when you startthinking about it, it's not your
data, it's the data of your customers.
How do you authenticate?
How do you manage access to it?
The current model where you have singleuser connecting to your customer database,
and you only do any kind of filtering andaccess controls in the application, As a
data person seemed absolutely terrifying,like people really the entire world.
(45:08):
This is how you're shipping applications.
I trust you with my data, andthis is how you secure it.
It sounded very scary.
So we wanted a better system atNile, and We were a bit naive.
We were like, okay,Postgres is very hackable.
You can write all those cool extensions.
(45:29):
How hard can it possibly be to writea model where it will look like you're
inserting everything into the same table?
It will still, you have oneconnection, you do insert into
and you put the tenant ID.
But behind the scenes, we actuallydetect which tenants it's for.
We route it to the right database.
(45:50):
When you run a query, if it's forone tenant, we route it correctly.
If it's for multiple tenants, wecollect all the results and give
you back a collective result.
So we basically set on the pathof re architecting Postgres,
which day one sounds like,how hard could it possibly be?
And then two years later,it's okay, we are still here.
(46:12):
Starting on an interesting journey,but some of the pieces are now in
place and we can see where it's going.
And yeah, that was, basicallywhat we ended up with.
Very cool.
I yeah.
Niall doing some absolutelyincredible stuff.
(46:32):
And I love how.
It's very, works great with Postgres.
If you're already a sophisticatedPostgres user, it takes away so much of
the boiler plate away of, and not thesegeneralized problems of how do you deal
with multi tenancy and applications and,creating isolation for certain users.
(46:54):
And I, I really love the templates thatyou have on your website too, that just
show how fast it is to deploy this app.
With the infrastructure that I alreadyuse that's very valuable, cause I don't
want to spin up all new types of I don'twant a new type of database and, even
worse, you don't, nobody wantsa new programming language,
(47:16):
absolutely.
Yeah.
Yeah.
It's great to just provide a fast pathfor users, not only to deploy, but also
scale and, in a, framework that's, that,that really does allow you to standardize
and grow with the infrastructurethat, that you already have.
I think that's, what's going to continueto be resilient because if you look
(47:39):
at the databases people are using.
10 years ago it's, yes, there'snew entrance and niche type of
databases, but the big players arestill very active Postgres being, of
course the largest open source one.
So yeah, very excited about your work.
(48:00):
I've been following your talks for along time, Sorry to the listeners for
the history lesson going back to Strata,but hopefully some of you were there
and able to appreciate it as well.
But Gwen, that's great.
Where can people followalong with your work?
And so there's a few places.
First of all, I always have andwill continue to blog a lot.
(48:21):
So these days it's on the Nile blog.
And no, I post, I'm very excitedabout AI, if you haven't noticed.
So I, and especially multi tenantAI, where you actually get to isolate
the tenants So I post a lot aboutthat and optimizations and so on.
And also some Postgresbasics from time to time.
(48:43):
And then I am less active onTwitter than I used to be.
But, and it's no longercalled Twitter anyway.
But you can still find me on the Excel.
And, and then yeah, I alsoam available on LinkedIn.
I normally accept connectionsand post interesting things and
(49:07):
respond to people who ping me.
Oh, and Niall has a Discord.
So if you want a live chat, I am there.
Because a lot of our users are there,I'm very responsive on our Discord.
That's incredible.
So many great ways to follow alongwith Gwen and stay in touch with her.
I totally recommend it for everyonewho wants to keep their skills nice and
(49:27):
sharp and updated and especially withawesome open source infrastructure where
Gwen's done a lot of work through bothher work at Confluent and now at Nile.
So Gwen, thank you so muchfor joining this episode.
I'm really excited to follow, continuefollowing along with you and thank
you to the listeners for tuning in.
(49:48):
Thank you so much.
It's been a pleasure.
I'm You
Popular Podcasts
Betrayal Weekly
Betrayal Weekly is back for a new season. Every Thursday, Betrayal Weekly shares first-hand accounts of broken trust, shocking deceptions, and the trail of destruction they leave behind. Hosted by Andrea Gunning, this weekly ongoing series digs into real-life stories of betrayal and the aftermath. From stories of double lives to dark discoveries, these are cautionary tales and accounts of resilience against all odds. From the producers of the critically acclaimed Betrayal series, Betrayal Weekly drops new episodes every Thursday. If you would like to share your story, you can reach out to the Betrayal Team by emailing them at betrayalpod@gmail.com and follow us on Instagram at @betrayalpod and @glasspodcasts. Please join our Substack for additional exclusive content, curated book recommendations, and community discussions. Sign up FREE by clicking this link Beyond Betrayal Substack. Join our community dedicated to truth, resilience, and healing. Your voice matters! Be a part of our Betrayal journey on Substack.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com