December 16, 2024 • 10 mins
In this episode, we dive into the world of queryable encryption with our security specialist. Discover how this innovative feature protects sensitive data while allowing for secure querying without exposing any information. Learn about its applications in industries like banking, healthcare, and government, and how it ensures that even in the event of a data breach, your crown jewels remain safe. We also discuss the recent advancements, including range queries and future enhancements like full-text search. Whether you're a developer or a security professional, this episode provides valuable insights into securing your data with MongoDB.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:07):
Welcome back to the show floor here at the dot Local London.
We do hope you're enjoying all the conversations and all the
guests that we've had today. Thank you for staying put with
us throughout the everything that we're doing today.
Now we earlier on today at the keynote, again expanding on what
we learned briefly at the keynote, Queryable Encryption
came up. I am.
(00:27):
I'm joined by Ken White, our security specialist here and Ken
Queryable Encryption. What's it all about?
South queryable encryption is a way to protect our customers
most sensible. So we see a lot of this in the
banking world, in, in the energyfields, in the DoD.
(00:50):
But really everyone has sensitive data, right?
Everyone has some kind of crown jewels that they want to
protect. And so the idea with gravel
encryption is that you can, you can save records in an encrypted
format, but the database doesn'tlearn what that is.
And they don't, it doesn't have keys.
And so if there's, if there's anattack, there's a compromise on
(01:13):
the database side. If there's, you know, someone
comes up and, and black blackmails one of our senior
engineers or holds a gun to our CE OS head and says, give me
this data from, you know, bank Xor whatever.
We say, fine, here's a, here's acopy of the data, but it's all
encrypted. Good luck with that.
So, so that's the that's the basic premise.
But the short version, because Iwas and I think it was at a
(01:35):
prior dot local in London that we launched variable encryption
a couple of years ago. It was, yeah.
We take the stuff, really. So there's a lot of things in
the odd world where you can kindof iterate and move quickly and
sort of move on the fly with cryptography and with security.
You want to be very deliberate and you really have really
rigorous, you know, engineering practices.
(01:57):
You want to analyze things very properly.
You want to have lots of third party reviews.
You want to be very funny and careful about what you do and so
we had it in a preview form for about a year and then we G8 it
last year. So what we just announced today
is that let me is that range supports and being able to do
(02:20):
ours, you know, searching for everyone over 25 to 50 and
really financial balance it's you know between 10,000 million
dollars, OK, every every audit record, every sales transaction
happened between this state and that date is supported, but in a
way that the database never understands what's being asked
for. And even if you have at ADBA
(02:42):
level problems, you don't learn anything.
And this isn't the thing that and I spent I, I don't know what
event we were at, but I shared abooth with Cynthia who was a
prior PM or probably still is the PM in the in the space, but,
and I remember poking her all the time going, how does it
work? How does it work?
And so to me, you know, in that entire query roundset, nothing
(03:03):
is exposed. It's like magic.
It's still being able to query, but nothing has been exposed and
your data secure in transit, at rest and within the query
itself, correct? That's right.
So even if, pardon me, even if the network is compromised, even
if TLS is broken, even if someone gets a copy of the disk
and steals it or, or encryption rest was compromised, it doesn't
(03:27):
matter because it's, it's similar to Indian encryption
like you'd find in WhatsApp or Signal where the idea is you've
got two parties communicating anyone that eavesdrops over the
network. If I haven't turned my phone off
and you've sent me a message, that message is sitting on a
server somewhere. But it doesn't matter if, you
know, if I don't log on for a little bit, if they had access
(03:47):
to the server, they still can't see it.
So it's a very similar idea to that.
What's unique about word on encryption is it was built from
the ground up for high performance data been searched.
So there's all kinds of encryption sort of libraries and
techniques out there, but very few are are meant for, you know,
generalized search for operational workload.
(04:09):
You mean there's a lot in the kind of academic world that are
kind of toy type. You know, use cases were like,
I've got 100 records and I've got 1 laptop and one database
server. But as soon as you get into
distributing systems, as soon asyou get into, you know, I've
got, you know, threaded web services.
I'm, I'm, I've got high key currency.
I've got tons of users, those things breakdown.
(04:32):
And then also there's some encrypted search techniques that
may take seconds or minutes to get an answer back.
Well, that's just not realistic for operation.
And I suppose a testament to howserious we take this is that we
have a research group for for cortography.
That's right. And, and I've built this and
when we put it out first, we also open sourced our methods,
(04:55):
right, because it needed to be up to scrutiny.
Yeah, exactly. So we worked with the group out
of Brown University called the Encrypted Systems Lab when we
built our first generation client side encryption.
And what they did that was really useful was they helped us
think about the model, what are the promises we can make?
What are the things that we can't guarantee.
You know what, it's important tounderstand what you don't get
(05:17):
from this particular security control.
That collaboration was really, really positive and they were
working a lot of, you know, other sort of advanced TAT.
Pardon me, no worries. And so about two years after we
launched, they had a major breakthrough.
We talked to them about it and then eventually we acquired the
(05:37):
company. So yeah, today we have an in
house cryptography research group.
It's headed up by Sandy Kamara and Tarita Wataz and people have
been in the field for, you know,in this case, over 20 years.
He literally pioneered the fieldof encrypted search when he was
back at Microsoft Research. And, and so a lot of the, the
knowledge, the basis, the sort of formalisms have been out
(06:01):
there for 1520 years. What we, you know, we're able to
do is to operationalize that into a practical database and
put things in place that made iteasy for developers to use
because a lot of times, you know, tech products put a huge
burden on the developer. They have to become cryptography
experts. There's in different encryption
(06:22):
modes and key sizes and all thissort of thing.
And we said, well, what, what ifwe made this very opinionated
based on conservative, you know,sort of cryptographer approved
values. So that basically you set up a
little boilerplate code. And then as a developer, if you
want to insert a document, you just write an insert statement
like you used to. You want to query a bunch of
documents. You should be able to query
(06:42):
without having to think too muchabout, you know, the underlying
crypto. And so, yeah, after that
acquisition, we refer to more cryptography researchers.
I think there's like 5 PhD cryptographers in the group.
They're publishing papers, they're leading, you know,
academic conferences. It is we now have like visiting
scientists and student interns. I mean, it's a proper full blown
(07:05):
R&D goods. It's.
Totally strategic for us given the clients that we have.
I mean, I know that, you know, certain institutions and certain
industries, financial, medical, etcetera are reluctant to go to
the cloud or we're reluctant to add.
But with this approach, essentially it's pretty much the
last barrier in being able to say we don't want our our data
(07:26):
being shipped around like this, correct?
Yeah, that's right. And so while we started with
some pretty sophisticated use cases, everybody's got crown
Jewelers, everybody has some kind of PII data or sensitive
data they want protected. But but our our sort of more
high end sophisticated customerssaid, look, we're not this isn't
some James Bond or science fiction scenario.
(07:47):
We have entire nation states that are targeting our platform.
When you're moving hundreds of billions of dollars a day
through capital markets, that's not fiction.
You, you really do have to kind of think about that.
And, and none of that to say that we've, we've sort of
developed some magic silver bullet that'll cure all
problems, but rather that we took a very disciplined, careful
(08:07):
approach of confidentiality, of keeping your secret secret,
keeping the secret from us. You don't have to trust us that
we'll do the right thing becausethere's also there's, there's
this sort of hackers, there's the hacker side, but then
there's the what happens if I'm compelled as a service provider?
What if we get a court order or some law enforcement thing or,
(08:30):
or you're under litigation and you know, a judge says I need a
forensic called Mongo DB. Give me a copy of this
customer's data and fine, we'll give you a copy of it, but it's
encrypted. Good luck with that.
But what it really means is thatthe customer controls their data
unless they supply the decryption in their keys, it it
(08:50):
doesn't matter whether again, our CEO is held hostage or
that's right. Even if somebody had root level
privileges on the virtual machine, even if they had
hypervisor level access, it's it's not, it's not.
They're not seeing the clear text data and they don't have
access to that being. Super secure.
So we we announced range queriesbefore that it was exact match
(09:12):
it was yeah. So range queries allows, as you
said, a couple of examples, you know, search for transactions
between this value and that value except as well too.
Is there more to come? Full text search for.
Oh, yeah, absolutely. So, so program encryption is an
is a platform for expressive search.
OK. So we're, you know, we're hoping
in the next, in the first half of next year to introduce tech
(09:35):
support, prefix FX substring support.
We'll Gea that after some reasonable amount of time and
then yeah, then then, then, thenit's open.
You know, we're potentially considering vector, we're
potentially considering, you know, geocode.
It's it's an expressive platform, so we're super excited
about it. Excellent.
And where do developers go to find out more?
Ken Sure, wherever they go today.
(09:56):
Yeah, so just query Mongo DB Queryable encryption.
We've got docs and tutorials andcode snippets you can copy and
paste if you Google the Mongo DBphotography group, lots of
technical papers. If you want to look at the
formal proofs, you want to understand the underlying math.
That's your source. Edison, excellent.
Well, listen, Ken, this has beenan eye opener.
(10:17):
I can't say that I still understand how it's done.
It looks like magic to me, but it's great to see it advancing
and what you've added with the range queries and I look forward
to seeing more. Thank you very much for joining
us on the Mongo to Be Live stream.
You're very welcome. Thank you.
Excellent. And from the show floor, we will
take a short break and we'll be back very soon with another
guest. Thank you.
Welcome back to the show floor here at the dot Local London.
We do hope you're enjoying all the conversations and all the
guests that we've had today. Thank you for staying put with
us throughout the everything that we're doing today.
Now we earlier on today at the keynote, again expanding on what
we learned briefly at the keynote, Queryable Encryption
came up. I am.
(00:27):
I'm joined by Ken White, our security specialist here and Ken
Queryable Encryption. What's it all about?
South queryable encryption is a way to protect our customers
most sensible. So we see a lot of this in the
banking world, in, in the energyfields, in the DoD.
(00:50):
But really everyone has sensitive data, right?
Everyone has some kind of crown jewels that they want to
protect. And so the idea with gravel
encryption is that you can, you can save records in an encrypted
format, but the database doesn'tlearn what that is.
And they don't, it doesn't have keys.
And so if there's, if there's anattack, there's a compromise on
(01:13):
the database side. If there's, you know, someone
comes up and, and black blackmails one of our senior
engineers or holds a gun to our CE OS head and says, give me
this data from, you know, bank Xor whatever.
We say, fine, here's a, here's acopy of the data, but it's all
encrypted. Good luck with that.
So, so that's the that's the basic premise.
But the short version, because Iwas and I think it was at a
(01:35):
prior dot local in London that we launched variable encryption
a couple of years ago. It was, yeah.
We take the stuff, really. So there's a lot of things in
the odd world where you can kindof iterate and move quickly and
sort of move on the fly with cryptography and with security.
You want to be very deliberate and you really have really
rigorous, you know, engineering practices.
(01:57):
You want to analyze things very properly.
You want to have lots of third party reviews.
You want to be very funny and careful about what you do and so
we had it in a preview form for about a year and then we G8 it
last year. So what we just announced today
is that let me is that range supports and being able to do
(02:20):
ours, you know, searching for everyone over 25 to 50 and
really financial balance it's you know between 10,000 million
dollars, OK, every every audit record, every sales transaction
happened between this state and that date is supported, but in a
way that the database never understands what's being asked
for. And even if you have at ADBA
(02:42):
level problems, you don't learn anything.
And this isn't the thing that and I spent I, I don't know what
event we were at, but I shared abooth with Cynthia who was a
prior PM or probably still is the PM in the in the space, but,
and I remember poking her all the time going, how does it
work? How does it work?
And so to me, you know, in that entire query roundset, nothing
(03:03):
is exposed. It's like magic.
It's still being able to query, but nothing has been exposed and
your data secure in transit, at rest and within the query
itself, correct? That's right.
So even if, pardon me, even if the network is compromised, even
if TLS is broken, even if someone gets a copy of the disk
and steals it or, or encryption rest was compromised, it doesn't
(03:27):
matter because it's, it's similar to Indian encryption
like you'd find in WhatsApp or Signal where the idea is you've
got two parties communicating anyone that eavesdrops over the
network. If I haven't turned my phone off
and you've sent me a message, that message is sitting on a
server somewhere. But it doesn't matter if, you
know, if I don't log on for a little bit, if they had access
(03:47):
to the server, they still can't see it.
So it's a very similar idea to that.
What's unique about word on encryption is it was built from
the ground up for high performance data been searched.
So there's all kinds of encryption sort of libraries and
techniques out there, but very few are are meant for, you know,
generalized search for operational workload.
(04:09):
You mean there's a lot in the kind of academic world that are
kind of toy type. You know, use cases were like,
I've got 100 records and I've got 1 laptop and one database
server. But as soon as you get into
distributing systems, as soon asyou get into, you know, I've
got, you know, threaded web services.
I'm, I'm, I've got high key currency.
I've got tons of users, those things breakdown.
(04:32):
And then also there's some encrypted search techniques that
may take seconds or minutes to get an answer back.
Well, that's just not realistic for operation.
And I suppose a testament to howserious we take this is that we
have a research group for for cortography.
That's right. And, and I've built this and
when we put it out first, we also open sourced our methods,
(04:55):
right, because it needed to be up to scrutiny.
Yeah, exactly. So we worked with the group out
of Brown University called the Encrypted Systems Lab when we
built our first generation client side encryption.
And what they did that was really useful was they helped us
think about the model, what are the promises we can make?
What are the things that we can't guarantee.
You know what, it's important tounderstand what you don't get
(05:17):
from this particular security control.
That collaboration was really, really positive and they were
working a lot of, you know, other sort of advanced TAT.
Pardon me, no worries. And so about two years after we
launched, they had a major breakthrough.
We talked to them about it and then eventually we acquired the
(05:37):
company. So yeah, today we have an in
house cryptography research group.
It's headed up by Sandy Kamara and Tarita Wataz and people have
been in the field for, you know,in this case, over 20 years.
He literally pioneered the fieldof encrypted search when he was
back at Microsoft Research. And, and so a lot of the, the
knowledge, the basis, the sort of formalisms have been out
(06:01):
there for 1520 years. What we, you know, we're able to
do is to operationalize that into a practical database and
put things in place that made iteasy for developers to use
because a lot of times, you know, tech products put a huge
burden on the developer. They have to become cryptography
experts. There's in different encryption
(06:22):
modes and key sizes and all thissort of thing.
And we said, well, what, what ifwe made this very opinionated
based on conservative, you know,sort of cryptographer approved
values. So that basically you set up a
little boilerplate code. And then as a developer, if you
want to insert a document, you just write an insert statement
like you used to. You want to query a bunch of
documents. You should be able to query
(06:42):
without having to think too muchabout, you know, the underlying
crypto. And so, yeah, after that
acquisition, we refer to more cryptography researchers.
I think there's like 5 PhD cryptographers in the group.
They're publishing papers, they're leading, you know,
academic conferences. It is we now have like visiting
scientists and student interns. I mean, it's a proper full blown
(07:05):
R&D goods. It's.
Totally strategic for us given the clients that we have.
I mean, I know that, you know, certain institutions and certain
industries, financial, medical, etcetera are reluctant to go to
the cloud or we're reluctant to add.
But with this approach, essentially it's pretty much the
last barrier in being able to say we don't want our our data
(07:26):
being shipped around like this, correct?
Yeah, that's right. And so while we started with
some pretty sophisticated use cases, everybody's got crown
Jewelers, everybody has some kind of PII data or sensitive
data they want protected. But but our our sort of more
high end sophisticated customerssaid, look, we're not this isn't
some James Bond or science fiction scenario.
(07:47):
We have entire nation states that are targeting our platform.
When you're moving hundreds of billions of dollars a day
through capital markets, that's not fiction.
You, you really do have to kind of think about that.
And, and none of that to say that we've, we've sort of
developed some magic silver bullet that'll cure all
problems, but rather that we took a very disciplined, careful
(08:07):
approach of confidentiality, of keeping your secret secret,
keeping the secret from us. You don't have to trust us that
we'll do the right thing becausethere's also there's, there's
this sort of hackers, there's the hacker side, but then
there's the what happens if I'm compelled as a service provider?
What if we get a court order or some law enforcement thing or,
(08:30):
or you're under litigation and you know, a judge says I need a
forensic called Mongo DB. Give me a copy of this
customer's data and fine, we'll give you a copy of it, but it's
encrypted. Good luck with that.
But what it really means is thatthe customer controls their data
unless they supply the decryption in their keys, it it
(08:50):
doesn't matter whether again, our CEO is held hostage or
that's right. Even if somebody had root level
privileges on the virtual machine, even if they had
hypervisor level access, it's it's not, it's not.
They're not seeing the clear text data and they don't have
access to that being. Super secure.
So we we announced range queriesbefore that it was exact match
(09:12):
it was yeah. So range queries allows, as you
said, a couple of examples, you know, search for transactions
between this value and that value except as well too.
Is there more to come? Full text search for.
Oh, yeah, absolutely. So, so program encryption is an
is a platform for expressive search.
OK. So we're, you know, we're hoping
in the next, in the first half of next year to introduce tech
(09:35):
support, prefix FX substring support.
We'll Gea that after some reasonable amount of time and
then yeah, then then, then, thenit's open.
You know, we're potentially considering vector, we're
potentially considering, you know, geocode.
It's it's an expressive platform, so we're super excited
about it. Excellent.
And where do developers go to find out more?
Ken Sure, wherever they go today.
(09:56):
Yeah, so just query Mongo DB Queryable encryption.
We've got docs and tutorials andcode snippets you can copy and
paste if you Google the Mongo DBphotography group, lots of
technical papers. If you want to look at the
formal proofs, you want to understand the underlying math.
That's your source. Edison, excellent.
Well, listen, Ken, this has beenan eye opener.
(10:17):
I can't say that I still understand how it's done.
It looks like magic to me, but it's great to see it advancing
and what you've added with the range queries and I look forward
to seeing more. Thank you very much for joining
us on the Mongo to Be Live stream.
You're very welcome. Thank you.
Excellent. And from the show floor, we will
take a short break and we'll be back very soon with another
guest. Thank you.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Bobby Bones Show
Listen to 'The Bobby Bones Show' by downloading the daily full replay.