March 7, 2025 • 67 mins
Welcome to MongoDB Podcast with host Shay McAlister! In this engaging episode, we dive into the transformative realm of AI in revenue operations, featuring our special guest, Fabio, Senior Software Development Engineer at MongoDB. Discover how AI is revolutionizing rev ops through Mops AI, a tool born from innovative hackathons. Fabio shares insights on leveraging MongoDB's own tools and advanced technologies like OpenAI and Atlas Vector Search to optimize tedious processes and enhance data utilization. Perfect for developers, AI enthusiasts, and tech innovators, this episode offers deep dives into practical AI applications, internal tool creation, and the future of generative AI in optimizing business processes. Tune in for a compelling look at how AI-powered innovation is shaping the way we handle data and drive revenue!
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:06):
Hello there and welcome to MongoDB TV.
I'm Shane McAllister. I'm a lead on the developer
Relations team here at Mongo DB.Today's show is entitled How AI
is Changing Rev OPS, The Story of Mops AI.
And today I'm joined by Fabio Fabio's, a Senior software
development Engineer here with Mongo DB going to take us
(00:27):
through with great insights intotoday's topic.
So Fabio, without further ado, you're very welcome to the show.
How are you? I'm good.
Thanks for having me, Shane. Thanks for this great
opportunity to bring Mopsy Eye, The story of Mopsy Eye and the
have UPS of Mongo B. Excellent, excellent.
And I very much look forward to to go through this.
(00:48):
Fabio, where, where are you joining us from?
I'm from Brazil, Sao Paulo. I am I, I changed everything
four years ago and I was always,we're talking about this early,
but yeah, from Brazil. Excellent.
So you're I'm, I'm in Ireland, as people could maybe gather by
the accent and anyone who's tuned into this before, but it's
(01:10):
it's pretty cold here. We got some snow over the last
couple of days, which is which is rare for Ireland, to be
perfectly honest. But you're at the opposite
extreme being in Brazil at the moment, Right, Fabio?
So and you told me you're near the beach.
So I'm super jealous. This is this is just not fair.
Yeah. Yeah, I, I change everything
here. So I'm living at the beach right
(01:31):
now. So it's, yeah, it's opposite
here. So it's warm here.
The heat is it's, it's like kindof something here for us.
Yeah, excellent. And I always try and I know that
you've been with MongoDB nearly four years or so now, but I, I
know and I know our audience is always super interested in the
journey of most of the guests that I have on.
(01:53):
So take us back a little bit about your journey towards Mongo
DB. What did you study?
What other companies did you work in?
What other roles did you have, Fabio?
All right, yeah. So I always focus my career in
software development like a software engineer, starting with
like companies like Sumo Technologies, almost close to
(02:17):
Oracle. Then I, I, I work with some
healthily healthcare and companies here in Brazil and
working as a software architect.So when I got this dispositions
working with across of of multiple companies as a software
architect, I started to bring one more one more I can do not
(02:42):
only in software development like building Houston
implementations, Houston software.
So solutions like mobile apps orserver side applications or UI
client sides. So what we I can do more?
And then I started to study machine learning AI and bring
the powerful of how can those tools can be implemented in a
(03:04):
way that we can increase the business.
And then I got like a change in my career, in my life and I
joined Mongo DB. So from the software engineer
culture to a marketing future, how can an engineer live inside
a marketing team? And I got this great opportunity
(03:25):
from Mongo DB. So how?
And I started to build and reorganize using all the soft
engineer concepts that I have all those years, those 20 years
of soft, how can I increase the quality of our integrations,
internal integrations, pipelines, data pipelines and
all those things that we need from our customers to internally
(03:46):
to our sales department and brings marketing high accurate
data quality. And then we got this huge bloom
of AI in the market. So how AI is shaping everything
that we have today? How can we add this AI as a
tool, not on a competitor inside, but as a tool?
(04:09):
And then I, I started to say, OK, so now I have to bring my
study. So I, I started studying a lot
how I can use all the concepts of like AI gene AI, only
generative applications, but butin a way that we can say, OK,
all the machine learning concepts that we have for
(04:30):
shaping like algorithms or increase the powerful of data
accurates. So, and then I got this
opportunity for MOP CI. We're going to, we're going to
go there on the MOP CI because we have like a great story
there. But that's the path that I that
I took from Mongo DB. OK.
And for the audience that we have, which is generally always
(04:51):
wide and varied, two things. First, Fabio, in the title we
say how AI is changing Rev OPS. There's a story of Mops AI.
Can you explain about Rev OPS and Mops AI in particular?
What, what do those two things mean to you and how do we
explain those to our audience today to give them an an idea of
what we're going through? And the other thing for me
(05:13):
actually is all too often when we've got internal MongoDB
people on the live streams that we do, we're showing off a new
product or a new feature, etcetera, and we're telling the
world how to use it. This is actually us using our
own products, using our own technology to help us do our own
operations as well too. So you know, what's the
(05:34):
background here? Why, why, why was this tooling
built? So that's three questions.
I'm sorry, Rev OPS, what does that mean?
MOPS AI, what does that mean to you?
And and what are we going to give us a high level background
as to to what you're building and and the significance of it?
All right. So let's start with with Rev
OPS. So Rev OPS means revenue
(05:54):
operations. So, and this means like we have
like a business strategy, like let's say, like a operation
model design for align things, process tools to increase the,
the the revenue internally usingQ standard applications, certain
strategy products that we can, we can, we can go and also align
(06:15):
the teams, streamline the process in a way that we can
decrease the cost, but also, like I said, increase our
revenue to be more strategic, tobe smart, to do much more with
less effort, less human effort. And that's part of AI twos, gene
AI twos that we have there. And then we have like the a
(06:38):
great opportunity like for MOP side.
Why MOP CI is on this highlight matter?
MOP CI was born in a very interesting part of Mong DB and
of my life here in Mong DB. I was invited to the sales
hackathon on New York and I was there with a great team of
solution architects there. I was the only engineer from
(07:00):
from marketing and I got this great opportunity to got 4
tasks. We had four tasks we divided and
we separate ourselves in in teams to tackle 4 and was a
competition from Mongo DB USA and EMEA and was good because I
was I was competing by doing thesame time was creating such a
(07:23):
good energy there for for building how to use Genni to
fix, to improve and to to to getsolutions from those four.
And what the, the, the, the one that I, that I got was like, how
can we use the competitive data and say, OK, how can we use the
GNI to get some takeaways, highlights and insights from the
(07:46):
competitor with our products? And then we, we started, I was
starting to, to do the hackathon72 hours there up in live
online, 72 hours without sleeping.
And I said, I have an idea. How can we decrease the time
spent for marketeers to build audience campaign data?
(08:10):
Because today my crites have, it's not a problem.
It's not an issue, but they havea requirement and necessity to
go to the data house and say, okay, I have this data.
How can we structure the data? How do I know the structure of
the data? I don't know.
So what's the process? I have to create tickets.
(08:30):
I have to create requirements for other teams, cross teams to
say, hey, can you get the data for me?
And that's the problem. Takes time, takes one week, two
weeks to bring the data, shake the data, reformat the data,
standardize the data. And then I say, OK, now I can
bring whatever I have to do hereas a task activity here.
And then Mop said, I said, why we cannot use AI to say go
(08:55):
there, give get the data for me export in this format here.
And that's what I was starting to do it.
But then we have other problems,security problems.
I cannot put the data inside theLLM.
I cannot just go there and say, hey, here's the data, open AI,
our Mongo, they didn't know. So what we did was to teach, to
(09:16):
fine tune, to train the AI to understand about the metadata.
So doesn't matter about the data, it's about the metadata,
how the definitions are. And then brings the powerful of
Mongo DB vector search because we use it entirely, completely
all the index search algorithm that we have there to search the
(09:38):
knowledge base that we built from the data sets, from the
machine learning data sets that we have there.
So we built, we got the that data sets there, we define it.
We wrote every single line thereof the definitions with the
thing. Amazing thing.
And then we said, OK, now the LLM can understand and transform
(10:01):
natural language prompts from the user to SQL, SQL queries.
And then we can execute the queries there, get the data.
OK, interesting. Excellent.
I, I love so going back to the fact that this was created
during a hackathon, I love that I, I, you know, at Mongo DB, we
(10:21):
tend to do these every now and then and we're really, I
suppose, you know, it's at the behest of, you know, the rest of
the management and the rest of the higher up team to say you
can have a week out. We do skunk works, we do
hackathons and there's been lotsof internal tools and external
tools that have been the output of these hackathons.
(10:41):
So, so that's brilliant. And you know, we do hackathons
in the community as well too. And anybody who has an
opportunity to join us for those.
So, you know, people say why do a hackathon?
Fabio, you're living proof as towhy we do these things is
because, as you said, 72 hours coding, probably lots of pizzas
and soft drinks, right? But you got the genesis of what
(11:03):
you're building, and that's amazing to me.
But the other amazing thing is, as you said, when when marketers
are trying to build a campaign and you said you've got to
gather the data, raise the tickets, all of that, That's,
you know, that's necessary work,but it's not the goal, right?
You just have to do these thingsin order to have the information
to your hand. And I think from the world of
(11:25):
generative AI that we've all been slightly occupying for the
last at least 18 months to two years now, I don't want Jen AI
to make songs and pictures. I want Jen AI to do the drudgery
boring things that I don't want to do.
So this is a perfect example of that.
It's it's been able to use the raw data structure that in a
(11:45):
format to which then you can usethe the intelligence and, and
create the campaigns around. Right.
Yeah. That's that's it.
That's the the major highlight point from Mobb say I So instead
of doing like, no, just generatethis text for me here.
That's the tone. No, let's just standardize the
data. Can you standardize the dot
(12:06):
function, the dot title? Can you access the sales force
and update this update that and streamline a data crisis inside
our our sales operations? Can we summarize the content in
a way that we can also pre validate for publishing like
social media or summarizations of Hey, can you check these
(12:29):
dashboards here internally here and check if we have like a
problem here in the in the last two, two weeks ago?
So mops, I can do all those things now and, and, and it's in
streamline all the notificationsto all our our our team to
understand, OK, we, we have likeproblems or not.
And also how to export the data in a way that we can say, OK, I,
(12:50):
I need to bring this audience here, but I want to generate why
we have this problem here. And we can explain this with the
data, but I don't know the data.But mops, I knows all the
definitions that can fetch the data for me, export and explain
in a, in a way that could be like a five years old kid and
will explain to me very simple. So that's, that's what how we
(13:11):
are shaping the things. I love it.
And you touched on it earlier there, you know, using Atlas
vector search obviously as a keycomponent of this.
What other core technologies andmethodologies are used in what
you've built here? We are using Chrome Software
Engineering Object Oriented programming software
(13:33):
architectory with consular principles of of of service
oriented architectory. We're using the whole package of
Mongo DB. We're using Mongo DB database
collections there. We're using Mongo DB charts for
monitoring. Also Atlas is the the base for
for us. Everything is, is on the
foundation of Atlas and, and, and internally we are using all
(13:57):
the, all the, the, the, the structure of our services like
Kubernetes and all those things there which should shape and
host all the applications. We are using Python I think is
the, I think is the basic language for machine learning
and all the things we have all their linguists too, but we
would choose Python here. And yeah, that's it.
(14:17):
That's what we are using today. OK.
And did you mention open AI as well too in the mix or?
Yeah, yeah, yeah. Azure Open ID.
Just open AI. Just open AI for now.
We are using, yeah, we're using certain models there that we
choose for primary and fall backto do the tasks that we have,
(14:40):
yeah. OK, OK, so again, this was part
of a hackathon, so very, very quick, very pulled together, but
obviously there was some challenges that you faced.
What was the stand out challenges at the beginning
anyway for you to overcome in building this?
Was the first challenge was how can I teach an LLM in the, let's
(15:05):
say two years ago to build SQL queries in a good quality?
Because the problem is OK, OK, one thing is the user wants, how
many malleable contacts do you have in the database?
That's a simple question for Marketeer.
How can you transform that line in a SQL query?
(15:28):
And the problem is that you haveto teach the LLM to understand
how. So we're starting to, to
benchmark the open eye models tounderstand how accurate they
will be. And then we got like 2 results
of query and then we started to to fine tune this kind of models
like 200. OK, so what's the engine?
What's the database engine that you have?
(15:49):
So teach SQL to the LLM because LLM is just a, we call the LLM
like a parrot, like it's just a talk with tax, large tax,
standard tax. And then you have to, to put
some definitions there to say, now you, you, you, you, you can
calculate day Times. Now you can do where clause or
you can do SQL counts and these things, things like that.
(16:11):
And to do that, it's basically, but for certain engines, it's
not. So we took like one month to
develop this, this, this approach.
And we're starting to implement and improve the model.
So we go from GPT 3.5, then we go to GPT 4, then we go to GPT
Turbo, then we go to GPT Mini O Mini.
(16:32):
And then we, we have now the better model for an open eye to,
to do this SQL task. But the way that we prompt
engineering and the way that we fine tune is different.
So that's what's the challenge, how we can teach, but in a way
that we can also fine tune and train the model 27 more.
(16:53):
So that was the point. Two questions on that from me
then obviously people who know MongoDB were no SQL relational
database. Why SQL queries in this
instance? Is that where the contact
information was stored? That's how we had to use it.
Yeah, yeah. The data that we have is
structure, you know, internally is in another way because it's
like all like like I said, S3 buckets here.
(17:16):
So we don't have like a databasestructure you have like this.
So we use engines to get the data for us using SQL because
it's just a simple way to to getthe data.
But all the data that we have like store the data for the UI,
for the service, for the business, everything is on the
is on because it's easy to to tofetch, it's easy to control,
(17:38):
it's easy to manage the disease to store.
This is how to prepare to visualize the data.
So. So only the data that is raw
data will be like in AWSS 3. Okay.
And then you mentioned obviouslyyou started this a while back
and you know the the models fromOpen AI obviously have advanced
in in the time. How how did you do your
(17:59):
evaluations there? You mentioned fine tuning and as
how did you do your evaluations as to whether the models were
good And then also did you face any tasks or complexities with
regard to migrating across newermodels as they got released?
Yeah. Whatever.
That was a fun story because what I did was to I wrote all
the questions and I wrote all the sequels and I just went
(18:23):
through the process. OK, so you benchmarked,
benchmarked. Everything and then I got a Co
working here that helped me to say so you will be my double
checker here because you are from analytics.
So I want you to understand if I'm doing correctly here.
So I double check and then we build this this definition, this
spreadsheet with all the benchmarks.
(18:44):
And then we start and I do, I did all the tasks on the MOPS AI
and got the results and compared.
And then I used the LLM MOPS AI to say, I asked this, that's my
answer, that's your answer, giveme a score.
And then we arrange the score there.
(19:05):
So they said what, what we have there, OK.
And then I'll say, OK, we are getting like a close to seven,
close to 8, close to 9, like thescore like 1:00 to 2:00 to
10:00. And that was the part that was
when we change the module from 3.5 to 4:00, we get huge like
quality. And then we change again to four
(19:26):
turbo. And then we, we, we keep
increasing and then we run. But every time that we run, we
have more benchmark, we have more use case.
And then we have to go to like aone hour, 2 hours test and then
three hours, 4 hour test again. So to, to understand what we
have there. So because it's like it's not
easy to to go to LLM and say, OK, let's build and generate
(19:51):
queries. Not, not it's not only that it's
not only easy to do that becauseyou face a lot of challenge
there. How the LLM will face your
context, face your definitions, your metadata to generate the,
the, the, the queries there. So you have to teach fine-tuned
to train, put some keywords and many keywords that will guide
(20:11):
the LLM to do specific things for you.
Yeah, which is a fairpoint because I think a lot of people
consider maybe that prompt engineering in a good way will
get you to where you need to be.But the fine tuning is
important. So as you said, you created your
own benchmark or ground truth. You know, I'm going to ask you
this. This is what I expect.
(20:33):
How far along have you got towards that?
Right. Yeah.
Perfect. Yeah, that's it.
Because. And that that that was obviously
quite a manual process at the beginning then Fabio for you.
Right. Yeah.
And, and, and, and it was good because in the beginning we
didn't have such things on the, on the market or, or some like
framework. Like today we have length,
graphic length chain. We, we started using those,
(20:55):
those, those frameworks. But today we don't use anymore
because we, we, we build our ownframework that I will, I will
demonstrate in a few minutes. But what we have is, but first
we have to benchmark. How do I know that the LM is
doing what I want the LM to do? And I have to, to get great
(21:16):
results there. Because if it were they I change
or, or of the user change the way that he's shaping the, the
context of the prompt, it will completely change.
But the LM will understand basedon the definition of the
metadata. So that's what, what, what I
was, that was my goal. So doesn't matter what the user
(21:37):
is trying to say, it's shaping the context and is highlighting
the points that I I need from the metadata.
OK. And you said, obviously there
we're going to see a little bit of this in action.
But before we do that, Fabio, you went and said, you know, it
was usually hard to put togethera campaign.
You needed the right data, you need to raise the right tickets.
(22:00):
And that obviously took a piece of time before these tools were
in place. How have things changed since
the the tool is in place now, the MOPS AI tooling in terms of
speeding up this and and making it easier and better to put
together? Oh, it's so we, we launch, we
launch the tool and so now it's internally it's, it's, it's,
(22:22):
it's working in a background wayon our on our internally chat.
And what we what, what the tool does is helping marketeers in
two channels. So, so today we have two
channels internally and one is more technical.
OK, Anyway, you can go there andprompt questions about summarize
(22:44):
A blog post from Mongo DB publicweb channel website, sorry, or a
case study or APDF or explain animage for me or something like
that. That can it can be like a simple
task and also be a good exploration.
They, they went OK, can you giveme the number of visitors on
this page here among the big close to more technical details.
(23:06):
And also we have other channel that we call that ask Mops
channel. So that's Mops channel is like
a, a way that anyone in marketing can go or anyone in
the company can go and say, hey guys, I'm having problems here.
I'm having issues. I cannot access this marketing
platform. I cannot access.
I don't have the permission for this.
I'm facing a problem with this. This is a critical or So what
(23:29):
multi I will do connect you to asubject matter expert create a
juratics. This is super highly priority is
like a blocker or a critical andcertain knowledge base.
So we have a knowledge base. So try to get the the the
content because sometimes marketing wants to look is a new
hire. So the new hire isn't is trying
(23:50):
to understand the process of marketing.
So he prompts a question there. Hey guys, what's the, the
concept of doing A to B and the mob say, I would say, OK, I can
help you go to this week page here internally and you can you
can access this, this, this, this concept.
So it will be close. We decrease the time.
So the speed of knowledge, the speed of context, we like
(24:13):
decrease so that we are decreasing the time and we are
bringing the AI to be close to what the the users, the magnet
tears want. So that's that.
So it's a, it's very, very good.OK.
And obviously you're very much reducing say repetitive non, you
know, non. I don't mean don't know what the
(24:35):
right word would be, kind of non.
Behaviors. Resultative tasks, you know,
yeah, the laborers time, tediousstuff that you've got to do to
pull these together. So so obviously and And how was
the initial reception when launched internally originally?
Was like a they like, I think was like a blow of their mind
because we're not now we're asking questions about we're
(24:59):
creating tickets to get the dataand now we can ask about it to
go to I don't know where the botwill will go, but we'll get the
data for me and export to Google.
She said, OK, in 10 seconds I have the data.
So I was like expecting one weekand then they are they are they
are. And also how we are generating
content like I want to summarizea blog post and go to the social
(25:21):
team and try to re preprocess this content to post on a social
media on MongoDB. How can I do that?
And then was like AI don't know two days, maybe 3 days.
Now it's like 1 hour or less than one hour because we can
bring the people to the table, say we have the pre content
(25:42):
here, let's work on this pre content, let's validate and then
we can try to publish or not. So I think that when you have
everything easy or pretty much easy, it's like a, it's a, it's
a word we go, Oh my God, is thatis that true?
But then you started to to ask. So now the, the, the problem
(26:02):
changed the problem. Now we want more features, We
want more. Can you do this now?
Can you do that? Can you automate this?
OK, excellent. And tell me this then too is
like with more and more questions being asked, for
example, do you have insights into maybe where there might be
bottlenecks? Does it create more of a
(26:24):
knowledge base depending on moreof the questions that have been
asked? Or you know, how does it how do
you improve this over time? Yeah.
So we have the, the feedback system internally, so the users
can feedback and thumbs down, thumbs up and add like a
description there. So what we are doing today is
getting this data and improve the, the way and also try to add
(26:49):
good questions, good answers from OPS AI to new related
questions. So like, if you prompt a
question today, what we're goingto do is, oh, Fabio just
answered one week ago a questionhere, a prompt, a question here
and Mopsy, I answered this. So check this out.
This might help you to understand what what we have in
(27:12):
the same context. So try to link the same thing
and use the feedback system to enhance the questions and and
answers. OK.
So, so there's an element of memory there.
Similar questions are surfacing up, a similar answer in the past
perhaps to speed everything up as well too and see if that's
appropriate. Yeah, OK, excellent.
(27:33):
Which all sounds fascinating. Fabio, you have something to
share and show in this regard with the audience then.
So right. Yeah, Yeah.
I want to, I want to share 22 simple features that we have
from the Somalization point thatI mentioned here.
Let me one second, let me do here.
(27:53):
No problem. And if anyone has any questions
while we're having a conversation here, please throw
them in the chat. I know Noor you were you were
asking about intern roles and Mongo to be slightly off topic,
but we do have an intern program.
I don't know where you're based.Go to the website, go and check
that out as well too. So it's great to see so many
(28:14):
people join us from different places.
But as we go through the demo and as maybe the conversation so
far has prompted some questions,drop them in the chat.
We'd be only too happy to engagewith them as as we go through.
So let me bring your screen up then.
So Fabio and you can tell us what we see here.
If you could do kind of control Plus or whatever to zoom the
(28:37):
bright to zoom the view a bit more would be great.
It's a little small. Can we?
Can you do that? I, I, I think because this is
the, this is the there's lack here.
OK, Yeah. OK, so let's let me try to, to,
to explain to you. So this is our, our Slack
(28:57):
sandbox here internally super clean here.
So what I want you to understandis that Mobtei is working here
right now in the background. So when I when I prompt here, I
will send a message here on the Mobtei chat here.
OK Mobtei will try to engage. Are you you're you on a Mac?
No. Yeah, I'm the Mac.
(29:20):
So do command plus there, if it's, it should zoom.
It should zoom the UI. Lovely.
Perfect. Perfectly.
Yeah. That makes perfect sense to our
viewers. Thank you.
Yeah. That's great.
So let's thanks. Thank you.
Let's centralized here. OK, So what are what are we
going to going to see here is the summarization feature here
from OPSEA. So I will prompt choose
(29:41):
summarization types here. So you, you, you, you guys will
see the result here. So 1 is this one here.
So let me just paste here so it summarizes this blog post here.
So this is a blog post, a publiclink from MongoDB website.
So you guys can you guys can seethere.
So what I wanted to do here is hit the press button here and
(30:02):
then mop say I will engage you see that you have eyes there on
the on the map place you. See that?
That's a nice touch. And what popularity will do here
is to summarize and get the content from me in a social tone
way. What this means?
So when I bring this here, so this means that I'm gonna see
this highlight here. So it will be like a
(30:25):
summarization here. So this is already ready for
social media because we have like minimal characters.
There are minimal lamp size for for for post.
And we have two here Twitter andlinkeds in here, as you can see
here. So I can bring this to Twitter
here to the X platform and and Ican bring the linkeds in here.
(30:46):
And also I can go to the docs soI can open the Google Docs.
Everything that was generated byMop CI will be stored in a
Google doc. So anyone can share this with
the team, work offline or onlinewith the team, with cross team
to engage to improve the contentand then bring this to life on
(31:06):
the social medias. So and the last part, so that's
the feedback part. When you OK, good, thank you.
Mop say hi or oh, this is not not good.
Let's improve this. We have an opportunity here.
So you can thumbs up or thumbs down here for us here.
So that's the feature that we wewe we call the summarization.
(31:27):
So you can summarize this and you can change the way if you
want, like you can change the prompt here in such a way like
the the other way that I have here.
So let me get another content here.
So I will close the right threadhere and I will paste this here.
So now instead of a blog post, Iwill do a YouTube video here.
(31:49):
So this is a public YouTube video from our our channel here.
So when I when I bring this to to to live here, it's just a
simple YouTube video from Mongo DB as you can see here.
So practical implementation withour architectory dot local
London 2024 and what Mopsy I would do here is OK, Sorry, I
(32:09):
did something here for me. So hey Fabio, I compiled your
video somewhere based on this, so into a Google document here.
Here's the link here. So I'm going to open this and I
will go back here one second. So you guys will see what we're
(32:29):
going to one second here. I want you to bring this to life
here for you guys. Just want to see where is the.
You could share another screen or another tab either and I can
swap out the the slack view for that one in the doc.
Yeah, I just just need to see where is the the I just lost the
(32:56):
the other where is the. No problem.
See, these are live demos. This is the beauty of.
It so I hope present here Let's see can you stop here and
present Sorry. Sure.
OK, OK, where is the window? I'm sorry guys.
(33:21):
I'm trying to to get the I'm trying to get the empower.
No worries. I mean, take your time.
Take your. Time.
Yeah. So let's begin.
This is the one. I think I'm going to screen this
and bring this to life. OK, All right.
(33:42):
So can you see my screen? You can, yeah, it's a little
small. Again, if you can command plus
and zoom in be cool. Perfect.
Yeah. Oh, no, not this.
Sorry. That's the other dot.
Sorry, nowhere. Oh, I'm trying to to do OK, so
let me, sorry, let me try to getthis all right.
(34:06):
This is the one. Excellent.
So this is the file that was generated by MOPS AI.
As you can see here OK. And are you seeing the video
here there? Yes, we can see the video info,
all the the metadata I suppose associated with the video at the
top anyway, yeah. OK, so, so that's the so that's
(34:28):
the metadata of the video. So that's the description that
was generated by the MOP CI explain the video based on what
the YouTube was provide to us and that's the part of that is
good for MOP CI. So we are seeing here the time
stamp. So we built the sections and the
chapters of the based on the timeline.
(34:49):
So 51 seconds an overview 2. So this was not there.
So mops I just generated based on the videos, extract the links
and create the chapters. So you can divide and create
sections inside the video to click there.
So it's good. Fabio, I did not know that we
had this and I create videos allthe time.
(35:12):
So I'm going to start using thisbecause it's it's one of the
things we don't do prop well in in developer relations that we
produce a lot of videos all the time, but we generally do not
upload them with this level of detail and chapters and
summarization. This is great.
This is this is news to me. Thank you.
And and, and the great part herethat we are trying to implement
(35:36):
here with the team is using the summarization that we have like
we struck the key point. We create the article here based
on what we have for like a text,if you want.
And the most important part is based on the video transcriptor
here that we got here. You can see that is another
language here the LM can create like things like that.
(35:57):
So you see that the out transcriptor here from the
YouTube channel. So we can add more language, we
can make the video available in other language like Italian,
Spanish, whatever Portuguese, Brazil, whatever language that
we have there. We can do that because then you
can organize the entire video and make the video available for
(36:20):
for a big audience there for allthe other captions that we have
on the video. So that's that.
That's what we we. Yeah, and, and something that's
a prime example of something that would have been incredibly
time consuming to do in the pastand that's brilliant.
So and an operation like that, Imean, obviously use Slack there
(36:40):
and the Mops AI tool to do this.I wasn't paying attention in how
quickly it responded and made that document, but it seemed to
be super quick. Right.
It is because what, what we whatthe the mop CI will do here is
to get the video transcript and based on the video transcript
that we have the base language every sometimes it's not
(37:00):
English. Sometimes could be like a video
can be prepared only for English, like only for Spanish.
I don't know. It could be like a video for dot
local in Mongo DB Spain, Barcelona or something like that
or Madrid. And then we have only the video
for that doesn't matter from upside.
We'll get this video and try to say, OK, now we have this video
(37:22):
and we'll prepare all the content for English.
But will that will be the transcription will still be on
the original language. Because if you ask for
translation, that's the feature that we are right now
implementing, trying to to use our translation platform to get
(37:42):
the content and say, OK, now let's translate this video from
Spanish to English, Spanish to Italian, Spanish to no to
Japanese or whatever available language that we have to make
this content available for the audience.
OK, excellent. So we've seen obviously the the
blog post and now a YouTube video.
(38:03):
What other essentially formats can be used or what other types
of files in MOPS AI? Oh.
OK, so to summarize, like if youwant to summarize things like
that, we can do. I can, I can bring the life of
this. Let's put this thing here.
(38:23):
Can I can I stop screening here and put the other one down you?
Can indeed all right, you can get your other screen.
We've we've impressed Romulo. I hope I pronounced that right
anyway, so he's he's very impressed at what he just saw.
Hopefully we can we can continue.
Thank you for the for the comments.
Let me do the sorry. I was like where is the Oh here.
(38:47):
OK, So what I want you to to bring is first is explained
here. So today we can do PDFs, Google
Docs, Google spreadsheets, also Microsoft files like docx or
Excel files. And most of the time what we are
(39:09):
doing is monitoring files from our platforms.
So we receive those files like CSVS is the most common and we
try to say, hey, Mopti, what is this what I have here?
What's the matrix that I have here on this data on this file?
What's the content here? And then explain to us this one
(39:29):
of the things that we have thereis can I, oh, it's not there.
I tried to. Is this your diagram?
Is this? Thank you.
Thank you. Yeah.
All right. So no worries, there you go.
So. What I want you to highlight how
powerful the MOPS AI was. So this is a high level
architecture here that we we we bring over the the monks.
(39:53):
So as you can see here, we have like starting from the front of
the bottom, you can see that we have a framework there with the
services that the internal services that we have there, the
common service, but also our foundation.
The LLM Foundation, the Azure Open eye, everything works.
So today is a plug and play architecture.
So one day if you want to OK, let's go to another provider, we
(40:15):
can do another provider that is just a plug and play situation
that we can we can do to avoid any kind of like a certain
breakdowns on the architecture. We have the raw LLM features.
What this means? That means that the basic
features that I, I this is like a spoon like where you have
summarization, how you do summarization, it doesn't
(40:35):
matter. You can do summarization in a
certain way. You can do classification, you
can standardize the data, you can output format standards if
you want. So this is the basic one, how,
how to, to speak, how to, to, tohow the LMV will speak to you.
Then you have like business LM features like like like we
(40:55):
demonstrate in a few minutes ago, like YouTube blog post,
case study, PDFSCSV, Google files, Microsoft files, whatever
we want. And we have the powerful of data
exploration, how we explore the data in such a way that we can
explain CSV files or go to a dairy house and search tables,
(41:16):
views or also search Mongo DB collections to get the data.
Say, hey, explain to me this, this, this or filter for this
data for this, this the time andget me the data for me.
Export this to a Google spreadsheet or something like
that and also the base and then the top level features that we
have is the task scheduler. So I build a task scaler because
(41:37):
we have certain tasks that needsto go like in a certain cadence
like a daily cadence or weekly cadence or specific like a
Saturday we need to run or and then we have the Mop CI API.
What this means, today we have abot, we use the Slack, we
integrate with Slack, but everything can be accessed
(41:59):
through this API. If one day we want to put the
MOPS AI in another situation, another host, it's just a matter
of integrate with the API, talk to the API, but that's one thing
that we can do there. Yeah.
And and look, but the the Slack interface is something that
(42:21):
people are very familiar with. It's a really good starting
point for a lot of these integrations and you know, a
really good way. Was there any particular
difficulties in adding it to Slack in the original in the
first place? In the 1st place, no, because
the way we are just slack with the just integration then with
(42:42):
forward the message from the user to to your application and
you process the message and reply back.
But when we started to do structure message output
structure message to our user, then the challenge is starts to
to to be fun because you you cango like that.
(43:04):
That's why I implemented the theoutput format standard module.
What is this? It's a markdown.
Can I reply in markdown? Can I reply in plain text?
Can I reply in links, structuredtopics what, what the user
wants? Sometimes the user doesn't
doesn't know what they want. So we go to a standard output.
(43:24):
But you know when, when you havelike a setting things like how
it's a business, but a business prompt or is a technical prompt,
you have a certain outputs that you you need to reply back
because that doesn't matter if it will be like highlight words
or something like that. So you can use LIKE for certain
things there. OK, OK, excellent.
(43:48):
You, you mentioned it earlier Fabio, with regard to security
etcetera that you were accessingthe data warehouse with the
metadata associated with it etcetera.
How you know what security pitfalls are here or potential
issues that you have avoided andand I suppose anybody trying to
build something similar should be aware of.
(44:10):
Yeah. So, so when we like like I said
early when we started to build Mops AI, the first thing is we
don't know nothing about the data.
So ours is logoing is we don't know nothing about the data only
from the meta data. So that's how we're is logoing.
Why that? Because.
I cannot. I need to put that on AT shirt.
Yeah, Yeah. I said something, I need to do
(44:32):
that because, because when you have like a, we have a lot of
data, we have powerful sensitivedata.
And the problem is PII. So, and not only PII is sharing
the data with a provider, even if is everything legal,
everything is OK, but it doesn'tmatter what we do then we have
(44:53):
to do is enclose the data, you know such a way in such layers
that LLM will not know about that.
So if you have to use the data exploration module or the
knowledge base module, anything that cuts the data and also
needs the LLM to explain the definition will be there, not
the data. So you have the results, you
(45:14):
have a number. I don't know could be like
5,000,000. I don't know what the number
means, but from the metadata that will give us something
that, OK, explain to me using the metadata.
So we teach and we, we train theLM to understand what is the
difference between a result in the context of the metadata.
Because I don't want, I don't want the data to be there.
(45:34):
I don't want to share the data with some, some LM that I, I was
not prepper, I was not there. I I don't know, I know who is in
the other end to understand what's what this means.
So that's the care of we have. And then the second point is our
users, our marketeers sometimes doesn't need to get PII data
(45:55):
because we already have here internally.
So if you want like a, oh, I want to export contacts leads,
you don't need the e-mail. You don't need that.
It's just that you need the datafor beauty the others, but you
don't need that because you already have that.
So certain points and Pi is willbe excluded and the data will
not be exported. Will not You'll be there in the
(46:17):
data set but it will not be exported.
So mobs, I understand extremely security guidelines and
following these extremely secureguidelines should understand the
data. OK.
OK. David just posted a question.
What do you call that security feature?
And I, I'm assuming and maybe David, you might want to follow
up, but I assume and you're talking about the fact that
(46:38):
we're only working with the metadata.
Yeah. Is that it or?
Did you pick that up differently?
Yeah, because, yeah, because when you, when you have, when we
build the security model, we putthe secret model as a layer and
everything that cuts the LLM, wefollow like some instructions
there. So if you wanted to export, we
follow certain export features there.
If you want to explain, we only you given the LM, we've only
(47:03):
given the the powerful of the metadata.
If you want to summarize the video you're going for the
instructions on on the social media.
So every everything has like a like a prompting instruction,
but also following the guidelines because we don't want
to just to give the data, we don't want to be generate
(47:25):
general standard solution. We have to be specific for each
kind of raw LLM feature that we revealed there.
OK, OK, David, I hope that answered your question there.
And I think that's one of the concerns that most people have
creating AI applications. Umm, you know, it is kind of,
you know, they don't want to share the data, they don't want
(47:47):
to be part of the corpus of training for an LLM owned by
somebody else. You know, and, and obviously we
see, you know, RAG solutions retrieval augmented generation
as being really good for that sort of thing whereby you've,
you've vectorized all of the data here, but you're using the
LLM to make sense of it as well too.
So that's a flavour of this in alittle bit, but you're doing it
(48:08):
mostly with the prompting Fabios, is that correct?
You're in terms of not exposing all of the data, just the bits
that you're interested. In yeah, yeah.
And that's that's that's what what I want to highlight here in
the my my last sharing here withthe audience here.
I want you to bring this this simple flow here and what this
(48:30):
flow means, that's how we are using.
You can see here that's how we are using or searching the data
exploration. So when we explore the data when
the user wants like a simple user prompter here how the
process will will do how MOPS AIwill face that.
So as you can see here we have all the checkings, all the
(48:51):
security here the search on the data house, we have the prepared
statement. What's the what's this means?
This means that every time that a question a query from the user
needs to be transformed from a natural language to a SQL query
or to a MongoDB query, that willbe like go for a process to is
(49:11):
this prompt engineer is harmful or not?
Is this have like static values or not?
What will what do is have like some PII here is asking for PII
or not. So then when past this, it'll be
super fast and then we're going to generate the query and then
the LLM will do not more anymorethere because they don't want to
(49:34):
do nothing there. I want to get the query I will
execute using our foundation framework and get the results
and explain to the user what we what we have using the LLM, but
using the definitions of the metadata that we have there.
So that's an explain here. So as you can see here, like the
exploration classification search on the vector search, the
(49:55):
opening eye, the checking of thevalues and explain the results.
So very simple, high level, but that's the structure of what we
have today. OK.
OK. David posted a follow up and
just wondering is, does this have a name in the industry kind
of this validation checking etcetera that you're going
(50:17):
through on the on the query thatyou're passing through?
In the industry I use this prepared statement is is it
something that we use for database engines that connect
frameworks like Java has one, Python has one, objected related
has others. The MongoDB connectors like
(50:39):
Pymongo has 1. So they they build this prepared
statement but in the security way.
I don't know if we have like a certain specific name for that,
but I I follow the guidelines from the from the database
principles like prepared statement for everything here.
OK, excellent. Well, look that you've gone
(51:00):
through all of the details. It's great to see it in action.
I think it was super quick. Anybody I think will agree on
that, particularly the video piece.
I I thought that was super fast.Whatever about summarizing A
blog post that is mainly text, but you know, I don't know how
long that video was that you used as your example, Fabio, but
for me, that's certainly a feature that I'm going to use.
(51:22):
What, if any future plans do youhave for Mops AI?
What have you not got to yet? Where are you hoping to go, if
you're able to share? Yeah, Right now what we are
doing is falling the first points that that you, you, you
highlight here instead of this generating content, how can we
increase the quality of our marketing platforms?
(51:44):
How can we act like Salesforce? How can we like we have a
feature today that we are tryingto standardize the data in the
Salesforce in a way that customers when we put the data
from our our source, the data will follow certain route paths
for the data to our internal systems.
(52:04):
And how can we shape, how can weorganize our connect data,
standardize that? Because today we have crossings
operating those data, trying to standardize the data and that's
not good because we are wasting resource, human resource there.
How can we use MOBS I to do certain things like that?
(52:25):
And also how can we use MOBS I to monitor, to alert, to notify,
to check, to analyze data from the dashboard?
Because when you build a dashboard, the dashboard will be
there, but you have to put someone there to understand.
Can we use MOPS AI to understandand to to understand how the
data is there was visualize it. Do you have an issue there?
(52:47):
So something like that. So we are, we're working to pull
like a smart behavior inside of our tools.
How can we can leave that? OK, OK, excellent.
One of the examples you had shared with me while we were
chatting to prepare for this live stream was the ability to
(53:08):
ask the performance of a particular web page etc as well.
Too fabulous. Is that correct?
Yes. When you ask how many visitors
we have on this lead page, yes, yeah, yeah, we have a certain
data set and you can ask Mopsy, I just say, hey, how many leads
do you have here on that Fister this space here and that that's
(53:31):
great because you can you can see web metrics there and that's
what we are we and that's one point that we are trying to
improve much more like put more data there, because sometimes
you want to see well, my audience is from oh, come from
this page from this dot local page, this registration page
(53:52):
here. Oh, this campaign here triggers
that. Oh, that's good.
So how can we know do we have the data, but in not super way.
If we have the data in a certaindata set, we can say, hey, mops
do that for me go there get the data from this page here.
How much visitors we have or distribute this in groups in
(54:12):
region demographic way or in a demographic way.
So you can, you can. Just prompted this to show chum
up. Say I am up.
Say I will do the work for you. Yeah, so, so like the video one,
this is another tool that I'm going to use.
So you're probably going to see lots of requests emanating from
Shane McAllister. But like in the past, like I'm
(54:33):
on the developer relations team,as I said in the introduction,
and we create a lot of the content that exists on the Mongo
DB Developer Centre. And in the past I'm, you know,
in Google Analytics, looking to see how certain pieces the
content are performing. We move to Amplitude a while
back. That's a relearning curve for
me. It's like, well, how do I find
out now? But now I realise that I can
(54:54):
just use MOPS AI to ask and see get my answers for me.
So I think this is as, as they say in the US in particular, you
know, dog fooding, right, using your own tools to to make things
better. And I think this is a prime
example because, you know, most of my live streams certainly
over the last year have been AI orientated.
(55:16):
And but This Is Us ourselves using AI to help our own
employees and our own internal teams to remove the drudgery of
some of these tasks and to make them more performant, which
obviously and it was in the title of of today's stream that
obviously leads to hopefully better revenue opportunities
then as well too, if things can be done quicker with more
(55:38):
accuracy, right? Yeah, perfect.
And, and, and that's the goal every quarter we are trying to
improve MOPS AI to do more tasksfor marketing and not only for
marketing, but for the MongoDB organization.
So how can today we are integrated with the sales team,
it's amazing. We are working with the sales
analytics team to understand howwe are moving every our data
(56:02):
from marketing to sales and how they are shaping the the data,
their way to use the data. So they are using our content,
they're using MOPS AI in other AI bots that are amazing bots
there on the sales side, amazingbots they are connecting.
So now MOP CI is an agent. And since it's an agent that was
built in a, in a time that agentwas not there.
(56:24):
Now we are integrating with other agents.
So we are talking about bots through APIs or through LLMS
talking exchange data and enhance their, their, their
their answers to the users because the audience on sales
will be like sales in markets will be marketed.
But how can we exchange the data, the flavors and how can we
(56:48):
use those datas to to say, OK, now marketing said this lead
here is a good lead. Here's a, is a point that you
can move this to Salesforce for you.
And how can we shape that? How can we afford that?
So that's good that that's the part that we are, we're working
right now to integrate both parts.
Okay, so the agents talking to agents, that's the agentic
(57:12):
future that you know now people talk about.
You know, people scare scaremonger and say the AI is
coming to take all our jobs, butthis is just going to make it
super simple, right? Um, in terms of, as I said
earlier, removing the drudgery and hopefully giving us back
something that we could be more proactive about, correct?
(57:32):
Yeah. Correct, because the future is
is the is this agent and like I said, mob say was building in a
in a time that agent was not there, the concept was not
there. We, we started using frameworks
to build something that now we don't need the frameworks
anymore. But not because the framework is
not good. It's amazing, like length, chain
(57:52):
length graphic, it's amazing framework, but we building it in
a way that we incorporate certain like I wrote a link in
the article that I can share later for you guys is how can we
use shared routing concept? So chat routing concept is a
concept from, from, from the, from the beginning of the genii.
(58:13):
How can we chat with the data and remember that sometimes in
the data? And then I started to think, how
can we use soft engineer concepts to understand more
about to build something that can be flexible.
So in the future, I, I want to add more flavors to the agent.
I don't want to stop the agent and deploy again.
I want to say, hey, here's another thing.
(58:35):
So that's why we have today summarizations classifications
with a long list of types of files.
We have image you can explain ifsomeone screenshot a error
inside like a Salesforce, you can go to them up and say, I
have this error here. Put the image there and MOPS, I
would try to say, oh, I see a problem here.
(58:56):
I can link to the knowledge basefor you and also tag a subject
matter expert to help you and assist you here right now.
So everything was like 4 weeks and now we have like minutes to
connect you to your problem to asolution.
And that's what what MOPS AI is.It's a, it's a just a system
(59:16):
that can help and you can add more.
That's why we are adding more and quickly on on the flavour on
the catalogue of Mobsang. Yeah, I love that a system that
can help and I think help us to the forefront of what you're
trying to do here. And as you said, remove the time
that it used to take to pull allof this together, to have the
(59:37):
expert on your shoulder to be able to escalate things if
necessary. So this is superb, Fabio, like
you've been embedded. You said you were in machine
learning before. You know, this generative AI
revolution we're in at the moment, the last two years, and
it's been only two years really.And at least in public
consciousness, I know AI has been around for a lot longer and
(59:57):
machine learning even more than that.
What excites you at the moment, given the work that you've done
here in the MOPS AI agent? What's exciting you about the
future, not necessarily in MOPS AI, but in general in the AI
space for. Me AGII think it's a it's a
concept right now. AGI I know that that it's not
(01:00:19):
like a opening eye. Just launch it.
Oh, we have like this new modulethat hits the 80 I don't know
with 80%. Sorry, sorry for the numbers.
A great score for AGI, but I think for me agent like talk
agents to agents, build more agents and how to automate this,
(01:00:43):
we'll have like a great impact when you don't need to build
business anymore. You have to allocate agents to
do the business for you in a certain way like what we are
doing today, like with using Mongo DB vector search, using
Atlas and Mongo DB vector searchto search the data for you, to
manage the data for you. It'll be the all the algorithm
(01:01:03):
available there and that's a great part.
And the second is AGI. When you hit the AGI phase, I
think I don't know if it'll be like one year or two years, I
don't know. But for me, the future will be
the there like I start with machine learning and try to
understand a whole list of, of concepts of AI there and like
(01:01:24):
deep learning, supervised learning, unsupervised learning.
And then when I, when I got the concept, I say, oh, now I got
it. Now I understand how the LM is
working or not. Like, like I said, it's just a
parrot talking about the data because just text this standard
text that patterns that needs tosearch and match.
(01:01:45):
But that's the the way. That's the way the LM say every
time that I see something, someone say, Hey, the LM is
thinking. No, no, the LM don't don't think
the LM the no reason that's thatthere was just a simple way to
to matching and standardize and and and do the the the pattern
and yeah, but yeah, so I think. I like that take on it.
(01:02:08):
I, I think you're, you mean you're obviously hit the nail on
the head and you're exposed to all of this as well, too.
I, I don't know where you go to learn and to keep up to date.
I listen to a lot of podcasts inthis space.
And there was one recently, I can't remember who it was, but
we're talking about AGI and they're saying, oh, you know, we
got, we're getting closer to AGI.
(01:02:28):
We're going to be able to think more like humans think.
And somebody responded and said,we still don't know how humans
think and how the human brain fully works.
So how the hell are we getting there?
But I know what they're saying. It's, it's becoming, it's, it's,
it's, it's a fascinating space. And I think you've summarized it
really well for me. The fact that you know this, the
(01:02:49):
inception of this was from a hackathon a while back into
something that's deployed into something that's saving people
time and something that does it in, in this instance, from what
you've shown me so far, two things that I needed to do.
So I'm now going to start to useit.
This has been superb. Fabio, any last comments for our
viewers today on this? We do have one link and I
(01:03:12):
apologize for the long URL, but essentially if you search for
Gen. AI powered video summarization
solution for Mongo DB, see an article there that Fabio was
involved in as well to showing some of the stuff that we were
doing in the background to thereas well too.
And as I said, most of our content, most of the things that
(01:03:35):
we do and build is up on our developer centre.
So developer.mongodb.com if you want to learn more.
Any last comments Fabio from from your good self in in terms
of maybe inspiration for other people to go out and build tools
such as this? I, I went, I went to highlight
one thing because from my experience, when I change the
(01:03:57):
path of my career from software engineering software act that
enterprise act that to work as an engineer in a marketing, that
was the most important decision that I did all my life.
And not because I'm here Mongo DB, but for, because Mongo DB
just opened doors and create this opportunity for me.
(01:04:18):
And with all the thing that I have on my back here is allowing
me to put the concept, put all these strengths and all these
skills that I have from soft engineering to build MOPS AI.
And not only MOPS AI, other tools, other applications that
we have here internally. But from the marketing
(01:04:40):
perspective, how amazing is to when you have this opportunity?
So if you, if you want to changesomething there, go ahead, study
yourself, analyze, go go deep dive into the the what, what I
did like four years ago, I started doing machine learning
in AI concept. And then I started to student,
(01:05:01):
get my MBA also and try to understand more about what,
what, what is this? What do we have here?
Because it's not the only the hype.
You don't don't, don't follow the hype of like frameworks or
products there. Take care, slow down, get the
concept first. Start doing what you want to do.
Experiment things here, experiment there and understand.
(01:05:23):
Benchmark everything, get the results.
And then you you, you get your insights.
Because one day, like we said here, could be AGI, could be
something different, could be, Idon't know, could be something
that we can we can do. We, we don't know yet, but
understand first because AI is amazing field, but we can do a
(01:05:43):
lot of things, but also we can do nothing because we don't
understand what they what we aregoing to to have with from the
LLM. We have a lot of LLMS there.
So yeah, that's, that's what I what I want to to highlight
here. Excellent and great words of
inspiration. And I think that's a really good
way to close out our live stream.
(01:06:05):
Fabio, thank you so much for joining me.
Do appreciate it and taking the time to to put this together and
to so eloquently go through how this was incepted at the
hackathon and how you built it and and then to show us the
demo. So do appreciate that and and
thank you to all our viewers forjoining in as well too, adding
your comments, etcetera. Appreciate that.
Please join us every Tuesday as we live stream lots of different
(01:06:29):
topics and we've got some great shows coming up.
But for now, from me in a very cold, freezing cold West of
Ireland to Fabio over in what's sunny Sao Paulo in Brazil, it's
been a pleasure, Fabio, to have you.
Thank you so much for joining me.
Thank you for this opportunity. Thanks for having me and thanks
for opening this great channel here among DB to explain the
(01:06:52):
story of Mops AI and what we build here.
And also we are currently building and improving and the
future will be amazing here. Excellent and that note as you
build more and more features in a number of months time, Fabio,
I'll certainly try and drag you back onto the live stream to
show us what you put together atthat point as well too.
(01:07:12):
But for now, this has been superb.
Thank you so much. It's been great to have you on
board. Great to have all our viewers.
And as I said, keep an eye on Mongo DB's, YouTube and LinkedIn
to keep up to date with future live streams.
But for now, thank you so much for joining us.
Do take care and catch you soon on another episode in the
future. Take everyone.
Thank you.
Hello there and welcome to MongoDB TV.
I'm Shane McAllister. I'm a lead on the developer
Relations team here at Mongo DB.Today's show is entitled How AI
is Changing Rev OPS, The Story of Mops AI.
And today I'm joined by Fabio Fabio's, a Senior software
development Engineer here with Mongo DB going to take us
(00:27):
through with great insights intotoday's topic.
So Fabio, without further ado, you're very welcome to the show.
How are you? I'm good.
Thanks for having me, Shane. Thanks for this great
opportunity to bring Mopsy Eye, The story of Mopsy Eye and the
have UPS of Mongo B. Excellent, excellent.
And I very much look forward to to go through this.
(00:48):
Fabio, where, where are you joining us from?
I'm from Brazil, Sao Paulo. I am I, I changed everything
four years ago and I was always,we're talking about this early,
but yeah, from Brazil. Excellent.
So you're I'm, I'm in Ireland, as people could maybe gather by
the accent and anyone who's tuned into this before, but it's
(01:10):
it's pretty cold here. We got some snow over the last
couple of days, which is which is rare for Ireland, to be
perfectly honest. But you're at the opposite
extreme being in Brazil at the moment, Right, Fabio?
So and you told me you're near the beach.
So I'm super jealous. This is this is just not fair.
Yeah. Yeah, I, I change everything
here. So I'm living at the beach right
(01:31):
now. So it's, yeah, it's opposite
here. So it's warm here.
The heat is it's, it's like kindof something here for us.
Yeah, excellent. And I always try and I know that
you've been with MongoDB nearly four years or so now, but I, I
know and I know our audience is always super interested in the
journey of most of the guests that I have on.
(01:53):
So take us back a little bit about your journey towards Mongo
DB. What did you study?
What other companies did you work in?
What other roles did you have, Fabio?
All right, yeah. So I always focus my career in
software development like a software engineer, starting with
like companies like Sumo Technologies, almost close to
(02:17):
Oracle. Then I, I, I work with some
healthily healthcare and companies here in Brazil and
working as a software architect.So when I got this dispositions
working with across of of multiple companies as a software
architect, I started to bring one more one more I can do not
(02:42):
only in software development like building Houston
implementations, Houston software.
So solutions like mobile apps orserver side applications or UI
client sides. So what we I can do more?
And then I started to study machine learning AI and bring
the powerful of how can those tools can be implemented in a
(03:04):
way that we can increase the business.
And then I got like a change in my career, in my life and I
joined Mongo DB. So from the software engineer
culture to a marketing future, how can an engineer live inside
a marketing team? And I got this great opportunity
(03:25):
from Mongo DB. So how?
And I started to build and reorganize using all the soft
engineer concepts that I have all those years, those 20 years
of soft, how can I increase the quality of our integrations,
internal integrations, pipelines, data pipelines and
all those things that we need from our customers to internally
(03:46):
to our sales department and brings marketing high accurate
data quality. And then we got this huge bloom
of AI in the market. So how AI is shaping everything
that we have today? How can we add this AI as a
tool, not on a competitor inside, but as a tool?
(04:09):
And then I, I started to say, OK, so now I have to bring my
study. So I, I started studying a lot
how I can use all the concepts of like AI gene AI, only
generative applications, but butin a way that we can say, OK,
all the machine learning concepts that we have for
(04:30):
shaping like algorithms or increase the powerful of data
accurates. So, and then I got this
opportunity for MOP CI. We're going to, we're going to
go there on the MOP CI because we have like a great story
there. But that's the path that I that
I took from Mongo DB. OK.
And for the audience that we have, which is generally always
(04:51):
wide and varied, two things. First, Fabio, in the title we
say how AI is changing Rev OPS. There's a story of Mops AI.
Can you explain about Rev OPS and Mops AI in particular?
What, what do those two things mean to you and how do we
explain those to our audience today to give them an an idea of
what we're going through? And the other thing for me
(05:13):
actually is all too often when we've got internal MongoDB
people on the live streams that we do, we're showing off a new
product or a new feature, etcetera, and we're telling the
world how to use it. This is actually us using our
own products, using our own technology to help us do our own
operations as well too. So you know, what's the
(05:34):
background here? Why, why, why was this tooling
built? So that's three questions.
I'm sorry, Rev OPS, what does that mean?
MOPS AI, what does that mean to you?
And and what are we going to give us a high level background
as to to what you're building and and the significance of it?
All right. So let's start with with Rev
OPS. So Rev OPS means revenue
(05:54):
operations. So, and this means like we have
like a business strategy, like let's say, like a operation
model design for align things, process tools to increase the,
the the revenue internally usingQ standard applications, certain
strategy products that we can, we can, we can go and also align
(06:15):
the teams, streamline the process in a way that we can
decrease the cost, but also, like I said, increase our
revenue to be more strategic, tobe smart, to do much more with
less effort, less human effort. And that's part of AI twos, gene
AI twos that we have there. And then we have like the a
(06:38):
great opportunity like for MOP side.
Why MOP CI is on this highlight matter?
MOP CI was born in a very interesting part of Mong DB and
of my life here in Mong DB. I was invited to the sales
hackathon on New York and I was there with a great team of
solution architects there. I was the only engineer from
(07:00):
from marketing and I got this great opportunity to got 4
tasks. We had four tasks we divided and
we separate ourselves in in teams to tackle 4 and was a
competition from Mongo DB USA and EMEA and was good because I
was I was competing by doing thesame time was creating such a
(07:23):
good energy there for for building how to use Genni to
fix, to improve and to to to getsolutions from those four.
And what the, the, the, the one that I, that I got was like, how
can we use the competitive data and say, OK, how can we use the
GNI to get some takeaways, highlights and insights from the
(07:46):
competitor with our products? And then we, we started, I was
starting to, to do the hackathon72 hours there up in live
online, 72 hours without sleeping.
And I said, I have an idea. How can we decrease the time
spent for marketeers to build audience campaign data?
(08:10):
Because today my crites have, it's not a problem.
It's not an issue, but they havea requirement and necessity to
go to the data house and say, okay, I have this data.
How can we structure the data? How do I know the structure of
the data? I don't know.
So what's the process? I have to create tickets.
(08:30):
I have to create requirements for other teams, cross teams to
say, hey, can you get the data for me?
And that's the problem. Takes time, takes one week, two
weeks to bring the data, shake the data, reformat the data,
standardize the data. And then I say, OK, now I can
bring whatever I have to do hereas a task activity here.
And then Mop said, I said, why we cannot use AI to say go
(08:55):
there, give get the data for me export in this format here.
And that's what I was starting to do it.
But then we have other problems,security problems.
I cannot put the data inside theLLM.
I cannot just go there and say, hey, here's the data, open AI,
our Mongo, they didn't know. So what we did was to teach, to
(09:16):
fine tune, to train the AI to understand about the metadata.
So doesn't matter about the data, it's about the metadata,
how the definitions are. And then brings the powerful of
Mongo DB vector search because we use it entirely, completely
all the index search algorithm that we have there to search the
(09:38):
knowledge base that we built from the data sets, from the
machine learning data sets that we have there.
So we built, we got the that data sets there, we define it.
We wrote every single line thereof the definitions with the
thing. Amazing thing.
And then we said, OK, now the LLM can understand and transform
(10:01):
natural language prompts from the user to SQL, SQL queries.
And then we can execute the queries there, get the data.
OK, interesting. Excellent.
I, I love so going back to the fact that this was created
during a hackathon, I love that I, I, you know, at Mongo DB, we
(10:21):
tend to do these every now and then and we're really, I
suppose, you know, it's at the behest of, you know, the rest of
the management and the rest of the higher up team to say you
can have a week out. We do skunk works, we do
hackathons and there's been lotsof internal tools and external
tools that have been the output of these hackathons.
(10:41):
So, so that's brilliant. And you know, we do hackathons
in the community as well too. And anybody who has an
opportunity to join us for those.
So, you know, people say why do a hackathon?
Fabio, you're living proof as towhy we do these things is
because, as you said, 72 hours coding, probably lots of pizzas
and soft drinks, right? But you got the genesis of what
(11:03):
you're building, and that's amazing to me.
But the other amazing thing is, as you said, when when marketers
are trying to build a campaign and you said you've got to
gather the data, raise the tickets, all of that, That's,
you know, that's necessary work,but it's not the goal, right?
You just have to do these thingsin order to have the information
to your hand. And I think from the world of
(11:25):
generative AI that we've all been slightly occupying for the
last at least 18 months to two years now, I don't want Jen AI
to make songs and pictures. I want Jen AI to do the drudgery
boring things that I don't want to do.
So this is a perfect example of that.
It's it's been able to use the raw data structure that in a
(11:45):
format to which then you can usethe the intelligence and, and
create the campaigns around. Right.
Yeah. That's that's it.
That's the the major highlight point from Mobb say I So instead
of doing like, no, just generatethis text for me here.
That's the tone. No, let's just standardize the
data. Can you standardize the dot
(12:06):
function, the dot title? Can you access the sales force
and update this update that and streamline a data crisis inside
our our sales operations? Can we summarize the content in
a way that we can also pre validate for publishing like
social media or summarizations of Hey, can you check these
(12:29):
dashboards here internally here and check if we have like a
problem here in the in the last two, two weeks ago?
So mops, I can do all those things now and, and, and it's in
streamline all the notificationsto all our our our team to
understand, OK, we, we have likeproblems or not.
And also how to export the data in a way that we can say, OK, I,
(12:50):
I need to bring this audience here, but I want to generate why
we have this problem here. And we can explain this with the
data, but I don't know the data.But mops, I knows all the
definitions that can fetch the data for me, export and explain
in a, in a way that could be like a five years old kid and
will explain to me very simple. So that's, that's what how we
(13:11):
are shaping the things. I love it.
And you touched on it earlier there, you know, using Atlas
vector search obviously as a keycomponent of this.
What other core technologies andmethodologies are used in what
you've built here? We are using Chrome Software
Engineering Object Oriented programming software
(13:33):
architectory with consular principles of of of service
oriented architectory. We're using the whole package of
Mongo DB. We're using Mongo DB database
collections there. We're using Mongo DB charts for
monitoring. Also Atlas is the the base for
for us. Everything is, is on the
foundation of Atlas and, and, and internally we are using all
(13:57):
the, all the, the, the, the structure of our services like
Kubernetes and all those things there which should shape and
host all the applications. We are using Python I think is
the, I think is the basic language for machine learning
and all the things we have all their linguists too, but we
would choose Python here. And yeah, that's it.
(14:17):
That's what we are using today. OK.
And did you mention open AI as well too in the mix or?
Yeah, yeah, yeah. Azure Open ID.
Just open AI. Just open AI for now.
We are using, yeah, we're using certain models there that we
choose for primary and fall backto do the tasks that we have,
(14:40):
yeah. OK, OK, so again, this was part
of a hackathon, so very, very quick, very pulled together, but
obviously there was some challenges that you faced.
What was the stand out challenges at the beginning
anyway for you to overcome in building this?
Was the first challenge was how can I teach an LLM in the, let's
(15:05):
say two years ago to build SQL queries in a good quality?
Because the problem is OK, OK, one thing is the user wants, how
many malleable contacts do you have in the database?
That's a simple question for Marketeer.
How can you transform that line in a SQL query?
(15:28):
And the problem is that you haveto teach the LLM to understand
how. So we're starting to, to
benchmark the open eye models tounderstand how accurate they
will be. And then we got like 2 results
of query and then we started to to fine tune this kind of models
like 200. OK, so what's the engine?
What's the database engine that you have?
(15:49):
So teach SQL to the LLM because LLM is just a, we call the LLM
like a parrot, like it's just a talk with tax, large tax,
standard tax. And then you have to, to put
some definitions there to say, now you, you, you, you, you can
calculate day Times. Now you can do where clause or
you can do SQL counts and these things, things like that.
(16:11):
And to do that, it's basically, but for certain engines, it's
not. So we took like one month to
develop this, this, this approach.
And we're starting to implement and improve the model.
So we go from GPT 3.5, then we go to GPT 4, then we go to GPT
Turbo, then we go to GPT Mini O Mini.
(16:32):
And then we, we have now the better model for an open eye to,
to do this SQL task. But the way that we prompt
engineering and the way that we fine tune is different.
So that's what's the challenge, how we can teach, but in a way
that we can also fine tune and train the model 27 more.
(16:53):
So that was the point. Two questions on that from me
then obviously people who know MongoDB were no SQL relational
database. Why SQL queries in this
instance? Is that where the contact
information was stored? That's how we had to use it.
Yeah, yeah. The data that we have is
structure, you know, internally is in another way because it's
like all like like I said, S3 buckets here.
(17:16):
So we don't have like a databasestructure you have like this.
So we use engines to get the data for us using SQL because
it's just a simple way to to getthe data.
But all the data that we have like store the data for the UI,
for the service, for the business, everything is on the
is on because it's easy to to tofetch, it's easy to control,
(17:38):
it's easy to manage the disease to store.
This is how to prepare to visualize the data.
So. So only the data that is raw
data will be like in AWSS 3. Okay.
And then you mentioned obviouslyyou started this a while back
and you know the the models fromOpen AI obviously have advanced
in in the time. How how did you do your
(17:59):
evaluations there? You mentioned fine tuning and as
how did you do your evaluations as to whether the models were
good And then also did you face any tasks or complexities with
regard to migrating across newermodels as they got released?
Yeah. Whatever.
That was a fun story because what I did was to I wrote all
the questions and I wrote all the sequels and I just went
(18:23):
through the process. OK, so you benchmarked,
benchmarked. Everything and then I got a Co
working here that helped me to say so you will be my double
checker here because you are from analytics.
So I want you to understand if I'm doing correctly here.
So I double check and then we build this this definition, this
spreadsheet with all the benchmarks.
(18:44):
And then we start and I do, I did all the tasks on the MOPS AI
and got the results and compared.
And then I used the LLM MOPS AI to say, I asked this, that's my
answer, that's your answer, giveme a score.
And then we arrange the score there.
(19:05):
So they said what, what we have there, OK.
And then I'll say, OK, we are getting like a close to seven,
close to 8, close to 9, like thescore like 1:00 to 2:00 to
10:00. And that was the part that was
when we change the module from 3.5 to 4:00, we get huge like
quality. And then we change again to four
(19:26):
turbo. And then we, we, we keep
increasing and then we run. But every time that we run, we
have more benchmark, we have more use case.
And then we have to go to like aone hour, 2 hours test and then
three hours, 4 hour test again. So to, to understand what we
have there. So because it's like it's not
easy to to go to LLM and say, OK, let's build and generate
(19:51):
queries. Not, not it's not only that it's
not only easy to do that becauseyou face a lot of challenge
there. How the LLM will face your
context, face your definitions, your metadata to generate the,
the, the, the queries there. So you have to teach fine-tuned
to train, put some keywords and many keywords that will guide
(20:11):
the LLM to do specific things for you.
Yeah, which is a fairpoint because I think a lot of people
consider maybe that prompt engineering in a good way will
get you to where you need to be.But the fine tuning is
important. So as you said, you created your
own benchmark or ground truth. You know, I'm going to ask you
this. This is what I expect.
(20:33):
How far along have you got towards that?
Right. Yeah.
Perfect. Yeah, that's it.
Because. And that that that was obviously
quite a manual process at the beginning then Fabio for you.
Right. Yeah.
And, and, and, and it was good because in the beginning we
didn't have such things on the, on the market or, or some like
framework. Like today we have length,
graphic length chain. We, we started using those,
(20:55):
those, those frameworks. But today we don't use anymore
because we, we, we build our ownframework that I will, I will
demonstrate in a few minutes. But what we have is, but first
we have to benchmark. How do I know that the LM is
doing what I want the LM to do? And I have to, to get great
(21:16):
results there. Because if it were they I change
or, or of the user change the way that he's shaping the, the
context of the prompt, it will completely change.
But the LM will understand basedon the definition of the
metadata. So that's what, what, what I
was, that was my goal. So doesn't matter what the user
(21:37):
is trying to say, it's shaping the context and is highlighting
the points that I I need from the metadata.
OK. And you said, obviously there
we're going to see a little bit of this in action.
But before we do that, Fabio, you went and said, you know, it
was usually hard to put togethera campaign.
You needed the right data, you need to raise the right tickets.
(22:00):
And that obviously took a piece of time before these tools were
in place. How have things changed since
the the tool is in place now, the MOPS AI tooling in terms of
speeding up this and and making it easier and better to put
together? Oh, it's so we, we launch, we
launch the tool and so now it's internally it's, it's, it's,
(22:22):
it's working in a background wayon our on our internally chat.
And what we what, what the tool does is helping marketeers in
two channels. So, so today we have two
channels internally and one is more technical.
OK, Anyway, you can go there andprompt questions about summarize
(22:44):
A blog post from Mongo DB publicweb channel website, sorry, or a
case study or APDF or explain animage for me or something like
that. That can it can be like a simple
task and also be a good exploration.
They, they went OK, can you giveme the number of visitors on
this page here among the big close to more technical details.
(23:06):
And also we have other channel that we call that ask Mops
channel. So that's Mops channel is like
a, a way that anyone in marketing can go or anyone in
the company can go and say, hey guys, I'm having problems here.
I'm having issues. I cannot access this marketing
platform. I cannot access.
I don't have the permission for this.
I'm facing a problem with this. This is a critical or So what
(23:29):
multi I will do connect you to asubject matter expert create a
juratics. This is super highly priority is
like a blocker or a critical andcertain knowledge base.
So we have a knowledge base. So try to get the the the
content because sometimes marketing wants to look is a new
hire. So the new hire isn't is trying
(23:50):
to understand the process of marketing.
So he prompts a question there. Hey guys, what's the, the
concept of doing A to B and the mob say, I would say, OK, I can
help you go to this week page here internally and you can you
can access this, this, this, this concept.
So it will be close. We decrease the time.
So the speed of knowledge, the speed of context, we like
(24:13):
decrease so that we are decreasing the time and we are
bringing the AI to be close to what the the users, the magnet
tears want. So that's that.
So it's a, it's very, very good.OK.
And obviously you're very much reducing say repetitive non, you
know, non. I don't mean don't know what the
(24:35):
right word would be, kind of non.
Behaviors. Resultative tasks, you know,
yeah, the laborers time, tediousstuff that you've got to do to
pull these together. So so obviously and And how was
the initial reception when launched internally originally?
Was like a they like, I think was like a blow of their mind
because we're not now we're asking questions about we're
(24:59):
creating tickets to get the dataand now we can ask about it to
go to I don't know where the botwill will go, but we'll get the
data for me and export to Google.
She said, OK, in 10 seconds I have the data.
So I was like expecting one weekand then they are they are they
are. And also how we are generating
content like I want to summarizea blog post and go to the social
(25:21):
team and try to re preprocess this content to post on a social
media on MongoDB. How can I do that?
And then was like AI don't know two days, maybe 3 days.
Now it's like 1 hour or less than one hour because we can
bring the people to the table, say we have the pre content
(25:42):
here, let's work on this pre content, let's validate and then
we can try to publish or not. So I think that when you have
everything easy or pretty much easy, it's like a, it's a, it's
a word we go, Oh my God, is thatis that true?
But then you started to to ask. So now the, the, the problem
(26:02):
changed the problem. Now we want more features, We
want more. Can you do this now?
Can you do that? Can you automate this?
OK, excellent. And tell me this then too is
like with more and more questions being asked, for
example, do you have insights into maybe where there might be
bottlenecks? Does it create more of a
(26:24):
knowledge base depending on moreof the questions that have been
asked? Or you know, how does it how do
you improve this over time? Yeah.
So we have the, the feedback system internally, so the users
can feedback and thumbs down, thumbs up and add like a
description there. So what we are doing today is
getting this data and improve the, the way and also try to add
(26:49):
good questions, good answers from OPS AI to new related
questions. So like, if you prompt a
question today, what we're goingto do is, oh, Fabio just
answered one week ago a questionhere, a prompt, a question here
and Mopsy, I answered this. So check this out.
This might help you to understand what what we have in
(27:12):
the same context. So try to link the same thing
and use the feedback system to enhance the questions and and
answers. OK.
So, so there's an element of memory there.
Similar questions are surfacing up, a similar answer in the past
perhaps to speed everything up as well too and see if that's
appropriate. Yeah, OK, excellent.
(27:33):
Which all sounds fascinating. Fabio, you have something to
share and show in this regard with the audience then.
So right. Yeah, Yeah.
I want to, I want to share 22 simple features that we have
from the Somalization point thatI mentioned here.
Let me one second, let me do here.
(27:53):
No problem. And if anyone has any questions
while we're having a conversation here, please throw
them in the chat. I know Noor you were you were
asking about intern roles and Mongo to be slightly off topic,
but we do have an intern program.
I don't know where you're based.Go to the website, go and check
that out as well too. So it's great to see so many
(28:14):
people join us from different places.
But as we go through the demo and as maybe the conversation so
far has prompted some questions,drop them in the chat.
We'd be only too happy to engagewith them as as we go through.
So let me bring your screen up then.
So Fabio and you can tell us what we see here.
If you could do kind of control Plus or whatever to zoom the
(28:37):
bright to zoom the view a bit more would be great.
It's a little small. Can we?
Can you do that? I, I, I think because this is
the, this is the there's lack here.
OK, Yeah. OK, so let's let me try to, to,
to explain to you. So this is our, our Slack
(28:57):
sandbox here internally super clean here.
So what I want you to understandis that Mobtei is working here
right now in the background. So when I when I prompt here, I
will send a message here on the Mobtei chat here.
OK Mobtei will try to engage. Are you you're you on a Mac?
No. Yeah, I'm the Mac.
(29:20):
So do command plus there, if it's, it should zoom.
It should zoom the UI. Lovely.
Perfect. Perfectly.
Yeah. That makes perfect sense to our
viewers. Thank you.
Yeah. That's great.
So let's thanks. Thank you.
Let's centralized here. OK, So what are what are we
going to going to see here is the summarization feature here
from OPSEA. So I will prompt choose
(29:41):
summarization types here. So you, you, you, you guys will
see the result here. So 1 is this one here.
So let me just paste here so it summarizes this blog post here.
So this is a blog post, a publiclink from MongoDB website.
So you guys can you guys can seethere.
So what I wanted to do here is hit the press button here and
(30:02):
then mop say I will engage you see that you have eyes there on
the on the map place you. See that?
That's a nice touch. And what popularity will do here
is to summarize and get the content from me in a social tone
way. What this means?
So when I bring this here, so this means that I'm gonna see
this highlight here. So it will be like a
(30:25):
summarization here. So this is already ready for
social media because we have like minimal characters.
There are minimal lamp size for for for post.
And we have two here Twitter andlinkeds in here, as you can see
here. So I can bring this to Twitter
here to the X platform and and Ican bring the linkeds in here.
(30:46):
And also I can go to the docs soI can open the Google Docs.
Everything that was generated byMop CI will be stored in a
Google doc. So anyone can share this with
the team, work offline or onlinewith the team, with cross team
to engage to improve the contentand then bring this to life on
(31:06):
the social medias. So and the last part, so that's
the feedback part. When you OK, good, thank you.
Mop say hi or oh, this is not not good.
Let's improve this. We have an opportunity here.
So you can thumbs up or thumbs down here for us here.
So that's the feature that we wewe we call the summarization.
(31:27):
So you can summarize this and you can change the way if you
want, like you can change the prompt here in such a way like
the the other way that I have here.
So let me get another content here.
So I will close the right threadhere and I will paste this here.
So now instead of a blog post, Iwill do a YouTube video here.
(31:49):
So this is a public YouTube video from our our channel here.
So when I when I bring this to to to live here, it's just a
simple YouTube video from Mongo DB as you can see here.
So practical implementation withour architectory dot local
London 2024 and what Mopsy I would do here is OK, Sorry, I
(32:09):
did something here for me. So hey Fabio, I compiled your
video somewhere based on this, so into a Google document here.
Here's the link here. So I'm going to open this and I
will go back here one second. So you guys will see what we're
(32:29):
going to one second here. I want you to bring this to life
here for you guys. Just want to see where is the.
You could share another screen or another tab either and I can
swap out the the slack view for that one in the doc.
Yeah, I just just need to see where is the the I just lost the
(32:56):
the other where is the. No problem.
See, these are live demos. This is the beauty of.
It so I hope present here Let's see can you stop here and
present Sorry. Sure.
OK, OK, where is the window? I'm sorry guys.
(33:21):
I'm trying to to get the I'm trying to get the empower.
No worries. I mean, take your time.
Take your. Time.
Yeah. So let's begin.
This is the one. I think I'm going to screen this
and bring this to life. OK, All right.
(33:42):
So can you see my screen? You can, yeah, it's a little
small. Again, if you can command plus
and zoom in be cool. Perfect.
Yeah. Oh, no, not this.
Sorry. That's the other dot.
Sorry, nowhere. Oh, I'm trying to to do OK, so
let me, sorry, let me try to getthis all right.
(34:06):
This is the one. Excellent.
So this is the file that was generated by MOPS AI.
As you can see here OK. And are you seeing the video
here there? Yes, we can see the video info,
all the the metadata I suppose associated with the video at the
top anyway, yeah. OK, so, so that's the so that's
(34:28):
the metadata of the video. So that's the description that
was generated by the MOP CI explain the video based on what
the YouTube was provide to us and that's the part of that is
good for MOP CI. So we are seeing here the time
stamp. So we built the sections and the
chapters of the based on the timeline.
(34:49):
So 51 seconds an overview 2. So this was not there.
So mops I just generated based on the videos, extract the links
and create the chapters. So you can divide and create
sections inside the video to click there.
So it's good. Fabio, I did not know that we
had this and I create videos allthe time.
(35:12):
So I'm going to start using thisbecause it's it's one of the
things we don't do prop well in in developer relations that we
produce a lot of videos all the time, but we generally do not
upload them with this level of detail and chapters and
summarization. This is great.
This is this is news to me. Thank you.
And and, and the great part herethat we are trying to implement
(35:36):
here with the team is using the summarization that we have like
we struck the key point. We create the article here based
on what we have for like a text,if you want.
And the most important part is based on the video transcriptor
here that we got here. You can see that is another
language here the LM can create like things like that.
(35:57):
So you see that the out transcriptor here from the
YouTube channel. So we can add more language, we
can make the video available in other language like Italian,
Spanish, whatever Portuguese, Brazil, whatever language that
we have there. We can do that because then you
can organize the entire video and make the video available for
(36:20):
for a big audience there for allthe other captions that we have
on the video. So that's that.
That's what we we. Yeah, and, and something that's
a prime example of something that would have been incredibly
time consuming to do in the pastand that's brilliant.
So and an operation like that, Imean, obviously use Slack there
(36:40):
and the Mops AI tool to do this.I wasn't paying attention in how
quickly it responded and made that document, but it seemed to
be super quick. Right.
It is because what, what we whatthe the mop CI will do here is
to get the video transcript and based on the video transcript
that we have the base language every sometimes it's not
(37:00):
English. Sometimes could be like a video
can be prepared only for English, like only for Spanish.
I don't know. It could be like a video for dot
local in Mongo DB Spain, Barcelona or something like that
or Madrid. And then we have only the video
for that doesn't matter from upside.
We'll get this video and try to say, OK, now we have this video
(37:22):
and we'll prepare all the content for English.
But will that will be the transcription will still be on
the original language. Because if you ask for
translation, that's the feature that we are right now
implementing, trying to to use our translation platform to get
(37:42):
the content and say, OK, now let's translate this video from
Spanish to English, Spanish to Italian, Spanish to no to
Japanese or whatever available language that we have to make
this content available for the audience.
OK, excellent. So we've seen obviously the the
blog post and now a YouTube video.
(38:03):
What other essentially formats can be used or what other types
of files in MOPS AI? Oh.
OK, so to summarize, like if youwant to summarize things like
that, we can do. I can, I can bring the life of
this. Let's put this thing here.
(38:23):
Can I can I stop screening here and put the other one down you?
Can indeed all right, you can get your other screen.
We've we've impressed Romulo. I hope I pronounced that right
anyway, so he's he's very impressed at what he just saw.
Hopefully we can we can continue.
Thank you for the for the comments.
Let me do the sorry. I was like where is the Oh here.
(38:47):
OK, So what I want you to to bring is first is explained
here. So today we can do PDFs, Google
Docs, Google spreadsheets, also Microsoft files like docx or
Excel files. And most of the time what we are
(39:09):
doing is monitoring files from our platforms.
So we receive those files like CSVS is the most common and we
try to say, hey, Mopti, what is this what I have here?
What's the matrix that I have here on this data on this file?
What's the content here? And then explain to us this one
(39:29):
of the things that we have thereis can I, oh, it's not there.
I tried to. Is this your diagram?
Is this? Thank you.
Thank you. Yeah.
All right. So no worries, there you go.
So. What I want you to highlight how
powerful the MOPS AI was. So this is a high level
architecture here that we we we bring over the the monks.
(39:53):
So as you can see here, we have like starting from the front of
the bottom, you can see that we have a framework there with the
services that the internal services that we have there, the
common service, but also our foundation.
The LLM Foundation, the Azure Open eye, everything works.
So today is a plug and play architecture.
So one day if you want to OK, let's go to another provider, we
(40:15):
can do another provider that is just a plug and play situation
that we can we can do to avoid any kind of like a certain
breakdowns on the architecture. We have the raw LLM features.
What this means? That means that the basic
features that I, I this is like a spoon like where you have
summarization, how you do summarization, it doesn't
(40:35):
matter. You can do summarization in a
certain way. You can do classification, you
can standardize the data, you can output format standards if
you want. So this is the basic one, how,
how to, to speak, how to, to, tohow the LMV will speak to you.
Then you have like business LM features like like like we
(40:55):
demonstrate in a few minutes ago, like YouTube blog post,
case study, PDFSCSV, Google files, Microsoft files, whatever
we want. And we have the powerful of data
exploration, how we explore the data in such a way that we can
explain CSV files or go to a dairy house and search tables,
(41:16):
views or also search Mongo DB collections to get the data.
Say, hey, explain to me this, this, this or filter for this
data for this, this the time andget me the data for me.
Export this to a Google spreadsheet or something like
that and also the base and then the top level features that we
have is the task scheduler. So I build a task scaler because
(41:37):
we have certain tasks that needsto go like in a certain cadence
like a daily cadence or weekly cadence or specific like a
Saturday we need to run or and then we have the Mop CI API.
What this means, today we have abot, we use the Slack, we
integrate with Slack, but everything can be accessed
(41:59):
through this API. If one day we want to put the
MOPS AI in another situation, another host, it's just a matter
of integrate with the API, talk to the API, but that's one thing
that we can do there. Yeah.
And and look, but the the Slack interface is something that
(42:21):
people are very familiar with. It's a really good starting
point for a lot of these integrations and you know, a
really good way. Was there any particular
difficulties in adding it to Slack in the original in the
first place? In the 1st place, no, because
the way we are just slack with the just integration then with
(42:42):
forward the message from the user to to your application and
you process the message and reply back.
But when we started to do structure message output
structure message to our user, then the challenge is starts to
to to be fun because you you cango like that.
(43:04):
That's why I implemented the theoutput format standard module.
What is this? It's a markdown.
Can I reply in markdown? Can I reply in plain text?
Can I reply in links, structuredtopics what, what the user
wants? Sometimes the user doesn't
doesn't know what they want. So we go to a standard output.
(43:24):
But you know when, when you havelike a setting things like how
it's a business, but a business prompt or is a technical prompt,
you have a certain outputs that you you need to reply back
because that doesn't matter if it will be like highlight words
or something like that. So you can use LIKE for certain
things there. OK, OK, excellent.
(43:48):
You, you mentioned it earlier Fabio, with regard to security
etcetera that you were accessingthe data warehouse with the
metadata associated with it etcetera.
How you know what security pitfalls are here or potential
issues that you have avoided andand I suppose anybody trying to
build something similar should be aware of.
(44:10):
Yeah. So, so when we like like I said
early when we started to build Mops AI, the first thing is we
don't know nothing about the data.
So ours is logoing is we don't know nothing about the data only
from the meta data. So that's how we're is logoing.
Why that? Because.
I cannot. I need to put that on AT shirt.
Yeah, Yeah. I said something, I need to do
(44:32):
that because, because when you have like a, we have a lot of
data, we have powerful sensitivedata.
And the problem is PII. So, and not only PII is sharing
the data with a provider, even if is everything legal,
everything is OK, but it doesn'tmatter what we do then we have
(44:53):
to do is enclose the data, you know such a way in such layers
that LLM will not know about that.
So if you have to use the data exploration module or the
knowledge base module, anything that cuts the data and also
needs the LLM to explain the definition will be there, not
the data. So you have the results, you
(45:14):
have a number. I don't know could be like
5,000,000. I don't know what the number
means, but from the metadata that will give us something
that, OK, explain to me using the metadata.
So we teach and we, we train theLM to understand what is the
difference between a result in the context of the metadata.
Because I don't want, I don't want the data to be there.
(45:34):
I don't want to share the data with some, some LM that I, I was
not prepper, I was not there. I I don't know, I know who is in
the other end to understand what's what this means.
So that's the care of we have. And then the second point is our
users, our marketeers sometimes doesn't need to get PII data
(45:55):
because we already have here internally.
So if you want like a, oh, I want to export contacts leads,
you don't need the e-mail. You don't need that.
It's just that you need the datafor beauty the others, but you
don't need that because you already have that.
So certain points and Pi is willbe excluded and the data will
not be exported. Will not You'll be there in the
(46:17):
data set but it will not be exported.
So mobs, I understand extremely security guidelines and
following these extremely secureguidelines should understand the
data. OK.
OK. David just posted a question.
What do you call that security feature?
And I, I'm assuming and maybe David, you might want to follow
up, but I assume and you're talking about the fact that
(46:38):
we're only working with the metadata.
Yeah. Is that it or?
Did you pick that up differently?
Yeah, because, yeah, because when you, when you have, when we
build the security model, we putthe secret model as a layer and
everything that cuts the LLM, wefollow like some instructions
there. So if you wanted to export, we
follow certain export features there.
If you want to explain, we only you given the LM, we've only
(47:03):
given the the powerful of the metadata.
If you want to summarize the video you're going for the
instructions on on the social media.
So every everything has like a like a prompting instruction,
but also following the guidelines because we don't want
to just to give the data, we don't want to be generate
(47:25):
general standard solution. We have to be specific for each
kind of raw LLM feature that we revealed there.
OK, OK, David, I hope that answered your question there.
And I think that's one of the concerns that most people have
creating AI applications. Umm, you know, it is kind of,
you know, they don't want to share the data, they don't want
(47:47):
to be part of the corpus of training for an LLM owned by
somebody else. You know, and, and obviously we
see, you know, RAG solutions retrieval augmented generation
as being really good for that sort of thing whereby you've,
you've vectorized all of the data here, but you're using the
LLM to make sense of it as well too.
So that's a flavour of this in alittle bit, but you're doing it
(48:08):
mostly with the prompting Fabios, is that correct?
You're in terms of not exposing all of the data, just the bits
that you're interested. In yeah, yeah.
And that's that's that's what what I want to highlight here in
the my my last sharing here withthe audience here.
I want you to bring this this simple flow here and what this
(48:30):
flow means, that's how we are using.
You can see here that's how we are using or searching the data
exploration. So when we explore the data when
the user wants like a simple user prompter here how the
process will will do how MOPS AIwill face that.
So as you can see here we have all the checkings, all the
(48:51):
security here the search on the data house, we have the prepared
statement. What's the what's this means?
This means that every time that a question a query from the user
needs to be transformed from a natural language to a SQL query
or to a MongoDB query, that willbe like go for a process to is
(49:11):
this prompt engineer is harmful or not?
Is this have like static values or not?
What will what do is have like some PII here is asking for PII
or not. So then when past this, it'll be
super fast and then we're going to generate the query and then
the LLM will do not more anymorethere because they don't want to
(49:34):
do nothing there. I want to get the query I will
execute using our foundation framework and get the results
and explain to the user what we what we have using the LLM, but
using the definitions of the metadata that we have there.
So that's an explain here. So as you can see here, like the
exploration classification search on the vector search, the
(49:55):
opening eye, the checking of thevalues and explain the results.
So very simple, high level, but that's the structure of what we
have today. OK.
OK. David posted a follow up and
just wondering is, does this have a name in the industry kind
of this validation checking etcetera that you're going
(50:17):
through on the on the query thatyou're passing through?
In the industry I use this prepared statement is is it
something that we use for database engines that connect
frameworks like Java has one, Python has one, objected related
has others. The MongoDB connectors like
(50:39):
Pymongo has 1. So they they build this prepared
statement but in the security way.
I don't know if we have like a certain specific name for that,
but I I follow the guidelines from the from the database
principles like prepared statement for everything here.
OK, excellent. Well, look that you've gone
(51:00):
through all of the details. It's great to see it in action.
I think it was super quick. Anybody I think will agree on
that, particularly the video piece.
I I thought that was super fast.Whatever about summarizing A
blog post that is mainly text, but you know, I don't know how
long that video was that you used as your example, Fabio, but
for me, that's certainly a feature that I'm going to use.
(51:22):
What, if any future plans do youhave for Mops AI?
What have you not got to yet? Where are you hoping to go, if
you're able to share? Yeah, Right now what we are
doing is falling the first points that that you, you, you
highlight here instead of this generating content, how can we
increase the quality of our marketing platforms?
(51:44):
How can we act like Salesforce? How can we like we have a
feature today that we are tryingto standardize the data in the
Salesforce in a way that customers when we put the data
from our our source, the data will follow certain route paths
for the data to our internal systems.
(52:04):
And how can we shape, how can weorganize our connect data,
standardize that? Because today we have crossings
operating those data, trying to standardize the data and that's
not good because we are wasting resource, human resource there.
How can we use MOBS I to do certain things like that?
(52:25):
And also how can we use MOBS I to monitor, to alert, to notify,
to check, to analyze data from the dashboard?
Because when you build a dashboard, the dashboard will be
there, but you have to put someone there to understand.
Can we use MOPS AI to understandand to to understand how the
data is there was visualize it. Do you have an issue there?
(52:47):
So something like that. So we are, we're working to pull
like a smart behavior inside of our tools.
How can we can leave that? OK, OK, excellent.
One of the examples you had shared with me while we were
chatting to prepare for this live stream was the ability to
(53:08):
ask the performance of a particular web page etc as well.
Too fabulous. Is that correct?
Yes. When you ask how many visitors
we have on this lead page, yes, yeah, yeah, we have a certain
data set and you can ask Mopsy, I just say, hey, how many leads
do you have here on that Fister this space here and that that's
(53:31):
great because you can you can see web metrics there and that's
what we are we and that's one point that we are trying to
improve much more like put more data there, because sometimes
you want to see well, my audience is from oh, come from
this page from this dot local page, this registration page
(53:52):
here. Oh, this campaign here triggers
that. Oh, that's good.
So how can we know do we have the data, but in not super way.
If we have the data in a certaindata set, we can say, hey, mops
do that for me go there get the data from this page here.
How much visitors we have or distribute this in groups in
(54:12):
region demographic way or in a demographic way.
So you can, you can. Just prompted this to show chum
up. Say I am up.
Say I will do the work for you. Yeah, so, so like the video one,
this is another tool that I'm going to use.
So you're probably going to see lots of requests emanating from
Shane McAllister. But like in the past, like I'm
(54:33):
on the developer relations team,as I said in the introduction,
and we create a lot of the content that exists on the Mongo
DB Developer Centre. And in the past I'm, you know,
in Google Analytics, looking to see how certain pieces the
content are performing. We move to Amplitude a while
back. That's a relearning curve for
me. It's like, well, how do I find
out now? But now I realise that I can
(54:54):
just use MOPS AI to ask and see get my answers for me.
So I think this is as, as they say in the US in particular, you
know, dog fooding, right, using your own tools to to make things
better. And I think this is a prime
example because, you know, most of my live streams certainly
over the last year have been AI orientated.
(55:16):
And but This Is Us ourselves using AI to help our own
employees and our own internal teams to remove the drudgery of
some of these tasks and to make them more performant, which
obviously and it was in the title of of today's stream that
obviously leads to hopefully better revenue opportunities
then as well too, if things can be done quicker with more
(55:38):
accuracy, right? Yeah, perfect.
And, and, and that's the goal every quarter we are trying to
improve MOPS AI to do more tasksfor marketing and not only for
marketing, but for the MongoDB organization.
So how can today we are integrated with the sales team,
it's amazing. We are working with the sales
analytics team to understand howwe are moving every our data
(56:02):
from marketing to sales and how they are shaping the the data,
their way to use the data. So they are using our content,
they're using MOPS AI in other AI bots that are amazing bots
there on the sales side, amazingbots they are connecting.
So now MOP CI is an agent. And since it's an agent that was
built in a, in a time that agentwas not there.
(56:24):
Now we are integrating with other agents.
So we are talking about bots through APIs or through LLMS
talking exchange data and enhance their, their, their
their answers to the users because the audience on sales
will be like sales in markets will be marketed.
But how can we exchange the data, the flavors and how can we
(56:48):
use those datas to to say, OK, now marketing said this lead
here is a good lead. Here's a, is a point that you
can move this to Salesforce for you.
And how can we shape that? How can we afford that?
So that's good that that's the part that we are, we're working
right now to integrate both parts.
Okay, so the agents talking to agents, that's the agentic
(57:12):
future that you know now people talk about.
You know, people scare scaremonger and say the AI is
coming to take all our jobs, butthis is just going to make it
super simple, right? Um, in terms of, as I said
earlier, removing the drudgery and hopefully giving us back
something that we could be more proactive about, correct?
(57:32):
Yeah. Correct, because the future is
is the is this agent and like I said, mob say was building in a
in a time that agent was not there, the concept was not
there. We, we started using frameworks
to build something that now we don't need the frameworks
anymore. But not because the framework is
not good. It's amazing, like length, chain
(57:52):
length graphic, it's amazing framework, but we building it in
a way that we incorporate certain like I wrote a link in
the article that I can share later for you guys is how can we
use shared routing concept? So chat routing concept is a
concept from, from, from the, from the beginning of the genii.
(58:13):
How can we chat with the data and remember that sometimes in
the data? And then I started to think, how
can we use soft engineer concepts to understand more
about to build something that can be flexible.
So in the future, I, I want to add more flavors to the agent.
I don't want to stop the agent and deploy again.
I want to say, hey, here's another thing.
(58:35):
So that's why we have today summarizations classifications
with a long list of types of files.
We have image you can explain ifsomeone screenshot a error
inside like a Salesforce, you can go to them up and say, I
have this error here. Put the image there and MOPS, I
would try to say, oh, I see a problem here.
(58:56):
I can link to the knowledge basefor you and also tag a subject
matter expert to help you and assist you here right now.
So everything was like 4 weeks and now we have like minutes to
connect you to your problem to asolution.
And that's what what MOPS AI is.It's a, it's a just a system
(59:16):
that can help and you can add more.
That's why we are adding more and quickly on on the flavour on
the catalogue of Mobsang. Yeah, I love that a system that
can help and I think help us to the forefront of what you're
trying to do here. And as you said, remove the time
that it used to take to pull allof this together, to have the
(59:37):
expert on your shoulder to be able to escalate things if
necessary. So this is superb, Fabio, like
you've been embedded. You said you were in machine
learning before. You know, this generative AI
revolution we're in at the moment, the last two years, and
it's been only two years really.And at least in public
consciousness, I know AI has been around for a lot longer and
(59:57):
machine learning even more than that.
What excites you at the moment, given the work that you've done
here in the MOPS AI agent? What's exciting you about the
future, not necessarily in MOPS AI, but in general in the AI
space for. Me AGII think it's a it's a
concept right now. AGI I know that that it's not
(01:00:19):
like a opening eye. Just launch it.
Oh, we have like this new modulethat hits the 80 I don't know
with 80%. Sorry, sorry for the numbers.
A great score for AGI, but I think for me agent like talk
agents to agents, build more agents and how to automate this,
(01:00:43):
we'll have like a great impact when you don't need to build
business anymore. You have to allocate agents to
do the business for you in a certain way like what we are
doing today, like with using Mongo DB vector search, using
Atlas and Mongo DB vector searchto search the data for you, to
manage the data for you. It'll be the all the algorithm
(01:01:03):
available there and that's a great part.
And the second is AGI. When you hit the AGI phase, I
think I don't know if it'll be like one year or two years, I
don't know. But for me, the future will be
the there like I start with machine learning and try to
understand a whole list of, of concepts of AI there and like
(01:01:24):
deep learning, supervised learning, unsupervised learning.
And then when I, when I got the concept, I say, oh, now I got
it. Now I understand how the LM is
working or not. Like, like I said, it's just a
parrot talking about the data because just text this standard
text that patterns that needs tosearch and match.
(01:01:45):
But that's the the way. That's the way the LM say every
time that I see something, someone say, Hey, the LM is
thinking. No, no, the LM don't don't think
the LM the no reason that's thatthere was just a simple way to
to matching and standardize and and and do the the the pattern
and yeah, but yeah, so I think. I like that take on it.
(01:02:08):
I, I think you're, you mean you're obviously hit the nail on
the head and you're exposed to all of this as well, too.
I, I don't know where you go to learn and to keep up to date.
I listen to a lot of podcasts inthis space.
And there was one recently, I can't remember who it was, but
we're talking about AGI and they're saying, oh, you know, we
got, we're getting closer to AGI.
(01:02:28):
We're going to be able to think more like humans think.
And somebody responded and said,we still don't know how humans
think and how the human brain fully works.
So how the hell are we getting there?
But I know what they're saying. It's, it's becoming, it's, it's,
it's, it's a fascinating space. And I think you've summarized it
really well for me. The fact that you know this, the
(01:02:49):
inception of this was from a hackathon a while back into
something that's deployed into something that's saving people
time and something that does it in, in this instance, from what
you've shown me so far, two things that I needed to do.
So I'm now going to start to useit.
This has been superb. Fabio, any last comments for our
viewers today on this? We do have one link and I
(01:03:12):
apologize for the long URL, but essentially if you search for
Gen. AI powered video summarization
solution for Mongo DB, see an article there that Fabio was
involved in as well to showing some of the stuff that we were
doing in the background to thereas well too.
And as I said, most of our content, most of the things that
(01:03:35):
we do and build is up on our developer centre.
So developer.mongodb.com if you want to learn more.
Any last comments Fabio from from your good self in in terms
of maybe inspiration for other people to go out and build tools
such as this? I, I went, I went to highlight
one thing because from my experience, when I change the
(01:03:57):
path of my career from software engineering software act that
enterprise act that to work as an engineer in a marketing, that
was the most important decision that I did all my life.
And not because I'm here Mongo DB, but for, because Mongo DB
just opened doors and create this opportunity for me.
(01:04:18):
And with all the thing that I have on my back here is allowing
me to put the concept, put all these strengths and all these
skills that I have from soft engineering to build MOPS AI.
And not only MOPS AI, other tools, other applications that
we have here internally. But from the marketing
(01:04:40):
perspective, how amazing is to when you have this opportunity?
So if you, if you want to changesomething there, go ahead, study
yourself, analyze, go go deep dive into the the what, what I
did like four years ago, I started doing machine learning
in AI concept. And then I started to student,
(01:05:01):
get my MBA also and try to understand more about what,
what, what is this? What do we have here?
Because it's not the only the hype.
You don't don't, don't follow the hype of like frameworks or
products there. Take care, slow down, get the
concept first. Start doing what you want to do.
Experiment things here, experiment there and understand.
(01:05:23):
Benchmark everything, get the results.
And then you you, you get your insights.
Because one day, like we said here, could be AGI, could be
something different, could be, Idon't know, could be something
that we can we can do. We, we don't know yet, but
understand first because AI is amazing field, but we can do a
(01:05:43):
lot of things, but also we can do nothing because we don't
understand what they what we aregoing to to have with from the
LLM. We have a lot of LLMS there.
So yeah, that's, that's what I what I want to to highlight
here. Excellent and great words of
inspiration. And I think that's a really good
way to close out our live stream.
(01:06:05):
Fabio, thank you so much for joining me.
Do appreciate it and taking the time to to put this together and
to so eloquently go through how this was incepted at the
hackathon and how you built it and and then to show us the
demo. So do appreciate that and and
thank you to all our viewers forjoining in as well too, adding
your comments, etcetera. Appreciate that.
Please join us every Tuesday as we live stream lots of different
(01:06:29):
topics and we've got some great shows coming up.
But for now, from me in a very cold, freezing cold West of
Ireland to Fabio over in what's sunny Sao Paulo in Brazil, it's
been a pleasure, Fabio, to have you.
Thank you so much for joining me.
Thank you for this opportunity. Thanks for having me and thanks
for opening this great channel here among DB to explain the
(01:06:52):
story of Mops AI and what we build here.
And also we are currently building and improving and the
future will be amazing here. Excellent and that note as you
build more and more features in a number of months time, Fabio,
I'll certainly try and drag you back onto the live stream to
show us what you put together atthat point as well too.
(01:07:12):
But for now, this has been superb.
Thank you so much. It's been great to have you on
board. Great to have all our viewers.
And as I said, keep an eye on Mongo DB's, YouTube and LinkedIn
to keep up to date with future live streams.
But for now, thank you so much for joining us.
Do take care and catch you soon on another episode in the
future. Take everyone.
Thank you.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Bobby Bones Show
Listen to 'The Bobby Bones Show' by downloading the daily full replay.