✨ Heads up! This episode features a demonstration of the SnapLogic UI and its AI Agent Creator towards the end. For the full visual experience, check out the video version on the Spotify app! ✨
(Episode Summary)Tired of tangled data spread across multiple clouds, on-premise systems, and the edge? In this episode, MongoDB's Shane McAllister sits down with Peter Ngai, Principal Architect at SnapLogic, to explore the future of data integration and management in today's complex tech landscape.
Dive into the challenges and solutions surrounding modern data architecture, including:
- Navigating the complexities of multi-cloud and hybrid cloud environments.
- The secrets to building flexible, resilient data ecosystems that avoid vendor lock-in.
- Strategies for seamless data integration and connecting disparate applications using low-code/no-code platforms like SnapLogic.
- Meeting critical data compliance, security, and sovereignty demands (think GDPR, HIPAA, etc.).
- How AI is revolutionizing data automation and providing faster access to insights (featuring SnapLogic's Agent Creator).
- The powerful synergy between SnapLogic and MongoDB, leveraging MongoDB both internally and for customer integrations.
- Real-world applications, from IoT data processing to simplifying enterprise workflows.
Whether you're an IT leader, data engineer, business analyst, or simply curious about cloud strategy, iPaaS solutions, AI in business, or simplifying your data stack, Peter offers invaluable insights into making data connectivity a driver, not a barrier, for innovation.
-
Keywords: Data Integration, Multi-Cloud, Hybrid Cloud, Edge Computing, SnapLogic, MongoDB, AI, Artificial Intelligence, Data Automation, iPaaS, Low-Code, No-Code, Data Architecture, Data Management, Cloud Data, Enterprise Data, API Integration, Data Compliance, Data Sovereignty, Data Security, Business Automation, ETL, ELT, Tech Stack Simplification, Peter Ngai, Shane McAllister.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:06):
Hello everyone and welcome to another MongoDB podcast live.
It's great to have you with us. I'm Shay McAllister and today
we've got a fantastic show as ever lined up for you.
We're diving into the one of themost exciting shifts in modern
data architecture, the move towards multi cloud, hybrid and
(00:26):
edge computing. And that's where Snap Logic
comes in. So it joining me today is Peter,
whose principal architect at Snap Logic.
And Peter is going to breakdown how Snap Logic is
revolutionizing the way businesses connect data across
clouds on Prem and beyond. We're going to talk about the
challenges that enterprises facein multi cloud and hybrid
(00:47):
environments and of course, how Mongo DB and Snap Logic work
together. And we're going to get a first
hand look at the Snap Logic UI and exclusive showcase of Snap
Logic's latest AI powered data automation in action.
Peter, you're very welcome to the show.
How are you? Hey, how are you?
Thanks, Shane. Thanks for having me on.
Not at all. It's great to have you.
(01:09):
So where are you joining us from, Peter?
I asked our guests, our viewers,to say where they are coming
from. Where are you based?
Sure thing. I'm based in the San Francisco
Bay Area, so more specifically if you're familiar with the
area, I'm I'm actually located in the East Bay.
That's where I live. Snap logic offices are actually
located on the peninsula, so that would be over to the to the
(01:30):
West side of the Bay. So, but currently I'm I'm home
so. Excellent.
So we're getting you at somewhatthe start of your day, tail end
of my day here in Ireland. And I know the US daylight
savings changed at the weekend right too.
Let's keep it current. So I think the scheduling of
this gave me a little bit of angst kind of going on my on my
5 or 8 hours on my 7 hours. What's what's the difference?
(01:55):
That's right, that's right, yeah.
And usually with the first week of daylight savings, it's a
little, it's a little bit sleepy, right?
Because like you lose an hour right in the spring of sleep
and, and, and it just takes me about a week to adjust to it.
So my kids, it takes any longer.But nonetheless, it's it's all
good. Yeah, Yeah.
(02:16):
Well, listen them it's great to have you join us, Peter, as
like, look, I, I talked about some of the things that we're
going to dive into in the introduction, but one of the key
things that I'm always interested in, I know that our
audience are as well too, is forany of the guests I have.
I love to know your path to date, your career path to date.
You know, how did you end up here?
(02:37):
What were you doing beforehand? What did you do all the way back
in college? You know, where have you been
since etcetera that took you to right now with your current role
as architect and Snap logic? Yeah, sure thing.
Well, I graduated from RIT. That stands for Rochester
Institute of Technology. That's located in Rochester, NY.
(02:58):
That was quite a few centuries ago, but I do hail from the New
York City area in tri-state areawhere I was born and raised.
And so naturally after graduating college, I stayed up
in Rochester. And if if anybody's familiar
with that city, there's, there'sbasically like Xerox and Kodak
were the were the major companies there.
(03:20):
So I weren't there for a number of years for both companies and
then moved back home to New YorkCity, worked on essentially Wall
Street for a number of years as an independent.
Then eventually it came time to,to raise a family.
My wife and I wanted to raise a family.
So we wanted to move out to finda place in, in the country where
it's a little bit different. So we, we came out here,
(03:44):
backpacked out to, you know, theSan Francisco Bay Area and just
kind of hunkered down and, and we've been here ever since about
2005. And so since I've worked at a
lot of, you know, I worked at some Fang companies and
essentially where I used to workprior in Fortune 500 companies,
now I find myself working for startups seems to be all the
(04:06):
thrill in the Bay Area. So that's how I wound up with
Snap Logic. Excellent.
You're bringing your Fortune 500knowledge back to the startups
basically then. So these are how the big
companies do it. This is what you, you, you need
to be thinking about as well, too.
It's interesting that you mentioned Rochester.
I, I, as I said, I'm from Ireland.
I've been traveling to the US quite a lot for work over the
(04:27):
years, but I'd say it was probably 20 odd years ago.
The last time I was in Rochester, I used to work for an
actual data storage company, a RAID systems company, and we
used to do our and what what they used to call accelerated
life testing of our systems there.
So they had a testing centre there that was unique, that
(04:48):
could do earthquake testing, allof this kind of high frequency
life, half life failures as well.
So I, I spent a few, few trips to Rochester, but that again,
long, long time ago. Perfect.
So you mentioned then culminating in snap logic.
Umm, tell us a little bit for the viewers who don't
(05:09):
necessarily know, tell us a little bit of, you know, around
Snap Logic, a kind of a high level overview of what Snap
Logic does, what the problem it's solving for its users and
in the kind of modern data ecosystem that we're in.
Sure thing, sure thing. So essentially Snap Logic is an
eye pass platform, right? So it's an integration platform
(05:32):
as a service. And what does that essentially
mean is that we help connect data sources in a company's
ecosystem one to another, also applying transformations in
between. In addition to data, we also
integrate applications across aswell, right?
So that's one advantage. So for example, if you wanted to
(05:55):
connect your, you know, Mongo database to like Salesforce upon
certain events that would certainly happen, then you can
certainly do that. And the advantage of snap logic
is that it's a basically a code free or a load code platform,
right? So it's ease of integration
(06:17):
there. OK, so you're kind of like an
interoperator between all of these platforms and, and
obviously, you know, I think particularly in large
enterprises, Mongo DB included as well too.
We, we use so many different platforms as well.
It's like, well, how do we connect all these things
together? And, and obviously from our own
perspective, we have integrations with our key cloud
(06:40):
partners and many other tech partners as well too.
But you're sitting in the middlespace then Peter and Snap logic,
you have to be almost agnostic. You just need to make sure you
can get, you know, from A to C through snap logic being to be
in the middle, right? That's right.
That's right. And it should be like a seamless
interaction too. And that's where it gets
(07:03):
somewhat tricky at times, right?So you have a lot of different
compliance space that changes a bit.
So you're talking about different environments as well.
So it's not just like point A topoint B, it's like you really
have to skip over like hurdles at certain times.
Yeah, I can imagine it's hard, you know and as maybe and I
(07:25):
don't know, but I would imagine your engineering team or very
much on top of the moving shifting, you know ecosystem
that's out there in terms of what people are building, what
people are adding and you know what knock on effect it might
have for snap logics infrastructure, right.
Yeah, exactly. Well, it would be to a certain
degree. I think a lot of it depends on
(07:47):
how you build the system, right?Because it's a very natural
problem space. I mean, if you look at
engineering and you had to connect one data source to
another, it's just a very natural inclination to like, I'm
going to write that one script that does that, right?
It's no one's paid job to reallydo it, first of all, right?
So it's just someone says I can write that in 5 minutes and they
do that. And the next thing you know,
(08:09):
someone changes something like a, a login or a password script.
I know what, I'm going to decouple my credentials into a
separate data store. Well, that's another piece,
right? And then, oh, the API change,
oh, that's another piece, right?So on and so forth.
So a lot of it really depends onhow you designed or architected
your system. And have to say, like App Logic
(08:29):
has done that really well where it's built upon a composable
architecture, right? There are there's always going
to be moving spaces in, in technology.
It's just a fact of life. And essentially, if you build it
right, you don't really have to change too much, but you would
need to append on to it, right? So it's built very well in that
(08:55):
degree where it's efficient and it scales well.
OK, so when you and I were chatting prior to to prepping
for this stream that we're doing, we talked a lot about
flexible data architecture and the kind of various, I suppose
you would call them pitfalls, but also, you know various
things to be concerned about as to how this.
But the flexibility, the advantages that gives you, if
(09:18):
you bear that in mind from the outset, talk a little bit about
that, like around, you know, avoiding things like vendor
locking and, you know, being compliance and being that
resilience that you just talked about.
That's right, that's right. And those are just some of the
key things of building a flexible arc data architecture,
right? Because in this day and age of,
(09:39):
of and there's also going to be,you know, the honest truth is
like economically, globally, we're all trying to save money,
right? We're trying to be more
efficient with what we have, right?
And we're also at the same time,we're also trying to maximize
what our expenses are, right? So there's gonna be a few key
principles that we look there for building flexible data
architecture, right? Is the first thing is that
(10:02):
companies really and there's andjust take a step back.
There's been a natural evolutionfrom data centers right to
clouds, public hypervisors. And now we're, we're embarking
on the next step, right? So where companies are starting
to notice that, yeah, there's there's the, the big cloud
(10:23):
providers like AWSGCP, Azure, soon and so forth.
But no companies really want to be locked in to like any one of
them, right? So they're trying to think
about, oh, how do I be cross cloud now?
How can I be cross cloud and on Prem or on the edge right as
well? So there's those factors in
(10:43):
there. There's also the the factor of
compliance, right So talk. To me, a little bit about on the
edge because I hear that a lot and I, I, I, I kind of know what
it is because we, we, I looked after a product that we
repurpose for edge stuff. But and as an edge database,
talk to maybe the viewers who mightn't be familiar with that
on the Edge might mean Peter. Sure thing.
(11:05):
Umm, so, so on the edge is essentially, if you have it,
umm, if you can think of that asa computing device that would
live, let's say for example, in your home, a Wi-Fi router would
be a perfect example, right? So that resides in your home,
right? If for those that have like
cable modems at home and such for Internet access, so inside
(11:27):
your Wi-Fi router, there's, there's certain intelligence
there. There's actually think of it as
a small computing engine in there that able, it's able to
perform some functions separate from the cloud, right?
Because everything in the cloud you're, you're we're always
thinking that we're just, you know, computing things in the
cloud and getting input and output from our homes where it's
(11:49):
like you took like a Wi-Fi router that's in your home, you
actually have a small computing devices there as well.
OK. So any devices that may be, you
know, an IoT device for example may also be edge device as well
like right Wi-Fi routers, littlelight switches, thermostat might
(12:10):
be in your home. I'm trying to stay away from
product names there, but. I get you.
We, we, I know when we were building our edge database
product, we were working with some of the the cellular
providers. So they were saying, look, when
your when your mobile device is making requests, you know, why
(12:31):
are we taking that all the way through the network when
sometimes the tower that received the request can
actually know what to do with it, right?
Exactly, Exactly. OK, And here's a here's a funny
little and I don't hear this term too too often anymore, but
like about five years ago, thereused to be this term called fog
computing and people like fog computing.
What's that? If you take a little
(12:53):
intelligence from the cloud and you bring it down to the edge,
it's like a fog. That's a new one.
That's a new one on me, but I love it.
I love the analogy. I think that I think that's
really good and it's great. If anyone else knows of any
other analogies like that, throwthem into the comments.
For those that joined and missedme, please add any questions you
(13:14):
might have for Peter as we make our way through the show.
It's great to see the folks joining us, you know from
Pakistan and Indonesia and Bangalore and Denver etcetera as
well too. So please do get involved.
When I asked you about age there, I had stopped you going
into the compliance part of how you were explaining what, you
(13:34):
know what's to be concerned about with flexible data
architectures, Peter. So talk to us a little bit about
compliance then. Sure thing.
So there's Prior to being a SNAPlogic, I used to work at a small
a healthcare company. And so we also used data talking
specifically about data in the compliance space in healthcare.
(13:58):
It was imperative that we neededto store our data on Prem, which
meant, you know, not in one of the public Capital Advisors
essentially. So we actually had Mongo
enterprise on premise, right. So that became very compliant as
well. Whereas the rest of our
computing was actually at the time on Azure, right.
(14:18):
So it was one of the public hypervisors, whereas our data
was actually on premise. So in industries such as like
healthcare and also finance, there would be some compliance
according to where the data is stored.
Also issues of topics of, excuseme, data sovereignty as well.
(14:38):
So there's certain data that areproduced in specific areas or
regions in the world that need to remain there as well.
So there's other types of issuesand topics that we that that
would come into play as well. Yeah, OK.
And being based in Europe, I'm particularly aware of the issues
because we have a very fragmented approach to data
(15:00):
sovereignty and compliance as well too.
That causes a ton of headaches, business opportunities at the
same time, but a ton of headaches for, for, for
companies as a whole, you know, and I'm assuming with with
regard to the compliance side ofthings.
The other factor then in the whole flexible data architecture
is just pure resiliency. That's why many people choose
(15:22):
multi cloud in the 1st place, right?
It's to, it's to have that resiliency either from a
physical perspective, a geolocation perspective,
etcetera as well, too, right, Peter?
That's correct. That's correct.
So a lot of companies are movingobviously depending on your
requirements towards resiliency or, or low RTO, low RPO, like
(15:43):
whether you have to be in different availability zones,
right? That's one, one area of like
resiliency. If that's good enough, then they
just, they're just basically in the same cloud provider, but
just in different availability zones.
They have to be in separate regions.
That's another requirement. And today there's even there's a
(16:06):
desire to be in, in multiple clouds as well, right?
So you have all those different factors at play.
OK, OK. And so that all of those
different factors at play are factors that really help Snap
Logic's position in that decoupling of of the
architecture that companies are kind of ending up with these
(16:29):
days. And as you said in your earlier
example, yes, it's fine. Somebody can write a piece of
code that can do something and then they kind of, as you said,
move chain, the login breaks andthen they got to decouple that
etcetera as well too. So obviously snap logic brings a
ton of benefits. I would imagine you know, given
the the huge decoupling, it's, it's also quicker to integrate,
(16:52):
maybe easier to scale, etcetera,more business value.
Talk to us a little bit about the kind of what you see, the
feedback from your customers, how it's how it's working for
them. Yeah, sure thing.
And that's a really good point, right?
Because it always, it's always seeds the idea whenever Snap
Logic is invited into a company to help integrate their systems,
(17:18):
right, always starts as a seed knowingly there.
It always branches out because it becomes such an efficient way
of, of performing this task thatit, it starts to just grow
because people, you know, companies find out that hey, I
just need to connect to this system, to this system.
(17:40):
I can write a script or I could just put together a small
pipeline in 5 minutes with snap logic and it's done, you know,
and it scales out really well and, and, and so that
proliferates through different business usages and it becomes
very efficient, right? Essentially the app integrations
(18:02):
and data integrations inside of companies usually doesn't start
as someone's like I use the termday job, but essentially it's
it's someone kind of takes it onas an extra, you know, an extra
function would want to try, but OK.
OK. And I suppose that might kind of
move us towards kind of where does Mongo DB fit into all of
(18:24):
this? Is that as well then, Peter, you
know, how does snap logic interact with Mongo DB?
What does Snap logic use Mongo DB for?
Sure, sure thing. So the main data store, well,
it, it integrates with Mongo DB in two facets, right?
The first one is that Mongo DB actually is the main data store,
(18:45):
right? For for, for snap logic, right?
So we chose in those sequels data store years ago to, to
house our data models, essentially all our assets and
operational metrics, all that isactually stored inside of our of
our Mongo clusters, right? So that's the first thing.
It's, it's acts as the backbone data store for Snap logic.
(19:09):
And now we also find out that there's a large customer base
that actually uses their Mongo data source that they would like
to integrate with their other systems, applications or their
other data source as well. So we we we have we we have
items called snaps. I'm trying to like see how it
(19:29):
can best explain this, but essentially.
And I know for those tuning in, we're, we are going to, you're
going to show this, we're going to open up the screen and share
the snap logic UI and, and, and go to that.
But that's intriguing though, because you're both a customer
and a partner of Mongo DB's essentially.
So you're using it internally for your own operations, but
(19:50):
you've obviously seen then your customers have data there that
needs to be connected to something else.
So hence you're you're also leveraging us as a partner.
Yes, that's correct. Perfect, perfect.
That's great. So I think so you touched on
snap packs. Do you want to talk a little bit
about what those? Let's set all the terms out, but
(20:12):
we're going to do show and tell,obviously in a few minutes,
yeah. Yeah, yeah, sure thing.
So. So if you think about how snap
logic is used to integrate systems in a company, it's what
is built in a workflow, which iscalled a pipeline, right?
So if you think about that as one executionable series of
(20:32):
steps that would comprise of a pipeline, let's say as simple as
read data from a file. That's one step in the transform
the data that's from the file into a separate, you know, a
separate format. That would be another step.
The third step is insert this data into Mongo.
(20:54):
That'd be the third step. So that comprises the pipeline.
Now each of the steps are also known as, as we call it a snap,
right, Because it's OK, OK, so yeah, so each of the snaps are
configurable as well. So it's it's pretty
integratable. And as you can think about,
there's hundreds of different snaps, right?
(21:16):
So and obviously you can snap them together however you want,
like like. I like it.
And obviously then the term snappacks is a is a pre built
sequence of snaps that you can leverage and is, you know, you
know, obviously kind of as you see customers doing certain
things that they require kind ofgoing, Oh, maybe is, is it the
(21:38):
case that you're looking at it going, maybe this is something
that all of our customers could benefit on.
Let's join all of these servicesup and give them a template.
Yeah, well, we do have template pipelines as well.
Snap pack is essentially a groupof snaps.
So for example, we have a Mongo snap pack.
(21:58):
In there we can connect to different Mongo data sources,
have different operations and such, right?
OK, OK, excellent. So, umm, I think we, we talked
obviously about those and obviously you know, all of the
introductions you spoke about the, the, the increasing
complexity that we live in, in a, in a kind of a multi cloud
(22:21):
and, and hybrid world as well then too Peter.
Umm, so I would imagine that then you know that complexity
and the way snap logic goes towards solving that has just
grown the business over time andand keeps growing it.
But I suppose in a roundabout way I'm saying the complexity of
today's environment is a benefitto snap logic because it it's
(22:45):
screaming out for the snap logicsolution applied to it, right?
Yeah, yeah, exactly. And if you look at the pace of
of technology evolution over theyears, it's just increasingly
faster, right? Every step.
And we're all familiar with the latest evolution of AI.
And, and if you look at the previous incantation to that, it
(23:07):
was just, it just seems shorter and shorter and shorter and
we're just exponentially going quicker and quicker and quicker.
So, yeah, staying in front of that curve, right?
It is certainly the question, right?
Yes, yeah, yeah, I'm surprised we got the 26 odd minutes into
the live stream before mentioning AI that usually it
(23:30):
usually comes up a lot earlier for for on most of my guests How
and I know we're going to see kind of an example of an agent
creator later on, but how has AIaffected Snap Logic's business
and and kind of how it thinks about things as well too.
What what changes has the AI brought about for the company as
(23:51):
a whole? Well, AI has always been part of
the fabric or the desire snap logic, even years before chapter
80 came out. Let's just say I think AI was
coined as machine learning. We had a staff of data
scientists that actually evolvedsnap logic and build upon it.
(24:15):
So we actually had predictive learnings of pipelines at that
time, even before chat ChatGPT. Yeah.
When AI came and now when when introduction of ChatGPT came
out, it became very conversational, right.
With the interface of of artificial intelligence and,
(24:35):
and, and, and the staying in front of that, essentially
that's what snap logic has done,right.
So, so essentially we become nowwe're not trying to be the LLM
or be the artificial intelligence engine.
We still want to be in that integration space, right.
(24:57):
So we enable connectivity to to the LLM and ease that
integration. And then there and I'll show
show this in a demo video, but essentially how one achieves
that very quickly in in a very low code environment.
OK, OK, excellent. I'm looking, I'm looking forward
(25:17):
to that. And again, I'm seeing the the
hellos coming in, in the comments.
It's great to have Andrew from Indonesia and folks from
Bangalore and Pakistan and Philadelphia.
Oscar, you're very welcome as well too.
Umm, you know, it's great to have you join us.
Do any questions you might have for Peter, please post them up
there. We'd love to entertain them.
So we talked a little bit about the multi cloud and the hybrid
(25:39):
world, et cetera. And you know, how you're
managing Snap via snap logic to connect data up with the
applications and platforms that need it, et cetera.
We mentioned compliance on the introductions.
How with regards to like, I presume you're kind of, you
know, my world is like an ETL, right?
(26:00):
Extract and transform and load. That's the way I'm, I'm looking
at some of this and I know it's,it's much, much bigger than
that. But that data in transit, how
does, where does SNAP logic fit in, in terms of being that
connector, in terms of compliance and security and
resilience there? Right, right, right, right.
So Snap Logic platform and how it's actually utilized, we're
(26:25):
very compliant with data governance and security, right.
So in dated governance, there's RBAC, permissioning, auditing,
full data metadata management right inside of security,
there's an encryption API security, of course Nexus
controls or SoC 2 compliant as well, right.
(26:48):
OK. In a compliance space where you
know, we're compliant with GDPR,HIPAA and of course, as I
mentioned, SoC 2 compliance. So, and I'm not sure if I spoke
about the architecture a little bit before, but essentially
there's there's a separation between how snap logic's
(27:10):
constructed, there's a separation between a control
plane and the data plane. OK, you know, I didn't quiz you
on that one. Tell us a little bit more about
that then. So, Peter.
Sure thing, sure thing. So if you think about the
control plane as being like yourmanagement console of your
system, right? And the data plane is
essentially think of this as your execution engine.
(27:34):
This is where the customer data would come to.
And essentially you would use a control plane to set up your
pipelines like I was talking about before.
OK, OK. You.
Use your data plane to essentially execute on those
pipelines right? So and the data plane can
reside. We offer what is known as a
Cloudplex. Right.
(27:56):
That that resides in the cloud. We, we can, we can, we can
assign you one of those or if you do choose, but we also
support what is known as a ground Plex, right?
So you can actually download thesoftware or if you want to run
it in a docker, we also have a docker image as well that you
can actually run a data plane either on your premise or in
(28:16):
your own cloud or your own tenant, essentially.
So we have both choices. And so in, in full compliance
like none of the no customer data would ever come to the
control plane essentially would stay, OK, that's your own
premise, right? So yes, it's very unique
(28:37):
architecture in that sense. OK, OK.
Now that makes sense and it becomes very clear now because I
would imagine that's probably one of the first things the
client, new client might say is,you know, hold on, Especially as
you mentioned earlier, people inthe healthcare sector and
obviously in the financial sector as well too.
And, and all of those areas where you know, that it's key
(28:58):
that, you know, data needs to have a ton of controls on it,
very, you know, very, very sensitive data, customer data,
the works. And, and I suppose look, Mongo
DB faces that as well too. And I think we've ticked most of
the boxes these days as well to make sure that usually when
these queries come up from potential clients, they can be
(29:19):
dismissed quite, quite easily byevery, all the scrutiny that
we've put ourselves to, as you mentioned, kind of the, the
Hipac and the Sock 2 and all of those things that Snaplogic has
assigned to as well. So we touched on, I get the
mechanism again, look in the demo, we're going to see this
really, I love the conversational piece of these
(29:40):
live streams, but I really like the demo.
So I'm looking forward to getting to that.
But before we do that, tell us alittle bit about, we've talked a
lot about how Snap logic works and how it helps companies,
etcetera. Tell us a little bit some of the
useful practical use cases that you might have from from from
customers, you know, can you talk to us a little bit about
(30:00):
how they might be used and leveraged before we get into the
demo portion? Yeah, sure.
So let's say, if we think about like joint ventures, especially
with both Mongo DB and Snap logic, I think it, it kind of
plays into the new compliance space and also the multi cloud
environments that we really wantto get into, right?
(30:21):
Because essentially that's wherecustomers are starting to go to
as well. There's actually like there,
there's a Gartner report saying that actually by the end of this
year, end of 2020, 590% of all large enterprises will be multi
cloud or in environment as well.So it, it that's just phenomenal
(30:42):
to hear about because because it's not just a fad, it's a
directional shift in technology.So if you think about that
space, you have to think about where data can then reside in
multiple data stores and multiple clouds on premise,
maybe even on my desktop back, my workstation that used to be
(31:04):
under my desk years ago. So just to have that flexibility
of being able to transfer data and transform data and also
integrate with applications as well across all that easily,
right? And officially is is like a mind
boggling puzzle that that's always evolving as well, right.
(31:26):
So I, I think that's probably one of the most ultimate use
cases. I, I, I see.
Hopefully that makes sense. I'm trying to.
I'm making a lot of assumptions without going to.
The yeah, no, no, it it definitely is.
And I look, I think throughout our conversation, we've touched
on a lot of it. You know, the, you spoke about
(31:47):
the IoT and the edge computing earlier as well too.
So collecting and, and processing that data nearby
before putting it back into the resulting database or, or
whatever the case might be as well too.
And we talked a lot about compliance and data sovereignty
as well. So I think that all makes sense.
So I think if if it suits you, Peter, why don't we, you know,
(32:11):
get into the meat of this and show a little bit of the snap
logic UI in action and how straightforward that is, is it
that suits? You yeah, sure thing, sure
thing. So I have a a video to show
after this but I wanted to just show this as a precursor.
Sure what? What you're looking at right now
is I'm logged into the stop logic control plane.
(32:32):
Right. So this user interface that you
essentially get. So if you look around this, it's
a little you, I'll come to this stuff in the middle in a second.
But essentially you'll see some of our functionalities and areas
here. So you can see pipelines, right?
OK. And then this takes a little bit
(32:52):
longer just to load, but essentially you have different
snaps. So these are different snap
packs I was talking about before.
OK, OK. The mongo DB snap pack you
actually see all the different different that are available
right? Perfect.
Yeah. And here is essentially what is
(33:13):
known as a pipeline, right? So there's essentially a start
and an end, right? So if you look at this, it's
it's like inside the configuration, you can see what
it's trying to read. This is just a sample pipeline
for for demo purposes. But essentially it's just you
see it's reading a file, a CSV file doesn't need any account
data. But if it was, you can input it
(33:34):
here, OK. And then it goes to the next
transformation step where it parses out and so on and so
forth. And one of the end results is
that it actually writes it out to a file called directory dot
Jason, right? OK so if I was to execute this,
basically it would read the file, transform it and then
(33:56):
basically write it to two different different files.
OK, OK. And usually I would execute this
and then demo this, but I would just wanted to throw this
concept out here because I wanted to show you something
that's very exciting coming fromSnap logic before I start this
and and taking what I showed youbefore, essentially you can
(34:19):
build an API driven pipeline. It's known as a trigger task,
right? You can also front load that
with, let's say you had a conversational user interface,
right? And then let's say you built
that and you wanted to integratethis into your ecosystem, right?
(34:40):
A lot of the stuff you can actually get off the shelf in
this one example I'm going to show you is, is using artificial
intelligence. It's going to connect to an LLM.
And basically an administrator can then purview and then use a
conversational interface and search for items about iPhones,
right? That's what all it is, and what
(35:02):
it would require is interrogation of databases, so
on and so forth. But no, there's not one line of
SQL that's actually written. Sorry about the SQL.
OK, we'll forgive you. But imagine if you will, it
could also be among the databaseas well, right?
So but the, but the concept hereis that no one has written any
(35:25):
programmatic query language, right, to view the, the
databases, but they're able to you're, you're able to build
this pipeline that uses AI to help provide the results that
you are seeking. So I'm just going to let this
play. This is going to go on for about
4 minutes. We're all showing the whole
(35:46):
thing but, but here we go. Here we have a database query
agent connected to a chat interface so that natural human
language could be used. This gives business analysts and
others greater and faster accessto data by removing the need to
create complex SQL expressions. Gone is the bottleneck of having
(36:07):
a single data team responsible for servicing all data query
request. Our database contains smartphone
product information, so let's ask a question about device
pricing for an iPhone. The AI agent processes our
request and we get back a response in straightforward
natural language. To build trust with AI
(36:27):
solutions, we added the ability to look behind the scenes at the
inner workings of the AI agent and show how it got the results.
Here we see that the database agent leveraging another agent
called Query Agent and passing it the parameter get the price
for iPhone 13. That agent processes our
(36:48):
original request and constructs ASQL query.
SQL expressions are fairly specific and exact in nature.
Things like spelling, capitalization, and spaces
matter. In this case the product search
was for iPhone 13 as one word, all lower case and without a
space. Unfortunately, we see that the
(37:10):
query returns no results. Instead of giving up, the
database agent tried again but expands the search parameters
and looks for any product with iPhone 13.
This then gets us a modified SQLexpression where the database
product names are normalized to all lower case.
Again, the query returns no results.
(37:33):
Not to be defeated, the agent further expands the query once
again, this time asking for all product where iPhone appears.
The resulting SQL expression normalizes all product names to
lower case and is looking for any product name with iPhone in
it. This time the query is
successful at retrieving information out of the database.
(37:57):
In the next steps, the information is analyzed and then
a response is crafted using natural language.
Let's go further behind the scenes on the workings of the
pipeline that makes this all happen.
In the Snap Logic Designer, we have a main parent pipeline that
defines the LLM prompts and controls the looping plus the
(38:17):
overall flow of the agent. The planning agent takes the
prompt from the parent and interfaces with the Amazon
Bedrock LLM to call the query agent where the query is
constructed and run. If no results are returned, the
flow goes back to the parent agent for prompt refinement.
(38:38):
You should notice that we haven't added any hard coded
structured rules or logic. The pipeline is basically taking
a prompt as input. Constructing ASQL query based on
the prompt does the search and if nothing comes back, devises a
new adjusted prompt for a new query.
(38:58):
This dynamic creation of prompts, responses and
evaluation of the response automatically refines the
answer. Once data has been successfully
retrieved, the next pass throughthe planning agent sends the
information to the data analysisand response agent.
There it is examined and a response will be crafted in
(39:19):
natural language by leveraging Snap Logic Agent Creator.
Create and deploy agents faster with any connector, pipeline or
API. Avoid vendor lock in with your
choice of LLMS and multimodal support.
Safe and secure with built in observability evaluation and
(39:40):
data security. Visit our website to learn more
about SNAP Logic and Agent Creator.
Excellent. No, thank you.
Thank you for that. It was I, I understand what you
mean now, but kind of we, we setthe scene, we showed a bit of
the UI and then we showed that in action, which which is great,
you know, so, so. All of that, as you said, done
(40:02):
with no code, with all of the snaps, etcetera.
But I loved how you engendered the trust behind it because you
could see what was happening behind each one.
I think that's really key for building these types of agentic
kind of applications is to be able to, yeah, we want, we know
the AI is clever enough to do it, but we also want to look
(40:26):
under the hood, right? We want to see what decisions
have been made. Tell us a little bit about how
that was kind of thought about internally and Snap Logic coming
up with this, because I think it's probably key for people's
kind of, I suppose, trusting of the agents in this instance,
right? Right, and just look, with any
(40:47):
type of new technologies and especially a a large
evolutionary step, it's very important to put in checks and
balances, right? Because there we AI, as
fascinating as it can be, you still need to check where it is,
right? Because obviously, right,
currently now there's still the paranoia of like AI is going to
(41:08):
take over the world and War of the Worlds is going to come and
computer reverses, right, the whole thing.
But, and so maybe it's checks and balance and a lot of that
may be real, right? But it's certainly a valid
concern. And just like any essentially AI
driven pipeline, so to say, you'd want to put in the checks
(41:31):
and balances as well, right? So you can actually put in pause
steps in there where you would require manual intervention to
inspect the results and then like continue another pipeline,
right? OK.
So, so essentially, yeah, that'sa very important point and and
and very a very valid concern, especially in today's day and
(41:51):
age. OK, I love the way in the demo
it refined the query a couple oftimes until it got, you know, it
surfaced up some of the results that it was looking for.
How do you like? Is there a limit to how you can,
you know, would keep the agent would keep trying to refine a
query to look at a result, or isthere a way to put guardrails on
(42:15):
the extent to which it might do something in that regard as
well? Peter, is that a case?
Yeah, exactly. It depends how you want to align
up your snaps, right? Let's say counters in the snap
saying, hey, make sure you know,if it comes back with no results
more than three times, then let's take this other route.
OK? So yeah, it's all configurable
(42:37):
in there. OK.
And obviously when you showed usthe the the snap UI there and
the snaps that were in there andthis the Mongo DB snap packs
that you had available as well too.
And ultimately, you know, the whole goal of some of kind of
getting the word out there aboutsnap logic is hopefully to drive
some attention towards that. How do people get started with
(42:59):
snap logic, like how easy it is to, you know, sign up for the
platform, join and and get to play around with that?
Is there, you know, how do you get people on board the
platform? Do you have amazing
documentation, tutorials, etcetera?
What way do you on board new customers?
Yeah. So, so there's both, right.
(43:19):
We, we actually have a wealth ofdocumentation online.
There's also there's an online community called Integration
Nation as well. If you, if you did, if you did a
search on Snap Logic IntegrationNation, there's an online
community as well. And you can also sign up for
free trial, right? So there's that as well.
And I think we'll share those links with the show notes, I
(43:43):
would assume, right so. Yeah, yeah, yeah.
Happy to put those out as well too, which which would be great.
So people can sign up for the free trial.
What is that? Is that a time limited trial or
is it a certain amount of snaps or pipelines or what?
What's the gauge on that, Peter?Yeah, exactly it is.
It's a 30 day free trial, OK. And only limited to certain snap
(44:05):
packs as well. I am curious if the AI ones are
in there, the pipe loops and stuff like that are included in
there. They may not have access to
certain advanced features such as like API management, which we
do have as well. That's OK.
Very up and coming feature that we do have.
I mean, it's been there already,but it's going to be a lot
(44:27):
stronger in the in the coming months as well.
I'm not sure that's included in the free trial, but I, I, I,
yeah, I'm sure you hear about it.
Unless. And in the the trial that they
might do, they can still connecttheir real proper data wherever
that might be up to their application or out to other
providers as well too, right? There's no constraints on what
(44:50):
they can do. It's it's just the time, right?
Exactly, exactly. Yeah, yeah.
So the the software is real, right?
So you can, I'm not, I'm not, I'm trying to remember if you do
get access to a ground Plex, which is basically downloading
the software onto your own premise, but you would certainly
have access to a cloud Plex, which is one that's managed by
(45:12):
Snaplogic, so. OK.
OK. Well, that makes sense.
So just go to the Snaplogic website to click the trial and
get in there. Is that the case?
Yeah, there's a certain link. I think it's snaplog dot dot dot
dot snaplogic.com free hyphen trial.
(45:32):
OK, free hyphen trial. Perfect.
I'll go to that URL right now while we're chatting, see if I
can grab it and and drop it intoour comments as well.
I've only only 5 available. No, I'm just kidding.
Not just maybe fine. Perfect, perfect.
So I know we only had a short piece of period of time we saw
(45:53):
the snap UI. For me, I thought when I saw the
list of snaps down the left handside that the fact that only the
A's fitted on the first screen before the scroll.
How many do you have? Like, do you know offhand,
Peter, how many snaps snap logichas at this point in time?
I'm trying to say I think it. I mean, I know it's well into
the hundreds I'm seeing. I'm, I'm trying to think of it's
(46:16):
surpassed 1000 mark or not. And then probably has and and
there's more being added every day, right.
So we also, not only do we develop additional snack packs,
we also have partners that are developing additional snack
packs as well. In addition to that, if you
wanted to develop your own snap pack, we actually have a
developers, you know, an SDK that would enable you to do that
(46:39):
as well. So if you wanted to create a
custom step, it's all there, right?
Obviously that would require some coding, but.
I found that URL, I just I just put it into the the comments
there as well too. I'll post it across into into
LinkedIn just shortly as well. At the same time, you can
(47:00):
certainly jump there. I suppose the key here to also
being the Mongo DB Podcast live.If you have your data on Mongo
DB, you can use all of those snap packs and everything else
that you have already populated up there to get playing around
quite quickly, right? Yes, that's right.
(47:20):
Excellent, excellent. What's excited you most about
kind of this is the question I ask most guests because I I'm
kind of keen, even from a personal perspective to see what
they tell me. The AI you mentioned, it's been
around for a long time, machine learning, everything else for
those of us who've been around alot, but you know, even in your
age and creator video there, it seemed incredibly powerful and
(47:44):
very, very straightforward to get up and running.
What's exciting you in general, not, not just in your own snap
logic space, but in general about AI these days, Peter?
I I'm really curious about the next step AI is going to take
the when, when we think about agents, right?
Those are essentially it's supposed to be autonomous
(48:08):
engines that run without any interactions, right?
So if you wanted it to do something for you, use ask the
agent, right? It's basically like if you can
think of it as a music agent, right?
Like what does a music agent do for you, right?
So they help you book gigs, theyhelp you market your your music,
(48:30):
right, so on and so forth. So looking at the next
evolutionary step of this is having everything be autonomous,
right? That's pretty wildly fascinating
what it could do, right? So, you know, can you, if you
could just imagine, just like, you know, hey, go, you know,
it's, it's, you know it, it's my, my wife's birthday coming up
(48:54):
right. So like, oh, I really need to
get her something and you know it's going to go and order it
for me. And you know your.
Wife's not going to like that example.
Now, Peter, that you're going touse AI to buy her her birthday
present. We have to think of a more
mundane example, right? Yeah, exactly, exactly.
I mean, I'll, I'll adjust, but yeah.
Yeah, no, I agree. I mean, I for me, the agentic
(49:17):
space is very interesting. I do hope it does take care of
the mundane things to give us more time to do the nice things.
Often, you know, all too often Isee these amazing AI demos that
they're doing the amazing cool stuff that you'd like to do
yourself as opposed to the mundane stuff that you'd like
taken away. But at the weekend I was away
(49:38):
and I'd made a bucking in a restaurant that we couldn't
make. We weren't going to get back on
time. I phoned up and it's said, you
know, the usual interface press 1, etcetera, etcetera.
I did that and was like, and this is the first time in real
life that happened to me. It was an AI voice agent.
What do you want to do? You want to cancel your
reservation Is that the reservation for tonight for four
(49:59):
people at 8:30? Yes, it is perfect.
Just confirm you want to cancel it it's cancelled and then done
so for something like that, AI is perfect because that's a
that's a non transaction. Basically it's like, you know,
this is a this is a poor thing. I need to cancel it.
I'm very sorry, I can't turn up,you know, whereas the flip side
(50:20):
of that, I want to book a table.I don't know, do I want to talk
to the AI per SE? I might want to talk to the the
the host or Hostess and, and learn a little bit more about
it. So we'll see.
We'll see. Yeah, that's right.
Just like the travel agent is a great example.
Like if you're in flight, yes, let's say that it's going to be
(50:40):
late, but you're going to miss your connection.
You know, wouldn't it be beautiful if you had an
autonomous agent actually rebookyou on another flight while
you're still flying on the firstflight, by the time you touch
down, it's seamless. Like, you know, because that's
probably one of the most paranoid paranoia events that
(51:02):
would happen when you touchdown like, Oh my gosh, my luggage.
What do I do? How do I connect to to my other
flight? How do I get to my destination?
Well, if that's all taken care of for you, that just it just,
it's just so much more convenient, right So.
Yeah, I, I think so. And look, I very much look
forward to that level of convenience hitting us all soon
and various, various kind of projects etcetera as well too.
(51:25):
Just before we wrap up, Peter, going back to, you know, at the
beginning, very keen on your career path to date etcetera and
where you've come from. I'm also keen in your role as as
principal architect with Snap Logic.
How do you learn yourself? How do you keep on top of the
changes that are happening in our landscape all of the time?
(51:46):
Where do you go to to keep on top of all of that news and
technology? Well, well, I, I do try to stay
connected with industry and different external partners like
yourself, Shane, I mean, 'cause yeah, you bring a lot of great
insight into and that helps feedideas and like the next
evolutions of designs and architectures that we do have in
(52:06):
house for Snap Logic. So I think that's, that's a key
opportune time, key opportune function is to stay connected
with third parties, right? And and if if it was hard to get
started there maybe going out tolike different IT networking
events in your area, if there isany or online forums as well,
(52:30):
right? It was a lot easier pre COVID
but but it's still possible today to do.
That it was it's it's coming back and I know look, yeah, I
think everybody experienced thatwe went through the collective
isolation of being stuck at home.
But I know from a Mongo DB perspective, we're, we're back
on the road with all of our dot local events in many locations
(52:53):
where I'm particularly involved on the developer relations team
here at Mongo DB and running ourthe, the third party events that
we're involved in. So these would be the developer
focused events, the language community events, the the Java,
the Python, the C# events that we, you know, have huge
developer communities in and we turn up with those.
So I agree with you, Peter. I think those sort of events I
(53:16):
think I used, you know, there was there's a serendipity
involved in turning up with these things and kind of having
those hallway conversations and and meeting those people that
you wouldn't necessarily kind ofbump into.
Because I think our inboxes are swamped with newsletters and
emails that we like to keep up with.
But I think it's often. Yeah, get out there and about is
(53:38):
a good way I find that I learn, you know, you're you're away
from the day job, you're paying pretty much 100% attention to
whomever might be speaking on stage if you're at an event.
I love that. So yeah, community events and
yeah, paying attention to what'sgoing on.
It is super hard to keep up withhow fast everything is moving
(53:58):
though. Peter, right?
It is, it is Frank, because today my car drives itself and
yeah, I can order food instantaneously.
So it's just, it's, it's, uh, the future is here, right?
So. Exactly, exactly.
Well, look, I think that's probably, uh, on those kind of
(54:19):
words. It's a good way to wrap up our
show. It's been great to have you on
board, Peter. Um, I thought, you know, what we
went through in terms of kind ofwhere snap logic fits in, in
this hybrid multi cloud world that we live in at the moment.
I think the the ability to kind of understand the need for
tooling like snap logic, coupledwith the fact that you showcased
(54:42):
the UI, which was to me the quick glance that I had of it,
incredibly intuitive and obviously, as you said, more
than 1000 or more snap. So I presume there's something
for everybody there. Anybody who's trying to connect
services together and not do, asyou said originally was write a
little bit of code that will do that, that will eventually
break, right? Or that that, as so often is the
(55:05):
case these days, that developer moves on to a different company
and nobody knows how that code was put together, right?
Exactly. And thanks for having me on,
Shane. I really appreciate this.
This has been wonderful. Not at all, no.
It's it's been great to have youon board and it's great to hear
that not only snap logic users of Mongo DB, but also connecting
customers to their use cases of Mongo DB as well too.
(55:28):
So it's nice to see the loop closed in that regard.
For those that are joining us, we, we threw in the comments a
link essentially they're to snaplogic.com/free Dash trial
that you can go in there time bounded of course, as most
trials are, but you can go in there fully function and get to
play around with it yourself. I would be remiss of me if I
(55:50):
didn't make it the odd plug for Mongo DB as well too.
So everything that we do within Mongo DB, this developer focused
either in the developer relations team or our
engineering team and product team, we put up our developer
tutorials on developer dot MongoDB dot com.
Please go there to have a, a good look around at what we
have. We've got some great filters
(56:11):
there. So you can find if it's a
language, if it's a cloud partner, if it's an integration
that you're looking for, by all means check that out as well
too. And if you some people said
they, I think in the comments, as I watch some people are just
getting started with Mongo DB etcetera as well too.
So if you need help, ourcommunity.mongodb.com is the
place to go. That's our forums where our dev
(56:34):
REL team, our engineers, our support people all hang out as
well too. So that's the end of my plugs.
But for me, Peter, it's been great to have you on board to
learn a lot more about Snap logic.
And it would be remiss of me notto do a shout out to your
colleague Dominic. Dominic I met at Reinvent, AWS
Reinvent. He's an ex Mongo DB person and
(56:56):
essentially the chat with him atReinvent going back to our talk
about events being serendipity, quick chat with Dominic at the
Mongo DB booth was, hey, can we get someone from Snap Logic on
the Mongo DB podcast live? And here you are.
So Peter, we're closing the loop.
Dominic, gratitude to get Peter on the show with me.
(57:16):
I appreciate that. I hope you enjoyed our slot
Peter, and you got everything that you wanted to do across in
our conversation. Yeah, it's been great, Shane.
Thank you so much and thank you,Dominic.
Yeah, thanks. We give him a shout out.
Well, listen, Peter, it's been brilliant.
I hope you know, I'll keep an eye on what's going on in Snap
Logic. And as you have new things or
(57:37):
new demos, just keep in touch. We'd love to get you back on the
show sometime in the in the not too distant future to show how
Snap Logic is, is kind of bringing new features to the
this space as well too, which I think is incredibly important.
As you said, it's a. It's a can be a messy space.
It doesn't need to be with the use of tools such as snap logic.
(57:58):
All right. Thank you, Shane.
Will do, will do definitely. Excellent.
Plus it's been my pleasure. Thank you so much, Peter, and
for me, Shane McAllister. Please tune in every Tuesday
mostly for you'll hear me or oneof my colleagues have great
guests such as Peter from Snap Logic on as well too.
We do appreciate everybody who joined and commented and we look
forward to having you back again.
(58:19):
But for now, for this stream, for this episode of Podcast
LIVE, thank you everybody. It's been great to have you take
care. Thank you.
Hello everyone and welcome to another MongoDB podcast live.
It's great to have you with us. I'm Shay McAllister and today
we've got a fantastic show as ever lined up for you.
We're diving into the one of themost exciting shifts in modern
data architecture, the move towards multi cloud, hybrid and
(00:26):
edge computing. And that's where Snap Logic
comes in. So it joining me today is Peter,
whose principal architect at Snap Logic.
And Peter is going to breakdown how Snap Logic is
revolutionizing the way businesses connect data across
clouds on Prem and beyond. We're going to talk about the
challenges that enterprises facein multi cloud and hybrid
(00:47):
environments and of course, how Mongo DB and Snap Logic work
together. And we're going to get a first
hand look at the Snap Logic UI and exclusive showcase of Snap
Logic's latest AI powered data automation in action.
Peter, you're very welcome to the show.
How are you? Hey, how are you?
Thanks, Shane. Thanks for having me on.
Not at all. It's great to have you.
(01:09):
So where are you joining us from, Peter?
I asked our guests, our viewers,to say where they are coming
from. Where are you based?
Sure thing. I'm based in the San Francisco
Bay Area, so more specifically if you're familiar with the
area, I'm I'm actually located in the East Bay.
That's where I live. Snap logic offices are actually
located on the peninsula, so that would be over to the to the
(01:30):
West side of the Bay. So, but currently I'm I'm home
so. Excellent.
So we're getting you at somewhatthe start of your day, tail end
of my day here in Ireland. And I know the US daylight
savings changed at the weekend right too.
Let's keep it current. So I think the scheduling of
this gave me a little bit of angst kind of going on my on my
5 or 8 hours on my 7 hours. What's what's the difference?
(01:55):
That's right, that's right, yeah.
And usually with the first week of daylight savings, it's a
little, it's a little bit sleepy, right?
Because like you lose an hour right in the spring of sleep
and, and, and it just takes me about a week to adjust to it.
So my kids, it takes any longer.But nonetheless, it's it's all
good. Yeah, Yeah.
(02:16):
Well, listen them it's great to have you join us, Peter, as
like, look, I, I talked about some of the things that we're
going to dive into in the introduction, but one of the key
things that I'm always interested in, I know that our
audience are as well too, is forany of the guests I have.
I love to know your path to date, your career path to date.
You know, how did you end up here?
(02:37):
What were you doing beforehand? What did you do all the way back
in college? You know, where have you been
since etcetera that took you to right now with your current role
as architect and Snap logic? Yeah, sure thing.
Well, I graduated from RIT. That stands for Rochester
Institute of Technology. That's located in Rochester, NY.
(02:58):
That was quite a few centuries ago, but I do hail from the New
York City area in tri-state areawhere I was born and raised.
And so naturally after graduating college, I stayed up
in Rochester. And if if anybody's familiar
with that city, there's, there'sbasically like Xerox and Kodak
were the were the major companies there.
(03:20):
So I weren't there for a number of years for both companies and
then moved back home to New YorkCity, worked on essentially Wall
Street for a number of years as an independent.
Then eventually it came time to,to raise a family.
My wife and I wanted to raise a family.
So we wanted to move out to finda place in, in the country where
it's a little bit different. So we, we came out here,
(03:44):
backpacked out to, you know, theSan Francisco Bay Area and just
kind of hunkered down and, and we've been here ever since about
2005. And so since I've worked at a
lot of, you know, I worked at some Fang companies and
essentially where I used to workprior in Fortune 500 companies,
now I find myself working for startups seems to be all the
(04:06):
thrill in the Bay Area. So that's how I wound up with
Snap Logic. Excellent.
You're bringing your Fortune 500knowledge back to the startups
basically then. So these are how the big
companies do it. This is what you, you, you need
to be thinking about as well, too.
It's interesting that you mentioned Rochester.
I, I, as I said, I'm from Ireland.
I've been traveling to the US quite a lot for work over the
(04:27):
years, but I'd say it was probably 20 odd years ago.
The last time I was in Rochester, I used to work for an
actual data storage company, a RAID systems company, and we
used to do our and what what they used to call accelerated
life testing of our systems there.
So they had a testing centre there that was unique, that
(04:48):
could do earthquake testing, allof this kind of high frequency
life, half life failures as well.
So I, I spent a few, few trips to Rochester, but that again,
long, long time ago. Perfect.
So you mentioned then culminating in snap logic.
Umm, tell us a little bit for the viewers who don't
(05:09):
necessarily know, tell us a little bit of, you know, around
Snap Logic, a kind of a high level overview of what Snap
Logic does, what the problem it's solving for its users and
in the kind of modern data ecosystem that we're in.
Sure thing, sure thing. So essentially Snap Logic is an
eye pass platform, right? So it's an integration platform
(05:32):
as a service. And what does that essentially
mean is that we help connect data sources in a company's
ecosystem one to another, also applying transformations in
between. In addition to data, we also
integrate applications across aswell, right?
So that's one advantage. So for example, if you wanted to
(05:55):
connect your, you know, Mongo database to like Salesforce upon
certain events that would certainly happen, then you can
certainly do that. And the advantage of snap logic
is that it's a basically a code free or a load code platform,
right? So it's ease of integration
(06:17):
there. OK, so you're kind of like an
interoperator between all of these platforms and, and
obviously, you know, I think particularly in large
enterprises, Mongo DB included as well too.
We, we use so many different platforms as well.
It's like, well, how do we connect all these things
together? And, and obviously from our own
perspective, we have integrations with our key cloud
(06:40):
partners and many other tech partners as well too.
But you're sitting in the middlespace then Peter and Snap logic,
you have to be almost agnostic. You just need to make sure you
can get, you know, from A to C through snap logic being to be
in the middle, right? That's right.
That's right. And it should be like a seamless
interaction too. And that's where it gets
(07:03):
somewhat tricky at times, right?So you have a lot of different
compliance space that changes a bit.
So you're talking about different environments as well.
So it's not just like point A topoint B, it's like you really
have to skip over like hurdles at certain times.
Yeah, I can imagine it's hard, you know and as maybe and I
(07:25):
don't know, but I would imagine your engineering team or very
much on top of the moving shifting, you know ecosystem
that's out there in terms of what people are building, what
people are adding and you know what knock on effect it might
have for snap logics infrastructure, right.
Yeah, exactly. Well, it would be to a certain
degree. I think a lot of it depends on
(07:47):
how you build the system, right?Because it's a very natural
problem space. I mean, if you look at
engineering and you had to connect one data source to
another, it's just a very natural inclination to like, I'm
going to write that one script that does that, right?
It's no one's paid job to reallydo it, first of all, right?
So it's just someone says I can write that in 5 minutes and they
do that. And the next thing you know,
(08:09):
someone changes something like a, a login or a password script.
I know what, I'm going to decouple my credentials into a
separate data store. Well, that's another piece,
right? And then, oh, the API change,
oh, that's another piece, right?So on and so forth.
So a lot of it really depends onhow you designed or architected
your system. And have to say, like App Logic
(08:29):
has done that really well where it's built upon a composable
architecture, right? There are there's always going
to be moving spaces in, in technology.
It's just a fact of life. And essentially, if you build it
right, you don't really have to change too much, but you would
need to append on to it, right? So it's built very well in that
(08:55):
degree where it's efficient and it scales well.
OK, so when you and I were chatting prior to to prepping
for this stream that we're doing, we talked a lot about
flexible data architecture and the kind of various, I suppose
you would call them pitfalls, but also, you know various
things to be concerned about as to how this.
But the flexibility, the advantages that gives you, if
(09:18):
you bear that in mind from the outset, talk a little bit about
that, like around, you know, avoiding things like vendor
locking and, you know, being compliance and being that
resilience that you just talked about.
That's right, that's right. And those are just some of the
key things of building a flexible arc data architecture,
right? Because in this day and age of,
(09:39):
of and there's also going to be,you know, the honest truth is
like economically, globally, we're all trying to save money,
right? We're trying to be more
efficient with what we have, right?
And we're also at the same time,we're also trying to maximize
what our expenses are, right? So there's gonna be a few key
principles that we look there for building flexible data
architecture, right? Is the first thing is that
(10:02):
companies really and there's andjust take a step back.
There's been a natural evolutionfrom data centers right to
clouds, public hypervisors. And now we're, we're embarking
on the next step, right? So where companies are starting
to notice that, yeah, there's there's the, the big cloud
(10:23):
providers like AWSGCP, Azure, soon and so forth.
But no companies really want to be locked in to like any one of
them, right? So they're trying to think
about, oh, how do I be cross cloud now?
How can I be cross cloud and on Prem or on the edge right as
well? So there's those factors in
(10:43):
there. There's also the the factor of
compliance, right So talk. To me, a little bit about on the
edge because I hear that a lot and I, I, I, I kind of know what
it is because we, we, I looked after a product that we
repurpose for edge stuff. But and as an edge database,
talk to maybe the viewers who mightn't be familiar with that
on the Edge might mean Peter. Sure thing.
(11:05):
Umm, so, so on the edge is essentially, if you have it,
umm, if you can think of that asa computing device that would
live, let's say for example, in your home, a Wi-Fi router would
be a perfect example, right? So that resides in your home,
right? If for those that have like
cable modems at home and such for Internet access, so inside
(11:27):
your Wi-Fi router, there's, there's certain intelligence
there. There's actually think of it as
a small computing engine in there that able, it's able to
perform some functions separate from the cloud, right?
Because everything in the cloud you're, you're we're always
thinking that we're just, you know, computing things in the
cloud and getting input and output from our homes where it's
(11:49):
like you took like a Wi-Fi router that's in your home, you
actually have a small computing devices there as well.
OK. So any devices that may be, you
know, an IoT device for example may also be edge device as well
like right Wi-Fi routers, littlelight switches, thermostat might
(12:10):
be in your home. I'm trying to stay away from
product names there, but. I get you.
We, we, I know when we were building our edge database
product, we were working with some of the the cellular
providers. So they were saying, look, when
your when your mobile device is making requests, you know, why
(12:31):
are we taking that all the way through the network when
sometimes the tower that received the request can
actually know what to do with it, right?
Exactly, Exactly. OK, And here's a here's a funny
little and I don't hear this term too too often anymore, but
like about five years ago, thereused to be this term called fog
computing and people like fog computing.
What's that? If you take a little
(12:53):
intelligence from the cloud and you bring it down to the edge,
it's like a fog. That's a new one.
That's a new one on me, but I love it.
I love the analogy. I think that I think that's
really good and it's great. If anyone else knows of any
other analogies like that, throwthem into the comments.
For those that joined and missedme, please add any questions you
(13:14):
might have for Peter as we make our way through the show.
It's great to see the folks joining us, you know from
Pakistan and Indonesia and Bangalore and Denver etcetera as
well too. So please do get involved.
When I asked you about age there, I had stopped you going
into the compliance part of how you were explaining what, you
(13:34):
know what's to be concerned about with flexible data
architectures, Peter. So talk to us a little bit about
compliance then. Sure thing.
So there's Prior to being a SNAPlogic, I used to work at a small
a healthcare company. And so we also used data talking
specifically about data in the compliance space in healthcare.
(13:58):
It was imperative that we neededto store our data on Prem, which
meant, you know, not in one of the public Capital Advisors
essentially. So we actually had Mongo
enterprise on premise, right. So that became very compliant as
well. Whereas the rest of our
computing was actually at the time on Azure, right.
(14:18):
So it was one of the public hypervisors, whereas our data
was actually on premise. So in industries such as like
healthcare and also finance, there would be some compliance
according to where the data is stored.
Also issues of topics of, excuseme, data sovereignty as well.
(14:38):
So there's certain data that areproduced in specific areas or
regions in the world that need to remain there as well.
So there's other types of issuesand topics that we that that
would come into play as well. Yeah, OK.
And being based in Europe, I'm particularly aware of the issues
because we have a very fragmented approach to data
(15:00):
sovereignty and compliance as well too.
That causes a ton of headaches, business opportunities at the
same time, but a ton of headaches for, for, for
companies as a whole, you know, and I'm assuming with with
regard to the compliance side ofthings.
The other factor then in the whole flexible data architecture
is just pure resiliency. That's why many people choose
(15:22):
multi cloud in the 1st place, right?
It's to, it's to have that resiliency either from a
physical perspective, a geolocation perspective,
etcetera as well, too, right, Peter?
That's correct. That's correct.
So a lot of companies are movingobviously depending on your
requirements towards resiliency or, or low RTO, low RPO, like
(15:43):
whether you have to be in different availability zones,
right? That's one, one area of like
resiliency. If that's good enough, then they
just, they're just basically in the same cloud provider, but
just in different availability zones.
They have to be in separate regions.
That's another requirement. And today there's even there's a
(16:06):
desire to be in, in multiple clouds as well, right?
So you have all those different factors at play.
OK, OK. And so that all of those
different factors at play are factors that really help Snap
Logic's position in that decoupling of of the
architecture that companies are kind of ending up with these
(16:29):
days. And as you said in your earlier
example, yes, it's fine. Somebody can write a piece of
code that can do something and then they kind of, as you said,
move chain, the login breaks andthen they got to decouple that
etcetera as well too. So obviously snap logic brings a
ton of benefits. I would imagine you know, given
the the huge decoupling, it's, it's also quicker to integrate,
(16:52):
maybe easier to scale, etcetera,more business value.
Talk to us a little bit about the kind of what you see, the
feedback from your customers, how it's how it's working for
them. Yeah, sure thing.
And that's a really good point, right?
Because it always, it's always seeds the idea whenever Snap
Logic is invited into a company to help integrate their systems,
(17:18):
right, always starts as a seed knowingly there.
It always branches out because it becomes such an efficient way
of, of performing this task thatit, it starts to just grow
because people, you know, companies find out that hey, I
just need to connect to this system, to this system.
(17:40):
I can write a script or I could just put together a small
pipeline in 5 minutes with snap logic and it's done, you know,
and it scales out really well and, and, and so that
proliferates through different business usages and it becomes
very efficient, right? Essentially the app integrations
(18:02):
and data integrations inside of companies usually doesn't start
as someone's like I use the termday job, but essentially it's
it's someone kind of takes it onas an extra, you know, an extra
function would want to try, but OK.
OK. And I suppose that might kind of
move us towards kind of where does Mongo DB fit into all of
(18:24):
this? Is that as well then, Peter, you
know, how does snap logic interact with Mongo DB?
What does Snap logic use Mongo DB for?
Sure, sure thing. So the main data store, well,
it, it integrates with Mongo DB in two facets, right?
The first one is that Mongo DB actually is the main data store,
(18:45):
right? For for, for snap logic, right?
So we chose in those sequels data store years ago to, to
house our data models, essentially all our assets and
operational metrics, all that isactually stored inside of our of
our Mongo clusters, right? So that's the first thing.
It's, it's acts as the backbone data store for Snap logic.
(19:09):
And now we also find out that there's a large customer base
that actually uses their Mongo data source that they would like
to integrate with their other systems, applications or their
other data source as well. So we we we have we we have
items called snaps. I'm trying to like see how it
(19:29):
can best explain this, but essentially.
And I know for those tuning in, we're, we are going to, you're
going to show this, we're going to open up the screen and share
the snap logic UI and, and, and go to that.
But that's intriguing though, because you're both a customer
and a partner of Mongo DB's essentially.
So you're using it internally for your own operations, but
(19:50):
you've obviously seen then your customers have data there that
needs to be connected to something else.
So hence you're you're also leveraging us as a partner.
Yes, that's correct. Perfect, perfect.
That's great. So I think so you touched on
snap packs. Do you want to talk a little bit
about what those? Let's set all the terms out, but
(20:12):
we're going to do show and tell,obviously in a few minutes,
yeah. Yeah, yeah, sure thing.
So. So if you think about how snap
logic is used to integrate systems in a company, it's what
is built in a workflow, which iscalled a pipeline, right?
So if you think about that as one executionable series of
(20:32):
steps that would comprise of a pipeline, let's say as simple as
read data from a file. That's one step in the transform
the data that's from the file into a separate, you know, a
separate format. That would be another step.
The third step is insert this data into Mongo.
(20:54):
That'd be the third step. So that comprises the pipeline.
Now each of the steps are also known as, as we call it a snap,
right, Because it's OK, OK, so yeah, so each of the snaps are
configurable as well. So it's it's pretty
integratable. And as you can think about,
there's hundreds of different snaps, right?
(21:16):
So and obviously you can snap them together however you want,
like like. I like it.
And obviously then the term snappacks is a is a pre built
sequence of snaps that you can leverage and is, you know, you
know, obviously kind of as you see customers doing certain
things that they require kind ofgoing, Oh, maybe is, is it the
(21:38):
case that you're looking at it going, maybe this is something
that all of our customers could benefit on.
Let's join all of these servicesup and give them a template.
Yeah, well, we do have template pipelines as well.
Snap pack is essentially a groupof snaps.
So for example, we have a Mongo snap pack.
(21:58):
In there we can connect to different Mongo data sources,
have different operations and such, right?
OK, OK, excellent. So, umm, I think we, we talked
obviously about those and obviously you know, all of the
introductions you spoke about the, the, the increasing
complexity that we live in, in a, in a kind of a multi cloud
(22:21):
and, and hybrid world as well then too Peter.
Umm, so I would imagine that then you know that complexity
and the way snap logic goes towards solving that has just
grown the business over time andand keeps growing it.
But I suppose in a roundabout way I'm saying the complexity of
today's environment is a benefitto snap logic because it it's
(22:45):
screaming out for the snap logicsolution applied to it, right?
Yeah, yeah, exactly. And if you look at the pace of
of technology evolution over theyears, it's just increasingly
faster, right? Every step.
And we're all familiar with the latest evolution of AI.
And, and if you look at the previous incantation to that, it
(23:07):
was just, it just seems shorter and shorter and shorter and
we're just exponentially going quicker and quicker and quicker.
So, yeah, staying in front of that curve, right?
It is certainly the question, right?
Yes, yeah, yeah, I'm surprised we got the 26 odd minutes into
the live stream before mentioning AI that usually it
(23:30):
usually comes up a lot earlier for for on most of my guests How
and I know we're going to see kind of an example of an agent
creator later on, but how has AIaffected Snap Logic's business
and and kind of how it thinks about things as well too.
What what changes has the AI brought about for the company as
(23:51):
a whole? Well, AI has always been part of
the fabric or the desire snap logic, even years before chapter
80 came out. Let's just say I think AI was
coined as machine learning. We had a staff of data
scientists that actually evolvedsnap logic and build upon it.
(24:15):
So we actually had predictive learnings of pipelines at that
time, even before chat ChatGPT. Yeah.
When AI came and now when when introduction of ChatGPT came
out, it became very conversational, right.
With the interface of of artificial intelligence and,
(24:35):
and, and, and the staying in front of that, essentially
that's what snap logic has done,right.
So, so essentially we become nowwe're not trying to be the LLM
or be the artificial intelligence engine.
We still want to be in that integration space, right.
(24:57):
So we enable connectivity to to the LLM and ease that
integration. And then there and I'll show
show this in a demo video, but essentially how one achieves
that very quickly in in a very low code environment.
OK, OK, excellent. I'm looking, I'm looking forward
(25:17):
to that. And again, I'm seeing the the
hellos coming in, in the comments.
It's great to have Andrew from Indonesia and folks from
Bangalore and Pakistan and Philadelphia.
Oscar, you're very welcome as well too.
Umm, you know, it's great to have you join us.
Do any questions you might have for Peter, please post them up
there. We'd love to entertain them.
So we talked a little bit about the multi cloud and the hybrid
(25:39):
world, et cetera. And you know, how you're
managing Snap via snap logic to connect data up with the
applications and platforms that need it, et cetera.
We mentioned compliance on the introductions.
How with regards to like, I presume you're kind of, you
know, my world is like an ETL, right?
(26:00):
Extract and transform and load. That's the way I'm, I'm looking
at some of this and I know it's,it's much, much bigger than
that. But that data in transit, how
does, where does SNAP logic fit in, in terms of being that
connector, in terms of compliance and security and
resilience there? Right, right, right, right.
So Snap Logic platform and how it's actually utilized, we're
(26:25):
very compliant with data governance and security, right.
So in dated governance, there's RBAC, permissioning, auditing,
full data metadata management right inside of security,
there's an encryption API security, of course Nexus
controls or SoC 2 compliant as well, right.
(26:48):
OK. In a compliance space where you
know, we're compliant with GDPR,HIPAA and of course, as I
mentioned, SoC 2 compliance. So, and I'm not sure if I spoke
about the architecture a little bit before, but essentially
there's there's a separation between how snap logic's
(27:10):
constructed, there's a separation between a control
plane and the data plane. OK, you know, I didn't quiz you
on that one. Tell us a little bit more about
that then. So, Peter.
Sure thing, sure thing. So if you think about the
control plane as being like yourmanagement console of your
system, right? And the data plane is
essentially think of this as your execution engine.
(27:34):
This is where the customer data would come to.
And essentially you would use a control plane to set up your
pipelines like I was talking about before.
OK, OK. You.
Use your data plane to essentially execute on those
pipelines right? So and the data plane can
reside. We offer what is known as a
Cloudplex. Right.
(27:56):
That that resides in the cloud. We, we can, we can, we can
assign you one of those or if you do choose, but we also
support what is known as a ground Plex, right?
So you can actually download thesoftware or if you want to run
it in a docker, we also have a docker image as well that you
can actually run a data plane either on your premise or in
(28:16):
your own cloud or your own tenant, essentially.
So we have both choices. And so in, in full compliance
like none of the no customer data would ever come to the
control plane essentially would stay, OK, that's your own
premise, right? So yes, it's very unique
(28:37):
architecture in that sense. OK, OK.
Now that makes sense and it becomes very clear now because I
would imagine that's probably one of the first things the
client, new client might say is,you know, hold on, Especially as
you mentioned earlier, people inthe healthcare sector and
obviously in the financial sector as well too.
And, and all of those areas where you know, that it's key
(28:58):
that, you know, data needs to have a ton of controls on it,
very, you know, very, very sensitive data, customer data,
the works. And, and I suppose look, Mongo
DB faces that as well too. And I think we've ticked most of
the boxes these days as well to make sure that usually when
these queries come up from potential clients, they can be
(29:19):
dismissed quite, quite easily byevery, all the scrutiny that
we've put ourselves to, as you mentioned, kind of the, the
Hipac and the Sock 2 and all of those things that Snaplogic has
assigned to as well. So we touched on, I get the
mechanism again, look in the demo, we're going to see this
really, I love the conversational piece of these
(29:40):
live streams, but I really like the demo.
So I'm looking forward to getting to that.
But before we do that, tell us alittle bit about, we've talked a
lot about how Snap logic works and how it helps companies,
etcetera. Tell us a little bit some of the
useful practical use cases that you might have from from from
customers, you know, can you talk to us a little bit about
(30:00):
how they might be used and leveraged before we get into the
demo portion? Yeah, sure.
So let's say, if we think about like joint ventures, especially
with both Mongo DB and Snap logic, I think it, it kind of
plays into the new compliance space and also the multi cloud
environments that we really wantto get into, right?
(30:21):
Because essentially that's wherecustomers are starting to go to
as well. There's actually like there,
there's a Gartner report saying that actually by the end of this
year, end of 2020, 590% of all large enterprises will be multi
cloud or in environment as well.So it, it that's just phenomenal
(30:42):
to hear about because because it's not just a fad, it's a
directional shift in technology.So if you think about that
space, you have to think about where data can then reside in
multiple data stores and multiple clouds on premise,
maybe even on my desktop back, my workstation that used to be
(31:04):
under my desk years ago. So just to have that flexibility
of being able to transfer data and transform data and also
integrate with applications as well across all that easily,
right? And officially is is like a mind
boggling puzzle that that's always evolving as well, right.
(31:26):
So I, I think that's probably one of the most ultimate use
cases. I, I, I see.
Hopefully that makes sense. I'm trying to.
I'm making a lot of assumptions without going to.
The yeah, no, no, it it definitely is.
And I look, I think throughout our conversation, we've touched
on a lot of it. You know, the, you spoke about
(31:47):
the IoT and the edge computing earlier as well too.
So collecting and, and processing that data nearby
before putting it back into the resulting database or, or
whatever the case might be as well too.
And we talked a lot about compliance and data sovereignty
as well. So I think that all makes sense.
So I think if if it suits you, Peter, why don't we, you know,
(32:11):
get into the meat of this and show a little bit of the snap
logic UI in action and how straightforward that is, is it
that suits? You yeah, sure thing, sure
thing. So I have a a video to show
after this but I wanted to just show this as a precursor.
Sure what? What you're looking at right now
is I'm logged into the stop logic control plane.
(32:32):
Right. So this user interface that you
essentially get. So if you look around this, it's
a little you, I'll come to this stuff in the middle in a second.
But essentially you'll see some of our functionalities and areas
here. So you can see pipelines, right?
OK. And then this takes a little bit
(32:52):
longer just to load, but essentially you have different
snaps. So these are different snap
packs I was talking about before.
OK, OK. The mongo DB snap pack you
actually see all the different different that are available
right? Perfect.
Yeah. And here is essentially what is
(33:13):
known as a pipeline, right? So there's essentially a start
and an end, right? So if you look at this, it's
it's like inside the configuration, you can see what
it's trying to read. This is just a sample pipeline
for for demo purposes. But essentially it's just you
see it's reading a file, a CSV file doesn't need any account
data. But if it was, you can input it
(33:34):
here, OK. And then it goes to the next
transformation step where it parses out and so on and so
forth. And one of the end results is
that it actually writes it out to a file called directory dot
Jason, right? OK so if I was to execute this,
basically it would read the file, transform it and then
(33:56):
basically write it to two different different files.
OK, OK. And usually I would execute this
and then demo this, but I would just wanted to throw this
concept out here because I wanted to show you something
that's very exciting coming fromSnap logic before I start this
and and taking what I showed youbefore, essentially you can
(34:19):
build an API driven pipeline. It's known as a trigger task,
right? You can also front load that
with, let's say you had a conversational user interface,
right? And then let's say you built
that and you wanted to integratethis into your ecosystem, right?
(34:40):
A lot of the stuff you can actually get off the shelf in
this one example I'm going to show you is, is using artificial
intelligence. It's going to connect to an LLM.
And basically an administrator can then purview and then use a
conversational interface and search for items about iPhones,
right? That's what all it is, and what
(35:02):
it would require is interrogation of databases, so
on and so forth. But no, there's not one line of
SQL that's actually written. Sorry about the SQL.
OK, we'll forgive you. But imagine if you will, it
could also be among the databaseas well, right?
So but the, but the concept hereis that no one has written any
(35:25):
programmatic query language, right, to view the, the
databases, but they're able to you're, you're able to build
this pipeline that uses AI to help provide the results that
you are seeking. So I'm just going to let this
play. This is going to go on for about
4 minutes. We're all showing the whole
(35:46):
thing but, but here we go. Here we have a database query
agent connected to a chat interface so that natural human
language could be used. This gives business analysts and
others greater and faster accessto data by removing the need to
create complex SQL expressions. Gone is the bottleneck of having
(36:07):
a single data team responsible for servicing all data query
request. Our database contains smartphone
product information, so let's ask a question about device
pricing for an iPhone. The AI agent processes our
request and we get back a response in straightforward
natural language. To build trust with AI
(36:27):
solutions, we added the ability to look behind the scenes at the
inner workings of the AI agent and show how it got the results.
Here we see that the database agent leveraging another agent
called Query Agent and passing it the parameter get the price
for iPhone 13. That agent processes our
(36:48):
original request and constructs ASQL query.
SQL expressions are fairly specific and exact in nature.
Things like spelling, capitalization, and spaces
matter. In this case the product search
was for iPhone 13 as one word, all lower case and without a
space. Unfortunately, we see that the
(37:10):
query returns no results. Instead of giving up, the
database agent tried again but expands the search parameters
and looks for any product with iPhone 13.
This then gets us a modified SQLexpression where the database
product names are normalized to all lower case.
Again, the query returns no results.
(37:33):
Not to be defeated, the agent further expands the query once
again, this time asking for all product where iPhone appears.
The resulting SQL expression normalizes all product names to
lower case and is looking for any product name with iPhone in
it. This time the query is
successful at retrieving information out of the database.
(37:57):
In the next steps, the information is analyzed and then
a response is crafted using natural language.
Let's go further behind the scenes on the workings of the
pipeline that makes this all happen.
In the Snap Logic Designer, we have a main parent pipeline that
defines the LLM prompts and controls the looping plus the
(38:17):
overall flow of the agent. The planning agent takes the
prompt from the parent and interfaces with the Amazon
Bedrock LLM to call the query agent where the query is
constructed and run. If no results are returned, the
flow goes back to the parent agent for prompt refinement.
(38:38):
You should notice that we haven't added any hard coded
structured rules or logic. The pipeline is basically taking
a prompt as input. Constructing ASQL query based on
the prompt does the search and if nothing comes back, devises a
new adjusted prompt for a new query.
(38:58):
This dynamic creation of prompts, responses and
evaluation of the response automatically refines the
answer. Once data has been successfully
retrieved, the next pass throughthe planning agent sends the
information to the data analysisand response agent.
There it is examined and a response will be crafted in
(39:19):
natural language by leveraging Snap Logic Agent Creator.
Create and deploy agents faster with any connector, pipeline or
API. Avoid vendor lock in with your
choice of LLMS and multimodal support.
Safe and secure with built in observability evaluation and
(39:40):
data security. Visit our website to learn more
about SNAP Logic and Agent Creator.
Excellent. No, thank you.
Thank you for that. It was I, I understand what you
mean now, but kind of we, we setthe scene, we showed a bit of
the UI and then we showed that in action, which which is great,
you know, so, so. All of that, as you said, done
(40:02):
with no code, with all of the snaps, etcetera.
But I loved how you engendered the trust behind it because you
could see what was happening behind each one.
I think that's really key for building these types of agentic
kind of applications is to be able to, yeah, we want, we know
the AI is clever enough to do it, but we also want to look
(40:26):
under the hood, right? We want to see what decisions
have been made. Tell us a little bit about how
that was kind of thought about internally and Snap Logic coming
up with this, because I think it's probably key for people's
kind of, I suppose, trusting of the agents in this instance,
right? Right, and just look, with any
(40:47):
type of new technologies and especially a a large
evolutionary step, it's very important to put in checks and
balances, right? Because there we AI, as
fascinating as it can be, you still need to check where it is,
right? Because obviously, right,
currently now there's still the paranoia of like AI is going to
(41:08):
take over the world and War of the Worlds is going to come and
computer reverses, right, the whole thing.
But, and so maybe it's checks and balance and a lot of that
may be real, right? But it's certainly a valid
concern. And just like any essentially AI
driven pipeline, so to say, you'd want to put in the checks
(41:31):
and balances as well, right? So you can actually put in pause
steps in there where you would require manual intervention to
inspect the results and then like continue another pipeline,
right? OK.
So, so essentially, yeah, that'sa very important point and and
and very a very valid concern, especially in today's day and
(41:51):
age. OK, I love the way in the demo
it refined the query a couple oftimes until it got, you know, it
surfaced up some of the results that it was looking for.
How do you like? Is there a limit to how you can,
you know, would keep the agent would keep trying to refine a
query to look at a result, or isthere a way to put guardrails on
(42:15):
the extent to which it might do something in that regard as
well? Peter, is that a case?
Yeah, exactly. It depends how you want to align
up your snaps, right? Let's say counters in the snap
saying, hey, make sure you know,if it comes back with no results
more than three times, then let's take this other route.
OK? So yeah, it's all configurable
(42:37):
in there. OK.
And obviously when you showed usthe the the snap UI there and
the snaps that were in there andthis the Mongo DB snap packs
that you had available as well too.
And ultimately, you know, the whole goal of some of kind of
getting the word out there aboutsnap logic is hopefully to drive
some attention towards that. How do people get started with
(42:59):
snap logic, like how easy it is to, you know, sign up for the
platform, join and and get to play around with that?
Is there, you know, how do you get people on board the
platform? Do you have amazing
documentation, tutorials, etcetera?
What way do you on board new customers?
Yeah. So, so there's both, right.
(43:19):
We, we actually have a wealth ofdocumentation online.
There's also there's an online community called Integration
Nation as well. If you, if you did, if you did a
search on Snap Logic IntegrationNation, there's an online
community as well. And you can also sign up for
free trial, right? So there's that as well.
And I think we'll share those links with the show notes, I
(43:43):
would assume, right so. Yeah, yeah, yeah.
Happy to put those out as well too, which which would be great.
So people can sign up for the free trial.
What is that? Is that a time limited trial or
is it a certain amount of snaps or pipelines or what?
What's the gauge on that, Peter?Yeah, exactly it is.
It's a 30 day free trial, OK. And only limited to certain snap
(44:05):
packs as well. I am curious if the AI ones are
in there, the pipe loops and stuff like that are included in
there. They may not have access to
certain advanced features such as like API management, which we
do have as well. That's OK.
Very up and coming feature that we do have.
I mean, it's been there already,but it's going to be a lot
(44:27):
stronger in the in the coming months as well.
I'm not sure that's included in the free trial, but I, I, I,
yeah, I'm sure you hear about it.
Unless. And in the the trial that they
might do, they can still connecttheir real proper data wherever
that might be up to their application or out to other
providers as well too, right? There's no constraints on what
(44:50):
they can do. It's it's just the time, right?
Exactly, exactly. Yeah, yeah.
So the the software is real, right?
So you can, I'm not, I'm not, I'm trying to remember if you do
get access to a ground Plex, which is basically downloading
the software onto your own premise, but you would certainly
have access to a cloud Plex, which is one that's managed by
(45:12):
Snaplogic, so. OK.
OK. Well, that makes sense.
So just go to the Snaplogic website to click the trial and
get in there. Is that the case?
Yeah, there's a certain link. I think it's snaplog dot dot dot
dot snaplogic.com free hyphen trial.
(45:32):
OK, free hyphen trial. Perfect.
I'll go to that URL right now while we're chatting, see if I
can grab it and and drop it intoour comments as well.
I've only only 5 available. No, I'm just kidding.
Not just maybe fine. Perfect, perfect.
So I know we only had a short piece of period of time we saw
(45:53):
the snap UI. For me, I thought when I saw the
list of snaps down the left handside that the fact that only the
A's fitted on the first screen before the scroll.
How many do you have? Like, do you know offhand,
Peter, how many snaps snap logichas at this point in time?
I'm trying to say I think it. I mean, I know it's well into
the hundreds I'm seeing. I'm, I'm trying to think of it's
(46:16):
surpassed 1000 mark or not. And then probably has and and
there's more being added every day, right.
So we also, not only do we develop additional snack packs,
we also have partners that are developing additional snack
packs as well. In addition to that, if you
wanted to develop your own snap pack, we actually have a
developers, you know, an SDK that would enable you to do that
(46:39):
as well. So if you wanted to create a
custom step, it's all there, right?
Obviously that would require some coding, but.
I found that URL, I just I just put it into the the comments
there as well too. I'll post it across into into
LinkedIn just shortly as well. At the same time, you can
(47:00):
certainly jump there. I suppose the key here to also
being the Mongo DB Podcast live.If you have your data on Mongo
DB, you can use all of those snap packs and everything else
that you have already populated up there to get playing around
quite quickly, right? Yes, that's right.
(47:20):
Excellent, excellent. What's excited you most about
kind of this is the question I ask most guests because I I'm
kind of keen, even from a personal perspective to see what
they tell me. The AI you mentioned, it's been
around for a long time, machine learning, everything else for
those of us who've been around alot, but you know, even in your
age and creator video there, it seemed incredibly powerful and
(47:44):
very, very straightforward to get up and running.
What's exciting you in general, not, not just in your own snap
logic space, but in general about AI these days, Peter?
I I'm really curious about the next step AI is going to take
the when, when we think about agents, right?
Those are essentially it's supposed to be autonomous
(48:08):
engines that run without any interactions, right?
So if you wanted it to do something for you, use ask the
agent, right? It's basically like if you can
think of it as a music agent, right?
Like what does a music agent do for you, right?
So they help you book gigs, theyhelp you market your your music,
(48:30):
right, so on and so forth. So looking at the next
evolutionary step of this is having everything be autonomous,
right? That's pretty wildly fascinating
what it could do, right? So, you know, can you, if you
could just imagine, just like, you know, hey, go, you know,
it's, it's, you know it, it's my, my wife's birthday coming up
(48:54):
right. So like, oh, I really need to
get her something and you know it's going to go and order it
for me. And you know your.
Wife's not going to like that example.
Now, Peter, that you're going touse AI to buy her her birthday
present. We have to think of a more
mundane example, right? Yeah, exactly, exactly.
I mean, I'll, I'll adjust, but yeah.
Yeah, no, I agree. I mean, I for me, the agentic
(49:17):
space is very interesting. I do hope it does take care of
the mundane things to give us more time to do the nice things.
Often, you know, all too often Isee these amazing AI demos that
they're doing the amazing cool stuff that you'd like to do
yourself as opposed to the mundane stuff that you'd like
taken away. But at the weekend I was away
(49:38):
and I'd made a bucking in a restaurant that we couldn't
make. We weren't going to get back on
time. I phoned up and it's said, you
know, the usual interface press 1, etcetera, etcetera.
I did that and was like, and this is the first time in real
life that happened to me. It was an AI voice agent.
What do you want to do? You want to cancel your
reservation Is that the reservation for tonight for four
(49:59):
people at 8:30? Yes, it is perfect.
Just confirm you want to cancel it it's cancelled and then done
so for something like that, AI is perfect because that's a
that's a non transaction. Basically it's like, you know,
this is a this is a poor thing. I need to cancel it.
I'm very sorry, I can't turn up,you know, whereas the flip side
(50:20):
of that, I want to book a table.I don't know, do I want to talk
to the AI per SE? I might want to talk to the the
the host or Hostess and, and learn a little bit more about
it. So we'll see.
We'll see. Yeah, that's right.
Just like the travel agent is a great example.
Like if you're in flight, yes, let's say that it's going to be
(50:40):
late, but you're going to miss your connection.
You know, wouldn't it be beautiful if you had an
autonomous agent actually rebookyou on another flight while
you're still flying on the firstflight, by the time you touch
down, it's seamless. Like, you know, because that's
probably one of the most paranoid paranoia events that
(51:02):
would happen when you touchdown like, Oh my gosh, my luggage.
What do I do? How do I connect to to my other
flight? How do I get to my destination?
Well, if that's all taken care of for you, that just it just,
it's just so much more convenient, right So.
Yeah, I, I think so. And look, I very much look
forward to that level of convenience hitting us all soon
and various, various kind of projects etcetera as well too.
(51:25):
Just before we wrap up, Peter, going back to, you know, at the
beginning, very keen on your career path to date etcetera and
where you've come from. I'm also keen in your role as as
principal architect with Snap Logic.
How do you learn yourself? How do you keep on top of the
changes that are happening in our landscape all of the time?
(51:46):
Where do you go to to keep on top of all of that news and
technology? Well, well, I, I do try to stay
connected with industry and different external partners like
yourself, Shane, I mean, 'cause yeah, you bring a lot of great
insight into and that helps feedideas and like the next
evolutions of designs and architectures that we do have in
(52:06):
house for Snap Logic. So I think that's, that's a key
opportune time, key opportune function is to stay connected
with third parties, right? And and if if it was hard to get
started there maybe going out tolike different IT networking
events in your area, if there isany or online forums as well,
(52:30):
right? It was a lot easier pre COVID
but but it's still possible today to do.
That it was it's it's coming back and I know look, yeah, I
think everybody experienced thatwe went through the collective
isolation of being stuck at home.
But I know from a Mongo DB perspective, we're, we're back
on the road with all of our dot local events in many locations
(52:53):
where I'm particularly involved on the developer relations team
here at Mongo DB and running ourthe, the third party events that
we're involved in. So these would be the developer
focused events, the language community events, the the Java,
the Python, the C# events that we, you know, have huge
developer communities in and we turn up with those.
So I agree with you, Peter. I think those sort of events I
(53:16):
think I used, you know, there was there's a serendipity
involved in turning up with these things and kind of having
those hallway conversations and and meeting those people that
you wouldn't necessarily kind ofbump into.
Because I think our inboxes are swamped with newsletters and
emails that we like to keep up with.
But I think it's often. Yeah, get out there and about is
(53:38):
a good way I find that I learn, you know, you're you're away
from the day job, you're paying pretty much 100% attention to
whomever might be speaking on stage if you're at an event.
I love that. So yeah, community events and
yeah, paying attention to what'sgoing on.
It is super hard to keep up withhow fast everything is moving
(53:58):
though. Peter, right?
It is, it is Frank, because today my car drives itself and
yeah, I can order food instantaneously.
So it's just, it's, it's, uh, the future is here, right?
So. Exactly, exactly.
Well, look, I think that's probably, uh, on those kind of
(54:19):
words. It's a good way to wrap up our
show. It's been great to have you on
board, Peter. Um, I thought, you know, what we
went through in terms of kind ofwhere snap logic fits in, in
this hybrid multi cloud world that we live in at the moment.
I think the the ability to kind of understand the need for
tooling like snap logic, coupledwith the fact that you showcased
(54:42):
the UI, which was to me the quick glance that I had of it,
incredibly intuitive and obviously, as you said, more
than 1000 or more snap. So I presume there's something
for everybody there. Anybody who's trying to connect
services together and not do, asyou said originally was write a
little bit of code that will do that, that will eventually
break, right? Or that that, as so often is the
(55:05):
case these days, that developer moves on to a different company
and nobody knows how that code was put together, right?
Exactly. And thanks for having me on,
Shane. I really appreciate this.
This has been wonderful. Not at all, no.
It's it's been great to have youon board and it's great to hear
that not only snap logic users of Mongo DB, but also connecting
customers to their use cases of Mongo DB as well too.
(55:28):
So it's nice to see the loop closed in that regard.
For those that are joining us, we, we threw in the comments a
link essentially they're to snaplogic.com/free Dash trial
that you can go in there time bounded of course, as most
trials are, but you can go in there fully function and get to
play around with it yourself. I would be remiss of me if I
(55:50):
didn't make it the odd plug for Mongo DB as well too.
So everything that we do within Mongo DB, this developer focused
either in the developer relations team or our
engineering team and product team, we put up our developer
tutorials on developer dot MongoDB dot com.
Please go there to have a, a good look around at what we
have. We've got some great filters
(56:11):
there. So you can find if it's a
language, if it's a cloud partner, if it's an integration
that you're looking for, by all means check that out as well
too. And if you some people said
they, I think in the comments, as I watch some people are just
getting started with Mongo DB etcetera as well too.
So if you need help, ourcommunity.mongodb.com is the
place to go. That's our forums where our dev
(56:34):
REL team, our engineers, our support people all hang out as
well too. So that's the end of my plugs.
But for me, Peter, it's been great to have you on board to
learn a lot more about Snap logic.
And it would be remiss of me notto do a shout out to your
colleague Dominic. Dominic I met at Reinvent, AWS
Reinvent. He's an ex Mongo DB person and
(56:56):
essentially the chat with him atReinvent going back to our talk
about events being serendipity, quick chat with Dominic at the
Mongo DB booth was, hey, can we get someone from Snap Logic on
the Mongo DB podcast live? And here you are.
So Peter, we're closing the loop.
Dominic, gratitude to get Peter on the show with me.
(57:16):
I appreciate that. I hope you enjoyed our slot
Peter, and you got everything that you wanted to do across in
our conversation. Yeah, it's been great, Shane.
Thank you so much and thank you,Dominic.
Yeah, thanks. We give him a shout out.
Well, listen, Peter, it's been brilliant.
I hope you know, I'll keep an eye on what's going on in Snap
Logic. And as you have new things or
(57:37):
new demos, just keep in touch. We'd love to get you back on the
show sometime in the in the not too distant future to show how
Snap Logic is, is kind of bringing new features to the
this space as well too, which I think is incredibly important.
As you said, it's a. It's a can be a messy space.
It doesn't need to be with the use of tools such as snap logic.
(57:58):
All right. Thank you, Shane.
Will do, will do definitely. Excellent.
Plus it's been my pleasure. Thank you so much, Peter, and
for me, Shane McAllister. Please tune in every Tuesday
mostly for you'll hear me or oneof my colleagues have great
guests such as Peter from Snap Logic on as well too.
We do appreciate everybody who joined and commented and we look
forward to having you back again.
(58:19):
But for now, for this stream, for this episode of Podcast
LIVE, thank you everybody. It's been great to have you take
care. Thank you.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Bobby Bones Show
Listen to 'The Bobby Bones Show' by downloading the daily full replay.